[fpc-devel] Parallel processing in the compiler

[Florian Klaempfl]

Am 06.09.2010 05:22, schrieb Hans-Peter Diettrich:
> Florian Klämpfl schrieb:
> 
>>> Right, that's how it *should* be designed. But try to find out why the
>>> code generation is added, when variables like current_filepos or
>>> current_tokenpos are moved into TModule (current_module) :-(
>>
>> Why should current_filepos and current_tokenpos go into TModule? They
>> can be perfectly threadvars.
> 
> Good point. So would it be sufficient to retype all such variables as
> threadvar, in order to make parallel processing (threads) possible?
> 
> I have no real idea, how in such a model the initialization of the
> threadvars has to be implemented. That's why I try to assign all
> state-related variables to a definite object, whose reference can be
> easily copied (or moved?) into any created thread.

In case of current_filepos, the initialization could be done by the
specific worker thread class doing e.g. code generation.

> 
> Then we also have to draw a border, between possible parallel and still
> required sequential processing. Sequential processing must be used for
> all output, be binary or log files. Also encountered errors must be
> reported from the originating thread to all related (main and other)
> threads, when they require to stop and shut down compilation in an
> orderly manner.

Yes, this won't get easy, but we are far away from something like this.

> 
>> How we can continue? I'll see if I find within the next week time (I
>> were on holiday for one week) to review the noglobals changes and how we
>> can split them into usable parts.
> 
> IMO the most important decision is about the general direction of the
> refactoring. Do we want more OO (encapsulation), more codegen
> separation, or what else. IMO encapsulation is the most useful first
> step, towards any other goal. 

I think we agreed on this already. And I also agree with you, that the
front end needs some redesign and it won't be easy (it took me almost 2
years to find the current back end design btw. ...)

> The current compiler "structure" is
> dictated by purely *formal* aspects (unit dependencies), and does not
> reflect the *logical* dependencies between objects, variables,
> procedures etc. 

The problem is (very simplified): in a compiler everything depends on
everything, so getting some structure is only possible with some
formalism, formalism is the last resort when logic does not work
anymore. Formalism is also something absolute while logic might be
rather subjective.

> This lack of logical structure, next lack of up-to-date
> documentation, 

Nobody volounteered yet to continue Carl's work of documenting the compiler.