[fpc-devel] Parallel processing in the compiler

[Hans-Peter Diettrich]

Jonas Maebe schrieb:
> On 04 Sep 2010, at 19:41, Hans-Peter Diettrich wrote:
> 
>> Every parallel processing requires that all related data is private
>> to every thread. Since some time I'm trying to eliminate such
>> (currently) global variables. Now ppudump turned into an show
>> stopper, due to the unsystematic and obscure unit dependencies in
>> the compiler:
> 
> I think it is quite systematic and logical that the ppu loading
> should not depend on code generator internals.

Right, that's how it *should* be designed. But try to find out why the 
code generation is added, when variables like current_filepos or 
current_tokenpos are moved into TModule (current_module) :-(


> You have to describe in your svn commit logs what you actually do.
> "NG: fixed make all" is not a useful commit message. You changed 149
> files in that commit, and did not describe what was done nor why:

I only commited these changes so that they don't get lost. Please review 
the preceding version, as mentioned in the Readme.

The last change was to remove ppudump from the Makefile, and this proved 
that so far only ppudump is sensitive to changes in the compiler internals.

> * what does all that {$ifdef fix} code do? What does it fix?

It's simply a flag for recent changes, which can be made unconditional 
after verification. The name was choosen to be enough unique, so that 
all affected locations can be found be searching for "fix".


> * you
> deleted the aasmsym unit and changed code that depended on it without
> any remarks why you did that. The comment in the header of that unit
> said: "Contains abstract assembler instructions for all processor 
> types, including routines which depend on the symbol table. These
> cannot be in aasmtai, because the symbol table units depend on that
> one."

I wanted to make code generation more abstract, so that it can be 
separated better from the remaining code. The GlobVar unit is intended 
as a place for all (previously) global variables, that can be used 
without importing further dependencies.

> You nevertheless did move that code to aasmtai and now there are
> indeed (implementation-level) circular dependencies between aasmtai
> and symsym. Adding extra circular dependencies to the compiler
> without any argument as to why this is required is not good.

I don't remember details. My experimental approach left these units 
unmodified, and still resulted in importing code generation (see above).

> * that
> commit also contains changes that have nothing to do with fixing
> "make all", such as fixing a typo in a comment, removing type
> redefinitions, commenting the assignment of tcgaddnode to caddnode in
> the init code of ncgadd.pas (and similar assignments in ncgmat),
> possibly an unrelated change in ppheap (which is wrongly indented),
> removing an unused local type definition from ppu.pas...

I tried to follow the coding style, to not comment anything in the code.

Please suggest an better and more systematic way of reducing the number 
of compiler hints, and supply a justification for redefinitions of 
system unit types.

> * I assume
> that all the code you added in psystem.pas is with the long term view
> of making the parser independent. However, that is a separate project
> and not part of removing global variables from the compiler

Both are related, since the "stateful" variables are related to 
current_module, and that's where the scanner and parser is involved, 
somehow.

> While reviewing your previous commits, I noticed that you also
> integrated your patch from
> http://bugs.freepascal.org/view.php?id=16888 although that one has
> nothing to do with removing the use of globals.
> 
> This way of working is not how we can keep the compiler development
> manageable for multiple people to work on.

I added some improvements and corrected typos, while waiting for 
comments on my work. It would be nice to know in *advance*, what should 
be avoided. The lack of response to my repeated review requests is not 
what I consider as "multiple people working on the same project".


In the meantime I started several projects, not commited to the 
repository, one of which (almost only) moves global variables into 
TModule, without breaking ppudump and affecting unrelated units. If you 
think that this is a better base for cooperation, I can try to add it as 
another branch, with no detailed history.

DoDi