[fpc-devel] Compiler trheads (was: NoGlobals branch)

[Hans-Peter Diettrich]

Marco van de Voort schrieb:
> In our previous episode, Hans-Peter Diettrich said:
>> How would you count an level, when multiple concurrent parser threads 
>> create further threads concurrently?
> 
> Hmm, I think a producer/consumer system with a queue for modules that can be
> compiled and N worker threads (specifiable by user)
>  
> So on start threads are created, and no new threads are created after?

That's not a solution :-(

*When* a parser becomes a thread, it can proceed only to the first 
"uses" clause - then it has to create further threads, one for each used 
unit, and has to wait for their completion. I.e. most threads are 
waiting for other threads to complete[1].

That behaviour makes it questionable, whether parsers should become 
threads at all. Furthermore it suggests (to me) a dynamic thread 
priorization, based on the number of other threads waiting for their 
completion[2]. At least we have to distinguish between processing the 
interface part (or loading the declarations from the ppu file), until 
which point all dependent threads (using that unit) are blocked. Once 
the interface information has become available, the dependent threads 
can continue parsing and generating object code (.o and .ppu files).

Brainstorming: What other models come into mind, for using threads in 
the compiler?


Your suggestion with a limited number of threads becomes interesting for 
the code generation, but this IMO deserves another modification of the 
workflow - currently all procedure bodies are processed "inline", during 
parsing, so that the procinfo can be destroyed on exit from parsing the 
procedure block. So we should have threads for parsing a procedure, with 
the strange result that parsing can happen in the mainstream, as well as 
in a thread. I just don't know what consequences will arise for the 
actual code, in such a splitted environment. Parsing inside a thread may 
require to synchronize several actions, what's not necessary during 
parsing in the mainstream (main thread).


[1] What synchronization methods exist for threads, suitable to block 
and resume threads before they have finished? With a fixed number of 
threads we IMO risk a stall of the entire compiler, as soon as each of 
these threads is blocked for *other* reasons (memory management, disk 
I/O...).

[2] How portable is a dynamic thread priorization?

DoDi