[fpc-devel] Compiler trheads (was: NoGlobals branch)

[Helmut Hartl]

  Am 12.08.10 11:23, schrieb Marco van de Voort:
>> [2] How portable is a dynamic thread priorization?
> You don't. Your control thread determines what module is compiled/resumed
> next when a worker thread comes available. There is no reason to do this via
> funky thread systems.
A single controlling thread does not scale very well. A better solution 
is that the
workers choose their work by themself. Or to use some sort of 
distribution queues.

After doing years of multithreaded development in our communication 
appliance
we found that apple came to a similar solution like our own, and took 
the time to write it down :-)

http://developer.apple.com/mac/library/documentation/General/Conceptual/ConcurrencyProgrammingGuide/ThreadMigration/ThreadMigration.html#//apple_ref/doc/uid/TP40008091-CH105-SW1

We have an custom written scheduler that operates on lock free 
distribution queues that feed exactly
or less that amount of threads that the machine is able to do really 
parallel.
We strive to use no locks at all by distributing the work(chunks) 
directly to a per worker thread queue,
and where not possible to use lock free algorithms. Ideally is to 
partition the data if it's known beforehand
or at an early stage. Also it is good to keep the working set data per 
thread and per core locally and don't move it from
core to core.

More infos:

About the speed of locks :
http://transact09.cs.washington.edu/19_paper.pdf

About  memory/cache/threading in general:
Google: Ulrich Drepper, What every programmer should know about memory

helmut