[fpc-devel] Compiler trheads

[Dimitri Smits]

----- "Florian Klaempfl" <florian at freepascal.org> schreef:

> Am 12.08.2010 08:12, schrieb Hans-Peter Diettrich:
> > Brainstorming: What other models come into mind, for using threads
> in
> > the compiler?
> 
> The "final" plan is to use a task queue with dependencies and all
> needed
> information. Such tasks may be: parse module X, generated code for
> module X, load module Y, search for module Y, assemble module X etc.
> Each task has a list of dependencies which much be done first before
> the
> task can be executed.
> 
> So not a task per unit or whatever is started but a couple of worker
> threads depending on the number of cores available, given command
> line
> switches etc. These worker threads look for task in the queue with
> all
> dependencies solved and when such a task is found, it is executed.
> The
> more cores we get during the next years, the finer we could split the
> tasks.

+1 for this approach.

BTW, Hans-Peter, those worker threads are better conceived then as general purpose tthread descendants.

Java (I know, but hear me out first) has at least 2 ways to get threads done:
* implement a Thread descendant and override the run/execute method with the task's code => this is no good for the above scenario
* implement a Runnable interface for your task's code and use with a thread instance to let it call it's run/execute method => this is place to look

in Delphi/FPC one could also do this (either with inheritance or interface implementation):

IWorkerTask = interface(IUNKNOWN) // or the default implementation, or as a class...
  procedure Execute();
  procedure SetFatalException(fatalException:TObject);
  procedure Done();
  function IsDone(): boolean;
end;

TWorkerThread = class(TThread)
private
  FWorkerTask: IWorkerTask;
public
  procedure Execute(); override;
  property Task: IWorkerTask read FWorkerTask write FWorkerTask;
end;

procedure TWorkerThread.Execute();
begin
  while not Terminated do
    if Task is nil then 
      self.Yield() // method that gives up rest of timeslice of thread
    else
    begin
      try
        Task.Execute();
      except 
        on o:TObject do
        begin
          Task.SetFatalException(o);
        end;
      end;
      Task := nil;
    end;
  end;
end;


Just a thought & example, the main thread could then do the marshalling over the available pool of workerthreads and tasks.

Mind you, in this approach you can NOT use "threadvars" for workertasks' data since you do not know if/when the task is resubmitted/remarshalled on other threads. All data should hang in the implementation of that IWorkerTask and be used as a proxy from there.

kind regards,
Dimitri Smits