[fpc-devel] Parallel Computing

[Peter Popov]

> As much as I do appreciate your comments, I really can't believe that  
> the GNU GCC community took the pain to fully integrate OpenMP that  
> deeply in their system that it can be used by just issuing a single  
> #pragma line, if there would be no benefit to be expected.
>
> -Michael

Actually, most people who are doing parallel computing for scientific  
purposes (the vast majority of prallel programs written so far) use MPI  
(http://en.wikipedia.org/wiki/Message_Passing_Interface). Language  
extensions are never too popular as far as shared memory  
machines/clusters/supercomputers are involved. The main reason is that  
such equipment is usally unique and the manufacturer typically provides  
only FORTRAN/C compiler, sometimes C++, and always an optimized MPI. There  
is virtually no chance that GCC would run well on a custom supercomputer.  
Moreover, parallel programmig is quite difficult, so usually there is no  
time to explore new language concepts.

Let me give you a flavour of how concurrency is implemented in ADA, using  
pascal syntax
(see http://www.seas.gwu.edu/~csci51/fall99/ada_task.html).

procedure TSomeClass.DoSomething;
type
   local_task1 = task; // Declaration of parallel task
   local_task2 = task(x: integer); // Declaration of parallel task
var
   res: Integer;

   // Implementation of first local task
   task local_task1;
   begin
   end;

   // Implementation of second local task
   task local_task2(x: integer);
   begin
   end;

var
   A: local_task1;
   B1: local_task2(10);
   B2: local_task2(20);

begin
// At the start of DoSometing, four processes run concurrently:
// DoSomething, 1 copy of local_task1 (A) and two copies of local_task2  
(B(10), B(20))
...
// DoSomething exits only after both A and B(01) and B(20) all are done
end;

The difference between a normal local procedure and a parallel task is  
that tasks are started at declaration time. Up to this point, the  
semantics is tha same as in ADA. Passing parameters to a task is done at  
declaration or by messages  
(http://www.seas.gwu.edu/~csci51/fall99/ada_task.html).

However, one can instead define a pointer to a task which is no longer  
started automatically:

type
   plocal_task2 = ^local_task2;

and then, start a new task at any time by doing:

var
   pB: plocal_task;
begin
   ....
   New(pB, plocal_task(30));
   ...
   Dispose(pB); // wait for pB to terminate
end;

In this way tasks can be made equiavalent to local procedures/functions.  
The idea is to encapsulate TThread. The advantage is that the following  
programming paradigm is significanlty easire to code:

We have a class with method DoSomething. DoSomething executes a series of  
complex (higher level) parallel tasks. These tasks need input from  
DoSomething and need access to its local variables (global with respect to  
each task). Whatever the local tasks do, eventually DoSomething patches  
the results and returns to the caller.

The main advantage is the the task will have access not only to  
DoSomething's local declarations but also to all the methods, fields and  
properties of TSomeClass. With threads, the descendant of TTHread which  
implements the task needs a pointer to TSomeClass in order to have access.  
This means additional work in constructors, etc and it is not clean.  
Relatively simple algortithms get spread along several methods which is  
not desireable.

I will try to refresh further my memories of ADA's parallel constructs and  
try to see if it could be useful.
I would also like to mention one language particularly good for  
distributed heterogeneous networks (clouds, as it is called these days):  
mpC.

Peter Popov