[fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

[Andrew Brunner]

On Wed, Oct 13, 2010 at 8:28 AM, Michael Van Canneyt
<michael at freepascal.org> wrote:
> Probably because it uses a heap manager per thread.
>
> You may try to use 'cmem', which will replace the heap manager with the C
> memory manager (one for the whole app, not per thread). That will allow you
> to test this hypothesis.

Ok.  Trying to speed up the creating of threads I have enabled cmem by
putting it as the first unit in the project.  cmem,cthreads, are the
first two units to be exact.

looking at the status of the process using the system monitor it shows
me that process switches from ptrace_stop to futex_wait_queue_me
(cycling).  None of my CPUs look to be all consumed so this is clearly
an inefficiency with the way threads are created and added internally
to the fpc rtl.  I tried upping the nice level to -15 and it does
speed up a little, it still takes minutes to create anything over
1,500 threads. Futexs can be avoided with interlocked calls to assign
pointer values. I really hate to see the application thread slowed
down to a crawl just because of memory or class creation.

Please (anyone is welcome) help me speed fpc thread creation up.  I
might be able to suggest improvements to the threading system.  Anyone
intimately know FPC and it's linux guts w/r/t/ TThread that is willing
to take a look at this with me?