[fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

[Michael Van Canneyt]

On Wed, 13 Oct 2010, Andrew Brunner wrote:

> On Wed, Oct 13, 2010 at 8:28 AM, Michael Van Canneyt
> <michael at freepascal.org> wrote:
>> Probably because it uses a heap manager per thread.
>>
>> You may try to use 'cmem', which will replace the heap manager with the C
>> memory manager (one for the whole app, not per thread). That will allow you
>> to test this hypothesis.
>
> Ok.  Trying to speed up the creating of threads I have enabled cmem by
> putting it as the first unit in the project.  cmem,cthreads, are the
> first two units to be exact.
>
> looking at the status of the process using the system monitor it shows
> me that process switches from ptrace_stop to futex_wait_queue_me
> (cycling).  None of my CPUs look to be all consumed so this is clearly
> an inefficiency with the way threads are created and added internally
> to the fpc rtl.

FPC doesn't have anything to say about CPU allocation. 
The threads are created by the C pthread library and Linux kernel.
They do the heavy work.

> I tried upping the nice level to -15 and it does
> speed up a little, it still takes minutes to create anything over
> 1,500 threads. Futexs can be avoided with interlocked calls to assign
> pointer values. I really hate to see the application thread slowed
> down to a crawl just because of memory or class creation.
>
> Please (anyone is welcome) help me speed fpc thread creation up.  I
> might be able to suggest improvements to the threading system.  Anyone
> intimately know FPC and it's linux guts w/r/t/ TThread that is willing
> to take a look at this with me?

Why do you think I answered you ? :-)

Michael.