[fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

[Vinzent Höfler]

On Fri, 15 Oct 2010 18:19:14 +0200, Andrew Brunner  
<andrew.t.brunner at gmail.com> wrote:

> On Fri, Oct 15, 2010 at 10:57 AM, Vinzent Höfler
> <JeLlyFish.software at gmx.net> wrote:
>
>> If you access it inside the execute method, you more or
>> less crash (or at least leak memory).
>
> You obviously had a problem with access ThreadID before it was assigned.
> Accessing it should not arrive at a RAV.

No, but trying to free the thread with an invalid handle via the
pthread_destroy() function does.

>  I'm not sure this is on topic...

It was the reason for the changes introduced into FPC. I don't
remember the exact bug report number.

>> Excerpt from the workaround I had written then:
>
>>      //-- Overwritten to ensure it starts in suspended state,  woken up
>>      //-- by @link(AfterConstruction) a little moment later.
>>      constructor Create;
>
> Exactly.  It was poor implementation.  You should have had a global
> barrier onExecute.  That would unlock the thread after everything you
> needed was readable.

If it's global, it would unlock *any* thread at that time.

> See my project attached a few emails back.  It
> demonstrates how things should be done with threads.  Now - fpc
> carries one private barrier per thread instance.

Sure, it's not the optimum design. But using one global barrier each
thread could block each other on creation, potentially introducing
excessive context switches. Which is not optimal, neither.

Maybe it's less expensive, yet this blocking is actually required.

> Developers of FPC should not always blame the development platform.

I don't think anyone did. IMO, the problem is neither FPC nor pthreads,
the problem is the required semantics of TThread.

> They are sometimes going to need to redirect an issue back onto the
> developer.  But I do recognize this is a delicate issue.  But slowing
> us all down for issues such as these...

The faster alternative is bound to fail with "FreeOnTerminate := True".

Perhaps - instead of looking into the pthreads library - you should
better take a look at the TThreads implementation and try to understand
*why* all the code there actually *is* there.

> Perhaps a wiki article on how to get threads to wait during execute
> would have an easier fix.

Yeah, right. Creating the semaphore to wait in the execute method is
bound to be faster than creating it before thread creation. Sure.

> But I make a case, that slowing down the entire platform for those who
> need help getting threads to synchronize makes for a poor movement
> collectively.  Perhaps a plethora of examples could help.

The reason wasn't "people need help getting threads to synchronize", the
reason was - TThread does not work as advertised under all circumstances.
And some of those circumstances were not at all controllable by the user.

I presume, most people rather have a *working* thread implementation than
a fast, but - for no apparent reason - occasionally crashing one due to
race conditions.

>> A similar workaround got into FPC a while later and it *was* for a
>> good reason: Safety.
>
> Anyone can make safety an issue.

Yes. But if you prefer crashing the application in favor of speed just
use C. :->

> IMO, cThreads has a lot of RFI. (Room for improvement).

No. TThread has. Last time I looked into CThreads it was a more or
less thin binding to the pthreads API.

>>> There is no need to block the thread from execution.
>>
>> Actually there currently is. Otherwise you ask for trouble.
>
> I disagree.  Completely.  But that's just because I'm not tied to any
> one way of thinking.

Try calling pthread_destroy() with an uninitialized handle. Maybe
that changes your way of thinking.

> I reviewed the pthreads code.

Oh, great. That probably was never done before. Sorry for my sarcasm.

> BTW: pthreads is what FPC uses to implement threading.

On Unix like targets.

> The barrier may be accessible therby eliminating
> the need for an additional instance being created inside TThread.

*What* barrier?


Vinzent.