[fpc-pascal] Re: Multi-threaded project with few locks (no Thread.waitfor). Memory consumption keeps increasing on Ubuntu 10.10 x64

Andrew Brunner andrew.t.brunner at gmail.com
Fri Oct 15 20:15:51 CEST 2010


On Fri, Oct 15, 2010 at 12:01 PM, Vinzent Höfler
<JeLlyFish.software at gmx.net> wrote:

>> Exactly.  It was poor implementation.  You should have had a global
>> barrier onExecute.  That would unlock the thread after everything you
>> needed was readable.
>
> If it's global, it would unlock *any* thread at that time.

Yes, belonging to a system with of collection of said threads where
"managing" system oversees
any worker threads.

> Sure, it's not the optimum design. But using one global barrier each
> thread could block each other on creation, potentially introducing
> excessive context switches. Which is not optimal, neither.

Ok.  Say a project manager is looking to give FPC a try for a highly
scale system that requires managing
1 million concurrent operations across say 30,000 threads.  Say it's a
10 core system with 32Gigs ram.  The first thing they are going to do
is compare a general case with TThreads.  As is - it's slow as crap.
They aren't going to want to re-implement the threading system.  They
aren't going to take FPC seriously as a viable compiler for parallel
computing.

Pascal offers people the ability to quickly read what they are seeing.
 IMO, FPC would be a great solution.  But with so many competing
platforms and with M$ literally pumping millions into their parallel
computing platform... Hey... It's probably worth just going with the
compiler du jour ;-)

> Maybe it's less expensive, yet this blocking is actually required.

It is not required if threading is implemented that ALREADY has a
barrier in this case see pthreads source. It does.

>IMO, the problem is neither FPC nor pthreads,
> the problem is the required semantics of TThread.

The problem is not with TThread.  The problem is with Suspend and
Resume features.

> The faster alternative is bound to fail with "FreeOnTerminate := True".

How did you draw this conclusion?

> Perhaps - instead of looking into the pthreads library - you should
> better take a look at the TThreads implementation and try to understand
> *why* all the code there actually *is* there.

I'm going to look at severe linear inefficiencies.  That is my nature. :-)

> Yeah, right. Creating the semaphore to wait in the execute method is
> bound to be faster than creating it before thread creation. Sure.

LOL.  Again, how in the world did you make the assumption, especially
given all the code I put out there regarding a manager overseeing all
the threads, that I would create a semaphore inside a worker thread's
execute body?  Did you read the uThreads.pas unit for this discussion?

> Yes. But if you prefer crashing the application in favor of speed just
> use C. :->

I would most likely prefer objective-c.

>> IMO, cThreads has a lot of RFI. (Room for improvement).
>
> No. TThread has. Last time I looked into CThreads it was a more or
> less thin binding to the pthreads API.

TThread has RFI.  But are you then suggesting you want to improve w/o
using libc !?!
IMO, using whatever we can would be great.

>>>> There is no need to block the thread from execution.
>>> Actually there currently is. Otherwise you ask for trouble.

We're going to disagree here.  It's just that simple.  If you found
yourself in the ranks with developers saying "What would anyone need
30,000 threads" camp, please ignore this entire discussion.  I beg
this with authority :-)

> Try calling pthread_destroy() with an uninitialized handle. Maybe
> that changes your way of thinking.

No, I pretty much question EVERYTHING.

> Oh, great. That probably was never done before. Sorry for my sarcasm.

My question is... Sorry to whom?

> *What* barrier?

Seriously?  There are four possibly five (depending on prospective)
barriers mentioned in the course of this entire discussion.  List them
and I will be able to oblige.



More information about the fpc-pascal mailing list