[fpc-pascal] msedb and fcl-db benchmarks

[Martin Schreiber]

On Tuesday 17 July 2007 23.58, Joost van der Sluis wrote:
> > So this makes your case, doesn't it. Martin argued that his dataset was
> > designed to work together with his visual controls.
>
> Yes, but it's an important part, because exactly this is the part that
> has changed. Except from this and the widestring-difference, there are
> no real differences. So I'm very curious know why his dataset works
> better with his visual controls.
> I want to know this, so we can remove the need for two times the same
> component. Or that's at least what I hope for.
>
What I need for the MSEgui components is 
- fast RECNO access for the DB grid proportional scrollbar mode (I think we 
talked about that when you changed TBufDataset to an linked list).
- internal indexes to switch sorting order in DB grids fast.
- calculated fields (which didn't work in TBufDataset at the time I needed 
them).
- internal calc fields (which are not implemented in TBufDataset up to now 
AFAIK).
- full BLOB and MEMO support (they where not fully functional in TSQLQuery at 
the time I needed them).
-"TClientDataset" functionality (not implemented in TBufDataset).
- the storage of strings as widestrings to prevent converting from/to unicode 
at every data access.
- the storage of strings as pointer to a variable length string to allow the 
MSEgui users to use long varchar fields without memory waste (TSQLQuery 
stored the popular PostgreSQL TEXT field with dsMaxStringSize = 8192 bytes).
- the limiting of the edited string length by character count and not byte 
count (think on multibyte encodings or current Linux installations with utf-8 
locals).
- and possible more which comes into mind at the moment...

> > I'm refering to very large buffered datasets held in memory by
> > middleware for fast access. It is well known for instance that Delphi's
> > TClientDataset chokes with say 100000 records, whereas different designs
> > scale much better, TkbmMemTable for instance. Not long ago Marco van der
> > Voort suggested (in another forum) to have a look at something like  his
> > "lightcontainers", apparently someting completely different  (than for
> > instance a TList based  dataset).
> > I've put forward this point  because  for datasets used with visual
> > controls speeds can't  be the issue at all. But for very large sets it
> > certainly is.
>
> Well, that's exactly the difference between mse's bufdataset and the
> fcl-db one. Tmsebufdataset is based on a TList, TBufDataset isn't. And I
> believe that at TList based dataset is always slower. (although it could
> save some memory)
>
The architecture of tmsebufdataset are dynamic arrays of pointer which hold 
pointers to the recordbuffers.
One for the "physical" order and one for every internal index.

Martin