[fpc-devel] TClientDataset (was: Dnamic packages support)

[Joost van der Sluis]

Op maandag 05-11-2007 om 10:44 uur [tijdzone +0100], schreef Michael Van
Canneyt:
> 
> On Sun, 4 Nov 2007, Joost van der Sluis wrote:
> 
> > Op zondag 04-11-2007 om 00:34 uur [tijdzone +0100], schreef Michael Van
> > Canneyt:
> > > 
> > > On Sat, 3 Nov 2007, Joost van der Sluis wrote:
> > > 
> > > > Op zaterdag 03-11-2007 om 19:57 uur [tijdzone +0100], schreef Michael
> > > > Van Canneyt:
> > > > > 
> > > > > Packages and the missing TClientDataset are the only 2 remaining
> > > > > things 
> > > > > stopping me from using FPC/Lazarus for my daytime job. All else is 
> > > > > covered or can be covered with a minimal amount of work. 
> > > > 
> > > > You'll have to explain the TClientDataset to me. Do you need a
> > > > symantical identical equivalent of TClientDataset?
> > > 
> > > I need all features offered by TClientDataset:
> > > 
> > > - 3-tier database handling, TProvider and all.
> > > - Local filtering
> > > - Local sorting/indexing
> > > - Locate()
> > > - Maintained Aggregates
> > > - Load/Save to file (briefcase model)
> > > 
> > > And all features are critical; We have about 3200 instances in 1 application,
> > > almost all of them using all of the features mentioned, maybe with exception
> > > of the aggregates.
> > > 
> > > > Or do you only miss some features in sqldb?
> > > 
> > > SQLDB will be good for our server tier, so I can drop the IBX.
> > > But it's not suitable for the client layer.
> > > 
> > > As I see it, TClientDataset should be built on top of TBufDataset.
> > > Local filtering is available as fas as I know (or did you put it 
> > > in TSQLQuery ?), and the indexing probably also can somehow be 
> > > managed. The locate can be done with a filter. The maintained
> > > aggregates would need to be implemented from scratch, as well
> > > as the packet handling and file handling...
> > 
> > Local filtering is in TBufDataset, locate also. The packet-handling is
> > also there?
> 
> So you can create a delta packet, it can be sent/applied/whatever, and
> you can integrate whatever changes were incorporated after the delta
> was applied ? I must have missed that ?

Yes. But you can only apply the 'delta-packet'. Or cancel it, offcourse
(ie: remove it). But that's because saving to disk and sending to
another TDataser or over the network isn't implemented at all.

> > Only no file-handling. But that's more because I didn't find it urgent
> > enough to implement. It should be very easy. And there are no local
> > indexes. 
> 
> The local indexes are one of THE most important things.
> The file handling is relatively easy, but should be designed well
> so it can handle various formats.
> 
> > 
> > Bu I know where to work on now. But there's also a design-issue.
> > 
> > You talked about a TProvider. But as TBufdataset is setup know, you
> > don't need any. For every kind of connection between TBufDataset and the
> > ouside world, you'll have to use another descendent of TBufDataset.
> 
> TProvider is needed to
> 1. Supply initial data.
> 2. Apply the deltas and send back reconciling data.

You don't need a TProvider for that. Take for example sqldb: TSQLQuery
does nothing more (or less) then a TProvider would do. TBufDataset does
the rest. (btw: TSQLQuert doesn't send back reconciling data, although
TBufdataset can handle it)

> > So we can add a TProviderDataset or something, that can connect to data
> > on an application-server. Is this ok to you? That way you always heve to
> > re-create your descendent of TBufDataset for every connection?
> 
> Hm. I'm not sure I understand this ?

Read the example above. See TSQLCustomQuery as the provider, TBufDataset
as the TClientDataset.

> > Or do you like the TClientDataset model better? So that you can connect
> > it to any TProvider you want - to get access to some data?
> 
> This is essential; Without this it is useless to me.

I need to know if you can also use the model above.

> > You're right that we could make a TClientDataset which inherits from
> > TBufDataset. But that kind of defeats the purpose of TBufDataset.
> 
> Nono. You should split out functionalities:
> 
> The TBufDataset should handle what it handles now, plus the local indexes
> and saving to file. In short: a complete in-memory dataset. It doesn't care
> where the actual data comes from; This is the job of a descendent. 
> (TSQLQuery, TClientDataset)
> 
> What TClientDataset adds to this is the ability to work with a TProvider:
> - Get records from a remote source (IAppProvider interface).
> - Send and Apply delta, integrate result back in buffer.

TBufDataset can already do this. A descendent has to:

Override Fetch, LoadField, LoadBlobintoBuffer and ApplyrecUpdate.
(integrating the results in the buffer is done by TBufDataset, just as
TClientDataset does)

So TClientDataset could override these functions simply in such a way
that they call the same functions, but then form an external provider.

ie:

function TClientDataset.Fetch : boolean;
begin
  Result := AProvider.Fetch;
end;

And all the like...

If you really need compatibility with the IAppProvider interface (why?)
then there's more work to do. But that was what I meant with my first
question. Do you need the functionality ot TClientDataset, or a
symantical equivalent?

> > Because if you do so, it would also be better to base TSQLQuery on
> > TDataset directly. Or else you're double-buffering your data when you
> > connect a TSQLQuery (TBufDataset) with a TProvider to a TClientDataset
> > (another TBufDataset).
> 
> You'll always have double buffering. Don't forget that TSQLQuery and TProvider
> are on the application server tier, and TClientDataset is on the client
> tier. The client tier doesn't have direct access to the database so
> TSQLQuery would be useless.

On the client side you're right. But on the server side... woudn't you
need both a TSQLQuery and a TClientDataset on the application-server?

> That's why a 'unidirectional' property in TSQLQuery/TBufDataset is needed: 
> in that case, only 2 buffers are needed: one for the current record and one 
> for edit/update/filter. Makes a HUGE difference on the server. IBX implemented 
> it especially for that.

What does TBufDataset do more then TDataset when that property is set to
True?

If you need a unidirectional dataset, don't use a TBufDataset. That's
also why I once made a TSQLQuery based on TDataset. 

Joost.