[fpc-devel] TField.AsString and Databases with UTF-8 charset

[Joost van der Sluis]

Op vrijdag 24-07-2009 om 12:49 uur [tijdzone +0200], schreef Graeme
Geldenhuys:
> Michael Van Canneyt wrote:
> > 
> >> That way, SqlDB can return copy(fieldvaluestring, 0, character_len)
> >> as the actual field text value, which trims off the padding of
> >> spaces.
> > 
> > If you look carefully, you'll see that the padding of spaces happens
> > in code in the case of a CHAR field. Maybe we should do something
> > about that.

We can add parameter for this in TSQLConnection, Delphi does the same,
it's a quite common approach. (ie: an option: should char-fields be
trimmed.

> In Summary:
> -------------
> There are two problems here.
> 
> 1) SqlDB and Firebird are reporting the wrong TParam.Size and 
> TField.Size results. SqlDB is using byte length instead of character length.

Which is should. You are thinking in a 'real-world' situation. But you
have to look at the TDataset/TStringField model. It allocates DataSize
bytes for each string per record. And, yes, in case of an utf-string,
that has to be 4*character-count. This works perfectly fine.

> 2) The UTF-8 implementation of Firebird is seriously flawed. Firebird 
> makes as if UTF-8 is a fixed byte algorithm and just returns rubbish 
> results from a Char(x) field and breaks the DDL rule of what the maximum 
> character length is. Using the metadata, SqlDB *can* fixes this by using 
> something like:  copy(fieldvalue, 0, MaxCharacterLength)

They made some decisions you would have done differently. But sqldb is
not there to hide db-features(bugs?) form the end-developer. It's only a
layer to connect to the database. Post-processing should be done by the
end-programmer, imho.

I don't have time to read the complete thread, but these are the basics.

Joost