[fpc-devel] TField.AsString and Databases with UTF-8 charset

[Michael Van Canneyt]

On Tue, 21 Jul 2009, Graeme Geldenhuys wrote:

> Michael Van Canneyt wrote:
>> 
>> I never use Char fields,
>
> I can live with that...
>
>
>> and I also don't use charsets in Firebird...
>
> ..but not with this! Here are just a few issues with using NONE as the 
> default charset in a Firebird database.
>
> * Firebird does NOT default to your system locale. It ignores
>  anything on you OS, so all databases on any OS in any locale
>  are treated the same.

No problem, since this is Windows, all with the same CP.

>
> * Firebird can't correctly uppercase, lowercase or no accent
>  non-ASCII characters

All this is handled on the client app. For the 4 fields in the database where it matters,
we have a 2nd (indexed) field which contains a lowercased value of the value of the original 
field. Takes slightly more room, but is much faster.

>
> * Firebird sorting of non-ASCII characters very wrong for   most occasions

Sorting never happens by firebird, all on client.

> * Nothing in the database itself documents how the bytes   stored should be 
> interpreted as characters

All handled by app.

>
> * When the same database is simultaneously or in succession   used by systems 
> with different system default character   sets, the data will silently change 
> meaning.

Not a problem in my context.

Just to say that using NONE doesn't need to be a problem in all cases.

In general, I think your approach is correct, though.

> Our system will use accented characters and sorting will be important. We 
> will also be accessing data from various locations (differing locales).
>
> I'm Google'ing now for "firebird charset utf8" and reading the various 
> articles. But I'll still be testing FBLib and IBX to see what they do and if 
> they handle it the same as SqlDB.

IBX does, as far as I can gather.

Michael.