[fpc-pascal] code example where AnsiString used in FCL (SqlDB) causes data loss

[Graeme Geldenhuys]

Hi,

Here is an example [proof if you will] of the problem. I wrote a small
test program that reads data from a Firebird database where the database
and field charset is set to UTF8.

I compile the program, then run it. No recompiles between the two runs.
The first run my system is set to have a UTF-8 locale. The second run is
where I set my system to have a ISO8859-1 (Latin-1) locale. The program
outputs the DefaultSystemCodePage to the console.

Because the locale changes the behaviour of String (aka AnsiString) in
the RTL and FCL, the first run works, but the second run corrupts my data.

Console output:

[unicode_test]$ export LANG=en_US.UTF-8
[unicode_test]$ ./unicodetest
65001

[unicode_test]$ export LANG=en_US.ISO8859-1
[unicode_test]$ ./unicodetest
28591


In my test program I write the data read from the database to a file
using TFileStream, thus console and file encoding settings will not
affect the data being written to file. TFileStream is simply writing bytes.

The “locale_utf8.png” screenshots shows the actual data in the database
on the left, and the read (and saved to a file “output.data”) data on
the right

Compiled with 64-bit FPC 3.0.1 (updated yesterday) on my FreeBSD 10.3
system. Firebird v2.5.4 is being used. I can supply a backup of the test
Firebird database too if needed - it is small.


I'm honestly trying very hard to understand the string changes
implemented in FPC 3.x, and the best way to use it going forward. In
this example I tried everything I learned from the recent mailing list
discussions. My concern with the usage of String/AnsiString still
stands, as this test program shows.

Regards,
  Graeme

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: project1.lpr
Type: text/typhon-project-source
Size: 1618 bytes
Desc: not available
URL: <http://lists.freepascal.org/pipermail/fpc-pascal/attachments/20160511/82eb6047/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: locale_utf8.png
Type: image/png
Size: 28941 bytes
Desc: not available
URL: <http://lists.freepascal.org/pipermail/fpc-pascal/attachments/20160511/82eb6047/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: locale_iso8859-1.png
Type: image/png
Size: 9827 bytes
Desc: not available
URL: <http://lists.freepascal.org/pipermail/fpc-pascal/attachments/20160511/82eb6047/attachment-0001.png>