[fpc-devel] TStringList.LoadFromFile and SavetoFile - file encoding support

[Graeme Geldenhuys]

On Tue, Feb 3, 2009 at 9:02 AM, Vincent Snijders
<vsnijders at vodafonevast.nl> wrote:
>
> I am a Lazarus developer, and I don't think I said it like that.

I wasn't pointing fingers to you Vincent. :-) I summarized what a few
people have said.

> LoadFromFile in a LCL control, you need to make sure they are valid UTF8
> strings. And honestly, it is only you who make sure that it is, because you
> know the initial encoding.

The problem is as follows.... Even though I am a long time developer,
I often have no clue what encoding a file is in when I look at the
file using Nautilus file manager. I often open a file in my preferred
text editor, look if it displays correctly, then look in the statusbar
area for what encoding the editor detected (at least my editor does
that nicely).

So even though you are using something as simple as the TMemo in LCL,
and LCL always wants UTF-8, how do you know what encoding to convert
from to UTF-8? If I give you various text files, each using one of the
following schemes: UTF-16, UTF-16BE, and UTF-16LE, UTF-32 and whatever
else I can find. Loading the file into a TStringList and then doing
UTF8Decode on each line.... will it display correctly in the TMemo?

Now what if the memo content is changed and then saved?  How does the
TMemo know which encoding to use (I would preferably like the same
encoding as before, not necessarily UTF-8). So if the file was
originally UTF-32, I don't want it to be UTF-8 afterwards.

If the TStringList.LoadFromFile(...) took a encoding parameter, it
could store that encoding option internally, so if you call
.SaveToFile(somefile.txt) later, it could use the same encoding as
used in LoadFromFile(), otherwise default to something like utf-8 if
no encoding was specified anywhere.

Regards,
  - Graeme -


_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/