[fpc-pascal] Unicode file routines proposal

[Marco van de Voort]

> Marco van de Voort schreef:
> At the borders of my I convert all strings to the 'internal type' and encoding and 
> use it like that. Kind of like we are doing nowadays to convert the line-endings in 
> text files.

I don't like this. This makes e.g. processing a database export on Unix
unnecessarily costly
 
> I see what you are trying to say, but having a string type that is UTF8
> encoded on one system and UTF16 encoded on another system, doesn't seem
> easy to work with to me, even if you name it for example RTLString. 

It should be possible to work in the native encoding. One doesn't want to
wrap _every_ function in _every_ header with conversions procs.

> > Note also that reliance on encoding is way less important, since fewer
> > people will be parsing through strings manually (simply because it is more
> > difficult)
> 
> Right, but they rely on not having to convert it all the time.

Well, they will have to do that with one string type too, at every external
barrier.

That also kills the benefit of choosing UTF-16 in the first place, since
Delphi code won't work on Unix without manually inserting a lot of
conversion code.
 
> ATM, all the client libs above the RTL have chosen one encoding, string type: LCL en 
>   fpGUI: UTF8, MseGui: widestring

That has nothing to do with these decisions. They chose that in the absence
of a good solution. This is about picking a good solution.
 
> So for those libs to interface with a platform dependent string type in
> the LCL, they would have to write platform dependent code.

You will have to anyway for any solution that only supports one encoding.