[fpc-pascal] Delphi / FPC and UTF8 BOM

[Jonas Maebe]

On 21 Oct 2008, at 19:29, Felipe Monteiro de Carvalho wrote:

> There has been a lot of discussion about this problem. What happens is
> that FPC wishes to always have ansistrings holding system locale
> encoded strings, it's impossible to have strings which store utf-8
> data as far as FPC is concerned.

And the reason is that
a) if you mix system and non-system encodings in ansistrings, then a  
bunch of string conversions between ansistrings and widestrings will  
go horribly wrong
b) if you only use a particular non-system encoding for ansistrings,  
then interfacing with OS routines will break down completely

It is possible to solve b) by manually adding necessary extra string  
conversions everywhere in the RTL where ansistrings are passed to OS  
routines, but that is a lot of work (both to implement and to  
maintain) and very error prone. Then it's indeed much cleaner to  
simply introduce a new string type which does not have to be  
compatible with the OS encoding.


Jonas