[fpc-pascal] Unicode chars losing information

[Michael Van Canneyt]

On Mon, 8 Mar 2021, Adriaan van Os via fpc-pascal wrote:

> Michael Van Canneyt wrote:
>> 
>> The output for me is the same, regardless of the -FcUTF-8 flag being 
>> present
>> or not: question marks.
>> 
>> But if I add
>> 
>> uses cwstring;
>> 
>> all will be well.
>> 
>> Rationale:
>> Without that, the RTL cannot convert whatever the compiler wrote in
>> the binary to UTF8 to display it on the console.
>> 
>> The compiler people will need to explain what exactly the compiler writes
>> with or without the flag.
>
> Well, this should at least produce a warning, if not an error. Silently 
> producing the wrong code is  not a good idea.

Strictly speaking, there is no wrong code produced:

You didn't configure your environment to deal correctly with Unicode.
You're using the default widestring manager, which simply skips any non-ascii
characters.

All this is documented in various places, for example:

https://www.freepascal.org/docs-html/rtl/system/unicodesupport.html

Michael.