[fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

[Dennis Poon]

> I have followed steps in the Wiki, etc., but to little avail, so I 
> have some questions for anyone who knows more than me:
> 1. What encoding "should" I be writing to the terminal?  from 
> experimenting with text files using the cat command in powershell, it 
> seems that local ("ANSI") encoding should be used.  This makes sense 
> since older versions of windows only supported local encodings.
> 2. Is there any reason why writing out data in the local encoding 
> (with write statements, etc.) should get corrupted?  For example is 
> some level of the RTL assuming something about the encoding? (I don't 
> think so, but...)
> 3. Is there a way to set the output to UTF8 so I can just write out 
> UTF8 and be done with it?
>
> Just to give an example:
> 1. I read in an SJIS CSV file, and display it on the screen, and it's 
> corrupted.
> 2. I convert it to UTF8 before displaying it, and it's still corrupted.
> 3.I cat the file to the screen and it's ok.
> 4. I write the output to a file instead of the console, and it's ok.
>

Please state the windows version you are using. XP or Windows 7?  I deal 
with chinese in my programs so I know your problems. The same delphi 5 
program works differently on XP and Windows 7.  Looks like Windows 7 has 
removed support for non unicode (I am not sure whether the Unicode it 
uses is UTF8, UTF16 or UTF32).

Seems that all filenames in XP are treated as unicode code. If you type 
a non unicode file name in Explorer, it will be auto converted to unicode.
Also, in XP, when text is copied to MS Office from other programs and 
vice versa, XP seems to do an automatic conversion to UTF-8 and vice versa.
That is if you copy some text in your program which is encoding in SJIS 
and paste it to Word, the text seems to be auto converted to unicode.

As for file handling, it seems some programs will read the first 2 bytes 
of the text file to determine the the encoding of the file. Google about it.

Sorry, I don't have exact answers to your questions. Just share some of 
my experience.

Dennis