[fpc-pascal] Re: UTF8 JSON library: was: [Lazarus] UTF-8 XML

[Luiz Americo Pereira Camara]

Em 24/6/2012 12:22, Reinier Olislagers escreveu:
> On 24-6-2012 17:05, Luiz Americo Pereira Camara wrote:
>
>> Hi,
>>
>> Thanks for your work. I plan to look at OAuth library soon
> No problems, if you have questions, please feel free to ask.

I will

>
>> About fpjson and UTF8, can you post an example showing the problem?
> The symptom is that the tweets, which are UTF8, get converted to my ANSI
> codepage on Windows. This means e.g. Greek or Japanese tweets can't be
> displayed or stored correctly.
>
> As mentioned in
> https://bitbucket.org/reiniero/fpctwit/src/efde73d5b1b3/fpjsonreadme.TXT
> I applied your patch from bug 22197.

I already read but got curious since that change was not supposed to 
handle this kind of situation

> Then, adapted line jsonscannerutf8.pp line 244 (FPC 2.7.1 jsonscanner.pp
> line 238/239), function DoFetchToken
> to convert incoming json string data to UTF8 instead of system codepage:
> e.g.
> from:
> // Takes care of conversion...
> S:=WideChar(StrToInt('$'+S));
> to:
> // Convert from Unicode codepoint in hex to UTF8... via UTF16:
> S:=Utf8Encode(WideString(WideChar(StrToInt('$'+S))));

This seems correct

>> Maybe i can help.
> It's working for me right now ;) Just wanted to note that I haven't
> tested converting objects to JSON output, just JSON strings to objects.
>
> Of course, if you have improvement suggestions, they're very welcome.

I isolated the problem and reported the issue. See 
http://bugs.freepascal.org/view.php?id=22310

Luiz