[fpc-devel] ansistrings and widestrings

[peter green]

in wondows terminology (which i presume is where the name ansistring comes
from) the windows code page which is often refered to in documentation as
the ansi code page CAN be multi byte.

http://www.microsoft.com/globaldev/reference/WinCP.mspx

more generally i belive an ansistring is usually intended to represent text
in the platforms local encoding. Whilst a widestring is meant to represent
text in utf-16.

The platforms local encoding may be a single byte encodeing (iso-8859-?
windows-125? etc) it may be a legacy mixed width encoding (EUC-?? SHIFT-JIS
BIG5 etc) or it may be a unicode transformation format which is a superset
of ascii (utf-8).

now for dependency reasons i belive that the default conversion functions
should remain a "dumb fallback" BUT i also belive that the function
prototypes should be designed in such a way as to allow the conversion
routines to be replaced with ones that can sesiblly handle the local
encoding.

i've created a page on the wiki for this issue at
http://www.freepascal.org/wiki/index.php/Widestrings


> -----Original Message-----
> From: fpc-devel-bounces at lists.freepascal.org
> [mailto:fpc-devel-bounces at lists.freepascal.org]On Behalf Of Alexey
> Barkovoy
> Sent: 05 January 2005 19:55
> To: FPC developers' list
> Subject: Re: [fpc-devel] ansistrings and widestrings
>
>
> Well functions are called ANSI to unicode and vice versa. ANSI is
> always single
> byte; by unicode people usually refer to utf16, not multibyte
> encoding and both
> Delphi and FPC define WideString as double byte strings. So semantically
> functions do that is required. IMHO then assigning widestring to
> ansistring
> noone should expect multibyte encoded result. Then you need utf-8
> you should
> call special functions.
>
> ----- Original Message -----
> From: "peter green" <plugwash at bircd.org>
> To: "FPC developers' list" <fpc-devel at lists.freepascal.org>
> Sent: Wednesday, January 05, 2005 8:32 PM
> Subject: RE: [fpc-devel] ansistrings and widestrings
>
>
> > ok i see a MAJOR problem with the semantics of those functions.
> >
> > they assume that one widechar is equivilent to one ansichar (that is the
> > source count of widechars will equal the destination count of
> ansichars or
> > the source count of widechars will equal the destination count of
> > ansichars).
> >
> > this is simply not the case for many encodings. (utf-8 sjis euc
> to name just
> > a few)
> >
> >
> >> -----Original Message-----
> >> From: fpc-devel-bounces at lists.freepascal.org
> >> [mailto:fpc-devel-bounces at lists.freepascal.org]On Behalf Of Michael Van
> >> Canneyt
> >> Sent: 05 January 2005 16:11
> >> To: FPC developers' list
> >> Subject: RE: [fpc-devel] ansistrings and widestrings
> >>
> >>
> >> On Wed, 5 Jan 2005, peter green wrote:
> >>
> >> > where are theese default versions located in the code?
> >> >
> >>
> >> In the inc directory of the rtl. wstrings.inc
> >>
> >> procedure Wide2AnsiMove(source:pwidechar;dest:pchar;len:SizeInt);
> >> procedure Ansi2WideMove(source:pchar;dest:pwidechar;len:SizeInt);
> >>
>
>
> _______________________________________________
> fpc-devel maillist  -  fpc-devel at lists.freepascal.org
> http://lists.freepascal.org/mailman/listinfo/fpc-devel