[fpc-devel] Re: EBCDIC (was On a port of Free Pascal to, the IBM 370)

[Tomas Hajny]

On Thu, February 2, 2012 18:27, Mark Morgan Lloyd wrote:
> steve smithers wrote:
>>> If you try to achieve a port by modifying all code that deals with
>>> characters you will fail. The amount of work becomes then far too big
>>> for
>>> a single person, and the modifications become too huge and wide-spread
>>> that you will raise objections for merging it with the SVN trunk.
>>
>> That's a good point, but I'm not really suggesting doing it all in one
>> hit.
>>
>>> In other words, do yourself a favour and keep ALL processing in ASCII.
>>> You
>>> can convert between ASCII & EBCDIC on input/output. That way the
>>> modifications in order to support EBCDIC are well concentrated in a
>>> single
>>> piece of code, which can be easily merged without risk of destabilizing
>>> the code base.
>>
>> You can't convert that way, at least not all the time.  There are always
>> going to be mixed string and binary files.
>
> I'd only expect the version control system to contain textfiles. Or are
> you saying that some IBM operating systems support files which contain
> both text and data (I'm aware of the Mac's OS and obviously OS/2 doing
> this, both of which FPC supports).
 .
 .

I guess that Steve meant something different. Regardless from the
underlying platform, a general binary file may still contain strings; they
are just not delimited by line end characters but possibly part of a fixed
structure or using something like ASN.1, etc. The difference is that you
cannot use something like GNU recode (or some equivalent available on
S/390) on such a file to convert it from EBCDIC to ASCII before accessing
such a file with a FPC program because this would break the real binary
parts inevitably. You can't leave the translation on the RTL either (at
least not fully - you can possibly still use some helper, of course, but
you need to take care of that explicitly in your program), because the
typical solution involving BlockRead would not perform any translation. If
you store the read data in a string, they're still EBCDIC. If you start
using RTL string manipulation functions on them (e.g. Val), they will
fail. You can still "convert between ASCII & EBCDIC on input" as suggested
in your previous comment, but you need to do it explicitly in your program
rather than automatically in the RTL.

BTW, this is the part where the Sven's suggestion of translation to
Unicode together with only using Unicode strings within the program
(except for the read buffer) would help; the question is how easy or
difficult is Unicode manager implementation on that "OS".

Tomas