[fpc-pascal] Unicodestrings and Assign(File)

[Jürgen Hestermann]

 > Given that, it doesn't really make sense to use the *A versions at all
 > in Windows (Unless you want to support very old Windows versions).
 > Thus the approach for Windows should be the opposite of Unix, I suppose.
 > UTF8 or ANSI gets converted to UTF16 and call the *W functions.

That's true. In general the data type with the most information 
(UnicodeString
in this case) should be used as much as possible and only when functions
require other (lower) data types a conversion should be done.
Otherwise Information gets lost.



 > I have the same time problem, but I will take a look at the source code,
 > I don't think it should be very difficult to figure out.

I had a look at the sources and found the following
(it's a bit longer, so if you are not interested just stop reading ;-)):



1.) The 2 fundamental file types filerec and textrec (for standard files and
text files) are defined in
fpc\fpc\rtl\inc\filerec.inc and
fpc\fpc\rtl\inc\textrec.inc
Both differ only by some buffer structures at the end but share the 
first part
(including file name). The file name is of type

array[0..255] of char;

and needs to be replaced by UnicodeString.
This requires other changes:


1.) The order of elements in the file data structures have to be changed.
The name has to become the first element as in:
-----------------------------------------------------------
      TextRec = Packed Record
                name      : UnicodeString;
                Handle    : THandle; // Handle has to be directly after 
name, see Assign!
                Mode      : longint;
                bufsize   : SizeInt;
                _private  : SizeInt;
                bufpos,
                bufend    : SizeInt;
                bufptr    : ^textbuf;
                openfunc,
                inoutfunc,
                flushfunc,
                closefunc : pointer;
                UserData  : array[1..32] of byte;
                LineEnd   : TLineEndStr;
                buffer    : textbuf;
                {$ifdef FPC_HAS_CPSTRING}
                CodePage  : TSystemCodePage;
                {$endif}
                End;
-----------------------------------------------------------
Reason: See next item 2.)


2.) The Assign routine in \fpc\rtl\inc\file.inc overwrites the
whole file data structure with zeros (using fillchar). The
name has to be omitted from this action because it is now
managed automatically. So Assign needs to be changed to:

-----------------------------------------------------------
Procedure Assign(out f:File;const NewName:string);
{  Assign Name to file f so it can be used with the file routines  }

Begin // Assign
FillChar(FileRec(f).Handle,SizeOf(FileRec)-Sizeof(FileRec(f).name),0);
FileRec(f).Handle := UnusedHandle;
FileRec(f).mode   := fmClosed;
FileRec(f).Name   := NewName;
End; // Assign
-----------------------------------------------------------


3.) The routine do_open in \fpc\rtl\win\sysfile.inc has to be
changed at 3 lines to (see "// THIS LINE CHANGED!")
-----------------------------------------------------------
procedure do_open(var f; p:UnicodeString; flags:longint);          // 
THIS LINE CHANGED!
{ filerec and textrec have both handle and mode as the first items so
  they could use the same routine for opening/creating.
  when (flags and $100)   the file will be append
  when (flags and $1000)  the file will be truncate/rewritten
  when (flags and $10000) there is no check for close (needed for 
textfiles) }
Const file_Share_Read   = $00000001;
      file_Share_Write  = $00000002;
      file_Share_Delete = $00000004;
Var   shflags,
      oflags,cd : longint;
      security  : TSecurityAttributes;

begin // do_open
DoDirSeparators(p);
{ close first if opened }
if ((flags and $10000)=0) then
   begin
   case filerec(f).mode of
      fminput,fmoutput,fminout :
         Do_Close(filerec(f).handle);
      fmclosed : ;
   else  {not assigned}
         inoutres:=102;
         exit;
   end; // of case
 end;
{ reset file handle }
filerec(f).handle := UnusedHandle;
{ convert filesharing }
shflags := 0;
if ((filemode and fmshareExclusive)=fmshareExclusive) then { no sharing }
else
   if (filemode=fmShareCompat) or ((filemode and 
fmshareDenyWrite)=fmshareDenyWrite) then
      shflags := file_Share_Read
   else
      if ((filemode and fmshareDenyRead)=fmshareDenyRead) then
         shflags := file_Share_Write
      else
         if ((filemode and fmshareDenyNone)=fmshareDenyNone) then
            shflags := {$ifdef WINCE}
                       { WinCE does not know file_share_delete }
                       file_Share_Read or file_Share_Write;
                       {$else WINCE}
                       fmShareDenyNoneFlags;
                       {$endif WINCE}
{ convert filemode to filerec modes }
case (flags and 3) of
   0 : begin
       filerec(f).mode := fminput;
       oflags          := longint(GENERIC_READ);
       end;
   1 : begin
       filerec(f).mode := fmoutput;
       oflags          := longint(GENERIC_WRITE);
       end;
   2 : begin
       filerec(f).mode := fminout;
       oflags          := longint(GENERIC_WRITE or GENERIC_READ);
       end;
   end; // of case
{ create it ? }
if (flags and $1000)<>0 then
   cd:=CREATE_ALWAYS
{ or Append/Open ? }
else
   cd:=OPEN_EXISTING;
{ empty name is special }
if p='' then                           // THIS LINE CHANGED!
   begin
   case FileRec(f).mode of
      fminput :
         FileRec(f).Handle := StdInputHandle;
      fminout, { this is set by rewrite }
      fmoutput :
         FileRec(f).Handle := StdOutputHandle;
      fmappend :
         begin
         FileRec(f).Handle := StdOutputHandle;
         FileRec(f).mode   := fmoutput; {fool fmappend}
         end;
      end; // of case
   exit;
   end;
security.nLength              := Sizeof(TSecurityAttributes);
security.bInheritHandle       := true;
security.lpSecurityDescriptor := nil;
filerec(f).handle := 
CreateFileW(pwidechar(p),oflags,shflags, at security,cd,FILE_ATTRIBUTE_NORMAL,0); 
// THIS LINE CHANGED!
{ append mode }
if ((flags and $100)<>0) and
   (filerec(f).handle<>0) and
   (filerec(f).handle<>UnusedHandle) then
   begin
   do_seekend(filerec(f).handle);
   filerec(f).mode:=fmoutput; {fool fmappend}
   end;
{ get errors }
{ handle -1 is returned sometimes !! (PM) }
if (filerec(f).handle=0) or (filerec(f).handle=UnusedHandle) then
   begin
   errno:=GetLastError;
   Errno2InoutRes;
   end;
end; // do_open
-----------------------------------------------------------



3.) Of course, there are lots of places where the file data structures
filerec and textrec are used in the code. But in most cases only non-changed
record elements are used (so no changes are required) and only seldomly
the name is used/changed or the whole data structure is modified.
So I think the above modifications require to check a lot of code but
only a few modifications (hopefully). I would be willing to do these
changes but I am not sure whether I have overlooked something obvious.
These are very fundamental changes which affect all operating systems
so it may have unexpected side affects.
I also need to get familiar with SVN.

Any comments?