[fpc-devel] UTF8 support for win\process.inc

[Michael Thompson]

G'day,

I'm on Windows 8/Trunk FPC

Been banging my head against the wall all weekend trying to get some UTF8
working with the LazUtils TProcessUTF8.  By the end of that I realized no
matter what I did there, I wasn't going to get anywhere.

I've now solved my problem using only TProcess.  See below for the modified
TProcess.Execute (win\process.inc)

Procedure TProcess.Execute;
Var
  i : Integer;
  PName,PDir,PCommandLine : PWideChar;     // Changed from PChar
  FEnv: pointer;
  FCreationFlags : Cardinal;
  FProcessAttributes : TSecurityAttributes;
  FThreadAttributes : TSecurityAttributes;
  FProcessInformation : TProcessInformation;
  FStartupInfo : STARTUPINFO;
  HI,HO,HE : THandle;
  Cmd : String;

begin
  PName:=Nil;
  PCommandLine:=Nil;
  PDir:=Nil;

  if (FApplicationName='') and (FCommandLine='') and (FExecutable='') then
    Raise EProcess.Create(SNoCommandline);
  if (FApplicationName<>'') then
    begin
    PName:=PWideChar(UTF8Decode(FApplicationName));   // Added UTF8Decode
    PCommandLine:=PWideChar(UTF8Decode(FCommandLine));// Added UTF8Decode
    end
  else If (FCommandLine<>'') then
    PCommandLine:=PWideChar(UTF8Decode(FCommandLine)) // Added UTF8Decode
  else if (Fexecutable<>'') then
    begin
    Cmd:=MaybeQuoteIfNotQuoted(Executable);
    For I:=0 to Parameters.Count-1 do
      Cmd:=Cmd+' '+MaybeQuoteIfNotQuoted(Parameters[i]);
    PCommandLine:=PWideChar(UTF8Decode(Cmd));         // Added UTF8Decode
    end;
  If FCurrentDirectory<>'' then
    PDir:=PWideChar(UTF8Decode(FCurrentDirectory));   // Added UTF8Decode
  if FEnvironment.Count<>0 then
    FEnv:=StringsToPChars(FEnvironment)
  else
    FEnv:=Nil;
  Try
    FCreationFlags:=GetCreationFlags(Self);
    InitProcessAttributes(Self,FProcessAttributes);
    InitThreadAttributes(Self,FThreadAttributes);
    InitStartupInfo(Self,FStartUpInfo);
    If poUsePipes in FProcessOptions then
      CreatePipes(HI,HO,HE,FStartupInfo,Not(poStdErrToOutPut in
FProcessOptions), FPipeBufferSize);
    Try
      If Not
CreateProcessW(PName,PCommandLine, at FProcessAttributes, at FThreadAttributes,
                   FInheritHandles,FCreationFlags,FEnv,PDir,FStartupInfo,
                   fProcessInformation) then
             // Changed to CreateProcessW
        Raise
EProcess.CreateFmt(SErrCannotExecute,[FCommandLine,GetLastError]);
      FProcessHandle:=FProcessInformation.hProcess;
      FThreadHandle:=FProcessInformation.hThread;
      FProcessID:=FProcessINformation.dwProcessID;
    Finally
      if POUsePipes in FProcessOptions then
        begin
        FileClose(FStartupInfo.hStdInput);
        FileClose(FStartupInfo.hStdOutput);
        if Not (poStdErrToOutPut in FProcessOptions) then
          FileClose(FStartupInfo.hStdError);
        CreateStreams(HI,HO,HE);
        end;
    end;
    FRunning:=True;
  Finally
    If FEnv<>Nil then
      FreeMem(FEnv);
  end;
  if not (csDesigning in ComponentState) and // This would hang the IDE !
     (poWaitOnExit in FProcessOptions) and
      not (poRunSuspended in FProcessOptions) then
    WaitOnExit;
end;

Basically I changed 3 x PChar to PWideChar, then when the PWideChars are
being set I do it via PWideChar(UTF8Decode(<string>))
Finally I changed CreateProcess to CreateProcessW.

I had thought I was going to need to change the datatypes for FCommandline
etc, but this wasn't the case.

With the limited testing I've done, this seems to successfully enable UTF8
support within TProcess, without the end user needing to change any of
their code.  Also, this doesn't seem to break any existing code that uses
TProcess.

Before I submit this as a patch though, I have concerns.  Mainly around a)
not knowing your development plans in this area and b) me not really
knowing unicode (just because this works, doesn't mean it's right).

A summary of my concerns:
* I note in redef.inc there is already a redirect for CreateProcess to
CreateProcessW.  This implies to me there is already a plan in mind, and I
might be cutting across that by directly calling CreateProcessW.
* My code doesn't take UTF16 into account.  In truth, I'm not sure how to
do that.
* I've only done limited testing on a Win 8 64bit PC set to Australian
regional settings.  Really I think I need to test this on a wide variety of
regional settings and across a variety of Win OS's...

Before I submit a patch based on this can anyone tell me if I'm on the
wrong track? (and if so, where)

Many thanks

Mike Thompson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freepascal.org/pipermail/fpc-devel/attachments/20140701/7a061e80/attachment.html>