[fpc-devel] Standard in/out/error of Windows GUI apps

[Henry Vermaak]

Hello,

Recently a customer asked whether he could get a GUI version of one of
our console apps because the flashing console is annoying him.  Fair
enough, I thought, this should be easy.  We do it with our C utilities
already.

I know that a writeln() in a GUI app crashes, from when someone left a
stray writeln() in FPC 3.0.x and our app started crashing when looking
for updates online.  Can someone explain to me what the use of this is?
If you know that a handle doesn't exist, why not just redirect the
output to 'nul'?

I set out to do this myself, when I discovered that fpc tries to show
anything that you write to stdout/stderr in a messagebox.  It does this
even when the handles are being redirected (to e.g. pipes) by the
calling application.  Relevant code in syswin.inc (procedure
SysInitStdIO):

if not IsConsole then
 begin
  AssignError(stderr);
  AssignError(StdOut);
  Assign(Output,'');
  Assign(Input,'');
  Assign(ErrOutput,'');
 end
...
...

Showing writeln output in a messagebox is questionable, but it doesn't
even do it for Output and ErrOutput.  The documentation says they are
aliases for stdout/stderr?  So not only does it ignore whichever
possibly valid handles in Std{Output|Input|Error}Handle, but it also
leaves ErrOutput pointing to StdOutputHandle.

For anyone interested, this is what I had to do so that a command like
"app > stdout.txt 2> stderr.txt" behaves as expected:

{$ifdef windows}
function CanOpenHandle(AHandle: THandle): Boolean;
begin
  Result := GetFileType(AHandle) in [FILE_TYPE_DISK, FILE_TYPE_PIPE, FILE_TYPE_CHAR];
end;

procedure RedirectStdIO(var AFile: Text; AHandle: THandle);
var
  CanOpen: Boolean;
begin
  CanOpen := CanOpenHandle(AHandle);
  AssignFile(AFile, IfThen(CanOpen, '', 'nul'));

  if AHandle = StdOutputHandle then begin
    Rewrite(AFile);
  end else if AHandle = StdErrorHandle then begin
    Rewrite(AFile);
    { Rewriting a file with a '' name will use StdOutputHandle, but we want
      StdErrorHandle, so hack it in. }
    if CanOpen then
      TextRec(AFile).Handle := AHandle;
  end else if AHandle = StdInputHandle then begin
    Reset(AFile);
  end;
end;

procedure RedirectWinGUIStdIO;
begin
  if IsConsole then
    Exit;

  RedirectStdIO(StdOut, StdOutputHandle);
  RedirectStdIO(Output, StdOutputHandle);

  RedirectStdIO(StdErr, StdErrorHandle);
  RedirectStdIO(ErrOutput, StdErrorHandle);

  RedirectStdIO(Input, StdInputHandle);
end;
{$endif}

Then call RedirectWinGUIStdIO at the start.  I hope it works for all
possibilities, my Windows skills are lacking.

Henry