Redirecting or capturing output from processes

- or How to add achievements in ScummVM.

Have you ever had the need to redirect output from a console application into your application, either for monitoring, integration or "user friendliness"?

I will walk through 2 scenarios - and sprinkle them with a bit of extras.

Getting the window handle of the console application, and then just set the parent to the handle of another windows control, is not redirecting or capturing anything. It just appears that you application has some "nerdy" thing going on - it seems that some people mistrust console windows (read here) - even open source is mistrusted these days (read here), how contradicting it that? :-D

Well back to the code - this:

procedure TForm1.Button1Click(Sender: TObject);
var
  H: HWND;
begin
  H := FindWindow('ConsoleWindowClass', 'C:\Windows\system32\cmd.exe');
  if H<>0 then
  begin
    WinApi.Windows.SetParent(H, Memo1.Handle);
    SetWindowPos(H, 0, 0, 0, ClientWidth, ClientHeight, SWP_ASYNCWINDOWPOS);
  end;
end;

would end up looking something like this:

Not really useful - and remember to terminate the process of the console when you close the form. Notice I did use a TMemo as place holder, you will see why later.

Now to something useful - a useful tool when working with window handles and figuring out what is going on in Windows - Greatis WinDowse.

Back to the scenarios of interest.

Scenario 1: Redirecting STDIN, STDOUT and STDERR from a process created/started by you.

Every process has 3 default handles (not windows handles) - an input, output and error (STDIN, STDOUT and STDERR). So if we as the previous example would like to know the version of the currently installed Java version - a bit hard to keep up with :-D

We first need to set the security attributes and create the pipe read and write handles. Then we need to fill the TStartupInfo record structure, to enable the redirection to our just create pipe handles. And then we create/start the process using the pipes, since the CreateProcess command has the ability to use these.

procedure TForm1.CaptureConsoleOutput(const cmd, param: String; CallBackProc:
  TArg<PAnsiChar>);
const
  secAttr: TSecurityAttributes = (
    nLength: SizeOf(TSecurityAttributes);
    bInheritHandle: True);
  bufSize = 2400;
var
  rPipe: THandle;
  wPipe: THandle;
  suiRec: TStartupInfo;
  piRec: TProcessInformation;
  dRun, dAvail, dRead: DWORD;
  rBuf: array [0..bufSize] of AnsiChar;
begin
  if CreatePipe(rPipe, wPipe, @secAttr, 0) then
  try
    FillChar(suiRec, SizeOf(TStartupInfo), #0);
    suiRec.cb := SizeOf(TStartupInfo);
    suiRec.dwFlags := STARTF_USESTDHANDLES or STARTF_USESHOWWINDOW;
    suiRec.wShowWindow := SW_HIDE;
    suiRec.hStdInput := rPipe;
    suiRec.hStdOutput := wPipe;
    suiRec.hStdError := wPipe;
    if CreateProcess(nil, PChar(cmd+' '+param), @secAttr, @secAttr, True,
      NORMAL_PRIORITY_CLASS, nil, nil, suiRec, piRec) then
    try
      repeat
        dRun := WaitForSingleObject(piRec.hProcess, 100);
        PeekNamedPipe(rPipe, nil, 0, nil, @dAvail, nil);
        if (dAvail > 0) then
        repeat
          dRead := 0;
          ReadFile(rPipe, rBuf[0], bufSize, dRead, nil);
          rBuf[dRead] := #0;
          OemToCharA(rBuf, rBuf);
          CallBackProc(rBuf);
        until (dRead < bufSize);
        Application.ProcessMessages;
      until (dRun <> WAIT_TIMEOUT);
    finally
      CloseHandle(piRec.hProcess);
      CloseHandle(piRec.hThread);
    end;
  finally
    CloseHandle(rPipe);
    CloseHandle(wPipe);
  end;
end;

Notice the use of the anonymous procedure type - defined as:

TArg<T> = reference to procedure(const Arg: T);

Which enables us from the button to call our capture procedure as follows:

CaptureConsoleOutput('java','-version',

      procedure(const Line: PAnsiChar)

      begin

        Memo1.Clear;

        Memo1.Lines.Add(String(Line));

      end);

Thanks goes to Lars Fosdal for that nice approach. Also be aware that if you call CreateProcess with the 2nd parameter as a string const you will get an AV - from the MSDN documentation:

The Unicode version of this function, CreateProcessW, can modify the contents of this string. Therefore, this parameter cannot be a pointer to read-only memory (such as a const variable or a literal string). If this parameter is a constant string, the function may cause an access violation.

So we now ended up with something like this:

You could of course in this case use the normal redirect and pipe commands, and end up with a file, that you then read and parsed - but not very elegant.

Scenario 2: Capturing STDOUT on a process not started by you.

Picture that you want to monitor a console output from a process not started by you or a sub process initiated from the program you are using. Then the previous scenario will not do.

As an example we will use the program ScummVM, monitor its debug output to figur out when certain things happens in a game.

If you do not know what ScummVM is you should go their homepage and download it to your favourite platform, grab one of the freeware games and enjoy the rest of the weekend :-). The team has over the years done a great job, reimplementing the engines of many (all) the old classics - like Monkey Island or Sierras Quests series.

A bit of background: I had just finished off an update of a translation for ScummVM, being ready for the upcoming 1.7.0 version, when I looked at the various command line options, and thought I could have some fun with that.

You can see a video here, of the PoC I did by hooking into the debug output - I did state it as a "fix" - but there is nothing broken in ScummVM - it was a "fix" to the itch I had for getting achievements in some older games - Just to make that clear :-D.

First we need to define a reference to the external function AttachConsole, that for some reason isn't included in the Winapi.Windows unit as I would have expected.

function AttachConsole(dwProcessId: DWORD): BOOL; stdcall; external kernel32 name 'AttachConsole';

We then create a function to return the STDOUT pipe handle, on the given process id for the console application.

function TForm1.AttachAndGetConsolewPipe(ProcessId: Cardinal): Cardinal;
begin
  if not AttachConsole(ProcessId) then
    raise Exception.Create('AttachConsole error: ' + IntToStr(GetLastError));
  Result := GetStdHandle(STD_OUTPUT_HANDLE);
  if Result = INVALID_HANDLE_VALUE then
    raise Exception.Create('GetStdHandle(STD_OUTPUT_HANDLE) error: ' + IntToStr(GetLastError));
end;

And a procedure to free it again.

procedure TForm1.DettachConsole;
begin
  if not FreeConsole then
    raise Exception.Create('FreeConsole error: ' + IntToStr(GetLastError));
end;

Note that since a process can only be associated with 1 console, the FreeConsole needs no process id. You can also take a look at the AllocConsole function.

Now we only need a function that reads the buffer from the STDOUT pipe (wPipe), using the GetConsoleScreenBufferInfo and the ReadConsoleOutput WinAPI functions.

function TForm1.ReadConsole(wPipe: Cardinal): TStringList;
var
  BufInfo: _CONSOLE_SCREEN_BUFFER_INFO;
  BufSize, BufCoord: _COORD;
  ReadRegion: _SMALL_RECT;
  Buf: Array of _CHAR_INFO;
  I, J: Integer;
  Line: AnsiString;
begin
  Result := TStringList.Create;
  ZeroMemory(@BufInfo, SizeOf(BufInfo));
  if not GetConsoleScreenBufferInfo(wPipe, BufInfo) then
    raise Exception.Create('GetConsoleScreenBufferInfo error: ' + IntToStr(GetLastError));
  SetLength(Buf, BufInfo.dwSize.X * BufInfo.dwSize.Y);
  BufSize.X := BufInfo.dwSize.X;
  BufSize.Y := BufInfo.dwSize.Y;
  BufCoord.X := 0;
  BufCoord.Y := 0;
  ReadRegion.Left := 0;
  ReadRegion.Top := 0;
  ReadRegion.Right := BufInfo.dwSize.X;
  ReadRegion.Bottom := BufInfo.dwSize.Y;
  if ReadConsoleOutput(wPipe, Pointer(Buf), BufSize, BufCoord, ReadRegion) then
  begin
    for I := 0 to BufInfo.dwSize.Y - 1 do
    begin
      Line := '';
      for J := 0 to BufInfo.dwSize.X - 1 do
        Line := Line + Buf[I * BufInfo.dwSize.X + J].AsciiChar;
      Result.Add(Line)
    end
  end
  else
    raise Exception.Create('ReadConsoleOutput error: ' + IntToStr(GetLastError));
end;

Finally we can on the 3rd button put the following code:

procedure TForm1.Button3Click(Sender: TObject);
var
  pid: Cardinal;
  sout: Cardinal;
begin
  H := FindWindow('ConsoleWindowClass', 'ScummVM Status Window');
  if H<>0 then
  begin
    GetWindowThreadProcessId(H, pid);
    sout := AttachAndGetConsolewPipe(pid);
    Memo1.Lines := ReadConsole(sout);
    DettachConsole;
  end;
end;

Then before we try our code, we want - as an example start scummvm.exe with the following parameters: -d6 queen (which requires that you have download ScummVM and the game "Flight of the Amazon Queen", and added the game), any other process would do, just figure out the window handle of the console window, and change the title parameter of the last FindWindow call.

We will stop for now, but using the ShowWindow call and a timer you could hide the console window, and continuous update your output memo, searching the text for hints/events - a simple pos function is probably the fastest.

In the video I originally used the AnimateWindow function to slide a "achievement" form (FormStyle=fsStayOnTop) in from the right, but the ScreenRecorder software wanted to be on the top, so....

Go and support the ScummVM project, and buy some of the classic games from gog.com through their site. BTW: You can also buy or upgrade any EMBT product by clicking the ad in the top right corner of my blog. :-)

Links:

Source code: RedirCMD