Job Objects

- or How to bundle and control your (child) processes better.

This is partly a follow up to my old post on processes - found here, and an introduction to Job Objects.

A Job Object is a container where you can attach processes to, so that they are controlled within the same context.

To see what processes consist of Job Objects, we could install Process Explorer from Sysinternals from here. 

After installation Jobs by default are not shown, so you must select "Options|Configure Colors...", and enable the Jobs to be shown.

We can see that more and more applications tend to contain more and more processes and bundle these up - browsers and applications like Teams springs to mind.

The list of benefits of a Job Object is long:

- Allow a group of processes to be managed as a one.

- Starting and terminating related external processes.

- They can be named. 

- If main process dies, Windows will terminate the child processes.

- Can be nested.

- Accounting info – I/O, CPU usage

- More and more used – since more application consists of many processes

- They can impose limits on its processes.

- CPU: Maximum processes active, time, Affinity, rate control

- Memory: minimum and maximum working set, commit maximum

- Network: Maximum bandwidth 

- I/O: Maximum rate, read and write bytes 

- UI: User and GDI handles, clipboard access, exiting windows, desktop switching/creation

Below is an example of a Job with limitations:

Example

To illustrate a Job Object I want my "main" application to create a Job Object, create a Process and attach that to the Job Object - I want to pass a callback to get the output from the STDERR steam and if the "child" terminates I also want to be notified.

In the FormCreate I do create an instance of TChildThread - more on that later - with the parameters of the command of the process to run and the callback procdure:

var

  ChildExeName: string;

begin

  // Start Child process and attach to this process via Job Object

  ChildExeName := TPath.Combine(ExtractFilePath(ParamStr(0)), 'ChildProcessC.exe');

  if FileExists(ChildExeName) then

  begin

    ChildExeName := '"'+ChildExeName+'"';

    System.UniqueString(ChildExeName);

    ChildThread := TChildThread.Create(ChildExeName, LogChildError);

    ChildThread.OnTerminate := DidChildTerminateUnexpected;

  end

  else

    memLog.Lines.Add('Error: Not able to find any child process exe to start.');

The OnTerminate event is assigned to the procedure that notifies me of an unexpected termination.

And in the FormDestroy I terminated the child process and close its ProcessInfo handles. Remembering to set the onTerminate to nil prior - since this is the expected termination:

  if Assigned(ChildThread) and (ChildThread.ProcessInfo.hProcess>0) then

  begin

    ChildThread.OnTerminate := nil;

    TerminateProcess(ChildThread.ProcessInfo.hProcess, 0);

    ChildThread.Free;

  end;

Back to the TChildThread:

  TChildThread = class(TThread)

  const

    bufSize = 2400;

  private

    FReadBuf: array [0..bufSize] of AnsiChar;

    FCmd: string;

    FCallbackProc: TOnCaptureProc;

    FProcessInfo: TProcessInformation;

    function GethProcess: TProcessInformation;

    procedure SendLogMsg;

  public

    constructor Create(const cmd: string; CallBackProc: TOnCaptureProc);

    destructor Destroy; override;

    procedure Execute; override;

    property ProcessInfo: TProcessInformation read GethProcess;

  end;

and the callback is of type:

TOnCaptureProc = reference to procedure(const Value:string);

The most interesting thing here is the Execute procedure:

procedure TChildThread.Execute;

const

  secAttr: TSecurityAttributes = (

    nLength: SizeOf(TSecurityAttributes);

    bInheritHandle: True);

var

  rPipe: THandle; // Could PeekNamedPipe this for normal console info

  wPipe: THandle;

  erPipe: THandle; // STDERR pipe

  ewPipe: THandle;

  startupInfo: TStartupInfo;

  dRun, dAvail, dRead: DWORD;

  jobObject: NativeUInt;

  jobLimitInfo: TJobObjectExtendedLimitInformation;

begin

  inherited;

  if CreatePipe(rPipe, wPipe, @secAttr, 0) and 

     CreatePipe(erPipe, ewPipe, @secAttr, 0) then

  try

    FillChar(startupInfo, SizeOf(TStartupInfo), #0);

    startupInfo.cb := SizeOf(TStartupInfo);

    startupInfo.dwFlags := STARTF_USESTDHANDLES or STARTF_USESHOWWINDOW;

    startupInfo.wShowWindow := SW_HIDE;

    startupInfo.hStdInput := rPipe;

    startupInfo.hStdOutput := wPipe;

    startupInfo.hStdError := ewPipe;

After having create the pipes to capture the STDERR (and STDIN and STDOUT), and set these in the StartupInfo structure, we create the Job Object and set some "Limits" for the Job Object.

    jobObject := CreateJobObject(nil, PChar(Format('Global\%d', [GetCurrentProcessID])));

    if jobObject <> 0 then

    begin

      jobLimitInfo.BasicLimitInformation.LimitFlags := JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;

      SetInformationJobObject(JobObject, JobObjectExtendedLimitInformation, @jobLimitInfo, SizeOf(TJobObjectExtendedLimitInformation));

    end;

Then we create the process and assign it to our Job Object

    if CreateProcess(nil, PChar(FCmd), @secAttr, @secAttr, True,   // !!!!

      CREATE_BREAKAWAY_FROM_JOB, nil, nil, startupInfo, FProcessInfo) then

    try

      if FProcessInfo.hProcess <> INVALID_HANDLE_VALUE then

        AssignProcessToJobObject(jobObject, FProcessInfo.hProcess);

And then we read from STDERR, and send any data back via the callback. 

      repeat

     dRun := WaitForSingleObject(FProcessInfo.hProcess, 100);

     PeekNamedPipe(erPipe, nil, 0, nil, @dAvail, nil);

     if (dAvail > 0) then

     repeat

       dRead := 0;

       ReadFile(erPipe, FReadBuf[0], bufSize, dRead, nil);

       FReadBuf[dRead] := #0;

       OemToCharA(FReadBuf, FReadBuf);

       Synchronize(SendLogMsg);

     until (dRead < bufSize);

   until (dRun <> WAIT_TIMEOUT);

And after termination, clean up and close the various handles.

    finally

      CloseHandle(FProcessInfo.hProcess);

      CloseHandle(FProcessInfo.hThread);

    end;

  finally

    CloseHandle(rPipe);

    CloseHandle(wPipe);

    CloseHandle(erPipe);

    CloseHandle(ewPipe);

  end;

end;

One important thing is to remember to set the Inheritable Handles to True - otherwise the handles to the "pipe" streams might not be what you expect - you want them typically to be the same as your "main" process.

Also note, that prior to Delphi 12, a needed Windows API const was missing, so you had to declare that yourself within your code:

const

  CREATE_BREAKAWAY_FROM_JOB       = $01000000;

  {$EXTERNALSYM CREATE_BREAKAWAY_FROM_JOB}

Documentation on Job Objects and of all the limit flag and other parameters found here: https://learn.microsoft.com/en-us/windows/win32/procthread/job-objects

Update: There will be a session at CodeRage 2025 - will put the link here when a replay is avaiable.

/Enjoy