Job Objects

- or How to bundle and control your (child) processes better.

This is partly a follow up to my old post on processes - found here, and an introduction to Job Objects.

A Job Object is a container where you can attach processes to, so that they are controlled within the same context.

To see what processes consist of Job Objects, we could install Process Explorer from Sysinternals from here. 

After installation Jobs by default are not shown, so you must select "Options|Configure Colors...", and enable the Jobs to be shown.

We can see that more and more applications tend to contain more and more processes and bundle these up - browsers and applications like Teams springs to mind.

The list of benefits of a Job Object is long:

- Allow a group of processes to be managed as a one.

- Starting and terminating related external processes.

- They can be named. 

- If main process dies, Windows will terminate the child processes.

- Can be nested.

- Accounting info – I/O, CPU usage

- More and more used – since more application consists of many processes

- They can impose limits on its processes.

- CPU: Maximum processes active, time, Affinity, rate control

- Memory: minimum and maximum working set, commit maximum

- Network: Maximum bandwidth 

- I/O: Maximum rate, read and write bytes 

- UI: User and GDI handles, clipboard access, exiting windows, desktop switching/creation

Below is an example of a Job with limitations:

Example

To illustrate a Job Object I want my "main" application to create a Job Object, create a Process and attach that to the Job Object - I want to pass a callback to get the output from the STDERR steam and if the "child" terminates I also want to be notified.

In the FormCreate I do create an instance of TChildThread - more on that later - with the parameters of the command of the process to run and the callback procdure:

var

  ChildExeName: string;

begin

  // Start Child process and attach to this process via Job Object

  ChildExeName := TPath.Combine(ExtractFilePath(ParamStr(0)), 'ChildProcessC.exe');

  if FileExists(ChildExeName) then

  begin

    ChildExeName := '"'+ChildExeName+'"';

    System.UniqueString(ChildExeName);

    ChildThread := TChildThread.Create(ChildExeName, LogChildError);

    ChildThread.OnTerminate := DidChildTerminateUnexpected;

  end

  else

    memLog.Lines.Add('Error: Not able to find any child process exe to start.');

The OnTerminate event is assigned to the procedure that notifies me of an unexpected termination.

And in the FormDestroy I terminated the child process and close its ProcessInfo handles. Remembering to set the onTerminate to nil prior - since this is the expected termination:

  if Assigned(ChildThread) and (ChildThread.ProcessInfo.hProcess>0) then

  begin

    ChildThread.OnTerminate := nil;

    TerminateProcess(ChildThread.ProcessInfo.hProcess, 0);

    ChildThread.Free;

  end;

Back to the TChildThread:

  TChildThread = class(TThread)

  const

    bufSize = 2400;

  private

    FReadBuf: array [0..bufSize] of AnsiChar;

    FCmd: string;

    FCallbackProc: TOnCaptureProc;

    FProcessInfo: TProcessInformation;

    function GethProcess: TProcessInformation;

    procedure SendLogMsg;

  public

    constructor Create(const cmd: string; CallBackProc: TOnCaptureProc);

    destructor Destroy; override;

    procedure Execute; override;

    property ProcessInfo: TProcessInformation read GethProcess;

  end;

and the callback is of type:

TOnCaptureProc = reference to procedure(const Value:string);

The most interesting thing here is the Execute procedure:

procedure TChildThread.Execute;

const

  secAttr: TSecurityAttributes = (

    nLength: SizeOf(TSecurityAttributes);

    bInheritHandle: True);

var

  rPipe: THandle; // Could PeekNamedPipe this for normal console info

  wPipe: THandle;

  erPipe: THandle; // STDERR pipe

  ewPipe: THandle;

  startupInfo: TStartupInfo;

  dRun, dAvail, dRead: DWORD;

  jobObject: NativeUInt;

  jobLimitInfo: TJobObjectExtendedLimitInformation;

begin

  inherited;

  if CreatePipe(rPipe, wPipe, @secAttr, 0) and 

     CreatePipe(erPipe, ewPipe, @secAttr, 0) then

  try

    FillChar(startupInfo, SizeOf(TStartupInfo), #0);

    startupInfo.cb := SizeOf(TStartupInfo);

    startupInfo.dwFlags := STARTF_USESTDHANDLES or STARTF_USESHOWWINDOW;

    startupInfo.wShowWindow := SW_HIDE;

    startupInfo.hStdInput := rPipe;

    startupInfo.hStdOutput := wPipe;

    startupInfo.hStdError := ewPipe;

After having create the pipes to capture the STDERR (and STDIN and STDOUT), and set these in the StartupInfo structure, we create the Job Object and set some "Limits" for the Job Object.

    jobObject := CreateJobObject(nil, PChar(Format('Global\%d', [GetCurrentProcessID])));

    if jobObject <> 0 then

    begin

      jobLimitInfo.BasicLimitInformation.LimitFlags := JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;

      SetInformationJobObject(JobObject, JobObjectExtendedLimitInformation, @jobLimitInfo, SizeOf(TJobObjectExtendedLimitInformation));

    end;

Then we create the process and assign it to our Job Object

    if CreateProcess(nil, PChar(FCmd), @secAttr, @secAttr, True,   // !!!!

      CREATE_BREAKAWAY_FROM_JOB, nil, nil, startupInfo, FProcessInfo) then

    try

      if FProcessInfo.hProcess <> INVALID_HANDLE_VALUE then

        AssignProcessToJobObject(jobObject, FProcessInfo.hProcess);

And then we read from STDERR, and send any data back via the callback. 

      repeat

     dRun := WaitForSingleObject(FProcessInfo.hProcess, 100);

     PeekNamedPipe(erPipe, nil, 0, nil, @dAvail, nil);

     if (dAvail > 0) then

     repeat

       dRead := 0;

       ReadFile(erPipe, FReadBuf[0], bufSize, dRead, nil);

       FReadBuf[dRead] := #0;

       OemToCharA(FReadBuf, FReadBuf);

       Synchronize(SendLogMsg);

     until (dRead < bufSize);

   until (dRun <> WAIT_TIMEOUT);

And after termination, clean up and close the various handles.

    finally

      CloseHandle(FProcessInfo.hProcess);

      CloseHandle(FProcessInfo.hThread);

    end;

  finally

    CloseHandle(rPipe);

    CloseHandle(wPipe);

    CloseHandle(erPipe);

    CloseHandle(ewPipe);

  end;

end;

One important thing is to remember to set the Inheritable Handles to True - otherwise the handles to the "pipe" streams might not be what you expect - you want them typically to be the same as your "main" process.

Also note, that prior to Delphi 12, a needed Windows API const was missing, so you had to declare that yourself within your code:

const

  CREATE_BREAKAWAY_FROM_JOB       = $01000000;

  {$EXTERNALSYM CREATE_BREAKAWAY_FROM_JOB}

Documentation on Job Objects and of all the limit flag and other parameters found here: https://learn.microsoft.com/en-us/windows/win32/procthread/job-objects

Update: There will be a session at CodeRage 2025 - will put the link here when a replay is avaiable.

/Enjoy

IAM Cloak, without the Dagger

 - or using a Keycloak setup as your Authentication solution in your native Delphi application.

With no reference to melodramatic intrigues, espionage, secret agents or a Marvel crime-fighting team involving experimental drugs, this post will just try to enlighten how fairly easy it is to get started with Keycloak as an authentication backend for your Delphi application.

This post is a continuation of some earlier post about IAM related stuff and my Auth component samples for PingOne, EntraID and now Keycloak on GitHub.

The posts can be found here:

• IAP Client, therefore IAM
• Just Ping someone!

GitHub gist/repos:

• IAPClient
• PingOneAuth
• EntraIDAuth
• KeycloakAuth

The EntraIDAuth was used in my Delphi Summit 2024 session: I can, therefore IAM.

Keycloak

Keycloak is a mature Open Source IAM solution, and I am by not means an expert - but I have been dabbling with some of the other provides - and this is as good as any.

The easiest way to get started is to install a docker container as described in the official documentation here.

There are other options and a lot more to it, but that short guide has you started, and we use that sample as the basis for the KeycloakAuth component, found in the last repo link above. Note that at the time of writing this post, it does only parse an access token for the "uniqueid", and the state parameter is by default added.

The state value is a GUID (or should I say UUID) - so to get "proper" UUID, I end up doing this:

if FUseState then
begin
  FStateValue := TGUID.NewGuid;
  URL := URL + '&state=' + GUIDToString(FStateValue).Trim(['{','}']).ToLower;
end;

Using the state parameter in the authorization code flow, helps to check that the client value sent matches the value in the response - to mitigate CSRF/XSRF (Cross-site request forgery) attacks. After that initial check one should then check the nonce given within the id_token.

But getting ahead of myself - to use and play around with my KeycloakAuth component - get it from GitHub as mentioned above - and install the component in the IDE (Delphi 12 package in repo).

As any of my other Auth components, this is based of TWebBrowser - and I hear you ask why not TEdgeBrowser? Well I had done some things similar years back before Edge was a thing, but since I do strongly suggest that the "SelectedEngine" property is set to "EdgeOnly" - TWebBrowser does internally use a TEdgeBrowser. It does mean you do need to install the Edge WebView2 SDK, and deploy the correct dll. Read all about it here.

Create a new VCL application and drop a KeycloakAuth component, align to client and set the following properties:

    RealmName = 'myrealm'
    AuthPath = 'http://localhost:8080/realms/'
    ClientId = 'myclient'
    RedirectUri = 'http://localhost:1234'
    AuthEndpoint = '/protocol/openid-connect/auth'
    TokenEndpoint = '/protocol/openid-connect/token'
    Scope = 'openid'
    UseState = False
    ResponseType = 'code'
    UserIdClaim = sub
    OnAuthenticated = KeycloakAuthenticated
    OnDenied = KeycloakDenied

Add an OnAuthenticated and OnDenied - like:

procedure TForm6.KeycloakAuthenticated(Sender: TObject);

begin

  ShowMessage('Welcome '+KeycloakAuth1.GreetName+'!'+

    sLineBreak+sLineBreak+'You have been authenticated as userId:'+

    KeycloakAuth1.UserId);

end;

procedure TForm6.KeycloakDenied(Sender: TObject);

begin

  ShowMessage('You have not been authenticated!');

end;

In the forms FormShow event call the components Authorize procedure, to start the flow.

I will add more capabilities to the component - such as nonce and code challenge, and cover all OpenID Connect response_type combinations.

To get a good grasp and overview of the OpenID Connect flows - take a look at this Medium post.

Happy 30th anniversary - Delphi! and see you at Delphi Summit 2025.

/Enjoy

Splitters Helpers

 - or how to find ways to add own helpers and keeping existing helpers.

Wanted to do a pun on Finders Keepers, but failed - so this post is a sample on how to add your own record helper - in this case for the string type.

Delphi/Object Pascal does not allow for multiple helpers for the same type to be available at the same time.

So in my little example I wanted a string function that would split a string by a char, but up till a given max length.

A scenario for that use could be if you need to feed a system that has limited fixed size fields, spanning over more fields - CompanyName1, CampanyName2 only being 30 chars each. And for readability and UI, you also need to consider not to split mid-word.

To overcome the issue with  the one helper active per type, defined your own matching type:

MyString = type string;

Define the new record helper for that type with its function:

MyStringHelper = record helper for MyString
  function SplitMaxProper(const ch: char; const len: Integer): TArray<string>;
end;

Since we also want to use the normal string helpers within the helper function, we need to do some casting when referring to the helpers type itself:

function MyStringHelper.SplitMaxProper(const ch: char; const len: Integer): TArray<string>;
var
  sl: TStringList;
begin
  var sidx := 0;
  var done := False;
  sl := TStringList.Create;
  try
    while (not done) do
    begin
      var delta := string(Self).LastIndexOf(ch, sidx+len, len);
      if (delta = -1) or (string(Self).Length-sidx <= len) then
      begin
        sl.Add(Trim(string(Self).Substring(sidx)));
        done := True;
      end
      else
        sl.Add(Trim(string(Self).Substring(sidx, delta-sidx)));
      sidx := delta;
    end;
    Result := sl.ToStringArray;
  finally
    sl.Free;
  end;
end;

And when using the new string helper we do need to cast to get to our new function:

procedure TForm6.btnSplitClick(Sender: TObject);
begin
  meSplitText.Clear;
  var len := StrToInt(edSplitLength.Text);
  var str := MyString(edTextToSplit.Text);
  meSplitText.Lines.AddStrings(str.SplitMaxProper(#32, len));
end;

Disclaimer: The code is done so that it does fix better in the narrow width of my blog layout, and the function might also need some optimization :)

A note on the LastIndexOf string helper functions, the current official documentation seems to me a bit unclear:

StartIndex specifies the offset in this 0-based string where the LastIndexOf method begins the search, and Count specifies the end offset where the search ends.

Do remember that the search in this case, of cause goes from right to left - so StartIndex would normally be the length of the string.

I started with a Stephen King book title pun, so I should also comment on the quote used in the image shown - a quote from the brilliant author Tom Holt and the book The Portable Door - which is highly recommended, even if you have watched the movie adaptation, which is also good - but different - since adapting Tom Holt's books is no trivial task.

/Enjoy

Just Ping someone!

- or a component for Ping Identity's authentication within your Delphi application.

It has often been the rule that native applications either authenticate the user by current OS user or by an application-centric user and role model - but that might for multiple reasons not be good enough anymore.

The old AD or LDAP lookups are being replaced by cloud IAM platforms, to control and secure the authentication of the identity of the user.

After the user is authenticated (and is authorised "access" your application), the application-centric roles flow can continue as-is.

One of these IAM cloud provides is PingIdentity, and they have a fairly extensive Postman collection for their PingOne Platform API - found here. Their developer documentation is also very useful - found here.

I have previously used various ways against various providers, but it seemed that when testing against PingOne with MFA enabled (Multi-Factor Authentication - which might include the annoying phone thing - that everyone uses) - I was hit by either a CORS issue or something else.

When using an OAuth2/OpenID Connect authentication, it does require that you setup and use a redirect uri, to tell the provider who is locally listening/waiting for the "response".

The idea with this type of authentication, is that the flow is handled securely within a web browser session, and the client does at no point in time know the login credentials - only when the user is authenticated do we need an "id token". So no "local" storage/handling of "passwords".

The listener part and possible timeouts has always annoyed me, so it seemed based on the flow, that I could handle it differently by just intercepting the local redirect call with the auth code - when the user is authenticated on the PingOne side - to get the id token needed.

Implementation

The TPingOneAuth component is a descendant of standard TWebBrowser overriding the IDocHostUIHandler interface, and adding a some properties.

The GetHostInfo override is to ensure that all redirects in the browser component is triggering the OnBeforeNavigate2 event - which then on the redirect auth code call, will get the wanted id token.

function TPingOneAuth.GetHostInfo(var pInfo: TDocHostUIInfo): HRESULT;
begin
  pInfo.cbSize := SizeOf(pInfo);
  pInfo.dwFlags := 0;
  pInfo.dwFlags := pInfo.dwFlags or DOCHOSTUIFLAG_NO3DBORDER;
  pInfo.dwFlags := pInfo.dwFlags or DOCHOSTUIFLAG_THEME;
  pInfo.dwFlags := pInfo.dwFlags or DOCHOSTUIFLAG_ENABLE_REDIRECT_NOTIFICATION;
  Result := S_OK;
end;

The id token is then parsed using Pablo Rossi's brilliant JOSE and JWT library, that might as well have been done using a call to one of the PingOne's Token Introspection endpoints.

Since we do need an user id - the OpenID Connect scope must include profile also.

To parse the custom claim a custom TJWTClaims class is added, containing the profile claims we want to read.

TPingOneClaims = class(TJWTClaims)
// Adding some given by the OpenID Connect scope: profile
private
  function GetPreferredUsername: string;
  procedure SetPreferredUsername(const Value: string);
  function GetGivenName: string;
  procedure SetGivenName(const Value: string);
  function GetFamilyName: string;
  procedure SetFamilyName(const Value: string);
public
  property PreferredUsername: string read GetPreferredUsername write SetPreferredUsername;
  property GivenName: string read GetGivenName write SetGivenName;
  property FamilyName: string read GetFamilyName write SetFamilyName;
end;

Usage

Install the TPingOneAuth component in the Delphi IDE, and set the library path - business as usual.

There is a small sample application in the GitHub repo, but steps are at follows:

Drop or Create the TPingOneAuth on a form, setting following properties:

• AuthEndpoint: /as/authorize
• AuthPath: https://auth.pingone.eu/
• ClientId:
• ClientSecret:
• EnvironmentId:
• RedirectUri:
• ResponseType: code
• Scope: openid profile
• TokenEndpoint: /as/token

All these values are found in the PingOne console, under the application you setup as a "authorization" reference to your native Delphi application.

Add code to the OnAuthenticated (and OnDenied) event(s) - were on a successful authentication the public properties UserId and GreetName can be useful.

To start the authentication just call the Authorize method - and with MFA enabled, you will see the prompt bellow, once you initially have given your credentials.

After finding your phone, and having swiped to authenticate in the PingOne mobile companion app, the OnAuthenticated event is called.

The code for the component and the sample is found here.

A thing to note:

When having MFA enabled - and you need to pair a device the first time around - the url Ping provides does contain some "strict mode" Javascript that IE will prompt you about twice, the solution to this is to set the SelectedEngine property to EdgeIfAvailable or EdgeOnly and deploy the EdgeView2 Runtime (WebView2Loader.dll) which can be found in GetIt as the EdgeView2 SDK.

/Enjoy

Use(s) less

- how to visualize your uses - for un-needed and wrong scoped units.

The "Uses" plugin installed and feed with a PAL report.

Disclaimer: Even if the above picture shows my usage of the code in the current state - the code/plugin is by no means done and complete - but I put this out for others to tinker with and benefit from.

One of the things that has been on my list of wish's for the Delphi IDE since forever, has been the ability to identify which units in your uses clause is either not needed, or should/could be moved from the interface section to the implementation section.

There seems to be good reasons for that feature not to be a 100% correct in it assumption in what is needed and what could be moved, and that might also be the reason Embarcadero has not included that feature yet, but..

I only need something that is good enough, and that can help me make the decision on what to clean up.

There are some third-party tools that attempt to do that, one being Peganza's Pascal Analyzer - and the free Lite version is sufficient to what I attempted to try here.

With the new CodeEditorEvents added to the OTAPI, it is now a lot easier to change the rendering of the code editor window in a safer way.

So this is what you need to do to get to a similar result - you need to download and install Pascal Analyzer (Lite).

Setup a project to handle your Delphi project, and you can optimizer the output from the analysis to only output the Uses.txt with only the Usage (USES1) - since that is the only needed.

I should mention that people more clever than me - could probably utilize the LSP or DelphiAST (or whatnot) - to make a more elegant solution - and they should :D.

So the prerequisite for the plugin is in place - a Uses.txt to parse. And to "refresh" the Uses.txt - you would have to run C:\Program Files\Peganza\Pascal Analyzer Lite\palcmd.exe <MyDocuments>\Pascal Analyzer Lite\Projects\<Myproject>.pap again.

The plugin contains code to parse Uses.txt and created a dictionary where each unit/module has a list of flagged units.

That dictionary is used to get the uses statements for the current unit, so that the renderer can either gray out or strikeout the unit name in the Uses clauses'.

As mentioned, the state of the code is more a prof of concept and does include code that is more a comment and an intent to what I had planned at that point of time - but it has now been laying around since summer, and I will not get this perfected in foreseeable future so it will have a better life set free.

The code can be found here: https://github.com/SteveNew/CodeEditorPaintTextPAL

IAP Client, therefore IAM

- or creating a simple Google IAP client using JOSE and a service account key file.

Google offers a massive amount of services and APIs to these, and I doubt anyone has the full overview unless they are in that domain naming space 24/7, and the same would be true for Amazon, Microsoft and other offerings.

In this example, I am using Googles IAP (Identity-Aware Proxy) to get an OIDC token that can be used to authorise the requests sent by a service account.

And a disclaimer: This is a conceptual example, for more secure and correct use one should not store any form of keys - so for that look into something like Workforce identity federation - which gives the same short-lived OIDC tokens, with the help of an identity provider. But for the purpose of the example - less is more.

First you would need to create or get the service account key file by using the Google Cloud console to create a new key for the service account, and download and store that in a safe place. The Grijjy guys did a similar example years back, where they used a PEM key based from the P12 file.

I just created a class to load our json key into to extract the values needed, and by using the JOSE library, I did a customer header and claim/payload - which was signed as the example given by the bash shell script in Googles documentation here: Get OpenID Connect Id token from key file.

Since the signing requires RS256, it does disqualify some of the other libraries, but JOSE does support that.

The Google API Client libraries and Cloud Client Libraries tend to use what they call the Application Default Credentials (ADC) strategy, and to mimic that, I added a bit of code that should cover the 3 desktop OSs.

You can read more about ADC here: Google Application Default Credentials

One requirement of the signing is the OpenSSL libraries - JOSE does use these. You could of course also "just" use the EVP_DigestSignXXX functions from the SSLEAY library, and wrap what you needed, but JOSE does such a nice job of that, so why bother.

Adding a scope will give an access_token, whereas without it you will only get the id_token which is the OIDC token.

I did add an expiration check on the OIDC token, and it does seem to do the job of not having to request the token more than needed - but it might not be the perfect way.

The OIDC token is used as a bearer token for the actual requests to the service the proxy works for, so I added some HTTP methods as sample - and they just return an IHTTPResponse interface.

So an example of use would be something like:

uses
  System.Net.HttpClient,
  FbC.IAPClient;

procedure TForm1.Button1Click(Sender: TObject);
const
  cSERVICE_ACCOUNT_KEY='lustrous-stack-342709-93cfcb2a8000.json';
  cIAP_CLIENT_ID='108584532133305403517';
  cURL='https://mytest.com';
var
  IAPClient: TIAPClient;
begin
  IAPClient := TIAPClient.Create(cSERVICE_ACCOUNT_KEY, cIAP_CLIENT_ID, cURL);
  try
    Memo1.Text := IAPClient.IdToken;
//    Memo1.Text := IAPClient.Get('/companies').ContentAsString();
  finally
    IAPClient.Free;
  end;
end;

The IdToken property should not be public, and the client code is just meant as a starting point - since it does solve the painful bit - signing and authentication.

The code for the conceptual IAPClient can be found as a gist here: Delphi Google IAPClient

Requirements are also:

JOSE library: https://github.com/paolo-rossi/delphi-jose-jwt

The appropriate OpenSSL dlls: https://github.com/IndySockets/OpenSSL-Binaries

An implementation of TOAuth2Authenticator with this might have been a good idea, but ...

Well I hope it will at least get you on the right track, if you need to go through an IAP. There is no check for an unsuccessful attempt to get the OIDC token - it will just return an empty string - so that is meant as homework.

/Enjoy

A SET up

- or how the for-in statement changed  recently.

Before Delphi 10.4, the statement:

for integer in [] do 

would operate on a set, but since then it now operates on an array. Which then also means a bit of different behaviour.

I had missed this change completely, and only recently gotten aware of it since we had a build issue - mixing source from different IDE versions. (BTW: Do not do that).

Since it previously worked on a set - the "list" when traversed would be an ordered list of values with an ordinal value no bigger than 255, and no more than 256 elements, that is now changed. I will list a few examples - illustrating the pre-10.4 behaviour compared to 10.4 and later.

Duplicates and ordering

var

  i: integer;

begin

  for i in [42, 10, 42, 3, 42, 7] do

    WriteLn(i.ToString);

end.

returns:

10.4+: 42, 10, 42, 3, 42, 7

pre10.4: 3,7,10,42

Range

var

  i: integer;

begin

  for i in [42, 365, MaxInt] do

    WriteLn(i.ToString);

end.

10.4+: 42, 365, MaxInt

pre10.4: Does not compile - In 10.3 you get an E1012 Constant expression violates subrange bounds (earlier you might also get an E2024)

To solve that you could replace it with something like:

uses

  System.Types,

  System.SysUtils;

var

  i: integer;

begin

  for i in TIntegerDynArray.Create(42, 365, MaxInt) do

    WriteLn(i.ToString);

end.

I do think the change is a step up, and does syntax-wise work more like one would expect, so a change for the greater good - but do be aware of the change of behaviour. Another benefit of a newer version is the usage of inline variable declaration and type inference, like in this case:

for var i in [42, 10, 42, 3, 42, 7] do

    WriteLn(i.ToString);

/Enjoy