Bringing the POX cross-platform

- or a few things on the extended RTTI and new inline variables presented in 10.3.

Having figured out the internals of the POX file format used in the game Siege of Avalon, and it's sibling from the modding community: Ashes of Avalon, Pillars of Avalon and Days of Ahoul to name the larger ones - enables to revise the tooling used for the engine - most of it closed source or requiring jumping through hoops while juggling.

So the first thing I brewed together is a POX file viewer/player with an export function. A preliminary test was done in the previous post, but I redid it using the FMX multi-device UI framework that comes with RAD Studio/Delphi/C++Builder - so that modders using for instance macOS, do not need to run in a virtual OS - and it all comes for "free" when using FMX.


Since it is WIP, I only want to go through a few parts of the code - but the full source can be found in the Src/New Tools folder in my fork of the Siege of Avalon GitHub repo here.

Also feel free to point at any errors or improvements that could be made - it only makes the code better and all of us smarter - and I am sure it can be done better :)

The POX files contains data for most assets the game engine uses - they typically have a "setting/property" part, in the form of a .ini file structure, and a run length encoded (RLE) part containing the pixel/bitmap data.

Ini file data structure from string

Since the ini file data is embedded in the POX file, we end up with a list of strings containing the info we want to parse, as if it was an .ini file. Since the constructor of TMemIniFile does require a filename, it is not given that we instead can populate its data, calling SetStrings, after having created it with an empty filename - like below:

ini := TMemIniFile.Create('');
try
  ini.SetStrings(iniData);
  var actionStr := ini.ReadString('HEADER', 'Actions', '');
finally
  ini.Free;
end;

There is a GetStrings as well, if you need to create data structured as .ini data, and just wants to end up with a list of strings;

Accessing memory using TMemoryStream

A similar issue, do we have with the raw RLE data already read from the POX file, since the header for every frame has a pointer to the RLE data that needs decoded, and just using a TMemoryStream seemed more readable that pointer arithmetic :). But what we need is protected, so the usual hack applies, by class inheritance to get access to call SetPointer:

TROMemoryStream = class(TMemoryStream)
private
  OrigPtr : Pointer;
  OrigSize : LongInt;
public
  procedure SetMemPointer(Ptr: Pointer; newSize: Longint);
  destructor Destroy; override;
end;

As in most things in life, you should put things back to the state you found them (or a better state), so we store the original properties as they were created, before we point the memory stream to the new memory location.

destructor TROMemoryStream.Destroy;
begin
  SetPointer ( OrigPtr, OrigSize );
  inherited;
end;

procedure TROMemoryStream.SetMemPointer(ptr: Pointer; newSize: Longint);
begin
  OrigPtr := Memory;
  OrigSize := Self.Size;
  SetPointer ( ptr, newSize );
end;

And use it like this:

rleData := TROMemoryStream.Create;
rleData.SetMemPointer(rle.DataPtr, rleSize);
rleData.Position := 0;
rleData.Read(&c, 1);
...

Inline variables and RTTI on enumerations

Since I do want the ability to "play" the frames ordered by the type of action they represent, and the direction they are heading - instead of just playing all frames in stored order - some frames are used more than once, and in different order. The ini data has this information - both which action that are available for the "asset" stored in the POX file, and what directions - of possible 9 it "moves" in.

So two enumerations is defined, that actually matches the stored properties/values in the ini data.

TActionEnum = ( Stand, Attack1, BowAttack, Cast, Pain, Death, Walk, Run, Default, Explode, Sit, Reveal, Hide );
TDirectionEnum = ( NWFrames, NNFrames, NEFrames, EEFrames, SEFrames, SSFrames, SWFrames, WWFrames );

and a list type for the frames, that gets added to a dictionary with a key expressing action and direction.

TFrameList = class(TList<Integer>);
movements: TObjectDictionary<Byte, TFrameList>;

An example of the ini data, shows that each action has 8 directions stored as a list of frames, and a "behavior" tag - which we want to ignore for now.

[Action Pain]
NWFrames=97,98,99,99,98,97,END
NNFrames=105,106,107,107,106,105,END
NEFrames=113,114,115,115,114,113,END
EEFrames=121,122,123,123,122,121,END
SEFrames=129,130,131,131,130,129,END
SSFrames=137,138,139,139,138,137,END
SWFrames=145,146,147,147,146,145,END
WWFrames=153,154,155,155,154,153,END

By going through the TActionEnum....

for var action := Low(TActionEnum) to High(TActionEnum) do
  addFrames(action);

...for each "Action", the TDirectionEnum directions "framelist" are read...

procedure addFrames(action: TActionEnum);
begin
  for var direction := Low(TDirectionEnum) to High(TDirectionEnum) do
  begin
    var Frames := ini.ReadString('Action '+TRttiEnumerationType.GetName(action), TRttiEnumerationType.GetName(direction), '');
    var i: integer;
    if Frames<>'' then
    begin
      var frameList: TFrameList := TFrameList.Create;
      for var str: string in Frames.Split([',']) do
      begin
        if TryStrToInt(str, i) then
          frameList.Add(i);
      end;
      movements.Add(ord(action)*10+ord(direction), frameList);
    end;
  end;
end;

..and converted into a list of integers, and added to the movements dictionary. Notice the use of TRttiEnumerationType from the RTTI unit - since we have the info anyway - let us use it.

So now we can "play" a specific list of frames by looking up by action+direction.

RLE decoding, colour convertion and pixels 

We need to do 3 things here, decode the run length encode data, convert the RGB565 pixelformat and then set the pixels colour in the bitmap.

To access the bitmap we need to map into a TBitmapData structure, set the pixels and unmap the data back - we add the bitmap to a list of bitmaps afterwards - and to show the bitmap we can just assign the bitmap to the images' bitmap property - so no need for draw or copy functions.

var
  i : integer;
  c : byte;
  colour: word;
  pxCol: TAlphaColorRec;
  rleData: TROMemoryStream;
  bmpData: TBitmapData;
begin
  pxCol.A := $FF;
  if bitmap.Map(TMapAccess.Write, bmpData) then
  begin
    var x: Integer := 0;
    var y: Integer := 0;
    rleData := TROMemoryStream.Create;
    rleData.SetMemPointer(rle.DataPtr, rleSize);
    rleData.Position := 0;
    rleData.Read(&c, 1);
    while (c > 0) and (c < 4) do
    begin
      case c of
        1 : begin // colour/pixel data
          rleData.Read(&i, 4);
          while i > 0 do
          begin
            rleData.Read(&colour, 2);
            pxCol.B := (Colour and $1F) shl 3;
            pxCol.G := ((Colour and $7E0) shr 5) shl 2;
            pxCol.R := ((Colour and $F800) shr 11) shl 3;
            bmpData.SetPixel(X+rle.AdjX, Y+rle.AdjY, pxCol.Color);
            inc(x);
            dec(i);
          end;
        end;
        2 : begin // add x offset
          rleData.Read(&i, 4);
          i := i div 2;
          inc(x, i);
        end;
        3 : inc(y); // new line
      end;
      rleData.Read(&c, 1);
    end;
    FreeAndNil(rleData);
    bitmap.Unmap(bmpData);
  end;
end;

The pixelformat convertion seems correct, but I do have a "alternative" at hand - where values were found by brute force:

R := (r * 527 + 23 ) shr 6;
G := (g * 259 + 33 ) shr 6;
B := (b * 527 + 23 ) shr 6;

And hopefully this is fast enough and correct - since going from $FFFF number of colors to $FFFFFF does give a bit of room for the originators of the POX algorithm to play some tricks on us.

The FMX TAlphaColorRec structure is actually quite nice and readable - not having to convert back and forth.

I do like to use inc(x, y) - instead of x := x + y - and sprinkled with the new inline variables can make things more compact and still very readable.

Two sinful things to minimize code - tags and string enumeration

In the initial code I put into the repo in a sec, both some features are missing and it does not handle some specific scenarios like some sprite object items - try SmallSack.pox - since it does not have any directions - just one image - but everything is on the TODO :)

It does also contain examples of code that I normally consider bad practices. Since this is a small simple tool to handle a simple task - this is a bit quick and dirty.

I mention them here so that you do not included this type of code into big life-preserving software :)

To get the ordinal value of the current selected action type I do like this:

currentAction := ord( TRttiEnumerationType.GetValue<TActionEnum>(TRadioButton(Sender).Text) );

Let us hope that I never decide to translate the UI - that will break the code. But I only need one line of code to handle all the 13 action type radiobuttons. What I might do is to make the list dynamic - driven by the enum.

This was also an example of using the RTTI the other way round - getting the enum value by name.

The second example is "tag programming" - I have the 8 (9) directional buttons, where I on the onClick set the chosen direction:

currentDirection := TSpeedButton(Sender).Tag;

So every button component has it's tag property set to the LSD (least significant digit - I think I just made that TLA up :D), of the key value needed to lookup the movements dictionary.

I would have taken a picture of a macOS build, but my wifes miniMac is not update-able after El Capitan.... So Apple is tightening the ropes - if I need to build for macOS/iOS in the near future - lets see if she wishes for a newer miniMac ;D

Enjoy.