Cure for POX found in Avalon

- or how reverse engineering makes a game engine portable.

One thing that can stop software from being portable and surviving the decay of time, is when closed and proprietary file formats block any chance of progress - the game Siege of Avalon has the POX.

I did pick up the ball earlier this year to move the Siege of Avalon OS game "engine", that was open sourced back in mid-2003, onto newer version of the compiler, and eventually to other platforms - a thing that had been the original plan for opening the source. Read about it here.

But after having succeeded with some of the initial issues - I hit the roadblock that might have been the cause that this game hasn't been "ported" yet - the POX file format.

The POX files contains data for most assets the game engine uses - they typically have a "setting/property" part, in the form of a .ini file structure, and a run length encoded (RLE) part containing the pixel/bitmap data.

That would be all fine and good, if it hadn't been for the usage of special "drivers" in the form of close-sourced DLLs - provided by a deceased third-party - and not part of the open sourced code. The main job of these are to decode the RLE image data and "draw" this to a bitplane.

So it is not that RLE is a way to encrypt anything, but just to optimize the storage and possibly performance - which made very much sense back when the game was done - and still does.

Basically does RLE mean that you "compress" repetitive data - by just storing the length of "what". But the way it is done, varies with what data you store - so no encoding is the same.

I had been looking at what I refer to as the DigiFX (that use the DLLs) pox in the code, and hoped that it would just go away, but no..

So having a few days off - I actually found my old IDA Pro Free - had actually motivated myself into how great it would be to get my lack of x86 assembly skills challenged. But I did a last search on the SoA forums - and tada! Someone had back in mid 2011, actually done what I planned to do, figuring out how the POX RLE data was decoded - and even left some C pseudo code:

void UnpackRLE(istream& fin, Bitmap* pixelmap, RLEHDR* rlehdr)

{

int x=0, y=0;

char c;

short colour;

int i;

while(1) {

c = fin.get();

switch(c) {

case 1: // colour/pixel data

for( fin.read(&i,4); i > 0; i-- ) {

fin.read(&colour, 2);

pixelmap->setPixel(x + rlehdr->AdjX , y + rlehdr->AdjY, colour);

x++;

}

break;

case 2: // add x offset

fin.read(&i, 4);

i >> 1;

x += i;

break;

case 3: // new line, carriage return (y++, x=0)

y++;

break;

case 0: // end of rle stream

return;

default:

throw;

}

}

}

That enabled me to finish my document on the "POX (Proprietary Object eXtension) file format" including some Pascal pseudo code and put that onto my SoAOS GitHub repo here.

There is still much left to do - but the pox of the POX is cured. I did a small PoC application as shown in the start - and I will just move that to FMX, and fix the colour/pixelformat issue - and test that it can handle all the various resource types.

I know that as an alternative it had been suggested/tried to just convert the POX files into a different "better" format - utilizing PNG files - that would also be a better format for SDL2 - using the alpha channel.

But the user would still have to run the conversion on Windows due to the DLLs, and since assets (apart from the first chapters) are not free (and there is also all the mods) - these could not be provided "converted".

My take on the work of the engine is also to keep it compatible with the original assets - so that you can copy the asset files to a new platform together with a fresh native "engine" :)

And do remember that reverse engineering, can be a good thing under fair use - and here we have a "clean room" example - someone provided me with the spec - the pseudo C code - and from there I could implement code, that could decode the data.

But always thread carefully - there could have been a EULA or TOC that would have prohibited you from solving the wish of interoperability :)

Read EFF.orgs "Coders’ Rights Project Reverse Engineering FAQ", here. 

/enjoy