ARX

Advanced Resource files

Hagen Möbius


Table of Contents
1. Introduction
1.1. What does ARX mean?
1.2. What is ARX?
1.3. What is Arxx
1.4. Purpose
1.4.1. Archive
1.4.2. Unified Data Source
1.4.3. Merging
1.4.4. Templates
2. Features
2.1. Compression in ARX
2.2. Structure in ARX
2.3. References in Arxx
3. Design Principles
3.1. ARX archives
3.2. Inheritance
3.3. Heap and Stack, Constructors and Destructors
4. Uses of ARX
4.1. Scenarios
4.1.1. General layout of scenarios
4.1.2. Scenario 1
4.1.3. Scenario 2
4.1.4. Scenario 3
5. ARX Server
5.1. What it is?
5.2. What is is not?
6. Structure
6.1. Relations
6.2. Structure to the items

1. Introduction

1.1. What does ARX mean?

ARX is not a Three-Letter-Acronym! Although "Advanced Resourcefiles" is probably a good start there is no corresponding word for the final X. ARX has been derived from the name "Arxx" which is the name for "Advanced Resourcefiles for C++" and names an implementatory concept in the programming language C++. In that line of thought, ARX is the abstraction of Arxx as seen from outside any implemtation or language specific details.

1.2. What is ARX?

First of all ARX defines a format for archive files. Within that definition lie the general concepts that make ARX "advanced" as the nature of these have to be supported by the file format. ARX is intended to help software developers manage and access huge repositories of static data for any context. It is supposed to imply the notions of data synchronization, referencing external data even in remote places like http or ftp servers, and merging multiple ARX archives into one. For all these features ARX' intended audience are game developers but it may fit in other data intensive contexts as well.

1.3. What is Arxx

Arxx is an ARX API for C++ - a definition for accessing ARX archives inside C++ programs. Arxx defines the C++ programmer interface to provide an object-oriented view on ARX archives, that is on a directory tree of data items. Mainly Arxx has two duties to fulfill:

  • It defines the C++ representation of the ARX features like data synchronization, merging and external data fetching.

  • Additionally it adds some concepts to ARX that make the use of ARX archive in C++ even more powerful. Mainly this includes assigning semantic meaning to binary data as stored or referenced in an ARX archive.

1.4. Purpose

ARX is the combination of four concepts with varying degrees of difficulty and usability but all with the intent to help software developers manage any static data their applications may require. These four concepts should be given names now so we can refer to them.

  • Archive

  • Unified Data Source

  • Merging

  • Templates (only in Arxx)

A short summary for every point will follow now.

1.4.1. Archive

The most predominant function of ARX is its ability to act as an archive. Even so, it certainly is also the most boring feature since this has been done many times before. Like a tar-ball, ARX can act as a packed directory tree. However, an ARX archive is not meant to be packed somewhere and, after downloading it or something, be unpacked into your local file system. ARX archives are generally intended to be loaded directly by applications, using libarxx in the C++ context to provide an object-oriented view to its content. Taking into considerations the needs of application programmers the items in an ARX archive also have additional properties like version number and type.

1.4.2. Unified Data Source

This is a very small part of ARX and not very important but it may help teams to develop applications using ARX in a comfortable manner or help the migration process to ARX. The idea is that instead of packing the actual data into the ARX archive you only save the location of the real data in URL form. When the application loads the ARX archive the data will be retrieved from the location indicated by the URI, given the ARX implentation knows how to handle a certain URI scheme or location. This way applications' data can stay up-to-date without much ado: they just load them down everytime (from http, ftp, cvs or svn repository or even P2P networks).

1.4.3. Merging

Merging can be considered the "evil" (tm) function of ARX. In short, Merging is the process of taking at least two ARX archives and merging their contents. Depending on the type of merge there may be collisions between the contents and resolving those will define the nature of the Merge operation. You could, for example, merge for unique IDs (the type of the merge) and resolving collisions (two Items having the same unique ID) by taking the Item with the higher version number. The nature of such a Merge is that you always get the most recent and all the data. Now imagine how you could merge a ARX archive, read from a local file, with an archive read from an HTTP or ARX server and imagine also that both, the local and the remote archive, can have Items which, instead of having the data locally, point to them by URL.

1.4.4. Templates

Originally this was a very important concept inside ARX but I had no time to think about it again yet and so it moved down to Arxx as I can only imagine this feature in C++. Templates are meant to further structure the data inside Items adding "semantic" meaning to binary data. This concept is almost purely implemenational in that it provides structured pseudo-typesafe access to binary data. Imagine a block of data being 16 bytes long. Now, reading a template definition from the ARX archive, the ARX implementation creates a template object. For example the definition says that these 16 bytes should be read as 4 times 4 byte unsigned integer and their names are "x", "y", "z" and "w". Now you could access the third integer part using the index "z" on the Item. The only interesting part about this probably is that templates should be able to nest so you can combine them in complex structures.