diff --git a/bemani/format/afp/README.md b/bemani/format/afp/README.md new file mode 100644 index 0000000..e244148 --- /dev/null +++ b/bemani/format/afp/README.md @@ -0,0 +1,210 @@ +# AFP Animation Format + +As far back as I can see, Konami has used an animation format that I refer to as +AFP in virtually all of their games. Basically, if it runs on a PC, it probably +has a variant of AFP running on it. This also goes for several of their commercial +PC titles, some console titles and mobile titles as well. The name "AFP" comes +from the library that implements most of the format but I have no idea what this +actually stands for. In very old PC-based arcade games such as The\*BishiBashi the +library code makes references to Flash. If it encounters a plain SWF file it returns +an error stating that old data is not supported and encouraging the artist to +convert the data to the "new" format. Many other similarities between SWF and AFP +are present including a legacy bytecode table that is identical to Flash, a tag ID +to string function that matches Flash exactly up to the point where Konami started +adding their own, and the fact that the core engine works identically down to the +affine transformations, additive and multiplicative colors, tag system, blending +modes, masking and a whole host of other things. + +At some point, most of the old SWF code was removed from the project. A modern +AFP library is almost unrecognizeable from a classic one, save for some basic +code that parses the still-supported tags. The legacy SWF bytecode parser has +been removed completely, support for recognizing vanilla SWF tags has been removed, +and support for bytecode that is interactive in nature has been removed leaving +only the shell of a bytecode engine that can update properties for effects such +as applying masks, setting objects visible or invisible and rewinding/advancing +individual sprites on the main canvas. While Konami appears to have put a decent +effort into making all of their format changes purely additive, if one was to +try to render an animation from an older game inside a newer game's engine it +would not be a surprise if the resulting graphics were subtly wrong. + +The AFP file format itself is not an all-encompassing container. It superficially +resembles a SWF file but has an entirely different header and different ways of +encoding the tags and optional features of the format. Curiously, one cannot decode +a chunk of AFP data without the corresponding BSI data which is simply a list of +locations to byteswap as well as the length to byteswap. This appears to be a +rudimentary way of hiding plain strings in the AFP data itself so that they can't +be found in a hex editor. The byteswap instructions are trivial as well as reversible. +Along with AFP data, one would need shape structures as well as textures. These are +not found inside the AFP data but are instead inside a parent container. For older +games this is a TXP2 file which contains special sections for texture sheets, +individual textures, shapes, fonts, AFP data and BSI data. For newer games this is +a standard Konami IFS file where the AFP data is found in the `afp/` folder, the +BSI data is found in the `afp/bsi/` folder, the shapes are found in the `geo/` +folder and the textures are found in the `tex/` folder. Aside from container format +differences the underlying files are identically parsed and have equivalent +meanings regardless of where one unpacks them from. + +The basic concept of how an animation is stored and represented is fairly simple. +Each animation has a specified number of frames and there is a lookup table in +the AFP header specifying which tags should be acted upon for each frame. To play +an animation, one must act upon each tag for a frame and then display or save the +resulting image. Tags have actions such as defining a shape or sprite for later +use, requests to place a previously defined shape or sprite on the canvas, +requests to update the properties of a previously placed object on the canvas, +requests to delete a previously placed object from the canvas and arbitrary +bytecode to execute as part of the frame's processing. Placed objects on the +canvas each have an associated additive and multiplicative color to apply to them +before blending, a blending mode such as normal, additive or subtractive, an +affine or perspective transform matrix used to determine how to position the +object and finally either a bounding box for a solid color rectangle, a reference +to a texture or a list of child objects to treat as a single sprite. Objects are +placed onto their own depth planes and rendered from lowest plane to highest plane +one by one until the final frame has been constructed. + +# Tags + +Since tags are the heart of the format (as they are in SWF files), I go into a bit +more detail about the crucial ones here. In the older versions of the format +vanilla SWF tags could also be used but they are documented elsewhere online and +in practice have never been seen inside an AFP animation. + +## AP2 Shape Tag + +The shape tag defines a shape (either a reference to a texture quad or a solid +color rectangle) and assigns it an internal ID for later use. Both texture quads +and solid rectangles are represented by the same shape structure which contains +the shape's bounding rectangle, UV vertex points and draw parameters (such as whether +this shape is a rectangle or a quad in the first place). If the shape is a texture +quad the structure will also include a string reference to the name of the texture +itself. The UV coordinates and bounding rectangle are always identical to the size +of the referenced texture in the case of texture quad shapes and appear to be in +the format simply to pass forward to a graphics card. Acting upon a shape tag means +adding the shape to an internal library of objects that can be placed on the canvas. + +## AP2 Image Tag + +This appears to be functionally identical to a shape, save for missing all of the +graphics card coordinates. Instead of pointing at a shape structure, image tags point +directly at a texture itself. Acting upon an image tag means adding the image to an +internal library of objects that can be placed on the canvas in a similar manner to +a shape. + +## AP2 Sprite Tag + +The sprite tag defines a sprite, which in SWF and AFP nomenclature just means an +embedded animation complete with its own list of frames and tags. In reality, the root +animation itself can be seen as a sprite tag, and in fact the format allows importing +another animation as a sprite to be placed on the canvas just like an image or shape. +Sprites are handy because they allow an animator to define things like characters with +their own shapes, transform and move the objects that make up those characters, and then +later place the character on the canvas just like a shape or image. As you might expect, +since a sprite is just a set of tags to act upon each frame, sprites can include their +own child sprites indefinitely. Acting upon a sprite tag means adding the sprite to +an internal library of objects that can be placed on the canvas. + +## AP2 Place Object Tag + +The place object tag is the meat of the format. Place object tags specify a placed +object ID to refer later to the placed object as well as a depth to place the object +on. They refer to sprites, images and shapes by their registered ID. They can come in +one of two flavors: create or update. In create mode, a new object is placed onto the +canvas at the specified depth. That object is looked up in the internal library of +objects that was previously added to in a shape, image or sprite tag. In update mode, +a previously placed object is looked up by object ID and depth and its properties are +updated to match the update tag's specifications. In both cases, the place object tag +can include a blend mode, an affine or projection transform, an additive and +multiplicative color and a few other properties. The blend mode and colors specify to +the renderer how to combine the pixels of the placed object with other objects that are +placed on the canvas. The transform specifies how the object is positioned, rotated and +stretched. Using a series of updates, an animator could rotate an object, move its +position on the canvas, change its transparency to fade it in or out and basically any +other operation. + +Acting upon a place object tag means placing a new object on the canvas to be +rendered this frame or updating an existing object with new properties which will be +rendered this frame. Objects that are already on the canvas and not updated by a +place object tag will keep their properties and be rendered identically on subsequent +frames. The only exception to this is placed sprites, which automatically advance to +the next frame of their own embedded animation in order to render correctly. + +## AP2 Remove Object Tag + +The remove object tag looks up an object previously placed by a place object tag and +removes it from the canvas. It can either remove an object by object ID and depth or +it can remove the last placed object on a particular depth. Acting upon a remove object +tag means removing an existing object from the canvas so that it is not rendered on +this frame or subsequent frames. + +## AP2 Do Action Tag + +The do action tag executes AFP bytecode in order to perform some action. This can mean +requesting a mask to be applied to a placed object, requesting a placed sprite be +advanced or rewound to a specific frame, requesting a placed object be made invisible +or a host of other useful effects. AFP animations use this to provide looping effects +on sprites since the format does not natively support loops. They also use this to +apply masks to placed objects as the format does not natively support defining a mask. +Originally do action tags could also provide interactivity such as in The\*BishiBashi +where the levels themselves were implemented entirely in AFP files and played using +bytecode to read inputs, check game scores and the like. Modern games have stripped +this functionality out and only use it to provide a few useful hooks such as the above +documented effects and things such as triggering sounds to play or informing other +systems what frame of the animation is being rendered. + +Acting upon a do action tag means using a bytecode interpreter to execute the bytecode +and apply the desired property changes to the currently placed objects on the canvas. +Note also that the place object tag supports bytecode triggers including an "on load" +trigger that requires bytecode to be executed when the object is placed. So, bytecode +can be found in two different locations but is executed identically either way. + +## AP2 Place Camera Tag + +The place camera tag registers a camera by specifying where in 3D the camera is and +the focal length from the camera to the render plane. The camera always looks straight +down at the canvas and the focal length in practice alwys matches the Z offset of the +camera. This is only used in conjunction with objects that have a perspective transform +to correctly render the object based on its depth in the Z plane. Acting upon a place +camera tag means storing where the camera is for future projection perspective renders +when the engine goes to display all of the placed objects for a frame. + +# Quirks + +No format that is this complex will be without its quirks. At some point, Konami +hacked in 3D perspective support to the format. This is extremely rudimentary +and only allows for a camera looking straight down at the canvas and for 2D +quads to be transformed using a 3D perspective transform. Its my best guess that +artists were tired of having to render out individual frames and specify them +in order to achieve correct 3D rotation of arbitrary objects so this system was +put in place. There is no culling, no normals, no depth buffer, FOV, nothing. It +exists simply as a way to specify what depth to project a 2D quad that has been +rotated in 3D space. This is used in newer games mostly for perspective-correct +rotation of objects such as records. + +Now, unlike the 2D portion which was based on Flash and has had some 15 or so +years of testing to mature, the 3D portion is much more hacked in and has a host +of quriks. For instance, it appears that they did not do perspective-based masking +correctly and as such some animations which should have masking animations applied +to them appear to do so incorrectly. They also appear to have a bug where specifying +3D perspective projection on an object in a sprite and then placing that sprite +down on the main canvas results in a 2D affine projection instead of a 3D one. +These are all subtle enough that its quite possible QA never caught the problem or +the artist complained and was told it was not a big enough deal to fix. + +I briefly considered having a compatibility quirks flag system in order to attempt +to fix these issues and present a renderer that exactly matches what games do. +The problem with that is that this renderer already attempts to be compatible +with about a decade and a half of format changes. Given how much of the format +has been ripped out for newer games, this is already an impossible task. I would +also have to spend a great deal more time trying to figure out exactly HOW they +got these things subtly wrong to replicate them versus understanding how the format +was intended to work. I've made the decision that it is not worth my time to do so. + +# References + +Much of the understanding of this format came from the various online SWF file format +references as so much of the format mirrors Flash even today. Detailed documentation +on the format itself is entirely missing but the code in this directory provides a +working reference implementation of the format. Wherever this document is vague or +confusing, please refer to the code in `swf.py` for how to parse AFP, `render.py` for +how to render AFP, `container.py` for how to parse TXP2 files and `geo.py` for how to +parse shape structures. Please also refer to `decompile.py` for a ton of details on +how the bytecode itself works.