bemanitools/doc/development.md

Development

This document is intended for developers interested in contributing to Bemanitools. Please read this document before you start developing.

Goals

We want you to understand what this project is about and its goals. The following list serves as a guidance for all developers to identify valuable contributions for this project. As the project evolves, these gaols might do as well.

• Allow running Konami arcade rhythm games, i.e. games of the Bemani series, on arbitrary hardware.
  • Emulate required software and hardware features.
  • Provide means to cope with incompatibility issues resulting from using a different software platform (e.g. version of Windows).
• Provide an API for custom interfaces and configuring fundamental application features.

Development environment

The following tooling is required in order to build this project.

Tooling

Linux / MacOSX

• git
• make
• mingw-w64
• clang-format
• wine (optional, for running tests or some quick testing without requiring a VM)

On MacOSX, you can use homebrew or macports to install these packages.

Windows

TODO

IDE

Ultimately, you are free to use whatever you feel comfortable with for development. The following is our preferred development environment which we run on a Linux distribution of our choice:

• Visual Studio Code with the following extensions
  • C/C++
  • C++ Intellisense
  • Clang-Format

Further tools for testing and debugging

• Debugger: Can be part of your reverse engineering IDE of your choice or stand-along like OllyDbg.
• apitrace: Trace render calls to graphics APIs like D3D and OpenGL. This tool allows you to record and re-play render calls of an application with frame-by-frame debugging. Very useful to analyze the render pipeline or debug graphicial glitches.

Building

Simply run make in the root folder:

make

All output is located in the build folder including the final bemanitools.zip package.

Note about using -j n option on make: This is currently considered broken/unreliable. Expect to run into odd issues like randomly changing unresolved dependency errors. If you attempted this, run a make clean before running make again.

Release building

A release build is a clean build including code formatting and testing. This can be executed by running the following command:

make release

Building with docker

You can also build bemanitools using docker which avoids having to setup a full development environment if you are just interested in building binaries for the latest changes. Naturally, this requires you to have the docker daemon installed. Then, run the following command from the root folder of the project:

make build-docker

Once completed successfully, the build output is located in the build/docker sub-folder.

Creating releases

For developers to create official releases with major and minor versioning:

1. Ensure that all everything you want to have for this release is merged into master.
2. Create a changelog based on aggregating and summerizing commit messages of whatever got added starting from the tag of the current version to the current head of master. Commit that changelog and push to master.
3. Create a tag with the next version number, e.g. for 5.28:

git tag v5.28

1. Push the tag upstream:

git push origin v5.28

1. Wait for the CI pipeline to finish building the release and check if everything's ok.
2. On GitLab, go to "Tags" in the repository, click on the commit ID below the tag in the list you just created.
3. Click on the "Pipelines" tab and click on the download button on the right and on "Download release artifacts".
4. Rename the downloaded zip to "bemanitools-v5.28.zip" and the replace v5.28 with the version you want to release.
5. Upload the release.
6. For publishing the release, create a post with the following contents:

v5.28
<insert link to uploaded zip here>

<additional comments or things to point out for this release>

Changelog copy-paste:
<insert copy-pasted changelog you created previously here>

Code formatting

To apply our code style using clang-format, simply run the following command:

make code-format

Please also refer to the section about which cannot be covered using clang-format.

Testing

This still needs to be improved/implemented properly to run the unit-tests easily. Currently, you have to be on either a Linux/MacOSX system and run

make run-tests

This executes all currently available unit-tests and reports to the terminal any errors. This requires wine to be installed.

Project structure

Now that your setup is ready to go, here is brief big picture of what you find in this project.

• build: This folder will be generated once you have run the build process.
• dist: Distribution related files such as (default) configuration files, shell scripts, etc.
• doc: Documentation for the tools and hooks as well as some development related docs.
• src: The source code
  • imports: Provides headers and import definitions for AVS libs and a few other dependencies.
  • main: The main source code with game specific hook libraries, hardware emulation and application fixes.
  • test: Unit tests for modules that are not required for hooking or the presence of a piece of hardware.
• .clang-format: Code style for clang-format.
• GNUmakefile: Our makefile to build this project.
• Module.mk: Defines the various libraries and exe files to build for the distribution packages.

Code style and guidelines

Please follow these guidelines to keep a consistent style for the code base. Furthermore, we provide some best practices that have shown to be helpful. Please read them and reach out to us if you have any concerns or valuable additions/changes.

Clang-format

The style we agreed on is provided as a clang-format file. Therefore, we use clang-format for autoformatting our code.

You can use clang-format from your terminal but when using Visual Studio Code, just install the extension. Apply formatting manually at the end or enable the "reformat on save" feature.

However, clang-format cannot provide guidance to cover all our style rules. Therefore, we ask you to stick to the "additional" guidelines in the following sections.

Additional code style guidelines

No trailing comments

// NOPE
int var = 1; // this is a variable

// OK
// this is a variable
int var = 1;

Comment style

• Use either // or /* ... */ for single line.
• Use /* ... */ for multiline comments.
• Use /* ... */ for documentation.

Examples:

// single line comment
int var = 1;

/* another single line comment */
int var2 = 2;

/* multi
   line
   comment */

/**
 * This is a function.
 */
int func(int a, int b);

Include guards

Provide include guards for every header file. The naming follows the namespacing of the module and the module name.

Example for bsthook/acio.h file:

#ifndef BSTHOOK_ACIO_H
#define BSTHOOK_ACIO_H

// ...

#endif

Empty line before and after control blocks

Control blocks include: if, if-else, for, while, do-while, switch

Makes the code more readible with control blocks being easily visible.

Example

int var = 1;

if (var == 2) {
    // ...
}

printf("%d\n", var);

Empty line at the start of blocks

There are situations when you want to have an empty line at the start of the block to enhance readability, for example:

if (a && 
    b && 
    c) {
    d = 5;
}

Here the assignment aligns with the conditions of the if-black making it hard to read. Better:

if (a && 
    b && 
    c) {

    d = 5;
}

Includes

• Always keep all includes at the top of a header and source file. Exception: Documentation before include guards on header file.
• Use < > for system-based includes, e.g. <stdio.h>.
• Use " " for project-based includes, e.g. "util/log.h".
• For project-based includes, always use the full path relative to the root folder (src/main, src/test), e.g. "util/log.h" and not "log.h" when being in another module in the "util" namespace.
• Sorting
  • System-based includes before project-based includes
  • Block group them by different namespaces
  • Lex sort block groups
  • Because windows header files are a mess, the sorting on system-based includes is not always applicable. Please add a comment when applicable and apply the necessary order.

Example for sorting

#include <stdio.h>
#include <string.h>
#include <windows.h>

#include "iidxhook-util/acio.h"
#include "iidxhook-util/d3d9.h"

#include "util/log.h"
#include "util/mem.h"

Documentation

In general, add comments where required to explain a certain piece of code. If is not self-explanatory about:

• Why is it implemented like this
• Very important details to understand how and why it is working that only you know
• A complex algorithm/logic

Make sure to add some comments or even an extended document. Avoid comments that explain trivial and obvious things like "enable feature X" before an if-block with a feature switch.

Especially if it comes to reverse-engineering efforts, comments or even a separate document is crucial to allow others to understand the depths you dived into.

Any extended notes about documentation some hardware, protocol, reverse-engineering a feature etc. can be stored in the doc/dev folder and stay with the repository. Make good use of that!

Header files

Document any enum, struct or function exposed by a header file. Documentation of static functions or variables in source modules is not required. Also provide documentation for the module.

Example for my-namespace/my-module.h

/**
 * Some example module to show you where documentation is expected.
 */
#ifndef MY_NAMESPACE_MY_MODULE_H
#define MY_NAMESPACE_MY_MODULE_H

/**
 * Very useful enum for things.
 */
enum my_enum {
    MY_NAMESPACE_MY_MODULE_MY_ENUM_VAL_1 = 1,
    MY_NAMESPACE_MY_MODULE_MY_ENUM_VAL_2 = 2,
}

/**
 * Some cool data structure.
 */
struct my_struct {
    int a;
    float b;
};

/**
 * This is my awesome function doing great things.
 *
 * Here are some details about it:
 * - Detail 1
 * - Detail 2
 *
 * @param a If > 0, something happens.
 * @param b Only positive values valid, makes sure fancy things happen.
 * @return Result of the computation X which is always > 0. -1 on error.
 */
int my_namespace_my_module_func(int a, int b);

#endif

Naming conventions

In general, try to keep names short but don't overdo it by using abbrevations of things you created. Sometimes this is not possible and we accept exceptions if there are no proper alternatives.

Namespacing

The folder names use lower-case names with dashes - as seperators, e.g. my-namespace.

Modules

Header and source files of modules use lower-case names with dashes - as seperators, e.g. my-module.c, *my-module.h.

The include guards contain the name of the namespace and module, see [here](#### Include guards).

Variables, functions, structs, enums and macros are namespace accordingly.

Variables

Snake-case with proper namespacing to namespace and module. Namespacing applies to static and global variables. Local variables are not namespaced.

// For namespace "ezusb", module "device", static variable in module
static HANDLE ezusb_device_handle;

// Local variable in some function
int buffer_size = 256;

Functions

Snake-case with proper namespacing to namespace and module for all functions that are not hook functions.

// For namespace "ezusb", module "device", static variable in module, init function
void ezusb_device_init(...);

// CreateFileA hook function inside module
HANDLE my_CreateFileA(...)
{
    // ...
}

Structs

Snake-case with proper namespacing to namespace and module.

// For namespace "ezusb", module "device" ctx struct
struct ezusb_device_ctx {
    // ...
}

Enums

Snake-case with proper namespacing to namespace and module. Upper-case for enum entries

// For namespace "ezusb", module "device" state enum
struct EZUSB_DEVICE_STATE {
    EZUSB_DEVICE_STATE_INIT = 0,
    EZUSB_DEVICE_STATE_RUNNING = 1,
    // ...
}

Macros

Upper-case with underscore as spacing, proper namespacing to namespace and module.

// For namespace "ezusb", module "device" vid
#define EZUSB_DEVICE_VID 0xFFFF

Multiple variable declarations

Only one declaration per line which also avoids various errors, for example:

// Nope
char a, b, b;

// Ok
char a;
char b;
char c;

Testing

We advice you to write unit tests for all modules that allow proper unit testing. This applies to modules that are not part of the actual hooking process and do not rely on external devices to be available. Add these tests to the src/test sub-folder.

This does not only speed up your own development but hardens the code base and avoids having to test these things by running the real applications, hence saving a lot of time and trouble.

Further best practices

• Avoid external dependencies like additional libraries. Bemanitools is extremely self-contained which enables high portability and control which is important for implementing various "hacks" to just make things work.
• If you see some module/function lacking documentation that you have to use/understand, add documentation once you figured out what the function/module is doing. This does not only help your future you but others reading the code.
• Keep documentation and readme files up-to-date. When introducing changes/adding new features, review existing documentation and apply necessary changes.

Misc

The core API interception code was ripped out, cleaned up a tiny bit and released on GitHub. BT5 will eventually be ported to use this external library in order to avoid maintaining the same code in two places at once.

https://github.com/decafcode/capnhook

This too is a little rudimentary; it doesn't come with any examples or even a README yet.