Cool_Tools/bemanitools
1
0
Fork 0
mirror of https://github.com/djhackersdev/bemanitools.git synced 2025-02-26 22:49:28 +01:00
Code Releases Activity

doc: Dev journal entry about IIDX engine

Browse Source 
Due to recent development of tools and discussions, this
is something that came up and has probably been overdue
for ages.
This commit is contained in:
icex2 2025-02-09 15:21:40 +01:00
parent ce1a004bf6
commit addcdf8d16
1 changed files with 360 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

360
doc/dev/journal/2025-02-09-iidx-engine.md Normal file
View File

@ -0,0 +1,360 @@
# IIDX engine overview and how game-play is impacted by hardware and software
Date: 2025-02-09 Author: icex2
This document provides information and explains the following:
* A sort-of taxonomy of the different major IIDX game engine versions
* How the core game-play part of the engine works regarding
* Threading model
* Main render loop
* Timing and synchronization
* Conclude with connecting various known game-play impacting symptoms to root causes and suggesting
solutions to:
* Song de-syncing
* Micro-stuttering and non-smooth note scrolling
This document is not claiming to be complete or 100% accurate. It is based on my own personal
research and understanding of the IIDX game engine. It is very likely that some of the information
is wrong or misunderstood. Feel free to raise any questions or concerns in an issue as I want
this document to be as accurate as possible.
## Taxonomy of IIDX game engine versions
With every version, there have been numerous changes and improvements to the game engine. The
following is a rough taxonomy that is focused on cabinet and hardware changes. As these are likely
the main driver for key changes to the software and game engine, they create fairly distinct
stages throughout the different versions of the game. The key attributes to be considered here are:
* Main PCB
* IO board
* Monitor
All of these had significant impact on key software features appearing or changing.
### Twinkle hardware era: 1st Style to 8th Style
* [Twinkle PCB](https://github.com/shizmob/arcade-docs/blob/main/konami/boards.md#twinkle)
* [Original rear projection monitor](https://github.com/shizmob/arcade-docs/blob/main/konami/products.md#iidx-rear-projection-monitor)
* Homogenous hardware and software
### 1st gen PC-based upgrade only: 9th Style
* 9th Style came as an upgrade kit only to existing twinkle-based cabinet hardware
* [KNM-845G3-A02 PCB](https://github.com/shizmob/arcade-docs/blob/main/konami/boards.md#knm-845g3-a02)
* [C02 EZUSB FX USB IO board](https://github.com/shizmob/arcade-docs/blob/main/konami/io.md#gec02-pwbaa)
* Magnetic card readers connected to ezusb IO board
### 1st gen PC-based with dedicated cabinet: 10th Style to DistorteD (13)
* From here on, old cabinets that received the "Twinkle hardware" upgrade kit could always receive software only
upgrades
* [CRT monitor](https://github.com/shizmob/arcade-docs/blob/main/konami/products.md#iidx-crt-monitor) with new dedicated
cabinets
* [D01 EZUSB FX USB IO board](https://github.com/shizmob/arcade-docs/blob/main/konami/io.md#d01-io)
* Slotted card readers starting version 13
### 2nd gen PC: GOLD (14) to Lincle (19)
* [FAB-e945-KN205](https://github.com/shizmob/arcade-docs/blob/main/konami/boards.md#fab-e945-kn205)
* All prior cabinet configurations can be upgraded with a PCB and software upgrade
* New LCD monitor starting version 15
* Further variants of the LCD monitor appear starting version 18
* [IO2 EZUSB FX2 USB IO board](https://github.com/shizmob/arcade-docs/blob/main/konami/io.md#usbio2) introduced with
version 14
* Switch to wave pass readers on version 19
### 3rd gen PC: Tricoro (20) to Sinobuz (24)
* All prior cabinet configurations can be upgraded with a PCB and software upgrade
* [ADE-HM65](https://github.com/shizmob/arcade-docs/blob/main/konami/boards.md#hm65)
* Game engine supports HD mode (720p)
* Many new LCD monitor variants appear with every new version
### 4th gen PC: CANNON BALLERS (25) to RESIDENT (30)
* All prior cabinet configurations can be upgraded with a PCB and software upgrade
* [ADE-6291](https://github.com/shizmob/arcade-docs/blob/main/konami/boards.md#ade-6291)
* [BIO2 IO board](https://github.com/shizmob/arcade-docs/blob/main/konami/boards.md#hm65)
* Upgrade cabinets required the [BIO2 sub-IO board](https://github.com/shizmob/arcade-docs/blob/main/konami/io.md#bio2-ldj-sub-io)
* Drop of SD mode support starting version 27
* Supporting 120 hz monitors starting version 27
* Many new LCD monitor variants appear with every new version
* 42" 120 hz for lightning cabinets starting version 27
* 42" 60 hz for non-lightning cabinets starting version 25
### 5th gen PC: RESIDENT (30) to EPOLIS (31)
* All prior cabinet configurations can be upgraded with a PCB and software upgrade
* [C300-xonarae](https://github.com/shizmob/arcade-docs/blob/main/konami/boards.md#c300-xonarae)
* Game engine supports FHD mode (1080p)
## Threading model and core responsibilities in PC-based games
Since the early days of 9th Style to today (as of time of writing this: EPOLIS), the threading model
hasn't changed significantly:
* Main and rendering thread
* Runs your typical D3D9 render loop
* Using D3D9ex starting version 27
* Starting and managing other threads
* Any file and game asset loading
* Handles all the core game logic of the different screens starting from the initial boot screen
to the core game-play screen
* IO thread
* Polls the main IO hardware and synchronizes the in-memory input and output state with the
hardware
* Executes commands issued by the main thread to the IO board such as dongle or card reader
commands (ezusb generation of IO only)
* Card reader thread (not applicable to versions 9 to 12)
* Runs the protocol to read/write the hardware of the readers
* Async execution of higher level commands issued by the main thread to the card reader hardware,
e.g. card read, card eject etc.
* Audio thread
* Streaming of audio data to the sound API
* DirectSound for versions 9 to 26
* WASAPI for versions 27 to 30 for non-lighting cabinets with ADE-6291 hardware
* ASIO for versions 27 and newer for lighting cabinets
* ASIO for version 30/31 and newer for non-lighting cabinets with C300-xonarae hardware
* Async execution of higher level commands issued by the main thread to the audio backend, e.g.
play audio, stop audio, etc.
* Network thread
* Network plumbing with the xrpc protocol
* Async execution of request-response commands issued by the main thread
## Main and rendering thread
With the game having transitioning to different screens throughout it's lifecycle, the main render
loop might contain different logic such as asset loading. The following focuses on the main
game-play screen, only. Common rendering logic still applies to other screens.
This follows your typical D3D9 single threaded rendering loop which boils down to the following:
* Begin the scene (`BeginScene`)
* Set the render target (`SetRenderTarget`)
* Clear the back buffer (`Clear`)
* Run engine step. This includes and is mixed with
* IO input evaluation
* Game engine state and (re-) drawing the scene
* Commands to the audio playback, e.g. play key sounds
* Setting IO outputs
* End the scene (`EndScene`)
* Target a minimum frame time
* Introduced with version 11
* Sleep (`Sleep` or `SleepEx`) to fill up the frame up to either 13 ms or 14 ms (depending on the
version)
* This was likely introduced to not have the game run at hundreds of FPS if v-sync was disabled
(probably when running in window mode for development/testing purposes)
* Swapping the back buffers (`Present`)
* D3D9 is configured with v-sync enabled by default and targets 60 hz
* With this configuration, the call to `Present` fills up the remaining frame time to target the
configured 60 hz refresh rate
## Core engine timing and synchronization
The following outlines and explains the core engine concepts related to timing and synchronization.
### The origins, twinkle hardware
* It appears that the game has been on the same codebase since it's inception.
* This can be reasoned by the following observations
* 9th style as the first PC-based game very buggy on initial release
* 9th style threading model very clunky with various thread settings showing they tried to
prioritize different processing on a single core CPU
* Coming from twinkle hardware that is a realtime system vs. a Windows XP system
* Fixed playstation 1 based hardware with a fixed refresh rate of 60.925 hz
* No need to assume getting anything else if the resource budget per frame is not exceeded
* All sub-systems were driven synchronously to that fixed time step
* No need for the engine to synchronize audio during the song. When the song is started
correctly, it stayed sync throughout the song
* It appears the developers struggled a lot with the significant platform shift to PC as indicated
by various core engine improvements throughout the years still
* Switches of hardware generations further complicated the situation as these forced them to further
drift away from how the game engine was initially designed on the twinkle system
### Frame time/timing and engine step
* The game's engine step and timing is frame based
* The main render loop defines the minimum valid timing window for the game engine to be able to
evaluate
* 60 hz/fps = 16.667 ms (rounded to three decimal places)
* 120 hz/fps = 8.333 ms (rounded to three decimal places), only relevant for lightning cabinets with their 120 hz
screen/mode
* Depending on the GPU hardware and configuration, the GPU might drive the screen not exactly at
perfectly 60 hz/fps = 16.667 ms
* As outlined in the [main render loop](#main-and-rendering-thread), the `Present` call is the
key function call to determine the frame time for the game engine
* Just having v-sync enabled and telling D3D9 to **target** 60 hz, it still depends on how the GPU
driver is implemented and how the GPU driver's implementation of D3D9 executes on the `Present`
call
* This is primarily determined by how the GPU driver is operating and driving connected display
hardware
* The following factors can contribute to that either directly or indirectly
* Different monitor types analog vs. digital (CRT vs. LCD)
* Different monitor connections VGA vs. DVI/HDMI/DP
* Different GPU drivers
* Different GPU hardware and vendors
* Different GPU settings
### Monitor check screen
The monitor check screen was introduced with version 14. This was a solution to the problem of not
guaranteeing a single fixed refresh rate throughout different GPU and monitor combinations. See
a detailed explanation in the previous sub-subsection about
[what influences the frame time](#frame-timetiming-and-engine-step) of the game.
The goal of the monitor check is to measure the frame time of the main render loop and use the
results for [driving the core engine step](#engine-step-and-synchronization).
The monitor check evolved slightly as different GPU and monitor hardware, and GPU drivers yielded
varying refresh rates that determined the frame times of the main game loop.
Actual values are outlined further down in the [engine step time](#engine-step-time) section.
There are two main versions of the monitor check, all of them have always been part of the boot
process:
* 14
* Measures a total of 3000 frames
* If total time this took < 50004 ms -> so average of 16.668 ms per frame = 59.9952 hz
* S-Video mode -> 59.95 hz (also shows S-VIDEO when the monitor check completes)
* Otherwise VGA mode -> 60.05 hz
* If VGA is determined, the game patches the chart event data when loading
* `event_timestamp * 0.99817199 + 0.40000001`
* When S-Video no charts are patched and the timing data in the chart is used as is
* 15 to 18
* Identical to 14 except that chart patching now happens in milliseconds instead of frames
* VGA mode: `16.65279`
* S-Video mode: `16.680567`
* `event_timestamp * (1.0 / frame_time) + 0.40000001)`
* 19
* Identical to previous version
* Monitor check screen shows current FPS on the last revision of that version. However, it doesn't
use the value shown there by any means. Assuming this was added before the introduction of the
new monitor check on 20.
* 20 to 26
* Runs on boot or when switching monitor modes SD vs. HD
* Measures only 1200 frames
* Takes the average refresh rate of 1200 frames
* Applies the measured avg. frame time directly to the charts before song
* 27 to 30
* Frame rate measuring identical to prior versions
* Added additional logic after the frame time measurements for handling of 120 hz displays
* Includes checking the monitor and if the framerate matches as expected
* 30/31 LDJ-010 upgrade
* Added NvDisplayConfigLDJ tool which runs on system start and configures a custom timing in the
NVIDIA GPU driver to ensure a fixed monitor refresh rate
* 31
* Same code as before
* Consider the first 460 frames of the measurement as warm-up frames
* Discard these for the actual measurement to get more stable frame times
* Use a base avg. value of 119.982 for the 120 hz monitors and 59.9 for 60 hz monitors. Further
frames measured are counted towards the average with the given base value
### Engine core game-play with visual and audio synchronization
#### Chart data
* Chart data encodes note timing and timing window information
* Chart data consists of *note events* encoding a timestamp and note information such as type
and key-sound information
* Timing windows are defined per chart
* Most infamous example: GAMBOL (7Key/Another)
* Game engine timestamp evaluation and note event timestamp encoding
* 9 to 14: Refresh rate in hz
* 15 to 31: Milliseconds (equivalent to assuming 1000 fps)
* Fundamentally, that doesn't change game play experience as it's just a different way of
evaluating the timestamps since refresh rate in hz/frame rate in fps can be converted lossless to
frame time in ms
#### Engine step time
The engine uses the following source for frame rate/time to drive internal logic on the following
versions
* 9 to 13: Hardcoded 59.95 hz
* 14: Either 59.95 hz or 60.05 hz determined by which value is determined to be closer to the
measured one during the monitor check screen on startup
* 15 to 19: 59.95 hz or 60.05 hz
* Monitor check on startup measures the frame time and picks the value closest to what's been
measured
* Monitor check result can be overridden with the *Output Type* setting in the operator menu
* *S-Video* to force to 59.95 hz
* *VGA* to force to 60.05 hz
* 20 to 31: Monitor check on boot determines the frame time and uses the result throughout the
entire game session
#### Engine step and synchronization
Per frame engine step advancing and synchronization during game-play
* Game-play needs synchronization of the following aspects
* Drawn content on the screen
* Background audio playback track
* Key sound effects
* The game engine synchronizes these only at the start of a song
* It takes known/determined refresh rate/frame time as the fixed time for every frame cycle
throughout the entire song
* It starts playback of the background audio track together with playback of the chart data
* With every engine step/frame it advances internal calculations by that fixed amount of time.
Thus, the game engine expects this to be constant/stable throughout an entire song
* The chart data is advanced every frame by the engine
* Key sound events are turned into audio playback on command as the chart advances
* No re-synchronization or check-pointing between main rendering thread and audio is happening
throughout the song
* Remark: The above does not consider any kind of offsetting and timing adjustments for the sake
of keeping this simple and focus on the absolute core concepts
## Known game-play impacting symptoms and their root causes
### Song de-syncing
Throughout playing a song, the audio starts de-syncing even it was fine when the song started
still.
This is caused by a mismatch of expected frame time by the game engine and actually delivered
frame time by the GPU driver. This can come as a stable frame time that just mismatches in general
or (highly) fluctuating frame times.
Since the game engine works on a fixed frame time and assumes that this is constant, this
can cause the chart event time to drift/fluctuate over time. Game-play timing starts to feel off
and key sounds are played off-sync to the background audio track
Highly fluctuating frame times are likely to be caused by some configuration of the GPU driver
settings and display combination:
* G-sync on modern displays causes known fluctuations
* V-sync not correctly configured/enforced or even vsync-off override in driver
* Incorrect display timings
Stable frame times but not having the correct ones that the game engine expects is a different
problem that can be addressed by:
* Knowing the [refresh rates expected by the game engine](#frame-timetiming-and-engine-step) of the
different game versions
* Configuring your GPU driver's display timings to match the expected frame times
* Or, patch the chart data with your custom display timings so the game engine gets to know these
* Note: Offline patching the chart data doesn't work for several game versions as the monitor
check does override this data, but not as flexible as required
* Use the chart patch feature of bemanitools instead
### Micro-stuttering and non-smooth note scrolling
This is a slightly different problem which isn't really timing related. However, it can be confusing
as one might assume that stuttering can be caused by the frame times that are visible to the game.
However, any kind of power states and energy saving isn't visible to the game and rendering loop
and typically cannot be seen in frame time metrics either. This results in non-smooth scrolling
and note stuttering during game-play even the game engine runs perfectly fine at the expected
and stable frame times.
These symptoms must be addressed by:
* Correct CPU power configuration on Windows (non-energy saving modes)
* Correct CPU C-states configuration in BIOS/UEFI
* These cannot be configured in Windows and are separate from Windows's power configuration
* GPU power configuration
* Configured via the driver's configuration utility
* Other stuff running in the background of the system causing fluctuations
* Disk I/O heavy applications
* Or anything creating a considerable amount of CPU or GPU load