Cool_Tools/bemanitools
1
0
Fork 0
mirror of https://github.com/djhackersdev/bemanitools.git synced 2024-12-12 06:21:05 +01:00
Code Releases Activity
70e8e2a965
bemanitools/doc/architecture.md

240 lines
8.2 KiB
Markdown
Raw Normal View History

Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
# Bemanitools architecture
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
This document gives you an overview of the architecture of Bemanitools (5). Why do we need this?
This helps document the various design decisions and how everything comes together, in the end. At
some point when things are evolving and other developers want to pick up the project, this might
give them answers to why some things were done well or not so well. This allows them to iterate on
the current design (document) to analyze if a change they want to apply might work out or not.
Anway, enough preface...I guess you got the idea. This document will be split into several sections
which address different aspects of the architecture.
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
## The big picture
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
TODO create a graphic that presents the key modules and ideas.
## Detouring library functions, IAT hooking
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
One of Bemanitools's goals is to avoid patching of executables, libraries or any game data and
instead rely on intercepting calls to libraries to patch bugs or introduce new features. To support
a game on a non native platform transparently, e.g. your home desktop, this allows us to intercept
I/O communication to emulate hardware or files/filesystem features the game expects to be available.
We create a so called "hook library/hook.dll" which gets injected to the target process (the game)
before the game runs any of its application code. This is well known by the terms of dll injection
and will not be further discussed here (google it). The injected dll will replace the function
addresses in the IATs to detour any calls to our own handler functions before, eventually, calling
the real function. This way, we can intercept the call and do cool things.
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
These features are covered by the modules in the following subfolders in src/main:
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- hook: Essentially, this is capnhook: https://github.com/decafcode/capnhook. General tools for
hooking Win32 API calls.
- hooklib: Some additional helper modules to take care of specific issues in Bemanitools, e.g. rs232
related stuff for ACIO.
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
Check the modules for details.
## Hooking and IRP
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
IRP stands for "I/O request packet" and is a kernel mode structure used in Windows drivers for
communication with the OS. The data structure describes an I/O request with parameters for that
request avoiding function calls with large number of arguments to a driver.
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
We make use of that "IRP pattern" by creating a flexible and maintainable abstraction layer for the
following hooking modules:
- iohook: Hook I/O (e.g. file) related calls
- d3d9: Hook d3d9 graphics API calls
An IRP handler is implemented to handle selected IRP calls based on the specified operation type.
IRP handlers can be chained which allows splitting up different features/interceptors to different
functions and modules allowing you to create a clear structure. At the end of the forwarding chain,
you the real API function that maps to the abstracted operation is called. However, a handler can
decide at any time to not forward calls which allows you to implement emulation of access to
selected files or I/O devices.
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
Modules that make use of this:
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- iidxhook-util/d3d9
- ezusb-emu/device
- ezusb2-emu/device
- acioemu/emu
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
Support pop'n music 15-18 * popnhook1 for pop'n 15 - 18 has been added * popnio has been added * inject.exe has new syntax for loading hook DLLs, `real.dll=hook.dll`. This will trigger an early IAT hook where it will load the EXE suspended without resolving imports, replace the reference to real.dll in the import table with hook.dll, and then resolve everything before launching. This allows for ezusb.dll to be hooked properly. * launcher.exe also has a new early IAT hook feature now. Use `-I real.dll=hook.dll`. The idea is the same as described above for inject.exe. * Updated ezusb constant namings based on what is visible in ezusb.dll's debug statements. The launcher.exe implementation of early IAT hooking means that someone can implement popnhook2.dll for 19 and above. I have tried pop'n music Sunny Park using a modified version of popnhook1 and it seems to work to some degree: the I/O check and security check returns OK which means the ezusb hooking used in popnhook1 is also working for the later games using `launcher.exe -I ezusb.dll=ezusb2-popn-shim.dll ...`. The process is rather invasive (manually resolving all imports means more chances to fail) so it has been implemented in such a way that the launcher will work the same as it has before as long as `-I` isn't specified. One questionable thing I am not confident about is the `texture_usage_fix` hack flag I added in the conf. As the comment says, pop'n music 16 will work in Windows XP without the flag being set, but the game will immediately crash on later OSes without the flag being set in my experience. No other games had this issue in my experience. Enabling it in other games doesn't seem to have any negative effects.
2022-06-11 00:46:59 +02:00
## Bemanitools's hook libraries, let's glue everything together
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Bemanitools dlls to be injected into target game processes are refered to as "hook dlls" and come in
different flavours targetting different games and often different versions of the same game
(series), for example:
- ddrhook: Hook dll for Dance Dance Revolution games
- iidxhook1-8: Hook dlls for Beatmania IIDX, we are currently at 8 different implementations due to
the various iterations the game went through, related to software and hardware.
- sdvxhook: Hook dll for SoundVoltex series
The following sub-sections will give you some brief insights on each hook implementation and what
modules were used.
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
### bsthook
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
TODO
### ddrhook
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
TODO
### iidxhook
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
IIDX went through so many hard- and software iterations, it's actually amazing that the development
team(s) refactored and improved parts of the game and hardware with each iteration. However, when
facing emulation and supporting compatibility to legacy OS platforms, it can't get any worse. On the
bright side, IIDX helped shaping Bemanitools a lot and created a solid foundation other games can
build on.
Because of that, we have 8 iidxhook implementations supporting sometimes different software
features/fixes and hardware. The following sub-sub-sections list the most relevant aspects and
modules to point out common and different higher level features.
architecture doc minor update
2019-10-07 20:12:33 +02:00
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Essentially, the main module file of each iidxhook implementation just glues the APIs of the modules
it requires together. An additional configuration layer allows users to tweak some of the features.
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
TODO go into some more detail about some differences in the hook modules, e.g. 18 to 19 look
identical -> but different AVS versions
Update changelog
2019-10-12 20:44:51 +02:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
#### iidxhook1 (9-12)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- Ezusb C02 I/O emulation
- Setupapi emulation
- Full security emulation with SRAM and round plugs
- Full serial emulation for magstripe card readers
- Full game essential I/O emulation
- d3d8 patching and extended features (superseded by d3d9 hook module + d3d8to9 wrapper)
- clock patching
- Font patching for Japanese chars
- Network related patches to enable eamuse to custom servers
- Filesystem patches to detour E and F backup drives for settings
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
#### iidxhook2 (13)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- Ezusb C02 I/O emulation
- Setupapi emulation
- Full security emulation with SRAM and round plugs
- Full game essential I/O emulation
- ACIO ICCA card reader emulation, slotted readers
- d3d8 patching and extended features (superseded by d3d9 hook module + d3d8to9 wrapper)
- clock patching
- Font patching for Japanese chars
- Network related patches to enable eamuse to custom servers
- Filesystem patches to detour E and F backup drives for settings
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
#### iidxhook3 (14-17)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- Ezusb IO2 I/O emulation
- Setupapi emulation
- Full security emulation with SRAM and round plugs
- Full game essential I/O emulation
- ACIO ICCA card reader emulation, slotted readers
- d3d9 patching and extended features
- Font patching for Japanese chars
- Network related patches to enable eamuse to custom servers
- Filesystem patches to detour E and F backup drives for settings
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
#### iidxhook4 (18)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- Ezusb IO2 I/O emulation
- Setupapi emulation
- Full game essential I/O emulation
- ACIO ICCA card reader emulation, slotted readers
- d3d9 patching and extended features
- Font patching for Japanese chars
- Filesystem patches to detour E and F backup drives for settings
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
iidxhook4-cn: Add support for IIDX Resort Anthem CN * Exe based game * No network functionality * No card readers checked/used
2022-06-12 17:27:53 +02:00
#### iidxhook4-cn (18 CN)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- Ezusb IO2 I/O emulation
- Setupapi emulation
- Full security emulation with SRAM and round plugs
- Full game essential I/O emulation
- d3d9 patching and extended features
- Font patching for Japanese chars
- Filesystem patches to detour E and F backup drives for settings
iidxhook4-cn: Add support for IIDX Resort Anthem CN * Exe based game * No network functionality * No card readers checked/used
2022-06-12 17:27:53 +02:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
#### iidxhook5 (19)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- Ezusb IO2 I/O emulation
- Setupapi emulation
- Full game essential I/O emulation
- ACIO ICCA card reader emulation, wave pass readers
- d3d9 patching and extended features
- Font patching for Japanese chars
- Filesystem patches to detour E and F backup drives for settings
iidxhook4-cn: Add support for IIDX Resort Anthem CN * Exe based game * No network functionality * No card readers checked/used
2022-06-12 17:27:53 +02:00
#### iidxhook5-cn (20 CN)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- Ezusb IO2 I/O emulation
- Setupapi emulation
- Full security emulation with SRAM and round plugs
- Full game essential I/O emulation
- d3d9 patching and extended features
- Font patching for Japanese chars
- Filesystem patches to detour E and F backup drives for settings
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
#### iidxhook6 (20)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- Ezusb IO2 I/O emulation
- Setupapi emulation
- Full game essential I/O emulation
- ACIO ICCA card reader emulation, wave pass readers
- d3d9 patching and extended features
- Font patching for Japanese chars
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
#### iidxhook7 (21-24)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- Ezusb IO2 I/O emulation
- Setupapi emulation
- Full game essential I/O emulation
- ACIO ICCA card reader emulation, wave pass readers
- d3d9 patching and extended features
- Font patching for Japanese chars
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
#### iidxhook8 (25-26)
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
- ACIO BIO2 I/O emulation
- Setupapi emulation
- Full game essential I/O emulation
- ACIO ICCA card reader emulation, wave pass readers
- d3d9 patching and extended features
- Font patching for Japanese chars
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
### jbhook
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
TODO
Support pop'n music 15-18 * popnhook1 for pop'n 15 - 18 has been added * popnio has been added * inject.exe has new syntax for loading hook DLLs, `real.dll=hook.dll`. This will trigger an early IAT hook where it will load the EXE suspended without resolving imports, replace the reference to real.dll in the import table with hook.dll, and then resolve everything before launching. This allows for ezusb.dll to be hooked properly. * launcher.exe also has a new early IAT hook feature now. Use `-I real.dll=hook.dll`. The idea is the same as described above for inject.exe. * Updated ezusb constant namings based on what is visible in ezusb.dll's debug statements. The launcher.exe implementation of early IAT hooking means that someone can implement popnhook2.dll for 19 and above. I have tried pop'n music Sunny Park using a modified version of popnhook1 and it seems to work to some degree: the I/O check and security check returns OK which means the ezusb hooking used in popnhook1 is also working for the later games using `launcher.exe -I ezusb.dll=ezusb2-popn-shim.dll ...`. The process is rather invasive (manually resolving all imports means more chances to fail) so it has been implemented in such a way that the launcher will work the same as it has before as long as `-I` isn't specified. One questionable thing I am not confident about is the `texture_usage_fix` hack flag I added in the conf. As the comment says, pop'n music 16 will work in Windows XP without the flag being set, but the game will immediately crash on later OSes without the flag being set in my experience. No other games had this issue in my experience. Enabling it in other games doesn't seem to have any negative effects.
2022-06-11 00:46:59 +02:00
### popnhook
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Support pop'n music 15-18 * popnhook1 for pop'n 15 - 18 has been added * popnio has been added * inject.exe has new syntax for loading hook DLLs, `real.dll=hook.dll`. This will trigger an early IAT hook where it will load the EXE suspended without resolving imports, replace the reference to real.dll in the import table with hook.dll, and then resolve everything before launching. This allows for ezusb.dll to be hooked properly. * launcher.exe also has a new early IAT hook feature now. Use `-I real.dll=hook.dll`. The idea is the same as described above for inject.exe. * Updated ezusb constant namings based on what is visible in ezusb.dll's debug statements. The launcher.exe implementation of early IAT hooking means that someone can implement popnhook2.dll for 19 and above. I have tried pop'n music Sunny Park using a modified version of popnhook1 and it seems to work to some degree: the I/O check and security check returns OK which means the ezusb hooking used in popnhook1 is also working for the later games using `launcher.exe -I ezusb.dll=ezusb2-popn-shim.dll ...`. The process is rather invasive (manually resolving all imports means more chances to fail) so it has been implemented in such a way that the launcher will work the same as it has before as long as `-I` isn't specified. One questionable thing I am not confident about is the `texture_usage_fix` hack flag I added in the conf. As the comment says, pop'n music 16 will work in Windows XP without the flag being set, but the game will immediately crash on later OSes without the flag being set in my experience. No other games had this issue in my experience. Enabling it in other games doesn't seem to have any negative effects.
2022-06-11 00:46:59 +02:00
TODO
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
### sdvxhook
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
TODO
## The ezusb (emulation) stack
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
TODO
## The ACIO (emulation) stack
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
TODO
## BT5 API
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
TODO
## AVS and launcher
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
Add initial draft of architecture document. Still has a bunch of TODOs where more information is required. This commit adds the first few sections and some information about iidxhook.
2019-10-07 20:02:32 +02:00
TODO
## Inject
chore: Apply formatting to all markdown docs
2024-01-29 23:18:21 +01:00
TODO
Copy Permalink