Merge pull request 'Initial D THE ARCADE support added' (#41) from Dniel97/artemis:idac into develop

Reviewed-on: https://gitea.tendokyu.moe/Hay1tsme/artemis/pulls/41

Dockerfile

@@ -15,6 +15,7 @@ COPY dbutils.py dbutils.py
 COPY read.py read.py
 ADD core core
 ADD titles titles
+ADD config config
 ADD logs logs
 ADD cert cert
 

changelog.md

@@ -8,6 +8,18 @@ Documenting updates to ARTEMiS, to be updated every time the master branch is pu
 ### Card Maker
 + Added support for maimai DX FESTiVAL PLUS
 
+## 20231001
+### Initial D THE ARCADE
++ Added support for Initial D THE ARCADE S2
+  + Story mode progress added
+  + Bunta Challenge/Touhou Project modes added
+  + Time Trials added
+  + Leaderboards added, but doesn't refresh sometimes
+  + Theory of Street mode added (with CPUs)
+  + Play Stamp/Timetrial events added
+  + Frontend to download profile added
+  + Importer to import profiles added
+
 ## 20230716
 ### General
 + Docker files added (#19)

docs/INSTALL_WINDOWS.md

@@ -10,11 +10,11 @@ This step-by-step guide assumes that you are using a fresh install of Windows 10
     3. Make sure that you enable "Create shortcuts for installed applications" and "Add Python to environment variables" and hit Install
 
 ## Install MySQL 8.0
-1. Download MySQL 8.0 Server : [Link](https://cdn.mysql.com//Downloads/MySQLInstaller/mysql-installer-web-community-8.0.31.0.msi)
+1. Download MySQL 8.0 Server : [Link](https://dev.mysql.com/get/Downloads/MySQLInstaller/mysql-installer-community-8.0.34.0.msi)
-2. Install mysql-installer-web-community-8.0.31.0.msi
+2. Install mysql-installer-web-community-8.0.34.0.msi
     1. Click on "Add ..." on the side
     2. Click on the "+" next to MySQL Servers
-    3. Make sure MySQL Server 8.0.29 - X64 is under the products to be installed.
+    3. Make sure MySQL Server 8.0.34 - X64 is under the products to be installed.
     4. Hit Next and Next once installed
     5. Select the configuration type "Development Computer"
     6. Hit Next
@@ -23,9 +23,10 @@ This step-by-step guide assumes that you are using a fresh install of Windows 10
     9. Leave everything under Windows Service as default and hit Next >
     10. Click on Execute and for it to finish and hit Next> and then Finish
 3. Open MySQL 8.0 Command Line Client and login as your root user
-4. Type those commands to create your user and the database
+4. Change `<Enter Password Here>` to a new password for the user aime, type those commands to create your user and the database
-```
+
-CREATE USER 'aime'@'localhost' IDENTIFIED BY 'MyStrongPass.';
+```sql
+CREATE USER 'aime'@'localhost' IDENTIFIED BY '<Enter Password Here>';
 CREATE DATABASE aime;
 GRANT Alter,Create,Delete,Drop,Index,Insert,References,Select,Update ON aime.* TO 'aime'@'localhost';
 FLUSH PRIVILEGES;
@@ -34,33 +35,50 @@ exit;
 
 ## Install Python modules
 1. Change your work path to the artemis-master folder using 'cd' and install the requirements:
-> pip install -r requirements.txt
 
-## Copy/Rename the folder example_config to config
+```shell
-
+pip install -r requirements.txt
-## Adjust /config/core.yaml
-
-1. Make sure to change the server listen_address to be set to your local machine IP (ex.: 192.168.1.xxx) 
-    - In case you want to run this only locally, set the following values:
 ```	
+
+## Copy/Rename the folder `example_config` to `config`
+
+## Adjust `config/core.yaml`
+
+1. Make sure to change the server `hostname` to be set to your local machine IP (ex.: 192.168.xxx.xxx) 
+    - In case you want to run this only locally, set the following values:
+
+```yaml
 server:
     listen_address: 0.0.0.0
 title: 
-    hostname: localhost
+    hostname: 192.168.xxx.xxx
+```
+
+1. Adjust the proper MySQL information you created earlier
+```yaml
+database:
+  host: "localhost"
+  username: "aime"
+  password: "<Enter Password Here>"
+  name: "aime"
 ```
-2. Adjust the proper MySQL information you created earlier
 3. Add the AimeDB key at the bottom of the file
 4. If the webui is needed, change the flag from False to True
 
 ## Create the database tables for ARTEMiS
-> python dbutils.py create
+
+```shell
+python dbutils.py create
+```
 
 ## Firewall Adjustements 
 Make sure the following ports are open both on your router and local Windows firewall in case you want to use this for public use (NOT recommended):
 > Port 80 (TCP), 443 (TCP), 8443 (TCP), 22345 (TCP), 8080 (TCP), 8090 (TCP) **webui, 8444 (TCP) **mucha
 
 ## Running the ARTEMiS instance
-> python index.py
+```shell
+python index.py
+```
 
 # Troubleshooting
 
@@ -78,6 +96,7 @@ Make sure the following ports are open both on your router and local Windows fir
 ## AttributeError: module 'collections' has no attribute 'Hashable'
 1. This means the pyYAML module is obsolete, simply rerun pip with the -U (force update) flag, as shown below.
     - Change your work path to the artemis-master (or artemis-develop) folder using 'cd' and run the following commands:
-```
+
+```shell
 pip install -r requirements.txt -U
 ```

docs/game_specific_info.md

@@ -6,6 +6,12 @@ the corresponding importer and database upgrades.
 **Important: The described database upgrades are only required if you are using an old database schema, f.e. still
 using the megaime database. Clean installations always create the latest database structure!**
 
+To upgrade the core database and the database for every game, execute:
+
+```shell
+python dbutils.py autoupgrade
+```
+
 # Table of content
 
 - [Supported Games](#supported-games)
@@ -16,6 +22,7 @@ using the megaime database. Clean installations always create the latest databas
     - [Card Maker](#card-maker)
     - [WACCA](#wacca)
     - [Sword Art Online Arcade](#sao)
+	- [Initial D THE ARCADE](#initial-d-the-arcade)
 
 
 # Supported Games
@@ -27,7 +34,7 @@ Games listed below have been tested and confirmed working.
 ### SDBT
 
 | Version ID | Version Name          |
-|------------|-----------------------|
+| ---------- | --------------------- |
 | 0          | CHUNITHM              |
 | 1          | CHUNITHM PLUS         |
 | 2          | CHUNITHM AIR          |
@@ -43,7 +50,7 @@ Games listed below have been tested and confirmed working.
 ### SDHD/SDBT
 
 | Version ID | Version Name        |
-|------------|---------------------|
+| ---------- | ------------------- |
 | 11         | CHUNITHM NEW!!      |
 | 12         | CHUNITHM NEW PLUS!! |
 | 13         | CHUNITHM SUN        |
@@ -83,9 +90,7 @@ crypto:
 
 ### Database upgrade
 
-Always make sure your database (tables) are up-to-date, to do so go to the `core/data/schema/versions` folder and see
+Always make sure your database (tables) are up-to-date:
-which version is the latest, f.e. `SDBT_4_upgrade.sql`. In order to upgrade to version 4 in this case you need to
-perform all previous updates as well:
 
 ```shell
 python dbutils.py --game SDBT upgrade
@@ -146,7 +151,7 @@ The songId is based on the actual ID within your version of Chunithm.
 ### SDCA
 
 | Version ID | Version Name                       |
-|------------|------------------------------------|
+| ---------- | ---------------------------------- |
 | 0          | crossbeats REV.                    |
 | 1          | crossbeats REV. SUNRISE            |
 | 2          | crossbeats REV. SUNRISE S2         |
@@ -167,7 +172,7 @@ The importer for crossbeats REV. will import Music.
 Config file is located in `config/cxb.yaml`.
 
 | Option                | Info                                                       |
-|------------------------|------------------------------------------------------------|
+| --------------------- | ---------------------------------------------------------- |
 | `hostname`            | Requires a proper `hostname` (not localhost!) to run       |
 | `ssl_enable`          | Enables/Disables the use of the `ssl_cert` and `ssl_key`   |
 | `port`                | Set your unsecure port number                              |
@@ -180,12 +185,12 @@ Config file is located in `config/cxb.yaml`.
 ### SDEZ
 
 | Game Code | Version ID | Version Name |
-|-----------|------------|-------------------------|
+| --------- | ---------- | ------------ |
 
 
 For versions pre-dx
 | Game Code | Version ID | Version Name            |
-|-----------|------------|-------------------------|
+| --------- | ---------- | ----------------------- |
 | SBXL      | 0          | maimai                  |
 | SBXL      | 1          | maimai PLUS             |
 | SBZF      | 2          | maimai GreeN            |
@@ -227,11 +232,12 @@ The importer for maimai Pre-DX will import Events and Music. Not all games will
 
 ### Database upgrade
 
-Always make sure your database (tables) are up-to-date, to do so go to the `core/data/schema/versions` folder and see which version is the latest, f.e. `SDEZ_2_upgrade.sql`. In order to upgrade to version 2 in this case you need to perform all previous updates as well:
+Always make sure your database (tables) are up-to-date:
 
 ```shell
 python dbutils.py --game SDEZ upgrade
 ```
+
 Pre-Dx uses the same database as DX, so only upgrade using the SDEZ game code!
 
 ## Hatsune Miku Project Diva
@@ -239,7 +245,7 @@ Pre-Dx uses the same database as DX, so only upgrade using the SDEZ game code!
 ### SBZV
 
 | Version ID | Version Name                    |
-|------------|---------------------------------|
+| ---------- | ------------------------------- |
 | 0          | Project Diva Arcade             |
 | 1          | Project Diva Arcade Future Tone |
 
@@ -260,7 +266,7 @@ the Shop, Modules and Customizations.
 Config file is located in `config/diva.yaml`.
 
 | Option               | Info                                                                                            |
-|----------------------|-------------------------------------------------------------------------------------------------|
+| -------------------- | ----------------------------------------------------------------------------------------------- |
 | `unlock_all_modules` | Unlocks all modules (costumes) by default, if set to `False` all modules need to be purchased   |
 | `unlock_all_items`   | Unlocks all items (customizations) by default, if set to `False` all items need to be purchased |
 
@@ -270,9 +276,7 @@ In order to use custom PV Lists, simply drop in your .dat files inside of /title
 
 ### Database upgrade
 
-Always make sure your database (tables) are up-to-date, to do so go to the `core/data/schema/versions` folder and see
+Always make sure your database (tables) are up-to-date:
-which version is the latest, f.e. `SBZV_4_upgrade.sql`. In order to upgrade to version 4 in this case you need to
-perform all previous updates as well:
 
 ```shell
 python dbutils.py --game SBZV upgrade
@@ -283,7 +287,7 @@ python dbutils.py --game SBZV upgrade
 ### SDDT
 
 | Version ID | Version Name               |
-|------------|----------------------------|
+| ---------- | -------------------------- |
 | 0          | O.N.G.E.K.I.               |
 | 1          | O.N.G.E.K.I. +             |
 | 2          | O.N.G.E.K.I. SUMMER        |
@@ -311,7 +315,7 @@ The importer for O.N.G.E.K.I. will all all Cards, Music and Events.
 Config file is located in `config/ongeki.yaml`.
 
 | Option           | Info                                                                                                           |
-|------------------|----------------------------------------------------------------------------------------------------------------|
+| ---------------- | -------------------------------------------------------------------------------------------------------------- |
 | `enabled_gachas` | Enter all gacha IDs for Card Maker to work, other than default may not work due to missing cards added to them |
 | `crypto`         | This option is used to enable the TLS Encryption                                                               |
 
@@ -328,9 +332,7 @@ crypto:
 
 ### Database upgrade
 
-Always make sure your database (tables) are up-to-date, to do so go to the `core/data/schema/versions` folder and see
+Always make sure your database (tables) are up-to-date:
-which version is the latest, f.e. `SDDT_4_upgrade.sql`. In order to upgrade to version 4 in this case you need to
-perform all previous updates as well:
 
 ```shell
 python dbutils.py --game SDDT upgrade
@@ -403,7 +405,7 @@ After that, on next login the present should be received (or whenever it suppose
 ### SDED
 
 | Version ID | Version Name    |
-|------------|-----------------|
+| ---------- | --------------- |
 | 0          | Card Maker 1.30 |
 | 1          | Card Maker 1.35 |
 
@@ -525,7 +527,7 @@ Gacha IDs up to 1140 will be loaded for CM 1.34 and all gachas will be loaded fo
 ### SDFE
 
 | Version ID | Version Name  |
-|------------|---------------|
+| ---------- | ------------- |
 | 0          | WACCA         |
 | 1          | WACCA S       |
 | 2          | WACCA Lily    |
@@ -548,7 +550,7 @@ The importer for WACCA will import all Music data.
 Config file is located in `config/wacca.yaml`.
 
 | Option             | Info                                                                        |
-|--------------------|-----------------------------------------------------------------------------|
+| ------------------ | --------------------------------------------------------------------------- |
 | `always_vip`       | Enables/Disables VIP, if disabled it needs to be purchased manually in game |
 | `infinite_tickets` | Always set the "unlock expert" tickets to 5                                 |
 | `infinite_wp`      | Sets the user WP to `999999`                                                |
@@ -557,7 +559,7 @@ Config file is located in `config/wacca.yaml`.
 
 ### Database upgrade
 
-Always make sure your database (tables) are up-to-date, to do so go to the `core/data/schema/versions` folder and see which version is the latest, f.e. `SDFE_3_upgrade.sql`. In order to upgrade to version 3 in this case you need to perform all previous updates as well:
+Always make sure your database (tables) are up-to-date:
 
 ```shell
 python dbutils.py --game SDFE upgrade
@@ -603,7 +605,7 @@ Below is a list of VIP rewards. Currently, VIP is not implemented, and thus thes
 ### SDEW
 
 | Version ID | Version Name |
-|------------|---------------|
+| ---------- | ------------ |
 | 0          | SAO          |
 
 
@@ -622,7 +624,7 @@ The importer for SAO will import all items, heroes, support skills and titles da
 Config file is located in `config/sao.yaml`.
 
 | Option          | Info                                                              |
-|--------------------|-----------------------------------------------------------------------------|
+| --------------- | ----------------------------------------------------------------- |
 | `hostname`      | Changes the server listening address for Mucha                    |
 | `port`          | Changes the listing port                                          |
 | `auto_register` | Allows the game to handle the automatic registration of new cards |
@@ -630,7 +632,7 @@ Config file is located in `config/sao.yaml`.
 
 ### Database upgrade
 
-Always make sure your database (tables) are up-to-date, to do so go to the `core/data/schema/versions` folder and see which version is the latest, f.e. `SDEW_1_upgrade.sql`. In order to upgrade to version 3 in this case you need to perform all previous updates as well:
+Always make sure your database (tables) are up-to-date:
 
 ```shell
 python dbutils.py --game SDEW upgrade
@@ -650,3 +652,134 @@ python dbutils.py --game SDEW upgrade
 - Midorica - Limited Network Support
 - Dniel97 - Helping with network base
 - tungnotpunk - Source
+
+## Initial D THE ARCADE
+
+### SDGT
+
+| Version ID | Version Name                  |
+| ---------- | ----------------------------- |
+| 0          | Initial D THE ARCADE Season 1 |
+| 1          | Initial D THE ARCADE Season 2 |
+
+**Important: Only version 1.50.00 (Season 2) is currently working and actively supported!**
+
+### Profile Importer
+
+In order to use the profile importer download the `idac_profile.json` file from the frontend
+and either directly use the folder path with `idac_profile.json` in it or specify the complete
+path to the `.json` file
+
+```shell
+python read.py --game SDGT --version <Version ID> --optfolder /path/to/game/download/folder
+```
+
+The importer for SDGT will import the complete profile data with personal high scores as well.
+
+### Config
+
+Config file is located in `config/idac.yaml`.
+
+| Option                        | Info                                                                                                        |
+| ----------------------------- | ----------------------------------------------------------------------------------------------------------- |
+| `ssl`                         | Enables/Disables the use of the `ssl_cert` and `ssl_key` (currently unsuported)                             |
+| `matching_host`               | IPv4 address of your PC for the Online Battle (currently unsupported)                                       |
+| `port_matching`               | Port number for the Online Battle Matching                                                                  |
+| `port_echo1/2`                | Port numbers for Echos                                                                                      |
+| `port_matching_p2p`           | Port number for Online Battle (currently unsupported)                                                       |
+| `stamp.enable`               | Enables/Disabled the play stamp events                                                                      |
+| `stamp.enabled_stamps`        | Define up to 3 play stamp events (without `.json` extension, which are placed in `titles/idac/data/stamps`) |
+| `timetrial.enable`           | Enables/Disables the time trial event                                                                       |
+| `timetrial.enabled_timetrial` | Define one! trial event (without `.json` extension, which are placed in `titles/idac/data/timetrial`)       |
+
+
+### Database upgrade
+
+Always make sure your database (tables) are up-to-date:
+
+```shell
+python dbutils.py --game SDGT upgrade
+```
+
+### Notes
+- Online Battle is not supported
+- Online Battle Matching is not supported
+
+### Item categories
+
+| Category ID | Category Name            |
+| ----------- | ------------------------ |
+| 1           | D Coin                   |
+| 3           | Car Dressup Token        |
+| 5           | Avatar Dressup Token     |
+| 6           | Tachometer               |
+| 7           | Aura                     |
+| 8           | Aura Color               |
+| 9           | Avatar Face              |
+| 10          | Avatar Eye               |
+| 11          | Avatar Mouth             |
+| 12          | Avatar Hair              |
+| 13          | Avatar Glasses           |
+| 14          | Avatar Face accessories  |
+| 15          | Avatar Body              |
+| 18          | Avatar Background        |
+| 21          | Chat Stamp               |
+| 22          | Keychain                 |
+| 24          | Title                    |
+| 25          | FullTune Ticket          |
+| 26          | Paper Cup                |
+| 27          | BGM                      |
+| 28          | Drifting Text            |
+| 31          | Start Menu BG            |
+| 32          | Car Color/Paint          |
+| 33          | Aura Level               |
+| 34          | FullTune Ticket Fragment |
+| 35          | Underneon Lights         |
+
+### TimeRelease Chapter:
+
+1. Story: 1, 2, 3, 4, 5, 6, 7, 8, 9, 19 (Chapter 10), (29 Chapter 11?)
+2. MF Ghost: 10, 11, 12, 13, 14, 15
+3. Bunta: 15, 16, 17, 18, 19, 20, (21, 21, 22?)
+4. Special Event: 23, 24, 25, 26, 27, 28 (Touhou Project)
+
+### TimeRelease Courses:
+
+
+| Course ID | Course Name               | Direction                |
+| --------- | ------------------------- | ------------------------ |
+| 0         | Akina Lake(秋名湖)        | CounterClockwise(左周り) |
+| 2         | Akina Lake(秋名湖)        | Clockwise(右周り)        |
+| 52        | Hakone(箱根)              | Downhill(下り)           |
+| 54        | Hakone(箱根)              | Hillclimb(上り)          |
+| 36        | Usui(碓氷)                | CounterClockwise(左周り) |
+| 38        | Usui(碓氷)                | Clockwise(右周り)        |
+| 4         | Myogi(妙義)               | Downhill(下り)           |
+| 6         | Myogi(妙義)               | Hillclimb(上り)          |
+| 8         | Akagi(赤城)               | Downhill(下り)           |
+| 10        | Akagi(赤城)               | Hillclimb(上り)          |
+| 12        | Akina(秋名)               | Downhill(下り)           |
+| 14        | Akina(秋名)               | Hillclimb(上り)          |
+| 16        | Irohazaka(いろは坂)       | Downhill(下り)           |
+| 18        | Irohazaka(いろは坂)       | Reverse(逆走)            |
+| 56        | Momiji Line(もみじライン) | Downhill(下り)           |
+| 58        | Momiji Line(もみじライン) | Hillclimb(上り)          |
+| 20        | Tsukuba(筑波)             | Outbound(往路)           |
+| 22        | Tsukuba(筑波)             | Inbound(復路)            |
+| 24        | Happogahara(八方ヶ原)     | Outbound(往路)           |
+| 26        | Happogahara(八方ヶ原)     | Inbound(復路)            |
+| 40        | Sadamine(定峰)            | Downhill(下り)           |
+| 42        | Sadamine(定峰)            | Hillclimb(上り)          |
+| 44        | Tsuchisaka(土坂)          | Outbound(往路)           |
+| 46        | Tsuchisaka(土坂)          | Inbound(復路)            |
+| 48        | Akina Snow(秋名雪)        | Downhill(下り)           |
+| 50        | Akina Snow(秋名雪)        | Hillclimb(上り)          |
+| 68        | Odawara(小田原)           | Forward(順走)            |
+| 70        | Odawara(小田原)           | Reverse(逆走)            |
+
+### Credits
+- Bottersnike: For the HUGE Reverse Engineering help
+- Kinako: For helping with the timeRelease unlocking of courses and special mode
+
+A huge thanks to all people who helped shaping this project to what it is now and don't want to be mentioned here.
+

example_config/idac.yaml

@@ -0,0 +1,22 @@
+server:
+  enable: True
+  loglevel: "info"
+  ssl: False
+  ssl_key: "cert/idac.key"
+  ssl_cert: "cert/idac.crt"
+  matching_host: "127.0.0.1"
+  port_matching: 20000
+  port_echo1: 20001
+  port_echo2: 20002
+  port_matching_p2p: 20003
+
+stamp:
+  enable: True
+  enabled_stamps: # max 3 play stamps
+    - "touhou_remilia_scarlet"
+    - "touhou_flandre_scarlet"
+    - "touhou_sakuya_izayoi"
+
+timetrial:
+  enable: True
+  enabled_timetrial: "touhou_remilia_scarlet"

readme.md

@@ -33,6 +33,9 @@ Games listed below have been tested and confirmed working. Only game versions ol
 + Sword Art Online Arcade (partial support)
     + Final
 
++ Initial D THE ARCADE
+    + Season 2
+
 ## Requirements
 - python 3 (tested working with 3.9 and 3.10, other versions YMMV)
 - pip

titles/idac/__init__.py

@@ -0,0 +1,12 @@
+from titles.idac.index import IDACServlet
+from titles.idac.const import IDACConstants
+from titles.idac.database import IDACData
+from titles.idac.read import IDACReader
+from titles.idac.frontend import IDACFrontend
+
+index = IDACServlet
+database = IDACData
+reader = IDACReader
+frontend = IDACFrontend
+game_codes = [IDACConstants.GAME_CODE]
+current_schema_version = 1

titles/idac/base.py

@@ -0,0 +1,16 @@
+import logging
+
+from core.config import CoreConfig
+from titles.idac.config import IDACConfig
+from titles.idac.const import IDACConstants
+from titles.idac.database import IDACData
+
+
+class IDACBase:
+    def __init__(self, core_cfg: CoreConfig, game_cfg: IDACConfig) -> None:
+        self.core_cfg = core_cfg
+        self.game_config = game_cfg
+        self.game = IDACConstants.GAME_CODE
+        self.version = IDACConstants.VER_IDAC_SEASON_1
+        self.data = IDACData(core_cfg)
+        self.logger = logging.getLogger("idac")

titles/idac/config.py

@@ -0,0 +1,121 @@
+from core.config import CoreConfig
+
+
+class IDACServerConfig:
+    def __init__(self, parent: "IDACConfig") -> None:
+        self.__config = parent
+
+    @property
+    def enable(self) -> bool:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "server", "enable", default=True
+        )
+
+    @property
+    def loglevel(self) -> int:
+        return CoreConfig.str_to_loglevel(
+            CoreConfig.get_config_field(
+                self.__config, "idac", "server", "loglevel", default="info"
+            )
+        )
+
+    @property
+    def ssl(self) -> bool:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "server", "ssl", default=False
+        )
+
+    @property
+    def ssl_cert(self) -> str:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "server", "ssl_cert", default="cert/title.crt"
+        )
+
+    @property
+    def ssl_key(self) -> str:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "server", "ssl_key", default="cert/title.key"
+        )
+
+    @property
+    def matching_host(self) -> str:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "server", "matching_host", default="127.0.0.1"
+        )
+
+    @property
+    def matching(self) -> int:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "server", "port_matching", default=20000
+        )
+
+    @property
+    def echo1(self) -> int:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "server", "port_echo1", default=20001
+        )
+
+    @property
+    def echo2(self) -> int:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "server", "port_echo2", default=20002
+        )
+
+    @property
+    def matching_p2p(self) -> int:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "server", "port_matching_p2p", default=20003
+        )
+
+
+class IDACStampConfig:
+    def __init__(self, parent: "IDACConfig") -> None:
+        self.__config = parent
+
+    @property
+    def enable(self) -> bool:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "stamp", "enable", default=True
+        )
+
+    @property
+    def enabled_stamps(self) -> list:
+        return CoreConfig.get_config_field(
+            self.__config,
+            "idac",
+            "stamp",
+            "enabled_stamps",
+            default=[
+                "touhou_remilia_scarlet",
+                "touhou_flandre_scarlet",
+                "touhou_sakuya_izayoi",
+            ],
+        )
+
+
+class IDACTimetrialConfig:
+    def __init__(self, parent: "IDACConfig") -> None:
+        self.__config = parent
+
+    @property
+    def enable(self) -> bool:
+        return CoreConfig.get_config_field(
+            self.__config, "idac", "timetrial", "enable", default=True
+        )
+
+    @property
+    def enabled_timetrial(self) -> str:
+        return CoreConfig.get_config_field(
+            self.__config,
+            "idac",
+            "timetrial",
+            "enabled_timetrial",
+            default="touhou_remilia_scarlet",
+        )
+
+
+class IDACConfig(dict):
+    def __init__(self) -> None:
+        self.server = IDACServerConfig(self)
+        self.stamp = IDACStampConfig(self)
+        self.timetrial = IDACTimetrialConfig(self)

titles/idac/const.py

@@ -0,0 +1,16 @@
+class IDACConstants():
+    GAME_CODE = "SDGT"
+
+    CONFIG_NAME = "idac.yaml"
+
+    VER_IDAC_SEASON_1 = 0
+    VER_IDAC_SEASON_2 = 1
+
+    VERSION_STRING = (
+        "Initial D THE ARCADE Season 1",
+        "Initial D THE ARCADE Season 2",
+    )
+
+    @classmethod
+    def game_ver_to_string(cls, ver: int):
+        return cls.VERSION_STRING[ver]

titles/idac/data/create_delivery_images.py

@@ -0,0 +1,38 @@
+import os
+import hashlib
+
+
+def prepare_images(image_folder="titles/idac/data/images"):
+    print(f"Preparing image delivery files in {image_folder}...")
+
+    for file in os.listdir(image_folder):
+        if file.endswith(".png") or file.endswith(".jpg"):
+            # dpg_name = "adv-" + file[:-4].upper()
+            dpg_name = file[:-4]
+            if file.endswith(".png"):
+                dpg_name += ".dpg"
+            else:
+                dpg_name += ".djg"
+
+            if os.path.exists(os.path.join(image_folder, dpg_name)):
+                continue
+            else:
+                with open(
+                    os.path.join(image_folder, file), "rb"
+                ) as original_image_file:
+                    original_image = original_image_file.read()
+                    image_hash = hashlib.md5(original_image).hexdigest()
+                    print(
+                        f"DPG for {file} not found, creating with hash {image_hash.upper()} ..."
+                    )
+                    md5_buf = bytes.fromhex(image_hash)
+                    dpg_buf = md5_buf + original_image
+                    dpg_name = "adv-" + image_hash.upper() + dpg_name[:-4]
+                    with open(os.path.join(image_folder, dpg_name), "wb") as dpg_file:
+                        dpg_file.write(dpg_buf)
+
+                    print(f"Created {dpg_name}.")
+
+
+# Call the function to execute it
+prepare_images()

titles/idac/database.py

@@ -0,0 +1,12 @@
+from core.data import Data
+from core.config import CoreConfig
+from titles.idac.schema.profile import IDACProfileData
+from titles.idac.schema.item import IDACItemData
+
+
+class IDACData(Data):
+    def __init__(self, cfg: CoreConfig) -> None:
+        super().__init__(cfg)
+
+        self.profile = IDACProfileData(cfg, self.session)
+        self.item = IDACItemData(cfg, self.session)

titles/idac/echo.py

@@ -0,0 +1,64 @@
+import logging
+from random import randbytes
+import socket
+
+from twisted.internet.protocol import DatagramProtocol
+from socketserver import BaseRequestHandler, TCPServer
+from typing import Tuple
+
+from core.config import CoreConfig
+from titles.idac.config import IDACConfig
+from titles.idac.database import IDACData
+
+
+class IDACEchoUDP(DatagramProtocol):
+    def __init__(self, cfg: CoreConfig, game_cfg: IDACConfig, port: int) -> None:
+        super().__init__()
+        self.port = port
+        self.core_config = cfg
+        self.game_config = game_cfg
+        self.logger = logging.getLogger("idac")
+
+    def datagramReceived(self, data, addr):
+        self.logger.info(
+            f"UDP Ping from from {addr[0]}:{addr[1]} -> {self.port} - {data.hex()}"
+        )
+        self.transport.write(data, addr)
+
+
+class IDACEchoTCP(BaseRequestHandler):
+    def __init__(
+        self, request, client_address, server, cfg: CoreConfig, game_cfg: IDACConfig
+    ) -> None:
+        self.core_config = cfg
+        self.game_config = game_cfg
+        self.logger = logging.getLogger("idac")
+        self.data = IDACData(cfg)
+        super().__init__(request, client_address, server)
+
+    def handle(self):
+        data = self.request.recv(1024).strip()
+        self.logger.debug(
+            f"TCP Ping from {self.client_address[0]}:{self.client_address[1]} -> {self.server.server_address[1]}: {data.hex()}"
+        )
+        self.request.sendall(data)
+        self.request.shutdown(socket.SHUT_WR)
+
+
+class IDACEchoTCPFactory(TCPServer):
+    def __init__(
+        self,
+        server_address: Tuple[str, int],
+        RequestHandlerClass,
+        cfg: CoreConfig,
+        game_cfg: IDACConfig,
+        bind_and_activate: bool = ...,
+    ) -> None:
+        super().__init__(server_address, RequestHandlerClass, bind_and_activate)
+        self.core_config = cfg
+        self.game_config = game_cfg
+
+    def finish_request(self, request, client_address):
+        self.RequestHandlerClass(
+            request, client_address, self, self.core_config, self.game_config
+        )

titles/idac/frontend.py

@@ -0,0 +1,142 @@
+import json
+import yaml
+import jinja2
+from os import path
+from twisted.web.util import redirectTo
+from twisted.web.http import Request
+from twisted.web.server import Session
+
+from core.frontend import FE_Base, IUserSession
+from core.config import CoreConfig
+from titles.idac.database import IDACData
+from titles.idac.schema.profile import *
+from titles.idac.schema.item import *
+from titles.idac.config import IDACConfig
+from titles.idac.const import IDACConstants
+
+
+class IDACFrontend(FE_Base):
+    def __init__(
+        self, cfg: CoreConfig, environment: jinja2.Environment, cfg_dir: str
+    ) -> None:
+        super().__init__(cfg, environment)
+        self.data = IDACData(cfg)
+        self.game_cfg = IDACConfig()
+        if path.exists(f"{cfg_dir}/{IDACConstants.CONFIG_NAME}"):
+            self.game_cfg.update(
+                yaml.safe_load(open(f"{cfg_dir}/{IDACConstants.CONFIG_NAME}"))
+            )
+        self.nav_name = "頭文字D THE ARCADE"
+        # TODO: Add version list
+        self.version = IDACConstants.VER_IDAC_SEASON_2
+
+        self.ticket_names = {
+            3: "car_dressup_points",
+            5: "avatar_points",
+            25: "full_tune_tickets",
+            34: "full_tune_fragments",
+        }
+
+    def generate_all_tables_json(self, user_id: int):
+        json_export = {}
+
+        idac_tables = {
+            profile,
+            config,
+            avatar,
+            rank,
+            stock,
+            theory,
+            car,
+            ticket,
+            story,
+            episode,
+            difficulty,
+            course,
+            trial,
+            challenge,
+            theory_course,
+            theory_partner,
+            theory_running,
+            vs_info,
+            stamp,
+            timetrial_event
+        }
+
+        for table in idac_tables:
+            sql = select(table).where(
+                table.c.user == user_id,
+            )
+
+            # check if the table has a version column
+            if "version" in table.c:
+                sql = sql.where(table.c.version == self.version)
+
+            # lol use the profile connection for items, dirty hack
+            result = self.data.profile.execute(sql)
+            data_list = result.fetchall()
+
+            # add the list to the json export with the correct table name
+            json_export[table.name] = []
+            for data in data_list:
+                tmp = data._asdict()
+                tmp.pop("id")
+                tmp.pop("user")
+                json_export[table.name].append(tmp)
+
+        return json.dumps(json_export, indent=4, default=str, ensure_ascii=False)
+
+    def render_GET(self, request: Request) -> bytes:
+        uri: str = request.uri.decode()
+
+        template = self.environment.get_template(
+            "titles/idac/frontend/idac_index.jinja"
+        )
+        sesh: Session = request.getSession()
+        usr_sesh = IUserSession(sesh)
+        user_id = usr_sesh.userId
+        # user_id = usr_sesh.user_id
+
+        # profile export
+        if uri.startswith("/game/idac/export"):
+            if user_id == 0:
+                return redirectTo(b"/game/idac", request)
+
+            # set the file name, content type and size to download the json
+            content = self.generate_all_tables_json(user_id).encode("utf-8")
+            request.responseHeaders.addRawHeader(
+                b"content-type", b"application/octet-stream"
+            )
+            request.responseHeaders.addRawHeader(
+                b"content-disposition", b"attachment; filename=idac_profile.json"
+            )
+            request.responseHeaders.addRawHeader(
+                b"content-length", str(len(content)).encode("utf-8")
+            )
+
+            self.logger.info(f"User {user_id} exported their IDAC data")
+            return content
+
+        profile_data, tickets, rank = None, None, None
+        if user_id > 0:
+            profile_data = self.data.profile.get_profile(user_id, self.version)
+            ticket_data = self.data.item.get_tickets(user_id)
+            rank = self.data.profile.get_profile_rank(user_id, self.version)
+
+            tickets = {
+                self.ticket_names[ticket["ticket_id"]]: ticket["ticket_cnt"]
+                for ticket in ticket_data
+            }
+
+        return template.render(
+            title=f"{self.core_config.server.name} | {self.nav_name}",
+            game_list=self.environment.globals["game_list"],
+            profile=profile_data,
+            tickets=tickets,
+            rank=rank,
+            sesh=vars(usr_sesh),
+            active_page="idac",
+        ).encode("utf-16")
+
+    def render_POST(self, request: Request) -> bytes:
+        pass

titles/idac/frontend/idac_index.jinja

@@ -0,0 +1,134 @@
+{% extends "core/frontend/index.jinja" %}
+{% block content %}
+<h1 class="mb-3">頭文字D THE ARCADE</h1>
+
+{% if sesh is defined and sesh["userId"] > 0 %}
+<div class="card mb-3">
+    <div class="card-body">
+        <div class="card-title">
+            <div class="d-flex justify-content-between flex-wrap flex-md-nowrap align-items-center">
+                <h3>{{ sesh["username"] }}'s Profile</h3>
+                <div class="btn-toolbar mb-2 mb-md-0">
+                    <div class="btn-group me-2">
+                        <!--<button type="button" class="btn btn-sm btn-outline-secondary">Share</button>-->
+                        <button type="button" data-bs-toggle="modal" data-bs-target="#export"
+                            class="btn btn-sm btn-outline-primary">Export</button>
+                    </div>
+                </div>
+            </div>
+        </div>
+        <!--<h4 class="card-subtitle mb-2 text-body-secondary">Card subtitle</h4>-->
+        {% if profile is defined and profile is not none %}
+        <div class="row d-flex justify-content-center h-100">
+            <div class="col col-lg-3 col-12">
+                <div class="card mb-3">
+                    <div class="card-body p-4">
+                        <h5>Information</h5>
+                        <hr class="mt-0 mb-4">
+                        <h6>Username</h6>
+                        <p class="text-muted">{{ profile.username }}</p>
+                        <h6>Cash</h6>
+                        <p class="text-muted">{{ profile.cash }} D</p>
+                        <h6>Grade</h6>
+                        <h4>
+                            {% set grade = rank.grade %}
+                            {% if grade >= 1 and grade <= 72 %}
+                                {% set grade_number = (grade - 1) // 9 %}
+                                {% set grade_letters = ['E', 'D', 'C', 'B', 'A', 'S', 'SS', 'X'] %}
+                                {{ grade_letters[grade_number] }}{{ 9 - ((grade-1) % 9) }}
+                            {% else %}
+                                Unknown
+                            {% endif %}
+                        </h4>
+                    </div>
+                </div>
+            </div>
+            <div class="col col-lg-9 col-12">
+                <div class="card mb-3">
+
+                    <div class="card-body p-4">
+                        <h5>Statistics</h5>
+                        <hr class="mt-0 mb-4">
+                        <div class="row pt-1">
+                            <div class="col-lg-4 col-md-6 mb-3">
+                                <h6>Total Plays</h6>
+                                <p class="text-muted">{{ profile.total_play }}</p>
+                            </div>
+                            <div class="col-lg-4 col-md-6 mb-3">
+                                <h6>Last Played</h6>
+                                <p class="text-muted">{{ profile.last_play_date }}</p>
+                            </div>
+                            <div class="col-lg-4 col-md-6 mb-3">
+                                <h6>Mileage</h6>
+                                <p class="text-muted">{{ profile.mileage / 1000}} km</p>
+                            </div>
+                        </div>
+                        {% if tickets is defined and tickets|length > 0 %}
+                        <h5>Tokens/Tickets</h5>
+                        <hr class="mt-0 mb-4">
+                        <div class="row pt-1">
+                            <div class="col-lg-3 col-md-6 mb-3">
+                                <h6>Avatar Tokens</h6>
+                                <p class="text-muted">{{ tickets.avatar_points }}/30</p>
+                            </div>
+                            <div class="col-lg-3 col-md-6 mb-3">
+                                <h6>Car Dressup Tokens</h6>
+                                <p class="text-muted">{{ tickets.car_dressup_points }}/30</p>
+                            </div>
+                            <div class="col-lg-3 col-md-6 mb-3">
+                                <h6>FullTune Tickets</h6>
+                                <p class="text-muted">{{ tickets.full_tune_tickets }}/99</p>
+                            </div>
+                            <div class="col-lg-3 col-md-6 mb-3">
+                                <h6>FullTune Fragments</h6>
+                                <p class="text-muted">{{ tickets.full_tune_fragments }}/10</p>
+                            </div>
+                        </div>
+                        {% endif %}
+                    </div>
+                </div>
+            </div>
+        </div>
+        {% else %}
+        <div class="alert alert-warning" role="alert">
+            You need to play 頭文字D THE ARCADE first to view your profile.
+        </div>
+        {% endif %}
+        <!--<a href="#" data-bs-toggle="modal" data-bs-target="#card-add" class="card-link">Add Card</a>-->
+    </div>
+</div>
+{% else %}
+<div class="alert alert-info" role="alert">
+    You need to be logged in to view this page. <a href="/gate">Login</a></a>
+</div>
+{% endif %}
+
+<div class="modal fade" id="export" tabindex="-1" aria-labelledby="export-label" aria-hidden="true">
+    <div class="modal-dialog">
+        <div class="modal-content">
+            <div class="modal-header">
+                <h1 class="modal-title fs-5" id="exort-label">Export Profile</h1>
+                <button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
+            </div>
+            <div class="modal-body">
+                Download your profile as a <strong>.json</strong> file in order to import it into your local ARTEMiS
+                database.
+                <div class="alert alert-warning mt-3" role="alert">
+                    {% if profile is defined and profile is not none %}
+                    Are you sure you want to export your profile with the username {{ profile.username }}?
+                    {% endif %}
+                </div>
+            </div>
+            <div class="modal-footer">
+                <button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
+                <button type="button" class="btn btn-primary" id="exportBtn">Download Profile</a>
+            </div>
+        </div>
+    </div>
+</div>
+
+<script type="text/javascript">
+    {% include "titles/idac/frontend/js/idac_scripts.js" %}
+</script>
+
+{% endblock content %}

titles/idac/frontend/js/idac_scripts.js

@@ -0,0 +1,10 @@
+$(document).ready(function () {
+    $('#exportBtn').click(function () {
+        window.location = "/game/idac/export";
+
+        // appendAlert('Successfully exported the profile', 'success');
+
+        // Close the modal on success
+        $('#export').modal('hide');
+    });
+});

titles/idac/index.py

@@ -0,0 +1,165 @@
+import json
+import traceback
+import inflection
+import yaml
+import logging
+import coloredlogs
+
+from os import path
+from typing import Dict, List, Tuple
+from logging.handlers import TimedRotatingFileHandler
+from twisted.web import server
+from twisted.web.http import Request
+from twisted.internet import reactor, endpoints
+
+from core.config import CoreConfig
+from core.utils import Utils
+from titles.idac.base import IDACBase
+from titles.idac.season2 import IDACSeason2
+from titles.idac.config import IDACConfig
+from titles.idac.const import IDACConstants
+from titles.idac.echo import IDACEchoUDP
+from titles.idac.matching import IDACMatching
+
+
+class IDACServlet:
+    def __init__(self, core_cfg: CoreConfig, cfg_dir: str) -> None:
+        self.core_cfg = core_cfg
+        self.game_cfg = IDACConfig()
+        if path.exists(f"{cfg_dir}/{IDACConstants.CONFIG_NAME}"):
+            self.game_cfg.update(
+                yaml.safe_load(open(f"{cfg_dir}/{IDACConstants.CONFIG_NAME}"))
+            )
+
+        self.versions = [
+            IDACBase(core_cfg, self.game_cfg),
+            IDACSeason2(core_cfg, self.game_cfg)
+        ]
+
+        self.logger = logging.getLogger("idac")
+        log_fmt_str = "[%(asctime)s] IDAC | %(levelname)s | %(message)s"
+        log_fmt = logging.Formatter(log_fmt_str)
+        fileHandler = TimedRotatingFileHandler(
+            "{0}/{1}.log".format(self.core_cfg.server.log_dir, "idac"),
+            encoding="utf8",
+            when="d",
+            backupCount=10,
+        )
+
+        fileHandler.setFormatter(log_fmt)
+
+        consoleHandler = logging.StreamHandler()
+        consoleHandler.setFormatter(log_fmt)
+
+        self.logger.addHandler(fileHandler)
+        self.logger.addHandler(consoleHandler)
+
+        self.logger.setLevel(self.game_cfg.server.loglevel)
+        coloredlogs.install(
+            level=self.game_cfg.server.loglevel, logger=self.logger, fmt=log_fmt_str
+        )
+
+    @classmethod
+    def is_game_enabled(cls, game_code: str, core_cfg: CoreConfig, cfg_dir: str) -> bool:
+        game_cfg = IDACConfig()
+
+        if path.exists(f"{cfg_dir}/{IDACConstants.CONFIG_NAME}"):
+            game_cfg.update(
+                yaml.safe_load(open(f"{cfg_dir}/{IDACConstants.CONFIG_NAME}"))
+            )
+
+        if not game_cfg.server.enable:
+            return False
+        
+        return True
+
+    def get_endpoint_matchers(self) -> Tuple[List[Tuple[str, str, Dict]], List[Tuple[str, str, Dict]]]:
+        return (
+            [], 
+            [("render_POST", "/SDGT/{version}/initiald/{category}/{endpoint}", {})]
+        )
+
+    def get_allnet_info(
+        self, game_code: str, game_ver: int, keychip: str
+    ) -> Tuple[bool, str, str]:
+        title_port_int = Utils.get_title_port(self.core_cfg)
+        t_port = f":{title_port_int}" if title_port_int and not self.core_cfg.server.is_using_proxy else ""
+
+        return (
+            f"",
+            # requires http or else it defaults to https
+            f"http://{self.core_cfg.title.hostname}{t_port}/{game_code}/{game_ver}/",
+        )
+
+    def render_POST(self, request: Request, game_code: int, matchers: Dict) -> bytes:
+        req_raw = request.content.getvalue()
+        internal_ver = 0
+        version = int(matchers['version'])
+        category = matchers['category']
+        endpoint = matchers['endpoint']
+        client_ip = Utils.get_ip_addr(request)
+
+        if version >= 100 and version < 140:  # IDAC Season 1
+            internal_ver = IDACConstants.VER_IDAC_SEASON_1
+        elif version >= 140 and version < 171:  # IDAC Season 2
+            internal_ver = IDACConstants.VER_IDAC_SEASON_2
+
+        header_application = self.decode_header(request.getAllHeaders())
+
+        req_data = json.loads(req_raw)
+
+        self.logger.info(f"v{version} {endpoint} request from {client_ip}")
+        self.logger.debug(f"Headers: {header_application}")
+        self.logger.debug(req_data)
+
+        # func_to_find = "handle_" + inflection.underscore(endpoint) + "_request"
+        func_to_find = "handle_"
+        func_to_find += f"{category.lower()}_" if not category == "" else ""
+        func_to_find += f"{endpoint.lower()}_request"
+
+        if not hasattr(self.versions[internal_ver], func_to_find):
+            self.logger.warning(f"Unhandled v{version} request {endpoint}")
+            return '{"status_code": "0"}'.encode("utf-8")
+
+        resp = None
+        try:
+            handler = getattr(self.versions[internal_ver], func_to_find)
+            resp = handler(req_data, header_application)
+
+        except Exception as e:
+            traceback.print_exc()
+            self.logger.error(f"Error handling v{version} method {endpoint} - {e}")
+            return '{"status_code": "0"}'.encode("utf-8")
+
+        if resp is None:
+            resp = {"status_code": "0"}
+
+        self.logger.debug(f"Response {resp}")
+        return json.dumps(resp, ensure_ascii=False).encode("utf-8")
+
+
+    def decode_header(self, data: Dict) -> Dict:
+        app: str = data[b"application"].decode()
+        ret = {}
+
+        for x in app.split(", "):
+            y = x.split("=")
+            ret[y[0]] = y[1].replace('"', "")
+
+        return ret
+
+    def setup(self):
+        if self.game_cfg.server.enable:
+            endpoints.serverFromString(
+                reactor,
+                f"tcp:{self.game_cfg.server.matching}:interface={self.core_cfg.server.listen_address}",
+            ).listen(server.Site(IDACMatching(self.core_cfg, self.game_cfg)))
+
+            reactor.listenUDP(
+                self.game_cfg.server.echo1,
+                IDACEchoUDP(self.core_cfg, self.game_cfg, self.game_cfg.server.echo1),
+            )
+            reactor.listenUDP(
+                self.game_cfg.server.echo2,
+                IDACEchoUDP(self.core_cfg, self.game_cfg, self.game_cfg.server.echo2),
+            )

titles/idac/matching.py

@@ -0,0 +1,72 @@
+import json
+import logging
+
+from typing import Dict
+from twisted.web import resource
+
+from core import CoreConfig
+from titles.idac.season2 import IDACBase
+from titles.idac.config import IDACConfig
+
+
+class IDACMatching(resource.Resource):
+    isLeaf = True
+
+    def __init__(self, cfg: CoreConfig, game_cfg: IDACConfig) -> None:
+        self.core_config = cfg
+        self.game_config = game_cfg
+        self.base = IDACBase(cfg, game_cfg)
+        self.logger = logging.getLogger("idac")
+
+        self.queue = 0
+
+    def get_matching_state(self):
+        if self.queue >= 1:
+            self.queue -= 1
+            return 0
+        else:
+            return 1
+
+    def render_POST(self, req) -> bytes:
+        url = req.uri.decode()
+        req_data = json.loads(req.content.getvalue().decode())
+        header_application = self.decode_header(req.getAllHeaders())
+        user_id = int(header_application["session"])
+
+        # self.getMatchingStatus(user_id)
+
+        self.logger.info(
+            f"IDAC Matching request from {req.getClientIP()}: {url} - {req_data}"
+        )
+
+        resp = {"status_code": "0"}
+        if url == "/regist":
+            self.queue = self.queue + 1
+        elif url == "/status":
+            if req_data.get("cancel_flag"):
+                self.queue = self.queue - 1
+                self.logger.info(
+                    f"IDAC Matching endpoint {req.getClientIP()} had quited"
+                )
+
+            resp = {
+                "status_code": "0",
+                # Only IPv4 is supported
+                "host": self.game_config.server.matching_host,
+                "port": self.game_config.server.matching_p2p,
+                "room_name": "INDTA",
+                "state": 1,
+            }
+
+        self.logger.debug(f"Response {resp}")
+        return json.dumps(resp, ensure_ascii=False).encode("utf-8")
+
+    def decode_header(self, data: Dict) -> Dict:
+        app: str = data[b"application"].decode()
+        ret = {}
+
+        for x in app.split(", "):
+            y = x.split("=")
+            ret[y[0]] = y[1].replace('"', "")
+
+        return ret

titles/idac/read.py

@@ -0,0 +1,161 @@
+import json
+import logging
+import os
+from typing import Any, Dict, List, Optional
+
+from read import BaseReader
+from core.data import Data
+from core.config import CoreConfig
+from titles.idac.const import IDACConstants
+from titles.idac.database import IDACData
+from titles.idac.schema.profile import *
+from titles.idac.schema.item import *
+
+
+class IDACReader(BaseReader):
+    def __init__(
+        self,
+        config: CoreConfig,
+        version: int,
+        bin_dir: Optional[str],
+        opt_dir: Optional[str],
+        extra: Optional[str],
+    ) -> None:
+        super().__init__(config, version, bin_dir, opt_dir, extra)
+        self.card_data = Data(config).card
+        self.data = IDACData(config)
+
+        try:
+            self.logger.info(
+                f"Start importer for {IDACConstants.game_ver_to_string(version)}"
+            )
+        except IndexError:
+            self.logger.error(f"Invalid Initial D THE ARCADE version {version}")
+            exit(1)
+
+    def read(self) -> None:
+        if self.bin_dir is None and self.opt_dir is None:
+            self.logger.error(
+                (
+                    "To import your profile specify the '--optfolder'",
+                    " path to your idac_profile.json file, exiting",
+                )
+            )
+            exit(1)
+
+        if self.opt_dir is not None:
+            if not os.path.exists(self.opt_dir):
+                self.logger.error(
+                    f"Path to idac_profile.json does not exist: {self.opt_dir}"
+                )
+                exit(1)
+
+            if os.path.isdir(self.opt_dir):
+                self.opt_dir = os.path.join(self.opt_dir, "idac_profile.json")
+
+            if not os.path.isfile(self.opt_dir) or self.opt_dir[-5:] != ".json":
+                self.logger.error(
+                    f"Path to idac_profile.json does not exist: {self.opt_dir}"
+                )
+                exit(1)
+
+            self.read_idac_profile(self.opt_dir)
+
+    def read_idac_profile(self, file_path: str) -> None:
+        self.logger.info(f"Reading profile from {file_path}...")
+
+        # read it as binary to avoid encoding issues
+        profile_data: Dict[str, Any] = {}
+        with open(file_path, "rb") as f:
+            profile_data = json.loads(f.read().decode("utf-8"))
+
+        if not profile_data:
+            self.logger.error("Profile could not be parsed, exiting")
+            exit(1)
+
+        access_code = None
+        while access_code is None:
+            access_code = input("Enter your 20 digits access code: ")
+            if len(access_code) != 20 or not access_code.isdigit():
+                access_code = None
+                self.logger.warning("Invalid access code, please try again.")
+
+        # check if access code already exists, if not create a new profile
+        user_id = self.card_data.get_user_id_from_card(access_code)
+        if user_id is None:
+            choice = input("Access code does not exist, do you want to create a new profile? (Y/n): ")
+            if choice.lower() == "n":
+                self.logger.info("Exiting...")
+                exit(0)
+
+            user_id = self.data.user.create_user()
+
+            if user_id is None:
+                self.logger.error("Failed to register user!")
+                user_id = -1
+
+            else:
+                card_id = self.data.card.create_card(user_id, access_code)
+
+                if card_id is None:
+                    self.logger.error("Failed to register card!")
+                    user_id = -1
+
+        if user_id == -1:
+            self.logger.error("Failed to create profile, exiting")
+            exit(1)
+
+        # table mapping to insert the data properly
+        tables = {
+            "idac_profile": profile,
+            "idac_profile_config": config,
+            "idac_profile_avatar": avatar,
+            "idac_profile_rank": rank,
+            "idac_profile_stock": stock,
+            "idac_profile_theory": theory,
+            "idac_user_car": car,
+            "idac_user_ticket": ticket,
+            "idac_user_story": story,
+            "idac_user_story_episode": episode,
+            "idac_user_story_episode_difficulty": difficulty,
+            "idac_user_course": course,
+            "idac_user_time_trial": trial,
+            "idac_user_challenge": challenge,
+            "idac_user_theory_course": theory_course,
+            "idac_user_theory_partner": theory_partner,
+            "idac_user_theory_running": theory_running,
+            "idac_user_vs_info": vs_info,
+            "idac_user_stamp": stamp,
+            "idac_user_timetrial_event": timetrial_event,
+        }
+
+        for name, data_list in profile_data.items():
+            # get the SQLAlchemy table object from the name
+            table = tables.get(name)
+            if table is None:
+                self.logger.warning(f"Unknown table {name}, skipping")
+                continue
+
+            for data in data_list:
+                # add user to the data
+                data["user"] = user_id
+
+                # check if the table has a version column
+                if "version" in table.c:
+                    data["version"] = self.version
+
+                sql = insert(table).values(
+                    **data
+                )
+
+                # lol use the profile connection for items, dirty hack
+                conflict = sql.on_duplicate_key_update(**data)
+                result = self.data.profile.execute(conflict)
+
+                if result is None:
+                    self.logger.error(f"Failed to insert data into table {name}")
+                    exit(1)
+
+                self.logger.info(f"Inserted data into table {name}")
+
+        self.logger.info("Profile import complete!")

titles/idac/schema/item.py

@@ -0,0 +1,983 @@
+from typing import Dict, Optional, List
+from sqlalchemy import (
+    Table,
+    Column,
+    UniqueConstraint,
+    PrimaryKeyConstraint,
+    and_,
+    update,
+)
+from sqlalchemy.types import Integer, String, TIMESTAMP, Boolean, JSON
+from sqlalchemy.schema import ForeignKey
+from sqlalchemy.engine import Row
+from sqlalchemy.sql import func, select
+from sqlalchemy.dialects.mysql import insert
+
+from core.data.schema import BaseData, metadata
+
+car = Table(
+    "idac_user_car",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("version", Integer, nullable=False),
+    Column("car_id", Integer),
+    Column("style_car_id", Integer),
+    Column("color", Integer),
+    Column("bureau", Integer),
+    Column("kana", Integer),
+    Column("s_no", Integer),
+    Column("l_no", Integer),
+    Column("car_flag", Integer),
+    Column("tune_point", Integer),
+    Column("tune_level", Integer, server_default="1"),
+    Column("tune_parts", Integer),
+    Column("infinity_tune", Integer, server_default="0"),
+    Column("online_vs_win", Integer, server_default="0"),
+    Column(
+        "pickup_seq", Integer, server_default="1"
+    ),  # the order in which the car was picked up
+    Column(
+        "purchase_seq", Integer, server_default="1"
+    ),  # the order in which the car was purchased
+    Column("color_stock_list", String(32)),
+    Column("color_stock_new_list", String(32)),
+    Column("parts_stock_list", String(48)),
+    Column("parts_stock_new_list", String(48)),
+    Column("parts_set_equip_list", String(48)),
+    Column("parts_list", JSON),
+    Column("equip_parts_count", Integer, server_default="0"),
+    Column("total_car_parts_count", Integer, server_default="0"),
+    Column("use_count", Integer, server_default="0"),
+    Column("story_use_count", Integer, server_default="0"),
+    Column("timetrial_use_count", Integer, server_default="0"),
+    Column("vs_use_count", Integer, server_default="0"),
+    Column("net_vs_use_count", Integer, server_default="0"),
+    Column("theory_use_count", Integer, server_default="0"),
+    Column("car_mileage", Integer, server_default="0"),
+    UniqueConstraint("user", "version", "style_car_id", name="idac_user_car_uk"),
+    mysql_charset="utf8mb4",
+)
+
+ticket = Table(
+    "idac_user_ticket",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("ticket_id", Integer),
+    Column("ticket_cnt", Integer),
+    UniqueConstraint("user", "ticket_id", name="idac_user_ticket_uk"),
+    mysql_charset="utf8mb4",
+)
+
+story = Table(
+    "idac_user_story",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("story_type", Integer),
+    Column("chapter", Integer),
+    Column("loop_count", Integer, server_default="1"),
+    UniqueConstraint("user", "chapter", name="idac_user_story_uk"),
+    mysql_charset="utf8mb4",
+)
+
+episode = Table(
+    "idac_user_story_episode",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("chapter", Integer),
+    Column("episode", Integer),
+    Column("play_status", Integer),
+    UniqueConstraint("user", "chapter", "episode", name="idac_user_story_episode_uk"),
+    mysql_charset="utf8mb4",
+)
+
+difficulty = Table(
+    "idac_user_story_episode_difficulty",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("episode", Integer),
+    Column("difficulty", Integer),
+    Column("play_count", Integer),
+    Column("clear_count", Integer),
+    Column("play_status", Integer),
+    Column("play_score", Integer),
+    UniqueConstraint(
+        "user", "episode", "difficulty", name="idac_user_story_episode_difficulty_uk"
+    ),
+    mysql_charset="utf8mb4",
+)
+
+course = Table(
+    "idac_user_course",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("course_id", Integer),
+    Column("run_counts", Integer, server_default="1"),
+    Column("skill_level_exp", Integer, server_default="0"),
+    UniqueConstraint("user", "course_id", name="idac_user_course_uk"),
+    mysql_charset="utf8mb4",
+)
+
+trial = Table(
+    "idac_user_time_trial",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("version", Integer, nullable=False),
+    Column("style_car_id", Integer),
+    Column("course_id", Integer),
+    Column("eval_id", Integer, server_default="0"),
+    Column("goal_time", Integer),
+    Column("section_time_1", Integer),
+    Column("section_time_2", Integer),
+    Column("section_time_3", Integer),
+    Column("section_time_4", Integer),
+    Column("mission", Integer),
+    Column("play_dt", TIMESTAMP, server_default=func.now()),
+    UniqueConstraint(
+        "user", "version", "course_id", "style_car_id", name="idac_user_time_trial_uk"
+    ),
+    mysql_charset="utf8mb4",
+)
+
+challenge = Table(
+    "idac_user_challenge",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("vs_type", Integer),
+    Column("play_difficulty", Integer),
+    Column("cleared_difficulty", Integer),
+    Column("story_type", Integer),
+    Column("play_count", Integer, server_default="1"),
+    Column("weak_difficulty", Integer, server_default="0"),
+    Column("eval_id", Integer),
+    Column("advantage", Integer),
+    Column("sec1_advantage_avg", Integer),
+    Column("sec2_advantage_avg", Integer),
+    Column("sec3_advantage_avg", Integer),
+    Column("sec4_advantage_avg", Integer),
+    Column("nearby_advantage_rate", Integer),
+    Column("win_flag", Integer),
+    Column("result", Integer),
+    Column("record", Integer),
+    Column("course_id", Integer),
+    Column("last_play_course_id", Integer),
+    Column("style_car_id", Integer),
+    Column("course_day", Integer),
+    UniqueConstraint(
+        "user", "vs_type", "play_difficulty", name="idac_user_challenge_uk"
+    ),
+    mysql_charset="utf8mb4",
+)
+
+theory_course = Table(
+    "idac_user_theory_course",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("course_id", Integer),
+    Column("max_victory_grade", Integer, server_default="0"),
+    Column("run_count", Integer, server_default="1"),
+    Column("powerhouse_lv", Integer),
+    Column("powerhouse_exp", Integer),
+    Column("played_powerhouse_lv", Integer),
+    Column("update_dt", TIMESTAMP, server_default=func.now()),
+    UniqueConstraint("user", "course_id", name="idac_user_theory_course_uk"),
+    mysql_charset="utf8mb4",
+)
+
+theory_partner = Table(
+    "idac_user_theory_partner",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("partner_id", Integer),
+    Column("fellowship_lv", Integer),
+    Column("fellowship_exp", Integer),
+    UniqueConstraint("user", "partner_id", name="idac_user_theory_partner_uk"),
+    mysql_charset="utf8mb4",
+)
+
+theory_running = Table(
+    "idac_user_theory_running",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("course_id", Integer),
+    Column("attack", Integer),
+    Column("defense", Integer),
+    Column("safety", Integer),
+    Column("runaway", Integer),
+    Column("trick_flag", Integer),
+    UniqueConstraint("user", "course_id", name="idac_user_theory_running_uk"),
+    mysql_charset="utf8mb4",
+)
+
+vs_info = Table(
+    "idac_user_vs_info",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column("user", ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade")),
+    Column("group_key", String(25)),
+    Column("win_flg", Integer),
+    Column("style_car_id", Integer),
+    Column("course_id", Integer),
+    Column("course_day", Integer),
+    Column("players_num", Integer),
+    Column("winning", Integer),
+    Column("advantage_1", Integer),
+    Column("advantage_2", Integer),
+    Column("advantage_3", Integer),
+    Column("advantage_4", Integer),
+    Column("select_course_id", Integer),
+    Column("select_course_day", Integer),
+    Column("select_course_random", Integer),
+    Column("matching_success_sec", Integer),
+    Column("boost_flag", Integer),
+    Column("vs_history", Integer),
+    Column("break_count", Integer),
+    Column("break_penalty_flag", Integer),
+    UniqueConstraint("user", "group_key", name="idac_user_vs_info_uk"),
+    mysql_charset="utf8mb4",
+)
+
+stamp = Table(
+    "idac_user_stamp",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("m_stamp_event_id", Integer),
+    Column("select_flag", Integer),
+    Column("stamp_masu", Integer),
+    Column("daily_bonus", Integer),
+    Column("weekly_bonus", Integer),
+    Column("weekday_bonus", Integer),
+    Column("weekend_bonus", Integer),
+    Column("total_bonus", Integer),
+    Column("day_total_bonus", Integer),
+    Column("store_battle_bonus", Integer),
+    Column("story_bonus", Integer),
+    Column("online_battle_bonus", Integer),
+    Column("timetrial_bonus", Integer),
+    Column("fasteststreetlegaltheory_bonus", Integer),
+    Column("collaboration_bonus", Integer),
+    Column("add_bonus_daily_flag_1", Integer),
+    Column("add_bonus_daily_flag_2", Integer),
+    Column("add_bonus_daily_flag_3", Integer),
+    Column("create_date_daily", TIMESTAMP, server_default=func.now()),
+    Column("create_date_weekly", TIMESTAMP, server_default=func.now()),
+    UniqueConstraint("user", "m_stamp_event_id", name="idac_user_stamp_uk"),
+    mysql_charset="utf8mb4",
+)
+
+timetrial_event = Table(
+    "idac_user_timetrial_event",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("timetrial_event_id", Integer),
+    Column("point", Integer),
+    UniqueConstraint("user", "timetrial_event_id", name="idac_user_timetrial_event_uk"),
+    mysql_charset="utf8mb4",
+)
+
+
+class IDACItemData(BaseData):
+    def get_random_user_car(self, aime_id: int, version: int) -> Optional[List[Row]]:
+        sql = (
+            select(car)
+            .where(and_(car.c.user == aime_id, car.c.version == version))
+            .order_by(func.rand())
+            .limit(1)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_random_car(self, version: int) -> Optional[List[Row]]:
+        sql = select(car).where(car.c.version == version).order_by(func.rand()).limit(1)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_car(
+        self, aime_id: int, version: int, style_car_id: int
+    ) -> Optional[List[Row]]:
+        sql = select(car).where(
+            and_(
+                car.c.user == aime_id,
+                car.c.version == version,
+                car.c.style_car_id == style_car_id,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_cars(
+        self, version: int, aime_id: int, only_pickup: bool = False
+    ) -> Optional[List[Row]]:
+        if only_pickup:
+            sql = select(car).where(
+                and_(
+                    car.c.user == aime_id,
+                    car.c.version == version,
+                    car.c.pickup_seq != 0,
+                )
+            )
+        else:
+            sql = select(car).where(
+                and_(car.c.user == aime_id, car.c.version == version)
+            )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_ticket(self, aime_id: int, ticket_id: int) -> Optional[Row]:
+        sql = select(ticket).where(
+            ticket.c.user == aime_id, ticket.c.ticket_id == ticket_id
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_tickets(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select(ticket).where(ticket.c.user == aime_id)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_story(self, aime_id: int, chapter_id: int) -> Optional[Row]:
+        sql = select(story).where(
+            and_(story.c.user == aime_id, story.c.chapter == chapter_id)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_stories(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select(story).where(story.c.user == aime_id)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_story_episodes(self, aime_id: int, chapter_id: int) -> Optional[List[Row]]:
+        sql = select(episode).where(
+            and_(episode.c.user == aime_id, episode.c.chapter == chapter_id)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_story_episode(self, aime_id: int, episode_id: int) -> Optional[Row]:
+        sql = select(episode).where(
+            and_(episode.c.user == aime_id, episode.c.episode == episode_id)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_story_episode_difficulties(
+        self, aime_id: int, episode_id: int
+    ) -> Optional[List[Row]]:
+        sql = select(difficulty).where(
+            and_(difficulty.c.user == aime_id, difficulty.c.episode == episode_id)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_courses(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select(course).where(course.c.user == aime_id)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_course(self, aime_id: int, course_id: int) -> Optional[Row]:
+        sql = select(course).where(
+            and_(course.c.user == aime_id, course.c.course_id == course_id)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_time_trial_courses(self, version: int) -> Optional[List[Row]]:
+        sql = select(trial.c.course_id).where(trial.c.version == version).distinct()
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_time_trial_user_best_time_by_course_car(
+        self, version: int, aime_id: int, course_id: int, style_car_id: int
+    ) -> Optional[Row]:
+        sql = select(trial).where(
+            and_(
+                trial.c.user == aime_id,
+                trial.c.version == version,
+                trial.c.course_id == course_id,
+                trial.c.style_car_id == style_car_id,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_time_trial_user_best_courses(
+        self, version: int, aime_id: int
+    ) -> Optional[List[Row]]:
+        # get for a given aime_id the best time for each course
+        subquery = (
+            select(
+                trial.c.version,
+                func.min(trial.c.goal_time).label("min_goal_time"),
+                trial.c.course_id,
+            )
+            .where(and_(trial.c.version == version, trial.c.user == aime_id))
+            .group_by(trial.c.course_id)
+            .subquery()
+        )
+
+        # now get the full row for each best time
+        sql = select(trial).where(
+            and_(
+                trial.c.version == subquery.c.version,
+                trial.c.goal_time == subquery.c.min_goal_time,
+                trial.c.course_id == subquery.c.course_id,
+                trial.c.user == aime_id,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_time_trial_best_cars_by_course(
+        self, version: int, course_id: int, aime_id: Optional[int] = None
+    ) -> Optional[List[Row]]:
+        subquery = (
+            select(
+                trial.c.version,
+                func.min(trial.c.goal_time).label("min_goal_time"),
+                trial.c.style_car_id,
+            )
+            .where(
+                and_(
+                    trial.c.version == version,
+                    trial.c.course_id == course_id,
+                )
+            )
+        )
+
+        if aime_id is not None:
+            subquery = subquery.where(trial.c.user == aime_id)
+        
+        subquery = subquery.group_by(trial.c.style_car_id).subquery()
+
+        sql = select(trial).where(
+            and_(
+                trial.c.version == subquery.c.version,
+                trial.c.goal_time == subquery.c.min_goal_time,
+                trial.c.style_car_id == subquery.c.style_car_id,
+                trial.c.course_id == course_id,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_time_trial_ranking_by_course(
+        self,
+        version: int,
+        course_id: int,
+        style_car_id: Optional[int] = None,
+        limit: Optional[int] = 10,
+    ) -> Optional[List[Row]]:
+        # get the top 10 ranking by goal_time for a given course which is grouped by user
+        subquery = select(
+            trial.c.version,
+            trial.c.user,
+            func.min(trial.c.goal_time).label("min_goal_time"),
+        ).where(and_(trial.c.version == version, trial.c.course_id == course_id))
+
+        # if wantd filter only by style_car_id
+        if style_car_id is not None:
+            subquery = subquery.where(trial.c.style_car_id == style_car_id)
+
+        subquery = subquery.group_by(trial.c.user).subquery()
+
+        sql = (
+            select(trial)
+            .where(
+                and_(
+                    trial.c.version == subquery.c.version,
+                    trial.c.user == subquery.c.user,
+                    trial.c.goal_time == subquery.c.min_goal_time,
+                ),
+            )
+            .order_by(trial.c.goal_time)
+        )
+
+        # limit the result if needed
+        if limit is not None:
+            sql = sql.limit(limit)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_time_trial_best_ranking_by_course(
+        self, version: int, aime_id: int, course_id: int
+    ) -> Optional[Row]:
+        sql = (
+            select(trial)
+            .where(
+                and_(
+                    trial.c.version == version,
+                    trial.c.user == aime_id,
+                    trial.c.course_id == course_id,
+                ),
+            )
+            .order_by(trial.c.goal_time)
+            .limit(1)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_challenge(
+        self, aime_id: int, vs_type: int, play_difficulty: int
+    ) -> Optional[Row]:
+        sql = select(challenge).where(
+            and_(
+                challenge.c.user == aime_id,
+                challenge.c.vs_type == vs_type,
+                challenge.c.play_difficulty == play_difficulty,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_challenges(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select(challenge).where(challenge.c.user == aime_id)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_best_challenges_by_vs_type(
+        self, aime_id: int, story_type: int = 4
+    ) -> Optional[List[Row]]:
+        subquery = (
+            select(
+                challenge.c.story_type,
+                challenge.c.user,
+                challenge.c.vs_type,
+                func.max(challenge.c.play_difficulty).label("last_play_lv"),
+            )
+            .where(
+                and_(challenge.c.user == aime_id, challenge.c.story_type == story_type)
+            )
+            .group_by(challenge.c.vs_type)
+        )
+
+        sql = (
+            select(
+                challenge.c.story_type,
+                challenge.c.vs_type,
+                challenge.c.cleared_difficulty.label("max_clear_lv"),
+                challenge.c.play_difficulty.label("last_play_lv"),
+                challenge.c.course_id,
+                challenge.c.play_count,
+            )
+            .where(
+                and_(
+                    challenge.c.user == subquery.c.user,
+                    challenge.c.vs_type == subquery.c.vs_type,
+                    challenge.c.play_difficulty == subquery.c.last_play_lv,
+                ),
+            )
+            .order_by(challenge.c.vs_type)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_theory_courses(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select(theory_course).where(theory_course.c.user == aime_id)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_theory_course_by_powerhouse_lv(
+        self, aime_id: int, course_id: int, powerhouse_lv: int, count: int = 3
+    ) -> Optional[List[Row]]:
+        sql = (
+            select(theory_course)
+            .where(
+                and_(
+                    theory_course.c.user != aime_id,
+                    theory_course.c.course_id == course_id,
+                    theory_course.c.powerhouse_lv == powerhouse_lv,
+                )
+            )
+            .order_by(func.rand())
+            .limit(count)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_theory_course(self, aime_id: int, course_id: int) -> Optional[List[Row]]:
+        sql = select(theory_course).where(
+            and_(
+                theory_course.c.user == aime_id, theory_course.c.course_id == course_id
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_theory_partners(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select(theory_partner).where(theory_partner.c.user == aime_id)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_theory_running(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select(theory_running).where(theory_running.c.user == aime_id)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_theory_running_by_course(
+        self, aime_id: int, course_id: int
+    ) -> Optional[Row]:
+        sql = select(theory_running).where(
+            and_(
+                theory_running.c.user == aime_id,
+                theory_running.c.course_id == course_id,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_vs_infos(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select(vs_info).where(vs_info.c.user == aime_id)
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_stamps(self, aime_id: int) -> Optional[List[Row]]:
+        sql = select(stamp).where(
+            and_(
+                stamp.c.user == aime_id,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_timetrial_event(self, aime_id: int, timetrial_event_id: int) -> Optional[Row]:
+        sql = select(timetrial_event).where(
+            and_(
+                timetrial_event.c.user == aime_id,
+                timetrial_event.c.timetrial_event_id == timetrial_event_id,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def put_car(self, aime_id: int, version: int, car_data: Dict) -> Optional[int]:
+        car_data["user"] = aime_id
+        car_data["version"] = version
+
+        sql = insert(car).values(**car_data)
+        conflict = sql.on_duplicate_key_update(**car_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_car: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_ticket(self, aime_id: int, ticket_data: Dict) -> Optional[int]:
+        ticket_data["user"] = aime_id
+
+        sql = insert(ticket).values(**ticket_data)
+        conflict = sql.on_duplicate_key_update(**ticket_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_ticket: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_story(self, aime_id: int, story_data: Dict) -> Optional[int]:
+        story_data["user"] = aime_id
+
+        sql = insert(story).values(**story_data)
+        conflict = sql.on_duplicate_key_update(**story_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_story: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_story_episode_play_status(
+        self, aime_id: int, chapter_id: int, play_status: int = 1
+    ) -> Optional[int]:
+        sql = (
+            update(episode)
+            .where(and_(episode.c.user == aime_id, episode.c.chapter == chapter_id))
+            .values(play_status=play_status)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            self.logger.warn(
+                f"put_story_episode_play_status: Failed to update! aime_id: {aime_id}"
+            )
+            return None
+        return result.lastrowid
+
+    def put_story_episode(
+        self, aime_id: int, chapter_id: int, episode_data: Dict
+    ) -> Optional[int]:
+        episode_data["user"] = aime_id
+        episode_data["chapter"] = chapter_id
+
+        sql = insert(episode).values(**episode_data)
+        conflict = sql.on_duplicate_key_update(**episode_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_story_episode: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_story_episode_difficulty(
+        self, aime_id: int, episode_id: int, difficulty_data: Dict
+    ) -> Optional[int]:
+        difficulty_data["user"] = aime_id
+        difficulty_data["episode"] = episode_id
+
+        sql = insert(difficulty).values(**difficulty_data)
+        conflict = sql.on_duplicate_key_update(**difficulty_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(
+                f"put_story_episode_difficulty: Failed to update! aime_id: {aime_id}"
+            )
+            return None
+        return result.lastrowid
+
+    def put_course(self, aime_id: int, course_data: Dict) -> Optional[int]:
+        course_data["user"] = aime_id
+
+        sql = insert(course).values(**course_data)
+        conflict = sql.on_duplicate_key_update(**course_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_course: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_time_trial(
+        self, version: int, aime_id: int, time_trial_data: Dict
+    ) -> Optional[int]:
+        time_trial_data["user"] = aime_id
+        time_trial_data["version"] = version
+
+        sql = insert(trial).values(**time_trial_data)
+        conflict = sql.on_duplicate_key_update(**time_trial_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_time_trial: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_challenge(self, aime_id: int, challenge_data: Dict) -> Optional[int]:
+        challenge_data["user"] = aime_id
+
+        sql = insert(challenge).values(**challenge_data)
+        conflict = sql.on_duplicate_key_update(**challenge_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_challenge: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_theory_course(
+        self, aime_id: int, theory_course_data: Dict
+    ) -> Optional[int]:
+        theory_course_data["user"] = aime_id
+
+        sql = insert(theory_course).values(**theory_course_data)
+        conflict = sql.on_duplicate_key_update(**theory_course_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_theory_course: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_theory_partner(
+        self, aime_id: int, theory_partner_data: Dict
+    ) -> Optional[int]:
+        theory_partner_data["user"] = aime_id
+
+        sql = insert(theory_partner).values(**theory_partner_data)
+        conflict = sql.on_duplicate_key_update(**theory_partner_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(
+                f"put_theory_partner: Failed to update! aime_id: {aime_id}"
+            )
+            return None
+        return result.lastrowid
+
+    def put_theory_running(
+        self, aime_id: int, theory_running_data: Dict
+    ) -> Optional[int]:
+        theory_running_data["user"] = aime_id
+
+        sql = insert(theory_running).values(**theory_running_data)
+        conflict = sql.on_duplicate_key_update(**theory_running_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(
+                f"put_theory_running: Failed to update! aime_id: {aime_id}"
+            )
+            return None
+        return result.lastrowid
+
+    def put_vs_info(self, aime_id: int, vs_info_data: Dict) -> Optional[int]:
+        vs_info_data["user"] = aime_id
+
+        sql = insert(vs_info).values(**vs_info_data)
+        conflict = sql.on_duplicate_key_update(**vs_info_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_vs_info: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_stamp(
+        self, aime_id: int, stamp_data: Dict
+    ) -> Optional[int]:
+        stamp_data["user"] = aime_id
+
+        sql = insert(stamp).values(**stamp_data)
+        conflict = sql.on_duplicate_key_update(**stamp_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(
+                f"putstamp: Failed to update! aime_id: {aime_id}"
+            )
+            return None
+        return result.lastrowid
+
+    def put_timetrial_event(
+        self, aime_id: int, time_trial_event_id: int, point: int
+    ) -> Optional[int]:
+        timetrial_event_data = {
+            "user": aime_id,
+            "timetrial_event_id": time_trial_event_id,
+            "point": point,
+        }
+
+        sql = insert(timetrial_event).values(**timetrial_event_data)
+        conflict = sql.on_duplicate_key_update(**timetrial_event_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(
+                f"put_timetrial_event: Failed to update! aime_id: {aime_id}"
+            )
+            return None
+        return result.lastrowid

titles/idac/schema/profile.py

@@ -0,0 +1,440 @@
+from typing import Dict, List, Optional
+from sqlalchemy import Table, Column, UniqueConstraint, PrimaryKeyConstraint, and_
+from sqlalchemy.types import Integer, String, TIMESTAMP, Boolean, JSON, BigInteger
+from sqlalchemy.engine.base import Connection
+from sqlalchemy.schema import ForeignKey
+from sqlalchemy.sql import func, select
+from sqlalchemy.engine import Row
+from sqlalchemy.dialects.mysql import insert
+
+from core.data.schema import BaseData, metadata
+from core.config import CoreConfig
+
+profile = Table(
+    "idac_profile",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("version", Integer, nullable=False),
+    Column("username", String(8)),
+    Column("country", Integer),
+    Column("store", Integer),
+    Column("team_id", Integer, server_default="0"),
+    Column("total_play", Integer, server_default="0"),
+    Column("daily_play", Integer, server_default="0"),
+    Column("day_play", Integer, server_default="0"),
+    Column("mileage", Integer, server_default="0"),
+    Column("asset_version", Integer, server_default="1"),
+    Column("last_play_date", TIMESTAMP, server_default=func.now()),
+    Column("mytitle_id", Integer, server_default="0"),
+    Column("mytitle_efffect_id", Integer, server_default="0"),
+    Column("sticker_id", Integer, server_default="0"),
+    Column("sticker_effect_id", Integer, server_default="0"),
+    Column("papercup_id", Integer, server_default="0"),
+    Column("tachometer_id", Integer, server_default="0"),
+    Column("aura_id", Integer, server_default="0"),
+    Column("aura_color_id", Integer, server_default="0"),
+    Column("aura_line_id", Integer, server_default="0"),
+    Column("bgm_id", Integer, server_default="0"),
+    Column("keyholder_id", Integer, server_default="0"),
+    Column("start_menu_bg_id", Integer, server_default="0"),
+    Column("use_car_id", Integer, server_default="1"),
+    Column("use_style_car_id", Integer, server_default="1"),
+    Column("bothwin_count", Integer, server_default="0"),
+    Column("bothwin_score", Integer, server_default="0"),
+    Column("subcard_count", Integer, server_default="0"),
+    Column("vs_history", Integer, server_default="0"),
+    Column("stamp_key_assign_0", Integer),
+    Column("stamp_key_assign_1", Integer),
+    Column("stamp_key_assign_2", Integer),
+    Column("stamp_key_assign_3", Integer),
+    Column("name_change_category", Integer, server_default="0"),
+    Column("factory_disp", Integer, server_default="0"),
+    Column("create_date", TIMESTAMP, server_default=func.now()),
+    Column("cash", Integer, server_default="0"),
+    Column("dressup_point", Integer, server_default="0"),
+    Column("avatar_point", Integer, server_default="0"),
+    Column("total_cash", Integer, server_default="0"),
+    UniqueConstraint("user", "version", name="idac_profile_uk"),
+    mysql_charset="utf8mb4",
+)
+
+# No point setting defaults since the game sends everything on profile creation anyway
+config = Table(
+    "idac_profile_config",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("config_id", Integer),
+    Column("steering_intensity", Integer),
+    Column("transmission_type", Integer),
+    Column("default_viewpoint", Integer),
+    Column("favorite_bgm", Integer),
+    Column("bgm_volume", Integer),
+    Column("se_volume", Integer),
+    Column("master_volume", Integer),
+    Column("store_battle_policy", Integer),
+    Column("battle_onomatope_display", Integer),
+    Column("cornering_guide", Integer),
+    Column("minimap", Integer),
+    Column("line_guide", Integer),
+    Column("ghost", Integer),
+    Column("race_exit", Integer),
+    Column("result_skip", Integer),
+    Column("stamp_select_skip", Integer),
+    UniqueConstraint("user", name="idac_profile_config_uk"),
+    mysql_charset="utf8mb4",
+)
+
+# No point setting defaults since the game sends everything on profile creation anyway
+avatar = Table(
+    "idac_profile_avatar",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("sex", Integer),
+    Column("face", Integer),
+    Column("eye", Integer),
+    Column("mouth", Integer),
+    Column("hair", Integer),
+    Column("glasses", Integer),
+    Column("face_accessory", Integer),
+    Column("body", Integer),
+    Column("body_accessory", Integer),
+    Column("behind", Integer),
+    Column("bg", Integer),
+    Column("effect", Integer),
+    Column("special", Integer),
+    UniqueConstraint("user", name="idac_profile_avatar_uk"),
+    mysql_charset="utf8mb4",
+)
+
+rank = Table(
+    "idac_profile_rank",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("version", Integer, nullable=False),
+    Column("story_rank_exp", Integer, server_default="0"),
+    Column("story_rank", Integer, server_default="1"),
+    Column("time_trial_rank_exp", Integer, server_default="0"),
+    Column("time_trial_rank", Integer, server_default="1"),
+    Column("online_battle_rank_exp", Integer, server_default="0"),
+    Column("online_battle_rank", Integer, server_default="1"),
+    Column("store_battle_rank_exp", Integer, server_default="0"),
+    Column("store_battle_rank", Integer, server_default="1"),
+    Column("theory_exp", Integer, server_default="0"),
+    Column("theory_rank", Integer, server_default="1"),
+    Column("pride_group_id", Integer, server_default="0"),
+    Column("pride_point", Integer, server_default="0"),
+    Column("grade_exp", Integer, server_default="0"),
+    Column("grade", Integer, server_default="1"),
+    Column("grade_reward_dist", Integer, server_default="0"),
+    Column("story_rank_reward_dist", Integer, server_default="0"),
+    Column("time_trial_rank_reward_dist", Integer, server_default="0"),
+    Column("online_battle_rank_reward_dist", Integer, server_default="0"),
+    Column("store_battle_rank_reward_dist", Integer, server_default="0"),
+    Column("theory_rank_reward_dist", Integer, server_default="0"),
+    Column("max_attained_online_battle_rank", Integer, server_default="1"),
+    Column("max_attained_pride_point", Integer, server_default="0"),
+    Column("is_last_max", Integer, server_default="0"),
+    UniqueConstraint("user", "version", name="idac_profile_rank_uk"),
+    mysql_charset="utf8mb4",
+)
+
+stock = Table(
+    "idac_profile_stock",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("version", Integer, nullable=False),
+    Column("mytitle_list", String(1024), server_default=""),
+    Column("mytitle_new_list", String(1024), server_default=""),
+    Column("avatar_face_list", String(255), server_default=""),
+    Column("avatar_face_new_list", String(255), server_default=""),
+    Column("avatar_eye_list", String(255), server_default=""),
+    Column("avatar_eye_new_list", String(255), server_default=""),
+    Column("avatar_hair_list", String(255), server_default=""),
+    Column("avatar_hair_new_list", String(255), server_default=""),
+    Column("avatar_body_list", String(255), server_default=""),
+    Column("avatar_body_new_list", String(255), server_default=""),
+    Column("avatar_mouth_list", String(255), server_default=""),
+    Column("avatar_mouth_new_list", String(255), server_default=""),
+    Column("avatar_glasses_list", String(255), server_default=""),
+    Column("avatar_glasses_new_list", String(255), server_default=""),
+    Column("avatar_face_accessory_list", String(255), server_default=""),
+    Column("avatar_face_accessory_new_list", String(255), server_default=""),
+    Column("avatar_body_accessory_list", String(255), server_default=""),
+    Column("avatar_body_accessory_new_list", String(255), server_default=""),
+    Column("avatar_behind_list", String(255), server_default=""),
+    Column("avatar_behind_new_list", String(255), server_default=""),
+    Column("avatar_bg_list", String(255), server_default=""),
+    Column("avatar_bg_new_list", String(255), server_default=""),
+    Column("avatar_effect_list", String(255), server_default=""),
+    Column("avatar_effect_new_list", String(255), server_default=""),
+    Column("avatar_special_list", String(255), server_default=""),
+    Column("avatar_special_new_list", String(255), server_default=""),
+    Column("stamp_list", String(255), server_default=""),
+    Column("stamp_new_list", String(255), server_default=""),
+    Column("keyholder_list", String(256), server_default=""),
+    Column("keyholder_new_list", String(256), server_default=""),
+    Column("papercup_list", String(255), server_default=""),
+    Column("papercup_new_list", String(255), server_default=""),
+    Column("tachometer_list", String(255), server_default=""),
+    Column("tachometer_new_list", String(255), server_default=""),
+    Column("aura_list", String(255), server_default=""),
+    Column("aura_new_list", String(255), server_default=""),
+    Column("aura_color_list", String(255), server_default=""),
+    Column("aura_color_new_list", String(255), server_default=""),
+    Column("aura_line_list", String(255), server_default=""),
+    Column("aura_line_new_list", String(255), server_default=""),
+    Column("bgm_list", String(255), server_default=""),
+    Column("bgm_new_list", String(255), server_default=""),
+    Column("dx_color_list", String(255), server_default=""),
+    Column("dx_color_new_list", String(255), server_default=""),
+    Column("start_menu_bg_list", String(255), server_default=""),
+    Column("start_menu_bg_new_list", String(255), server_default=""),
+    Column("under_neon_list", String(255), server_default=""),
+    UniqueConstraint("user", "version", name="idac_profile_stock_uk"),
+    mysql_charset="utf8mb4",
+)
+
+theory = Table(
+    "idac_profile_theory",
+    metadata,
+    Column("id", Integer, primary_key=True, nullable=False),
+    Column(
+        "user",
+        ForeignKey("aime_user.id", ondelete="cascade", onupdate="cascade"),
+        nullable=False,
+    ),
+    Column("version", Integer, nullable=False),
+    Column("play_count", Integer, server_default="0"),
+    Column("play_count_multi", Integer, server_default="0"),
+    Column("partner_id", Integer),
+    Column("partner_progress", Integer),
+    Column("partner_progress_score", Integer),
+    Column("practice_start_rank", Integer, server_default="0"),
+    Column("general_flag", Integer, server_default="0"),
+    Column("vs_history", Integer, server_default="0"),
+    Column("vs_history_multi", Integer, server_default="0"),
+    Column("win_count", Integer, server_default="0"),
+    Column("win_count_multi", Integer, server_default="0"),
+    UniqueConstraint("user", "version", name="idac_profile_theory_uk"),
+    mysql_charset="utf8mb4",
+)
+
+
+class IDACProfileData(BaseData):
+    def __init__(self, cfg: CoreConfig, conn: Connection) -> None:
+        super().__init__(cfg, conn)
+        self.date_time_format_ext = (
+            "%Y-%m-%d %H:%M:%S.%f"  # needs to be lopped off at [:-5]
+        )
+        self.date_time_format_short = "%Y-%m-%d"
+
+    def get_profile(self, aime_id: int, version: int) -> Optional[Row]:
+        sql = select(profile).where(
+            and_(
+                profile.c.user == aime_id,
+                profile.c.version == version,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_different_random_profiles(
+        self, aime_id: int, version: int, count: int = 9
+    ) -> Optional[Row]:
+        sql = (
+            select(profile)
+            .where(
+                and_(
+                    profile.c.user != aime_id,
+                    profile.c.version == version,
+                )
+            )
+            .order_by(func.rand())
+            .limit(count)
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchall()
+
+    def get_profile_config(self, aime_id: int) -> Optional[Row]:
+        sql = select(config).where(
+            and_(
+                config.c.user == aime_id,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_profile_avatar(self, aime_id: int) -> Optional[Row]:
+        sql = select(avatar).where(
+            and_(
+                avatar.c.user == aime_id,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_profile_rank(self, aime_id: int, version: int) -> Optional[Row]:
+        sql = select(rank).where(
+            and_(
+                rank.c.user == aime_id,
+                rank.c.version == version,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_profile_stock(self, aime_id: int, version: int) -> Optional[Row]:
+        sql = select(stock).where(
+            and_(
+                stock.c.user == aime_id,
+                stock.c.version == version,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def get_profile_theory(self, aime_id: int, version: int) -> Optional[Row]:
+        sql = select(theory).where(
+            and_(
+                theory.c.user == aime_id,
+                theory.c.version == version,
+            )
+        )
+
+        result = self.execute(sql)
+        if result is None:
+            return None
+        return result.fetchone()
+
+    def put_profile(
+        self, aime_id: int, version: int, profile_data: Dict
+    ) -> Optional[int]:
+        profile_data["user"] = aime_id
+        profile_data["version"] = version
+
+        sql = insert(profile).values(**profile_data)
+        conflict = sql.on_duplicate_key_update(**profile_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_profile: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_profile_config(self, aime_id: int, config_data: Dict) -> Optional[int]:
+        config_data["user"] = aime_id
+
+        sql = insert(config).values(**config_data)
+        conflict = sql.on_duplicate_key_update(**config_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(
+                f"put_profile_config: Failed to update! aime_id: {aime_id}"
+            )
+            return None
+        return result.lastrowid
+
+    def put_profile_avatar(self, aime_id: int, avatar_data: Dict) -> Optional[int]:
+        avatar_data["user"] = aime_id
+
+        sql = insert(avatar).values(**avatar_data)
+        conflict = sql.on_duplicate_key_update(**avatar_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(
+                f"put_profile_avatar: Failed to update! aime_id: {aime_id}"
+            )
+            return None
+        return result.lastrowid
+
+    def put_profile_rank(
+        self, aime_id: int, version: int, rank_data: Dict
+    ) -> Optional[int]:
+        rank_data["user"] = aime_id
+        rank_data["version"] = version
+
+        sql = insert(rank).values(**rank_data)
+        conflict = sql.on_duplicate_key_update(**rank_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_profile_rank: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_profile_stock(
+        self, aime_id: int, version: int, stock_data: Dict
+    ) -> Optional[int]:
+        stock_data["user"] = aime_id
+        stock_data["version"] = version
+
+        sql = insert(stock).values(**stock_data)
+        conflict = sql.on_duplicate_key_update(**stock_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(f"put_profile_stock: Failed to update! aime_id: {aime_id}")
+            return None
+        return result.lastrowid
+
+    def put_profile_theory(
+        self, aime_id: int, version: int, theory_data: Dict
+    ) -> Optional[int]:
+        theory_data["user"] = aime_id
+        theory_data["version"] = version
+
+        sql = insert(theory).values(**theory_data)
+        conflict = sql.on_duplicate_key_update(**theory_data)
+        result = self.execute(conflict)
+
+        if result is None:
+            self.logger.warn(
+                f"put_profile_theory: Failed to update! aime_id: {aime_id}"
+            )
+            return None
+        return result.lastrowid