1
0
mirror of synced 2025-02-15 09:52:35 +01:00

35 Commits

Author SHA1 Message Date
beerpsi
d6d98d20cb fix: typing shenanigans 2024-12-12 20:47:34 +07:00
beerpsi
476a911df9 [database] fix invalid transaction being left open 2024-11-25 20:13:51 +07:00
beerpsi
58a5177a30 use SQL's limit/offset pagination for nextIndex/maxCount requests (#185)
Instead of retrieving the entire list of items/characters/scores/etc. at once (and even store them in memory), use SQL's `LIMIT ... OFFSET ...` pagination so we only take what we need.

Currently only CHUNITHM uses this, but this will also affect maimai DX and O.N.G.E.K.I. once the PR is ready.

Also snuck in a fix for CHUNITHM/maimai DX's `GetUserRivalMusicApi` to respect the `userRivalMusicLevelList` sent by the client.

### How this works

Say we have a `GetUserCharacterApi` request:

```json
{
    "userId": 10000,
    "maxCount": 700,
    "nextIndex": 0
}
```

Instead of getting the entire character list from the database (which can be very large if the user force unlocked everything), add limit/offset to the query:

```python
select(character)
.where(character.c.user == user_id)
.order_by(character.c.id.asc())
.limit(max_count + 1)
.offset(next_index)
```

The query takes `maxCount + 1` items from the database to determine if there is more items than can be returned:

```python
rows = ...

if len(rows) > max_count:
    # return only max_count rows
    next_index += max_count
else:
    # return everything left
    next_index = -1
```

This has the benefit of not needing to load everything into memory (and also having to store server state, as seen in the [`SCORE_BUFFER` list](2274b42358/titles/chuni/base.py (L13)).)

Reviewed-on: https://gitea.tendokyu.moe/Hay1tsme/artemis/pulls/185
Co-authored-by: beerpsi <beerpsi@duck.com>
Co-committed-by: beerpsi <beerpsi@duck.com>
2024-11-16 19:10:29 +00:00
beerpsi
789d50c406 use AsyncSession directly
see the warnings in https://docs.sqlalchemy.org/en/14/orm/extensions/asyncio.html#using-asyncio-scoped-session
2024-11-14 13:10:14 +07:00
beerpsi
bc7524c8fc fix: make database async 2024-11-14 12:36:22 +07:00
Midorica
b7a006f7ee core: pushing changes regarding MySQL ssl toggle that is now mandatory 2024-11-12 10:53:02 -05:00
Hay1tsme
1072a9d63b database: fix create_revision_auto 2024-04-02 16:43:13 -04:00
Kevin Trocolli
b63c2c2d12 dbutils: add autocreate option to revision 2024-03-11 21:24:57 -04:00
Kevin Trocolli
c4deff9d1c add database downgrading 2024-03-04 00:50:51 -05:00
Kevin Trocolli
bd97428166 frontend: fix account and owner creation (fixes #108) 2024-03-02 17:48:48 -05:00
Kevin Trocolli
fc3f0900b3 dbutils: add create-revision command 2024-01-11 20:48:27 -05:00
Kevin Trocolli
19fd008598 fix legacy upgrade script 2024-01-11 20:39:29 -05:00
Hay1tsme
261d09aaef dbutils: add legacy migration 2024-01-09 17:11:49 -05:00
Hay1tsme
edd3ce8ead move to alembic 2024-01-09 13:57:59 -05:00
Hay1tsme
88a1462304 logger: change from warn to warning 2023-08-08 10:17:56 -04:00
Kevin Trocolli
b943807904 core: add columns to machine table, bump to v5 2023-07-23 22:21:41 -04:00
Kevin Trocolli
2f13596885 fix db ignoring port in config, createing database no longer runs over version 2023-07-15 00:15:14 -04:00
Kevin Trocolli
6c155a5e48 database: add static variables to prevent having multiple sessions 2023-07-08 00:01:52 -04:00
Kevin Trocolli
858b101a36 dbutils: add command to show versions 2023-06-24 13:14:40 -04:00
Kevin Trocolli
238d437519 reformat with black in preperation for merge to master 2023-04-23 21:04:52 -04:00
Kevin Trocolli
0dc96f33e1 database: don't set schema version if autoupdate fails 2023-04-15 03:13:14 -04:00
Kevin Trocolli
4102ba21fc database: remove print 2023-04-15 03:07:15 -04:00
Kevin Trocolli
9895068125 database: fix autoupdate 2023-04-15 01:31:52 -04:00
Kevin Trocolli
a416fb09e1 dbutils: version can now be left black to auto-upgrade to latest 2023-04-15 00:13:04 -04:00
Kevin Trocolli
188be2dfc1 database: add autoupgrade command 2023-03-18 02:12:58 -04:00
Kevin Trocolli
a791142f95 database: add check for current_schema_version 2023-03-12 22:37:44 -04:00
Kevin Trocolli
ea14f105d5 database: skip games that lack a database member 2023-03-12 00:26:48 -05:00
Hay1tsme
a76bb94eb1 let black do it's magic 2023-03-09 11:38:58 -05:00
Hay1tsme
f5d4f519d3 database: add create-owner, migrate-card, and cleanup commands 2023-03-04 00:04:47 -05:00
Hay1tsme
279f48dc0c frontend: fix login, remove frontend_session in favor of twisted sessions 2023-03-03 21:31:23 -05:00
Hay1tsme
2a6842db24 remove db old-to-new migration 2023-03-03 17:03:19 -05:00
Hay1tsme
db6b950c29 add partial frontend 2023-02-19 15:40:25 -05:00
Hay1tsme
97d16365df carry over database functions from megaime 2023-02-19 14:52:20 -05:00
Hay1tsme
f18e939dd0 change to using txroutes 2023-02-16 17:13:41 -05:00
Hay1tsme
32879491f4 Begin the process of transitioning from megaime 2023-02-16 00:06:42 -05:00