Jennifer Taylor
4790385022
Fix a logic error in if statement unwrapping, remove duplicated code in favor of a better true/false detection algorithm.
2021-05-04 02:30:37 +00:00
Jennifer Taylor
108d7c228d
Handle having bytecode with no actual entries.
2021-05-04 02:29:53 +00:00
Jennifer Taylor
54b4c88d65
It seems that some bytecode leaves extra stuff on the stack sometimes, I think this is fine?
2021-05-04 02:29:27 +00:00
Jennifer Taylor
70c7448aa0
Fix nasty bug where sometimes we would have gotos without corresponding labels.
2021-05-02 03:50:48 +00:00
Jennifer Taylor
336ef6fc3d
Check in some beta code for a new style of if detection that does not work yet.
2021-05-02 03:50:19 +00:00
Jennifer Taylor
7493db034f
Add in basic throw, test code generation of mostly the same code as we tested the code graph with.
2021-05-02 03:49:35 +00:00
Jennifer Taylor
829597a871
Address the fact that I never handled END actions (they end processing as they are encountered).
2021-05-02 03:48:38 +00:00
Jennifer Taylor
261c3d7fbd
Start a unit test framework for decompilation, in the hopes of eventually tracking down a logic bug in if extraction.
2021-05-02 03:48:07 +00:00
Jennifer Taylor
820a86845d
Fix shallowest successor find when multiple nodes lead to the same location, fix if swap optimization and add BITAND and NOT_BITAND if support.
2021-04-26 03:24:47 +00:00
Jennifer Taylor
8cf219cb25
Avoid duplicating labels outside of and inside of do-while statements. Avoid going to or generating negative labels (artificially inserted nodes).
2021-04-26 03:24:24 +00:00
Jennifer Taylor
b7523b5521
Fix if processing by getting rid of negation early optimization. There was a path that
...
was negating but not being checked when doing the final render, leaving some if statements
inverted. Moved the optimization to a final pass for more generalized correctness.
2021-04-26 02:50:19 +00:00
Jennifer Taylor
2d4f6cadfb
Much more complicated stack walking algorithm that chains the stack through chunks and also calculates when temporary variables are needed.
2021-04-26 01:28:48 +00:00
Jennifer Taylor
4d0bfe5637
Fix an edge case in if detection where we reused IDs incorrectly by stopping the reuse of IDs. Also, convert returns much earlier in the stack.
2021-04-26 01:27:51 +00:00
Jennifer Taylor
28ce17b996
Apparently some BishiBashi levels use an even older AP2 version.
2021-04-26 01:27:10 +00:00
Jennifer Taylor
06bd9d1245
Fix logic error where we lost if statement jump points because __gather_chunks was too-aggressively dropping end pointers.
2021-04-26 01:26:50 +00:00
Jennifer Taylor
6f78668e9a
Fix up exceptions and add more invariant checks to loop generation code.
2021-04-26 01:25:32 +00:00
Jennifer Taylor
5761474bec
Enforce another invariant for dominator calculation that would have caught another hard-to-find bug.
2021-04-26 01:24:25 +00:00
Jennifer Taylor
a09ad70de5
Improve a lot of exception message, enforce several more invariants in
...
graph generation that would have caught previous bugs, fix possible
issue with loop/dominator code if the entry code chunk was the beginning
of a loop.
2021-04-26 01:23:54 +00:00
Jennifer Taylor
44f5e5cdcc
Fix control flow graph splitting function to handle functions that jump to the end offset as a way of exiting early.
2021-04-26 01:23:08 +00:00
Jennifer Taylor
6323ef9adf
Implment several more opcodes in actual decompilation step.
2021-04-26 01:22:39 +00:00
Jennifer Taylor
6052deed3c
Refactor code generation step to allow for nested function definitions, actually hook those up.
2021-04-26 01:22:14 +00:00
Jennifer Taylor
3a9b3a7b3c
Pass indentation context into expression rendering for upcoming function definition support.
2021-04-26 01:21:42 +00:00
Jennifer Taylor
351e7060b8
Allow subclasses of verbose manager to check verbose state directly.
2021-04-26 01:21:03 +00:00
Jennifer Taylor
51e27c0cff
Enhance goto eliminiation to remove all unneeded gotos.
2021-04-24 19:37:14 +00:00
Jennifer Taylor
964d6f082c
Clean up types on If statement, stop using strings to pass information.
2021-04-24 19:36:58 +00:00
Jennifer Taylor
534ab20f98
Add dead code eliminiation so dominators doesn't fail. Fix a tricky jump case with if statements.
2021-04-24 18:13:07 +00:00
Jennifer Taylor
de84379ad0
Implement a lot more opcodes for expressions, fix an if conditional, improve while display.
2021-04-24 18:11:20 +00:00
Jennifer Taylor
cc27d2418e
Fix register display on assignment, add a few optimizing passes to clean up code.
2021-04-24 18:10:27 +00:00
Jennifer Taylor
f1aea996c4
Handle if-goto pattern that I previously neglected.
2021-04-24 18:09:44 +00:00
Jennifer Taylor
6221d0273b
Overhauled statement eval system so we always get back a list of statements.
2021-04-24 18:08:04 +00:00
Jennifer Taylor
b0778e1110
Convert expression rendering to an explicit function so we can retain __repr__ for debugging.
2021-04-24 18:07:01 +00:00
Jennifer Taylor
f2761b90b0
Restructure a lot of types to fit an expression/statement model and making a lot of stuff make more sense.
2021-04-24 18:05:58 +00:00
Jennifer Taylor
19c6de1fcc
Actually decompile basic bytecode. There's much left to do, but some basics work!
2021-04-24 18:05:03 +00:00
Jennifer Taylor
aebc277b62
Add a graph walker at the very end to verify that we've handled all control paths correctly, fix an edge case in If detection.
2021-04-24 18:03:24 +00:00
Jennifer Taylor
6aa04b0c1b
Implement recursive if detection.
2021-04-24 18:02:26 +00:00
Jennifer Taylor
6e34d2647e
Implement loop break/continue/goto processing.
2021-04-24 18:01:25 +00:00
Jennifer Taylor
738fce36c9
Lots more implementation done on decompiler, including better control flow and loop detection.
2021-04-24 18:00:13 +00:00
Jennifer Taylor
b77ccdd5b9
Beginning of an AFP ByteCode decompiler, starting with a massive code reorg and a control flow graph analyzer.
2021-04-24 17:59:36 +00:00
Jennifer Taylor
011b84aceb
Actually parse bytecode into an intermediate representation.
2021-04-21 03:58:53 +00:00
Jennifer Taylor
f209bcbe54
Exit early if we're done rendering the main animation, allow background color override.
2021-04-21 01:06:48 +00:00
Jennifer Taylor
47525837cd
Allow partial renders by hitting ctrl-c.
2021-04-21 00:01:35 +00:00
Jennifer Taylor
cc25c3c8dc
Don't draw shapes/sprites with zero scaling factor.
2021-04-20 23:33:43 +00:00
Jennifer Taylor
05a85abce8
Convert some more exceptions to warnings for best-effort renderings.
2021-04-20 23:19:14 +00:00
Jennifer Taylor
4c7ac0f744
Rework engine so multiple of the same sprite can be placed down by place item tags.
2021-04-20 21:41:28 +00:00
Jennifer Taylor
81c4496269
Implement what I think are the correct blending modes. Fixes some older Pop'n animations.
2021-04-20 21:41:09 +00:00
Jennifer Taylor
5a499a0f2c
Fix rotation origin so some bishi bashi levels will render.
2021-04-17 23:33:04 +00:00
Jennifer Taylor
d1cf978e12
Change rendering output style, allow series of png files to be rendered.
2021-04-17 23:32:47 +00:00
Jennifer Taylor
8cd85f26e9
Separate blend and color arguments for ease of experimenting.
2021-04-17 23:32:30 +00:00
Jennifer Taylor
ebc86019ae
Implement color blending, add fast path back to rendering shapes.
2021-04-17 23:32:10 +00:00
Jennifer Taylor
e1c6ad429c
Implement proper affine transformations instead of just translation.
2021-04-17 23:31:36 +00:00
Jennifer Taylor
c6e19d0dfa
Overhaul clip playback engine to allow clips to start when they're placed, not when they're created.
2021-04-17 23:31:08 +00:00
Jennifer Taylor
830f32814e
Clean up the code some in preparation for trying to get affine transformations working.
2021-04-17 23:30:46 +00:00
Jennifer Taylor
700e63a001
Convert some more exceptions to warnings to allow attempts to render some BishiBashi content.
2021-04-16 21:28:53 +00:00
Jennifer Taylor
de5dfd2421
Hook up TXP2 container to renderer, provide a "list" option to list out possible rendering paths in a container.
2021-04-16 21:08:41 +00:00
Jennifer Taylor
d4faa9f7d8
Fix path issues for a few miscelaneous tools on Windows.
2021-04-16 03:51:16 +00:00
Jennifer Taylor
42f57e10d7
Include origin point in updated tags.
2021-04-15 23:24:46 +00:00
Jennifer Taylor
3ad60064f9
Convert some exceptions to warnings so files can be partially rendered.
2021-04-15 23:20:27 +00:00
Jennifer Taylor
d835c46320
Support nested animations as well as update object requests.
2021-04-15 23:19:56 +00:00
Jennifer Taylor
3f5760343a
Always filter out unnecessary exported tags, even when rendering the default animation.
2021-04-15 23:18:54 +00:00
Jennifer Taylor
1683c8ecdd
Initial stab at an AFP animation renderer. It can render some basic animations from Pop'n Music!
2021-04-15 23:18:33 +00:00
Jennifer Taylor
3941b7e602
Fix a few more typos in object placement, values make a lot more sense now.
2021-04-15 23:17:45 +00:00
Jennifer Taylor
61603b0e63
Codify my guess as to what's going on with the mysterious point.
2021-04-15 23:17:15 +00:00
Jennifer Taylor
544baae387
Fix a few small bugs with place object handling.
2021-04-15 23:16:45 +00:00
Jennifer Taylor
db36298f22
Slightly better IFS extraction and shape documentation, given my understanding of the relationship between the two.
2021-04-15 23:16:03 +00:00
Jennifer Taylor
520181785d
Vastly improved IFS extraction which names afp/bsi/geo files correctly.
2021-04-14 01:08:13 +00:00
Jennifer Taylor
d486cfa54e
Finish handling push types and a few miscelaneous constants.
2021-04-14 01:07:33 +00:00
Jennifer Taylor
142ee31d56
Finish initial RE on bytecode, every opcode is now accounted for.
2021-04-14 01:07:01 +00:00
Jennifer Taylor
79b31c7fa2
Hook up parsed objects to all of SWF except for the bytecode mess.
2021-04-12 03:10:31 +00:00
Jennifer Taylor
8d686dcdb7
Assert on unknown data inside GE2D structures, so we don't miss something important.
2021-04-12 03:10:14 +00:00
Jennifer Taylor
9f6b9eb7d6
Begin parsing AFP data into useful structures.
2021-04-12 03:09:57 +00:00
Jennifer Taylor
9925b2f6b0
Add an extract option to write afp/bsi/geo files out of TXP2 containers similar to IFS containers.
2021-04-11 20:45:40 +00:00
Jennifer Taylor
522f8eaa29
Refactor verbose debugging to its own class, clean up classes that used it.
2021-04-11 20:45:17 +00:00
Jennifer Taylor
897e779b20
Refactor coverage stuff out to its own utility class, clean up code that used it.
2021-04-11 20:44:55 +00:00
Jennifer Taylor
30a51f48e6
Massive juggling of core AFP/AP2 implementation into its own files.
2021-04-11 20:44:31 +00:00
Jennifer Taylor
f1294df839
Split types up to generic types and AP2-specific types.
2021-04-11 20:44:11 +00:00
Jennifer Taylor
cef8bbfa83
Separate types out into their own module.
2021-04-11 20:43:51 +00:00
Jennifer Taylor
749f2bab0d
Move AFP library in preparation of refactoring.
2021-04-11 20:43:27 +00:00
Jennifer Taylor
fce04a2493
Implement a few more bytecode parsers, implement a few more push types, fix advancing past tags which are not 4-byte aligned in size.
2021-04-10 03:47:15 +00:00
Jennifer Taylor
61bd70f803
Implement some of edit text tags.
2021-04-10 03:46:52 +00:00
Jennifer Taylor
72f81e4f58
Fix typing mistake not caught on previous AP2 check-in.
2021-04-08 01:05:48 +00:00
Jennifer Taylor
a3f247f422
Clean up node a little bit:
...
- Remove redundant attribute creation function.
- Remove redundant len field.
- Add the rest of the missing types to the constructor array.
- Reorganize a bit.
2021-04-08 01:05:20 +00:00
Jennifer Taylor
6b547f72ec
Implement a decent chunk more bytecode, implement font tag.
2021-04-07 04:36:55 +00:00
Jennifer Taylor
2c35ca1cda
Implement unknown point parser to make DDR PS3 files parseable.
2021-04-07 00:45:08 +00:00
Jennifer Taylor
3109794607
Add in expanded property table that works with newer Pop'n/IIDX data.
2021-04-07 00:22:42 +00:00
Jennifer Taylor
689a997079
Support 1-byte integer PUSH from tricoro.
2021-04-06 23:01:42 +00:00
Jennifer Taylor
32cc4a071c
Update with working RE for all remaining known data pieces including properties and object events.
2021-04-06 22:24:36 +00:00
Jennifer Taylor
eda60c065c
Teach struct tool how to print line numbers with offsets, handy for decoding enums.
2021-04-06 22:24:16 +00:00
Jennifer Taylor
cddadd8114
Teach struct utility to take an --end or a --count for decoding values.
2021-04-06 22:24:00 +00:00
Jennifer Taylor
fa49513381
Fix ability to parse DDR PS3 AFP files, add a bit of info gleaned from looking at those files.
2021-04-06 05:46:53 +00:00
Jennifer Taylor
b118cd1109
Add a ton of RE for bytecodes, now capable of parsing several Bishi Bashi levels.
2021-04-06 04:56:59 +00:00
Jennifer Taylor
50b5ae8616
Fix a few more todos, slight renaming based on better understanding.
2021-04-05 16:28:34 +00:00
Jennifer Taylor
0bfec7655c
Last few object bits implemented enough to parse Pop'n Music, a few more annotations.
2021-04-05 00:52:10 +00:00
Jennifer Taylor
d26f3e0238
Teach struct how to deal with NULL pointers, better output of parsed structures.
2021-04-05 00:51:34 +00:00
Jennifer Taylor
c7e739c6e2
Support BINXML data with no property bodies.
2021-04-05 00:50:30 +00:00
Jennifer Taylor
5adb36d5b2
Fix a few comments in old XML parser.
2021-04-05 00:50:08 +00:00
Jennifer Taylor
887d4dc657
Add a mostly-complete parser for shapes, which are the missing link between object placement in AP2 files and textures/regions elsewhere.
2021-04-03 05:30:19 +00:00
Jennifer Taylor
98197641c9
Add an almost-complete object instantiation/updating parser.
2021-04-03 05:29:37 +00:00
Jennifer Taylor
c6a138c0ea
Add AP2 action table, codify understanding of frames, minor adjustments.
2021-04-03 05:28:57 +00:00
Jennifer Taylor
cdfb6cc21f
Allow loading/parsing AFP/BSI files extracted from IFS files.
2021-04-03 05:27:09 +00:00
Jennifer Taylor
4a3068686a
Fix help display and fix skipping padding bytes ('x' character).
2021-04-03 05:26:48 +00:00
Jennifer Taylor
bfa7384f2b
Update tag parsing with better guesses as to the unknown tag sections.
2021-04-03 05:26:25 +00:00
Jennifer Taylor
0e86248fd1
Support sprites which are basically recursive tag sections.
2021-04-03 05:26:03 +00:00
Jennifer Taylor
d688314d1f
Restructure string decoding, fix non-ascii strings, at least parse out the last remaining tag bytes we don't understand.
2021-04-03 05:25:27 +00:00
Jennifer Taylor
7beb518484
Fix a TODO, add a bit more RE.
2021-03-30 04:50:51 +00:00
Jennifer Taylor
02d2c9cd5a
Got a bit more of the AFP base format figured out. Now we can see shape definitions and imported/exported tags properly.
2021-03-30 04:50:05 +00:00
Jennifer Taylor
8e8bcbf822
Juggle AFP file a bit.
2021-03-30 04:49:37 +00:00
Jennifer Taylor
ec88cc866e
Fix lint errors that crept in.
2021-03-19 00:47:56 +00:00
Jennifer Taylor
152787f6d6
Fix error on startup caused by incorrect types and new SqlAlchemy.
2021-03-19 00:47:37 +00:00
Cammy
ccba50e2cf
Add event id notes, update Museca Plus stuff.
2021-02-06 22:23:55 -05:00
kami-poi
5e7061c23b
Fix logic error in eacoin.py
2021-01-24 12:38:22 -05:00
Jennifer Taylor
07a1d09cfa
Initial work to reverse engineering the AFP flash derivative.
2021-01-15 01:03:59 +00:00
Jennifer Taylor
69ab6723fe
Far more powerful struct printer that can handle complex structures with pointers.
2021-01-15 01:03:29 +00:00
Jennifer Taylor
6768b1417f
Better 1555/565/4444 color extraction that scales colors to full 8-bit range.
2020-12-21 23:41:58 +00:00
Jennifer Taylor
47330ca37d
Fix DXT1/DXT5 handling for DDR PS3 files.
2020-12-21 23:41:58 +00:00
seth
5522b80994
MUSECA plus: fix read script
2020-12-21 13:35:22 -05:00
Jennifer Taylor
e70b74f21a
Slight optimization of python lz77 code.
2020-12-20 04:06:41 +00:00
Jennifer Taylor
c431e30d3b
Remove a few redundant code paths in LZ77 C++ implementation, fix a bug preventing us from finding the optimal compression.
2020-12-20 03:38:34 +00:00
Jennifer Taylor
6ab6dc89c4
Support one more texture format for repacking.
2020-12-17 22:06:14 +00:00
Jennifer Taylor
68c1b7cc63
Encode unicode in JSX differently to attempt to fix issue with unicode translation on some non-utf8 systems.
2020-12-17 21:47:36 +00:00
Jennifer Taylor
b7bd09bb62
Remove magic strings and use constants instead.
2020-12-17 20:16:30 +00:00
seth
416904100b
API: enable omnimix for only MUSECA and IIDX
2020-12-17 14:57:20 -05:00
seth
d32c8fafa6
Code style
2020-12-17 14:57:20 -05:00
seth
349a34d25d
MUSECA PLUS: Webui fix
2020-12-17 14:57:20 -05:00
seth
13c0d02e45
MUSECA: finishing touches
2020-12-17 14:57:20 -05:00
seth
85622dbc9b
MUSECA 1+1/2: mission stuff
2020-12-17 14:57:20 -05:00
seth
f8f511250c
Fix linting errors
2020-12-17 14:57:20 -05:00
seth
1a4a231066
MUSECA: API and webui fixes for Museca PLUS
2020-12-17 14:57:20 -05:00
seth
82a67e0792
MUSECA: Support Museca Plus modpack
2020-12-17 14:57:20 -05:00
Jennifer Taylor
37e01e0a3e
Rewrite LZ77 compressor in C++ (like decompressor) for another massive speed boost.
2020-12-14 22:15:20 +00:00
Jennifer Taylor
fd3868073f
Add a few more known node types.
2020-11-30 22:57:46 +00:00
Jennifer Taylor
0ea53660b4
Support repacking a few more texture formats.
2020-11-30 18:16:02 +00:00
Jennifer Taylor
c6e395b8a5
Add ability to update textures from individual sprites as well as whole texture images.
2020-11-29 00:48:44 +00:00
Jennifer Taylor
0c4cb8df5c
Support splitting texturemaps into individual sprites.
2020-11-29 00:47:25 +00:00
Jennifer Taylor
31ad2856fd
Add a much faster C++ implementation of Lz77 decompress. We can achieve approximately a 10x speedup decompressing with this enabled.
2020-11-28 19:05:27 +00:00
Jennifer Taylor
c1fad0caeb
Skip tracking coverage when we aren't printing verbose debugging for a slight speedup.
2020-11-28 19:05:03 +00:00
Jennifer Taylor
d2d4f9d1e9
Fix bug where some textures weren't referenced properly by BishiBashi in updated AFP files.
2020-11-27 21:53:01 +00:00
Jennifer Taylor
d3d0bab2f5
Reorganize some debug prints to help figure out correct file writing order.
2020-11-27 21:52:30 +00:00
Jennifer Taylor
476c459272
Add debugging printer so we can verify contents identical before and after a null update.
2020-11-27 17:47:48 +00:00
Jennifer Taylor
27afc98ee5
Skip recompressing textures we don't change, don't try to write a file unless we successfully generated it.
2020-11-24 00:03:28 +00:00
Jennifer Taylor
526a462ef4
Overwrite original AFP file instead of making a temporary copy.
2020-11-23 22:33:12 +00:00
Jennifer Taylor
f6b90ea014
Add feature to enable overlay bounds generation for visualizing regions in textures.
2020-11-23 20:56:05 +00:00
Jennifer Taylor
8acdb1cefc
Update documentation for other texture formats that we semi-support.
2020-11-23 20:55:38 +00:00
Jennifer Taylor
ff147a37b1
Support an additional texture format found in updated BishiBashi JP data, document mapping to D3D9 structures.
2020-11-23 20:55:00 +00:00
Jennifer Taylor
db6a6739fc
Implement the first bits of what's necessary to update textures.
2020-11-12 05:03:52 +00:00
Jennifer Taylor
41ea293897
Faster, but less than ideal, large file compression as well as start byte GC.
2020-11-12 05:03:33 +00:00
Jennifer Taylor
88042a7e9e
Write back textures and regions when saving files.
2020-11-12 05:03:03 +00:00
Jennifer Taylor
42c3362a3c
Fix off-by-one error in LZ77 implementation rarely resulting in truncated files on inflate.
2020-11-12 05:02:37 +00:00
Jennifer Taylor
a5966a6382
Fix some lint and typing issues uncovered by mypy/flake8 upgrades.
2020-11-11 05:14:36 +00:00
Jennifer Taylor
ffa70e372b
Ability to write back out simple files with the data we parsed.
2020-11-11 03:39:51 +00:00
Jennifer Taylor
32d1909137
Support round-tripping a few more bits of various files, to increase our eventual success of injecting translated T*BB textures.
2020-11-11 03:39:21 +00:00
