Merge branch 'CrazyRedMachine:main' into main

omnimix/BM2DXFontScript.md

@@ -0,0 +1,35 @@
+# BM2DXFontScript format
+
+BM2DXFontScript is the internal string formatting engine of the game.
+
+Some of popnhax options use it as well.
+
+It works a bit like printf so %s is our placeholder for the actual song/artist name (and since the function call is what it is, we cannot add new variables).
+
+The different ways to alter texts are the following:
+```
+ [f:]   [/f]      anything other than 0 seems to break SJIS characters (maybe something to switch to another codepage for titles like Τέλος ?)
+ [b:]   [/b]      Boldness
+ [pos:] [/pos]    Position (seems to add a margin both left and right)
+ [p:]   [/p]      Padding (space between letters)
+*[c:]   [/c]      Color (rgb)
+ [ol:]  [/ol]     OutLine
+*[olc:] [/olc]    OutLine Color (rgb)
+ [ds:]  [/ds]     Drop Shadow (adds a copy of same text behind, on bottom right)
+*[dsc:] [/dsc]    Drop Shadow Color (rgb)
+ [sx:]  [/sx]     Size x (text width) 0-100
+ [sy:]  [/sy]     Size y (text height) 0-100
+ [rz:]  [/rz]     Rotation along z axis (italic/slanted text)
+ [sz:]  [/sz]     Size (looks much bigger than sx and sy)
+ [a:]   [/a]      Unknown (alpha?)
+ [r:]   [/r]      Unknown
+ [br:]            Unknown (and the closing bracket [/br] doesn't exist.. line break? brightness?)
+```
+
+`*` indicates the value is rgb as hex (eg. [c:ff0000] for bright red)
+
+other tags all take a single numerical value to set effect intensity
+
+Unknown tags have no effect in the context of the songlist.. 
+
+TODO: Fuzz another part of the code which uses the format and see the effect

omnimix/README.md

@@ -4,6 +4,21 @@ Refer to [omnimix_db.md](omnimix_db.md)
 
 # Omnimix tools
 
+## fix_backgrounds.py
+
+Fixes musicdb xmls for correct background display in unilab+.
+
+(Unilab has changed the mask bitfield to add a "force load background" entry. This script will check all songs with a bg_xxxx.ifs file and update the xmls accordingly)
+
+Usage:
+1. Copy fix_backgrounds.py in data_mods folder
+2. Run the script
+3. Verify the backgrounds are working as expected
+
+**Note**: All modified XMLs are first copied to `data_mods\_fix_backgrounds_backup`. If you encounter any issue, copy this folder contents back into data_mods and overwrite when prompted.
+
+This will restore the initial state.
+
 ## ida_find_addrs.py
 
 IDA script tested in 6.6 and 7.x.
@@ -73,4 +88,4 @@ Example: `python3 verify_data.py --input-dll popn22.dll --input-xml db/patches_2
 - ~~As of time of writing, the latest version of ifstools (1.14) will not extract jacket.ifs properly on Windows due to NTFS's case-insensitivity, resulting in 3 images being overwritten with data that won't work in-game. You can extract on a *nix system to get the correct jacket images if you see a green block in place of the jackets for the affected songs.~~
   - Not pushed out to pypi yet, but this has already been fixed in master and will be included in the next release of ifstools where you can use the `--rename-dupes` flag (thanks mon!).
 
-- Character database editing is slightly restrictive at the moment due to not being able to add new entries to the flavor table. When trying to add new entries to the flavor table there is a high chance of the game crashing in my experience. My guess is that there are some more places that should be patched that I have not found yet. This is a technical issue that could be solved with more work but it's more of a stretch goal than a main goal for this project so I put in a bandaid to make sure that the flavor table never expands. This issue is also why some unlocked characters will turn into Nyami alts.
+- Character database editing is slightly restrictive at the moment due to not being able to add new entries to the flavor table. When trying to add new entries to the flavor table there is a high chance of the game crashing in my experience. My guess is that there are some more places that should be patched that I have not found yet. This is a technical issue that could be solved with more work but it's more of a stretch goal than a main goal for this project so I put in a bandaid to make sure that the flavor table never expands. This issue is also why some unlocked characters will turn into Nyami alts.

omnimix/fix_backgrounds.py

@@ -0,0 +1,89 @@
+import glob
+import os
+from os import walk
+from os import listdir
+from os.path import isdir, join
+import shutil
+import re
+
+
+def handle_mask(folder,songids):
+    for musicdb_filepath in glob.glob(os.path.join(folder, "*musicdb*.xml")):
+        has_changes = False
+        print("    processing",musicdb_filepath)
+        fixedfile = ""
+        musicdb_filehandle = open(musicdb_filepath,encoding='cp932')
+        content = musicdb_filehandle.read()
+        result = re.search("<music id=.*?</mask",content, re.DOTALL)
+        offset = 0
+        while(result is not None):
+            #songid
+            prefixlen = len("<music id\"")
+            songid_end = content[(offset+result.start()+prefixlen+1):(offset+result.start()+prefixlen+20)].index('"')+1
+            extract_songid = content[(offset+result.start()+prefixlen+1):(offset+result.start()+prefixlen+songid_end)]
+            if extract_songid not in songids:
+                #print match as is
+                fixedfile += content[(offset):(offset+result.end()+1)]
+                offset += result.end()+1
+                result = re.search("<music id=.*?</mask",content[offset:], re.DOTALL)
+                continue
+            print("        processing songid",extract_songid)
+            #mask
+            prefixlen = len("<mask __type=\"u32\">")
+            mask_begin = content[(offset+result.start()):(offset+result.end())].index("<mask __type=\"u32\">")
+            mask_end = content[(offset+result.start()+mask_begin+prefixlen+1):(offset+result.start()+mask_begin+prefixlen+50)].index('<')+1
+            extract_mask = int(content[(offset+result.start()+mask_begin+prefixlen):(offset+result.start()+mask_begin+prefixlen+mask_end)])
+            fixed_mask = extract_mask|0x100
+            if extract_mask != fixed_mask:
+                print("            fixing mask",extract_mask,"->",fixed_mask)
+                newcontent = content[(offset+result.start()+mask_begin+prefixlen):(offset+result.start()+mask_begin+prefixlen+mask_end)].replace(str(extract_mask), str(fixed_mask))
+                has_changes = True
+                #print match with updated flag
+                fixedfile += content[(offset):(offset+result.start()+mask_begin+prefixlen)]
+                fixedfile += newcontent
+                fixedfile += content[(offset+result.start()+mask_begin+prefixlen+mask_end):(offset+result.end()+1)]
+            else:
+                print("            mask already includes the load_background flag")
+                #print match as is
+                fixedfile += content[(offset):(offset+result.end()+1)]
+            offset += result.end()+1
+            result = re.search("<music id=.*?</mask",content[offset:], re.DOTALL)
+        #print rest of file
+        fixedfile += content[offset:]
+        musicdb_filehandle.close()
+        if has_changes:
+            print("    Commit changes to file")
+            #first make a backup
+            newpath = os.path.join("_fix_backgrounds_backup",musicdb_filepath)
+            if not os.path.exists(newpath):
+                os.makedirs(os.path.join("_fix_backgrounds_backup",folder),exist_ok=True)
+                shutil.copyfile(musicdb_filepath, newpath)
+            else:
+                print("ERROR: a backup already exists for this file, aborting")
+                exit(0)
+            with open(musicdb_filepath, "w",encoding='cp932',newline='\n') as new_file:
+                new_file.write(fixedfile)
+
+bg_diff_path = os.path.join("tex","system","bg_diff_ifs")
+
+folders = []
+
+if os.path.exists(os.path.join("_fix_backgrounds_backup")):
+    print("WARNING: a backup path already exists (maybe you run the script already?)")
+    input("Press Enter to continue anyways, or Ctrl+C to abort...")
+
+for (dirpath, dirnames, filenames) in walk("."):
+    folders.extend(dirnames)
+    break
+
+for folder in folders:
+    print("Processing",folder,"...")
+    prefix_len = len(os.path.join(folder, bg_diff_path))+len("/bg_")
+    list = glob.glob(os.path.join(folder, bg_diff_path, "*.ifs"))
+    songids = []
+    for item in list:
+        songid = item[prefix_len:-4]
+        songids.append(songid)
+    handle_mask(folder,songids)
+    print("Done.")
+