diff --git a/README.txt b/README.txt index 0ba188e..6059124 100755 --- a/README.txt +++ b/README.txt @@ -1,178 +1,178 @@ -README.txt - -By Ross Ridge -Pubic Domain - -@(#) mymc README.txt 1.6 12/10/04 19:18:08 - - -This file describes mymc, a utility for manipulating PlayStation 2 -memory card images as used by the emulator PCSX2. Its main purpose is -to allow save games to be imported and exported to and from these -images. Both MAX Drive and EMS (.psu) save files are fully supported, -however save files in the SharkPort/X-Port and Code Breaker formats -can only be imported and not exported. In addition to these basic -functions, mymc can also perform a number of other operations, like -creating new memory card images, viewing their contents, and adding -and extracting individual files. - -A simple, hopefully easy to use, graphicial user interface (GUI) is -provided, but it's limitted to only basic operations. More advanced -opterations require the use of a command line tool. To install mymc, -unpack the downloaded ZIP archive to a new directory on your machine. -You can then run the GUI version of mymc by openning that newn -directory with Windows Explorer and double clicking on the "mymc-gui" -icon. To make it easier to access, you can drag the "mymc-gui" icon -to either your Desktop, Start Menu or Quick Launch toolbar. Make sure -if you do so, that you create a shortcut to "mymc-gui.exe". If you -copy the file instead, the program won't work. - -The command line utility can be invoked from the Windows Command -Prompt by using the "mymc" command. The executable "mymc.exe" and -number of support files and these file must kept together in the same -directory. To run the command you need to either add the directory -where you unpacked the distribution to your PATH or type the full -pathname of the executable. For example if you unpacked mymc to a -directory named "c:\mymc" you need to enter "c:\mymc\mymc.exe" to run -the program. - -The second important thing to note is that mymc is only "alpha" -quality software. This means that has is been released without -extensive testing and may be unreliable. While it works fine for me, -the author, it might not work as well for you. For that reason you -should be careful how you use it, and prepared for the eventuality of -it corrupting your save game images or producing garbage save files. -If you're worried about this, one make things safer is to use two -memory card images. Use the first image to load and save your games -with under PCSX2, and the second image to import and export saves -games using mysc. Then use the PS2 browser to copy files between two -card images. - - -GUI TUTORIAL -============ - -The GUI for mymc is should be easy to use. After starting mymc, you -can select the PS2 memory card image you want to work with by -selecting the "Open" command by pressing the first button on the -toolbar. You can then import a save file clicking on the Import -toolbar button. To export a save files, first select it and then -press the Export button. You can delete a save file permanently from -your memory card, by selecting the "Delete" command from the File -menu. - -Do not try to use mymc to modify a memory card image while PCSX2 is -running. Doing so will corrupt your memory card. - - -COMMAND LINE TUTORIAL -===================== - -The basic usage template for mysc is "mymc memcard.ps2 command". The -first argument, "memcard.ps2" is the filename of the memory card image -while "command" is the name of the command you wish to use on the -image. So for example, assuming you've installed mymc in "c:\mymc" -and you've installed PCSX2 in "c:\pcsx2" you could enter the following -command to see the contents of the memory card in the emulator's slot -1: - - c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 dir - -You would see output something like this: - - BASLUS-20678USAGAS00 UNLIMITED SAGA - 154KB Not Protected SYSTEMDATA - - BADATA-SYSTEM Your System - 5KB Not Protected Configuration - - BASLUS-20488-0000D SOTET<13>060:08 - 173KB Not Protected Arias - - 7,800 KB Free - -This is the simple "user friendly" way to view the contents of a -memory card. It displays the same information you can see using the -PlayStation 2 memory card browser. On the right is name of each save, -and on the left is the size and protection status of the save. Also -on the left is one bit of information you won't see in the browser, -the directory name of the save file. PlayStation 2 saves are actually -a collection of different files all stored in a single directory on -the memory card. This is important information, because you need to -know it to export save files. - -As mentioned above, if you know the directory name of a save, you can -export it. Exporting a save creates a save file in either the EMS -(.psu) or MAX Drive (.max) format. You can then transfer the save to -real PS2 memory using the appropriate tools. You can also send the -saves to someone else to use or just keep them on your hard drive as a -backup. The following command demonstrates how to export a save in -the EMS format using mymc: - - c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 export BASLUS-20448-0000D - -This will create a file called "BASLUS-20448-0000D.psu" in the current -directory. To create a file in the MAX format instead, use the export -command's -m option: - - c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 export -m BASLUS-20448-0000D - -This creates a file named "BASLUS-20448-0000D.max". Note the "-m" -option that appears after the "export" command. - -Importing save files is similar. The save file type is auto-detected, -so you don't need use an "-m" option with MAX Drive saves. Here's a -couple of examples using each format: - - c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 import BASLUS-20035.psu - c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 import 20062_3583_GTA3.max - - -ADVANCED NOTES -============== - - - To get general help with the command line utility use the "-h" - global option (eg. "mymc -h"). To get help with a specific - command use the "-h" option with that command (eg. "mymc x - import -h"). In this later case, you need to specify a memory - card image file, but it's ignored and so doesn't need to exist. - - - Both executables in the Windows version, "mymc.exe" and - "mymc-gui.exe" do the same thing and support the same options. - The difference is that "mymc" is console application, while - "mymc-gui" is a Windows appliction. Currently, using "mymc" - to start the GUI will result in a fair amount debug messages - being printed that are normally not seen "mymc-gui" is used. - - - It's possible to use mymc create images that are bigger (or - smaller) than standard PS2 memory cards. Be very careful if you - do this, not all games may be compatible with such images. - - - The bad block list on images is ignored. Since memory card - images created with either PCSX2 or mymc won't have any bad - blocks, this shouldn't be a problem unless you've somehow - extracted a complete image from a real memory card and expect to - copy it back. - - - The PS2 only uses at most 8,000 KB of a memory card, but there - is actually 8,135 KB of allocatable space on a standard - error-free memory card. The extra 135 KB is reserved so that - memory card with bad blocks don't appear to have less space than - memory cards with fewer or no bad blocks. Since there are no - bad blocks on memory card images, mymc uses the full capacity - provided by standard memory cards. - - -PYTHON SOURCE DISTRIBUTION -========================== - -The "source code" distribution of mymc is provided for users of Linux -and other non-Windows operating systems. It uses the same Python code -that the Windows distribution is built with (using py2exe) and -supports all the same functionality. One big difference is that the -Windows DLL "mymcsup.dll" is not included and as a result compressing -and decompressing MAX Drive saves will be as much as 100 times slower. -The GUI mode is hasn't been extensively tested on non-Windows systems, -and the 3D display of save file icons requires the DLL. The Python -source version should support big-endian machines, but this hasn't -been tested. +README.txt + +By Ross Ridge +Pubic Domain + +@(#) mymc README.txt 1.6 12/10/04 19:18:08 + + +This file describes mymc, a utility for manipulating PlayStation 2 +memory card images as used by the emulator PCSX2. Its main purpose is +to allow save games to be imported and exported to and from these +images. Both MAX Drive and EMS (.psu) save files are fully supported, +however save files in the SharkPort/X-Port and Code Breaker formats +can only be imported and not exported. In addition to these basic +functions, mymc can also perform a number of other operations, like +creating new memory card images, viewing their contents, and adding +and extracting individual files. + +A simple, hopefully easy to use, graphicial user interface (GUI) is +provided, but it's limitted to only basic operations. More advanced +opterations require the use of a command line tool. To install mymc, +unpack the downloaded ZIP archive to a new directory on your machine. +You can then run the GUI version of mymc by openning that newn +directory with Windows Explorer and double clicking on the "mymc-gui" +icon. To make it easier to access, you can drag the "mymc-gui" icon +to either your Desktop, Start Menu or Quick Launch toolbar. Make sure +if you do so, that you create a shortcut to "mymc-gui.exe". If you +copy the file instead, the program won't work. + +The command line utility can be invoked from the Windows Command +Prompt by using the "mymc" command. The executable "mymc.exe" and +number of support files and these file must kept together in the same +directory. To run the command you need to either add the directory +where you unpacked the distribution to your PATH or type the full +pathname of the executable. For example if you unpacked mymc to a +directory named "c:\mymc" you need to enter "c:\mymc\mymc.exe" to run +the program. + +The second important thing to note is that mymc is only "alpha" +quality software. This means that has is been released without +extensive testing and may be unreliable. While it works fine for me, +the author, it might not work as well for you. For that reason you +should be careful how you use it, and prepared for the eventuality of +it corrupting your save game images or producing garbage save files. +If you're worried about this, one make things safer is to use two +memory card images. Use the first image to load and save your games +with under PCSX2, and the second image to import and export saves +games using mysc. Then use the PS2 browser to copy files between two +card images. + + +GUI TUTORIAL +============ + +The GUI for mymc is should be easy to use. After starting mymc, you +can select the PS2 memory card image you want to work with by +selecting the "Open" command by pressing the first button on the +toolbar. You can then import a save file clicking on the Import +toolbar button. To export a save files, first select it and then +press the Export button. You can delete a save file permanently from +your memory card, by selecting the "Delete" command from the File +menu. + +Do not try to use mymc to modify a memory card image while PCSX2 is +running. Doing so will corrupt your memory card. + + +COMMAND LINE TUTORIAL +===================== + +The basic usage template for mysc is "mymc memcard.ps2 command". The +first argument, "memcard.ps2" is the filename of the memory card image +while "command" is the name of the command you wish to use on the +image. So for example, assuming you've installed mymc in "c:\mymc" +and you've installed PCSX2 in "c:\pcsx2" you could enter the following +command to see the contents of the memory card in the emulator's slot +1: + + c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 dir + +You would see output something like this: + + BASLUS-20678USAGAS00 UNLIMITED SAGA + 154KB Not Protected SYSTEMDATA + + BADATA-SYSTEM Your System + 5KB Not Protected Configuration + + BASLUS-20488-0000D SOTET<13>060:08 + 173KB Not Protected Arias + + 7,800 KB Free + +This is the simple "user friendly" way to view the contents of a +memory card. It displays the same information you can see using the +PlayStation 2 memory card browser. On the right is name of each save, +and on the left is the size and protection status of the save. Also +on the left is one bit of information you won't see in the browser, +the directory name of the save file. PlayStation 2 saves are actually +a collection of different files all stored in a single directory on +the memory card. This is important information, because you need to +know it to export save files. + +As mentioned above, if you know the directory name of a save, you can +export it. Exporting a save creates a save file in either the EMS +(.psu) or MAX Drive (.max) format. You can then transfer the save to +real PS2 memory using the appropriate tools. You can also send the +saves to someone else to use or just keep them on your hard drive as a +backup. The following command demonstrates how to export a save in +the EMS format using mymc: + + c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 export BASLUS-20448-0000D + +This will create a file called "BASLUS-20448-0000D.psu" in the current +directory. To create a file in the MAX format instead, use the export +command's -m option: + + c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 export -m BASLUS-20448-0000D + +This creates a file named "BASLUS-20448-0000D.max". Note the "-m" +option that appears after the "export" command. + +Importing save files is similar. The save file type is auto-detected, +so you don't need use an "-m" option with MAX Drive saves. Here's a +couple of examples using each format: + + c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 import BASLUS-20035.psu + c:\mymc\mymc c:\pcsx2\memcards\Mcd001.ps2 import 20062_3583_GTA3.max + + +ADVANCED NOTES +============== + + - To get general help with the command line utility use the "-h" + global option (eg. "mymc -h"). To get help with a specific + command use the "-h" option with that command (eg. "mymc x + import -h"). In this later case, you need to specify a memory + card image file, but it's ignored and so doesn't need to exist. + + - Both executables in the Windows version, "mymc.exe" and + "mymc-gui.exe" do the same thing and support the same options. + The difference is that "mymc" is console application, while + "mymc-gui" is a Windows appliction. Currently, using "mymc" + to start the GUI will result in a fair amount debug messages + being printed that are normally not seen "mymc-gui" is used. + + - It's possible to use mymc create images that are bigger (or + smaller) than standard PS2 memory cards. Be very careful if you + do this, not all games may be compatible with such images. + + - The bad block list on images is ignored. Since memory card + images created with either PCSX2 or mymc won't have any bad + blocks, this shouldn't be a problem unless you've somehow + extracted a complete image from a real memory card and expect to + copy it back. + + - The PS2 only uses at most 8,000 KB of a memory card, but there + is actually 8,135 KB of allocatable space on a standard + error-free memory card. The extra 135 KB is reserved so that + memory card with bad blocks don't appear to have less space than + memory cards with fewer or no bad blocks. Since there are no + bad blocks on memory card images, mymc uses the full capacity + provided by standard memory cards. + + +PYTHON SOURCE DISTRIBUTION +========================== + +The "source code" distribution of mymc is provided for users of Linux +and other non-Windows operating systems. It uses the same Python code +that the Windows distribution is built with (using py2exe) and +supports all the same functionality. One big difference is that the +Windows DLL "mymcsup.dll" is not included and as a result compressing +and decompressing MAX Drive saves will be as much as 100 times slower. +The GUI mode is hasn't been extensively tested on non-Windows systems, +and the 3D display of save file icons requires the DLL. The Python +source version should support big-endian machines, but this hasn't +been tested. diff --git a/gui.py b/gui.py index b5d2220..d14324e 100755 --- a/gui.py +++ b/gui.py @@ -1,953 +1,963 @@ -# -# gui.py -# -# By Ross Ridge -# Public Domain -# - -"""Graphical user-interface for mymc.""" - -_SCCS_ID = "@(#) mymc gui.py 1.4 12/10/04 18:51:51\n" - -import os -import sys -import struct -import cStringIO -import time - -# Work around a problem with mixing wx and py2exe -if os.name == "nt" and hasattr(sys, "setdefaultencoding"): - sys.setdefaultencoding("mbcs") -import wx - -import ps2mc -import ps2save -import guires - -try: - import ctypes - import mymcsup - D3DXVECTOR3 = mymcsup.D3DXVECTOR3 - D3DXVECTOR4 = mymcsup.D3DXVECTOR4 - D3DXVECTOR4_ARRAY3 = mymcsup.D3DXVECTOR4_ARRAY3 - - def mkvec4arr3(l): - return D3DXVECTOR4_ARRAY3(*[D3DXVECTOR4(*vec) - for vec in l]) -except ImportError: - mymcsup = None - -lighting_none = {"lighting": False, - "vertex_diffuse": False, - "alt_lighting": False, - "light_dirs": [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]], - "light_colours": [[0, 0, 0, 0], [0, 0, 0, 0], - [0, 0, 0, 0]], - "ambient": [0, 0, 0, 0]} - -lighting_diffuse = {"lighting": False, - "vertex_diffuse": True, - "alt_lighting": False, - "light_dirs": [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]], - "light_colours": [[0, 0, 0, 0], [0, 0, 0, 0], - [0, 0, 0, 0]], - "ambient": [0, 0, 0, 0]} - -lighting_icon = {"lighting": True, - "vertex_diffuse": True, - "alt_lighting": False, - "light_dirs": [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]], - "light_colours": [[0, 0, 0, 0], [0, 0, 0, 0], - [0, 0, 0, 0]], - "ambient": [0, 0, 0, 0]} - -lighting_alternate = {"lighting": True, - "vertex_diffuse": True, - "alt_lighting": True, - "light_dirs": [[1, -1, 2, 0], - [-1, 1, -2, 0], - [0, 1, 0, 0]], - "light_colours": [[1, 1, 1, 1], - [1, 1, 1, 1], - [0.7, 0.7, 0.7, 1]], - "ambient": [0.5, 0.5, 0.5, 1]} - -lighting_alternate2 = {"lighting": True, - "vertex_diffuse": False, - "alt_lighting": True, - "light_dirs": [[1, -1, 2, 0], - [-1, 1, -2, 0], - [0, 4, 1, 0]], - "light_colours": [[0.7, 0.7, 0.7, 1], - [0.7, 0.7, 0.7, 1], - [0.2, 0.2, 0.2, 1]], - "ambient": [0.3, 0.3, 0.3, 1]} - -camera_default = [0, 4, -8] -camera_high = [0, 7, -6] -camera_near = [0, 3, -6] -camera_flat = [0, 2, -7.5] - -def get_dialog_units(win): - return win.ConvertDialogPointToPixels((1, 1))[0] - -def single_title(title): - """Convert the two parts of an icon.sys title into one string.""" - - title = title[0] + " " + title[1] - return u" ".join(title.split()) - -def _get_icon_resource_as_images(name): - ico = guires.resources[name] - images = [] - f = cStringIO.StringIO(ico) - count = struct.unpack("= size[0] and sz[1] >= size[1]: - if ((best_size[0] < size[0] or best_size[1] < size[1]) - or sz[0] * sz[1] < best_size[0] * best_size[1]): - best = img - best_size = sz - elif sz[0] * sz[1] > best_size[0] * best_size[1]: - best = img - best_size = sz - img = best.Rescale(size[0], size[1], wx.IMAGE_QUALITY_HIGH) - return wx.BitmapFromImage(img) - - -class dirlist_control(wx.ListCtrl): - """Lists all the save files in a memory card image.""" - - def __init__(self, parent, evt_focus, evt_select, config): - self.config = config - self.selected = set() - self.evt_select = evt_select - wx.ListCtrl.__init__(self, parent, wx.ID_ANY, - style = wx.LC_REPORT) - wx.EVT_LIST_COL_CLICK(self, -1, self.evt_col_click) - wx.EVT_LIST_ITEM_FOCUSED(self, -1, evt_focus) - wx.EVT_LIST_ITEM_SELECTED(self, -1, self.evt_item_selected) - wx.EVT_LIST_ITEM_DESELECTED(self, -1, self.evt_item_deselected) - - def _update_dirtable(self, mc, dir): - self.dirtable = table = [] - enc = "unicode" - if self.config.get_ascii(): - enc = "ascii" - for ent in dir: - if not ps2mc.mode_is_dir(ent[0]): - continue - dirname = "/" + ent[8] - s = mc.get_icon_sys(dirname) - if s == None: - continue - a = ps2save.unpack_icon_sys(s) - size = mc.dir_size(dirname) - title = ps2save.icon_sys_title(a, encoding = enc) - table.append((ent, s, size, title)) - - def update_dirtable(self, mc): - self.dirtable = [] - if mc == None: - return - dir = mc.dir_open("/") - try: - self._update_dirtable(mc, dir) - finally: - dir.close() - - def cmp_dir_name(self, i1, i2): - return self.dirtable[i1][0][8] > self.dirtable[i2][0][8] - - def cmp_dir_title(self, i1, i2): - return self.dirtable[i1][3] > self.dirtable[i2][3] - - def cmp_dir_size(self, i1, i2): - return self.dirtable[i1][2] > self.dirtable[i2][2] - - def cmp_dir_modified(self, i1, i2): - m1 = list(self.dirtable[i1][0][6]) - m2 = list(self.dirtable[i2][0][6]) - m1.reverse() - m2.reverse() - return m1 > m2 - - def evt_col_click(self, event): - col = event.m_col - if col == 0: - cmp = self.cmp_dir_name - elif col == 1: - cmp = self.cmp_dir_size - elif col == 2: - cmp = self.cmp_dir_modified - elif col == 3: - cmp = self.cmp_dir_title - self.SortItems(cmp) - return - - def evt_item_selected(self, event): - self.selected.add(event.GetData()) - self.evt_select(event) - - def evt_item_deselected(self, event): - self.selected.discard(event.GetData()) - self.evt_select(event) - - def update(self, mc): - """Update the ListCtrl according to the contents of the - memory card image.""" - - self.ClearAll() - self.selected = set() - self.InsertColumn(0, "Directory") - self.InsertColumn(1, "Size") - self.InsertColumn(2, "Modified") - self.InsertColumn(3, "Description") - li = self.GetColumn(1) - li.SetAlign(wx.LIST_FORMAT_RIGHT) - li.SetText("Size") - self.SetColumn(1, li) - - self.update_dirtable(mc) - - empty = len(self.dirtable) == 0 - self.Enable(not empty) - if empty: - return - - for (i, a) in enumerate(self.dirtable): - (ent, icon_sys, size, title) = a - li = self.InsertStringItem(i, ent[8]) - self.SetStringItem(li, 1, "%dK" % (size / 1024)) - m = ent[6] - m = ("%04d-%02d-%02d %02d:%02d" - % (m[5], m[4], m[3], m[2], m[1])) - self.SetStringItem(li, 2, m) - self.SetStringItem(li, 3, single_title(title)) - self.SetItemData(li, i) - - du = get_dialog_units(self) - for i in range(4): - self.SetColumnWidth(i, wx.LIST_AUTOSIZE) - self.SetColumnWidth(i, self.GetColumnWidth(i) + du) - self.SortItems(self.cmp_dir_name) - - -class icon_window(wx.Window): - """Displays a save file's 3D icon. Windows only. - - The rendering of the 3D icon is handled by C++ code in the - mymcsup DLL which subclasses this window. This class mainly - handles configuration options that affect how the 3D icon is - displayed. - """ - - ID_CMD_ANIMATE = 201 - ID_CMD_LIGHT_NONE = 202 - ID_CMD_LIGHT_ICON = 203 - ID_CMD_LIGHT_ALT1 = 204 - ID_CMD_LIGHT_ALT2 = 205 - ID_CMD_CAMERA_FLAT = 206 - ID_CMD_CAMERA_DEFAULT = 207 - ID_CMD_CAMERA_NEAR = 209 - ID_CMD_CAMERA_HIGH = 210 - - light_options = {ID_CMD_LIGHT_NONE: lighting_none, - ID_CMD_LIGHT_ICON: lighting_icon, - ID_CMD_LIGHT_ALT1: lighting_alternate, - ID_CMD_LIGHT_ALT2: lighting_alternate2} - - camera_options = {ID_CMD_CAMERA_FLAT: camera_flat, - ID_CMD_CAMERA_DEFAULT: camera_default, - ID_CMD_CAMERA_NEAR: camera_near, - ID_CMD_CAMERA_HIGH: camera_high} - - def append_menu_options(self, win, menu): - menu.AppendCheckItem(icon_window.ID_CMD_ANIMATE, - "Animate Icons") - menu.AppendSeparator() - menu.AppendRadioItem(icon_window.ID_CMD_LIGHT_NONE, - "Lighting Off") - menu.AppendRadioItem(icon_window.ID_CMD_LIGHT_ICON, - "Icon Lighting") - menu.AppendRadioItem(icon_window.ID_CMD_LIGHT_ALT1, - "Alternate Lighting") - menu.AppendRadioItem(icon_window.ID_CMD_LIGHT_ALT2, - "Alternate Lighting 2") - menu.AppendSeparator() - menu.AppendRadioItem(icon_window.ID_CMD_CAMERA_FLAT, - "Camera Flat") - menu.AppendRadioItem(icon_window.ID_CMD_CAMERA_DEFAULT, - "Camera Default") - menu.AppendRadioItem(icon_window.ID_CMD_CAMERA_NEAR, - "Camera Near") - menu.AppendRadioItem(icon_window.ID_CMD_CAMERA_HIGH, - "Camera High") - - wx.EVT_MENU(win, icon_window.ID_CMD_ANIMATE, - self.evt_menu_animate) - wx.EVT_MENU(win, icon_window.ID_CMD_LIGHT_NONE, - self.evt_menu_light) - wx.EVT_MENU(win, icon_window.ID_CMD_LIGHT_ICON, - self.evt_menu_light) - wx.EVT_MENU(win, icon_window.ID_CMD_LIGHT_ALT1, - self.evt_menu_light) - wx.EVT_MENU(win, icon_window.ID_CMD_LIGHT_ALT2, - self.evt_menu_light) - - wx.EVT_MENU(win, icon_window.ID_CMD_CAMERA_FLAT, - self.evt_menu_camera) - wx.EVT_MENU(win, icon_window.ID_CMD_CAMERA_DEFAULT, - self.evt_menu_camera) - wx.EVT_MENU(win, icon_window.ID_CMD_CAMERA_NEAR, - self.evt_menu_camera) - wx.EVT_MENU(win, icon_window.ID_CMD_CAMERA_HIGH, - self.evt_menu_camera) - - def __init__(self, parent, focus): - self.failed = False - wx.Window.__init__(self, parent) - if mymcsup == None: - self.failed = True - return - r = mymcsup.init_icon_renderer(focus.GetHandle(), - self.GetHandle()) - if r == -1: - print "init_icon_renderer failed" - self.failed = True - return - - self.config = config = mymcsup.icon_config() - config.animate = True - - self.menu = wx.Menu() - self.append_menu_options(self, self.menu) - self.set_lighting(self.ID_CMD_LIGHT_ALT2) - self.set_camera(self.ID_CMD_CAMERA_DEFAULT) - - wx.EVT_CONTEXT_MENU(self, self.evt_context_menu) - - def __del__(self): - if mymcsup != None: - mymcsup.delete_icon_renderer() - - def update_menu(self, menu): - """Update the content menu according to the current config.""" - - menu.Check(icon_window.ID_CMD_ANIMATE, self.config.animate) - menu.Check(self.lighting_id, True) - menu.Check(self.camera_id, True) - - def load_icon(self, icon_sys, icon): - """Pass the raw icon data to the support DLL for display.""" - - if self.failed: - return - - if icon_sys == None or icon == None: - r = mymcsup.load_icon(None, 0, None, 0) - else: - r = mymcsup.load_icon(icon_sys, len(icon_sys), - icon, len(icon)) - if r != 0: - print "load_icon", r - self.failed = True - - def _set_lighting(self, lighting, vertex_diffuse, alt_lighting, - light_dirs, light_colours, ambient): - if self.failed: - return - config = self.config - config.lighting = lighting - config.vertex_diffuse = vertex_diffuse - config.alt_lighting = alt_lighting - config.light_dirs = mkvec4arr3(light_dirs) - config.light_colours = mkvec4arr3(light_colours) - config.ambient = D3DXVECTOR4(*ambient) - if mymcsup.set_config(config) == -1: - self.failed = True - - def set_lighting(self, id): - self.lighting_id = id - self._set_lighting(**self.light_options[id]) - - def set_animate(self, animate): - if self.failed: - return - self.config.animate = animate - if mymcsup.set_config(self.config) == -1: - self.failed = True - - def _set_camera(self, camera): - if self.failed: - return - self.config.camera = mymcsup.D3DXVECTOR3(*camera) - if mymcsup.set_config(self.config) == -1: - self.failed = True - - def set_camera(self, id): - self.camera_id = id - self._set_camera(self.camera_options[id]) - - def evt_context_menu(self, event): - self.update_menu(self.menu) - self.PopupMenu(self.menu) - - def evt_menu_animate(self, event): - self.set_animate(not self.config.animate) - - def evt_menu_light(self, event): - self.set_lighting(event.GetId()) - - def evt_menu_camera(self, event): - self.set_camera(event.GetId()) - -class gui_config(wx.Config): - """A class for holding the persistant configuration state.""" - - memcard_dir = "Memory Card Directory" - savefile_dir = "Save File Directory" - ascii = "ASCII Descriptions" - - def __init__(self): - wx.Config.__init__(self, "mymc", "Ross Ridge", - style = wx.CONFIG_USE_LOCAL_FILE) - - def get_memcard_dir(self, default = None): - return self.Read(gui_config.memcard_dir, default) - - def set_memcard_dir(self, value): - return self.Write(gui_config.memcard_dir, value) - - def get_savefile_dir(self, default = None): - return self.Read(gui_config.savefile_dir, default) - - def set_savefile_dir(self, value): - return self.Write(gui_config.savefile_dir, value) - - def get_ascii(self, default = False): - return bool(self.ReadInt(gui_config.ascii, int(bool(default)))) - - def set_ascii(self, value): - return self.WriteInt(gui_config.ascii, int(bool(value))) - -def add_tool(toolbar, id, label, ico): - tbsize = toolbar.GetToolBitmapSize() - bmp = get_icon_resource_bmp(ico, tbsize) - return toolbar.AddLabelTool(id, label, bmp, shortHelp = label) - -class gui_frame(wx.Frame): - """The main top level window.""" - - ID_CMD_EXIT = wx.ID_EXIT - ID_CMD_OPEN = wx.ID_OPEN - ID_CMD_EXPORT = 103 - ID_CMD_IMPORT = 104 - ID_CMD_DELETE = wx.ID_DELETE - ID_CMD_ASCII = 106 - - def message_box(self, message, caption = "mymc", style = wx.OK, - x = -1, y = -1): - return wx.MessageBox(message, caption, style, self, x, y) - - def error_box(self, msg): - return self.message_box(msg, "Error", wx.OK | wx.ICON_ERROR) - - def mc_error(self, value, filename = None): - """Display a message box for EnvironmentError exeception.""" - - if filename == None: - filename = getattr(value, "filename") - if filename == None: - filename = self.mcname - if filename == None: - filename = "???" - - strerror = getattr(value, "strerror", None) - if strerror == None: - strerror = "unknown error" - - return self.error_box(filename + ": " + strerror) - - def __init__(self, parent, title, mcname = None): - self.f = None - self.mc = None - self.mcname = None - self.icon_win = None - - size = (750, 350) - if mymcsup == None: - size = (500, 350) - wx.Frame.__init__(self, parent, wx.ID_ANY, title, size = size) - - wx.EVT_CLOSE(self, self.evt_close) - - self.config = gui_config() - self.title = title - - self.SetIcons(get_icon_resource("mc4.ico")) - - wx.EVT_MENU(self, self.ID_CMD_EXIT, self.evt_cmd_exit) - wx.EVT_MENU(self, self.ID_CMD_OPEN, self.evt_cmd_open) - wx.EVT_MENU(self, self.ID_CMD_EXPORT, self.evt_cmd_export) - wx.EVT_MENU(self, self.ID_CMD_IMPORT, self.evt_cmd_import) - wx.EVT_MENU(self, self.ID_CMD_DELETE, self.evt_cmd_delete) - wx.EVT_MENU(self, self.ID_CMD_ASCII, self.evt_cmd_ascii) - - filemenu = wx.Menu() - filemenu.Append(self.ID_CMD_OPEN, "&Open...", - "Opens an existing PS2 memory card image.") - filemenu.AppendSeparator() - self.export_menu_item = filemenu.Append( - self.ID_CMD_EXPORT, "&Export...", - "Export a save file from this image.") - self.import_menu_item = filemenu.Append( - self.ID_CMD_IMPORT, "&Import...", - "Import a save file into this image.") - self.delete_menu_item = filemenu.Append( - self.ID_CMD_DELETE, "&Delete") - filemenu.AppendSeparator() - filemenu.Append(self.ID_CMD_EXIT, "E&xit") - - optionmenu = wx.Menu() - self.ascii_menu_item = optionmenu.AppendCheckItem( - self.ID_CMD_ASCII, "&ASCII Descriptions", - "Show descriptions in ASCII instead of Shift-JIS") - - - wx.EVT_MENU_OPEN(self, self.evt_menu_open); - - self.CreateToolBar(wx.TB_HORIZONTAL) - self.toolbar = toolbar = self.GetToolBar() - tbsize = (32, 32) - toolbar.SetToolBitmapSize(tbsize) - add_tool(toolbar, self.ID_CMD_OPEN, "Open", "mc2.ico") - toolbar.AddSeparator() - add_tool(toolbar, self.ID_CMD_IMPORT, "Import", "mc5b.ico") - add_tool(toolbar, self.ID_CMD_EXPORT, "Export", "mc6a.ico") - toolbar.Realize() - - self.statusbar = self.CreateStatusBar(2, - style = wx.ST_SIZEGRIP) - self.statusbar.SetStatusWidths([-2, -1]) - - panel = wx.Panel(self, wx.ID_ANY, (0, 0)) - - self.dirlist = dirlist_control(panel, - self.evt_dirlist_item_focused, - self.evt_dirlist_select, - self.config) - if mcname != None: - self.open_mc(mcname) - else: - self.refresh() - - sizer = wx.BoxSizer(wx.HORIZONTAL) - sizer.Add(self.dirlist, 2, wx.EXPAND) - sizer.AddSpacer(5) - - icon_win = None - if mymcsup != None: - icon_win = icon_window(panel, self) - if icon_win.failed: - icon_win.Destroy() - icon_win = None - self.icon_win = icon_win - - if icon_win == None: - self.info1 = None - self.info2 = None - else: - self.icon_menu = icon_menu = wx.Menu() - icon_win.append_menu_options(self, icon_menu) - optionmenu.AppendSubMenu(icon_menu, "Icon Window") - title_style = wx.ALIGN_RIGHT | wx.ST_NO_AUTORESIZE - - self.info1 = wx.StaticText(panel, -1, "", - style = title_style) - self.info2 = wx.StaticText(panel, -1, "", - style = title_style) - # self.info3 = wx.StaticText(panel, -1, "") - - info_sizer = wx.BoxSizer(wx.VERTICAL) - info_sizer.Add(self.info1, 0, wx.EXPAND) - info_sizer.Add(self.info2, 0, wx.EXPAND) - # info_sizer.Add(self.info3, 0, wx.EXPAND) - info_sizer.AddSpacer(5) - info_sizer.Add(icon_win, 1, wx.EXPAND) - - sizer.Add(info_sizer, 1, wx.EXPAND | wx.ALL, - border = 5) - - menubar = wx.MenuBar() - menubar.Append(filemenu, "&File") - menubar.Append(optionmenu, "&Options") - self.SetMenuBar(menubar) - - - panel.SetSizer(sizer) - panel.SetAutoLayout(True) - sizer.Fit(panel) - - self.Show(True) - - if self.mc == None: - self.evt_cmd_open() - - def _close_mc(self): - if self.mc != None: - try: - self.mc.close() - except EnvironmentError, value: - self.mc_error(value) - self.mc = None - if self.f != None: - try: - self.f.close() - except EnvironmentError, value: - self.mc_error(value) - self.f = None - self.mcname = None - - def refresh(self): - try: - self.dirlist.update(self.mc) - except EnvironmentError, value: - self.mc_error(value) - self._close_mc() - self.dirlist.update(None) - - mc = self.mc - - self.toolbar.EnableTool(self.ID_CMD_IMPORT, mc != None) - self.toolbar.EnableTool(self.ID_CMD_EXPORT, False) - - if mc == None: - status = "No memory card image" - else: - free = mc.get_free_space() / 1024 - limit = mc.get_allocatable_space() / 1024 - status = "%dK of %dK free" % (free, limit) - self.statusbar.SetStatusText(status, 1) - - def open_mc(self, filename): - self._close_mc() - self.statusbar.SetStatusText("", 1) - if self.icon_win != None: - self.icon_win.load_icon(None, None) - - f = None - try: - f = file(filename, "r+b") - mc = ps2mc.ps2mc(f) - except EnvironmentError, value: - if f != None: - f.close() - self.mc_error(value, filename) - self.SetTitle(self.title) - self.refresh() - return - - self.f = f - self.mc = mc - self.mcname = filename - self.SetTitle(filename + " - " + self.title) - self.refresh() - - def evt_menu_open(self, event): - self.import_menu_item.Enable(self.mc != None) - selected = self.mc != None and len(self.dirlist.selected) > 0 - self.export_menu_item.Enable(selected) - self.delete_menu_item.Enable(selected) - self.ascii_menu_item.Check(self.config.get_ascii()) - if self.icon_win != None: - self.icon_win.update_menu(self.icon_menu) - - def evt_dirlist_item_focused(self, event): - if self.icon_win == None: - return - - mc = self.mc - - i = event.GetData() - (ent, icon_sys, size, title) = self.dirlist.dirtable[i] - self.info1.SetLabel(title[0]) - self.info2.SetLabel(title[1]) - - a = ps2save.unpack_icon_sys(icon_sys) - try: - mc.chdir("/" + ent[8]) - f = mc.open(a[15], "rb") - try: - icon = f.read() - finally: - f.close() - except EnvironmentError, value: - print "icon failed to load", value - self.icon_win.load_icon(None, None) - return - - self.icon_win.load_icon(icon_sys, icon) - - def evt_dirlist_select(self, event): - self.toolbar.EnableTool(self.ID_CMD_IMPORT, self.mc != None) - self.toolbar.EnableTool(self.ID_CMD_EXPORT, - len(self.dirlist.selected) > 0) - - def evt_cmd_open(self, event = None): - fn = wx.FileSelector("Open Memory Card Image", - self.config.get_memcard_dir(""), - "Mcd001.ps2", "ps2", "*.ps2", - wx.FD_FILE_MUST_EXIST | wx.FD_OPEN, - self) - if fn == "": - return - self.open_mc(fn) - if self.mc != None: - dirname = os.path.dirname(fn) - if os.path.isabs(dirname): - self.config.set_memcard_dir(dirname) - - def evt_cmd_export(self, event): - mc = self.mc - if mc == None: - return - - selected = self.dirlist.selected - dirtable = self.dirlist.dirtable - sfiles = [] - for i in selected: - dirname = dirtable[i][0][8] - try: - sf = mc.export_save_file("/" + dirname) - longname = ps2save.make_longname(dirname, sf) - sfiles.append((dirname, sf, longname)) - except EnvironmentError, value: - self.mc_error(value. dirname) - - if len(sfiles) == 0: - return - - dir = self.config.get_savefile_dir("") - if len(selected) == 1: - (dirname, sf, longname) = sfiles[0] - fn = wx.FileSelector("Export " + dirname, - dir, longname, "psu", - "EMS save file (.psu)|*.psu" - "|MAXDrive save file (.max)" - "|*.max", - (wx.FD_OVERWRITE_PROMPT - | wx.FD_SAVE), - self) - if fn == "": - return - try: - f = file(fn, "wb") - try: - if fn.endswith(".max"): - sf.save_max_drive(f) - else: - sf.save_ems(f) - finally: - f.close() - except EnvironmentError, value: - self.mc_error(value, fn) - return - - dir = os.path.dirname(fn) - if os.path.isabs(dir): - self.config.set_savefile_dir(dir) - - self.message_box("Exported " + fn + " successfully.") - return - - dir = wx.DirSelector("Export Save Files", dir, parent = self) - if dir == "": - return - count = 0 - for (dirname, sf, longname) in sfiles: - fn = os.path.join(dir, longname) + ".psu" - try: - f = file(fn, "wb") - sf.save_ems(f) - f.close() - count += 1 - except EnvironmentError, value: - self.mc_error(value, fn) - if count > 0: - if os.path.isabs(dir): - self.config.set_savefile_dir(dir) - self.message_box("Exported %d file(s) successfully." - % count) - - - def _do_import(self, fn): - sf = ps2save.ps2_save_file() - f = file(fn, "rb") - try: - ft = ps2save.detect_file_type(f) - f.seek(0) - if ft == "max": - sf.load_max_drive(f) - elif ft == "psu": - sf.load_ems(f) - elif ft == "cbs": - sf.load_codebreaker(f) - elif ft == "sps": - sf.load_sharkport(f) - elif ft == "npo": - self.error_box(fn + ": nPort saves" - " are not supported.") - return - else: - self.error_box(fn + ": Save file format not" - " recognized.") - return - finally: - f.close() - - if not self.mc.import_save_file(sf, True): - self.error_box(fn + ": Save file already present.") - - def evt_cmd_import(self, event): - if self.mc == None: - return - - dir = self.config.get_savefile_dir("") - fd = wx.FileDialog(self, "Import Save File", dir, - wildcard = ("PS2 save files" - " (.cbs;.psu;.max;.sps;.xps)" - "|*.cbs;*.psu;*.max;*.sps;*.xps" - "|All files|*.*"), - style = (wx.FD_OPEN | wx.FD_MULTIPLE - | wx.FD_FILE_MUST_EXIST)) - if fd == None: - return - r = fd.ShowModal() - if r == wx.ID_CANCEL: - return - - success = None - for fn in fd.GetPaths(): - try: - self._do_import(fn) - success = fn - except EnvironmentError, value: - self.mc_error(value, fn) - - if success != None: - dir = os.path.dirname(success) - if os.path.isabs(dir): - self.config.set_savefile_dir(dir) - self.refresh() - - def evt_cmd_delete(self, event): - mc = self.mc - if mc == None: - return - - selected = self.dirlist.selected - dirtable = self.dirlist.dirtable - - dirnames = [dirtable[i][0][8] - for i in selected] - if len(selected) == 1: - title = dirtable[list(selected)[0]][3] - s = dirnames[0] + " (" + single_title(title) + ")" - else: - s = ", ".join(dirnames) - if len(s) > 200: - s = s[:200] + "..." - r = self.message_box("Are you sure you want to delete " - + s + "?", - "Delete Save File Confirmation", - wx.YES_NO) - if r != wx.YES: - return - - for dn in dirnames: - try: - mc.rmdir("/" + dn) - except EnvironmentError, value: - self.mc_error(value, dn) - - mc.check() - self.refresh() - - def evt_cmd_ascii(self, event): - self.config.set_ascii(not self.config.get_ascii()) - self.refresh() - - def evt_cmd_exit(self, event): - self.Close(True) - - def evt_close(self, event): - self._close_mc() - self.Destroy() - -def run(filename = None): - """Display a GUI for working with memory card images.""" - - wx_app = wx.PySimpleApp() - frame = gui_frame(None, "mymc", filename) - return wx_app.MainLoop() - -if __name__ == "__main__": - import gc - gc.set_debug(gc.DEBUG_LEAK) - - run("test.ps2") - - gc.collect() - for o in gc.garbage: - print - print o - if type(o) == ps2mc.ps2mc_file: - for m in dir(o): - print m, getattr(o, m) - - -# while True: -# for o in gc.garbage: -# if type(o) == ps2mc.ps2mc_file: -# for m in dir(o): -# if getattr(o, m) == None: -# continue -# if (m == "__del__" -# or m == "__class__" -# or m == "__dict__" -# or m == "__weakref__"): -# continue -# print m -# setattr(o, m, None) -# o = None -# break -# break -# del gc.garbage[:] -# gc.collect() +# +# gui.py +# +# By Ross Ridge +# Public Domain +# + +"""Graphical user-interface for mymc.""" + +_SCCS_ID = "@(#) mymc gui.py 1.8 22/02/05 19:20:59\n" + +import os +import sys +import struct +import cStringIO +import time +from functools import partial + +# Work around a problem with mixing wx and py2exe +if os.name == "nt" and hasattr(sys, "setdefaultencoding"): + sys.setdefaultencoding("mbcs") +import wx + +import ps2mc +import ps2save +import guires + +try: + import ctypes + import mymcicon + D3DXVECTOR3 = mymcicon.D3DXVECTOR3 + D3DXVECTOR4 = mymcicon.D3DXVECTOR4 + D3DXVECTOR4_ARRAY3 = mymcicon.D3DXVECTOR4_ARRAY3 + + def mkvec4arr3(l): + return D3DXVECTOR4_ARRAY3(*[D3DXVECTOR4(*vec) + for vec in l]) +except ImportError: + mymcicon = None + +lighting_none = {"lighting": False, + "vertex_diffuse": False, + "alt_lighting": False, + "light_dirs": [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]], + "light_colours": [[0, 0, 0, 0], [0, 0, 0, 0], + [0, 0, 0, 0]], + "ambient": [0, 0, 0, 0]} + +lighting_diffuse = {"lighting": False, + "vertex_diffuse": True, + "alt_lighting": False, + "light_dirs": [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]], + "light_colours": [[0, 0, 0, 0], [0, 0, 0, 0], + [0, 0, 0, 0]], + "ambient": [0, 0, 0, 0]} + +lighting_icon = {"lighting": True, + "vertex_diffuse": True, + "alt_lighting": False, + "light_dirs": [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]], + "light_colours": [[0, 0, 0, 0], [0, 0, 0, 0], + [0, 0, 0, 0]], + "ambient": [0, 0, 0, 0]} + +lighting_alternate = {"lighting": True, + "vertex_diffuse": True, + "alt_lighting": True, + "light_dirs": [[1, -1, 2, 0], + [-1, 1, -2, 0], + [0, 1, 0, 0]], + "light_colours": [[1, 1, 1, 1], + [1, 1, 1, 1], + [0.7, 0.7, 0.7, 1]], + "ambient": [0.5, 0.5, 0.5, 1]} + +lighting_alternate2 = {"lighting": True, + "vertex_diffuse": False, + "alt_lighting": True, + "light_dirs": [[1, -1, 2, 0], + [-1, 1, -2, 0], + [0, 4, 1, 0]], + "light_colours": [[0.7, 0.7, 0.7, 1], + [0.7, 0.7, 0.7, 1], + [0.2, 0.2, 0.2, 1]], + "ambient": [0.3, 0.3, 0.3, 1]} + +camera_default = [0, 4, -8] +camera_high = [0, 7, -6] +camera_near = [0, 3, -6] +camera_flat = [0, 2, -7.5] + +def get_dialog_units(win): + return win.ConvertDialogToPixels((1, 1))[0] + +def single_title(title): + """Convert the two parts of an icon.sys title into one string.""" + + title = title[0] + " " + title[1] + return u" ".join(title.split()) + +def _get_icon_resource_as_images(name): + ico = guires.resources[name] + images = [] + f = cStringIO.StringIO(ico) + count = struct.unpack("= size[0] and sz[1] >= size[1]: + if ((best_size[0] < size[0] or best_size[1] < size[1]) + or sz[0] * sz[1] < best_size[0] * best_size[1]): + best = img + best_size = sz + elif sz[0] * sz[1] > best_size[0] * best_size[1]: + best = img + best_size = sz + img = best.Rescale(size[0], size[1], wx.IMAGE_QUALITY_HIGH) + return wx.Bitmap(img) + + +class dirlist_control(wx.ListCtrl): + """Lists all the save files in a memory card image.""" + + def __init__(self, parent, evt_focus, evt_select, config): + self.config = config + self.selected = set() + self.evt_select = evt_select + wx.ListCtrl.__init__(self, parent, wx.ID_ANY, + style = wx.LC_REPORT) + self.Bind(wx.EVT_LIST_COL_CLICK, self.evt_col_click) + self.Bind(wx.EVT_LIST_ITEM_FOCUSED, evt_focus) + self.Bind(wx.EVT_LIST_ITEM_SELECTED, self.evt_item_selected) + self.Bind(wx.EVT_LIST_ITEM_DESELECTED, + self.evt_item_deselected) + + def _update_dirtable(self, mc, dir): + self.dirtable = table = [] + enc = "unicode" + if self.config.get_ascii(): + enc = "ascii" + for ent in dir: + if not ps2mc.mode_is_dir(ent[0]): + continue + dirname = "/" + ent[8] + s = mc.get_icon_sys(dirname) + if s == None: + continue + a = ps2save.unpack_icon_sys(s) + size = mc.dir_size(dirname) + title = ps2save.icon_sys_title(a, encoding = enc) + table.append((ent, s, size, title)) + + def update_dirtable(self, mc): + self.dirtable = [] + if mc == None: + return + dir = mc.dir_open("/") + try: + self._update_dirtable(mc, dir) + finally: + dir.close() + + def get_dir_name(self, i): + return self.dirtable[i][0][8] + + def get_dir_title(self, i): + return self.dirtable[i][3] + + def get_dir_size(self, i): + return self.dirtable[i][2] + + def get_dir_modified(self, i): + m = list(self.dirtable[i][0][6]) + m.reverse() + return m + + def sort_items(self, key): + def cmp(i1, i2): + a1 = key(i1) + a2 = key(i2) + if a1 < a2: + return -1 + if a1 > a2: + return 1 + return 0 + self.SortItems(cmp) + + def evt_col_click(self, event): + col = event.GetColumn() + if col == 0: + key = self.get_dir_name + elif col == 1: + key = self.get_dir_size + elif col == 2: + key = self.get_dir_modified + elif col == 3: + key = self.get_dir_title + self.sort_items(key) + return + + def evt_item_selected(self, event): + self.selected.add(event.GetData()) + self.evt_select(event) + + def evt_item_deselected(self, event): + self.selected.discard(event.GetData()) + self.evt_select(event) + + def update(self, mc): + """Update the ListCtrl according to the contents of the + memory card image.""" + + self.ClearAll() + self.selected = set() + self.InsertColumn(0, "Directory") + self.InsertColumn(1, "Size") + self.InsertColumn(2, "Modified") + self.InsertColumn(3, "Description") + li = self.GetColumn(1) + li.SetAlign(wx.LIST_FORMAT_RIGHT) + li.SetText("Size") + self.SetColumn(1, li) + + self.update_dirtable(mc) + + empty = (len(self.dirtable) == 0) + self.Enable(not empty) + if empty: + return + + for (i, a) in enumerate(self.dirtable): + (ent, icon_sys, size, title) = a + li = self.InsertItem(i, ent[8]) + self.SetItem(li, 1, "%dK" % (size / 1024)) + m = ent[6] + self.SetItem(li, 2, ("%04d-%02d-%02d %02d:%02d" + % (m[5], m[4], m[3], m[2], m[1]))) + self.SetItem(li, 3, single_title(title)) + self.SetItemData(li, i) + + du = get_dialog_units(self) + for i in range(4): + self.SetColumnWidth(i, wx.LIST_AUTOSIZE) + self.SetColumnWidth(i, self.GetColumnWidth(i) + du) + self.sort_items(self.get_dir_name) + + +class icon_window(wx.Window): + """Displays a save file's 3D icon. Windows only. + + The rendering of the 3D icon is handled by C++ code in the + mymcicon DLL which subclasses this window. This class mainly + handles configuration options that affect how the 3D icon is + displayed. + """ + + ID_CMD_ANIMATE = 201 + ID_CMD_LIGHT_NONE = 202 + ID_CMD_LIGHT_ICON = 203 + ID_CMD_LIGHT_ALT1 = 204 + ID_CMD_LIGHT_ALT2 = 205 + ID_CMD_CAMERA_FLAT = 206 + ID_CMD_CAMERA_DEFAULT = 207 + ID_CMD_CAMERA_NEAR = 209 + ID_CMD_CAMERA_HIGH = 210 + + light_options = {ID_CMD_LIGHT_NONE: lighting_none, + ID_CMD_LIGHT_ICON: lighting_icon, + ID_CMD_LIGHT_ALT1: lighting_alternate, + ID_CMD_LIGHT_ALT2: lighting_alternate2} + + camera_options = {ID_CMD_CAMERA_FLAT: camera_flat, + ID_CMD_CAMERA_DEFAULT: camera_default, + ID_CMD_CAMERA_NEAR: camera_near, + ID_CMD_CAMERA_HIGH: camera_high} + + def append_menu_options(self, win, menu): + menu.AppendCheckItem(icon_window.ID_CMD_ANIMATE, + "Animate Icons") + menu.AppendSeparator() + menu.AppendRadioItem(icon_window.ID_CMD_LIGHT_NONE, + "Lighting Off") + menu.AppendRadioItem(icon_window.ID_CMD_LIGHT_ICON, + "Icon Lighting") + menu.AppendRadioItem(icon_window.ID_CMD_LIGHT_ALT1, + "Alternate Lighting") + menu.AppendRadioItem(icon_window.ID_CMD_LIGHT_ALT2, + "Alternate Lighting 2") + menu.AppendSeparator() + menu.AppendRadioItem(icon_window.ID_CMD_CAMERA_FLAT, + "Camera Flat") + menu.AppendRadioItem(icon_window.ID_CMD_CAMERA_DEFAULT, + "Camera Default") + menu.AppendRadioItem(icon_window.ID_CMD_CAMERA_NEAR, + "Camera Near") + menu.AppendRadioItem(icon_window.ID_CMD_CAMERA_HIGH, + "Camera High") + + bind_menu = partial(win.Bind, wx.EVT_MENU) + + bind_menu(self.evt_menu_animate, None, + icon_window.ID_CMD_ANIMATE) + + bind_menu_light = partial(bind_menu, self.evt_menu_light, None) + bind_menu_light(icon_window.ID_CMD_LIGHT_NONE) + bind_menu_light(icon_window.ID_CMD_LIGHT_ICON) + bind_menu_light(icon_window.ID_CMD_LIGHT_ALT1) + bind_menu_light(icon_window.ID_CMD_LIGHT_ALT2) + + bind_menu_camera = partial(bind_menu, + self.evt_menu_camera, None) + bind_menu_camera(icon_window.ID_CMD_CAMERA_FLAT) + bind_menu_camera(icon_window.ID_CMD_CAMERA_DEFAULT) + bind_menu_camera(icon_window.ID_CMD_CAMERA_NEAR) + bind_menu_camera(icon_window.ID_CMD_CAMERA_HIGH) + + def __init__(self, parent, focus): + self.failed = False + wx.Window.__init__(self, parent) + if mymcicon == None: + self.failed = True + return + r = mymcicon.init_icon_renderer(focus.GetHandle(), + self.GetHandle()) + if r == -1: + print "init_icon_renderer failed" + self.failed = True + return + + self.config = config = mymcicon.icon_config() + config.animate = True + + self.menu = wx.Menu() + self.append_menu_options(self, self.menu) + self.set_lighting(self.ID_CMD_LIGHT_ALT2) + self.set_camera(self.ID_CMD_CAMERA_DEFAULT) + + self.Bind(wx.EVT_CONTEXT_MENU, self.evt_context_menu) + + def __del__(self): + if mymcicon != None: + mymcicon.delete_icon_renderer() + + def update_menu(self, menu): + """Update the content menu according to the current config.""" + + menu.Check(icon_window.ID_CMD_ANIMATE, self.config.animate) + menu.Check(self.lighting_id, True) + menu.Check(self.camera_id, True) + + def load_icon(self, icon_sys, icon): + """Pass the raw icon data to the support DLL for display.""" + + if self.failed: + return + + if icon_sys == None or icon == None: + r = mymcicon.load_icon(None, 0, None, 0) + else: + r = mymcicon.load_icon(icon_sys, len(icon_sys), + icon, len(icon)) + if r != 0: + print "load_icon", r + self.failed = True + + def _set_lighting(self, lighting, vertex_diffuse, alt_lighting, + light_dirs, light_colours, ambient): + if self.failed: + return + config = self.config + config.lighting = lighting + config.vertex_diffuse = vertex_diffuse + config.alt_lighting = alt_lighting + config.light_dirs = mkvec4arr3(light_dirs) + config.light_colours = mkvec4arr3(light_colours) + config.ambient = D3DXVECTOR4(*ambient) + if mymcicon.set_config(config) == -1: + self.failed = True + + def set_lighting(self, id): + self.lighting_id = id + self._set_lighting(**self.light_options[id]) + + def set_animate(self, animate): + if self.failed: + return + self.config.animate = animate + if mymcicon.set_config(self.config) == -1: + self.failed = True + + def _set_camera(self, camera): + if self.failed: + return + self.config.camera = mymcicon.D3DXVECTOR3(*camera) + if mymcicon.set_config(self.config) == -1: + self.failed = True + + def set_camera(self, id): + self.camera_id = id + self._set_camera(self.camera_options[id]) + + def evt_context_menu(self, event): + self.update_menu(self.menu) + self.PopupMenu(self.menu) + + def evt_menu_animate(self, event): + self.set_animate(not self.config.animate) + + def evt_menu_light(self, event): + self.set_lighting(event.GetId()) + + def evt_menu_camera(self, event): + self.set_camera(event.GetId()) + +class gui_config(wx.Config): + """A class for holding the persistant configuration state.""" + + memcard_dir = "Memory Card Directory" + savefile_dir = "Save File Directory" + ascii = "ASCII Descriptions" + + def __init__(self): + wx.Config.__init__(self, "mymc", "Ross Ridge", + style = wx.CONFIG_USE_LOCAL_FILE) + + def get_memcard_dir(self, default = None): + return self.Read(gui_config.memcard_dir, default) + + def set_memcard_dir(self, value): + return self.Write(gui_config.memcard_dir, value) + + def get_savefile_dir(self, default = None): + return self.Read(gui_config.savefile_dir, default) + + def set_savefile_dir(self, value): + return self.Write(gui_config.savefile_dir, value) + + def get_ascii(self, default = False): + return bool(self.ReadInt(gui_config.ascii, int(bool(default)))) + + def set_ascii(self, value): + return self.WriteInt(gui_config.ascii, int(bool(value))) + +def add_tool(toolbar, id, label, ico): + tbsize = toolbar.GetToolBitmapSize() + bmp = get_icon_resource_bmp(ico, tbsize) + return toolbar.AddTool(id, label, bmp, shortHelp = label) + +class gui_frame(wx.Frame): + """The main top level window.""" + + ID_CMD_EXIT = wx.ID_EXIT + ID_CMD_OPEN = wx.ID_OPEN + ID_CMD_EXPORT = 103 + ID_CMD_IMPORT = 104 + ID_CMD_DELETE = wx.ID_DELETE + ID_CMD_ASCII = 106 + + def message_box(self, message, caption = "mymc", style = wx.OK, + x = -1, y = -1): + return wx.MessageBox(message, caption, style, self, x, y) + + def error_box(self, msg): + return self.message_box(msg, "Error", wx.OK | wx.ICON_ERROR) + + def mc_error(self, value, filename = None): + """Display a message box for EnvironmentError exeception.""" + + if filename == None: + filename = getattr(value, "filename") + if filename == None: + filename = self.mcname + if filename == None: + filename = "???" + + strerror = getattr(value, "strerror", None) + if strerror == None: + strerror = "unknown error" + + return self.error_box(filename + ": " + strerror) + + def __init__(self, parent, title, mcname = None): + self.f = None + self.mc = None + self.mcname = None + self.icon_win = None + + size = (750, 350) + if mymcicon == None: + size = (500, 350) + wx.Frame.__init__(self, parent, wx.ID_ANY, title, size = size) + + self.Bind(wx.EVT_CLOSE, self.evt_close) + + self.config = gui_config() + self.title = title + + self.SetIcons(get_icon_resource("mc4.ico")) + + bind_menu = (lambda handler, id: + self.Bind(wx.EVT_MENU, handler, None, id)) + bind_menu(self.evt_cmd_exit, self.ID_CMD_EXIT) + bind_menu(self.evt_cmd_open, self.ID_CMD_OPEN) + bind_menu(self.evt_cmd_export, self.ID_CMD_EXPORT) + bind_menu(self.evt_cmd_import, self.ID_CMD_IMPORT) + bind_menu(self.evt_cmd_delete, self.ID_CMD_DELETE) + bind_menu(self.evt_cmd_ascii, self.ID_CMD_ASCII, ) + + filemenu = wx.Menu() + filemenu.Append(self.ID_CMD_OPEN, "&Open...", + "Opens an existing PS2 memory card image.") + filemenu.AppendSeparator() + self.export_menu_item = filemenu.Append( + self.ID_CMD_EXPORT, "&Export...", + "Export a save file from this image.") + self.import_menu_item = filemenu.Append( + self.ID_CMD_IMPORT, "&Import...", + "Import a save file into this image.") + self.delete_menu_item = filemenu.Append( + self.ID_CMD_DELETE, "&Delete") + filemenu.AppendSeparator() + filemenu.Append(self.ID_CMD_EXIT, "E&xit") + + optionmenu = wx.Menu() + self.ascii_menu_item = optionmenu.AppendCheckItem( + self.ID_CMD_ASCII, "&ASCII Descriptions", + "Show descriptions in ASCII instead of Shift-JIS") + + + self.Bind(wx.EVT_MENU_OPEN, self.evt_menu_open); + + self.CreateToolBar(wx.TB_HORIZONTAL) + self.toolbar = toolbar = self.GetToolBar() + tbsize = (32, 32) + toolbar.SetToolBitmapSize(tbsize) + add_tool(toolbar, self.ID_CMD_OPEN, "Open", "mc2.ico") + toolbar.AddSeparator() + add_tool(toolbar, self.ID_CMD_IMPORT, "Import", "mc5b.ico") + add_tool(toolbar, self.ID_CMD_EXPORT, "Export", "mc6a.ico") + toolbar.Realize() + + self.statusbar = self.CreateStatusBar(2, + style = wx.STB_SIZEGRIP) + self.statusbar.SetStatusWidths([-2, -1]) + + panel = wx.Panel(self, wx.ID_ANY, (0, 0)) + + self.dirlist = dirlist_control(panel, + self.evt_dirlist_item_focused, + self.evt_dirlist_select, + self.config) + if mcname != None: + self.open_mc(mcname) + else: + self.refresh() + + sizer = wx.BoxSizer(wx.HORIZONTAL) + sizer.Add(self.dirlist, 2, wx.EXPAND) + sizer.AddSpacer(5) + + icon_win = None + if mymcicon != None: + icon_win = icon_window(panel, self) + if icon_win.failed: + icon_win.Destroy() + icon_win = None + self.icon_win = icon_win + + if icon_win == None: + self.info1 = None + self.info2 = None + else: + self.icon_menu = icon_menu = wx.Menu() + icon_win.append_menu_options(self, icon_menu) + optionmenu.AppendSubMenu(icon_menu, "Icon Window") + title_style = wx.ALIGN_RIGHT | wx.ST_NO_AUTORESIZE + + self.info1 = wx.StaticText(panel, -1, "", + style = title_style) + self.info2 = wx.StaticText(panel, -1, "", + style = title_style) + # self.info3 = wx.StaticText(panel, -1, "") + + info_sizer = wx.BoxSizer(wx.VERTICAL) + info_sizer.Add(self.info1, 0, wx.EXPAND) + info_sizer.Add(self.info2, 0, wx.EXPAND) + # info_sizer.Add(self.info3, 0, wx.EXPAND) + info_sizer.AddSpacer(5) + info_sizer.Add(icon_win, 1, wx.EXPAND) + + sizer.Add(info_sizer, 1, wx.EXPAND | wx.ALL, + border = 5) + + menubar = wx.MenuBar() + menubar.Append(filemenu, "&File") + menubar.Append(optionmenu, "&Options") + self.SetMenuBar(menubar) + + + panel.SetSizer(sizer) + panel.SetAutoLayout(True) + sizer.Fit(panel) + + self.Show(True) + + if self.mc == None: + self.evt_cmd_open() + + def _close_mc(self): + if self.mc != None: + try: + self.mc.close() + except EnvironmentError, value: + self.mc_error(value) + self.mc = None + if self.f != None: + try: + self.f.close() + except EnvironmentError, value: + self.mc_error(value) + self.f = None + self.mcname = None + + def refresh(self): + try: + self.dirlist.update(self.mc) + except EnvironmentError, value: + self.mc_error(value) + self._close_mc() + self.dirlist.update(None) + + mc = self.mc + + self.toolbar.EnableTool(self.ID_CMD_IMPORT, mc != None) + self.toolbar.EnableTool(self.ID_CMD_EXPORT, False) + + if mc == None: + status = "No memory card image" + else: + free = mc.get_free_space() / 1024 + limit = mc.get_allocatable_space() / 1024 + status = "%dK of %dK free" % (free, limit) + self.statusbar.SetStatusText(status, 1) + + def open_mc(self, filename): + self._close_mc() + self.statusbar.SetStatusText("", 1) + if self.icon_win != None: + self.icon_win.load_icon(None, None) + + f = None + try: + f = file(filename, "r+b") + mc = ps2mc.ps2mc(f) + except EnvironmentError, value: + if f != None: + f.close() + self.mc_error(value, filename) + self.SetTitle(self.title) + self.refresh() + return + + self.f = f + self.mc = mc + self.mcname = filename + self.SetTitle(filename + " - " + self.title) + self.refresh() + + def evt_menu_open(self, event): + self.import_menu_item.Enable(self.mc != None) + selected = self.mc != None and len(self.dirlist.selected) > 0 + self.export_menu_item.Enable(selected) + self.delete_menu_item.Enable(selected) + self.ascii_menu_item.Check(self.config.get_ascii()) + if self.icon_win != None: + self.icon_win.update_menu(self.icon_menu) + + def evt_dirlist_item_focused(self, event): + if self.icon_win == None: + return + + mc = self.mc + + i = event.GetData() + (ent, icon_sys, size, title) = self.dirlist.dirtable[i] + self.info1.SetLabel(title[0]) + self.info2.SetLabel(title[1]) + + a = ps2save.unpack_icon_sys(icon_sys) + try: + mc.chdir("/" + ent[8]) + f = mc.open(a[15], "rb") + try: + icon = f.read() + finally: + f.close() + except EnvironmentError, value: + print "icon failed to load", value + self.icon_win.load_icon(None, None) + return + + self.icon_win.load_icon(icon_sys, icon) + + def evt_dirlist_select(self, event): + self.toolbar.EnableTool(self.ID_CMD_IMPORT, self.mc != None) + self.toolbar.EnableTool(self.ID_CMD_EXPORT, + len(self.dirlist.selected) > 0) + + def evt_cmd_open(self, event = None): + fn = wx.FileSelector("Open Memory Card Image", + self.config.get_memcard_dir(""), + "Mcd001.ps2", "ps2", "*.ps2", + wx.FD_FILE_MUST_EXIST | wx.FD_OPEN, + self) + if fn == "": + return + self.open_mc(fn) + if self.mc != None: + dirname = os.path.dirname(fn) + if os.path.isabs(dirname): + self.config.set_memcard_dir(dirname) + + def evt_cmd_export(self, event): + mc = self.mc + if mc == None: + return + + selected = self.dirlist.selected + dirtable = self.dirlist.dirtable + sfiles = [] + for i in selected: + dirname = dirtable[i][0][8] + try: + sf = mc.export_save_file("/" + dirname) + longname = ps2save.make_longname(dirname, sf) + sfiles.append((dirname, sf, longname)) + except EnvironmentError, value: + self.mc_error(value. dirname) + + if len(sfiles) == 0: + return + + dir = self.config.get_savefile_dir("") + if len(selected) == 1: + (dirname, sf, longname) = sfiles[0] + fn = wx.FileSelector("Export " + dirname, + dir, longname, "psu", + "EMS save file (.psu)|*.psu" + "|MAXDrive save file (.max)" + "|*.max", + (wx.FD_OVERWRITE_PROMPT + | wx.FD_SAVE), + self) + if fn == "": + return + try: + f = file(fn, "wb") + try: + if fn.endswith(".max"): + sf.save_max_drive(f) + else: + sf.save_ems(f) + finally: + f.close() + except EnvironmentError, value: + self.mc_error(value, fn) + return + + dir = os.path.dirname(fn) + if os.path.isabs(dir): + self.config.set_savefile_dir(dir) + + self.message_box("Exported " + fn + " successfully.") + return + + dir = wx.DirSelector("Export Save Files", dir, parent = self) + if dir == "": + return + count = 0 + for (dirname, sf, longname) in sfiles: + fn = os.path.join(dir, longname) + ".psu" + try: + f = file(fn, "wb") + sf.save_ems(f) + f.close() + count += 1 + except EnvironmentError, value: + self.mc_error(value, fn) + if count > 0: + if os.path.isabs(dir): + self.config.set_savefile_dir(dir) + self.message_box("Exported %d file(s) successfully." + % count) + + + def _do_import(self, fn): + sf = ps2save.ps2_save_file() + f = file(fn, "rb") + try: + ft = ps2save.detect_file_type(f) + f.seek(0) + if ft == "max": + sf.load_max_drive(f) + elif ft == "psu": + sf.load_ems(f) + elif ft == "cbs": + sf.load_codebreaker(f) + elif ft == "sps": + sf.load_sharkport(f) + elif ft == "npo": + self.error_box(fn + ": nPort saves" + " are not supported.") + return + else: + self.error_box(fn + ": Save file format not" + " recognized.") + return + finally: + f.close() + + if not self.mc.import_save_file(sf, True): + self.error_box(fn + ": Save file already present.") + + def evt_cmd_import(self, event): + if self.mc == None: + return + + dir = self.config.get_savefile_dir("") + fd = wx.FileDialog(self, "Import Save File", dir, + wildcard = ("PS2 save files" + " (.cbs;.psu;.max;.sps;.xps)" + "|*.cbs;*.psu;*.max;*.sps;*.xps" + "|All files|*.*"), + style = (wx.FD_OPEN | wx.FD_MULTIPLE + | wx.FD_FILE_MUST_EXIST)) + if fd == None: + return + r = fd.ShowModal() + if r == wx.ID_CANCEL: + return + + success = None + for fn in fd.GetPaths(): + try: + self._do_import(fn) + success = fn + except EnvironmentError, value: + self.mc_error(value, fn) + + if success != None: + dir = os.path.dirname(success) + if os.path.isabs(dir): + self.config.set_savefile_dir(dir) + self.refresh() + + def evt_cmd_delete(self, event): + mc = self.mc + if mc == None: + return + + selected = self.dirlist.selected + dirtable = self.dirlist.dirtable + + dirnames = [dirtable[i][0][8] + for i in selected] + if len(selected) == 1: + title = dirtable[list(selected)[0]][3] + s = dirnames[0] + " (" + single_title(title) + ")" + else: + s = ", ".join(dirnames) + if len(s) > 200: + s = s[:200] + "..." + r = self.message_box("Are you sure you want to delete " + + s + "?", + "Delete Save File Confirmation", + wx.YES_NO) + if r != wx.YES: + return + + for dn in dirnames: + try: + mc.rmdir("/" + dn) + except EnvironmentError, value: + self.mc_error(value, dn) + + mc.check() + self.refresh() + + def evt_cmd_ascii(self, event): + self.config.set_ascii(not self.config.get_ascii()) + self.refresh() + + def evt_cmd_exit(self, event): + self.Close(True) + + def evt_close(self, event): + self._close_mc() + self.Destroy() + +def run(filename = None): + """Display a GUI for working with memory card images.""" + + wx_app = wx.App() + frame = gui_frame(None, "mymc", filename) + return wx_app.MainLoop() + +if __name__ == "__main__": + import gc + gc.set_debug(gc.DEBUG_LEAK) + + run("test.ps2") + + gc.collect() + for o in gc.garbage: + print + print o + if type(o) == ps2mc.ps2mc_file: + for m in dir(o): + print m, getattr(o, m) + + +# while True: +# for o in gc.garbage: +# if type(o) == ps2mc.ps2mc_file: +# for m in dir(o): +# if getattr(o, m) == None: +# continue +# if (m == "__del__" +# or m == "__class__" +# or m == "__dict__" +# or m == "__weakref__"): +# continue +# print m +# setattr(o, m, None) +# o = None +# break +# break +# del gc.garbage[:] +# gc.collect() diff --git a/guires.py b/guires.py index 9c819d8..3ae4438 100755 --- a/guires.py +++ b/guires.py @@ -1,185 +1,185 @@ -resources = { - "mc4.ico": ( - "AAABAAIAICAQAAAAAADoAgAAJgAAADAwAAEAAAAAqA4AAA4DAAAoAAAAIAAAAEAAAAABAAQAAAAA\n" - "AIACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAgAAAAICAAIAAAACAAIAAgIAAAICAgADAwMAA\n" - "AAD/AAD/AAAA//8A/wAAAP8A/wD//wAA////AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAcABwAAAAAAAAAAAAAAAABwAABwAAAAAAAAAAAAAAAHAAYAB3d3d3dwAAAAAAAAcABm\n" - "YAB3d3dwAAAAAAAABwAGZmYAB3d3AAAAAAAAAHAAZmZmYAB4iIiIiIiAAAcABmZmZgAAB/+P+q/4\n" - "gABwAGZmZmAAAABu7u7u6IAHAAZmZmYAAAAABu7u7u6AAABmZmZgBwcAAABu7u7ugAAGZmZmAHBw\n" - "AAAABu7u7oAAAGZmYAcHBwAAAABu7u6ABwAGZgBwcHBwAAAABu7ugAAAAGAABwcHAABAAADu7oAA\n" - "AAAAAHB3cAAEAAAG7u6AAAAAAAAAB3AAwAAAbu7ugAAABwAAAAAABAAABu7u7oAAAAhgAAAAAMAH\n" - "AG7u7u6AAAAI5gAAAAAAdwbu7u7ugAAACO5gAAxAAABu7u7u7oAAAAju5gAEzAAG7u7u7u6AAAAI\n" - "7u5gAAwAbu7u7u7ugAAACO7u5gAABu7u7u7u7oAAAAju7u5gAG7u7u7u7u6AAAAI7u7u5gbu7u7u\n" - "7u7ugAAACI7u7u7u7u7u7u7u6IAAAAiIiIiIiIiIiIiIiIiAAAAIiIiIiIiIiIiIiIiIgAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA////////////g////wH///4AAH/8AAH/+AAD//AA\n" - "AAHgAAABwAAAAYAAAAGAAAABgAAAAYAAAAGAAAAB4AAAAfAAAAH4AAAB+AAAAfgAAAH4AAAB+AAA\n" - "AfgAAAH4AAAB+AAAAfgAAAH4AAAB+AAAAfgAAAH4AAAB//////////8oAAAAMAAAAGAAAAABAAgA\n" - "AAAAAIAKAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAgAAAAICAAIAAAACAAIAAgIAAAMDAwADA\n" - "3MAA8MqmAAAgQAAAIGAAACCAAAAgoAAAIMAAACDgAABAAAAAQCAAAEBAAABAYAAAQIAAAECgAABA\n" - "wAAAQOAAAGAAAABgIAAAYEAAAGBgAABggAAAYKAAAGDAAABg4AAAgAAAAIAgAACAQAAAgGAAAICA\n" - "AACAoAAAgMAAAIDgAACgAAAAoCAAAKBAAACgYAAAoIAAAKCgAACgwAAAoOAAAMAAAADAIAAAwEAA\n" - "AMBgAADAgAAAwKAAAMDAAADA4AAA4AAAAOAgAADgQAAA4GAAAOCAAADgoAAA4MAAAODgAEAAAABA\n" - "ACAAQABAAEAAYABAAIAAQACgAEAAwABAAOAAQCAAAEAgIABAIEAAQCBgAEAggABAIKAAQCDAAEAg\n" - "4ABAQAAAQEAgAEBAQABAQGAAQECAAEBAoABAQMAAQEDgAEBgAABAYCAAQGBAAEBgYABAYIAAQGCg\n" - "AEBgwABAYOAAQIAAAECAIABAgEAAQIBgAECAgABAgKAAQIDAAECA4ABAoAAAQKAgAECgQABAoGAA\n" - "QKCAAECgoABAoMAAQKDgAEDAAABAwCAAQMBAAEDAYABAwIAAQMCgAEDAwABAwOAAQOAAAEDgIABA\n" - "4EAAQOBgAEDggABA4KAAQODAAEDg4ACAAAAAgAAgAIAAQACAAGAAgACAAIAAoACAAMAAgADgAIAg\n" - "AACAICAAgCBAAIAgYACAIIAAgCCgAIAgwACAIOAAgEAAAIBAIACAQEAAgEBgAIBAgACAQKAAgEDA\n" - "AIBA4ACAYAAAgGAgAIBgQACAYGAAgGCAAIBgoACAYMAAgGDgAICAAACAgCAAgIBAAICAYACAgIAA\n" - "gICgAICAwACAgOAAgKAAAICgIACAoEAAgKBgAICggACAoKAAgKDAAICg4ACAwAAAgMAgAIDAQACA\n" - "wGAAgMCAAIDAoACAwMAAgMDgAIDgAACA4CAAgOBAAIDgYACA4IAAgOCgAIDgwACA4OAAwAAAAMAA\n" - "IADAAEAAwABgAMAAgADAAKAAwADAAMAA4ADAIAAAwCAgAMAgQADAIGAAwCCAAMAgoADAIMAAwCDg\n" - "AMBAAADAQCAAwEBAAMBAYADAQIAAwECgAMBAwADAQOAAwGAAAMBgIADAYEAAwGBgAMBggADAYKAA\n" - "wGDAAMBg4ADAgAAAwIAgAMCAQADAgGAAwICAAMCAoADAgMAAwIDgAMCgAADAoCAAwKBAAMCgYADA\n" - "oIAAwKCgAMCgwADAoOAAwMAAAMDAIADAwEAAwMBgAMDAgADAwKAA8Pv/AKSgoACAgIAAAAD/AAD/\n" - "AAAA//8A/wAAAP8A/wD//wAA////AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAKQAAKQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAApAAAAACkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACk\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKQAAAAAAAAApAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAApAAAAAAACQAAAACkpKSkpKSkpKSk\n" - "pKSkpAAAAAAAAAAAAAAAAAAAAAAAAAAAAACkAAAAAAAJCQkAAACkpKSkpKSkpKSkpKQAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAKQAAAAAAAkJCQkJAAAAAKSkpKSkpKQAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAApAAAAAAACQkJCQkJCQAAAACkpKSkpKQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACkAAAA\n" - "AAAJCQkJCQkJCQkAAAAApPf39/f39/f39/f39/f39/f39/cAAAAAAAAAAKQAAAAAAAkJCQkJCQkJ\n" - "CQkJAAAAAKQHBwf3BwcH9/f39wcHB/f39/cAAAAAAAAApAAAAAAACQkJCQkJCQkJCQkAAAAAAACk\n" - "9vb39vb29/r69/b29vf39/cAAAAAAACkAAAAAAAJCQkJCQkJCQkJCQAAAAAAAAAApPf39/f39/f3\n" - "9/f39/f39/cAAAAAAKQAAAAAAAkJCQkJCQkJCQkJAAAAAAAAAAAAAKT39/f39/f39/f39/f39/cA\n" - "AAAApAAAAAAACQkJCQkJCQkJCQkAAAAAAAAAAAAAAAD3/v7+/v7+/v7+/v739/cAAACkAAAAAAAJ\n" - "CQkJCQkJCQkJCQAApAAApAAAAAAAAAAAB/7+/v7+/v7+/v7+9/cAAAAAAAAAAAkJCQkJCQkJCQkJ\n" - "AACkAACkAAAAAAAAAAAAAAf+/v7+/v7+/v7+9/cAAAAAAAAACQkJCQkJCQkJCQkAAKQAAKQAAAAA\n" - "AAAAAAAAAAAH/v7+/v7+/v7+9/cAAACkAAAAAAkJCQkJCQkJCQAApAAApAAAAKQAAAAAAAAAAAAA\n" - "B/7+/v7+/v7+9/cAAAAApAAAAAAJCQkJCQkJAACkAACkAAAApACkAAAAAAAAAAAAAAf+/v7+/v7+\n" - "9/cAAAAAAACkAAAACQkJCQkAAKQAAKQAAACkAKQAAAAAAAAAAAAAAAAH/v7+/v7+9/cAAAAAAAAA\n" - "AAAAAAkJCQAAAAAApAAAAAAApAAAAAAAAADAAAAAAAAAB/7+/v7+9/cAAAAAAAAAAKQAAAAJAAAA\n" - "AACkAAAKpKQAAAAAAAAAAMAAAAAAAAAAAP7+/v7+9/cAAAAAAAAAAAAAAAAAAAAAAKQAAACkAKSk\n" - "AAAAAAAAwAAAAAAAAAAAB/7+/v7+9/cAAAAAAAAAAACkAAAAAAAAAAAAAACkpACkAAAAAADAAAAA\n" - "AAAAAAAH/v7+/v7+9/cAAAAAAAAAAAD3pAAAAAAAAAAAAAAApKQKAAAAAMDAAAAAAAAAAAf+/v7+\n" - "/v7+9/cAAAAAAAAAAAD39wcAAAAAAAAAAAAAAAAAAAAAwAAAAAAAAAAAB/7+/v7+/v7+9/cAAAAA\n" - "AAAAAAD39/4HAAAAAAAAAAAAAAAAAADAAAAACqQAAAAH/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+\n" - "BwAAAAAAAAAAAAAAAMDAAAAKpAoAAAf+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/gcAAAAAAAAA\n" - "AAAAAAAAAAqkpAAAB/7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v4HAAAAAAAAAAAAAAAAAKQK\n" - "AAAH/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+BwAAAAAAwADAAAAAAAAAAAf+/v7+/v7+\n" - "/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/gcAAAAAAMDAwAAAAAAAB/7+/v7+/v7+/v7+/v7+9/cA\n" - "AAAAAAAAAAD39/7+/v7+/v4HAAAAwAAAwAAAAAAH/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD3\n" - "9/7+/v7+/v7+BwAAAAAAwAAAAAf+/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/v7+\n" - "/gcAAAAAAAAAB/7+/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/v7+/v4HAAAAAAAH\n" - "/v7+/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/v7+/v7+BwAAAAf+/v7+/v7+/v7+\n" - "/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/v7+/v7+/gcAB/7+/v7+/v7+/v7+/v7+/v7+/v7+\n" - "9/cAAAAAAAAAAAD39/7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAA\n" - "AAD39/f+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v739/cAAAAAAAAAAAD3+/v79/f3\n" - "9/f39/f39/f39/f39/f39/f39/f39/f39/f39/f39/cAAAAAAAAAAAD30NDQ9/f39/f39/f39/f3\n" - "9/f39/f39/f39/f39/f396SkpKSk9/cAAAAAAAAAAAD3+ff59/f39/f39/f39/f39/f39/f39/f3\n" - "9/f39/f39/f39/f39/cAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP///////wAA////\n" - "////AAD//h////8AAP/8D////wAA//gP////AAD/8Af///8AAP/gAAAP/wAA/8AAAD//AAD/gAAB\n" - "//8AAP8AAAH//wAA/gAAAAADAAD8AAAAAAMAAPgAAAAAAwAA8AAAAAADAADgAAAAAAMAAMAAAAAA\n" - "AwAAgAAAAAADAACAAAAAAAMAAIAAAAAAAwAAgAAAAAADAADAAAAAAAMAAPAAAAAAAwAA+AAAAAAD\n" - "AAD8AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+AAAAAAMA\n" - "AP4AAAAAAwAA/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA\n" - "/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+\n" - "AAAAAAMAAP///////wAA////////AAD///////8AAA==\n" - ).decode("base64_codec"), - "mc5b.ico": ( - "AAABAAEAICAQAAAAAADoAgAAFgAAACgAAAAgAAAAQAAAAAEABAAAAAAAgAIAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAACAAACAAAAAgIAAgAAAAIAAgACAgAAAgICAAMDAwAAAAP8AAP8AAAD//wD/AAAA\n" - "/wD/AP//AAD///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAcABwAAAAAAAAAAAAAAAABwAAAAAAAAAAAAAAAAAA\n" - "AHAAYAAAAAAAAAAAAAAAAAcABmYAAAAAAAAAAAAAAKBwAGZmYABwAAAAAAAAAACqAAZmZmYABwAA\n" - "AAAAAAAAqqBmZmZgAABwAAAAAAAAB6qqZmZmAAAABwAACqqqqqqiqqZmYAAAAABwAAqqqqqqoiqq\n" - "ZgBwcAAABwAKoiIiIiIiqqAHBwAAAABwCqIiIiIiIiqqcHBwAAAABwqiIiIiIiIqqgcHBwAAAAAK\n" - "oiIiIiIiqqBwcHAABAAACqqqqqqiKqoHB3cAAEAAAAqqqqqqoqqgAAB3AAwAAAcAAAAAAKqqAAAA\n" - "AABAAABwAAAAAACqpwAAAAAMAHAHAAAAAAAAqgBwAAAAAAdwcAAAAAAAAKAABwAAxAAABwAAAAAA\n" - "AAAAAABwAEzAAHAAAAAAAAAAAAAABwAAwAcAAAAAAAAAAAAAAABwAABwAAAAAAAAAAAAAAAABwAH\n" - "AAAAAAAAAAAAAAAAAABwcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAD///////////////////////8H///+B////AP///gB///QAH//wAA//8AAH/+A\n" - "AA+AAAAHgAAAA4AAAAGAAAAAgAAAAIAAAACAAAAAgAAAAP/AAAH/wAAD/8wAB//eAA///wAf//+A\n" - "P///wH///+D////x/////////////////w==\n" - ).decode("base64_codec"), - "mc2.ico": ( - "AAABAAEAMDAAAQAAAACoDgAAFgAAACgAAAAwAAAAYAAAAAEACAAAAAAAgAoAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAACAAACAAAAAgIAAgAAAAIAAgACAgAAAwMDAAMDcwADwyqYAACBAAAAgYAAAIIAA\n" - "ACCgAAAgwAAAIOAAAEAAAABAIAAAQEAAAEBgAABAgAAAQKAAAEDAAABA4AAAYAAAAGAgAABgQAAA\n" - "YGAAAGCAAABgoAAAYMAAAGDgAACAAAAAgCAAAIBAAACAYAAAgIAAAICgAACAwAAAgOAAAKAAAACg\n" - "IAAAoEAAAKBgAACggAAAoKAAAKDAAACg4AAAwAAAAMAgAADAQAAAwGAAAMCAAADAoAAAwMAAAMDg\n" - "AADgAAAA4CAAAOBAAADgYAAA4IAAAOCgAADgwAAA4OAAQAAAAEAAIABAAEAAQABgAEAAgABAAKAA\n" - "QADAAEAA4ABAIAAAQCAgAEAgQABAIGAAQCCAAEAgoABAIMAAQCDgAEBAAABAQCAAQEBAAEBAYABA\n" - "QIAAQECgAEBAwABAQOAAQGAAAEBgIABAYEAAQGBgAEBggABAYKAAQGDAAEBg4ABAgAAAQIAgAECA\n" - "QABAgGAAQICAAECAoABAgMAAQIDgAECgAABAoCAAQKBAAECgYABAoIAAQKCgAECgwABAoOAAQMAA\n" - "AEDAIABAwEAAQMBgAEDAgABAwKAAQMDAAEDA4ABA4AAAQOAgAEDgQABA4GAAQOCAAEDgoABA4MAA\n" - "QODgAIAAAACAACAAgABAAIAAYACAAIAAgACgAIAAwACAAOAAgCAAAIAgIACAIEAAgCBgAIAggACA\n" - "IKAAgCDAAIAg4ACAQAAAgEAgAIBAQACAQGAAgECAAIBAoACAQMAAgEDgAIBgAACAYCAAgGBAAIBg\n" - "YACAYIAAgGCgAIBgwACAYOAAgIAAAICAIACAgEAAgIBgAICAgACAgKAAgIDAAICA4ACAoAAAgKAg\n" - "AICgQACAoGAAgKCAAICgoACAoMAAgKDgAIDAAACAwCAAgMBAAIDAYACAwIAAgMCgAIDAwACAwOAA\n" - "gOAAAIDgIACA4EAAgOBgAIDggACA4KAAgODAAIDg4ADAAAAAwAAgAMAAQADAAGAAwACAAMAAoADA\n" - "AMAAwADgAMAgAADAICAAwCBAAMAgYADAIIAAwCCgAMAgwADAIOAAwEAAAMBAIADAQEAAwEBgAMBA\n" - "gADAQKAAwEDAAMBA4ADAYAAAwGAgAMBgQADAYGAAwGCAAMBgoADAYMAAwGDgAMCAAADAgCAAwIBA\n" - "AMCAYADAgIAAwICgAMCAwADAgOAAwKAAAMCgIADAoEAAwKBgAMCggADAoKAAwKDAAMCg4ADAwAAA\n" - "wMAgAMDAQADAwGAAwMCAAMDAoADw+/8ApKCgAICAgAAAAP8AAP8AAAD//wD/AAAA/wD/AP//AAD/\n" - "//8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKQAAKQAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAACkpKSkpKSkpAAAAACkpKSkpKSkpKSkpKSkpKSkpKSkpKSk\n" - "pKQAAAAAAAAAAAAAAKSkpKSkpKQAAAAAAAAApKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkAAAAAAAA\n" - "AAAAAKSkpKSkpAcAAAAAAAAAB6SkpKSkpKSkpKSkpKSkpKSkpKSkpKSkAAAAAAAAAAAAAKSkpKSk\n" - "BwAAAAAACQAAAACkpKSkpKSkpKSkpKSkpKSkpKQAAACkAAAAAAAAAAAAAKSkpKQHAAAAAAAJCQkA\n" - "AAAHpKSkpKSkpKSkpKSkpKSkpKQAAACkAAAAAAAAAAAAAKSkpAcAAAAAAAkJCQkJAAAAAAekpKSk\n" - "pKSkpKSkpKSkpKQAAACkAAAAAAAAAAAAAKSkBwAAAAAACQkJCQkJCQAAAAAHpKSkpKSkpKSkpKSk\n" - "pKSkpKSkAAAAAAAAAAAAAKQAAAAAAAAJCQkJCQkJCQkAAAAAB6SkpKSkpKSkpKSkpKSkpKSkAAAA\n" - "AAAAAAAAAKQAAAAAAAkJCQkJCQkJCQkJAAAAAAekpKSkpKSkpKSkpKSkpKSkAAAAAAAAAAAApAAA\n" - "AAAACQkJCQkJCQkJCQkAAAAAAAAHpKSkpKSkpKSkpKSkpKSkAAAAAAAAAACkAAAAAAAJCQkJCQkJ\n" - "CQkJCQAAAAAAAAAAB6SkpKSkpKSkpKSkpKSkAAAAAAAAAKQAAAAAAAkJCQkJCQkJCQkJAAAAAAAA\n" - "AAAAAAekpKSkpKSkpKSkpKSkAAAAAAAApAAAAAAACQkJCQkJCQkJCQkAAAAAAAAAAAAAAAAHpKSk\n" - "pKSkpKSkpKSkAAAAAACkAAAAAAAJCQkJCQkJCQkJCQAApAAApAAAAAAAAAAAB6SkpKSkpKSkpKSk\n" - "AAAAAAAAAAAAAAkJCQkJCQkJCQkJAACkAACkAAAAAAAAAAAAAAekpKSkpKSkpKSkAAAAAAAAAAAA\n" - "CQkJCQkJCQkJCQkAAKQAAKQAAAAAAAAAAAAAAAAHpKSkpKSkpKSkAAAAAACkAAAAAAkJCQkJCQkJ\n" - "CQAApAAApAAAAKQAAAAAAAAAAAAAB6SkpKSkpKSkAAAAAAAApAAAAAAJCQkJCQkJAACkAACkAAAA\n" - "pACkAAAAAAAAAAAAAAekpKSkpKSkAAAAAAAAAACkAAAACQkJCQkAAKQAAKQAAACkAKQAAAAAAAAA\n" - "AAAAAAAHpKSkpKSkAAAAAAAAAAAAAAAAAAkJCQAAAAAApAAAAAAApAAAAAAAAADAAAAAAAAAB6Sk\n" - "pKSkAAAAAAAAAAAAAAAAAAAJAAAAAACkAAAKpKQAAAAAAAAAAMAAAAAAAAAAAKSkpKSkAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAKQAAACkAKSkAAAAAAAAwAAAAAAAAAAAB6SkpKSkAAAAAAAAAAAAAKQAAAAA\n" - "AAAAAAAAAACkpACkAAAAAADAAAAAAAAAAAAHpKSkpKSkAAAAAAAAAAAAAKSkBwAAAAAAAAAAAAAA\n" - "pKQKAAAAAMDAAAAAAAAAAAekpKSkpKSkAAAAAAAAAAAAAKSkpAcAAAAAAAAAAAAAAAAAAAAAwAAA\n" - "AAAAAAAAB6SkpKSkpKSkAAAAAAAAAAAAAKSkpKQHAAAAAAAAAAAAAAAAAADAAAAACqQAAAAHpKSk\n" - "pKSkpKSkAAAAAAAAAAAAAKSkpKSkBwAAAAAAAAAAAAAAAMDAAAAKpAoAAAekpKSkpKSkpKSkAAAA\n" - "AAAAAAAAAKSkpKSkpAcAAAAAAAAAAAAAAAAAAAqkpAAAB6SkpKSkpKSkpKSkAAAAAAAAAAAAAKSk\n" - "pKSkpKQHAAAAAAAAAAAAAAAAAKQKAAD3pKSkpKSkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9wAA\n" - "AAAAwADAAAAAAAAAAPf39/f396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/cAAAAAAMDAwAAA\n" - "AAAAB6SkpPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f3AAAAwAAAwAAAAACkpKSkpPf3\n" - "96SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f39wAAAAAAwAAAAPekpKSkpPf396SkpKSkpKSk\n" - "AAAAAAAAAAAAAKSkpKSkpKSk9/f39/cAAAAAAAAA9/ekpKSkpPf396SkpKSkpKSkAAAAAAAAAAAA\n" - "AKSkpKSkpKSk9/f39/f3AAAAAAD39/ekpKSkpPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk\n" - "9/f39/f39wAAAPf39/ekpKSkpPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f39/f39/cA\n" - "9/f39/ekpKSkpPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f39/f39/f39/f39/ekpKSk\n" - "pPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f39/f39/f39/f39/ekpKSkpPf396SkpKSk\n" - "pKQAAAAAAAAAAAAAAKSkpKSkpKSk9/f39/f39/f39/f39/ekpKSkpPf396SkpKSkpAAA9gAAAAAA\n" - "AAAAAACkpKSkpKSk9/f39/f39/f39/f39/f39/f39/f396SkpKSkAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA////////AAD///////8AAP///////wAA////////\n" - "AAD//w////8AAP8AAAAADwAA/gAAAAAHAAD+AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAAcA\n" - "AP4AAAAABwAA/gAAAAAHAAD+AAAAAAcAAPwAAAAABwAA+AAAAAAHAADwAAAAAAcAAOAAAAAABwAA\n" - "wAAAAAAHAADAAAAAAAcAAMAAAAAABwAAwAAAAAAHAADgAAAAAAcAAPgAAAAABwAA/AAAAAAHAAD+\n" - "AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAAcAAP4A\n" - "AAAABwAA/gAAAAAHAAD+AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAAcAAP4AAAAABwAA/gAA\n" - "AAAHAAD+AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAA8AAP4AAAAAGwAA/wAAAAA/AAD/////\n" - "//sAAP///////wAA\n" - ).decode("base64_codec"), - "mc6a.ico": ( - "AAABAAEAICAQAAAAAADoAgAAFgAAACgAAAAgAAAAQAAAAAEABAAAAAAAgAIAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAACAAACAAAAAgIAAgAAAAIAAgACAgAAAgICAAMDAwAAAAP8AAP8AAAD//wD/AAAA\n" - "/wD/AP//AAD///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAABwAHAAAAAAAAAAAAAAAAAHAAAAAAAAAAAAAAAAAAAAcABg\n" - "AAAAAAAAAAAAAAAABwAGZgAAAAAAAAAAAAAAAHAAZmZgAHAAAACgAAAAAAcABmZmZgAHAAAAqgAA\n" - "AABwAGZmZmAAAHAAAKqgAAAHAAZmZmYAAAAHAACqqgAAcABmZmZgAAqqqqqqoqqgAAAGZmZmAHB6\n" - "qqqqqqIqqgAAZmZmYAcHCqIiIiIiIqqgAAZmZgBwcHqiIiIiIiIqqnAAZmAHBwcKoiIiIiIiKqoA\n" - "AAYAAHBweqIiIiIiIqqgAAAAAAcHdwqqqqqqoiqqAAAAAAAAAHcKqqqqqqKqoAAAAHAAAAAAAEAA\n" - "AHCqqgAAAAAHAAAAAAwAcAcAqqAAAAAAAHAAAAAAB3BwAKoAAAAAAAAHAADEAAAHAACgAAAAAAAA\n" - "AHAATMAAcAAAAAAAAAAAAAAHAADABwAAAAAAAAAAAAAAAHAAAHAAAAAAAAAAAAAAAAAHAAcAAAAA\n" - "AAAAAAAAAAAAAHBwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" - "AAAAAAAAAAAAAAD//////////////////////wf///4H///8A///+AH///AAf3/gAD8/wAAfH4AA\n" - "Dw8AAAAHAAAAAwAAAAEAAAAAAAAAAMAAAAHgAAAD8AAAB/AAAQ/4AAMf/AAHP/4AD3//AB///4A/\n" - "///Af///4P////H//////////////////w==\n" - ).decode("base64_codec"), -} +resources = { + "mc4.ico": ( + "AAABAAIAICAQAAAAAADoAgAAJgAAADAwAAEAAAAAqA4AAA4DAAAoAAAAIAAAAEAAAAABAAQAAAAA\n" + "AIACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAgAAAAICAAIAAAACAAIAAgIAAAICAgADAwMAA\n" + "AAD/AAD/AAAA//8A/wAAAP8A/wD//wAA////AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAcABwAAAAAAAAAAAAAAAABwAABwAAAAAAAAAAAAAAAHAAYAB3d3d3dwAAAAAAAAcABm\n" + "YAB3d3dwAAAAAAAABwAGZmYAB3d3AAAAAAAAAHAAZmZmYAB4iIiIiIiAAAcABmZmZgAAB/+P+q/4\n" + "gABwAGZmZmAAAABu7u7u6IAHAAZmZmYAAAAABu7u7u6AAABmZmZgBwcAAABu7u7ugAAGZmZmAHBw\n" + "AAAABu7u7oAAAGZmYAcHBwAAAABu7u6ABwAGZgBwcHBwAAAABu7ugAAAAGAABwcHAABAAADu7oAA\n" + "AAAAAHB3cAAEAAAG7u6AAAAAAAAAB3AAwAAAbu7ugAAABwAAAAAABAAABu7u7oAAAAhgAAAAAMAH\n" + "AG7u7u6AAAAI5gAAAAAAdwbu7u7ugAAACO5gAAxAAABu7u7u7oAAAAju5gAEzAAG7u7u7u6AAAAI\n" + "7u5gAAwAbu7u7u7ugAAACO7u5gAABu7u7u7u7oAAAAju7u5gAG7u7u7u7u6AAAAI7u7u5gbu7u7u\n" + "7u7ugAAACI7u7u7u7u7u7u7u6IAAAAiIiIiIiIiIiIiIiIiAAAAIiIiIiIiIiIiIiIiIgAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA////////////g////wH///4AAH/8AAH/+AAD//AA\n" + "AAHgAAABwAAAAYAAAAGAAAABgAAAAYAAAAGAAAAB4AAAAfAAAAH4AAAB+AAAAfgAAAH4AAAB+AAA\n" + "AfgAAAH4AAAB+AAAAfgAAAH4AAAB+AAAAfgAAAH4AAAB//////////8oAAAAMAAAAGAAAAABAAgA\n" + "AAAAAIAKAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAgAAAAICAAIAAAACAAIAAgIAAAMDAwADA\n" + "3MAA8MqmAAAgQAAAIGAAACCAAAAgoAAAIMAAACDgAABAAAAAQCAAAEBAAABAYAAAQIAAAECgAABA\n" + "wAAAQOAAAGAAAABgIAAAYEAAAGBgAABggAAAYKAAAGDAAABg4AAAgAAAAIAgAACAQAAAgGAAAICA\n" + "AACAoAAAgMAAAIDgAACgAAAAoCAAAKBAAACgYAAAoIAAAKCgAACgwAAAoOAAAMAAAADAIAAAwEAA\n" + "AMBgAADAgAAAwKAAAMDAAADA4AAA4AAAAOAgAADgQAAA4GAAAOCAAADgoAAA4MAAAODgAEAAAABA\n" + "ACAAQABAAEAAYABAAIAAQACgAEAAwABAAOAAQCAAAEAgIABAIEAAQCBgAEAggABAIKAAQCDAAEAg\n" + "4ABAQAAAQEAgAEBAQABAQGAAQECAAEBAoABAQMAAQEDgAEBgAABAYCAAQGBAAEBgYABAYIAAQGCg\n" + "AEBgwABAYOAAQIAAAECAIABAgEAAQIBgAECAgABAgKAAQIDAAECA4ABAoAAAQKAgAECgQABAoGAA\n" + "QKCAAECgoABAoMAAQKDgAEDAAABAwCAAQMBAAEDAYABAwIAAQMCgAEDAwABAwOAAQOAAAEDgIABA\n" + "4EAAQOBgAEDggABA4KAAQODAAEDg4ACAAAAAgAAgAIAAQACAAGAAgACAAIAAoACAAMAAgADgAIAg\n" + "AACAICAAgCBAAIAgYACAIIAAgCCgAIAgwACAIOAAgEAAAIBAIACAQEAAgEBgAIBAgACAQKAAgEDA\n" + "AIBA4ACAYAAAgGAgAIBgQACAYGAAgGCAAIBgoACAYMAAgGDgAICAAACAgCAAgIBAAICAYACAgIAA\n" + "gICgAICAwACAgOAAgKAAAICgIACAoEAAgKBgAICggACAoKAAgKDAAICg4ACAwAAAgMAgAIDAQACA\n" + "wGAAgMCAAIDAoACAwMAAgMDgAIDgAACA4CAAgOBAAIDgYACA4IAAgOCgAIDgwACA4OAAwAAAAMAA\n" + "IADAAEAAwABgAMAAgADAAKAAwADAAMAA4ADAIAAAwCAgAMAgQADAIGAAwCCAAMAgoADAIMAAwCDg\n" + "AMBAAADAQCAAwEBAAMBAYADAQIAAwECgAMBAwADAQOAAwGAAAMBgIADAYEAAwGBgAMBggADAYKAA\n" + "wGDAAMBg4ADAgAAAwIAgAMCAQADAgGAAwICAAMCAoADAgMAAwIDgAMCgAADAoCAAwKBAAMCgYADA\n" + "oIAAwKCgAMCgwADAoOAAwMAAAMDAIADAwEAAwMBgAMDAgADAwKAA8Pv/AKSgoACAgIAAAAD/AAD/\n" + "AAAA//8A/wAAAP8A/wD//wAA////AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAKQAAKQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAApAAAAACkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACk\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKQAAAAAAAAApAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAApAAAAAAACQAAAACkpKSkpKSkpKSk\n" + "pKSkpAAAAAAAAAAAAAAAAAAAAAAAAAAAAACkAAAAAAAJCQkAAACkpKSkpKSkpKSkpKQAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAKQAAAAAAAkJCQkJAAAAAKSkpKSkpKQAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAApAAAAAAACQkJCQkJCQAAAACkpKSkpKQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACkAAAA\n" + "AAAJCQkJCQkJCQkAAAAApPf39/f39/f39/f39/f39/f39/cAAAAAAAAAAKQAAAAAAAkJCQkJCQkJ\n" + "CQkJAAAAAKQHBwf3BwcH9/f39wcHB/f39/cAAAAAAAAApAAAAAAACQkJCQkJCQkJCQkAAAAAAACk\n" + "9vb39vb29/r69/b29vf39/cAAAAAAACkAAAAAAAJCQkJCQkJCQkJCQAAAAAAAAAApPf39/f39/f3\n" + "9/f39/f39/cAAAAAAKQAAAAAAAkJCQkJCQkJCQkJAAAAAAAAAAAAAKT39/f39/f39/f39/f39/cA\n" + "AAAApAAAAAAACQkJCQkJCQkJCQkAAAAAAAAAAAAAAAD3/v7+/v7+/v7+/v739/cAAACkAAAAAAAJ\n" + "CQkJCQkJCQkJCQAApAAApAAAAAAAAAAAB/7+/v7+/v7+/v7+9/cAAAAAAAAAAAkJCQkJCQkJCQkJ\n" + "AACkAACkAAAAAAAAAAAAAAf+/v7+/v7+/v7+9/cAAAAAAAAACQkJCQkJCQkJCQkAAKQAAKQAAAAA\n" + "AAAAAAAAAAAH/v7+/v7+/v7+9/cAAACkAAAAAAkJCQkJCQkJCQAApAAApAAAAKQAAAAAAAAAAAAA\n" + "B/7+/v7+/v7+9/cAAAAApAAAAAAJCQkJCQkJAACkAACkAAAApACkAAAAAAAAAAAAAAf+/v7+/v7+\n" + "9/cAAAAAAACkAAAACQkJCQkAAKQAAKQAAACkAKQAAAAAAAAAAAAAAAAH/v7+/v7+9/cAAAAAAAAA\n" + "AAAAAAkJCQAAAAAApAAAAAAApAAAAAAAAADAAAAAAAAAB/7+/v7+9/cAAAAAAAAAAKQAAAAJAAAA\n" + "AACkAAAKpKQAAAAAAAAAAMAAAAAAAAAAAP7+/v7+9/cAAAAAAAAAAAAAAAAAAAAAAKQAAACkAKSk\n" + "AAAAAAAAwAAAAAAAAAAAB/7+/v7+9/cAAAAAAAAAAACkAAAAAAAAAAAAAACkpACkAAAAAADAAAAA\n" + "AAAAAAAH/v7+/v7+9/cAAAAAAAAAAAD3pAAAAAAAAAAAAAAApKQKAAAAAMDAAAAAAAAAAAf+/v7+\n" + "/v7+9/cAAAAAAAAAAAD39wcAAAAAAAAAAAAAAAAAAAAAwAAAAAAAAAAAB/7+/v7+/v7+9/cAAAAA\n" + "AAAAAAD39/4HAAAAAAAAAAAAAAAAAADAAAAACqQAAAAH/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+\n" + "BwAAAAAAAAAAAAAAAMDAAAAKpAoAAAf+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/gcAAAAAAAAA\n" + "AAAAAAAAAAqkpAAAB/7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v4HAAAAAAAAAAAAAAAAAKQK\n" + "AAAH/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+BwAAAAAAwADAAAAAAAAAAAf+/v7+/v7+\n" + "/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/gcAAAAAAMDAwAAAAAAAB/7+/v7+/v7+/v7+/v7+9/cA\n" + "AAAAAAAAAAD39/7+/v7+/v4HAAAAwAAAwAAAAAAH/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD3\n" + "9/7+/v7+/v7+BwAAAAAAwAAAAAf+/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/v7+\n" + "/gcAAAAAAAAAB/7+/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/v7+/v4HAAAAAAAH\n" + "/v7+/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/v7+/v7+BwAAAAf+/v7+/v7+/v7+\n" + "/v7+/v7+/v7+9/cAAAAAAAAAAAD39/7+/v7+/v7+/v7+/gcAB/7+/v7+/v7+/v7+/v7+/v7+/v7+\n" + "9/cAAAAAAAAAAAD39/7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+9/cAAAAAAAAA\n" + "AAD39/f+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v7+/v739/cAAAAAAAAAAAD3+/v79/f3\n" + "9/f39/f39/f39/f39/f39/f39/f39/f39/f39/f39/cAAAAAAAAAAAD30NDQ9/f39/f39/f39/f3\n" + "9/f39/f39/f39/f39/f396SkpKSk9/cAAAAAAAAAAAD3+ff59/f39/f39/f39/f39/f39/f39/f3\n" + "9/f39/f39/f39/f39/cAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP///////wAA////\n" + "////AAD//h////8AAP/8D////wAA//gP////AAD/8Af///8AAP/gAAAP/wAA/8AAAD//AAD/gAAB\n" + "//8AAP8AAAH//wAA/gAAAAADAAD8AAAAAAMAAPgAAAAAAwAA8AAAAAADAADgAAAAAAMAAMAAAAAA\n" + "AwAAgAAAAAADAACAAAAAAAMAAIAAAAAAAwAAgAAAAAADAADAAAAAAAMAAPAAAAAAAwAA+AAAAAAD\n" + "AAD8AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+AAAAAAMA\n" + "AP4AAAAAAwAA/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA\n" + "/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+AAAAAAMAAP4AAAAAAwAA/gAAAAADAAD+\n" + "AAAAAAMAAP///////wAA////////AAD///////8AAA==\n" + ).decode("base64_codec"), + "mc5b.ico": ( + "AAABAAEAICAQAAAAAADoAgAAFgAAACgAAAAgAAAAQAAAAAEABAAAAAAAgAIAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAACAAACAAAAAgIAAgAAAAIAAgACAgAAAgICAAMDAwAAAAP8AAP8AAAD//wD/AAAA\n" + "/wD/AP//AAD///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAcABwAAAAAAAAAAAAAAAABwAAAAAAAAAAAAAAAAAA\n" + "AHAAYAAAAAAAAAAAAAAAAAcABmYAAAAAAAAAAAAAAKBwAGZmYABwAAAAAAAAAACqAAZmZmYABwAA\n" + "AAAAAAAAqqBmZmZgAABwAAAAAAAAB6qqZmZmAAAABwAACqqqqqqiqqZmYAAAAABwAAqqqqqqoiqq\n" + "ZgBwcAAABwAKoiIiIiIiqqAHBwAAAABwCqIiIiIiIiqqcHBwAAAABwqiIiIiIiIqqgcHBwAAAAAK\n" + "oiIiIiIiqqBwcHAABAAACqqqqqqiKqoHB3cAAEAAAAqqqqqqoqqgAAB3AAwAAAcAAAAAAKqqAAAA\n" + "AABAAABwAAAAAACqpwAAAAAMAHAHAAAAAAAAqgBwAAAAAAdwcAAAAAAAAKAABwAAxAAABwAAAAAA\n" + "AAAAAABwAEzAAHAAAAAAAAAAAAAABwAAwAcAAAAAAAAAAAAAAABwAABwAAAAAAAAAAAAAAAABwAH\n" + "AAAAAAAAAAAAAAAAAABwcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAD///////////////////////8H///+B////AP///gB///QAH//wAA//8AAH/+A\n" + "AA+AAAAHgAAAA4AAAAGAAAAAgAAAAIAAAACAAAAAgAAAAP/AAAH/wAAD/8wAB//eAA///wAf//+A\n" + "P///wH///+D////x/////////////////w==\n" + ).decode("base64_codec"), + "mc2.ico": ( + "AAABAAEAMDAAAQAAAACoDgAAFgAAACgAAAAwAAAAYAAAAAEACAAAAAAAgAoAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAACAAACAAAAAgIAAgAAAAIAAgACAgAAAwMDAAMDcwADwyqYAACBAAAAgYAAAIIAA\n" + "ACCgAAAgwAAAIOAAAEAAAABAIAAAQEAAAEBgAABAgAAAQKAAAEDAAABA4AAAYAAAAGAgAABgQAAA\n" + "YGAAAGCAAABgoAAAYMAAAGDgAACAAAAAgCAAAIBAAACAYAAAgIAAAICgAACAwAAAgOAAAKAAAACg\n" + "IAAAoEAAAKBgAACggAAAoKAAAKDAAACg4AAAwAAAAMAgAADAQAAAwGAAAMCAAADAoAAAwMAAAMDg\n" + "AADgAAAA4CAAAOBAAADgYAAA4IAAAOCgAADgwAAA4OAAQAAAAEAAIABAAEAAQABgAEAAgABAAKAA\n" + "QADAAEAA4ABAIAAAQCAgAEAgQABAIGAAQCCAAEAgoABAIMAAQCDgAEBAAABAQCAAQEBAAEBAYABA\n" + "QIAAQECgAEBAwABAQOAAQGAAAEBgIABAYEAAQGBgAEBggABAYKAAQGDAAEBg4ABAgAAAQIAgAECA\n" + "QABAgGAAQICAAECAoABAgMAAQIDgAECgAABAoCAAQKBAAECgYABAoIAAQKCgAECgwABAoOAAQMAA\n" + "AEDAIABAwEAAQMBgAEDAgABAwKAAQMDAAEDA4ABA4AAAQOAgAEDgQABA4GAAQOCAAEDgoABA4MAA\n" + "QODgAIAAAACAACAAgABAAIAAYACAAIAAgACgAIAAwACAAOAAgCAAAIAgIACAIEAAgCBgAIAggACA\n" + "IKAAgCDAAIAg4ACAQAAAgEAgAIBAQACAQGAAgECAAIBAoACAQMAAgEDgAIBgAACAYCAAgGBAAIBg\n" + "YACAYIAAgGCgAIBgwACAYOAAgIAAAICAIACAgEAAgIBgAICAgACAgKAAgIDAAICA4ACAoAAAgKAg\n" + "AICgQACAoGAAgKCAAICgoACAoMAAgKDgAIDAAACAwCAAgMBAAIDAYACAwIAAgMCgAIDAwACAwOAA\n" + "gOAAAIDgIACA4EAAgOBgAIDggACA4KAAgODAAIDg4ADAAAAAwAAgAMAAQADAAGAAwACAAMAAoADA\n" + "AMAAwADgAMAgAADAICAAwCBAAMAgYADAIIAAwCCgAMAgwADAIOAAwEAAAMBAIADAQEAAwEBgAMBA\n" + "gADAQKAAwEDAAMBA4ADAYAAAwGAgAMBgQADAYGAAwGCAAMBgoADAYMAAwGDgAMCAAADAgCAAwIBA\n" + "AMCAYADAgIAAwICgAMCAwADAgOAAwKAAAMCgIADAoEAAwKBgAMCggADAoKAAwKDAAMCg4ADAwAAA\n" + "wMAgAMDAQADAwGAAwMCAAMDAoADw+/8ApKCgAICAgAAAAP8AAP8AAAD//wD/AAAA/wD/AP//AAD/\n" + "//8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKQAAKQAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAACkpKSkpKSkpAAAAACkpKSkpKSkpKSkpKSkpKSkpKSkpKSk\n" + "pKQAAAAAAAAAAAAAAKSkpKSkpKQAAAAAAAAApKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkAAAAAAAA\n" + "AAAAAKSkpKSkpAcAAAAAAAAAB6SkpKSkpKSkpKSkpKSkpKSkpKSkpKSkAAAAAAAAAAAAAKSkpKSk\n" + "BwAAAAAACQAAAACkpKSkpKSkpKSkpKSkpKSkpKQAAACkAAAAAAAAAAAAAKSkpKQHAAAAAAAJCQkA\n" + "AAAHpKSkpKSkpKSkpKSkpKSkpKQAAACkAAAAAAAAAAAAAKSkpAcAAAAAAAkJCQkJAAAAAAekpKSk\n" + "pKSkpKSkpKSkpKQAAACkAAAAAAAAAAAAAKSkBwAAAAAACQkJCQkJCQAAAAAHpKSkpKSkpKSkpKSk\n" + "pKSkpKSkAAAAAAAAAAAAAKQAAAAAAAAJCQkJCQkJCQkAAAAAB6SkpKSkpKSkpKSkpKSkpKSkAAAA\n" + "AAAAAAAAAKQAAAAAAAkJCQkJCQkJCQkJAAAAAAekpKSkpKSkpKSkpKSkpKSkAAAAAAAAAAAApAAA\n" + "AAAACQkJCQkJCQkJCQkAAAAAAAAHpKSkpKSkpKSkpKSkpKSkAAAAAAAAAACkAAAAAAAJCQkJCQkJ\n" + "CQkJCQAAAAAAAAAAB6SkpKSkpKSkpKSkpKSkAAAAAAAAAKQAAAAAAAkJCQkJCQkJCQkJAAAAAAAA\n" + "AAAAAAekpKSkpKSkpKSkpKSkAAAAAAAApAAAAAAACQkJCQkJCQkJCQkAAAAAAAAAAAAAAAAHpKSk\n" + "pKSkpKSkpKSkAAAAAACkAAAAAAAJCQkJCQkJCQkJCQAApAAApAAAAAAAAAAAB6SkpKSkpKSkpKSk\n" + "AAAAAAAAAAAAAAkJCQkJCQkJCQkJAACkAACkAAAAAAAAAAAAAAekpKSkpKSkpKSkAAAAAAAAAAAA\n" + "CQkJCQkJCQkJCQkAAKQAAKQAAAAAAAAAAAAAAAAHpKSkpKSkpKSkAAAAAACkAAAAAAkJCQkJCQkJ\n" + "CQAApAAApAAAAKQAAAAAAAAAAAAAB6SkpKSkpKSkAAAAAAAApAAAAAAJCQkJCQkJAACkAACkAAAA\n" + "pACkAAAAAAAAAAAAAAekpKSkpKSkAAAAAAAAAACkAAAACQkJCQkAAKQAAKQAAACkAKQAAAAAAAAA\n" + "AAAAAAAHpKSkpKSkAAAAAAAAAAAAAAAAAAkJCQAAAAAApAAAAAAApAAAAAAAAADAAAAAAAAAB6Sk\n" + "pKSkAAAAAAAAAAAAAAAAAAAJAAAAAACkAAAKpKQAAAAAAAAAAMAAAAAAAAAAAKSkpKSkAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAKQAAACkAKSkAAAAAAAAwAAAAAAAAAAAB6SkpKSkAAAAAAAAAAAAAKQAAAAA\n" + "AAAAAAAAAACkpACkAAAAAADAAAAAAAAAAAAHpKSkpKSkAAAAAAAAAAAAAKSkBwAAAAAAAAAAAAAA\n" + "pKQKAAAAAMDAAAAAAAAAAAekpKSkpKSkAAAAAAAAAAAAAKSkpAcAAAAAAAAAAAAAAAAAAAAAwAAA\n" + "AAAAAAAAB6SkpKSkpKSkAAAAAAAAAAAAAKSkpKQHAAAAAAAAAAAAAAAAAADAAAAACqQAAAAHpKSk\n" + "pKSkpKSkAAAAAAAAAAAAAKSkpKSkBwAAAAAAAAAAAAAAAMDAAAAKpAoAAAekpKSkpKSkpKSkAAAA\n" + "AAAAAAAAAKSkpKSkpAcAAAAAAAAAAAAAAAAAAAqkpAAAB6SkpKSkpKSkpKSkAAAAAAAAAAAAAKSk\n" + "pKSkpKQHAAAAAAAAAAAAAAAAAKQKAAD3pKSkpKSkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9wAA\n" + "AAAAwADAAAAAAAAAAPf39/f396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/cAAAAAAMDAwAAA\n" + "AAAAB6SkpPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f3AAAAwAAAwAAAAACkpKSkpPf3\n" + "96SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f39wAAAAAAwAAAAPekpKSkpPf396SkpKSkpKSk\n" + "AAAAAAAAAAAAAKSkpKSkpKSk9/f39/cAAAAAAAAA9/ekpKSkpPf396SkpKSkpKSkAAAAAAAAAAAA\n" + "AKSkpKSkpKSk9/f39/f3AAAAAAD39/ekpKSkpPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk\n" + "9/f39/f39wAAAPf39/ekpKSkpPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f39/f39/cA\n" + "9/f39/ekpKSkpPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f39/f39/f39/f39/ekpKSk\n" + "pPf396SkpKSkpKSkAAAAAAAAAAAAAKSkpKSkpKSk9/f39/f39/f39/f39/ekpKSkpPf396SkpKSk\n" + "pKQAAAAAAAAAAAAAAKSkpKSkpKSk9/f39/f39/f39/f39/ekpKSkpPf396SkpKSkpAAA9gAAAAAA\n" + "AAAAAACkpKSkpKSk9/f39/f39/f39/f39/f39/f39/f396SkpKSkAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA////////AAD///////8AAP///////wAA////////\n" + "AAD//w////8AAP8AAAAADwAA/gAAAAAHAAD+AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAAcA\n" + "AP4AAAAABwAA/gAAAAAHAAD+AAAAAAcAAPwAAAAABwAA+AAAAAAHAADwAAAAAAcAAOAAAAAABwAA\n" + "wAAAAAAHAADAAAAAAAcAAMAAAAAABwAAwAAAAAAHAADgAAAAAAcAAPgAAAAABwAA/AAAAAAHAAD+\n" + "AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAAcAAP4A\n" + "AAAABwAA/gAAAAAHAAD+AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAAcAAP4AAAAABwAA/gAA\n" + "AAAHAAD+AAAAAAcAAP4AAAAABwAA/gAAAAAHAAD+AAAAAA8AAP4AAAAAGwAA/wAAAAA/AAD/////\n" + "//sAAP///////wAA\n" + ).decode("base64_codec"), + "mc6a.ico": ( + "AAABAAEAICAQAAAAAADoAgAAFgAAACgAAAAgAAAAQAAAAAEABAAAAAAAgAIAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAACAAACAAAAAgIAAgAAAAIAAgACAgAAAgICAAMDAwAAAAP8AAP8AAAD//wD/AAAA\n" + "/wD/AP//AAD///8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAABwAHAAAAAAAAAAAAAAAAAHAAAAAAAAAAAAAAAAAAAAcABg\n" + "AAAAAAAAAAAAAAAABwAGZgAAAAAAAAAAAAAAAHAAZmZgAHAAAACgAAAAAAcABmZmZgAHAAAAqgAA\n" + "AABwAGZmZmAAAHAAAKqgAAAHAAZmZmYAAAAHAACqqgAAcABmZmZgAAqqqqqqoqqgAAAGZmZmAHB6\n" + "qqqqqqIqqgAAZmZmYAcHCqIiIiIiIqqgAAZmZgBwcHqiIiIiIiIqqnAAZmAHBwcKoiIiIiIiKqoA\n" + "AAYAAHBweqIiIiIiIqqgAAAAAAcHdwqqqqqqoiqqAAAAAAAAAHcKqqqqqqKqoAAAAHAAAAAAAEAA\n" + "AHCqqgAAAAAHAAAAAAwAcAcAqqAAAAAAAHAAAAAAB3BwAKoAAAAAAAAHAADEAAAHAACgAAAAAAAA\n" + "AHAATMAAcAAAAAAAAAAAAAAHAADABwAAAAAAAAAAAAAAAHAAAHAAAAAAAAAAAAAAAAAHAAcAAAAA\n" + "AAAAAAAAAAAAAHBwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\n" + "AAAAAAAAAAAAAAD//////////////////////wf///4H///8A///+AH///AAf3/gAD8/wAAfH4AA\n" + "Dw8AAAAHAAAAAwAAAAEAAAAAAAAAAMAAAAHgAAAD8AAAB/AAAQ/4AAMf/AAHP/4AD3//AB///4A/\n" + "///Af///4P////H//////////////////w==\n" + ).decode("base64_codec"), +} diff --git a/lzari.py b/lzari.py index f80f6f0..de71086 100755 --- a/lzari.py +++ b/lzari.py @@ -1,789 +1,789 @@ -# -# lzari.py -# -# By Ross Ridge -# - -""" -Implementation of Haruhiko Okumura's LZARI data compression algorithm -in Python. Largely based on LZARI.C, one key difference is the use of -a two level dicitionary look up during compression rather than -LZARI.C's binary search tree. -""" - -_SCCS_ID = "@(#) mysc lzari.py 1.6 12/10/04 19:07:53\n" - -import sys -import array -import binascii -import string -import time -from bisect import bisect_right -from math import log - -try: - import ctypes - import mymcsup -except ImportError: - mymcsup = None - -hexlify = binascii.hexlify - -__ALL__ = ['lzari_codec', 'string_to_bit_array', 'bit_array_to_string'] - -# -# Fundamental constants of the LZARI compression alogorithm. -# -# Changing any of these values will create an incompatible implementation. -# - -HIST_LEN = 4096 -MIN_MATCH_LEN = 3 -MAX_MATCH_LEN = 60 - -ARITH_BITS = 15 -QUADRANT1 = 1 << ARITH_BITS -QUADRANT2 = QUADRANT1 * 2 -QUADRANT3 = QUADRANT1 * 3 -QUADRANT4 = QUADRANT1 * 4 -MAX_CUM = QUADRANT1 - 1 -MAX_CHAR = (256 + MAX_MATCH_LEN - MIN_MATCH_LEN + 1) - -# -# Other constants specific to this implementation -# - -MAX_SUFFIX_CHAIN = 50 # limit on how many identical suffixes to try to match - -#def debug(value, msg): -# print "@@@ %s %04x" % (msg, value) -debug = lambda value, msg: None - -_tr_16 = string.maketrans("0123456789abcdef", - "\x00\x01\x02\x03" - "\x10\x11\x12\x13" - "\x20\x21\x22\x23" - "\x30\x31\x32\x33") -_tr_4 = string.maketrans("0123", - "\x00\x01" - "\x10\x11") -_tr_2 = string.maketrans("01", "\x00\x01") - -def string_to_bit_array(s): - """Convert a string to an array containing a sequence of bits.""" - s = binascii.hexlify(s).translate(_tr_16) - s = binascii.hexlify(s).translate(_tr_4) - s = binascii.hexlify(s).translate(_tr_2) - a = array.array('B', s) - return a - -_tr_rev_2 = string.maketrans("\x00\x01", "01") -_tr_rev_4 = string.maketrans("\x00\x01" - "\x10\x11", - "0123") -_tr_rev_16 = string.maketrans("\x00\x01\x02\x03" - "\x10\x11\x12\x13" - "\x20\x21\x22\x23" - "\x30\x31\x32\x33", - "0123456789abcdef") -def bit_array_to_string(a): - """Convert an array containing a sequence of bits to a string.""" - remainder = len(a) % 8 - if remainder != 0: - a.fromlist([0] * (8 - remainder)) - s = a.tostring() - s = binascii.unhexlify(s.translate(_tr_rev_2)) - s = binascii.unhexlify(s.translate(_tr_rev_4)) - return binascii.unhexlify(s.translate(_tr_rev_16)) - -def _match(src, pos, hpos, mlen, end): - mlen += 1 - if not src.startswith(src[hpos : hpos + mlen], pos): - return None - for i in range(mlen, end): - if src[pos + i] != src[hpos + i]: - return i - return end - -def _rehash_table2(src, chars, head, next, next2, hist_invalid): - p = head - table2 = {} - l = [] - while p > hist_invalid: - l.append(p) - p = next[p % HIST_LEN] - l.reverse() - for p in l: - p2 = p + MIN_MATCH_LEN - key2 = src[p2 : p2 + chars] - head2 = table2.get(key2, hist_invalid) - next2[p % HIST_LEN] = head2 - table2[key2] = p - return table2 - -class lzari_codec(object): - # despite the name this does not implement a codec compatible - # with Python's codec system - - def init(self, decode): - self.high = QUADRANT4 - self.low = 0 - if decode: - self.code = 0 - # reverse the order of sym_cum so bisect_right() can - # be used for faster searching - self.sym_cum = range(0, MAX_CHAR + 1) - else: - self.shifts = 0 - self.char_to_symbol = range(1, MAX_CHAR + 1) - self.sym_cum = range(MAX_CHAR, -1, -1) - self.next_table = [None] * HIST_LEN - self.next2_table = [None] * HIST_LEN - self.suffix_table = {} - - self.symbol_to_char = [0] + range(MAX_CHAR) - self.sym_freq = [0] + [1] * MAX_CHAR - self.position_cum = [0] * (HIST_LEN + 1) - a = 0 - for i in range(HIST_LEN, 0, -1): - a = a + 10000 / (200 + i) - self.position_cum[i - 1] = a - - def search(self, table, x): - c = 1 - s = len(table) - 1 - while True: - a = (s + c) / 2 - if table[a] <= x: - s = a - else: - c = a + 1 - if c >= s: - break - return c - - def update_model_decode(self, symbol): - # A compatible implemention to the one used while compressing. - - sym_freq = self.sym_freq - sym_cum = self.sym_cum - - if self.sym_cum[MAX_CHAR] >= MAX_CUM: - c = 0 - for i in range(MAX_CHAR, 0, -1): - self.sym_cum[MAX_CHAR - i] = c - a = (self.sym_freq[i] + 1) / 2 - self.sym_freq[i] = a - c += a - self.sym_cum[MAX_CHAR] = c - freq = sym_freq[symbol] - new_symbol = symbol - while self.sym_freq[new_symbol - 1] == freq: - new_symbol -= 1 - # new_symbol = sym_freq.index(freq) - if new_symbol != symbol: - symbol_to_char = self.symbol_to_char - swap_char = symbol_to_char[new_symbol] - char = symbol_to_char[symbol] - symbol_to_char[new_symbol] = char - symbol_to_char[symbol] = swap_char - sym_freq[new_symbol] = freq + 1 - for i in range(MAX_CHAR - new_symbol + 1, MAX_CHAR + 1): - sym_cum[i] += 1 - - def update_model_encode(self, symbol): - sym_freq = self.sym_freq - sym_cum = self.sym_cum - - if sym_cum[0] >= MAX_CUM: - c = 0 - for i in range(MAX_CHAR, 0, -1): - sym_cum[i] = c - a = (sym_freq[i] + 1) / 2 - sym_freq[i] = a - c += a - sym_cum[0] = c - freq = sym_freq[symbol] - new_symbol = symbol - while sym_freq[new_symbol - 1] == freq: - new_symbol -= 1 - if new_symbol != symbol: - debug(new_symbol, "a") - swap_char = self.symbol_to_char[new_symbol] - char = self.symbol_to_char[symbol] - self.symbol_to_char[new_symbol] = char - self.symbol_to_char[symbol] = swap_char - self.char_to_symbol[char] = new_symbol - self.char_to_symbol[swap_char] = symbol - sym_freq[new_symbol] += 1 - for i in range(new_symbol): - sym_cum[i] += 1 - - def decode_char(self): - high = self.high - low = self.low - code = self.code - sym_cum = self.sym_cum - - _range = high - low - max_cum_freq = sym_cum[MAX_CHAR] - n = ((code - low + 1) * max_cum_freq - 1) / _range - i = bisect_right(sym_cum, n, 1) - high = low + sym_cum[i] * _range / max_cum_freq - low += sym_cum[i - 1] * _range / max_cum_freq - symbol = MAX_CHAR + 1 - i - - while True: - if low < QUADRANT2: - if low < QUADRANT1 or high > QUADRANT3: - if high > QUADRANT2: - break - else: - low -= QUADRANT1 - code -= QUADRANT1 - high -= QUADRANT1 - else: - low -= QUADRANT2 - code -= QUADRANT2 - high -= QUADRANT2 - low *= 2 - high *= 2 - code = code * 2 + self.in_iter() - - ret = self.symbol_to_char[symbol] - self.high = high - self.low = low - self.code = code - self.update_model_decode(symbol) - return ret - - def decode_position(self): - _range = self.high - self.low - max_cum = self.position_cum[0] - pos = self.search(self.position_cum, - ((self.code - self.low + 1) - * max_cum - 1) / _range) - 1 - self.high = (self.low + - self.position_cum[pos] * _range / max_cum) - self.low += self.position_cum[pos + 1] * _range / max_cum - while True: - if self.low < QUADRANT2: - if (self.low < QUADRANT1 - or self.high > QUADRANT3): - if self.high > QUADRANT2: - return pos - else: - self.low -= QUADRANT1 - self.code -= QUADRANT1 - self.high -= QUADRANT1 - else: - self.low -= QUADRANT2 - self.code -= QUADRANT2 - self.high -= QUADRANT2 - self.low *= 2 - self.high *= 2 - self.code = self.in_iter() + self.code * 2 - - def add_suffix_1(self, pos, find): - # naive implemention used for testing - - if not find: - return (None, 0) - src = self.src - mlen = min(1000, self.max_match, len(src) - pos) - hist_start = max(pos - HIST_LEN, 0) - while mlen >= MIN_MATCH_LEN: - i = src.rfind(src[pos : pos + mlen], hist_start, pos) - if i != -1: - assert (src[pos : pos + mlen] - == src[i: i + mlen]) - return (i, mlen) - mlen -= 1 - return (None, -1) - - def add_suffix_2(self, pos, find): - # a two level dictionary look up that leverages Python's - # built-in dicts to get something that's hopefully faster - # than implementing binary trees in completely in Python. - - src = self.src - suffix_table = self.suffix_table - max_match = min(self.max_match, len(src) - pos) - - mlen = -1 - mpos = None - - hist_invalid = pos - HIST_LEN - 1 - modpos = pos % HIST_LEN - pos2 = pos + MIN_MATCH_LEN - - key = src[pos : pos2] - a = suffix_table.get(key) - if a != None: - next = self.next_table - next2 = self.next2_table - - [count, head, table2, chars] = a - - pos3 = pos2 + chars - key2 = src[pos2 : pos3] - min_match2 = MIN_MATCH_LEN + chars - if find: - p = table2.get(key2, hist_invalid) - maxmlen = max_match - min_match2 - while p > hist_invalid and mlen != maxmlen: - p3 = p + min_match2 - if mpos == None and p3 <= pos: - mpos = p - mlen = 0 - if p3 >= pos: - p = next2[p % HIST_LEN] - continue - rlen = _match(src, pos3, p3, mlen, - min(maxmlen, pos - p3)) - if rlen != None: - mpos = p - mlen = rlen - p = next2[p % HIST_LEN] - if mpos != None: - mlen += min_match2 - elif find: - p = head - maxmlen = min(chars, max_match - MIN_MATCH_LEN) - i = 0 - while (p > hist_invalid and i < 50000 - and mlen < maxmlen): - assert i < count - i += 1 - p2 = p + MIN_MATCH_LEN - l2 = pos - p2 - if mpos == None and l2 >= 0: - mpos = p - mlen = 0 - if l2 <= 0: - p = next[p % HIST_LEN] - continue - if l2 > maxmlen: - l2 = maxmlen - m = mlen + 1 - if src.startswith(src[p2 : p2 + m], - pos2): - mpos = p - for j in range(m, l2): - if (src[pos2 + j] - != src[p2 + j]): - mlen = j - break - else: - mlen = l2 - #rlen = _match(src, pos2, p2, mlen, l2) - #if rlen != None: - # mpos = p - # mlen = rlen - p = next[p % HIST_LEN] - - if mpos != None: - mlen += MIN_MATCH_LEN - - count += 1 - new_chars = int(log(count, 2)) - # new_chars = 50 - new_chars = min(new_chars, max_match - MIN_MATCH_LEN) - if new_chars > chars: - chars = new_chars - table2 = _rehash_table2(src, chars, head, - next, next2, - hist_invalid) - - next[modpos] = head - head = pos - - key2 = src[pos2 : pos2 + chars] - head2 = table2.get(key2, hist_invalid) - next2[modpos] = head2 - table2[key2] = pos - - a[0] = count - a[1] = head - a[2] = table2 - a[3] = chars - else: - self.next_table[modpos] = hist_invalid - self.next2_table[modpos] = hist_invalid - key2 = "" - # key2 = src[pos2 : pos2 + 1] - suffix_table[key] = [1, pos, {key2: pos}, len(key2)] - - p = pos - HIST_LEN - if p >= 0: - p2 = p + MIN_MATCH_LEN - key = src[p : p2] - a = suffix_table[key] - (count, head, table2, chars) = a - count -= 1 - if count == 0: - assert head == p - del suffix_table[key] - else: - key2 = src[p2 : p2 + chars] - if table2[key2] == p: - del table2[key2] - a[0] = count - assert (mpos == None - or src[pos : pos + mlen] == src[mpos : mpos + mlen]) - return (mpos, mlen) - - def _add_suffix(self, pos, find): - r = self.add_suffix_2(pos, find) - start_pos = self.start_pos - if find and r[0] != None: - print ("%4d %02x %4d %2d" - % (pos - start_pos, ord(self.src[pos]), - r[0] - start_pos, r[1])) - else: - print ("%4d %02x" - % (pos - start_pos, ord(self.src[pos]))) - return r - - add_suffix = add_suffix_2 - - def output_bit(self, bit): - self.append_bit(bit) - bit ^= 1 - for i in range(self.shifts): - self.append_bit(bit) - self.shifts = 0 - - def encode_char(self, char): - low = self.low - high = self.high - sym_cum = self.sym_cum - - symbol = self.char_to_symbol[char] - range = high - low - - high = low + range * sym_cum[symbol - 1] / sym_cum[0] - low += range * sym_cum[symbol] / sym_cum[0] - debug(high, "high"); - debug(low, "low"); - while True: - if high <= QUADRANT2: - self.output_bit(0) - elif low >= QUADRANT2: - self.output_bit(1) - low -= QUADRANT2 - high -= QUADRANT2 - elif low >= QUADRANT1 and high <= QUADRANT3: - self.shifts += 1 - low -= QUADRANT1 - high -= QUADRANT1 - else: - break - low *= 2 - high *= 2 - self.low = low - self.high = high - self.update_model_encode(symbol) - - def encode_position(self, position): - position_cum = self.position_cum - low = self.low - high = self.high - - range = high - low - high = low + range * position_cum[position] / position_cum[0] - low += range * position_cum[position + 1] / position_cum[0] - - debug(high, "high"); - debug(low, "low"); - while True: - if high <= QUADRANT2: - self.output_bit(0) - elif low >= QUADRANT2: - self.output_bit(1) - low -= QUADRANT2 - high -= QUADRANT2 - elif low >= QUADRANT1 and high <= QUADRANT3: - self.shifts += 1 - low -= QUADRANT1 - high -= QUADRANT1 - else: - break - low *= 2 - high *= 2 - - self.low = low - self.high = high - - def encode(self, src, progress = None): - """Compress a string.""" - - length = len(src) - if length == 0: - return "" - - out_array = array.array('B') - self.out_array = out_array - self.append_bit = out_array.append - - self.init(False) - - max_match = min(MAX_MATCH_LEN, length) - self.max_match = max_match - self.src = src = "\x20" * max_match + src - - in_length = len(src) - - self.start_pos = max_match - - for in_pos in range(max_match): - self.add_suffix(in_pos, False) - in_pos += 1 - last_percent = -1 - while in_pos < in_length: - if progress: - percent = (in_pos - max_match) * 100 / length - if percent != last_percent: - sys.stderr.write("%s%3d%%\r" - % (progress, percent)) - last_percent = percent - debug(ord(src[in_pos]), "src") - (match_pos, match_len) = self.add_suffix(in_pos, True) - if match_len < MIN_MATCH_LEN: - self.encode_char(ord(src[in_pos])) - else: - debug(in_pos - match_pos - 1, "match_pos") - debug(match_len, "match_len") - self.encode_char(256 - MIN_MATCH_LEN - + match_len) - self.encode_position(in_pos - match_pos - 1) - for i in range(match_len - 1): - in_pos += 1 - self.add_suffix(in_pos, False) - in_pos += 1 - - self.shifts += 1 - if self.low < QUADRANT1: - self.output_bit(0) - else: - self.output_bit(1) - - #for k, v in sorted(self.suffix_table.items()): - # count, head, table2, chars = v - # print hexlify(k), count, head, len(table2), chars - - if progress: - sys.stderr.write("%s100%%\n" % progress) - - return bit_array_to_string(out_array) - - def decode(self, src, out_length, progress = None): - """Decompress a string.""" - - a = string_to_bit_array(src) - a.fromlist([0] * 32) # add some extra bits - self.in_iter = iter(a).next - - out = array.array('B', "\0") * out_length - outpos = 0 - - self.init(True) - - self.code = 0 - for i in range(ARITH_BITS + 2): - self.code += self.code + self.in_iter() - - hist_pos = HIST_LEN - MAX_MATCH_LEN - history = [0x20] * hist_pos + [0] * MAX_MATCH_LEN - - decode_char = self.decode_char - last_percent = -1 - last_time = time.time() - while outpos < out_length: - if progress: - percent = outpos * 100 / out_length - if percent != last_percent: - now = time.time() - if now - last_time >= 1: - sys.stderr.write("%s%3d%%\r" - % (progress, percent)) - last_percent = percent - last_time = now - char = decode_char() - if char >= 0x100: - pos = self.decode_position() - length = char - 0x100 + MIN_MATCH_LEN - base = (hist_pos - pos - 1) % HIST_LEN - for off in range(length): - a = history[(base + off) % HIST_LEN] - out[outpos] = a - outpos += 1 - history[hist_pos] = a - hist_pos = (hist_pos + 1) % HIST_LEN - else: - out[outpos] = char - outpos += 1 - history[hist_pos] = char - hist_pos = (hist_pos + 1) % HIST_LEN - - self.in_iter = None - if progress: - sys.stderr.write("%s100%%\n" % progress) - return out.tostring() - -if mymcsup == None: - def decode(src, out_length, progress = None): - return lzari_codec().decode(src, out_length, progress) - - def encode(src, progress = None): - return lzari_codec().encode(src, progress) -else: - mylzari_decode = mymcsup.mylzari_decode - mylzari_encode = mymcsup.mylzari_encode - mylzari_free_encoded = mymcsup.mylzari_free_encoded - - def decode(src, out_length, progress = None): - out = ctypes.create_string_buffer(out_length) - if (mylzari_decode(src, len(src), out, out_length, progress) - == -1): - raise ValueError, "compressed input is corrupt" - return ctypes.string_at(out, out_length) - - def encode(src, progress = None): - (r, compressed, comp_len) = mylzari_encode(src, len(src), - progress) - # print r, compressed.value, comp_len - if r == -1: - raise MemoryError, "out of memory during compression" - if compressed.value == None: - return "" - ret = ctypes.string_at(compressed.value, comp_len.value) - mylzari_free_encoded(compressed) - return ret; - -def main2(args): - import struct - import os - - src = file(args[2], "rb").read() - lzari = lzari_codec() - out = file(args[3], "wb") - start = os.times() - if args[1] == "c": - dest = lzari.encode(src) - now = os.times() - out.write(struct.pack("L", len(src))) - else: - dest = lzari.decode(src[4:], - struct.unpack("L", src[:4])[0]) - now = os.times() - out.write(dest) - out.close() - print "time:", now[0] - start[0], now[1] - start[1], now[4] - start[4] - - -def _get_hotshot_lineinfo(filename): - import hotshot.log - log = hotshot.log.LogReader(filename) - timings = {} - for what, loc, tdelta in log: - if what == hotshot.log.LINE: - a = timings.get(loc) - if a == None: - timings[loc] = [1, tdelta] - else: - a[0] += 1 - a[1] += tdelta - return timings.items() - -def _dump_hotshot_lineinfo(log): - a = sorted(_get_hotshot_lineinfo(log)) - total_count = sum((time[0] - for (loc, time) in a)) - total_time = sum((time[1] - for (loc, time) in a)) - for (loc, [count, time]) in a: - print ("%8d %6.3f%% %8d %6.3f%%" - % (time, time * 100.0 / total_time, - count, count * 100.0 / total_count)), - print "%s:%d(%s)" % loc - -def _dump_hotshot_lineinfo2(log): - cur = None - a = sorted(_get_hotshot_lineinfo(log)) - total_count = sum((time[0] - for (loc, time) in a)) - total_time = sum((time[1] - for (loc, time) in a)) - for ((filename, lineno, fn), [count, time]) in a: - if cur != filename: - if cur != None and f != None: - for line in f: - print line[:-1] - f.close() - try: - f = file(filename, "r") - except OSError: - f = None - cur = filename - l = 0 - print "#", filename - if f != None: - while l < lineno: - print f.readline()[:-1] - l += 1 - print ("# %8d %6.3f%% %8d %6.3f%%" - % (time, time * 100.0 / total_time, - count, count * 100.0 / total_count)) - if cur != None and f != None: - for line in f: - print line[:-1] - f.close() - -def main(args): - import os - - if args[1] == "pc": - import profile - pr = profile.Profile() - for i in range(5): - print pr.calibrate(100000) - return - elif args[1] == "p": - import profile - ret = 0 - # profile.Profile.bias = 5.26e-6 - profile.runctx("ret = main2(args[1:])", - globals(), locals()) - return ret - elif args[1].startswith("h"): - import hotshot, hotshot.stats - import warnings - - warnings.filterwarnings("ignore") - tmp = os.tempnam() - try: - l = args[1].startswith("hl") - p = hotshot.Profile(tmp, l) - ret = p.runcall(main2, args[1:]) - p.close() - p = None - if l: - if args[1] == "hl2": - _dump_hotshot_lineinfo2(tmp) - else: - _dump_hotshot_lineinfo(tmp) - else: - hotshot.stats.load(tmp).print_stats() - finally: - try: - os.remove(tmp) - except OSError: - pass - return ret - - return main2(args) - -if __name__ == '__main__': - sys.exit(main(sys.argv)) - +# +# lzari.py +# +# By Ross Ridge +# + +""" +Implementation of Haruhiko Okumura's LZARI data compression algorithm +in Python. Largely based on LZARI.C, one key difference is the use of +a two level dicitionary look up during compression rather than +LZARI.C's binary search tree. +""" + +_SCCS_ID = "@(#) mymc lzari.py 1.6 12/10/04 19:07:53\n" + +import sys +import array +import binascii +import string +import time +from bisect import bisect_right +from math import log + +try: + import ctypes + import mymcsup +except ImportError: + mymcsup = None + +hexlify = binascii.hexlify + +__ALL__ = ['lzari_codec', 'string_to_bit_array', 'bit_array_to_string'] + +# +# Fundamental constants of the LZARI compression alogorithm. +# +# Changing any of these values will create an incompatible implementation. +# + +HIST_LEN = 4096 +MIN_MATCH_LEN = 3 +MAX_MATCH_LEN = 60 + +ARITH_BITS = 15 +QUADRANT1 = 1 << ARITH_BITS +QUADRANT2 = QUADRANT1 * 2 +QUADRANT3 = QUADRANT1 * 3 +QUADRANT4 = QUADRANT1 * 4 +MAX_CUM = QUADRANT1 - 1 +MAX_CHAR = (256 + MAX_MATCH_LEN - MIN_MATCH_LEN + 1) + +# +# Other constants specific to this implementation +# + +MAX_SUFFIX_CHAIN = 50 # limit on how many identical suffixes to try to match + +#def debug(value, msg): +# print "@@@ %s %04x" % (msg, value) +debug = lambda value, msg: None + +_tr_16 = string.maketrans("0123456789abcdef", + "\x00\x01\x02\x03" + "\x10\x11\x12\x13" + "\x20\x21\x22\x23" + "\x30\x31\x32\x33") +_tr_4 = string.maketrans("0123", + "\x00\x01" + "\x10\x11") +_tr_2 = string.maketrans("01", "\x00\x01") + +def string_to_bit_array(s): + """Convert a string to an array containing a sequence of bits.""" + s = binascii.hexlify(s).translate(_tr_16) + s = binascii.hexlify(s).translate(_tr_4) + s = binascii.hexlify(s).translate(_tr_2) + a = array.array('B', s) + return a + +_tr_rev_2 = string.maketrans("\x00\x01", "01") +_tr_rev_4 = string.maketrans("\x00\x01" + "\x10\x11", + "0123") +_tr_rev_16 = string.maketrans("\x00\x01\x02\x03" + "\x10\x11\x12\x13" + "\x20\x21\x22\x23" + "\x30\x31\x32\x33", + "0123456789abcdef") +def bit_array_to_string(a): + """Convert an array containing a sequence of bits to a string.""" + remainder = len(a) % 8 + if remainder != 0: + a.fromlist([0] * (8 - remainder)) + s = a.tostring() + s = binascii.unhexlify(s.translate(_tr_rev_2)) + s = binascii.unhexlify(s.translate(_tr_rev_4)) + return binascii.unhexlify(s.translate(_tr_rev_16)) + +def _match(src, pos, hpos, mlen, end): + mlen += 1 + if not src.startswith(src[hpos : hpos + mlen], pos): + return None + for i in range(mlen, end): + if src[pos + i] != src[hpos + i]: + return i + return end + +def _rehash_table2(src, chars, head, next, next2, hist_invalid): + p = head + table2 = {} + l = [] + while p > hist_invalid: + l.append(p) + p = next[p % HIST_LEN] + l.reverse() + for p in l: + p2 = p + MIN_MATCH_LEN + key2 = src[p2 : p2 + chars] + head2 = table2.get(key2, hist_invalid) + next2[p % HIST_LEN] = head2 + table2[key2] = p + return table2 + +class lzari_codec(object): + # despite the name this does not implement a codec compatible + # with Python's codec system + + def init(self, decode): + self.high = QUADRANT4 + self.low = 0 + if decode: + self.code = 0 + # reverse the order of sym_cum so bisect_right() can + # be used for faster searching + self.sym_cum = range(0, MAX_CHAR + 1) + else: + self.shifts = 0 + self.char_to_symbol = range(1, MAX_CHAR + 1) + self.sym_cum = range(MAX_CHAR, -1, -1) + self.next_table = [None] * HIST_LEN + self.next2_table = [None] * HIST_LEN + self.suffix_table = {} + + self.symbol_to_char = [0] + range(MAX_CHAR) + self.sym_freq = [0] + [1] * MAX_CHAR + self.position_cum = [0] * (HIST_LEN + 1) + a = 0 + for i in range(HIST_LEN, 0, -1): + a = a + 10000 / (200 + i) + self.position_cum[i - 1] = a + + def search(self, table, x): + c = 1 + s = len(table) - 1 + while True: + a = (s + c) / 2 + if table[a] <= x: + s = a + else: + c = a + 1 + if c >= s: + break + return c + + def update_model_decode(self, symbol): + # A compatible implemention to the one used while compressing. + + sym_freq = self.sym_freq + sym_cum = self.sym_cum + + if self.sym_cum[MAX_CHAR] >= MAX_CUM: + c = 0 + for i in range(MAX_CHAR, 0, -1): + self.sym_cum[MAX_CHAR - i] = c + a = (self.sym_freq[i] + 1) / 2 + self.sym_freq[i] = a + c += a + self.sym_cum[MAX_CHAR] = c + freq = sym_freq[symbol] + new_symbol = symbol + while self.sym_freq[new_symbol - 1] == freq: + new_symbol -= 1 + # new_symbol = sym_freq.index(freq) + if new_symbol != symbol: + symbol_to_char = self.symbol_to_char + swap_char = symbol_to_char[new_symbol] + char = symbol_to_char[symbol] + symbol_to_char[new_symbol] = char + symbol_to_char[symbol] = swap_char + sym_freq[new_symbol] = freq + 1 + for i in range(MAX_CHAR - new_symbol + 1, MAX_CHAR + 1): + sym_cum[i] += 1 + + def update_model_encode(self, symbol): + sym_freq = self.sym_freq + sym_cum = self.sym_cum + + if sym_cum[0] >= MAX_CUM: + c = 0 + for i in range(MAX_CHAR, 0, -1): + sym_cum[i] = c + a = (sym_freq[i] + 1) / 2 + sym_freq[i] = a + c += a + sym_cum[0] = c + freq = sym_freq[symbol] + new_symbol = symbol + while sym_freq[new_symbol - 1] == freq: + new_symbol -= 1 + if new_symbol != symbol: + debug(new_symbol, "a") + swap_char = self.symbol_to_char[new_symbol] + char = self.symbol_to_char[symbol] + self.symbol_to_char[new_symbol] = char + self.symbol_to_char[symbol] = swap_char + self.char_to_symbol[char] = new_symbol + self.char_to_symbol[swap_char] = symbol + sym_freq[new_symbol] += 1 + for i in range(new_symbol): + sym_cum[i] += 1 + + def decode_char(self): + high = self.high + low = self.low + code = self.code + sym_cum = self.sym_cum + + _range = high - low + max_cum_freq = sym_cum[MAX_CHAR] + n = ((code - low + 1) * max_cum_freq - 1) / _range + i = bisect_right(sym_cum, n, 1) + high = low + sym_cum[i] * _range / max_cum_freq + low += sym_cum[i - 1] * _range / max_cum_freq + symbol = MAX_CHAR + 1 - i + + while True: + if low < QUADRANT2: + if low < QUADRANT1 or high > QUADRANT3: + if high > QUADRANT2: + break + else: + low -= QUADRANT1 + code -= QUADRANT1 + high -= QUADRANT1 + else: + low -= QUADRANT2 + code -= QUADRANT2 + high -= QUADRANT2 + low *= 2 + high *= 2 + code = code * 2 + self.in_iter() + + ret = self.symbol_to_char[symbol] + self.high = high + self.low = low + self.code = code + self.update_model_decode(symbol) + return ret + + def decode_position(self): + _range = self.high - self.low + max_cum = self.position_cum[0] + pos = self.search(self.position_cum, + ((self.code - self.low + 1) + * max_cum - 1) / _range) - 1 + self.high = (self.low + + self.position_cum[pos] * _range / max_cum) + self.low += self.position_cum[pos + 1] * _range / max_cum + while True: + if self.low < QUADRANT2: + if (self.low < QUADRANT1 + or self.high > QUADRANT3): + if self.high > QUADRANT2: + return pos + else: + self.low -= QUADRANT1 + self.code -= QUADRANT1 + self.high -= QUADRANT1 + else: + self.low -= QUADRANT2 + self.code -= QUADRANT2 + self.high -= QUADRANT2 + self.low *= 2 + self.high *= 2 + self.code = self.in_iter() + self.code * 2 + + def add_suffix_1(self, pos, find): + # naive implemention used for testing + + if not find: + return (None, 0) + src = self.src + mlen = min(1000, self.max_match, len(src) - pos) + hist_start = max(pos - HIST_LEN, 0) + while mlen >= MIN_MATCH_LEN: + i = src.rfind(src[pos : pos + mlen], hist_start, pos) + if i != -1: + assert (src[pos : pos + mlen] + == src[i: i + mlen]) + return (i, mlen) + mlen -= 1 + return (None, -1) + + def add_suffix_2(self, pos, find): + # a two level dictionary look up that leverages Python's + # built-in dicts to get something that's hopefully faster + # than implementing binary trees in completely in Python. + + src = self.src + suffix_table = self.suffix_table + max_match = min(self.max_match, len(src) - pos) + + mlen = -1 + mpos = None + + hist_invalid = pos - HIST_LEN - 1 + modpos = pos % HIST_LEN + pos2 = pos + MIN_MATCH_LEN + + key = src[pos : pos2] + a = suffix_table.get(key) + if a != None: + next = self.next_table + next2 = self.next2_table + + [count, head, table2, chars] = a + + pos3 = pos2 + chars + key2 = src[pos2 : pos3] + min_match2 = MIN_MATCH_LEN + chars + if find: + p = table2.get(key2, hist_invalid) + maxmlen = max_match - min_match2 + while p > hist_invalid and mlen != maxmlen: + p3 = p + min_match2 + if mpos == None and p3 <= pos: + mpos = p + mlen = 0 + if p3 >= pos: + p = next2[p % HIST_LEN] + continue + rlen = _match(src, pos3, p3, mlen, + min(maxmlen, pos - p3)) + if rlen != None: + mpos = p + mlen = rlen + p = next2[p % HIST_LEN] + if mpos != None: + mlen += min_match2 + elif find: + p = head + maxmlen = min(chars, max_match - MIN_MATCH_LEN) + i = 0 + while (p > hist_invalid and i < 50000 + and mlen < maxmlen): + assert i < count + i += 1 + p2 = p + MIN_MATCH_LEN + l2 = pos - p2 + if mpos == None and l2 >= 0: + mpos = p + mlen = 0 + if l2 <= 0: + p = next[p % HIST_LEN] + continue + if l2 > maxmlen: + l2 = maxmlen + m = mlen + 1 + if src.startswith(src[p2 : p2 + m], + pos2): + mpos = p + for j in range(m, l2): + if (src[pos2 + j] + != src[p2 + j]): + mlen = j + break + else: + mlen = l2 + #rlen = _match(src, pos2, p2, mlen, l2) + #if rlen != None: + # mpos = p + # mlen = rlen + p = next[p % HIST_LEN] + + if mpos != None: + mlen += MIN_MATCH_LEN + + count += 1 + new_chars = int(log(count, 2)) + # new_chars = 50 + new_chars = min(new_chars, max_match - MIN_MATCH_LEN) + if new_chars > chars: + chars = new_chars + table2 = _rehash_table2(src, chars, head, + next, next2, + hist_invalid) + + next[modpos] = head + head = pos + + key2 = src[pos2 : pos2 + chars] + head2 = table2.get(key2, hist_invalid) + next2[modpos] = head2 + table2[key2] = pos + + a[0] = count + a[1] = head + a[2] = table2 + a[3] = chars + else: + self.next_table[modpos] = hist_invalid + self.next2_table[modpos] = hist_invalid + key2 = "" + # key2 = src[pos2 : pos2 + 1] + suffix_table[key] = [1, pos, {key2: pos}, len(key2)] + + p = pos - HIST_LEN + if p >= 0: + p2 = p + MIN_MATCH_LEN + key = src[p : p2] + a = suffix_table[key] + (count, head, table2, chars) = a + count -= 1 + if count == 0: + assert head == p + del suffix_table[key] + else: + key2 = src[p2 : p2 + chars] + if table2[key2] == p: + del table2[key2] + a[0] = count + assert (mpos == None + or src[pos : pos + mlen] == src[mpos : mpos + mlen]) + return (mpos, mlen) + + def _add_suffix(self, pos, find): + r = self.add_suffix_2(pos, find) + start_pos = self.start_pos + if find and r[0] != None: + print ("%4d %02x %4d %2d" + % (pos - start_pos, ord(self.src[pos]), + r[0] - start_pos, r[1])) + else: + print ("%4d %02x" + % (pos - start_pos, ord(self.src[pos]))) + return r + + add_suffix = add_suffix_2 + + def output_bit(self, bit): + self.append_bit(bit) + bit ^= 1 + for i in range(self.shifts): + self.append_bit(bit) + self.shifts = 0 + + def encode_char(self, char): + low = self.low + high = self.high + sym_cum = self.sym_cum + + symbol = self.char_to_symbol[char] + range = high - low + + high = low + range * sym_cum[symbol - 1] / sym_cum[0] + low += range * sym_cum[symbol] / sym_cum[0] + debug(high, "high"); + debug(low, "low"); + while True: + if high <= QUADRANT2: + self.output_bit(0) + elif low >= QUADRANT2: + self.output_bit(1) + low -= QUADRANT2 + high -= QUADRANT2 + elif low >= QUADRANT1 and high <= QUADRANT3: + self.shifts += 1 + low -= QUADRANT1 + high -= QUADRANT1 + else: + break + low *= 2 + high *= 2 + self.low = low + self.high = high + self.update_model_encode(symbol) + + def encode_position(self, position): + position_cum = self.position_cum + low = self.low + high = self.high + + range = high - low + high = low + range * position_cum[position] / position_cum[0] + low += range * position_cum[position + 1] / position_cum[0] + + debug(high, "high"); + debug(low, "low"); + while True: + if high <= QUADRANT2: + self.output_bit(0) + elif low >= QUADRANT2: + self.output_bit(1) + low -= QUADRANT2 + high -= QUADRANT2 + elif low >= QUADRANT1 and high <= QUADRANT3: + self.shifts += 1 + low -= QUADRANT1 + high -= QUADRANT1 + else: + break + low *= 2 + high *= 2 + + self.low = low + self.high = high + + def encode(self, src, progress = None): + """Compress a string.""" + + length = len(src) + if length == 0: + return "" + + out_array = array.array('B') + self.out_array = out_array + self.append_bit = out_array.append + + self.init(False) + + max_match = min(MAX_MATCH_LEN, length) + self.max_match = max_match + self.src = src = "\x20" * max_match + src + + in_length = len(src) + + self.start_pos = max_match + + for in_pos in range(max_match): + self.add_suffix(in_pos, False) + in_pos += 1 + last_percent = -1 + while in_pos < in_length: + if progress: + percent = (in_pos - max_match) * 100 / length + if percent != last_percent: + sys.stderr.write("%s%3d%%\r" + % (progress, percent)) + last_percent = percent + debug(ord(src[in_pos]), "src") + (match_pos, match_len) = self.add_suffix(in_pos, True) + if match_len < MIN_MATCH_LEN: + self.encode_char(ord(src[in_pos])) + else: + debug(in_pos - match_pos - 1, "match_pos") + debug(match_len, "match_len") + self.encode_char(256 - MIN_MATCH_LEN + + match_len) + self.encode_position(in_pos - match_pos - 1) + for i in range(match_len - 1): + in_pos += 1 + self.add_suffix(in_pos, False) + in_pos += 1 + + self.shifts += 1 + if self.low < QUADRANT1: + self.output_bit(0) + else: + self.output_bit(1) + + #for k, v in sorted(self.suffix_table.items()): + # count, head, table2, chars = v + # print hexlify(k), count, head, len(table2), chars + + if progress: + sys.stderr.write("%s100%%\n" % progress) + + return bit_array_to_string(out_array) + + def decode(self, src, out_length, progress = None): + """Decompress a string.""" + + a = string_to_bit_array(src) + a.fromlist([0] * 32) # add some extra bits + self.in_iter = iter(a).next + + out = array.array('B', "\0") * out_length + outpos = 0 + + self.init(True) + + self.code = 0 + for i in range(ARITH_BITS + 2): + self.code += self.code + self.in_iter() + + hist_pos = HIST_LEN - MAX_MATCH_LEN + history = [0x20] * hist_pos + [0] * MAX_MATCH_LEN + + decode_char = self.decode_char + last_percent = -1 + last_time = time.time() + while outpos < out_length: + if progress: + percent = outpos * 100 / out_length + if percent != last_percent: + now = time.time() + if now - last_time >= 1: + sys.stderr.write("%s%3d%%\r" + % (progress, percent)) + last_percent = percent + last_time = now + char = decode_char() + if char >= 0x100: + pos = self.decode_position() + length = char - 0x100 + MIN_MATCH_LEN + base = (hist_pos - pos - 1) % HIST_LEN + for off in range(length): + a = history[(base + off) % HIST_LEN] + out[outpos] = a + outpos += 1 + history[hist_pos] = a + hist_pos = (hist_pos + 1) % HIST_LEN + else: + out[outpos] = char + outpos += 1 + history[hist_pos] = char + hist_pos = (hist_pos + 1) % HIST_LEN + + self.in_iter = None + if progress: + sys.stderr.write("%s100%%\n" % progress) + return out.tostring() + +if mymcsup == None: + def decode(src, out_length, progress = None): + return lzari_codec().decode(src, out_length, progress) + + def encode(src, progress = None): + return lzari_codec().encode(src, progress) +else: + mylzari_decode = mymcsup.mylzari_decode + mylzari_encode = mymcsup.mylzari_encode + mylzari_free_encoded = mymcsup.mylzari_free_encoded + + def decode(src, out_length, progress = None): + out = ctypes.create_string_buffer(out_length) + if (mylzari_decode(src, len(src), out, out_length, progress) + == -1): + raise ValueError, "compressed input is corrupt" + return ctypes.string_at(out, out_length) + + def encode(src, progress = None): + (r, compressed, comp_len) = mylzari_encode(src, len(src), + progress) + # print r, compressed.value, comp_len + if r == -1: + raise MemoryError, "out of memory during compression" + if compressed.value == None: + return "" + ret = ctypes.string_at(compressed.value, comp_len.value) + mylzari_free_encoded(compressed) + return ret; + +def main2(args): + import struct + import os + + src = file(args[2], "rb").read() + lzari = lzari_codec() + out = file(args[3], "wb") + start = os.times() + if args[1] == "c": + dest = lzari.encode(src) + now = os.times() + out.write(struct.pack("L", len(src))) + else: + dest = lzari.decode(src[4:], + struct.unpack("L", src[:4])[0]) + now = os.times() + out.write(dest) + out.close() + print "time:", now[0] - start[0], now[1] - start[1], now[4] - start[4] + + +def _get_hotshot_lineinfo(filename): + import hotshot.log + log = hotshot.log.LogReader(filename) + timings = {} + for what, loc, tdelta in log: + if what == hotshot.log.LINE: + a = timings.get(loc) + if a == None: + timings[loc] = [1, tdelta] + else: + a[0] += 1 + a[1] += tdelta + return timings.items() + +def _dump_hotshot_lineinfo(log): + a = sorted(_get_hotshot_lineinfo(log)) + total_count = sum((time[0] + for (loc, time) in a)) + total_time = sum((time[1] + for (loc, time) in a)) + for (loc, [count, time]) in a: + print ("%8d %6.3f%% %8d %6.3f%%" + % (time, time * 100.0 / total_time, + count, count * 100.0 / total_count)), + print "%s:%d(%s)" % loc + +def _dump_hotshot_lineinfo2(log): + cur = None + a = sorted(_get_hotshot_lineinfo(log)) + total_count = sum((time[0] + for (loc, time) in a)) + total_time = sum((time[1] + for (loc, time) in a)) + for ((filename, lineno, fn), [count, time]) in a: + if cur != filename: + if cur != None and f != None: + for line in f: + print line[:-1] + f.close() + try: + f = file(filename, "r") + except OSError: + f = None + cur = filename + l = 0 + print "#", filename + if f != None: + while l < lineno: + print f.readline()[:-1] + l += 1 + print ("# %8d %6.3f%% %8d %6.3f%%" + % (time, time * 100.0 / total_time, + count, count * 100.0 / total_count)) + if cur != None and f != None: + for line in f: + print line[:-1] + f.close() + +def main(args): + import os + + if args[1] == "pc": + import profile + pr = profile.Profile() + for i in range(5): + print pr.calibrate(100000) + return + elif args[1] == "p": + import profile + ret = 0 + # profile.Profile.bias = 5.26e-6 + profile.runctx("ret = main2(args[1:])", + globals(), locals()) + return ret + elif args[1].startswith("h"): + import hotshot, hotshot.stats + import warnings + + warnings.filterwarnings("ignore") + tmp = os.tempnam() + try: + l = args[1].startswith("hl") + p = hotshot.Profile(tmp, l) + ret = p.runcall(main2, args[1:]) + p.close() + p = None + if l: + if args[1] == "hl2": + _dump_hotshot_lineinfo2(tmp) + else: + _dump_hotshot_lineinfo(tmp) + else: + hotshot.stats.load(tmp).print_stats() + finally: + try: + os.remove(tmp) + except OSError: + pass + return ret + + return main2(args) + +if __name__ == '__main__': + sys.exit(main(sys.argv)) + diff --git a/mymc.py b/mymc.py index 54f3262..51068ec 100755 --- a/mymc.py +++ b/mymc.py @@ -1,812 +1,825 @@ -# -# mymc.py -# -# By Ross Ridge -# Public Domain -# - -"""A utility for manipulating PS2 memory card images.""" - -_SCCS_ID = "@(#) mysc mymc.py 1.12 12/10/04 19:09:16\n"[:-1] - -import sys -import os -import time -import optparse -import textwrap -import binascii -import string -from errno import EEXIST, EIO - -import ps2mc -import ps2save -from ps2mc_dir import * -from round import * -import verbuild - -class subopt_error(Exception): - pass - -io_error = ps2mc.io_error - -if os.name == "nt": - import codecs - - class file_wrap(object): - """ wrap a file-like object with a new encoding attribute. """ - - def __init__(self, f, encoding): - object.__setattr__(self, "_f", f) - object.__setattr__(self, "encoding", encoding) - - def __getattribute__(self, name): - if name == "encoding": - return object.__getattribute__(self, name) - return getattr(object.__getattribute__(self, "_f"), - name) - - def __setattr__(self, name, value): - if name == "encoding": - raise TypeError, "readonly attribute" - return setattr(object.__getattribute__(self, "_f"), - name, value) - - for name in ["stdin", "stdout", "stderr"]: - f = getattr(sys, name) - cur = getattr(f, "encoding", None) - if cur == "ascii" or cur == None: - f = file_wrap(f, "mbcs") - else: - try: - codecs.lookup(cur) - except LookupError: - f = file_wrap(f, "mbcs") - setattr(sys, name, f) - - -if os.name in ["nt", "os2", "ce"]: - from glob import glob -else: - # assume globing is done by the shell - glob = lambda pattern: [pattern] - - -def glob_args(args, globfn): - ret = [] - for arg in args: - match = globfn(arg) - if len(match) == 0: - ret.append(arg) - else: - ret += match - return ret - -def _copy(fout, fin): - """copy the contents of one file to another""" - - while True: - s = fin.read(1024) - if s == "": - break - fout.write(s) - - -def do_ls(cmd, mc, opts, args, opterr): - mode_bits = "rwxpfdD81C+KPH4" - - if len(args) == 0: - args = ["/"] - - out = sys.stdout - args = glob_args(args, mc.glob) - for dirname in args: - dir = mc.dir_open(dirname) - try: - if len(args) > 1: - sys.stdout.write("\n" + dirname + ":\n") - for ent in dir: - mode = ent[0] - if (mode & DF_EXISTS) == 0: - continue - for bit in range(0, 15): - if mode & (1 << bit): - out.write(mode_bits[bit]) - else: - out.write("-") - if opts.creation_time: - tod = ent[3] - else: - tod = ent[6] - tm = time.localtime(tod_to_time(tod)) - out.write(" %7d %04d-%02d-%02d" - " %02d:%02d:%02d %s\n" - % (ent[2], - tm.tm_year, tm.tm_mon, tm.tm_mday, - tm.tm_hour, tm.tm_min, tm.tm_sec, - ent[8])) - finally: - dir.close() - - -def do_add(cmd, mc, opts, args, opterr): - if len(args) < 1: - opterr("Filename required.") - if opts.directory != None: - mc.chdir(opts.directory) - for src in glob_args(args, glob): - f = open(src, "rb") - dest = os.path.basename(src) - out = mc.open(dest, "wb") - _copy(out, f) - out.close() - f.close() - -def do_extract(cmd, mc, opts, args, opterr): - if len(args) < 1: - opterr("Filename required.") - - if opts.directory != None: - mc.chdir(opts.directory) - - close_out = False - out = None - if opts.output != None: - if opts.use_stdout: - opterr("The -o and -p options are mutually exclusive.") - dont_close_out = True - out = file(opts.output, "wb") - elif opts.use_stdout: - out = sys.stdout - - try: - for filename in glob_args(args, mc.glob): - f = mc.open(filename, "rb") - try: - if out != None: - _copy(out, f) - continue - a = filename.split("/") - o = file(a[-1], "wb") - try: - _copy(o, f) - finally: - o.close() - finally: - f.close() - finally: - if close_out: - out.close() - -def do_mkdir(cmd, mc, opts, args, opterr): - if len(args) < 1: - opterr("Directory required.") - - for filename in args: - mc.mkdir(filename) - -def do_remove(cmd, mc, opts, args, opterr): - if len(args) < 1: - opterr("Filename required.") - - for filename in args: - mc.remove(filename) - -def do_import(cmd, mc, opts, args, opterr): - if len(args) < 1: - opterr("Filename required.") - - args = glob_args(args, glob) - if opts.directory != None and len(args) > 1: - opterr("The -d option can only be used with a" - "single savefile.") - - for filename in args: - sf = ps2save.ps2_save_file() - f = file(filename, "rb") - try: - ftype = ps2save.detect_file_type(f) - f.seek(0) - if ftype == "max": - sf.load_max_drive(f) - elif ftype == "psu": - sf.load_ems(f) - elif ftype == "cbs": - sf.load_codebreaker(f) - elif ftype == "sps": - sf.load_sharkport(f) - elif ftype == "npo": - raise io_error, (EIO, "nPort saves" - " are not supported.", - filename) - else: - raise io_error, (EIO, "Save file format not" - " recognized", filename) - finally: - f.close() - dirname = opts.directory - if dirname == None: - dirname = sf.get_directory()[8] - print "Importing", filename, "to", dirname - if not mc.import_save_file(sf, opts.ignore_existing, - opts.directory): - print (filename + ": already in memory card image," - " ignored.") - -#re_num = re.compile("[0-9]+") - -def do_export(cmd, mc, opts, args, opterr): - if len(args) < 1: - opterr("Directory name required") - - if opts.overwrite_existing and opts.ignore_existing: - opterr("The -i and -f options are mutually exclusive.") - - args = glob_args(args, mc.glob) - if opts.output_file != None: - if len(args) > 1: - opterr("Only one directory can be exported" - " when the -o option is used.") - if opts.longnames: - opterr("The -o and -l options are mutually exclusive.") - - if opts.directory != None: - os.chdir(opts.directory) - - for dirname in args: - sf = mc.export_save_file(dirname) - filename = opts.output_file - if opts.longnames: - filename = (ps2save.make_longname(dirname, sf) - + "." + opts.type) - if filename == None: - filename = dirname + "." + opts.type - - if not opts.overwrite_existing: - exists = True - try: - open(filename, "rb").close() - except EnvironmentError: - exists = False - if exists: - if opts.ignore_existing: - continue - raise io_error(EEXIST, "File exists", filename) - - f = file(filename, "wb") - try: - print "Exporing", dirname, "to", filename - - if opts.type == "max": - sf.save_max_drive(f) - else: - sf.save_ems(f) - finally: - f.close() - -def do_delete(cmd, mc, opts, args, opterr): - if len(args) < 1: - opterr("Directory required.") - - for dirname in args: - mc.rmdir(dirname) - -def do_setmode(cmd, mc, opts, args, opterr): - set_mask = 0 - clear_mask = ~0 - for (opt, bit) in [(opts.read, DF_READ), - (opts.write, DF_WRITE), - (opts.execute, DF_EXECUTE), - (opts.protected, DF_PROTECTED), - (opts.psx, DF_PSX), - (opts.pocketstation, DF_POCKETSTN), - (opts.hidden, DF_HIDDEN)]: - if opt != None: - if opt: - set_mask |= bit - else: - clear_mask ^= bit - - value = opts.hex_value - if set_mask == 0 and clear_mask == ~0: - if value == None: - opterr("At least one option must be given.") - if value.startswith("0x") or value.startswith("0X"): - value = int(value[2:], 16) - else: - value = int(value, 16) - else: - if value != None: - opterr("The -X option can't be combined with" - " other options.") - - for arg in glob_args(args, mc.glob): - ent = mc.get_dirent(arg) - if value == None: - ent[0] = (ent[0] & clear_mask) | set_mask - # print "new %04x" % ent[0] - else: - ent[0] = value - mc.set_dirent(arg, ent) - -def _get_ps2_title(mc, enc): - s = mc.get_icon_sys("."); - if s == None: - return None - a = ps2save.unpack_icon_sys(s) - return ps2save.icon_sys_title(a, enc) - -def _get_psx_title(mc, savename, enc): - mode = mc.get_mode(savename) - if mode == None or not mode_is_file(mode): - return None - f = mc.open(savename) - s = f.read(128) - if len(s) != 128: - return None - (magic, icon, blocks, title) = struct.unpack("<2sBB64s28x32x", s) - if magic != "SC": - return None - return [ps2save.shift_jis_conv(zero_terminate(title), enc), ""] - -def do_dir(cmd, mc, opts, args, opterr): - if len(args) != 0: - opterr("Incorrect number of arguments.") - f = None - dir = mc.dir_open("/") - try: - for ent in list(dir)[2:]: - dirmode = ent[0] - if not mode_is_dir(dirmode): - continue - dirname = "/" + ent[8] - mc.chdir(dirname) - length = mc.dir_size("."); - enc = getattr(sys.stdout, "encoding", None) - if dirmode & DF_PSX: - title = _get_psx_title(mc, ent[8], enc) - else: - title = _get_ps2_title(mc, enc) - if title == None: - title = ["Corrupt", ""] - protection = dirmode & (DF_PROTECTED | DF_WRITE) - if protection == 0: - protection = "Delete Protected" - elif protection == DF_WRITE: - protection = "Not Protected" - elif protection == DF_PROTECTED: - protection = "Copy & Delete Protected" - else: - protection = "Copy Protected" - - type = None - if dirmode & DF_PSX: - type = "PlayStation" - if dirmode & DF_POCKETSTN: - type = "PocketStation" - if type != None: - protection = type - - print "%-32s %s" % (ent[8], title[0]) - print ("%4dKB %-25s %s" - % (length / 1024, protection, title[1])) - print - finally: - if f != None: - f.close() - dir.close() - - free = mc.get_free_space() / 1024 - if free > 999999: - free = "%d,%03d,%03d" % (free / 1000000, free / 1000 % 1000, - free % 1000) - elif free > 999: - free = "%d,%03d" % (free / 1000, free % 1000) - else: - free = "%d" % free - - print free + " KB Free" - -def do_df(cmd, mc, opts, args, opterr): - if len(args) != 0: - opterr("Incorrect number of arguments.") - print mc.f.name + ":", mc.get_free_space(), "bytes free." - -def do_check(cmd, mc, opts, args, opterr): - if len(args) != 0: - opterr("Incorrect number of arguments.") - if mc.check(): - print "No errors found." - return 0 - return 1 - -def do_format(cmd, mcname, opts, args, opterr): - if len(args) != 0: - opterr("Incorrect number of arguments.") - pages_per_card = ps2mc.PS2MC_STANDARD_PAGES_PER_CARD - if opts.clusters != None: - pages_per_cluster = (ps2mc.PS2MC_CLUSTER_SIZE - / ps2mc.PS2MC_STANDARD_PAGE_SIZE) - pages_per_card = opts.clusters * pages_per_cluster - params = (not opts.no_ecc, - ps2mc.PS2MC_STANDARD_PAGE_SIZE, - ps2mc.PS2MC_STANDARD_PAGES_PER_ERASE_BLOCK, - pages_per_card) - - if not opts.overwrite_existing: - exists = True - try: - file(mcname, "rb").close() - except EnvironmentError: - exists = False - if exists: - raise io_error, (EEXIST, "file exists", mcname) - - f = file(mcname, "w+b") - try: - ps2mc.ps2mc(f, True, params).close() - finally: - f.close() - -def do_gui(cmd, mcname, opts, args, opterr): - if len(args) != 0: - opterr("Incorrect number of arguments.") - - try: - import gui - except ImportError: - write_error(None, "GUI not available") - return 1 - - gui.run(mcname) - return 0 - -def do_create_pad(cmd, mc, opts, args, opterr): - length = mc.clusters_per_card - if len(args) > 1: - length = int(args[1]) - pad = "\0" * mc.cluster_size - f = mc.open(args[0], "wb") - try: - for i in xrange(length): - f.write(pad) - finally: - f.close() - - -def do_frob(cmd, mc, opts, args, opterr): - mc.write_superblock() - -_trans = string.maketrans("".join(map(chr, range(32))), " " * 32) - -def _print_bin(base, s): - for off in range(0, len(s), 16): - print "%04X" % (base + off), - a = s[off : off + 16] - for b in a: - print "%02X" % ord(b), - print "", a.translate(_trans) - -def _print_erase_block(mc, n): - ppb = mc.pages_per_erase_block - base = n * ppb - for i in range(ppb): - s = mc.read_page(base + i) - _print_bin(i * mc.page_size, s) - print - -def do_print_good_blocks(cmd, mc, opts, args, opterr): - print "good_block2:" - _print_erase_block(mc, mc.good_block2) - print "good_block1:" - _print_erase_block(mc, mc.good_block1) - -def do_ecc_check(cmd, mc, opts, args, opterr): - for i in range(mc.clusters_per_card * mc.pages_per_cluster): - try: - mc.read_page(i) - except ps2mc.ecc_error: - print "bad: %05x" % i - -opt = optparse.make_option - -# -# Each value in the dictionary is a tuple consisting of: -# - function implementing the command -# - mode to use to open the ps2 save file -# - help description of the command -# - list of options supported by the command -# -cmd_table = { - "ls": (do_ls, "rb", - "[directory ...]", - "List the contents of a directory.", - [opt("-c", "--creation-time", action="store_true", - help = "Display creation times.")]), - "extract": (do_extract, "rb", - "filename ...", - "Extract files from the memory card.", - [opt("-o", "--output", metavar = "FILE", - help = 'Extract file to "FILE".'), - opt("-d", "--directory", - help = 'Extract files from "DIRECTORY".'), - opt("-p", "--use-stdout", action="store_true", - help = "Extract files to standard output.")]), - "add": (do_add, "r+b", - "filename ...", - "Add files to the memory card.", - [opt("-d", "--directory", - help = 'Add files to "directory".')]), - "mkdir": (do_mkdir, "r+b", - "directory ...", - "Make directories.", - []), - "remove": (do_remove, "r+b", - "filename ...", - "Remove files and directories.", - []), - "import": (do_import, "r+b", - "savefile ...", - "Import save files into the memory card.", - [opt("-i", "--ignore-existing", action="store_true", - help = ("Ignore files that already exist" - "on the image.")), - opt("-d", "--directory", metavar="DEST", - help = 'Import to "DEST".')]), - "export": (do_export, "rb", - "directory ...", - "Export save files from the memory card.", - [opt("-f", "--overwrite-existing", action = "store_true", - help = "Overwrite any save files already exported."), - opt("-i", "--ignore-existing", action = "store_true", - help = "Ingore any save files already exported."), - opt("-o", "--output-file", metavar = "filename", - help = 'Use "filename" as the name of the save file.'), - opt("-d", "--directory", metavar = "directory", - help = 'Export save files to "directory".'), - opt("-l", "--longnames", action = "store_true", - help = ("Generate longer, more descriptive," - " filenames.")), - opt("-p", "--ems", action = "store_const", - dest = "type", const = "psu", default = "psu", - help = "Use the EMS .psu save file format. [default]"), - opt("-m", "--max-drive", action = "store_const", - dest = "type", const = "max", - help = "Use the MAX Drive save file format.")]), - "delete": (do_delete, "r+b", - "dirname ...", - "Recursively delete a directory (save file).", - []), - "set": (do_setmode, "r+b", - "filename ...", - "Set mode flags on files and directories", - [opt("-p", "--protected", action="store_true", - help = "Set copy protected flag"), - opt("-P", "--psx", action="store_true", - help = "Set PSX flag"), - opt("-K", "--pocketstation", action="store_true", - help = "Set PocketStation flag"), - opt("-H", "--hidden", action="store_true", - help = "Set hidden flag"), - opt("-r", "--read", action="store_true", - help = "Set read allowed flag"), - opt("-w", "--write", action="store_true", - help = "Set write allowed flag"), - opt("-x", "--execute", action="store_true", - help = "Set executable flag"), - opt("-X", "--hex-value", metavar="mode", - help = 'Set mode to "mode".')]), - "clear": (do_setmode, "r+b", - "filename ...", - "Clear mode flags on files and directories", - [opt("-p", "--protected", action="store_false", - help = "Clear copy protected flag"), - opt("-P", "--psx", action="store_false", - help = "Clear PSX flag"), - opt("-K", "--pocketstation", action="store_false", - help = "Clear PocketStation flag"), - opt("-H", "--hidden", action="store_false", - help = "Clear hidden flag"), - opt("-r", "--read", action="store_false", - help = "Clear read allowed flag"), - opt("-w", "--write", action="store_false", - help = "Clear write allowed flag"), - opt("-x", "--execute", action="store_false", - help = "Clear executable flag"), - opt("-X", dest="hex_value", default=None, - help = optparse.SUPPRESS_HELP)]), - "dir": (do_dir, "rb", - None, - "Display save file information.", - []), - "df": (do_df, "rb", - None, - "Display the amount free space.", - []), - "check": (do_check, "rb", - "", - "Check for file system errors.", - []), - "format": (do_format, None, - "", - "Creates a new memory card image.", - [opt("-c", "--clusters", type="int", - help = "Size in clusters of the memory card."), - opt("-f", "--overwrite-existing", action="store_true", - help = "Overwrite any existing file"), - opt("-e", "--no-ecc", action="store_true", - help = "Create an image without ECC")]), - "gui": (do_gui, None, - "", - "Starts the graphical user interface.", - []), -} - -# -# secret commands for debugging purposes. -# -debug_cmd_table = { - "frob": (do_frob, "r+b", - "", - None, - []), - "print_good_blocks": (do_print_good_blocks, "rb", - "", - None, - []), - "ecc_check": (do_ecc_check, "rb", - "", - None, - []), - "create_pad": (do_create_pad, "r+b", - "", - None, - []) -} - -del opt # clean up name space - - -def write_error(filename, msg): - if filename == None: - sys.stderr.write(msg + "\n") - else: - sys.stderr.write(filename + ": " + msg + "\n") - -class suboption_parser(optparse.OptionParser): - def exit(self, status = 0, msg = None): - if msg: - sys.stderr.write(msg) - raise subopt_error, status - -class my_help_formatter(optparse.IndentedHelpFormatter): - """A better formatter for optparser's help message""" - - def format_description(self, description): - if not description: - return "" - desc_width = self.width - self.current_indent - indent = " " * self.current_indent - lines = [] - for line in description.split('\n'): - ii = indent - si = indent - if line.startswith("\t"): - line = line[1:] - ii = indent + " " * 4 - si = ii + " " * line.find(":") + 2 - line = textwrap.fill(line, desc_width, - initial_indent = ii, - subsequent_indent = si) - lines.append(line) - return "\n".join(lines) + "\n" - -def main(): - prog = sys.argv[0].decode(sys.getdefaultencoding(), "replace") - usage = "usage: %prog [-ih] memcard.ps2 command [...]" - description = ("Manipulate PS2 memory card images.\n\n" - "Supported commands: ") - for cmd in sorted(cmd_table.keys()): - description += "\n " + cmd + ": " + cmd_table[cmd][3] - - version = ("mymc " - + verbuild.MYMC_VERSION_MAJOR - + "." + verbuild.MYMC_VERSION_BUILD - + " (" + _SCCS_ID + ")") - - optparser = optparse.OptionParser(prog = prog, usage = usage, - description = description, - version = version, - formatter = my_help_formatter()) - optparser.add_option("-D", dest = "debug", action = "store_true", - default = False, help = optparse.SUPPRESS_HELP) - optparser.add_option("-i", "--ignore-ecc", action = "store_true", - help = "Ignore ECC errors while reading.") - - optparser.disable_interspersed_args() - (opts, args) = optparser.parse_args() - - if len(args) == 0: - try: - import gui - except ImportError: - gui = None - if gui != None: - gui.run() - sys.exit(0) - - if len(args) < 2: - optparser.error("Incorrect number of arguments.") - - if opts.debug: - cmd_table.update(debug_cmd_table) - cmd = args[1] - if cmd not in cmd_table: - optparser.error('Command "%s" not recognized.' % cmd) - (fn, mode, usage_args, description, optlist) = cmd_table[cmd] - - usage = "%prog" - if len(optlist) > 0: - usage += " [options]" - if usage_args != None: - usage += " " + usage_args - subprog = prog + " memcard.ps2 " + cmd - subopt_parser = suboption_parser(prog = subprog, usage = usage, - description = description, - option_list = optlist) - subopt_parser.disable_interspersed_args() - - f = None - mc = None - ret = 0 - mcname = args[0] - - try: - (subopts, subargs) = subopt_parser.parse_args(args[2:]) - try: - if mode == None: - ret = fn(cmd, mcname, subopts, subargs, - subopt_parser.error) - else: - f = file(mcname, mode) - mc = ps2mc.ps2mc(f, opts.ignore_ecc) - ret = fn(cmd, mc, subopts, subargs, - subopt_parser.error) - finally: - if mc != None: - mc.close() - if f != None: - # print "f.close()" - f.close() - - except EnvironmentError, value: - if getattr(value, "filename", None) != None: - write_error(value.filename, value.strerror) - ret = 1 - elif getattr(value, "strerror", None) != None: - write_error(mcname, value.strerror) - ret = 1 - else: - # something weird - raise - if opts.debug: - raise - - except subopt_error, (ret,): - pass - - except (ps2mc.error, ps2save.error), value: - fn = getattr(value, "filename", None) - if fn == None: - fn = mcname - write_error(fn, str(value)) - if opts.debug: - raise - ret = 1 - - if ret == None: - ret = 0 - - return ret - -sys.exit(main()) - +# +# mymc.py +# +# By Ross Ridge +# Public Domain +# + +"""A utility for manipulating PS2 memory card images.""" + +_SCCS_ID = "@(#) mymc mymc.py 1.13 22/01/15 01:04:45\n"[:-1] + +import sys +import os +import time +import optparse +import textwrap +import binascii +import string +from errno import EEXIST, EIO + +#import gc +#gc.set_debug(gc.DEBUG_LEAK) + +import ps2mc +import ps2save +from ps2mc_dir import * +from round import * +import verbuild + +class subopt_error(Exception): + pass + +io_error = ps2mc.io_error + +if os.name == "nt": + import codecs + + class file_wrap(object): + """ wrap a file-like object with a new encoding attribute. """ + + def __init__(self, f, encoding): + object.__setattr__(self, "_f", f) + object.__setattr__(self, "encoding", encoding) + + def __getattribute__(self, name): + if name == "encoding": + return object.__getattribute__(self, name) + return getattr(object.__getattribute__(self, "_f"), + name) + + def __setattr__(self, name, value): + if name == "encoding": + raise TypeError, "readonly attribute" + return setattr(object.__getattribute__(self, "_f"), + name, value) + + for name in ["stdin", "stdout", "stderr"]: + f = getattr(sys, name) + cur = getattr(f, "encoding", None) + if cur == "ascii" or cur == None: + f = file_wrap(f, "mbcs") + else: + try: + codecs.lookup(cur) + except LookupError: + f = file_wrap(f, "mbcs") + setattr(sys, name, f) + + +if os.name in ["nt", "os2", "ce"]: + from glob import glob +else: + # assume globing is done by the shell + glob = lambda pattern: [pattern] + + +def glob_args(args, globfn): + ret = [] + for arg in args: + match = globfn(arg) + if len(match) == 0: + ret.append(arg) + else: + ret += match + return ret + +def _copy(fout, fin): + """copy the contents of one file to another""" + + while True: + s = fin.read(1024) + if s == "": + break + fout.write(s) + + +def do_ls(cmd, mc, opts, args, opterr): + mode_bits = "rwxpfdD81C+KPH4" + + if len(args) == 0: + args = ["/"] + + out = sys.stdout + args = glob_args(args, mc.glob) + for dirname in args: + dir = mc.dir_open(dirname) + try: + if len(args) > 1: + sys.stdout.write("\n" + dirname + ":\n") + for ent in dir: + mode = ent[0] + if (mode & DF_EXISTS) == 0: + continue + for bit in range(0, 15): + if mode & (1 << bit): + out.write(mode_bits[bit]) + else: + out.write("-") + if opts.creation_time: + tod = ent[3] + else: + tod = ent[6] + tm = time.localtime(tod_to_time(tod)) + out.write(" %7d %04d-%02d-%02d" + " %02d:%02d:%02d %s\n" + % (ent[2], + tm.tm_year, tm.tm_mon, tm.tm_mday, + tm.tm_hour, tm.tm_min, tm.tm_sec, + ent[8])) + finally: + dir.close() + + +def do_add(cmd, mc, opts, args, opterr): + if len(args) < 1: + opterr("Filename required.") + if opts.directory != None: + mc.chdir(opts.directory) + for src in glob_args(args, glob): + f = open(src, "rb") + dest = os.path.basename(src) + out = mc.open(dest, "wb") + _copy(out, f) + out.close() + f.close() + +def do_extract(cmd, mc, opts, args, opterr): + if len(args) < 1: + opterr("Filename required.") + + if opts.directory != None: + mc.chdir(opts.directory) + + close_out = False + out = None + if opts.output != None: + if opts.use_stdout: + opterr("The -o and -p options are mutually exclusive.") + dont_close_out = True + out = file(opts.output, "wb") + elif opts.use_stdout: + out = sys.stdout + + try: + for filename in glob_args(args, mc.glob): + f = mc.open(filename, "rb") + try: + if out != None: + _copy(out, f) + continue + a = filename.split("/") + o = file(a[-1], "wb") + try: + _copy(o, f) + finally: + o.close() + finally: + f.close() + finally: + if close_out: + out.close() + +def do_mkdir(cmd, mc, opts, args, opterr): + if len(args) < 1: + opterr("Directory required.") + + for filename in args: + mc.mkdir(filename) + +def do_remove(cmd, mc, opts, args, opterr): + if len(args) < 1: + opterr("Filename required.") + + for filename in args: + mc.remove(filename) + +def do_import(cmd, mc, opts, args, opterr): + if len(args) < 1: + opterr("Filename required.") + + args = glob_args(args, glob) + if opts.directory != None and len(args) > 1: + opterr("The -d option can only be used with a" + "single savefile.") + + for filename in args: + sf = ps2save.ps2_save_file() + f = file(filename, "rb") + try: + ftype = ps2save.detect_file_type(f) + f.seek(0) + if ftype == "max": + sf.load_max_drive(f) + elif ftype == "psu": + sf.load_ems(f) + elif ftype == "cbs": + sf.load_codebreaker(f) + elif ftype == "sps": + sf.load_sharkport(f) + elif ftype == "npo": + raise io_error, (EIO, "nPort saves" + " are not supported.", + filename) + else: + raise io_error, (EIO, "Save file format not" + " recognized", filename) + finally: + f.close() + dirname = opts.directory + if dirname == None: + dirname = sf.get_directory()[8] + print "Importing", filename, "to", dirname + if not mc.import_save_file(sf, opts.ignore_existing, + opts.directory): + print (filename + ": already in memory card image," + " ignored.") + +#re_num = re.compile("[0-9]+") + +def do_export(cmd, mc, opts, args, opterr): + if len(args) < 1: + opterr("Directory name required") + + if opts.overwrite_existing and opts.ignore_existing: + opterr("The -i and -f options are mutually exclusive.") + + args = glob_args(args, mc.glob) + if opts.output_file != None: + if len(args) > 1: + opterr("Only one directory can be exported" + " when the -o option is used.") + if opts.longnames: + opterr("The -o and -l options are mutually exclusive.") + + if opts.directory != None: + os.chdir(opts.directory) + + for dirname in args: + sf = mc.export_save_file(dirname) + filename = opts.output_file + if opts.longnames: + filename = (ps2save.make_longname(dirname, sf) + + "." + opts.type) + if filename == None: + filename = dirname + "." + opts.type + + if not opts.overwrite_existing: + exists = True + try: + open(filename, "rb").close() + except EnvironmentError: + exists = False + if exists: + if opts.ignore_existing: + continue + raise io_error(EEXIST, "File exists", filename) + + f = file(filename, "wb") + try: + print "Exporing", dirname, "to", filename + + if opts.type == "max": + sf.save_max_drive(f) + else: + sf.save_ems(f) + finally: + f.close() + +def do_delete(cmd, mc, opts, args, opterr): + if len(args) < 1: + opterr("Directory required.") + + for dirname in args: + mc.rmdir(dirname) + +def do_setmode(cmd, mc, opts, args, opterr): + set_mask = 0 + clear_mask = ~0 + for (opt, bit) in [(opts.read, DF_READ), + (opts.write, DF_WRITE), + (opts.execute, DF_EXECUTE), + (opts.protected, DF_PROTECTED), + (opts.psx, DF_PSX), + (opts.pocketstation, DF_POCKETSTN), + (opts.hidden, DF_HIDDEN)]: + if opt != None: + if opt: + set_mask |= bit + else: + clear_mask ^= bit + + value = opts.hex_value + if set_mask == 0 and clear_mask == ~0: + if value == None: + opterr("At least one option must be given.") + if value.startswith("0x") or value.startswith("0X"): + value = int(value[2:], 16) + else: + value = int(value, 16) + else: + if value != None: + opterr("The -X option can't be combined with" + " other options.") + + for arg in glob_args(args, mc.glob): + ent = mc.get_dirent(arg) + if value == None: + ent[0] = (ent[0] & clear_mask) | set_mask + # print "new %04x" % ent[0] + else: + ent[0] = value + mc.set_dirent(arg, ent) + +def do_rename(cmd, mc, opts, args, opterr): + if len(args) != 2: + opterr("Old and new names required") + mc.rename(args[0], args[1]) + +def _get_ps2_title(mc, enc): + s = mc.get_icon_sys("."); + if s == None: + return None + a = ps2save.unpack_icon_sys(s) + return ps2save.icon_sys_title(a, enc) + +def _get_psx_title(mc, savename, enc): + mode = mc.get_mode(savename) + if mode == None or not mode_is_file(mode): + return None + f = mc.open(savename) + s = f.read(128) + if len(s) != 128: + return None + (magic, icon, blocks, title) = struct.unpack("<2sBB64s28x32x", s) + if magic != "SC": + return None + return [ps2save.shift_jis_conv(zero_terminate(title), enc), ""] + +def do_dir(cmd, mc, opts, args, opterr): + if len(args) != 0: + opterr("Incorrect number of arguments.") + f = None + dir = mc.dir_open("/") + try: + for ent in list(dir)[2:]: + dirmode = ent[0] + if not mode_is_dir(dirmode): + continue + dirname = "/" + ent[8] + mc.chdir(dirname) + length = mc.dir_size("."); + enc = getattr(sys.stdout, "encoding", None) + if dirmode & DF_PSX: + title = _get_psx_title(mc, ent[8], enc) + else: + title = _get_ps2_title(mc, enc) + if title == None: + title = ["Corrupt", ""] + protection = dirmode & (DF_PROTECTED | DF_WRITE) + if protection == 0: + protection = "Delete Protected" + elif protection == DF_WRITE: + protection = "Not Protected" + elif protection == DF_PROTECTED: + protection = "Copy & Delete Protected" + else: + protection = "Copy Protected" + + type = None + if dirmode & DF_PSX: + type = "PlayStation" + if dirmode & DF_POCKETSTN: + type = "PocketStation" + if type != None: + protection = type + + print "%-32s %s" % (ent[8], title[0]) + print ("%4dKB %-25s %s" + % (length / 1024, protection, title[1])) + print + finally: + if f != None: + f.close() + dir.close() + + free = mc.get_free_space() / 1024 + if free > 999999: + free = "%d,%03d,%03d" % (free / 1000000, free / 1000 % 1000, + free % 1000) + elif free > 999: + free = "%d,%03d" % (free / 1000, free % 1000) + else: + free = "%d" % free + + print free + " KB Free" + +def do_df(cmd, mc, opts, args, opterr): + if len(args) != 0: + opterr("Incorrect number of arguments.") + print mc.f.name + ":", mc.get_free_space(), "bytes free." + +def do_check(cmd, mc, opts, args, opterr): + if len(args) != 0: + opterr("Incorrect number of arguments.") + if mc.check(): + print "No errors found." + return 0 + return 1 + +def do_format(cmd, mcname, opts, args, opterr): + if len(args) != 0: + opterr("Incorrect number of arguments.") + pages_per_card = ps2mc.PS2MC_STANDARD_PAGES_PER_CARD + if opts.clusters != None: + pages_per_cluster = (ps2mc.PS2MC_CLUSTER_SIZE + / ps2mc.PS2MC_STANDARD_PAGE_SIZE) + pages_per_card = opts.clusters * pages_per_cluster + params = (not opts.no_ecc, + ps2mc.PS2MC_STANDARD_PAGE_SIZE, + ps2mc.PS2MC_STANDARD_PAGES_PER_ERASE_BLOCK, + pages_per_card) + + if not opts.overwrite_existing: + exists = True + try: + file(mcname, "rb").close() + except EnvironmentError: + exists = False + if exists: + raise io_error, (EEXIST, "file exists", mcname) + + f = file(mcname, "w+b") + try: + ps2mc.ps2mc(f, True, params).close() + finally: + f.close() + +def do_gui(cmd, mcname, opts, args, opterr): + if len(args) != 0: + opterr("Incorrect number of arguments.") + + try: + import gui + except ImportError: + write_error(None, "GUI not available") + return 1 + + gui.run(mcname) + return 0 + +def do_create_pad(cmd, mc, opts, args, opterr): + length = mc.clusters_per_card + if len(args) > 1: + length = int(args[1]) + pad = "\0" * mc.cluster_size + f = mc.open(args[0], "wb") + try: + for i in xrange(length): + f.write(pad) + finally: + f.close() + + +def do_frob(cmd, mc, opts, args, opterr): + mc.write_superblock() + +_trans = string.maketrans("".join(map(chr, range(32))), " " * 32) + +def _print_bin(base, s): + for off in range(0, len(s), 16): + print "%04X" % (base + off), + a = s[off : off + 16] + for b in a: + print "%02X" % ord(b), + print "", a.translate(_trans) + +def _print_erase_block(mc, n): + ppb = mc.pages_per_erase_block + base = n * ppb + for i in range(ppb): + s = mc.read_page(base + i) + _print_bin(i * mc.page_size, s) + print + +def do_print_good_blocks(cmd, mc, opts, args, opterr): + print "good_block2:" + _print_erase_block(mc, mc.good_block2) + print "good_block1:" + _print_erase_block(mc, mc.good_block1) + +def do_ecc_check(cmd, mc, opts, args, opterr): + for i in range(mc.clusters_per_card * mc.pages_per_cluster): + try: + mc.read_page(i) + except ps2mc.ecc_error: + print "bad: %05x" % i + +opt = optparse.make_option + +# +# Each value in the dictionary is a tuple consisting of: +# - function implementing the command +# - mode to use to open the ps2 save file +# - help description of the command +# - list of options supported by the command +# +cmd_table = { + "ls": (do_ls, "rb", + "[directory ...]", + "List the contents of a directory.", + [opt("-c", "--creation-time", action="store_true", + help = "Display creation times.")]), + "extract": (do_extract, "rb", + "filename ...", + "Extract files from the memory card.", + [opt("-o", "--output", metavar = "FILE", + help = 'Extract file to "FILE".'), + opt("-d", "--directory", + help = 'Extract files from "DIRECTORY".'), + opt("-p", "--use-stdout", action="store_true", + help = "Extract files to standard output.")]), + "add": (do_add, "r+b", + "filename ...", + "Add files to the memory card.", + [opt("-d", "--directory", + help = 'Add files to "directory".')]), + "mkdir": (do_mkdir, "r+b", + "directory ...", + "Make directories.", + []), + "remove": (do_remove, "r+b", + "filename ...", + "Remove files and directories.", + []), + "import": (do_import, "r+b", + "savefile ...", + "Import save files into the memory card.", + [opt("-i", "--ignore-existing", action="store_true", + help = ("Ignore files that already exist" + " on the image.")), + opt("-d", "--directory", metavar="DEST", + help = 'Import to "DEST".')]), + "export": (do_export, "rb", + "directory ...", + "Export save files from the memory card.", + [opt("-f", "--overwrite-existing", action = "store_true", + help = "Overwrite any save files already exported."), + opt("-i", "--ignore-existing", action = "store_true", + help = "Ingore any save files already exported."), + opt("-o", "--output-file", metavar = "filename", + help = 'Use "filename" as the name of the save file.'), + opt("-d", "--directory", metavar = "directory", + help = 'Export save files to "directory".'), + opt("-l", "--longnames", action = "store_true", + help = ("Generate longer, more descriptive," + " filenames.")), + opt("-p", "--ems", action = "store_const", + dest = "type", const = "psu", default = "psu", + help = "Use the EMS .psu save file format. [default]"), + opt("-m", "--max-drive", action = "store_const", + dest = "type", const = "max", + help = "Use the MAX Drive save file format.")]), + "delete": (do_delete, "r+b", + "dirname ...", + "Recursively delete a directory (save file).", + []), + "set": (do_setmode, "r+b", + "filename ...", + "Set mode flags on files and directories", + [opt("-p", "--protected", action="store_true", + help = "Set copy protected flag"), + opt("-P", "--psx", action="store_true", + help = "Set PSX flag"), + opt("-K", "--pocketstation", action="store_true", + help = "Set PocketStation flag"), + opt("-H", "--hidden", action="store_true", + help = "Set hidden flag"), + opt("-r", "--read", action="store_true", + help = "Set read allowed flag"), + opt("-w", "--write", action="store_true", + help = "Set write allowed flag"), + opt("-x", "--execute", action="store_true", + help = "Set executable flag"), + opt("-X", "--hex-value", metavar="mode", + help = 'Set mode to "mode".')]), + "clear": (do_setmode, "r+b", + "filename ...", + "Clear mode flags on files and directories", + [opt("-p", "--protected", action="store_false", + help = "Clear copy protected flag"), + opt("-P", "--psx", action="store_false", + help = "Clear PSX flag"), + opt("-K", "--pocketstation", action="store_false", + help = "Clear PocketStation flag"), + opt("-H", "--hidden", action="store_false", + help = "Clear hidden flag"), + opt("-r", "--read", action="store_false", + help = "Clear read allowed flag"), + opt("-w", "--write", action="store_false", + help = "Clear write allowed flag"), + opt("-x", "--execute", action="store_false", + help = "Clear executable flag"), + opt("-X", dest="hex_value", default=None, + help = optparse.SUPPRESS_HELP)]), + "rename": (do_rename, "r+b", + "oldname newname", + "Rename a file or directory", + []), + "dir": (do_dir, "rb", + None, + "Display save file information.", + []), + "df": (do_df, "rb", + None, + "Display the amount free space.", + []), + "check": (do_check, "rb", + "", + "Check for file system errors.", + []), + "format": (do_format, None, + "", + "Creates a new memory card image.", + [opt("-c", "--clusters", type="int", + help = "Size in clusters of the memory card."), + opt("-f", "--overwrite-existing", action="store_true", + help = "Overwrite any existing file"), + opt("-e", "--no-ecc", action="store_true", + help = "Create an image without ECC")]), + "gui": (do_gui, None, + "", + "Starts the graphical user interface.", + []), +} + +# +# secret commands for debugging purposes. +# +debug_cmd_table = { + "frob": (do_frob, "r+b", + "", + None, + []), + "print_good_blocks": (do_print_good_blocks, "rb", + "", + None, + []), + "ecc_check": (do_ecc_check, "rb", + "", + None, + []), + "create_pad": (do_create_pad, "r+b", + "", + None, + []) +} + +del opt # clean up name space + + +def write_error(filename, msg): + if filename == None: + sys.stderr.write(msg + "\n") + else: + sys.stderr.write(filename + ": " + msg + "\n") + +class suboption_parser(optparse.OptionParser): + def exit(self, status = 0, msg = None): + if msg: + sys.stderr.write(msg) + raise subopt_error, status + +class my_help_formatter(optparse.IndentedHelpFormatter): + """A better formatter for optparser's help message""" + + def format_description(self, description): + if not description: + return "" + desc_width = self.width - self.current_indent + indent = " " * self.current_indent + lines = [] + for line in description.split('\n'): + ii = indent + si = indent + if line.startswith("\t"): + line = line[1:] + ii = indent + " " * 4 + si = ii + " " * line.find(":") + 2 + line = textwrap.fill(line, desc_width, + initial_indent = ii, + subsequent_indent = si) + lines.append(line) + return "\n".join(lines) + "\n" + +def main(): + prog = sys.argv[0].decode(sys.getdefaultencoding(), "replace") + usage = "usage: %prog [-ih] memcard.ps2 command [...]" + description = ("Manipulate PS2 memory card images.\n\n" + "Supported commands: ") + for cmd in sorted(cmd_table.keys()): + description += "\n " + cmd + ": " + cmd_table[cmd][3] + + version = ("mymc " + + verbuild.MYMC_VERSION_MAJOR + + "." + verbuild.MYMC_VERSION_BUILD + + " (" + _SCCS_ID + ")") + + optparser = optparse.OptionParser(prog = prog, usage = usage, + description = description, + version = version, + formatter = my_help_formatter()) + optparser.add_option("-D", dest = "debug", action = "store_true", + default = False, help = optparse.SUPPRESS_HELP) + optparser.add_option("-i", "--ignore-ecc", action = "store_true", + help = "Ignore ECC errors while reading.") + + optparser.disable_interspersed_args() + (opts, args) = optparser.parse_args() + + if len(args) == 0: + try: + import gui + except ImportError: + gui = None + if gui != None: + optparser.destroy() + gui.run() + sys.exit(0) + + if len(args) < 2: + optparser.error("Incorrect number of arguments.") + + if opts.debug: + cmd_table.update(debug_cmd_table) + cmd = args[1] + if cmd not in cmd_table: + optparser.error('Command "%s" not recognized.' % cmd) + (fn, mode, usage_args, description, optlist) = cmd_table[cmd] + + usage = "%prog" + if len(optlist) > 0: + usage += " [options]" + if usage_args != None: + usage += " " + usage_args + subprog = prog + " memcard.ps2 " + cmd + subopt_parser = suboption_parser(prog = subprog, usage = usage, + description = description, + option_list = optlist) + subopt_parser.disable_interspersed_args() + + f = None + mc = None + ret = 0 + mcname = args[0] + + try: + (subopts, subargs) = subopt_parser.parse_args(args[2:]) + try: + if mode == None: + ret = fn(cmd, mcname, subopts, subargs, + subopt_parser.error) + else: + f = file(mcname, mode) + mc = ps2mc.ps2mc(f, opts.ignore_ecc) + ret = fn(cmd, mc, subopts, subargs, + subopt_parser.error) + finally: + if mc != None: + mc.close() + if f != None: + # print "f.close()" + f.close() + + except EnvironmentError, value: + if getattr(value, "filename", None) != None: + write_error(value.filename, value.strerror) + ret = 1 + elif getattr(value, "strerror", None) != None: + write_error(mcname, value.strerror) + ret = 1 + else: + # something weird + raise + if opts.debug: + raise + + except subopt_error, (ret,): + pass + + except (ps2mc.error, ps2save.error), value: + fn = getattr(value, "filename", None) + if fn == None: + fn = mcname + write_error(fn, str(value)) + if opts.debug: + raise + ret = 1 + + if ret == None: + ret = 0 + + return ret + +sys.exit(main()) + diff --git a/ps2mc.py b/ps2mc.py index 6165229..8e3ca21 100755 --- a/ps2mc.py +++ b/ps2mc.py @@ -1,1873 +1,1985 @@ -# -# ps2mc.py -# -# By Ross Ridge -# Public Domain -# - -"""Manipulate PS2 memory card images.""" - -_SCCS_ID = "@(#) mysc ps2mc.py 1.10 12/10/04 19:10:35\n" - -import sys -import array -import struct -from errno import EACCES, ENOENT, EEXIST, ENOTDIR, EISDIR, EROFS, ENOTEMPTY,\ - ENOSPC, EIO, EBUSY -import fnmatch -import traceback - -from round import * -from ps2mc_ecc import * -from ps2mc_dir import * -import ps2save - -PS2MC_MAGIC = "Sony PS2 Memory Card Format " -PS2MC_FAT_ALLOCATED_BIT = 0x80000000 -PS2MC_FAT_CHAIN_END = 0xFFFFFFFF -PS2MC_FAT_CHAIN_END_UNALLOC = 0x7FFFFFFF -PS2MC_FAT_CLUSTER_MASK = 0x7FFFFFFF -PS2MC_MAX_INDIRECT_FAT_CLUSTERS = 32 -PS2MC_CLUSTER_SIZE = 1024 -PS2MC_INDIRECT_FAT_OFFSET = 0x2000 - -PS2MC_STANDARD_PAGE_SIZE = 512 -PS2MC_STANDARD_PAGES_PER_CARD = 16384 -PS2MC_STANDARD_PAGES_PER_ERASE_BLOCK = 16 - -class error(Exception): - pass - -class io_error(error, IOError): - def __init__(self, *args, **kwargs): - IOError.__init__(self, *args, **kwargs) - - def __str__(self): - if getattr(self, "strerror", None) == None: - return str(self.args) - if getattr(self, "filename", None) != None: - return self.filename + ": " + self.strerror - return self.strerror - -class path_not_found(io_error): - def __init__(self, filename): - io_error.__init__(self, ENOENT, "path not found", filename) - -class file_not_found(io_error): - def __init__(self, filename): - io_error.__init__(self, ENOENT, "file not found", filename) - -class dir_not_found(io_error): - def __init__(self, filename): - io_error.__init__(self, ENOENT, "directory not found", - filename) - -class dir_index_not_found(io_error, IndexError): - def __init__(self, filename, index): - msg = "index (%d) past of end of directory" % index - io_error.__init__(self, ENOENT, msg, filename) - -class corrupt(io_error): - def __init__(self, msg, f = None): - filename = None - if f != None: - filename = getattr(f, "name") - io_error.__init__(self, EIO, msg, filename) - -class ecc_error(corrupt): - def __init__(self, msg, filename = None): - corrupt.__init__(self, msg, filename) - -if sys.byteorder == "big": - def unpack_32bit_array(s): - a = array.array('I', s) - a.byteswap() - - def pack_32bit_array(a): - a = a[:] - a.byteswap() - return a.tostring() -else: - def unpack_32bit_array(s): - #if isinstance(s, str): - # a = array.array('L') - # a.fromstring(s) - # return a - return array.array('I', s) - - def pack_32bit_array(a): - return a.tostring() - -def unpack_superblock(s): - sb = struct.unpack("<28s12sHHHHLLLLLL8x128s128sbbxx", s) - sb = list(sb) - sb[12] = unpack_32bit_array(sb[12]) - sb[13] = unpack_32bit_array(sb[13]) - return sb - -def pack_superblock(sb): - sb = list(sb) - sb[12] = pack_32bit_array(sb[12]) - sb[13] = pack_32bit_array(sb[13]) - return struct.pack("<28s12sHHHHLLLLLL8x128s128sbbxx", *sb) - -unpack_fat = unpack_32bit_array -pack_fat = pack_32bit_array - -class lru_cache(object): - def __init__(self, length): - self._lru_list = [[i - 1, None, None, i + 1] - for i in range(length + 1)] - self._index_map = {} - - def dump(self): - lru_list = self._lru_list - i = 0 - while i != len(self._lru_list): - print "%d: %s, " % (i, str(lru_list[i][1])), - i = lru_list[i][3] - print - print self._index_map - - def _move_to_front(self, i): - lru_list = self._lru_list - first = lru_list[0] - i2 = first[3] - if i != i2: - elt = lru_list[i] - prev = lru_list[elt[0]] - next = lru_list[elt[3]] - prev[3] = elt[3] - next[0] = elt[0] - elt[0] = 0 - elt[3] = i2 - lru_list[i2][0] = i - first[3] = i - - def add(self, key, value): - lru_list = self._lru_list - index_map = self._index_map - ret = None - if key in index_map: - i = index_map[key] - # print "add hit ", key, i - elt = lru_list[i] - else: - # print "add miss", key - i = lru_list[-1][0] - elt = lru_list[i] - old_key = elt[1] - if old_key != None: - del index_map[old_key] - ret = (old_key, elt[2]) - index_map[key] = i - elt[1] = key - elt[2] = value - self._move_to_front(i) - - return ret - - def get(self, key, default = None): - i = self._index_map.get(key) - if i == None: - # print "get miss", key - return default - # print "get hit ", key, i - ret = self._lru_list[i][2] - self._move_to_front(i) - return ret - - def items(self): - return [(elt[1], elt[2]) - for elt in self._lru_list[1 : -1] - if elt[2] != None] - -class fat_chain(object): - """A class for accessing a file's FAT entries as a simple sequence.""" - - def __init__(self, lookup_fat, first): - self.lookup_fat = lookup_fat - self._first = first - self.offset = 0 - self._prev = None - self._cur = first - - def __getitem__(self, i): - # not iterable - offset = self.offset - if i == offset: - # print "@@@ fat_chain[] cur:", i, self._cur - return self._cur - elif i == offset - 1: - assert self._prev != None - # print "@@@ fat_chain[] prev:", i, self._prev - return self._prev - if i < offset: - if i == 0: - # print "@@@ fat_chain[] first", i, self._first - return self._first - offset = 0 - prev = None - cur = self._first - else: - prev = self._prev - cur = self._cur - # print "@@@ fat_chain[] distance", i - offset - while offset != i: - next = self.lookup_fat(cur) - if next == PS2MC_FAT_CHAIN_END: - break; - if next & PS2MC_FAT_ALLOCATED_BIT: - next &= ~PS2MC_FAT_ALLOCATED_BIT - else: - # corrupt - next = PS2MC_FAT_CHAIN_END - break - - offset += 1 - prev = cur - cur = next - self.offset = offset - self._prev = prev - self._cur = cur - # print "@@@ offset, prev, cur:", offset, prev, cur - # print "@@@ fat_chain[]", i, next - return next - - def __len__(self): - old_prev = self._prev - old_cur = self._cur - old_offset = self.offset - i = self.offset - while self[i] != PS2MC_FAT_CHAIN_END: - i += 1 - self._prev = old_prev - self._cur = old_cur - self.offset = old_offset - return i - -class ps2mc_file(object): - """A file-like object for accessing a file in memory card image.""" - - def __init__(self, mc, dirloc, first_cluster, length, mode, - name = None): - # print "ps2mc_file.__init__", name, self - self.mc = mc - self.length = length - self.first_cluster = first_cluster - self.dirloc = dirloc - self.fat_chain = None - self._pos = 0 - self.buffer = None - self.buffer_cluster = None - self.softspace = 0 - if name == None: - self.name = "" - else: - self.name = name - self.closed = False - - if mode == None or len(mode) == 0: - mode = "rb" - self.mode = mode - self._append = False - self._write = False - if mode[0] == "a": - self._append = True - elif mode[0] != "w" or ("+" not in self.mode): - self._write = True - - def _find_file_cluster(self, n): - if self.fat_chain == None: - self.fat_chain = self.mc.fat_chain(self.first_cluster) - return self.fat_chain[n] - - def read_file_cluster(self, n): - if n == self.buffer_cluster: - return self.buffer - cluster = self._find_file_cluster(n) - # print "@@@ read_file_cluster", self.dirloc, n, cluster, repr(self.name) - if cluster == PS2MC_FAT_CHAIN_END: - return None - self.buffer = self.mc.read_allocatable_cluster(cluster) - self.buffer_cluster = n - return self.buffer - - def _extend_file(self, n): - mc = self.mc - cluster = mc.allocate_cluster() - # print "@@@ extending file", n, cluster - if cluster == None: - return None - if n == 0: - self.first_cluster = cluster - self.fat_chain = None - # print "@@@ linking", self.dirloc, "->", cluster - mc.update_dirent(self.dirloc, self, cluster, - None, False) - else: - prev = self.fat_chain[n - 1] - # print "@@@ linking", prev, "->", cluster - mc.set_fat(prev, cluster | PS2MC_FAT_ALLOCATED_BIT) - return cluster - - def write_file_cluster(self, n, buf): - mc = self.mc - cluster = self._find_file_cluster(n) - if cluster != PS2MC_FAT_CHAIN_END: - mc.write_allocatable_cluster(cluster, buf) - self.buffer = buf - self.buffer_cluster = n - return True - - cluster_size = mc.cluster_size - file_cluster_end = div_round_up(self.length, cluster_size) - - if (cluster < file_cluster_end - or len(self.fat_chain) != file_cluster_end): - raise corrupt, ("file length doesn't match cluster" - " chain length", mc.f) - - for i in range(file_cluster_end, n): - cluster = self._extend_file(i) - if cluster == None: - if i != file_cluster_end: - self.length = (i - 1) * cluster_size - mc.update_dirent(self.dirloc, self, - None, self.length, - True) - return False - mc.write_allocatable_cluster(cluster, - ["\0"] * cluster_size) - - cluster = self._extend_file(n) - if cluster == None: - return False - - mc.write_allocatable_cluster(cluster, buf) - self.buffer = buf - self.buffer_cluster = n - return True - - def update_notify(self, first_cluster, length): - if self.first_cluster != first_cluster: - self.first_cluster = first_cluster - self.fat_chain = None - self.length = length - self.buffer = None - self.buffer_cluster = None - - def read(self, size = None, eol = None): - if self.closed: - raise ValueError, "file is closed" - - pos = self._pos - cluster_size = self.mc.cluster_size - if size == None: - size = self.length - size = max(min(self.length - pos, size), 0) - ret = "" - while size > 0: - off = pos % cluster_size - l = min(cluster_size - off, size) - buf = self.read_file_cluster(pos / cluster_size) - if buf == None: - break - if eol != None: - i = buf.find(eol, off, off + l) - if i != -1: - l = off - i + 1 - size = l - pos += l - self._pos = pos - ret += buf[off : off + l] - size -= l - return ret - - def write(self, out, _set_modified = True): - if self.closed: - raise ValueError, "file is closed" - - cluster_size = self.mc.cluster_size - pos = self._pos - if self._append: - pos = self.length - elif not self._write: - raise io_error, (EACCES, "file not opened for writing", - self.name) - - size = len(out) - # print "@@@ write", pos, size - i = 0 - while size > 0: - cluster = pos / cluster_size - off = pos % cluster_size - l = min(cluster_size - off, size) - s = out[i : i + l] - pos += l - if l == cluster_size: - buf = s - else: - buf = self.read_file_cluster(cluster) - if buf == None: - buf = "\0" * cluster_size - buf = buf[:off] + s + buf[off + l:] - if not self.write_file_cluster(cluster, buf): - raise io_error, (ENOSPC, - "out of space on image", - self.name) - self._pos = pos - # print "@@@ pos", pos - new_length = None - if pos > self.length: - new_length = self.length = pos - self.mc.update_dirent(self.dirloc, self, None, - new_length, _set_modified) - - i += l - size -= l - - def close(self): - # print "ps2mc_file.close", self.name, self - if self.mc != None: - self.mc.notify_closed(self.dirloc, self) - self.mc = None - self.fat_chain = None - self.buffer = None - - def next(self): - r = self.readline() - if r == "": - raise StopIteration - return r - - def readline(self, size = None): - return self.read(size, "\n") - - def readlines(self, sizehint): - return [line for line in self] - - def seek(self, offset, whence = 0): - if self.closed: - raise ValueError, "file is closed" - - if whence == 1: - base = self._pos - elif whence == 2: - base = self.length - else: - base = 0 - pos = max(base + offset, 0) - self._pos = pos - - def tell(self): - if self.closed: - raise ValueError, "file is closed" - return self._pos - - def __enter__(self): - return - - def __exit__(self, a, b, c): - self.close() - return - - # def __del__(self): - # # print "ps2mc_file.__del__", self - # if self.mc != None: - # self.mc.notify_closed(self.dirloc, self) - # self.mc = None - # self.fat_chain = None - -class ps2mc_directory(object): - """A sequence and iterator object for directories.""" - - def __init__(self, mc, dirloc, first_cluster, length, - mode = "rb", name = None): - self.f = ps2mc_file(mc, dirloc, first_cluster, - length * PS2MC_DIRENT_LENGTH, mode, name) - - def __iter__(self): - start = self.tell() - if start != 0: - start -= 1 - self.seek(start) - self._iter_end = start - return self - - def write_raw_ent(self, index, ent, set_modified): - # print "@@@ write_raw_ent", index - self.seek(index) - self.f.write(pack_dirent(ent), - _set_modified = set_modified) - - def next(self): - # print "@@@ next", self.tell(), self.f.name - dirent = self.f.read(PS2MC_DIRENT_LENGTH) - if dirent == "": - if 0 == self._iter_end: - raise StopIteration - self.seek(0) - dirent = self.f.read(PS2MC_DIRENT_LENGTH) - elif self.tell() == self._iter_end: - raise StopIteration - return unpack_dirent(dirent) - - def seek(self, offset, whence = 0): - self.f.seek(offset * PS2MC_DIRENT_LENGTH, whence) - - def tell(self): - return self.f.tell() / PS2MC_DIRENT_LENGTH - - def __len__(self): - return self.f.length / PS2MC_DIRENT_LENGTH - - def __getitem__(self, index): - # print "@@@ getitem", index, self.f.name - self.seek(index) - dirent = self.f.read(PS2MC_DIRENT_LENGTH) - if len(dirent) != PS2MC_DIRENT_LENGTH: - raise dir_index_not_found(self.f.name, index) - return unpack_dirent(dirent) - - def __setitem__(self, index, new_ent): - ent = self[index] - mode = ent[0] - if (mode & DF_EXISTS) == 0: - return - if new_ent[0] != None: - mode = ((new_ent[0] & ~(DF_FILE | DF_DIR | DF_EXISTS)) - | (mode & (DF_FILE | DF_DIR | DF_EXISTS))) - ent[0] = mode - if new_ent[1] != None: - ent[1] = new_ent[1] - if new_ent[3] != None: - ent[3] = new_ent[3] - if new_ent[6] != None: - ent[6] = new_ent[6] - if new_ent[7] != None: - ent[7] = new_ent[7] - self.write_raw_ent(index, ent, False) - - def close(self): - # print "ps2mc_directory.close", self - self.f.close() - self.f = None - - def __del__(self): - # print "ps2mc_directory.__del__", self - if self.f != None: - self.f.close() - self.f = None - -class _root_directory(ps2mc_directory): - """Wrapper for the cached root directory object. - - The close() method is disabled so the cached object can be reused.""" - - def __init__(self, mc, dirloc, first_cluster, length, - mode = "r+b", name = "/"): - ps2mc_directory.__init__(self, mc, dirloc, first_cluster, - length, mode, name) - - def close(self): - pass - - def real_close(self): - ps2mc_directory.close(self) - -class ps2mc(object): - """A PlayStation 2 memory card filesystem implementation. - - The close() method must be called when the object is no longer needed, - otherwise cycles that can't be collected by the garbage collector - will remain.""" - - open_files = None - fat_cache = None - - def _calculate_derived(self): - self.spare_size = div_round_up(self.page_size, 128) * 4 - self.raw_page_size = self.page_size + self.spare_size - self.cluster_size = self.page_size * self.pages_per_cluster - self.entries_per_cluster = (self.page_size - * self.pages_per_cluster / 4) - - limit = (min(self.good_block2, self.good_block1) - * self.pages_per_erase_block - / self.pages_per_cluster - - self.allocatable_cluster_offset) - self.allocatable_cluster_limit = limit - - def __init__(self, f, ignore_ecc = False, params = None): - self.open_files = {} - self.fat_cache = lru_cache(12) - self.alloc_cluster_cache = lru_cache(64) - self.modified = False - self.f = None - self.rootdir = None - - f.seek(0) - s = f.read(0x154) - if len(s) != 0x154 or not s.startswith(PS2MC_MAGIC): - if (params == None): - raise corrupt, ("Not a PS2 memory card image", - f) - self.f = f - self.format(params) - else: - sb = unpack_superblock(s) - self.version = sb[1] - self.page_size = sb[2] - self.pages_per_cluster = sb[3] - self.pages_per_erase_block = sb[4] - self.clusters_per_card = sb[6] - self.allocatable_cluster_offset = sb[7] - self.allocatable_cluster_end = sb[8] - self.rootdir_fat_cluster = sb[9] - self.good_block1 = sb[10] - self.good_block2 = sb[11] - self.indirect_fat_cluster_list = sb[12] - self.bad_erase_block_list = sb[13] - - self._calculate_derived() - - self.f = f - self.ignore_ecc = False - - try: - self.read_page(0) - self.ignore_ecc = ignore_ecc - except ecc_error: - # the error might be due the fact the file - # image doesn't contain ECC data - self.spare_size = 0 - self.raw_page_size = self.page_size - ignore_ecc = True - - # sanity check - root = self._directory(None, 0, 1) - dot = root[0] - dotdot = root[1] - root.close() - if (dot[8] != "." or dotdot[8] != ".." - or not mode_is_dir(dot[0]) or not mode_is_dir(dotdot[0])): - raise corrupt, "Root directory damaged." - - self.fat_cursor = 0 - self.curdir = (0, 0) - - def write_superblock(self): - s = pack_superblock((PS2MC_MAGIC, - self.version, - self.page_size, - self.pages_per_cluster, - self.pages_per_erase_block, - 0xFF00, - self.clusters_per_card, - self.allocatable_cluster_offset, - self.allocatable_cluster_end, - self.rootdir_fat_cluster, - self.good_block1, - self.good_block2, - self.indirect_fat_cluster_list, - self.bad_erase_block_list, - 2, - 0x2B)) - s += "\x00" * (self.page_size - len(s)) - self.write_page(0, s) - - page = "\xFF" * self.raw_page_size - self.f.seek(self.good_block2 * self.pages_per_erase_block - * self.raw_page_size) - for i in range(self.pages_per_erase_block): - self.f.write(page) - - self.modified = False - return - - def format(self, params): - """Create (format) a new memory card image.""" - - (with_ecc, page_size, - pages_per_erase_block, param_pages_per_card) = params - - if pages_per_erase_block < 1: - raise error, ("invalid pages per erase block (%d)" - % page_size) - - pages_per_card = round_down(param_pages_per_card, - pages_per_erase_block) - cluster_size = PS2MC_CLUSTER_SIZE - pages_per_cluster = cluster_size / page_size - clusters_per_erase_block = (pages_per_erase_block - / pages_per_cluster) - erase_blocks_per_card = pages_per_card / pages_per_erase_block - clusters_per_card = pages_per_card / pages_per_cluster - epc = cluster_size / 4 - - if (page_size < PS2MC_DIRENT_LENGTH - or pages_per_cluster < 1 - or pages_per_cluster * page_size != cluster_size): - raise error, "invalid page size (%d)" % page_size - - good_block1 = erase_blocks_per_card - 1 - good_block2 = erase_blocks_per_card - 2 - first_ifc = div_round_up(PS2MC_INDIRECT_FAT_OFFSET, - cluster_size) - - allocatable_clusters = clusters_per_card - (first_ifc + 2) - fat_clusters = div_round_up(allocatable_clusters, epc) - indirect_fat_clusters = div_round_up(fat_clusters, epc) - if indirect_fat_clusters > PS2MC_MAX_INDIRECT_FAT_CLUSTERS: - indirect_fat_clusters = PS2MC_MAX_INDIRECT_FAT_CLUSTERS - fat_clusters = indirect_fat_clusters * epc - allocatable_clusters = fat_clusters * epc - - allocatable_cluster_offset = (first_ifc - + indirect_fat_clusters - + fat_clusters) - allocatable_cluster_end = (good_block2 - * clusters_per_erase_block - - allocatable_cluster_offset) - if allocatable_cluster_end < 1: - raise error, ("memory card image too small" - " to be formatted") - - ifc_list = unpack_fat("\0\0\0\0" - * PS2MC_MAX_INDIRECT_FAT_CLUSTERS) - for i in range(indirect_fat_clusters): - ifc_list[i] = first_ifc + i - - self.version = "1.2.0.0" - self.page_size = page_size - self.pages_per_cluster = pages_per_cluster - self.pages_per_erase_block = pages_per_erase_block - self.clusters_per_card = clusters_per_card - self.allocatable_cluster_offset = allocatable_cluster_offset - self.allocatable_cluster_end = allocatable_clusters - self.rootdir_fat_cluster = 0 - self.good_block1 = good_block1 - self.good_block2 = good_block2 - self.indirect_fat_cluster_list = ifc_list - bebl = "\xFF\xFF\xFF\xFF" * 32 - self.bad_erase_block_list = unpack_32bit_array(bebl) - - self._calculate_derived() - - self.ignore_ecc = not with_ecc - erased = "\0" * page_size - if not with_ecc: - self.spare_size = 0 - else: - ecc = "".join(["".join(map(chr, s)) - for s in ecc_calculate_page(erased)]) - erased += ecc + "\0" * (self.spare_size - len(ecc)) - - self.f.seek(0) - for page in range(pages_per_card): - self.f.write(erased) - - self.modified = True - - first_fat_cluster = first_ifc + indirect_fat_clusters - remainder = fat_clusters % epc - for i in range(indirect_fat_clusters): - base = first_fat_cluster + i * epc - buf = unpack_fat(range(base, base + epc)) - if (i == indirect_fat_clusters - 1 - and remainder != 0): - del buf[remainder:] - buf.fromlist([0xFFFFFFFF] * (epc - remainder)) - self._write_fat_cluster(ifc_list[i], buf) - - - # go through the fat backwards for better cache usage - for i in range(allocatable_clusters - 1, - allocatable_cluster_end - 1, -1): - self.set_fat(i, PS2MC_FAT_CHAIN_END) - for i in range(allocatable_cluster_end - 1, 0, -1): - self.set_fat(i, PS2MC_FAT_CLUSTER_MASK) - self.set_fat(0, PS2MC_FAT_CHAIN_END) - - self.allocatable_cluster_end = allocatable_cluster_end - - now = tod_now() - s = pack_dirent((DF_RWX | DF_DIR | DF_0400 | DF_EXISTS, - 0, 2, now, - 0, 0, now, 0, ".")) - s += "\0" * (cluster_size - len(s)) - self.write_allocatable_cluster(0, s) - dir = self._directory((0, 0), 0, 2, "wb", "/") - dir.write_raw_ent(1, (DF_WRITE | DF_EXECUTE | DF_DIR | DF_0400 - | DF_HIDDEN | DF_EXISTS, - 0, 0, now, - 0, 0, now, 0, ".."), False) - dir.close() - - self.flush() - - def read_page(self, n): - # print "@@@ page", n - f = self.f - f.seek(self.raw_page_size * n) - page = f.read(self.page_size) - if len(page) != self.page_size: - raise corrupt, ("attempted to read past EOF" - " (page %05X)" % n, f) - if self.ignore_ecc: - return page - spare = f.read(self.spare_size) - if len(spare) != self.spare_size: - raise corrupt, ("attempted to read past EOF" - " (page %05X)" % n, f) - (status, page, spare) = ecc_check_page(page, spare) - if status == ECC_CHECK_FAILED: - raise ecc_error, ("Unrecoverable ECC error (page %d)" - % n) - return page - - def write_page(self, n, buf): - f = self.f - f.seek(self.raw_page_size * n) - self.modified = True - if len(buf) != self.page_size: - raise error, ("internal error: write_page:" - " %d != %d" % (len(buf), self.page_size)) - f.write(buf) - if self.spare_size != 0: - a = array.array('B') - for s in ecc_calculate_page(buf): - a.fromlist(s) - a.tofile(f) - f.write("\0" * (self.spare_size - len(a))) - - def read_cluster(self, n): - pages_per_cluster = self.pages_per_cluster - cluster_size = self.cluster_size - if self.spare_size == 0: - self.f.seek(cluster_size * n) - return self.f.read(cluster_size) - n *= pages_per_cluster - if pages_per_cluster == 2: - return self.read_page(n) + self.read_page(n + 1) - return "".join(map(self.read_page, - range(n, n + pages_per_cluster))) - - def write_cluster(self, n, buf): - pages_per_cluster = self.pages_per_cluster - cluster_size = self.cluster_size - if self.spare_size == 0: - self.f.seek(cluster_size * n) - if len(buf) != cluster_size: - raise error, ("internal error: write_cluster:" - " %d != %d" % (len(buf), - cluster_size)) - return self.f.write(buf) - n *= pages_per_cluster - pgsize = self.page_size - for i in range(pages_per_cluster): - self.write_page(n + i, buf[i * pgsize - : i * pgsize + pgsize]) - - - def _add_fat_cluster_to_cache(self, n, fat, dirty): - old = self.fat_cache.add(n, [fat, dirty]) - if old != None: - (n, [fat, dirty]) = old - if dirty: - self.write_cluster(n, pack_fat(fat)) - - def _read_fat_cluster(self, n): - v = self.fat_cache.get(n) - if v != None: - # print "@@@ fat hit", n - return v[0] - # print "@@@ fat miss", n - fat = unpack_fat(self.read_cluster(n)) - self._add_fat_cluster_to_cache(n, fat, False) - return fat - - def _write_fat_cluster(self, n, fat): - self._add_fat_cluster_to_cache(n, fat, True) - - def flush_fat_cache(self): - if self.fat_cache == None: - return - for (n, v) in self.fat_cache.items(): - [fat, dirty] = v - if dirty: - self.write_cluster(n, pack_fat(fat)) - v[1] = False - - def _add_alloc_cluster_to_cache(self, n, buf, dirty): - old = self.alloc_cluster_cache.add(n, [buf, dirty]) - if old != None: - (n, [buf, dirty]) = old - if dirty: - n += self.allocatable_cluster_offset - self.write_cluster(n, buf) - - def read_allocatable_cluster(self, n): - a = self.alloc_cluster_cache.get(n) - if a != None: - # print "@@@ cache hit", n - return a[0] - # print "@@@ cache miss", n - buf = self.read_cluster(n + self.allocatable_cluster_offset) - self._add_alloc_cluster_to_cache(n, buf, False) - return buf - - def write_allocatable_cluster(self, n, buf): - self._add_alloc_cluster_to_cache(n, buf, True) - - def flush_alloc_cluster_cache(self): - if self.alloc_cluster_cache == None: - return - for (n, a) in self.alloc_cluster_cache.items(): - [buf, dirty] = a - if dirty: - n += self.allocatable_cluster_offset - self.write_cluster(n, buf) - a[1] = False - - def read_fat_cluster(self, n): - indirect_offset = n % self.entries_per_cluster - dbl_offset = n / self.entries_per_cluster - indirect_cluster = self.indirect_fat_cluster_list[dbl_offset] - indirect_fat = self._read_fat_cluster(indirect_cluster) - cluster = indirect_fat[indirect_offset] - return (self._read_fat_cluster(cluster), cluster) - - def read_fat(self, n): - if n < 0 or n >= self.allocatable_cluster_end: - raise io_error, (EIO, - "FAT cluster index out of range" - " (%d)" % n) - offset = n % self.entries_per_cluster - fat_cluster = n / self.entries_per_cluster - (fat, cluster) = self.read_fat_cluster(fat_cluster) - return (fat, offset, cluster) - - def lookup_fat(self, n): - (fat, offset, cluster) = self.read_fat(n) - return fat[offset] - - def set_fat(self, n, value): - (fat, offset, cluster) = self.read_fat(n) - fat[offset] = value - self._write_fat_cluster(cluster, fat) - - def allocate_cluster(self): - epc = self.entries_per_cluster - allocatable_cluster_limit = self.allocatable_cluster_limit - - end = div_round_up(allocatable_cluster_limit, epc) - remainder = allocatable_cluster_limit % epc - - while self.fat_cursor < end: - (fat, cluster) = self.read_fat_cluster(self.fat_cursor) - if (self.fat_cursor == end - 1 - and remainder != 0): - n = min(fat[:remainder]) - else: - n = min(fat) - if (n & PS2MC_FAT_ALLOCATED_BIT) == 0: - offset = fat.index(n) - fat[offset] = PS2MC_FAT_CHAIN_END - self._write_fat_cluster(cluster, fat) - ret = self.fat_cursor * epc + offset - # print "@@@ allocated", ret - return ret - self.fat_cursor += 1 - return None - - def fat_chain(self, first_cluster): - return fat_chain(self.lookup_fat, first_cluster) - - def file(self, dirloc, first_cluster, length, mode, name = None): - """Create a new file-like object for a file.""" - - f = ps2mc_file(self, dirloc, first_cluster, length, mode, name) - if dirloc == None: - return - open_files = self.open_files - if dirloc not in open_files: - open_files[dirloc] = [None, set([f])] - else: - open_files[dirloc][1].add(f) - return f - - def directory(self, dirloc, first_cluster, length, - mode = None, name = None): - return ps2mc_directory(self, dirloc, first_cluster, length, - mode, name) - - def _directory(self, dirloc, first_cluster, length, - mode = None, name = None): - # print "@@@ _directory", dirloc, first_cluster, length - if first_cluster != 0: - return self.directory(dirloc, first_cluster, length, - mode, name) - if dirloc == None: - dirloc = (0, 0) - assert dirloc == (0, 0) - if self.rootdir != None: - return self.rootdir - dir = _root_directory(self, dirloc, 0, length, "r+b", "/") - l = dir[0][2] - if l != length: - dir.real_close() - dir = _root_directory(self, dirloc, 0, l, "r+b", "/") - self.rootdir = dir - return dir - - def _get_parent_dirloc(self, dirloc): - """Get the dirloc of the parent directory of the - file or directory refered to by dirloc""" - - cluster = self.read_allocatable_cluster(dirloc[0]) - ent = unpack_dirent(cluster[:PS2MC_DIRENT_LENGTH]) - return (ent[4], ent[5]) - - def _dirloc_to_ent(self, dirloc): - """Get the directory entry of the file or directory - refered to by dirloc""" - - dir = self._directory(None, dirloc[0], dirloc[1] + 1, - name = "_dirloc_to_ent temp") - ent = dir[dirloc[1]] - dir.close() - return ent - - def _opendir_dirloc(self, dirloc, mode = "rb"): - """Open the directory that is refered to by dirloc""" - - ent = self._dirloc_to_ent(dirloc) - return self._directory(dirloc, ent[4], ent[2], - name = "_opendir_dirloc temp") - - def _opendir_parent_dirloc(self, dirloc, mode = "rb"): - """Open the directory that contains the file or directory - refered to by dirloc""" - - return self._opendir_dirloc(self._get_parent_dirloc(dirloc), - mode) - - def update_dirent_all(self, dirloc, thisf, new_ent): - # print "@@@ update_dirent", dirloc - # print "@@@ new_ent", new_ent - opened = self.open_files.get(dirloc, None) - if opened == None: - files = [] - dir = None - else: - dir, files = opened - if dir == None: - dir = self._opendir_parent_dirloc(dirloc, "r+b") - if opened != None: - opened[0] = dir - - ent = dir[dirloc[1]] - # print "@@@ old_ent", ent - - is_dir = ent[0] & DF_DIR - - if is_dir and thisf != None and new_ent[2] != None: - new_ent = list(new_ent) - new_ent[2] /= PS2MC_DIRENT_LENGTH - - # print "len: ", ent[2], new_ent[2] - - modified = changed = notify = False - for i in range(len(ent)): - new = new_ent[i] - if new != None: - if new != ent[i]: - ent[i] = new - changed = True - if i == 6: - modified = True - if i in [2, 4]: - notify = True - - # Modifying a file causes the modification time of - # both the file and the file's directory to updated, - # however modifying a directory never updates the - # modification time of the directory's parent. - if changed: - dir.write_raw_ent(dirloc[1], ent, - (modified and not is_dir)) - - - if notify: - for f in files: - if f != thisf: - f.update_notfiy(ent[4], ent[2]) - if opened == None: - dir.close() - - def update_dirent(self, dirloc, thisf, first_cluster, length, - modified): - if modified: - modified = tod_now() - else: - if first_cluster == None and length == None: - return - modified = None - self.update_dirent_all(dirloc, thisf, - (None, None, length, None, - first_cluster, None, modified, None, - None)) - - def notify_closed(self, dirloc, thisf): - if self.open_files == None or dirloc == None: - return - a = self.open_files.get(dirloc, None) - if a == None: - return - self.flush() - dir, files = a - files.discard(thisf) - if len(files) == 0: - # print "@@@ notify_closed", dir - if dir != None: - dir.close() - del self.open_files[dirloc] - - def search_directory(self, dir, name): - """Search dir for name.""" - - # start the search where the last search ended. - start = dir.tell() - 1 - if start == -1: - start = 0 - for i in range(start, len(dir)) + range(0, start): - try: - ent = dir[i] - except IndexError: - raise corrupt("Corrupt directory", dir.f) - - if ent[8] == name and (ent[0] & DF_EXISTS): - return (i, ent) - return (None, None) - - def create_dir_entry(self, parent_dirloc, name, mode): - """Create a new directory entry in a directory.""" - - # print "@@@ create_dir_ent", parent_dirloc, name - dir_ent = self._dirloc_to_ent(parent_dirloc) - dir = self._directory(parent_dirloc, dir_ent[4], dir_ent[2], - "r+b") - l = len(dir) - # print "@@@ len", l - assert l >= 2 - for i in range(l): - ent = dir[i] - if (ent[0] & DF_EXISTS) == 0: - break - else: - i = l - - dirloc = (dir_ent[4], i) - # print "@@@ dirloc", dirloc - now = tod_now() - if mode & DF_DIR: - mode &= ~DF_FILE - cluster = self.allocate_cluster() - length = 1 - else: - mode |= DF_FILE - mode &= ~DF_DIR - cluster = PS2MC_FAT_CHAIN_END - length = 0 - ent[0] = mode | DF_EXISTS - ent[1] = 0 - ent[2] = length - ent[3] = now - ent[4] = cluster - ent[5] = 0 - ent[6] = now - ent[7] = 0 - ent[8] = name[:32] - dir.write_raw_ent(i, ent, True) - dir.close() - - if mode & DF_FILE: - # print "@@@ ret", dirloc, ent - return (dirloc, ent) - - dirent = pack_dirent((DF_RWX | DF_0400 | DF_DIR | DF_EXISTS, - 0, 0, now, dirloc[0], dirloc[1], - now, 0, ".")) - dirent += "\0" * (self.cluster_size - PS2MC_DIRENT_LENGTH) - self.write_allocatable_cluster(cluster, dirent) - dir = self._directory(dirloc, cluster, 1, "wb", - name = "") - dir.write_raw_ent(1, (DF_RWX | DF_0400 | DF_DIR | DF_EXISTS, - 0, 0, now, - 0, 0, - now, 0, ".."), False) - dir.close() - ent[2] = 2 - # print "@@@ ret", dirloc, ent - return (dirloc, ent) - - def delete_dirloc(self, dirloc, truncate, name): - """Delete or truncate the file or directory given by dirloc.""" - - if dirloc == (0, 0): - raise io_error, (EACCES, - "cannot remove root directory", - name) - if dirloc[1] in [0, 1]: - raise io_error, (EACCES, - 'cannot remove "." or ".." entries', - name) - - if dirloc in self.open_files: - raise io_error, (EBUSY, - "cannot remove open file", filename) - - epc = self.entries_per_cluster - - ent = self._dirloc_to_ent(dirloc) - cluster = ent[4] - if truncate: - ent[2] = 0 - ent[4] = PS2MC_FAT_CHAIN_END - ent[6] = tod_now() - else: - ent[0] &= ~DF_EXISTS - self.update_dirent_all(dirloc, None, ent) - - while cluster != PS2MC_FAT_CHAIN_END: - if cluster / epc < self.fat_cursor: - self.fat_cursor = cluster / epc - next_cluster = self.lookup_fat(cluster) - if next_cluster & PS2MC_FAT_ALLOCATED_BIT == 0: - # corrupted - break - next_cluster &= ~PS2MC_FAT_ALLOCATED_BIT - self.set_fat(cluster, next_cluster) - if next_cluster == PS2MC_FAT_CHAIN_END_UNALLOC: - break - cluster = next_cluster - - def path_search(self, pathname): - """Parse and resolve a pathname. - - Return a tuple containing a tuple containing three - values. The first is either the dirloc of the file or - directory, if it exists, otherwise it's the dirloc the - pathname's parent directory, if that exists otherwise - it's None. The second component is directory entry - for pathname if it exists, otherwise None. The third - is a boolean value that's true if the pathname refers - a directory.""" - - components = pathname.split("/") - if len(components) < 1: - # could return curdir - return (None, None, False) - - dirloc = self.curdir - if components[0] == "": - dirloc = (0, 0) - if dirloc == (0, 0): - rootent = self.read_allocatable_cluster(0) - ent = unpack_dirent(rootent[:PS2MC_DIRENT_LENGTH]) - dir_cluster = 0 - dir = self._directory(dirloc, dir_cluster, ent[2], - name = "") - else: - ent = self._dirloc_to_ent(dirloc) - dir = self._directory(dirloc, ent[4], ent[2], - name = "") - - for s in components: - # print "@@@", dirloc, repr(s), dir == None, ent - if s == "": - continue - - if dir == None: - # tried to traverse a file or a - # non-existent directory - return (None, None, False) - - if s == "" or s == ".": - continue - if s == "..": - dotent = dir[0] - dir.close() - dirloc = (dotent[4], dotent[5]) - ent = self._dirloc_to_ent(dirloc) - dir = self._directory(dirloc, ent[4], ent[2], - name - = "") - continue - - dir_cluster = ent[4] - (i, ent) = self.search_directory(dir, s) - dir.close() - dir = None - - if ent == None: - continue - - dirloc = (dir_cluster, i) - if ent[0] & DF_DIR: - dir = self._directory(dirloc, ent[4], ent[2], - name - = "") - - if dir != None: - dir.close() - - return (dirloc, ent, dir != None) - - def open(self, filename, mode = "r"): - """Open a file, returning a new file-like object for it.""" - - (dirloc, ent, is_dir) = self.path_search(filename) - # print "@@@ open", (dirloc, ent) - if dirloc == None or (ent == None and is_dir): - raise path_not_found, filename - if is_dir: - raise io_error, (EISDIR, "not a regular file", - filename) - if ent == None: - if mode[0] not in "wa": - raise file_not_found, filename - name = filename.split("/")[-1] - (dirloc, ent) = self.create_dir_entry(dirloc, name, - DF_FILE | DF_RWX - | DF_0400); - self.flush() - elif mode[0] == "w": - self.delete_dirloc(dirloc, True, filename) - ent[4] = PS2MC_FAT_CHAIN_END - ent[2] = 0 - return self.file(dirloc, ent[4], ent[2], mode, filename) - - def dir_open(self, filename, mode = "rb"): - (dirloc, ent, is_dir) = self.path_search(filename) - if dirloc == None: - raise path_not_found, filename - if ent == None: - raise dir_not_found, filename - if not is_dir: - raise io_error, (ENOTDIR, "not a directory", filename) - return self.directory(dirloc, ent[4], ent[2], mode, filename) - - def mkdir(self, filename): - (dirloc, ent, is_dir) = self.path_search(filename) - if dirloc == None: - raise path_not_found, filename - if ent != None: - raise io_error, (EEXIST, "directory exists", filename) - a = filename.split("/") - name = a.pop() - while name == "": - name = a.pop() - self.create_dir_entry(dirloc, name, DF_DIR | DF_RWX | DF_0400) - self.flush() - - def _is_empty(self, dirloc, ent, filename): - """Check if a directory is empty.""" - - dir = self._directory(dirloc, ent[4], ent[2], "rb", - filename) - try: - for i in range(2, len(dir)): - if dir[i][0] & DF_EXISTS: - return False - finally: - dir.close() - return True - - def remove(self, filename): - """Remove a file or empty directory.""" - - (dirloc, ent, is_dir) = self.path_search(filename) - if dirloc == None: - raise path_not_found, filename - if ent == None: - raise file_not_found, filename - if is_dir: - if ent[4] == 0: - raise io_error, (EACCES, - "cannot remove" - " root directory") - if not self._is_empty(dirloc, ent, filename): - raise io_error, (ENOTEMPTY, - "directory not empty", - filename) - self.delete_dirloc(dirloc, False, filename) - self.flush() - - def chdir(self, filename): - (dirloc, ent, is_dir) = self.path_search(filename) - if dirloc == None: - raise path_not_found, filename - if ent == None: - raise dir_not_found, filename - if not is_dir: - raise io_error, (ENOTDIR, "not a directory", filename) - self.curdir = dirloc - - def get_mode(self, filename): - """Get mode bits of a file. - - Returns None if the filename doesn't exist, rather than - throwing a error.""" - - (dirloc, ent, is_dir) = self.path_search(filename) - if ent == None: - return None - return ent[0] - - def get_dirent(self, filename): - """Get the raw directory entry tuple for a file.""" - - (dirloc, ent, is_dir) = self.path_search(filename) - if dirloc == None: - raise path_not_found, filename - if ent == None: - raise file_not_found, filename - return ent - - def set_dirent(self, filename, new_ent): - """Set various directory entry fields of a file. - - Not all fields can be changed. If a field in new_ent - is set to None then is not changed.""" - - (dirloc, ent, is_dir) = self.path_search(filename) - if dirloc == None: - raise path_not_found, filename - if ent == None: - raise file_not_found, filename - dir = self._opendir_parent_dirloc(dirloc) - try: - dir[dirloc[1]] = new_ent - finally: - dir.close() - self.flush() - return ent - - def import_save_file(self, sf, ignore_existing, dirname = None): - """Copy the contents a ps2_save_file object to a directory. - - If ingore_existing is true and the directory being imported - to already exists then False is returned instead of raising - an error. If dirname is given then the save file is copied - to that directory instead of the directory specified by - the save file. - """ - - dir_ent = sf.get_directory() - if dirname == None: - dir_ent_name = dir_ent[8] - dirname = "/" + dir_ent[8] - else: - if dirname == "": - raise path_not_found, dirname - - # remove trailing slashes - dirname = dirname.rstrip("/") - if dirname == "": - dirname = "/" - dir_ent_name = dirname.split("/")[0] - - (root_dirloc, ent, is_dir) = self.path_search(dirname) - if root_dirloc == None: - raise path_not_found, dirname - if ent != None: - if ignore_existing: - return False - raise io_error, (EEXIST, "directory exists", dirname) - mode = DF_DIR | (dir_ent[0] & ~DF_FILE) - - (dir_dirloc, ent) = self.create_dir_entry(root_dirloc, - dir_ent_name, - mode) - try: - assert dirname != "/" - dirname = dirname + "/" - for i in range(dir_ent[2]): - (ent, data) = sf.get_file(i) - mode = DF_FILE | (ent[0] & ~DF_DIR) - (dirloc, ent) \ - = self.create_dir_entry(dir_dirloc, - ent[8], mode) - # print "@@@ file", dirloc, ent[4], ent[2] - f = self.file(dirloc, ent[4], ent[2], "wb", - dirname + ent[8]) - try: - f.write(data) - finally: - f.close() - except EnvironmentError: - type, what, where = sys.exc_info() - try: - try: - for i in range(dir_ent[2]): - (ent, data) = sf.get_file(i) - # print "@@@ remove", ent[8] - self.remove(dirname + ent[8]) - except EnvironmentError, why: - # print "@@@ failed", why - pass - - try: - # print "@@@ remove dir", dirname - self.remove(dirname) - except EnvironmentError, why: - # print "@@@ failed", why - pass - raise type, what, where - finally: - del where - - # set modes and timestamps to those of the save file - - dir = self._opendir_dirloc(dir_dirloc, "r+b") - try: - for i in range(dir_ent[2]): - dir[i + 2] = sf.get_file(i)[0] - finally: - dir.close() - - dir = self._opendir_dirloc(root_dirloc, "r+b") - try: - dir[dir_dirloc[1]] = dir_ent - finally: - dir.close() - - self.flush() - return True - - def export_save_file(self, filename): - (dir_dirloc, dirent, is_dir) = self.path_search(filename) - if dir_dirloc == None: - raise path_not_found, filename - if dirent == None: - raise dir_not_found, filename - if not is_dir: - raise io_error, (ENOTDIR, "not a directory", filename) - if dir_dirloc == (0, 0): - raise io_error, (EACCES, "can't export root directory", - filename) - sf = ps2save.ps2_save_file() - files = [] - f = None - dir = self._directory(dir_dirloc, dirent[4], dirent[2], - "rb", filename) - try: - for i in range(2, dirent[2]): - ent = dir[i] - if not mode_is_file(ent[0]): - print ("warning: %s/%s is not a file," - " ingored." - % (dirent[8], ent[8])) - continue - f = self.file((dirent[4], i), ent[4], ent[2], - "rb") - data = f.read(ent[2]) - f.close() - assert len(data) == ent[2] - files.append((ent, data)) - finally: - if f != None: - f.close() - dir.close() - dirent[2] = len(files) - sf.set_directory(dirent) - for (i, (ent, data)) in enumerate(files): - sf.set_file(i, ent, data) - return sf - - def _remove_dir(self, dirloc, ent, dirname): - """Recurse over a directory tree to remove it. - If not "", dirname must end with a slash (/).""" - - first_cluster = ent[4] - length = ent[2] - dir = self._directory(dirloc, first_cluster, length, - "rb", dirname) - try: - ents = list(enumerate(dir)) - finally: - dir.close() - for (i, ent) in ents[2:]: - mode = ent[0] - if not (mode & DF_EXISTS): - continue - if mode & DF_DIR: - self._remove_dir((first_cluster, i), ent, - dirname + ent[8] + "/") - else: - # print "deleting", dirname + ent[8] - self.delete_dirloc((first_cluster, i), False, - dirname + ent[8]) - self.delete_dirloc(dirloc, False, dirname) - - def rmdir(self, dirname): - """Recursively delete a directory.""" - - (dirloc, ent, is_dir) = self.path_search(dirname) - if dirloc == None: - raise path_not_found, dirname - if ent == None: - raise dir_not_found, dirname - if not is_dir: - raise io_error, (ENOTDIR, "not a directory", dirname) - if dirloc == (0, 0): - raise io_error, (EACCES, "can't delete root directory", - dirname) - - if dirname != "" and dirname[-1] != "/": - dirname += "/" - self._remove_dir(dirloc, ent, dirname) - - def get_free_space(self): - """Returns the amount of free space in bytes.""" - - free = 0 - for i in xrange(self.allocatable_cluster_end): - if (self.lookup_fat(i) & PS2MC_FAT_ALLOCATED_BIT) == 0: - free += 1 - return free * self.cluster_size - - def get_allocatable_space(self): - """Returns the total amount of allocatable space in bytes.""" - return self.allocatable_cluster_limit * self.cluster_size - - def _check_file(self, fat, first_cluster, length): - cluster = first_cluster - i = 0 - while cluster != PS2MC_FAT_CHAIN_END: - if cluster < 0 or cluster >= len(fat): - return "invalid cluster in chain" - if fat[cluster]: - return "cross linked chain" - i += 1 - # print cluster, - fat[cluster] = 1 - next = self.lookup_fat(cluster) - if next == PS2MC_FAT_CHAIN_END: - break - if (next & PS2MC_FAT_ALLOCATED_BIT) == 0: - return "unallocated cluster in chain" - cluster = next & ~PS2MC_FAT_ALLOCATED_BIT - file_cluster_end = div_round_up(length, self.cluster_size) - if i < file_cluster_end: - return "chain ends before end of file" - elif i > file_cluster_end: - return "chain continues after end of file" - return None - - def _check_dir(self, fat, dirloc, dirname, ent): - why = self._check_file(fat, ent[4], - ent[2] * PS2MC_DIRENT_LENGTH) - if why != None: - print "bad directory:", dirname + ":", why - return False - ret = True - first_cluster = ent[4] - length = ent[2] - dir = self._directory(dirloc, first_cluster, length, - "rb", dirname) - dot_ent = dir[0] - if dot_ent[8] != ".": - print "bad directory:", dirname + ': missing "." entry' - ret = False - if (dot_ent[4], dot_ent[5]) != dirloc: - print "bad directory:", dirname + ': bad "." entry' - ret = False - if dir[1][8] != "..": - print "bad directory:", (dirname - + ': missing ".." entry') - ret = False - for i in xrange(2, length): - ent = dir[i] - mode = ent[0] - if not (mode & DF_EXISTS): - continue - if mode & DF_DIR: - if not self._check_dir(fat, (first_cluster, i), - dirname + ent[8] + "/", - ent): - ret = False - else: - why = self._check_file(fat, ent[4], ent[2]) - if why != None: - print "bad file:", (dirname + ent[8] - + ":"), why - ret = False - - dir.close() - return ret - - def check(self): - """Run a simple file system check. - - Any problems found are reported to stdout.""" - - ret = True - - fat_len = int(str(self.allocatable_cluster_end)) - if not isinstance(fat_len, int): - raise error, "Memory card image too big to check." - - fat = array.array('B', [0]) * fat_len - - cluster = self.read_allocatable_cluster(0) - ent = unpack_dirent(cluster[:PS2MC_DIRENT_LENGTH]) - ret = self._check_dir(fat, (0, 0), "/", ent) - - lost_clusters = 0 - for i in xrange(self.allocatable_cluster_end): - a = self.lookup_fat(i) - if (a & PS2MC_FAT_ALLOCATED_BIT) and not fat[i]: - print i, - lost_clusters += 1 - if lost_clusters > 0: - print - print "found", lost_clusters, "lost clusters" - ret = False - - return ret - - def _glob(self, dirname, components): - pattern = components[0] - if len(components) == 1: - if pattern == "": - return [dirname] - dir = self.dir_open(dirname) - try: - return [dirname + ent[8] - for ent in dir - if ((ent[0] & DF_EXISTS) - and (ent[8] not in [".", ".."] - or ent[8] == pattern) - and fnmatch.fnmatchcase(ent[8], - pattern))] - finally: - dir.close() - if pattern == "": - return self._glob(dirname + "/", components[1:]) - if dirname == "": - dir = self.dir_open(".") - else: - dir = self.dir_open(dirname) - try: - ret = [] - for ent in dir: - name = ent[8] - if ((ent[0] & DF_EXISTS) == 0 - or (ent[0] & DF_DIR) == 0): - continue - if name == "." or name == "..": - if pattern != name: - continue - elif not fnmatch.fnmatchcase(name, pattern): - continue - ret += self._glob(dirname + name + "/", - components[1:]) - finally: - dir.close() - return ret - - def glob(self, pattern): - if pattern == "": - return [] - ret = self._glob("", pattern.split("/")) - # print pattern, "->", ret - return self._glob("", pattern.split("/")) - - def get_icon_sys(self, dirname): - """Get contents of a directory's icon.sys file, if it exits.""" - - icon_sys = dirname + "/icon.sys" - mode = self.get_mode(icon_sys) - if mode == None or not mode_is_file(mode): - return None - f = self.open(icon_sys, "rb") - s = f.read(964) - f.close() - if len(s) == 964 and s[0:4] == "PS2D": - return s; - return None - - def dir_size(self, dirname): - """Calculate the total size of the contents of a directory.""" - - dir = self.dir_open(dirname) - try: - length = round_up(len(dir) * PS2MC_DIRENT_LENGTH, - self.cluster_size) - for ent in dir: - if mode_is_file(ent[0]): - length += round_up(ent[2], - self.cluster_size) - elif (mode_is_dir(ent[0]) - and ent[8] not in [".", ".."]): - length += self.dir_size(dirname + "/" - + ent[8]) - finally: - dir.close() - return length - - def flush(self): - self.flush_alloc_cluster_cache() - self.flush_fat_cache() - if self.modified: - self.write_superblock() - self.f.flush() - - def close(self): - """Close all open files. - - Disconnects, but doesn't close the file object used - access the raw image. After this method has been - called on a ps2mc object, it can no longer be used.""" - - # print "ps2mc.close" - try: - f = self.f - if f == None or getattr(f, "closed", False): - # print "closed" - return - open_files = self.open_files - # print "open_files", open_files - if open_files != None: - # this is complicated by the fact as - # files are closed they will remove - # themselves from the list of open files - for (dir, files) in open_files.values(): - for f in list(files): - f.close() - while len(open_files) > 0: - (k, v) = open_files.popitem() - (dir, files) = v - if dir != None: - dir.close() - if self.rootdir != None: - self.rootdir.close() - if self.fat_cache != None: - self.flush() - finally: - self.open_files = None - self.fat_cache = None - self.f = None - self.rootdir = None - - def __del__(self): - # print "ps2mc.__del__" - try: - self.close() - except: - sys.stderr.write("ps2mc.__del__: \n") - traceback.print_exc() +# +# ps2mc.py +# +# By Ross Ridge +# Public Domain +# + +"""Manipulate PS2 memory card images.""" + +_SCCS_ID = "@(#) mymc ps2mc.py 1.11 22/01/15 01:17:07\n" + +import sys +import array +import struct +from errno import EACCES, ENOENT, EEXIST, ENOTDIR, EISDIR, EROFS, ENOTEMPTY,\ + ENOSPC, EIO, EBUSY, EINVAL +import fnmatch +import traceback + +from round import * +from ps2mc_ecc import * +from ps2mc_dir import * +import ps2save + +PS2MC_MAGIC = "Sony PS2 Memory Card Format " +PS2MC_FAT_ALLOCATED_BIT = 0x80000000 +PS2MC_FAT_CHAIN_END = 0xFFFFFFFF +PS2MC_FAT_CHAIN_END_UNALLOC = 0x7FFFFFFF +PS2MC_FAT_CLUSTER_MASK = 0x7FFFFFFF +PS2MC_MAX_INDIRECT_FAT_CLUSTERS = 32 +PS2MC_CLUSTER_SIZE = 1024 +PS2MC_INDIRECT_FAT_OFFSET = 0x2000 + +PS2MC_STANDARD_PAGE_SIZE = 512 +PS2MC_STANDARD_PAGES_PER_CARD = 16384 +PS2MC_STANDARD_PAGES_PER_ERASE_BLOCK = 16 + +class error(Exception): + pass + +class io_error(error, IOError): + def __init__(self, *args, **kwargs): + IOError.__init__(self, *args, **kwargs) + + def __str__(self): + if getattr(self, "strerror", None) == None: + return str(self.args) + if getattr(self, "filename", None) != None: + return self.filename + ": " + self.strerror + return self.strerror + +class path_not_found(io_error): + def __init__(self, filename): + io_error.__init__(self, ENOENT, "path not found", filename) + +class file_not_found(io_error): + def __init__(self, filename): + io_error.__init__(self, ENOENT, "file not found", filename) + +class dir_not_found(io_error): + def __init__(self, filename): + io_error.__init__(self, ENOENT, "directory not found", + filename) + +class dir_index_not_found(io_error, IndexError): + def __init__(self, filename, index): + msg = "index (%d) past of end of directory" % index + io_error.__init__(self, ENOENT, msg, filename) + +class corrupt(io_error): + def __init__(self, msg, f = None): + filename = None + if f != None: + filename = getattr(f, "name") + io_error.__init__(self, EIO, msg, filename) + +class ecc_error(corrupt): + def __init__(self, msg, filename = None): + corrupt.__init__(self, msg, filename) + +if sys.byteorder == "big": + def unpack_32bit_array(s): + a = array.array('I', s) + a.byteswap() + return a + + def pack_32bit_array(a): + a = a[:] + a.byteswap() + return a.tostring() +else: + def unpack_32bit_array(s): + #if isinstance(s, str): + # a = array.array('L') + # a.fromstring(s) + # return a + return array.array('I', s) + + def pack_32bit_array(a): + return a.tostring() + +def unpack_superblock(s): + sb = struct.unpack("<28s12sHHHHLLLLLL8x128s128sbbxx", s) + sb = list(sb) + sb[12] = unpack_32bit_array(sb[12]) + sb[13] = unpack_32bit_array(sb[13]) + return sb + +def pack_superblock(sb): + sb = list(sb) + sb[12] = pack_32bit_array(sb[12]) + sb[13] = pack_32bit_array(sb[13]) + return struct.pack("<28s12sHHHHLLLLLL8x128s128sbbxx", *sb) + +unpack_fat = unpack_32bit_array +pack_fat = pack_32bit_array + +def pathname_split(pathname): + if pathname == "": + return (None, False, False) + components = pathname.split("/") + return ([name + for name in components + if name != ""], + components[0] != "", + components[-1] == "") + +class lru_cache(object): + def __init__(self, length): + self._lru_list = [[i - 1, None, None, i + 1] + for i in range(length + 1)] + self._index_map = {} + + def dump(self): + lru_list = self._lru_list + i = 0 + while i != len(self._lru_list): + print "%d: %s, " % (i, str(lru_list[i][1])), + i = lru_list[i][3] + print + print self._index_map + + def _move_to_front(self, i): + lru_list = self._lru_list + first = lru_list[0] + i2 = first[3] + if i != i2: + elt = lru_list[i] + prev = lru_list[elt[0]] + next = lru_list[elt[3]] + prev[3] = elt[3] + next[0] = elt[0] + elt[0] = 0 + elt[3] = i2 + lru_list[i2][0] = i + first[3] = i + + def add(self, key, value): + lru_list = self._lru_list + index_map = self._index_map + ret = None + if key in index_map: + i = index_map[key] + # print "add hit ", key, i + elt = lru_list[i] + else: + # print "add miss", key + i = lru_list[-1][0] + elt = lru_list[i] + old_key = elt[1] + if old_key != None: + del index_map[old_key] + ret = (old_key, elt[2]) + index_map[key] = i + elt[1] = key + elt[2] = value + self._move_to_front(i) + + return ret + + def get(self, key, default = None): + i = self._index_map.get(key) + if i == None: + # print "get miss", key + return default + # print "get hit ", key, i + ret = self._lru_list[i][2] + self._move_to_front(i) + return ret + + def items(self): + return [(elt[1], elt[2]) + for elt in self._lru_list[1 : -1] + if elt[2] != None] + +class fat_chain(object): + """A class for accessing a file's FAT entries as a simple sequence.""" + + def __init__(self, lookup_fat, first): + self.lookup_fat = lookup_fat + self._first = first + self.offset = 0 + self._prev = None + self._cur = first + + def __getitem__(self, i): + # not iterable + offset = self.offset + if i == offset: + # print "@@@ fat_chain[] cur:", i, self._cur + return self._cur + elif i == offset - 1: + assert self._prev != None + # print "@@@ fat_chain[] prev:", i, self._prev + return self._prev + if i < offset: + if i == 0: + # print "@@@ fat_chain[] first", i, self._first + return self._first + offset = 0 + prev = None + cur = self._first + else: + prev = self._prev + cur = self._cur + # print "@@@ fat_chain[] distance", i - offset + while offset != i: + next = self.lookup_fat(cur) + if next == PS2MC_FAT_CHAIN_END: + break; + if next & PS2MC_FAT_ALLOCATED_BIT: + next &= ~PS2MC_FAT_ALLOCATED_BIT + else: + # corrupt + next = PS2MC_FAT_CHAIN_END + break + + offset += 1 + prev = cur + cur = next + self.offset = offset + self._prev = prev + self._cur = cur + # print "@@@ offset, prev, cur:", offset, prev, cur + # print "@@@ fat_chain[]", i, next + return next + + def __len__(self): + old_prev = self._prev + old_cur = self._cur + old_offset = self.offset + i = self.offset + while self[i] != PS2MC_FAT_CHAIN_END: + i += 1 + self._prev = old_prev + self._cur = old_cur + self.offset = old_offset + return i + +class ps2mc_file(object): + """A file-like object for accessing a file in memory card image.""" + + def __init__(self, mc, dirloc, first_cluster, length, mode, + name = None): + # print "ps2mc_file.__init__", name, self + self.mc = mc + self.length = length + self.first_cluster = first_cluster + self.dirloc = dirloc + self.fat_chain = None + self._pos = 0 + self.buffer = None + self.buffer_cluster = None + self.softspace = 0 + if name == None: + self.name = "" + else: + self.name = name + self.closed = False + + if mode == None or len(mode) == 0: + mode = "rb" + self.mode = mode + self._append = False + self._write = False + if mode[0] == "a": + self._append = True + elif mode[0] != "w" or ("+" not in self.mode): + self._write = True + + def _find_file_cluster(self, n): + if self.fat_chain == None: + self.fat_chain = self.mc.fat_chain(self.first_cluster) + return self.fat_chain[n] + + def read_file_cluster(self, n): + if n == self.buffer_cluster: + return self.buffer + cluster = self._find_file_cluster(n) + # print "@@@ read_file_cluster", self.dirloc, n, cluster, repr(self.name) + if cluster == PS2MC_FAT_CHAIN_END: + return None + self.buffer = self.mc.read_allocatable_cluster(cluster) + self.buffer_cluster = n + return self.buffer + + def _extend_file(self, n): + mc = self.mc + cluster = mc.allocate_cluster() + # print "@@@ extending file", n, cluster + if cluster == None: + return None + if n == 0: + self.first_cluster = cluster + self.fat_chain = None + # print "@@@ linking", self.dirloc, "->", cluster + mc.update_dirent(self.dirloc, self, cluster, + None, False) + else: + prev = self.fat_chain[n - 1] + # print "@@@ linking", prev, "->", cluster + mc.set_fat(prev, cluster | PS2MC_FAT_ALLOCATED_BIT) + return cluster + + def write_file_cluster(self, n, buf): + mc = self.mc + cluster = self._find_file_cluster(n) + if cluster != PS2MC_FAT_CHAIN_END: + mc.write_allocatable_cluster(cluster, buf) + self.buffer = buf + self.buffer_cluster = n + return True + + cluster_size = mc.cluster_size + file_cluster_end = div_round_up(self.length, cluster_size) + + if (cluster < file_cluster_end + or len(self.fat_chain) != file_cluster_end): + raise corrupt, ("file length doesn't match cluster" + " chain length", mc.f) + + for i in range(file_cluster_end, n): + cluster = self._extend_file(i) + if cluster == None: + if i != file_cluster_end: + self.length = (i - 1) * cluster_size + mc.update_dirent(self.dirloc, self, + None, self.length, + True) + return False + mc.write_allocatable_cluster(cluster, + ["\0"] * cluster_size) + + cluster = self._extend_file(n) + if cluster == None: + return False + + mc.write_allocatable_cluster(cluster, buf) + self.buffer = buf + self.buffer_cluster = n + return True + + def update_notify(self, first_cluster, length): + if self.first_cluster != first_cluster: + self.first_cluster = first_cluster + self.fat_chain = None + self.length = length + self.buffer = None + self.buffer_cluster = None + + def read(self, size = None, eol = None): + if self.closed: + raise ValueError, "file is closed" + + pos = self._pos + cluster_size = self.mc.cluster_size + if size == None: + size = self.length + size = max(min(self.length - pos, size), 0) + ret = "" + while size > 0: + off = pos % cluster_size + l = min(cluster_size - off, size) + buf = self.read_file_cluster(pos / cluster_size) + if buf == None: + break + if eol != None: + i = buf.find(eol, off, off + l) + if i != -1: + l = off - i + 1 + size = l + pos += l + self._pos = pos + ret += buf[off : off + l] + size -= l + return ret + + def write(self, out, _set_modified = True): + if self.closed: + raise ValueError, "file is closed" + + cluster_size = self.mc.cluster_size + pos = self._pos + if self._append: + pos = self.length + elif not self._write: + raise io_error, (EACCES, "file not opened for writing", + self.name) + + size = len(out) + # print "@@@ write", pos, size + i = 0 + while size > 0: + cluster = pos / cluster_size + off = pos % cluster_size + l = min(cluster_size - off, size) + s = out[i : i + l] + pos += l + if l == cluster_size: + buf = s + else: + buf = self.read_file_cluster(cluster) + if buf == None: + buf = "\0" * cluster_size + buf = buf[:off] + s + buf[off + l:] + if not self.write_file_cluster(cluster, buf): + raise io_error, (ENOSPC, + "out of space on image", + self.name) + self._pos = pos + # print "@@@ pos", pos + new_length = None + if pos > self.length: + new_length = self.length = pos + self.mc.update_dirent(self.dirloc, self, None, + new_length, _set_modified) + + i += l + size -= l + + def close(self): + # print "ps2mc_file.close", self.name, self + if self.mc != None: + self.mc.notify_closed(self.dirloc, self) + self.mc = None + self.fat_chain = None + self.buffer = None + + def next(self): + r = self.readline() + if r == "": + raise StopIteration + return r + + def readline(self, size = None): + return self.read(size, "\n") + + def readlines(self, sizehint): + return [line for line in self] + + def seek(self, offset, whence = 0): + if self.closed: + raise ValueError, "file is closed" + + if whence == 1: + base = self._pos + elif whence == 2: + base = self.length + else: + base = 0 + pos = max(base + offset, 0) + self._pos = pos + + def tell(self): + if self.closed: + raise ValueError, "file is closed" + return self._pos + + def __enter__(self): + return + + def __exit__(self, a, b, c): + self.close() + return + + # def __del__(self): + # # print "ps2mc_file.__del__", self + # if self.mc != None: + # self.mc.notify_closed(self.dirloc, self) + # self.mc = None + # self.fat_chain = None + +class ps2mc_directory(object): + """A sequence and iterator object for directories.""" + + def __init__(self, mc, dirloc, first_cluster, length, + mode = "rb", name = None): + self.f = ps2mc_file(mc, dirloc, first_cluster, + length * PS2MC_DIRENT_LENGTH, mode, name) + + def __iter__(self): + start = self.tell() + if start != 0: + start -= 1 + self.seek(start) + self._iter_end = start + return self + + def write_raw_ent(self, index, ent, set_modified): + # print "@@@ write_raw_ent", index + self.seek(index) + self.f.write(pack_dirent(ent), + _set_modified = set_modified) + + def next(self): + # print "@@@ next", self.tell(), self.f.name + dirent = self.f.read(PS2MC_DIRENT_LENGTH) + if dirent == "": + if 0 == self._iter_end: + raise StopIteration + self.seek(0) + dirent = self.f.read(PS2MC_DIRENT_LENGTH) + elif self.tell() == self._iter_end: + raise StopIteration + return unpack_dirent(dirent) + + def seek(self, offset, whence = 0): + self.f.seek(offset * PS2MC_DIRENT_LENGTH, whence) + + def tell(self): + return self.f.tell() / PS2MC_DIRENT_LENGTH + + def __len__(self): + return self.f.length / PS2MC_DIRENT_LENGTH + + def __getitem__(self, index): + # print "@@@ getitem", index, self.f.name + self.seek(index) + dirent = self.f.read(PS2MC_DIRENT_LENGTH) + if len(dirent) != PS2MC_DIRENT_LENGTH: + raise dir_index_not_found(self.f.name, index) + return unpack_dirent(dirent) + + def __setitem__(self, index, new_ent): + ent = self[index] + mode = ent[0] + if (mode & DF_EXISTS) == 0: + return + if new_ent[0] != None: + mode = ((new_ent[0] & ~(DF_FILE | DF_DIR | DF_EXISTS)) + | (mode & (DF_FILE | DF_DIR | DF_EXISTS))) + ent[0] = mode + for i in [1, 3, 6, 7, 8]: # ???, created, modifed, attr + if new_ent[i] != None: + ent[i] = new_ent[i] + self.write_raw_ent(index, ent, False) + + def close(self): + # print "ps2mc_directory.close", self + self.f.close() + self.f = None + + def __del__(self): + # print "ps2mc_directory.__del__", self + if self.f != None: + self.f.close() + self.f = None + +class _root_directory(ps2mc_directory): + """Wrapper for the cached root directory object. + + The close() method is disabled so the cached object can be reused.""" + + def __init__(self, mc, dirloc, first_cluster, length, + mode = "r+b", name = "/"): + ps2mc_directory.__init__(self, mc, dirloc, first_cluster, + length, mode, name) + + def close(self): + pass + + def real_close(self): + ps2mc_directory.close(self) + +class ps2mc(object): + """A PlayStation 2 memory card filesystem implementation. + + The close() method must be called when the object is no longer needed, + otherwise cycles that can't be collected by the garbage collector + will remain.""" + + open_files = None + fat_cache = None + + def _calculate_derived(self): + self.spare_size = div_round_up(self.page_size, 128) * 4 + self.raw_page_size = self.page_size + self.spare_size + self.cluster_size = self.page_size * self.pages_per_cluster + self.entries_per_cluster = (self.page_size + * self.pages_per_cluster / 4) + + limit = (min(self.good_block2, self.good_block1) + * self.pages_per_erase_block + / self.pages_per_cluster + - self.allocatable_cluster_offset) + self.allocatable_cluster_limit = limit + + def __init__(self, f, ignore_ecc = False, params = None): + self.open_files = {} + self.fat_cache = lru_cache(12) + self.alloc_cluster_cache = lru_cache(64) + self.modified = False + self.f = None + self.rootdir = None + + f.seek(0) + s = f.read(0x154) + if len(s) != 0x154 or not s.startswith(PS2MC_MAGIC): + if (params == None): + raise corrupt, ("Not a PS2 memory card image", + f) + self.f = f + self.format(params) + else: + sb = unpack_superblock(s) + self.version = sb[1] + self.page_size = sb[2] + self.pages_per_cluster = sb[3] + self.pages_per_erase_block = sb[4] + self.clusters_per_card = sb[6] + self.allocatable_cluster_offset = sb[7] + self.allocatable_cluster_end = sb[8] + self.rootdir_fat_cluster = sb[9] + self.good_block1 = sb[10] + self.good_block2 = sb[11] + self.indirect_fat_cluster_list = sb[12] + self.bad_erase_block_list = sb[13] + + self._calculate_derived() + + self.f = f + self.ignore_ecc = False + + try: + self.read_page(0) + self.ignore_ecc = ignore_ecc + except ecc_error: + # the error might be due the fact the file + # image doesn't contain ECC data + self.spare_size = 0 + self.raw_page_size = self.page_size + ignore_ecc = True + + # sanity check + root = self._directory(None, 0, 1) + dot = root[0] + dotdot = root[1] + root.close() + if (dot[8] != "." or dotdot[8] != ".." + or not mode_is_dir(dot[0]) or not mode_is_dir(dotdot[0])): + raise corrupt, "Root directory damaged." + + self.fat_cursor = 0 + self.curdir = (0, 0) + + def write_superblock(self): + s = pack_superblock((PS2MC_MAGIC, + self.version, + self.page_size, + self.pages_per_cluster, + self.pages_per_erase_block, + 0xFF00, + self.clusters_per_card, + self.allocatable_cluster_offset, + self.allocatable_cluster_end, + self.rootdir_fat_cluster, + self.good_block1, + self.good_block2, + self.indirect_fat_cluster_list, + self.bad_erase_block_list, + 2, + 0x2B)) + s += "\x00" * (self.page_size - len(s)) + self.write_page(0, s) + + page = "\xFF" * self.raw_page_size + self.f.seek(self.good_block2 * self.pages_per_erase_block + * self.raw_page_size) + for i in range(self.pages_per_erase_block): + self.f.write(page) + + self.modified = False + return + + def format(self, params): + """Create (format) a new memory card image.""" + + (with_ecc, page_size, + pages_per_erase_block, param_pages_per_card) = params + + if pages_per_erase_block < 1: + raise error, ("invalid pages per erase block (%d)" + % page_size) + + pages_per_card = round_down(param_pages_per_card, + pages_per_erase_block) + cluster_size = PS2MC_CLUSTER_SIZE + pages_per_cluster = cluster_size / page_size + clusters_per_erase_block = (pages_per_erase_block + / pages_per_cluster) + erase_blocks_per_card = pages_per_card / pages_per_erase_block + clusters_per_card = pages_per_card / pages_per_cluster + epc = cluster_size / 4 + + if (page_size < PS2MC_DIRENT_LENGTH + or pages_per_cluster < 1 + or pages_per_cluster * page_size != cluster_size): + raise error, "invalid page size (%d)" % page_size + + good_block1 = erase_blocks_per_card - 1 + good_block2 = erase_blocks_per_card - 2 + first_ifc = div_round_up(PS2MC_INDIRECT_FAT_OFFSET, + cluster_size) + + allocatable_clusters = clusters_per_card - (first_ifc + 2) + fat_clusters = div_round_up(allocatable_clusters, epc) + indirect_fat_clusters = div_round_up(fat_clusters, epc) + if indirect_fat_clusters > PS2MC_MAX_INDIRECT_FAT_CLUSTERS: + indirect_fat_clusters = PS2MC_MAX_INDIRECT_FAT_CLUSTERS + fat_clusters = indirect_fat_clusters * epc + allocatable_clusters = fat_clusters * epc + + allocatable_cluster_offset = (first_ifc + + indirect_fat_clusters + + fat_clusters) + allocatable_cluster_end = (good_block2 + * clusters_per_erase_block + - allocatable_cluster_offset) + if allocatable_cluster_end < 1: + raise error, ("memory card image too small" + " to be formatted") + + ifc_list = unpack_fat("\0\0\0\0" + * PS2MC_MAX_INDIRECT_FAT_CLUSTERS) + for i in range(indirect_fat_clusters): + ifc_list[i] = first_ifc + i + + self.version = "1.2.0.0" + self.page_size = page_size + self.pages_per_cluster = pages_per_cluster + self.pages_per_erase_block = pages_per_erase_block + self.clusters_per_card = clusters_per_card + self.allocatable_cluster_offset = allocatable_cluster_offset + self.allocatable_cluster_end = allocatable_clusters + self.rootdir_fat_cluster = 0 + self.good_block1 = good_block1 + self.good_block2 = good_block2 + self.indirect_fat_cluster_list = ifc_list + bebl = "\xFF\xFF\xFF\xFF" * 32 + self.bad_erase_block_list = unpack_32bit_array(bebl) + + self._calculate_derived() + + self.ignore_ecc = not with_ecc + erased = "\0" * page_size + if not with_ecc: + self.spare_size = 0 + else: + ecc = "".join(["".join(map(chr, s)) + for s in ecc_calculate_page(erased)]) + erased += ecc + "\0" * (self.spare_size - len(ecc)) + + self.f.seek(0) + for page in range(pages_per_card): + self.f.write(erased) + + self.modified = True + + first_fat_cluster = first_ifc + indirect_fat_clusters + remainder = fat_clusters % epc + for i in range(indirect_fat_clusters): + base = first_fat_cluster + i * epc + buf = unpack_fat(range(base, base + epc)) + if (i == indirect_fat_clusters - 1 + and remainder != 0): + del buf[remainder:] + buf.fromlist([0xFFFFFFFF] * (epc - remainder)) + self._write_fat_cluster(ifc_list[i], buf) + + + # go through the fat backwards for better cache usage + for i in range(allocatable_clusters - 1, + allocatable_cluster_end - 1, -1): + self.set_fat(i, PS2MC_FAT_CHAIN_END) + for i in range(allocatable_cluster_end - 1, 0, -1): + self.set_fat(i, PS2MC_FAT_CLUSTER_MASK) + self.set_fat(0, PS2MC_FAT_CHAIN_END) + + self.allocatable_cluster_end = allocatable_cluster_end + + now = tod_now() + s = pack_dirent((DF_RWX | DF_DIR | DF_0400 | DF_EXISTS, + 0, 2, now, + 0, 0, now, 0, ".")) + s += "\0" * (cluster_size - len(s)) + self.write_allocatable_cluster(0, s) + dir = self._directory((0, 0), 0, 2, "wb", "/") + dir.write_raw_ent(1, (DF_WRITE | DF_EXECUTE | DF_DIR | DF_0400 + | DF_HIDDEN | DF_EXISTS, + 0, 0, now, + 0, 0, now, 0, ".."), False) + dir.close() + + self.flush() + + def read_page(self, n): + # print "@@@ page", n + f = self.f + f.seek(self.raw_page_size * n) + page = f.read(self.page_size) + if len(page) != self.page_size: + raise corrupt, ("attempted to read past EOF" + " (page %05X)" % n, f) + if self.ignore_ecc: + return page + spare = f.read(self.spare_size) + if len(spare) != self.spare_size: + raise corrupt, ("attempted to read past EOF" + " (page %05X)" % n, f) + (status, page, spare) = ecc_check_page(page, spare) + if status == ECC_CHECK_FAILED: + raise ecc_error, ("Unrecoverable ECC error (page %d)" + % n) + return page + + def write_page(self, n, buf): + f = self.f + f.seek(self.raw_page_size * n) + self.modified = True + if len(buf) != self.page_size: + raise error, ("internal error: write_page:" + " %d != %d" % (len(buf), self.page_size)) + f.write(buf) + if self.spare_size != 0: + a = array.array('B') + for s in ecc_calculate_page(buf): + a.fromlist(s) + a.tofile(f) + f.write("\0" * (self.spare_size - len(a))) + + def read_cluster(self, n): + pages_per_cluster = self.pages_per_cluster + cluster_size = self.cluster_size + if self.spare_size == 0: + self.f.seek(cluster_size * n) + return self.f.read(cluster_size) + n *= pages_per_cluster + if pages_per_cluster == 2: + return self.read_page(n) + self.read_page(n + 1) + return "".join(map(self.read_page, + range(n, n + pages_per_cluster))) + + def write_cluster(self, n, buf): + pages_per_cluster = self.pages_per_cluster + cluster_size = self.cluster_size + if self.spare_size == 0: + self.f.seek(cluster_size * n) + if len(buf) != cluster_size: + raise error, ("internal error: write_cluster:" + " %d != %d" % (len(buf), + cluster_size)) + return self.f.write(buf) + n *= pages_per_cluster + pgsize = self.page_size + for i in range(pages_per_cluster): + self.write_page(n + i, buf[i * pgsize + : i * pgsize + pgsize]) + + + def _add_fat_cluster_to_cache(self, n, fat, dirty): + old = self.fat_cache.add(n, [fat, dirty]) + if old != None: + (n, [fat, dirty]) = old + if dirty: + self.write_cluster(n, pack_fat(fat)) + + def _read_fat_cluster(self, n): + v = self.fat_cache.get(n) + if v != None: + # print "@@@ fat hit", n + return v[0] + # print "@@@ fat miss", n + fat = unpack_fat(self.read_cluster(n)) + self._add_fat_cluster_to_cache(n, fat, False) + return fat + + def _write_fat_cluster(self, n, fat): + self._add_fat_cluster_to_cache(n, fat, True) + + def flush_fat_cache(self): + if self.fat_cache == None: + return + for (n, v) in self.fat_cache.items(): + [fat, dirty] = v + if dirty: + self.write_cluster(n, pack_fat(fat)) + v[1] = False + + def _add_alloc_cluster_to_cache(self, n, buf, dirty): + old = self.alloc_cluster_cache.add(n, [buf, dirty]) + if old != None: + (n, [buf, dirty]) = old + if dirty: + n += self.allocatable_cluster_offset + self.write_cluster(n, buf) + + def read_allocatable_cluster(self, n): + a = self.alloc_cluster_cache.get(n) + if a != None: + # print "@@@ cache hit", n + return a[0] + # print "@@@ cache miss", n + buf = self.read_cluster(n + self.allocatable_cluster_offset) + self._add_alloc_cluster_to_cache(n, buf, False) + return buf + + def write_allocatable_cluster(self, n, buf): + self._add_alloc_cluster_to_cache(n, buf, True) + + def flush_alloc_cluster_cache(self): + if self.alloc_cluster_cache == None: + return + for (n, a) in self.alloc_cluster_cache.items(): + [buf, dirty] = a + if dirty: + n += self.allocatable_cluster_offset + self.write_cluster(n, buf) + a[1] = False + + def read_fat_cluster(self, n): + indirect_offset = n % self.entries_per_cluster + dbl_offset = n / self.entries_per_cluster + indirect_cluster = self.indirect_fat_cluster_list[dbl_offset] + indirect_fat = self._read_fat_cluster(indirect_cluster) + cluster = indirect_fat[indirect_offset] + return (self._read_fat_cluster(cluster), cluster) + + def read_fat(self, n): + if n < 0 or n >= self.allocatable_cluster_end: + raise io_error, (EIO, + "FAT cluster index out of range" + " (%d)" % n) + offset = n % self.entries_per_cluster + fat_cluster = n / self.entries_per_cluster + (fat, cluster) = self.read_fat_cluster(fat_cluster) + return (fat, offset, cluster) + + def lookup_fat(self, n): + (fat, offset, cluster) = self.read_fat(n) + return fat[offset] + + def set_fat(self, n, value): + (fat, offset, cluster) = self.read_fat(n) + fat[offset] = value + self._write_fat_cluster(cluster, fat) + + def allocate_cluster(self): + epc = self.entries_per_cluster + allocatable_cluster_limit = self.allocatable_cluster_limit + + end = div_round_up(allocatable_cluster_limit, epc) + remainder = allocatable_cluster_limit % epc + + while self.fat_cursor < end: + (fat, cluster) = self.read_fat_cluster(self.fat_cursor) + if (self.fat_cursor == end - 1 + and remainder != 0): + n = min(fat[:remainder]) + else: + n = min(fat) + if (n & PS2MC_FAT_ALLOCATED_BIT) == 0: + offset = fat.index(n) + fat[offset] = PS2MC_FAT_CHAIN_END + self._write_fat_cluster(cluster, fat) + ret = self.fat_cursor * epc + offset + # print "@@@ allocated", ret + return ret + self.fat_cursor += 1 + return None + + def fat_chain(self, first_cluster): + return fat_chain(self.lookup_fat, first_cluster) + + def file(self, dirloc, first_cluster, length, mode, name = None): + """Create a new file-like object for a file.""" + + f = ps2mc_file(self, dirloc, first_cluster, length, mode, name) + if dirloc == None: + return + open_files = self.open_files + if dirloc not in open_files: + open_files[dirloc] = [None, set([f])] + else: + open_files[dirloc][1].add(f) + return f + + def directory(self, dirloc, first_cluster, length, + mode = None, name = None): + return ps2mc_directory(self, dirloc, first_cluster, length, + mode, name) + + def _directory(self, dirloc, first_cluster, length, + mode = None, name = None): + # print "@@@ _directory", dirloc, first_cluster, length + if first_cluster != 0: + return self.directory(dirloc, first_cluster, length, + mode, name) + if dirloc == None: + dirloc = (0, 0) + assert dirloc == (0, 0) + if self.rootdir != None: + return self.rootdir + dir = _root_directory(self, dirloc, 0, length, "r+b", "/") + l = dir[0][2] + if l != length: + dir.real_close() + dir = _root_directory(self, dirloc, 0, l, "r+b", "/") + self.rootdir = dir + return dir + + def _get_parent_dirloc(self, dirloc): + """Get the dirloc of the parent directory of the + file or directory refered to by dirloc""" + + cluster = self.read_allocatable_cluster(dirloc[0]) + ent = unpack_dirent(cluster[:PS2MC_DIRENT_LENGTH]) + return (ent[4], ent[5]) + + def _dirloc_to_ent(self, dirloc): + """Get the directory entry of the file or directory + refered to by dirloc""" + + dir = self._directory(None, dirloc[0], dirloc[1] + 1, + name = "_dirloc_to_ent temp") + ent = dir[dirloc[1]] + dir.close() + return ent + + def _opendir_dirloc(self, dirloc, mode = "rb"): + """Open the directory that is refered to by dirloc""" + + ent = self._dirloc_to_ent(dirloc) + return self._directory(dirloc, ent[4], ent[2], + name = "_opendir_dirloc temp") + + def _opendir_parent_dirloc(self, dirloc, mode = "rb"): + """Open the directory that contains the file or directory + refered to by dirloc""" + + return self._opendir_dirloc(self._get_parent_dirloc(dirloc), + mode) + + def update_dirent_all(self, dirloc, thisf, new_ent): + # print "@@@ update_dirent", dirloc + # print "@@@ new_ent", new_ent + opened = self.open_files.get(dirloc, None) + if opened == None: + files = [] + dir = None + else: + dir, files = opened + if dir == None: + dir = self._opendir_parent_dirloc(dirloc, "r+b") + if opened != None: + opened[0] = dir + + ent = dir[dirloc[1]] + # print "@@@ old_ent", ent + + is_dir = ent[0] & DF_DIR + + if is_dir and thisf != None and new_ent[2] != None: + new_ent = list(new_ent) + new_ent[2] /= PS2MC_DIRENT_LENGTH + + # print "len: ", ent[2], new_ent[2] + + modified = changed = notify = False + for i in range(len(ent)): + new = new_ent[i] + if new != None: + if new != ent[i]: + ent[i] = new + changed = True + if i == 6: + modified = True + if i in [2, 4]: + notify = True + + # Modifying a file causes the modification time of + # both the file and the file's directory to updated, + # however modifying a directory never updates the + # modification time of the directory's parent. + if changed: + dir.write_raw_ent(dirloc[1], ent, + (modified and not is_dir)) + + + if notify: + for f in files: + if f != thisf: + f.update_notfiy(ent[4], ent[2]) + if opened == None: + dir.close() + + def update_dirent(self, dirloc, thisf, first_cluster, length, + modified): + if modified: + modified = tod_now() + else: + if first_cluster == None and length == None: + return + modified = None + self.update_dirent_all(dirloc, thisf, + (None, None, length, None, + first_cluster, None, modified, None, + None)) + + def notify_closed(self, dirloc, thisf): + if self.open_files == None or dirloc == None: + return + a = self.open_files.get(dirloc, None) + if a == None: + return + self.flush() + dir, files = a + files.discard(thisf) + if len(files) == 0: + # print "@@@ notify_closed", dir + if dir != None: + dir.close() + del self.open_files[dirloc] + + def search_directory(self, dir, name): + """Search dir for name.""" + + # start the search where the last search ended. + start = dir.tell() - 1 + if start == -1: + start = 0 + for i in range(start, len(dir)) + range(0, start): + try: + ent = dir[i] + except IndexError: + raise corrupt("Corrupt directory", dir.f) + + if ent[8] == name and (ent[0] & DF_EXISTS): + return (i, ent) + return (None, None) + + def create_dir_entry(self, parent_dirloc, name, mode): + """Create a new directory entry in a directory.""" + + if name == "": + raise file_not_found, name + + # print "@@@ create_dir_ent", parent_dirloc, name + dir_ent = self._dirloc_to_ent(parent_dirloc) + dir = self._directory(parent_dirloc, dir_ent[4], dir_ent[2], + "r+b") + l = len(dir) + # print "@@@ len", l + assert l >= 2 + for i in range(l): + ent = dir[i] + if (ent[0] & DF_EXISTS) == 0: + break + else: + i = l + + dirloc = (dir_ent[4], i) + # print "@@@ dirloc", dirloc + now = tod_now() + if mode & DF_DIR: + mode &= ~DF_FILE + cluster = self.allocate_cluster() + length = 1 + else: + mode |= DF_FILE + mode &= ~DF_DIR + cluster = PS2MC_FAT_CHAIN_END + length = 0 + ent[0] = mode | DF_EXISTS + ent[1] = 0 + ent[2] = length + ent[3] = now + ent[4] = cluster + ent[5] = 0 + ent[6] = now + ent[7] = 0 + ent[8] = name[:32] + dir.write_raw_ent(i, ent, True) + dir.close() + + if mode & DF_FILE: + # print "@@@ ret", dirloc, ent + return (dirloc, ent) + + dirent = pack_dirent((DF_RWX | DF_0400 | DF_DIR | DF_EXISTS, + 0, 0, now, dirloc[0], dirloc[1], + now, 0, ".")) + dirent += "\0" * (self.cluster_size - PS2MC_DIRENT_LENGTH) + self.write_allocatable_cluster(cluster, dirent) + dir = self._directory(dirloc, cluster, 1, "wb", + name = "") + dir.write_raw_ent(1, (DF_RWX | DF_0400 | DF_DIR | DF_EXISTS, + 0, 0, now, + 0, 0, + now, 0, ".."), False) + dir.close() + ent[2] = 2 + # print "@@@ ret", dirloc, ent + return (dirloc, ent) + + def delete_dirloc(self, dirloc, truncate, name): + """Delete or truncate the file or directory given by dirloc.""" + + if dirloc == (0, 0): + raise io_error, (EACCES, + "cannot remove root directory", + name) + if dirloc[1] in [0, 1]: + raise io_error, (EACCES, + 'cannot remove "." or ".." entries', + name) + + if dirloc in self.open_files: + raise io_error, (EBUSY, + "cannot remove open file", filename) + + epc = self.entries_per_cluster + + ent = self._dirloc_to_ent(dirloc) + cluster = ent[4] + if truncate: + ent[2] = 0 + ent[4] = PS2MC_FAT_CHAIN_END + ent[6] = tod_now() + else: + ent[0] &= ~DF_EXISTS + self.update_dirent_all(dirloc, None, ent) + + while cluster != PS2MC_FAT_CHAIN_END: + if cluster / epc < self.fat_cursor: + self.fat_cursor = cluster / epc + next_cluster = self.lookup_fat(cluster) + if next_cluster & PS2MC_FAT_ALLOCATED_BIT == 0: + # corrupted + break + next_cluster &= ~PS2MC_FAT_ALLOCATED_BIT + self.set_fat(cluster, next_cluster) + if next_cluster == PS2MC_FAT_CHAIN_END_UNALLOC: + break + cluster = next_cluster + + def path_search(self, pathname): + """Parse and resolve a pathname. + + Return a tuple containing a tuple containing three + values. The first is either the dirloc of the file or + directory, if it exists, otherwise it's the dirloc the + pathname's parent directory, if that exists otherwise + it's None. The second component is directory entry + for pathname if it exists, otherwise its dummy entry + with the first element set to 0, and the last element + set to the final component of the pathname. The third + is a boolean value that's true if the pathname refers + a directory.""" + + # print "@@@ path_search", repr(pathname) + if pathname == "": + return (None, None, False) + + (components, relative, is_dir) = pathname_split(pathname) + + dirloc = (0, 0) + if relative: + dirloc = self.curdir + + tmpname = "" + _directory = self._directory + + if dirloc == (0, 0): + rootent = self.read_allocatable_cluster(0) + ent = unpack_dirent(rootent[:PS2MC_DIRENT_LENGTH]) + dir_cluster = 0 + dir = _directory(dirloc, dir_cluster, ent[2], + name = tmpname) + else: + ent = self._dirloc_to_ent(dirloc) + dir = _directory(dirloc, ent[4], ent[2], + name = tmpname) + + for s in components: + # print "@@@", dirloc, repr(s), dir == None, ent + + if dir == None: + # tried to traverse a file or a + # non-existent directory + return (None, (0, 0, 0, 0, 0, 0, 0, 0, None), + False) + + if s == ".": + continue + if s == "..": + dotent = dir[0] + dir.close() + dirloc = (dotent[4], dotent[5]) + ent = self._dirloc_to_ent(dirloc) + dir = _directory(dirloc, ent[4], ent[2], + name = tmpname) + continue + + dir_cluster = ent[4] + (i, ent) = self.search_directory(dir, s) + dir.close() + dir = None + + if ent == None: + continue + + dirloc = (dir_cluster, i) + if ent[0] & DF_DIR: + dir = _directory(dirloc, ent[4], ent[2], + name = tmpname) + + if dir != None: + dir.close() + is_dir = True + elif ent != None: + is_dir = False + + if ent == None: + ent = (0, 0, 0, 0, 0, 0, 0, 0, components[-1]) + + return (dirloc, ent, is_dir) + + def open(self, filename, mode = "r"): + """Open a file, returning a new file-like object for it.""" + + (dirloc, ent, is_dir) = self.path_search(filename) + # print "@@@ open", (dirloc, ent) + if dirloc == None: + raise path_not_found, filename + if is_dir: + raise io_error, (EISDIR, "not a regular file", + filename) + if ent[0] == 0: + if mode[0] not in "wa": + raise file_not_found, filename + name = ent[8] + (dirloc, ent) = self.create_dir_entry(dirloc, name, + DF_FILE | DF_RWX + | DF_0400); + self.flush() + elif mode[0] == "w": + self.delete_dirloc(dirloc, True, filename) + ent[4] = PS2MC_FAT_CHAIN_END + ent[2] = 0 + return self.file(dirloc, ent[4], ent[2], mode, filename) + + def dir_open(self, filename, mode = "rb"): + (dirloc, ent, is_dir) = self.path_search(filename) + if dirloc == None: + raise path_not_found, filename + if ent[0] == 0: + raise dir_not_found, filename + if not is_dir: + raise io_error, (ENOTDIR, "not a directory", filename) + return self.directory(dirloc, ent[4], ent[2], mode, filename) + + def mkdir(self, filename): + (dirloc, ent, is_dir) = self.path_search(filename) + if dirloc == None: + raise path_not_found, filename + if ent[0] != 0: + raise io_error, (EEXIST, "directory exists", filename) + name = ent[8] + self.create_dir_entry(dirloc, name, DF_DIR | DF_RWX | DF_0400) + self.flush() + + def _is_empty(self, dirloc, ent, filename): + """Check if a directory is empty.""" + + dir = self._directory(dirloc, ent[4], ent[2], "rb", + filename) + try: + for i in range(2, len(dir)): + if dir[i][0] & DF_EXISTS: + return False + finally: + dir.close() + return True + + def remove(self, filename): + """Remove a file or empty directory.""" + + (dirloc, ent, is_dir) = self.path_search(filename) + if dirloc == None: + raise path_not_found, filename + if ent[0] == 0: + raise file_not_found, filename + if is_dir: + if ent[4] == 0: + raise io_error, (EACCES, + "cannot remove" + " root directory") + if not self._is_empty(dirloc, ent, filename): + raise io_error, (ENOTEMPTY, + "directory not empty", + filename) + self.delete_dirloc(dirloc, False, filename) + self.flush() + + def chdir(self, filename): + (dirloc, ent, is_dir) = self.path_search(filename) + if dirloc == None: + raise path_not_found, filename + if ent[0] == 0: + raise dir_not_found, filename + if not is_dir: + raise io_error, (ENOTDIR, "not a directory", filename) + self.curdir = dirloc + + def get_mode(self, filename): + """Get mode bits of a file. + + Returns None if the filename doesn't exist, rather than + throwing a error.""" + + (dirloc, ent, is_dir) = self.path_search(filename) + if ent[0] == 0: + return None + return ent[0] + + def get_dirent(self, filename): + """Get the raw directory entry tuple for a file.""" + + (dirloc, ent, is_dir) = self.path_search(filename) + if dirloc == None: + raise path_not_found, filename + if ent[0] == 0: + raise file_not_found, filename + return ent + + def set_dirent(self, filename, new_ent): + """Set various directory entry fields of a file. + + Not all fields can be changed. If a field in new_ent + is set to None then is not changed.""" + + (dirloc, ent, is_dir) = self.path_search(filename) + if dirloc == None: + raise path_not_found, filename + if ent[0] == 0: + raise file_not_found, filename + dir = self._opendir_parent_dirloc(dirloc, "r+b") + try: + new_ent = list(new_ent) + new_ent[8] = None + dir[dirloc[1]] = new_ent + finally: + dir.close() + self.flush() + return ent + + def is_ancestor(self, dirloc, olddirloc): + while True: + if dirloc == olddirloc: + return True + if dirloc == (0, 0): + return False + dirloc = self._get_parent_dirloc(dirloc) + + def rename(self, oldpathname, newpathname): + (olddirloc, oldent, is_dir) = self.path_search(oldpathname) + if olddirloc == None: + raise path_not_found, oldpathname + if oldent[0] == 0: + raise file_not_found, oldpathname + + if olddirloc == (0, 0): + raise io_error, (EINVAL, + "cannot rename root directory", + oldpathname) + if olddirloc in self.open_files: + raise io_error, (EBUSY, "cannot rename open file", + newname) + + (newparentdirloc, newent, x) = self.path_search(newpathname) + if newparentdirloc == None: + raise path_not_found, newpathname + if newent[0] != 0: + raise io_error, (EEXIST, "file exists", newpathname) + newname = newent[8] + + oldparentdirloc = self._get_parent_dirloc(olddirloc) + if oldparentdirloc == newparentdirloc: + dir = self._opendir_dirloc(oldparentdirloc, "r+b") + try: + dir[olddirloc[1]] = (None, None, None, None, + None, None, None, None, + newname) + finally: + dir.close() + return + + if is_dir and self.is_ancestor(newparentdirloc, olddirloc): + raise io_error, (EINVAL, "cannot move directory" + " beneath itself", oldpathname) + + + newparentdir = None + newent = None + try: + tmpmode = (oldent[0] & ~DF_DIR) | DF_FILE + + (newdirloc, newent) \ + = self.create_dir_entry(newparentdirloc, + newname, tmpmode) + + newent[:8] = oldent[:8] + newparentdir = self._opendir_dirloc(newparentdirloc) + newparentdir.write_raw_ent(newdirloc[1], newent, True) + newent = None + + oldent[0] &= ~DF_EXISTS + self.update_dirent_all(olddirloc, None, oldent) + + except: + if newent != None: + self.delete_dirloc(newdirloc, False, + newpathname) + finally: + if newparentdir != None: + newparentdir.close() + + if not is_dir: + return + + newdir = self._opendir_dirloc(newdirloc) + try: + dotent = list(newdir[0]) + dotent[4:6] = newdirloc + newdir.write_raw_ent(0, dotent, False) + finally: + newdir.close() + + + def import_save_file(self, sf, ignore_existing, dirname = None): + """Copy the contents a ps2_save_file object to a directory. + + If ingore_existing is true and the directory being imported + to already exists then False is returned instead of raising + an error. If dirname is given then the save file is copied + to that directory instead of the directory specified by + the save file. + """ + + dir_ent = sf.get_directory() + if dirname == None: + dirname = "/" + dir_ent[8] + + (root_dirloc, ent, is_dir) = self.path_search(dirname) + if root_dirloc == None: + raise path_not_found, dirname + if ent[0] != 0: + if ignore_existing: + return False + raise io_error, (EEXIST, "directory exists", dirname) + name = ent[8] + mode = DF_DIR | (dir_ent[0] & ~DF_FILE) + + (dir_dirloc, ent) = self.create_dir_entry(root_dirloc, + name, mode) + try: + assert dirname != "/" + dirname = dirname + "/" + for i in range(dir_ent[2]): + (ent, data) = sf.get_file(i) + mode = DF_FILE | (ent[0] & ~DF_DIR) + (dirloc, ent) \ + = self.create_dir_entry(dir_dirloc, + ent[8], mode) + # print "@@@ file", dirloc, ent[4], ent[2] + f = self.file(dirloc, ent[4], ent[2], "wb", + dirname + ent[8]) + try: + f.write(data) + finally: + f.close() + except EnvironmentError: + type, what, where = sys.exc_info() + try: + try: + for i in range(dir_ent[2]): + (ent, data) = sf.get_file(i) + # print "@@@ remove", ent[8] + self.remove(dirname + ent[8]) + except EnvironmentError, why: + # print "@@@ failed", why + pass + + try: + # print "@@@ remove dir", dirname + self.remove(dirname) + except EnvironmentError, why: + # print "@@@ failed", why + pass + raise type, what, where + finally: + del where + + # set modes and timestamps to those of the save file + + dir = self._opendir_dirloc(dir_dirloc, "r+b") + try: + for i in range(dir_ent[2]): + dir[i + 2] = sf.get_file(i)[0] + finally: + dir.close() + + dir = self._opendir_dirloc(root_dirloc, "r+b") + try: + a = dir_ent[:] + a[8] = None # don't change the name + dir[dir_dirloc[1]] = a + finally: + dir.close() + + self.flush() + return True + + def export_save_file(self, filename): + (dir_dirloc, dirent, is_dir) = self.path_search(filename) + if dir_dirloc == None: + raise path_not_found, filename + if dirent[0] == 0: + raise dir_not_found, filename + if not is_dir: + raise io_error, (ENOTDIR, "not a directory", filename) + if dir_dirloc == (0, 0): + raise io_error, (EACCES, "can't export root directory", + filename) + sf = ps2save.ps2_save_file() + files = [] + f = None + dir = self._directory(dir_dirloc, dirent[4], dirent[2], + "rb", filename) + try: + for i in range(2, dirent[2]): + ent = dir[i] + if not mode_is_file(ent[0]): + print ("warning: %s/%s is not a file," + " ingored." + % (dirent[8], ent[8])) + continue + f = self.file((dirent[4], i), ent[4], ent[2], + "rb") + data = f.read(ent[2]) + f.close() + assert len(data) == ent[2] + files.append((ent, data)) + finally: + if f != None: + f.close() + dir.close() + dirent[2] = len(files) + sf.set_directory(dirent) + for (i, (ent, data)) in enumerate(files): + sf.set_file(i, ent, data) + return sf + + def _remove_dir(self, dirloc, ent, dirname): + """Recurse over a directory tree to remove it. + If not "", dirname must end with a slash (/).""" + + first_cluster = ent[4] + length = ent[2] + dir = self._directory(dirloc, first_cluster, length, + "rb", dirname) + try: + ents = list(enumerate(dir)) + finally: + dir.close() + for (i, ent) in ents[2:]: + mode = ent[0] + if not (mode & DF_EXISTS): + continue + if mode & DF_DIR: + self._remove_dir((first_cluster, i), ent, + dirname + ent[8] + "/") + else: + # print "deleting", dirname + ent[8] + self.delete_dirloc((first_cluster, i), False, + dirname + ent[8]) + self.delete_dirloc(dirloc, False, dirname) + + def rmdir(self, dirname): + """Recursively delete a directory.""" + + (dirloc, ent, is_dir) = self.path_search(dirname) + if dirloc == None: + raise path_not_found, dirname + if ent[0] == 0: + raise dir_not_found, dirname + if not is_dir: + raise io_error, (ENOTDIR, "not a directory", dirname) + if dirloc == (0, 0): + raise io_error, (EACCES, "can't delete root directory", + dirname) + + if dirname != "" and dirname[-1] != "/": + dirname += "/" + self._remove_dir(dirloc, ent, dirname) + + def get_free_space(self): + """Returns the amount of free space in bytes.""" + + free = 0 + for i in xrange(self.allocatable_cluster_end): + if (self.lookup_fat(i) & PS2MC_FAT_ALLOCATED_BIT) == 0: + free += 1 + return free * self.cluster_size + + def get_allocatable_space(self): + """Returns the total amount of allocatable space in bytes.""" + return self.allocatable_cluster_limit * self.cluster_size + + def _check_file(self, fat, first_cluster, length): + cluster = first_cluster + i = 0 + while cluster != PS2MC_FAT_CHAIN_END: + if cluster < 0 or cluster >= len(fat): + return "invalid cluster in chain" + if fat[cluster]: + return "cross linked chain" + i += 1 + # print cluster, + fat[cluster] = 1 + next = self.lookup_fat(cluster) + if next == PS2MC_FAT_CHAIN_END: + break + if (next & PS2MC_FAT_ALLOCATED_BIT) == 0: + return "unallocated cluster in chain" + cluster = next & ~PS2MC_FAT_ALLOCATED_BIT + file_cluster_end = div_round_up(length, self.cluster_size) + if i < file_cluster_end: + return "chain ends before end of file" + elif i > file_cluster_end: + return "chain continues after end of file" + return None + + def _check_dir(self, fat, dirloc, dirname, ent): + why = self._check_file(fat, ent[4], + ent[2] * PS2MC_DIRENT_LENGTH) + if why != None: + print "bad directory:", dirname + ":", why + return False + ret = True + first_cluster = ent[4] + length = ent[2] + dir = self._directory(dirloc, first_cluster, length, + "rb", dirname) + dot_ent = dir[0] + if dot_ent[8] != ".": + print "bad directory:", dirname + ': missing "." entry' + ret = False + if (dot_ent[4], dot_ent[5]) != dirloc: + print "bad directory:", dirname + ': bad "." entry' + ret = False + if dir[1][8] != "..": + print "bad directory:", (dirname + + ': missing ".." entry') + ret = False + for i in xrange(2, length): + ent = dir[i] + mode = ent[0] + if not (mode & DF_EXISTS): + continue + if mode & DF_DIR: + if not self._check_dir(fat, (first_cluster, i), + dirname + ent[8] + "/", + ent): + ret = False + else: + why = self._check_file(fat, ent[4], ent[2]) + if why != None: + print "bad file:", (dirname + ent[8] + + ":"), why + ret = False + + dir.close() + return ret + + def check(self): + """Run a simple file system check. + + Any problems found are reported to stdout.""" + + ret = True + + fat_len = int(str(self.allocatable_cluster_end)) + if not isinstance(fat_len, int): + raise error, "Memory card image too big to check." + + fat = array.array('B', [0]) * fat_len + + cluster = self.read_allocatable_cluster(0) + ent = unpack_dirent(cluster[:PS2MC_DIRENT_LENGTH]) + ret = self._check_dir(fat, (0, 0), "/", ent) + + lost_clusters = 0 + for i in xrange(self.allocatable_cluster_end): + a = self.lookup_fat(i) + if (a & PS2MC_FAT_ALLOCATED_BIT) and not fat[i]: + print i, + lost_clusters += 1 + if lost_clusters > 0: + print + print "found", lost_clusters, "lost clusters" + ret = False + + return ret + + def _globdir(self, dirname, components, is_dir): + pattern = components[0] + if dirname == "": + dir = self.dir_open(".") + else: + dir = self.dir_open(dirname) + try: + return [dirname + ent[8] + for ent in dir + if ((ent[0] & DF_EXISTS) + and (not is_dir or (ent[8] & DF_DIR)) + and (ent[8] not in [".", ".."] + or ent[8] == pattern) + and fnmatch.fnmatchcase(ent[8], + pattern))] + finally: + dir.close() + + def _glob(self, dirname, components, is_dir): + pattern = components[0] + components = components[1:] + + if len(components) == 1: + _glob = self._globdir + else: + _glob = self._glob + + if dirname == "": + dir = self.dir_open(".") + else: + dir = self.dir_open(dirname) + try: + ret = [] + for ent in dir: + name = ent[8] + if ((ent[0] & DF_EXISTS) == 0 + or (ent[0] & DF_DIR) == 0): + continue + if name == "." or name == "..": + if pattern != name: + continue + elif not fnmatch.fnmatchcase(name, pattern): + continue + ret += _glob(dirname + name + "/", + components, is_dir) + finally: + dir.close() + return ret + + def glob(self, pattern): + if pattern == "": + return [""] + (components, relative, isdir) = pathname_split(pattern) + if len(components) == 0: + return ["/"] + if relative: + dirname = "" + else: + dirname = "/" + if len(components) == 1: + ret = self._globdir(dirname, components, isdir) + else: + ret = self._glob(dirname, components, isdir) + # print pattern, "->", ret + return ret + + def get_icon_sys(self, dirname): + """Get contents of a directory's icon.sys file, if it exits.""" + + icon_sys = dirname + "/icon.sys" + mode = self.get_mode(icon_sys) + if mode == None or not mode_is_file(mode): + return None + f = self.open(icon_sys, "rb") + s = f.read(964) + f.close() + if len(s) == 964 and s[0:4] == "PS2D": + return s; + return None + + def dir_size(self, dirname): + """Calculate the total size of the contents of a directory.""" + + dir = self.dir_open(dirname) + try: + length = round_up(len(dir) * PS2MC_DIRENT_LENGTH, + self.cluster_size) + for ent in dir: + if mode_is_file(ent[0]): + length += round_up(ent[2], + self.cluster_size) + elif (mode_is_dir(ent[0]) + and ent[8] not in [".", ".."]): + length += self.dir_size(dirname + "/" + + ent[8]) + finally: + dir.close() + return length + + def flush(self): + self.flush_alloc_cluster_cache() + self.flush_fat_cache() + if self.modified: + self.write_superblock() + self.f.flush() + + def close(self): + """Close all open files. + + Disconnects, but doesn't close the file object used + access the raw image. After this method has been + called on a ps2mc object, it can no longer be used.""" + + # print "ps2mc.close" + try: + f = self.f + if f == None or getattr(f, "closed", False): + # print "closed" + return + open_files = self.open_files + # print "open_files", open_files + if open_files != None: + # this is complicated by the fact as + # files are closed they will remove + # themselves from the list of open files + for (dir, files) in open_files.values(): + for f in list(files): + f.close() + while len(open_files) > 0: + (k, v) = open_files.popitem() + (dir, files) = v + if dir != None: + dir.close() + if self.rootdir != None: + self.rootdir.close() + if self.fat_cache != None: + self.flush() + finally: + self.open_files = None + self.fat_cache = None + self.f = None + self.rootdir = None + + def __del__(self): + # print "ps2mc.__del__" + try: + self.close() + except: + sys.stderr.write("ps2mc.__del__: \n") + traceback.print_exc() diff --git a/ps2mc_dir.py b/ps2mc_dir.py index 9d21e07..2ab2c6e 100755 --- a/ps2mc_dir.py +++ b/ps2mc_dir.py @@ -1,134 +1,134 @@ -# -# ps2mc_dir.py -# -# By Ross Ridge -# Public Domain -# - -"""Functions for working with PS2 memory card directory entries.""" - -_SCCS_ID = "@(#) mysc ps2mc_dir.py 1.4 12/10/04 19:11:08\n" - -import struct -import time -import calendar - -PS2MC_DIRENT_LENGTH = 512 - -DF_READ = 0x0001 -DF_WRITE = 0x0002 -DF_EXECUTE = 0x0004 -DF_RWX = DF_READ | DF_WRITE | DF_EXECUTE -DF_PROTECTED = 0x0008 -DF_FILE = 0x0010 -DF_DIR = 0x0020 -DF_O_DCREAT = 0x0040 -DF_0080 = 0x0080 -DF_0100 = 0x0100 -DF_O_CREAT = 0x0200 -DF_0400 = 0x0400 -DF_POCKETSTN = 0x0800 -DF_PSX = 0x1000 -DF_HIDDEN = 0x2000 -DF_4000 = 0x4000 -DF_EXISTS = 0x8000 - -def zero_terminate(s): - """Truncate a string at the first NUL ('\0') character, if any.""" - - i = s.find('\0') - if i == -1: - return s - return s[:i] - -# mode, ???, length, created, -# fat_cluster, parent_entry, modified, attr, -# name -_dirent_fmt = "> 1)) - a = (a ^ (a >> 2)) - a = (a ^ (a >> 4)) - return a & 1 - -def _make_ecc_tables(): - parity_table = [_parityb(b) - for b in range(256)] - cpmasks = [0x55, 0x33, 0x0F, 0x00, 0xAA, 0xCC, 0xF0] - - column_parity_masks = [None] * 256 - for b in range(256): - mask = 0 - for i in range(len(cpmasks)): - mask |= parity_table[b & cpmasks[i]] << i - column_parity_masks[b] = mask - - return parity_table, column_parity_masks - -_parity_table, _column_parity_masks = _make_ecc_tables() - -def _ecc_calculate(s): - "Calculate the Hamming code for a 128 byte long string or byte array." - - if not isinstance(s, array.array): - a = array.array('B') - a.fromstring(s) - s = a - column_parity = 0x77 - line_parity_0 = 0x7F - line_parity_1 = 0x7F - for i in range(len(s)): - b = s[i] - column_parity ^= _column_parity_masks[b] - if _parity_table[b]: - line_parity_0 ^= ~i - line_parity_1 ^= i - return [column_parity, line_parity_0 & 0x7F, line_parity_1] - -def _ecc_check(s, ecc): - """Detect and correct any single bit errors. - - The parameters "s" and "ecc", the data and expected Hamming code - repectively, must be modifiable sequences of integers and are - updated with the corrected values if necessary.""" - - computed = ecc_calculate(s) - if computed == ecc: - return ECC_CHECK_OK - - #print - #_print_bin(0, s.tostring()) - #print "computed %02x %02x %02x" % tuple(computed) - #print "actual %02x %02x %02x" % tuple(ecc) - - # ECC mismatch - - cp_diff = (computed[0] ^ ecc[0]) & 0x77 - lp0_diff = (computed[1] ^ ecc[1]) & 0x7F - lp1_diff = (computed[2] ^ ecc[2]) & 0x7F - lp_comp = lp0_diff ^ lp1_diff - cp_comp = (cp_diff >> 4) ^ (cp_diff & 0x07) - - #print "%02x %02x %02x %02x %02x" % (cp_diff, lp0_diff, lp1_diff, - # lp_comp, cp_comp) - - if lp_comp == 0x7F and cp_comp == 0x07: - print "corrected 1" - # correctable 1 bit error in data - s[lp1_diff] ^= 1 << (cp_diff >> 4) - return ECC_CHECK_CORRECTED - if ((cp_diff == 0 and lp0_diff == 0 and lp1_diff == 0) - or _popcount(lp_comp) + _popcount(cp_comp) == 1): - print "corrected 2" - # correctable 1 bit error in ECC - # (and/or one of the unused bits was set) - ecc[0] = computed[0] - ecc[1] = computed[1] - ecc[2] = computed[2] - return ECC_CHECK_CORRECTED - - # uncorrectable error - return ECC_CHECK_FAILED - -def ecc_calculate_page(page): - """Return a list of the ECC codes for a PS2 memory card page.""" - return [ecc_calculate(page[i * 128 : i * 128 + 128]) - for i in range(div_round_up(len(page), 128))] - -def ecc_check_page(page, spare): - "Check and correct any single bit errors in a PS2 memory card page." - - failed = False - corrected = False - - #chunks = [(array.array('B', page[i * 128 : i * 128 + 128]), - # map(ord, spare[i * 3 : i * 3 + 3])) - # for i in range(div_round_up(len(page), 128))] - - chunks = [] - for i in range(div_round_up(len(page), 128)): - a = array.array('B') - a.fromstring(page[i * 128 : i * 128 + 128]) - chunks.append((a, map(ord, spare[i * 3 : i * 3 + 3]))) - - r = [ecc_check(s, ecc) - for (s, ecc) in chunks] - ret = ECC_CHECK_OK - if ECC_CHECK_CORRECTED in r: - # rebuild sector and spare from the corrected versions - page = "".join([a[0].tostring() - for a in chunks]) - spare = "".join([chr(a[1][i]) - for a in chunks - for i in range(3)]) - ret = ECC_CHECK_CORRECTED - if ECC_CHECK_FAILED in r: - ret = ECC_CHECK_FAILED - return (ret, page, spare) - -if mymcsup == None: - ecc_calculate = _ecc_calculate - ecc_check = _ecc_check -else: - # _c_ubyte_p = ctypes.POINTER(ctypes.c_ubyte) - def ecc_calculate(s): - aecc = array.array('B', "\0\0\0") - cecc = ctypes.c_ubyte.from_address(aecc.buffer_info()[0]) - mymcsup.ecc_calculate(s, len(s), cecc) - return list(aecc) - - def ecc_check(s, ecc): - cs = ctypes.c_ubyte.from_address(s.buffer_info()[0]) - # print "%08X" % s.buffer_info()[0] - aecc = array.array('B', ecc) - cecc = ctypes.c_ubyte.from_address(aecc.buffer_info()[0]) - ret = mymcsup.ecc_check(cs, len(s), cecc) - ecc[0] = aecc[0] - ecc[1] = aecc[1] - ecc[2] = aecc[2] - return ret - +# +# ps2mc_ecc.py +# +# By Ross Ridge +# Public Domain +# + +""" +Routines for calculating the Hamming codes, a simple form of error +correcting codes (ECC), as used on PS2 memory cards. +""" + +_SCCS_ID = "@(#) mymc ps2mc_ecc.py 1.4 07/12/17 02:34:04\n" + +import array + +from round import div_round_up + +try: + import ctypes + import mymcsup +except ImportError: + mymcsup = None + +__ALL__ = ["ECC_CHECK_OK", "ECC_CHECK_CORRECTED", "ECC_CHECK_FAILED", + "ecc_calculate", "ecc_check", + "ecc_calculate_page", "ecc_check_page"] + +ECC_CHECK_OK = 0 +ECC_CHECK_CORRECTED = 1 +ECC_CHECK_FAILED = 2 + +def _popcount(a): + count = 0 + while a != 0: + a &= a - 1 + count += 1 + return count + +def _parityb(a): + a = (a ^ (a >> 1)) + a = (a ^ (a >> 2)) + a = (a ^ (a >> 4)) + return a & 1 + +def _make_ecc_tables(): + parity_table = [_parityb(b) + for b in range(256)] + cpmasks = [0x55, 0x33, 0x0F, 0x00, 0xAA, 0xCC, 0xF0] + + column_parity_masks = [None] * 256 + for b in range(256): + mask = 0 + for i in range(len(cpmasks)): + mask |= parity_table[b & cpmasks[i]] << i + column_parity_masks[b] = mask + + return parity_table, column_parity_masks + +_parity_table, _column_parity_masks = _make_ecc_tables() + +def _ecc_calculate(s): + "Calculate the Hamming code for a 128 byte long string or byte array." + + if not isinstance(s, array.array): + a = array.array('B') + a.fromstring(s) + s = a + column_parity = 0x77 + line_parity_0 = 0x7F + line_parity_1 = 0x7F + for i in range(len(s)): + b = s[i] + column_parity ^= _column_parity_masks[b] + if _parity_table[b]: + line_parity_0 ^= ~i + line_parity_1 ^= i + return [column_parity, line_parity_0 & 0x7F, line_parity_1] + +def _ecc_check(s, ecc): + """Detect and correct any single bit errors. + + The parameters "s" and "ecc", the data and expected Hamming code + repectively, must be modifiable sequences of integers and are + updated with the corrected values if necessary.""" + + computed = ecc_calculate(s) + if computed == ecc: + return ECC_CHECK_OK + + #print + #_print_bin(0, s.tostring()) + #print "computed %02x %02x %02x" % tuple(computed) + #print "actual %02x %02x %02x" % tuple(ecc) + + # ECC mismatch + + cp_diff = (computed[0] ^ ecc[0]) & 0x77 + lp0_diff = (computed[1] ^ ecc[1]) & 0x7F + lp1_diff = (computed[2] ^ ecc[2]) & 0x7F + lp_comp = lp0_diff ^ lp1_diff + cp_comp = (cp_diff >> 4) ^ (cp_diff & 0x07) + + #print "%02x %02x %02x %02x %02x" % (cp_diff, lp0_diff, lp1_diff, + # lp_comp, cp_comp) + + if lp_comp == 0x7F and cp_comp == 0x07: + print "corrected 1" + # correctable 1 bit error in data + s[lp1_diff] ^= 1 << (cp_diff >> 4) + return ECC_CHECK_CORRECTED + if ((cp_diff == 0 and lp0_diff == 0 and lp1_diff == 0) + or _popcount(lp_comp) + _popcount(cp_comp) == 1): + print "corrected 2" + # correctable 1 bit error in ECC + # (and/or one of the unused bits was set) + ecc[0] = computed[0] + ecc[1] = computed[1] + ecc[2] = computed[2] + return ECC_CHECK_CORRECTED + + # uncorrectable error + return ECC_CHECK_FAILED + +def ecc_calculate_page(page): + """Return a list of the ECC codes for a PS2 memory card page.""" + return [ecc_calculate(page[i * 128 : i * 128 + 128]) + for i in range(div_round_up(len(page), 128))] + +def ecc_check_page(page, spare): + "Check and correct any single bit errors in a PS2 memory card page." + + failed = False + corrected = False + + #chunks = [(array.array('B', page[i * 128 : i * 128 + 128]), + # map(ord, spare[i * 3 : i * 3 + 3])) + # for i in range(div_round_up(len(page), 128))] + + chunks = [] + for i in range(div_round_up(len(page), 128)): + a = array.array('B') + a.fromstring(page[i * 128 : i * 128 + 128]) + chunks.append((a, map(ord, spare[i * 3 : i * 3 + 3]))) + + r = [ecc_check(s, ecc) + for (s, ecc) in chunks] + ret = ECC_CHECK_OK + if ECC_CHECK_CORRECTED in r: + # rebuild sector and spare from the corrected versions + page = "".join([a[0].tostring() + for a in chunks]) + spare = "".join([chr(a[1][i]) + for a in chunks + for i in range(3)]) + ret = ECC_CHECK_CORRECTED + if ECC_CHECK_FAILED in r: + ret = ECC_CHECK_FAILED + return (ret, page, spare) + +if mymcsup == None: + ecc_calculate = _ecc_calculate + ecc_check = _ecc_check +else: + # _c_ubyte_p = ctypes.POINTER(ctypes.c_ubyte) + def ecc_calculate(s): + aecc = array.array('B', "\0\0\0") + cecc = ctypes.c_ubyte.from_address(aecc.buffer_info()[0]) + mymcsup.ecc_calculate(s, len(s), cecc) + return list(aecc) + + def ecc_check(s, ecc): + cs = ctypes.c_ubyte.from_address(s.buffer_info()[0]) + # print "%08X" % s.buffer_info()[0] + aecc = array.array('B', ecc) + cecc = ctypes.c_ubyte.from_address(aecc.buffer_info()[0]) + ret = mymcsup.ecc_check(cs, len(s), cecc) + ecc[0] = aecc[0] + ecc[1] = aecc[1] + ecc[2] = aecc[2] + return ret + diff --git a/ps2save.py b/ps2save.py index 0c90a09..465de20 100755 --- a/ps2save.py +++ b/ps2save.py @@ -1,630 +1,632 @@ -# -# ps2save.py -# -# By Ross Ridge -# Public Domain -# -# A simple interface for working with various PS2 save file formats. -# - -_SCCS_ID = "@(#) mysc ps2save.py 1.7 12/10/04 19:17:16\n" - -import sys -import os -import string -import struct -import binascii -import array -import zlib - -from round import div_round_up, round_up -from ps2mc_dir import * -from sjistab import shift_jis_normalize_table - -try: - import lzari -except ImportError: - lzari = None - -PS2SAVE_MAX_MAGIC = "Ps2PowerSave" -PS2SAVE_SPS_MAGIC = "\x0d\0\0\0SharkPortSave" -PS2SAVE_CBS_MAGIC = "CFU\0" -PS2SAVE_NPO_MAGIC = "nPort" - -# This is the initial permutation state ("S") for the RC4 stream cipher -# algorithm used to encrpyt and decrypt Codebreaker saves. -PS2SAVE_CBS_RC4S = [0x5f, 0x1f, 0x85, 0x6f, 0x31, 0xaa, 0x3b, 0x18, - 0x21, 0xb9, 0xce, 0x1c, 0x07, 0x4c, 0x9c, 0xb4, - 0x81, 0xb8, 0xef, 0x98, 0x59, 0xae, 0xf9, 0x26, - 0xe3, 0x80, 0xa3, 0x29, 0x2d, 0x73, 0x51, 0x62, - 0x7c, 0x64, 0x46, 0xf4, 0x34, 0x1a, 0xf6, 0xe1, - 0xba, 0x3a, 0x0d, 0x82, 0x79, 0x0a, 0x5c, 0x16, - 0x71, 0x49, 0x8e, 0xac, 0x8c, 0x9f, 0x35, 0x19, - 0x45, 0x94, 0x3f, 0x56, 0x0c, 0x91, 0x00, 0x0b, - 0xd7, 0xb0, 0xdd, 0x39, 0x66, 0xa1, 0x76, 0x52, - 0x13, 0x57, 0xf3, 0xbb, 0x4e, 0xe5, 0xdc, 0xf0, - 0x65, 0x84, 0xb2, 0xd6, 0xdf, 0x15, 0x3c, 0x63, - 0x1d, 0x89, 0x14, 0xbd, 0xd2, 0x36, 0xfe, 0xb1, - 0xca, 0x8b, 0xa4, 0xc6, 0x9e, 0x67, 0x47, 0x37, - 0x42, 0x6d, 0x6a, 0x03, 0x92, 0x70, 0x05, 0x7d, - 0x96, 0x2f, 0x40, 0x90, 0xc4, 0xf1, 0x3e, 0x3d, - 0x01, 0xf7, 0x68, 0x1e, 0xc3, 0xfc, 0x72, 0xb5, - 0x54, 0xcf, 0xe7, 0x41, 0xe4, 0x4d, 0x83, 0x55, - 0x12, 0x22, 0x09, 0x78, 0xfa, 0xde, 0xa7, 0x06, - 0x08, 0x23, 0xbf, 0x0f, 0xcc, 0xc1, 0x97, 0x61, - 0xc5, 0x4a, 0xe6, 0xa0, 0x11, 0xc2, 0xea, 0x74, - 0x02, 0x87, 0xd5, 0xd1, 0x9d, 0xb7, 0x7e, 0x38, - 0x60, 0x53, 0x95, 0x8d, 0x25, 0x77, 0x10, 0x5e, - 0x9b, 0x7f, 0xd8, 0x6e, 0xda, 0xa2, 0x2e, 0x20, - 0x4f, 0xcd, 0x8f, 0xcb, 0xbe, 0x5a, 0xe0, 0xed, - 0x2c, 0x9a, 0xd4, 0xe2, 0xaf, 0xd0, 0xa9, 0xe8, - 0xad, 0x7a, 0xbc, 0xa8, 0xf2, 0xee, 0xeb, 0xf5, - 0xa6, 0x99, 0x28, 0x24, 0x6c, 0x2b, 0x75, 0x5d, - 0xf8, 0xd3, 0x86, 0x17, 0xfb, 0xc0, 0x7b, 0xb3, - 0x58, 0xdb, 0xc7, 0x4b, 0xff, 0x04, 0x50, 0xe9, - 0x88, 0x69, 0xc9, 0x2a, 0xab, 0xfd, 0x5b, 0x1b, - 0x8a, 0xd9, 0xec, 0x27, 0x44, 0x0e, 0x33, 0xc8, - 0x6b, 0x93, 0x32, 0x48, 0xb6, 0x30, 0x43, 0xa5] - -class error(Exception): - """Base for all exceptions specific to this module.""" - pass - -class corrupt(error): - """Corrupt save file.""" - - def __init__(self, msg, f = None): - fn = None - if f != None: - fn = getattr(f, "name", None) - self.filename = fn - error.__init__(self, "Corrupt save file: " + msg) - -class eof(corrupt): - """Save file is truncated.""" - - def __init__(self, f = None): - corrupt.__init__(self, "Unexpected EOF", f) - -class subdir(corrupt): - def __init__(self, f = None): - corrupt.__init__(self, "Non-file in save file.", f) - -# -# Table of graphically similar ASCII characters that can be used -# as substitutes for Unicode characters. -# -char_substs = { - u'\u00a2': u"c", - u'\u00b4': u"'", - u'\u00d7': u"x", - u'\u00f7': u"/", - u'\u2010': u"-", - u'\u2015': u"-", - u'\u2018': u"'", - u'\u2019': u"'", - u'\u201c': u'"', - u'\u201d': u'"', - u'\u2032': u"'", - u'\u2212': u"-", - u'\u226a': u"<<", - u'\u226b': u">>", - u'\u2500': u"-", - u'\u2501': u"-", - u'\u2502': u"|", - u'\u2503': u"|", - u'\u250c': u"+", - u'\u250f': u"+", - u'\u2510': u"+", - u'\u2513': u"+", - u'\u2514': u"+", - u'\u2517': u"+", - u'\u2518': u"+", - u'\u251b': u"+", - u'\u251c': u"+", - u'\u251d': u"+", - u'\u2520': u"+", - u'\u2523': u"+", - u'\u2524': u"+", - u'\u2525': u"+", - u'\u2528': u"+", - u'\u252b': u"+", - u'\u252c': u"+", - u'\u252f': u"+", - u'\u2530': u"+", - u'\u2533': u"+", - u'\u2537': u"+", - u'\u2538': u"+", - u'\u253b': u"+", - u'\u253c': u"+", - u'\u253f': u"+", - u'\u2542': u"+", - u'\u254b': u"+", - u'\u25a0': u"#", - u'\u25a1': u"#", - u'\u3001': u",", - u'\u3002': u".", - u'\u3003': u'"', - u'\u3007': u'0', - u'\u3008': u'<', - u'\u3009': u'>', - u'\u300a': u'<<', - u'\u300b': u'>>', - u'\u300a': u'<<', - u'\u300b': u'>>', - u'\u300c': u'[', - u'\u300d': u']', - u'\u300e': u'[', - u'\u300f': u']', - u'\u3010': u'[', - u'\u3011': u']', - u'\u3014': u'[', - u'\u3015': u']', - u'\u301c': u'~', - u'\u30fc': u'-', -} - -def shift_jis_conv(src, encoding = None): - """Convert Shift-JIS strings to a graphically similar representation. - - If encoding is "unicode" then a Unicode string is returned, otherwise - a string in the encoding specified is returned. If necessary, - graphically similar characters are used to replace characters not - exactly representable in the desired encoding. - """ - - if encoding == None: - encoding = sys.getdefaultencoding() - if encoding == "shift_jis": - return src - u = src.decode("shift_jis", "replace") - if encoding == "unicode": - return u - a = [] - for uc in u: - try: - uc.encode(encoding) - a.append(uc) - except UnicodeError: - for uc2 in shift_jis_normalize_table.get(uc, uc): - a.append(char_substs.get(uc2, uc2)) - - return u"".join(a).encode(encoding, "replace") - -def rc4_crypt(s, t): - """RC4 encrypt/decrypt the string t using the permutation s. - - Returns a byte array.""" - - s = array.array('B', s) - t = array.array('B', t) - j = 0 - for ii in range(len(t)): - i = (ii + 1) % 256 - j = (j + s[i]) % 256 - (s[i], s[j]) = (s[j], s[i]) - t[ii] ^= s[(s[i] + s[j]) % 256] - return t - -# def sps_check(s): -# """Calculate the checksum for a SharkPort save.""" -# -# h = 0 -# for c in array.array('B', s): -# h += c << (h % 24) -# h &= 0xFFFFFFFF -# return h - -def unpack_icon_sys(s): - """Unpack an icon.sys file into a tuple.""" - - # magic, title offset, ... - # [14] title, normal icon, copy icon, del icon - a = struct.unpack("<4s2xH4x" - "L" "16s16s16s16s" "16s16s16s" "16s16s16s" "16s" - "68s64s64s64s512x", s) - a = list(a) - for i in range(3, 7): - a[i] = struct.unpack("<4L", a[i]) - a[i] = map(hex, a[i]) - for i in range(7, 14): - a[i] = struct.unpack("<4f", a[i]) - a[14] = zero_terminate(a[14]) - a[15] = zero_terminate(a[15]) - a[16] = zero_terminate(a[16]) - a[17] = zero_terminate(a[17]) - return a - -def icon_sys_title(icon_sys, encoding = None): - """Extract the two lines of the title stored in an icon.sys tuple.""" - - offset = icon_sys[1] - title = icon_sys[14] - title2 = shift_jis_conv(title[offset:], encoding) - title1 = shift_jis_conv(title[:offset], encoding) - return (title1, title2) - -def _read_fixed(f, n): - """Read a string of a fixed length from a file.""" - - s = f.read(n) - if len(s) != n: - raise eof, f - return s - -def _read_long_string(f): - """Read a string prefixed with a 32-bit length from a file.""" - - length = struct.unpack("= 964: - return unpack_icon_sys(data[:964]) - return None - - def load_ems(self, f): - """Load EMS (.psu) save files.""" - - cluster_size = 1024 - - dirent = unpack_dirent(_read_fixed(f, PS2MC_DIRENT_LENGTH)) - dotent = unpack_dirent(_read_fixed(f, PS2MC_DIRENT_LENGTH)) - dotdotent = unpack_dirent(_read_fixed(f, PS2MC_DIRENT_LENGTH)) - if (not mode_is_dir(dirent[0]) - or not mode_is_dir(dotent[0]) - or not mode_is_dir(dotdotent[0]) - or dirent[2] < 2): - raise corrupt, ("Not a EMS (.psu) save file.", f) - - dirent[2] -= 2 - self.set_directory(dirent) - - for i in range(dirent[2]): - ent = unpack_dirent(_read_fixed(f, - PS2MC_DIRENT_LENGTH)) - if not mode_is_file(ent[0]): - raise subdir, f - flen = ent[2] - self.set_file(i, ent, _read_fixed(f, flen)) - _read_fixed(f, round_up(flen, cluster_size) - flen) - - - def save_ems(self, f): - cluster_size = 1024 - - dirent = self.dirent[:] - dirent[2] += 2 - f.write(pack_dirent(dirent)) - f.write(pack_dirent((DF_RWX | DF_DIR | DF_0400 | DF_EXISTS, - 0, 0, dirent[3], - 0, 0, dirent[3], 0, "."))) - f.write(pack_dirent((DF_RWX | DF_DIR | DF_0400 | DF_EXISTS, - 0, 0, dirent[3], - 0, 0, dirent[3], 0, ".."))) - - for i in range(dirent[2] - 2): - (ent, data) = self.get_file(i) - f.write(pack_dirent(ent)) - if not mode_is_file(ent[0]): - # print ent - # print hex(ent[0]) - raise error, "Directory has a subdirectory." - f.write(data) - f.write("\0" * (round_up(len(data), cluster_size) - - len(data))) - f.flush() - - def _load_max_drive_2(self): - (length, s) = self._compressed - self._compressed = None - - if lzari == None: - raise error, ("The lzari module is needed to " - " decompress MAX Drive saves.") - s = lzari.decode(s, length, - "decompressing " + self.dirent[8] + ": ") - dirlen = self.dirent[2] - timestamp = self.dirent[3] - off = 0 - for i in range(dirlen): - if len(s) - off < 36: - raise eof, f - (l, name) = struct.unpack(" 0 and title[0][-1] != ' ': - iconsysname = title[0] + " " + title[1].strip() - else: - iconsysname = title[0] + title[1].rstrip() - s = "" - dirent = self.dirent - for i in range(dirent[2]): - (ent, data) = self.get_file(i) - if not mode_is_file(ent[0]): - raise error, "Non-file in save file." - s += struct.pack("= 2 - and dotent[8] == "." and dotdotent[8] == ".."): - return "psu" - return None - -# -# Set up tables of illegal and problematic characters in file names. -# -_bad_filename_chars = ("".join(map(chr, range(32))) - + "".join(map(chr, range(127, 256)))) -_bad_filename_repl = "_" * len(_bad_filename_chars) - -if os.name in ["nt", "os2", "ce"]: - _bad_filename_chars += '<>:"/\\|' - _bad_filename_repl += "()_'___" - _bad_filename_chars2 = _bad_filename_chars + "?* " - _bad_filename_repl2 = _bad_filename_repl + "___" -else: - _bad_filename_chars += "/" - _bad_filename_repl += "_" - _bad_filename_chars2 = _bad_filename_chars + "?*'&|:[<>] \\\"" - _bad_filename_repl2 = _bad_filename_repl + "______(())___" - -_filename_trans = string.maketrans(_bad_filename_chars, _bad_filename_repl); -_filename_trans2 = string.maketrans(_bad_filename_chars2, _bad_filename_repl2); - -def fix_filename(filename): - """Replace illegal or problematic characters from a filename.""" - return filename.translate(_filename_trans) - -def make_longname(dirname, sf): - """Return a string containing a verbose filename for a save file.""" - - icon_sys = sf.get_icon_sys() - title = "" - if icon_sys != None: - title = icon_sys_title(icon_sys, "ascii") - title = title[0] + " " + title[1] - title = " ".join(title.split()) - crc = binascii.crc32("") - for (ent, data) in sf: - crc = binascii.crc32(data, crc) - if len(dirname) >= 12 and (dirname[0:2] in ("BA", "BJ", "BE", "BK")): - if dirname[2:6] == "DATA": - title = "" - else: - #dirname = dirname[2:6] + dirname[7:12] - dirname = dirname[2:12] - - return fix_filename("%s %s (%08X)" - % (dirname, title, crc & 0xFFFFFFFF)) - +# +# ps2save.py +# +# By Ross Ridge +# Public Domain +# +# A simple interface for working with various PS2 save file formats. +# + +_SCCS_ID = "@(#) mymc ps2save.py 1.8 22/01/15 01:25:25\n" + +import sys +import os +import string +import struct +import binascii +import array +import zlib + +from round import div_round_up, round_up +from ps2mc_dir import * +from sjistab import shift_jis_normalize_table + +try: + import lzari +except ImportError: + lzari = None + +PS2SAVE_MAX_MAGIC = "Ps2PowerSave" +PS2SAVE_SPS_MAGIC = "\x0d\0\0\0SharkPortSave" +PS2SAVE_CBS_MAGIC = "CFU\0" +PS2SAVE_NPO_MAGIC = "nPort" + +# This is the initial permutation state ("S") for the RC4 stream cipher +# algorithm used to encrpyt and decrypt Codebreaker saves. +PS2SAVE_CBS_RC4S = [0x5f, 0x1f, 0x85, 0x6f, 0x31, 0xaa, 0x3b, 0x18, + 0x21, 0xb9, 0xce, 0x1c, 0x07, 0x4c, 0x9c, 0xb4, + 0x81, 0xb8, 0xef, 0x98, 0x59, 0xae, 0xf9, 0x26, + 0xe3, 0x80, 0xa3, 0x29, 0x2d, 0x73, 0x51, 0x62, + 0x7c, 0x64, 0x46, 0xf4, 0x34, 0x1a, 0xf6, 0xe1, + 0xba, 0x3a, 0x0d, 0x82, 0x79, 0x0a, 0x5c, 0x16, + 0x71, 0x49, 0x8e, 0xac, 0x8c, 0x9f, 0x35, 0x19, + 0x45, 0x94, 0x3f, 0x56, 0x0c, 0x91, 0x00, 0x0b, + 0xd7, 0xb0, 0xdd, 0x39, 0x66, 0xa1, 0x76, 0x52, + 0x13, 0x57, 0xf3, 0xbb, 0x4e, 0xe5, 0xdc, 0xf0, + 0x65, 0x84, 0xb2, 0xd6, 0xdf, 0x15, 0x3c, 0x63, + 0x1d, 0x89, 0x14, 0xbd, 0xd2, 0x36, 0xfe, 0xb1, + 0xca, 0x8b, 0xa4, 0xc6, 0x9e, 0x67, 0x47, 0x37, + 0x42, 0x6d, 0x6a, 0x03, 0x92, 0x70, 0x05, 0x7d, + 0x96, 0x2f, 0x40, 0x90, 0xc4, 0xf1, 0x3e, 0x3d, + 0x01, 0xf7, 0x68, 0x1e, 0xc3, 0xfc, 0x72, 0xb5, + 0x54, 0xcf, 0xe7, 0x41, 0xe4, 0x4d, 0x83, 0x55, + 0x12, 0x22, 0x09, 0x78, 0xfa, 0xde, 0xa7, 0x06, + 0x08, 0x23, 0xbf, 0x0f, 0xcc, 0xc1, 0x97, 0x61, + 0xc5, 0x4a, 0xe6, 0xa0, 0x11, 0xc2, 0xea, 0x74, + 0x02, 0x87, 0xd5, 0xd1, 0x9d, 0xb7, 0x7e, 0x38, + 0x60, 0x53, 0x95, 0x8d, 0x25, 0x77, 0x10, 0x5e, + 0x9b, 0x7f, 0xd8, 0x6e, 0xda, 0xa2, 0x2e, 0x20, + 0x4f, 0xcd, 0x8f, 0xcb, 0xbe, 0x5a, 0xe0, 0xed, + 0x2c, 0x9a, 0xd4, 0xe2, 0xaf, 0xd0, 0xa9, 0xe8, + 0xad, 0x7a, 0xbc, 0xa8, 0xf2, 0xee, 0xeb, 0xf5, + 0xa6, 0x99, 0x28, 0x24, 0x6c, 0x2b, 0x75, 0x5d, + 0xf8, 0xd3, 0x86, 0x17, 0xfb, 0xc0, 0x7b, 0xb3, + 0x58, 0xdb, 0xc7, 0x4b, 0xff, 0x04, 0x50, 0xe9, + 0x88, 0x69, 0xc9, 0x2a, 0xab, 0xfd, 0x5b, 0x1b, + 0x8a, 0xd9, 0xec, 0x27, 0x44, 0x0e, 0x33, 0xc8, + 0x6b, 0x93, 0x32, 0x48, 0xb6, 0x30, 0x43, 0xa5] + +class error(Exception): + """Base for all exceptions specific to this module.""" + pass + +class corrupt(error): + """Corrupt save file.""" + + def __init__(self, msg, f = None): + fn = None + if f != None: + fn = getattr(f, "name", None) + self.filename = fn + error.__init__(self, "Corrupt save file: " + msg) + +class eof(corrupt): + """Save file is truncated.""" + + def __init__(self, f = None): + corrupt.__init__(self, "Unexpected EOF", f) + +class subdir(corrupt): + def __init__(self, f = None): + corrupt.__init__(self, "Non-file in save file.", f) + +# +# Table of graphically similar ASCII characters that can be used +# as substitutes for Unicode characters. +# +char_substs = { + u'\u00a2': u"c", + u'\u00b4': u"'", + u'\u00d7': u"x", + u'\u00f7': u"/", + u'\u2010': u"-", + u'\u2015': u"-", + u'\u2018': u"'", + u'\u2019': u"'", + u'\u201c': u'"', + u'\u201d': u'"', + u'\u2032': u"'", + u'\u2212': u"-", + u'\u226a': u"<<", + u'\u226b': u">>", + u'\u2500': u"-", + u'\u2501': u"-", + u'\u2502': u"|", + u'\u2503': u"|", + u'\u250c': u"+", + u'\u250f': u"+", + u'\u2510': u"+", + u'\u2513': u"+", + u'\u2514': u"+", + u'\u2517': u"+", + u'\u2518': u"+", + u'\u251b': u"+", + u'\u251c': u"+", + u'\u251d': u"+", + u'\u2520': u"+", + u'\u2523': u"+", + u'\u2524': u"+", + u'\u2525': u"+", + u'\u2528': u"+", + u'\u252b': u"+", + u'\u252c': u"+", + u'\u252f': u"+", + u'\u2530': u"+", + u'\u2533': u"+", + u'\u2537': u"+", + u'\u2538': u"+", + u'\u253b': u"+", + u'\u253c': u"+", + u'\u253f': u"+", + u'\u2542': u"+", + u'\u254b': u"+", + u'\u25a0': u"#", + u'\u25a1': u"#", + u'\u2605': u"*", + u'\u2606': u"*", + u'\u3001': u",", + u'\u3002': u".", + u'\u3003': u'"', + u'\u3007': u'0', + u'\u3008': u'<', + u'\u3009': u'>', + u'\u300a': u'<<', + u'\u300b': u'>>', + u'\u300a': u'<<', + u'\u300b': u'>>', + u'\u300c': u'[', + u'\u300d': u']', + u'\u300e': u'[', + u'\u300f': u']', + u'\u3010': u'[', + u'\u3011': u']', + u'\u3014': u'[', + u'\u3015': u']', + u'\u301c': u'~', + u'\u30fc': u'-', +} + +def shift_jis_conv(src, encoding = None): + """Convert Shift-JIS strings to a graphically similar representation. + + If encoding is "unicode" then a Unicode string is returned, otherwise + a string in the encoding specified is returned. If necessary, + graphically similar characters are used to replace characters not + exactly representable in the desired encoding. + """ + + if encoding == None: + encoding = sys.getdefaultencoding() + if encoding == "shift_jis": + return src + u = src.decode("shift_jis", "replace") + if encoding == "unicode": + return u + a = [] + for uc in u: + try: + uc.encode(encoding) + a.append(uc) + except UnicodeError: + for uc2 in shift_jis_normalize_table.get(uc, uc): + a.append(char_substs.get(uc2, uc2)) + + return u"".join(a).encode(encoding, "replace") + +def rc4_crypt(s, t): + """RC4 encrypt/decrypt the string t using the permutation s. + + Returns a byte array.""" + + s = array.array('B', s) + t = array.array('B', t) + j = 0 + for ii in range(len(t)): + i = (ii + 1) % 256 + j = (j + s[i]) % 256 + (s[i], s[j]) = (s[j], s[i]) + t[ii] ^= s[(s[i] + s[j]) % 256] + return t + +# def sps_check(s): +# """Calculate the checksum for a SharkPort save.""" +# +# h = 0 +# for c in array.array('B', s): +# h += c << (h % 24) +# h &= 0xFFFFFFFF +# return h + +def unpack_icon_sys(s): + """Unpack an icon.sys file into a tuple.""" + + # magic, title offset, ... + # [14] title, normal icon, copy icon, del icon + a = struct.unpack("<4s2xH4x" + "L" "16s16s16s16s" "16s16s16s" "16s16s16s" "16s" + "68s64s64s64s512x", s) + a = list(a) + for i in range(3, 7): + a[i] = struct.unpack("<4L", a[i]) + a[i] = map(hex, a[i]) + for i in range(7, 14): + a[i] = struct.unpack("<4f", a[i]) + a[14] = zero_terminate(a[14]) + a[15] = zero_terminate(a[15]) + a[16] = zero_terminate(a[16]) + a[17] = zero_terminate(a[17]) + return a + +def icon_sys_title(icon_sys, encoding = None): + """Extract the two lines of the title stored in an icon.sys tuple.""" + + offset = icon_sys[1] + title = icon_sys[14] + title2 = shift_jis_conv(title[offset:], encoding) + title1 = shift_jis_conv(title[:offset], encoding) + return (title1, title2) + +def _read_fixed(f, n): + """Read a string of a fixed length from a file.""" + + s = f.read(n) + if len(s) != n: + raise eof, f + return s + +def _read_long_string(f): + """Read a string prefixed with a 32-bit length from a file.""" + + length = struct.unpack("= 964: + return unpack_icon_sys(data[:964]) + return None + + def load_ems(self, f): + """Load EMS (.psu) save files.""" + + cluster_size = 1024 + + dirent = unpack_dirent(_read_fixed(f, PS2MC_DIRENT_LENGTH)) + dotent = unpack_dirent(_read_fixed(f, PS2MC_DIRENT_LENGTH)) + dotdotent = unpack_dirent(_read_fixed(f, PS2MC_DIRENT_LENGTH)) + if (not mode_is_dir(dirent[0]) + or not mode_is_dir(dotent[0]) + or not mode_is_dir(dotdotent[0]) + or dirent[2] < 2): + raise corrupt, ("Not a EMS (.psu) save file.", f) + + dirent[2] -= 2 + self.set_directory(dirent) + + for i in range(dirent[2]): + ent = unpack_dirent(_read_fixed(f, + PS2MC_DIRENT_LENGTH)) + if not mode_is_file(ent[0]): + raise subdir, f + flen = ent[2] + self.set_file(i, ent, _read_fixed(f, flen)) + _read_fixed(f, round_up(flen, cluster_size) - flen) + + + def save_ems(self, f): + cluster_size = 1024 + + dirent = self.dirent[:] + dirent[2] += 2 + f.write(pack_dirent(dirent)) + f.write(pack_dirent((DF_RWX | DF_DIR | DF_0400 | DF_EXISTS, + 0, 0, dirent[3], + 0, 0, dirent[3], 0, "."))) + f.write(pack_dirent((DF_RWX | DF_DIR | DF_0400 | DF_EXISTS, + 0, 0, dirent[3], + 0, 0, dirent[3], 0, ".."))) + + for i in range(dirent[2] - 2): + (ent, data) = self.get_file(i) + f.write(pack_dirent(ent)) + if not mode_is_file(ent[0]): + # print ent + # print hex(ent[0]) + raise error, "Directory has a subdirectory." + f.write(data) + f.write("\0" * (round_up(len(data), cluster_size) + - len(data))) + f.flush() + + def _load_max_drive_2(self): + (length, s) = self._compressed + self._compressed = None + + if lzari == None: + raise error, ("The lzari module is needed to " + " decompress MAX Drive saves.") + s = lzari.decode(s, length, + "decompressing " + self.dirent[8] + ": ") + dirlen = self.dirent[2] + timestamp = self.dirent[3] + off = 0 + for i in range(dirlen): + if len(s) - off < 36: + raise eof, f + (l, name) = struct.unpack(" 0 and title[0][-1] != ' ': + iconsysname = title[0] + " " + title[1].strip() + else: + iconsysname = title[0] + title[1].rstrip() + s = "" + dirent = self.dirent + for i in range(dirent[2]): + (ent, data) = self.get_file(i) + if not mode_is_file(ent[0]): + raise error, "Non-file in save file." + s += struct.pack("= 2 + and dotent[8] == "." and dotdotent[8] == ".."): + return "psu" + return None + +# +# Set up tables of illegal and problematic characters in file names. +# +_bad_filename_chars = ("".join(map(chr, range(32))) + + "".join(map(chr, range(127, 256)))) +_bad_filename_repl = "_" * len(_bad_filename_chars) + +if os.name in ["nt", "os2", "ce"]: + _bad_filename_chars += '<>:"/\\|?*' + _bad_filename_repl += "()_'_____" + _bad_filename_chars2 = _bad_filename_chars + " " + _bad_filename_repl2 = _bad_filename_repl + "_" +else: + _bad_filename_chars += "/" + _bad_filename_repl += "_" + _bad_filename_chars2 = _bad_filename_chars + "?*'&|:[<>] \\\"" + _bad_filename_repl2 = _bad_filename_repl + "______(())___" + +_filename_trans = string.maketrans(_bad_filename_chars, _bad_filename_repl); +_filename_trans2 = string.maketrans(_bad_filename_chars2, _bad_filename_repl2); + +def fix_filename(filename): + """Replace illegal or problematic characters from a filename.""" + return filename.translate(_filename_trans) + +def make_longname(dirname, sf): + """Return a string containing a verbose filename for a save file.""" + + icon_sys = sf.get_icon_sys() + title = "" + if icon_sys != None: + title = icon_sys_title(icon_sys, "ascii") + title = title[0] + " " + title[1] + title = " ".join(title.split()) + crc = binascii.crc32("") + for (ent, data) in sf: + crc = binascii.crc32(data, crc) + if len(dirname) >= 12 and (dirname[0:2] in ("BA", "BJ", "BE", "BK")): + if dirname[2:6] == "DATA": + title = "" + else: + #dirname = dirname[2:6] + dirname[7:12] + dirname = dirname[2:12] + + return fix_filename("%s %s (%08X)" + % (dirname, title, crc & 0xFFFFFFFF)) + diff --git a/round.py b/round.py index d34c040..1d81ef9 100755 --- a/round.py +++ b/round.py @@ -1,21 +1,21 @@ -# -# round.py -# -# By Ross Ridge -# Public Domain -# -# Simple rounding functions. -# - -_SCCS_ID = "@(#) mysc round.py 1.3 07/04/17 02:10:27\n" - -def div_round_up(a, b): - return (a + b - 1) / b - -def round_up(a, b): - return (a + b - 1) / b * b - -def round_down(a, b): - return a / b * b - - +# +# round.py +# +# By Ross Ridge +# Public Domain +# +# Simple rounding functions. +# + +_SCCS_ID = "@(#) mymc round.py 1.3 07/04/17 02:10:27\n" + +def div_round_up(a, b): + return (a + b - 1) / b + +def round_up(a, b): + return (a + b - 1) / b * b + +def round_down(a, b): + return a / b * b + + diff --git a/sjistab.py b/sjistab.py index 3376721..e635f3a 100755 --- a/sjistab.py +++ b/sjistab.py @@ -1,2 +1,2 @@ -# automatically generated -shift_jis_normalize_table = {u'\uff81': u'\u30c1', u'\u3000': u' ', u'\uff85': u'\u30ca', u'\uff06': u'&', u'\uff89': u'\u30ce', u'\uff0a': u'*', u'\uff8d': u'\u30d8', u'\uff0e': u'.', u'\uff91': u'\u30e0', u'\uff12': u'2', u'\uff95': u'\u30e6', u'\uff16': u'6', u'\uff99': u'\u30eb', u'\u309b': u' \u3099', u'\uff1a': u':', u'\uff9d': u'\u30f3', u'\uff03': u'#', u'\uff1e': u'>', u'\uff22': u'B', u'\uff26': u'F', u'\uff2a': u'J', u'\u222c': u'\u222b\u222b', u'\uff2e': u'N', u'\uff32': u'R', u'\uff36': u'V', u'\uff3a': u'Z', u'\uff3e': u'^', u'\uff42': u'b', u'\uff46': u'f', u'\uff4a': u'j', u'\uff4e': u'n', u'\uff52': u'r', u'\uff56': u'v', u'\uff5a': u'z', u'\uff62': u'\u300c', u'\uffe5': u'\xa5', u'\uff66': u'\u30f2', u'\uff6a': u'\u30a7', u'\uff6e': u'\u30e7', u'\uff72': u'\u30a4', u'\uff76': u'\u30ab', u'\uff7a': u'\u30b3', u'\uff7e': u'\u30bb', u'\uff01': u'!', u'\uff82': u'\u30c4', u'\uff05': u'%', u'\uff86': u'\u30cb', u'\uff09': u')', u'\uff8a': u'\u30cf', u'\uff8e': u'\u30db', u'\uff11': u'1', u'\uff92': u'\u30e1', u'\uff15': u'5', u'\uff96': u'\u30e8', u'\uff19': u'9', u'\uff9a': u'\u30ec', u'\uff1d': u'=', u'\u309c': u' \u309a', u'\uff9e': u'\u3099', u'\uff21': u'A', u'\uff25': u'E', u'\uff29': u'I', u'\xa8': u' \u0308', u'\uff2d': u'M', u'\uff31': u'Q', u'\u2033': u'\u2032\u2032', u'\uff35': u'U', u'\xb4': u' \u0301', u'\uff39': u'Y', u'\uff3d': u']', u'\uff41': u'a', u'\uff45': u'e', u'\uff49': u'i', u'\uff4d': u'm', u'\uff51': u'q', u'\uff55': u'u', u'\uff59': u'y', u'\uff5d': u'}', u'\uff61': u'\u3002', u'\uff65': u'\u30fb', u'\uff69': u'\u30a5', u'\uff6d': u'\u30e5', u'\uff71': u'\u30a2', u'\uff75': u'\u30aa', u'\uff79': u'\u30b1', u'\uff7d': u'\u30b9', u'\uff83': u'\u30c6', u'\uff04': u'$', u'\uff87': u'\u30cc', u'\uff08': u'(', u'\uff8b': u'\u30d2', u'\uff0c': u',', u'\uff8f': u'\u30de', u'\uff10': u'0', u'\uff93': u'\u30e2', u'\uff14': u'4', u'\uff97': u'\u30e9', u'\uff18': u'8', u'\uff9b': u'\u30ed', u'\uff1c': u'<', u'\uff9f': u'\u309a', u'\uff20': u'@', u'\uff24': u'D', u'\u2026': u'...', u'\uff28': u'H', u'\uff2c': u'L', u'\uff30': u'P', u'\uff34': u'T', u'\uff38': u'X', u'\uff3c': u'\\', u'\uff40': u'`', u'\uff44': u'd', u'\uff48': u'h', u'\uff4c': u'l', u'\uff50': u'p', u'\uff54': u't', u'\uff58': u'x', u'\uff5c': u'|', u'\uffe3': u' \u0304', u'\uff64': u'\u3001', u'\uff68': u'\u30a3', u'\uff6c': u'\u30e3', u'\uff70': u'\u30fc', u'\uff74': u'\u30a8', u'\uff78': u'\u30af', u'\uff7c': u'\u30b7', u'\uff80': u'\u30bf', u'\u2103': u'\xb0C', u'\uff84': u'\u30c8', u'\uff88': u'\u30cd', u'\uff0b': u'+', u'\uff8c': u'\u30d5', u'\uff0f': u'/', u'\uff90': u'\u30df', u'\uff13': u'3', u'\uff94': u'\u30e4', u'\uff17': u'7', u'\uff98': u'\u30ea', u'\uff1b': u';', u'\uff9c': u'\u30ef', u'\uff1f': u'?', u'\uff23': u'C', u'\u2025': u'..', u'\uff27': u'G', u'\u212b': u'\xc5', u'\uff2f': u'O', u'\uff33': u'S', u'\uff37': u'W', u'\uff3b': u'[', u'\uff3f': u'_', u'\uff43': u'c', u'\uff47': u'g', u'\uff4b': u'k', u'\uff4f': u'o', u'\uff53': u's', u'\uff57': u'w', u'\uff5b': u'{', u'\uff63': u'\u300d', u'\uff67': u'\u30a1', u'\uff6b': u'\u30a9', u'\uff6f': u'\u30c3', u'\uff73': u'\u30a6', u'\uff77': u'\u30ad', u'\uff7b': u'\u30b5', u'\uff2b': u'K', u'\uff7f': u'\u30bd'} +# automatically generated +shift_jis_normalize_table = {u'\uff81': u'\u30c1', u'\u3000': u' ', u'\uff85': u'\u30ca', u'\uff06': u'&', u'\uff89': u'\u30ce', u'\uff0a': u'*', u'\uff8d': u'\u30d8', u'\uff0e': u'.', u'\uff91': u'\u30e0', u'\uff12': u'2', u'\uff95': u'\u30e6', u'\uff16': u'6', u'\uff99': u'\u30eb', u'\u309b': u' \u3099', u'\uff1a': u':', u'\uff9d': u'\u30f3', u'\uff03': u'#', u'\uff1e': u'>', u'\uff22': u'B', u'\uff26': u'F', u'\uff2a': u'J', u'\u222c': u'\u222b\u222b', u'\uff2e': u'N', u'\uff32': u'R', u'\uff36': u'V', u'\uff3a': u'Z', u'\uff3e': u'^', u'\uff42': u'b', u'\uff46': u'f', u'\uff4a': u'j', u'\uff4e': u'n', u'\uff52': u'r', u'\uff56': u'v', u'\uff5a': u'z', u'\uff62': u'\u300c', u'\uffe5': u'\xa5', u'\uff66': u'\u30f2', u'\uff6a': u'\u30a7', u'\uff6e': u'\u30e7', u'\uff72': u'\u30a4', u'\uff76': u'\u30ab', u'\uff7a': u'\u30b3', u'\uff7e': u'\u30bb', u'\uff01': u'!', u'\uff82': u'\u30c4', u'\uff05': u'%', u'\uff86': u'\u30cb', u'\uff09': u')', u'\uff8a': u'\u30cf', u'\uff8e': u'\u30db', u'\uff11': u'1', u'\uff92': u'\u30e1', u'\uff15': u'5', u'\uff96': u'\u30e8', u'\uff19': u'9', u'\uff9a': u'\u30ec', u'\uff1d': u'=', u'\u309c': u' \u309a', u'\uff9e': u'\u3099', u'\uff21': u'A', u'\uff25': u'E', u'\uff29': u'I', u'\xa8': u' \u0308', u'\uff2d': u'M', u'\uff31': u'Q', u'\u2033': u'\u2032\u2032', u'\uff35': u'U', u'\xb4': u' \u0301', u'\uff39': u'Y', u'\uff3d': u']', u'\uff41': u'a', u'\uff45': u'e', u'\uff49': u'i', u'\uff4d': u'm', u'\uff51': u'q', u'\uff55': u'u', u'\uff59': u'y', u'\uff5d': u'}', u'\uff61': u'\u3002', u'\uff65': u'\u30fb', u'\uff69': u'\u30a5', u'\uff6d': u'\u30e5', u'\uff71': u'\u30a2', u'\uff75': u'\u30aa', u'\uff79': u'\u30b1', u'\uff7d': u'\u30b9', u'\uff83': u'\u30c6', u'\uff04': u'$', u'\uff87': u'\u30cc', u'\uff08': u'(', u'\uff8b': u'\u30d2', u'\uff0c': u',', u'\uff8f': u'\u30de', u'\uff10': u'0', u'\uff93': u'\u30e2', u'\uff14': u'4', u'\uff97': u'\u30e9', u'\uff18': u'8', u'\uff9b': u'\u30ed', u'\uff1c': u'<', u'\uff9f': u'\u309a', u'\uff20': u'@', u'\uff24': u'D', u'\u2026': u'...', u'\uff28': u'H', u'\uff2c': u'L', u'\uff30': u'P', u'\uff34': u'T', u'\uff38': u'X', u'\uff3c': u'\\', u'\uff40': u'`', u'\uff44': u'd', u'\uff48': u'h', u'\uff4c': u'l', u'\uff50': u'p', u'\uff54': u't', u'\uff58': u'x', u'\uff5c': u'|', u'\uffe3': u' \u0304', u'\uff64': u'\u3001', u'\uff68': u'\u30a3', u'\uff6c': u'\u30e3', u'\uff70': u'\u30fc', u'\uff74': u'\u30a8', u'\uff78': u'\u30af', u'\uff7c': u'\u30b7', u'\uff80': u'\u30bf', u'\u2103': u'\xb0C', u'\uff84': u'\u30c8', u'\uff88': u'\u30cd', u'\uff0b': u'+', u'\uff8c': u'\u30d5', u'\uff0f': u'/', u'\uff90': u'\u30df', u'\uff13': u'3', u'\uff94': u'\u30e4', u'\uff17': u'7', u'\uff98': u'\u30ea', u'\uff1b': u';', u'\uff9c': u'\u30ef', u'\uff1f': u'?', u'\uff23': u'C', u'\u2025': u'..', u'\uff27': u'G', u'\u212b': u'\xc5', u'\uff2f': u'O', u'\uff33': u'S', u'\uff37': u'W', u'\uff3b': u'[', u'\uff3f': u'_', u'\uff43': u'c', u'\uff47': u'g', u'\uff4b': u'k', u'\uff4f': u'o', u'\uff53': u's', u'\uff57': u'w', u'\uff5b': u'{', u'\uff63': u'\u300d', u'\uff67': u'\u30a1', u'\uff6b': u'\u30a9', u'\uff6f': u'\u30c3', u'\uff73': u'\u30a6', u'\uff77': u'\u30ad', u'\uff7b': u'\u30b5', u'\uff2b': u'K', u'\uff7f': u'\u30bd'} diff --git a/verbuild.py b/verbuild.py index 68c0d52..71b2f66 100755 --- a/verbuild.py +++ b/verbuild.py @@ -1,2 +1,2 @@ -MYMC_VERSION_BUILD = r'''6''' -MYMC_VERSION_MAJOR = r'''2''' +MYMC_VERSION_BUILD = r'''7''' +MYMC_VERSION_MAJOR = r'''2'''