Improve CPK support

README.md

@@ -0,0 +1,24 @@
+# Sonic Audio Tools
+Sonic Audio Tools is a set of tools for editing the audio formats seen in Sonic the Hedgehog games. Currently, it's mostly focused on CRIWARE's audio formats, this means that these tools can be used to edit files from any game, as long as they are CSB, CPK, ACB or AWB.
+## Building
+1. Clone from [GitHub](https://github.com/blueskythlikesclouds/SonicAudioTools.git) `git clone https://github.com/blueskythlikesclouds/SonicAudioTools.git`
+2. Open the solution in Visual Studio. (Visual Studio 2015 or later is required.)
+3. Install the missing NuGet packages.
+4. Build the solution.
+
+## Projects
+For more information about the projects, visit the [wiki](https://github.com/blueskythlikesclouds/SonicAudioTools/wiki) page.
+### [SonicAudioLib](https://github.com/blueskythlikesclouds/SonicAudioTools/tree/master/Source/SonicAudioLib)
+Core class library of the solution. Handles the IO work of the other projects.
+### [AcbEditor](https://github.com/blueskythlikesclouds/SonicAudioTools/tree/master/Source/AcbEditor)
+A tool to replace audio data in ACB files. Handles both ACBs and AWBs. Old ACB versions which had CPKs for storage are also supported.
+### [CsbBuilder](https://github.com/blueskythlikesclouds/SonicAudioTools/tree/master/Source/CsbBuilder)
+A fully functional CSB creator. It's currently able to load any CSB file as long as they contain ADX files. It also always builds in the latest CSB version, which appeared probably around 2010.
+### [CsbEditor](https://github.com/blueskythlikesclouds/SonicAudioTools/tree/master/Source/CsbEditor)
+A tool to replace audio data in CSB files. Handles both CSBs and CPKs.
+
+## Releases
+The compiled executables can be found in the [Release](https://github.com/blueskythlikesclouds/SonicAudioTools/tree/master/Release) directory. This directory will be removed when the projects get released under the [Releases](https://github.com/blueskythlikesclouds/SonicAudioTools/releases) page.
+
+## License
+See [LICENSE.txt](https://github.com/blueskythlikesclouds/SonicAudioTools/blob/master/LICENSE.txt)

Source/CsbBuilder/CsbBuilder.csproj

@@ -221,6 +221,9 @@
   <ItemGroup>
     <None Include="Resources\Template_16x.png" />
   </ItemGroup>
+  <ItemGroup>
+    <Content Include="NAudio.LICENSE.txt" />
+  </ItemGroup>
   <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
   <!-- To modify your build process, add your task inside one of the targets below and uncomment it. 
        Other similar extension points exist, see Microsoft.Common.targets.

Source/CsbBuilder/NAudio.LICENSE.txt

@@ -0,0 +1,31 @@
+Microsoft Public License (Ms-PL)
+
+This license governs use of the accompanying software. If you use the software, you accept this license. If you do not accept the license, do not use the software.
+
+1. Definitions
+
+The terms "reproduce," "reproduction," "derivative works," and "distribution" have the same meaning here as under U.S. copyright law.
+
+A "contribution" is the original software, or any additions or changes to the software.
+
+A "contributor" is any person that distributes its contribution under this license.
+
+"Licensed patents" are a contributor's patent claims that read directly on its contribution.
+
+2. Grant of Rights
+
+(A) Copyright Grant- Subject to the terms of this license, including the license conditions and limitations in section 3, each contributor grants you a non-exclusive, worldwide, royalty-free copyright license to reproduce its contribution, prepare derivative works of its contribution, and distribute its contribution or any derivative works that you create.
+
+(B) Patent Grant- Subject to the terms of this license, including the license conditions and limitations in section 3, each contributor grants you a non-exclusive, worldwide, royalty-free license under its licensed patents to make, have made, use, sell, offer for sale, import, and/or otherwise dispose of its contribution in the software or derivative works of the contribution in the software.
+
+3. Conditions and Limitations
+
+(A) No Trademark License- This license does not grant you rights to use any contributors' name, logo, or trademarks.
+
+(B) If you bring a patent claim against any contributor over patents that you claim are infringed by the software, your patent license from such contributor to the software ends automatically.
+
+(C) If you distribute any portion of the software, you must retain all copyright, patent, trademark, and attribution notices that are present in the software.
+
+(D) If you distribute any portion of the software in source code form, you may do so only under this license by including a complete copy of this license with your distribution. If you distribute any portion of the software in compiled or object code form, you may only do so under a license that complies with this license.
+
+(E) The software is licensed "as-is." You bear the risk of using it. The contributors give no express warranties, guarantees or conditions. You may have additional consumer rights under your local laws which this license cannot change. To the extent permitted under your local laws, the contributors exclude the implied warranties of merchantability, fitness for a particular purpose and non-infringement. 

Source/CsbEditor/Program.cs

@@ -129,6 +129,7 @@ namespace CsbEditor
                     }
 
                     CriCpkArchive cpkArchive = new CriCpkArchive();
+                //cpkArchive.EnableMask = true;
 
                     CriTable csbFile = new CriTable();
                     csbFile.Load(csbPath);

Source/SonicAudioCmd/Program.cs

@@ -20,7 +20,11 @@ namespace SonicAudioCmd
     {
         static void Main(string[] args)
         {
-            // What are you looking at? There's nothing here. Go away.
+            using (Stream source = File.OpenRead(args[0]))
+            using (Stream destination = File.Create(args[0] + "-unmask"))
+            {
+                Methods.MaskCriTable(source, destination, source.Length);
+            }
         }
     }
 }

Source/SonicAudioLib/Archive/CriCpkArchive.cs

@@ -54,6 +54,7 @@ namespace SonicAudioLib.Archive
     {
         private ushort align = 1;
         private CriCpkMode mode = CriCpkMode.FileName;
+        private bool enableMask = false;
 
         public ushort Align
         {
@@ -86,6 +87,19 @@ namespace SonicAudioLib.Archive
             }
         }
 
+        public bool EnableMask
+        {
+            get
+            {
+                return enableMask;
+            }
+
+            set
+            {
+                enableMask = value;
+            }
+        }
+
         public string Comment { get; set; }
 
         public override void Read(Stream source)
@@ -94,17 +108,38 @@ namespace SonicAudioLib.Archive
             {
                 reader.Read();
 
-                // The older CPK versions don't have the CpkMode field, and the other stuff. I will add
+                bool latest = reader.ContainsField("CpkMode");
-                // support for the older versions when someone reports it. I need file examples.
-                ushort version = reader.GetUInt16("Version");
-                ushort revision = reader.GetUInt16("Revision");
 
-                if (version != 7 && revision != 2)
+                if (latest)
                 {
-                    throw new Exception($"This CPK file version ({version}.{revision}) isn't supported yet! Please report the error with the file if you want support for this CPK version.");
+                    mode = (CriCpkMode)reader.GetUInt32("CpkMode");
                 }
 
-                mode = (CriCpkMode)reader.GetUInt32("CpkMode");
+                else
+                {
+                    bool tocEnabled = reader.GetUInt64("TocOffset") > 0;
+                    bool itocEnabled = reader.GetUInt64("ItocOffset") > 0;
+
+                    if (tocEnabled && !itocEnabled)
+                    {
+                        mode = CriCpkMode.FileName;
+                    }
+
+                    else if (!tocEnabled && itocEnabled)
+                    {
+                        mode = CriCpkMode.Id;
+                    }
+
+                    else if (tocEnabled && itocEnabled)
+                    {
+                        mode = CriCpkMode.FileNameAndId;
+                    }
+
+                    else
+                    {
+                        mode = CriCpkMode.None;
+                    }
+                }
 
                 // No need to waste time, stop right there.
                 if (mode == CriCpkMode.None)
@@ -131,7 +166,7 @@ namespace SonicAudioLib.Archive
                             entry.Name = tocReader.GetString("FileName");
                             entry.Length = tocReader.GetUInt32("FileSize");
                             entry.Position = (long)tocReader.GetUInt64("FileOffset");
-                            entry.Id = tocReader.GetUInt32("ID");
+                            entry.Id = latest ? tocReader.GetUInt32("ID") : tocReader.GetUInt32("Info");
                             entry.Comment = tocReader.GetString("UserString");
                             entry.IsCompressed = entry.Length != tocReader.GetUInt32("ExtractSize");
 
@@ -154,7 +189,7 @@ namespace SonicAudioLib.Archive
                         }
                     }
 
-                    if (mode == CriCpkMode.FileNameAndId)
+                    if (mode == CriCpkMode.FileNameAndId && latest)
                     {
                         using (CriTableReader itocReader = CriCpkSection.Open(source, itocPosition))
                         {
@@ -229,7 +264,7 @@ namespace SonicAudioLib.Archive
         {
             VldPool vldPool = new VldPool(Align, 2048);
 
-            using (CriCpkSection cpkSection = new CriCpkSection(destination, "CPK "))
+            using (CriCpkSection cpkSection = new CriCpkSection(destination, "CPK ", enableMask))
             {
                 cpkSection.Writer.WriteStartTable("CpkHeader");
 
@@ -319,8 +354,8 @@ namespace SonicAudioLib.Archive
 
                     var orderedEntries = entries.OrderBy(entry => entry.Name).ToList();
 
-                    using (CriCpkSection tocSection = new CriCpkSection(tocMemoryStream, "TOC "))
+                    using (CriCpkSection tocSection = new CriCpkSection(tocMemoryStream, "TOC ", enableMask))
-                    using (CriCpkSection etocSection = new CriCpkSection(etocMemoryStream, "ETOC"))
+                    using (CriCpkSection etocSection = new CriCpkSection(etocMemoryStream, "ETOC", enableMask))
                     {
                         tocSection.Writer.WriteStartTable("CpkTocInfo");
 
@@ -363,7 +398,7 @@ namespace SonicAudioLib.Archive
                     {
                         itocMemoryStream = new MemoryStream();
 
-                        using (CriCpkSection itocSection = new CriCpkSection(itocMemoryStream, "ITOC"))
+                        using (CriCpkSection itocSection = new CriCpkSection(itocMemoryStream, "ITOC", enableMask))
                         {
                             itocSection.Writer.WriteStartTable("CpkExtendId");
 
@@ -386,7 +421,7 @@ namespace SonicAudioLib.Archive
                 {
                     itocMemoryStream = new MemoryStream();
 
-                    using (CriCpkSection itocSection = new CriCpkSection(itocMemoryStream, "ITOC"))
+                    using (CriCpkSection itocSection = new CriCpkSection(itocMemoryStream, "ITOC", enableMask))
                     {
                         itocSection.Writer.WriteStartTable("CpkItocInfo");
 
@@ -598,18 +633,11 @@ namespace SonicAudioLib.Archive
                 uint tableLength = EndianStream.ReadUInt32(source);
                 uint unknown = EndianStream.ReadUInt32(source);
 
-                try
-                {
-                    return CriTableReader.Create(new Substream(source, source.Position, tableLength));
-                }
 
-                catch
+                return CriTableReader.Create(new Substream(source, source.Position, tableLength));
-                {
-                    throw new Exception("CPK file decryption needs to be implemented or it's an unknown error. Please report this error with the file.");
-                }
             }
 
-            public CriCpkSection(Stream destination, string signature)
+            public CriCpkSection(Stream destination, string signature, bool enableMask)
             {
                 this.destination = destination;
                 headerPosition = destination.Position;
@@ -618,7 +646,7 @@ namespace SonicAudioLib.Archive
                 EndianStream.WriteUInt32(destination, byte.MaxValue);
                 destination.Seek(8, SeekOrigin.Current);
 
-                writer = CriTableWriter.Create(destination, new CriTableWriterSettings() { LeaveOpen = true });
+                writer = CriTableWriter.Create(destination, new CriTableWriterSettings() { LeaveOpen = true, EnableMask = enableMask });
             }
         }
     }

Source/SonicAudioLib/CriMw/CriTableReader.cs

@@ -78,7 +78,27 @@ namespace SonicAudioLib.CriMw
 
             if (EndianStream.ReadCString(source, 4) != CriTableHeader.Signature)
             {
-                throw new Exception("No @UTF signature found.");
+                // try to decrypt (currently only for CPK files since those are the only examples I have)
+                source.Seek(-4, SeekOrigin.Current);
+
+                MemoryStream unmaskedSource = new MemoryStream();
+                Methods.MaskCriTable(source, unmaskedSource, source.Length);
+
+                // try again
+                unmaskedSource.Seek(0, SeekOrigin.Begin);
+
+                if (EndianStream.ReadCString(unmaskedSource, 4) != CriTableHeader.Signature)
+                {
+                    throw new Exception("No @UTF signature found.");
+                }
+
+                // Close the old stream
+                if (!leaveOpen)
+                {
+                    source.Close();
+                }
+
+                source = unmaskedSource;
             }
 
             header.Length = ReadUInt32() + 0x8;

Source/SonicAudioLib/CriMw/CriTableWriter.cs

@@ -126,6 +126,13 @@ namespace SonicAudioLib.CriMw
             WriteUInt16(header.NumberOfFields);
             WriteUInt16(header.RowLength);
             WriteUInt32(header.NumberOfRows);
+
+            if (settings.EnableMask)
+            {
+                destination.Position = headerPosition;
+                Methods.MaskCriTable(destination, header.Length);
+            }
+
             destination.Seek(previousPosition, SeekOrigin.Begin);
         }
 
@@ -608,6 +615,7 @@ namespace SonicAudioLib.CriMw
         private bool leaveOpen = false;
         private Encoding encodingType = Encoding.GetEncoding("shift-jis");
         private bool removeDuplicateStrings = true;
+        private bool enableMask = false;
 
         public uint Align
         {
@@ -679,6 +687,19 @@ namespace SonicAudioLib.CriMw
             }
         }
 
+        public bool EnableMask
+        {
+            get
+            {
+                return enableMask;
+            }
+
+            set
+            {
+                enableMask = value;
+            }
+        }
+
         public static CriTableWriterSettings AdxSettings
         {
             get

Source/SonicAudioLib/IO/Methods.cs

@@ -2,7 +2,7 @@
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
-using System.Threading.Tasks;
+using System.IO;
 
 namespace SonicAudioLib.IO
 {
@@ -17,5 +17,38 @@ namespace SonicAudioLib.IO
 
             return value;
         }
+
+        // This one masks the source to destination
+        public static void MaskCriTable(Stream source, Stream destination, long length)
+        {
+            uint currentXor = 25951;
+            long currentPosition = source.Position;
+
+            while (source.Position < currentPosition + length)
+            {
+                byte maskedByte = (byte)(EndianStream.ReadByte(source) ^ currentXor);
+                currentXor *= 16661;
+
+                EndianStream.WriteByte(destination, maskedByte);
+            }
+        }
+
+        // This one masks the source to itself
+        public static void MaskCriTable(Stream source, long length)
+        {
+            if (source.CanRead && source.CanWrite)
+            {
+                uint currentXor = 25951;
+                long currentPosition = source.Position;
+
+                while (source.Position < currentPosition + length)
+                {
+                    byte maskedByte = (byte)(EndianStream.ReadByte(source) ^ currentXor);
+                    currentXor *= 16661;
+
+                    EndianStream.WriteByteAt(source, maskedByte, source.Position - 1);
+                }
+            }
+        }
     }
 }

Source/SonicAudioLib/Properties/AssemblyInfo.cs

@@ -32,4 +32,4 @@ using System.Runtime.InteropServices;
 // You can specify all the values or you can default the Build and Revision Numbers 
 // by using the '*' as shown below:
 // [assembly: AssemblyVersion("1.0.*")]
-[assembly: AssemblyVersion("0.1.*")]
+[assembly: AssemblyVersion("0.3.*")]