Cool_Tools/vgmstream
1
0
Fork 0
mirror of https://github.com/vgmstream/vgmstream.git synced 2024-11-12 01:30:49 +01:00
Code Issues Packages Projects Releases Wiki Activity

docs: add build info for libs and tweaks

Browse Source
This commit is contained in:
bnnm 2023-01-15 21:11:08 +01:00
parent 35fcbf2201
commit 6bcdec9c48
3 changed files with 716 additions and 127 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

634
doc/BUILD-LIB.md Normal file
View File

@ -0,0 +1,634 @@
# vgmstream lib build help
This document explains how to build various external dependencies used in vgmstream. Like *vgmstream*, most external libs use C and need to be compiled as such.
See [BUILD](BUILD.md#external-libraries) for a description of each lib first.
## Intro
Guide is mainly geared towards **Windows** DLLs, as a reference for later updates. For **Linux/Mac**, libs are already included when using *CMake*, but you can mostly follow this with minor tweaks (like using default install folders) to create linkable `.so` libs, should you need to.
Unless mentioned, their latest version should be ok to use, though included DLLs may be a bit older. Each lib is compiled using a *recommended version*, but most should work with recent versions (FFMpeg may rarely change the *API* though). Most libs don't provide official pre-compiled *binaries*, or only for certain versions, so we need to compile them ourselves.
### Requirements
Guide assumes you followed the steps above to install Git, GCC (MingW or MSYS2) or Visual Studio, but you'll need GCC most of the time. This guide uses the command line to describe repeatable steps, though for MSVC DLLs you may open `.sln` and compile manually. Both MSVC's and GCC's DLLs should work fine and with comparable performance.
MSVC commands use `msbuild.exe`, which can be called by opening VS's *x86 Native Tools for VS 20xx* console, found on Windows start menu. It can also be added to regular `PATH` for CMD/MSYS2 as well, but since location changes around you may need to download *vswhere* (locator) to find it:
```bat
curl --location https://github.com/Microsoft/vswhere/releases/download/2.6.7/vswhere.exe --output vswhere.exe
REM #path to MSBuild
vswhere.exe -latest -products * -requires Microsoft.Component.MSBuild -find MSBuild\**\Bin\MSBuild.exe
REM set PATH=%PATH%;C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin
```
On Windows 10, `curl` is included by default but you may need to get it first (or just manually download and unzip files).
### Tools and PATHs
When using Mingw+Git you may add their location to Windows' `PATH` variable, so programs like `gcc.exe` work as-is in Windows CMD without writting the full path. This can be done temporarily by writting in the command line: `set PATH=C:\(path-to-mingw)\mingw32\bin;C:\Git\usr\bin;%PATH%` (for example), or in Windows' system variables panel.
For MSYS2 commands should work (after installing relevant programs) by opening the `msys2/mingw32.exe` console.
**Important** though is that some libs (like mpg123) only work properly when GCC/MSYS2 paths goes *before* Windows default path, since some program names conflict otherwise.
### standalone MinGW tweaks
MinGW has `mingw32-make.exe` but most scripts expect plain `make`. Generally you can pass a `MAKE=mingw32-make` variable with the full name, but may be easier to clone/rename it to `make.exe`.
Also for autotool-based buils, it's best to put files (and dev tools) in folders paths *without spaces* as some scripts still get tripped by that. For the sake of the steps below you for (for example) run commands in `C:\vgmstream-dlls\sources`.
### autotools
Several libs use *autotools* (a collection of scripts that guess system's config), that typically works by doing:
```sh
# creates a custom Makefile(s) based on current system
sh ./configure [params]
# default, compiles code using generated Makefile(s) (resulting .dll/files in some internal folder)
make [params]
# copies resulting files to some pre-defined dir, and compiles if plain make wasn't called first
# optionally, "make install-strip" does the same, but "strips" binaries (good)
make install
make install-strip
# cleans compiled code (important when changing ./configure options)
make clean
```
While mainly geared towards *Linux* using GCC-like compilers, works on *Windows* as long as typical *Linux* utils and compilers can be found `PATH`. Not all *configure* are created equal so there is some fiddling around depending on lib.
Usually, *autoconf* DLLs are generated with *debug symbols* by default. This can be fixed by calling's GCC `strip (dll)`, generally done automatically or when using `make install-strip`.
On Linux `install` is used to actually *install* libs on system dirs (so `--prefix` is rarely used), while on Windows is just to copy DLLs, `.h` (used in C code), linker libs (not needed) and other stuff to final dir.
You can call multiple *targets* in a single line `make clean install-strip` is the same as `make clean` and `make install` (which in turn calls plain `make` / default). That's the theory, but at some libs don't properly handle this.
### autotools config
*autotools* are **very** fragile and picky, beware when changing stuff.Check other flags by calling `sh ./configure --help`, but changing some of the steps will likely cause odd issues, *autotools are not consistent between libs*.
Common *configure*/Makefile params:
- `--build=...`: current compilation environment (autodetected, but may fail in outdated libs)
- `--host=...`: target system (same as build by default), can be forced for cross-compilation.
- `--target=...`: target binary (same as host by default). Not really needed.
- how much `build/host/target` matter depends on lib, included always by default.
- `--target-os/--arch/--etc`: depending on script, to ease config
- `--disable-static --enable-shared`: only needed config as appropriate (varies)
- `--prefix=/c/vgmstream-dlls/(varies)/`: *make install* dir (where libs are copied), to simplify we'll use a fixed one
- defaults to `/c/Git/usr/local` (standalone) or MSYS2's `/usr/local` folders if not set
- if you don't call *make install* DLLs will be still there, inside `.libs` subdirs usually (will need to `strip`)
- most projects (except FFmpeg only?) need a full path (`/path/...`) and won't accept a relative one (`./subdir/...`)
- `MAKE=mingw32-make`: may be passed for mingw32 to work properly (otherwise rename `mingw32-make.exe` to `make.exe`)
- `CFLAGS` / `AM_CFLAGS` / `LDFLAGS` / `AM_LDFLAGS`: extra compiler/linker flags
Compiler/linker flags are very important yet have big gotchas:
- pass `-m32`/`-64` to the compiler for 32/64-bit output
- this may be autodetected and set in some environments
- pass `-static-libgcc` to the linker to remove Mingw-w64 DLL dependencies (not needed in 64-bit DLLs?)
- `CFLAGS` / `LDFLAGS` on *configure* will usually (**not always**) *add* to the default ones
- `CFLAGS` on `make` will usually *overwrite* default ones (such as `-O2` optimizations)
- `AM_CFLAGS` on `make` should work together with `CFLAGS`, but actually aren't always passed to all `.c`
- `LDFLAGS` on `make` may *overwrite* default ones, but often aren't set
- `AM_LDFLAGS` on `make` should work together with `LDFLAGS`, but some libs don't support them, or *libtool* (hellspawn script that internally generates final `.dll`) sometimes only reads `LDFLAGS`
However, *those flags aren't consistent between libs*, meaning in one using *configure* + `CFLAGS` adds to existing CFLAGS, other overwrites them. So, scripts below may look inconsistent, but they certain flags for a reason.
### Xiph's releases and exports
Sometimes we use "official releases" sources rather than using Git's sources. Both should be the same, but releases have pre-generated *./configure*, while Git needs to call `autogen.sh` that calls `autoreconf` that generates a base `configure` script. Since getting `autoreconf` working on **Windows** without MSYS2 requires extra steps (not described), Xiph's releases are recommended.
When building a DLL/lib compiler sets *exported symbols* (functions). Xiph's *autoconf* may generate DLLs correctly, butdon't detect Mingw/Win config properly and export all symbols by default. This is fixed manually, but there may be better ways to handle it (to be researched).
### Shared libs details
Roughly, a `.dll` is a Windows "shared library"; Linux equivalent would be a `.so` file. First, `.c` files are compiled into objects (`.o` in GCC, `.obj` in MSCV), then can be made into a `.dll`. Later, when a program needs that DLL (or rather, it's functions), a compiler can use it as long as some conditions are met.
DLL must *export symbols* (functions), which on a Windows's DLL is done with:
- adding `__declspec(dllexport)` to a function (usually done with `#define EXPORT ...` and similar ways)
- using a `.def` module definition file
- if neither of the above is used, GCC exports every function by default (not great=
Then, to *link* (refer to) a DLL compiler usually needs helper files (`.dll.a` in GCC, `.lib` in MSVC). DLL's are copied to vgmstream's source, while helper files are created on compile time from `.dll`+`.def` (see *ext_libs/Makefile* for GCC and `ext_libs.vcxproj` for MSVC).
DLLs also *links* to standard C lib (MingW: `msvcrt.dll`, MSVC: `msvcrt(version).dll`). On Windows there are multiple versions of this *runtime*, but DLLs may include (part of) it with certain compiler/linker flags. This means there are subtle differences between compiler's generated DLLs, but for libs (that only do limited stuff) they don't matter much.
### Static libs details
*vgmstream* uses external DLLs to support extra codecs, but it's clunkier and less user-friendly needing a bunch of extra DLLs around. Ideally *vgmstream* could use *static libs* instead (eliminating the need of DLLs), but it's complex and not done at the moment.
To make static libs, all objects (`.o`/`.obj`) are integrated to an archive (`.a` in GCC, `.lib` in MSVC) then this can use used by compiler. However, unlike DLLs, mixing static libs from one compiler with another is harder due to compiler dependencies that aren't a problem with DLLs. For example, by default Mingw's static libs may depend on `libmingwex.a` and would need that lib if used with MSVC.
One could have static libs for each compiler, but not all projects can be compiled with MSVC or GCC, also being a lot of extra work. Incidentally, C++ DLLs/libs can't be easily shared between MSVC and GCC (unless carefully prepared to be so), unlike plain C libs that are mostly compatible.
### 32 and 64-bit
Maybe obvious, but programs and DLLs can be compiled as 32-bit or 64-bit, but you can't mix 64-bit programs and 32-bit DLLs (on **Windows** mixing DLLs will usually result on error `0xc00007b`). Compilers, being programs, can be 32 or 64-bit as well.
Both 32/64-bit GCC can compile 32-bit or 64-bit without issues, using the `-m32`/`-m64` flags (because GCC is able to create win32/win64/arm32/arm64/etc executables regardless of itself being 32/64-bit), while MSVC probably does as well, but just in case install the 64-bit version that handles both and decides what to generate based on info from `.vcxproj` files.
## Compiling external libs
### mpg123
Must use *autotools*, though some scripts may simplify the process (`makedll.sh`, `windows-builds.sh x86/x86-64`).
#### Source
```bat
curl --location https://sourceforge.net/projects/mpg123/files/mpg123/1.31.1/mpg123-1.31.1.tar.bz2 --output mpg123-1.31.1.tar.bz2
tar -xf mpg123-1.31.1.tar.bz2
cd mpg123-1.31.1
```
#### With GCC
*Notes*:
- if `make` ends with a libtool message of "syntax error near token", make sure GCC/MSYS2/Linux utils go *first* in `PATH` (`set PATH=C:\(...);%PATH%`).
- `make install-strip` throws an error and fails to copy `.h` but seems to properly strip DLLs (regular `install` is used to copy `.h`).
```bat
REM # 32-bit DLL
sh ./configure --host=mingw32 --disable-static --enable-shared --with-cpu=x86 --prefix=/c/vgmstream-dlls/out/mpg123-32 CFLAGS="-m32" LDFLAGS="-static-libgcc"
make clean install install-strip
REM # 64-bit DLL
sh ./configure --host=mingw64 --disable-static --enable-shared --with-cpu=x86-64 --prefix=/c/vgmstream-dlls/out/mpg123-64 CFLAGS="-m64" LDFLAGS="-static-libgcc"
make clean install install-strip
```
#### With MSVC
Untested/not possible.
### libg719_decode
Use MSVC and `g719.sln`, or GCC and the Makefile included.
#### Source
```bat
git clone https://github.com/kode54/libg719_decode
git -C libg719_decode checkout da90ad8a676876c6c47889bcea6a753f9bbf7a73
cd libg719_decode
```
#### With GCC
```bat
REM # 32-bit DLL
mkdir C:\vgmstream-dlls\out\g719-32\
make clean shared EXTRA_CFLAGS="-m32" EXTRA_LDFLAGS="-m32 -static-libgcc" OUTPUT_DIR=C:\vgmstream-dlls\out\g719-32\
REM # 64-bit DLL
mkdir C:\vgmstream-dlls\out\g719-64\
make clean shared EXTRA_CFLAGS="-m64" EXTRA_LDFLAGS="-m64 -static-libgcc" OUTPUT_DIR=C:\vgmstream-dlls\out\g719-64\
```
#### With MSVC
```bat
REM # 32-bit DLL
MSBuild.exe g719.sln /p:Platform=Win32 /p:Configuration=Release /p:WindowsTargetPlatformVersion=10.0 /p:PlatformToolset=v142
mkdir C:\vgmstream-dlls\out\g719-32
copy /B .\Release\libg719_decode.dll C:\vgmstream-dlls\out\g719-32\libg719_decode.dll
REM # 64-bit DLL
MSBuild.exe g719.sln /p:Platform=x64 /p:Configuration=Release /p:WindowsTargetPlatformVersion=10.0 /p:PlatformToolset=v142
mkdir C:\vgmstream-dlls\out\g719-64
copy /B .\x64\Release\libg719_decode.dll C:\vgmstream-dlls\out\g719-64\libg719_decode.dll
REM add /t:Clean to the above to clean up compilation
```
### LibAtrac9
Use MSCV and `libatrac9.sln`, or GCC and the Makefile included.
#### Source
```bat
git clone https://github.com/Thealexbarney/LibAtrac9
git -C LibAtrac9 checkout 6a9e00f6c7abd74d037fd210b6670d3cdb313049
cd LibAtrac9/C
```
#### With GCC
**NOTE**: on Windows `mkdir` clashes and needs full path
```bat
REM # 32-bit DLL
make clean shared SFLAGS="-O2 -m32" MKDIR="/Git/usr/bin/mkdir -p" BINDIR=C:\vgmstream-dlls\out\atrac9-32 SHARED_NAME=C:\vgmstream-dlls\out\atrac9-32\libatrac9.dll
REM make clean shared SFLAGS="-O2 -m32" MKDIR="/Git/usr/bin/mkdir -p" BINDIR=C:\vgmstream-dlls\out\atrac9-32 SHARED_FILENAME=libatrac9.dll
REM # 64-bit DLL
make clean shared SFLAGS="-O2 -m64" MKDIR="/Git/usr/bin/mkdir -p" BINDIR=C:\vgmstream-dlls\out\atrac9-64 SHARED_NAME=C:\vgmstream-dlls\out\atrac9-64\libatrac9.dll
```
#### With MSVC
```bat
REM # 32-bit DLL
MSBuild.exe libatrac9.sln /p:Platform=x86 /p:Configuration=Release /p:WindowsTargetPlatformVersion=10.0 /p:PlatformToolset=v142
mkdir C:\vgmstream-dlls\out\atrac9-32
copy /B .\Release\libatrac9.dll C:\vgmstream-dlls\out\atrac9-32\libatrac9.dll
REM # 64-bit DLL
MSBuild.exe libatrac9.sln /p:Platform=x64 /p:Configuration=Release /p:WindowsTargetPlatformVersion=10.0 /p:PlatformToolset=v142
mkdir C:\vgmstream-dlls\out\atrac9-64
copy /B .\x64\Release\libatrac9.dll C:\vgmstream-dlls\out\atrac9-64\libatrac9.dll
REM add /t:Clean to the above to clean up compilation
```
**NOTE**
Some `libatrac9.vcxproj` x64 config may be outdated. In MSBuild +15 (VS +2017) you can force changes by creating a file named `Directory.Build.props` nearby. Also possible to pass this with /p:ForceImportBeforeCppTargets=(file.prop), but only works with full paths. There is no command line support to change CL (MSVC's compile) options other than this.
```
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemDefinitionGroup>
<ClCompile>
<ExceptionHandling>Sync</ExceptionHandling>
<RuntimeLibrary>MultiThreaded</RuntimeLibrary>
<FloatingPointModel>Fast</FloatingPointModel>
</ClCompile>
</ItemDefinitionGroup>
</Project>
```
### libvorbis
Should be buildable with *autotools* (Git releases need to use `autogen.sh` first) or MSVC (projects in `./win32/`, may not be up to date). *CMake* may work as well.
Methods below create 3 DLL: `libogg.dll`, `libvorbis.dll` and `libvorbisfile.dll` (also `libvorbisenc.dll`, unneeded), plus static libs (`.a`). However Vorbis/Ogg DLL support in vgmstream was originally added using a combined DLL from *RareWares* (https://www.rarewares.org/ogg-libraries.php) simply called `libvorbis.dll`, so separate DLLs can't be used at the moment and we'll need to fix that.
**TODO**: should restrict exported symbols (unsure how libvorbis does it, here it's manually done on last step)
#### Sources
``` bat
curl --location http://downloads.xiph.org/releases/ogg/libogg-1.3.5.zip --output libogg-1.3.5.zip
tar -xf libogg-1.3.5.zip
cd libogg-1.3.5
```
```
curl --location http://downloads.xiph.org/releases/vorbis/libvorbis-1.3.7.zip --output libvorbis-1.3.7.zip
tar -xf libvorbis-1.3.7.zip
cd libvorbis-1.3.7
```
#### With GCC
*Notes*:
- `-ffast-math` isn't enabled by default but seems safe and commonly done elsewhere
**libogg**
```bat
REM # 32-bit DLL + static lib
sh ./configure --build=mingw32 --enable-static --enable-shared --prefix=/c/vgmstream-dlls/out/ogg-32 CFLAGS="-m32" LDFLAGS="-static-libgcc"
make clean install-strip
REM # 64-bit DLL + static lib
sh ./configure --build=mingw64 --enable-static --enable-shared --prefix=/c/vgmstream-dlls/out/ogg-64 CFLAGS="-m64" LDFLAGS="-static-libgcc"
make clean install-strip
```
**libvorbis/libvorbisfile**
```bat
REM # 32-bit DLL + static lib
sh ./configure --host=mingw32 --enable-static --enable-shared --disable-docs --prefix=/c/vgmstream-dlls/out/vorbis-32 --with-ogg=/c/vgmstream-dlls/out/ogg-32 CFLAGS="-m32 -ffast-math" LDFLAGS="-static-libgcc"
make clean install-strip
sh ./configure --host=mingw64 --enable-static --enable-shared --disable-docs --prefix=/c/vgmstream-dlls/out/vorbis-64 --with-ogg=/c/vgmstream-dlls/out/ogg-64 CFLAGS="-m64 -ffast-math" LDFLAGS="-static-libgcc"
make clean install-strip
```
**libvorbix mix**
There no official (autoconf) way to bundle all 3 libs above into one DLL, as we need, but we can do it manually (ugly but what can you do).
Commands, based on how generated makefile creates dlls (-Wl are 'linker' commands):
- `-shared`: create a .so/.dll
- `(file).def`: limits exported DLL symbols/functions (no dllexports so gcc can't tell otherwise)
- `-s`: strips DLL (removes symbols)
- `-o (name)`: output
- `-Wl,--whole-archive (multiple .a) -Wl,--no-whole-archive`: adds all `.o` inside those `.a` (archives)
- otherwise could use `ar xo (archive.s)` to unzip and manually pass all `.o` inside to gcc
- `-Wl,--enable-auto-image-base`: for DLLs, improves loading?
- `-Wl,--out-implib=(name).dll.a`: creates a helper link .lib, though we don't really need it
- `-Wl,--output-def=(name).def`: can be included to create a final export file (to compare)
This needs `libvorbis.def` from *vgmstream/ext_libs*, and doesn't include `libvorbisenc` code.
```bat
cd C:\vgmstream-dlls\out
REM # 32-bit DLL
gcc -m32 -shared -s libvorbis.def -o vorbis-32/libvorbis.dll -Wl,--whole-archive ogg-32/lib/libogg.a vorbis-32/lib/libvorbisfile.a vorbis-32/lib/libvorbis.a -Wl,--no-whole-archive -Wl,--enable-auto-image-base
REM # 64-bit DLL
gcc -m64 -shared -s libvorbis.def -o vorbis-64/libvorbis.dll -Wl,--whole-archive ogg-64/lib/libogg.a vorbis-64/lib/libvorbisfile.a vorbis-64/lib/libvorbis.a -Wl,--no-whole-archive -Wl,--enable-auto-image-base
```
#### With MSVC/CMake
Untested.
### libcelt
FSB uses two incompatible, older libcelt versions. Both libraries export the same symbols so normally can't coexist together. To get them working we need to make sure symbols are renamed first. This may be solved in various ways:
- using dynamic loading (LoadLibrary) but for portability it isn't an option
- may be possible to link+rename using .def files, but those are mainly used on **Windows**
- Linux/Mingw's objcopy to (supposedly) rename DLL symbols
- Use GCC's preprocessor to rename functions on compile
- Directly rename functions in source code
We'll use autotools with GCC preprocessor renaming. On **Windows** steps are described below; on **Linux** you can use CMake that patch celt libs automatically (or follow the steps removing Windows-only config).
#### Source
**celt-0.6.1**
```bat
curl --location http://downloads.us.xiph.org/releases/celt/celt-0.6.1.tar.gz --output celt-0.6.1.tar.gz
tar -xf celt-0.6.1.tar.gz
cd celt-0.6.1
```
**celt-0.11.0**
```bat
curl --location http://downloads.us.xiph.org/releases/celt/celt-0.11.0.tar.gz --output celt-0.11.0.tar.gz
tar -xf celt-0.11.0.tar.gz
cd celt-0.11.0
```
#### With GCC
*Notes*:
- on **Windows** exports need to be fixed (undefines `CELT_BUILD` and defines `WIN32` to allow `dllexport`)
- uses `sed` (Linux text replacer) to change `#define`
- despite the name, `CELT_BUILD` seems to be used only to detect shared builds
- on **Windows** *celt-0.6.1*'s *libtool* somehow removes `-static-libgcc` from `LDFLAGS`, meaning DLLs depend on Mingw's libs
- partially fixed using `-Wc,-static-libgcc`, but will cause various errors
- uses `sed` (Linux text replacer) to insert `-static-libgcc` into *libtool* calls
- detects build environment incorrectly (outdated scripts)
- uses `mingw32` even on 64-bit due to older *configure*
- `CFLAGS` on *configure* overwrite defaults, also no `AM_LDFLAGS`
- in theory passing `--enable-custom-modes` in *configure* would be equivalent to `-DCUSTOM_MODES=1` in 0.11.0 but doesn't seem to work?
- `SUBDIRS="libcelt" DIST_SUBDIRS="libcelt"` forces Makefile to compile libcelt only and ignore tests (no other way to disable)
- otherwise (buggy?) *configure* may detect *libogg* + compiles extra stuff (not an issue on Windows but for completeness)
- `--disable-oggtest`: oggtest are test utils, not related to *libogg* detection
- when compiling GCC may complain about missing `ec_log`, but it seems correctly defined in `entcode.c`and included in the lib and not a cause of issues.
- `-no-undefined` is necessary for this or the DLL won't be created
- uses preprocessor renaming (encoder functions aren't needed but for completion)
**TODO**: would be better to rename DLL output (since it's part of the DLL and makes .def simpler) but who knows what exact command is used
**celt-0.6.1**
```bat
REM # 32-bit DLL
sh ./configure --build=mingw32 --disable-static --enable-shared --disable-oggtest --prefix=/c/vgmstream-dlls/out/celt0061-32
sed -i -e "s/#define CELT_BUILD.*/#undef CELT_BUILD/g" config.h
sed -i -e "s/compiler_flags -o/compiler_flags -static-libgcc -o/g" libtool
make clean
make SUBDIRS="libcelt" DIST_SUBDIRS="libcelt" LDFLAGS="-m32 -static-libgcc -no-undefined" AM_CFLAGS="-m32 -DWIN32 -Dcelt_decode=celt_decode_0061 -Dcelt_decoder_create=celt_decoder_create_0061 -Dcelt_decoder_destroy=celt_decoder_destroy_0061 -Dcelt_mode_create=celt_mode_create_0061 -Dcelt_mode_destroy=celt_mode_destroy_0061 -Dcelt_mode_info=celt_mode_info_0061 -Dcelt_decode_float=celt_decode_float_0061 -Dcelt_decoder_ctl=celt_decoder_ctl_0061 -Dcelt_encode=celt_encode_0061 -Dcelt_encode_float=celt_encode_float_0061 -Dcelt_encoder_create=celt_encoder_create_0061 -Dcelt_encoder_ctl=celt_encoder_ctl_0061 -Dcelt_encoder_destroy=celt_encoder_destroy_0061 -Dcelt_header_from_packet=celt_header_from_packet_0061 -Dcelt_header_init=celt_header_init_0061 -Dcelt_header_to_packet=celt_header_to_packet_0061"
make install-strip SUBDIRS="libcelt" DIST_SUBDIRS="libcelt"
REM # 64-bit DLL
sh ./configure --build=mingw32 --disable-static --enable-shared --disable-oggtest --prefix=/c/vgmstream-dlls/out/celt0061-64
sed -i -e "s/#define CELT_BUILD.*/#undef CELT_BUILD/g" config.h
sed -i -e "s/compiler_flags -o/compiler_flags -static-libgcc -o/g" libtool
make clean
make SUBDIRS="libcelt" DIST_SUBDIRS="libcelt" LDFLAGS="-m64 -static-libgcc -no-undefined" AM_CFLAGS="-m64 -DWIN32 -Dcelt_decode=celt_decode_0061 -Dcelt_decoder_create=celt_decoder_create_0061 -Dcelt_decoder_destroy=celt_decoder_destroy_0061 -Dcelt_mode_create=celt_mode_create_0061 -Dcelt_mode_destroy=celt_mode_destroy_0061 -Dcelt_mode_info=celt_mode_info_0061 -Dcelt_decode_float=celt_decode_float_0061 -Dcelt_decoder_ctl=celt_decoder_ctl_0061 -Dcelt_encode=celt_encode_0061 -Dcelt_encode_float=celt_encode_float_0061 -Dcelt_encoder_create=celt_encoder_create_0061 -Dcelt_encoder_ctl=celt_encoder_ctl_0061 -Dcelt_encoder_destroy=celt_encoder_destroy_0061 -Dcelt_header_from_packet=celt_header_from_packet_0061 -Dcelt_header_init=celt_header_init_0061 -Dcelt_header_to_packet=celt_header_to_packet_0061"
make install-strip SUBDIRS="libcelt" DIST_SUBDIRS="libcelt"
```
**celt-0.11.0**
```bat
REM # 32-bit DLL
sh ./configure --build=mingw32 --disable-static --enable-shared --disable-oggtest --prefix=/c/vgmstream-dlls/out/celt0110-32
sed -i -e "s/#define CELT_BUILD.*/#undef CELT_BUILD/g" config.h
make clean
make SUBDIRS="libcelt" DIST_SUBDIRS="libcelt" LDFLAGS="-m32 -static-libgcc -no-undefined" AM_CFLAGS="-m32 -DWIN32 -DCUSTOM_MODES=1 -Dcelt_decode=celt_decode_0110 -Dcelt_decoder_create_custom=celt_decoder_create_custom_0110 -Dcelt_decoder_destroy=celt_decoder_destroy_0110 -Dcelt_mode_create=celt_mode_create_0110 -Dcelt_mode_destroy=celt_mode_destroy_0110 -Dcelt_mode_info=celt_mode_info_0110 -Dcelt_decode_float=celt_decode_float_0110 -Dcelt_decoder_create=celt_decoder_create_0110 -Dcelt_decoder_ctl=celt_decoder_ctl_0110 -Dcelt_decoder_get_size=celt_decoder_get_size_0110 -Dcelt_decoder_get_size_custom=celt_decoder_get_size_custom_0110 -Dcelt_decoder_init=celt_decoder_init_0110 -Dcelt_decoder_init_custom=celt_decoder_init_custom_0110 -Dcelt_encode=celt_encode_0110 -Dcelt_encode_float=celt_encode_float_0110 -Dcelt_encoder_create=celt_encoder_create_0110 -Dcelt_encoder_create_custom=celt_encoder_create_custom_0110 -Dcelt_encoder_ctl=celt_encoder_ctl_0110 -Dcelt_encoder_destroy=celt_encoder_destroy_0110 -Dcelt_encoder_get_size=celt_encoder_get_size_0110 -Dcelt_encoder_get_size_custom=celt_encoder_get_size_custom_0110 -Dcelt_encoder_init=celt_encoder_init_0110 -Dcelt_encoder_init_custom=celt_encoder_init_custom_0110 -Dcelt_header_from_packet=celt_header_from_packet_0110 -Dcelt_header_init=celt_header_init_0110 -Dcelt_header_to_packet=celt_header_to_packet_0110 -Dcelt_strerror=celt_strerror_0110"
make install-strip SUBDIRS="libcelt" DIST_SUBDIRS="libcelt"
REM # 64-bit DLL
sh ./configure --build=mingw32 --disable-static --enable-shared --disable-oggtest --prefix=/c/vgmstream-dlls/out/celt0110-64
sed -i -e "s/#define CELT_BUILD.*/#undef CELT_BUILD/g" config.h
make clean
make SUBDIRS="libcelt" DIST_SUBDIRS="libcelt" LDFLAGS="-m64 -static-libgcc -no-undefined" AM_CFLAGS="-m64 -DWIN32 -DCUSTOM_MODES=1 -Dcelt_decode=celt_decode_0110 -Dcelt_decoder_create_custom=celt_decoder_create_custom_0110 -Dcelt_decoder_destroy=celt_decoder_destroy_0110 -Dcelt_mode_create=celt_mode_create_0110 -Dcelt_mode_destroy=celt_mode_destroy_0110 -Dcelt_mode_info=celt_mode_info_0110 -Dcelt_decode_float=celt_decode_float_0110 -Dcelt_decoder_create=celt_decoder_create_0110 -Dcelt_decoder_ctl=celt_decoder_ctl_0110 -Dcelt_decoder_get_size=celt_decoder_get_size_0110 -Dcelt_decoder_get_size_custom=celt_decoder_get_size_custom_0110 -Dcelt_decoder_init=celt_decoder_init_0110 -Dcelt_decoder_init_custom=celt_decoder_init_custom_0110 -Dcelt_encode=celt_encode_0110 -Dcelt_encode_float=celt_encode_float_0110 -Dcelt_encoder_create=celt_encoder_create_0110 -Dcelt_encoder_create_custom=celt_encoder_create_custom_0110 -Dcelt_encoder_ctl=celt_encoder_ctl_0110 -Dcelt_encoder_destroy=celt_encoder_destroy_0110 -Dcelt_encoder_get_size=celt_encoder_get_size_0110 -Dcelt_encoder_get_size_custom=celt_encoder_get_size_custom_0110 -Dcelt_encoder_init=celt_encoder_init_0110 -Dcelt_encoder_init_custom=celt_encoder_init_custom_0110 -Dcelt_header_from_packet=celt_header_from_packet_0110 -Dcelt_header_init=celt_header_init_0110 -Dcelt_header_to_packet=celt_header_to_packet_0110 -Dcelt_strerror=celt_strerror_0110"
make install-strip
```
Resulting DLLs need to be renamed to `libcelt-0061.dll` and `libcelt-0110.dll`, and may need to create a `.def` file with `gendef (name).dll` (in theory this is done by passing `-Wl,--output-def=libcelt.def` to `AM_FLAGS` but seems to fail). vgmstream also needs `celt.h`, `celt_types.h`, `celt_header.h` with renamed functions, but a custom `.h` with minimal symbols is already included in source.
#### With MSVC
Untested/not possible.
### libspeex
Should be buildable with *autotools* (Git releases need to use `autogen.sh` first) or MSVC (projects in `./win32/`, may not be up to date).
#### Source
```bat
curl --location http://downloads.us.xiph.org/releases/speex/speex-1.2.1.tar.gz --output speex-1.2.1.tar.gz
tar -xf speex-1.2.1.tar.gz
cd speex-1.2.1
```
#### With GCC
*Notes*:
- on **Windows** exports need to be fixed (swaps Linux exports with Windows' `dllexport`)
- `CFLAGS` on *configure* overwrite defaults
```bat
REM # 32-bit DLL
sh ./configure --host=mingw32 --disable-static --enable-shared --prefix=/c/vgmstream-dlls/out/speex-32
sed -i -e "s/#define EXPORT .*/#define EXPORT __declspec(dllexport)/g" config.h
make clean install-strip LDFLAGS="-m32 -static-libgcc" AM_CFLAGS="-m32"
REM # 64-bit DLL
sh ./configure --host=mingw64 --disable-static --enable-shared --prefix=/c/vgmstream-dlls/out/speex-64
sed -i -e "s/#define EXPORT .*/#define EXPORT __declspec(dllexport)/g" config.h
make clean install-strip LDFLAGS="-m64 -static-libgcc" AM_CFLAGS="-m64"
```
#### With MSVC
Untested/outdated.
### libopus
This is used below by FFmpeg (but can be disabled), as a static lib (`.a`/`.lib`) rather than DLL.
If you wonder why use it through FFmpeg instead of directly, all work was already done for FFmpeg's opus so it was faster easier this way.
#### Source
```bat
curl --location https://archive.mozilla.org/pub/opus/opus-1.3.1.tar.gz --output opus-1.3.1.tar.gz
tar -xf opus-1.3.1.tar.gz
cd opus-1.3.1
```
#### With GCC
**Notes**
- `CFLAGS` on *configure* overwrite defaults
- remove `--prefix` for pkg-config to work properly?
```bat
REM # 32-bit lib
sh ./configure --host=mingw32 --enable-static --disable-shared --disable-doc --disable-extra-programs --prefix=/c/vgmstream-dlls/out/opus-32
make clean install-strip LDFLAGS="-m32 -static-libgcc" AM_CFLAGS="-m32"
REM # 64-bit lib
sh ./configure --host=mingw32 --enable-static --disable-shared --disable-doc --disable-extra-programs --prefix=/c/vgmstream-dlls/out/opus-64
make clean install-strip LDFLAGS="-m64 -static-libgcc" AM_CFLAGS="-m64"
```
#### With MSVC
**Notes**:
- if you have called *configure* first, delete `config.h` on root or you may get odd errors
```bat
del config.h
cd win32/VS2015
REM # 32-bit lib
MSBuild.exe opus.sln /p:Platform=Win32 /p:Configuration=Release /p:WindowsTargetPlatformVersion=10.0 /p:PlatformToolset=v142
REM # 64-bit lib
MSBuild.exe opus.sln /p:Platform=x64 /p:Configuration=Release /p:WindowsTargetPlatformVersion=10.0 /p:PlatformToolset=v142
```
### FFmpeg
vgmstream's FFmpeg builds for **Windows** and static builds for **Linux** remove many unnecessary parts of FFmpeg to trim down its gigantic size, and, on Windows, are also built with the "-vgmstream" suffix to prevent clashing with other plugins. Current options can be seen in `ffmpeg_options.txt`. Shared **Linux** builds usually link to system FFmpeg without issues, while standard FFmpeg DLLs may work (renamed to -vgmstream).
FFmpeg can be compiled with *libopus* (external lib) rather than internal *opus*. This is used because FFmpeg decodes incorrectly Opus files used some in games (mostly low bitrate). In older versions this was audibly wrong, but currently the difference shouldn't be that much, but still not that accurate compared with *libopus* (PCM sample diffs of +5000), so *vgmstream* enables it. Getting *libopus* recognized can be unwieldly, so internal *opus* is a decent enough substitute (remove `--enable-libopus` and change `libopus` to `opus` in `--enable-decoder`).
GCC and MSVC need `yasm.exe` somewhere in `PATH` to properly compile/optimize: https://yasm.tortall.net (add `--disable-yasm` to *configure* options to disable, may decrease performance).
FFmpeg uses separates DLLs, that depend on each other like this:
- avutil: none (uses bcrypt only in Win7+, could be be patched out)
- swresample: avutil
- avformat: avcodec, avutil
- avcodec: avutil, swresample
#### Source
```bat
# clone only current tag's "depth" as otherwise FFmpeg history is pretty big
git clone https://git.ffmpeg.org/ffmpeg.git --depth 1 --branch n5.1.2
cd ffmpeg
```
#### libopus and pkg-config
FFmpeg uses *pkg-config* (a kind of "installed lib manager") to detect pre-compiled *libopus*. On Linux it should detect *libopus* after `make install` with default `--prefix`, or adding opus's `--prefix` path to `PKG_CONFIG_PATH`. On Windows, MSYS2 *probably* works the same.
However when compiling with MSVC it's not clear how to mix Windows-style `.lib` and *pkg-config*, though should be possible (*media-autobuild_suite* project does it?). For now as a temp hack, we can force FFmpeg to skip pkg-config and manually pass lib's location. But if you can get *pkg-config* to work ignore "pkg-config hack" steps (**TODO**: to be researched).
#### With GCC
**Notes**:
- *do not* call `make install` directly without `make` first, doesn't properly execute needed dependencies (won't have version numbers)
```sh
# read current options (removing comments and line breaks); change file path if needed (or manually copy options below)
FFMPEG_OPTIONS=`sed -e '/^#/d' ../vgmstream/ext_libs/ffmpeg_options.txt`
echo $FFMPEG_OPTIONS
# PKG-CONFIG HACK: disables pkg-config in FFmpeg's configure (use only if *configure* throws a pkg-config error)
sed -i -e "s/require_pkg_config libopus/: #require_pkg_config libopus/g" configure
# PKG-CONFIG HACK (untested): pass the following to configure
# --extra-cflags="-I(path to opus's /include)" --extra-ldflags="-L(path to opus's /lib) -lopus -static-libgc"
# 32-bit DLL
sh ./configure $FFMPEG_OPTIONS --target-os=mingw32 --arch=x86 --enable-custom-modes --extra-ldflags="-static-libgcc" --prefix=/c/vgmstream-dlls/out/ffmpeg-32
make clean
make
make install
# 64-bit DLL
sh ./configure $FFMPEG_OPTIONS --target-os=mingw32 --arch=x86_64 --enable-custom-modes --extra-ldflags="-static-libgcc" --prefix=/c/vgmstream-dlls/out/ffmpeg-64
make clean
make
make install
```
#### With MSVC
Supported but also needs *autotools* with MSYS2, and several hoops:
- have MSVC's `cl.exe`/`link.exe` compiler/linker in `PATH` (`cl.exe` returns info)
- it's kinda hard to find out so best would be opening VS's *x86 Native Tools for VS 20xx* console, found on Windows start menu
- (libopus only) temp add opus's include and lib paths into MSVC's `INCLUDE` and `LIB` console variables
- `set INCLUDE=%LIB%;C:\(path to include)`
- `set LIB=%LIB%;C:\(path to lib)`
- open MSYS2's ming32 console keeping PATH:
- `C:\msys64\msys2_shell.cmd -mingw32 -use-full-path` (32-bit)
- `C:\msys64\msys2_shell.cmd -mingw64 -use-full-path` (64-bit)
- a new console should open, you can close *x86 Native Tools Command Prompt* now
- check `which cl` is found (shows VS's path)
- check `which link` is MSVC's and not `/usr/bin/link.exe`
- temp rename wrong `link.exe` if needed: `mv /usr/bin/link.exe /usr/bin/link.exe.bak`
- beware as *mingw32* and *ming64* consoles may have different settings/links
- make sure git/gcc/make/autotools/mingw/pkg-config(pkgconf?)/mingw-w64 are installed in MSYS2
- make sure yasm is installed (`pacman -S yasm`)
- (libopus only but kinda optional) install pkg-config
- get *libopus*, compile it with VS (to get a `.lib`) as described before
- get FFmpeg's source and enter it
- call *configure/make/make install* with options described above, changing:
- `--target-os=mingw32` to `--target-os=win32 --toolchain=msvc`
- if you get "*compiler cannot create executables*" errors make sure that:
- you have open the *mingw32* (32-bit) or *mingw64* (64-bit) console
- `which link` is properly set
- *libopus*'s `opus.lib` path ib `LIB` is correctly set
- if you get pkg-config errors, try hack to disable it (see below)
- if you still get errors, try disabling libopus (remove `--enable-libopus` and change `libopus` to `opus` in `--enable-decoder`)
- if you still get errors, try deleting opusffmpeg dirs and *carefully* redo the steps
- missing a single step or changing stuff will likely cause issues!
After a while (+5-10min) you should get DLLs. When compiling 64-bit DLLs, open *x64 Native Tools for VS 20xx* console instead, compile *libopus* 64-bit and set `--target-os=win64`.
Reportedly this helper project works (automates all of the above):
- https://github.com/m-ab-s/media-autobuild_suite
`ffmpeg_options.txt` and `media-autobuild_suite.ini` (outdated?) for it can be found in *vgmstream* source.
In theory adding Git+Mingw+yasm in `PATH` inside the VS console would work (temp include: `set PATH=%PATH%;C:\Git\usr\bin;C:\(mingw-path)\mingw32\bin;C:\yasm`), but seems to have issues with existing `awk`.
Extra info:
- http://ffmpeg.org/platform.html
- https://trac.ffmpeg.org/wiki/CompilationGuide/MSVC
- https://github.com/OpenChemistry/tomviz-superbuild/blob/master/projects/win32/BuildFFMPEG.md
##### 32-bit scripts
**x86 Native Tools Command Prompt**
```bat
REM compile libopus first: download, enter win32/VS2015, call MSBuild.exe, etc (detailed above)
set INCLUDE=%INCLUDE%;C:\vgmstream-dlls\sources\opus-1.3.1\include
set LIB=%LIB%;C:\vgmstream-dlls\sources\opus-1.3.1\win32\VS2015\Win32\Release
C:\msys64\msys2_shell.cmd -mingw32 -use-full-path
```
**MSYS2's mingw32 console**
```sh
# download FFmpeg first, etc
cd /c/vgmstream-dlls/sources/ffmpeg
# read current options (removing comments and line breaks); change file path if needed (or manually copy options below)
FFMPEG_OPTIONS=`sed -e '/^#/d' ../vgmstream/ext_libs/ffmpeg_options.txt`
echo $FFMPEG_OPTIONS
# PKG-CONFIG HACK: disables pkg-config in FFmpeg's configure (use only if *configure* throws a pkg-config error)
sed -i -e "s/require_pkg_config libopus/: #require_pkg_config libopus/g" configure
# opus should be on INCLUDE/LIB path now, otherwise try:
# --extra-cflags="-I."-I(full windows-style path)
# --extra-ldflags="(full windows-style path)\opus.lib"
sh ./configure $FFMPEG_OPTIONS --target-os=win32 --toolchain=msvc --arch=x86 --extra-ldflags="opus.lib" --prefix=/c/vgmstream-dlls/out/ffmpeg-32
make clean
make
make install
```
##### 64-bit scripts
**x64 Native Tools Command Prompt**
```bat
REM compile libopus first: download, enter win32/VS2015, call MSBuild.exe, etc (detailed above)
set INCLUDE=%INCLUDE%;C:\vgmstream-dlls\sources\opus-1.3.1\include
set LIB=%LIB%;C:\vgmstream-dlls\sources\opus-1.3.1\win32\VS2015\x64\Release
C:\msys64\msys2_shell.cmd -mingw64 -use-full-path
```
**MSYS2's mingw32 console**
```sh
# download FFmpeg first, etc
cd /c/vgmstream-dlls/sources/ffmpeg
# read current options (removing comments and line breaks); change file path if needed (or manually copy options below)
FFMPEG_OPTIONS=`sed -e '/^#/d' ../vgmstream/ext_libs/ffmpeg_options.txt`
echo $FFMPEG_OPTIONS
# PKG-CONFIG HACK: disables pkg-config in FFmpeg's configure (use only if *configure* throws a pkg-config error)
sed -i -e "s/require_pkg_config libopus/: #require_pkg_config libopus/g" configure
# opus should be on INCLUDE/LIB path now, otherwise try:
# --extra-cflags="-I."-I(full windows-style path)
# --extra-ldflags="(full windows-style path)\opus.lib"
sh ./configure $FFMPEG_OPTIONS --target-os=win64 --toolchain=msvc --arch=x86_64 --extra-ldflags="opus.lib" --prefix=/c/vgmstream-dlls/out/ffmpeg-64
make clean
make
make install
```

207
doc/BUILD.md
View File

@ -1,7 +1,6 @@
# vgmstream build help
This document explains how to build each of vgmstream's components and libraries.
## Compilation requirements
vgmstream can be compiled using one of several build scripts that are available in this repository. Components are detailed below, but if you are new to development you probably want one of these:
- **Windows**: [simple scripts](#simple-scripts-builds) + [Visual Studio 2019](#microsofts-visual-c-msvc--visual-studio--msbuild-compiler)
@ -50,8 +49,11 @@ make
- May try https://formulae.brew.sh/formula/vgmstream instead (not part of this project)
### Windows
- Install Visual Studio: https://www.visualstudio.com/downloads/ (for C/C++, with "MFC support" and "ATL support")
- Make a file called `msvc-build.config.ps1` in vgmstream's root, with your installed toolset and SDK:
- Install Visual Studio (VS): https://www.visualstudio.com/downloads/ (for C/C++, include "MFC support" and "ATL support")
- Currently defaults to VS 2019 and Win 10/11
- If you don't have VS 2019 and Win 10/11 you'll need to change project's defaults
- Manually open .sln and change them in Visual Studio
- Or make a file called `msvc-build.config.ps1` in vgmstream's source root, with your installed toolset and SDK:
```ps1
# - toolsets: "" (default), "v140" (VS 2015), "v141" (VS 2017), "v141_xp" (XP support), "v142" (VS 2019), etc
# - sdks: "" (default), "7.0" (Win7 SDK), "8.1" (Win8 SDK), "10.0" (Win10 SDK), etc
@ -68,15 +70,25 @@ This guide is mainly geared towards beginner devs, introducing concepts in steps
### GCC / Make (compiler)
Common C compiler, most development is done with this.
On **Windows** you need one of these somewhere in PATH:
- MinGW-w64 (32bit version), in any config (for example: gcc-8.1.0, i686, win32, sjlj)
- Use this for easier standalone executables
- [Sourceforce project](https://sourceforge.net/projects/mingw-w64/)
- [Latest online MinGW installer](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/installer/mingw-w64-install.exe/download).
- Or download and unzip the [portable MinGW package](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/8.1.0/threads-win32/sjlj/i686-8.1.0-release-win32-sjlj-rt_v6-rev0.7z/download)
- Or get from [alt builds](https://github.com/niXman/mingw-builds-binaries/releases/) (may be more recent versions)
- MSYS2 with the MinGW-w64_shell (32bit) package: https://msys2.github.io/
- Resulting binaries may depend on `msys*.dll`.
On **Windows** you need one of these:
- Standalone MinGW-w64, with **MSVCRT** runtime:
- Example config: gcc-12.2.x, **i686**/x86_64 (32-bit), **win32**/posix (threads), **dward**/sehs
- UCRT runtime is better but only installed by default in Windows 10
- [Sourceforce project](https://sourceforge.net/projects/mingw-w64/) (max 8.x, outdated)
- [Alt builds](https://github.com/niXman/mingw-builds-binaries/) (12.x)
- [Various flavors plus Clang](https://winlibs.com/) (12.x)
- Despite the name and config used, works and creates files for 32 or 64-bit Windows as needed
- Also get Git for Windows (described later), as it includes program that make compiling with GCC easier.
- Make sure the following are on Windows' `PATH` variable:
- `C:\mingw-w64\i686-12.2.0-release-win32-sjlj-rt_v10-rev0\mingw32\bin` (path to GCC compiler)
- `C:\Git\usr\bin` (from Git, extra Linux utils for Windows)
- You can add temp PATH vars in CMD/.bat by typing `set PATH=%PATH%;C:\(path)\`)
- [MSYS2](https://www.msys2.org/) environment, with extra required packages
- Open the `msys2/mingw32.exe` (or `msys2/mingw64.exe`s) console
- using regular `msys2/msys2.exe` may create binaries that depend on `msys*.dll`?
- may need to install dependencies using:
- `pacman -S git gcc make autotools` (also mingw-w64-i686/mingw-w64-x86_64?)
On **Linux** it should be included by default in the distribution, or can be easily installed using the distro's package manager (for example `sudo apt-get install gcc g++ make`).
@ -255,7 +267,7 @@ With no extra libs (or only some) enabled vgmstream works fine, but some advance
## Compiling modules
### CLI (test.exe/vgmstream-cli) / Winamp plugin (in_vgmstream) / XMPlay plugin (xmp-vgmstream)
### CLI (vgmstream-cli) / Winamp plugin (in_vgmstream) / XMPlay plugin (xmp-vgmstream)
**With GCC/Clang**: there are various ways to build it, each with some differences; you probably want CMake described below.
@ -274,8 +286,9 @@ You may try CMake instead as it may be simpler and handle libs better. See the b
**Windows** CMD .bat example (with some debugging on):
```bat
prompt $P$G$_$S
set PATH=C:\Program Files (x86)\Git\usr\bin;%PATH%
set PATH=C:\Program Files (x86)\mingw-w64\i686-5.4.0-win32-sjlj-rt_v5-rev0\mingw32\bin;%PATH%
set PATH=%PATH%;C:\mingw\i686-12.2.0-release-win32-sjlj-rt_v10-rev0\mingw32\bin
set PATH=%PATH%;C:\Git\usr\bin
cd vgmstream
@ -294,9 +307,6 @@ If you get build errors, remember you need to adjust compiler/SDK in the `.sln`.
CMake can also be used instead to create project files (no particular benefit).
#### Notes
While the official name for the CLI tool is `vgmstream-cli`, on **Windows**, `test.exe` is used instead for historical reasons. If you want to reuse it for your project, it's probably better to rename it to `vgmstream-cli.exe`.
### foobar2000 plugin (foo\_input\_vgmstream)
Requires MSVC (foobar/SDK only links to MSVC C++ DLLs). To build in Visual Studio, run `./msvc-build-init.bat`, open `vgmstream_full.sln` and compile. To build from the command line, just run `./msvc-build.bat`.
@ -320,8 +330,8 @@ You can also manually use the command line to compile with MSBuild, if you don't
prompt $P$G$_$S
REM MSVC ~2015
REM set PATH=%PATH%;C:\Program Files (x86)\MSBuild\14.0\Bin;%PATH%
REM Latest(?) MSVC
REM set PATH=%PATH%;C:\Program Files (x86)\MSBuild\14.0\Bin
REM Latest(?) MSVC (may also open "Native Tools Command Prompt for VS 20xx" in Windows' start menu)
set PATH=%PATH%;C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin
cd vgmstream
@ -464,149 +474,94 @@ Support for some codecs is done with external libs, instead of copying their cod
- not all licenses used by libs may allow to copy their code
- simplifies maintenance and updating
They are compiled in their own sources, and the resulting binary is linked by vgmstream using a few of their symbols.
Currently repo contains pre-compiled external libraries for **Windows** (32-bit Windows DLLs), while other systems link to system libraries. Ideally vgmstream could use libs compiled as static code (thus eliminating the need of DLLs), but involves a bunch of changes.
Below is a quick explanation of each library and how to compile binaries from them (for **Windows**). Unless mentioned, their latest version should be ok to use, though included DLLs may be a bit older.
MSVC needs a .lib helper to link .dll files, but libs below usually only create .dll (and maybe .def). Instead, those .lib are automatically generated during build step in `ext_libs.vcxproj` from .dll+.def, using lib.exe tool.
They are compiled in their own sources, and the resulting binary is linked by vgmstream using a few of their symbols (see [BUILD-LIB](BUILD-LIB.md) doc).
Currently vgmstream's repository contains pre-compiled external DLL libraries for **Windows**, while other systems link to system libraries or include static copies using CMake.
### libvorbis
Adds support for Vorbis, inside Ogg as `.ogg` (plain or encrypted) or custom variations like `.wem`, `.fsb`, `.ogl`, etc.
- Source: http://downloads.xiph.org/releases/vorbis/libvorbis-1.3.6.zip
- Sources:
- http://downloads.xiph.org/releases/vorbis/libvorbis-1.3.7.zip (for base vorbis decoding)
- http://downloads.xiph.org/releases/ogg/libogg-1.3.5.zip (for ogg support)
- Official Windows binaries: none
- Commonly used compilations: https://www.rarewares.org/ogg-libraries.php (32-bit only, fusing libogg+libvorbis+libvorbisfile)
- Version: 1.3.7
- DLL: `libvorbis.dll`
- lib: `-lvorbis -lvorbisfile`
- licensed under the 3-clause BSD license.
Should be buildable with MSVC (in /win32 dir are .sln files) or autotools (use `autogen.sh`).
- licensed under the 3-clause BSD license
### mpg123
Adds support for MPEG (MP1/MP2/MP3), used in formats that may have custom MPEG like `.ahx`, `.msf`, `.xvag`, `.scd`, etc.
- Source: https://sourceforge.net/projects/mpg123/files/mpg123/1.25.10/
- Builds: http://www.mpg123.de/download/win32/1.25.10/
- Source: https://sourceforge.net/projects/mpg123/files/mpg123/1.31.1/mpg123-1.31.1.tar.bz2
- Git mirror: https://github.com/madebr/mpg123 (commit `aec901b7a636b6eb61e03a87ff3547c787e8c693`)
- SVN repository: svn://scm.orgis.org/mpg123/trunk (version: ?)
- Official Windows binaries:
- https://www.mpg123.de/download/win32/1.31.0/mpg123-1.31.0-x86.zip (32-bit)
- https://www.mpg123.de/download/win64/1.31.0/mpg123-1.31.0-x86-64.zip (64-bit)
- Version: 1.32.0 / 1.31.1
- DLL: `libmpg123-0.dll`
- lib: `-lmpg123`
- licensed under the LGPL v2.1
Must use autotools (sh configure, make, make install), though some scripts simplify the process: `makedll.sh`, `windows-builds.sh`.
### libg719_decode
Adds support for ITU-T G.719 (standardization of Polycom Siren 22), used in a few Namco `.bnsf` games.
- Source: https://github.com/kode54/libg719_decode
- Source: https://github.com/kode54/libg719_decode (commit `da90ad8a676876c6c47889bcea6a753f9bbf7a73`)
- Official Windows binaries: none
- Version: latest
- DLL: `libg719_decode.dll`
- lib: ---
- unknown license (possibly invalid and Polycom's)
Use MSVC (use `g719.sln`). It can be built with GCC too, for example, using [the CMake script from this repository](../ext_libs/libg719_decode/CMakeLists.txt).
- unknown license (reference decoder, possibly not valid/Polycom's)
### FFmpeg
Adds support for multiple codecs: ATRAC3 (`.at3`), ATRAC3plus (`.at3`), XMA1/2 (`.xma`), WMA v1 (`.wma`), WMA v2 (`.wma`), WMAPro (`.xwma`), AAC (`.mp4`, `.aac`), Bink (`.bik`), AC3/SPDIF (`.ac3`), Opus (`.opus`), Musepack (`.mpc`), FLAC (`.flac`), etc.
- Source: https://github.com/FFmpeg/FFmpeg/
- DLLs: `avcodec-vgmstream-58.dll`, `avformat-vgmstream-58.dll`, `avutil-vgmstream-56.dll`, `swresample-vgmstream-3.dll`
Adds support for multiple codecs: ATRAC3 (`.at3`), ATRAC3plus (`.at3`), XMA1/2 (`.xma`), WMA v1 (`.wma`), WMA v2 (`.wma`), WMAPro (`.xwma`), AAC (`.mp4`, `.aac`), Bink (`.bik`), Smacker (`.smk`), AC3/SPDIF (`.ac3`), Opus (`.opus`), Musepack (`.mpc`), FLAC (`.flac`), etc. Vorbis, MPEG and PCM and a few others are also included for rare cases.
- Source: https://git.ffmpeg.org/ffmpeg.git (tag `n5.1.2`)
- Git mirror: https://github.com/FFmpeg/FFmpeg/ (tag `n5.1.2`)
- Official Windows binaries: none
- Version: n5.1.2
- DLLs: `avcodec-vgmstream-59.dll`, `avformat-vgmstream-59.dll`, `avutil-vgmstream-57.dll`, `swresample-vgmstream-4.dll`
- lib: `-lavcodec -lavformat -lavutil -lswresample`
- primarily licensed under the LGPL v2.1 or later, with portions licensed under the GPL v2
vgmstream's FFmpeg builds for **Windows** and static builds for **Linux** remove many unnecessary parts of FFmpeg to trim down its gigantic size, and, on Windows, are also built with the "vgmstream-" prefix to avoid clashing with other plugins. Current options can be seen in `ffmpeg_options.txt`. Shared **Linux** builds usually link to system FFmpeg without issues.
Note that the options above use *libopus*, but you can use FFmpeg's *Opus* by removing `--enable-libopus` and changing `--enable-decoder`'s `libopus` to `opus`. libopus is preferable since FFmpeg's Opus decoding is buggy in some files.
For GCC simply use [autotools](#autotools-builds), passing to `./configure` the above options.
For MSCV it can be done through a helper: https://github.com/jb-alvarado/media-autobuild_suite
Both may need yasm somewhere in PATH to properly compile: https://yasm.tortall.net
### libopus
Indirectly used by FFmpeg for improved Opus (`.opus` and variants) support.
- Source: https://archive.mozilla.org/pub/opus/opus-1.3.1.tar.gz
- Git mirror: https://github.com/xiph/opus (tag `v1.3.1`)
- Official Windows binaries: none
- Version: v1.3.1
- DLLs: (part of FFmpeg)
- lib: (part of FFmpeg)
- licensed under a variant of the BSD license: https://opus-codec.org/license/
### LibAtrac9
Adds support for ATRAC9, used in `.at9` and other formats for the PS4 and Vita.
- Source: https://github.com/Thealexbarney/LibAtrac9
- Source: https://github.com/Thealexbarney/LibAtrac9 (commit `6a9e00f6c7abd74d037fd210b6670d3cdb313049`)
- Official Windows binaries:
- https://github.com/Thealexbarney/LibAtrac9/releases (32-bit only, outdated)
- Version: latest
- DLL: `libatrac9.dll`
- lib: `-latrac9` / `-l:libatrac9.a`
- licensed under the MIT license
Use MSCV and `libatrac9.sln`, or GCC and the Makefile included.
### libcelt
Adds support for FSB CELT versions 0.6.1 and 0.11.0, used in a handful of older `.fsb`.
- Source (0.6.1): http://downloads.us.xiph.org/releases/celt/celt-0.6.1.tar.gz
- Git mirror: https://gitlab.xiph.org/xiph/celt (commit `8ccf148573277b983692e15d5f0753081f806bea`)
- Source (0.11.0): http://downloads.xiph.org/releases/celt/celt-0.11.0.tar.gz
- Git mirror: https://gitlab.xiph.org/xiph/celt (commit `0b405d1170122c859faab435405666506d52fa2e`)
- Official Windows binaries: none
- Versions: 0.6.1, 0.11.0
- DLL: `libcelt-0061.dll`, `libcelt-0110.dll`
- lib: `-lcelt-0061` `-lcelt-0110` / `-l:libcelt-0110.a` `-l:libcelt-0061.a`
- licensed under the MIT license
FSB uses two incompatible, older libcelt versions. Both libraries export the same symbols so normally can't coexist together. To get them working we need to make sure symbols are renamed first. This may be solved in various ways:
- using dynamic loading (LoadLibrary) but for portability it isn't an option
- It may be possible to link+rename using .def files
- Linux/Mingw's objcopy to (supposedly) rename DLL symbols
- Use GCC's preprocessor to rename functions on compile
- Rename functions in the source code directly.
To compile we'll use autotools with GCC preprocessor renaming:
- in the celt-0.6.1 dir:
```bat
# creates Makefiles with Automake
sh.exe ./configure --build=mingw32 --prefix=/c/celt0.6.1/bin/ --exec-prefix=/c/celt-0.6.1/bin/
# LDFLAGS are needed to create the .dll (Automake whining)
# CFLAGS rename a few CELT functions (we don't import the rest so they won't clash)
mingw32-make.exe clean
mingw32-make.exe LDFLAGS="-no-undefined" AM_CFLAGS="-Dcelt_decode=celt_0061_decode -Dcelt_decoder_create=celt_0061_decoder_create -Dcelt_decoder_destroy=celt_0061_decoder_destroy -Dcelt_mode_create=celt_0061_mode_create -Dcelt_mode_destroy=celt_0061_mode_destroy -Dcelt_mode_info=celt_0061_mode_info"
```
- in the celt-0.11.0 dir:
```bat
# creates Makefiles with Automake
sh.exe ./configure --build=mingw32 --prefix=/c/celt-0.11.0/bin/ --exec-prefix=/c/celt-0.11.0/bin/
# LDFLAGS are needed to create the .dll (Automake whining)
# CFLAGS rename a few CELT functions (notice one is different vs 0.6.1), CUSTOM_MODES is also a must.
mingw32-make.exe clean
mingw32-make.exe LDFLAGS="-no-undefined" AM_CFLAGS="-DCUSTOM_MODES=1 -Dcelt_decode=celt_0110_decode -Dcelt_decoder_create_custom=celt_0110_decoder_create_custom -Dcelt_decoder_destroy=celt_0110_decoder_destroy -Dcelt_mode_create=celt_0110_mode_create -Dcelt_mode_destroy=celt_0110_mode_destroy -Dcelt_mode_info=celt_0110_mode_info"
```
- take the .dlls from ./bin/bin, and rename libcelt.dll to libcelt-0061.dll and libcelt-0110.dll respectively.
- you need to create a .def file for those DLL with the renamed simbol names above
- finally the includes. libcelt gives "celt.h" "celt_types.h" "celt_header.h", but since we renamed a few functions we have a simpler custom .h with minimal renamed symbols.
For **Linux**, you can use CMake that similarly patch celt libs automatically.
You can also get them from the official git (https://gitlab.xiph.org/xiph/celt) call `./autogen.sh` first, then pass call configure/make with renames (see `./make-build.sh`).
Instead of passing `-DCUSTOM_MODES=1` to `make` you can pass `--enable-custom-codes` to *./configure*. There is also `--disable-oggtests`, `--disable-static/shared` and typical config. Note that if *./configure* finds Ogg in your system it'll try to build encoder/decoder test `tools` (that depend on libogg). There is no official way disable that or compile `libcelt` only, but you can force it by calling `make SUBDIRS=libcelt DIST_SUBDIRS=libcelt`, in case you have dependency issues.
### libspeex
Adds support for Speex (inside custom containers), used in a few *EA* formats (`.sns`, `.sps`) for voices.
- Source: http://downloads.us.xiph.org/releases/speex/speex-1.2.0.tar.gz
- DLL: `libspeex.dll`
- Source: http://downloads.us.xiph.org/releases/speex/speex-1.2.1.tar.gz
- Git: https://gitlab.xiph.org/xiph/speex (tag `Speex-1.2.1`)
- Github: https://github.com/xiph/speex/releases/tag/Speex-1.2.1
- Official Windows binaries:
- http://downloads.xiph.org/releases/speex/speex-1.2beta3-win32.zip (32-bit only, outdated)
- Version: Speex-1.2.1 (latest)
- DLL: `libspeex-1.dll`
- lib: `-lspeex`
- licensed under the Xiph.Org variant of the BSD license.
https://www.xiph.org/licenses/bsd/speex/
Should be buildable with MSVC (in /win32 dir are .sln files, but not up to date and may need to convert .vcproj to vcxproj) or autotools (use `autogen.sh`, or script below).
You can also find a release on Github (https://github.com/xiph/speex/releases/tag/Speex-1.2.0). It has newer timestamps and some different helper files vs Xiph's release, but actual lib should be the same. Notably, Github's release *needs* `autogen.sh` that calls `autoreconf` to generate a base `configure` script, while Xiph's pre-includes `configure`. Since getting autoreconf working on **Windows** can be quite involved, Xiph's release is recommended on that platform.
**Windows** CMD example:
```bat
set PATH=%PATH%;C:\mingw\i686-8.1.0-release-win32-sjlj-rt_v6-rev0\mingw32\bin
set PATH=%PATH%;C:\Git\usr\bin
sh ./configure --host=mingw32 --prefix=/c/celt-0.11.0/bin/ --exec-prefix=/c/celt-0.11.0/bin/
mingw32-make.exe LDFLAGS="-no-undefined -static-libgcc" MAKE=mingw32-make.exe
mingw32-make.exe MAKE=mingw32-make.exe install
```
If all goes well, use generated .DLL in ./bin/bin (may need to rename to libspeex.dll) and ./win32/libspeex.def, and speex folder with .h in bin/include.
### maiatrac3plus
This lib was used as an alternate for ATRAC3PLUS decoding. Now this is handled by FFmpeg, though some code remains for now.
It was a straight-up decompilation from Sony's libs (presumably those found in SoundForge), without any clean-up or actual reverse engineering, thus legally and morally dubious.
It doesn't do encoder delay properly, but on the other hand decoding is 100% accurate unlike FFmpeg (probably inaudible though).
So, don't use it unless you have a very good reason.
- licensed under the Xiph.Org variant of the BSD license: https://www.xiph.org/licenses/bsd/speex/

2
doc/USAGE.md
View File

@ -20,7 +20,7 @@ Put the following files somewhere Windows can find them:
- `libatrac9.dll`
- `libcelt-0061.dll`
- `libcelt-0110.dll`
- `libspeex.dll`
- `libspeex-1.dll`
For command line (`vgmstream-cli.exe`) and XMPlay this means in the directory with the main
`.exe`, or possibly a directory in the PATH variable.