Cool_Tools/vgmstream
1
0
Fork 0
mirror of https://github.com/vgmstream/vgmstream.git synced 2024-09-24 19:28:24 +02:00
Code Issues Packages Projects Releases Wiki Activity
978dab14e6
vgmstream/doc/BUILD.md
bnnm 978dab14e6 doc
2021-07-29 17:36:59 +02:00

22 KiB
Raw Blame History

vgmstream build help

This document explains how to build each of vgmstream's components and libs.

Compilation requirements

Because each module has different quirks one can't use a single tool for everything. You should be able to build most using a standard compiler (GCC/MSVC/Clang) in any typical OS (Windows/Linux/Mac) using common build helpers (scripts/CMake/autotools).

64-bit support may work but has been minimally tested, since main use of vgmstream is plugins for 32-bit players (should work but extra codec libs are included for 32-bit only ATM).

GCC / Make

Common C compiler, most development is done with this.

On Windows you need one of these somewhere in PATH:

On Linux it should be included by default in the distribution, or can be easily installed using the distro's package manager (for example sudo apt-get install gcc g++ make).

Microsoft's Visual C++ (MSVC) / Visual Studio / MSBuild

Alt C compiler, auto-generated builds for Windows use this.

Bundled in either:

Visual Studio Community should work (free), but must register after trial period. Even after trial you can still use MSBuild, command-line tool that actually does all the building. You can also find and install "Visual C++ Build Tools" without Visual Studio IDE (try google as MS's links tend to move around).

Older versions of MSVC (2010 and before) have limited C support, while reportedly beta/new versions aren't always very stable. Some very odd issues affecting only MSVC have been found and fixed before. Keep in mind if you run into problems.

Clang

Reportedly works fine on Mac and may used as a replacement for GCC without issues.

Should be usable on Linux and possibly Windows with CMake.

For other makefiles may work, though may need to set compiler vars appropriately (CC=clang AR=llvm-ar).

Git

Optional, used to generate version numbers:

Git also comes with typical Linux utils compiled for Windows (in the usr\bin dir), that can help when compiling some extra components on Windows.

CMake

Optional, can be used to compile vgmstream's modules instead of regular scripts. Needs v3.6 or later:

If you wish to use CMake, see CMAKE.md. Some extra info is only mentioned in this doc though.

autotools

Optional, used by some modules (mainly Audacious for Linux, and external libs).

For Windows you must include GCC, and Linux's sh tool in some form in PATH. The simplest would be installing MinGW-w64 for GCC.exe, and Git for sh.exe, and making PATH point their bin dir.

  • ex. C:\i686-8.1.0-release-win32-sjlj-rt_v6-rev0\mingw32\bin and C:\Git\usr\bin
  • Both must be installed/copied in a dir without spaces (with spaces autoconf seemingly works but creates buggy files)
  • If you don't have Git, try compiled GNU tools for Windows (http://gnuwin32.sourceforge.net/packages.html)

A useful trick for Windows is that you can alter PATH variable temporarly in .bat scripts (PATH is used to call programs in Windows without having to write full path to .exe)

set PATH=%PATH%;C:\i686-8.1.0-release-win32-sjlj-rt_v6-rev0\mingw32\bin
set PATH=%PATH%;C:\Git\usr\bin
gcc.exe (...)

For Linux, those may be included already, or install with a package manager (sudo apt-get install autoconf automake libtool).

Typical usage involves ./configure (creates makefiles) + Make (compiles) + Make install (copies results), but varies slightly depending on module/lib (explained later).

External libs using autotools can be compiled on Windows too, try using sh.exe ./configure, mingw32-make.exe, mingw32-make.exe install instead. Also for older libs, call sh.exe ./configure with either --build=mingw32, --host=mingw32 or --target-os=mingw32 (varies) for older configure. You may also need to use mingw32-make.exe LDFLAGS="-no-undefined -static-libgcc" MAKE=mingw32-make.exe so that .dll are correctly generated.

Extra libs

Optional, add codec or extra functionalities. See External libraries for full info.

On Windows most libs are pre-compiled and included to simplify building (since they can be quite involved to compile).

On Linux you usually need dev packages of each (for example libao-dev for vgmstream123, audacious-dev for Audacious, and so on) and they should be picked by CMake/autotool scripts.

Compiling modules

CLI (test.exe/vgmstream-cli) / Winamp plugin (in_vgmstream) / XMPlay plugin (xmp-vgmstream)

With GCC/Clang: there are various ways to build it, each with some differences; you probably want CMake described below.

Simplest way is using the ./Makefile in the root folder, see inside for options. For compilation flags check the Makefile in each folder. You may need to manually rebuild if you change a .h file (use make clean). On Windows this will build with external libs enabled, but Linux can't ATM.

Also, on Linux you can't build in_vgmstream and xmp-vgmstream (given they are Windows DLLs...). Makefiles have been used in the past to cross-compile from Linux with MingW headers though, but can't generate native Win code at the moment (should be fixable with some effort).

Autotools should build and install it as vgmstream-cli, this is explained in detail in the Audacious section. It enables (some) extra codecs. Some Linux distributions like Arch Linux include pre-patched vgmstream with most libraries, you may want that instead: https://aur.archlinux.org/packages/vgmstream-kode54-git/

If you use Mac (or Linux), there is a Homebrew script that may automate the process (uses CMake): https://formulae.brew.sh/formula/vgmstream

You may try CMake instead as it may be simpler and handle libs better. Some older distros may not work though (CMake version needs to recognize FILTER command). You may also need to install resulting artifacts manually (check ./cli dir). Check the CMAKE.md doc for some extra info too.

sudo apt-get update
sudo apt-get install -y libmpg123-dev libvorbis-dev libspeex-dev
sudo apt-get install -y libavformat-dev libavcodec-dev libavutil-dev libswresample-dev
sudo apt-get install -y libao-dev audacious-dev
sudo apt-get install -y cmake

git clone https://github.com/vgmstream/vgmstream
cd vgmstream

cmake .
make

Windows CMD example (with some debugging on):

prompt $P$G$_$S
set PATH=C:\Program Files (x86)\Git\usr\bin;%PATH%
set PATH=C:\Program Files (x86)\mingw-w64\i686-5.4.0-win32-sjlj-rt_v5-rev0\mingw32\bin;%PATH%

cd vgmstream

mingw32-make.exe vgmstream_cli -f Makefile ^
 VGM_ENABLE_MAIATRAC3PLUS=0 ^
 EXTRA_CFLAGS="-DVGM_DEBUG_OUTPUT -g -Wimplicit-function-declaration" ^
 SHELL=sh.exe CC=gcc.exe AR=ar.exe STRIP=strip.exe DLLTOOL=dlltool.exe WINDRES=windres.exe ^
 STRIP=echo ^
 1> ../vgmstream-stdout.txt 2> ../vgmstream-stderr.txt

With MSVC: To build in Visual Studio, run ./init-build.bat, open ./vgmstream_full.sln and compile. To build from the command line, run ./build.bat.

The build script will automatically handle obtaining dependencies and making the project changes listed in the foobar2000 section (you may need to get some PowerShell .NET packages).

You can also call MSBuild directly in the command line (see the foobar2000 section for dependencies and examples)

notes

While the official name for the CLI tool is vgmstream-cli, test.exe is used on Windows for historical reasons. If you want to reuse it for your own project it's probably better renaming to vgmstream-cli.exe.

foobar2000 plugin (foo_input_vgmstream)

Requires MSVC (foobar/SDK only links to MSVC C++ DLLs) and these dependencies:

The following project modifications are required:

  • For foobar2000_ATL_helpers add ../../../WTL/Include to the compilers's additional includes

FDK-AAC/QAAC can be enabled adding VGM_USE_MP4V2 and VGM_USE_FDKAAC in the compiler/linker options and the project dependencies, otherwise FFmpeg is used instead to support .mp4. FDK-AAC Support is limited so FFmpeg is recommended.

You can also manually use the command line to compile with MSBuild, if you don't want to touch the .vcxproj files, register VS after trial, get PowerShell dependencies for the build script, or only have VC++/MSBuild tools.

Windows CMD example for foobar2000:

prompt $P$G$_$S
set PATH=C:\Program Files (x86)\Git\usr\bin;%PATH%
set PATH=C:\Program Files (x86)\MSBuild\14.0\Bin;%PATH%

cd vgmstream

set CL=/I"C:\projects\WTL\Include"
set LINK="C:\projects\foobar\foobar2000\shared\shared.lib"

msbuild fb2k/foo_input_vgmstream.vcxproj ^
 /t:Clean

msbuild fb2k/foo_input_vgmstream.vcxproj ^
 /t:Build ^
 /p:Platform=Win32 ^
 /p:PlatformToolset=v140 ^
 /p:Configuration=Release ^
 /p:DependenciesDir=../..

Audacious plugin

Requires the dev version of Audacious (and dependencies), automake/autoconf or CMake, and gcc/make (C++11). It must be compiled and installed into Audacious, where it should appear in the plugin list as "vgmstream".

The plugin needs Audacious 3.5 or higher. New Audacious releases can break plugin compatibility so it may not work with the latest version unless adapted first.

libvorbis/libmpg123/libspeex will be used if found, while FFmpeg and other external libraries aren't enabled at the moment, thus some formats won't work (build scripts need to be fixed).

Windows builds aren't supported at the moment (should be possible but there are complex dependency chains).

If you get errors during the build phase we probably forgot some #ifdef needed for Audacious, notify and should be quickly fixed.

Take note of other plugins stealing extensions (see README). To change Audacious's default priority for vgmstream you can make with CFLAG AUDACIOUS_VGMSTREAM_PRIORITY n (where N is a number where 10=lowest)

Terminal example, assuming a Ubuntu-based Linux distribution:

# build setup

# default requirements
sudo apt-get update
sudo apt-get install gcc g++ make
sudo apt-get install autoconf automake libtool
sudo apt-get install git
# vgmstream dependencies
sudo apt-get install libmpg123-dev libvorbis-dev libspeex-dev
#sudo apt-get install libavformat-dev libavcodec-dev libavutil-dev libswresample-dev
# Audacious player and dependencies
sudo apt-get install audacious
sudo apt-get install audacious-dev libglib2.0-dev libgtk2.0-dev libpango1.0-dev
# vgmstream123 dependencies (optional)
sudo apt-get install libao-dev

# check Audacious version >= 3.5
pkg-config --modversion audacious

# base vgmstream build
git clone https://github.com/vgmstream/vgmstream
cd vgmstream

# main vgmstream build (if you get errors here please report)
./bootstrap
./configure
make -f Makefile.autotools

# copy to audacious plugins (note that this will also install "libvgmstream",
# vgmstream-cli and vgmstream123, so they can be invoked from the terminal)
sudo make -f Makefile.autotools install

# update global libvgmstream.so.0 refs
sudo ldconfig

# start audacious in verbose mode to check if it was installed correctly
audacious -V

# if all goes well no "ERROR (something) referencing libvgmstream should show 
# in the terminal log, then go to menu services > plugins > input tab and check
# vgmstream is there (you can start audacious normally next time)

# uninstall if needed
sudo make -f Makefile.autotools uninstall

# optional post-cleanup
make -f Makefile.autotools clean
find . -name ".deps" -type d -exec rm -r "{}" \;
./unbootstrap
## WARNING, removes *all* untracked files not in .gitignore
git clean -fd

To update vgmstream it's probably easiest to remove the vgmstream folder and start again from base vgmstream build step, since updates often require a full rebuild anyway.

Instead of autotools you can try building with CMake. Some older distros may not work though (CMake version needs to recognize FILTER command). You may need to install resulting artifacts manually (check ./audacious dir).

sudo apt-get update
sudo apt-get install -y libmpg123-dev libvorbis-dev libspeex-dev
sudo apt-get install -y libavformat-dev libavcodec-dev libavutil-dev libswresample-dev
sudo apt-get install -y libao-dev audacious-dev
sudo apt-get install -y cmake

git clone https://github.com/vgmstream/vgmstream
cd vgmstream

cmake .
make

vgmstream123 player

Should be buildable with Autotools/CMake by following the same steps as listen in the Audacious section (requires libao-dev).

Windows builds are possible with libao.dll and libao includes (found elsewhere) through the Makefile, but some features are disabled.

libao is licensed under the GPL v2 or later.

External libraries

Support for some codecs is done with external libs, instead of copying their code in vgmstream. There are various reasons for this:

  • each lib may have complex or conflicting ways to compile that aren't simple to replicate
  • their sources can be quite big and undesirable to include in full
  • libs usually only compile with either GCC or MSVC, while vgmstream supports both compilers, so linking to the generated binary (compatible) is much easier
  • not all licenses used by libs may allow to copy their code
  • simplifies maintenance and updating

They are compiled in their own sources, and the resulting binary is linked by vgmstream using a few of their symbols.

Currently repo contains pre-compiled external libraries for Windows (32-bit Windows DLLs), while other systems link to system libraries. Ideally vgmstream could use libs compiled as static code (thus eliminating the need of DLLs), but involves a bunch of changes.

Below is a quick explanation of each library and how to compile binaries from them (for Windows). Unless mentioned, their latest version should be ok to use, though included DLLs may be a bit older.

MSVC needs a .lib helper to link .dll files, but libs below usually only create .dll (and maybe .def). Instead, those .lib are automatically generated during build step in ext_libs.vcxproj from .dll+.def, using lib.exe tool.

libvorbis

Adds support for Vorbis (inside Ogg and custom containers).

Should be buildable with MSVC (in /win32 dir are .sln files) or autotools (use autogen.sh).

mpg123

Adds support for MPEG (MP1/MP2/MP3).

Must use autotools (sh configure, make, make install), though some scripts simplify the process: makedll.sh, windows-builds.sh.

libg719_decode

Adds support for ITU-T G.719 (standardization of Polycom Siren 22).

Use MSVC (use g719.sln). It can be built with GCC too, but you'll need to manually create scripts/makefiles.

FFmpeg

Adds support for multiple codecs: ATRAC3, ATRAC3plus, XMA1/2, WMA v1, WMA v2, WMAPro, AAC, Bink, AC3/SPDIF, Opus, Musepack, FLAC, etc (also Vorbis and MPEG for certain cases).

  • Source: https://github.com/FFmpeg/FFmpeg/
  • DLLs: avcodec-vgmstream-58.dll, avformat-vgmstream-58.dll, avutil-vgmstream-56.dll, swresample-vgmstream-3.dll
  • primarily licensed under the LGPL v2.1 or later, with portions licensed under the GPL v2

vgmstream's FFmpeg builds remove many unnecessary parts of FFmpeg to trim down its gigantic size, and are also built with the "vgmstream-" preffix (to avoid clashing with other plugins). Current options can be seen in ffmpeg_options.txt.

For GCC simply use autotools (configure, make, make install), passing to configure the above options.

For MSCV it can be done through a helper: https://github.com/jb-alvarado/media-autobuild_suite

Both may need yasm somewhere in PATH to properly compile: https://yasm.tortall.net

LibAtrac9

Adds support for ATRAC9.

Use MSCV and libatrac9.sln, or GCC and the Makefile included.

libcelt

Adds support for FSB CELT versions 0.6.1 and 0.11.0.

FSB uses two incompatible, older libcelt versions. Both libraries export the same symbols so normally can't coexist together. To get them working we need to make sure symbols are renamed first. This may be solved in various ways:

  • using dynamic loading (LoadLibrary) but for portability it isn't an option
  • It may be possible to link+rename using .def files
  • Linux/Mingw's objcopy to (supposedly) rename DLL symbols
  • Use GCC's preprocessor to rename functions on compile
  • Rename functions in the source code directly.

To compile we'll use autotools with GCC preprocessor renaming:

  • in the celt-0.6.1 dir: 
    # creates Makefiles with Automake
sh.exe ./configure --build=mingw32 --prefix=/c/celt-0.11.0/bin/  --exec-prefix=/c/celt-0.11.0/bin/

# LDFLAGS are needed to create the .dll (Automake whinning)
# CFLAGS rename a few CELT functions (we don't import the rest so they won't clash)
mingw32-make.exe clean
mingw32-make.exe LDFLAGS="-no-undefined" AM_CFLAGS="-Dcelt_decode=celt_0061_decode -Dcelt_decoder_create=celt_0061_decoder_create -Dcelt_decoder_destroy=celt_0061_decoder_destroy -Dcelt_mode_create=celt_0061_mode_create -Dcelt_mode_destroy=celt_0061_mode_destroy -Dcelt_mode_info=celt_0061_mode_info"
  • in the celt-0.11.0 dir: 
    # creates Makefiles with Automake
sh.exe ./configure --build=mingw32 --prefix=/c/celt-0.11.0/bin/  --exec-prefix=/c/celt-0.11.0/bin/

# LDFLAGS are needed to create the .dll (Automake whinning)
# CFLAGS rename a few CELT functions (notice one is different vs 0.6.1), CUSTOM_MODES is also a must.
mingw32-make.exe clean
mingw32-make.exe LDFLAGS="-no-undefined" AM_CFLAGS="-DCUSTOM_MODES=1 -Dcelt_decode=celt_0110_decode -Dcelt_decoder_create_custom=celt_0110_decoder_create_custom -Dcelt_decoder_destroy=celt_0110_decoder_destroy -Dcelt_mode_create=celt_0110_mode_create -Dcelt_mode_destroy=celt_0110_mode_destroy -Dcelt_mode_info=celt_0110_mode_info"
  • take the .dlls from ./bin/bin, and rename libcelt.dll to libcelt-0061.dll and libcelt-0110.dll respectively.
  • you need to create a .def file for those DLL with the renamed simbol names above
  • finally the includes. libcelt gives "celt.h" "celt_types.h" "celt_header.h", but since we renamed a few functions we have a simpler custom .h with minimal renamed symbols.

libspeex

Adds support for Speex (inside custom containers).

Should be buildable with MSVC (in /win32 dir are .sln files, but not up to date and may need to convert .vcproj to vcxproj) or autotools (use autogen.sh, or script below).

You can also find a release on Github (https://github.com/xiph/speex/releases/tag/Speex-1.2.0). It has newer timestamps and some different helper files vs Xiph's release, but actual lib should be the same. Notably, Github's release needs autogen.sh that calls autoreconf to generate a base configure script, while Xiph's pre-includes configure. Since getting autoreconf working on Windows can be quite involved, Xiph's release is recommended.

Windows CMD example:

set PATH=%PATH%;C:\Git\usr\bin
set PATH=%PATH%;C:\i686-8.1.0-release-win32-sjlj-rt_v6-rev0\mingw32\bin

sh ./configure --host=mingw32 --prefix=/c/celt-0.11.0/bin/  --exec-prefix=/c/celt-0.11.0/bin/
mingw32-make.exe LDFLAGS="-no-undefined -static-libgcc" MAKE=mingw32-make.exe
mingw32-make.exe MAKE=mingw32-make.exe install

If all goes well, use generated .DLL in ./bin/bin (may need to rename to libspeex.dll) and ./win32/libspeex.def, and speex folder with .h in bin/include.

maiatrac3plus

This lib was used as an alternate for ATRAC3PLUS decoding. Now this is handled by FFmpeg, though some code remains for now.

It was a straight-up decompilation from Sony's libs (presumably those found in SoundForge), without any clean-up or actual reverse engineering, thus legally and morally dubious.

It doesn't do encoder delay properly, but on the other hand decoding is 100% accurate unlike FFmpeg (probably inaudible though).

So, don't use it unless you have a very good reason.