For this series, we'll be using this demonstration scene as the example starting point for VRSL AudioLink. As stated before, most of this information can be found in the Project Setup Guide, so do ensure you read that guide first before coming to this one. You can use whatever scene you may wish, but do note that there are 3 important things setup for this scene:
- The
VRSL-LocalUIControlPanelprefab is present in this scene. As stated in the project setup guide, this is where most of the global settings for VRSL will be located. - The
Directional Light (For Depth)prefab is also present in this scene. This adds the depth-texture to the scene, which allows for most of VRSL's effects to work properly. - A Layer called
PostProcessinghas been added. The default Post Processing prefabs in VRSL look for a layer called PostProcessing and will automatically assign itself to it if it can find it. Once you've done that, ensure that aPost-process Layercomponent is added to your main camera with the layer parameter on it set to yourPostProcessingLayer.
You can see all of this here in this image:
Setting Up the Control Panel
Before we jump into AudioLink, we need to set up the VRSL control panel to be in AudioLink Mode. To do this, open the control panel by hitting this button and the window should appear before you:
Next, since we are not using DMX mode in this project we will go ahead and disable it and all of its features. If you intend to use both DMX and AudioLink in your project, go ahead and skip this step, otherwise, hit the purple DMX button, which should turn it red like this:
Using the Control Panel
Now that the Control Panel is ready to go, we'll begin setting up AudioLink by spawning the appropriate prefabs. To do this, click the Show Options button on the VRSL Control Panel, which should drop down a few more options like this:
Next, hit the Show AudioLink Prefab Spawn Buttons, which will show a list of useful buttons to spawn all the prefabs you may need for your VRSL AudioLink project.
Once this menu is open, you'll see a large list of buttons. These buttons spawn both AudioLink and VRSL AudioLink prefabs. This guide assumes you're starting completely from zero, So the first thing we'll do is add the AudioLink prefab itself. You can do that from the control panel directly by hitting the Spawn Standard AudioLink Prefab button:
Which will spawn the AudioLink prefab into your world like so:
Afterward, we can go ahead and move to spawn a controller for AudioLink. There are buttons for bot the standard AudioLink Controller that comes with AudioLink and the modified AudioLink Controller specifically for VRSL. For this tutorial, we will be using the modified AudioLinkController-WithVRSLSmoothing. To spawn it, click the `Spawn VRSL AudioLink Controller Button:
Once zoomed in on the controller after it spawned, it should look something like this:
Afterwhich, you will want to ensure that the controller is connected to your AudioLink prefab. To do this, open the dropdown button on the AudioLinkController-WithVRSLSmoothing prefab in your hierarchy to access its children.
From there click on the AudioLinkController object to view its script in the inspector.
Finally, drag the AudioLink prefab from your hierarchy into the "AudioLink" parameter of the AudioLinkController script.
Connecting AudioLink to the USharp Video Player
For the rest of this tutorial to make sense, we will need a source of audio to feed into AudioLink. The rest of this page will be focused on getting the USharp Video Player to work both in Editor and in VRChat with AudioLink. If you already know how to set up your video player or audio source with AudioLink, go ahead and skip to the next chapter.
So, let's start by dragging the Standard USharp Video Prefab into the world. You can do this by going to Assets/USharpVideo in your projects folder, then dragging the USharpVideo.prefab into your scene. You should get something like this:
Once we have the prefab in, we can go ahead and set a default video to use for our testing. (Note: This video must be a video and not a livestream as the following steps to get videos to work in Editor will not work for livestreams). I picked a video that had a strong drum start so that I can see how the bass and lower mids react rather quickly.
Getting Usharp Video To Work In The Editor
NOTE: THIS TRICK MAY OR MAY NOT WORK FOR YOU DEPENDING ON HOW YOU HAVE CERTAIN DLLS INSTALLED. THIS TRICK INVOLVES MODIFYING A SMALL EDITOR SCRIPT THAT CAME WITH USHARP VIDEO. (Re-Implementing a fix that wasn't included in the unitypackage version of UsharpVideo.)
If you attempt to try to play your scene in Editor now, most likely, you will have UsharpVideo spitting errors out at you like this:
This is because one of UsharpVideo's Editor scripts is looking for the youtube-dl.exe file that usually comes installed with VRChat. This is what VRChat uses to playback youtube videos in the Unity Video Player. Due to certain circumstances that are too complex to explain, VRChat now instead uses a similar file called yt-dlp.exe. What we need to fix this error is to tell the editor script to look for yt-dlp.exe instead of youtube-dl.exe.
After leaving play mode... in your project window, go to the search bar and search for EditorURLResolverShim.cs. This is the file we're going to be editing. Double-clicking the file in unity will open it in whatever default text editor has been set for Unity (Usually Visual Studio). You can also right-click the file and choose "Show In Explorer" to see the file in windows and open it with whatever text editor of your choice.
Once you have opened the file in your text editor of your choice, you'll want to travel to line 30, which should say something along the lines of youtubeDLPath = string.Join("\\", splitPath.Take(splitPath.Length - 2)) + "\\VRChat\\VRChat\\Tools\\youtube-dl.exe"; as pictured here:
What we need to change is the youtube-dll.exe part towards the end and replace it with yt-dlp.exe. The line should now read youtubeDLPath = string.Join("\\", splitPath.Take(splitPath.Length - 2)) + "\\VRChat\\VRChat\\Tools\\yt-dlp.exe"; like so:
Once you've confirmed the changes, save the file (either with CTL + S or hitting File>Save and not "Save As") and close your text editor.
If you hit play now, the video should now start to play inside the Editor!:
This will not affect how the video player works in-game, only in the editor. This modification is very useful for testing your AudioLink setup with multiple videos with various audio qualities. Some other video players may already have this feature implemented, but since we are using USharp Video as a baseline, this modification will be necessary.
Usharp Video Into AudioLink
Here is where we set up Usharp Video to properly connect to AudioLink while supporting both the video player in Usharp Video as well as its Stream player, which is accessed by hitting the Stream/Video switch on the USharp Video Player's in-game UI menu.
Reducing The Number of Audio Sources
To begin, we need to reduce the number of audio sources on the Usharp Player down to one. To do that, we're going to make the VideoAudioSource object under the AudioSources object able to support both the Unity Video player and the AVPro Stream Player.
First, under the UsharpVideo object in the hierarchy, click the Internals object to the VideoPlayerManager component attached to it in the inspector. In that component, reduce the Audio Sources property's size down to 1 so that only the VideoAudioSource object is listed there.
Afterward, we'll set the Audio Sources property's size back to 3 now. This will populate the list with 3 copies of the VideoAudioSource object. We need to do this as the UsharpVideoPlayer is looking for 2 more audio source components for the stream player specifically.
Next, under the AudioSources object in the hierarchy, click on its arrow to look at that object's children and click on the VideoAudioSource object to view it in the inspector. In the inspector, click the Add Component button to add a component. In the search dialog that pops up after clicking the Add Component button, search for VRC AVPro Video Speaker and add it to the object.
You should now see this component in the VideoAudioSource object:
And if you haven't already, make sure to set the Mode parameter of the component to Stereo Mix to ensure that the audio source plays both the left and right channels of the incoming signal.
Next, we will need to remove the VRC AVPro Video Speaker components from the two StreamAudioSource components as they will not be needed and may cause conflicts. To do this, go to both the left and right StreamAudioSource objects and right click on the title of the VRC AVPro Video Speaker to open a dialog menu. In that menu, click Remove Component. Because this is a prefab, the component will still be visible but be greyed out and says (Removed) at the end of its title. Do not worry though, the component is actually removed, but Unity has the original state of the prefab stored, so you this allows you to revert this change if you wanted to in the future. The steps should be like this:
Finally, you should now be able to disable the two StreamAudioSource objects themselves as they will no longer be needed either.
Attaching The Audio Source to AudioLink
We are now ready to attach the audio source to the AudioLink object and give it a test.
To do this we, simply drag our VideoAudioSource object into the the Audio Source parameter on our AudioLink object like so:
After you've done this, we'll want to go ahead and disable spatializing on our the object. This is for the sake of simplicity as it will allow us to hear the music and see how AudioLink reacts anywhere in the world. To remove spatialization, on the same VideoAudioSource object, there will be another script above the VRC AVPro Video Speaker script called the VRC SPatial Audio Source script. in that script, under the Advanced Options drop down, there should be a checkbox that says Enable Spatialization. Go ahead and uncheck that, and that will for the audio source to be 2D instead of 3D.
We are now ready for our first official test! Since we have set up youtube playback inside the editor, we can check for compatibility with the basic video player within the editor by dragging our AudioLinkController into view and seeing if it reacts to the audio. When you are ready, hit the play button and see for any movement from the controller's UI.
If you see the controller reacting like this then that means you have set the controller up properly! The next thing we will want to test is if the stream player also works. Unfortunately, we'll have to do that within the game itself, so leave play mode and get ready to test it in-game!
If you haven't already, make sure that you have a VRCWorld object inside your scene as well as make sure you have all your initial VRChat world settings enabled and go ahead and hit the Build & Test button after you've logged into the VRChat SDK panel
In-game we can see that the video player is still working as expected:
Now we want to go ahead and test the stream player. Switch the player to stream mode as such. This will stop the current youtube video from playing and switch it to stream mode.
Go on twitch or youtube and find a stream that is currently live that is playing music. There are many streams out there that play music 24/7 so its recommended to give one of those a try. For this tutorial, we're going to test the Monstercat TV stream on twitch, which runs 24/7 music from their library of stream-friendly music. So click the URL link and add it to the video player and it should start playing automatically:
And now, after we wait for it to load and play any ads, we should hear the sound coming through and AudioLink reacting to it properly just like the video player!:
Finishing Up
We are now ready to start adding VRSL fixtures to the scene. To recap, this is everything we've done so far:
- We have set up the VRSL Control Panel to be in AudioLink mode
- We have added the AudioLink object and AudioLink Controller to the scene.
- We have added the USharp Video Player and configured it to work in Editor.
- We have connected the USharp Video Player to AudioLink while ensuring both Stream and Video Player compatibility.
If you have ensured you have followed the steps, here is the link to the next chapter/page: