Downloading audio files from Azure blob storage and playing them in Mixed Reality apps

Intro

All but the most trivial apps have some kind of back end. In the past, I have written about accessing all kinds of services from HoloLens and Mixed Reality apps, But apart from mere data, you can download all kind of media from external sources and use those in you Mixed Reality apps. This is highly useful if you want to change used media, use different files for like instructions depending on some factor. Or heck, some random background music. In fact, what I will be describing is not even Mixed Reality specific – the principle can be used in any Unity app, although the code that sits around to demonstrate the workings definitely only works for Mixed Reality.

Oh, and by the way – for us Microsoft geeks “back end” equals to “Microsoft Azure” – if it’s not for the competitive pricing, then for the way Microsoft makes it easy for developers to get going (heaven knows this was quite different in the early days). But to be clear: this will work with any backend that hosts files.

This post will be 2-part: the first part concentrates on the actual technique of downloading and playing audio files, the second part will explain the ‘floating audio player’ I built to make this easily demonstrable. The floating part in ‘floating player’ should be taken in the most literal way possible:

You can find it in the demo project. You can just jump to there if you want to skip all my mumbling.

Global overview

If you open the project in Unity, you will not see one but three floating players, next to each other. They will all attempt to play on of my ringtones – an excerpt from the Doctor Who theme song (one is a nerd or one is not). But it will attempt to use one of three different audio formats – the well-known MP3, Ogg Vorbis, and WAVE audio format (aka ye good ole’ WAV).

To hear them, either build the app or just hit the play button. If you choose the second option, you will need to attach an XBox One controller to you PC, to steer the cursor and click on the buttons. You will notice the Ogg and the WAV file playing nicely. The mp3 one won’t. In fact, the player just disappears before you even can get to it. We will get to that later.

Using UnityWebRequest

In previous posts I have shown you either how to use the Unity WWW class, or how to resort to pure UWP code and use HttpClient/HttpRequestMessage. It seems like the WWW class is being deprecated, although I have no official information on that. However,the new (third) kid on the block seems to be UnityWebRequest.

This is a bit on an oddball. It uses a handler. Generally, a UnityWebRequest looks like this:

var request = UnityWebRequest.Get(url);
request.downloadHandler = handler;
yield return request.SendWebRequest();

The second line is optional – if the you don’t set the handler, it’s the default DownloadHandler class, which sports a “text” property you can query. This is very useful for accessing data services like an API app on Azure App Service. If you want to download audio files, you will need a DownloadHandlerAudioClip handler.

Putting it in code

To make this all work easily, I have created the following base class for download loading media:

using System.Collections;
using UnityEngine;
using UnityEngine.Networking;

public abstract class BaseMediaLoader : MonoBehaviour
{
    public string MediaUrl;

    private string _currentMediaUrl;

    protected virtual void Update()
    {
        if (_currentMediaUrl != MediaUrl)
        {
            _currentMediaUrl = MediaUrl;
            StartCoroutine(StartLoadMedia());
        }
    }

    protected abstract IEnumerator StartLoadMedia();

    protected IEnumerator ExecuteRequest(string url, DownloadHandler handler)
    {
        var request = UnityWebRequest.Get(url);
        request.downloadHandler = handler;
        yield return request.SendWebRequest();
    }
}

MediaUrl is an url that points to a place where the actual audio file can be downloaded – in this case, as SAS link to a file in my own Azure blob storage. Every time Update is called, it checks if the MediaUrl has been changed, and if so, it starts the StartLoadMedia in the background. ExecuteRequest is a helper method that can be used from a StartLoadMedia. Like this:

using System.Collections;
using UnityEngine;
using UnityEngine.Networking;

public class SoundPlaybackController : BaseMediaLoader
{
    public AudioSource Audio;

    public AudioType TypeAudio = AudioType.OGGVORBIS;

    protected override IEnumerator StartLoadMedia()
    {
        yield return LoadMediaFromUrl(MediaUrl);
    }

    private IEnumerator LoadMediaFromUrl(string url)
    {
        var handler = new DownloadHandlerAudioClip(url, TypeAudio);
        yield return ExecuteRequest(url, handler);
        if (handler.audioClip.length > 0)
        {
            Audio.clip = handler.audioClip;
        }
    }
}

This is a very reduced version of the SoundPlaybackController that is in the demo project, concentrating only on the actual downloading and playing of the data.

The override of StartLoadMedia simply calls StartLoadMedia, which proceeds to create a DownloadHandlerAudioClip handler. Now the odd thing is, this handler wants and url as well as the request. Why this is so, I have no idea. Also notice the fact you need to supply the handler with the type of audio you are going to download – there’s a AudioType enumeration for that.

If the audio has been downloaded successfully, you only have to set the “clip” property of an AudioSource to the handler’s “audioClip” property, and call the Audiosource’s “Play” method. And you are good to go.

Audio types are important

Since the TypeAudio property is public, the Unity editor makes a nice dropdown for us to select the type of audio:

I’ll be the first one to admit I haven’t heard of most of these file types, let alone know them. Actually, before I started, I only knew MP3 and WAV. I learned to know Ogg Vorbis. And for a good reason too. I already mentioned the fact the left (mp3) player disappears when you start the code. The SoundPlaybackController that I actually created (not the simple version above) hides the UI of the whole player while actually downloading the audio file. You might have noticed an error in the Unity status bar when you run the code. That actually says:

Streaming of 'mp3?st=2018-02-02t15%3a51%3a00z&se=2020-02-03t17%3a51%3a00z&sp=rl&sv=2017-04-17&sr=b&sig=kymvql5q1yyr%2bqilcxhfc3popwo56vd0ejibennldzw%3d' on this platform is not supported
UnityEngine.Networking.DownloadHandlerAudioClip:get_audioClip()
<LoadMediaFromUrl>c__Iterator0:MoveNext() (at Assets/App/Scripts/SoundPlaybackController.cs:34)
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

The code in that player crashes, and never gets to showing the UI again. Well, that’s a bummer. Unity simply does not seem to support MP3 on ‘this platform’, which is apparently UWP. You can convert it to WAV, but unless you like to burn a lot of battery downloading stuff, you must be pretty much out of your mind doing so – Ogg Voribs is a much better option, as this simple list of files shows.

And Audacity, that good old workhorse of audio-artists and podcasters all around the globe makes conversion easy, so why not use it, right.

Conclusion

The very short version of this blog post: use UnityWebRequest and DownloadHandlerAudioClip to download an audio clip from Azure blob storage, and if you value your users’ bandwidth, their devices’ battery life, and your own sanity – use Ogg Vorbis audio files.

I have not tried all the other audio files types, simply because I did not have to do so – Ogg Vorbis works fine. WAV too, but is absurdly big compared to MP3 and Ogg Vorbis. In addition, I have no idea what those audio types are, how I should create/convert them, and why I should use them. To paraphrase Star Trek’s TOS medical officer Dr. Leonard “Bones” McCoy – I am a developer, not an audio engineer. 

Next time, I will explain the workings of the floating audio player around this code.