How to Send Voice over Unity Networking - UNET

Question

I am able to Record and Run Audio from my Microphone using this code snippet:

void OnGUI() {
        if (GUI.Button(new Rect(0, 0, 100, 20), "record")) {
           // Microphone.End(Microphone.devices[0]);
            audsourceAttached.clip = Microphone.Start(Microphone.devices[0], false, 3, 44100);
        }
        if (GUI.Button(new Rect(0, 30, 100, 20), "Stop"))
        {
            //Microphone.End(Microphone.devices[0]);
            audsourceAttached.Play();
        }
    }

Now I am willing to send this audio across Network (on my local network-another instance of the game). How can I do this?

I tried to find this on google but unable to find any useful solution/links.

How can I send audio through microphone like real-time Chat in a game/etc.

Coburn · Answer

TL;DR: You can't do it natively in Unity

Microphone.Start returns an AudioClip which has 3 load types, none of which fit your use case. All of them (even the .Streaming one) require the entire clip to be on the users computer already. Using AudioClip.GetData() and sending a file over the network (in Unity, using Networking.NetworkTransport.Send()) would be horribly unresponsive and can only actually occur after the recording finishes its clip. This ignores audio codecs, using multiple threads (Unity isn't thread safe), and dealing with the real time nature of audio device data.

What you need is a way to get the audio device data in real time which Unity provides no mechanism for and therefore cannot support RTC natively. This answer comes to the same conclusion

Third party solutions exist on the asset store. Here's one such solution mentioned in the above linked answer: Dissonance Voice Chat. Going through the files list it uses a native .dll, the Opus audio codec, and a bunch of C# files for interfacing with the native code.

Rolling your own, which I do not recommend, would require doing the following:

Using a separate C# System.Threading thread to process your audio so that your framerate and audio playback aren't coupled together.
Getting the audio data isn't trivial in C#. .NET uses DirectSound which is a Windows things.  Mono doesn't support this. This means you either need to implement a cross platform solution yourself natively, or rely on a library that already does.
Transporting it over the network requires a socket using .NET sockets. These seem robust and are supported in Mono.
Your sockets should transport as a P2P multicast. This allows you to send one audio stream to multiple people all at once. It also skips a setup like Client ==> Server ==> Client which adds extra latency.
You would also want a codec to make sure that you're not just sending uncompressed audio willy-nilly. There are many out there which most likely requires interfacing with another native library.

How to Send Voice over Unity Networking - UNET

One Answer

Add your own answers!

Ask a Question