Audio - Multiple listeners for split screen multiplayer?
I'm working as a sound designer on a game in 4.8 that features split screen local multiplayer. From what I can tell there doesn't seem to be any support for multiple audio listeners, so only player 1 can hear 3d sound. Ideally, each player would have their own listener. At the moment we're just having to utilize 2d sound as much as we can but we're reaching a point where it's becoming very difficult. I'm working with Fmod, which definitely supports multiple listeners so any mixing or polyphony issues could be handled from there.
So is there any support in 4.8 for multiple audio listeners? I'm hoping we've just missed something!
asked Jul 28 '15 at 10:55 AM in Using UE4
Hey guys, I'm not 100% sure how FMOD's integration deals with multiple listeners, but the vanilla UE4 audio engine does support multiple listeners. The way UE4 audio deals with 3d audio is that sounds will spatialize relative to the closest listener. I believe this is the behavior of multiple listeners with FMOD too, if I recall when I programmed with FMOD in the past. This makes sense -- otherwise, you'd get crazy double-triggered audio as it played 2 times for each listener.
If you peruse, for example, FAudioDevice in AudioDevice.cpp, you will see that we have a Listeners array that has at least one default listener.
The way UE4 supports multiple listeners for local split screen is the following. The underlying audio device code assumes a single primary listener (0th index in the listener array) but positions sounds relative to their closest listener. In other words, a sounds absolute emitter coordinate (absolute relative to the map coordinates) is used to find the closest listener. Then the sound is "rebased" relative to the closest emitter -- all it's distance attenuations, spatialization, etc, are computed relative to that emitter. But the underlying audio engine doesn't know or care that it's actually spatialized relative to a different listener than the primary listener. I didn't design this system, but I think it's actually pretty clever.
If you know C++, the code that does this rebasing is in ActiveSound.cpp:
The multiple listeners are automatically updated based on the "viewport index" -- i.e. the the viewport on the world. Each viewport automatically has a listener. The code that does this is in GameViewportClient.cpp:
Where SetListener is implemented as:
answered Jul 28 '15 at 08:52 PM
The FMOD integration has the similar logic for determining listener orientation from the player position to the built in audio in UE4. We then pass that info into the FMOD API.
Internally FMOD's support for multiple listeners is done slightly differently to how Aaron describes the inbuilt audio. There isn't anything special about listener 0. Instead we have logic in our panner so that it supports being given multiple positions. The gain is calculated based on the closest listener, but there is also an envelopment that depends on the difference of the relative orientations. That smooths over any changes in panning as the listeners move closer or further out from the emitter.
However, a silly bug crept into in the integration, so the number of listeners was always set as 1. This will be fixed FMOD integration next patch release.
answered Jul 30 '15 at 01:53 AM
Follow this question
Once you sign in you will be able to subscribe for any updates here