x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Audio - Multiple listeners for split screen multiplayer?

Hi,

I'm working as a sound designer on a game in 4.8 that features split screen local multiplayer. From what I can tell there doesn't seem to be any support for multiple audio listeners, so only player 1 can hear 3d sound. Ideally, each player would have their own listener. At the moment we're just having to utilize 2d sound as much as we can but we're reaching a point where it's becoming very difficult. I'm working with Fmod, which definitely supports multiple listeners so any mixing or polyphony issues could be handled from there.

So is there any support in 4.8 for multiple audio listeners? I'm hoping we've just missed something!

Thanks, Matt

Product Version: UE 4.8
Tags:
more ▼

asked Jul 28 '15 at 10:55 AM in Using UE4

avatar image

mgriffin89
21 1 2 4

avatar image CelestialPanda Jul 28 '15 at 12:26 PM

I too am interested in how to make audio work better in splitscreen, blueprints preferable.

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

2 answers: sort voted first

Hey guys, I'm not 100% sure how FMOD's integration deals with multiple listeners, but the vanilla UE4 audio engine does support multiple listeners. The way UE4 audio deals with 3d audio is that sounds will spatialize relative to the closest listener. I believe this is the behavior of multiple listeners with FMOD too, if I recall when I programmed with FMOD in the past. This makes sense -- otherwise, you'd get crazy double-triggered audio as it played 2 times for each listener.

If you peruse, for example, FAudioDevice in AudioDevice.cpp, you will see that we have a Listeners array that has at least one default listener.

The way UE4 supports multiple listeners for local split screen is the following. The underlying audio device code assumes a single primary listener (0th index in the listener array) but positions sounds relative to their closest listener. In other words, a sounds absolute emitter coordinate (absolute relative to the map coordinates) is used to find the closest listener. Then the sound is "rebased" relative to the closest emitter -- all it's distance attenuations, spatialization, etc, are computed relative to that emitter. But the underlying audio engine doesn't know or care that it's actually spatialized relative to a different listener than the primary listener. I didn't design this system, but I think it's actually pretty clever.

If you know C++, the code that does this rebasing is in ActiveSound.cpp:

     // splitscreen support:
     // we always pass the 'primary' listener (viewport 0) to the sound nodes and the underlying audio system
     // then move the AudioComponent's CurrentLocation so that its position relative to that Listener is the same as its real position is relative to the closest Listener
     const FListener& Listener = AudioDevice->Listeners[ 0 ];
 
     int32 ClosestListenerIndex = 0;
 
     if (AudioDevice->Listeners.Num() > 0)
     {
         SCOPE_CYCLE_COUNTER( STAT_AudioFindNearestLocation );
         ClosestListenerIndex = FindClosestListener(AudioDevice->Listeners);
     }
 
     const FListener& ClosestListener = AudioDevice->Listeners[ ClosestListenerIndex ];
 
 // SNIP
 
     // if the closest listener is not the primary one, transform CurrentLocation
     if( ClosestListenerIndex != 0 )
     {
         ParseParams.Transform = ParseParams.Transform * ClosestListener.Transform.Inverse() * Listener.Transform;
     }
 

The multiple listeners are automatically updated based on the "viewport index" -- i.e. the the viewport on the world. Each viewport automatically has a listener. The code that does this is in GameViewportClient.cpp:

                                 uint32 ViewportIndex = PlayerViewMap.Num() - 1;
                                 AudioDevice->SetListener(ViewportIndex, ListenerTransform, (View->bCameraCut ? 0.f : GetWorld()->GetDeltaSeconds()), PlayerAudioVolume, PlayerInteriorSettings);
 

Where SetListener is implemented as:

 void FAudioDevice::SetListener( const int32 InViewportIndex, const FTransform& InListenerTransform, const float InDeltaSeconds, class AAudioVolume* Volume, const FInteriorSettings& InteriorSettings )
 {
     FTransform ListenerTransform = InListenerTransform;
     
     if (!ensureMsgf(ListenerTransform.IsValid(), TEXT("Invalid listener transform provided to AudioDevice")))
     {
         // If we have a bad transform give it something functional if totally wrong
         ListenerTransform = FTransform::Identity;
     }
 
     if( InViewportIndex >= Listeners.Num() )
     {
         UE_LOG(LogAudio, Log, TEXT( "Resizing Listeners array: %d -> %d" ), Listeners.Num(), InViewportIndex );
         Listeners.AddZeroed( InViewportIndex - Listeners.Num() + 1 );
     }
 
     Listeners[ InViewportIndex ].Velocity = InDeltaSeconds > 0.f ? 
                                             (ListenerTransform.GetTranslation() - Listeners[ InViewportIndex ].Transform.GetTranslation()) / InDeltaSeconds
                                             : FVector::ZeroVector;
 
     Listeners[ InViewportIndex ].Transform = ListenerTransform;
 
     Listeners[ InViewportIndex ].ApplyInteriorSettings(Volume, InteriorSettings);
 }




more ▼

answered Jul 28 '15 at 08:52 PM

avatar image

Minus_Kelvin STAFF
912 23 4 31

avatar image Ungalyant Sep 18 '15 at 04:10 PM

Out of curiosity, was a system where audio time is progressed but a channel isn't used ever developed?

I'm currently working towards a system where I'd like audio to play based on a world time, the mathematics of how to work out what time that should be is fine, but I'm unable to establish where a piece of audio has its "start playing again" function for the player entering the falloff.

I'd also like to know if there's a simple way to separate the creation of an Audio Listener from GameViewportClient? I'd like to have a second Audio Listener without everything else that comes with the viewport client as it's simply going to be an abstracted 2nd position that the player can listen from (hopefully with the ability to hear from both listeners simultaneously).

avatar image Minus_Kelvin STAFF Sep 24 '15 at 05:53 PM

This should probably be a separate post since it's not exactly related to split-screen.

Out of curiosity, was a system where audio time is progressed but a channel isn't used ever developed?

If you're talking about virtual voices, then no. The new audio engine will support that, but that's not going to be available for a while.

I'm currently working towards a system... ...entering the falloff.

Not really sure what you're talking about here...

I'd also like to know if there's a simple way to separate the creation of an Audio Listener from GameViewportClient? I'd like to have a second Audio Listener without everything else that comes with the viewport client as it's simply going to be an abstracted 2nd position that the player can listen from (hopefully with the ability to hear from both listeners simultaneously).

You probably don't want to hear from 2 listeners at the same time. A "listener" is really only used to calculate spatialization -- to get both the panning and the distance-attenuation values for a playing sound. Usually, for split-screen, as mentioned here, the "closest" listener is used to do the spatialization calculations from. If you heard a sound play for both listeners you would get a ton of double-triggering and I guarantee that it would not only sound very bad but it would use more resources than you expect -- i.e. 2x the number of voices.

As for detaching the listener from the GameViewportClient -- UE4 doesn't support this now but it would be trivial to detach it or make the listener an independent component attachable to any other component. I do not think this would be a good idea for most cases since you'd get issues of sounds being spatialized behind you but seeing the sound-source in front of you, etc.

avatar image Minus_Kelvin STAFF Sep 24 '15 at 05:54 PM

One thing that is a common approach for "listeners" in 3rd person games is to separate the distance-attenuation and panning calculations of the listener. 1) do the distance attenuation based on the distance of sounds to the controlled character and 2) do panning based on position of the camera (i.e. GameViewportClient). This way you avoid the problem of sounds being too quiet when your camera is pulled away from your character (you're always above the action) and you avoid the problem of sounds spatializing behind you when they're actually in front of you (which is unsettling). We don't support this mode in UE4 now but I want to add it at some point. Should be easy.

avatar image Ungalyant Sep 29 '15 at 03:38 PM

Thanks for getting back to me, not sure at this point if I should post a new question or respond here?

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

The FMOD integration has the similar logic for determining listener orientation from the player position to the built in audio in UE4. We then pass that info into the FMOD API.

Internally FMOD's support for multiple listeners is done slightly differently to how Aaron describes the inbuilt audio. There isn't anything special about listener 0. Instead we have logic in our panner so that it supports being given multiple positions. The gain is calculated based on the closest listener, but there is also an envelopment that depends on the difference of the relative orientations. That smooths over any changes in panning as the listeners move closer or further out from the emitter.

However, a silly bug crept into in the integration, so the number of listeners was always set as 1. This will be fixed FMOD integration next patch release.

more ▼

answered Jul 30 '15 at 01:53 AM

avatar image

Geoff Carlton
6 1 1 2

avatar image Minus_Kelvin STAFF Jul 30 '15 at 02:05 AM

Hey Geoff, thanks for replying!

avatar image mgriffin89 Jul 30 '15 at 09:01 AM

Thanks guys! Brilliant news :)

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question