[Audio] MaxConcurrentPlayCount with AutoActivated ambient sounds

vipeout · March 30, 2016, 1:10pm

Hi,

I’ll try describing the issue as short as possible. I have several ambient sounds on map that I want to fire and forget, basic Ambient Sound with Auto Activate enabled (default setting). There may be however many of those on map and I want to do it properly - not spam the audio buffer which causes nasty distortions and other issues. So I set the MaxConcurrentPlayCount to 4.

What I have noticed, is that if there are more AmbientSounds using that specific SoundCue on map than the limit it has, those above the limit will not be fired and will never be fired basically rendering them useless, unless I write my own system that enables sounds as the listener moves around the map, but that’s what the engine should do I believe.

What really happens on the included test map:

First sounds are set up ok and we’re all happy.
Limit is reached
Next sound is not getting activated because of the limit
Next sounds are also never enabled
We have several map environment sounds that do not work

Theoretically it’s what should happen, but thing is, that the check for whether to activate given sound does not take into account how close the listener is and whether the sound will actually be heard or is it too far and will not be heard. In the test map there are 6 sounds, all far enough from player to never be heard. Regardless of that fact, 2 sounds are always broken and do not play.

Sounds that have 0 volume and are cut out should not be included in this particular counter. I want to avoid playing 4 simultaneously, not avoid having more than 4 on the map (this I can do easily myself).

What I’ve also noticed, is that when those sounds Auto Play, the listener is not yet initialized, therefore it’s location equals to (0,0,0) which might mess stuff up as well, unless in the next frame the system reacts accordingly (as if the listener was teleported)

For now I’ve increased the limit very high, but problem is I do not know how many sounds will be on the map. We may reach the limit again someday. Also, it is far from ideal, since with this setting I can fill whole audio buffer with just a single ambient Sound Cue, which is what I wanted to use that variable for - to prevent that from happening.

I have looked a bit here and there, unfortunately found no fix for the issue, thus the post. I hope I explained it clear enough for you to follow and that the sample map will show the issue on your side as well. From what I can tell, you’re rewriting Audio system anyway, so this may be worth looking into for the future releases.

Used binary version 4.9.2 to reproduce the problem. Migration to 4.11p8 proved impossible (heavy Sound Cue changes I see), therefore I’ve recreated the project from scratch. Both versions added. Both versions are bugged in the same way. I used stat soundwaves and stat soundcues to determine whether the sound plays or not. Sound I’ve added in the project is under the Creative Commons 0 License, which means - no copyrights whatsoever.

anonymous_user_6e7b421a · March 30, 2016, 4:58pm

Hello vipeout,

I grabbed this information from a bug report that dealt with Listener and Max Concurrency you might find very informative and verbose.

Until the new concurrency resolution rule, “priority” only came into play when sorting the order of playing voices. Since there’s 32 voices, that meant it didn’t really do anything until more than 32 voices were trying to play so to hear its effect you’d have to have a lot of sounds playing (and thus its hard to really tell whats going on).

The new concurrency feature now uses the same priority value to resolve max concurrency – i.e. if 10 sounds are trying to play with the same shared concurrency group/setting with a max-count set to 9, and the “resolution rule” is set to StopLowestPriority, then it will sort the active sounds playing in that concurrency group according to their priority. Then, if the new sound’s priority is higher than the lowest-sorted sound, then the lowest-sorted sound is stopped and the new sound play-request is allowed to play.

How this interacts with the focus feature is if a sound has focus applied to it, it may be scaling the sound’s focus based on the angle of the 3d sound and the listener (where the focus settings are set with the attenuation settings uobject or overriden locally). What that means is that if a sound tries to play with same concurrency group and an “out-of-focus” sound has its priority reduced a bit, then that out-of-focus sound will have a much higher chance of getting stopped in favor of the new sound (which may be in-focus).

To give concrete numbers, lets take an example of a situation. There are 4 3d looping sounds playing A, B, C, D in the same concurrency setting. (note: easiest to test this if they’re all looping)
The concurrency setting has a max concurrency count of 4 so any new sound in the concurrency group will trigger the resolution rule)

Each sound is using the same sound-attenuation setting object which has the following focus settings:

FocusAzimuth = 15
NonFocusAzimuth = 45
FocusPriorityScale = 1.0
NonFocusPriorityScale = 0.1
(note: 0 azimuth angle is directly in front, 180 is directly behind, and 90 is directly to both sides)

The arrangement of the sounds are:

A = Left (-90 degrees azimuth, out-of-focus)
B = Center (0 degrees azimuth, in-focus)
C = Right (90 degrees azimuth, out-of-focus)
D = Behind (180 degrees azimuth, out-of-focus)

The 4 sound’s individual priority settings are:

Priority_A = 1.0
Priority_B = 2.0
Priority_C = 3.0
Priority_D = 4.0

(note: increasing priority so we can easily predict what we’d expect)

With the focus priority scales taken into account, their scaled resultant priorities are:

ResultantPriority = SoundPriority * FocusPriority
ResultantPriority_A = 1.0 * 0.1 = 0.1
ResultantPriority_B = 2.0 * 1.0 = 2.0 (in focus)
ResultantPriority_C = 3.0 * 0.1 = 0.3
ResultantPriority_D = 4.0 * 0.1 = 0.4

Play a new one-shot sound with same concurrency group with priority 1.0, the resolution rule (described above) triggers and it will stop sound A because it’s now lowest priority.
Stop all sounds and then replay all 4 sounds to reset the situation.

Now, rotate listener/player to be facing a new direction, e.g. A. and do the above math, their priorities are:

ResultantPriority_A = 1.0 * 1.0 = 1.0 (in focus)
ResultantPriority_B = 2.0 * 0.1 = 0.2
ResultantPriority_C = 3.0 * 0.1 = 0.3
ResultantPriority_D = 4.0 * 0.1 = 0.4

Now when the 5th sound to play in the group plays (the one-shot), it will end up stopping sound B instead of A from before.
Repeat in other arrangements and other focus settings.

Cheers,

vipeout · March 31, 2016, 7:09am

Thank you for the detailed response, it will surely come in handy during my dealing with the audio system that’s ahead of me.

Thing is, in my case all the sounds have the same priority - it’s just an environment sound, ambient. I’m making a kind of game where I do not want to additionally specify any focus angles - I want the player to be able to hear the sounds behind him the same as the ones ahead, just attenuated and spatialized properly.

I do also set the listeners location to be different than camera’s location.

Regardless of the above - if I put 6 sounds on map and set max concurrency to 4, 2 sounds will be ignored when auto-playing and will never be enabled by the engine if I come closer to them. The only way it will work is if I manually enable it, which is something I don’t want to have to deal with with ambient sounds that are present on map 24/7.

I have to split the comment, because it’s too long.

vipeout · March 31, 2016, 7:10am

Let me visualize a simple case for my map:

Circles are Ambient Sounds with AutoPlay set to true, star is the listener location, we’re looking at the map from the top. The listener can hear 4 sounds around him.

Let’s skip the part where listener is not initialized and starts at 0,0,0. I have max concurrency set to 4 and have 9 sounds on the map. What happens on begin play is this:

Sounds 1,2,3,4 are within the concurrency limit and are activated
Sounds 5,6,7,8,9 are above the limit and are not activated.
Sound 1 and 4 are too far for the player to hear yet they take up the concurrency slots
Player is within the range of 2 and 3, so these 2 sounds play and you hear them well, but you do not hear 5 and 6 because they were never activated. Already a bugged situation

Now we move and the listener moves to new location and situation is as follows:

This is no longer begin play so we have the situation like this:

Sounds 1,2,3,4 were previously activated. The player in this position can hear only sound 4. They were never turned off.
Sounds 5,6,7,8,9 were never activated and are not being activated when the player moves closer. Therefore the player doesn’t hear 5,7,8. Another bugged situation.

If the player moved to bottom right corner, he wouldn’t hear anything.

I don’t want the engine to turn on/off the sounds that have auto play, I want those to play all the time just as it normally happens, but be cut out from the concurrency rule if they are not heard, ie. they are too far to be heard / have 0 volume and do not contribute to the sound buffer (32 slots).

The example you gave me specifies the behavior when you ‘Play’ the sound. That’s great for setting up SFXes but doesn’t help me at all with my current issue.

Minus_Kelvin · April 4, 2016, 1:10am

Before getting into anything complicated, it doesn’t sound like you’ve tried any of the concurrency resolution rules that involve “resolving based on distance”. I.e. if you hit the max limit, only play the closest sounds. That should help your situation if you haven’t tried it.

For the more robust concurrency system in 4.11, I added new resolution rules (like resolve based on volume-weighted priority then distance, and resolve based on volume). The new concurrency system also supports different sounds using the same concurrency group. It sounds like you’re using the same sound-cue so it should work ok if you use the existing resolve-based on distance resolution rule.

Minus_Kelvin · April 4, 2016, 1:15am

One more thing to note is that pre 4.11, if a sound is “out of range” it’ll actually not play the sound (i.e. technically won’t be taking any concurrency slots). 4.11 actually changes that and will play a sound at 0-volume if there are available (i.e. empty) slots available in the max channel/voice count (which is 32 by default). Stopping all sounds out of range resulted in weird issues with the sound getting restarted as you walk in and out of range of the sound, which can sound very bad in some instances. It also allows for the same sort of behavior you would get with a virtual voice feature and can maintain high-priority sounds that go to 0-volume due to active mixes, etc.

vipeout · April 4, 2016, 6:39am

Before getting into anything complicated, it doesn’t sound like you’ve tried any of the concurrency resolution rules that involve “resolving based on distance”. I.e. if you hit the max limit, only play the closest sounds. That should help your situation if you haven’t tried it.

I did and it is used in the project I have attached. ‘Stop farthest then oldest’. Does not help.

One more thing to note is that pre 4.11, if a sound is “out of range” it’ll actually not play the sound (i.e. technically won’t be taking any concurrency slots).

I might be wrong, but I believe that this is not the case. It does take up slots, because 4.9 and 4.11 project both have the same exact issue. There is no difference between them (or at least a noticeable difference without looking into the code). The result is the same.

Once we migrate to 4.11 I will look into it, but it seems like it’s broken there too, taking the project I’ve attached as an example. If you had some time please try that project out. Maybe I’m missing something, in which case the question will remain here answered properly, or maybe there is something to it.

Thanks for taking the time to reply to me.

Sdruvie · November 6, 2018, 3:28pm

I know this post is old, but I’m having the same issue, and I was wondering if you had found a solution.

vipeout · November 7, 2018, 7:37am

This post is old, the issue still persists in 4.17.2 and I have found no workaround for this. Basically I was ignored or not understood in this thread and decided to give up. What I ended up doing is ignoring the issue and pretending that it works. We shipped the game with that bug in it and are now preparing a console port. There’s simply no time in my schedule to delve that deep into audio system. I can try reproducing it in a clean project for 4.17 if there is will on the Epic’s side to fix it.

Sdruvie · November 7, 2018, 9:56am

I’m on 4.19, and I have the same problem. I’ve tried to find a solution with no luck (the audios with 0 volume seems to take one slot). Changing the concurrency settings like “Stop farthest the oldest” does not help me neither.
I wonder if there is something that I’m missing. Otherwise, how big games do ambient sound? By default it’s 16 slots for the same ambient sound, so, I’m surprised that there aren’t more people with this issue.

Vipeout, have you set up the max count for concurrency to a higher number? Can you tell if this is a major problem for performance?

vipeout · November 7, 2018, 10:19am

Setting it to higher number is a ‘fix’ but I can’t really think of a worse fix - they will take up all available hardware audio slots (default 32) at the start of a map and keep them claimed for the entire duration of the game. Depending on the remaining sound settings - you may never hear any other sound, or a new sound will kill such ambient and the ambient will never play, unless retriggered again.

Yes, ambient sound with start on begin play will take up 1 slot, even if so far away that the volume is 0. Default engine setting is 16 sounds simultaneously, yes. We usually reduce this to 4, but that depends on the sound.

The workaround I can think of is keeping those ambient sounds disabled on start (so the issue does not exist) and detecting the distance yourself and enabling / disabling when you are close to them. This is what the engine basically does (although in a different way) so I didn’t really feel like duplicating it’s job while I have a game to make. I guessed it’s going to be fixed in some revision… apparently I was wrong.

There is AudioQuality value (SetAudioQualityLevel in GameUserSettings) - either expose it to menu for the player to choose (we’ll do it this way one day) or hardcode to something higher. I never tested above 64, though. 64 works ok for PC, consoles is another story - you’ll have to verify with hardware, but I’d stick to 32 there.

As for performance - generally it is ok for PC, you’re not really going to notice audio thread much unless you’re loading sounds late - long (or many) streaming sounds (for movie players or special sound wave setup), or have them encoded and are reading the compressed, encoded version in runtime. We are compressing it - it takes much less space so is loaded faster (shorter loading screen) and takes up less RAM. It is decoded on CPU though, so the more audio channels you allow, the more audio files may need decoding simultaneously. Choose wisely. If you’re PC only and worry about CPU - skip codecs.

Sdruvie · November 7, 2018, 5:03pm

The workaround I’ve done is mostly what you’ve said. It didn’t take more than some minutes to set up, As you said, I have the feeling that I’m doing what Unreal is already doing, but stoping the audio manually by distance empties one slot.

I’ve created a blueprint that only contains the audio and a sphere trigger (with the radius of the max attenuation). When the overlap with the player begins, I start the audio. When it ends, I stop it. I’ve set the audio as a public variable so I can change it from the editor.

I’ll do test performances the next week, but I think it’s not going to be a problem. I hope this gets fixed, or someone found a real solution.