Hi, when streaming realtime audio in android, I randomly have a crash after few seconds.
After some investigation in engine source code, I’ve found that this is likely a concurrency problem in SoundWaveStreaming.cpp, where method void USoundWaveStreaming::QueueAudio gets interrupted by a callback which in end empties an array used, causing an exception when invoking: &QueuedAudio[ Position ] (Array index out of bounds)
Basically, I’ve added some more logs, and every crash shows following pattern:
05-07 00:04:43.568: D/UE4(22601): [2015.05.06-22.04.43:571][ 0]LogSoundWaveStreaming:Warning: USoundWaveStreaming::QueueAudio Position:28160
05-07 00:04:43.568: D/UE4(22601): [2015.05.06-22.04.43:572][ 0]LogSoundWaveStreaming:Warning: USoundWaveStreaming::QueueAudio2 Position:28160
05-07 00:04:43.628: D/UE4(22601): [2015.05.06-22.04.43:636][ 0]LogSoundWaveStreaming:Warning: USoundWaveStreaming::QueueAudio Position:30080
05-07 00:04:43.628: D/UE4(22601): [2015.05.06-22.04.43:638][ 0]LogSoundWaveStreaming:Warning: USoundWaveStreaming::GeneratePCMData SamplesNeeded 8192 > SamplesAvailable 16000
05-07 00:04:43.628: D/UE4(22601): Array index out of bounds: 30080 from an array of size 15616
code affected in SoundWaveStreaming.cpp is following:
void USoundWaveStreaming::QueueAudio( const uint8* AudioData, const int32 BufferSize )
{
if (BufferSize == 0 || !ensure( ( BufferSize % sizeof( int16 ) ) == 0 ))
{
return;
}
const int32 Position = QueuedAudio.AddUninitialized( BufferSize );
UE_LOG( LogSoundWaveStreaming, Warning, TEXT("USoundWaveStreaming::QueueAudio Position:%d"), Position);
FMemory::Memcpy( &QueuedAudio[ Position ], AudioData, BufferSize );
UE_LOG( LogSoundWaveStreaming, Warning, TEXT("USoundWaveStreaming::QueueAudio2 Position:%d"), Position);
}
int32 USoundWaveStreaming::GeneratePCMData( uint8* PCMData, const int32 SamplesNeeded )
{
int32 SamplesAvailable = QueuedAudio.Num() / sizeof( int16 );
UE_LOG( LogSoundWaveStreaming, Warning, TEXT("USoundWaveStreaming::GeneratePCMData SamplesNeeded %d > SamplesAvailable %d"), SamplesNeeded, SamplesAvailable);
// if delegate is bound and we don't have enough samples, call it so system can supply more
if (SamplesNeeded > SamplesAvailable && OnSoundWaveStreamingUnderflow.IsBound())
{
OnSoundWaveStreamingUnderflow.Execute(this, SamplesNeeded);
// Update available samples
SamplesAvailable = QueuedAudio.Num() / sizeof( int16 );
}
if (SamplesAvailable > 0 && SamplesNeeded > 0)
{
const int32 SamplesToCopy = FMath::Min<int32>( SamplesNeeded, SamplesAvailable );
const int32 BytesToCopy = SamplesToCopy * sizeof( int16 );
FMemory::Memcpy( ( void* )PCMData, &QueuedAudio[ 0 ], BytesToCopy );
QueuedAudio.RemoveAt( 0, BytesToCopy );
return BytesToCopy;
}
return 0;
}
What happen is:
- While void USoundWaveStreaming::QueueAudio is running, thread gets preempted by callback void FSLESSoundSource::OnRequeueBufferCallback in AndroidAudioSource.cpp, which in turn calls int32 USoundWaveStreaming::GeneratePCMData. (you can see that USoundWaveStreaming::QueueAudio2 never gets printed)
- In int32 USoundWaveStreaming::GeneratePCMData command QueuedAudio.RemoveAt( 0, BytesToCopy ); empties part of queue, so when thread gets back to FMemory::Memcpy( &QueuedAudio[ Position ], AudioData, BufferSize ); in void USoundWaveStreaming::QueueAudio array &QueuedAudio[ Position ] causes exception discussed.
Synchronising two methods above (e.g. making void USoundWaveStreaming::QueueAudio call “monolithic” and not preemptable would probably solve issue, unless there is some better lower level solution in AndroidAudioSource.cpp.
How can I achieve this in unreal engine code? FScopeLock? Any fix?
Yes, I’ve been digging a bit it mac and android code and it’s kind of though to fully understand it. I have been working on voice capture module for android up to now, and I have a mostly working module. You can check 