UE-63843 - New Audio engine crashing (No steam audio)

Mostly related to UE-63843.
No Steam audio etc, only “AudioDeviceModuleName=AudioMixerXAudio2”

For some reason this happens on a build we run on a win 2016 server, while building locally from source works for my computer …

EDIT: I also posted this into a forum topic I made:
It seems that the reason I get the crashes is the VS version.
The system that compiled a working build used the Visual Studio 2017 14.13.26128 toolchain since I never updated after I installed it, while the other machines had a recent toolchain (Visual Studio 2017 14.15.26726).
(4.20.3 was also tested on one of the “broken” machines, but the fixes did not affect my problem)

Using the old audio engine seems to work.
Editor, standalone from the editor works.

The game just crashes on start and (sometimes) that gets printed to the log:

[2018.09.19-16.50.18:861][  0]LogWindowsTextInputMethodSystem: Display: IME system now deactivated.
[2018.09.19-16.50.18:863][  0]LogContentStreaming: Texture pool size now 1500 MB
[2018.09.19-16.50.19:043][  3]LogAudio: Warning: Waited 71.076149ms for audio thread.
[2018.09.19-16.50.19:079][  3]LogAudio: Warning: Waited 107.021347ms for audio thread.
[2018.09.19-16.50.19:116][  3]LogAudio: Warning: Waited 143.921143ms for audio thread.
[2018.09.19-16.50.19:152][  3]LogAudio: Warning: Waited 179.884277ms for audio thread.
[2018.09.19-16.50.19:156][  3]LogThreadingWindows: Error: Runnable thread AudioThread crashed.
[2018.09.19-16.50.19:157][  3]LogWindows: Error: === Critical error: ===
[2018.09.19-16.50.19:157][  3]LogWindows: Error: 
[2018.09.19-16.50.19:158][  3]LogWindows: Error: Fatal error!
[2018.09.19-16.50.19:158][  3]LogWindows: Error: 
[2018.09.19-16.50.19:160][  3]LogWindows: Error: Unhandled Exception: EXCEPTION_ACCESS_VIOLATION writing address 0xadcbcaad
[2018.09.19-16.50.19:160][  3]LogWindows: Error: 
[2018.09.19-16.50.19:161][  3]LogWindows: Error: [Callstack] 0x00007ff6b0cf10d2 to.exe!<lambda_8946ade225da5b99e0250639c68d5c0c>::operator()() [d:\to4\src\to4_ue4.20\engine\source\runtime\audiomixer\private\audiomixersubmix.cpp:149]
[2018.09.19-16.50.19:163][  3]LogWindows: Error: [Callstack] 0x00007ff6b0d26803 to.exe!Audio::FMixerSubmix::PumpCommandQueue() [d:\to4\src\to4_ue4.20\engine\source\runtime\audiomixer\private\audiomixersubmix.cpp:493]
[2018.09.19-16.50.19:164][  3]LogWindows: Error: [Callstack] 0x00007ff6b0d23113 to.exe!Audio::FMixerSubmix::ProcessAudio() [d:\to4\src\to4_ue4.20\engine\source\runtime\audiomixer\private\audiomixersubmix.cpp:509]
[2018.09.19-16.50.19:165][  3]LogWindows: Error: [Callstack] 0x00007ff6b0d1fdbb to.exe!Audio::FMixerDevice::OnProcessAudioStream() [d:\to4\src\to4_ue4.20\engine\source\runtime\audiomixer\private\audiomixerdevice.cpp:448]
[2018.09.19-16.50.19:165][  3]LogWindows: Error: [Callstack] 0x00007ff6b0d1cf2a to.exe!Audio::FOutputBuffer::MixNextBuffer() [d:\to4\src\to4_ue4.20\engine\source\runtime\audiomixer\private\audiomixer.cpp:128]
[2018.09.19-16.50.19:166][  3]LogWindows: Error: [Callstack] 0x00007ff6b0d30b29 to.exe!Audio::IAudioMixerPlatformInterface::RunInternal() [d:\to4\src\to4_ue4.20\engine\source\runtime\audiomixer\private\audiomixer.cpp:469]
[2018.09.19-16.50.19:166][  3]LogWindows: Error: [Callstack] 0x00007ff6b0d30a2e to.exe!Audio::IAudioMixerPlatformInterface::Run() [d:\to4\src\to4_ue4.20\engine\source\runtime\audiomixer\private\audiomixer.cpp:508]
[2018.09.19-16.50.19:167][  3]LogWindows: Error: [Callstack] 0x00007ff6ae00ed00 to.exe!FRunnableThreadWin::Run() [d:\to4\src\to4_ue4.20\engine\source\runtime\core\private\windows\windowsrunnablethread.cpp:76]
[2018.09.19-16.50.19:167][  3]LogWindows: Error: [Callstack] 0x00007ff6ae009dfd to.exe!FRunnableThreadWin::GuardedRun() [d:\to4\src\to4_ue4.20\engine\source\runtime\core\private\windows\windowsrunnablethread.cpp:25]
[2018.09.19-16.50.19:168][  3]LogWindows: Error: [Callstack] 0x00007ffe26d63034 KERNEL32.DLL!UnknownFunction []
[2018.09.19-16.50.19:168][  3]LogWindows: Error: [Callstack] 0x00007ffe297f1461 ntdll.dll!UnknownFunction []
[2018.09.19-16.50.19:169][  3]LogWindows: Error: 
[2018.09.19-16.50.19:169][  3]LogWindows: Error: Crash in runnable thread AudioThread
[2018.09.19-16.50.19:188][  3]LogExit: Executing StaticShutdownAfterError
[2018.09.19-16.50.19:188][  3]LogAudio: Warning: Waited 215.857651ms for audio thread.
[2018.09.19-16.50.19:190][  3]LogWindows: FPlatformMisc::RequestExit(1)
[2018.09.19-16.50.19:199][  3]Log file closed, 09/19/18 18:50:19

When attaching to the process this is what vs prints

>	[Inlineframe] TO.exe!FWindowsPlatformAtomics::InterlockedDecrement(volatile int *) Zeile 65	C++
 	[Inlineframe] TO.exe!SharedPointerInternals::FReferenceControllerOps<1>::ReleaseSharedReference(SharedPointerInternals::FReferenceControllerBase *) Zeile 279	C++
 	[Inlineframe] TO.exe!SharedPointerInternals::FSharedReferencer<1>::{dtor}() Zeile 445	C++
 	[Inlineframe] TO.exe!TSharedPtr<Audio::FMixerSubmix,1>::operator=(const TSharedPtr<Audio::FMixerSubmix,1> &) Zeile 755	C++
 	TO.exe!Audio::FMixerSubmix::AddChildSubmix::__l2::<Lambda>() Zeile 149	C++
 	[Inlineframe] TO.exe!UE4Function_Private::TFunctionRefBase<TFunction<void __cdecl(void)>,void __cdecl(void)>::operator()() Zeile 290	C++
 	TO.exe!Audio::FMixerSubmix::PumpCommandQueue() Zeile 493	C++
 	TO.exe!Audio::FMixerSubmix::ProcessAudio(const ESubmixChannelFormat ParentChannelType, TArray<float,TAlignedHeapAllocator<16> > & OutAudioBuffer) Zeile 509	C++
 	TO.exe!Audio::FMixerDevice::OnProcessAudioStream(TArray<float,TAlignedHeapAllocator<16> > & Output) Zeile 448	C++
 	TO.exe!Audio::FOutputBuffer::MixNextBuffer() Zeile 128	C++
 	TO.exe!Audio::IAudioMixerPlatformInterface::RunInternal() Zeile 469	C++
 	TO.exe!Audio::IAudioMixerPlatformInterface::Run() Zeile 508	C++
 	TO.exe!FRunnableThreadWin::Run() Zeile 76	C++
 	TO.exe!FRunnableThreadWin::GuardedRun() Zeile 25	C++

I made an edit since I think I found the “reason” for my current problem…

Can you link the forum topic you made? I’m getting a very similar interlocked decrement crash in the audio engine.

The forum thread was unresolved.
The issue seems to be related to the toolchain I used and EPIC already added a workaround to the engine for 4.21.