[4.15.2] Resizing while a media is open crash game

Hi,
i recently ported to 4.15.2 (from 4.14.3 ) and i discovered that if i resize the viewport while simulating on my main menu ( for all selected viewport, new editor viewport (pie), standalone modes)

after investigation, it seam it due to a media player opened on the menu ( i have a background video ), unlinking the “open media” node avoid crash

Crash is “random” but after 5 or 6 resize i almost always get it to crash.
No UE crash windows appear, only Windows “game stop working”
Attaching a debuger got me this :

Assertion failed:
IsInRenderingThread()
Sync\Engine\Source\Runtime\MediaAssets\Private\Misc\MediaTextureResource.cpp
[Line: 564]

/Saved/Log file end with :

[2017.05.05-12.15.02:355][740]LogRenderer: Reallocating scene render targets to
support 676x384 NumSamples 1
(Frame:2815).
[2017.05.05-12.15.02:436][740]LogRenderer: Reallocating scene render targets to
support 1140x644 NumSamples 1
(Frame:2816).
[2017.05.05-12.15.02:512][740]LogRenderer: Reallocating scene render targets to
support 1740x980 NumSamples 1
(Frame:2817).
[2017.05.05-12.15.02:577][740]LogRenderer: Reallocating scene render targets to
support 1808x1016 NumSamples 1
(Frame:2818).
[2017.05.05-12.19.54:194][740]LogRenderer: Reallocating scene render targets to
support 1800x1012 NumSamples 1
(Frame:2819).

other info : engine binaries are from luncher and it’s a c++/blueprint project

I did a blueprint only test project and it crash too, i can send it to a dev if needed

i don’t know if it was working good in 4.14 ( but i think i would have noticed it )

Hey Firefly74,

If you could provide me with the test project in 4.14.3, I could test there and upgrade it to 4.15.2 and check against that version.

From reading the logs, it looks like the resizing is also modifying Render Targets which requires memory, and that memory consumption might not be getting cleared when you resize each time. In turn, this eventually fills up your VRAM and causes your project to run out of memory and crash.

I would need a way to reproduce it on my end to know for sure. Along with the project, could you also send me your ‘dxdiag’?

Thank you,

Hi Andrew !

No need to migrate, i reproduced the crash on a clean blueprintonly 4.15.2 project,

here it is :

here the dx diag :

steps:

Open crashVideoTest.uproject

Load MainMenu.umap

Play (in current viewport or PIE )

Resize viewport ( don’t be scared to try hard, play with size without releasing mouse, sometime it take some retries )

Same problem here in 4.15.1 while trying to switch between stereo rendering on and off. Hoping to see a fix for this!

Edit: here is the call stack

UE4Editor-MediaAssets.dll!FMediaTextureResource::ConvertResource(const FMediaTextureResource::FResource & Resource) Line 564	C++
UE4Editor-MediaAssets.dll!FMediaTextureResource::SwapResource() Line 808	C++
UE4Editor-MediaAssets.dll!FMediaTextureResource::DisplayBuffer() Line 78	C++
UE4Editor-MediaAssets.dll!UMediaTexture::DisplayTextureSinkBuffer(FTimespan __formal) Line 170	C++
UE4Editor-WmfMedia.dll!FWmfMediaTracks::HandleMediaSamplerSample(const unsigned char * Buffer, unsigned int Size, FTimespan Duration, FTimespan Time, EMediaTrackType TrackType) Line 1304	C++
UE4Editor-WmfMedia.dll!TBaseRawMethodDelegateInstance<0,FWmfMediaTracks,void __cdecl(unsigned char const * __ptr64,unsigned int,FTimespan,FTimespan),enum EMediaTrackType>::ExecuteIfSafe(const unsigned char * <Params_0>, unsigned int <Params_1>, FTimespan <Params_2>, FTimespan <Params_3>) Line 648	C++
UE4Editor-WmfMedia.dll!TBaseMulticastDelegate<void,unsigned char const * __ptr64,unsigned int,FTimespan,FTimespan>::Broadcast(const unsigned char * <Params_0>, unsigned int <Params_1>, FTimespan <Params_2>, FTimespan <Params_3>) Line 937	C++
UE4Editor-WmfMedia.dll!FWmfMediaSampler::OnProcessSample(const _GUID & MajorMediaType, unsigned long SampleFlags, __int64 SampleTime, __int64 SampleDuration, const unsigned char * SampleBuffer, unsigned long SampleSize) Line 101	C++
[External Code]

This is a known issue. It will be fixed in Media Framework 3.0, which is currently scheduled for 4.18. Unfortunately, it is not a quick fix that you can merge, because it required a redesign of how video textures are processed.

Glad to hear that there is a fix for this in an upcoming release, but do you have any tips or workarounds for projects that need to ship prior to 4.17’s release? Thanks for the help!

I don’t have a workaround, but I can tell you what the problem is. The following code does not work as intended while resizing the viewport:

void FMediaTextureResource::DisplayBuffer()
{
	if (State != EState::Initialized)
	{
		return;
	}

	if (IsInRenderingThread())
	{
		SwapResource();
	}
	else
	{
		RenderThreadTasks.Enqueue([this]() {
			SwapResource();
		});
	}
}

The problem is that during resolution changes, the render device is being recreated. During this short period of time, the IsInRenderingThread() function returns true for all threads, including threads the Engine doesn’t even own! That is because it has the following unfortunate definition:

CORE_API bool IsInRenderingThread()
{
	return !GRenderingThread || GIsRenderingThreadSuspended || (FPlatformTLS::GetCurrentThreadId() == GRenderingThread->GetThreadID());
}

During viewport resizing, the end result is true no matter what. This is certainly a bug, and it’s on the to-do list, but so far only the Media Framework is affected, so it’s not super high priority right now.

There are two possible solutions for the 4.15 Media Framework case, neither of which we have actually explored, because the DisplayBuffer code (and most of the remaining API in MediaTextureResource) actually goes away in MF 3.0.

The first is to check in DisplayBuffer whether the render threads are suspended. This sounds easy, but it can be a little tricky (and probably quite ugly), because we must prevent the Render thread from transitioning while we’re inside DisplayBuffer, so some kind of global/shared lock is required.

The second is to suspend the thread that DisplayBuffer is running on while the Render thread is transitioning. This is also not trivial and may not be possible on all platforms.

If I had to chose, I’d hack in a global scope lock. Of course, it would be best if you could wait a couple more months, and then this problem will just go away.

Thank you for the prompt reply! Not quite what I’d hoped to hear, but good to be pointed in the right direction. Thanks for all of your help!

Unfortunately, that is not a complete solution. You also need to prevent the render thread from suspending while the code inside the if-statement is running. That is the hard part.

By checking for GIsRenderingThreadSuspended before executing the sample rendering code, you have shrunk the window of opportunity in which crashes can happen. It is less likely to crash, but it’s still possible.

Hi guys,
I changed

if (IsInRenderingThread())
To
if (IsInRenderingThread() && !GIsRenderingThreadSuspended)

And I didn’t get crash reports related to FMediaTextureResource::DisplayBuffer since 10.08.2017.
@gmpreussner what do you think about it ?

Regards
Piotr

Do you guys have an ETA on when this bug may be fixed? Is it in issues.unrealengine.com?

well, he said 4.18, targeted release is Q4 2017, i don’t think we will get more infos…
Let’s hope media framework make it to this release though …

Oh I thought this question was from 2015 lmao! I was thinking that 4.18 was an oddly specific ETA haha!