Media Framework - WmfMedia audio lag

We are experiencing a persistent audio lag (about 0.5 to 1.0 second delay of audio track compared to video track) when playing any mp4 video using the Media Player. This happens even when opening the Media Player asset in the editor and playing any of the Media Source from there (without actually going into play mode).

UE version is 4.20.1. We were able to reproduce this on 3 different PCs and using several different videos, including this one:
https://dnnrz1gqa.blob.core.windows.net/portals/attachments/Engine/MediaFramework/HowTo/StreamMediaSource/Infiltrator%20Demo.mp4?sr=b&si=DNNFileManagerPolicy&sig=F%2BBpnYueeQTUTSW0nCRSrXEfr35LIawe7C3MQoe2%2FPI%3D

Using File Media Source instead of Stream Media Source (after downloading video on disk) doesn’t change anything.

I tried every setting I could find with no success. The only setting that made a difference is “Native Audio Out” in the WmfMedia plugin project settings. It did eliminate the lag, but prevent Pause from working (using “Pause” BP node on the Media Player returns false and video continues to play) so this is not a solution for us unfortunately.

For reference, here is our setup for using Media Player in play mode (though I doubt it has any relevance):

  • Media Player asset
  • A Media Texture points to Media Player asset
  • UMG Widget has an Image that displays the Media Texture above
  • On scene there is an Actor with MediaSoundComponent that points to the same Media Player
  • In play mode user interacts with UMG widget and initiates video playback by clicking

Lag is the same no matter if it’s in actual standalone built game, in the MediaPlayer window in Editor or in PIE.

Has anyone else experienced this? Is there any solution other than writing custom Decoder Plugin?

Edit: tried using the current master of VlcMedia plugin with no success. The lag remains but with this decoder videos appear to be distorted now. Looks like this issue may be related to some core UE4 code for playing the audio and not the decoders.

1 Like

Same here, 4.20.2. And even that “Native Audio Out” is not usable because it not work in shipping build at all.

I recommend you to use this: Unreal Engine Community

have you found a solution for this? I’m experiencing the same on 4.20.3. WmfMedia. If native out selected there is no latency but unfortunately that does apply to a shipping build.

If I remember correctly then I found no soulution how to fix that. I had to play movie without sound and at the same time start playing wav with movie sound (previously separated from movie). Not ideal, but usable.

Here’s a potential solution.

I had a terrible lag that got worse as the video progressed. So I tried a different video format. I found that when I switched from H264 MP4 to H265 (HEVC), the audio lag disappeared and everything was nicely synched.

Hi I have similar issues… for me de-sync arise when pausing and restarting the video. I get different behaviors on console and I was able to put some duck tape around the issue, but long term an improvement to MediaUtils has to be done.

I investigated a bit the code and I would like to make the following suggestion to Epic :

Normally, when implementing a/v sync for codec, the video sink needs to slave to the audio sink. The reason for this is that it is less noticeable to skip a video frame than to skip an audio frame (it will cause discontinuities in the signal and pop or glitch).

Looking at UMediaSoundComponent, the opposite was implemented. By setting DynamicRateAdjustment to true, you would allow the audio track to adjust it’s playback rate according to the video sink. When you slave audio to video and dynamically resample it, this will create pitch fluctuation in the audio (unless you implement a time stretching algorithm, and even that would introduce artefacts). This could be ok to implement AV sync this way if you have only speech in your audio signal, however if you have music playing this could affect the pitch of your music and cause weird artifacts.

Looking at the architecture of MediaPlayerFacade, it allows to add multiple video sinks and multiple audio sinks. Either, epic remove the functionality of having multiple video sinks and audio sinks and makes the video sink slave to the audio sink… or they come up with the concept of audio master sink and make it possible to video sinks to synchronize to the audio sink.

Eventually if Epic does not come with a fix, I will fix it myself, however since we will integrate other version of UE4 in our code base before shipping, I can live with the duck tape solution for now.