Unreal Engine Defaults to Headless Mode

Build Type: GitHub. Source build of stable branch.

Build version: 4.19 (GitHub 4.19 branch commit 3f28d8781885eb16b23fec92dffee0f47ecc0e6e)

Detailed description of the issue: Since 4.18, opening any Unreal Project that uses C++ fails with a completely unhelpful error message.

This includes projects created with 4.18 or 4.19 as well as projects upgraded from 4.17 and earlier (the project upgrade system is broken as well, see https://answers.unrealengine.com/questions/718575/, but this is a separate issue).

Looking at the source code, the text No is implied. is only printed by GetHeadlessMessageBoxMessage() which itself is only called if SDL would fail to initialize:

https://github.com/EpicGames/UnrealEngine/blob/8e4560b8c22b309e73ff0ce90377742c3dfe13cc/Engine/Source/Runtime/ApplicationCore/Private/Linux/LinuxPlatformApplicationMisc.cpp#L61-L67

https://github.com/EpicGames/UnrealEngine/blob/8e4560b8c22b309e73ff0ce90377742c3dfe13cc/Engine/Source/Runtime/ApplicationCore/Private/Linux/LinuxPlatformApplicationMisc.cpp#L205-L213

Do observe, however, that SDL does initialize successfully in the attached screenshot and that other SDL applications are running fine on this system.

Repro Steps:

  1. Download Unreal Engine 4.18 or Unreal Engine 4.19 from GitHub on any Linux system
  2. Build it (Setup.sh, GenerateProjectFiles.sh, make)
  3. Run Unreal Engine, in the project/template window, create a new C++ project
  4. The project will be created but not built, thus fail to open. No error message will be displayed.
  5. Any attempts at opening the project thereafter yield the unhelpful error message below.

Occurs 100% of the time.

System Specs:
Gentoo Linux, 2x Xeon E5-2680, 16 GiB RAM, GTX 1080
KDE Plasma 5.11.5, NVidia proprietary drivers 390.42
GCC 6.4, CLang 5.0.1 w/LLD 5.0.1

It is the second of the two code locations I posted, SDL_ShowMessageBox(), that fails. The SDL function returns -1.

The error message reported by SDL is “No message system available.”

https://github.com/EpicGames/UnrealEngine/blob/8e4560b8c22b309e73ff0ce90377742c3dfe13cc/Engine/Source/ThirdParty/SDL2/SDL-gui-backend/src/video/SDL_video.c#L3960-L3969

A workaround for those who don’t mind ugly hacks:

  1. edit Engine/Source/Runtime/ApplicationCore/Private/Linux/LinuxPlatformApplicationMisc.cpp

  2. add the following code snippet near line 208:

    if (SDL_ShowMessageBox(&MessageBoxData, &ButtonPressed) == -1)
    

    {
    FString Message = GetHeadlessMessageBoxMessage(MsgType, Caption, Text, Answer);
    UE_LOG(LogLinux, Warning, TEXT("%s"), *Message);

        UE_LOG(LogLinux, Warning, TEXT("Answer yes? (Y/N)"));
        char answer[10];
        scanf("%s", answer);
        if((answer[0] == 'y') || (answer[0] == 'Y')) {
                Answer = EAppReturnType::Yes;
        } else if((answer[0] == 'n') || (answer[0] == 'N')) {
                Answer = EAppReturnType::No;
        }
    

    }

Launching Unreal Engine from the shell, this will then present you with a prompt to answer Y or N. Answering Y will compile the C++ project, open the editor and everything will work from there.

Hello,

We’ve recently made a switch to a new bug reporting method using a more structured form. Please visit the link below for more details and report the issue using the new Bug Submission Form. Feel free to continue to use this thread for community discussion around the issue.

https://forums.unrealengine.com/unreal-engine/announcements-and-releases/1410408-unreal-engine-bug-submission-form

Thanks

While you seem to have worked the problem around, could you please try breaking into X11_ShowMessageBox() and seeing why it fails? There must be something specific to your X server that SDL2 doesn’t handle. What are you running - X.Org, XWayland, anything else?

I’m running vanilla X11 R7.7 (X.Org server 1.19.5) with KDE Plasma desktop 5.11.5. I’ll try to debug into the SDL code to see which of the calls to the windowing system fails.

in SDL_x11messagebox.c, there’s a function X11_ShowMessageBox(), which for some reason calls fork and then tries to read from the piped process (line #858). The call to read() from the pipe is what fails.