[Crash] Race condition in FDirectoryWatcherWindows destructor

There is a race condition in the destructor of FDirectoryWatcherWindows which causes a crash on exit if any other code does an alertable wait on any kernel handle (such as an external DLL).

Steps to reproduce:

Version is UE 4.1 running DebugEditor on Win64 (Windows 7)

(My original repro involves external code, but you can easily force the race to happen by inserting a few lines into the destructor)

Insert these lines at end of destructor in FDirectoryWatcherWindows::~FDirectoryWatcherWindows DirectoryWatcher.cpp after ensure(NumRequests == 0); line

HANDLE hEvent = CreateEventEx(nullptr, nullptr, CREATE_EVENT_MANUAL_RESET, EVENT_ALL_ACCESS);

while (WaitForSingleObjectEx(hEvent, 10 * 1000, 1) != WAIT_TIMEOUT)
{

}

CloseHandle(hEvent);

This creates an event, does an alterable wait for 10 seconds until it times out.

  1. Run editor loading any project
  2. Exit editor
  3. Crash!

Callstack of crash;

>	UE4Editor-DirectoryWatcher-Win64-Debug.dll!FDirectoryWatchRequestWindows::ProcessChange(unsigned int Error, unsigned int NumBytes) Line 176	C++
 	UE4Editor-DirectoryWatcher-Win64-Debug.dll!FDirectoryWatchRequestWindows::ChangeNotification(unsigned long Error, unsigned long NumBytes, _OVERLAPPED * InOverlapped) Line 271	C++
 	ntdll.dll!KiUserApcDispatcher()	Unknown
 	ntdll.dll!NtWaitForSingleObject()	Unknown
 	KernelBase.dll!WaitForSingleObjectEx()	Unknown
 	UE4Editor-DirectoryWatcher-Win64-Debug.dll!FDirectoryWatcherWindows::~FDirectoryWatcherWindows() Line 43	C++
 	UE4Editor-DirectoryWatcher-Win64-Debug.dll!FDirectoryWatcherWindows::`scalar deleting destructor'(unsigned int)	C++
 	UE4Editor-DirectoryWatcher-Win64-Debug.dll!FDirectoryWatcherModule::ShutdownModule() Line 21	C++
 	UE4Editor-Core-Win64-Debug.dll!FModuleManager::UnloadModule(const FName InModuleName, bool bIsShutdown) Line 442	C++
 	UE4Editor-Core-Win64-Debug.dll!FModuleManager::UnloadModulesAtShutdown() Line 560	C++
 	UE4Editor-Win64-Debug.exe!FEngineLoop::Exit() Line 1860	C++
 	UE4Editor-Win64-Debug.exe!EngineExit() Line 64	C++
 	UE4Editor-Win64-Debug.exe!`GuardedMain'::`2'::EngineLoopCleanupGuard::~EngineLoopCleanupGuard() Line 92	C++
 	UE4Editor-Win64-Debug.exe!GuardedMain(const wchar_t * CmdLine, HINSTANCE__ * hInInstance, HINSTANCE__ * hPrevInstance, int nCmdShow) Line 143	C++
 	UE4Editor-Win64-Debug.exe!WinMain(HINSTANCE__ * hInInstance, HINSTANCE__ * hPrevInstance, char * __formal, int nCmdShow) Line 196	C++

Cause:

CancelIoEx is not an immediate operation. See the docs:

“The cancel operation for all pending I/O operations issued by the calling process for the specified file handle was successfully requested. The application must not free or reuse the OVERLAPPED structure associated with the canceled I/O operations until they have completed. The thread can use the GetOverlappedResult function to determine when the I/O operations themselves have been completed.”

The problem is the destructor is destroying the object that contains the OVERLAPPED data structure for the operation, and the callback gets called during the alertable wait and attempts to write to the deleted object.

Solution:

The destructor should wait until all pending operations are complete before destroying the request objects.

Thank you very much for the detailed description of the problem and solution! It is now fixed in CL#2068271 by waiting for the operation to complete before continuing on to delete the request, as you had described.