How do you write and run C++ tests?

Development Programming & Scripting
1

I’ve had a read of https://docs.unrealengine.com/latest/INT/Programming/Automation/TechnicalGuide/index.html

…and I’ve found a few things in the forums about it, like Need Help with Automation Testing in 4.6 - C++ - Unreal Engine Forums

However, I still fail to understand a few basic things about tests for the engine.

  1. There are lots of flags for tests in Engine/Source/Runtime/Core/Public/Misc/AutomationTest.h. Which ones are for what?

Specifically, for example, say I have:

  • A set of blue print functions, that I want to test outside the editor what is the appropriate flag?
    (eg. Foo(a, b) → A + B; I want to assert the behavior of this function is correct; it has no external dependencies beyond its direct inputs)

  • A component that does various things on update and I want to create an actor that has the component associated with it, and then assert some in-game behaviour for it. What flag is appropriate? (ATF_Game?)

At a technical level, what do these flags actually do?

  1. **Why does the hot reload not reload tests? **

I literally have to restart the editor to get the automation tool to load the new versions of the tests.

I’ve tried disconnecting from the session, reconnecting, etc. and no luck.

  1. Given that you have to restart the editor to rerun tests, how do you invoke them from the command line?

There’s some suggestion you can use the RunUAT tool for this, but I can’t find what combination of flags to pass it.

I do not want to generate a build of the whole project, and assume that I have already used Build.sh to confirm that the tests compile; I literally just want to run the automation tool and get the results (ideally in a text format that can easily be parsed).

2

fwiw if anyone else finds this question, the closest I ever got to an answer was: unreal engine4 - How do you run tests from the command line? - Stack Overflow

Didn’t work for me.

Just write your own tests in your own C++ test framework and trigger them yourself manually.

3

Are you just mocking the UE objects?

4

I would love an answer to this, I have been trying to figure out automation testing for the last day and a half, and my test is currently building, but I can’t find it anywhere in the automation tool.

5

A set of blue print functions, that I want to test outside the editor what is the appropriate flag? (eg. Foo(a, b) → A + B; […])

This is a super fast test, so you can add SmokeFilter. Also, you don’t require any context, so you can pass the entire ApplicationContextMask rather than specifying this test can be run in a Game context, Editor context etc.

Your final flags parameter would therefore look like this: ApplicationContextMask | SmokeFilter.

The in-code documentation in AutomationTest.h of the flags is actually pretty good for these enum values.

Why does the hot reload not reload tests?

There is a bug open for this: Unreal Engine Issues and Bug Tracker (UE-25350) and marked as “backlogged”, so fingers crossed that this will work some day.

Given that you have to restart the editor to rerun tests, how do you invoke them from the command line?

Find the answer to that here: Running Command Line Automation Tests 4.10? - C++ - Epic Developer Community Forums