I am creating functional tests for blueprints using the automation framework. The issue I’m having is that every functional test generates a “game preview” window that must be closed before the next test can be run.
How can tests be set up so that all tests can be run without requiring user any interaction?
Example Scenario
Create two blueprint functional tests based on the “FunctionalTest” blueprint class.
Add both tests to a test level.
Open the Session Frontend → Automation tab.
Select the level. (All tests in the level are automatically selected.)
Use either “Start Tests” or “Run Level Tests” to start the tests.
What Happens
A game preview window opens up for the first test. That first test completes but I have to close the preview window before the next test will even start. So if a level contains 20 tests, I have to close 20 game preview windows in order to run all of the tests.
Hi SteveFSP: for clarity, closing the preview window is not considered a cancellation by the FTest framework. To stop the currently-running tests when using the Automation Frontend, you need to click the Stop Tests button.
The issue turned out to be a bug in one of the tests that was hidden by a bug in the automation framework.
The test bug was that one of the completion paths did not end in a “Finish Test” node. So the test never completed. This was hidden by the fact that the all tests were showing as having succeed. Basically, the automation framework can show a test as successful even when the test is cancelled by closing the game preview window before the test is marked as finished.
One way to avoid running into this issue is to make sure the timeouts on all tests are reasonable. (The default timeout is 60 seconds.) Then always wait for the timeout so you don’t risk triggering the framework bug.
just figured this one out. in a blueprint derived from FunctionalTest there is a TimeLimit property that you can set. i believe the actual call is SetTimeLimit.