x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Office Holiday

Epic Games' offices will be on holiday from June 22nd to July 7th. During this period support will be limited. Our offices will reopen on Monday, July 8th. 

Hot-reload does not re-compile Automation Tests

I have also tested this in 4.9.

Repro:

  1. Create C++ project

  2. Create a simple automation test as per this gist.

  3. Run the test via the in-editor Frontend and observe the console output.

  4. Change the UE_LOG output to some other phrase.

  5. Hit the 'compile' button in-editor.

  6. Run the test again and notice that it has the same output as before the hot-reload.

You can try this with any sort of change if for some reason console outputs aren't hot-reloaded by default. The results are the same.

This is a huge problem for the speed and practicality of conducting regular unit tests. At the moment, I have to reboot the editor every single time I change a test. It takes too long to be practical or useful.

Thanks!

Product Version: UE 4.11 Preview
Tags:
more ▼

asked Jan 12 '16 at 05:51 PM in Bug Reports

avatar image

HateDread
453 41 47 72

avatar image Doug E ♦♦ STAFF Jan 13 '16 at 02:36 PM

Hey HateDread-

Where exactly are you adding this code? Is this going into an Actor class or the game mode class or somewhere else? What exactly is FHotReloadExampleTest? Can you provide more information on how you setup your step 2?

Cheers

Doug Wilson

avatar image HateDread Jan 13 '16 at 02:42 PM

Ahh, sorry Doug! I've been looking at the Automation system so much that I had just presumed the universe knew what I was talking about.

That goes into its own .cpp file in the Private/Tests directory, as per this page. FHotReloadExampleTest was just a name I came up with for this unit test. It should show up in the Frontend -> Automation section, with each word in the 2nd parameter of the marco separated by the '.' character being a deeper level in the test hierarchy.

For example, I have the following macro in my Private/Tests/ViewTests.cpp file:

 IMPLEMENT_SIMPLE_AUTOMATION_TEST(FViewLocationIsConsistent, "RTG.Time.ViewLocationIsConsistent", (EAutomationTestFlags::ATF_ApplicationMask | EAutomationTestFlags::ATF_SmokeTest))

And this shows up in the Frontend like so:

alt text

avatar image HateDread Jan 13 '16 at 02:56 PM

Let me know if there's anything else you need :) I've probably glossed over a detail or two!

avatar image ColdIronCoderYZ Apr 28 '16 at 02:41 AM

Hi, does this work for you in 4.11.2? This is what I have: https://gist.github.com/ColdIronCoderYZ/e8a6c08940d911f34c2dd440dc98ffbe, it's inside my project's source/project_name/private/Test folder. It does not show up in frontend... not sure what I'm missing?

avatar image alpaka Jun 28 '17 at 06:37 PM

Just in case anyone is looking for a sorta workaround to this: I played around a bit with hot reloading modules that contain tests and while you cannot change an existing test, adding a new one seems to work fine (recompile module from the editor - or, if the test is in your main module, it also works when building from VS - then hit Refresh Tests in the session frontend).

So when you want to change an existing test to make it pass or whatever, you can change its name from Test to something else (I use Test2, Test3, etc.). Unfortunately, the old test class stays around as a ghost until you restart the editor, so it messes up the "Run All tests in a category" functionality. Note that you can have multiple test classes in a single source file so genuinely adding new tests is not a big issue.

While this is obviously not a perfect solution, it's somewhat workable and is at least better than having to restart the editor all the time. Once in a while, you can clean up all the numbers and just restart the editor.

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

2 answers: sort voted first

After further editing the code you provided I was able to get the project to compile and had the same outcome when running the automation test. I have entered a report (UE-25350) for further investigation.

Cheers

Doug Wilson.

more ▼

answered Jan 13 '16 at 03:05 PM

avatar image Doug E ♦♦ STAFF Jan 19 '16 at 07:19 PM

Hey HateDread-

It was brought to my attention that any code that is not a UCLASS type, including automation code, is not hot-reloaded. The ticket I entered regarding this issue is still open in case this can be addressed in the future however there is no timeline for when that may be. For now what you're experiencing is the expected result.

avatar image clarkelaprairie Mar 09 '16 at 08:55 PM

Hi Doug,

I'm a game developer trying to write quality code using automation testing. I'm running into the same issue as HateDread.

This is obviously not a feature, it is bug. The current state of UE4's automation testing framework makes it exceedingly difficult for game developers to write tested, defect free code. If this is the expected result, please exceed my expectations.

For more information on the problem, see this forum post: https://forums.unrealengine.com/showthread.php?84659-Automation-Testing-Should-be-on-the-Roadmap

Looking forward to hearing you, Clarke

avatar image Stephen Ellis ♦♦ STAFF Mar 21 '16 at 03:56 PM

Hi all,

Thanks for the feedback. The automation systems seem to be an area that could benefit from some additional attention, and I'll bring up the topic with our engineers soon to discuss any plans.

Cheers

avatar image mrooney Dec 13 '16 at 05:24 PM

Haya, Ran into this problem also and just wanted to add a bit more data that I didn't notice in the bug report. The tests do recompile, they just don't get reloaded. not sure if that's any help at all.

Also just kinda wanted to +1 this bug because it does make establishing new unit tests a little more painful than it needs to be. Not a huge issue, but a big enough issue that it makes me frown a little every time I have to restart to update the session frontend with my newly built tests.

avatar image zhennTil Aug 13 '16 at 03:26 PM

This issue also affects UE 4.12.

avatar image jimmyt1988 Apr 16 '19 at 07:32 PM

and 4.22 :(

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

I found a workaround that allows you to enable hot reloading per test at least. The nature of the bug and the workaround are described in my blog post about it: https://vhiterabbit.com/2018/03/02/ue4-code-fixing-test-hot-reload/

more ▼

answered Mar 02 '18 at 07:46 PM

avatar image

Squareys
101 2 7 7

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question