x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

HISM UpdateInstanceTransform crashes (with repro)


Background

For a few major versions now, HISM (Hierarchical Instanced Static Mesh) has been particularly crash-prone during run-time updates, i.e. when UpdateInstanceTransform, AddInstance, etc are used on a HISM component after the game has begun. The crash typically happens in the rendering thread due to a HISM "ClusterTree" being invalid.

After chasing this for ages I finally have a clean simple repro project that demonstrates exactly what triggers this HISM crash.


Repro

Run the sample project, it should crash within 8-10 seconds. If not, just wait a bit or try a few times again. This bug is timing related so it behaves a little differently each time you run it. The relevant code is in HISMCrashSimulator.cpp's Tick function.

HISM is likely to crash whenever UpdateInstanceTransform is called with a location that is exactly the same as the current location of the instance. Furthermore, calling AddInstanceTransform around the same time practically guarantees that the crash will happen (that's what the repro project does).

In other words, if you need to update only the scale or rotation of a HISM instance and leave the location intact (or if you simply pass the instance's current transform back to the update function) it eventually crashes and especially so if a new instance is also added at or around the same time.


Crash details

The crash occurs in the rendering thread at HierarchicalInstancedStaticMeshComponent.cpp in the function UpdateInstanceTreeBoundsInternal_RenderThread. Specifically - FClusterNode* Node = &ClusterTree[0]; which produces undefined behavior because ClusterTree's memory is invaild.

Sometimes it crashes right there, other times further ahead, but perhaps the most frightening issue of all is when it freezes the entire unreal process inside some WinEvent thread wait routine after which it is very difficult to even come out and close the program!


Workaround

HISM's code treats "In-place-updates" (i.e. location unchanged) as a special scenario, this is where the issues manifest.

So the workaround is to add a tiny offset to the location so we can bypass that scenario, like this:

 float tinyOffset = KINDA_SMALL_NUMBER * 10;
 newTransform.AddToTranslation(FVector(0, 0, tinyOffset));
 
 // newTransform's scale or rotation can now be manipulated as desired 
 // and UpdateInstanceTransform can be called safely

Obviously far from ideal, but until there's a fix I really needed a way to stabilize my release builds so I could move forward and perhaps others using HISM may be in a similar situation as well.

I hope this report helps in fixing the issue!

HISM is one of my favorite features in Unreal Engine and the possibilities are endless when you add dynamic behaviors to it (that's where the tricky bugs usually come in). I hope to see HISM become even more awesome over time!

Thank you.

Product Version: UE 4.11
Tags:
more ▼

asked Apr 28 '16 at 01:20 PM in Bug Reports

avatar image

VSZ
929 24 28 54

avatar image Adam Davis STAFF Apr 28 '16 at 05:13 PM

HI VSZ,

  • Does this occur in a clean, blank project with no additional content or is it limited to one project?

  • What steps can I take to reproduce this on my end?

  • Are you doing this all in c++ or in blueprints?

avatar image VSZ Apr 28 '16 at 05:23 PM

I've already provided a simple repro project in the first section of this report! :)

The repro project uses C++ but if my analysis of the bug is correct it doesn't matter whether C++ or blueprints are used.

avatar image VSZ Apr 29 '16 at 10:34 AM

One important update: Further testing reveals the workaround I provided in this report isn't 100% reliable :( It greatly helps in reducing crashes but doesn't eliminate it.

Adam, let me know if the repro project link in the OP is not visible to you for some reason or if the instructions in the Repro section above are not clear. This is a simple blank project with just one tiny C++ class in it.

Thank you so much for your time! My release builds are in jeopardy until I can prevent this crash somehow.

avatar image Burnz Feb 09 '17 at 04:03 AM

This crashes here :

 bool UHierarchicalInstancedStaticMeshComponent::UpdateInstanceTransform(int32 InstanceIndex, const FTransform& NewInstanceTransform, bool bWorldSpace, bool bMarkRenderStateDirty, bool bTeleport)
 {
     if (!PerInstanceSMData.IsValidIndex(InstanceIndex))
     {
         return false;
     }
 
     if (IsAsyncBuilding())
     {
         // invalidate the results of the current async build we need to modify the tree
         bConcurrentRemoval = true;
     }
 
 int32 RenderIndex = InstanceReorderTable[InstanceIndex];

InstanceReorderTable can have a size of 0 so it causes a crash.

(comments are locked)
10|2000 characters needed characters left

1 answer: sort voted first

Hey VSZ,

Firstly I would like to preface by saying, by updating these HISMC on EventTick, it needs to run on every single game cycle. That means you can't even draw a frame until the BP is done doing whatever you told it to do, which in your case add another HISMC. Secondly, you are using a for loop to also update the number of instances per tick, which is equally as dangerous. Remember that when making anything on the Event Graph, do your very best to keep things event-based, rather than being driven by Tick and you'll be fine.

Instead of running on EventTick, try using a Timeline and set it to 'Auto-Play' which you can then set a custom time dilation to control when your HISMC updates. This keeps your blueprints needing to wait for the event tick to complete before being able to tick again. With that said, I was able to load your project and did not experience a crash. Could you provide me with your 'dxdiag' so I can take a look at your system specifications?

Once you attempt the suggested alterations to how you call the HISMC to be updated, let me know if you still experience the same issues/crash and we can continue with the troubleshooting process.

Thank you,

Andrew Hurley

more ▼

answered May 06 '16 at 08:23 PM

avatar image VSZ May 06 '16 at 09:38 PM

Hi Andrew, I'm certainly not using EventTick or For loops for any HISM work in my actual game! Things are done via conventional event based paradigms.

The repro project was written in that way only because it was the easiest way I could demo the crash with some degree of certainty (and indeed, it still ended up not crashing for you :) ). Btw did you try the level a couple of times for more than 10-15 seconds? Reproducible on multiple machines for me (Dx diag attached)


Some background on how my game uses HISM might be relevant: it's a simulation that uses HISM to model individual plant parts (leaf/stalk/etc) in large numbers. These undergo lifecycle changes that trigger transform updates (mainly to hide instances).

Definitely possible that two unrelated entities ended up calling a HISM update for at least two consecutive ticks. Crash is hard to reproduce but occurs within 5 minutes of gameplay. I tried a centralized queue to control update frequency but admittedly I never tried going slower than at least one update each tick (I need to support a "High-speed simulation" feature of stuff growing and receding at anywhere between 1x to 500x times normal speed). I totally understand such usecases may be beyond what HISM was meant for. I just wish it wouldn't crash the entire application though!

Note:- My builds are stable now after applying workarounds but I think HISM should guard itself against crashes better, hence this bug report.

Workaround #1 - see bug report. Workaround #2 - There are some meshes that HISM just doesn't like regardless of avoiding "in-place" updates. I just use plain ISM for those now.

link text

dxdiag.txt (72.2 kB)
avatar image hackalyze Sep 25 '16 at 01:17 AM

Just wanted to chime in saying I'm experiencing the same issue. In my case, I'm frequently updating the transforms of various HISM instances, as well as clearing all instances and adding new ones. I tracked it down to the same issue that VSZ reported here. The rendering thread is trying to access ClusterTree which could be invalidated at any time in the game thread. There are two places in UpdateInstanceTreeBoundsInternal_RenderThread that retrieve an index of ClusterTree that I've seen crashes at in 4.13

avatar image Burnz Feb 09 '17 at 04:05 AM

We have the same issue. Our blueprint calls UpdateInstanceTransform() and crashes on the dedicated server. The InstanceReorderTable can have a size of 0 while PerInstanceSMData.IsValidIndex(InstanceIndex) can be true.

avatar image AndrewHurley Feb 15 '17 at 07:16 PM

There was a bug entered for a crash dealing with HISMC crashing when called to update.

UE-36188

It has been marked as fixed for 4.14 so I would test your setup in that build to see if it still occurs.

Thanks,

Andrew H

(comments are locked)
10|2000 characters needed characters left
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question