As a quick background on what I’m doing… I’m working on a dual contouring based voxel engine which currently runs with up to ~1500 chunks. Each chunk uses a heavily customized version of the ProceduralMeshComponent, or PMC, to render and provide collision.
First, currently the ProceduralMeshComponent (PMC) implements the interface IInterface_CollisionDataProvider to provide the mesh data to PhysX which then gets baked by PhysX on the game thread. It appears this works by using the same baking framework that is meant for the editor.
What I’m investigating doing is making the PhysX baking callable from any thread by supplying the data, and then giving the result as a part of the BodySetup so that I don’t slow down the game thread with PhysX cooking. I see where the PhysXFormats has AllowParallelBuild return false. Is there something within PhysX/UE4 that keeps from cooking on alternate threads?
Also, specific to my project… I started in C# and as a part of it built custom collision detection logic into a heavily modified version of Jitter Physics. I would like if possible to pull over my custom collision code as I was able to cope with far more complicated objects than what the compound shapes PhysX uses can cope with. I would like to know how practical it would be to modify PhysX/UE4 to basically add a custom shape and it’s associated collision detection. I don’t want to try to replace PhysX but more just give it another shape, like what convex hull is. Is there a better way to do this than modifying PhysX itself?
There is more to this, but as an example the ships in Space Engineers being made of a uniform grid of blocks I can quickly cull out most non colliding blocks between 2 ships far faster than what the built in compound shape can do. In testing in UE I get major frame rate hits colliding bodies of more than ~ 10k blocks but I was able to handle well north of a few million in each ship in my custom detection system.
I don’t believe we have done any tests to see if we can cook collision geometry in a background thread. Generally things are not thread safe unless they have been designed that way, so there will probably be some work to support that, and you may need to talk to NVIDIA support about it as well.
Unfortunately at the moment I don’t believe it is possible to create your own new mid- or narrow-phase for collision detection in PhysX. I know this is something NVIDIA want to add in the future though. Their source is available though, so you could modify PhysX yourself to add it.
Were you able to improve the runtime PhysX baking performance after all? This is the only thing dragging my procedural generation engine’s performance down at the moment and I would love to hear from anyone who managed to improve this process… At least knowing about the possibility of improving it would be helpful before jumping into the adventure of going through a good portion of the UE4 and PhysX code base.
Some numbers about my case: Without enabling collision on any of the generated meshes, I am able to generate a terrain of 768x768x128 blocks in volume (together with a good number of procedural trees and quite a bit of foliage that has a lot of vertices to avoid translucency) and put all of those onto screen under 1,750M CPU cycles (0.5 sec) on my I5-4590. Moreover, although the collision is not enabled, I still generate the required FTriMeshCollisionData array during this process. When I enable collision, the amount of CPU cycles required to put the exact same geometry onto screen goes up to 13,000M. That is 7 times the processing power required to generate and render the same geometry.
These numbers don’t add up in my mind. From noise generation to merging the generated voxels in order to create leaner meshes out of them and to putting those meshes onto screen, the amount of work done in a procedural generation pipeline is absolutely massive. The fact that baking PhsyX data out of ready-provided collision mesh data takes 6 times longer just doesn’t make sense. Clearly the baking process wasn’t designed to be done in real time, hence wasn’t optimized accordingly. That’s why I am hoping that there should be a good amount of room for improvement and eager to hear what others have done about it…
If you want to, come find me in the Unreal Slack chat. I’ve done a couple minimal improvements so far that might or might not help you, but I’m planning on looking into quite a bit more soon.
Also, there’s pretty good reasons the baking takes a little bit of time.I’m sure like anything it could be faster, and I’m sure they don’t put as much time into optimizing the baking as they do everything else, but from what I’ve seen so far there’s a couple bigger issues.
I spoke to NVIDIA about PhysX triangle mesh cook times. They mentioned that there are some flags you can set in C++ to improve cook times. From their number, setting PxMeshCookingHint::eCOOKING_PERFORMANCE seems to make things about 4x times. You can also use eDISABLE_CLEAN_MESH and eDISABLE_ACTIVE_EDGES_PRECOMPUTE which together give another approx 2x speed up. Note that this might impact quality of physics simulation collision with the mesh though.
I’m going to add a task for us to expose this as an option on ProceduralMeshComponent, so users can set it when doing lots of runtime collision creation.
@JamesG That sounds like it would help quite a bit! I had seen the eDISABLE_CLEAN_MESH before and had thought about trying to enable that since I can output a clean mesh with my mesh generator from the voxels. So that would be amazing to get access to those. I’m c Right now I’m spending up to 8ms per frame baking collision, so I had been looking into moving baking completely off the game thread, and I’ve seen from the Nvidia documentation where it appears to be safe to move it to a separate thread, but I know that will depend also on what’s within UE4 that’s run at the same time, which I haven’t had time to look into yet. Here soon I’m probably going to investigate that myself unless you know of anything that will preclude it. Thanks James for looking into this!
@Ales.Borovicka From what investigation I have been able to do, it appears currently UE4 uses the offline cooking system which means to be able to do that wouldn’t the PhysX cooker for runtime have to be moved somewhere else so it’s not buried in the offline cooking support? Also do you know if the cooker could be on a different thread if it used that path to insert back into the scene instead of serializing/deserializing like the engine does currently? Thanks for your input as well!
Please note, that it is recommended to validate the mesh in debug configuration when flag eDISABLE_CLEAN_MESH is used.
So in debug before create mesh please call:
bool validateTriangleMesh(const PxTriangleMeshDesc& desc)
Cooking can be done on a different thread afaik, there is a mutex guarding the triangle mesh insertion into meshes hash map which should be the only part that can cause trouble. So it should be safe to call it from a different thread.
These are all great news guys, thanks for the time and the effort. I’ll spend some time looking in to your suggestions in the coming days and come back with some numbers.
Although in practice things will probably not add up perfectly linearly, if I look at the provided numbers it seems like the potential improvement in the runtime baking performance could be anywhere between 4x to 10x. If indeed the practical gain falls into that range, do you guys think we could enable runtime collision baking for mobile as well?
I haven’t tried to use UE4 on mobile yet, but the question I would have with baking on mobile isn’t so much the performance but whether the PhysX cooker is even supported on mobile? Assuming it is, I think the only way it’s really practical on mobile would be to get it out of the GT. My mobile development has been mostly limited to simple apps where I haven’t had to really push the hardware, so I may be wrong.
I’m looking into this now and I’m curious where/ how you actually set those flags. I can see on the getVerticesForModification() call that is says if I wanted the face indexes to be the same as the supplied mesh then I need to set eWELD_VERTICES = 0, and eDISABLE_CLEAN_MESH = 1. But I’ve yet to find how to set these for cooking.