I have a normal direction and a location plucked from one of the verts that make up this triangle. I would like to rotate the normal based on it spherical location.
Okay It took me a little while to figure out what exactly you where asking but I think I understand it. You are trying to move something (like a character) around the inside of a sphere. While I currently do not know the full answer to your question I do know that if you have the normal of your triangle which I assume the triangles of your sphere are facing the inside, then your triangle normal will always be facing the center of the sphere and so you can transform your normal into a rotation by calling
myRotation = myNormal.Rotation();
Now this will give you a direction always facing toward the center of the sphere. Using your Normal as the “Up” direction and you should theoretical have a forward direction which these two combined vectors/rotations may give you your answer via Cross Product, Dot Product, or something else… it’s been a while.
I think you’re close. I’m trying to figure out a way to better explain it. The triangle in question could be any geometry inside the sphere. It’s not facing the center of the sphere (0,0,0). I’m tyring to rewrite the normal evaluation inside of the Recast.cpp which is what checks to see if the world geometry normals exceed the allowable limit for walkable ground. If it does then it’s flagged as not walkable and Navmesh isn’t generated there (or at least isn’t marked as walkable). So the triangle is any world geometry that is going to be checked in the navmesh system. Currently the limit is 89 degrees, it is checked by looking at the up axis normal value.
What I want to do is check this normal value but relative to it’s location inside the sphere. I figured that because I have a vertex location from the triangles I could take that, and derive a rotational location inside of the sphere. This rotation I could then use to Inverse Transform the normal so that it is evaluated as if it was sitting on the ground in a normal flat world. It would still have it’s relative normal direction, only now it would fit within the 89 degree limit (if in fact it is relatively less than 89 degrees).
I’ve done a similar thing with the CharacterMovementComponent so that my character can walk around on walls/ ceiling, but in that situation I was able to use the Characters rotation to perform the inverse transform method. Now I want to do the same thing but using a vertex location within the sphere. I believe you are correct though in that I should be able to Cross or Dot product the Normals Vector and the Vector pointing toward the center. Though I’m a slightly confused as how to go about that. And which one does what again 