x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Normal Map Workflow with Xnormal is not working.

Using the Unreal 4 Documentation found here **HERE**, I am unable to bake a good normal map using the documentation's settings. I have to change the settings quite drastically to achieve a bake that works, and even then, the bake is not quite perfect, with some surface shading errors still occuring. My workflow: High / Low / Cage / and Triangulate in 3Dsmax 2013 and 2014(tried both), 1 smoothing group for the mesh. Reset xforms on meshes. Export triangulated .fbx 2013 files to xnormal with same exact settings as documentation, baked using exported normals and cage. Import same triangulated .fbx into UE4, broken normals appear in the model previewer and in viewport.

I should note, the hard line that appears at the edge facing the camera, is present in the high poly. That is working as intended in the bake. The shading errors with the red X's are the problem areas.

alt text

Product Version: Not Selected
Tags:
more ▼

asked Mar 21 '14 at 04:36 PM in Bug Reports

avatar image

Quack
11 1 3 6

avatar image Adam Davis STAFF Mar 25 '14 at 02:29 PM

Hi Quack,

I am attempting to re-produce your issue here and have been unable thus far to do so. Would you mind walking me through the exact steps you took so I can better assist you? Since you have already provided your assets I just need your import steps to the Unreal Editor. Do you have this issue on any other meshes that you have imported or that are currently in the Editor? Thank you and have a great day!

Adam

avatar image Quack Mar 28 '14 at 09:30 PM

Hey Adam,

Thanks for the response!

I created 2 cubes on two different computers from scratch, high poly with beveled edges, low poly. I used every setting exactly as your documentation says, except for when you import into unreal. The import dialog prompt in your documentation is different then in Unreal4. You can not choose explicit normals any more, you only have a choice to calculate tangents, import normals only, or import normals and tangents. I am assuming that 'import normals' means explicit normals?

Regardless, the only way to get an ok bake is to not follow your documentation. Your documentation states to uncheck the "Tangents and Binormals" FBX checkbox when exporting to xnormal, then to unreal. UN-Checking this box breaks the normal map bake. This image is from another polycount member with the same issue ( http://www.polycount.com/forum/attachment.php?attachmentid=16371&d=1395448501 )

To get a good bake, I have to check the fbx box for Tangents and Binormals and then make sure on import to use "Import Normals and Tangents. Otherwise I get those triangle errors.

I have tried this on two different computers with 3 different meshes and gotten the same issues. I can get the normals to look good, just not following the recommended workflow.

avatar image Adam Davis STAFF Mar 28 '14 at 09:56 PM

Hi Quack,

After discussing this issue with colleagues, we discovered that these documents may be describing a workflow that no longer applies. It is being taken into consideration by the developers. Thank you for pointing this out!

Have a great day!

Adam

avatar image Quack Mar 28 '14 at 09:57 PM

Ok cool. I just wanted to make sure the results I am getting with exporting with tangents and binormals and going through xnormal is the optimal path.

Again, thanks for the help.

avatar image zeroprey0 STAFF Apr 03 '14 at 01:49 AM

I just corrected a discrepancy with how UE4 uses tangents to how xNormal does. This further corrects things. A better document explaining the correct way of generating normal maps for UE4 should be coming.

avatar image SonKim Apr 11 '14 at 07:12 PM

Will this fix be in UE4.1 ? I'm also having the same problems as Quack, but I'll wait for the fix to try baking from Modo to Xnormal to UE4.

avatar image DennisGlowacki Apr 16 '14 at 05:29 PM

I was about to post about this. Here is a test I did: Epic's Xnormal documentation vs a workflow I have in Maya http://content.screencast.com/users/cubedude89/folders/Jing/media/7fb0f10d-501f-4ef3-a022-661f75eaee19/2014-04-07_0140.png

Here is a document I wrote of my maya workflow incase anyone might find it useful. https://docs.google.com/document/d/1EJ3HtTaE7FILx3dFQ5M3WIvHSxTs8RiISd457f_Wkko/edit

avatar image SonKim Jun 13 '14 at 10:52 PM

I'm still having the same issue mention by Quack in UE4.2. I tried going from Modo to Xnormal to UE4, and Maya 2015 to Xnormal to UE4 - both instance I get that nasty triangle artifact. So I started looking at the difference between 2 similar files both with 1 smoothing group, one where UE4 generates the tangents & binormals and the other with imported tangents & binormal, there was a huge difference on 2 of the vertex(though I only highlight 1 spot in the pic), might this be what's causing the triangle bug?:alt text

Here are my results, as you can see I get a triangle on 2 of the corners:

alt text

So my workflow for the red cube is:

  1. Maya 2015: Export all mesh(low,high,cage)using FBX 2014 with 1 smoothing group(with export smoothing group ON and Tangents & Binormals OFF) for the cage and low res mesh, all transform freeze for all meshes and only the cage and low res mesh are triangulated

  2. Baking is done in Xnormal 3.18.8 with Mikk tspace with "Compute binormal in the pixel shader" checked. Low res = use exported normals, High res= used exported normals,use external cage file provided in the zip. Bake to 16 bit TIFF at 2048 X 2048 pixel. Normal map Swizzle Coordinates = X+ Y- Z+.

  3. Take the output normal map TIFF file into Photoshop, change it to 8bits/channel(Image>Mode>8Bits/Channel),save file as PNG no compression.

  4. Import the same low res mesh into UE4 with the "import normal" option.

  5. Import PNG normal map into UE4

  6. Create a basic Material with color, metalic,roughness and apply the nomal map to the normal slot.

Here are the files: http://www.firedrive.com/file/1D4671885E09284B

2triangle.jpg (82.1 kB)
avatar image Lovecraft_K ♦♦ STAFF Jun 16 '14 at 05:12 PM

Hi SonKim -

Please try running again but do not triangulate your cage. See if the same error occurs and let us know. Using your assets I was also able to eliminate this issue by breaking up the Cube unwrap to add padding between each island.

We are continuing to look into this.

Eric Ketchum

avatar image SonKim Jun 16 '14 at 11:03 PM

Hi Eric Ketchum,

Thanks for the answer! Secondly, the low res and cage has to be the same when using Xnormal(so I we would need to triangulate the cage, since the low res is triangulated). I think the confusion comes from the following line:

"One is to give your model a single smoothing group. This will create a very gradient heavy normal map but few seams on UV edges."

The latter part "few seams on the UV edges" make its sound like you can get away(in my case) with 1 UV island for my cube. That's appears to be a incorrect assumption.

I guess the Normal Map Creation Guide, needs to add some additional information like, if you are using 1 smoothing group but have sharp corners(90 degree turns) on your mesh, you'll need to split the UVs at those edges.

This isn't ideal because it creates more UV vertices, compare to using a single UV island.

This should be mention in the guide also another option for those with access to 3DSmax/Maya you can export "Smoothing Groups" and "Tangents and Binormals", you'll be able to use 1 smoothing group and 1 UV island, yet still get perfect normal map shading. Baking done in Xnormal.

Another option on the horizon is handplane3D(www.handplane3d.com) which should let you use 1 smoothing group, lay out the UVs how you see fit(1 or multiple UV island) by just using "Import Normals" in UE4. UE4 support is coming to handplane3d.

My cube(1 smoothing group,multiple UV island, Import Normals): alt text

Quack's "cube"(1 smoothing group,multiple UV island, Import Normals):alt text

My cube(1 smoothing group,1 UV island, Import Normals and Tangents): alt text

ue4_redcube2.jpg (399.6 kB)
ue4_quack2.jpg (265.8 kB)
avatar image Hejden Sep 26 '14 at 09:35 PM

I've been struggling to get this working today, but I've found a solution that got the desired results. If you have a low poly mesh consisting of just six faces (a cube) where the high poly mesh has some nice rounded bevels, you need to isolate each face in the low poly cube's UVs to get a nice bevel at the right place.

When I unfolded the UV of the cube into one single UV island and one smoothing group, I got the exact same artifacts with a diagonal line in the shading. When I split the faces into separate islands, the artifacts disappeared.

The normal map renderer (xNormal, Maya, 3dsmax etc) uses the smoothing group layout of the low poly mesh in its calculation process. If you have one smoothing group for your cube, you're essentially telling the normal map renderer that your cube is round. The renderer will then create a normal map that tries to even out the roundness of your low poly cube to match the geometry of your high poly mesh. But with a normal map of limited bit depth (and compression on top of that), I guess it's very hard for it to match the smoothing group shading perfectly; hence the artifacts.

avatar image Quack Sep 27 '14 at 09:12 PM

The fix you describe is not the solution to this problem.

The problem you ran into is standard normal map creating steps.

The problem I encountered is a different problem, highlight by my image.

You gave good advice, but it doesn't apply to this issue. I am posting this, not to say you are wrong, but to make sure that people who come here with my problem are focused towards the answer.

It's been many months since I encountered this issue and will take the time tonight to see if it has been fixed and post a resolution.

avatar image JedTheKrampus Oct 14 '14 at 05:31 PM

Here's the skinny on things. There's a particular edge case with UE4's tangent and bitangent calculation code that creates a problem where the tangents are split when you choose to import normals and calculate tangents for your static mesh. Other MikkTSpace implementations (Blender, Xnormal, Toolbag 2) don't have the same issue, and if you bake a map in Xnormal it will show up fine in either Blender or Toolbag 2 (as long as you have "Calculate binormal in the pixel shader" checked.) See the image below. yuck, split tangents

If you import the .fbx into Blender, then export the .fbx from Blender with its calculated tangents and bitangents with the v7 binary exporter, Unreal will import the correct tangent space rather than calculating the incorrect one, and will display the mesh with correctly mapped normals. blender mesh on right

I'm working on stepping through the code to see if I can figure out the problem, but I think you guys at Epic should be able to find it faster than I can. I do believe that it's a problem with your FBX importer's tangent calculation, and it's probably a really silly edge case. I've also put a brief summary in the official FBX problem reporting thread. I think we can fix this bug once and for all.

avatar image Farfarer Oct 15 '14 at 09:53 AM

I'm having similar issues with FBX files from Modo. Exporting with vertex normals and no tangents/binormals (Modo doesn't support exporting tangents/binormals anyway), then telling UE4 to calculate the tangents, results in several tangent splits where there should be none.

You can see the very simple cube mesh here in Modo, with UV boundaries highlighted (top). And the resulting mesh in UE4 (bottom) has several extraneous tangent splits (circled in blue) causing very ugly seams across the mesh (highlighted in dotted purple).

Normal map baked in xNormal using a cage and "compute binormals in pixel shader" enabled. All other programs I load this into that support MikkTSpace work perfectly fine.

Tangent split example.

ue4_fbx_issues.png (609.1 kB)
avatar image JedTheKrampus Oct 15 '14 at 02:43 PM

In my own tests, it looks like the 90-degree angles have more to do with breaking the tangents than the UV splits.

alt text

avatar image Farfarer Oct 15 '14 at 08:49 PM

I'm not sure. I've tried some tests based on that and I can't seem to get 90 degree angles to do anything strange. I get tons of tangent splits where I have UV splits, though. Give this mesh a look over, should show what I mean.

Bottom row is just a bunch of planes that meet at different angles. Next row up is those joined up to triangles at the side. Next row up is the same, but unwrapped with UV splits and BAM tons of tangent splits. Top one is them as closed surfaces and there's tangent splits with those too because of the UV splits.

Angle Test Mesh

avatar image Quack Oct 27 '14 at 08:30 PM

I do triangulate before all of my bakes. But I may have missed it. Because of the fact that this is still open, I will devote a bit of time tonight to try with the latest version of Unreal.

I recently imported my District9 mech with which I stress tested edges to see how far I could push it, and the normal map seemed almost on par as it did in Toolbag2.

I will report back.

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

1 answer: sort voted first

I've put up a pull request on GitHub that has a fix in it. Please give it a try and let me know if anything else needs to be done for this fix to be integrated upstream for 4.6.

Pull request 561

more ▼

answered Oct 29 '14 at 04:34 PM

avatar image

JedTheKrampus
68 2 6 9

avatar image Farfarer Oct 29 '14 at 06:22 PM

I've tested this locally and it works perfectly with xNormal bakes and FBX files exported from MODO (meshes with mirrored UVs, too). Completely solves all the issues outlined here.

Nice work, Spencer :D

avatar image Lovecraft_K ♦♦ STAFF Dec 12 '14 at 09:13 PM

In case any one has missed the GitHub comments:

"An update here -- after an internal review by artists MikkTSpace is now enabled by default on both new and existing static meshes. When you update to 4.7 you won't have to do anything special. The engine will automatically recompute tangents with MikkTSpace, assuming the asset has "Recompute Tangents" checked.

Skeletal meshes are a bit trickier because we can only handle import time. We'll update that code as well but have to do a separate review process.

Thanks for your patience :)" - Nick Penwarden

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

Hello Quack and Everyone Else -

I have been trying to get a really good reproduction of this particular reported issue. From my investigation, this appears to be functioning exactly as intended in UE4 and in all the modeling programs. Before everyone roars in outrage, let me explain what I am seeing and how I got there so we can talk from the same space.

To begin, I think everyone understands how the vertex normal is derived, but to clarify the vertex normal is the unit vector that is perpendicular to the surface (for our discussion this is an ok definition, though vertex normals are slightly different than typical surface normals). From my investigation I cannot see where the vertex normal either derived or calculated comes into the engine as incorrect (I should note here that I am using 4.5.1 for my tests). If anyone is having Vertex Normals computing or derived (imported) coming into the engine as incorrect please post pictures of the vertex normal in your modeling program and the vertex normal in the static mesh editor and, if possible, upload the fbx of your mesh.

The problem that people are having, it seems from reading responses here and over on Polycount, is that the tangents are seeming to import correctly and are calculated incorrectly. In either case the math behind this is very simple to checkout, the tangent of the vertex should run along the + U unwrap starting from the vertex, or more technically, it is the vector which shows increasing U space along each surface. This is very important to understand as no matter what we do to our unwraps the UV space is always perpendicular to each other and always runs 0 to 1.

When you are calculating the tangents, the engine does not look at your smoothing information from Blender, Modo, Max or Maya because you are telling it not to look at it. All that it has to go off of is the unwrap and the vertex itself. Often times this will cause the engine to treat the vertex as multiple vertices because of how you have unwrapped that mesh. Ultimately though the engine looks at the vertex in teh context of your unwrap and assigns as many normals and tangents as it needs.

As a case study of this I used a Pyramid, like JedtheKrampus does over at Polycount. I am going to be showing you Maya images but I confirmed this behavior in both Max(2013) and Blender(2.72). To begin here is my mesh, it shows you hard edges: Hard_Maya

And here is the same mesh completely Smoothed in maya: Smooth_Maya

Now the unwrap for the mesh: Unwrap

So the smoothed mesh brought into UE4 with imported Normals and Tangents looks like this: Smooth_UE4

As you can see the Normals and tangents are identical between Maya and the engine. Now lets look at the Calculated Normals: Calculated_UE4

Looking at the unwrap shown with the Calculated normals and you can see the tangents is based on vertex and unwraps planes connected to the vertex that are not merely triangulated quads, so the bottom plane counts in calculation terms as one plane.

So from the examples I have seen, the tangents appear correct when calculated based on the various unwraps and meshes.

If you feel I've missed something or if you have evidence that the tangents are not calculating as I describe above, please let us know with screenshots and uploading an FBX.

Thank You

Eric Ketchum


Regarding the initial question from Quack - I would ask as I did earlier but I believe I did not address you directly and I apologize for that, did you triangulate your mesh before running normals? Often time triangulating your mesh before running a normals calculation will result in a normal where the triangulation is running an opposing direction which will cause the original triangulation to show up as a render error. But we have worked on and continue to work on updating the documentation so if you feel some workflow we are using is incorrect, please let us know.

more ▼

answered Oct 27 '14 at 08:09 PM

avatar image

Lovecraft_K ♦♦ STAFF
36.6k 703 260 737

avatar image Farfarer Oct 27 '14 at 08:28 PM

Hmm, I'm not convinced. I actually fixed the issue in my local fork of the engine code.

It's also telling that this only happens with static meshes and that the skeletal mesh calculates tangents correctly for each vertex.

Edit: Also, you are saying that the engine goes off just the unwrap and the vertex, but looking through the code, it goes off the vertex position, normal, UVs and the faces smoothing groups. (And if it wasn't using this information, then it would be doing it wrong.)

If you check my last image towards the end of this thread (currently 3rd post from the bottom) you'll see it's blending the tangents for the cube correctly; http://www.polycount.com/forum/showthread.php?t=141659

And I have an FBX here that clearly and reproducably shows the issue; https://answers.unrealengine.com/questions/122279/fbx-static-mesh-incorrect-blending-of-tangents.html

I'll try and post up a version from my copy of the engine soon to show what it should look like. Or you could just rig it and import it as a skeletal mesh.

avatar image JedTheKrampus Oct 28 '14 at 08:19 PM

Import that pyramid as a skeletal mesh, calculating tangents, and see if it has the same tangents. If it doesn't (which I really doubt it would, since I've read your code), your tangent space code isn't even consistent within your engine.

Also, your pyramid has a different UV layout from mine, which would of course lead to different tangent vectors. Try exporting that pyramid from Blender, with smoothing off (i.e. explicit split normals) and tangents exported, and import the tangents. This will give you a good idea of the tangents that the reference MikkTSpace implementation would calculate.

avatar image JedTheKrampus Nov 03 '14 at 05:41 PM

Hi, I've updated my pull request that fixes these issues for both static and skeletal meshes by incorporating the reference MikkTSpace implementation. If you could find a guy to help get it merged for 4.6 I would really appreciate it. This is a really important thing to get right, and Mikkelsen's, James', and my implementation has a lot of real benefits, including lower vertex counts for static meshes that have imported tangents and more reliable normal map mirroring that's guaranteed to be seamless (!) without needing to overlay a detail normal map. Both James and I have tested this extensively and it passes.

Pull request in question, please test it internally and get it merged for 4.6!

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question