x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Lightmass: Disable Self Shadowing on Static Meshes?

Is there a way to disable self-shadowing on statically-lit meshes?

It would be a super nice feature to have for foliage since lightmap shadows look especially bad on foliage built using 2d planes for leaves/fronds.

Right now I can only disabled shadow casing entirely which is not what I want, ideally I want this:

  1. Static Mesh/foliage casts shadow on world and other props

  2. Does not cast a shadow on itself

  3. Can receive shadows cast by other meshes/foliage

Is there a way to do this right now in Unreal?

Thanks

Product Version: UE 4.14
Tags:
more ▼

asked Jan 10 '17 at 01:49 AM in Using UE4

avatar image

brisck1
100 6 14 19

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

3 answers: sort voted first

Since the question hasn't really been answered. I believe the most simple and non-technical way to achieve this, is by having two objects. The tree itself, with" cast shadow" disabled, and another hidden shadow proxy, which would simply be a more optimised version of the mesh, thats set to invisible, but does cast a shadow. Hope this helps some people.

more ▼

answered May 21 '18 at 01:07 AM

avatar image

LorenzFischer
0

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

Hi brisck1,

Self-shadowing cannot be disabled, but one thing you may want look into is how addressed this very same issue in our GDC Kite Demo.

We used the Render to Texture Toolset to render out a depth texture for our tree imposter and then set everything up via the material so that the tree would have more accurate shadow casting and not appear like a billboard plane casting a shadow at these far distances.

The two additional pages for this tool you'll want to look at are the Render 3D Imposter Sprites and the Render a Normalized Depth Map pages. You'll want to render out the Imposter sprites so that when the camera is moving around these camera facing sprites will then shift between these imposters so that the view is accurate to the the location and the camera. The last page for the Normalized depth is so that you can capture accurate shadows for the sprites as they rotate. It can be a lot of setup, but it can be the difference between seeing a plane cast a shadow and something that blends more nicely for these billboards.

-Tim

more ▼

answered Jan 10 '17 at 03:37 PM

avatar image

Tim Hobson ♦♦ STAFF
48.7k 888 103 880

avatar image brisck1 Jan 10 '17 at 08:40 PM

Hi Tim, thanks so much for your help!

The Impostor stuff should be really useful for my lowest LOD and I have a few questions regarding that (see below).

However, the main problem I'm facing is with the full detail models + static lighting & shadows. Currently, I'm attempting a stylised art direction for my trees, using bent normals to make them look more volumetric. This works fine with shadows disabled, however, when shadow casting is enabled, the trees cast shadows onto themselves and the 2d cards I use for the fronds/leaves become very obvious:

Current Shadows

Here you can see the difference between Shadows enabled (left) and disabled (right). With a Stationary directional light baked shadows are super sharp and a bit messy, and with a static directional light they are softer but the overall problem still remains.

Here's a mock-up of what I'd imagine disabled self shadowing would look like:

Self Shadowing Disabled

The reason I can't disable shadows completely is that it breaks the scene lighting - it just doesn't look right when you're walking through a forest where no shadows are being cast on the surrounding environment.

Maybe there's a better way for me to approach this problem that I'm not seeing though?

Regarding the impostor stuff, I have a couple of quick questions:

  1. Do impostor sprites and normalized depth maps work well with baked static lighting?

  2. If I'm using static lighting for my trees and have an importer sprite as a LOD, is there anything I should do in regards to my lightmap UV's on the main/impostor LOD to get it to light properly (since multiple lightmap UVs is not supported for foliage right now)

avatar image Tim Hobson ♦♦ STAFF Jan 11 '17 at 06:42 PM

There may be some redundancy in my answers because I'm just answering as I go along. Sorry for that.

This works fine with shadows disabled, however, when shadow casting is enabled, the trees cast shadows onto themselves and the 2d cards I use for the fronds/leaves become very obvious:

I've never tried it with static lighting, but if you're using a masked material for your setup for the imposter sprites I wouldn't think it should matter here. You may need to check the option for Cast as masked shadow in the material settings though. In Kite Demo where we only used Dynamic lighting the PixelDepthOffset_Foliage node or use the VolumeShadowMap material function as a way to adjust the shadowing. It may be best to set this particular material not to cast shadows in the Static Mesh Editor under that Material Element since it can be faked in the material for casting on the actor.

For billboards you shouldn't worry as much about completely accurate shadow casting for these because usually at that distance the player won't be able to make out ever fine detail. If the tree mesh itself, and not the billboard, is casting a static shadow that will still be on the ground.

With a Stationary directional light baked shadows are super sharp and a bit messy, and with a static directional light they are softer but the overall problem still remains.

With your stationary Directional Light make sure that you've set a Dynamic Shadow Distance for the shadows. By default it is set to 0. At farther distances these will fade out anyhow and shift to the static shadow, so this shouldn't be a concern for the billboards if they are being used with a static mesh as the higher LODs.

avatar image Tim Hobson ♦♦ STAFF Jan 11 '17 at 06:42 PM

it just doesn't look right when you're walking through a forest where no shadows are being cast on the surrounding environment.

In Kite this was handled by using Distance Fields Shadows. You also have the option to enable Far Cascades for your directional light. This will use farther distances to have some cascaded shadows.

Regarding the impostor stuff, I have a couple of quick questions:

  • Do impostor sprites and normalized depth maps work well with baked static lighting?

  • If I'm using static lighting for my trees and have an importer sprite as a LOD, is there anything I should do in regards to my lightmap UV's on the main/impostor LOD to get it to light properly (since multiple lightmap UVs is not supported for foliage right now)

Haven't done any static lighting with them, but suspect it should work. Possibly some settings I've mentioned above that need to be enabled or options used.

No specific setup that I'm aware of for the imposter that needs to be setup for the UV. With the baked shadows from the mesh down to the billboard you'll probably see some bake issues. I think this doesn't come up that often is because foliage like trees that move and sway with the wind typically don't get bake shadows because it doesn't look natural. For the imposter sprite it may be better to just control everything via the material and use an unlit material that has these depth textures with the normalized shadowing. Use the emissive output to control the brightness of the material so that it's not so blown out. I think it's just a lot of finding ways to fake the effect or either not use the imposter sprites all together and have a significantly lower LOD rather than a billboard.

avatar image brisck1 Jan 13 '17 at 04:52 AM

Hi Tim,

Thanks for your help and apologies for the delay in getting back to you. I wanted to try out your suggestions fully to make sure I understood them before I responded.

I tried what you suggested with depth maps on my foliage but I think there is some misunderstanding in what I am trying to do - most of your suggestions seem to be for improving shadows, where instead, I actually want to remove them:

Basically my issue is not about getting good quality self shadowing on my foliage, (the goal here isn't realism, but stylization) the problem is that any self-shadowing on my foliage breaks the nice, volumetric, stylized shading from the bent vertex normals of my model:

alt text

All I want to be able to do is find a way to disable self-shadowing on my model, ideally via lightmass baked lighing (static/stationary) without disabling shadow casting entirely - Hence this mock-up image:

alt text

Hope this helps clarify things. I realised that disabling self-shadowing in Deferred Rendering is not easy (although they managed it in CryEngine somehow) but failing that, i'm guessing it must be possible for Lightmass baked lighting?

Thanks again :)

avatar image Tim Hobson ♦♦ STAFF Jan 13 '17 at 09:13 PM

There's no way to disable self-shadowing in UE4 so often you have to workaround this by using the texture plugged into a multiplier going to the emissive input. You can use a constant value to control the emissive nature so that you can override this self-shadowing so it's not as intense.

If you have a sample project with one of these trees I that is stylized I could probably look and give you more direct feedback playing around with some settings and show you some examples.

Alternative methods would be to use an Unlit material and just control the diffuse in the emissive input as I described above.

Also looking at the material for the billboard and disabling shadow casting under the material option, but with that, you lose shadow casting from for this billboard for dynamic lights. But at far distances that shouldn't matter. And if you're using statically baked lighting anyway for these The LOD0 mesh would have it's baked shadow on the ground and not the billboard. This also does not affect distance fields since the DF mesh is based on the LOD0 mesh and not the LODs down the hierarchy.

Anyway, if you have a sample you don't mind sharing you can post here or send me privately on the forums and I can take a more in-depth look to give you feedback.

avatar image brisck1 Jan 14 '17 at 01:14 AM

Hi Tim,

That's sad to hear, I was holding out hope that it might still be possible in Lightmass since it's baked and non-dynamic.

I've put together a sample project for you with some example trees (not pictured in this thread). One has two-sided polys with a single-sided material, the other is has single sided polys with a two-sided material.

This includes a test map which demonstrates different options turned on and off (shadow casting and mobility) - see attached file for project.

test level

Take a look and let me know what you think.

Thanks! link text

testlevel.jpg (91.1 kB)
avatar image Tim Hobson ♦♦ STAFF Jan 17 '17 at 03:22 PM

Thanks! I'll take a look when I get time at the end of this week or early next week. Lots of stuff happening this week! (4.15 previews just started, Preparation for my training stream on lighting next week, documentation work, etc)

That's sad to hear, I was holding out hope that it might still be possible in Lightmass since it's baked and non-dynamic.

It's still possible to use lightmass, just I think the best course here is to not use it on the billboard by disabling shadow casting for that part of the mesh. Any baked shadow for the LODs before the billboard should still be baked. That should work, in theory. I just need to see the setup and work it out.

avatar image brisck1 Jan 17 '17 at 09:32 PM

Thanks Tim, appreciate that, please let me know how you get on :)

avatar image brisck1 Jan 25 '17 at 06:29 PM

Hi Tim, I was just wondering if you'd had a chance yet to look at this problem?

avatar image Tim Hobson ♦♦ STAFF Jan 26 '17 at 05:32 PM

Sorry, not yet. Busy week with releases and my time split between another team. I'll try to get to it by tomorrow, but unfortunately, I can't promise that I will be able to.

avatar image brisck1 Jan 30 '17 at 03:06 AM

Ok thank you, any time you can spare is of great help - im still struggling very hard to overcome this

avatar image ziga.osolin Jul 12 '17 at 06:31 PM

I would also like to know if there has been some progress on these issues.

avatar image [HP] Dec 29 '17 at 08:38 PM

Hey,

Just wondering if there's any progress/news on this?

Thanks, HP

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

Hi all, I know this post is kind of old, but I wanted to chime in. To achieve the effect you're looking for, you may want to look into vertex normal editing in Maya. This is how most artists achieve this "stylized tree" look that you're after.

Example: https://www.artstation.com/artwork/9QGVy

I hope this helps anyone else who might be on a whiled goose chase for answers :)

more ▼

answered Jan 31 '18 at 09:52 AM

avatar image

bobericko
36 1 4 5

(comments are locked)
10|2000 characters needed characters left
Viewable by all users
Your answer
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question