Landscape visibility tool breaks physical material detection

When using the landscape visibility tool (to cut holes in the landscape), this results in traces not detecting the physical material for that landscape component.

This breaks my system for footsteps.

Any ideas of how to fix this (or any ideas of a work around)?

I believe this is because the visibility tool essentially applies a mask to the landscape material, so the mask is present across the whole component, and is what gets returned in HitInfo when doing a trace.

Any ideas on how to lower the importance of the visibility mask?