Make chunks not interact with the pawn

I have a brick that breaks into chunks. I make the chunks disappear after a few seconds but in the mean time they seem to be physics actors and try as I might I can’t turn them off. I tried setting the bricks collision channel to ignore the pawn and event disable the collision on the destructible but that doesn’t do anything for the chunks. I don’t see anyway to get a reference to the chunks. How can I do this?