How do I remove Third Party binaries from a package?

I am trying to package a game for deployment on the Oculus store. However, Oculus doesn’t like that Unreal packages with any of the .dll files in the Binaries/Third Party folder. How can I package without these .dll files?