Multiplayer VR: how to facilitate players using with and without VR

Is there any way to create a game where one player uses VR (Oculus) and another set of players don’t? At the moment we can only get all players to use VR or not use it.