How to use Vive Controller as a virtual camera at the same time the HMD is working

I found tons of example videos(like these :

but I haven’t found a proper tutorial. It seems easy and everyone seems to be doing well but I’m new to Unreal and I’m struggling with it for weeks.

What I’m trying to do is, the player will use only a HMD and a controller, and the other controller will be attached to the camera as a Virtual camera. When I play, I want one screen to display a HMD view and another screen to display a Virtual camera view.

I understand how to track the controller as a camera. I made the camera component as a child of the controller. But I coudn’t find out how to make them work at the same time. The position and rotation value of HMD is mixed with the value of the controller and it keeps getting messy.

To sum up,

  • 1st player(walking around in the level) : one HMD, one controller(left)

  • 2nd player(virtual camera): view from the other controller(right) attached to real camera

  • play at the same time

If the first player uses the controller to teleport, I hope the second player will also move along.

How do I modify the blueprint on a VR template to make this possible?

Hey 28agarwaen,

I’d be happy to help you out with this. I made the blueprints for the MR video in your Pulse Design Group link.

What you’re saying about mixed inputs absolutely makes sense. Are you setting your virtual camera controller to operate as a 2nd player? You will then need to launch the program as 2 players instead of 1.

Epic says there are some better MR video resources on the way. Have you looked at what may be available in 4.19 or 4.20?

Thank you for your answer. Of course I checked that blueprint.
But there were some missing parts and I had to figure out.
What does it mean to launch the program as two players? That was the most obscure part. I found a way to use spectator mode and it worked anyway but that way doesn’t require a projection box like yours. Tell me how you did it please.

I should preface this by saying that I made the MR video back in like 4.12 or something, so there are probably a lot of redundancies and deprecated blueprints at play. Furthermore, I’ve learned a ton since then and my approach was super sloppy.

The first player would operate as normal with 1 controller. The Vive Tracker was set up in the 1st player’s BP (note: this probably doesn’t need to even be in the 1st player’s BP anymore) and a “scene capture component” node was attached to the tracker. That scene capture component would output a texture, which is fed into this blueprint that is placed underground or far away in a secluded part of the scene:

Then, make sure your camera rig blueprint in your scene (from that 2nd picture) is set to be player 2. Lastly, go to the dropdown next to “play” and choose 2 players:

246402-snip4.png

At this point, you should have 2 screens pop open when you play - one for the VR user, one for non-VR. If I recall correctly, we had some issues getting the 1st player to actually render VR to the headset when launching with 2 players, but that may be resolved at this point. Then we simply screen captured with OBS to record the video.

Hope that helps,