x

Search in
Sort by:

Question Status:

Search help

  • Simple searches use one or more words. Separate the words with spaces (cat dog) to search cat,dog or both. Separate the words with plus signs (cat +dog) to search for items that may contain cat but must contain dog.
  • You can further refine your search on the search results page, where you can search by keywords, author, topic. These can be combined with each other. Examples
    • cat dog --matches anything with cat,dog or both
    • cat +dog --searches for cat +dog where dog is a mandatory term
    • cat -dog -- searches for cat excluding any result containing dog
    • [cats] —will restrict your search to results with topic named "cats"
    • [cats] [dogs] —will restrict your search to results with both topics, "cats", and "dogs"

Office Holiday

Epic Games' offices will be on holiday from June 22nd to July 7th. During this period support will be limited. Our offices will reopen on Monday, July 8th. 

Mixed Reality Capture Framework 4.20

Hi everyone!

I couldn't find a Q/Thread about this so I started one.

I have been playing around with the new Mixed Reality Capture Framework plugin that came with the 4.20 preview. This is only in beta at the moment, but it would be awesome to get it working somehow. I have gotten as far as getting my camera feed overlaying the virtual camera. I am using a green screen. So far so good.

My plan is to move my camera around the VR player while it comps live. I am using a tracker attached to my camera and feeding the location to the virtual camera.

My problem is currently that the camera feed is live and working but the virtual camera is not moving real time. So it is like I am comping over a still image of my virtual environment from where the camera was positioned when I pressed play.

I cant find any documentation on this plugin yet, but I was hoping someone has tested this new feature and could give some tips and tricks?

I am using HTC vive and trackers.

Thank you.

Product Version: Not Selected
Tags:
more ▼

asked Jun 14 '18 at 10:58 AM in Using UE4

avatar image

KatrineHoloVR
6 1 2 3

avatar image Jonas M Jun 14 '18 at 03:39 PM

Can you please explain how you set this up with 4.20? I have a working build in 4.19. So I'm not sure what 4.20 brings to the table, or how to utilize it.

capture.png (533.6 kB)
avatar image KatrineHoloVR Jun 21 '18 at 09:50 AM

Hi Jonas

I am sorry for a late reply, I have been away from my work desktop and still am but I will try to explain from my memory. Sorry if this is old news for you!

I used the actor blueprint that came in the plugin, you can find it in the plugins content folder or search for it when you create a new actor blueprint.

That has a camera in it. In the camera details panel you have a media player and a material. I opened the media player and assigned the webcam I was using. Then I made an instance of the material and customised the parameters till I got a keyer I was happy with, then applied that material instance to the camera.

Now I got the composition as a texture that I can use as a texture for spectator view or as a material in game for the player to see.

I haven't gotten as far as using the plugins blueprints yet, but I am trying to play around with them a bit but I haven't got a desired result yet and I dont know what half of them do!

I ended up waiting for more documentation on this and made a chroma key in 4.19.

Hope it helps!

avatar image mebalzer Jun 27 '18 at 06:04 AM

I am in the same boat. I was able to use the Mixed Reality Framework when it first came out be because they offered a Mixed Reality Calibration map. This sadly was not included in 4.20 and bringing it over from 4.18 does not work since they have changed many of the CPP libraries so they are not compatible.

I have gotten as far as you have and also wish there were more details since you still need to calibrate our controller or camera to the scene. :(

avatar image 如果爱 Aug 02 '18 at 03:32 AM

hello,can you share your simple mixed reality project to us 。and what hardwave did you use。i can not find any camera in media player. i used avermedia gc 550 and gopro camera. i can find them in obs. expect your tutorial。

avatar image mebalzer Aug 02 '18 at 05:25 AM

Unreal has finally released 4.20 and a link to how to use their Mixed Reality Framework as well as downloading a executable for calibration is listed below. If you don't need to calibrate your controllers, you can skip that and go right to adjusting for your camera. The sad thing, is the only thing I was able to get to work with it was Logitech C920. If you have a Magewell USB3 V2 they claim it works as well and then you can hook up any HDMI camera to it. Sadly my Magewell USB3 V1 does not work and the Magewell USB-3 V2 is $299 on Amazon and for now I have been using the C920 with okay results. Sadly the C920 is 4:2:0 MP4, so the chrominance component has only 1/2 the resolution of the luminance which means key edge is pretty ragged. I am testing this when it comes in Friday (https://www.amazon.com/gp/product/B07DLB33NG/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1) I will reply back if it works. (Sadly the same results as the Magewell USB-3 V1)

Here is link to Unreal's Mixed Reality Tutorials - https://docs.unrealengine.com/en-us/Platforms/MR

avatar image fdomw Oct 11 '18 at 09:48 AM

Hey - how did you get on with the new capture card that you were testing?

I am in the same position. I have tried magewell and Blackmagic capture cards in unreal but without success. Logitech works like you say.

I am pretty sure it is because their drivers aren't compatible with windows media foundation.

I've also downloaded the VLC player unreal plugin but at the moment it shows in the system but doesn't seem to work (VLC picks up my Logitech c920 and my capture cards but the VLC plugin doesn't (I switched off all other media player plugins to check).

Be great to hear if you have found any workarounds - I'll also post if I manage to get anything working....!

avatar image mebalzer Oct 11 '18 at 08:39 PM

It worked great as a Windows Media Foundation Device, but no go within Unreal with similar results to the Magewell USB-3 V1. I moved onto other things, and reluctant to buy the newer Magewell version since I am hoping someone (or myself) makes some changes to allow it to work correctly. As mentioned it shows up within the Editor, but freezes and shows a 50Hz frame rate when you go to preview mode.

(comments are locked)
10|2000 characters needed characters left
Viewable by all users

0 answers: sort voted first
Be the first one to answer this question
toggle preview:

Up to 5 attachments (including images) can be used with a maximum of 5.2 MB each and 5.2 MB total.

Follow this question

Once you sign in you will be able to subscribe for any updates here

Answers to this question