We've been experimenting with Mixed Reality (MR) capture methods to showcase some of our VR modules. The above video is our experimental result, and was presented at the 3D Med Australia 2018 Conference in Melbourne, Australia.
In VR development, one challenge we constantly encounter is communicating the VR experience, especially in settings where an in-person demonstration would be impractical.
One solution we've found was to simply capture the user's Point-of-View (POV); that is, mirroring what the user in VR is seeing/doing. We've used this approach before when demonstrating our Operating Room and Medical Imaging VR modules:
Medical imaging VR module
Operating room VR module
This approach gets the job done, but it doesn't quite capture the experience of immersion in VR. Moreover, observing the first-person view may be a bit disorienting or even nauseating, due to the variable camera position.
Another approach is to utilize Mixed Reality (MR). Simply put, this approach puts us in a third-person view, where we're able to observe the user and the VR environment they're in at the same time.
Third-person view, seeing both the user and the VR environment they're immersed in
This approach is surprisingly easy to set up - with the help of a green screen studio (for those of you in Toronto, there's one free to rent at the Toronto Reference Library), as well as an Unity plugin known as the LIV compositor, we were set to go.
Greenscreen setup
View from a 'virtual camera' within VR
Compositing within the LIV plugin
Special thanks to our surgical fellow Dr. Lawrence Lau for patiently testing this out with us.
- The TVASurg team