The TVASurg team has always been keen on learning and adapting new technology, with the goal of documenting and presenting the complex and intricate work done by our HPB surgeons in Toronto.
As such, our team has started to develop content in Virtual Reality (VR), and explore how the immersive experience that VR brings could be implemented into surgical and anatomical education. Click here for a video that documents our latest VR experiment - our very own Surgical Anatomy VR Module.
Liver Anatomy VR Module
The first functioning VR module we created was an offshoot of a project developed by our summer student intern, Matan Berson's project. The original goal of the project was to develop a framework for web-friendly, interactive 3D anatomy modules that would supplement each of our 40+ cases in the atlas.
Concurrently, the team had gained approval to purchase and build a custom PC rig, both to help with our 3D animation renders, but also to start exploring development in VR. The headset we chose for development was the HTC Vive.
The software Matan used for his project, Unity3D, is also capable of developing VR content. This meant that the 3D anatomy modules he was developing could potentially be ported into VR. Moreover, one of our chief editors Dr. Paul Greig mentioned a potential stage to showcase VR technology–his bi-annual liver anatomy lecture, given to new surgical trainees.
We decided to dedicate our full team effort to developing this VR module, and the result was a successful 45-minute VR lecture presented by Dr. Greig, which we've posted online as one our featured talks.
Google Cardboard
Soon after the lecture, we've received quite an amount of visitors around our cubicle area, including an enthusiastic group of high school students from the David Suzuki Secondary School.
As a next step, we started wondering whether it was possible to deliver VR content to audiences in a more portable manner, without the hassle of having them tethered to the VR equipment at our cubicle.
The answer we found was Google Cardboard, an inexpensive solution that transforms a smartphone (Android or iPhone) into a VR headset.
Using a new workflow, we developed both Android/iPhone apps in VR, and were able to distribute them to our internal testers through email links. In the near future, we hope to distribute these modules through the Google Play Store and the Apple App Store, to accompany our monthly surgical video releases.
Surgical Anatomy VR Module
As we continued to refine the VR Liver Anatomy Module using the HTC Vive headset, one of the most common observations we had was testers instinctively wanting to grab virtual objects with their hands, instead of relying on the controllers that came with the HTC Vive.
We decided to address this user preference using the Leap Motion controller, a device that senses and captures hand movement and gestures in 3D space, which also supports development in VR using Unity3D.
Grabbing, moving and rotating virtual 3D models
Mini anatomical control console
Viewing surgical footage within VR
Adjusting size of virtual 3D models
Viewing CT scans within VR
Jumping to other surgical cases
Our experiments (documented above) not only allowed users to move and rotate objects in VR space with just their hands, but we also introduced new VR control elements that help users navigate the anatomical details of our models.
These experiments helped us develop the Surgical Anatomy VR Module, which will allow us to explore the anatomy of any of our 40+ cases documented in our atlas in VR, using our hands as controllers.
As developers, we are excited to be participating in the thrilling world of VR development. However, as storytellers, we also want to make sure that the technology itself is not an end-product, but more of a tool used to deliver an experience.
We encourage you to leave us a comment, or send us an email on your thoughts on the role of VR in surgical education. Please subscribe to our newsletter if you'd like to receive monthly notifications of our latest updates, and we'll see you in our December post.
-The TVASurg Team