Recently we’ve embarked on a self-assigned task to experiment with some new ideas and explore some under-utilized tools in our surgical animation pipeline repertoire. We’ve acquired some animal organs from a local butcher shop (sometimes referred to as “offals”), and testing our knowledge of comparative vertebrate anatomy, have been dissecting them to make the perfect subjects for an on-going photogrammetry project.
A primary consideration throughout this adventure has been photography, specifically, implementing and improving upon our biological and surgical photography skills. Setting up our subjects and paying careful attention to our lighting arrangements is critical to obtaining the best possible photos to be used in photogrammetry reconstructions. Soft, even lighting from a diffuse light source is the goal; light boxes, light diffusers and overcast sky light are the best sources in this regard.
It would be remiss to discuss best practices in photogrammetry without referring to the work of Carla Schroer from Cultural Heritage Imaging, whose detailed photogrammetry process is a goldmine of information and should be considered required reading before exploring photogrammetry for scientific applications. By implementing some of the tips gleaned from CHI's photogrammetry tutorials, we've seen significant improvements in our reconstructions.
We’ve discussed the art & science of photogrammetry before, which allows you to produce a 3D model from a series of photographs. By taking photos of a subject from multiple angles with a consistent pose and lighting, software will use the parallax between the photos to determine the depth and volume of the object.
Producing 3D models using this method has its advantages and drawbacks: the greatest advantage being that you are working with a photo-realistic, highly-detailed reproduction of your subject. There are disadvantages as well though, and they include highly-dense 3D meshes (3D models with very high polygon counts), which can be cumbersome to manipulate and may cause some programs to crash while performing otherwise simple functions. The need to re-topologize (re-build the model’s geometry in a more simplified way) and sometimes to re-map UVs (to re-assign photographic textures for accurate colour representation), means that the photogrammetry production process is not necessarily a time-saver. When photographic capture is optimized though, and experienced 3D artists are at hand, the results bring a new level of accuracy to 3D modelling.
In an on-going effort to implement photogrammetry into surgical 3D animation, we are using these animal organ studies to evaluate the ability to create reconstructions from subjects which are traditionally considered not ideal for photogrammetry: wet biological material with reflective surfaces, bright highlights and dark shadowy crevices. These pilot studies will allow us to troubleshoot and refine our process so that down the road we can integrate this technique into our surgical teaching videos.
Through trial-and-error we’ve been able to produce some promising results. Here are two examples, one of a dissected lamb heart, and one of a dissected lamb liver:
We’ll continue to experiment with what this technology can offer for surgical education, and share more of our discoveries with you in the future!
– TVASurg Team