Catfish Visualization
Teaching users about catfish morphology 🐟
Full Stack Dev
Feb 2018 - July 2020
Overview
At Brown University's Brainerd Lab, I had the opportunity to work with my mentor, Aaron Olsen, on various projects motivated by the question of how suction feeding behaviors have caused evolutionary diversification of musculoskeletal systems in ray-finned fish.
With the help and guidance of Aaron, I learned to use XROMM (X-Ray of moving morphology) technology and various 3D segmenting and mesh manipulation programs such as 3D Slicer and HOROS.
Published Software Package
Introduction
Understanding science through visuals
One of the projects we worked on was an interactive browser-based catfish visualization. Suction feeding fish rapidly expand their mouths to create a pressure gradient, forcing water and prey inside. To better understand this phenomenon, the fish skulls' skeletal elements are studied using x-ray and CT imaging techniques.
However, we wanted to go a step further by building an interactive tool where a user could change the conformation of a fish's bones depending on a biologically accurate linkage model connecting each bone. This would allow anyone to visualize how a catfish skull's components would move under different conditions.
Implementation
Bringing it all to life
Creating the Meshes
The 3D meshes we used for the visualization were generated using real fish bones. This was done by CT scanning the fish, which produced DICOM (Digital Imaging and Communications in Medicine) files. These files could then be imported into a program such as 3D Slicer or HOROS, which allow for a user to isolate bones from the rest of the fish anatomy, and export the resulting selections as 3D meshes.
Making the Meshes presentable
To maximize the simplicity of the visualization while keeping scientific accuracy, we cleaned up the meshes a bit and decimated them to include less polygons and thus optimize our program. The bones were then kept in their initial CT scan positions as to not compromise accuracy. Each bone is its own mesh, which allowed us many options in how to use them moving forward (visualization, color-coding, 3D printing, etc.)
Physical Model
To help us better understand the morphology of the catfish skull, we decided to create a 3D printed version of the skull. This would allow us to more critically evaluate how different components of the skull would move in relation to one another when creating the digital model.
Because we already had the 3D obj files for each bone, we were able to 3D print each bone to scale. To simulate ligaments between bones, we used small woven polyester ribbons anchored by micro-screws. The final result allowed us to manipulate the skull "by hand" and visualize its general motion patterns.
The Application
When creating the virtual model, we wanted to allow users to play around with different motion patterns to see how the skeletal elements interacted with one another, as they would with the physical model. Aaron wrote a linkage model that handled the physics of motion and kinematics between bones given observed properties in the actual organisms.
I then helped him expand upon the svgViewR package to include interpolation functionality so that when users interact with different sliders, they can view different forms of movement in the model. They can choose to play all motion types at once, or play them individually to view breakdowns of the motion patterns.
Reflection
This interdisciplinary project was my first foray into scientitifc visualization. Taking real world data and creating a medium by which to present it to others for their own understanding is truly special. Working with Aaron was a huge step in my journey as a product designer and developer. This body of work inspired me to pursue countless other projects that helped me develop my skillset as a creator (and I am an expert in catfish skull segmentation, a title held by a very limited few 😉).
Recommended 🔎
