Ever since the age of 6, I've been in love with Taekwondo. I joined the Taekwondo team at
Brown University and became an instructor. In the Spring of 2020, I was working at the
Brainerd Biomechanics and Morphology lab studying the motion data of nonhuman organisms. My
mentor asked me if I was interested in creating an independent project, and the Taekwondo
Visualizer was born.
Quantifying Martial Arts
The first step in creating the project was the idea: what was I
make? Naturally, there were other questions that accompanied this one: What goal
trying to accomplish with this visualization? Who would the potential audience of such a
visualization be? What user pain points would I need to address?
An interactive 3D martial arts visualizer seemed extremely useful, since
most online martial arts tutorials tend to be in two dimensional video formats. With the
help of my mentor, I decided to create a 3D visualization using Unity.
This would allow a
user to select different techniques, view them from any viewpoint, and learn about the
breakdowns of each individual technique to learn more than by simply viewing a technique
executed via video.
The science behind the visuals
Motion Capture System
Initially, the idea was to create a visualization in Virtual
would allow for the viewer to see the model kicking right in front of them.
The first step
of this was to collect the data. To do this, we created a motion capture rig from scratch
that would allow us to capture real-world motion data.
We created a mocap suit with radio-dense beads that could be
XROMM tracking software. We then set
up 3 GoPro cameras to capture the motion from three
different viewpoints. The idea was to capture the motion data from various kicks, track
them, export the 3D transformations, and import them as animations in Unity.
An Unexpected Roadblock
Though the motion capture system worked and we had a solid
place, the project came to a halt due to the Covid-19 pandemic-- it became impossible to
access our motion capture system at the time.
Because of our excitement for the project, we
felt it would be better to pivot to a new workflow than to abandon the
project altogether, so we
went back to the drawing board.
The next idea was to use photogrammetry to generate 3D meshes of
athletes in various poses throughout a kick. However, this idea soon became logistically
challenging for various reasons.
Mainly, the athlete would be required to hold a position
for enough time to allow for many photos to be taken to feed into the photogrammetry
software. This would sacrifice accuracy in the motion, since a technique being held
statically is much different from one in motion.
We finally settled on animating character skeletons based on the
of taekwondo athletes from video. This meant I would use two viewpoints of an athlete
completing the motion, and animate a character rig to match.
This proved to be both a time
efficient and accurate solution, so I moved forward with it. This allowed me to animate
various kicks without the need to run motion data through the variety of software required
for our previous workflow.
With the animations created, I then created a wireframe of what I
the final application to look like.
After some ideation and brainstorming sessions, we came
up with a variety of core features that could be implemented. The focus of the wireframe was
entirely ease of use and user-friendliness in accessing these features.
With an idea of what the UI would look like, and a sense of the user
experience we wishes to create from the core features list, I created a very basic mockup to
see what colors would work well with the wireframe design.
Now that the original ideas had been fleshed out, it was time to create the
application itself in Unity. I proceeded to implement our key features in the program, using
C# for the backend.
The final product provided a clean and easily digestible interface that
would allow the user to focus on visualizing techniques, as opposed to wrestling with
Interactive Playback bar
Individual kick sub-technique breakdowns
Technique tip popups
Scrollable technique selection list
Change scale of ground, or toggle its visibility
Visualize individual limb velocities
Center of mass
Calculate center of mass and visualize via sphere projected onto ground
Skeletal View and Center of Mass
There have been so many
positive takeaways from this project. The ideation and iterative processes in designing this
extremely valuable in tackling the project in manageable chunks rather than diving into it
Integrating real-world data into the Unity engine allows for an easy way to present the data
via various platforms such as virtual reality, in-browser ports, or even mobile
This project was a huge lesson in how to visualize data for users in a fun and intuitive way. Not
to mention, it was fun to make! Thanks for reading! 💥