Taekwondo Visualizer
Visualizing Martial Arts 🥋
Full Stack Dev
January 2020 - May 2020
Why Taekwondo?
Ever since the age of 6, I've been in love with Taekwondo. I joined the Taekwondo team at Brown University and became an instructor. In the Spring of 2020, I was working at the Brainerd Biomechanics and Morphology lab studying the motion data of nonhuman organisms. My mentor asked me if I was interested in creating an independent project, and the Taekwondo Visualizer was born.
Introduction
Quantifying Martial Arts
The first step in creating the project was the idea: what was I going to make? Naturally, there were other questions that accompanied this one: What goal was I trying to accomplish with this visualization? Who would the potential audience of such a visualization be? What user pain points would I need to address?
An interactive 3D martial arts visualizer seemed extremely useful, since most online martial arts tutorials tend to be in two dimensional video formats. With the help of my mentor, I decided to create a 3D visualization using Unity.
This would allow a user to select different techniques, view them from any viewpoint, and learn about the breakdowns of each individual technique to learn more than by simply viewing a technique executed via video.
Data Collection
The science behind the visuals
Motion Capture System
Initially, the idea was to create a visualization in Virtual Reality that would allow for the viewer to see the model kicking right in front of them.
The first step of this was to collect the data. To do this, we created a motion capture rig from scratch that would allow us to capture real-world motion data.
We created a mocap suit with radio-dense beads that could be tracked using XROMM tracking software. We then set up 3 GoPro cameras to capture the motion from three different viewpoints. The idea was to capture the motion data from various kicks, track them, export the 3D transformations, and import them as animations in Unity.
An Unexpected Roadblock
Though the motion capture system worked and we had a solid workflow in place, the project came to a halt due to the Covid-19 pandemic-- it became impossible to access our motion capture system at the time.
Because of our excitement for the project, we felt it would be better to pivot to a new workflow than to abandon the project altogether, so we went back to the drawing board.
Iteration
The next idea was to use photogrammetry to generate 3D meshes of Taekwondo athletes in various poses throughout a kick. However, this idea soon became logistically challenging for various reasons.
Mainly, the athlete would be required to hold a position for enough time to allow for many photos to be taken to feed into the photogrammetry software. This would sacrifice accuracy in the motion, since a technique being held statically is much different from one in motion.
We finally settled on animating character skeletons based on the positions of taekwondo athletes from video. This meant I would use two viewpoints of an athlete completing the motion, and animate a character rig to match.
This proved to be both a time efficient and accurate solution, so I moved forward with it. This allowed me to animate various kicks without the need to run motion data through the variety of software required for our previous workflow.
Design
With the animations created, I then created a wireframe of what I wanted the final application to look like.
After some ideation and brainstorming sessions, we came up with a variety of core features that could be implemented. The focus of the wireframe was entirely ease of use and user-friendliness in accessing these features.
With an idea of what the UI would look like, and a sense of the user experience we wishes to create from the core features list, I created a very basic mockup to see what colors would work well with the wireframe design.
Development
Now that the original ideas had been fleshed out, it was time to create the application itself in Unity. I proceeded to implement our key features in the program, using C# for the backend.
The final product provided a clean and easily digestible interface that would allow the user to focus on visualizing techniques, as opposed to wrestling with confusing UI.
Core Features
- Responsive camera
- Interactive Playback bar
- Individual kick sub-technique breakdowns
- Technique tip popups
-
Interactive menu
- Scrollable technique selection list
- Cosmetic options
- Change scale of ground, or toggle its visibility
- Add textures
- Toggle modes
- Skeletal view
- Visualize individual limb velocities
- Center of mass
- Calculate center of mass and visualize via sphere projected onto ground
The Product
Playback
Skeletal View and Center of Mass
Cosmetic Changes
Reflection
There have been so many positive takeaways from this project. The ideation and iterative processes in designing this application were extremely valuable in tackling the project in manageable chunks rather than diving into it blindly.
Integrating real-world data into the Unity engine allows for an easy way to present the data via various platforms such as virtual reality, in-browser ports, or even mobile applications.
This project was a huge lesson in how to visualize data for users in a fun and intuitive way. Not to mention, it was fun to make! Thanks for reading! 💥
Recommended 🔎
