In the United States alone, there are half a million competitive, and 8 million school-level high school student athletes. The massive majority have no access to affordable coaching in their respective sports.
My personal project aims to create and implement software capable of analyzing athletes and provide feedback.
First, empathize with the expected consumers of this app. I interviewed a multitude of athletes across various sports.
Athlete Testimonies
“Coaching nowadays, regardless of what sport, is expensive … If you want to take it seriously in any competitive manner. In addition to equipment cost, private and group lessons are inaccessible and are becoming more and more expensive” – Vincent (Badminton)
“Coaching is extremely pricy and expensive. It is difficult to find affordable and accessible coaching.” – Felix (Hockey)
“Lessons are really hard to find and way too expensive so I love the idea of your project.” – Josh (Bodybuilder)
After these interviews, I then created a basic project outline and transposed it into Notion, an AI powered calendar app, most used for managing projects. My notion project page can be found at the bottom of the page or here
First step of the process was to choose a stack for the software to be built on. For simplicity, I chose python, centered around the use of mostly Mediapipe but some OpenCV libraries for computer vision.
Then, I created what is now known as the “Live Presentation” function as a prototype. The program takes a video as input and uses the libraries to draw the skeletons in real time.
This “skeleton” prototype was completed by March 11th, as shown in the Notion calendar, and was the first step. After this proof of concept, it was off to the races. To analyze this data, I had to first transpose these landmarks into usable data. Here, I was faced with whether I should use .json or .csv formatting.
| JSON | CSV | |
| Size/Transmission | More verbose, larger file size | Lightweight and more efficient. |
| Readability | Strong, supports nested structures for increased readability. | Flat format with rows and columns. |
| Ease of Automation | Easily parsed with other coding languages. | Simple parsing but difficult to automate because of inconsistencies. |
| Error Handling/Ease of use | Extremely strict, difficult to find errors. | Very forgiving but possible formatting issues |
| Cost | 8-10x higher than CSV | ~$2-3 / 1 minutes |
The largest factor for this was cost, so I chose to go with csv. However, it was still too expensive so I had to optimize it. Firstly, the data was being truncated at 10 decimal points, causing unnecessary precision. So, I cut it to 3 decimals to retain the majority of the information. Then, I added frame interval, (min 5 max 20) as I found a lot of repeated information by analyzing each frame individually in the csv. After this, I conducted tests comparing the accuracy of the csv before and after and found that it lost almost no information at all (99.8%).
An example of the final csv can be found here
Then I created modules and INPUT/OUTPUT files for organization and the ability to re-analyze output files without the need for a new csv file. Then, I used pillow to create an extremely basic GUI which allows the user to select a video through a dropdown, choose frame intervals, enable/disable presentation mode, and enable a demo mode which uses the front camera to draw skeletons live.
For the backend analysis, I wanted to create a form of Artificial Intelligence that is capable of efficiently and cheaply analyze the csv files. For the prototype, I packaged the program with OPENAI’s API with an extremely specific prompt as well as optional context statements(shown above). The cost of the API was extremely cheap after all csv optimizations, with all prototype runs (~25 15 sec videos) costing around $0.85
Lastly, I implemented a basic landmark system to assist in finding accurate perspectives. However, I found that it did not benefit the accuracy meaningfully and required significantly more processing power.
Testing
After some preliminary tests, it seems as if the coaching is both effective and accurate. Additionally, it agrees with what other coaches are saying and provides accurate and useful exercises/drills.
Next Steps
Firstly, I must finish the AI analysis specific to sports, with options of selecting models designed for specific movements.
Moving forward, I will be creating a more accessible and intuitive user interface as well as porting the app to IOS.. I will also be creating an alternate version of the software that analyzes teams instead of individual players for coaches to use.
Notion Calendar Link: https://www.notion.so/Fusion-Personal-Project-1a5c9e2849ab80e5b3c8ece4c8f27cda?pvs=4
A link to the source code and the testing footage used can be found here: https://drive.google.com/file/d/1-Q5tdt8B53DZBy9-cMTdTv-mtEP0Ohml/view?usp=sharing
Leave a Reply to mcrompton Cancel reply