IPFit.

BrainHack is a hackathon that is hosted by Defence Science and Technology Agency (DSTA), a technology organisation that supports the Singapore Armed Forces (SAF).

DSTA actively look for ways to improve the life of servicemen and in the summer of 2022, this meant looking into helping the largest, inactive part of the SAF, the NSMen.

I joined this hackathon along with my friends with the intention of learning new technologies. The hackathon hit home when we realised that it revolved around solving the needs of NSMen, as some of us were NSMen ourselves.

We were asked to present a solution to improve the quality of life of NSMen by leveraging digital technologies on mobile platforms.

A common problem faced by NSMen is scoring in the Individual Physical Proficiency Test (IPPT), a test that consists of 3 stations, push-ups, sit-ups and a 2.4km run. Participants are awarded Gold, Silver, Pass with Incentive, or Pass depending on their performance in the test.

The ELISS machine, which is a machine used to determine one's repetition count in the push-up and sit-up station, leaves no leeway for form discrepancies in either station, as a result, a lot of IPPT participants are penalised for bad form.

Thus, we chose to focus on the physical training aspect of NSMen, specifically push-ups and sit-ups. We thought that it would be helpful for future IPPT participants to be able to check their forms for push-ups and sit-ups with their smartphones, whenever and wherever. It is natural to be out of shape after their 2 years of service, hence are not able to perform in their IPPTs to the best of their ability.

IPFit is an easy-to-use Artificial Intelligence powered fitness app designed for NSMen that provides the user with personalised workout plans and an easy to use form corrector for push-ups and sit-ups. This app would be the first of its kind on the market as well.

While we all participated in ideation, implementation of ideas and visual design phases, I was mainly responsible for the visual design, UI/UX, front-end programming of the app.

Ideation

We brainstormed ideas, product sketches and prioritized important features for our app.

App Design

I was responsible for contributing to establishing a visual design guideline (typography, colours) for the app’s user interface.

Front-End Development

I was also responsible for the implementation of the website design via front-end technologies.

After coming to a consensus on how to solve the problem and the UI design of the app, I started to work on the front-end code of the app, which took the bulk of my time.

This was my first time using React.JS for a project, and implementing reuable components was a game-changer for me, especially since coming from vanilla JavaScript and HTML.

Implementing the horizontal scrollbar menu and the sticky exercise header bar upon scrolling were one of the harder features that I had to implement in this project.

We utilised MoveNet, a Computer Vision model that detects 17 keypoints of a body and can run client-sided at 50+ fps on modern laptops and phones.

Push-up posture detection.

Points used for left side detection: 13 for knee, 11 for hip, 5 for shoulder

Points used for right side detection: 14 for knee, 12 for hip, 6 for shoulder

The range of angle for "correct" push-ups was between 172 degrees amnd 190 degrees. Any angle outside this range would be deemed as "incorrect" and user would be prompted to fix their posture.

Sit-up Posture Detection

Points used for left side detection: 15 for feet, 13 for knee, 11 for hip, 7 for elbow

Points used for right side detection: 16 for feet, 14 for knee, 12 for hip, 8 for elbow

A sit-up would be labeled as "correct" if the distance between your elbow and knee at the upright position was lesser than 15% of the width of the video stream, and the hip and your feet were at approximately the same level throughout the motion of the exercise.

Documentation for Model Used

Hackathons are a lot of fun. We were constantly forced to think outside the box, and give thought to aspects of product design that a programmer wouldn't necessarily be required to think about.

Using TensorFlow.JS' joint detection library, we can correct the postures of more than just push-ups and sit-ups. As long as the camera of one's mobile device captures the body of the user, the exercises that could be added to our app are endless.

It was really rewarding to know that we finished amongst the top 10 teams, with over a 100 participating teams, in our first hackathon. Big thanks to DTSA for making our group's first hackathon a great and memorable experience!