Auscult AI
Summary of metrics
82.1%
Overall accuracy of best performing LSTM model on 234 test samples
15
Participants tested using AuscultAI on Android and iOS
Technologies: Flutter, Python, Flask, Tensorflow, Librosa
Tags: Mobile App Development, Machine Learning, Audio Classification, Heart Audio Signals, Thesis
As part of completing my bachelor's in computer science, I was tasked to lead and manage my group's thesis. Our concept revolved around bringing accessible heart diagnostics to the Iloilo City public. It works as a preliminary evaluation tool that can assist a physician's standard check-up evaluation.
The work resulted in the design and development of Auscult AI, a mobile application that classifies the heart for abnormality. The app records audio via the phone's external microphone, sends it to a trained machine learning model, and returns its classification and confidence level. The class label can either be normal or, if abnormal, be a murmur, extra systole, extra heart sound, or artifact. The labels were based on previous literature that tackled heart audio signal classification.
Our lead UI/UX, Gem Win, designed the branding and Figma prototype. With my input, she followed an approach to simplicity and clarity. Designing the interface for middle to old age participants required that element emphasis be used only to highlight function importance and guide users to understand a process. A good example is how she designed the instructions on how to record the heart using the app. Succinct labels and clear visuals made it easy for users to onboard and confidently start recording.
While she designed the interfaces, I was the one who developed it. I chose Flutter because of familiarity. It was also faster to produce a working app thanks to the plethora of widgets on pub.dev. The most important ones were the audio player and audio visualizer. While the app was intended for Android, Gem helped to configure it for the iPhone. We were then able to test the app on both devices, with a total of 15 participants sampled and evaluated by cardiologists for their heart audio recordings.
Training the classification model was rigorous. Aaron, together with Ralph Pablo and Car Torres, developed an ETL pipeline to organize the process from data cleaning to data extraction and transformation. The final model we configured was a bi-directional LSTM, capable of analyzing sound patterns forward and backward. Our metric on overall accuracy reached as high as 82.1% against 234 test audio samples. The best performing model was then later utilized for the app's classification of heart audio recordings.
