Abstract Search

ISEF | Projects Database | Finalist Abstract

Back to Search Results | Print PDF

A Holistic Multi-Modal GenAI System: Early Detection and Predictive Treatment Monitoring Using Chest X-rays and Cough Audio for Tuberculosis

Booth Id:
ENBM019

Category:
Biomedical Engineering

Year:
2024

Finalist Names:
Suda, Chandra (School: Bentonville High School)

Abstract:
Tuberculosis (TB), a bacterial infectious disease, is one of the top 10 causes of mortality worldwide in low-income countries, resulting in approximately 10 million new infections and 1.4 million deaths. I updated the ML model to include multi-modality (text and audio input) as well as additional feature extraction (mel-spectrograms) and data augmentation (IR-convolution). I created a mobile and web app to integrate the model, and the results are available within 15 seconds. After preliminary testing, the model achieved an area under the receiving operator characteristic curve (AUROC) of 88%, surpassing the WHO requirements for screening tests. I also integrated my new research on chest radiography (CXR) tools. I created a novel 2D convolutional neural network (2D-CNN) to identify and forecast subsequent incident TB using CXR. Predicting the risk of active TB long before symptoms enables preventive treatment that can be administered earlier (6-12 months before symptoms). I trained the model on the O2 high-performance cluster. I implemented Focal Loss and class weightage to counteract the high-class imbalance. The optimal 2D-CNN included a Gated Activation Unit and a Multi-Head Self-Attention (heads=8), performing with a specificity of 93.7%, a sensitivity of 75.8%, and an AUC ROC of 83.7%, showcasing the strong potential for using CXRs in contact tracing. I have integrated these two models into a generative AI system powered by a custom LLM I created using the LLaMa2 open-source and a Retrieval Augumentaed Generation vector database. This provides more up-to-date knowledge and specific facts for the LLM, creating a comprehensive patient treatment and monitoring system, and increasing access to quality healthcare worldwide, especially in low/middle-income areas.