Booth Id:
TMED061
Category:
Translational Medical Science
Year:
2024
Finalist Names:
Vivek, Nithika (School: Del Norte High School)
Abstract:
The visual similarity of melanoma and seborrheic keratosis has made it difficult for elderly patients with disabilities to know when to seek medical attention, contributing to the metastasis of melanoma. In this project, we use multimodal deep learning techniques to distinguish between melanoma and seborrheic keratosis. Background research and past studies to diagnose skin disorders like acne versus eczema led to the hypothesis that the ResNet50 model combined with structured metadata would provide the best accuracy in distinguishing between the two skin conditions due to its ability to perform accurate texture analysis such as border patterns. The strategy is three-fold: (1) utilize patient image data to train and test three deep learning models using transfer learning (ResNet50, InceptionV3, and VGG16) and one author constructed model, (2) use patient metadata to train and test a deep learning metadata model, and (3) use non linear least squares analysis to specify optimal weights for combining predictions from the best performing image model with those from the metadata model. The accuracy of the combined model was 87% on test data with no lesion ID overlap between test and train. Results from this experiment could be used to eliminate late diagnosis of melanoma via easy, ubiquitous access to a smartphone app. Seeking early attention is vital to prevent metastasis. Future work can extend multimodal inputs to include text data in addition to the existing inputs (subjective data pertaining to how the patient felt over a certain period of time) thereby allowing the deep learning model to learn from real hospital settings.