Booth Id:
SFTD026
Category:
Software Design
Year:
2026
Finalist Names:
Joseph, Ethan (School: Engineering and Science University Magnet School)
Abstract:
Alzheimer’s disease (AD) is a progressive neurodegenerative disorder characterized by heterogeneous cognitive decline, including episodic memory loss, semantic impairment, and visuospatial dysfunction. While existing assistive technologies primarily emphasize reactive safety, they lack proactive, context-aware semantic support and often rely on cloud-based processing, raising privacy concerns. This study presents the development and evaluation of a fully offline, privacy-aware multimodal mobile assistant designed to mitigate object and face recognition deficits in individuals with AD.
The system integrates MobileVLM for on-device multimodal reasoning, a FaceNet-based biometric pipeline with MTCNN face detection for prosopagnosia support, and ChromaDB for on-device retrieval-augmented generation (RAG). All components operate locally using performance optimizations to ensure efficient execution within resource-constrained mobile environments. A voice interface enables hands-free interaction using native on-device speech recognition.
Evaluation across a large-scale corpus of clinical records demonstrated that the RAG-based framework effectively retrieved contextual data, significantly outperforming standard keyword-filtered LLM approaches. Passive linguistic monitoring successfully detected markers of mild cognitive impairment progression with high reliability. Real-time face identification demonstrated near-perfect classification accuracy while maintaining low-latency inference.
These results demonstrate the feasibility of a fully localized “memory prosthesis” that provides proactive semantic assistance while preserving patient privacy. The framework establishes a scalable foundation for integrating edge-based multimodal AI into Alzheimer’s care.
Awards Won:
Association for the Advancement of Artificial Intelligence: AAAI Membership for the School Libraries of All 8 Winners (in-kind award / part of 1st-3rd prize and honorable mentions' prize)
Association for the Advancement of Artificial Intelligence: Third Award of $500