Booth Id:
SFTD003T
Category:
Software Design
Year:
2026
Finalist Names:
Gunes, Sinan (School: Batman Turk Telekom Anatolian High School)
Bilir, Eyup (School: Batman Turk Telekom Anatolian High School)
Aygun, Eren (School: Batman Turk Telekom Anatolian High School)
Abstract:
Severe Speech and Motor Impairment (SSMI), including conditions like advanced-stage ALS, progressively traps individuals in their bodies. This loss of muscle control and communication ability significantly reduces quality of life and increases social isolation. Current eye-tracking assistive systems are often expensive, complex to set up, and reliant on external hardware, which severely limits their accessibility for disadvantaged individuals. This project utilizes a standard smartphone camera and a multi-stage vision pipeline. Raw frames undergo grayscale conversion, contrast enhancement, and Gaussian blurring to maximize pupil-sclera distinction. Leveraging MediaPipe at the native layer, the system extracts facial landmarks, while the Perspective-n-Point (PnP) algorithm enables 3D head pose estimation for posture-invariant tracking. Stable control is achieved through linear regression and consecutive frame averaging. A Visual tab provides pictogram-based communication for those with low literacy. Furthermore, a Social tab operates as a messaging platform, empowering isolated patients to text distant relatives and connect with peers. For maximum privacy, an on-device Local LLM generates context-aware sentences, supporting 10 languages. Evaluated on 108 participants (including 8 SSMI patients), the system achieved 96.3% detection accuracy (94.1% with reflective glasses) and 93.1% communication success. Ultimately, these findings prove that smartphones can serve as a highly scalable, cost-effective, and accessible alternative to traditional AAC systems.
Awards Won:
Fourth Award of $600
Association for the Advancement of Artificial Intelligence: AAAI Membership for the School Libraries of All 8 Winners (in-kind award / part of 1st-3rd prize and honorable mentions' prize)