Booth Id:
SOFT043
Category:
Systems Software
Year:
2024
Finalist Names:
Sun, Xiangzhou (School: Webb School of California)
Abstract:
People with Amyotrophic Lateral Sclerosis (PALS), who have severe motor and speech impairments, mostly rely on their eyes and assistive technology to communicate. However, existing high-tech products are expensive and hard to access, while low-tech products are inefficient and restrictive.
To mitigate the limitations, this research proposes GazeLink, a low-cost mobile application for PALS to communicate efficiently with only eye movements. First, the system recognizes user eye gestures like left or up with machine learning and a template-matching algorithm. Then, it converts the eye gestures to words through a keyboard that supports English, Spanish, and Chinese. For efficiency, the system employs Large Language Models (LLMs) to generate a suitable sentence with words typed by the user and the context. Finally, the system provides text-to-speech and social media post services for both verbal and digital eye-gesture communication.
Simulations conclude that sentence generation with LLMs can improve text-entry rate by 81% while maintaining 90% of semantic similarity. Usability studies with 30 participants show that GazeLink can recognize eye gestures with 94.1% accuracy in varying lighting. After rapidly learning the user interface in under 10 attempts, first-time participants typed sentences of various lengths with their eyes at 15.1 words per minute, which is 7.2x faster than the common low-tech solution E-Tran.
Experiments demonstrate GazeLink’s efficiency, learnability, and accuracy in eye-gesture text entry. The system is extremely affordable (<$0.1 a month), portable, and easily accessible online. It also supports different users, lighting, smartphones, and languages. Product testing with PALS and personalized LLM models will be the next step.