Booth Id:
ROBO043T
Category:
Robotics and Intelligent Machines
Year:
2026
Finalist Names:
Ahmed, Mohamed (School: Shree Cutchi Leva Patel Samaj School)
Singh, Tejveer (School: Shree Cutchi Leva Patel Samaj School)
Murigi, Sean (School: Shree Cutchi Leva Patel Samaj School)
Abstract:
Kenyan Sign Language (KSL) remains absent from AI research despite serving 600,000 Deaf Kenyans. Existing recognition systems, including those for ASL and BSL, fail under real-world conditions by assuming fixed camera angles, controlled lighting, and unoccluded signers. No multi-angle KSL dataset or robustness study under environmental degradation exists in the published literature.
This study presents MARS-KSL, a framework to maintain accuracy under real-world Kenyan conditions including classrooms, hospitals, and public spaces. A structured dataset of 50 signs was collected from 8 signers across five camera angles (0°, 30°, 45°, 60°, 90°), three lighting conditions, and two background complexity levels, constituting the first multi-angle KSL dataset in the literature.
Hand and body landmarks were extracted using MediaPipe and normalised through a view-invariant coordinate transformation anchored to the mid-shoulder axis, reducing inter-angle variance prior to model input. A hybrid LSTM-Transformer architecture processed the normalised sequences, with LSTM layers capturing temporal dynamics and Transformer layers weighting the most informative frames.
The hybrid model achieved 86.3% recognition accuracy at frontal angle, with drops of only 4.6 and 11.5 percentage points at 45° and 90° respectively, compared to a 23.1 point drop in the LSTM-only baseline at 90°. View-invariant normalisation reduced hand landmark variance by 61.2% (t(41) = 12.83, p < 0.001, Cohen's d = 1.94). Real-time inference reached 23.4 fps on a mobile device.
These results demonstrate that view-invariant landmark normalisation significantly reduces the accuracy penalty of off-axis viewing, establishing a reproducible framework for low-resource African sign language recognition systems.
Awards Won:
Fourth Award of $600