Booth Id:
SOFT036
Category:
Systems Software
Year:
2022
Finalist Names:
Warren, Glen (School: Dearborn Center for Math, Science, and Technology)
Abstract:
As technology increasingly impacts daily life, technological interaction is critical to the future of equity and health. In an effort to advance this interaction, the researcher sought to design a versatile and customizable system for computer control, powered by webcam based gesture recognition. Replacing or augmenting traditional mouse control with a flexible computer vision system could offer more versatile and natural control. A non-contact system could also reduce repetitive motion injuries, promoting equity for those with hand injuries or disabilities.
Python was selected as the language for the system due to available gesture recognition and computer control libraries. Open Computer Vision (OpenCV) was a critical library for the system’s base recognition, predicting hand position via landmarks. Open source python libraries enabled creation of a user interface for customization of control features.
The final prototype proved promising but unrefined. The system succeeded in detecting hand position changes and executing computer control within 10 milliseconds. This was the case for most functions under normal lighting conditions, and proved functional for real-time computer control. However, some sensitive landmark positions and rigid gesture recognition caused misinterpretation of gestures. The prototype also required specific hand movements which harmed its ergonomic potential, failing to remove hand stresses.
This project revealed that camera-powered gesture recognition with current technology is promising but requires refinement to resolve recognition errors and improve natural interaction. The findings support future research and improvement, anticipating future benefits for versatility, customization, ergonomics, and equity of computer control.