Abstract Search

ISEF | Projects Database | Finalist Abstract

Back to Search Results | Print PDF

Artificial Intelligence Sense for the Blind to Play Video Games using Their Tongue

Booth Id:

Earth and Environmental Sciences


Finalist Names:
Silva, Nipun

Blindness forms a significant barrier to participate in physical activities including entertainment. Playing video games is one way of having fun. At present the current video game technology for blind is relying on sound cues, BUT NO SENSE OF VISION. We are familiar with augmentation or substitute of one sense for another such as eye glasses. This invention is using Tongue sensor as touch sense to take in information normally intended for another sense called VISION. The software that runs video game named as ‘GAMESTUDIO’ divided into 2 categories, RGD (Real Game Display) and GDT (Game display on Tongue), and both are divided into 100 pixels. The system comprises control device, signal processing unit, and tongue sensor with 100 electrodes. When playing video game, it capture the image of the display, remove the background color, and change into black and white picture using the software which create using C# and C++ programming languages. After analyzing the basic color of the image (Black, White and Grey) image convert into low and high resolution electrical pulse and send signals to the tongue sensor unit of 100 electrodes. These stimulate the tongue to represent the shape of the image which the computer displays. Each electrode corresponds to a set of pixels. Black pixels have strong pulse, grey pixels weak pulse and white pixels give no signals. Densely packed nerves on the tongue surface receive the incoming signals feels like a tingle or vibration on the tongue. Those signals then remain unclear and send to the brain’s visual cortex and the touch data of tongue surface is interpreted and build up the video and recreated the image from analysis of the impulse pattern, allow the user to play video game. Blind user might need to practice about 10-20 hours.

Awards Won:
Fourth Award of $500