Booth Id:
SFTD061
Category:
Software Design
Year:
2026
Finalist Names:
Suzuki, Hanna (School: Bedford High School)
Abstract:
This project advances audio augmentation in augmented reality (AR) by developing a music-theoretic spatial sonification method. Although AR is increasingly used in education and entertainment, most research and applications rely almost exclusively on visual augmentation, with limited meaningful audio. Using astronomy as a testbed, this project addresses the gap by augmenting celestial objects such as constellations with musical representations derived from their astronomical features (e.g., position on the celestial sphere, size, brightness, and local deep-sky objects).
By translating astronomical data into musically coherent compositions based on music-theoretic principles (e.g., tonality and functional harmony), the method allows learners to perceive and explore celestial objects through auditory modalities. An accompanying AR device extends this experience beyond the classroom by supporting immersive interaction with celestial objects under the night sky through a virtual spatial soundscape. The device performs real-time object identification and audio-guided search by rendering 3D sounds that appear to originate from celestial objects.
A user study with 116 participants shows that the method is effective even for novices, with consistently high accuracy across recognition tasks (92-93%). Participants also exhibit a 68 percentage-point increase in perceived connection between constellations and their music, along with strong engagement, increased motivation, and perceived educational value of multimodal learning. Field testing demonstrates the feasibility of the integrated system, including reliable constellation identification and search, responsive spatial audio rendering, and immersive audio-guided exploration under nighttime observation conditions.
Awards Won:
Fourth Award of $600
Association for Computing Machinery: Fourth Award of $500