Booth Id:
TECA010
Category:
Technology Enhances the Arts
Year:
2026
Finalist Names:
Lee, Jiho (School: North London Collegiate School Jeju)
Abstract:
While technology has revolutionised music education, visually impaired musicians remain marginalised due to the visual-centric design of existing practice platforms. To restore artistic independence, this project developed Synchronome, a novel music practice application integrating real-time audio analysis with auditory-tactile feedback to enable independent accuracy enhancement without visual reliance. The Optical Music Recognition (OMR) mechanism involves converting uploaded PDF scores into an encoded MusicXML structure, enabling pitch, rhythm, and notation data extraction. Incoming user performance audio is analysed in real time using a Digital Signal Processor (DSP), where waveform manipulation algorithms extract fundamental frequencies and determine note onsets for continuous comparison with the expected score data. Upon the detection of user deviations in pitch and rhythm, immediate feedback is delivered through a specialised haptic signal system alongside complementary auditory cues, allowing users to quickly recognise and correct errors. In addition, aggregated performance data are used to create personalised feedback via a generative artificial intelligence model, delivered post-performance through Text-to-Speech (TTS) output. Prototype testing confirmed the system’s ability to analyse and detect pitch and rhythm deviations with low latency, achieving a mean error detection accuracy of 95%. The haptic response system and TTS output also functioned reliably, supporting users in interpreting feedback. These positive results ultimately demonstrate that Synchronome’s multimodal, real-time feedback system not only improves technical music proficiency but fosters an inclusive musical society, proving how technology can reconstruct independent artistic expression.
Awards Won:
Third Award of $1,200