Booth Id:
TECA016
Category:
Technology Enhances the Arts
Year:
2024
Finalist Names:
Huang, Henry (School: Palos Verdes Peninsula High School)
Abstract:
Classical music education is a complex field requiring the teacher to evaluate a student based on audio and visual information. However, this approach can be error prone and biased. This project proposes a framework of a 3-component piano performance analyzer that can provide feedback to students without the input of an instructor.
The core component for analyzing the performance was focused on in this project. Training data was synthesized from existing piano performance datasets by randomly generating errors within the pieces. Two classes of errors were categorized, with simple errors describing one note, and high-level errors describing a series of notes. To predict these errors, multi-class and multi-label networks composed of difference modules and inception modules inspired by GoogleNet were created using 1D Convolutions.
The model for predicting simple errors was able to reach an accuracy level of 97%, where accuracy was measured in how closely the output of the model matched the error codes for the piece. The accuracy of the high-level error model peaked at 90%.
The results validate the current architecture of the analyzer component. Including the other components for generating an optimal performance and translating the student performance would complete the full model. For the analyzer component, future models could experiment with using a deeper network or combining it with a large language model to generate text descriptions of critique.