Booth Id:
TECA005
Category:
Technology Enhances the Arts
Year:
2024
Finalist Names:
Lu, Minghan (School: Shanghai Foreign Language School Affiliated to SISU)
Abstract:
To tackle the decline of the art of shadow puppetry, this project provides a systematic way of preserving and promoting the art form. The procedure includes repairing fragmented recordings, capturing key points from the processed videos, and perform the content of the videos in real life using a mechanical structure.
This project first endeavors to introduce an innovative AI-driven video generation technique specializes at transforming static images into continuous and dynamic video sequences. The model architecture comprises an image pair generator leveraging StyleGAN2 to produce sequential images; motion codes derived from the Jacobian matrix and low-rank factorization of the generator; and a video generator that iteratively manipulates image pairs using the motion codes. Results show that this model can be effectively used to repair shadow puppetry videos, and can serve as a novel way of art creation as well.
Using AI action recognition technology on the repaired videos, key-points controlling the shadow puppetry characters are extracted from the repaired or generated videos. Then, a mechanical structure made up of three cross-slide tables controlling puppetry characters according to the obtained key-points is designed. With that, we are able to re-enact the way of performance of shadow puppetry in real life. This provides a real and beautiful shadow puppetry experience, while the content of the performance is the same as generated videos.
In this way, a new, easy to use and advanced way of preserving, creating and performing the art of shadow puppetry is obtain. It not only can be used to help promote the beauty of the art and gain public attention and love, but also serve as an important step of modernization of this time-honored art.