Booth Id:
ROBO033
Category:
Robotics and Intelligent Machines
Year:
2021
Finalist Names:
Hua, Michelle (School: Cranbrook Kingswood School)
Abstract:
With a wide range of applications in artificial intelligence (AI), human action recognition is one of the most attractive yet challenging research fields. It has the potential to foster natural human-computer interaction, increase public safety, aid physical therapy, conduct sport analysis, etc. Human action is a spatio-temporal sequence with strong interdependencies between the spatial geometry and temporal dynamics of motion. Many AI methods have been proposed for human action recognition; however, in existing literature, there is a lack of synergy in investigating spatial geometry and temporal dynamics in a joint representation and embedding space. Therefore, I propose a novel dilated Silhouette Convolutional Neural Network (SCNN) for action recognition from a video. In SCNN, spatial geometric information of the moving human is modeled with silhouettes extracted from each frame of the video. The silhouettes are stacked along time and resampled to form a 3D point cloud, a unified spatio-temporal representation of the video action. With dilated silhouette convolutions, SCNN learns co-occurrence features from low-level geometric silhouettes and their temporal dynamics and constructs a unified convolutional embedding space, where spatial and temporal properties are integrated effectively. Geometry-based SCNN, complemented by an image-based approach, significantly improves the distinctiveness of learned features from motion. Experiment results on the public JHMDB, HMDB, and UCF101 datasets demonstrate the effectiveness of my method, outperforming all other state-of-the-art algorithms. I further integrate my SCNN model into a real-world product, an iOS app, to recognize actions and provide real-time coaching in fitness and physical therapy.
Awards Won:
First Award of $5,000
George D. Yancopoulos Innovator Award
Association for the Advancement of Artificial Intelligence: First Award of $1,500