Booth Id:
ROBO059
Category:
Robotics and Intelligent Machines
Year:
2019
Finalist Names:
Meng, Kevin (School: Plano West Senior High School)
Abstract:
Overcoming the visual barrier and developing “see-through vision” has been one of mankind’s long-standing dreams. Unlike visible light, Radio Frequency (RF) signals penetrate opaque obstructions and reflect highly off humans.
This project created a breakthrough artificial intelligence model that can be trained to reconstruct continuous video of a 15-point human skeleton even through visual occlusion using RF signals.
The AI training process adopted a student/teacher learning procedure inspired by the Feynman Technique for Learning. Video frames and RF data were first collected simultaneously using a co-located setup containing an optical camera and an RF antenna array transceiver. Next, the video frames were processed with a computer-vision-based gait analysis “teacher” module to generate ground-truth human skeletons for each frame. Then, the same type of skeleton was predicted from corresponding RF data using a “student” deep-learning model consisting of a Residual Convolutional Neural Network (CNN), Region Proposal Network (RPN), and Recurrent Neural Network with Long-Short Term Memory (RNN-LSTM) that 1) extracted spatial features from RF images, 2) detected all people present in a scene, and 3) aggregated information over multiple time-steps, respectively.
After reducing error between teacher-provided ground-truths and student-created predictions over thousands of training iterations, the model was shown to be capable of accurately and completely predicting the pose of any human behind visual obstruction solely using RF signals.
Primary academic contributions include the novel many-to-many imaging methodology, unique RPN/RNN-LSTM integration, newly proposed objective function, and original training pipeline. A simulator was also created.
Awards Won:
First Award of $3,000
Association for the Advancement of Artificial Intelligence: First Award of $1,500
Office of Naval Research on behalf of the United States Navy and Marine Corps: The Chief of Naval Research Scholarship Award of $10,000
National Security Agency Research Directorate : First Place Award "Cyber Pioneer" of $1000
Association for Computing Machinery: Fourth Award of $500
Intel ISEF Best of Category Award of $5,000