Booth Id:
ROBO031
Category:
Robotics and Intelligent Machines
Year:
2022
Finalist Names:
Lin, Yutang (School: Beijing 101 Middle School)
Abstract:
Memory impairment caused by Alzheimer's disease seriously affects the daily life and even safety of the elderly. For example, if the emergency medicine cannot be found in time, the valuable emergency time window will be missed. Existing object finding technology based on either computer vision or wireless IoT cannot solve the problem of object occlusion and long time-span. Moreover, it is generally believed that human behavior is the fundamental reason for the invisibility and position change of objects. In this project, we propose a new object finding method based on neural-symbolic reasoning, which predicts object position by inferring both human-object interaction and object state change. In the neural perception part, human actions based on 3D skeleton representation are recognized by a carefully-designed recurrent network. While in the symbolic reasoning part, the interaction rules between human, target object and context objects (e.g., table and other containers) are modeled by a probabilistic logic framework. For model training, we implement joint neural-symbolic optimization by seamlessly integrating the parameter learning of probabilistic logic programming with back propagation of neural network for behavior recognition. To verify the effectiveness of the proposed method, we collect an object finding dataset including 17 sets of 4-camera videos for total 15GB. The comparison results on this dataset demonstrate that our method can find objects more efficiently and accurately in complex scenes with long time-span and occlusion than traditional computer vision-based methods. Our neural-symbolic framework can also provide better interpretability and stronger generalization capabilities.