Bio-LSTM: A Biomechanically Inspired Recurrent Neural Network for 3D Pedestrian Pose and Gait Prediction
Xiaoxiao Du · Ram Vasudevan · Matthew Johnson-Roberson
![Fig. 3: An illustration for our proposed network. This illustration is inspired by the network diagram in [45] with the network architecture modified for our specific design. The inputs and outputs of the network are vectors of SMPL paramet](/_next/image?url=%2Fpapers%2Fbio-lstm%2Fimage.webp&w=3840&q=75&dpl=dpl_8ygxTXqcm6LhJeKCMTS1uNCnV7Kk)
TL;DR
A recurrent network trained with gait periodicity, body symmetry, and ground-force cues predicts multi-pedestrian 3D body pose and location at urban intersections.
BRIEF
Self-driving vehicles need to anticipate where pedestrians will go and how their bodies will move. This work presents a recurrent neural network that takes noisy 3D pose and location estimates from earlier frames and predicts future full-body poses and global positions for several pedestrians at once, up to 45 m from the cameras. Training uses biomechanical cues: the periodic rhythm of walking, left-right body symmetry, and changes in ground reaction force over a gait cycle. Tests on PedX, a dataset from busy urban intersections, show accurate and consistent predictions.
Abstract
In applications, such as autonomous driving, it is important to understand, infer, and anticipate the intention and future behavior of pedestrians. This ability allows vehicles to avoid collisions and improve ride safety and quality. This letter proposes a biomechanically inspired recurrent neural network that can predict the location and three-dimensional (3-D) articulated body pose of pedestrians in a global coordinate frame, given 3-D poses and locations estimated in prior frames with inaccuracy. The proposed network is able to predict poses and global locations for multiple pedestrians simultaneously, for pedestrians up to 45 m from the cameras (urban intersection scale). The outputs of the proposed network are full-body 3-D meshes represented in skinned multi-person linear model parameters. The proposed approach relies on a novel objective function that incorporates the periodicity of human walking (gait), the mirror symmetry of the human body, and the change of ground reaction forces in a human gait cycle. This letter presents prediction results on the PedX dataset, a large-scale, in-the-wild data set collected at real urban intersections with heavy pedestrian traffic. Results show that the proposed network can successfully learn the characteristics of pedestrian gait and produce accurate and consistent 3-D pose predictions.