PedX: Benchmark Dataset for Metric 3D Pose Estimation of Pedestrians in Complex Urban Intersections
Wonhui Kim · Manikandasriram Srinivasan Ramanagopal · Charles Barto · Ming-Yuan Yu · Karl Rosaen · Nick Goumas · Ram Vasudevan · Matthew Johnson-Roberson
TL;DR
PedX is a stereo and LiDAR dataset of pedestrians at urban intersections, with metric 3D body models produced by a multimodal fitting method.
BRIEF
Self-driving systems need to know where pedestrians are and how their bodies are posed, especially at busy intersections where people and vehicles interact closely. PedX collects more than 5,000 pairs of 12MP stereo images and LiDAR scans from three four-way stop intersections, with 2D labels and 3D pedestrian labels in a global frame. A model-fitting method combines image and LiDAR constraints with shape and temporal priors to produce metric 3D body models automatically. A motion capture experiment validates these models, and automated 2D labeling can replace manual annotation.
Abstract
This letter presents a novel dataset titled PedX, a large-scale multimodal collection of pedestrians at complex urban intersections. PedX consists of more than 5 000 pairs of high-resolution (12MP) stereo images and LiDAR data along with providing two-dimensional (2-D) image labels and 3-D labels of pedestrians in a global coordinate frame. Data were captured at three four-way stop intersections with heavy pedestrian-vehicle interaction. We also present a 3-D model fitting algorithm for automatic labeling harnessing constraints across different modalities and novel shape and temporal priors. All annotated 3-D pedestrians are localized into the real-world metric space, and the generated 3-D models are validated using a motion capture system configured in a controlled outdoor environment to simulate pedestrians in urban intersections. We also show that the manual 2-D image labels can be replaced by state-of-the-art automated labeling approaches, thereby facilitating automatic generation of large scale datasets.