Pedestrian Planar LiDAR Pose (PPLP) Network for Oriented Pedestrian Detection Based on Planar LiDAR and Monocular Images
Fan Bu · Trinh Le · Xiaoxiao Du · Ram Vasudevan · Matthew Johnson-Roberson
TL;DR
A low-cost pipeline combining a 2D planar LiDAR and one camera detects pedestrians and their orientation about as well as 3D-LiDAR methods.
BRIEF
Robots and self-driving cars need to know where pedestrians are and which way they face, but most detectors depend on expensive 3D LiDAR. This work uses a cheaper 2D planar LiDAR together with a single camera. One network estimates each person's body orientation from the image, another turns the LiDAR scan into an occupancy grid and proposes 3D boxes, and a final network combines both. On two datasets, accuracy is similar to or better than 3D-LiDAR-based methods, indoors and outdoors.
Abstract
Pedestrian detection is an important task for human-robot interaction and autonomous driving applications. Most previous pedestrian detection methods rely on data collected from three-dimensional (3D) Light Detection and Ranging (LiDAR) sensors in addition to camera imagery, which can be expensive to deploy. In this letter, we propose a novel Pedestrian Planar LiDAR Pose Network (PPLP Net) based on two-dimensional (2D) LiDAR data and monocular camera imagery, which offers a far more affordable solution to the oriented pedestrian detection problem. The proposed PPLP Net consists of three sub-networks: an orientation detection network (OrientNet), a Region Proposal Network (RPN), and a PredictorNet. The OrientNet leverages state-of-the-art neural-network-based 2D pedestrian detection algorithms, including Mask R-CNN and ResNet, to detect the Bird's Eye View (BEV) orientation of each pedestrian. The RPN transfers 2D LiDAR point clouds into occupancy grid map and uses a frustum-based matching strategy for estimating non-oriented 3D pedestrian bounding boxes. Outputs from both OrientNet and RPN are passed through the PredictorNet for a final regression. The overall outputs of our proposed network are 3D bounding box locations and orientation values for all pedestrians in the scene. We present oriented pedestrian detection results on two datasets, the CMU Panoptic Dataset and a newly collected FCAV M-Air Pedestrian (FMP) Dataset, and show that our proposed PPLP network based on 2D LiDAR and monocular camera achieves similar or better performance to previous state-of-the-art 3D-LiDAR-based pedestrian detection methods in both indoor and outdoor environments.