SilhoNet: An RGB Method for 6D Object Pose Estimation

Gideon Billings · Matthew Johnson-Roberson

Fig. 1: Overview of the SilhoNet pipeline for silhouette prediction and 6D object pose estimation. The 3D translation is predicted in parallel with the silhouettes. The predicted unoccluded silhouette is fed into a second stage network to p — FIGURE FROM THE PAPER

TL;DR

SilhoNet predicts object silhouettes from a single RGB image and regresses 6D pose from them, outperforming two monocular baselines on YCB-Video.

BRIEF

Robots that manipulate objects need to know where each object is and how it is oriented, a 6D pose. Depth sensors help, but cost or the working environment sometimes rule them out, leaving only a single color camera. The method predicts an object's silhouette, including an occlusion mask, and its 3D translation from a cropped region of the image. The orientation is then regressed from the predicted silhouette. On the YCB-Video dataset, it performs better overall than two state-of-the-art monocular pose estimation networks.

Abstract

Autonomous robot manipulation involves estimating the translation and orientation of the object to be manipulated as a 6-degree-of-freedom (6D) pose. Methods using RGB-D data have shown great success in solving this problem. However, there are situations where cost constraints or the working environment may limit the use of RGB-D sensors. When limited to monocular camera data only, the problem of object pose estimation is very challenging. In this work, we introduce a novel method called SilhoNet that predicts 6D object pose from monocular images. We use a Convolutional Neural Network (CNN) pipeline that takes in Region of Interest (ROI) proposals to simultaneously predict an intermediate silhouette representation for objects with an associated occlusion mask and a 3D translation vector. The 3D orientation is then regressed from the predicted silhouettes. We show that our method achieves better overall performance on the YCB-Video dataset than two state-of-the art networks for 6D pose estimation from monocular image input.