V-PRISM: Probabilistic Mapping of Unknown Tabletop Scenes

Herbert Wright · Weiming Zhi · Matthew Johnson-Roberson · Tucker Hermans

Figure 1: Our method takes a segmented (top left) point cloud observation (top right) and builds a continuous probabilistic map. This map can be used to reconstruct the scene (bottom left) or measure uncertainty about the scene (bottom righ — FIGURE FROM THE PAPER

TL;DR

V-PRISM builds probabilistic 3D segmentation maps of tabletop scenes from one segmented RGB-D image, giving object shapes and uncertainty.

BRIEF

Robots that manipulate objects on a table need a compact model of the scene, and they need to know how sure that model is. Many existing reconstruction methods give a shape but no reliable measure of uncertainty, which matters for safe motion planning. V-PRISM treats tabletop mapping as a multi-class classification problem over 3D space. From one segmented RGB-D image it builds a map with occupancy, object labels, and uncertainty. It outperforms continuous reconstruction methods that do not reason about objects in a multi-class way.

Abstract

The ability to construct concise scene representations from sensor input is central to the field of robotics. This paper addresses the problem of robustly creating a 3D representation of a tabletop scene from a segmented RGB-D image. These representations are then critical for a range of downstream manipulation tasks. Many previous attempts to tackle this problem do not capture accurate uncertainty, which is required to subsequently produce safe motion plans. In this paper, we cast the representation of 3D tabletop scenes as a multi-class classification problem. To tackle this, we introduce V-PRISM, a framework and method for robustly creating probabilistic 3D segmentation maps of tabletop scenes. Our maps contain both occupancy estimates, segmentation information, and principled uncertainty measures. We evaluate the robustness of our method in (1) procedurally generated scenes using open-source object datasets, and (2) real-world tabletop data collected from a depth camera. Our experiments show that our approach outperforms alternative continuous reconstruction approaches that do not explicitly reason about objects in a multi-class formulation.