A probabilistic framework for intrinsic image decomposition from RGB-D streams
Wonhui Kim · M. Johnson-Roberson
TL;DR
Intrinsic image decomposition for RGB-D video is cast as recursive Bayesian filtering, giving more temporally consistent results than frame-by-frame optimization.
BRIEF
Shadows and lighting change how an object looks, which can confuse robots that try to recognize or match objects indoors. Intrinsic image decomposition separates a scene into surface color and shading. Most methods process each frame on its own, solving a costly optimization every time. This work treats decomposition of an RGB-D video stream as a Bayesian filtering problem, using past frames to predict the current one. Results are more consistent over time, and the filter fits into the recursive loops common in robot perception.
Abstract
Lighting and shading can have a great impact on a robot's ability to recognize, match, and classify objects in indoor scenes. Additionally, the realism of augmented reality applications benefit greatly from understanding the lighting and shading in a scene. To aid in extracting this information from a mobile platform we introduce a novel framework to solve the intrinsic image decomposition problem for RGB-D streams. In our pipeline, the task is formulated as a Bayesian estimation problem. Compared to frame-based methods that must solve a full conditional random field (CRF) optimization problem at each time step, our framework can utilize the knowledge of past frames to predict the intrinsic images at a given frame. Our approach produces more reliable and consistent predictions over time, and our filtering-based framework achieves significant performance gains. Furthermore, our framework can be easily integrated into standard perception loops in many robotic systems that use a similar recursive filtering structure. We show qualitative results on real data and generate quantitative results using ground truth from a photorealistic synthetic dataset produced using a state-of-the-art ray tracer and high fidelity 3D model.