Efficient Construction of Implicit Surface Models From a Single Image for Motion Generation

Wei-Teng Chu · Tianyi Zhang · Matthew Johnson-Roberson · Weiming Zhi

Fig. 1: We present Fast Image-to-Neural Surface (FINS), an efficient framework ($\sim$10s on consumer-grade hardware) that can reconstruct high-fidelity surfaces and SDF fields based on sparse or even a single image. Top row: Input RGB imag — FIGURE FROM THE PAPER

TL;DR

FINS builds a neural signed distance field from a single image in about ten seconds, enabling robot surface-following motion.

BRIEF

Robots often need a distance field of nearby surfaces to plan safe motions, but building one with neural implicit methods usually takes many views and long training. FINS reconstructs a surface and its signed distance field from a single RGB image, or a few images, in roughly ten seconds on consumer hardware. It pairs a hash-grid encoder and small prediction heads with a fast approximate second-order optimizer, and uses pre-trained 3D foundation models to estimate geometry from the image. The authors report better speed and accuracy than baselines, and show a robot arm following reconstructed surfaces.

Abstract

Implicit representations have been widely applied in robotics for obstacle avoidance and path planning. In this paper, we explore the problem of constructing an implicit distance representation from a single image. Past methods for implicit surface reconstruction, such as NeuS and its variants generally require a large set of multi-view images as input, and require long training times. In this work, we propose Fast Image-to-Neural Surface (FINS), a lightweight framework that can reconstruct high-fidelity surfaces and SDF fields based on a single or a small set of images. FINS integrates a multi-resolution hash grid encoder with lightweight geometry and color heads, making the training via an approximate second-order optimizer highly efficient and capable of converging within a few seconds. Additionally, we achieve the construction of a neural surface requiring only a single RGB image, by leveraging pre-trained foundation models to estimate the geometry inherent in the image. Our experiments demonstrate that under the same conditions, our method outperforms state-of-the-art baselines in both convergence speed and accuracy on surface reconstruction and SDF field estimation. Moreover, we demonstrate the applicability of FINS for robot surface following tasks and show its scalability to a variety of benchmark datasets. Code is publicly available at https://github.com/waynechu1109/FINS.