RecGS: Removing Water Caustic with Recurrent Gaussian Splatting

Tianyi Zhang · Weiming Zhi · Braden Meyers · Nelson Durrant · Kaining Huang · Joshua G. Mangelson · Corina Barbălată · Matthew Johnson-Roberson

Fig. 3: Our proposed recurrent 3DGS workflow: we build a vanilla 3DGS model with caustics in the images first, then find the residual between captured image and rendered view. We run 2D FFT on the residual image, and reconstruct it with onl — FIGURE FROM THE PAPER

TL;DR

RecGS repeatedly fits a 3D Gaussian Splatting model and filters the residual to strip water caustics from underwater robot imagery of the seafloor.

BRIEF

In shallow water, sunlight refracted by surface waves paints shifting bright patterns, called caustics, onto the seafloor. These patterns corrupt images taken by underwater robots. Common fixes filter images in 2D or rely on networks trained on annotated data, and both struggle with real 3D terrain. The authors build a 3D Gaussian Splatting model from a robot's image sequence, compare it with the captured images, and low-pass filter the residual to isolate caustics. They subtract that estimate and repeat. Experiments show cleaner, more evenly lit seafloor imagery than the alternatives.

Abstract

Water caustics are commonly observed in seafloor imaging data from shallow-water areas. Traditional methods that remove caustic patterns from images often rely on 2D filtering or pre-training on an annotated dataset, hindering the performance when generalizing to real-world seafloor data with 3D structures. In this letter, we present a novel method Recurrent Gaussian Splatting (RecGS), which takes advantage of today's photorealistic 3D reconstruction technology, 3D Gaussian Splatting (3DGS), to separate caustics from seafloor imagery. With a sequence of images taken by an underwater robot, we build 3DGS recurrently and decompose the caustic with low-pass filtering in each iteration. In the experiments, we analyze and compare with different methods, including joint optimization, 2D filtering, and deep learning approaches. The results show that our proposed RecGS paradigm can effectively separate the caustic from the seafloor, improving the visual appearance, and can be potentially applied on more problems with inconsistent illumination.