Beyond NeRF Underwater: Learning Neural Reflectance Fields for True Color Correction of Marine Imagery

Tianyi Zhang · Matthew Johnson-Roberson

Fig. 1: Observing an underwater scene from different altitudes results in variating color distribution over the RGB channels. Such observations encode the physics of light-water interactions. Our proposed model leverages this cue to restore — FIGURE FROM THE PAPER

TL;DR

A neural reflectance field combined with a physical water model restores true color in underwater images, with better color consistency than existing methods.

BRIEF

Water absorbs and scatters light, so underwater photos lose their true colors, which makes it hard for biologists and geographers to study the seafloor. This work recovers the true color (albedo) of a scene by learning a neural reflectance field together with a simple physical model of the water. The model accounts for light attenuation with distance and backscatter, and a logistic regression separates water from scene. On synthetic and real data, it gave more consistent colors than existing methods.

Abstract

Underwater imagery often exhibits distorted coloration as a result of light-water interactions, which complicates the study of benthic environments in marine biology and geography. In this research, we propose an algorithm to restore the true color (albedo) in underwater imagery by jointly learning the effects of the medium and neural scene representations. Our approach models water effects as a combination of light attenuation with distance and backscattered light. The proposed neural scene representation is based on a neural reflectance field model, which learns albedos, normals, and volume densities of the underwater environment. We introduce a logistic regression model to separate water from the scene and apply distinct light physics during training. Our method avoids the need to estimate complex backscatter effects in water by employing several approximations, enhancing sampling efficiency and numerical stability during training. The proposed technique integrates underwater light effects into a volume rendering framework with end-to-end differentiability. Experimental results on both synthetic and real-world data demonstrate that our method effectively restores true color from underwater imagery, outperforming existing approaches in terms of color consistency.