DEEP ROBOT OPTICAL PERCEPTION · VANDERBILT UNIVERSITY

Teaching robots to see, model, and act in a three-dimensional world.

We build perception and learning systems for robots that work in complex, dynamic environments — from the seafloor, to city streets, to the cluttered workbench.

RECONSTRUCTING SURFACEpts 448 / 448 · mesh 75%

Latest papers

All papers →
Figure 1: Pre-contact online monitoring via action-conditioned latent world models. While a visuomotor policy executes an action chunk, ContactGuard conditions a frozen latent world model on the same planned chunk at an anchor $k$ steps bef
AUG 2026 · ARXIV 2608.13438
ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models

ContactGuard predicts future visual latents from a policy's planned actions to abort likely contact failures before they happen, without modifying the policy.

G. Zheng, M. Johnson-Roberson, W. Zhi
Fig. 2: Overview of the Rewind-IL framework. (Left) Offline Staging: Successful policy rollouts are used to construct a viable CP threshold for failure detection. Concurrently, a VLM extracts meaningful keyframes from demonstration videos t
IEEE ROBOTICS AND AUTOMATION LETTERS 2026 · ARXIV 2604.16683
Rewind-IL: Online Failure Detection and State Respawning for Imitation Learning

Rewind-IL detects when an imitation-learning policy is failing using its own inconsistency, then rewinds the robot to a verified safe state and restarts.

G. Zheng, S. Seenivasan, M. Johnson-Roberson, W. Zhi
Figure 1: Rethinking geometric ambiguity for 3D spatial understanding. (a) Ambiguous layered scenes can contain multiple visible surfaces along one ray, while single-depth supervision records only one biased layer. (b) This collapse turns m
ECCV 2026 · ARXIV 2606.29600
One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models

Depth foundation models pick different layers in transparent scenes, and a training-free Laplacian input transform can shift which layer some models report.

X. Xu, F. Xue, X. Li, H. Li, … M. Johnson-Roberson

Research

Themes →
01
Reliable manipulation

Failure detection, recovery, and motion generation for learned visuomotor policies.

02
3D scene understanding

Reconstruction, segmentation, and depth from images — and knowing when geometry is ambiguous.

03
Underwater robotics

AUV mapping, seafloor reconstruction, color correction, and generative underwater terrain.

04
Autonomy in the field

Perception for self-driving and delivery robots operating among people.

People

A small group with a long line of alumni now at NVIDIA, Meta, Waymo, and Aurora, and in faculty posts at Stony Brook, LSU, and Sydney.