Rewind-IL: Online Failure Detection and State Respawning for Imitation Learning
Gehan Zheng · Sanjay Seenivasan · Matthew Johnson-Roberson · Weiming Zhi
TL;DR
Rewind-IL detects when an imitation-learning policy is failing using its own inconsistency, then rewinds the robot to a verified safe state and restarts.
BRIEF
Robots trained by imitation learning can perform long, multi-step manipulation tasks, but once they drift away from what they saw in demonstrations, they often keep producing plausible-looking actions without recovering. Rewind-IL adds a safeguard that needs no failure data and no retraining. It watches whether overlapping action chunks agree with each other to detect trouble, then returns the robot to the latest safe intermediate state, identified from demonstrations with a vision-language model, and restarts the policy. Tests in real-world and simulated tasks show improved reliability.
Abstract
Imitation learning has enabled robots to acquire complex visuomotor manipulation skills from demonstrations, but deployment failures remain a major obstacle, especially for long-horizon action-chunked policies. Once execution drifts off the demonstration manifold, these policies often continue producing locally plausible actions without recovering from the failure. Existing runtime monitors either require failure data, over-trigger under benign feature drift, or stop at failure detection without providing a recovery mechanism. We present Rewind-IL, a training-free online safeguard framework for generative action-chunked imitation policies. Rewind-IL combines a zero-shot failure detector based on Temporal Inter-chunk Discrepancy Estimate (TIDE), calibrated with split conformal prediction, with a state-respawning mechanism that returns the robot to a semantically verified safe intermediate state. Offline, a vision-language model identifies recovery checkpoints in demonstrations, and the frozen policy encoder is used to construct a compact checkpoint feature database. Online, Rewind-IL monitors self-consistency in overlapping action chunks, tracks similarity to the checkpoint library, and, upon failure, rewinds execution to the latest verified safe state before restarting inference from a clean policy state. Experiments on real-world and simulated long-horizon manipulation tasks, including transfer to flow-matching action-chunked policies, demonstrate that policy-internal consistency coupled with semantically grounded respawning offers a practical route to improved reliability in imitation learning. Supplemental materials are available at https://sjay05.github.io/rewind-il