Modeling Camera Effects to Improve Deep Vision for Real and Synthetic Data
Alexandra Carlson · Katherine A. Skinner · Ram Vasudevan · M. Johnson-Roberson
TL;DR
A physically based pipeline that simulates camera effects in training images makes vehicle detectors more robust and better at generalizing across datasets.
BRIEF
Object detectors trained on urban driving images often fail when the camera changes, because lens and sensor effects alter how the same scene looks. Most augmentation work varies weather or occlusion, not the sensor. This work builds an automated, physically based pipeline that adds chromatic aberration, blur, exposure changes, noise, and color cast to both real and synthetic images. Training with these augmented images improves the robustness and generalization of object detection across several benchmark vehicle datasets.
Abstract
Recent work has focused on generating synthetic imagery and augmenting real imagery to increase the size and variability of training data for learning visual tasks in urban scenes. This includes increasing the occurrence of occlusions or varying environmental and weather effects. However, few have addressed modeling the variation in the sensor domain. Unfortunately, varying sensor effects can degrade performance and generalizability of results for visual tasks trained on human annotated datasets. This paper proposes an efficient, automated physically-based augmentation pipeline to vary sensor effects – specifically, chromatic aberration, blur, exposure, noise, and color cast – across both real and synthetic imagery. In particular, this paper illustrates that augmenting training datasets with the proposed pipeline improves the robustness and generalizability of object detection on a variety of benchmark vehicle datasets.