Pixel-Wise Motion Deblurring of Thermal Videos
Manikandasriram Srinivasan Ramanagopal · Zixu Zhang · Ram Vasudevan · Matthew Johnson-Roberson
TL;DR
Thermal camera blur comes from pixel thermal inertia, so deblurring each pixel over time with a LASSO formulation outperforms spatial deblurring methods.
BRIEF
Thermal cameras based on uncooled microbolometers let robots see in the dark, but their images blur badly when the camera or scene moves. The blur comes from the thermal inertia of each pixel, not from a spatial kernel, so standard deblurring methods do not work reliably. The authors treat each pixel's time signal on its own and recover it as a sparse LASSO problem, solved quickly with quadratic programming and a high frame rate. Pre-trained object detectors performed better on images restored this way than on images from other tested methods.
Abstract
Uncooled microbolometers can enable robots to see in the absence of visible illumination by imaging the "heat" radiated from the scene. Despite this ability to see in the dark, these sensors suffer from significant motion blur. This has limited their application on robotic systems. As described in this paper, this motion blur arises due to the thermal inertia of each pixel. This has meant that traditional motion deblurring techniques, which rely on identifying an appropriate spatial blur kernel to perform spatial deconvolution, are unable to reliably perform motion deblurring on thermal camera images. To address this problem, this paper formulates reversing the effect of thermal inertia at a single pixel as a Least Absolute Shrinkage and Selection Operator (LASSO) problem which we can solve rapidly using a quadratic programming solver. By leveraging sparsity and a high frame rate, this pixel-wise LASSO formulation is able to recover motion deblurred frames of thermal videos without using any spatial information. To compare its quality against state-of-the-art visible camera based deblurring methods, this paper evaluated the performance of a family of pre-trained object detectors on a set of images restored by different deblurring algorithms. All evaluated object detectors performed systematically better on images restored by the proposed algorithm rather than any other tested, state-of-the-art methods.