SilhoNet-Fisheye: Adaptation of A ROI Based Object Pose Estimation Network to Monocular Fisheye Images

Gideon Billings · Matthew Johnson-Roberson

Fig. 1: Overview of the three different SilhoNet [1] adaptations for processing full fisheye images. The Baesline method processes the raw fisheye image directly through the unmodified network. The Projective variant processes the raw fishe — FIGURE FROM THE PAPER

TL;DR

SilhoNet is adapted to fisheye images by projecting each region of interest onto a tangent plane, and an underwater fisheye pose dataset, UWHandles, is released.

BRIEF

Most deep-learning methods for estimating an object's 3D pose assume an ordinary perspective camera, yet fisheye lenses are common on robots and strongly distort objects away from the image center. This work adapts SilhoNet, a region-of-interest pose estimator, to full fisheye images. Features inside each region are mapped through a spherical representation onto a tangent plane centered on the region, which corrects the distortion. The authors also release UWHandles, an underwater fisheye dataset with 6D pose and bounding box annotations.

Abstract

There has been much recent interest in deep learning methods for monocular image based object pose estimation. While object pose estimation is an important problem for autonomous robot interaction with the physical world, and the application space for monocular-based methods is expansive, there has been little work on applying these methods with fisheye imaging systems. Also, little exists in the way of annotated fisheye image datasets on which these methods can be developed and tested. The research landscape is even more sparse for object detection methods applied in the underwater domain, fisheye image based or otherwise. In this work, we present a novel framework for adapting a ROI-based 6D object pose estimation method to work on full fisheye images. The method incorporates the gnomic projection of regions of interest from an intermediate spherical image representation to correct for the fisheye distortions. Further, we contribute a fisheye image dataset, called UWHandles, collected in natural underwater environments, with 6D object pose and 2D bounding box annotations.