A framework for enhanced localization of marine mammals using auto-detected video and wearable sensor data fusion
Joaquin Gabaldon · Ding Zhang · K. Barton · M. Johnson-Roberson · K. A. Shorter
TL;DR
Dolphins are localized by fusing automatically detected video positions with wearable inertial sensor data through Kalman-style filters and iSAM2 pose-graph smoothing.
BRIEF
Knowing where individual animals are over time helps researchers study how they behave alone and in groups. This work tracks dolphins under professional care by combining two sources: overhead video and wearable motion sensors. Video analysis detects each dolphin's position automatically using blob detection. Those positions are merged with magnetic, angular rate, gravity and inertial measurements through drift-correcting filters, then refined with a pose-graph smoother (iSAM2) to produce a localization estimate for the animal.
Abstract
Accurate biological agent localization offers the opportunity for both researchers and institutions to gain new knowledge about individual and group behaviors of biosystems. This paper presents a sensor-fusion approach for tracking biological agents, combining the data from automated video logging with magnetic, angular rate, and gravity (MARG) and inertial measurement unit (IMU) data, with professionally managed dolphins as the representative example. Our method of video logging allows for accurate and automated dolphin location detection using a combination of Laplacian of Gaussian (LoG) and multi-orientation elliptical blob detection. These data are combined with MARG/IMU measurements to generate a localization estimate through a series of drift-correcting Kalman and gradient-descent filters, finalized with Incremental Smoothing and Mapping (iSAM2) pose-graph localization.