Financialized methods for market-based multi-sensor fusion
J. Abernethy · M. Johnson-Roberson
TL;DR
Sensor fusion is framed as a prediction market where algorithms trade, and the price equilibrium matches hand-designed fusion on KITTI pedestrian detection.
BRIEF
Modern autonomous vehicles combine many sensors, but sensors get added, move, fail, or report at different times. This work treats fusion as a market. Each sensor-processing algorithm acts as a profit-seeking trader, new data enters through transactions, and the combined estimate is the resulting price equilibrium. The design draws on prediction-market research, which is built to pool information from diverse sources without central coordination. Tested on pedestrian detection with RGB and LIDAR data from KITTI, it performs comparably to state-of-the-art hand-designed fusion methods.
Abstract
Autonomous systems rely on an increasing number of input sensors of various modalities, and the problem of sensor fusion has received attention for many years. Autonomous system architectures are becoming more complex with time, and the number and placement of sensors will be modified regularly, sensors will fail for many reasons, information will arrive asynchronously, and the system will need to adjust to rapidly changing environments. To address these issues we propose a new paradigm for fusing information from multiple sources that draws from the rich of field pertaining to financial markets, particularly recent research on prediction market design. Among the many benefits of this financialized approach is that, both in theory and in practice, markets are well-equipped to robustly synthesize information from diverse sources in a decentralized fashion. Our framework poses sensor processing algorithms as profit-seeking market participants, data is incorporated via financial transactions, and the joint estimation is represented as a price equilibrium. We use pedestrian detection as a motivating application. Pedestrian detection is a well studied field and essential to autonomous driving. Real world fusion results are presented on RGB and LIDAR data from the KITTI Vision Benchmark Suite. We demonstrate we can achieve comparable performance to state-of-the-art hand designed fusion techniques using the proposed approach.