Discovering salient regions on 3D photo-textured maps: Crowdsourcing interaction data from multitouch smartphones and tablets
Matthew Johnson-Roberson · Mitch Bryson · Bertrand Douillard · Oscar R. Pizarro · Stefan B. Williams
TL;DR
Logged touch interactions on a mobile app for underwater 3D maps can be mined to find salient regions, validated against eye-tracking data.
BRIEF
Underwater robots collect large photo-textured 3D maps of the seafloor, but it is hard to know which parts of those maps people actually find interesting. This work built a smartphone and tablet app that lets users explore such maps, and logged hundreds of thousands of touch interactions. Two models estimate saliency from that behavior: one measures how long points stay on screen, and the other uses camera velocity with a Hidden Markov model. Results were compared with standard visual saliency methods and checked against gaze-tracking data.
Abstract
Abstract This paper presents a system for crowdsourcing saliency interest points for 3D photo-textured maps rendered on smartphones and tablets. An app was created that is capable of interactively rendering 3D reconstructions gathered with an Autonomous Underwater Vehicle. Through hundreds of thousands of logged user interactions with the models we attempt to data-mine salient interest points. To this end we propose two models for calculating saliency from human interaction with the data. The first uses the view frustum of the camera to track the amount of time points are on screen. The second uses the velocity of the camera as an indicator of saliency and uses a Hidden Markov model to learn the classification of salient and non-salient points. To provide a comparison to existing techniques several traditional visual saliency approaches are applied to orthographic views of the models' photo-texturing. The results of all approaches are validated with human attention ground truth gathered using a remote gaze-tracking system that recorded the locations of the person's attention while exploring the models.