Simulation and Real-World Evaluation of Attack Detection Schemes

Matthew Porter · Arnav Joshi · Pedro Hespanhol · Anil Aswani · Matthew Johnson-Roberson · Ram Vasudevan

Fig. 1: This paper describes a real-world implementation of malicious attack detection on a Segway Robotics Mobility Platform performing a path-following task (top). An additive attack is applied to this system as shown in the block diagram — FIGURE FROM THE PAPER

TL;DR

A method sets Dynamic Watermarking attack detectors to a chosen false alarm rate and measures attacker capability, tested against classic detectors in simulation and on a Segway.

BRIEF

Cyber-physical systems such as mobile robots can be attacked by someone who tampers with their sensor data. Dynamic Watermarking detects many such attacks, but until now it had no way to be tuned to a chosen false alarm rate or to measure what an attacker could still get away with. This work gives methods to quantify attacker capability, approximate it numerically, and design a detector for a target false alarm rate. Tests in simulation and on a Segway robot compare it with three classical detectors.

Abstract

A variety of anomaly detection schemes have been proposed to detect malicious attacks to Cyber-Physical Systems. Among these schemes, Dynamic Watermarking methods have been proven highly effective at detecting a wide range of attacks. Unfortunately, in contrast to other anomaly detectors, no method has been presented to design a Dynamic Watermarking detector to achieve a user-specified false alarm rate, or subsequently evaluate the capabilities of an attacker under such a selection. This paper describes methods to measure the capability of an attacker, to numerically approximate this metric, and to design a Dynamic Watermarking detector that can achieve a user-specified rate of false alarms. The performance of the Dynamic Watermarking detector is compared to three classical anomaly detectors in simulation and on a real-world platform. These experiments illustrate that the attack capability under the Dynamic Watermarking detector is comparable to those of classic anomaly detectors. Importantly, these experiments also make clear that the Dynamic Watermarking detector is consistently able to detect attacks that the other class of detectors are unable to identify.