Evaluation Framework for Anomaly Detection
Project Overview
Anomaly detection techniques are challenging to evaluate, especially when developed using different testbeds and conditions. Smart built environments generate heterogeneous sensor data including temperature readings, status commands, energy consumption metrics, and encrypted metadata. A core challenge is that testbed development has often been secondary to anomaly detection technique creation, making it difficult to compare results across different systems. This project addresses this gap by systematically examining the relationship between testbed design and detection performance, providing guidance, datasets, and tooling to enable more rigorous and reproducible evaluation of anomaly detection methods across heterogeneous smart home environments. The work is conducted in collaboration with PETRAS, the Building Research Establishment, and GCHQ, ensuring that findings are applicable to real-world security and operational contexts within the built environment.
The project pursues four key objectives to improve evaluation rigour. These include conducting a comprehensive literature review on IoT anomaly detection testbeds, identifying smart home testbed characteristics that affect detection quality, developing techniques for data capture, annotation, and modelling, and generating realistic synthetic datasets to benchmark cyber-physical attack detection while measuring trade-offs against live detection approaches.
By combining academic research expertise with industry and government insight, the project produces evaluation frameworks that reflect practical demands. These frameworks address the challenge of securing smart built environments by providing standardised approaches to testing anomaly detection systems, enabling meaningful comparisons across different implementations and deployment contexts.
Team
Partners
