PETRAS National Centre of Excellence for IoT Systems Cybersecurity is a consortium of eleven leading UK universities which will work together over the next three years to explore critical issues in privacy, ethics, trust, reliability, acceptability, and security.
Evaluation Framework for Anomaly Detection
Developing an Evaluation Framework for Anomaly Detection within Built Environments
Smart Built Environments are composed of physical and digital infrastructure and aims to improve data-driven decision-making and provide faster and cheaper operation and maintenance (e.g., better whole-life value). They are increasingly more vulnerable to cyber-physical attacks. Anomaly detection techniques are traditionally used to detect any abnormal behaviours. Anomaly detection is a broad field with a rich history where many different techniques have been developed. Out of those, a subset of techniques is focused on real-time anomaly detection. Another subset of techniques focuses on sensor data based on real-time anomaly detection. A key challenge of anomaly detections in the context of built environments is that they are heterogeneous in nature produced by different sensing devices in an unorderly fashion. Some data types are sensor values (e.g. temperature 23C). Other data types could be status or commands (e.g., ON/OFF, 0-1). Some data types could be energy consumption. There are also encrypted data where the actual content is unknown but the metadata available (e.g., packet destination, packet size, frequency co communication). Developing anomaly detection techniques within such context require comprehensive testbeds (or at least datasets collected from a comprehensive testbed). However, currently, no significant emphasis has been put on developing testbeds that can be used to develop, evaluate and compare anomaly detection techniques.
Developing a testbed has always been treated as a secondary task as the development of anomaly detection takes priority. The impact of a testbed’s characteristics and properties towards the anomaly detection techniques developed using them is largely unknown and less studied. The fundamental problem with generating synthetic environments is that in order to be realistic, a large amount of data must be generated in order to provide a convincing pattern of life for the simulated network, as well as give the appearance of longevity (the network must not appear to have been recently generated). Further, anomaly detection techniques are challenging to evaluate, especially when developed using different testbeds and conditions. This project aims to develop a comprehensive framework to evaluate the capabilities of a given anomaly detection technique. The project objectives are:
- Conduct a literature review to determine how testbeds are built to evaluate IoT-based anomaly detection techniques.
- Identify characteristics and properties of smart home testbeds that impact the quality of the anomaly detection techniques developed using them.
- Develop techniques to capture, annotate and model data from smart home testbeds to support comparable anomaly detection techniques development.
- Develop techniques to generate realistic synthetic datasets compared to real-time live anomaly detection and measure the trade-offs of both approaches.
Team
Partners
PETRAS 2
Building Research Establishment (BRE)
The Building Research Establishment (BRE) is a centre of building science in the United Kingdom, owned by a charitable organisation, the BRE Trust. BRE provides research, advice, training, testing, certification and standards for public and private sector organisations in the UK and abroad.
Government Communications Headquarters (GCHQ)
Government Communications Headquarters, commonly known as GCHQ, is an intelligence and security organisation responsible for providing signals intelligence and information assurance to the government and armed forces of the United Kingdom.