Internet of Things Garage

Evaluation Framework for Anomaly Detection

Developing an Evaluation Framework for Anomaly Detection within Built Environments

Smart Built Environments are composed of physical and digital infrastructure and aims to improve data-driven decision-making and provide faster and cheaper operation and maintenance (e.g., better whole-life value). They are increasingly more vulnerable to cyber-physical attacks. Anomaly detection techniques are traditionally used to detect any abnormal behaviours. Anomaly detection is a broad field with a rich history where many different techniques have been developed. Out of those, a subset of techniques is focused on real-time anomaly detection. Another subset of techniques focuses on sensor data based on real-time anomaly detection. A key challenge of anomaly detections in the context of built environments is that they are heterogeneous in nature produced by different sensing devices in an unorderly fashion. Some data types are sensor values (e.g. temperature 23C). Other data types could be status or commands (e.g., ON/OFF, 0-1). Some data types could be energy consumption. There are also encrypted data where the actual content is unknown but the metadata available (e.g., packet destination, packet size, frequency co communication). Developing anomaly detection techniques within such context require comprehensive testbeds (or at least datasets collected from a comprehensive testbed). However, currently, no significant emphasis has been put on developing testbeds that can be used to develop, evaluate and compare anomaly detection techniques.

Developing a testbed has always been treated as a secondary task as the development of anomaly detection takes priority. The impact of a testbed’s characteristics and properties towards the anomaly detection techniques developed using them is largely unknown and less studied. The fundamental problem with generating synthetic environments is that in order to be realistic, a large amount of data must be generated in order to provide a convincing pattern of life for the simulated network, as well as give the appearance of longevity (the network must not appear to have been recently generated). Further, anomaly detection techniques are challenging to evaluate, especially when developed using different testbeds and conditions. This project aims to develop a comprehensive framework to evaluate the capabilities of a given anomaly detection technique. The project objectives are:




PETRAS National Centre of Excellence for IoT Systems Cybersecurity is a consortium of eleven leading UK universities which will work together over the next three years to explore critical issues in privacy, ethics, trust, reliability, acceptability, and security.

Building Research Establishment (BRE)

The Building Research Establishment (BRE) is a centre of building science in the United Kingdom, owned by a charitable organisation, the BRE Trust. BRE provides research, advice, training, testing, certification and standards for public and private sector organisations in the UK and abroad.

Government Communications Headquarters (GCHQ)

Government Communications Headquarters, commonly known as GCHQ, is an intelligence and security organisation responsible for providing signals intelligence and information assurance to the government and armed forces of the United Kingdom.