Utilising Flow Aggregation to Classify Benign Imitating Attacks

Cyber-attacks continue to grow, both in terms of volume and sophistication. This is aided by an increase in available computational power, expanding attack surfaces, and advancements in the human understanding of how to make attacks undetectable. Unsurprisingly, machine learning is utilised to defend against these attacks. In many applications, the choice of features is more important than the choice of model. A range of studies have, with varying degrees of success, attempted to discriminate between benign traffic and well-known cyber-attacks. The features used in these studies are broadly similar and have demonstrated their effectiveness in situations where cyber-attacks do not imitate benign behaviour. To overcome this barrier, in this manuscript, we introduce new features based on a higher level of abstraction of network traffic. Specifically, we perform flow aggregation by grouping flows with similarities. This additional level of feature abstraction benefits from cumulative information, thus qualifying the models to classify cyber-attacks that mimic benign traffic. The performance of the new features is evaluated using the benchmark CICIDS2017 dataset, and the results demonstrate their validity and effectiveness. This novel proposal will improve the detection accuracy of cyber-attacks and also build towards a new direction of feature extraction for complex ones.

Travel flow aggregation: Nationally scalable methods for interactive and online visualisation of transport behaviour at the road network level

Origin–destination datasets representing millions of travel desire lines and routes are common in transport planning, but visualising such datasets is challenging. Existing methods often produce illegible results, low spatial resolution, or only a relative indication of the variation of flow on each road. This paper presents a new open-source algorithm called overline along with an accompanying method, to efficiently convert disparate geographical transport data into a policy-relevant summary form. Specifically, overline aggregates many individual routes into a route network map. These vector and raster maps provide total flow counts for each road and junction and are scalable to regional or national datasets. The method is demonstrated by visualising four million routes for a publicly accessible web mapping application, the Propensity to Cycle Tool, across the whole of England and Wales.

Anomaly Detection and Attribution in Network

In this article, we address the problem of not only id entifying phenomena, but also attributing the phenomenon to the movement that induces it. This causes to a combinatorial optimisation problem, which is prohibitively expensive. Instead we design two anomaly detection algorithms that are small in complexity. The first is based on the system for cross-entropy (CE), which identifies flow anomalies and labels flow anomalies. The second algorithm detects anomalies through GLRT on aggregated flow transformation a compact low-dimensional representation of raw traffic flows. The two algorithms complement each other and allow the network operator to use the algorithm for flow aggregation first so that device irregularities can be identified easily. After discovery of an exception, the user Can analyse further that individual flows are anomalous using CE-based algorithm. We perform extensive performance tests and trials on synthetic and semi-synthetic data with our algorithms, as well as real Internet traffic data gathered from the MAWI database, and finally make recommendations as to their usability.

Anomaly Detection and Attribution in Network

In this article, we address the problem of not only id entifying phenomena, but also attributing the phenomenon to the movement that induces it. This causes to a combinatorial optimisation problem, which is prohibitively expensive. Instead we design two anomaly detection algorithms that are small in complexity. The first is based on the system for cross-entropy (CE), which identifies flow anomalies and labels flow anomalies. The second algorithm detects anomalies through GLRT on aggregated flow transformation a compact low-dimensional representation of raw traffic flows. The two algorithms complement each other and allow the network operator to use the algorithm for flow aggregation first so that device irregularities can be identified easily. After discovery of an exception, the user Can analyse further that individual flows are anomalous using CE-based algorithm. We perform extensive performance tests and trials on synthetic and semi-synthetic data with our algorithms, as well as real Internet traffic data gathered from the MAWI database, and finally make recommendations as to their usability.