Detecting Persistent User Behavior Using Probabilistic Counting in Network-Wide View

Persistent user behavior monitoring, which deals with finding users that occur persistently over a measurement period, is one hot topic in traffic measurement. It is significant for many applications, such as anomaly detection. Former works concentrate on monitoring frequent user behavior, such as users occurring frequently either over one measurement period or on one monitor. They have paid little attention to detect persistent user behavior over a long measurement period on multiple monitors. However, persistent users do not necessarily appear frequently in a short measurement period, but appear persistently in a long measurement period. Due to limited resource on monitors, it is not practical to collect a tremendous amount of network traffic in a long measurement period on one single monitor. Moreover, since network attackers deliberately send packets flowing through the entire managed network, it is difficult to detect abnormal behavior on one single monitor. To solve the above challenges, a novel method for detecting persistent user behavior called DPU is proposed, and it contains both online distributed traffic collection in a long measurement period on multiple monitors and offline centralized user behavior detection on the central server. The key idea of DPU is that we design the compact distributed synopsis data structure to collect the relevant information with users occurring in a long measurement period on each monitor, and we can reconstruct user IDs using simple calculations and bit settings to find users with persistent behavior on the basis of estimated occurrence frequency of users on the central server when user IDs are unknown in advance. The experiments are conducted on real traffic to evaluate the performance of detecting persistent user behavior, and the experimental results illustrate that our method can improve about 30% estimation accuracy, 40% detection precision, and accelerate about 3 times in comparison with the related method.

An Efficient Indexing Scheme for Network Traffic Collection and Retrieval System

Historical network traffic retrieval, both at the packet and flow level, has been applied in many fields of network security, such as network traffic analysis and network forensics. To retrieve specific packets from a vast number of packet traces, it is an effective solution to build indexes for the query attributes. However, it brings challenges of storage consumption and construction time overhead for packet indexing. To address these challenges, we propose an efficient indexing scheme called IndexWM based on the wavelet matrix data structure for packet indexing. Moreover, we design a packet storage format based on the PcapNG format for our network traffic collection and retrieval system, which can speed up the extraction of index data from packet traces. Offline experiments on randomly generated network traffic and actual network traffic are performed to evaluate the performance of the proposed indexing scheme. We choose an open-source and widely used bitmap indexing scheme, FastBit, for comparison. Apart from the native bitmap compression method Word-Aligned Hybrid (WAH), we implement an efficient bitmap compression method Scope-Extended COMPAX (SECOMPAX) in FastBit for performance evaluation. The comparison results show that our scheme outperforms the selected bitmap indexing schemes in terms of time consumption, storage consumption and retrieval efficiency.

Model and Algorithms for User Identification by Network Traffic

This paper proposes a method of user identification by network traffic. We describe the information model created, as well as the implementation of each of the proposed problem solving stages. During the network traffic collection stage, a method of capturing network packets on the user's device using specialized software is used. The information obtained is further filtered by removing redundant data. During the object feature descriptor construction stage, we extract and describe the characteristics of network sessions from which the behavioral habits of users are derived. Classification of users according to the extracted characteristics of the network sessions is performed using machine learning techniques. When analyzing the test results, the most appropriate machine learning algorithms for solving the problem of user identification by network traffic were proposed, such as: logistic regression, decision trees, SVM with a linear hyperplane and the boosting method. The accuracy of the above methods was more than 95%. The results proved that it is possible to identify a particular user with a sufficiently high accuracy based on the characteristics of the data transmitted through the network, without examining the contents of the transmitted packets. Comparison of the developed model has shown that the proposed model of user identification by network traffic works as effectively as the existing analogues.

Real Network Traffic Collection and Deep Learning for Mobile App Identification

The proliferation of mobile devices over recent years has led to a dramatic increase in mobile traffic. Demand for enabling accurate mobile app identification is coming as it is an essential step to improve a multitude of network services: accounting, security monitoring, traffic forecasting, and quality-of-service. However, traditional traffic classification techniques do not work well for mobile traffic. Besides, multiple machine learning solutions developed in this field are severely restricted by their handcrafted features as well as unreliable datasets. In this paper, we propose a framework for real network traffic collection and labeling in a scalable way. A dedicated Android traffic capture tool is developed to build datasets with perfect ground truth. Using our established dataset, we make an empirical exploration on deep learning methods for the task of mobile app identification, which can automate the feature engineering process in an end-to-end fashion. We introduce three of the most representative deep learning models and design and evaluate our dedicated classifiers, namely, a SDAE, a 1D CNN, and a bidirectional LSTM network, respectively. In comparison with two other baseline solutions, our CNN and RNN models with raw traffic inputs are capable of achieving state-of-the-art results regardless of TLS encryption. Specifically, the 1D CNN classifier obtains the best performance with an accuracy of 91.8% and macroaverage F-measure of 90.1%. To further understand the trained model, sample-specific interpretations are performed, showing how it can automatically learn important and advanced features from the uppermost bytes of an app’s raw flows.

Using the CVP Traffic Detection Model at Road-Section Applies to Traffic Information Collection and Monitor - the Case Study

This paper proposes a using Cellular-Based Vehicle Probe (CVP) at road-section (RS) method to detect and setup a model for traffic flow information (info) collection and monitor. There are multiple traffic collection devices including CVP, ETC-Based Vehicle Probe (EVP), Vehicle Detector (VD), and CCTV as traffic resources to serve as road condition info for predicting the traffic jam problem, monitor and control. The main project has been applied at Tai # 2 Ghee-Jing roadway connects to Wan-Li section as a trial field on fiscal year of 2017-2018. This paper proposes a man-flow turning into traffic-flow with Long-Short Time Memory (LTSM) from recurrent neural network (RNN) model. We also provide a model verification and validation methodology with RNN for cross verification of system performance.