UNR-IDD Dataset

IMPORTANT

You may redistribute, republish, and mirror the UNR-IDD dataset in any form. However, any use or redistribution of the data must include a citation to the UNR-IDD paper in the following link: https://ieeexplore.ieee.org/abstract/document/10059640

Please click here to download the dataset

Abstract

With the expanded applications of modern-day networking, network infrastructures are at risk from cyber-attacks and intrusions. Multiple datasets have been proposed in the literature that can be used to create Machine Learning (ML) based Network Intrusion Detection Systems (NIDS). However, many of these datasets suffer from sub-optimal performance and do not adequately and effectively represent all types of intrusions.  Another problem with these datasets is the low accuracy of tail classes. To address these issues,  we propose the University of Nevada - Reno Intrusion Detection Dataset (UNR-IDD) that provides researchers with a wider range of samples and scenarios. 

Introduction

Modern computer networks and their connected applications have reached unprecedented growth with implementations like the internet of things, smart homes, and software-defined networks. However, this has also increased the potential for network intrusions, which are a continuous threat to network infrastructures as they attempt to compromise the major principles of computing systems: availability, authority, confidentiality, and integrity.  These threats are difficult to detect unaided, as they display indistinguishable network traffic patterns as normal functionality.  To provide enhanced protection against intrusions, the usage of machine learning for NIDS has gained traction in the last decade as various open-sourced datasets have been proposed and established by multiple research groups globally.

 However, a common problem that has been identified with many of these datasets is inadequate modeling of tail classes. Another limitation of the current datasets is that they mostly depend on flow level statistics, which can limit the transferability of the NIDS solutions to other network configurations. Lastly, some existing datasets suffer from incomplete or missing records. These records or samples must be ignored or dropped from the overall dataset, which leads to sub-optimal performance. 

To address these limitations, we propose the University of Nevada - Reno Intrusion Detection Dataset (UNR-IDD), a NIDS dataset. The main difference between UNR-IDD and existing datasets is that UNR-IDD consists primarily of network port statistics. These refer to the observed port metrics recorded in switch/router ports within a networking environment. The dataset also includes delta port statistics which indicates the change in magnitude of observed port statistics within a time interval. Compared to datasets that primarily use flow level statistics, these port statistics can provide a fine-grained analysis of network flows from the port level as decisions are made at the port level versus the flow level. This can lead to rapid identification of potential intrusions. We also address the limitation of the presence of tail classes. Our dataset ensures that there are enough samples for ML classifiers to achieve high F-Measure scores, uniquely. Our proposed dataset also ensures that there are no missing network metrics and that all data samples are filled. 

Data Collection Setup

Flow Simulation

IPerf is used to create TCP and UDP data streams simulating network flows in virtual and real networks using dummy payloads. By using the Mininet API and IPerf, we created a Python script to simulate realistic network flows. Once every 5 seconds, we initiated Iperf traffic between a randomly chosen source-destination host pair with a bandwidth of 10 Mbps and a duration of 5 seconds. We then simulate flows under normal and intrusion conditions to gather data in every possible scenario. To ensure that each normal and anomaly category is minimally variable and adequately represented, we execute the same number of flows while simulating each scenario.

Data Collection

We create a custom application to collect and log the available statistics that are captured periodically (once in every 5 seconds) from OpenFlow (OF) switches. The statistics are collected through by means of OFPPortStatsRequest and OFPPortStatsReply messages between controller and switches. The delta port statistics are computed on the controller side by taking the difference between the last two collected data instances. We create a key-value map of this data by gathering it from the data storage service, using the "Device Service" API provided by ONOS. After this, we logged the map of the collected statistics to a Javascript Object Notation (.json) file with a name N_i.json. 

Captured Data Features

Labels

The following intrusions are simulated in the data collection phase:

• TCP-SYN Flood: A Distributed Denial of Service (DDoS) attack where attackers target hosts by initiating many Transmission Control Protocol (TCP) handshake processes without waiting for the response from the target node. By doing so, the target device's resources are consumed as it has to keep allocating some memory space for every new TCP request.

• Port scan: An attack in which attackers scan available ports on a host device to learn information about the services, versions, and even security mechanisms that are running on that host. 

• Flow Table Overflow: An attack that targets network switches/routers where attacks compromise the functionality of a switch/router by consuming the flow tables that forward packets with illegitimate flow entries and rules so that legitimate flow entries and rules cannot be installed. 

• Blackhole: An attack that targets network switches/routers to discard the packets that pass through, instead of relaying them on to the next hop. 

• Traffic Diversion: A attack that targets network switches/routers to reroute the direction of packets away from their destination, intending to increase travel time and/or spying on network traffic through a man-in-the-middle scenario.

These intrusion types were selected for this dataset as they are common cyber attacks that can occur in any networking environment. Also, these intrusion types cover attacks that can be launched on both network devices and end hosts.  This dataset can be broken down into two different machine learning classification problems: binary and multi-class classification.