Demystifying the role of public intrusion datasets: A replication study of DoS network traffic data

Public intrusion datasets are contributing to make security research accessible to a large community of users, but are often trusted and reused neglecting the actual impact of the attacks therein on victim services. This paper documents a study aimed to assess whether the attacks provided by public datasets are impactful on their targets. DoS traffic data from five public datasets (CICIDS2017, ISCXIDS2012, NDSec-1 2016, MILCOM 2016 and SUEE 2017) are replayed, monitoring the performance of the victim server under different defense, configuration and load conditions. The obtained results show a partial ineffectiveness of the attacks of the datasets in the presence of defense mechanisms and suitable server configurations. These results pave the way for the construction of more rigorous datasets, collected on documented and realistic server configurations and reflecting actual traffic conditions under normative operations and disruptive attacks.