A LIGHT-WEIGHT SCADA TESTBED FOR MACHINE-LEARNING BASED CYBER-SECURITY EXPERIMENTS

Abstract

A rapid rise in cyber-attacks on Cyber Physical Systems (CPS) has been observed in the last decade. It becomes even more concerning that many ofthese attacks were on critical infrastructures, where some ofthem indeed succeeded and resulted into significant physical damages and financial losses. Such integrated systems are complex in nature, and due to that, loop holes do emerge and conventional security measures do not fulfil the security needs of such systems. As a result, many of the CPS have experienced several incidents of cyber-attacks. It is considered both difficult and financially infeasible to gain access to such CPS setup for researchers to conduct their real hardware. This is due to the system being restricted to work on or being tests on very costly to establish such a setup in a research lab. In consequence, researcher use simulations or experimental testbeds capable of providing flexible, scalable and interoperable platforms for executing various cybersecurity experiments. Such alternative options are considered as highly in need by all stakeholders, especially by the research community. Existing simulators are constrained by either being a dedicated stand-alone power grid system or water desalination simulator for the physical systems, such being dedicated for simulating the cyber components, such as a network as a plant, or simulator. Furthermore, existing co-simulators are at their infancy level and lag the flexibility needed for executing a variety ofresearch scenarios in CPS. Existing testbed design also have limited scalability due to physical and financial limitations. Moreover, testbed-based works evaluate performance ofthe system while considering only a single type of cyber-attack, while the impact of more than one attack on the system is not thoroughly studies. In this work, a new container-based testbed is presented for Supervisory Control lightweight, scalable, flexible and portable and Data Acquisition (SCADA) system platform that is capable for executing a variety of cybersecurity experiments. Through the proposed testbed, two types of cyber-attacks are generated, namely Address Man in the middle (M1TM) as a Resolution Protocol (ARP) spoofing that represents attack, and network scanning which represents traffic generated by both ARP spoofing and network scanning are captured and furth classification models i.e. decision tree model and random forest model. Performance of both the classification models is evaluated through a series of experiments where both the models perform quite well when evaluated with a single type of cyber-attack. However, when the same classification models are faced with two types of cyber attacks, the false alarm rate tends to increase when evaluating using a decision tree classification model i.e. 1.29%. In contrast, the un-detection rate tends to increase when evaluated with a random forest based classifier i.e. 0.34%. In general, we observed high false positives incidents for ARP traffic and a high number of false negative incidents for reconnaissance traffic. For reconnaissance traffic both the models report a high number of miss rates in which the model fails to classify attack traffic as an attack and lets them pass as a normal traffic, hence degrading their performance. Based on the obtained results, the decision tree based classifiers is a good option to consider with 0.016% un-detection rate for detecting multiple types of cyber-attacks where its false negative incidents is quite low as compared to that shown by random forest classifier.