Simulation-based Attack Injection at System-levelSimulation-based Attack Injection at System-level provides an opportunity of injecting attacks on the system level. Different parts of the system and its interconnections can be verified and validated by using this technique. The complete system behaviour can be analysed when a certain sub-system is under the influence of attacks. While conducting field tests could be costly and sometimes life-threatening, simulation-based tests provide a wide range of advantages, such as lower testing costs, adaptation of tests to a variety of traffic scenarios, and avoiding the life-threatening situations. This method could span over various tools such as SUMO (Simulation of Urban Mobility), VEINS (vehicular network simulator) and INET allowing different aspects of the system to be evaluated.https://repo.valu3s.eu/method/simulation-based-attack-injection-at-system-levelhttps://repo.valu3s.eu/@@site-logo/logo_valu3s_green_transparent.png
Simulation-based Attack Injection at System-level
Simulation-based Attack Injection at System-level provides an opportunity of injecting attacks on the system level. Different parts of the system and its interconnections can be verified and validated by using this technique. The complete system behaviour can be analysed when a certain sub-system is under the influence of attacks. While conducting field tests could be costly and sometimes life-threatening, simulation-based tests provide a wide range of advantages, such as lower testing costs, adaptation of tests to a variety of traffic scenarios, and avoiding the life-threatening situations. This method could span over various tools such as SUMO (Simulation of Urban Mobility), VEINS (vehicular network simulator) and INET allowing different aspects of the system to be evaluated.
The purpose of simulation-based attack injection at system-level is to evaluate the system’s cybersecurity properties by injecting attacks, e.g., by using simulator control commands during target system simulations.
System-level simulation is comprised of hardware and software models of a Cyber Physical System (CPS). The attack injection could be performed on different abstraction layers such as logical, functional, hardware, software, or system level. In this case, we focus on the simulation-based attack injection at the system level.
In our case, the simulation-based attack injection at system level is done through injection of attacks on communication system modelled in Veins simulator together with traffic simulators. Simulators of interest are SUMO, Veins and Veins-INET [SAI2][SAI3]. The simulation-based attack injection at system level using simulators may be used for security testing of automated systems such as autonomous vehicles [SAI1].
Simulation-based attack injection is a V&V method where the attacks are injected into system software in a simulation environment. This type of attack injection is applicable when:
A system model is available at the early stages of system development.
A software/prototype is available to run in a simulation environment. This type of testing is called Software in the Loop (SiL) testing and the software under evaluation is called SIL component*1 [SAI4]
The hardware is not available.
The software needs to be verified and validated in a simulation environment.
Simulation-based attack injection is useful for both development and deployment stages to identify and resolve different types of vulnerabilities relevant for each stage.
*1 A SiL component is an executable code written for a specific system, adjusted to run only in a simulation environment for software testing. This type of testing is useful especially when the hardware is not existing, when it is in the development phase, or when the verification results are required in short span of time. The latter could be facilitated by parallel execution of the tests in a cluster. Hardware requirements are taken away (e.g., end-to-end protection) from the SiL component so that it can run in a completely simulated or model-based environment. Note that the SiL testing is complemented by Hardware in the Loop (HiL) testing, when the hardware is available, in order to also evaluate the system when the software resides in the intended hardware, such as a particular mechatronic system.
The simulation-based attack injection at the system level can be useful for:
End-to-end resilience assessment of a complete system,especially in edge case scenarios*1
Introducing attacks in different parts of a system such as sensors, functions, and actuators to evaluate that specific part or even a complete system behaviour.
Introducing attacks in automated system communications, which may be hard to do through other verification methods.
It is possible to introduce multiple attacks which are valid for multiple layers of a vehicular communication system by using this method.
Measurements from simulation-based attack injection may be useful in later V&V activities.
*1The edge cases are realised by injecting attacks in the system to create a test scenario that is otherwise rarely tested or testable in the real-world.
The simulation-based attack injection at system level is limited to the injection of attack in simulations only, so it is not possible to evaluate the actual physical system.
The use of simulation-based attack injection techniques for ML-based systems showed promising results in the initial experimentation. However, there is a need to further explore this test technique for ML or deep learning-based systems.
[SAI1] Eduardo dos Santos et al., “Towards a Simulation-based Framework for the Security Testing of Autonomous Vehicles”
[SAI2] Michael Behrisch, Laura Bieker et al., “SUMO – Simulation of Urban Mobility, An Overview”, Institute of Transportation Systems, German Aerospace Center, Rutherfordstr. 2, 12489 Berlin, Germany.
[SAI3] D. Eckhoff and C. Sommer, “A Multi-channel IEEE 1609.4 and 802.11p EDCA model for the Veins framework,” in Proceedings of 5th ACM/ICST international conference on simulation tools and techniques for communications, networks and systems: 5th ACM/ICST international workshop on OMNet++.(Desenzano, Italy, 19-23 March, 2012). OMNeT, 2012.
