System-level simulation is comprised of hardware and software models of a cyber physical system (CPS). The attack injection could be performed on different abstraction layers such as logical, functional, hardware, software, or system level. In this case, we focus on the simulation-based attack injectionat the system level.

The simulation-based attack injection at system level using simulators may be used for security testing of automated systems such as autonomous vehicles [SAI1].

Multiple attack models valid for multi-layer vehicular communication in the simulation-based attack injection at system level is done through injection of attacks on the vehicular communication system modelled in Veins (Vehicles in Network Simulation). The communication layers that are modelled in the veins framework are MAC and PHY layers INET [SAI3]. A couple of wireless channel models are also implemented in the veins framework, which is built on top of the OMNet++ simulator [SAI9] .

Our V&V method allows to add simulators and simulation frameworks to perform the overall system simulations as per test requirements. Some simulators and simulation frameworks that can be added to perform complete system simulations are the SUMO vehicle simulator [SAI2], Veins framework [SAI3], which simulates vehicular communications, and Veins-INET framework, which also simulates vehicular communication with additional communication protocols. A high-level diagram of different simulators and simulation frameworks that can be added to the base simulator (i.e., the OMNeT++ simulator) together with the attack simulation engine is numbered in the figure below.

                        Figure ‎2.8: Simulation-Based Attack Injection at System-level simulation environment.

Incorporating pre- and post-injection analyses [SAI5] [SAI6] [SAI7] [SAI8] into simulation-based attack injection to reduce the attack space. This will reduce the time, cost and effort needed to perform simulation-based attack injection at system level. The pre-injection analysis is done before any attack injection experiments are performed while post-injection uses the results of previous attack injection experiments. These techniques require detailed knowledge of the target system for efficient implementation.

Pre-injection analysis in simulation-based attack injection at the system level can be done by investigating the system and its environment before running any attack injection experiment. The purpose is to design an effective test campaign (i.e., a set of test experiments) in advance.  In case of an automated vehicle system, some factors that are imperative to perform pre-injection analysis are scenario (actions, events, goals), scene (dynamics elements and scenery) and situation (intersections, highways) etc. By analysing the above-mentioned factors, it can be possible to reduce the number of test cases without affecting the validity of the results. For example, in the intersection scenarios, the low-speed ones are the most interesting, while in the highway driving scenarios the higher speed ones are the more relevant.

Post-injection analysis in simulation-based attack injection is performed on previous attack injection results, resulting in reduced campaigns for subsequent attack injection rounds. The purpose is to run fewer experiments and still achieve equivalent results. For example, the results connected to the repeating driving patterns of the vehicles in a scenario might be the same, and for subsequent attack injection campaigns, the number of experiments might be reduced based on the result analysis of the first test campaign focusing on a single repeat of the driving patterns.