The workflow describes the application of the V&V method for Simulation-based fault and attack injection. This method allows fault- and attack-injection mechanisms to evaluate the system’s cybersecurity and safety properties e.g., by using simulator control commands during target system simulations.

The workflow has four inputs: scenario database, test requirements, fault and attack model library and a target simulation system. These inputs are described below.

The Scenario database has a set of scenarios that are inputs to the ComFASE execution flow. Each scenario defines e.g., a network of roads, vehicle maneuvers and their interactions. From the scenario database, a specific scenario can be selected based on the test requirements.

An intrinsic part of this simulation method is its fault and attack model library, which stores all the faults and attacks that could be modeled by the fault and attack injector. The user could then select a model from the library and inject that into the target system.

The list of test requirements is another input to the execution flow allowing the user to configure the tests and analyze the results. The requirements come from the test department, the stakeholders, or the customers.

The analyze system under test activity determines the fault and attack space structure and the details of the system under test, such as the core components of the system, working methodology, protocols used, and vulnerabilities if any. This information about the system could be used to perform pre-injection analysis to reduce the fault and attack space. Following are the activities within the method (i.e., Simulation-Based Fault and Attack Injection at System-level Improved).

After analyzing the system, the scenario is selected and customized to perform the test campaigns.

After customizing a scenario, the golden run can be performed. The data logged during the golden run may serve as a reference for analyses of the injections. As part of the activities performed within a golden run, detailed traffic data generated from the simulation system is logged in a database for offline analysis.

After executing the golden run activity, an attack injection campaign would need to be configured. This activity consists of setting parameters such as the attack’s type, value, initiation time, and duration for the specific scenario selected. The result of this activity is a test campaign that consists of a set of attack injection experiments.

After the attack injection campaign is configured, the attack injection experiments are performed in the target system according to the detailed configurations set in the test campaign. In case of simulation crash during an experiment run, results are stored in a database for result analysis. The simulation is then reset, and the test campaign continues by conducting the next experiment planned. In case of no simulation crashes, the results of the experiments, including the traffic data logged during the simulation are stored in a database for result analysis. Once the entire test campaign is finished, the results stored in the database are analyzed to evaluate the impact of attacks on the system.