Simulation-Based Fault Injection at System-level with additional fault models valid for multiple IVC layers

'Simulation-based Fault Injection at System-level with additional fault models valid for multiple IVC layers' provides an opportunity of injecting faults on the system level. The complete system behavior can be analyzed when a certain sub-system is under the influence of faults. This method could span over various tools such as SUMO (Simulation of Urban Mobility), VEINS (vehicular network simulator) and INET allowing different aspects of the system to be evaluated.

Method Purpose Short description of the method's purpose and main benefits

The purpose of simulation-based fault injection at system-level is to evaluate system’s dependability by injecting faults, e.g., using simulator control commands during target system simulations. Like the method, “Simulation-Based Attack Injection at System-level with additional attack models valid for multiple layers of vehicular communication”, this improved method also allows to inject faults on multiple layers of vehicular communication system such as, Application, MAC and PHY layers as well as on wireless channels modelled for inter-vehicle communication. Moreover, this method provides the capability to the user to perform pre- and post-injection analysis based on the previous fault injection test results and analysis. This way, the fault space of upcoming test campaign can be iteratively reduced which results in to reduced time, cost, and overall V&V effort.

Method description A detailed description of the method

System-level simulation is comprised of hardware and software models of a cyber physical system (CPS). The fault injection could be performed on different abstraction layers such as logical, functional, hardware, software or system level etc [SFI1]. In this case, we focus on the simulation-based fault injection at system level.

In our case the simulation-based fault injection at system level is done through injection of faults on wireless communication systems modelled in Veins and Veins-INET simulators together with SUMO traffic simulator [SFI3] [SFI4]. The simulation-based fault injection at system level using simulators may be used for safety testing of automated systems such as autonomous vehicles [SFI2].

Simulation-based fault injection is a V&V method where faults are injected into system model or system software in a simulation environment.

Multiple attack models valid for multi-layer vehicular communication in the simulation-based fault injection at system level is done through injection of fault on the vehicular communication system modelled in Veins (Vehicles in Network Simulation).

The communication layers that are modelled in veins framework are, MAC and PHY layers as well as the wireless channel models. Veins framework itself is built on top of OMNet++ simulator [SFI10].

Our V&V method allows to add simulators and simulation frameworks to perform the overall system simulations as per test requirements. Some simulators and simulation framework that can be added to perform complete system simulations are, SUMO vehicle simulator, Veins framework that simulates the vehicular communication network and Veins-INET framework that also simulates vehicular communication network etc [SFI3] [SFI4] [SFI11]. Furthermore, it should then be possible to simulate a variety of system components, environmental conditions and AD/ADAS features that cannot be simulated using one simulator alone.

Incorporating Fault injection pre- and post-injection analyses [SFI6] [SFI7] [SFI8] [SFI9] into simulation-based fault injection to reduce the fault space. This will reduce the time, cost and effort needed to perform simulation-based fault injection at system level. Pre-injection analysis is done before any fault injection experiments are performed while post-injection uses the results of previous fault injection experiments. These techniques require detailed knowledge of the target system for efficient implementation.

Pre-injection analysis in simulation-based fault injection at system-level, can be done through investigating the system and its environment before running any fault injection experiment. The purpose is to design an effective test campaign (i.e., a set of test experiments) in advance. In case of an automated vehicle system, some factors that are imperative to perform pre-injection analysis are scenario (actions, events, goals), scene (dynamics elements and scenery) and situation (intersections, highways) etc.

Post-injection analysis in simulation-based fault injection is performed on previous fault injection results, to reduce test campaigns for subsequent fault injection campaigns. The purpose is to run less experiments and still achieve acceptable and high-quality results.

Method Strengths A listof the strengths of the method

The method inherits, and further improves, the strengths from the base-method i.e., Simulation-Based Fault Injection at System-level.

Method Limitations The limitations of the method

The method inherits the limitations from the base-method i.e., Simulation-Based Fault Injection at System-level.

Method References Bibliography references to papers about the method.

[SFI1] M.-C. Hsueh, T.K. Tsai, and R.K. Iyer, “Fault Injection Techniques and Tools,” Computer, vol. 40, no. 4, pp. 75-82, Apr. 1997.
[SFI2] S. Jha et al., “AVFI: Fault Injection for Autonomous Vehicles,” in Proc. 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), pp. 55–56. DOI: 10.1109/DSN-W.2018.00027
[SFI3] Michael Behrisch, Laura Bieker et al., “SUMO – Simulation of Urban Mobility, An Overview”, Institute of Transportation Systems, German Aerospace Center, Rutherfordstr. 2, 12489 Berlin, Germany.
[SFI4] D. Eckhoff and C. Sommer, “A Multi-channel IEEE 1609.4 and 802.11p EDCA model for the Veins framework,” in Proceedings of 5^th ACM/ICST international conference on simulation tools and techniques for communications, networks and systems: 5th ACM/ICST international workshop on OMNet++.(Desenzano, Italy, 19-23 March, 2012). OMNeT, 2012.
[SFI6] J. Grinschgl, A. Krieg, C. Steger, R. Weiss, H. Bock and J. Haid, "Efficient fault emulation using automatic pre-injection memory access analysis," 2012 IEEE International SOC Conference, Niagara Falls, NY, 2012, pp. 277-282. DOI: 10.1109/socc.2012.6398361
[SFI7] B. Sangchoolie, F. Ayatolahi, R. Johansson and J. Karlsson, "A Comparison of Inject-on-Read and Inject-on-Write in ISA-Level Fault Injection," 2015 11th European Dependable Computing Conference (EDCC), Paris, 2015, pp. 178-189.
[SFI8] Czeck, Edward W. and Daniel P. Siewiorek. “Observations on the Effects of Fault Manifestation as a Function of Workload.” IEEE Trans. Computers 41 (1992): 559-566. DOI: 1109/12.142682
[SFI9] Folkesson P., Karlsson J. (1999) Considering Workload Input Variations in Error Coverage Estimation. In: Hlavička J., Maehle E., Pataricza A. (eds) Dependable Computing — EDCC-3. EDCC 1999. Lecture Notes in Computer Science, vol 1667. Springer, Berlin, Heidelberg.
[SFI10] OMNeT++ Simulation Models and Tools; https://omnetpp.org/download/models-and-tools.
[SFI11] Veins-INET simulation framework; https://veins.car2x.org/documentation/modules/#veins_inet

Method Dimensions

Evaluation Environment Type Type of environment where the method is applied. It can be more than one type of environment. (Dimension 1)

Evaluation Type Type of evaluation performed with that method. If a method is hybrid, more than one type can be selected. (Dimension 2)

Type of Component Under Evaluation Type of components that can be evaluated with that method. It can be more than one type. (Dimension 3)

Evaluation Stage The evaluation stage where the method is used. (Dimension 5)

Purpose of the Component Under Evaluation The type of purpose that can be evaluated by a method. It can be more than one type. (Dimension 6)

Type of Requirement Under Evaluation The type of requirements that a method is able to evaluate. It can be more than one type. (Dimension 7)

Evaluation Performance Indicator The type of evaluation criteria or KPI that a method is able to improve. It can be more than one type. (Dimension 8)

Relations

Related Methods A method could be related to other methods.

Tools A method can use zero or more tools.

Part Method A method (when it is a combination: Combination of method artefact that is a specialization of a V&V Method) could be composed of a set of V&V Methods.

Test Case or Verification and Validation activity Test Case or V&V activity performed in a Use Case. We will link to the method to the test cases where the method is used.

Standards A method can be linked with standards.

Context

Workflow

Contents

There are currently no items in this folder.