For automated vehicles to be safely released to the market, it must be shown that cybersecurity threats do not jeopardise safety. As it is virtually impossible to validate an automated vehicle against all possible scenarios it will face in the real world, least of all in combination with security threats, there is a need to balance the representativeness of the tests and the reliable performance indicators. The method describes a way to define testing and validation procedures of ADS features that can combine tests both in simulation and in real environments, such as test tracks, into a complete assessment.

This mostly manual method could be used to establish independent tests to evaluate and compare the ability of ADSs to withstand security threats that can affect their safe operation. The method (see figure ‎below) consists of:

1. Extracting enough information from the feature description to facilitate categorisation of feature classes for an ADS under test. With feature class, we mean generic descriptions of ADS features (e.g., Automated Lane Keeping Systems (ALKS)) that match functionality offered by several vendors.
2. Development of an appropriate test suite for test facilities to assess cybersecurity, matching the feature class and sensor setup (e.g., attacks on camera, lidar, radar, satellite navigation and v2x). Attack the vehicle on a test track and capture the post-attack behaviour i.e., record what the vehicle does after the attack.
3. Co-simulation of post-attack behaviour with critical traffic scenarios (that is relevant for the feature class) to evaluate safety criteria. This is the step that that need tool support and with potential to be highly automated.
4. Evaluate coverage and update the test suite if needed and assess if the safety criterion holds.

The methodology generates tests that have been shown to be relevant to establishing a baseline for cybersecurity of an ADS. The proposed process is somewhat different from a more traditional approach where the combinatorial explosion renders test coverage unfeasible when considering all types of attacks and traffic scenarios. In contrast, this approach does not provide complete coverage but instead aims to build confidence. The co-simulation approach identifies the critical scenarios that need to be tested and makes the risks in the validation testing predictable, thus enabling a proactive strategy to address the hazards. The identified critical scenarios, comprised of relevant attacks in representative traffic conditions, may well be orchestrated and evaluated to form a comparable independent cybersecurity assessment.

Methodology of tests that could support establishing cybersecurity-informed safety in an ADS