UC2_Eval_SCP_5

Potential Impact of Incidents and Attacks- Used this criterion to evaluate results from runtime verification tests. The executed tests deal with the impact of jamming attacks that could introduce additional delays in the communication system which could result in the unpleasant consequence including disruption of the service, and damage to the vehicle or the surrounding environment.
UC2
Potential Impact of Incidents and Attacks- Used this criterion to evaluate results from runtime verification tests. The executed tests deal with the impact of jamming attacks that could introduce additional delays in the communication system which could result in the unpleasant consequence including disruption of the service, and damage to the vehicle or the surrounding environment.
Evaluation Criteria for Safety, Cybersecurity, and Privacy (SCP)
Potential impact of incidents and attacks

The UC2_Eval_SCP_5 is related to the impact of the faults introduced or the jamming attacks on the transmission link which causes communication delay or total loss of the communication. This SCP requirement is tightly connected to the UC2_Eval_SCP_1. Hence the explanation and results presented for  UC2_Eval_SCP_1 are also valid for UC2_Eval_SCP_5.

So, to assess the teleoperation system's ability to handle delayed data or connection loss, we conducted comprehensive tests. These tests involved introducing various delays into the transmission link to simulate faults or congestion within the communication system. Additionally, we intentionally simulated complete connection loss of the transmission link, representing a Denial-of-Service (DoS) attack on the communication system. Our objective was to verify the system's resilience and performance under challenging conditions.

Upon analyzing the test results, we made an intriguing observation: as the attack duration increased, the vehicle's tolerance for delays decreased, leading to an earlier triggering of the vehicle's safe stop mechanism. This indicates that prolonged attack durations render the system more susceptible to delays and disruptions in the communication link. Furthermore, as the delay in the transmission link increased, we noticed a gradual disconnection between the remote station and the 'ecu,' as illustrated in the accompanying figure.

These findings underscore the critical role of maintaining a reliable and low-latency communication channel to ensure the smooth operation of the teleoperation system. They emphasize the imperative need for robust countermeasures to mitigate the potential impact of attacks and disruptions on the communication infrastructure. By comprehending the system's behavior under various attack scenarios, we can develop effective strategies to enhance its resilience and improve its ability to handle delayed data or connection loss efficiently.

DelayAttack.jpg

The detailed results from the automated testing will also be available soon and provided upon request.

For in-depth analysis, we created and executed five test suites which are as follows,

1- DoS attack where both video and command channels are disconnected.

2- DoS attack where only the command channel is disconnected.

3- DoS attack where only the video channel is disconnected.

4- Delay attack where the communication on both video and command channels is delayed.

5- Delay attack where the communication on command channels is delayed. 

6- Delay attack where the communication video channel is delayed.

The purpose of these test suites is to verify and validate the two main safety mechanisms (aka fallback mechanism) which are implemented in the teleoperation system.  Following are the safety mechanisms that we used for result classification,

1- Vehicle Safe Stop: when the communication is delayed to a certain duration, the teleoperated vehicles must reduce their speed for ensuring safety.  The safe stop can be a slight safe stop which low deceleration rates or the safe stop can trigger a high deceleration rate (emergency braking) depending on the delay introduced.  the threshold for safe stop activation is when the communication delay is between 150ms and 1500ms.

2- Vehicle Disconnect: If the communication delay exceeds the 1500ms threshold, the disconnection between the remote station and the ECU must be initiated. 

We also automated the test configuration and execution process to facilitate the simulation-based testing.

Following are the test configuration values we have used for detailed analysis and evaluation:

Test scenario: The "top speed" test scenario is selected for testing the safety mechanisms of the system. In this scenario, the speed starts to accelerate to achieve a maximum speed which is 50 km/h, and then decelerates until the vehicle is stopped. The total duration of the test is 15 seconds. See the figure below which depicts the test scenario.

Attack start time: We choose the attack start time from 1 second to 7 seconds with the step of 1 second. The reason behind this selection is that the speed of the vehicle stabilizes at 1 second and 7 seconds was selected for the better coverage of the test scenario. See the figure below which depicts the test scenario.

Attack duration: The total of seven attack durations are selected for each attack start time. 

Attack value: There are 24 attack values selected in total. The minimum selected value is 50 ms and the maximum value is 1800 ms. The safe stop should activate between 50 ms and 150 ms. Delays of more than 150 ms should initiate the disconnection as mentioned above.

The total test cases that we ran are 1680 which takes approximately 32 hours to complete.

topspeedscen.JPG

Contents