UC2_Eval_VV_8

The effort needed for testing –The person-hours required for teleoperation system verification and validation are measured for the baseline testing (i.e., physical testing) and compared to simulation-based testing. The time and effort of testing greatly improved and have a positive impact on the overall efficiency.

Description of the evaluation

Use Case Id

Test case Test case where the V&V method has been used to improve a evaluation criteria. The test case where the V&V process improvement has been achieved

Requirement The validated requirement where a VALU3S V&V method has been used.

Evaluation Scenario The evaluation scenario where the V&V evaluation result has been obtained

V&V method

V&V Tool

Evaluation Criteria for V&V Processes

VALU3S V&V process Evaluation Criteria

Quantitative results Quantitative results , it can include the formula used to measure the value based on D5.2 evaluation criterias

Real-world testing involves utilizing physical vehicles and creating a secure testing environment, which introduces various logistical and safety challenges. In contrast, simulation-based testing eliminates these concerns. However, the complexity of the system being tested can present difficulties in setting up a realistic simulation-based test environment.

Considering the specific requirements and execution process of the tests, we have estimated that verifying the functional requirements of the teleoperation system in the real world requires approximately one day (480 minutes), including all necessary preparations. This time allocation is roughly distributed as follows: 180 minutes for preparation tasks such as equipment setup and vehicle retrieval, 120 minutes for the actual testing, and 60 minutes for concluding activities. Consequently, a single test typically takes around 96 minutes to complete.

In comparison, executing the same test in a simulation-based environment is estimated to take 30 minutes or less, significantly reducing the time required."

This revised version provides a more precise description of the challenges associated with real-world testing and the benefits of simulation-based testing. It also provides more concise and specific information regarding the time required for real-world and simulation-based tests.

Testing time effort.JPG

Long description

It is important to note that the previously mentioned time for real-world testing refers to the estimated duration required to conduct one test for a particular attack model.

For more comprehensive and detailed test campaigns, conducting tests in the real world may require additional time. Furthermore, the complexity of the system and the number of attack models being evaluated can significantly impact the overall testing duration.

Considering these factors, a simulation environment is highly recommended for running comprehensive test campaigns. Simulation offers the advantage of scalability, allowing for the efficient execution of a wide range of test scenarios and attack models. It enables researchers to explore different parameters, configurations, and edge cases without the time constraints and logistical challenges associated with real-world testing.

By utilizing a simulation environment, researchers can thoroughly evaluate the system's performance and robustness, covering a broader spectrum of attack scenarios and system variations.

Contents

Testing time effort.JPG