Safety violations on signal (light) detection.

Validation of the CV system for railway light signal detection (fixed or flashing red, orange and green lights) under low light and/or bad meteorological conditions in different trips made on the same route.

Id Each requirement and test-case has a unique ID with running numbers based on the corresponding use case. In order to avoid name clashing between identifiers of requirements and test-cases, which can potentially lead to confusion, the letter ‘R’ is used to denote a requirement ID and the letters ‘TC’ are used to denote a test-case ID. For example, UC2_R_1 is the ID of the first requirement of use case 2, and UC3_TC_2 is the ID of the second test-case of use case 3.

Evaluation Scenario Textual Description Each evaluation scenario has a concise description, which is limited to one sentence to keep it as concise as possible.

Evaluation Scenario Rationale Risk he rationale / risk contains an explanation of what needs to be evaluated, referring to the “What” stated in the work plan.

Evaluation Scenario Owner The owner is the partner acting as the main contributor to this evaluation scenario

Evaluation Scenario Requirements

Evaluation Scenario Requirements List

V&V Methods

V&V Methods List of the methods used in this evaluation scenario

VALU3S Framework

Evaluation Environment Type Type of environment where the method is applied. It can be more than one type of environment.(Dimension 1)

Evaluation Type Type of evaluation performed with that method. If a method is hybrid, more than one type can be selected.(Dimension 2)

Type of Component Under Evaluation Type of components that can be evaluated with that method. It can be more than one type.(Dimension 3)

Evaluation Stage The evaluation stage where the method is used.(Dimension 5)

Purpose of the Component Under Evaluation The type of purpose that can be evaluated by a method. It can be more than one type.(Dimension 6)

Type of Requirement Under Evaluation The type of requirements that a method is able to evaluate. It can be more than one type.(Dimension 7)

Evaluation Performance Indicator The type of evaluation criteria or KPI that a method is able to improve. It can be more than one type.(Dimension 8)

Contents

UC9_R_1

Tests must be executed under different driving and environment conditions

UC9_R_2

The framework must provide tools to prepare the datasets for the validation tests in a semi-automatic way

UC9_R_3

Information about all objects detected by the system during the execution of a validation test must be recorded (in a file) for further analysis.

UC9_R_4

Evidence of the execution of validation tests for light signals detection must be recorded

UC9_R_5

Accuracy metrics must be calculated for each test execution, comparing the test execution results and the expected results, defined by the ground truth for the journey.