Safety violations on speed restriction signs detection

Validation of the CV system for railway speed restriction signs detection under low light and/or bad meteorological conditions in different trips made on the same route.

Id Each requirement and test-case has a unique ID with running numbers based on the corresponding use case. In order to avoid name clashing between identifiers of requirements and test-cases, which can potentially lead to confusion, the letter ‘R’ is used to denote a requirement ID and the letters ‘TC’ are used to denote a test-case ID. For example, UC2_R_1 is the ID of the first requirement of use case 2, and UC3_TC_2 is the ID of the second test-case of use case 3.

Evaluation Scenario Textual Description Each evaluation scenario has a concise description, which is limited to one sentence to keep it as concise as possible.

Evaluation Scenario Rationale Risk he rationale / risk contains an explanation of what needs to be evaluated, referring to the “What” stated in the work plan.

Evaluation Scenario Owner The owner is the partner acting as the main contributor to this evaluation scenario

Evaluation Scenario Requirements

Evaluation Scenario Requirements List

V&V Methods

V&V Methods List of the methods used in this evaluation scenario

VALU3S Framework

Evaluation Environment Type Type of environment where the method is applied. It can be more than one type of environment.(Dimension 1)

Evaluation Type Type of evaluation performed with that method. If a method is hybrid, more than one type can be selected.(Dimension 2)

Type of Component Under Evaluation Type of components that can be evaluated with that method. It can be more than one type.(Dimension 3)

Evaluation Stage The evaluation stage where the method is used.(Dimension 5)

Purpose of the Component Under Evaluation The type of purpose that can be evaluated by a method. It can be more than one type.(Dimension 6)

Type of Requirement Under Evaluation The type of requirements that a method is able to evaluate. It can be more than one type.(Dimension 7)

Evaluation Performance Indicator The type of evaluation criteria or KPI that a method is able to improve. It can be more than one type.(Dimension 8)

Contents

UC9_R_6

Validation tests under different driving and environment conditions

UC9_R_7

The framework must provide tools to prepare the datasets for the validation tests in a semi-automatic way.

UC9_R_8

Information about all objects detected by the system during the execution of a validation test must be recorded (in a file) for further analysis.

UC9_R_9

Accuracy metrics must be calculated for each test execution, comparing the test execution results and the expected results, defined by the ground truth for each journey.