Computer vision system's operating limitations identification

Comparison of railway signal (both light signal and speed restriction signs) detection results for validation tests carried out under different conditions

Id Each requirement and test-case has a unique ID with running numbers based on the corresponding use case. In order to avoid name clashing between identifiers of requirements and test-cases, which can potentially lead to confusion, the letter ‘R’ is used to denote a requirement ID and the letters ‘TC’ are used to denote a test-case ID. For example, UC2_R_1 is the ID of the first requirement of use case 2, and UC3_TC_2 is the ID of the second test-case of use case 3.

Evaluation Scenario Textual Description Each evaluation scenario has a concise description, which is limited to one sentence to keep it as concise as possible.

Evaluation Scenario Rationale Risk he rationale / risk contains an explanation of what needs to be evaluated, referring to the “What” stated in the work plan.

Evaluation Scenario Owner The owner is the partner acting as the main contributor to this evaluation scenario

Evaluation Scenario Requirements

Evaluation Scenario Requirements List

V&V Methods

V&V Methods List of the methods used in this evaluation scenario

VALU3S Framework

Evaluation Environment Type Type of environment where the method is applied. It can be more than one type of environment.(Dimension 1)

Evaluation Type Type of evaluation performed with that method. If a method is hybrid, more than one type can be selected.(Dimension 2)

Type of Component Under Evaluation Type of components that can be evaluated with that method. It can be more than one type.(Dimension 3)

Evaluation Stage The evaluation stage where the method is used.(Dimension 5)

Purpose of the Component Under Evaluation The type of purpose that can be evaluated by a method. It can be more than one type.(Dimension 6)

Type of Requirement Under Evaluation The type of requirements that a method is able to evaluate. It can be more than one type.(Dimension 7)

Evaluation Performance Indicator The type of evaluation criteria or KPI that a method is able to improve. It can be more than one type.(Dimension 8)

Contents

UC9_R_10

An analysis of the results obtained during the tests carried out for a set of validation tests shall provide information to determine the conditions for a safe operation of the system.