Relationships with other web-repo artefacts

Related V&V Methods A tool could be based on several V&V methods.

• V&V of machine learning-based systems using simulators The traditional methods for verification and validation of a rule-based system is not effective for testing fuzzy machine learning based models. For safety reasons these models are first tested in simulators and the goal is to close the gap between these simulated environments and the real world.

Standards A method can be linked with standards.

Use cases Use cases where the tool has been used

• UC1 - Intelligent Traffic Surveillance Wihin the use case, CAMEA is planning to investigate smart and mostly wireless sensors (cameras, radars, etc.) in terms of testing and verification of quality requirements such as reliability and security for applications built on the Unicam platform. The VALU3S framework shall be used as an input generator covering various potentially interesting situations as data collection in real traffic environments is costly and does not guarantee full coverage of the input space. This could be problematic when training ML-based algorithms for safety critical systems, i.e., novel approaches to support robustness must be explored. Based on this, within the framework, the behaviour of such systems will be analysed, tested and validated against defined targets (e.g. dependability or trustworthiness). This should result in decreased development effort and subsequent deployment, and result in better maintainability of systems in the field.

Functional Toolchain Functional connection of tools for the implementation of a specific evaluation scenario/workflow. Example: the output of one tool can be used by the other tool(s) in order to obtain a different information (e.g., CIROS FERAL, MARVER >-> SRVT, etc)

• CARLA An open-source autonomous driving simulator.
• Esmini Environment Simulator Minimalistic
• BeamNG An open-world vehicle simulation video game developed and published by Bremen-based video game developer BeamNG GmbH
• LGSVL SVL SIMULATOR - An end-to-end autonomous vehicle simulation platform
• OpenRoadEd Simple application designed to create both logical (OpenDRIVE standard) and geometrical (OpenSceneGraph) descriptions of road networks.
• ScenarioGenerator ScenarioGenerator performs search-based software testing by automatic generation of various traffic scenarios in simulator to verify and validate the intended performance of ML based components in CPS systems.

Improvement Classification

Evaluation Criteria for Safety, Cybersecurity, and Privacy (SCP)

Accuracy of simulated sensor output, Simulation environment functionality, Simulation environment quality

Evaluation Criteria for V&V Processes

Open source - Goals

open Source

No

Goals Tool goals

Functional Requirements

Contents

There are currently no items in this folder.