Comparing Simulation with Physical Verification and Validation in a Maritime Test Field

The steadily increasing complexity of maritime systems substantially raised the need for advanced verification and validation (V&V) as well as certification methods. Extensive simulation-based certification adds new opportunities to existing physical testing. Compared with simulation, field tests are extremely time-consuming and therefore expensive. Furthermore, relevant close-range situations between ships or environmental impacts (e.g. certain types of bad weather situation) are impossible to perform in the field for safety reasons and the uncontrollability of the environment or simply the amount of experiments needed. Systems in the maritime domain (like products for navigation assistance, sensors, communication equipment etc.) are typically not used isolated but as part of a complex setup. More and more sensors and actuators are integrated to provide data for various systems or information services on board a ship and ashore. Since such systems are typically continuously evolving during their service lifetime, the development and maintenance of maritime systems (e.g. bridge systems) need to considered in its usage context that includes interconnected systems and external services, sensors and actuators. CPSoS (Cyber-Physical System of Systems) demand innovative approaches for distributed optimization, novel distributed management and control methodologies that can also deal with partially autonomous systems, and must be resilient to faults or cyber-attacks. In addition, CPSoS engineering no longer maintains the former strict separation between the engineering phases and actual operation. Instead, integrated approaches for the design- and operation- phase are required to cover the full lifecycle by modelling, simulation, validation, and verification (V&V). Thus, prospectively, it will be necessary to monitor the system formation and to conduct a final assessment of the system by means of a suitable application of test cases in a controlled and comprehensible manner. These systems have an emerging behavior and cannot entirely defined during the design phase. At this point it becomes apparent that conventional unit, integration and system tests are no longer sufficient to fully cover and validate the functional limits of Cyber-Physical System of Systems. An acceptable test coverage cannot be achieved with these methods for such systems. In this paper the authors present a use case of collision-regulation compliance checker to compare virtual (i.e. simulation-based) V&V, physical (i.e. in-situ testing) V&V and hybrid, mixed-reality V&V.