Performance Verification Module Design for Safety Operation Support Service of MASS

1. Concept of safety operation support service

SOS service consists of ‘Autonomous navigation support service’, ‘Marine accident response service’, ‘Berthing and mooring support service’, ‘Cargo operation and ship entry support service’, ‘Condition monitoring support service’, and ‘Port state control (PSC) inspection support service’.

[Fig. 1]

Autonomous navigation support service process model.

[Fig. 2]

Marine accident response service implementation function.

First, the ‘Autonomous navigation support service’ aims to provide the optimal route to MASS. The optimal route is based on the passage plan information provided by the MASS. It is created by considering the traffic situation of the relevant sea area and the risk with the incoming and outgoing ships. In particular, this service aims to contribute to the prevention of marine accidents in the vicinity of harbors by applying an algorithm for determining the possibility of collision with MASS.

Second, the ‘Marine accident response service’ aims to minimize the damage of additional accidents through prompt response by promptly disseminate to the relevant organizations when the ship is faced with a marine accident during voyage or when it recognizes the degree of exposure to the risk of a marine accident. In particular, this service includes a function that enables timely response by MASS remote control center and the MASS operators by recognizing the occurrence of accidents such as collision and capsizing in advance based on the ship’s heel angle data.

Third, ‘Berthing and mooring support service’ measures the ship’s speed and distance from the berth in real time using the camera and LiDAR (Light Detection and Ranging) installed on the berth, and provides it for the relevant parties to use. In addition, this service monitors soot or fire smoke generated by ships moored at the berth, and supports MASS to properly respond to emergencies occurring at the berth.

[Fig. 3]

Berthing and mooring support service conceptual diagram.

Fourth, ‘Cargo operation and ship entry support service’ establishes an information linkage platform with the port logistics process to monitor the cargo handling status at the berth where the MASS will berth, and to predict the time required for cargo operation. The predicted information is provided to the related users of MASS and the land, and the user makes decisions such as adjusting the scheduled time of boarding pilot and berthing of MASS using the received information.

[Fig. 4]

Cargo operation and ship entry support service process model.

Fifth, ‘Condition monitoring support service’ provides CBM (Condition Based Monitoring) information and PMS (Planned Maintenance System) information on five major equipment in the form of an integrated GUI (Graphic User Interface) for the purpose of supporting optimal condition monitoring between ships, shipping companies, and manufacturers. This service provides an intuitive and user-friendly GUI and dashboard for the efficient management of ship systems on the shore side and MASS.

[Fig. 5]

Condition monitoring support service conceptual diagram.

Finally, ‘PSC inspection support service’ aims to establish a database of various agreements related to PSC inspection, a database of PSC defect codes defined in Tokyo-MOU, and a PSC checklist system that provides inspection items optimized for PSC inspection target vessels. Various databases are provided to autonomous vessel operators, shipping companies, and related organizations so that they can effectively prepare for PSC inspections of MASS.

[Fig. 6]

PSC inspection support service process model

2. Service Verification Strategy

Since SOS service is software-based, performance verification of the service program must be accompanied. In general, performance verification is largely divided into functional verification and non-functional verification.

Functional verification means testing the function of software and documents such as requirement specification are used as test guidelines. It confirms that a desired output is generated when a specific input is provided using a black box testing technique. It includes different types of functional verification testing, such as unit testing, integration testing, functional testing, acceptance testing, smoke testing, sanity testing, regression tests, integration testing and more.

On the other hand, non-functional verification checks whether the operation of software or system meets non-functional requirements. The non-functional verification is as important as the functional verification because it involves everything but the functional part of the software. It includes different types of non-functional verification testing, such as security testing, usability testing, reliability testing, compatibility testing and more.

The procedure for both verification methods is as follows.

First, identify the function expected to be performed by the software. Second, generate input data according to the function specification. Finally, run the test case and check the output data to make sure the software is working properly.

The verification steps of both tests are the same, but there are differences in who performs the tests, what is the intent of the test, and what tools and technologies are used to perform the tests. Functional verification is mainly performed by quality assurance (QA) testers and check accuracy, efficiency, and business applicability. Non-function verification uses not only QA testers but also general users and testing tools, and check efficiency, response time, ease of use, ease of learning, security, etc.

1. Functional Verification Model

Functional verification checks whether the implemented functions of each services operate normally and the performance evaluation criteria is satisfied. This model is designed to identify verification items for each service according to specification of service requirements.

• A. Autonomous navigation support service
•  (1) Specification of service requirement
•   - It should be possible to monitor the route status of other ships through auto identification system (AIS) information
•   - It should be possible to plan the route on the electronic nautical chart (ENC), and it should provide general information about the route, section information, geographic information, and information about the schedule
•   - Inputs, outputs, and events occurring in the system must all be logged
•  (2) Identified verification items
•   - Ship collision avoidance function
•   - Real-time route exchange function
•   - Information display function
•   - Information transmission function
• B. Marine accident response service
•  (1) Specification of service requirement
•   - Information should be exchanged according to a set process, and there should be no missing information
•   - In the event of an accident, the necessary information should be selected in advance and a function should be provided so that it can be changed at any time
•   - It should include a function to request and provide necessary information in case of an accident
•   - A timer that can measure the situation propagation time should be configured
•  (2) Identified verification items
•   - Situation propagation function
•   - Accident information transmission/reception function
• C. Berthing and mooring support service
•  (1) Specification of service requirement
•   - It is necessary to provide an image of the target ship’s berthing
•   - The speed information of the target ship and distance information from the berth must be provided
•   - In the mooring situation, it is necessary to provide information on the detection results of workers present in the target berth and the detected fire
•  (2) Identified verification items
•   - Berthing support function
•   - Smoke and fire detection function
•   - Worker detection function
•   - Image and information provision function
• D. Cargo operation and ship entry support service
•  (1) Specification of service requirement
•   - The estimated time of completion of berthing of the vessel must be indicated
•   - Estimated cargo operation time must be indicated
•   - Estimated container cargo operation completion time should be indicated.
•   - Information such as cargo throughput, occupancy, and waiting rate must be displayed by terminal, berth, and ship
•   - Container work information management items should be displayed
•   - Container and external vehicle movement tracking management items and road operation status should be displayed
•  (2) Identified verification items
•   - Cargo operation support function
•   - Container operation time indication function
•   - Cargo related information display function
•   - Cargo tracking management information display function
•   - Arrival and departure support function
•   - Ship’s expected berthing completion time display function
• E. Condition monitoring support service
•  (1) Specification of service requirement
•   - Overall information (temperature, viscosity, pressure, etc.) of engine and generator fuel, lubricant, cooling water, air, exhaust gas, etc. should be indicated
•   - Various items such as major pumps and purifiers should be displayed
•   - The overall details of the basic piping system should be indicated
•  (2) Identified verification items
•   - Engine, generator, pump, purifier, piping system, etc. related information display function
• F. PSC inspection support service
•  (1) Specification of service requirement
•   - It should be possible to inquire the ship status for PSC inspection
•   - It should be possible to designate PSC target ships
•   - It should create PSC checklist for PSC target ships
•   - It should be possible to input PSC inspection result
•   - The PSC checklist should provide links to relevant agreements
•  (2) Identified verification items
•   - PSC related information inquiry function
•   - PSC target ship designation function
•   - PSC checklist creation function
•   - PSC inspection result input function

In addition, common verification items for all services include a test for responding to multiple ships and a test for responding to changes in weather and environment.

2. Non-Functional Verification Model

Non-functional verification model is designed to identify which test methods are suitable for verifying the services and to derive suitable verification items for each service.

A. Recovery testing

Recovery testing is a test technique to evaluate whether software is normally terminated and data is appropriate when a system failure occurs. When a failure occurs in hardware, software, communication network, etc. of the six types of service, it diagnoses and verifies whether the failure is repaired. The test items are divided into three categories as follows.

•  (1) Disability diagnosis test
•   - Check whether it is possible to identify the failure caused by intentionally generating the failure for each failure item
•  (2) Failure recovery test
•   - Check manuals to deal with failures and the ability to recover from failures
•  (3) Service normalization test
•   - Measure the recovery time from failure and check if service function operation is possible after recovery from failure

B. Reliability testing

Reliability testing is a testing technique performed to verify that software provides the same output over a specific period of time in a given environment. In order to verify the six types of services are continuously and stably provided, a test is performed to find out the change in performance due to long-term operation.

C. Usability testing

Usability testing is a method for real users to test and evaluate software. By observing how users actually use the software, developers can identify and fix problem areas (Lewis, 2006).

The users of the implemented six types of services are diverse, such as MASS operators, institutions, shipping companies, pilots, etc. Also, the information used in the services varies according to users. Therefore, users for each service and necessary information according to users are classified as shown in Table 1. Through the test, it is checked whether the necessary information is provided to the user appropriately, and whether the visibility, readability, and ease of operation of the information are appropriate for use.

<Table 1>

Classification of users and necessary information for usability testing