The Korean Society Fishries And Sciences Education

Journal Archive


[ Article ]
The Journal of the Korean Society for Fisheries and Marine Sciences Education - Vol. 34, No. 1, pp. 66-75
Abbreviation: J Kor Soc Fish Mar Edu.
ISSN: 1229-8999 (Print) 2288-2049 (Online)
Print publication date 28 Feb 2022
Received 14 Dec 2021 Revised 21 Jan 2022 Accepted 27 Jan 2022

Performance Verification Module Design for Safety Operation Support Service of MASS
Serng Bae MOON ; Won Jae YANG ; Wonjin CHOI*
Korea Maritime and Ocean University(professor)
Mokpo National Maritime University(professor)
*Korea Maritime and Ocean University(student)

자율운항선박 안전운항지원 서비스 성능 검증 모듈 설계
문성배 ; 양원재 ; 최원진*
Correspondence to : 051-410-4280,

Funding Information ▼


Safety operation support service (SOS service) is composed of six services to efficiently support Maritime autonomous surface ships (MASS) such as navigation, arrival and departure, cargo operation, inspection, etc. Since this service is developed based on software, performance verification of the service program should be accompanied. There are two methods to verify software. The first is a functional test, which verifies the function of the software and the second is a non-functional test, which verifies the rest except for the function. This study designed modules for performance verification of SOS service through the two test methods. The functional test identifies the implemented function based on the requirement specification for each service and to determine whether the function operates. The non-functional test determines the adequacy of the rest of the performance except for the function such as recovery ability, reliability, usability, etc. Especially for usability testing, user for each service and essential information for each user was classified to improve the efficiency of testing.

Keywords: Maritime autonomous surface ships, Safety operation support service, Software testing, Functional testing, Non-functional testing

Ⅰ. Introduction

Technology development related to maritime autonomous surface ships (MASS) is being promoted in earnest, led by advanced countries such as Korea, Norway, Finland, Japan, and the United States. For example, Finland succeeded in testing the fully autonomous passenger ship ‘falco’ for the first time in the world in December 2018. Japanese shipping company NYK succeeded in sea trial of ‘Iris Leader’ in 2019, a 20,000-ton class car carrier to which the automatic avoidance system SSR was applied (NYK Line, 2019).

Various studies such as collision avoidance, communication, remote control, and support systems have been conducted in Korea (Kim and Lee, 2018; Lee, 2018; Kim et al., 2019; Kang and Park, 2019; Bae et al., 2020). But there are limitations to being performed alone at the shipyards or research institutes because high-level experiments and verifications are required.

The Ministry of Trade, Industry and Energy and the Ministry of Oceans and Fisheries are jointly developing technologies related to MASS and an intelligent autonomous navigation system with the goal of leading international standards from 2020 (KASS project, 2020). This research and development is largely divided into 13 sub-tasks. One of the sub-tasks is safety operation support service (SOS service), develops technology for remote operation of MASS. The service consists of six modules related to arrival and departure, cargo operation, condition monitoring, inspection, etc. It aims to increase the safety and efficiency of MASS (Jang et al., 2020).

SOS service is developed based on software, and the software is complex and prone to errors and faults. In order for the development to proceed in the right direction, the software must be tested properly and in the right order.

There are many different ways to test software because software testing spans over the whole development cycle and requires many considerations, such as product characteristics, process organization, available resources and skills, and budget constraints (Baresi and Pezzè, 2006).

The aim of the research is to design suitable verification modules for SOS service among various software testing methods. Because research on MASS is just beginning, the software test method for MASS is not standardized, and it is difficult to find research related studies. This study intends to propose methods for verifying the SOS service by dividing it into functional and non-functional verification models. The functional verification model is designed to identify and test service items related to the requirements specification, and the non-functional verification model is designed to identify and test the non-functional requirements such as recovery, reliability, and usability.

Ⅱ. Methods
1. Concept of safety operation support service

SOS service consists of ‘Autonomous navigation support service’, ‘Marine accident response service’, ‘Berthing and mooring support service’, ‘Cargo operation and ship entry support service’, ‘Condition monitoring support service’, and ‘Port state control (PSC) inspection support service’.

[Fig. 1] 
Autonomous navigation support service process model.

[Fig. 2] 
Marine accident response service implementation function.

First, the ‘Autonomous navigation support service’ aims to provide the optimal route to MASS. The optimal route is based on the passage plan information provided by the MASS. It is created by considering the traffic situation of the relevant sea area and the risk with the incoming and outgoing ships. In particular, this service aims to contribute to the prevention of marine accidents in the vicinity of harbors by applying an algorithm for determining the possibility of collision with MASS.

Second, the ‘Marine accident response service’ aims to minimize the damage of additional accidents through prompt response by promptly disseminate to the relevant organizations when the ship is faced with a marine accident during voyage or when it recognizes the degree of exposure to the risk of a marine accident. In particular, this service includes a function that enables timely response by MASS remote control center and the MASS operators by recognizing the occurrence of accidents such as collision and capsizing in advance based on the ship’s heel angle data.

Third, ‘Berthing and mooring support service’ measures the ship’s speed and distance from the berth in real time using the camera and LiDAR (Light Detection and Ranging) installed on the berth, and provides it for the relevant parties to use. In addition, this service monitors soot or fire smoke generated by ships moored at the berth, and supports MASS to properly respond to emergencies occurring at the berth.

[Fig. 3] 
Berthing and mooring support service conceptual diagram.

Fourth, ‘Cargo operation and ship entry support service’ establishes an information linkage platform with the port logistics process to monitor the cargo handling status at the berth where the MASS will berth, and to predict the time required for cargo operation. The predicted information is provided to the related users of MASS and the land, and the user makes decisions such as adjusting the scheduled time of boarding pilot and berthing of MASS using the received information.

[Fig. 4] 
Cargo operation and ship entry support service process model.

Fifth, ‘Condition monitoring support service’ provides CBM (Condition Based Monitoring) information and PMS (Planned Maintenance System) information on five major equipment in the form of an integrated GUI (Graphic User Interface) for the purpose of supporting optimal condition monitoring between ships, shipping companies, and manufacturers. This service provides an intuitive and user-friendly GUI and dashboard for the efficient management of ship systems on the shore side and MASS.

[Fig. 5] 
Condition monitoring support service conceptual diagram.

Finally, ‘PSC inspection support service’ aims to establish a database of various agreements related to PSC inspection, a database of PSC defect codes defined in Tokyo-MOU, and a PSC checklist system that provides inspection items optimized for PSC inspection target vessels. Various databases are provided to autonomous vessel operators, shipping companies, and related organizations so that they can effectively prepare for PSC inspections of MASS.

[Fig. 6] 
PSC inspection support service process model

2. Service Verification Strategy

Since SOS service is software-based, performance verification of the service program must be accompanied. In general, performance verification is largely divided into functional verification and non-functional verification.

Functional verification means testing the function of software and documents such as requirement specification are used as test guidelines. It confirms that a desired output is generated when a specific input is provided using a black box testing technique. It includes different types of functional verification testing, such as unit testing, integration testing, functional testing, acceptance testing, smoke testing, sanity testing, regression tests, integration testing and more.

On the other hand, non-functional verification checks whether the operation of software or system meets non-functional requirements. The non-functional verification is as important as the functional verification because it involves everything but the functional part of the software. It includes different types of non-functional verification testing, such as security testing, usability testing, reliability testing, compatibility testing and more.

The procedure for both verification methods is as follows.

First, identify the function expected to be performed by the software. Second, generate input data according to the function specification. Finally, run the test case and check the output data to make sure the software is working properly.

The verification steps of both tests are the same, but there are differences in who performs the tests, what is the intent of the test, and what tools and technologies are used to perform the tests. Functional verification is mainly performed by quality assurance (QA) testers and check accuracy, efficiency, and business applicability. Non-function verification uses not only QA testers but also general users and testing tools, and check efficiency, response time, ease of use, ease of learning, security, etc.

Ⅲ. Results
1. Functional Verification Model

Functional verification checks whether the implemented functions of each services operate normally and the performance evaluation criteria is satisfied. This model is designed to identify verification items for each service according to specification of service requirements.

  • A. Autonomous navigation support service
  •  (1) Specification of service requirement
  •   - It should be possible to monitor the route status of other ships through auto identification system (AIS) information
  •   - It should be possible to plan the route on the electronic nautical chart (ENC), and it should provide general information about the route, section information, geographic information, and information about the schedule
  •   - Inputs, outputs, and events occurring in the system must all be logged
  •  (2) Identified verification items
  •   - Ship collision avoidance function
  •   - Real-time route exchange function
  •   - Information display function
  •   - Information transmission function
  • B. Marine accident response service
  •  (1) Specification of service requirement
  •   - Information should be exchanged according to a set process, and there should be no missing information
  •   - In the event of an accident, the necessary information should be selected in advance and a function should be provided so that it can be changed at any time
  •   - It should include a function to request and provide necessary information in case of an accident
  •   - A timer that can measure the situation propagation time should be configured
  •  (2) Identified verification items
  •   - Situation propagation function
  •   - Accident information transmission/reception function
  • C. Berthing and mooring support service
  •  (1) Specification of service requirement
  •   - It is necessary to provide an image of the target ship’s berthing
  •   - The speed information of the target ship and distance information from the berth must be provided
  •   - In the mooring situation, it is necessary to provide information on the detection results of workers present in the target berth and the detected fire
  •  (2) Identified verification items
  •   - Berthing support function
  •   - Smoke and fire detection function
  •   - Worker detection function
  •   - Image and information provision function
  • D. Cargo operation and ship entry support service
  •  (1) Specification of service requirement
  •   - The estimated time of completion of berthing of the vessel must be indicated
  •   - Estimated cargo operation time must be indicated
  •   - Estimated container cargo operation completion time should be indicated.
  •   - Information such as cargo throughput, occupancy, and waiting rate must be displayed by terminal, berth, and ship
  •   - Container work information management items should be displayed
  •   - Container and external vehicle movement tracking management items and road operation status should be displayed
  •  (2) Identified verification items
  •   - Cargo operation support function
  •   - Container operation time indication function
  •   - Cargo related information display function
  •   - Cargo tracking management information display function
  •   - Arrival and departure support function
  •   - Ship’s expected berthing completion time display function
  • E. Condition monitoring support service
  •  (1) Specification of service requirement
  •   - Overall information (temperature, viscosity, pressure, etc.) of engine and generator fuel, lubricant, cooling water, air, exhaust gas, etc. should be indicated
  •   - Various items such as major pumps and purifiers should be displayed
  •   - The overall details of the basic piping system should be indicated
  •  (2) Identified verification items
  •   - Engine, generator, pump, purifier, piping system, etc. related information display function
  • F. PSC inspection support service
  •  (1) Specification of service requirement
  •   - It should be possible to inquire the ship status for PSC inspection
  •   - It should be possible to designate PSC target ships
  •   - It should create PSC checklist for PSC target ships
  •   - It should be possible to input PSC inspection result
  •   - The PSC checklist should provide links to relevant agreements
  •  (2) Identified verification items
  •   - PSC related information inquiry function
  •   - PSC target ship designation function
  •   - PSC checklist creation function
  •   - PSC inspection result input function

In addition, common verification items for all services include a test for responding to multiple ships and a test for responding to changes in weather and environment.

2. Non-Functional Verification Model

Non-functional verification model is designed to identify which test methods are suitable for verifying the services and to derive suitable verification items for each service.

A. Recovery testing

Recovery testing is a test technique to evaluate whether software is normally terminated and data is appropriate when a system failure occurs. When a failure occurs in hardware, software, communication network, etc. of the six types of service, it diagnoses and verifies whether the failure is repaired. The test items are divided into three categories as follows.

  •  (1) Disability diagnosis test
  •   - Check whether it is possible to identify the failure caused by intentionally generating the failure for each failure item
  •  (2) Failure recovery test
  •   - Check manuals to deal with failures and the ability to recover from failures
  •  (3) Service normalization test
  •   - Measure the recovery time from failure and check if service function operation is possible after recovery from failure

B. Reliability testing

Reliability testing is a testing technique performed to verify that software provides the same output over a specific period of time in a given environment. In order to verify the six types of services are continuously and stably provided, a test is performed to find out the change in performance due to long-term operation.

C. Usability testing

Usability testing is a method for real users to test and evaluate software. By observing how users actually use the software, developers can identify and fix problem areas (Lewis, 2006).

The users of the implemented six types of services are diverse, such as MASS operators, institutions, shipping companies, pilots, etc. Also, the information used in the services varies according to users. Therefore, users for each service and necessary information according to users are classified as shown in Table 1. Through the test, it is checked whether the necessary information is provided to the user appropriately, and whether the visibility, readability, and ease of operation of the information are appropriate for use.

<Table 1> 
Classification of users and necessary information for usability testing
Service User Necessary information
Autonomous navigation
support service
- MASS operator Collision avoidance : presence of course change for collision avoidance, new course, estimated time to return to the route
Returning to the route : distance track distance (XTD), course and distance to the target waypoint, distance and time to the pilot station
- MASS remote control center Overall route plan of MASS, current course and speed, last waypoint, next waypoint, presence and reason of course change (external force, collision avoidance), changed route plan information (changed course, changed waypoint), estimated time to pilot station
Marine accident
response service
- Disaster response organization Types of marine accidents and time/location of occurrence, operation information of nearby ships, weather conditions in the accident area, extent of accident damage, accident level (minor, major, hazardous, catastrophic), requirements and content of marine accident support
- MASS operator Presence and content of marine accident support from onshore response organizations
Berthing and mooring
support service
- MASS operator
- Pilot
Berthing speed and acceleration, distance to the berth (bow and stern)
- Shipping company Distance from vessel to berth, berthing completion status, fire and smoke detection information
- MASS remote control center Berthing completion status, fire and smoke detection information
Cargo operation and ship
entry support service
- Shipping company
- MASS operator
Before arrival : Estimated time of completion of cargo work for berthing ships
Before departure : Container terminal cargo handling status (normal, delayed, suspended), estimated time to complete cargo work onboard
- Pilot
- Auxiliary service provider
(tug boat, pilot boat, etc.)
Before arrival : Estimated time of completion of cargo work for berthing ships
Before departure : Estimated time to complete cargo work onboard
Condition monitoring
support service
- MASS remote control center
- Shipping company
Status information of engines, generators, pumps, purifiers, piping system, etc.
PSC inspection
support service
- MASS operator
- Shipping company
- PSC inspector
MASS PSC inspection schedule, inspection items, corrective action requirements for each defect item, inspection history, PSC checklist

Ⅳ. Conclusion

Software testing is the process of verifying and validating that service designed and implemented according to requirements behaves as expected. This is to minimize the risk of development mistakes, as it is impossible to create perfect software (KSA, 2018).

SOS service developed to support MASS is software-based, and software testing is required to reduce development mistakes. Although there are countless different ways how to improve software, it is not possible to perform all test items. So, this study proposed functional and non-functional model for SOS service. The functional verification tests the sanity of the software function. For this, the verification items are identified according to the specification of service requirement. The non-functional verification tests the recovery ability, reliability, and usability of the software. In the usability test, users for each service are classified according to necessary information to increase efficiency.

This study suggested ways to verify the performance of SOS service from the developer’s point of view and the user’s point of view. The proposed functional and non-functional models identified the test methods required for each service and the items according to the test methods, but it did not suggest specific test scenarios. So, we intend to develop additional verification scenarios through future research to increase the degree of completion and satisfaction of the proposed performance verification methods.


This research was supported by the 'Development of Autonomous Ship Technology (20200615)' funded by the Ministry of Oceans and Fisheries (MOF, Korea).

1. Bae SH, Jun M and Jang EK(2020). Development of Scenarios for Evaluating the Collision Avoidance Performance of Maritime Autonomous Surface Ships in the Ocean. Journal of Fishier and Marine Educational Research, 32(5), 1115~1124.
2. James R. Lewis(2006). Usability Testing. IBM Software Group, 1~10.
3. Jang HS, Jo YH, Han GM and Song SH(2020). Introduction of six types of Safety Operation Support Service for Autonomous Ships. Proceedings of the Korean Institute of Navigation and Port Research Conference 2020, 116~118.
4. Kang WS and Park YS(2019). A Basic Study on the Application of Wireless Communication Technology in Vehicular Environment (V2X) for Maritime Autonomous Surface Ships. Korean Association of Maritime Police Science, 9(2), 267~288.
5. KASS Project(2020). Korea Autonomous Surface Ship Project.
6. Kim WO, Hong JH, Kim DH and Bae JY(2019). Development of Autonomous Navigation Support System using Maritime Traffic Risk Assessment Model. Journal of Fishier and Marine Educational Research, 31(1), 257~263.
7. Kim WO and LEE CH(2018). A Study on the Development of Collision Prevention Support System for Autonomous Surface Ship. Journal of Fishier and Marine Educational Research, 30(1), 227~235.
8. Korea Standards Association(2018). Software and systems engineering – Software testing – Part 1: Concepts and definitions. 13~21.
9. Lee KI(2018). Remote Control Management System for Autonomous Ship. Journal of Korea Convergence Society, 9(11), 45~51.
10. Luciano Baresi and Mauro Pezzè(2006). An Introduction to Software Testing. Electronic Notes in Theoretical Computer Science, 148(1), 89~111.
11. NYK Line(2019). NYK Conducts World’s First Maritime Autonomous Surface Ships Trial.