JP2025028789A - Information Processing System - Google Patents

Abstract

An information processing system for safely and efficiently avoiding obstacles while navigating is provided.
[Solution] The information processing system disclosed herein includes a control unit that performs information processing to avoid obstacles and navigate to a destination, and the control unit performs a parameter determination process that determines parameters related to the distance the ship will move away from an obstacle when avoiding the obstacle, based on obstacle information including position information and size information of the obstacle on or underwater.
[Selected Figure] Figure 1

Description

This disclosure relates to an information processing system.

Conventionally, when a ship navigates, it is necessary to avoid obstacles such as floating objects on the water surface and other ships, and technology to do so is required. For example, Patent Document 1 discloses a system that integrates sensor data from multiple sensors to detect foreign objects present on the ship's path of travel and assist the ship in avoiding collisions.

JP 2019-036010 A

However, if a ship navigates a route that is farther away from an obstacle than necessary in order to avoid it, it will have to take a detour, which can lead to poor time and fuel efficiency.

Therefore, this disclosure has been made in consideration of the above problems, and its purpose is to provide an information processing system for safely and efficiently avoiding obstacles while navigating.

According to the present disclosure, a control unit that executes information processing for avoiding obstacles and navigating to a destination is provided,
An information processing system is provided in which the control unit executes a parameter determination process to determine parameters related to the distance the ship will move away from an obstacle when avoiding the obstacle, based on obstacle information including position information and size information of the obstacle on or underwater.

This disclosure provides an information processing system for safely and efficiently avoiding obstacles while navigating.

FIG. 1 is a diagram illustrating an information processing system according to an embodiment of the present disclosure. FIG. 2 is a diagram showing a ship to which an example of an information processing system according to the embodiment is applied. 11 is a flowchart illustrating a series of controls in the information processing system according to the embodiment. FIG. 11 is a diagram showing an example of an appropriate obstacle avoidance distance according to the embodiment; FIG.

A preferred embodiment of the present disclosure will be described in detail below with reference to the attached drawings. Note that in this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals to avoid redundant description.

Figure 1 is an example of an information processing system (hereinafter also simply referred to as "system") according to this embodiment. The system of this embodiment can be installed on any type of vessel, and is particularly suitable for relatively small vessels. Specifically, the system of this embodiment can be adopted in fishing boats, water taxis, small multipurpose vessels, pleasure fishing boats, passenger boats, transportation boats, work boats, fireboats/patrol boats, pleasure yachts, pleasure motorboats, special work vessels, etc., and can also be adopted in large passenger ships, tankers, cargo ships, etc.

The system of this embodiment includes a control unit that executes information processing to avoid obstacles and navigate to a destination. The control unit executes a parameter determination process to determine parameters related to the distance the ship will move away from an obstacle when avoiding the obstacle, based on obstacle information including the position information and size information of the obstacle on or underwater.

The system of this embodiment is an information processing system 1 for safely and efficiently avoiding obstacles while a ship is navigating. Specifically, by determining parameters related to the distance to be kept away from an obstacle based on information on the position and size of the obstacle, it is possible to safely avoid the obstacle while preventing a decrease in efficiency by not making any unnecessary detours.

This system 1 includes, for example, a data acquisition unit 10, a control unit 20, a memory unit 30, an input unit 40, an output unit 50, a communication unit 60, and a ship operation control unit 70. Each component is connected by wire or wirelessly and can communicate with each other. In this example, the control unit 20, the memory unit 30, the input unit 40, the output unit 50, and the communication unit 60 are implemented in an information processing device (computer), but this is not limited to the above, and each component may be provided independently.

The data acquisition unit 10 includes a plurality of data acquisition means, such as various sensors and cameras, and acquires various sensor data and image data. The data acquisition unit may include, for example, a camera, a radar device such as Lidar (Light detection and ranging), a marine radar, or a millimeter wave radar, an ultrasonic ranging sensor, a GNSS module, or a QZSS module, a positioning device such as an AIS (Automatic Identification System), an inertial measurement unit (IMU), an inertial sensor, a wind direction sensor, a wind speed sensor, a speed sensor (ground speed sensor, water speed sensor), a direction sensor, an acceleration sensor, a gyro sensor, a magnetic compass, a satellite compass, a temperature sensor, a humidity sensor, a pressure sensor, an altitude sensor, an infrared sensor, and the like. The data acquisition unit 10 can acquire information on the current position (coordinates), attitude (direction such as the bow direction), moving speed, moving direction, and turning amount of the ship, information on the surrounding environment, and information on obstacles such as other ships and people. AIS is a system for transmitting and receiving various information between ships and between ships and land facilities, and can wirelessly exchange ship information such as the position, attitude (course), speed, and destination of each ship. The direction sensor may be a magnetic direction sensor that uses geomagnetism to calculate the bow direction, a gyrocompass, a GPS compass, etc.

The control unit 20 includes a processor (computing device) such as a CPU, and is configured to be able to execute various information processing operations based on programs stored in the memory unit 30.

The control unit 20 executes a parameter determination process to determine parameters related to the distance the ship will keep from an obstacle when avoiding the obstacle, based on obstacle information including position information, type information, and size information of the obstacle on or underwater. The control unit 20 may execute a parameter determination process to determine parameters related to the distance the ship will keep from an obstacle when avoiding the obstacle, based on obstacle information including position information, type information, and size information of the obstacle on or underwater.

Here, the parameter related to the distance the ship is away from an obstacle when avoiding the obstacle (appropriate avoidance distance: distance at which the obstacle can be avoided safely and efficiently) can be, for example, a numerical value expressed as the distance [m] from the obstacle. This parameter is associated with the position information, type information, and size information of the obstacle and stored in advance in the memory unit. Alternatively, the parameter may be calculated by applying the position information, type information, and size information of the obstacle to a calculation formula stored in advance in the memory unit. For example, if the position of the obstacle is 50 m ahead of the ship (position information), the type of obstacle is a person (type information), and the size of the obstacle is a planar area of 0.1 m^2 and a height of 1 m (size information), the numerical value of the appropriate avoidance distance corresponding to these conditions (for example, 10 m, etc.) can be determined based on the information in the memory unit.

Here, FIG. 4 shows an example of the appropriate avoidance distance for each of obstacles A and B located in the traveling direction of the ship 100 in a plan view. The appropriate avoidance distance is indicated by a circle (dash line) centered on the center point of obstacle A and obstacle B. The appropriate avoidance distance can be expressed, for example, as either the distance d1 from the outer surface of the obstacle or the distance d2 from the center of the obstacle. Alternatively, depending on the shape of the obstacle, the appropriate avoidance distance may be set so that a range indicating the shape and size of the hull (including the ship and other ships) expressed in a plan view as a polygon (a pentagon like the ship 100 and obstacle C in FIG. 4, or a triangle, a rectangle, a hexagon, etc.) generates a route that passes outside the separation distance from the obstacle. In the case of a polygon, the line (area) indicating the appropriate avoidance distance has a shape offset by a certain distance from the polygon (the intersection part has an arc shape centered on the intersection), as shown in obstacle C in FIG. 4. In this way, by setting the appropriate avoidance distance according to the shape and size of the obstacle, the same effect can be expected even when the separation distance acquired from the storage unit is applied to ships of different sizes. For example, when the size of the obstacle is small (for example, the area in plan view is 1 m^2 or less, the height from the water surface is 1 m or less), the appropriate avoidance distance may be set small (for example, 5 m, 10 m, etc.), and the larger the size of the obstacle (for example, the area in plan view is 10 m^2 or more), the larger the appropriate avoidance distance may be set (for example, 20 m, 50 m, etc.). Also, for example, when the obstacle is a person, the appropriate avoidance distance may be set large (for example, 20 m, 50 m, etc.), and when the obstacle is a floating object such as a buoy, driftwood, seaweed, etc., the appropriate avoidance distance may be set small (for example, 5 m, 10 m, etc.). The appropriate avoidance distance may be determined based on appropriate avoidance distance information stored in advance in the storage unit according to at least one of the type and size of the obstacle. In this case, the information on the type and size of the obstacle and the information on the appropriate avoidance distance are stored in advance in association with each other. Furthermore, if the obstacle is another ship, the appropriate avoidance distance may be calculated based on the speed and direction of movement of the other ship in addition to the position and size information. The speed and direction of movement of the obstacle, such as the other ship, may be obtained from the AIS, or by combining information from a distance sensor such as a camera and LiDAR, or a camera and RADAR.

The parameter related to the appropriate avoidance distance is not limited to the numerical value of the distance [m] from the obstacle itself, but may be, for example, a variable (score) associated with the distance [m] from the obstacle. Specifically, it may be a score expressed on a scale of 1 to 10, and any association can be made, such as a score of 1 indicating a distance of 5 m, a score of 2 indicating a distance of 10 m, and a score of 3 indicating a distance of 50 m.

The obstacle information can include at least any of the following: information acquired from the data acquisition unit 10, information generated by the control unit 20, information pre-stored in the memory unit 30, information input via the input unit 40, and information received from an external device, another ship, etc. via the communication unit 60.

The control unit 20 can further determine the above parameters based on the moving speed information of the obstacle. For example, the parameters can be set so that the appropriate avoidance distance increases as the moving speed of the obstacle increases. The moving speed information of the obstacle and the parameter information can be associated with each other in advance and stored in the storage unit. In addition, a calculation formula showing the relationship between the moving speed information and the parameters may be stored in the storage unit.

The control unit 20 can further determine the above parameters based on the moving direction information of the obstacle. For example, if the moving direction of the obstacle is toward the ship, the parameters can be set so that the appropriate avoidance distance is greater than if the moving direction of the obstacle is away from the ship. The moving direction information of the obstacle and the parameter information can be associated with each other in advance and stored in the storage unit. A calculation formula showing the relationship between the moving direction information and the parameters may also be stored in the storage unit. The moving direction may be a numerical value of the moving direction (angle) in a specific coordinate system based on, for example, the bow direction, or a numerical value of the moving direction (angle) in a coordinate system based on the north direction.

The control unit 20 can also determine the above parameters based on the attitude (direction) of the obstacle. For example, if the attitude (bow direction) of a ship as an obstacle is facing the own ship, the parameters can be set so that the appropriate avoidance distance is greater than when the bow direction is the same as the own ship. If the obstacle is a person using a surfboard or paddle boat, the parameters can be determined in a similar manner according to the direction (the direction of an object used by a person, such as a surfboard, canoe, yacht, or paddle board) and the direction of the person's face. Such information on the attitude of the obstacle and information on the parameters may be associated with each other in advance and stored in the memory unit.

The control unit 20 can execute a navigation route generation process that generates navigation route information for avoiding obstacles and navigating to a destination based on parameters. The navigation route generation process generates navigation route information that is an appropriate avoidance distance away from obstacles based on position information (coordinates) of the start position (e.g., the ship's current position), position information (coordinates) of the destination, 3D map information including the seabed topography, and obstacle information including parameters. In this case, the appropriate avoidance distance is determined based on not only the obstacle's position information but also the obstacle's size information and, if necessary, type information, thereby enabling safe and efficient navigation without taking longer routes than necessary.

The control unit 20 may generate a navigation route using a route search algorithm stored in the storage unit. For example, a cost (numerical value) may be set for each node, which is the smallest unit area obtained by dividing a map (nautical chart), and a search may be performed so that the cost from the start node to the destination node is minimized, thereby generating a navigation route. The cost may be an evaluation value indicating the degree of safety, and is basically set based on the water depth, distance from land, etc. For example, the shallower the water depth, the higher the cost value, and the closer to land, etc., which is an area that cannot be navigated, the higher the cost value. In setting the cost, a parameter based on the appropriate avoidance distance may be used. For example, the closer a node is to an obstacle, the higher the cost value added to each node, and the cost of a node that is more than the appropriate avoidance distance is set to be less than a predetermined value. In this way, the numerical value of the cost may be a parameter related to the appropriate avoidance distance. In this case, the predetermined value may be 0 or any value set in advance. A search is performed so that the cost from the starting point to the destination is minimized using an estimated value of the cost (distance) from each node to the destination. The cost of each node used in route search may be set using any other information in addition to obstacle information and map information (including water depth information and unnavigable area information). For example, the cost may be determined based on meteorological information (including wind direction, wind speed, etc.), ocean current information, wave size information, etc. For example, the higher the wind speed, the higher the cost, the higher the cost of a node where the wind is blowing in the opposite direction to the direction of movement, the higher the cost of a node where an ocean current is occurring in the opposite direction to the direction of movement, and the higher the cost of a node where the waves are larger. The relationship between each of these condition information (parameters) such as meteorological information (including wind direction, wind speed, etc.), ocean current information, and wave size and the numerical value of the cost added to each node is associated and stored in advance in the storage unit.

The control unit 20 may generate a navigation route using a simulation program in the memory unit. For example, when the ship moves along a planned navigation route (proposed route) at a preset speed (constant speed or preset variable speed), the control unit 20 may determine the timing at which the ship approaches each obstacle, whether the ship will come into contact with the obstacle, the distance to the obstacle when the ship approaches the obstacle, and whether the ship approaches the obstacle by more than the appropriate avoidance distance (possibility of appropriate avoidance). By repeatedly executing the simulation program while changing the planned route and the ship's speed, the ship can navigate at an appropriate distance from the obstacle and generate (search) a navigation route (including speed information) that can reach the destination in the shortest time. Machine learning based on a predetermined learning model may be used to generate such a navigation route.

The control unit 20 can execute a ship information estimation process to estimate the ship's position information and the ship's attitude information based on the data (first data) from the data acquisition unit 10. The ship's position information can be estimated using coordinate information as the positioning data acquired by the GNSS module as the current position. The ship's position information may be estimated by a self-position estimation process based on a comparison between map data (three-dimensional point cloud data) acquired from Lidar and map information stored in the memory unit. The ship's position information may be expressed as two-dimensional or three-dimensional coordinate information on a specific coordinate system, and may be expressed in latitude and longitude. The attitude of the ship is expressed as orientation (angle) information in a specific two-dimensional or three-dimensional coordinate system. The attitude information may be expressed in east-west-north orientation.

Here, the position information of the ship can be three-dimensional coordinate information (or two-dimensional coordinate information) of the position of a specific point on the ship. The specific point can be, for example, the center point of the ship, the center of gravity, the installation position of a positioning satellite antenna such as GNSS, or any other predetermined position. The control unit 20 can calculate the position information of the ship based on the position of a data acquisition means such as a positioning satellite antenna, information on the relative positional relationship with the specific point (prestored in the memory unit), and the position information acquired from the data acquisition means. Note that when the position of the data acquisition means is set as the position of the ship, such calculation processing is not necessary.

The ship's attitude information may also be based on a stationary attitude on the water surface, and changes from the reference attitude (yaw, roll, pitch directions) may be detected. In a planar view (two dimensions), the ship's attitude direction may be the direction the bow is pointing or any other preset direction. In three dimensions, the ship's attitude information may be based on a direction parallel to the horizontal plane, but a direction upward or downward at any preset angle relative to the horizontal plane may also be used as the reference attitude.

The control unit 20 can execute a ship information correction process to correct the ship's position information and the ship's attitude information based on the data (second data) from the data acquisition unit 10. The control unit 20 can correct the ship's position information and the ship's attitude information based on speed data acquired from, for example, Lidar or a camera.

The control unit 20 can execute a speed estimation process to estimate the ship's speed based on data from the data acquisition unit 10. For example, the control unit 20 can calculate the distance traveled in a given time from position information from a given time ago (0.01 seconds, 0.1 seconds, 1 second, etc.) and current position information, and estimate the speed from information on the distance traveled in the given time. The speed data may be information from a speed sensor serving as the data acquisition unit 10.

The control unit 20 can execute an obstacle information estimation process that estimates the position and size of an obstacle based on the ship's position information and ship's attitude information corrected in the ship's information correction process, and the data (third data) from the data acquisition unit 10. The control unit 20 may estimate the type information of an obstacle based on the ship's position information and ship's attitude information corrected in the ship's information correction process, and the data (third data) from the data acquisition unit 10. The control unit 20 may estimate the attitude (direction) of the obstacle in the obstacle information estimation process.

The control unit 20 can estimate the type of obstacle by, for example, selecting one of the candidate types of obstacles stored in advance in the memory unit 30. Information on the type of obstacle can include type information such as ships, specific ship classifications (fishing boats, water taxis, multipurpose boats, recreational fishing boats, passenger boats, transportation boats, work boats, fireboats/patrol boats, pleasure yachts, pleasure motorboats, special work boats, etc.), marine structures, people (including swimming, surfing, yachts, paddle boards, etc.), rocks, shores, driftwood, seaweed, marine life, etc. Information on the type of obstacle is stored in association with size information (area, height).

The control unit 20 can estimate the position, type, size, and posture of an obstacle by, for example, image analysis of image data acquired by a camera. In the obstacle information estimation process, the control unit 20 may estimate at least one of the position, type, and size of the obstacle using a machine learning technique using a learning model that has been trained in advance. When the control unit 20 estimates the position of an obstacle based on image data acquired by a camera, after estimating the three-dimensional position (coordinates) of the obstacle in the camera coordinate system, the camera coordinate system can be converted to a world coordinate system by a coordinate conversion process based on the camera position (coordinates) and shooting direction information, and the position (coordinates) of the obstacle on the world coordinate system can be calculated. The control unit 20 can estimate the bow direction of a ship as an obstacle, the direction of a person's face (front direction), and the direction (direction of travel) of an object used, such as a surfboard, by image analysis of image data from the camera or analysis of three-dimensional data (point cloud data, model data, etc.).

The memory unit 30 may store candidate types of obstacles in advance in association with the size (information on area and height in a plan view) and size range (upper and lower limits) corresponding to each type. For example, the range of a ship's area can be 3 m^2 or more and 9200 m^2 or less, and the range of a ship's height (height above the water surface) can be 0.5 m or more and 100 m or less. The range of a person's area can be 0.2 m^2 or more and 5 m^2 or less, and the range of a person's height (height above the water surface) can be 0.1 m or more and 3 m or less.

The memory unit 30 stores information on appropriate avoidance distances associated with obstacle information such as the position, type, and size of the obstacle. The memory unit 30 may also store information on a calculation formula for calculating the appropriate avoidance distance. For example, the appropriate avoidance distance may be calculated according to the size (planar area) of the ship as an obstacle. The parameters used in the calculation may include information on the type of obstacle, movement speed, movement direction, etc. The memory unit 30 may also include information on a program that simulates movement along the ship's navigation route (planned movement route) and whether or not there will be contact with an obstacle.

The memory unit 30 stores information about each data acquisition means. Information about the data acquisition means can be, for example, the position of the data acquisition means on the ship (e.g., the position relative to a specific point such as the center of the ship or the center of gravity), the relative attitude (e.g., the direction relative to the bow direction of the ship), etc., but is not limited to this.

The control unit 20 may estimate the moving speed and moving direction of the obstacle based on the ship's position information and ship's attitude information corrected by the ship's information correction process, the ship's moving speed information, moving direction information, and data (third data) from the data acquisition unit 10. The control unit 20 can acquire moving speed information and moving direction information of obstacles such as other ships relative to the ship based on data from Lidar. The control unit 20 can also acquire moving speed information and moving direction information of obstacles using data from a radar device.

In the obstacle information estimation process, the control unit 20 can analyze the image data acquired by the camera to estimate information such as the type, size, position, shape, and color of the obstacle. The control unit 20 may also estimate the information of the obstacle based on any information of the obstacle estimated from the image data and the obstacle-related information previously stored in the storage unit. For example, the control unit 20 may estimate the size of the obstacle from the image data, and refer to the storage unit to determine (estimate) a candidate obstacle associated with a size range that matches the size as the type of the obstacle. The control unit 20 may also analyze the image data acquired by the camera to estimate the type of the obstacle, and refer to the storage unit to determine (estimate) a size candidate associated with the type as the size of the obstacle. The type, size, and position of the obstacle may be estimated based on any one or a combination of detection data acquired from a radar device and detection data from Lidar, without being limited to image data acquired by the camera. For example, 3D data of an obstacle can be generated from distance data to multiple points on the obstacle using Lidar, and the 3D shape, type, size, position, etc. of the obstacle can be estimated based on the 3D data. In this way, the control unit 20 can select from among the obstacle candidates by comparing the information of the obstacle candidates stored in the memory unit 30 with the data on the obstacle acquired from the data acquisition unit 10, and estimate the type and size of the detected obstacle. Note that the control unit 20 can also estimate the position, type, and size of the obstacle based only on the data from the data acquisition unit 10.

The control unit 20 can store various information such as generated information in the memory unit 30, update the information in the memory unit 30, output from the output unit 50, and transmit to external devices, other ships, etc. via the communication unit 60. The control unit 20 can also accept information acquired by the data acquisition unit 10, user operation information input via the input unit 40, and information received by the communication unit 60, and perform information processing based on the information.

The memory unit 30 can store various information stored in advance, information acquired from the data acquisition unit 10, user operation information input via the input unit 40, information received from external devices or other ships via the communication unit 60, information generated by the control unit 20, etc., and is composed of a non-volatile memory, a hard disk, etc. The memory unit 30 can store information on various parameters indicating the shape of the ship (3D model data, etc.), characteristics (characteristics related to output, speed, and turning amount), information on obstacles, 2D or 3D map information, navigation history information, navigation route information, tidal information, etc.

The input unit 40 accepts input information based on operations from the user. The input unit 40 is composed of mechanical buttons, switches, operating levers, touch panels, etc. The input unit 40 may also be equipped with a microphone that allows voice input.

The output unit 50 outputs various information in the form of images (video), audio, etc. The output unit 50 may include, for example, a display unit such as an LCD monitor or touch panel that displays images, an audio output unit such as a speaker that displays audio, a vibration generating unit (vibration device) that generates vibrations, etc.

The communication unit 60 is connected to a network such as the Internet or wireless communication, and can transmit data to external devices (servers, control devices, etc.) or other ships, and receive data from external devices.

The vessel operation control unit 70 may include an auto-rudder that controls steering, an auto-throttle that controls the power unit (engine, motor, etc.) for rotating the propeller or thruster, etc. The vessel operation control unit 70 can control the operation of the vessel 70 (movements such as forward movement, backward movement, left and right movement, turning, etc.) based on instruction information from at least one of the control unit 20, the memory unit 30, the input unit 40, and the communication unit 60, and can navigate the vessel along a predetermined route.

Figure 2 shows an example of the present system installed on a ship 100. In the example of Figure 2, the system includes a lidar, a camera, a GNSS antenna, an AIS antenna, and a wind speed and direction meter as the data acquisition unit 10, a control unit as the control unit 20, a monitor as the output unit 50, and an auto-rudder and auto-throttle as the ship operation control unit 70. In the example of Figure 2, the lidar and camera are installed on the front (bow side) and rear (stern side) of the ship 100, respectively, allowing data around the ship 100, particularly the front and rear sides, to be detected with high accuracy. The configuration of the present system installed on a ship is not limited to the illustrated example, and other sensors, etc. may be included.

Figure 3 shows an example of a process flow for estimating obstacle information in the system of this embodiment.

The control unit 20 executes a process (S101) of acquiring obstacle information including position information and size information (and type information, if necessary) of an obstacle on or underwater, and a process (S102) of determining parameters relating to the distance the ship will move away from the obstacle when avoiding the obstacle, based on the obstacle information.

The control unit 20 may execute a process for generating a navigation route or a process for updating a pre-stored navigation route based on the parameters determined in S102 (S103).

The control unit 20 may execute an output information generation process (S104) that generates output information. The output information may be image data to be displayed on the display unit, audio data to be output, etc. The image data may include text or image information on the navigation route from the current position to the destination, the position, type, and size of an obstacle, and an indication of the appropriate avoidance distance using numerical values, lines, color differences, etc. Specifically, the monitor may display a map showing the line indicating the navigation route, the position of an obstacle, and the appropriate avoidance distance, or the type and size of an obstacle may be displayed in text or image, or a voice may be output from a speaker to warn the driver. In addition, when the ship is steered manually, the output information may be maneuvering information (data on steering, propulsion output, etc.). The output information may also be data for automatically navigating according to the navigation route information by applying it to an autopilot system.

The control unit 20 can also execute a ship information estimation process that estimates ship position information and ship attitude information based on first data acquired from at least one data acquisition means, a ship information correction process that corrects ship position information and ship attitude information based on second data acquired from at least one data acquisition means, an obstacle information estimation process that estimates the position, type, size, etc. of an obstacle based on the ship position information and ship attitude information corrected in the ship information correction process and third data acquired from at least one data acquisition means, and an output information generation process that generates output information based on the information on the position, type, size, etc. of the obstacle estimated in the obstacle information estimation process. The first data, second data, and third data may each include data acquired from a common data acquisition means. Furthermore, the first data, second data, and third data may each include data acquired from one data acquisition means or from multiple data acquisition means.

In the ship information estimation process, for example, the ship's three-dimensional position is estimated based on coordinate information from a GNSS antenna as a data acquisition means, and the ship's three-dimensional attitude information is estimated based on orientation data acquired from an inertial measurement unit. The ship information estimation process may be performed based on data acquired from only one data acquisition means, but from the perspective of improving estimation accuracy, it is preferable to perform the estimation process based on data acquired from multiple data acquisition means.

In the ship information correction process, the ship position information and ship attitude information estimated in are corrected based on speed information acquired from at least one of Lidar and a camera. The control unit 20 estimates the current position and attitude (a predetermined time after the time) using, for example, the position (coordinates), movement direction information, and movement speed information at a time point a predetermined time ago (0.01 seconds, 0.1 seconds, 1 second, etc.). The estimated position and attitude information is then compared with the ship position information and ship attitude information estimated in the ship information estimation process. If they do not match, the ship position information and ship attitude information can be corrected by adopting the numerical value between the two as accurate current position and current attitude information. Note that the ship information correction process may be performed in a different manner, and for example, the ship position information and ship attitude information may be corrected using position information and attitude information calculated by the ship position and attitude estimation process based on image data from a camera, or a combination of these. In addition, from the viewpoint of increasing the accuracy of the correction process, it is preferable to perform the correction process based on data acquired from multiple data acquisition means.

Next, in the obstacle information estimation process, for example, the position and attitude of the camera are estimated (the position and attitude of the camera in the ship's coordinate system) based on the ship's position information and ship's attitude information corrected in the ship's information correction process, and the position of the obstacle in the world coordinate system is estimated by coordinate conversion using the relative position data of the obstacle in the camera coordinate system. In addition, the camera image data is analyzed to estimate the type and size of the obstacle. The position, type and size of the obstacle may be estimated using the detection data acquired from the radar device alone or in combination.

As described above, in the system of this embodiment, the control unit 20 executes a process to determine parameters related to the distance the ship should move away from an obstacle when avoiding the obstacle, based on obstacle information including the obstacle's position information on or underwater and size information. In this way, by determining parameters related to the appropriate avoidance distance based on obstacle information including not only the obstacle's position information but also the obstacle's size information, safe and efficient navigation is possible without making unnecessarily long detours.

The parameter determination process may also determine parameters based on the obstacle's movement speed information and movement direction information. In this case, the appropriate avoidance distance can be determined in more detail.

In the system of this embodiment, the information on the size of the obstacle may include information on the height from the water surface to the top of the obstacle. With this configuration, for example, it is possible to determine an appropriate avoidance distance according to the height of the obstacle. It is also possible to improve the accuracy of estimating the type of obstacle.

In the system of this embodiment, the type information of an obstacle may include information indicating that the obstacle is another ship or a person. This makes it possible to determine an appropriate avoidance distance according to type information, such as another ship or person that requires special caution. It is also possible to notify the user that the obstacle is another ship or person, urging them to be careful.

In the system of this embodiment, the control unit may execute a navigation route generation process that generates navigation route information for avoiding obstacles and traveling to a destination based on parameters. This configuration allows navigation based on a navigation route that takes into account an appropriate avoidance distance, enabling safer and more efficient navigation.

In the system of this embodiment, the own ship information estimated by the own ship information correction process is corrected by the own ship information correction process before the obstacle information estimation process is performed, thereby improving the accuracy of obstacle estimation, and as a result, it becomes easier to avoid contact with obstacles, thereby improving the safety of the ship. Furthermore, according to this embodiment, the above process is repeatedly performed at predetermined intervals such as 0.01 seconds, 0.1 seconds, etc., so that obstacle information can be estimated with high accuracy essentially in real time.

In the system of this embodiment, the obstacle information estimation process may include a data correction process that corrects the third data based on the ship's position information and ship's attitude information corrected in the ship's information correction process. That is, for example, the detection data of the radar device, image data from the camera, data from Lidar, etc. as the third data may be corrected based on the corrected ship's position information and ship's attitude information, thereby further improving the accuracy of obstacle estimation. The correction process may include a so-called clustering process, etc.

In the system of this embodiment, the output information may include information on the distance the ship should be away from an obstacle, based on information on the position, type, and size of the obstacle. With this configuration, the ship can navigate while safely and efficiently avoiding obstacles.

In the system of this embodiment, the information on the size of the obstacle may include information on the height from the water surface to the top of the obstacle. With this configuration, for example, it is possible to determine the distance to avoid depending on the height of the obstacle, and to improve the accuracy of estimating the type of obstacle.

In the system of this embodiment, the obstacle information estimation process may include a process for determining whether the type of obstacle is another ship or a person. With this configuration, for example, it is possible to inform the user that there is another ship or person that requires special attention, urging them to be careful or allowing them to avoid the obstacle safely.

In the system of this embodiment, the first data may include positioning data received from a positioning satellite and data acquired from an inertial measurement unit. With this configuration, the ship information estimation process can be performed quickly and efficiently.

In the system of this embodiment, the second data may include speed data acquired from at least one of the Lidar and the camera. With this configuration, even if positioning data received from a positioning satellite cannot be acquired, the ship information correction process can be performed with high accuracy.

In the system of this embodiment, the third data may include data acquired from Lidar. With this configuration, for example, it is possible to estimate information such as the position, type, size, and shape of an obstacle with high accuracy using the three-dimensional point cloud data acquired from Lidar.

In the system of this embodiment, the third data may include image data acquired from a camera. With this configuration, for example, information such as the relative position, type, size, and shape of an obstacle can be estimated with high accuracy by image analysis of the image data.

In the system of this embodiment, in environments where signals from positioning satellites cannot be received, such as under bridges, or where signal interference occurs, i.e., where GNSS and the like do not function, self-location estimation processing can be performed based on data from at least one of the Lidar and the camera.

When the control unit 20 determines based on the signal from the data acquisition unit 10 that a signal from a positioning satellite has not been received, the control unit 20 may notify the ship by image or sound from the output unit 50 that a signal from a positioning satellite has not been received, or may automatically switch the method of ship's position estimation processing. The switching process of the ship's position estimation processing method may be, for example, switching from self-position estimation processing based on data such as GNSS to self-position estimation processing based on data from Lidar and a camera. Then, when it is determined that a signal from a positioning satellite is being received again, the method of ship's position estimation processing may be returned to the original method based on data from the positioning satellite.

In the system of this embodiment, the control unit 20 may determine the type of data to be used for each process based on priority information previously stored in the storage unit 30. For example, in the obstacle information estimation process, the priority can be set in advance such that the combination of camera image data and Lidar data is given the highest priority, followed by camera image data only, Lidar data only, and radar device data only. The priority can be set, for example, taking into consideration high estimation accuracy and low information processing load.

In the system of this embodiment, the control unit 20 may repeatedly determine whether each data acquisition means is functioning properly based on the presence or absence of a signal from the data acquisition means, and may determine the type of data to use for each process based on the determination result. For example, if it is determined that GNSS data cannot be acquired, the control unit 20 may execute a ship information estimation process using data acquired by Lidar to estimate the ship's position information and attitude information.

Information on the determination result as to whether each data acquisition means is functioning properly may be output from the output unit 50 as an image, sound, signal, etc.

The system of this embodiment can be installed when a new ship is manufactured, but it can also be implemented on an existing ship by installing various sensors, cameras, drive units, power sources, information processing units, and other components.

The system of this embodiment can be installed on a newly constructed ship, but can also be installed (retrofitted) on an existing ship. That is, the data acquisition unit 10, control unit 20, memory unit 30, input unit 40, output unit 50, communication unit 60, and ship operation control unit 70 may all or partly be formed as a system unit, and the system unit may be installed on an existing ship at a later time.

In the system of this embodiment, for example, meteorological information and ocean current information may be acquired via the data acquisition unit 10 or the communication unit 60. In that case, for example, the accuracy of estimating the ship or obstacles can be further improved by using the meteorological information and ocean current information in at least one of the above-mentioned ship information estimation process, ship information correction process, and obstacle information estimation process.

In the system of this embodiment, output information regarding obstacles generated by the control unit, and information such as weather information and ocean current information acquired via the data acquisition unit 10 or the communication unit 60 may be transmitted to other ships. In this case, for example, other ships can be notified of emergencies such as natural disasters and accidents. Information may also be shared with rescue teams and the coast guard to respond to distress accidents and emergencies at sea.

In the system of this embodiment, the current position and attitude of the ship may be predicted based on the ship's speed data at a specific point in time in the past, as described above, or the accuracy of the ship's position information and attitude estimation may be improved by predicting the ship's movements using machine learning or the like from information such as parameters indicating the ship's characteristics and navigation history.

In the system of this embodiment, the control unit may delete unnecessary obstacle information when the obstacle information acquired from the multiple data acquisition means is duplicated. In other words, the control unit may perform a duplication determination process to determine whether the multiple obstacle information estimated (detected) by the multiple data acquisition means is the same obstacle, and when it is determined that there are two or more obstacle information for one obstacle, it may delete the obstacle information other than one and leave only one obstacle information in the storage unit. The duplication determination process can determine that the position information (coordinates) of another ship acquired by the first data acquisition means (AIS, etc.) and the position information of the other ship (obstacle) estimated from the data of the second data acquisition means (camera, Lidar, radar device) are the same, or that the value (distance) indicating the difference between these two points (coordinates) is within a predetermined threshold range. On the other hand, when it is determined that the value (distance) indicating the difference between the two points (coordinates) is outside the predetermined threshold range (exceeds the threshold), it can determine that they are two different obstacles and maintain the two obstacle information without deleting it. The first data acquisition means and the second data acquisition means may be only one data acquisition means or a combination of multiple data acquisition means. Also, the first data acquisition means and the second data acquisition means are expected to be different data acquisition means, but some of them may be the same data acquisition means.

In the system of this embodiment, instead of or in combination with the method of avoiding an obstacle by setting an appropriate avoidance distance, a method of avoiding a collision by slowing down (including stopping) the ship and waiting for the obstacle to pass, or a method of avoiding a collision by first crossing the direction of the obstacle's movement based on the relative position and relative speed to the obstacle, can be included. In this case, costs are set for all selectable options, and the control unit can calculate the costs for all avoidance methods based on the above-mentioned algorithm and select the route with the lowest cost. In other words, it is not limited to the case where the ship's speed is constant, but it is possible to generate a movement route that reaches the destination at the minimum cost while securing the appropriate avoidance distance by decelerating and accelerating the speed. In addition, the information on the acceleration (combined deceleration and acceleration per unit of time) of deceleration and acceleration, and the parameters of the lower limit and upper limit of the speed can be set according to the type of hull, operating capacity, weight, size, etc., and can be stored in the memory unit in advance.

Furthermore, in the system of this embodiment, a cost-related parameter may be separately set to preferentially select an avoidance route that complies with laws and regulations regarding ship traffic. Specifically, the cost may be set to prioritize the right side as seen from the ship's own ship. For example, when trying to avoid an obstacle (another ship) approaching at high speed from the ship's starboard side, the cost of the ship crossing first can be set to be higher by a predetermined value than the avoidance method of the ship slowing down and waiting for the obstacle to pass. In this case, the cost of slowing down or stopping the ship to wait for the obstacle to pass and the cost of the ship going around the rear of the obstacle to avoid the obstacle may be calculated and compared. On the other hand, when trying to avoid an obstacle approaching at high speed from the ship's port side, the cost of the ship crossing first can be set to be lower than the avoidance method of slowing down and waiting for the obstacle to pass.

Although the preferred embodiment of the present disclosure has been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field of the present disclosure can conceive of various modified or revised examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

The device described in this specification may be realized as a single device, or may be realized by multiple devices (e.g., cloud servers) some or all of which are connected via a network. For example, the control unit and the storage unit may be realized by different servers connected to each other via a network.

The series of processes performed by the device described in this specification may be realized using software, hardware, or a combination of software and hardware. A computer program for realizing each function of the control unit according to this embodiment may be created and implemented in a PC or the like. A computer-readable recording medium on which such a computer program is stored may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. The computer program may also be distributed, for example, via a network, without using a recording medium.

In addition, the processes described in this specification using flowchart diagrams do not necessarily have to be performed in the order shown. Some processing steps may be performed in parallel. Also, additional processing steps may be employed, and some processing steps may be omitted.

Furthermore, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology disclosed herein may achieve other effects that are apparent to a person skilled in the art from the description in this specification, in addition to or in place of the above effects.

Note that the following configurations also fall within the technical scope of the present disclosure.
(Item 1)
A control unit is provided that executes information processing for avoiding obstacles and navigating to a destination,
The control unit of the information processing system executes a parameter determination process to determine parameters related to the distance the ship will move away from an obstacle when avoiding the obstacle, based on obstacle information including position information and size information of the obstacle on or underwater.
(Item 2)
2. The information processing system according to item 1, wherein the parameter determination process further determines the parameters based on type information of the obstacle.
(Item 3)
3. The information processing system according to item 1 or 2, wherein the parameter determination process further determines the parameter based on at least one of moving speed information and moving direction information of the obstacle.
(Item 4)
3. The information processing system according to item 1 or 2, wherein the size information of the obstacle includes information on the height from the water surface to the top of the obstacle.
(Item 5)
3. The information processing system according to item 1 or 2, wherein the obstacle type information includes information indicating that the obstacle is another ship or a person.
(Item 6)
3. The information processing system according to item 1 or 2, wherein the control unit executes a navigation route generation process to generate navigation route information for avoiding the obstacle and traveling to the destination based on the parameters.
(Item 7)
The control unit is
3. The information processing system according to item 1 or 2, further comprising:
(Item 8)
8. The information processing system according to item 7, wherein the output information includes image data to be displayed on a display unit or audio data to be output as audio.

REFERENCE SIGNS LIST 1 Information processing system 10 Data acquisition unit 20 Control unit 30 Memory unit 40 Input unit 50 Output unit 60 Communication unit 70 Ship operation control unit

Claims (8)

A control unit is provided that executes information processing for avoiding obstacles and navigating to a destination,
The control unit of the information processing system executes a parameter determination process to determine parameters related to the distance the ship will move away from an obstacle when avoiding the obstacle, based on obstacle information including position information and size information of the obstacle on or underwater. The information processing system according to claim 1, wherein the parameter determination process further determines the parameters based on the type information of the obstacle. The information processing system according to claim 1 or 2, wherein the parameter determination process further determines the parameters based on at least one of the movement speed information and the movement direction information of the obstacle. The information processing system according to claim 1 or 2, wherein the size information of the obstacle includes information on the height from the water surface to the top of the obstacle. The information processing system according to claim 1 or 2, wherein the obstacle type information includes information indicating that the obstacle is another ship or a person. The information processing system according to claim 1 or 2, wherein the control unit executes a navigation route generation process that generates navigation route information for avoiding the obstacles and traveling to the destination based on the parameters. The control unit is
The information processing system according to claim 1 , further comprising: an output information generating process for generating output information based on the parameters. The information processing system according to claim 7, wherein the output information includes image data to be displayed on a display unit or audio data to be output as audio.

Images