CN105785990B - Ship mooring system and obstacle recognition method based on panoramic looking-around - Google Patents

Abstract

A ship parking system based on panoramic surround view includes a main control module, a battery connected with the main control module and a mooring control module, a display module connected with the main control module, a ship speed sensor and a gyroscope sensor, and four connected with the main control module. a camera, the main control module includes a main controller, a power supply circuit connected to the main controller, a memory, four image correction and signal conditioning modules connected to the four cameras, and the ship speed sensor and the gyro respectively. The main control module also has a cluster analysis module. The cluster analysis module adopts an adaptive threshold level according to the CHT characteristics of the panoramic image obstacles stored in the memory and provided to the cluster analysis module. The linear support vector machine analyzes the safety and convenience of the current ready-to-berth environment, so as to judge whether it is necessary to call the berthing control module for berthing.

Description

Ship Parking System and Obstacle Recognition Method Based on Panoramic Surround View

technical field

The invention relates to the field of automatic control, in particular to a ship parking system based on panoramic surround view.

Background technique

With the development of economy and society, the shipbuilding industry is also used in many aspects of social industries, such as tourism, transportation, and even military. Even experienced captains can't quite understand the specifics of the waters. However, visual information is the main technical means to realize environmental perception and monitoring and system intelligence. Different from the traditional visual environment perception system with a smaller field of view, panoramic vision can realize a large field of view monitoring within a range of 360 degrees in the horizontal direction and 240 degrees in the vertical direction, and its wide viewing angle provides convenience for monitoring the surrounding environment.

Therefore, it is necessary to provide an automatic mooring system for unmanned boats based on three-dimensional panoramic view.

SUMMARY OF THE INVENTION

In order to solve the above problems, one aspect of the present invention provides a vessel parking system based on panoramic surround view, characterized in that: the vessel parking system based on panoramic surround view includes a main control module, a battery connected to the main control module, and a mooring control module. , a display module, a ship speed sensor, a gyroscope sensor and four cameras connected with the main control module, the main control module includes a main controller, a power circuit and a memory respectively connected with the main controller , four image correction and signal conditioning modules connected with four cameras and two signal conditioning circuits respectively connected with the ship speed sensor and the gyroscope sensor, the main control module also has a cluster analysis module, the memory It is connected with the cluster analysis module, and the cluster analysis module is connected with the mooring control module.

Further, the four cameras are respectively installed on the front, rear, left and right outer surfaces of the hull, and the cameras are respectively equipped with waterproof devices and have rotatable wide-angle lenses.

Further, the present application provides an obstacle identification method: the working method of the vessel parking system based on panoramic surround view includes the following steps: first, four cameras respectively collect real-time real-time images of the hull and the front, rear, left and right of the ship image, and transmit the real-time image to the main controller; the ship speed sensor and the gyroscope sensor collect the speed and running direction of the ship respectively; the four image correction and signal conditioning modules collect the real-time images collected by the four cameras The picture is corrected and signal processed, and then the four corrected images are spliced to obtain a panoramic image around the hull; the display module displays the panoramic image around the hull; the two signal conditioning circuits adjust the speed of the ship. The information collected by the sensor and the gyroscope sensor is corrected and signal processed to obtain the speed and specific orientation of the ship; the collected information of the ship is stored in the memory and the data of the cluster analysis is provided to the cluster analysis module. Panoramic images; the cluster analysis module uses adaptive threshold cascade linear support vector machines to identify different types of obstacles according to the characteristics of obstacles in the panoramic images stored in the memory and provides the cluster analysis module to the cluster analysis module, and provides a safety assessment of the current parking environment. The clustering algorithm with favorable characteristics of convenience and convenience is used to analyze, so as to judge whether it is necessary to call the berthing control module for berthing.

Further, the working method of the vessel mooring system based on panoramic surround view provided by the present application also includes a clustering module working method, and the clustering module working method is as follows, using the improved CHT feature to extract the obstacle feature T in the panoramic surround view image,

T=G(p _c )t(s(p _c -p ₀ ),s(p _c -p ₁ ),...,s(p _c -p ₇ ))

in,

In the formula, the t() function represents the joint distribution function of the sign difference value; s(p _c -p _i ) represents the sign difference value between the current pixel point p _c and the i-th neighborhood point p _i , G(p _c ) is a standard normal Gaussian distribution function.

The adaptive threshold cascade linear support vector machine is used to identify the CHT features of different types of obstacles, and the CHT features are extracted from all samples to form a training set, and the linear SVM is used to classify them to obtain a hyperplane classifier;

Each sample has an output value of the obstacle type m about the hyperplane classifier, which is calculated as follows:

Among them, w ^* , b ^* are the parameters of the hyperplane classifier respectively; x _n is the CHT feature vector of the nth sample, and each output value is added with a one-dimensional 0-1 uniformly distributed random number, which is mapped to a two-dimensional space, After the training of the first-layer classifier is completed, the samples that are easy to be classified are eliminated according to the adaptive threshold method, and the hard samples are reserved as the training samples of the second-layer classifier, and so on, until the last layer of classification, and finally the obstacle type is obtained at the same time. Determine whether to anchor or not

Compared with the prior art, the present application has the following beneficial effects: the vessel mooring system based on the panoramic view of the present application has high reliability, high measurement accuracy, convenient installation and debugging, low power consumption, and easy popularization and promotion.

Description of drawings

1 is a schematic diagram of a panoramic view vessel mooring system in an embodiment of the present invention;

Fig. 2 is an adaptive threshold cascade classifier training process diagram in an embodiment of the present invention;

3 is a sample diagram of decomposition of 8 neighborhood CHT feature detection windows with a radius of 1 in the embodiment of the present invention;

Fig. 4 is the spatial relationship diagram in the CHT feature extraction process;

Fig. 5 is the pixel amplitude value in the CHT feature extraction process;

Fig. 6 is the symbol difference value in the CHT feature extraction process;

Figure 7 is the weight template in the CHT feature extraction process.

Detailed ways

Figure 1 shows a vessel parking system based on panoramic surround view. The vessel parking system based on panoramic surround view includes a main control module 1, a battery 2 connected to the main control module 1, a mooring control module 3, and a main control module 1. The connected display module 4, the ship speed sensor 5 and the gyroscope sensor 6 and the four cameras 7 connected with the main control module 1, the main control module 1 comprises the main controller 11, the power supply connected with the main controller 1 respectively Circuit 12, memory 13, four image correction and signal conditioning modules 14 connected to the four cameras 7, and two signal conditioning circuits 15 respectively connected to the ship speed sensor 5 and the gyro sensor 6, the main control module 1 also has a cluster analysis module 16, the memory 13 is connected with the cluster analysis module 16, and the cluster analysis module 16 is connected with the mooring control module 3.

The battery 2 is used to provide power to the main control module 1 , and the power supply circuit 12 described in the main control module 1 is used to provide power to the main controller 11 .

The mooring control module 3 includes a manual control module 31 and an automatic control module 32 . The ship speed sensor 5 and the gyroscope sensor 6 are respectively fixed on the surface of the ship's hull, and are used to collect the running speed and running direction of the ship. The four cameras 7 are respectively a first camera 7A, a second camera 7B, a third camera 7C and a fourth camera 7D, and the first camera 7A, the second camera 7B, the third camera 7C and the fourth camera 7D are respectively It is installed on the front, rear, left and right outer surfaces of the hull, and the first camera 7A, the second camera 7B, the third camera 7C and the fourth camera 7D are all rotatable wide-angle lenses with waterproof devices. The first camera 7A, the second camera 7B, the third camera 7C and the fourth camera 7D are respectively used to collect real-time images of the hull and the front, rear, left and right sides of the ship.

The memory 13 is used to store all the information of the ship passing through the ship collected by the ship speed sensor 5 and the gyro sensor 6 .

The four image correction and signal conditioning modules 14 are respectively an image correction and signal conditioning module 14A, an image correction and signal conditioning module 14B, an image correction and signal conditioning module 14C, and an image correction and signal conditioning module 14D. The module 14A, the image correction and signal conditioning module 14B, the image correction and signal conditioning module 14C and the image correction and signal conditioning module 14D are respectively connected to the first camera 7A, the second camera 7B, the third camera 7C and the fourth camera 7D, respectively, Correction and signal processing are performed on the images collected by the first camera 7A, the second camera 7B, the third camera 7C and the fourth camera 7D, and then the four corrected images are stitched together, so as to obtain the surrounding area of the hull. panoramic image. The output end of the main controller is connected with the input end of the display module for displaying panoramic images around the hull.

The two signal conditioning circuits 15 are respectively a first signal conditioning circuit 15A and a second signal conditioning circuit 15B. The first signal conditioning circuit 15A and the second signal conditioning circuit 15B are respectively used to connect the ship speed sensor 5 and the gyroscope The information collected by the sensor 6 is corrected and signal processed to obtain the running speed and specific orientation of the ship.

The cluster analysis module 16 uses adaptive threshold cascade linear support vector machine to identify different types of obstacles according to the CHT features of the panoramic image obstacles stored in the memory 13 and provided to the cluster analysis module 16, and is ready to park at the moment. The safety and convenience of the environment are analyzed to determine whether it is necessary to call the berthing control module 3 for berthing. If the result obtained by the cluster analysis module 16 is that the current environment is suitable for berthing, the berthing control module 3 is called, and the berthing control module 16 correspondingly calls the manual control module 31 or the automatic control module 32 for manual control The ship is parked under the state of the state or the ship is parked under the automatic control state.

The working method of the vessel mooring system based on panoramic surround view in this application includes the following steps:

First, the four cameras 7 collect the real-time images of the hull and the front, rear, left and right of the ship respectively, and transmit the real-time images to the main controller;

The ship speed sensor 5 and the gyroscope sensor 6 respectively collect the traveling speed and the running direction of the ship;

The four image correction and signal conditioning modules 14 perform correction and signal processing on the real-time images collected by the four cameras 7, and then perform stitching processing on the four corrected images to obtain a panoramic image around the hull;

The display module 4 displays panoramic images around the hull;

The two signal conditioning circuits 15 perform correction and signal processing on the information collected by the ship speed sensor 5 and the gyroscope sensor 6 to obtain the ship's running speed and specific orientation;

The storage 13 stores the collected vessel information and provides the cluster analysis data to the cluster analysis module 16;

The cluster analysis module 16 analyzes the safety and convenience of the currently prepared parking environment according to the panoramic image obstacle CHT features stored in the memory 13 and provided to the cluster analysis module 16, thereby judging whether it is necessary to call the mooring vessel. Control module 3, to moor the ship.

The improved CHT feature is used to extract the obstacle feature T in the panoramic surround view image.

T=G(p _c )t(s(p _c -p ₀ ),s(p _c -p ₁ ),...,s(p _c -p ₇ ))

in:

In the formula, the t() function represents the joint distribution function of the sign difference value; s(p _c -p _i ) represents the sign difference value between the current pixel point p _c and the i-th neighborhood point p _i . G(p _c ) is a standard normal Gaussian distribution function.

As shown in Fig. 2, in order to improve the obstacle detection performance, an adaptive threshold cascaded linear support vector machine is adopted to identify the CHT features of different types of obstacles. The CHT features are extracted from all samples to form a training set, and the linear SVM is used to classify them to obtain a hyperplane classifier. Each sample has an output value for the classifier, the obstacle type m, which is calculated as follows:

Among them, w ^* and b ^* are the parameters of the hyperplane classifier respectively; xn is the CHT feature vector of the _nth sample. In order to make the output value have two-dimensional visualization characteristics, a one-dimensional 0-1 uniformly distributed random number is added to each output value and mapped to a two-dimensional space. After the training of the first-layer classifier is completed, the samples that are easy to be classified are eliminated according to the adaptive threshold method, and the samples that are difficult to be separated are retained as the training samples of the second-layer classifier. And so on, until the last layer of classification, and finally get the obstacle type and determine whether to anchor or not.

If the analysis result of the cluster analysis module 16 is that the current environment is suitable for berthing, the berthing control module 3 is called, and the berthing control module 3 correspondingly calls the manual control module 31 or the automatic control module 32 to perform the manual control state If the analysis result of the cluster analysis module 16 is that the current environment is not suitable for berthing, the berthing control module 3 will not be called, and the berthing will not be performed.

The above-mentioned CHT feature is a texture feature with strong expressive ability. The feature extraction process of 8-neighborhood CHT with radius 1 is shown in Fig. 3. And as shown in Figure 4, it is the spatial relationship diagram between the currently calculated pixel point p _c and the neighboring points, which are sequentially numbered clockwise: p ₀ ~ p ₇ ; Figure 5 shows the grayscale amplitude of the current pixel point . According to the amplitude relationship, the sign difference value between pixels can be obtained, as shown in equations (1) and (2):

T=t(s(p _c -p ₀ ),s(p _c -p ₁ ),...,s(p _c -p ₇ )) (1)

in:

In the formula, the t() function represents the joint distribution function of the sign difference value; s(p _c -p _i ) represents the sign difference value between the current pixel point p _c and the i-th neighborhood point p _i . As shown in Fig. 6, it is the corresponding symbol difference value of all neighboring points. Figure 7 shows the weight template for CHT feature calculation. At this time, the corresponding CHT value of the current pixel p _c can be calculated by formula (3):

In the formula, N is the number of neighboring pixels, R is the radius of CHT calculation, and the CHT value of the current pixel is 47. It is not difficult to see that the CHT value is only related to the relative relationship between the pixel points, and has nothing to do with the specific pixel amplitude.

Claims (2)

1. an obstacle identification method based on a panoramic view-based vessel parking system comprising a main control module, a battery connected with the main control module and a mooring control module, and a display connected with the main control module module, a ship speed sensor, a gyroscope sensor, and four cameras connected to the main control module, the main control module includes a main controller, a power circuit respectively connected to the main controller, a memory, and four cameras connected to the four cameras. an image correction and signal conditioning module and two signal conditioning circuits respectively connected to the ship speed sensor and the gyroscope sensor, the main control module also has a cluster analysis module, the memory is connected to the cluster analysis module, so The cluster analysis module is connected with the mooring control module; the four cameras are respectively installed on the front, rear, left and right outer surfaces of the hull; It is characterized in that: the obstacle identification method comprises the following steps: First, the four cameras capture the real-time images of the hull and the front, rear, left and right sides of the ship respectively, and transmit the real-time images to the main controller; The ship speed sensor and the gyroscope sensor respectively collect the traveling speed and the running azimuth of the ship as the ship information; The four image correction and signal conditioning modules perform correction and signal processing on the real-time images collected by the four cameras, and then perform stitching processing on the four corrected images to obtain a panoramic image around the hull; The display module displays a panoramic image around the hull; The two signal conditioning circuits correct and process the ship information collected by the ship speed sensor and the gyroscope sensor to obtain the ship's running speed and specific orientation; storing the collected vessel information in the memory and providing the cluster analysis module with the data of the cluster analysis, that is, the panoramic image; The cluster analysis module uses the adaptive threshold cascade linear support vector machine to identify different types of obstacles according to the characteristics of the panoramic images stored in the memory and provides the cluster analysis module to the cluster analysis module, and has a good understanding of the safety and convenience of the currently prepared parking environment. The clustering algorithm with favorable characteristics of sexuality is used to analyze, so as to judge whether it is necessary to call the berthing control module to berth the ship. 2. The obstacle identification method as claimed in claim 1, characterized in that: the clustering module working method is as follows: adopt the improved CHT feature to extract the obstacle feature T in the panoramic surround view image, T=G(p _c )t(s(p _c -p ₀ ),s(p _c -p ₁ ),...,s(p _c -p ₇ )) in, In the formula, the t() function represents the joint distribution function of the sign difference value; s(p _c -p _i ) represents the sign difference value between the current pixel point p _c and the i-th neighborhood point p _i , G(p _c ) is the standard normal Gaussian distribution function; Adopt adaptive threshold cascade linear support vector machine to identify CHT features of different types of obstacles; Extract CHT features from all samples to form a training set, and use linear SVM to classify them to obtain a hyperplane classifier; Each sample has an output value of the obstacle type m about the hyperplane classifier, which is calculated as follows: Among them, w ^* , b ^* are the parameters of the hyperplane classifier respectively; x _n is the CHT feature vector of the nth sample, and each output value is added with a one-dimensional 0-1 uniformly distributed random number, which is mapped to a two-dimensional space, After the training of the first-layer classifier is completed, the remaining samples after removing some samples according to the adaptive threshold method are used as the training samples of the second-layer classifier, and so on, until the last layer is classified, and finally the obstacle type is obtained and it is judged whether it is not. Anchor.