CN118397108B - A calibration method for combining underwater acoustic and optical information - Google Patents

Abstract

The invention provides a calibration method for underwater acousto-optic information combination, and belongs to the technical field of underwater acoustics and optical imaging. The invention establishes a camera imaging model according to the pinhole imaging principle, utilizes a checkerboard calibration plate as a standard reference object, and solves an internal matrix of the camera through coordinate conversion and a maximum likelihood methodAnd a distortion matrix; according to Zhang Zhengyou calibration method, using a function in openCV to obtain a rotation matrix and a translation matrix between a world coordinate system and a camera coordinate system which take sonar as an origin; and calculating the conversion relation among the four coordinate systems by using an imaging model of the camera, thereby completing calibration. According to the sonar image acquisition method, the sonar image can be obtained only by obtaining the position coordinates of the target object, so that the sonar image acquisition cost is reduced, and the algorithm calculation complexity is low. By the establishment of the method, the combination of acoustic information and optical information is realized, and the purpose of improving the underwater imaging quality is achieved.

Description

A calibration method for combining underwater acoustic and optical information

Technical Field

The invention belongs to the technical field of underwater acoustic and optical imaging, and in particular relates to a calibration method for combining underwater acoustic and optical information.

Background Art

In marine exploration activities, autonomous underwater vehicles (AUV) and remote operated vehicles (ROV) are the main detection tools, usually equipped with sonar and cameras. Sonar is not affected by the turbidity of the water and can detect targets at a relatively long distance. The camera can intuitively obtain high-resolution target images, but due to the scattering of light in seawater and the influence of the medium on its absorption, the optical image is blurred, the color is distorted, and the imaging distance is severely limited. Autonomous underwater vehicles (AUV) and remote operated vehicles (ROV) approach the target and adjust the light brightness according to the target position information and the collected image information. These two imaging technologies are complementary. Therefore, the acoustic-optical fusion technology has developed rapidly in recent years and has been widely used in the underwater field. It is a very potential method to improve the overall imaging capability.

At present, the main research direction of underwater acoustic and optical fusion is the underwater acoustic and optical image spatial registration algorithm, which is mainly divided into region-based and feature-based registration methods.

The region-based underwater photoacoustic image registration method mainly uses the grayscale statistical information of the two photoacoustic images to obtain the registration measurement function by searching for its optimal global parameters, thereby realizing the spatial registration of the two images. This traversal search and matching algorithm has high computational complexity and matching accuracy, and the matching accuracy is related to the quality of the original data, requiring a large overlapping area between the images.

Feature-based registration methods are mainly based on feature descriptor-based registration methods and deep learning-based registration methods. Feature-based registration algorithms are affected by the significant structural differences of underwater acoustic and optical images. For example, if the target object is rotated or has different proportions in the acoustic and optical images, the robustness of the feature descriptor will be reduced and the registration accuracy will be affected. For deep learning-based registration methods, a large number of data sets are required to train a network with strong generalization ability. Therefore, feature-based registration methods require a unified underwater acoustic and optical data set. Underwater optical scene data sets are easy to obtain, while sonar image acquisition experiments are costly.

Summary of the invention

Based on the above problems to be solved urgently, the present invention proposes a multi-physical field calibration method, which converts the coordinates of the target object detected by sonar, that is, the coordinates in the world coordinate system with sonar as the origin, into the pixel coordinates of the target object in the optical image through a camera calibration method, thereby realizing the combination of acoustic information and optical information and achieving the purpose of improving the quality of underwater imaging.

The present invention proposes a calibration method for combining underwater sound and light information, comprising the following steps:

Step 1: Establish a camera imaging model based on the pinhole imaging principle, and set the sonar as the origin World coordinate system , camera coordinate system , image physical coordinate system , image pixel coordinate system ;

Step 2: Use a checkerboard calibration plate as a standard reference. By changing its position, the camera collects multiple images. Given the size of the checkerboard and the size of each grid, establish the relationship between the image physical coordinates of the corner points on the checkerboard and the two-dimensional image coordinates of the corresponding image points in the camera coordinate system. After coordinate transformation and maximum likelihood method, solve the camera internal matrix and the distortion matrix;

Step 3: According to Zhang Zhengyou calibration method, use the solved camera internal matrix And the distortion matrix and the known world coordinates of the target feature points and the pixel coordinates of the target feature points in the image, use the function in openCV to get the rotation matrix and translation matrix between the world coordinate system with the sonar as the origin and the camera coordinate system;

Step 4: Use the imaging model of the camera to calculate the transformation relationship between the four coordinate systems; that is, derive the mathematical relationship between the coordinates of the feature points of the target object in the world coordinate system and its pixel coordinates on the imaging plane of the camera, and convert the internal matrix , distortion matrix, rotation matrix and translation matrix are substituted into the relationship to obtain the coordinates of the target position identified by the known sonar. and the pixel coordinates of the target The mapping relationship between them is established to complete the calibration.

Preferably, in step 1, the sonar is set as the origin World coordinate system , camera coordinate system , image physical coordinate system , image pixel coordinate system , specifically:

Sonar is the coordinate origin Establishing the world coordinate system ; , and The axis satisfies the right-hand screw rule;

Set the camera coordinate system , with the camera optical center As the origin of the coordinate system, a straight line passing through the optical center and perpendicular to the imaging plane is drawn axis, Axis and The plane formed by the axes is parallel to the imaging plane to establish the camera coordinate system;

Set the image physical coordinate system , the image physical coordinate system is established on the two-dimensional imaging surface of the camera, and the origin is the intersection of the camera optical axis and the imaging surface, axis The axes are parallel to the camera coordinate system axis axis;

Set the image pixel coordinate system ; The image pixel coordinate system takes the upper left corner of the image as the origin, axis The axis is the coordinate axis, the unit of the coordinate axis is pixel, and the horizontal coordinate of the pixel is With vertical coordinate Respectively represent the row and column numbers in the image array.

Preferably, the specific process of step 2 is:

A checkerboard calibration plate with known dimensions and grid size is used as a standard reference. To ensure the accuracy of calibration, N images of the checkerboard calibration plate at different positions and angles are collected.

Use the built-in calibrateCamera function in the openCV library for calibration. The calibration program reads the pixel coordinates and world coordinates of the corner points. The pixel coordinates of the corner points are the pixel coordinates of the corner points in the acquired image; the world coordinates of the corner points are the world coordinates of the corner points in a world coordinate system established with the chessboard calibration plate as the z plane, the upper left corner of the chessboard as the origin, and the length and width of the chessboard as the x and y axes respectively;

The principle of camera calibration is:

in , Respectively and , , Respectively represent the number of pixels per unit distance in the horizontal and vertical directions, Represents the focal length, in the camera coordinate system The pixel coordinates of the intersection of the axis and the image are ;

During calibration, the world coordinate system The plane is defined on the calibration plate plane. represents the world coordinates of the corner points of the chessboard in this world coordinate system, are the pixel coordinates of the corner points of the chessboard; for The projection matrix; Represents the extrinsic parameter matrix of the camera; and Respectively represent the rotation matrix and translation matrix of the world coordinate system defined during calibration relative to the camera coordinate system; the internal matrix of the camera is calculated according to the maximum likelihood method and the distortion matrix.

Preferably, the specific process of step 3 is:

Obtain the world coordinates of the feature points of the target object in the world coordinate system with the sonar as the origin; obtain the pixel coordinates of the feature points of the target object;

The camera internal parameters are solved The built-in solvePnP function in openCV is used to find the rotation vector and translation matrix using the distortion matrix, the world coordinates of the known target feature points, and the pixel coordinates of the target feature points in the image. , and use the Rodrigues function to convert the rotation vector into a rotation matrix .

Preferably, the world coordinates of the feature points of the target object in the world coordinate system with the sonar as the origin are obtained by using any one of the following three methods:

Measuring with a tape measure , and The coordinates on the axis represent the world coordinates of the feature points of the target object in the world coordinate system with the sonar as the origin;

Or use 2D sonar to obtain images, measure polar coordinates, and use a tape measure to obtain The coordinates on the axis are converted into the polar coordinates obtained by the two-dimensional sonar. , The world coordinates on the axis, combined The world coordinates on the axis are obtained to obtain the world coordinates of the feature points of the target object in the world coordinate system with the sonar as the origin;

Or directly use three-dimensional sonar measurement to obtain the world coordinates of the target feature points in the world coordinate system with the sonar as the origin.

Preferably, the specific process of step 4 is:

S1, the coordinates of the target position identified by the known sonar Convert to camera coordinates ;

S2, the homogeneous coordinates of the target position identified by the sonar in the camera coordinate system Convert to homogeneous coordinates in the image's physical coordinate system , through the pinhole camera model:

in, is the focal length of the camera;

S3, derive the coordinates in the camera coordinate system Coordinates in the same pixel coordinate system The relationship between the two is established and the coordinate transformation is completed; including the following processes:

S31, when converting, first consider the offset, assuming that the camera coordinate system The pixel coordinates of the intersection of the axis and the image are , then the point in the image physical coordinate system is The coordinates of the resulting image in the pixel coordinate system are:

S32, converting physical units and pixel units:

in, , Respectively represent the number of pixels per unit distance in the horizontal and vertical directions, focal length The unit is ;

S33, use , express and , we can get:

Get the coordinates in the camera coordinate system The relationship between the coordinates in the pixel coordinate system:

Since the intrinsic parameter matrix of the camera is:

Then we have:

S34, the coordinates in the camera coordinate system Convert to pixel coordinates ; The internal matrix With the obtained camera coordinates Multiply and normalize the resulting matrix by dividing each element by , the first two lines of the matrix are the obtained pixel coordinates; the undistortPoints function in the openCV library function is used to correct the distortion of the obtained pixel coordinates, and the corrected pixel coordinates are rounded to obtain the final pixel coordinates of the target object. .

Preferably, the specific process of S1 is:

Convert the known target's coordinates in the world coordinate system with the sonar as the origin into homogeneous coordinates and transpose them and rotate the matrix and Connect to form a new matrix ,Right now:

in, , They are the rotation matrix and translation matrix of the world coordinate system with the sonar as the origin relative to the camera coordinate system, is a 3×3 orthogonal identity matrix with 3 independent variables, is a 3×1 translation matrix, is a 4×4 camera extrinsic parameter matrix;

According to the relationship between the coordinates of the target position identified by the sonar and the coordinates in the camera coordinate system, the matrix Multiply the homogeneous coordinates of the known target in the world coordinate system with the sonar as the origin to obtain the homogeneous coordinates in the camera coordinate system ; The relationship between the two is:

in, Yes The homogeneous coordinates in the world coordinate system with the sonar as the origin are: Yes Homogeneous coordinates in the camera coordinate system.

Compared with the prior art, the present invention has the following beneficial effects:

The experimental results show that by inputting the world coordinates of different checkerboard corner points and visualizing the calculated pixel coordinates, it can be seen that they are basically consistent with the characteristic points of the target object on the image. The accurate acoustic-optical physical field calibration method can be used for new light and acoustic joint detection technology. Specifically, it can be used to calculate the exposure of the target area based on the pixel coordinates of the target in the image, thereby adjusting the energy delivery of the light source, improving the underwater imaging quality, and providing a good light field environment and visual effects for underwater robots to operate in different water environments.

The method proposed in the present invention makes full use of the advantages of Zhang Zhengyou's calibration method, avoids the problems of high-precision calibration objects and cumbersome operation, and achieves higher accuracy than other calibration methods, and can also avoid the problem of high cost. The method is simple to make, takes lens distortion into account during the calibration process, increases the accuracy of calibration, and also has good stability.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of the overall process flow of the present invention.

FIG2 is a schematic diagram of a camera imaging model.

FIG. 3 is a schematic diagram of a checkerboard calibration plate in an embodiment.

FIG. 4 is an image obtained by sonar in the embodiment.

FIG. 5 is an optical image corresponding to the sonar image in the embodiment.

FIG. 6 is a sonar image of the world coordinates in the world coordinate system with the sonar as the origin in the embodiment.

FIG. 7 is a diagram showing the result of converting the world coordinates of the first target feature point into pixel coordinates in the embodiment.

FIG. 8 is a diagram showing the result of converting the world coordinates of the feature point of the second target object into pixel coordinates in the embodiment.

FIG. 9 is a diagram showing the result of converting the third corner point into pixel coordinates in the embodiment.

DETAILED DESCRIPTION

The present invention proposes a method for converting the coordinates of a target object detected by sonar (i.e., the coordinates in the world coordinate system with sonar as the origin) into the pixel coordinates of the target object in an optical image. The sonar image used in this method only needs to be able to obtain the position coordinates of the target object, which reduces the cost of obtaining sonar images, and the algorithm calculation complexity is not high. By establishing this method, the combination of acoustic information and optical information is achieved, thereby achieving the purpose of improving the quality of underwater imaging. The overall process is shown in Figure 1:

Step 1: Establish a camera imaging model based on the pinhole imaging principle, and set the sonar as the origin World coordinate system , camera coordinate system , image physical coordinate system , image pixel coordinate system ;

Step 2: Use a checkerboard calibration plate as a standard reference. By changing its position, the camera collects multiple images. Given the size of the checkerboard and the size of each grid, establish the relationship between the image physical coordinates of the corner points on the checkerboard and the two-dimensional image coordinates of the corresponding image points in the camera coordinate system. After coordinate transformation and maximum likelihood method, solve the camera internal matrix and the distortion matrix;

Step 3: According to Zhang Zhengyou calibration method, use the solved camera internal matrix And the distortion matrix and the known world coordinates of the target feature points and the pixel coordinates of the target feature points in the image, use the function in openCV to get the rotation matrix and translation matrix between the world coordinate system with the sonar as the origin and the camera coordinate system;

Step 4: Use the imaging model of the camera to calculate the transformation relationship between the four coordinate systems; that is, derive the mathematical relationship between the coordinates of the feature points of the target object in the world coordinate system and its pixel coordinates on the imaging plane of the camera, and convert the internal matrix , distortion matrix, rotation matrix and translation matrix are substituted into the relationship to obtain the coordinates of the target position identified by the known sonar. and the pixel coordinates of the target The mapping relationship between them is established to complete the calibration.

The invention will be further described below in conjunction with specific embodiments.

1. Camera imaging model establishment:

The established camera imaging model is shown in Figure 2.

Set the world coordinate system , with the sonar as the coordinate origin Establish the world coordinate system. Axis and The plane formed by the axes is parallel to the ground. Axis perpendicular to Axis and The plane formed by the axes establishes the world coordinate system. , and The shaft satisfies the right-hand screw rule.

Set the camera coordinate system , with the camera optical center As the origin of the coordinate system, a straight line passing through the optical center and perpendicular to the imaging plane is drawn axis, Axis and The camera coordinate system is established by making the plane formed by the axes parallel to the imaging plane.

Set the image physical coordinate system , the image physical coordinate system is established on the two-dimensional imaging surface of the camera, and the origin is the intersection of the camera optical axis and the imaging surface, axis The axes are parallel to the camera coordinate system axis axis.

Set the image pixel coordinate system The image pixel coordinate system takes the upper left corner of the image as its origin. axis The axis is the coordinate axis, the unit of the coordinate axis is pixel (pixel), and the horizontal coordinate of the pixel With vertical coordinate Respectively represent the row and column numbers in the image array.

2. Solve the camera internal matrix And the distortion matrix:

The general principle of camera calibration is:

in , Respectively and , , Represents the number of pixels per unit distance in the horizontal and vertical directions respectively , Represents the focal length, in the camera coordinate system The pixel coordinates of the intersection of the axis and the image are For the convenience of calculation, the world coordinate system is calibrated The plane is defined on the calibration plate plane. represents the world coordinates of the corner points of the chessboard in this world coordinate system, are the pixel coordinates of the corner points of the chessboard. for The projection matrix. Represents the extrinsic parameter matrix of the camera. and They represent the rotation matrix and translation matrix of the world coordinate system relative to the camera coordinate system defined during calibration. The specific steps of solving the camera intrinsic parameter matrix using the calibration principle include:

The calibration plate is made of aluminum oxide and has a checkerboard grid. , the side length of each square grid is 20mm. The camera model used is Deepsea IPMSC-3105, and the camera resolution is 1080p. In order to ensure the accuracy of calibration, 14 images of the checkerboard calibration board at different positions and angles are collected. Figure 3 is a schematic diagram of the checkerboard calibration board.

The software used for camera calibration is python, and the built-in calibrateCamera function in the openCV library is used for calibration. The calibration program reads the pixel coordinates and world coordinates of the corner points, and calculates the intrinsic parameters and distortion matrix of the camera based on the maximum likelihood estimation.

Run the calibration program to obtain the calibration results. In this embodiment, the internal parameter matrix is

Save the calibration results in the file camera.pkl.

3. Solve the rotation matrix and translation matrix between the world coordinate system with the sonar as the origin and the camera coordinate system:

According to the camera calibration principle, the rotation matrix between the world coordinate system with the sonar as the origin and the camera coordinate system is solved using the intrinsic parameter matrix and distortion matrix obtained by calibration. and translation matrix The specific steps are:

S1, obtain the world coordinates of the feature points of the target object in the world coordinate system with the sonar as the origin.

The image was obtained using two-dimensional sonar. Figure 4 is the image obtained by sonar.

Polar coordinates were obtained by measuring with ProViewer software and using a tape measure. The coordinates on the axis are converted into the polar coordinates obtained by the two-dimensional sonar. , The world coordinates on the axis, combined The world coordinates on the axis are obtained to obtain the world coordinates of the target feature point in the world coordinate system with the sonar as the origin. The conversion relationship between the polar coordinates and the world coordinates of the target feature point is:

In the formula, It is the distance between the feature point of the target object and the sonar measured by the ProViewer software. It is the angle at which the feature point of the target deviates from the central axis of the sonar.

S2, obtaining the pixel coordinates of the feature points of the target object.

The optical image corresponding to the sonar image is obtained by using a camera. Figure 5 is the optical image corresponding to the sonar image.

Python is used to manually select the pixel coordinates of the feature points of the target object. The pixel coordinates of each feature point are read 5 times, and the average value is taken to obtain the final pixel coordinates to reduce the error of manual calibration.

S3, use the solvePnP function built into openCV to find the rotation vector and translation matrix , and use the Rodrigues function to convert the rotation vector into a rotation matrix The rotation vector is:

Convert to rotation matrix:

The translation matrix is:

.

4. Use the camera's imaging model to calculate the conversion relationship between the four coordinate systems and complete the calibration:

S1, the coordinates of the target position identified by the known sonar Convert to camera coordinates ;

Convert the known target's coordinates in the world coordinate system with the sonar as the origin into homogeneous coordinates and transpose them and rotate the matrix and Connect to form a new matrix ,Right now:

in , , They are the rotation matrix and translation matrix of the world coordinate system with the sonar as the origin relative to the camera coordinate system, is a 3×3 orthogonal identity matrix with 3 independent variables, is a 3×1 translation matrix, is a 4×4 camera extrinsic parameter matrix;

According to the relationship between the coordinates of the target position identified by the sonar and the coordinates in the camera coordinate system, the matrix Multiply the homogeneous coordinates of the known target in the world coordinate system with the sonar as the origin to obtain the homogeneous coordinates in the camera coordinate system ; The relationship between the two is:

in, Yes The homogeneous coordinates in the world coordinate system with the sonar as the origin are: Yes Homogeneous coordinates in the camera coordinate system.

S2, the homogeneous coordinates of the target position identified by the sonar in the camera coordinate system Convert to homogeneous coordinates in the image's physical coordinate system , through the pinhole camera model:

in, is the focal length of the camera;

S3, derive the coordinates in the camera coordinate system Coordinates in the same pixel coordinate system The relationship between the two is established and the coordinate transformation is completed; including the following processes:

S31, when converting, first consider the offset, assuming that the camera coordinate system The pixel coordinates of the intersection of the axis and the image are , then the point in the image physical coordinate system is The coordinates of the resulting image in the pixel coordinate system are:

S32, converting physical units and pixel units:

in, , Represents the number of pixels per unit distance in the horizontal and vertical directions respectively ,focal length The unit is ;

S33, use , express and , we can get:

Get the coordinates in the camera coordinate system The relationship between the coordinates in the pixel coordinate system:

Since the intrinsic parameter matrix of the camera is:

Then we have:

S34, the coordinates in the camera coordinate system Convert to pixel coordinates ; The internal matrix With the obtained camera coordinates Multiply and normalize the resulting matrix by dividing each element by , the first two lines of the matrix are the obtained pixel coordinates; the undistortPoints function in the openCV library function is used to correct the distortion of the obtained pixel coordinates, and the corrected pixel coordinates are rounded to obtain the final pixel coordinates of the target object. .

5. Experimental results:

In this embodiment, the coordinate input of the first, second and third feature points in the sonar image of FIG6 in the world coordinate system with the sonar as the origin is read, and the calculated pixel coordinates are represented in the optical images of FIG7, FIG8 and FIG9. The calculated pixel coordinates are visualized according to the image, and it can be seen that they are basically consistent with the feature points on the image.

The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims (6)

1. A calibration method for combining underwater sound and light information, characterized in that it comprises the following steps: Step 1: Establish a camera imaging model based on the pinhole imaging principle, and set the sonar as the origin World coordinate system , camera coordinate system , image physical coordinate system , image pixel coordinate system ; Step 2: Use a checkerboard calibration plate as a standard reference. By changing its position, the camera collects multiple images. Given the size of the checkerboard and the size of each grid, establish the relationship between the image physical coordinates of the corner points on the checkerboard and the two-dimensional image coordinates of the corresponding image points in the camera coordinate system. After coordinate transformation and maximum likelihood method, solve the camera internal matrix and the distortion matrix; Step 3: According to Zhang Zhengyou calibration method, use the solved camera internal matrix And the distortion matrix and the known world coordinates of the target feature points and the pixel coordinates of the target feature points in the image, use the function in openCV to get the rotation matrix and translation matrix between the world coordinate system with the sonar as the origin and the camera coordinate system; Step 4: Use the imaging model of the camera to calculate the transformation relationship between the four coordinate systems; derive the mathematical relationship between the coordinates of the feature points of the target object in the world coordinate system and its pixel coordinates on the camera imaging plane, and convert the internal matrix , distortion matrix, rotation matrix and translation matrix are substituted into the relationship to obtain the coordinates of the target position identified by the known sonar. and the pixel coordinates of the target The mapping relationship between them is used to complete the calibration; The specific process is: S1, the coordinates of the target position identified by the known sonar Convert to camera coordinates ; S2, the homogeneous coordinates of the target position identified by the sonar in the camera coordinate system Convert to homogeneous coordinates in the image's physical coordinate system , through the pinhole camera model: in, is the focal length of the camera; S3, derive the coordinates in the camera coordinate system Coordinates in the same pixel coordinate system The relationship between the two is established and the coordinate transformation is completed; including the following processes: S31, when converting, first consider the offset, assuming that the camera coordinate system The pixel coordinates of the intersection of the axis and the image are , then the point in the image physical coordinate system is The coordinates of the resulting image in the pixel coordinate system are: S32, converting physical units and pixel units: in, , Respectively represent the number of pixels per unit distance in the horizontal and vertical directions, focal length The unit is ; S33, use , express and , we can get: Get the coordinates in the camera coordinate system The relationship between the coordinates in the pixel coordinate system: Since the intrinsic parameter matrix of the camera is: Then we have: S34, the coordinates in the camera coordinate system Convert to pixel coordinates ; The internal matrix With the obtained camera coordinates Multiply and normalize the resulting matrix by dividing each element by , the first two lines of the matrix are the obtained pixel coordinates; the undistortPoints function in the openCV library function is used to correct the distortion of the obtained pixel coordinates, and the corrected pixel coordinates are rounded to obtain the final pixel coordinates of the target object. . 2. A calibration method for combining underwater sound and light information according to claim 1, characterized in that the sonar is set as the origin in step 1. World coordinate system , camera coordinate system , image physical coordinate system , image pixel coordinate system , specifically: Sonar is the coordinate origin Establishing the world coordinate system ; , and The axis satisfies the right-hand screw rule; Set the camera coordinate system , with the camera optical center As the origin of the coordinate system, a straight line passing through the optical center and perpendicular to the imaging plane is drawn axis, Axis and The plane formed by the axes is parallel to the imaging plane to establish the camera coordinate system; Set the image physical coordinate system , the image physical coordinate system is established on the two-dimensional imaging surface of the camera, and the origin is the intersection of the camera optical axis and the imaging surface, axis The axes are parallel to the camera coordinate system axis axis; Set the image pixel coordinate system ; The image pixel coordinate system takes the upper left corner of the image as the origin, axis The axis is the coordinate axis, the unit of the coordinate axis is pixel, and the horizontal coordinate of the pixel is With vertical coordinate Respectively represent the row and column numbers in the image array. 3. A calibration method for combining underwater sound and light information according to claim 1, characterized in that the specific process of step 2 is: A checkerboard calibration plate with known dimensions and grid size is used as a standard reference. To ensure the accuracy of calibration, N images of the checkerboard calibration plate at different positions and angles are collected. Use the built-in calibrateCamera function in the openCV library for calibration. The calibration program reads the pixel coordinates and world coordinates of the corner points. The pixel coordinates of the corner points are the pixel coordinates of the corner points in the acquired image; the world coordinates of the corner points are the world coordinates of the corner points in a world coordinate system established with the chessboard calibration plate as the z plane, the upper left corner of the chessboard as the origin, and the length and width of the chessboard as the x and y axes respectively; The principle of camera calibration is: in , Respectively and , , Respectively represent the number of pixels per unit distance in the horizontal and vertical directions, Represents the focal length, in the camera coordinate system The pixel coordinates of the intersection of the axis and the image are ; During calibration, the world coordinate system The plane is defined on the calibration plate plane. represents the world coordinates of the corner points of the chessboard in this world coordinate system, are the pixel coordinates of the corner points of the chessboard; for The projection matrix; Represents the extrinsic parameter matrix of the camera; and Respectively represent the rotation matrix and translation matrix of the world coordinate system defined during calibration relative to the camera coordinate system; the internal matrix of the camera is calculated according to the maximum likelihood method and the distortion matrix. 4. A calibration method for combining underwater sound and light information according to claim 1, characterized in that the specific process of step 3 is: Obtain the world coordinates of the feature points of the target object in the world coordinate system with the sonar as the origin; obtain the pixel coordinates of the feature points of the target object; The camera internal parameters are solved The built-in solvePnP function in openCV is used to find the rotation vector and translation matrix using the distortion matrix, the world coordinates of the known target feature points, and the pixel coordinates of the target feature points in the image. , and use the Rodrigues function to convert the rotation vector into a rotation matrix . 5. A calibration method for combining underwater sound and light information according to claim 4, characterized in that the world coordinates of the feature points of the target object in the world coordinate system with the sonar as the origin are obtained by using any one of the following three methods: Measuring with a tape measure , and The coordinates on the axis represent the world coordinates of the feature points of the target object in the world coordinate system with the sonar as the origin; Or use 2D sonar to obtain images, measure polar coordinates, and use a tape measure to obtain The coordinates on the axis are converted into the polar coordinates obtained by the two-dimensional sonar. , The world coordinates on the axis, combined The world coordinates on the axis are obtained to obtain the world coordinates of the feature points of the target object in the world coordinate system with the sonar as the origin; Or directly use three-dimensional sonar measurement to obtain the world coordinates of the target feature points in the world coordinate system with the sonar as the origin. 6. A calibration method for combining underwater sound and light information according to claim 1, characterized in that the specific process of S1 is: Convert the known target's coordinates in the world coordinate system with the sonar as the origin into homogeneous coordinates and transpose them and rotate the matrix and Connect to form a new matrix ,Right now: in, , They are the rotation matrix and translation matrix of the world coordinate system with the sonar as the origin relative to the camera coordinate system, is a 3×3 orthogonal identity matrix with 3 independent variables, is a 3×1 translation matrix, is a 4×4 camera extrinsic parameter matrix; According to the relationship between the coordinates of the target position identified by the sonar and the coordinates in the camera coordinate system, the matrix Multiply the homogeneous coordinates of the known target in the world coordinate system with the sonar as the origin to obtain the homogeneous coordinates in the camera coordinate system ; The relationship between the two is: in, Yes The homogeneous coordinates in the world coordinate system with the sonar as the origin are: Yes Homogeneous coordinates in the camera coordinate system.