CN106023193B - A kind of array camera observation procedure detected for body structure surface in turbid media - Google Patents

Abstract

The invention discloses a kind of array camera observation procedures detected for body structure surface in turbid media, using camera array device, observed range can be greatly shortened in the case that visual field size is constant, imaging definition is improved, to realize the purpose that body structure surface blur-free imaging is carried out in turbid media.This method includes the following steps：Each camera is demarcated using encoded point scaling board, while obtains the spin matrix and translation matrix of the inner parameter matrix of each camera, distortion parameter matrix, camera coordinates system and world coordinate system；Calculate camera array ideal inner parameter matrix；Calculate image homograph matrix；Camera photocentre correction calculates；Lens distortion calibration calculates；The mapping relations of single camera image and array image, that is, look-up table calculates；Body structure surface high resolution imaging shows and detects.

Description

An Array Camera Observation Method for Structural Surface Detection in Turbid Media

technical field

The invention relates to an array camera observation method for structural surface detection in turbid media, in particular to a high-definition structural surface observation method for turbid media realized by digital image processing technology and camera calibration technology.

Background technique

The structure will be affected by various conditions during the service process, which will cause different degrees of damage to the structure and affect its normal use. Therefore, the safety detection of the structure is of great significance, and the surface detection is one of the important aspects. Surface inspection refers to the measurement of the geometric dimensions of the overall structure and local structures, the detection and measurement of structural defects, etc.

Under normal conditions, a single camera is generally used to detect the surface of the structure, but under some complex conditions such as turbid media, if the observation distance is long, it is difficult to obtain a clear appearance image due to the influence of the light transmission medium; but the observation distance is short , the field of view becomes smaller. At present, most camera arrays use long-distance imaging, and the image clarity is difficult to guarantee. So far, there has not been an array camera method that can be used for structural surface detection in turbid media.

Contents of the invention

Technical problem: The present invention provides an array camera observation method for structural surface detection in turbid media, which is simple to operate, easy to implement, and can display structure surface images in real time during the detection process.

Technical solution: The array camera observation method for structural surface detection in turbid media of the present invention includes the following steps:

1) Array camera calibration: place the code point calibration plate at the position of the measured structure plane, and all the cameras in the array collect the calibration plate images synchronously to obtain a set of image 0; translate the calibration plate along a direction perpendicular to its plane for a known distance, The array camera synchronously collects the images of the calibration plate to obtain a set of image 1; the image 0 and image 1 collected by each camera are calibrated using a two-step method to obtain all the parameters of the camera, and the camera parameters include: the internal parameter matrix A _i , lens distortion parameter matrix D _i , rotation matrix R _i =[R _i,x R _i,y R _i,z ] and translation matrix T _i =[T _{i, x} T _{i, y} T _{i, z} ], the internal parameter matrix is Wherein f _{i, x} , f _{i, y} are the horizontal and vertical equivalent focal lengths of the ith camera, i=1, 2, ..., m, m is the number of cameras, s _i is the tilt factor, c _{i, x} , c _{i, y} are the pixel coordinates of the intersection point of the optical axis of the lens and the target surface;

2) The array camera collects the structural surface image synchronously, and calculates the homography transformation matrix H _i of the structural surface image collected by each camera according to the following relation:

A _ideal R _ideal ＝H _i A _i R _i

3) First calculate the world coordinates of each camera optical center Where C _i =[C _{i, x} C _{i, y} C _{i, z} ], then calculate the corrected distance C _{ideal, z} from the optical center to the surface of the structure, C _{ideal, z} is C _{i in the world coordinates of the optical center of all cameras,} mean value of _z ;

4) Taking the camera in the upper left corner of the camera array as camera 1, calculate the horizontal and vertical pixel translations x _trans , y _trans of the optical centers of all cameras relative to the optical center of camera 1 according to the following formula:

x _trans = (C _{i, x} −C _{1, x} )×f _{ideal, x} /C _{ideal, z}

y _trans = (C _i,y -C _1,y )×f _ideal,y /C _ideal,z

5) Traverse all points in the array image in the following manner, and calculate and determine the mapping relationship between the structure surface image collected by a single camera and the array image: For any point on the array image, all cameras obtained according to step 4) are relative to camera 1 Calculate the image position of the point at the i-th camera, and then use the corrected distance from the optical center to the structure surface obtained in step 3) to correct the optical center position of each camera, using the step 2) Homography transformation correction is performed on the obtained homography transformation matrix, and finally distortion correction is carried out using the lens distortion parameter matrix obtained in step 1);

The optical center position correction is performed according to the following formula:

x _ideal - c _{ideal, x} = (xi _- c _{ideal, x} ) × C _{i, z} / C _{ideal, z}

y _ideal -c _ideal,y =(y _i -c _ideal,y )×C _i,z /C _ideal,z

Among them, x _i and y _i are the image coordinates of any point before correction, and x _ideal and y _ideal are the image coordinates of any point after correction;

6) Calculate the pixel value of each image point in the array image through the mapping relationship obtained in step 5), and thereby obtain the array image to perform array image display and structure surface detection in real time.

Further, in the method of the present invention, in step 1), the lens distortion parameter matrix includes 6th-order radial distortion parameters K ₁ , K ₂ , K ₃ , K ₄ , K ₅ , K ₆ and 2nd-order tangential distortion parameters Matrix P ₁ , P ₂ .

Further, in the method of the present invention, in the step 2), the calibration plate image includes but not limited to code points.

By adopting the camera array device, the method of the invention shortens the observation distance under the condition that the size of the field of view remains unchanged, improves the imaging definition in the turbid medium, and finally realizes the purpose of detecting the structure surface in the turbid medium.

Beneficial effect: compared with the prior art, the present invention has the following advantages:

(1) Intuitive detection of the surface of the structure under test. Compared with other detection technologies such as acoustic imaging, the present invention adopts optical imaging technology, which can directly observe the original image of the structure surface, which is more intuitive and effective.

(2) The structure surface imaging is clearer. The traditional single-camera surface detection imaging distance is relatively long and the field of view is small; however, the present invention uses an array camera device, which not only increases the field of view, but also shortens the observation distance and reduces the influence of turbid media on imaging clarity. Improving the imaging clarity of the structure surface.

(3) The operation is simple and convenient. The present invention adopts the coding point, and only needs two steps to complete the calibration. Since the fixed array camera device is used, it only needs to be calibrated once when leaving the factory, and there is no need to calibrate again when the structure surface is detected afterwards.

(4) Real-time detection of the structure surface. The present invention can directly calculate the pixel value of the array image by constructing a lookup table, and is suitable for multi-thread technology parallel operation; compared with the method of image splicing, the calculation efficiency is higher.

Description of drawings

Figure 1 is the code point calibration plate, which is a standard part with known dimensions.

Fig. 2 is a flow chart of the inventive method.

Detailed ways

The present invention will be further described below in conjunction with embodiment and accompanying drawing.

Prepare a camera array device: the cameras are regularly arranged on a rigid frame according to the grid arrangement, and the image resolution, aperture, focal length and other camera and lens parameters of all cameras are the same, and the field of view of a single camera is about 8cm×6cm. The observation distance is about 11cm, the horizontal distance between adjacent cameras is slightly smaller than the width of a single camera's field of view, and the vertical distance is slightly smaller than the height of a single camera's field of view, and the cameras should be kept as parallel as possible; this arrangement can make full use of The image resolution of the camera can ensure a certain overlapping area between adjacent cameras, thereby ensuring the continuity of the final array image. If the structural surface inspection is carried out under underwater conditions, the camera should be equipped with a waterproof device. In order to ensure clear imaging under low light conditions, at least 4 LED lights should be arranged around each camera to increase the brightness.

An array camera observation method for structural surface detection in turbid media includes the following steps:

1) Array camera calibration: place the code point calibration plate shown in Figure 1 on the plane of the structure to be measured, use the mobile workstation to control all cameras in the array to collect the calibration plate images synchronously, and obtain a set of images 0; place the calibration plate along its plane Translate the known distance in the vertical direction, and the array camera collects the images of the calibration board synchronously to obtain a set of image 1; the image 0 and image 1 collected by each camera are calibrated by two-step method to obtain all the parameters of the camera; this method only It takes two steps to calibrate all the parameters of the camera, and multiple measurements can be realized without re-calibration after one calibration. The code point calibration plate is used as the calibration pattern, which is characterized in that each feature point can be uniquely identified, and the image coordinates and world coordinates of each feature point during the calibration process can be uniquely determined. The calibration plate here includes but is not limited to code point calibration plate. The camera parameters include: internal parameter matrix A _i , lens distortion parameter matrix D _i , rotation matrix R _i =[R _{i , x} R _{i , y} R _{i , z} ] and translation matrix T _i =[T _{i, x} T _{i, y} T _{i, z} ]. internal parameter matrix Among them, f _{i, x} , f _{i, y} are the horizontal and vertical equivalent focal lengths of the ith camera, i=1, 2, ..., m (m is the number of cameras), and s _i is the tilt factor , ci _{, x} , ci _{, y} are the pixel coordinates of the intersection point of the optical axis of the lens and the target surface; the lens distortion parameter matrix generally includes six-order radial distortion parameters K ₁ , K ₂ , K ₃ , K ₄ , K ₅ , K ₆ and the second-order tangential distortion parameter matrices P ₁ , P ₂ , and the lens distortion parameter matrix are mainly used to correct image distortion caused by lens distortion.

2) The array camera collects the structural surface image synchronously, and calculates the homography transformation matrix H _i of the structural surface image collected by each camera according to the following relation:

A _ideal R _ideal ＝H _i A _i R _i

3) Due to the installation error of the cameras, the optical centers of all the cameras cannot be on the same plane, so it is necessary to correct the positions of the optical centers of the cameras. First calculate the world coordinates of each camera optical center where C _i =[C _i,x C _i,y C _i,z ]. Then calculate the corrected distance from the optical center to the structure surface as C _{ideal, z} , where C _{ideal, z} is the average value of C _{i, z} in the world coordinates of the optical centers of all cameras.

4) Taking the camera in the upper left corner of the camera array as camera 1, calculate the horizontal and vertical pixel translations x _trans , y _trans of the optical centers of all cameras relative to the optical center of camera 1 according to the following formula:

x _trans = (C _{i, x} −C _{1, x} )×f _{ideal, x} /C _{ideal, z}

y _trans = (C _i,y -C _1,y )×f _ideal,y /C _ideal,z

5) Traverse all points in the array image in the following manner, and calculate and determine the mapping relationship between the structure surface image collected by a single camera and the array image: For any point on the array image, all cameras obtained according to step 4) are relative to camera 1 Calculate the image position of the point at the i-th camera, and then use the distance from the corrected optical center to the structural plane obtained in step 3) to correct the optical center position of each camera, and use the step 2 ) to perform homography transformation correction on the homography transformation matrix obtained, and finally use the lens distortion parameter matrix obtained in step 1) to perform distortion correction. All points in the array image are traversed, and the mapping relationship, that is, the lookup table, is calculated.

The optical center position correction is performed according to the following formula:

x _ideal - c _{ideal, x} = (xi _- c _{ideal, x} ) × C _{i, z} / C _{ideal, z}

y _ideal -c _ideal,y =(y _i -c _ideal,y )×C _i,z /C _ideal,z

Among them, x _i and y _i are the image coordinates of any point before correction, and x _ideal and y _ideal are the image coordinates of any point after correction.

6) Quickly calculate the pixel value of each image point in the array image through the mapping relationship obtained in step 5), thereby obtaining the array image. The array camera device only needs to be calibrated once before leaving the factory to obtain the mapping relationship in step 5), and the mapping relationship can be directly used in the subsequent detection process, and the multi-threading technology can be used to calculate and display the structure surface array image in real time.

The foregoing embodiments are only preferred implementations of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principles of the present invention, several improvements and equivalent replacements can be made, which are important to the rights of the present invention. Technical solutions requiring improvement and equivalent replacement all fall within the protection scope of the present invention.

Claims (3)

1. an array camera observation method for structural surface detection in turbid media, it is characterized in that, the method comprises the following steps: 1) Array camera calibration: place the code point calibration plate at the position of the measured structure plane, and all the cameras in the array collect the calibration plate images synchronously to obtain a set of image 0; translate the calibration plate along a direction perpendicular to its plane for a known distance, The array camera synchronously collects the images of the calibration plate to obtain a set of image 1; the image 0 and image 1 collected by each camera are calibrated using a two-step method to obtain all the parameters of the camera, and the camera parameters include: the internal parameter matrix A _i , lens distortion parameter matrix D _i , rotation matrix R _i =[R _i,x R _i,y R _i,z ] and translation matrix T _i =[T _{i, x} T _{i, y} T _{i, z} ], the internal parameter matrix is Wherein f _{i, x} , f _{i, y} are the horizontal and vertical equivalent focal lengths of the ith camera, i=1, 2, ..., m, m is the number of cameras, s _i is the tilt factor, c _{i, x} , c _{i, y} are the pixel coordinates of the intersection point of the optical axis of the lens and the target surface; 2) The array camera collects the structural surface image synchronously, and calculates the homography transformation matrix H _i of the structural surface image collected by each camera according to the following relation: Where f _{ideal, x} , f _{ideal, y} , s _ideal are the average values of all cameras f _{i, x} , f _{i, y} , s _i , c _{ideal, x} and c _{ideal, y} are the image homography correction of each camera Afterwards, the coordinates of the largest ideal principal point in the imaging area, R _ideal is the identity matrix, 3) First calculate the world coordinates of each camera optical center Where C _i =[C _{i, x} C _{i, y} C _{i, z} ], then calculate the corrected distance C _{ideal, z} from the optical center to the surface of the structure, C _{ideal, z} is C _{i in the world coordinates of the optical center of all cameras,} mean value of _z ; 4) Taking the camera in the upper left corner of the camera array as camera 1, calculate the horizontal and vertical pixel translations x _trans , y _trans of the optical centers of all cameras relative to the optical center of camera 1 according to the following formula: x _trans = (C _{i, x} −C _{1, x} )×f _{ideal, x} /C _{ideal, z} y _trans = (C _i,y -C _1,y )×f _ideal,y /C _ideal,z 5) Traverse all points in the array image in the following manner, and calculate and determine the mapping relationship between the structure surface image collected by a single camera and the array image: For any point on the array image, all cameras obtained according to step 4) are relative to camera 1 Calculate the image position of the point at the i-th camera, and then use the corrected distance from the optical center to the structure surface obtained in step 3) to correct the optical center position of each camera, using the step 2) Homography transformation correction is performed on the obtained homography transformation matrix, and finally distortion correction is carried out using the lens distortion parameter matrix obtained in step 1); The optical center position correction is performed according to the following formula: x _ideal - c _{ideal, x} = (xi _- c _{ideal, x} ) × C _{i, z} / C _{ideal, z} y _ideal -c _ideal,y =(y _i -c _ideal,y )×C _i,z /C _ideal,z Among them, x _i and y _i are the image coordinates of any point before correction, and x _ideal and y _ideal are the image coordinates of any point after correction; 6) Calculate the pixel value of each image point in the array image through the mapping relationship obtained in step 5), thereby obtaining the array image. 2. The array camera observation method for structural surface detection in turbid media according to claim 1, characterized in that, in the step 1), the lens distortion parameter matrix includes six-order radial distortion parameters K ₁ , K ₂ , K ₃ , K ₄ , K ₅ , K ₆ and the second-order tangential distortion parameter matrices P ₁ , P ₂ . 3. The array camera observation method for structural surface detection in turbid media according to claim 1 or 2, characterized in that, in the step 1), the calibration plate image includes but not limited to code points.