CN112033286B - A measurement method of a structural six-degree-of-freedom motion measurement system based on binocular vision - Google Patents

Abstract

The invention discloses a binocular vision-based structural six-degree-of-freedom motion measurement system and a measurement method thereof. Step 1: pasting a round target on the surface of the structure, and arranging at least three measuring points by taking the center of the round target as the measuring points; step 2: calibrating the vision measuring system by using a checkerboard calibration method; and step 3: capturing target motion with two cameras; and 4, step 4: carrying out image point coordinate optimization on the target motion captured in the step 3; and 5: and finishing the six-degree-of-freedom measurement of the structure. The method aims to solve the problems of inconvenient technical operation, low measurement precision, higher cost, low applicability, complex operation and low engineering practicability when the traditional measurement method is used for processing six-degree-of-freedom motion measurement.

Description

A measurement method of a structural six-degree-of-freedom motion measurement system based on binocular vision

technical field

The invention belongs to the technical field of structural six-degree-of-freedom motion measurement, and particularly relates to a measurement method for a structural six-degree-of-freedom motion measuring system based on binocular vision.

Background technique

At present, measurement methods can be divided into contact measurement and non-contact measurement. The measurement methods such as pull-wire displacement meter, dial indicator, linear variable differential pressure sensor are common contact measurement methods. These methods have high precision and high reliability, but these methods have troublesome sensor layout. , low efficiency, greatly affected by the site and the environment. When the measurement conditions are harsh, such as severe deformation of components, high temperature or severe temperature changes, the accuracy of the contact sensor is difficult to guarantee.

Non-contact measurement is not in direct contact with components, has little impact on the structure, high measurement efficiency, can work stably under complex conditions and provide high accuracy, and is becoming a focus of attention of scholars. Computer vision measurement method is an important non-contact measurement method. The visual measurement method can provide abundant measurement information, realize real-time high-precision measurement, and be easy to operate. Nowadays, computer technology and camera technology are increasingly applied to the field of measurement.

In the wave test, the six-degree-of-freedom motion that needs to be measured, that is, the three-dimensional displacement and three-dimensional rotation angle of the structure are measured. It is difficult to obtain the six-degree-of-freedom motion of the structure by traditional measurement methods. Moreover, in the test, the structure needs to be arranged on the water surface, and it is difficult to arrange the contact sensor.

SUMMARY OF THE INVENTION

The purpose of the invention is to solve the problems of inconvenient technical operation, low measurement accuracy, high cost, low applicability, complex operation and low engineering practicability when the traditional measurement method handles the six-degree-of-freedom motion measurement. The measurement method of the structural six-degree-of-freedom motion measurement system of binocular vision.

The present invention is achieved through the following technical solutions:

A measurement method for a structural six-degree-of-freedom motion measurement system based on binocular vision, the measurement system includes two cameras and a computer, the two cameras are connected with a computer through a network cable, and the computer is connected with a synchronization trigger , send the acquisition signal to the camera synchronously through the computer-controlled synchronization trigger;

The two cameras are mounted on a tripod, so that the center of the images of the two cameras is aligned with the structure to be tested.

A measurement method of a six-degree-of-freedom motion measurement system based on binocular vision, the configuration method of the measurement system comprises the following steps:

Step 1: Paste a circular target on the surface of the structure, take the center of the circular target as the measuring point, and arrange at least three measuring points; two cameras are arranged in front of the measuring point, and the two cameras are respectively set in the left front of the measuring point and the right front of the measuring point. The measuring point is within the field of view of the left camera and the right camera;

Step 2: Use the checkerboard calibration method to calibrate the visual measurement system; arrange the checkerboard calibration plate in the field of view, and select the plane on which the checkerboard calibration plate is located when the calibration plate is close to the surface of the structure as the structural reference. noodle;

Step 3: Use two cameras to capture the movement of the target; collect moving images, complete the identification of the circular target in the first image, and track and capture the movement of the circular target during the movement, and obtain the measuring points on the images of the left camera and the right camera. the pixel coordinates of ;

Step 4: Perform image point coordinate optimization on the target motion captured in step 3; calculate the corresponding epipolar line of the matching image point between the left camera and the right camera from the calibration result, and optimize the image point coordinate by the epipolar line equation. image point coordinates;

Step 5: Complete the six-degree-of-freedom measurement of the structure; calculate the three-dimensional displacement of the measuring point from the optimized image point coordinates, select three measuring points to calculate the three-dimensional rotation angle of the structure, and complete the six-degree-of-freedom measurement of the structure.

Further, the step 2 is specifically: calculating the internal parameter matrix A _l of the left camera of the visual measurement system and the internal parameter matrix A _r of the right camera from the collected calibration image; the left camera reference system is converted to the right camera. The rotation matrix R _l2r of the reference frame and the translation vector T _l2r ; the structural reference plane is converted to the rotation matrix R and the translation vector T of the left camera reference frame;

The expressions of A _l , A _r , R _l2r and T _l2r are:

分别为左侧相机在图像平面u、v轴上的等效焦距，

分别为右侧相机在u、v轴上的等效焦距，(u_l,v_l)和(u_r,v_r)分别为左侧相机主点和右侧相机主点在图像平面的坐标，r_i(i＝1,2...,9)为左侧相机参考系到右侧相机参考系的旋转矩阵参数,t_x,t_y,t_z分别为左侧相机参考系到右侧相机参考系的平移矩阵参数。in,

are the equivalent focal lengths of the left camera on the u and v axes of the image plane, respectively,

are the equivalent focal lengths of the right camera on the u and v axes, respectively, (u _l , v _l ) and (u _r , v _r ) are the coordinates of the left camera principal point and the right camera principal point on the image plane, respectively, r _i (i=1,2...,9) is the rotation matrix parameter from the left camera reference frame to the right camera reference frame, t _x , _ty , t _z are the left camera reference frame to the right camera respectively The translation matrix parameter of the reference frame.

Further, the specific process of capturing the movement of the target in the step 3 is:

Step 3.1: In the first image captured by the left camera, frame the measurement point area, and use the Niblack algorithm to binarize the image;

Step 3.2: Calculate the equivalent eccentricity of the closed area in the binarized image and compare it with the set threshold;

Step 3.3: Use the Canny-Zernike algorithm to extract the sub-pixel edge of the circular target, fit the extracted sub-pixel edge to obtain the ellipse equation, and calculate the ellipse center coordinate from the ellipse equation, and use the ellipse center coordinate as the left camera measuring point. initial coordinates;

Step 3.4: Input the displacement of the estimated measuring point to obtain the search sub-area of each measuring point;

Step 3.5: Use the same method as Step 3.1 to Step 3.4 to complete the timing matching of the images captured by the right camera, and complete the matching of the images captured by the left camera and the right camera according to the measurement point number; The right camera image matches the coordinates p _l , _pr of the image point.

Further, the specific process of step 3.5 is, by collecting the internal parameter matrix A _l of the left camera, the internal parameter matrix A _r of the right camera, the left camera is converted to the rotation matrix R _l2r and translation vector of the right camera. T _l2r calculates the fundamental matrix F of the camera,

F=A _r ^-T [T _l2r ] _× R _l2r A _l ^-1

From the fundamental matrix F and the coordinates p _l and p _r of the matched image points, the equations l _l and l _r of the corresponding polar lines are obtained:

l _l = _Fpr

l _r = Fp _l

Calculate the distances d _l and d _r from the image point to the epipolar line from the equations l _l and l _r of the epipolar line and the coordinates p _l and _pr of the image point; The direction vectors of the line are D _l , D _r , and the coordinates p′ _l and p′ _r of the image point after optimization are:

Further, the specific process of the six-degree-of-freedom measurement method of the structure in step 5 is: firstly calculate the normalized coordinates of the optimized image point X _nl =[x _nl ,y _nl ] ^T ,X _nr =[x _nr ,y _nr ] ^T :

Then, the three-dimensional coordinates X=[x _w , y _w , z _w ] ^T of the measuring point in the left camera reference system are obtained:

In order to convert the coordinate system to the structural reference plane, the 3D coordinates P of the final survey point are obtained:

P=R ^-1 (XT)

After obtaining the three-dimensional coordinates of each measuring point, select three measuring points and name them as measuring point 1, measuring point 2, and measuring point 3, and obtain the initial three-dimensional coordinates of the three measuring points as Pi ( _i =1,2,3), The three-dimensional coordinates at a certain moment in the movement process are P _i ' (i=1, 2, 3); with measuring point 2 as the rotation center, the three-dimensional displacement and three-dimensional rotation angle of the structure are calculated from these three measuring points. The three-dimensional displacement and three-dimensional rotation angle are represented by the rotation matrix r and the translation vector t;

Therefore:

t=P' ₂ -P ₂

P′ _i =r·P _i +t

and get:

[P' ₁ -t, P' ₂ -t, P' ₃ -t]=r·[P ₁ , P ₂ , P ₃ ]

From the above formula, the rotation matrix r can be solved, and the rotation matrix r can be decomposed into roll α, pitch β and bow γ, thereby obtaining the six-degree-of-freedom measurement result of the structure.

The beneficial effects of the present invention are:

Using the visual measurement method, the visual measurement of the six-degree-of-freedom motion of the structure is effectively completed. The initial recognition and tracking of the circular target by the structural six-degree-of-freedom motion vision measurement system is realized, and the sub-pixel coordinates of the center of the circular target are obtained, which meets the requirements of high-precision measurement.

The final step in visual measurement is 3D reconstruction. The accuracy of 3D reconstruction directly affects the accuracy of measurement. Due to the accumulation of errors in the process of calibration and image target positioning, how to eliminate errors reasonably is the key content to improve the accuracy of three-dimensional measurement of spatial point coordinates. The invention optimizes the coordinates of the image point before performing the three-dimensional reconstruction, and can effectively improve the measurement accuracy.

Description of drawings

FIG. 1 is a schematic diagram of the hardware structure of the system of the present invention.

FIG. 2 is an interface diagram of the measurement software of the present invention.

Figure 3 is a flow chart of the measurement software of the present invention.

FIG. 4 is a diagram of the recognition result of the circular target in the first image of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

A measurement method for a structural six-degree-of-freedom motion measurement system based on binocular vision, the measurement system includes two cameras and a computer, the two cameras are connected with a computer through a network cable, and the computer is connected with a synchronization trigger , send the acquisition signal to the camera synchronously through the computer-controlled synchronization trigger;

The two cameras are mounted on a tripod, and the postures of the cameras are adjusted so that the center of the images of the two cameras is aligned with the structure to be measured.

A measurement method of a six-degree-of-freedom motion measurement system based on binocular vision, the configuration method of the measurement system comprises the following steps:

Step 1: Paste a circular target on the surface of the structure, take the center of the circular target as the measuring point, and arrange at least three measuring points; two cameras are arranged in front of the measuring point, and the two cameras are respectively set in the left front of the measuring point and the right front of the measuring point. The measuring point is within the field of view of the left camera and the right camera;

Step 2: Use the checkerboard calibration method to calibrate the visual measurement system; arrange the checkerboard calibration plate in the field of view, change the posture of the checkerboard calibration plate, and use the left camera and the right camera to collect multiple images for camera calibration , select when the calibration plate is close to the surface of the structure, the plane where the checkerboard calibration plate is located is the reference surface of the structure;

Step 3: Use two cameras to capture the movement of the target; collect moving images, complete the identification of the circular target in the first image, and track and capture the movement of the circular target during the movement, and obtain the measuring points on the images of the left camera and the right camera. the pixel coordinates of ;

Step 4: Perform image point coordinate optimization on the target motion captured in step 3; calculate the corresponding epipolar line of the matching image point between the left camera and the right camera from the calibration result, and optimize the image point coordinate by the epipolar line equation. image point coordinates;

Step 5: Complete the six-degree-of-freedom measurement of the structure; calculate the three-dimensional displacement of the measuring point from the optimized image point coordinates, select three measuring points to calculate the three-dimensional rotation angle of the structure, and complete the six-degree-of-freedom measurement of the structure.

Further, the step 2 is specifically: calculating the internal parameter matrix A _l of the left camera of the visual measurement system and the internal parameter matrix A _r of the right camera from the collected calibration image; the left camera reference system is converted to the right camera. The rotation matrix R _l2r of the reference frame and the translation vector T _l2r ; the structural reference plane is converted to the rotation matrix R and the translation vector T of the left camera reference frame;

The expressions of A _l , A _r , R _l2r and T _l2r are:

Further, the specific process of capturing the target movement in the step 3, that is, realizing the tracking and identification of the circular target, is:

Step 3.1: In the first image captured by the left camera, frame the measurement point area, and use the Niblack algorithm to binarize the image;

Step 3.2: Calculate the equivalent eccentricity of the closed area in the binarized image and compare it with the set threshold; if the closed area and an ellipse have the same area distance, the eccentricity of the ellipse is the ellipse of the closed area. the equivalent eccentricity. If the equivalent eccentricity of the closed area is less than the set threshold, the area will be removed; if the equivalent eccentricity of the closed area is greater than the set threshold, the closed area will be considered as a round target image;

Step 3.3: Use the Canny-Zernike algorithm to extract the sub-pixel edge of the circular target, fit the extracted sub-pixel edge to obtain the ellipse equation, and calculate the ellipse center coordinate from the ellipse equation, and use the ellipse center coordinate as the left camera measuring point. initial coordinates;

Step 3.4: Input the displacement of the estimated measuring point to obtain the search sub-area of each measuring point; use the IC-GN algorithm in the search sub-area to complete the timing matching of the images collected by the left camera;

Step 3.5: Use the same method as Step 3.1 to Step 3.4 to complete the timing matching of the images captured by the right camera, and complete the matching of the images captured by the left camera and the right camera according to the measurement point number; The right camera image matches the coordinates p _l , _pr of the image point.

Further, the specific process of step 3.5 is, by collecting the internal parameter matrix A _l of the left camera, the internal parameter matrix A _r of the right camera, the left camera is converted to the rotation matrix R _l2r and translation vector of the right camera. T _l2r calculates the fundamental matrix F of the camera,

F=A _r ^-T [T _l2r ] _× R _l2r A _l ^-1

From the fundamental matrix F and the coordinates p _l and p _r of the matched image points, the equations l _l and l _r of the corresponding polar lines are obtained:

l _l = _Fpr

l _r = Fp _l

Calculate the distances d _l and d _r from the image point to the epipolar line from the equations l _l and l _r of the epipolar line and the coordinates p _l and _pr of the image point; The direction vectors of the line are D _l , D _r , and the coordinates p′ _l and p′ _r of the image point after optimization are:

Further, the specific process of the six-degree-of-freedom measurement method of the structure in step 5 is: firstly calculate the normalized coordinates of the optimized image point X _nl =[x _nl ,y _nl ] ^T ,X _nr =[x _nr ,y _nr ] ^T :

Then, the three-dimensional coordinates X=[x _w , y _w , z _w ] ^T of the measuring point in the left camera reference system are obtained:

In order to convert the coordinate system to the structural reference plane, the 3D coordinates P of the final survey point are obtained:

P=R ^-1 (XT)

After obtaining the three-dimensional coordinates of each measuring point, select three measuring points and name them as measuring point 1, measuring point 2, and measuring point 3, and obtain the initial three-dimensional coordinates of the three measuring points as Pi ( _i =1,2,3), The three-dimensional coordinates at a certain moment in the movement process are P _i ' (i=1, 2, 3); with measuring point 2 as the rotation center, the three-dimensional displacement and three-dimensional rotation angle of the structure are calculated from these three measuring points. The three-dimensional displacement and three-dimensional rotation angle are represented by the rotation matrix r and the translation vector t;

Therefore:

t=P' ₂ -P ₂

P′ _i =r·P _i +t

and get:

[P' ₁ -t, P' ₂ -t, P' ₃ -t]=r·[P ₁ , P ₂ , P ₃ ]

From the above formula, the rotation matrix r can be solved, and the rotation matrix r can be decomposed into roll α, pitch β and bow γ, thereby obtaining the six-degree-of-freedom measurement result of the structure.

The equipment used in the system is a CCD camera, model AVTGX1050, using the GigEVision interface protocol. In order to apply to different measurement ranges, M3Z1228C-MP series zoom lens and LM5JCM series fixed focus lens are used. The camera image data is connected to a gigabit network card through an RJ-45 network cable interface, and the model of the gigabit network card is PCIe-PoE74. The manufacturer of the PCIe-PoE74 capture card is ADLINK Technology Co., Ltd. It can be connected to 4 Gigabit Ethernet cables at the same time to meet the data bandwidth of two cameras during high-speed capture. The computer model is a Dell OptiPlex7070 commercial desktop computer, the computer CPU is clocked at 3GHz, and the memory size is 32G. The frame grabber is connected to the computer through the PCIe interface on the computer motherboard.

The interface of the six-degree-of-freedom motion vision measurement software is shown in Figure 2. Use the measurement software to control the camera to complete the six-degree-of-freedom motion measurement. The six-degree-of-freedom structural motion visual measurement software can realize the functions of image acquisition, real-time image display and storage, structural six-degree-of-freedom motion analysis and calculation result output.

Figure 3 shows the process of visual measurement of structural six-degree-of-freedom motion. According to the measurement needs, the structural six-degree-of-freedom motion vision measurement software is mainly divided into four modules, namely image acquisition, camera calibration, target tracking, and result output. There are corresponding operation buttons and input boxes under each module. Users can input corresponding parameters according to actual conditions, and then the measurement results of six free motions can be obtained.

Claims (4)

1. A measuring method of a structure six-degree-of-freedom motion measuring system based on binocular vision is characterized in that the measuring system comprises two cameras and a computer, the two cameras are connected with the computer through a network cable, the computer is connected with a synchronous trigger, and the synchronous trigger is controlled by the computer to synchronously send acquisition signals to the cameras;

the two cameras are arranged on a tripod, so that the picture centers of the two cameras are aligned to the structure to be detected;

the configuration method of the measuring system comprises the following steps:

step 1: pasting a round target on the surface of the structure, and arranging at least three measuring points by taking the center of the round target as the measuring points; two cameras are arranged in front of the measuring point, the two cameras are respectively arranged in the left front of the measuring point and the right front of the measuring point, and the measuring point is in the visual fields of the left camera and the right camera;

step 2: calibrating the vision measuring system by using a checkerboard calibration method; arranging the chessboard pattern calibration plate in a view field, and selecting a plane where the chessboard pattern calibration plate is positioned as a structure reference surface when the calibration plate is tightly attached to the structure surface;

and step 3: capturing target motion with two cameras; collecting a moving image, completing the identification of the circle target in the first image, tracking and capturing the motion of the circle target in the motion process, and obtaining pixel coordinates of a measuring point on the left camera image and the right camera image;

and 4, step 4: carrying out image point coordinate optimization on the target motion captured in the step 3; calculating corresponding polar lines of matched image points of the left camera and the right camera according to the calibration result, and optimizing the coordinates of the image points by a polar line equation to obtain optimized coordinates of the image points;

the specific process of the step 4 is that an internal parameter matrix A of the left camera is acquired_lInner parameter matrix A of the right-hand camera_rLeft-hand camera to right-hand camera rotation matrix R_l2rAnd a translation vector T_l2rCalculating to obtain a basic matrix F, F ═ A of the camera_r ^-T[T_l2r]_×R_l2rA_l ^-1

From the basic matrix F and the coordinates p of the matching image points_l、p_rSolving the equation l corresponding to the polar line_l、l_r：

l_l＝Fp_r

l_r＝Fp_l

Equation l from polar line_l、l_rCoordinates p with the image point_l、p_rDetermining the distance d from the image point to the epipolar line_lAnd d_r(ii) a The left camera and the right camera cross the image point to be taken as the vertical line of the polar line, and the direction vector pointing to the polar line from the image point is D_l，D_rCoordinates p 'of post-optimization image points'_l、p′_rComprises the following steps:

and 5: completing the six-degree-of-freedom measurement of the structure; and calculating the three-dimensional displacement of the measuring points by the optimized image point coordinates, and selecting three measuring points to calculate the three-dimensional corner of the structure to finish the six-degree-of-freedom measurement of the structure.

2. The measurement method according to claim 1, wherein step 2 is to calculate an intrinsic parameter matrix A of a left camera of the vision measurement system from the acquired calibration image_lInner parameter matrix A of the right-hand camera_r(ii) a Left camera reference frame rotationRotation matrix R transformed to the reference frame of the right camera_l2rAnd a translation vector T_l2r(ii) a Converting the structural reference surface into a rotation matrix R and a translation vector T of a left camera reference system;

A_l、A_r、R_l2rand T_l2rThe expression of (c) is:

wherein,

respectively equivalent focal lengths of the left camera on the u and v axes of an image plane,

the equivalent focal lengths of the right camera on the u and v axes respectively, (u)_l,v_l) And (u)_r,v_r) The coordinates of the left camera principal point and the right camera principal point in the image plane, r_i(i 1,2.., 9) is a rotation matrix parameter from the left camera reference frame to the right camera reference frame, t_x,t_y,t_zThe parameters of the translation matrix from the left camera reference frame to the right camera reference frame are respectively.

3. The measurement method according to claim 1, wherein the step 3 of capturing the target motion comprises the following specific processes:

step 3.1: selecting a point area in a first image collected by a left camera, and binarizing the image by using a Niblack algorithm;

step 3.2: calculating the equivalent eccentricity of a closed area in the binary image, and comparing the equivalent eccentricity with a set threshold value;

step 3.3: extracting sub-pixel edges of the circular target by adopting a Canny-Zernike algorithm, fitting the extracted sub-pixel edges to obtain an elliptic equation, calculating by using the elliptic equation to obtain an elliptic center coordinate, and taking the elliptic center coordinate as an initial coordinate of a left camera measuring point;

step 3.4: inputting the displacement of the predicted measuring points to obtain a search subarea of each measuring point;

step 3.5: completing the time sequence matching of the images acquired by the right camera by using the same method as the steps from 3.1 to 3.4, and completing the matching of the images acquired by the left camera and the right camera according to the measuring point numbers; thereby obtaining the coordinates p of the matched image points of the left camera image and the right camera image_l、p_r。

4. The measurement method according to claim 1, wherein the step 5 structure six-degree-of-freedom measurement method comprises the following specific processes: firstly, calculating the normalized coordinate X of the optimized image point_nl＝[x_nl,y_nl]^T,X_nr＝[x_nr,y_nr]^T：

Then, three-dimensional coordinates X ═ X [ X ] of the measuring points in the reference system of the left camera is obtained_w,y_w,z_w]^T：

To convert the coordinate system to the structural reference surface, the three-dimensional coordinates P of the final measurement point are obtained:

P＝R^-1(X-T)

after the three-dimensional coordinates of each measuring point are obtained, three measuring points are selected and named as measuring point 1, measuring point 2 and measuring point 3, and the initial three-dimensional coordinates of the three measuring points are obtained and are P_i(i is 1,2,3), and the three-dimensional coordinate at a certain moment in the motion process is P_i' (i ═ 1,2, 3); to be provided withMeasuring point 2 is a rotation center, three-dimensional displacement and three-dimensional rotation angle of the structure are calculated by the three measuring points, and the three-dimensional displacement and the three-dimensional rotation angle are expressed by using a rotation matrix r and a translation vector t;

this gives:

t＝P′₂-P₂

P′_i＝r·P_i+t

further obtaining:

[P′₁-t，P′₂-t，P′₃-t]＝r·[P₁，P₂，P₃]

the rotation matrix r can be solved by the above formula, and the rotation matrix r is decomposed into rolling alpha, pitching beta and yawing gamma, thereby obtaining the six-degree-of-freedom measurement result of the structure.

Images