CN115311358A - Long-distance continuous pose high-speed video measurement method - Google Patents

Abstract

The invention relates to a long-distance continuous pose high-speed video measurement method, which comprises the following steps: step 1, resolving an observed value by adopting a multi-binocular continuous observation combined beam method adjustment; step 2, processing the resolving process in the step 1) by adopting an adaptive strain weight strategy of depth distance and difference residual constraint to obtain a continuous pose parameter measurement result; and 3, optimizing the continuous pose parameter measurement result obtained in the step 2) by adopting dynamic inversion of position and pose parameters based on plane fitting. Compared with the prior art, the method has the advantages that the high-precision continuous measurement of the dynamic position and attitude parameters of the long-distance moving target can be realized, the measurement positioning precision is better than 3mm, and the precision is improved by over 30% compared with the precision of international famous vision measurement software.

Description

A high-speed video measurement method for long-distance continuous pose and orientation

technical field

The invention relates to a dynamic parameter measurement technology of an aerospace engineering model, in particular to a long-distance continuous pose high-speed video measurement method.

Background technique

With the rapid development of aerospace engineering, the high-precision measurement of long-distance continuous position and attitude parameters in ground verification test experiments has become a frontier research topic. Using traditional contact measurement methods, such as fixing contact sensors such as displacement meters, accelerometers, and strain gauges on the surface of the measurement target to obtain information such as displacement, acceleration, and strain, such methods have limited range, small measurement areas, and even damage. Model structure and other defects. In order to overcome the above defects, non-contact sensors represented by cameras and laser sensors are adopted, and laser technology has defects such as single measuring point, high price, and low acquisition frequency. Therefore, the camera is used to record the changing state of the moving object in detail, and then The measurement method of obtaining the dynamic parameters of the object to be measured is more widely used by accurately measuring the three-dimensional space coordinate change and three-dimensional deformation of the target feature point through the photogrammetry analysis method.

In the process of measuring dynamic parameters of aerospace engineering models, due to the long moving process of the target in the depth direction, the relative distance between the target and the camera in different observation segments is different. attitude, resulting in low overall positioning accuracy and inconsistent observation accuracy at long distances.

Contents of the invention

The purpose of the present invention is to provide a long-distance continuous pose high-speed video measurement method in order to overcome the above-mentioned defects in the prior art.

The purpose of the present invention can be achieved through the following technical solutions:

According to one aspect of the present invention, a long-distance continuous pose high-speed video measurement method is provided, the method comprising the following steps:

Step 1, using multi-binocular continuous observation joint beam adjustment to solve the observation value;

Step 2, using the adaptive variable weight strategy of depth distance and difference residual constraints to process the solution process of step 1), and obtain continuous pose parameter measurement results;

In step 3, the continuous pose parameter measurement results obtained in step 2) are optimized by using the dynamic inversion of position and pose parameters based on plane fitting.

As a preferred technical solution, the step 1, multi-binocular continuous observation joint beam adjustment process is specifically:

101. In photogrammetry, the principle of least squares is used to minimize the cumulative sum of squares of the residual distance between the back projection and the original image point, which is expressed in the form of the following least squares problem:

Where (u _i,j ,v _i,j ) is the pixel coordinate of the jth three-dimensional point with distortion obtained from the actual observation on the image plane captured by the i-th camera; π(Cam _i ,P _j )=(u _{(i ,j)distorted} ,v _{(i,j)distorted} ), which means that the jth three-dimensional point in the object space is transformed by the perspective transformation of the inner and outer square matrix of the i-th camera, and the lens distortion parameter offset of the camera is taken into account. Backprojection pixel coordinates, e, m, n are the backprojection residual, the number of cameras and the number of 3D points respectively;

102. Use formula (1) to describe the long-distance continuous pose calculation process as multiple independent beam adjustment processes, and the expression is changed to:

Among them, BA _k represents the local beam adjustment process of the k-th segment, and m _k represents the camera set and the observed object-space 3D point set in the binocular vision measurement system of the k-th segment, respectively.

103, using the joint beam method to adjust the solution and improve the formula (2) to obtain:

Among them, m′ _k-1 represents the object-space three-dimensional point set repeated by the camera in the binocular vision measurement system of the k-th segment and repeated in the binocular vision measurement system of the k-1 segment, and BA _Global represents the joint beam adjustment process.

As a preferred technical solution, the adaptive weighting strategy for depth distance and adjustment residual constraints in step 2 includes introducing iterative reweighting and introducing a depth change weighting strategy in the adjustment process.

As a preferred technical solution, the introduction of iterative reweighting is: continuously adjusting the weight of a certain observation value in the iterative solution of bundle adjustment optimization optimization.

As a preferred technical solution, the weight of constantly adjusting a certain observation value in the iterative solution of beam adjustment optimization optimization is specifically:

Based on the Huber robust loss function, the following iterative reweighting operation is performed on an observation value before each iteration optimization:

where w _r(k) represents the robust weight iteratively added to the k-th observation value, and r _k represents the bi-norm of the back-projection error calculated by the error equation for the k-th observation value, that is, r _k ＝‖e _k ‖ ₂ , δ represents the outlier threshold.

As a preferred technical solution, the abnormal point threshold value is 0.001.

As a preferred technical solution, the introduction of the depth change weighting strategy in the adjustment process is as follows: the reciprocal of the depth distance between the observed values at different depths and the baseline of the binocular vision system is used as the weighting basis to fully consider the It is an objective fact that the larger point depth value leads to lower observation accuracy.

As a preferred technical solution, the described way of determining the weight is specifically:

where w _z(k) represents the depth change weight of the k-th observation, D _k represents the depth distance between the k-th observation and the baseline of the binocular vision system, and D _δ represents the depth distance threshold. The above formula shows that only for a specific depth Observations outside the range are weighted.

As a preferred technical solution, the D _δ is set as the depth distance between the spatial intersection of the optical axes of the two cameras and the baseline of the binocular vision system, and this distance can be easily calculated by the intersection of the initial parameters of the two cameras.

As a preferred technical solution, described step 3 is specifically:

With reference to the three-dimensional coordinates of the four artificial markers given in the initial state, the relationship between the rotation and translation of the fitting plane relative to the initial state at each moment can be expressed by the rigid body transformation relationship between the four marker points as:

where R _k and t _k are the translation and rotation parameters of the model surface relative to the reference initial state at time k, respectively, and X ₀ , Y ₀ , Z ₀ , X _k , Y _k , and Z _k are the three-dimensional coordinate.

Compared with the prior art, the present invention can realize high-precision continuous measurement of the dynamic position and attitude parameters of long-distance moving targets, and the measurement and positioning accuracy is better than 3mm, which is more than 30% higher than that of internationally renowned visual measurement software.

Description of drawings

Figure 1 is a schematic diagram of the joint bundle adjustment method;

Figure 2 is a schematic diagram of dynamic inversion of position and attitude parameters based on plane fitting;

Figure 3 is a schematic diagram of the layout of the multi-binocular vision measurement system;

Fig. 4 is a schematic diagram of a method for laying out a three-dimensional control field in space;

Fig. 5 is a schematic diagram of the calculation results of the pose parameters at 20° upward;

Figure 6 is a schematic diagram of the calculation results of the pose parameters at 20° to the right;

Fig. 7 is a schematic diagram of the calculation results of left and right nutation pose parameters.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

1. Outline of the invention

With the rapid development of aerospace engineering, it has become a frontier research topic to measure the dynamic pose parameters of aerospace models with high precision in ground verification test experiments. In order to meet the high-precision non-contact measurement requirements of long-distance continuous pose parameters, a long-distance continuous pose estimation method based on adaptive variable weight combined with beam adjustment is proposed. The multi-binocular vision measurement system long-distance continuous observation joint beam adjustment method adopts the adaptive variable weight least squares adjustment strategy, and combines the dynamic inversion technology of position and attitude parameters based on plane fitting to optimize the accuracy of dynamic parameter measurement results. precision. In order to verify the reliability of the proposed long-distance pose parameter measurement method, this patent carried out a satellite rendezvous and ground test high-speed video pose measurement experiment. The experimental results show that using the measurement method proposed in this patent can realize the dynamic position and attitude of long-distance moving targets High-precision continuous measurement of parameters, measurement and positioning accuracy is better than 3mm, which is more than 30% higher than that of internationally renowned visual measurement software.

2. Long-distance continuous pose high-speed video measurement method

The long-distance continuous pose high-speed video measurement method proposed in this patent mainly includes two parts: the depth distance and the residual constraint weighting of the long-distance continuous observation combined beam adjustment method and the dynamic inversion of position and attitude parameters based on plane fitting. .

2.1 Multi-binocular continuous observation joint beam adjustment

In photogrammetry, the least squares principle is used to minimize the cumulative sum of squares of the distance between the back projection and the original image point, that is, the smallest sum of squares of the back projection error. This method is called Bundle Adjustment, which can be as follows The form of the least squares problem is expressed as:

In the formula, (u _i,j ,v _i,j ) is the distorted pixel coordinates of the jth three-dimensional point actually observed on the image plane captured by the i-th camera; π(Cam _i ,P _j )=(u _{(i,j)distorted} ,v _{(i,j)distorted} ), which means that the jth 3D point in the object space is transformed by the perspective transformation of the inner and outer square matrix of the i-th camera, and the lens distortion parameter offset of the camera is considered. The resulting backprojected pixel coordinates.

In the long-distance continuous pose measurement, the traditionally commonly used multi-camera segmental measurement method uses the observation of each segment of the binocular vision measurement system as a separate calculation unit, and performs local beam adjustment in each segment of the observation range to obtain The external parameters of the stereo camera and the spatial three-dimensional information of the target point to be measured are finally stitched together to obtain the long-distance whole-process calculation results. The above-mentioned calculation process can be described as multiple independent beams by using formula 2.1 Normal adjustment process:

In the formula, BA _k represents the local beam adjustment process of the k-th segment, and m _k represents the camera set and the observed object-space 3D point set in the binocular vision measurement system of the k-th segment, respectively. The use of this calculation method will cause the poses of the individual stereo pairs to be independent of each other, and it cannot be guaranteed that they are consistent with the actual relative poses in the overall space coordinate system, resulting in a lack of integrity in the spatial three-dimensional information of the target points to be measured. There will be discontinuities in the splicing of the global results, which will eventually make the settlement results not robust.

In view of the fact that the binocular vision measurement system of the latter stage will overlap the observation range of the binocular vision system of the previous stage at a far distance in the depth direction (as shown in Figure 1), this patent proposes a joint global beam adjustment method for multi-binocular continuous observation The solution strategy makes full use of the constraint information provided by adjacent repeated observations.

Based on the above strategy, the joint bundle adjustment solution can be obtained by improving formula 2.2:

In the formula, m′ _k-1 represents the object-space three-dimensional point set repeated by the camera in the binocular vision measurement system of the k-th segment and that in the binocular vision measurement system of the k-1st segment.

2.2 Adaptive weighting strategy for depth distance and difference residual constraints

The proposed adaptive variable weight least squares adjustment strategy is based on the idea of iterative reweighting and depth change weighting, and uses depth distance and adjustment residual constraints to perform adaptive weighting. The basic idea of introducing the iterative reweighted least squares method is to continuously adjust the weight of a certain observation value in the iterative solution of bundle adjustment optimization optimization, so as to reduce the influence of abnormal points on the final global solution result. This patent is based on the Huber robust loss function, and performs the following iterative reweighting operation on a certain observation value before each iteration optimization:

In the formula, w _r(k) represents the robust weight iteratively added to the k-th observation value, and r _k represents the bi-norm of the back-projection error calculated by the error equation of the k-th observation value, that is, r _k ＝∥e _k ‖ ₂ , δ represents the outlier threshold (the commonly used experience value of 0.001 is chosen in this patent).

In addition, the baseline B and the camera focal length f are constant values. Assuming that the parallax deviation is Δd, considering the influence of the depth Z itself (that is, the depth range) on the measurement accuracy, it can be deduced that:

From the formula 2.5, it is easy to know that the distance between the target and the lens is different, and the measurement accuracy is not the same. In the depth direction of the observation field of view, the deeper the point with the same name, the smaller the parallax in the stereo pair, that is, the pixel coordinates of the observation point at a deeper depth on the stereo pair slightly change, It will cause a large deviation in the depth direction.

Based on the above-mentioned relevant observation experience and theoretical research basis, this patent proposes to introduce a depth change weighting strategy in the adjustment process, and the reciprocal of the depth distance between the observation values at different depths and the baseline of the binocular vision system is used as the weighting basis. Fully consider the objective fact that the observation accuracy is reduced due to the large depth value of the target point, and avoid it from greatly affecting the iterative optimization process of the combined beam adjustment, so that the overall adjustment results are more accurate and stable.

In view of the fact that most of the observed depths are within the relatively high-precision observation range during specific observation tasks, and only some distant observation points may have the above-mentioned errors, this patent adopts the following weighting method:

In the formula, w _z(k) represents the depth change weight of the k-th observation value, D _k represents the depth distance between the k-th observation value and the baseline of the binocular vision system, and D _δ represents the depth distance threshold. The above formula shows that only for Observations outside a specified depth range are weighted. According to measurement experience, the binocular vision measurement system has generally the best measurement accuracy at the intersection of the optical axes of the two cameras, so this patent sets D _δ as the distance between the intersection of the optical axes of the two cameras and the baseline of the binocular vision system The depth distance of , which can be easily calculated by the intersection of the initial parameters of the two cameras.

In summary, the proposed adaptive variable weight least squares adjustment strategy combines the robust weights updated during each iterative adjustment process, and the depth change weight determined by the spatial three-dimensional coordinates of all observation points. Work together on the above joint bundle adjustment process.

2.3 Dynamic inversion of position and attitude parameters based on plane fitting

In general, the target model can be approximated as an ideal rigid body, and the dynamic posture of the model at different moments during the motion process can be equivalent to the rigid body rotation and translation relationship between the posture and the reference initial state, that is, the body composed of the model surface The rotation and translation parameters of the coordinate system relative to the reference initial state at each moment are substitute quantities. The surface of the model in the real scene is relatively flat, and the plane fitted by the four artificial marker points pasted on the surface is equivalent to replace the entire model surface, so the translation and rotation attitude angle parameters of the model relative to the reference initial state at each moment It can be replaced by the rotation and translation relationship between the planes fitted by artificial signs, that is, the problem of solving the dynamic parameters of the model is transformed into the dynamic inversion of the position and attitude parameters of the planes fitted by artificial signs. The schematic diagram of the principle is shown in Figure 2 .

With reference to the three-dimensional coordinates of the four artificial markers given in the initial state, the relationship between the rotation and translation of the fitting plane at each moment relative to the initial state can be expressed by the rigid body transformation relationship between the four marker points:

In the formula, R _k and t _k are the translation and rotation parameters of the model surface relative to the reference initial state at time k, respectively. There are 6 unknown parameters in total. Based on the least squares fitting principle, each moment can be obtained by high-speed video measurement The three-dimensional coordinate relationship corresponding to the four artificial marker points is solved to obtain R _k and t _k , and then R _k is calculated according to the order of the "ZYX" rotation axis to recover the corresponding roll angle roll, pitch angle yaw, and yaw angle pitch.

3. Experiment and experimental results

3.1 Layout of multi-binocular vision measurement system

Taking the satellite rendezvous and docking ground surface test high-speed video pose measurement as the specific experimental scene, the satellite rendezvous and docking model will slide forward about 45m along the guide rail of the ground test site. A total of 3 pairs of binocular vision measurement systems were deployed at intervals of 15m to record the entire experimental conditions. The baseline length between the two cameras in each individual binocular vision measurement system was calculated to be about 6.75m, and the length of each camera The angle between the optical axis and the baseline is set to about 67° to ensure that each individual binocular vision measurement system can clearly observe and record the movement changes of objects within a depth of field of about 15m in the effective field of view ahead. The specific measurement The layout of the system is shown in Figure 3.

In order to provide the coordinate system benchmark for the calculation of the camera's external attitude in the multi-binocular vision measurement system, a stable three-dimensional space control field needs to be deployed within the observation field of view. The specific layout is shown in Figure 4. The real three-dimensional space coordinates of all control points are obtained by observation with a total station before the experiment (accuracy is submillimeter level).

3.2 Accuracy analysis of high-speed video measurement system

In order to verify the measurement and positioning accuracy of the high-speed video measurement pose estimation method proposed in this patent, the control points in the three-dimensional control field can be divided into two parts, that is, one part is used as a stable control point to participate in the joint beam adjustment calculation, and the other part is used as Checkpoints participate in the accuracy assessment. In addition, in order to verify the effectiveness of the multi-binocular vision measurement system combined beam adjustment method proposed in this patent with the adaptive variable weight least squares strategy, this patent also cooperates with the segmental solution method of the internationally renowned visual measurement software Photomodeler The comparison was carried out, and the specific results are shown in Table 1.

Table 1

From the comparison of the measurement accuracy of the check points in the control field, it can be seen that the root mean square error of the measurement accuracy obtained based on the combined beam adjustment strategy proposed in this patent is less than 2mm in all directions, and the total positioning accuracy It is about 3mm, which is equivalent to 0.2mm/m at a camera field of view of about 15m. Compared with the segmental calculation method of the internationally renowned visual measurement software Photomodeler, the positioning accuracy has increased by 34.72% in the X, Y, and Z directions. %, 36.53% and 19.88%, the total positioning accuracy has increased by 31%, which proves the effectiveness of the adaptive variable weight joint beam adjustment pose estimation method proposed in this patent, and also verifies that high-speed video measurement can meet satellite rendezvous Accuracy requirements for long-distance pose measurement for ground surface testing.

3.3 Analysis of model pose parameter calculation results

Using the camera parameters restored by the combined beam adjustment strategy proposed in this patent, the three-dimensional coordinates of the four tracking points on the model surface at each moment in space can be accurately calculated, and then the dynamic reflection of position and attitude parameters based on plane fitting can be used The result of dynamic parameter calculation is obtained by performing technique. This patent lists the calculation results of the pose parameters of several satellite rendezvous and docking models in different motion states. "Moving" 3 kinds of motion states.

From the above high-speed video measurement dynamic parameter calculation results, it can be seen that the dynamic inversion of position and attitude parameters based on plane fitting can more accurately represent the actual motion state of the model, and the change range of the attitude angle is the same as the set 20° change range. Maintain a consistent and stable periodic change, and the overall position and attitude inversion results are relatively smooth.

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of various equivalents within the technical scope disclosed in the present invention. Modifications or replacements shall all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (10)

1. A long-distance continuous pose high-speed video measurement method is characterized in that the method may further comprise the steps: Step 1, using multi-binocular continuous observation joint beam adjustment to solve the observation value; Step 2, using the adaptive variable weight strategy of depth distance and difference residual constraints to process the solution process of step 1), and obtain continuous pose parameter measurement results; In step 3, the continuous pose parameter measurement results obtained in step 2) are optimized by using the dynamic inversion of position and pose parameters based on plane fitting. 2. A kind of long-distance continuous pose high-speed video measurement method according to claim 1, is characterized in that, described step 1, multi-binocular continuous observation joint beam adjustment process is specifically: 101. In photogrammetry, the principle of least squares is used to minimize the cumulative sum of squares of the residual distance between the back projection and the original image point, which is expressed in the form of the following least squares problem:

Where (u _{i, j} , v _{i, j} ) is the pixel coordinates of the jth three-dimensional point with distortion that is actually observed on the image plane captured by the i-th camera; π(Cam _i , P _j )=(u _{(i , j)distorted} , v _{(i, j)distorted} ), which means that the jth three-dimensional point in the object space is transformed by the perspective transformation of the inner and outer square matrix of the i-th camera, and the lens distortion parameter offset of the camera is taken into account. Backprojection pixel coordinates, e, m, n are the backprojection residual, the number of cameras and the number of 3D points respectively; 102. Use formula (1) to describe the long-distance continuous pose calculation process as multiple independent beam adjustment processes, and the expression is changed to:

Among them, BA _k represents the local beam adjustment process of the k-th segment, and m _k represents the camera set and the observed object-space 3D point set in the binocular vision measurement system of the k-th segment respectively; 103, using the joint beam method to adjust the solution and improve the formula (2) to obtain:

Among them, m′ _k-1 represents the object-space three-dimensional point set repeated by the camera in the binocular vision measurement system of the k-th segment and repeated in the binocular vision measurement system of the k-1 segment, and BA _Global represents the joint beam adjustment process. 3. A kind of long-distance continuous pose high-speed video measurement method according to claim 1, it is characterized in that, the self-adaptive weight-changing strategy of depth distance and difference residual constraint in described step 2 comprises introducing iterative reweighting And introduce a depth change weighting strategy in the adjustment process. 4. A kind of long-distance continuous pose high-speed video measurement method according to claim 3, characterized in that, the introduction of iterative re-weighting is: constantly adjusting a certain observation value in the iterative solution of beam adjustment optimization weight. 5. A kind of long-distance continuous pose high-speed video measurement method according to claim 4, it is characterized in that, described in the beam adjustment optimization iterative solution, the weight of constantly adjusting a certain observation value is specifically: Based on the Huber robust loss function, the following iterative reweighting operation is performed on an observation value before each iteration optimization:

where w _r(k) represents the robust weight of the iterative re-addition of the k-th observation value, and r _k represents the bi-norm of the back-projection error calculated by the error equation for the k-th observation value, that is, r _k =||e _k || ₂ , δ represents the outlier threshold. 6. A long-distance continuous pose high-speed video measurement method according to claim 5, characterized in that, the value of the abnormal point threshold is 0.001. 7. A kind of long-distance continuous pose high-speed video measurement method according to claim 3, characterized in that, the introduction of depth change weighting strategy in the adjustment process is as follows: the observation values of different depths are calculated according to their relationship with double The reciprocal of the depth distance between the baselines of the visual system is used as the weighting basis to fully consider the objective fact that the observation accuracy is reduced due to the large depth value of the target point. 8. A kind of long-distance continuous pose high-speed video measurement method according to claim 7, is characterized in that, described weighting mode is specifically:

where w _z(k) represents the depth change weight of the k-th observation, D _k represents the depth distance between the k-th observation and the baseline of the binocular vision system, and D _δ represents the depth distance threshold. The above formula shows that only for a specific depth Observations outside the range are weighted. 9. A kind of long-distance continuous pose high-speed video measurement method according to claim 7, it is characterized in that, described Dδ is set as the depth distance between two camera optical axis space intersections to the binocular vision system baseline , the distance can be easily calculated by the intersection of the initial parameters of the two cameras. 10. A kind of long-distance continuous pose high-speed video measurement method according to claim 1, is characterized in that, described step 3 is specifically: With reference to the three-dimensional coordinates of the four artificial markers given in the initial state, the relationship between the rotation and translation of the fitting plane relative to the initial state at each moment can be expressed by the rigid body transformation relationship between the four marker points as:

where R _k and t _k are the translation and rotation parameters of the model surface relative to the reference initial state at time k, respectively, and X ₀ , Y ₀ , Z ₀ , X _k , Y _k , and Z _k are the three-dimensional coordinate.

Images