CN109579695B - Part measuring method based on heterogeneous stereoscopic vision - Google Patents

Abstract

The invention proposes a method for measuring parts based on heterogeneous stereo vision. The measurement method adopts one telecentric industrial camera and two ordinary industrial cameras, and performs correlation processing on the images captured by the cameras, so that the binocular stereo vision and the The monocular telecentric vision is effectively combined, and the principle error of the ordinary optical system is compensated by telecentric imaging, and the advantages of binocular vision and the high measurement accuracy of telecentric vision are integrated into one, so as to solve the problem of low 3D non-contact measurement accuracy on the production site. It has the technical effect of high data reliability and reliable measurement results.

Description

A Parts Measurement Method Based on Heterogeneous Stereo Vision

technical field

The invention relates to the field of part size measurement, in particular to a part measurement method based on heterogeneous stereo vision.

Background technique

The research on parts inspection based on machine vision began to emerge in the 1990s, and now it has gradually entered various industrial fields, and the measurement methods and methods have also developed rapidly. Machine vision is to use machines instead of human eyes to make measurements and judgments. The image capture device CMOS or CCD converts the captured target into an image signal, and transmits it to a dedicated image processing system. According to pixel distribution, brightness, color and other information, it is converted into an image signal. Digitized signals; the image system performs various operations on these signals to extract the characteristics of the target, and then controls the on-site equipment actions according to the results of the discrimination.

Prior art 1:

Inspired by the human eyes perceiving the depth of the environment through the principle of parallax, the binocular stereo vision measurement system simultaneously observes the measured part from two points in space (as shown in Figure 1), and obtains two images of the measured part under different viewing angles. The registration relationship between the pixels between the two images, the positional deviation between the pixels is calculated by the principle of triangulation, the depth information of any point in the three-dimensional space is solved, and finally the three-dimensional topography of the measured part is reconstructed, and the multi-element measurement of the part is carried out. .

In response to the measurement of the stepped axis, Wu Xiang of Changchun University of Technology established a set of binocular vision measurement system, which can measure multiple elements of the measured part. The experimental results show that the minimum and maximum measurement errors of the binocular measurement system are 0.1mm and 0.9mm respectively (Wu Xiang. Research on key technologies of part size measurement system based on binocular vision [D]. Changchun: Changchun University of Technology, 2017. ). Zhang Junyong of Wuhan University of Science and Technology proposed a multi-dimension measurement method and system for parts based on binocular vision. Through a series of processes such as calibration, epipolar correction, epipolar constraint matching, 3D fitting, and 3D reconstruction, the measured parts are multi-dimensional. Three-dimensional measurement of elements (Zhang Junyong, Wu Shiqian, Xu Lihong. A multi-dimension measurement method and system for parts based on binocular vision: China, CN107588721A[P]. 2018.01.16.). Zhao Yan of Beihang University, etc. focused on the three-dimensional measurement of narrow space targets, and proposed a narrow space binocular vision measurement and positioning device and method, (Zhao Yan, Su Qinghua, Wu Falin, Yang Kui, Zhang Shaochen. A narrow space binocular vision measurement Positioning device and method: China, ZL201210191014.7[P].2012.11.05.)

Binocular vision measurement is a method based on the principle of bionics. It has many advantages and is theoretically very suitable for application in on-line, non-contact geometric accuracy inspection and quality control in manufacturing sites. However, in fact, the application of binocular vision is not as good as that of other methods, and the most critical constraint is its measurement accuracy. The accuracy of most binocular systems is only sub-millimeter level.

Prior art 2:

The theoretical basis of structured light vision measurement is the principle of laser triangulation. By projecting the structural light source known by the spatial mathematical model to the measured part, the machine vision system projects the image of the part and the structured light (as shown in Figure 2), and the spatial mathematical model of the structured light and the imaging mathematical model of the machine vision system are simultaneously solved. The recovery depth information can be obtained. There are four types of light projection modes: light, plane, curved surface and beam. Therefore, there are also four types of common structured light vision measurement systems.

Xiong Weinan of Harbin Institute of Technology established a structured light vision measurement system. When the object distance is 250mm, the measurement range of the system is not less than 200mm×200mm. The system was used to perform repeated measurements on standard gauge blocks with thickness and width of 8.845mm and 20.000mm, respectively. In the thickness measurement experiment, the maximum and minimum measurement errors are 50 μm and 30 μm, respectively; in the width measurement experiment, the maximum and minimum measurement errors are 50 μm and 18 μm, respectively (Xiong Weinan. Development of 3D Structured Light Vision Measurement Equipment [D]. Harbin: Harbin University of Technology, 2017.). Zhang Congpeng of Hubei University of Technology, etc. invented a method and device for measuring the chamfer of parts. The method mainly includes the process of calibrating the structured light vision system by the plane model method, feature image acquisition and preprocessing, feature extraction and feature parameter calculation. It has the application characteristics of high precision, high adaptability and high efficiency (Zhang Congpeng, Hou Bo, Cao Wenzheng, Lu Lei, Ren Shiyu. A method and device for measuring the chamfer of parts based on structured light vision: China, CN105783786A[P]. 2016.07 .20.). The structured light measurement method and its device invented by He Wantao of Heilongjiang University of Science and Technology focus on solving the problem that high reflectivity parts are difficult to measure (He Wantao, Guo Yanyan, Che Xiangqian, etc. A three-dimensional measurement of surface structured light for high reflectivity parts Apparatus and Methods: China, ZL201310717211.2[P].2016.09.07.). Tan Qingchang et al. of Jilin University used a structured light vision system to measure the radial runout error of shaft parts. Based on the establishment of a structured light vision model of radial runout error, the two-step calibration method of Zhang Zhengyou plane and the template matching method were used to measure the visual sensor and structured light. After calibration, the spatial coordinates of the intersection of the structured light and the surface of the part are solved through the measurement model (Tan Qingchang, Bao Haojing, Zhang Yachao, et al. On-line measurement method of radial runout error of shaft parts based on structured light vision: China, CN107101582A [P ].2017.08.29.).

Structured light vision and its improved methods both use light sources with known poses for active illumination, and only need to extract features from one image to restore image depth information, avoiding the registration of left and right images in binocular vision. Therefore, the measurement accuracy of structured light vision has been significantly improved compared with binocular vision. Nevertheless, the measurement accuracy of structured light vision still remains at the silk level. Moreover, compared with binocular vision, the point cloud density of structured light vision is significantly reduced, which is not conducive to the 3D reconstruction of small features of parts. In addition, the calibration of structured light and high-performance lasers and their cost remain to be studied.

Prior art 3:

Polyocular vision and binocular vision have the same theoretical basis and working principle, and are formed by adding multiple imaging sensors to the original binocular vision. Multi-eye vision simultaneously observes the measured part from three or more points in the space, and obtains multiple images of the measured part under different visions. The mathematical model of the depth of any point can then reconstruct the three-dimensional appearance of the measured part and carry out multi-element measurement of the part. Compared with binocular vision, multi-eye vision introduces more constraints and theoretically has higher measurement accuracy. The simplest form of multi-eye vision is trinocular vision, and its use in industry is on the rise.

Ye et al. introduced a third camera into the binocular system to form a trinocular vision measurement system, adding more constraints for stereo matching, reducing the uncertainty of binocular vision system matching, eliminating interference information, and effectively improving the accuracy of results ( YePan,Li Li,Jin Wei-Qi,Jiang Yu-Tong.Research on imaging ranging algorithm baseon constraint matching of trinocular vision[C].//Proc.SPIE 9301,IPTA 2014:Image Processing and Pattern Recognition.Beijing,China, May 13-15, 2014.). Lu and Shao established a trinocular system, designed a spherical calibration element, and obtained translation and rotation matrices through singular value decomposition. The results show that the relative accuracy of the trinocular system is improved to 0.105%, and the root mean square error is 0.026mm, which proves the accuracy and robustness of the system (Lu Rui, Shao Mingwei. Sphere-based calibration method for trinocularvision sensor[J].Opt . Lasers Eng., 2017, 90:119-127.). Aiming at the problem of low accuracy of binocular vision measurement of large swinging objects, Zhizao Future (Beijing) Robot System Technology Co., Ltd. proposed a target tracking method and device based on trinocular vision (Han Yexin. Trinocular Vision Recognition Tracking Device and Method: China, CN107507231A[P].2017.12.22.).

Inspired by the multi-sensor fusion measurement technology, by introducing more imaging sensors on the basis of the binocular system, a multi-eye vision system with trinocular or more is established, which not only reduces the measurement uncertainty, but also to a certain extent. The measurement accuracy is also improved. However, the accuracy of the trinocular system still cannot meet the needs of high-precision measurement of mechanical parts. The reason is that many cameras in the multi-eye vision system use the pinhole imaging model, and the principle errors such as parallax and distortion caused by ordinary lenses still cannot be eliminated.

SUMMARY OF THE INVENTION

The present invention is aimed at the shortcomings of the prior art, and provides a method for measuring parts based on heterogeneous stereo vision. The method effectively combines binocular stereo vision and monocular telecentric vision, and compensates for ordinary optical systems through telecentric imaging. It integrates the advantages of binocular vision and the advantages of high measurement accuracy of telecentric vision, and solves the problem of low 3D non-contact measurement accuracy on the production site.

The technical scheme adopted by the present invention is as follows:

A method for measuring parts based on heterogeneous stereo vision, comprising the following steps:

S1. Determine the positions of the No. 1 telecentric industrial camera, No. 2 general industrial camera and No. 3 general industrial camera according to the size of the measured part and the depth of field of the camera, and maintain the positional relationship between the cameras to calibrate the cameras. Make the camera image the part under test to obtain left image, right image and telecentric image;

S2. Register the "left-right" image pair and the "left-right-telecentric" image pair through a process including image preprocessing, feature extraction, and feature matching;

S3. According to the registration relationship between the left image and the right image, as well as the internal and external parameters calibrated by ordinary industrial cameras, construct a mathematical model of the binocular vision system, and obtain the depth information of the feature points of the left image and the right image;

S4. Group the feature points possessed by the "left-right-telecentric" image as a group of 2 feature points, and calculate the absolute difference of the depth of each group of 2 feature points according to the depth information obtained in S3, If the absolute difference is greater than the preset threshold, exclude the group;

S5. Project the feature points retained in S4 to the focal plane of the telecentric industrial camera, and calculate the distances between the two projection points in each group along the horizontal and vertical coordinate directions;

S6. According to the distance obtained in S5, correct the internal and external parameters calibrated in S1, and reconstruct the double binocular according to the registration relationship between the left image and the right image obtained in S2 and the corrected internal and external parameters of the ordinary industrial camera. Mathematical model of the visual system to obtain the updated depth information of the feature points of the left and right images;

S7. According to the updated depth information, construct a three-dimensional model of the measured part, carry out three-dimensional feature recognition, and extract outline elements of the measured part, so as to realize the measurement of the geometric features of the measured part.

Preferably, in S1, the telecentric industrial camera is calibrated in the following manner: the telecentric camera is made to image a calibration piece of known size, the ratio of the actual size of the calibration piece to the pixel size is obtained, and the equivalent pixel of the telecentric camera is obtained. .

Preferably, in S1, the ordinary industrial camera is calibrated in the following manner: after the ordinary industrial camera is installed, the ordinary industrial camera is calibrated by the Zhang Zhengyou plane calibration method, and the internal and external parameters of the ordinary industrial camera are determined based on the calibration result.

Preferably, in S4, extract the feature points in the left image, the right image and the telecentric image, find the feature with the same name of the left image and the right image, measure the Euclidean distance between the two feature points, if the absolute difference is greater than the preset Threshold to remove this group of feature points.

Preferably, the functional process of the part measurement method is as follows:

Establish the world coordinate system o _w -x _w y _w z _w , the camera coordinate system of the No. 1 telecentric industrial camera o _1c -x _1c y _1c z _1c , the camera coordinate system of the No. 2 ordinary industrial camera o _2c -x _2c y _2c z _2c , the camera coordinate system of the No. 3 ordinary industrial camera o _3c -x _3c y _3c z _3c ;

The world coordinates of a point P _i on the measured part are (x _i , y _i , z _i ), and the coordinates of this point in the camera coordinate system No. 1, No. 2 and No. 3 are:

where R _j and T _j represent the coordinate transformation relationship from the world coordinate system to the jth (j=1, 2, 3) imaging system camera coordinate system, also known as imaging system external parameters;

表示P_i在第j个成像系统中的像，其坐标为(u_ij,v_ij)，其中，j＝1,2,3，则像点与空间点的关系为：use

represents the image of P _i in the jth imaging system, and its coordinates are (u _ij , v _ij ), where j=1, 2, 3, the relationship between the image point and the space point is:

where α _j , β _j , γ _j , u _j and v _j are the internal parameters of the imaging system, and (u _ij , v _ij ) are the pixel coordinates of the image point P _ij ;

Extracting feature points from the images collected by the imaging systems No. 1, No. 2 and No. 3, and establishing corresponding feature point combination sets A ₁ , A ₂ and A ₃ ;

为正整数，采用搜索策略和测度函数在A₃中找出其“同名”特征a_3t，

分别将a_2s和a_3t存放到新集合B₂和B₃中；Take any element a _2s from the set A ₂ ,

is a positive integer, use search strategy and measure function to find its "same name" feature a _3t in A ₃ ,

Store a _2s and a _3t in new sets B ₂ and B ₃ respectively;

在B₃中有其“同名”特征b_3s，采用搜索策略和测度函数，在集合A₁中找到b_2s的“同名”特征a_1r，

利用测度函数验证a_1r和b_3s是否是一对“同名”特征，如果是则将a_1r、b_2s、b_3s分别存放到新集合C₁、C₂和C₃，如果不是则从集合B₂中重新任取一个元素b_2t，

且s≠t，重复以上步骤，直至取完集合B₂中的全部元素；Take any element b _2s from the set B ₂ ,

In B ₃ there is its "same name" feature b _3s , using search strategy and measure function, find the "same name" feature a _1r of b _2s in set A ₁ ,

Use the measure function to verify whether a _1r and b _3s are a pair of "same name" features, if so, store a _1r , b _2s , and b _3s in new sets C ₁ , C ₂ and C ₃ respectively, if not, from set B Re-select an element b _2t from ₂ ,

And s≠t, repeat the above steps until all elements in the set B ₂ are taken;

Combine equations (1) and (3) and rewrite them into the following form, and write the spatial coordinates to be found on the left side of the equation:

in the formula

Let m=|C ₂ |, c _2i ∈ C ₂ , c _3i ∈ C ₃ , i=1,2,...,m, the pixel coordinates of the feature point c _2i are (u _i2 ,v _i2 ), that is, j=2 ; The pixel coordinates of the feature point c _3i are (u _i3 , v _i3 ), that is, j=3; Substitute the pixel coordinates into formula (4), h _jk and

In formula (5), the spatial coordinates (x _i , y _i , z _i ) are unknown, and the rest of the parameters are known, so formula (5) is rewritten into formula (6):

G _i x _i =g _i (6)

In the formula, x _i =[x _i , y _i , z _i ] ^T , and g _i and G _i are shown in formula (7) and formula (8) respectively:

以求解得到的空间坐标为元素定义一个新的集合D；从集合D中任取两个元素

和

p,q＝1,2,…,m，且p≠q，计算z坐标的差值

如果δz的绝对值小于给定的非负常数τ，转到集合C₁，找到与空间点x_p和x_q所对应的像素点，它们的像素坐标分别是(u_p1,v_p1)和(u_q1,v_q1)，利用1号成像系统测量得到：Solving Equation (6) by the least square method, we get

Define a new set D with the obtained spatial coordinates as elements; take any two elements from the set D

and

p,q=1,2,...,m, and p≠q, calculate the difference of the z coordinate

If the absolute value of δz is less than the given non-negative constant τ, go to the set C ₁ and find the pixels corresponding to the spatial points x _p and x _q , whose pixel coordinates are (u _p1 ,v _p1 ) and ( u _q1 ,v _q1 ), measured by the No. 1 imaging system:

In the formula, β _- and β _⊥ are the equivalent pixels in the horizontal and vertical directions of the No. 1 imaging system, respectively, in mm/pixel;

应满足：The confidence level is taken as 95%, and the expanded uncertainty of No. 1 imaging system is denoted as U ₉₅ , the true value of d _k

Should satisfy:

改变p和q的取值，重复以上过程，直到x_i(i＝1,2,…m)全部都建立起联系，所以在式(9)和式(10)中采用了下标k，其取值范围是[1,n]，n应不小于m-1；In the formula,

Change the values of p and q and repeat the above process until all x _i (i=1, 2,...m) are connected, so the subscript k is used in equations (9) and (10), which The value range is [1,n], and n should not be less than m-1;

Equation (6) is equivalent to the nonlinear unconstrained extreme value problem shown in Equation (11):

The measurement accuracy of the binocular vision system can reach 1/10mm. Within the depth of field of the telecentric vision system, based on the above two points, all equations (10) are used as constraints, and equation (11) is transformed into a nonlinear constraint extreme value question:

求解式(12)获得更新后的解

将

及其像素坐标代入到式(1)和式(3)中，更新相机的内、外参数为

及T_j；将更新后的相机内外参数代入到式(6)中，并用式(6)对集合B₂和B₃中的元素计算其三维坐标。Take the solution of the least squares method as the initial value, namely

Solve equation (12) to obtain the updated solution

Will

and its pixel coordinates are substituted into equations (1) and (3), and the internal and external parameters of the updated camera are

and T _j ; substitute the updated internal and external parameters of the camera into formula (6), and use formula (6) to calculate the three-dimensional coordinates of the elements in sets B ₂ and B ₃ .

The present invention is compared with the prior art, and the implementation effect of the present invention is as follows:

The present invention organically combines the prior art 1 with the telecentric vision system, obtains the measurement result through the fusion of multiple sensor data, increases the utilization rate of system information, enhances the reliability of the data, and improves the reliability of the system .

The present invention does not need to use a high-performance laser light source, neither the problem of high cost of the laser light source nor the problem of reduced point cloud density due to the interval movement of the structured light.

The tertiary camera introduced in the present invention is a telecentric industrial camera, which has the characteristics of constant magnification, no parallax, small image distortion, etc., and high measurement accuracy.

Description of drawings

FIG. 1 is a measurement principle diagram of a binocular stereo vision system in the prior art 1 .

FIG. 2 is a schematic diagram of the prior art 2 centerline structured light vision measurement.

FIG. 3 is a measurement principle diagram of a polyocular vision system in the prior art 3. FIG.

Figure 4 is a schematic structural diagram of the present invention.

FIG. 5 is a schematic diagram of the principle of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Of course, the described embodiments are only some, but not all, embodiments of the present invention.

Referring to Figures 4 and 5, in the figure:

1(a): No. 1 telecentric industrial camera;

2(a): No. 2 ordinary industrial camera;

3(a): No. 3 ordinary industrial camera;

1(b): the image plane of 1(a);

2(b): the image plane of 2(a);

3(b): the image plane of 3(a);

4: Measured parts;

5: Any measured point p on the measured part;

6: Any measured point q on the measured part;

7: The image point of the measured point P in 1(a);

8: The image point of the measured point q in 1(a);

9: The image point of the measured point q in 2(b);

10: The image point of the measured point P in the imaging system 3;

11: Installation base of heterogeneous stereo vision systems.

Before measuring, according to the size of the measured part and the depth of field of the camera, determine the positions of the No. 1 telecentric industrial camera, No. 2 general industrial camera and No. 3 general industrial camera, and maintain the positional relationship between the cameras, so as to establish a The three-dimensional measurement space shown in Figure 4 is then calibrated for the above three cameras.

o _w -x _w y _w z _w is the world coordinate system, o _1c -x _1c y _1c z _1c is the camera coordinate system of the No. 1 telecentric industrial camera, and o _2c -x _2c y _2c z _2c is the No. 2 ordinary industrial camera The camera coordinate system of , o _3c -x _3c y _3c z _3c is the camera coordinate system of the No. 3 general industrial camera.

For a point on the measured part of P _i , its world coordinates are (x _i , y _i , z _i ), and the coordinates of the point in the camera coordinate system No. 1, No. 2 and No. 3 are:

In the formula, R _j and T _j represent the coordinate transformation relationship from the world coordinate system to the jth (j=1, 2, 3) imaging system camera coordinate system, which are also called imaging system external parameters.

表示P_i在第j个成像系统中的像，其坐标为(u_ij,v_ij)，其中，j＝1,2,3，则像点与空间点的关系为：use

represents the image of P _i in the jth imaging system, and its coordinates are (u _ij , v _ij ), where j=1, 2, 3, the relationship between the image point and the space point is:

的像素坐标。where α _j , β _j , γ _j , u _j and v _j are the internal parameters of the imaging system, (u _ij , v _ij ) are the image points

pixel coordinates.

Usually, the No. 1 telecentric imaging system can measure not by calibrating its internal and external parameters, but by calibrating its equivalent pixels, that is, by imaging a calibration part of known size, to obtain the actual size of the calibration part and the pixel size. ratio. There are many solutions to the calibration problem of No. 2 and No. 3 imaging systems, such as Zhang Zhengyou's plane calibration method (Zhang ZhengYou. A Flexible New Technique for Camera Calibration [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11) : 1330-1334.). Now it is assumed that the three imaging systems have been calibrated, that is, the equivalent pixels of the No. 1 imaging system, and the internal and external parameters of the No. 2 and No. 3 imaging systems are known.

Three cameras are used to image the part under test, and the left image, right image and telecentric image are obtained. It is assumed that the images collected by the No. 1, No. 2 and No. 3 imaging systems are image1, image2 and image3 respectively. Now extract feature points from image1, image2 and image3 and establish corresponding feature descriptors, such as the classic feature detection and feature descriptor algorithm SIFT (Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints [J]. International Journal of Computer Vision, 2004, 60(2):91-110.). The elements of the sets A ₁ , A ₂ and A ₃ are the feature points of image1 , image2 and image3 , respectively.

采用搜索策略和测度函数在A₃中找出其“同名”特征a_3t

也就是进行特征的检索与配准。常用的搜索策略有KD树及其改进算法，比如BBF(Best-Bin-First，BBF)搜索策略(Beis J S,Lowe D G.Shape indexingusing approximate nearest-neighbour search in high-dimensional spaces[C]//Conference on Computer Vision and Pattern Recognition,Puerto Rico,USA,17-19June1997:1000-1006.)。最常用的测度函数是欧氏距离。理论上a_2s和a_3t的欧式距离应为零，实际上它们的欧式距离通常是不为零的。因此，常取一个比较小的非负数ε，当a_2s和a_3t的欧式距离小于ε时，判定a_2s和a_3t是一对“同名”特征。分别将a_2s和a_3t存放到新集合B₂和B₃中。Take any element a _2s from the set A ₂

Use search strategy and measure function to find its "same name" feature a _3t in A ₃

That is, feature retrieval and registration. Commonly used search strategies include KD tree and its improved algorithm, such as BBF (Best-Bin-First, BBF) search strategy (Beis JS, Lowe D G. Shape indexing using approximate nearest-neighbour search in high-dimensional spaces[C]// Conference on Computer Vision and Pattern Recognition, Puerto Rico, USA, 17-19 June 1997: 1000-1006.). The most commonly used measure function is the Euclidean distance. In theory, the Euclidean distance of a _2s and a _3t should be zero, but in fact their Euclidean distance is usually not zero. Therefore, a relatively small non-negative number ε is often chosen. When the Euclidean distance between a _2s and a _3t is less than ε, it is determined that a _2s and a _3t are a pair of "same name" features. Store a _2s and a _3t into new sets B ₂ and B ₃ , respectively.

为正整数，在B₃中有其“同名”特征是b_3s。按照上一步特征检索和配准算法，在集合A₁中找到b_2s的“同名”特征a_1r

利用测度函数验证a_1r和b_3s是否是一对“同名”特征，如果是则将a_1r、b_2s、b_3s分别存放到新集合C₁、C₂和C₃，如果不是则从集合B₂中重新任取一个元素b_2t(

且s≠t)，重复以上步骤，直至取完集合B₂中的全部元素。Take any element b _2s from set B ₂

is a positive integer and has its "same name" feature in B ₃ as b _3s . According to the feature retrieval and registration algorithm in the previous step, find the "same name" feature a _1r of b _2s in the set A ₁

Use the measure function to verify whether a _1r and b _3s are a pair of "same name" features, if so, store a _1r , b _2s , and b _3s in new sets C ₁ , C ₂ and C ₃ respectively, if not, from set B Re-select an element b _2t from ₂ (

and s≠t), repeat the above steps until all elements in the set B ₂ are taken.

Combine formula (1) and formula (3) and rewrite them into the following form, and at the same time write the spatial coordinates to be found on the left side of the equation:

in the formula

Let m=|C ₂ |, c _2i ∈ C ₂ , c _3i ∈ C ₃ , i=1,2,...,m. The pixel coordinates of the feature point c _2i are (u _i2 ,v _i2 ), that is, j=2 ; The pixel coordinates of the feature point c _3i are (u _i3 , v _i3 ), that is, j=3. Substitute the pixel coordinates into equation (4), h _jk and

Equation (5) is that the space coordinates (x _i , y _i , z _i ) are unknown, and the rest of the parameters are known. Rewrite formula (5) into formula (6):

G _i x _i =g _i (6)

In the formula, x _i =[x _i , y _i , z _i ] ^T , and g _i and G _i are shown in formula (7) and formula (8) respectively:

以求解得到的空间坐标为元素定义一个新的集合D。从集合D中任取两个元素

和

(p,q＝1,2,…,m，且p≠q)，计算z坐标的差值

如果δz的绝对值小于给定的非负常数τ，转到集合C₁，找到与空间点x_p和x_q所对应的像素点，它们的像素坐标分别是(u_p1,v_p1)和(u_q1,v_q1)，利用1号成像系统测量得到：Solving Equation (6) by the least square method, we get

Define a new set D with the obtained spatial coordinates as elements. Take any two elements from the set D

and

(p,q=1,2,...,m, and p≠q), calculate the difference of the z coordinate

If the absolute value of δz is less than the given non-negative constant τ, go to the set C ₁ and find the pixels corresponding to the spatial points x _p and x _q , whose pixel coordinates are (u _p1 ,v _p1 ) and ( u _q1 ,v _q1 ), measured by the No. 1 imaging system:

In the formula, β _- and β _⊥ are the equivalent pixels in the horizontal and vertical directions of the imaging system No. 1, respectively, in mm/pixel.

应满足：The confidence level is taken as 95%, and the expanded uncertainty of No. 1 imaging system is denoted as U ₉₅ , the true value of d _k

Should satisfy:

In the formula,

Change the values of p and q and repeat the above process until all x _i (i=1, 2,...m) are connected, so the subscript k is used in equations (9) and (10), which The value range is [1,n], and n should not be less than m-1.

Equation (6) is equivalent to the nonlinear unconstrained extreme value problem shown in Equation (11):

The optical performance of the telecentric vision system is good, and its measurement accuracy is much higher than that of the binocular vision system. In addition, the measurement accuracy of the binocular vision system can reach 1/10mm, which is within the depth of field range of the telecentric vision system. Therefore, based on the above two points, all equations (10) are used as constraints, and equation (11) is transformed into a nonlinear constrained extreme value problem:

求解式(12)获得更新后的解

将

及其像素坐标代入到式(1)和式(3)中，更新相机的内、外参数为

及T_j。将更新后的相机内外参数代入到式(6)中，并用式(6)对集合B₂和B₃中的元素计算其三维坐标。Take the solution of the least squares method as the initial value, namely

Solve equation (12) to obtain the updated solution

Will

and its pixel coordinates are substituted into equations (1) and (3), and the internal and external parameters of the updated camera are

and T _j . Substitute the updated internal and external parameters of the camera into formula (6), and use formula (6) to calculate the three-dimensional coordinates of the elements in sets B ₂ and B ₃ .

Finally, the 3D model of the part is reconstructed using the 3D coordinates obtained in the previous step, and 3D multi-element measurement is performed.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (5)

1. A part measuring method based on heterogeneous stereoscopic vision is characterized by comprising the following steps:

s1, determining the positions of a No. 1 telecentric industrial camera, a No. 2 ordinary industrial camera and a No. 3 ordinary industrial camera according to the size of the part to be measured and the depth of field of the cameras, keeping the position relation among the cameras, calibrating the cameras, and enabling the cameras to image the part to be measured to obtain a left image, a right image and a telecentric image;

s2, registering the left-right image pair and the left-right-telecentric image pair through processes including image preprocessing, feature extraction and feature matching;

s3, constructing a binocular vision system mathematical model according to the registration relation of the left image and the right image and the internal and external parameters calibrated by the common industrial camera to obtain the depth information of the characteristic points of the left image and the right image;

s4, grouping the feature points of the left-right-telecentric image into a group according to each 2 feature points, calculating the absolute difference of the depths of the 2 feature points in each group according to the depth information obtained in S3, and excluding the group if the absolute difference is greater than a preset threshold;

s5, projecting the feature points reserved in the S4 to a focal plane of a telecentric industrial camera, and calculating the distance of 2 projection points in each group along the horizontal and vertical coordinate directions;

s6, correcting the internal and external parameters marked in S1 according to the distance obtained in S5, reconstructing a binocular vision system mathematical model according to the registration relation of the left image and the right image obtained in S2 and the corrected internal and external parameters of the common industrial camera, and obtaining the depth information of the feature points of the updated left image and the updated right image;

and S7, constructing a three-dimensional model of the measured part according to the updated depth information, performing three-dimensional feature recognition, and extracting outline elements of the measured part, thereby realizing the measurement of the geometric features of the measured part.

2. The method for measuring parts based on heterogeneous stereoscopic vision according to claim 1, wherein in S1, the telecentric industrial camera is calibrated as follows: and imaging the calibration piece with the known size by the telecentric camera, and acquiring the ratio of the actual size of the calibration piece to the pixel size to obtain the equivalent pixel of the telecentric camera.

3. The method for measuring parts based on heterogeneous stereoscopic vision according to claim 1, wherein in S1, a normal industrial camera is calibrated as follows: after the common industrial camera is installed, the common industrial camera is calibrated by adopting a Zhangyingyou plane calibration method, and internal and external parameters of the common industrial camera are determined based on a calibration result.

4. The method for measuring parts based on heterogeneous stereoscopic vision according to claim 1, wherein in S4, feature points in the left image, the right image and the telecentric image are extracted, the homonymous features of the left image and the right image are found, the euclidean distance between the two feature points is measured, and if the absolute difference value is greater than a preset threshold value, the set of feature points is rejected.

5. The part measuring method based on the heterogeneous stereovision according to claim 1, wherein the function process of the part measuring method is as follows:

establishing a world coordinate system o_w-x_wy_wz_wCamera coordinate system o of No. 1 telecentric industrial camera_1c-x_1cy_1cz_1cCamera coordinate system o of general industrial camera No. 2_2c-x_2cy_2cz_2cCamera coordinate system o of general industrial camera No. 3_3c-x_3cy_3cz_3c；

A certain point P on the part to be measured_iWorld coordinate is (x)_i,y_i,z_i) The coordinates of the point under the coordinate systems of the No. 1, No. 2 and No. 3 cameras are respectively as follows:

in the formula R_jAnd T_jRepresenting the coordinate transformation relation from a world coordinate system to a j (j is 1,2,3) th imaging system camera coordinate system, which is also called an imaging system external parameter;

by using

Represents P_iThe image in the jth imaging system has a coordinate of (u)_ij,v_ij) Where j is 1,2,3, the relationship between the image point and the spatial point is:

in the formula of alpha_j、β_j、γ_j、u_jAnd v_jIs an imaging system intrinsic parameter, (u)_ij,v_ij) Is a picture point

The pixel coordinates of (a);

extracting the characteristic points of the images collected by the No. 1, No. 2 and No. 3 imaging systems, and establishing a corresponding characteristic point combination set A₁、A₂And A₃；

From set A₂In which an element a is arbitrarily selected_2s，

For positive integers, search strategy and measure function are adopted in A₃Find out the 'same name' feature a_3t，

Respectively a to_2sAnd a_3tDeposit to New set B₂And B₃Performing the following steps;

from set B₂In any one of the elements b_2s，

In B₃In which there is its "same name" feature b_3sUsing search strategy and measure function in set A₁In (b) is found_2sCharacteristic of "same name" of (a)_1r，

Validating a with a measure function_1rAnd b_3sWhether it is a pair of "same name" features, if so, a_1r、b_2s、b_3sAre respectively stored in a new set C₁、C₂And C₃From set B if not₂In the process of re-choosing an element b_2t，

And s is not equal to t, the steps are repeated until the collection B is completely taken₂All of the elements in (1);

the equations (1) and (3) are put together and rewritten as follows, and the spatial coordinates to be found are written to the left side of the equations:

in the formula

k＝1,2,3

Let m be | C₂|，c_2i∈C₂，c_3i∈C₃I is 1,2, …, m, characteristic point c_2iHas a pixel coordinate of (u)_i2,v_i2) I.e., j is 2; characteristic point c_3iHas a pixel coordinate of (u)_i3,v_i3) I.e., j is 3; substituting the pixel coordinates into equations (4) and h_jkAnd

spatial coordinate (x) in formula (5)_i,y_i,z_i) Unknown, and the remaining parameters are known, and rewriting formula (5) into formula (6):

G_ix_i＝g_i (6)

in the formula x_i＝[x_i,y_i,z_i]^T，g_iAnd G_iSee, respectively, formula (7) and formula (8):

solving the formula (6) by the least square method to obtain

Defining a new set D by taking the space coordinate obtained by solving as an element; taking two elements from set D

And

p, q ≠ 1,2, …, m, and p ≠ q, calculating the difference in z-coordinates

If the absolute value of z is less than a given non-negative constant τ, go to set C₁Find and space point x_pAnd x_qThe corresponding pixel points have pixel coordinates of (u)_p1,v_p1) And (u)_q1,v_q1) And measuring by using a No. 1 imaging system to obtain:

in the formula, beta_-And beta_⊥Equivalent pixels in the horizontal and vertical directions of the No. 1 imaging system, respectively, in mm/pixel units;

the confidence level is 95 percent, and the expansion uncertainty of the No. 1 imaging system is U₉₅，d_kTrue value of

It should satisfy:

in the formula (I), the compound is shown in the specification,

changing the values of p and q, and repeating the process until x_iAll (i ═ 1,2, … m) are linked, so in formula (9) and formula (10) the subscript k is used, which ranges from [1, n ═ n]N should be not less than m-1;

equation (6) is equivalent to the nonlinear unconstrained extremum problem shown in equation (11):

the measurement accuracy of the binocular vision system can reach 1/10mm, and in the field depth range of the telecentric vision system, based on the two points, the whole formula (10) is used as a constraint condition, and the formula (11) is changed into a nonlinear constraint extreme value problem:

using the solution of least squares as initial values, i.e.

Solving equation (12) to obtain an updated solution

Will be provided with

And the pixel coordinates are substituted into the formula (1) and the formula (3) to update the internal and external parameters of the camera to be

And T_j(ii) a Substituting the updated camera internal and external parameters into the formula (6), and using the formula (6) to set B₂And B₃The element in (3) calculates its three-dimensional coordinates.

Images