CN103913131A

CN103913131A - Free curve method vector measurement method based on binocular vision

Info

Publication number: CN103913131A
Application number: CN201410149149.6A
Authority: CN
Inventors: 刘巍; 李肖; 马鑫; 贾振元; 尚志亮; 张洋; 李晓东; 高航
Original assignee: Dalian University of Technology
Current assignee: Dalian University of Technology
Priority date: 2014-04-14
Filing date: 2014-04-14
Publication date: 2014-07-09
Anticipated expiration: 2034-04-14
Also published as: CN103913131B

Abstract

The invention relates to a binocular vision-based free-form surface method vector measurement method, which belongs to the field of computer vision measurement, and relates to a binocular vision-based curved surface method vector measurement method. In the measurement method, the projection pattern projected by the laser projection device to the free-form surface is composed of two orthogonal straight lines and four circular light spots located on the two lines; the two straight lines intersect at one point, and the four circular light spots are equal in size and uniform in size. distributed on the same circle. The binocular vision system is used to collect the image of the projected pattern, and the curvature of a small neighborhood of the curved surface is estimated through the threshold judgment condition based on the distance, so as to select different normal vector measurement schemes. The measurement method takes into account the curvature of the neighborhood of the point to be measured, has high measurement flexibility and strong adaptability, and can realize online high-efficiency measurement of the normal vector of any point on the free-form surface. And the method is simple, and the algorithm is easy to realize.

Description

A vector measurement method based on binocular vision for free-form surface method

技术领域technical field

本发明属于计算机视觉测量领域，涉及一种基于双目视觉的曲面法矢量测量方法。The invention belongs to the field of computer vision measurement, and relates to a vector measurement method based on binocular vision for surface method.

背景技术Background technique

曲面零部件如涡轮叶片、喇叭、曲面腔体、列车蒙皮等正日益成为各应用领域产品不可缺少的重要构成部分。对于这些零件，曲面法矢成为重要的测量参数。蒙皮作为飞机典型复合材料曲面零件具有壁薄、形状复杂、随机变形大、材料性质各向异性、尺寸范围大等特点。蒙皮柔性钻铆工艺对钻铆垂直度有很高要求，而飞机蒙皮铆接件钻孔时，由加工预制件铺放、热压固化以及零件自身重力所引起的变形和定位、协调、夹紧时各种误差的积累使得实际待铆点处法矢量与三维数字模型中理论法矢量产生偏差。若铆接点处法向精度超出范围，会使孔加工质量下降、连接形式为强迫连接、连接处产生应力集中以及蒙皮表面不光滑，进而影响铆接质量和飞机气动外形，并最终导致飞机使役性能下降。因此，如何实现飞机蒙皮表面任意一点法矢量的在线高精度、高效率测量成为亟需解决的重要难题。Curved parts such as turbine blades, horns, curved cavities, train skins, etc. are increasingly becoming an indispensable and important part of products in various application fields. For these parts, the surface normal vector becomes an important measurement parameter. As a typical composite material surface part of aircraft, skin has the characteristics of thin wall, complex shape, large random deformation, anisotropy of material properties, and large size range. The skin flexible drilling and riveting process has high requirements on the verticality of the drilling and riveting, and when the aircraft skin riveting parts are drilled, the deformation, positioning, coordination, clamping, etc. Accumulation of various errors during tightening causes deviation between the normal vector at the actual to-be-riveted point and the theoretical normal vector in the 3D digital model. If the normal accuracy of the riveting point exceeds the range, the hole processing quality will be reduced, the connection form will be forced connection, stress concentration will be generated at the connection, and the skin surface will be rough, which will affect the quality of riveting and the aerodynamic shape of the aircraft, and eventually lead to aircraft service performance. decline. Therefore, how to realize the online high-precision and high-efficiency measurement of the normal vector of any point on the aircraft skin surface has become an important problem that needs to be solved urgently.

李原、余剑锋发明的专利号为CN201120358775的“一种测量自由曲面任意点处法向矢量的装置”发明一种利用球形触头在曲面上画出相交的曲线，利用曲线的切矢求取法向量的测量方法。此种方法为接触式测量，对于随机变形曲面其法矢量求取精度比较低。姚振强、胡永祥发明的专利号为CN102248450A的“用于大曲率半径曲面法向矢量快速检测方法”发明了利用光学测量技术实现两假想正交平面与曲面交线的获取,利用这两条交线在待测点的法矢量的叉乘积求取曲面一点的法向量,但该方法没有考虑曲面一点处的曲率大小,而单一的利用一种方法求取法矢量,效率低、柔性差。The patent No. CN201120358775 invented by Li Yuan and Yu Jianfeng is "a device for measuring the normal vector at any point on a free-form surface". Invented a method of using a spherical contact to draw intersecting curves on a curved surface, and using the tangent vector of the curve to obtain the method Vector measurement method. This method is a contact measurement, and the accuracy of calculating the normal vector for a randomly deformed surface is relatively low. Yao Zhenqiang and Hu Yongxiang's patent No. CN102248450A "A method for fast detection of normal vectors on curved surfaces with large curvature radii" invented the use of optical measurement technology to obtain the intersection of two imaginary orthogonal planes and curved surfaces. The cross product of the normal vector of the point to be measured is used to obtain the normal vector of a point on the surface, but this method does not consider the curvature of a point on the surface, and a single method is used to obtain the normal vector, which has low efficiency and poor flexibility.

发明内容Contents of the invention

本发明要解决的技术难题是克服现有技术的缺陷，发明一种基于双目视觉的自由曲面法矢量测量方法，采用由双目视觉系统以及投影图案组成的测量系统进行自由曲面任一点法矢量的测量。通过比较所设阈值与待测点到拟合平面的距离大小，选取不同测量方案完成法矢量的高精度快速测量。该测量方法考虑的待测点邻域的曲率的大小，柔性比较高，可实现曲面任一点法矢量的在线高精度与高效率测量。The technical problem to be solved in the present invention is to overcome the defects of the prior art, and to invent a free-form surface method vector measurement method based on binocular vision, which uses a measurement system composed of a binocular vision system and a projection pattern to measure the free-form surface normal vector at any point. Measurement. By comparing the set threshold with the distance from the point to be measured to the fitting plane, different measurement schemes are selected to complete the high-precision and fast measurement of the normal vector. The measurement method considers the size of the curvature of the neighborhood of the point to be measured, and has relatively high flexibility, which can realize online high-precision and high-efficiency measurement of the normal vector of any point on the surface.

本发明所采取的技术方案是一种基于双目视觉的自由曲面法矢量测量方法，其特征是：测量方法中,由激光投影装置向自由曲面投影的投影图案由两条正交直线L₁、L₂与位于两条线上的四个圆形光斑G构成；两直线交点为P，四个圆形光斑大小相等且均布在同一圆周上；利用双目视觉系统采集投影图案的图像；通过基于距离的阈值判定条件，估计曲面一点小邻域范围的曲率大小，以选取不同的法矢量测量方案：当d≤ε时，选用拟合平面的法向量逼近曲面待测点法矢量；当d＞ε时，则利用拟合后的两条空间曲线在待测点处切矢量的叉乘积求取待测点处的法矢量；测量方法的具体步骤如下:The technical solution adopted by the present invention is a free-form surface method vector measurement method based on binocular vision _. L ₂ is composed of four circular light spots G located on two lines; the intersection point of the two straight lines is P, and the four circular light spots are equal in size and evenly distributed on the same circumference; the image of the projected pattern is collected by the binocular vision system; through Based on the distance threshold judgment condition, estimate the curvature of a small neighborhood of the surface to select different normal vector measurement schemes: when d≤ε, select the normal vector of the fitting plane to approximate the normal vector of the surface to be measured; when d When >ε, then utilize two space curves after fitting to obtain the normal vector at the point to be measured by the cross product of the tangent vector at the point to be measured; the concrete steps of the measurement method are as follows:

（1）基于阈值灰度重心法的光斑中心提取(1) Spot center extraction based on threshold gray-scale center of gravity method

本发明采用Canny算子结合灰度重心法对光斑中心进行高精度提取，灰度图像I(i,j)中目标S的灰度重心为:The present invention uses the Canny operator in combination with the gray-scale center of gravity method to extract the spot center with high precision, and the gray-scale center of gravity of the target S in the gray-scale image I (i, j) is:

$\{\begin{matrix} {X x}_{k k} = = \frac{\underset{((i i,, j j)) &Element; &Element; S S}{Σ Σ} i i \times \times W W ((i i,, j j))}{\underset{((i i,, j j)) &Element; &Element; S S}{Σ Σ} W W ((i i,, j j))} \\ {Y Y}_{k k} = = \frac{\underset{((i i,, j j)) &Element; &Element; S S}{Σ Σ} j j \times \times W W ((i i,, j j))}{\underset{((i i,, j j)) &Element; &Element;}{Σ Σ} W W ((i i,, j j))} \end{matrix} - - - - - - ((11))$

式中，(X_k,Y_k)为第k个光斑中心点的图像坐标；W(i,j)为所设定的权值；考虑实际背景和目标之间的灰度信息状况，本发明采用阈值灰度重心法，其权值W(i,j)定义为：In the formula, (X _k , Y _k ) is the image coordinate of the center point of the kth light spot; W(i, j) is the set weight; considering the gray information status between the actual background and the target, the present invention Using the threshold gray-scale center of gravity method, its weight W(i,j) is defined as:

$\begin{matrix} W W ((i i,, j j)) = = \{\begin{matrix} I I ((i i,, j j)) \\ 00 \end{matrix} \\ ((I I ((i i,, j j)) > > T T)) \\ ((I I ((i i,, j j)) \leq \leq T T)) \end{matrix} - - - - - - ((22))$

其中，T为区分目标和背景的阈值；灰度重心取W(i,j)＝I(i,j)；Among them, T is the threshold for distinguishing the target from the background; the gray center of gravity is W(i,j)=I(i,j);

（2）光斑中心点的匹配和重建(2) Matching and reconstruction of spot center point

在完成光斑中心点的提取后对左右摄像机采集的图像上的光斑中心点进行匹配与重建；匹配方法如下：After completing the extraction of the center point of the light spot, match and reconstruct the center point of the light spot on the images collected by the left and right cameras; the matching method is as follows:

首先采用Hartley提出的改进八点归一化算法计算左右摄像机的基本矩阵F，然后通过左右两高速摄像机所采集的二维数字图像之间极线约束关系进行光斑中心点的初匹配，假设左图像光斑中心点x_i'与右图像光斑中心点x_i''相匹配，即两光斑中心点满足极限约束条件，极限约束方程可表示如下：First, the improved eight-point normalization algorithm proposed by Hartley is used to calculate the basic matrix F of the left and right cameras, and then the initial matching of the center point of the spot is performed through the epipolar constraint relationship between the two-dimensional digital images collected by the left and right high-speed cameras. The center point x _i ' of the light spot matches the center point x _i' ' of the light spot in the right image, that is, the center points of the two light spots satisfy the limit constraint condition, and the limit constraint equation can be expressed as follows:

${x x}_{i i}^{' ' T T} F f {x x}_{{i i}^{' '}}^{' '} = = 00 - - - - - - ((33))$

其中，x_i'为左摄像机采集的图像光斑中心点的像面坐标；x_i''为与x_i'相匹配由右摄像机所采集图像光斑中心点的像面坐标；F为两摄像机之间的基本矩阵；Among them, x _i 'is the image plane coordinates of the center point of the image spot collected by the left camera; x _i '' is the image plane coordinates of the center point of the image spot collected by the right camera matching with x _i '; F is the distance between the two cameras the basic matrix of

在此基础上对左右图像中所有满足极限约束条件的光斑中心点进行三维重建以得到光斑中心点在世界坐标系下的三维坐标值，重建公式如下：On this basis, three-dimensional reconstruction is performed on all the center points of the light spots in the left and right images that meet the limit constraints to obtain the three-dimensional coordinates of the center points of the light spots in the world coordinate system. The reconstruction formula is as follows:

$\{\begin{matrix} {x x}_{i i} = = \frac{{zX Z}_{i i}^{' '}}{{f f}_{11}} \\ {y the y}_{i i} = = \frac{{zY Z}_{i i}^{' '}}{{f f}_{11}} \\ {z z}_{i i} = = \frac{{f f}_{11} (({f f}_{22} {t t}_{y the y} - - {Y Y}_{{i i}^{' '}}^{' '} {t t}_{z z}))}{{Y Y}_{11} (({r r}_{77} {X x}_{i i}^{' '} + + {r r}_{88} {Y Y}_{i i}^{' '} + + {r r}_{99} {f f}_{11})) - - {f f}_{22} (({r r}_{44} {X x}_{i i}^{' '} + + {r r}_{55} {Y Y}_{i i}^{' '} + + {r r}_{66} {f f}_{11}))} \end{matrix} - - - - - - ((44))$

其中，x_i'＝(X_i',Y_i')，X_i'，Y_i'分别为左摄像机采集的图像光斑中心点x_i'在像面坐标系下的横、纵坐标；x_i''＝(X_i'',Y_i'')，X_i''，Y_i''分别为右摄像机采集的图像光斑中心点x_i‘'在像面坐标系下的横、纵坐标；(x_i,y_i,z_i)为由两匹配光斑中心点x_i'、x_i‘'重建出来的空间标记点的三维坐标；f₁、f₂分别为左右摄像机的焦距；为连接左右摄像机关系的旋转矩阵，[t_xt_yt_z]是右摄像机相对于左摄像机的平移矩阵；Among them, x _i '=(X _i ', Y _i '), Xi _' , Y _i ' are respectively the horizontal and vertical coordinates of the center point x _i ' of the image spot collected by the left camera in the image plane coordinate system; x _{i '} '=(X _i' ', Y _i' '), Xi _' ', Y _i' ' are respectively the abscissa and ordinate of the image spot center point x _i' ' collected by the right camera in the image plane coordinate system; ( _xi , y _i , _zi ) are the three-dimensional coordinates of the spatial marker points reconstructed from the center points x _i ' and xi _' ' of the two matching spots; f ₁ and f ₂ are the focal lengths of the left and right cameras respectively; is the rotation matrix connecting the left and right cameras, [t _x t _y t _z ] is the translation matrix of the right camera relative to the left camera;

（3）曲率判定(3) Curvature Judgment

1)最小二乘法拟合空间平面1) The least squares method to fit the space plane

以重建出的四个光斑中心点在世界坐标系的三维坐标值为基础，利用最小二乘法拟合空间平面，步骤如下：Based on the three-dimensional coordinate values of the reconstructed center points of the four spots in the world coordinate system, the least square method is used to fit the space plane. The steps are as follows:

平面方程的一般表达式为：The general expression for the plane equation is:

$\{\begin{matrix} Ax Ax + + By By + + Cz Cz + + D D. = = 00,, ((C C &NotEqual; &NotEqual; 00)) \\ z z = = - - \frac{A A}{C C} x x - - \frac{B B}{C C} y the y - - \frac{D D.}{C C} \end{matrix} - - - - - - ((55))$

其中(A,B,C)为平面的法向向量；D为原点到平面的距离；记 $a_{0} = - \frac{A}{D}, a_{1} = - \frac{B}{D}, a_{2} = - \frac{C}{D};$ 则z=a₀x+a₁y+a₂；Among them (A, B, C) is the normal vector of the plane; D is the distance from the origin to the plane; record $a_{0} = - \frac{A}{D.}, a_{1} = - \frac{B}{D.}, a_{2} = - \frac{C}{D.};$ Then z=a ₀ x+a ₁ y+a ₂ ;

选用最小二乘法利用n个点(n≥3):(x_i,y_i,z_i),i＝0,1…，,n-1拟合上述平面，则使： $S = Σ_{i = 0}^{n - 1} {(a_{0} x + a_{1} y + a_{2} - z)}^{2}$ 最小;Use the least squares method to use n points (n≥3): (xi _, y _, _zi ), i=0, 1...,, n-1 to fit the above plane, then make: $S = Σ_{i = 0}^{no - 1} {(a_{0} x + a_{1} the y + a_{2} - z)}^{2}$ minimum;

其中S为点到直线的距离的平方和；Where S is the sum of the squares of the distance from the point to the line;

要使S取得最小值，应满足：K＝0,1,2；即：To make S obtain the minimum value, it should satisfy: K=0,1,2; namely:

$\{\begin{matrix} Σ Σ 22 (({a a}_{00} {x x}_{i i} + + {a a}_{11} {y the y}_{i i} + + {a a}_{22} - - {z z}_{i i})) {x x}_{i i = = 00} \\ Σ Σ 22 (({a a}_{11} {x x}_{i i} + + {a a}_{11} {y the y}_{i i} + + {a a}_{22} - - {z z}_{i i})) {y the y}_{i i} = = 00 \\ Σ Σ 22 (({a a}_{11} {x x}_{i i} + + {a a}_{11} {y the y}_{i i} + + {a a}_{22} - - {z z}_{i i})) = = 00 \end{matrix} - - - - - - ((66))$

将四个重建光斑中心的空间三维坐标(x_i,y_i,z_i),i＝0,1,2,3带入上述方程组求得a₀,a₁,a₂；Put the three-dimensional coordinates (x _i , y _i , z _i ) of the centers of the four reconstruction spots into the above equations to obtain a ₀ , a ₁ , a ₂ ;

即拟合平面的方程为：z＝a₀x+a₁y+a₂；空间拟合平面的法向量为: That is, the equation of the fitting plane is: z=a ₀ x+a ₁ y+a ₂ ; the normal vector of the space fitting plane is:

2)求待测点P′到拟合平面的距离2) Find the distance from the point P' to be measured to the fitting plane

空间一点到平面的距离公式可表示为：The distance formula from a point to a plane in space can be expressed as:

其中,S_平面为空间拟合平面的方程；d为待测点到平面的距离；P′＝(x′,y′,z′)为待测点在世界坐标系下的坐标；Q＝(x_q,y_q,z_q)为拟合平面上的任意一点；ε为所设阈值；当d≤ε时，认为待测点P′小邻域曲面范围内曲率变化不大；当d＞ε时则认为待测点P′小邻域曲面范围内曲率变化较大；Among them, the S _plane is the equation of the space fitting plane; d is the distance from the point to be measured to the plane; P'=(x',y',z') is the coordinate of the point to be measured in the world coordinate system; Q=( x _q , y _q , z _q ) is any point on the fitting plane; ε is the set threshold; when d≤ε, it is considered that the curvature of the measured point P′ does not change much within the small neighborhood surface range; when d＞ When ε, it is considered that the curvature changes in the small neighborhood surface of the point P′ to be measured is relatively large;

（4）法矢量求解(4) Normal vector solution

基于距离阈值约束的法矢量测量方案选择准则，情况一：若位于曲面上的待测点P′到空间拟合平面的距离满足d≤ε，则认为待测点P′小邻域曲面范围内曲率变化不大，此时认为平面的法向量就是曲面上待测点的法矢量，即 $\overset{&RightArrow;}{n} = (a_{0} a_{1}, - 1);$ Selection criterion of normal vector measurement scheme based on distance threshold constraints, case 1: if the distance from the point P′ on the surface to the space fitting plane satisfies d≤ε, then the point P′ is considered to be within the small neighborhood surface range The curvature does not change much, and the normal vector of the plane is considered to be is the normal vector of the point to be measured on the surface, namely $\overset{&Right Arrow;}{no} = (a_{0} a_{1}, - 1);$

情况二：若曲面上待测点P′到空间拟合平面的距离d＞ε，则认为待测点P′小邻域曲面范围内曲率变化较大，其近似曲面可能是球面、抛物面、马鞍面等其它二次曲面，此时选用投影到曲面上的两条空间曲线进行法矢量的求解；其步骤如下：Situation 2: If the distance d>ε from the point P′ to be measured on the surface to the space fitting plane, it is considered that the curvature of the point P′ to be measured varies greatly within the small neighborhood surface range, and the approximate surface may be a sphere, a parabola, or a saddle surface and other quadratic surfaces, at this time, two space curves projected onto the surface are selected to solve the normal vector; the steps are as follows:

1)激光条纹中心线点的提取、匹配与重建1) Extraction, matching and reconstruction of laser stripe centerline points

本发明采用基于方向模板的激光条纹中心线检测方法，分别在水平、垂直、左倾斜45。、右倾斜45。方向上布置大小固定方向可变的模板，分别记为K₀、K₁、K₂、K₃，用这四个模板对二维数字图像每一行分别进行处理；以对_i行处理为例，对于K₀模板有：The present invention adopts a laser stripe centerline detection method based on a direction template, which is inclined 45 degrees horizontally, vertically and leftward respectively. , Tilt 45 to the right. Templates with fixed size and variable direction are arranged in the direction, which are respectively recorded as K ₀ , K ₁ , K ₂ , and K ₃ , and these four templates are used to process each row of the two-dimensional digital image separately; taking the processing of row _i as an example, For the K ₀ template there are:

$\begin{matrix} \{\begin{matrix} {H h}_{j j} = = {Σ Σ}_{s the s = = 00}^{M m - - 11} {Σ Σ}_{t t = = 00}^{N N - - 11} {K K}_{00} [[s the s]] [[t t]] C C [[i i - - \frac{M m}{22} + + S S]] [[j j - - \frac{N N}{22} + + t t]],, j j = = \frac{N N}{22},, \frac{N N + + 11}{22},, . . . . . .,, col col - - 11 \\ {H h}_{g g 00} = = max max (({H h}_{\frac{N N}{22},,} {H h}_{\frac{N N}{22} + + 11},, . . . . . .,, {H h}_{col col - - 11})),, \frac{N N}{22} \leq \leq j j \leq \leq col col - - 11 \end{matrix} - - - - - - ((88)) \end{matrix}$

其中M为模板所对应的行数；N为模板对应的列数；K₀[s][]t≥0；表示点的灰度值；相应的对于模板K₁、K₂、K₃有H_g1、H_g2、H_g3；求取H_g＝max(H_g0,H_g1,H_g2,H_g3)，则有第i行激光条纹的中心点的位置在点g处；用该方法对二维数字图像进行逐行逐像素检测可完成激光条纹中心线的提取；在完成激光条纹中心线提取的基础上，采用与本发明(2)中光斑中心点匹配和重建相同的方法进行激光条纹中心点的匹配和重建，得到激光条纹中心线点在世界坐标系下的三维坐标值；Where M is the number of rows corresponding to the template; N is the number of columns corresponding to the template; K ₀ [s][]t≥0; express The gray value of the point; correspondingly for the templates K ₁ , K ₂ , K ₃ there are H _g1 , H _g2 , H _g3 ; if H _g = max(H _g0 , H _g1 , H _g2 , H _g3 ), then there is The position of the center point of the i-th row of laser stripes is at point g; using this method to detect the two-dimensional digital image line by line and pixel by pixel can complete the extraction of the centerline of the laser stripe; on the basis of completing the extraction of the centerline of the laser stripe, use Carry out the matching and reconstruction of the center point of the laser stripe in the same method as the matching and reconstruction of the center point of the light spot in (2) of the present invention, and obtain the three-dimensional coordinate value of the center line point of the laser stripe under the world coordinate system;

2)三次B样条曲线拟合两空间曲线2) Cubic B-spline curve fitting two space curves

本发明采用三次B样条曲线拟合两空间激光条纹曲线，B样条曲线分段函数表达式为：The present invention adopts cubic B-spline curve to fit the two-space laser fringe curve, and the expression of the segmental function of the B-spline curve is:

$\{\begin{matrix} {c c}_{11} : : p p = = {P P}_{00} \cdot &Center Dot; {N N}_{1010}^{33} + + {P P}_{11} \cdot &Center Dot; {N N}_{1111}^{33} + + {P P}_{22} \cdot &Center Dot; {N N}_{1212}^{33} + + {P P}_{33} {N N}_{1313}^{33} & u u &Element; &Element; [[{u u}_{22},, {u u}_{33}]] \\ {c c}_{22} : : p p = = {P P}_{11} \cdot &Center Dot; {N N}_{2020}^{33} + + {P P}_{22} \cdot \cdot {N N}_{21 twenty one}^{33} + + {P P}_{33} \cdot \cdot {N N}_{22 twenty two}^{33} + + {P P}_{44} {N N}_{23 twenty three}^{33} & u u &Element; &Element; [[{u u}_{33},, {u u}_{44}]] \\ {c c}_{11} : : p p = = {P P}_{22} \cdot &Center Dot; {N N}_{3030}^{33} + + {P P}_{33} \cdot \cdot {N N}_{3131}^{33} + + {P P}_{44} \cdot &Center Dot; {N N}_{3232}^{33} + + {P P}_{55} {N N}_{3333}^{33} & u u &Element; &Element; [[{u u}_{44},, {u u}_{55}]] \end{matrix} - - - - - - ((99))$

其中P_i(i＝0,1…5)分别表示控制顶点；N_ij(i＝1…3,j＝0,1…4)表示基函数；设有两摄像机重建出的曲线上离散点为b₁,b₂,???,b_n；其中前i个点位于c₁段上，k-i个点位于c₂段上，n-j个点位于c₃段上，则将几点代入上述方程组得：Among them, P _i (i=0,1...5) represent control vertices respectively; N _ij (i=1...3, j=0,1...4) represent basis functions; if the discrete points on the curve reconstructed by two cameras are b ₁ ,b ₂ ,???,b _n ; where the first i points are located on segment c ₁ , ki points are located on segment c ₂ , and nj points are located on segment c ₃ , then the points are substituted into the above equations have to:

令M为左边的系数矩阵，P为所求的控制顶点的所组成的向量，p为三维重建的激光条纹中心线点，上述方程简写为：Let M be the coefficient matrix on the left, P be the vector composed of the control vertices to be sought, and p be the centerline point of the three-dimensionally reconstructed laser stripe, the above equation can be abbreviated as:

M·P＝p (11)M·P=p (11)

由此可得到拟合的法方程为：From this, the fitted normal equation can be obtained as:

M'·M·P＝M'·p (12)M'·M·P＝M'·p (12)

为提高交点附近曲线的拟合精度，对上述方程引入权值；加权后的方程为：In order to improve the fitting accuracy of the curve near the intersection point, a weight is introduced to the above equation; the weighted equation is:

(M'·H'·M·P)＝(M'·H')·M·p (13)(M'·H'·M·P)＝(M'·H')·M·p (13)

通过此加权方程即可求取两曲线的方程。在此基础上分别求取两曲线在待测点两个方向的切矢，记为则所求法矢量为： The equations of the two curves can be obtained through this weighted equation. On this basis, the tangent vectors of the two curves in the two directions of the point to be measured are obtained respectively, which are denoted as Then the normal vector obtained is:

本发明的有益效果是所发明的测量方法为非接触、柔性强、实时性高。可适用于曲面不同点的在线高效率测量，且其方法简单，算法易于实现。The beneficial effect of the invention is that the invented measuring method is non-contact, strong in flexibility and high in real-time performance. It is suitable for online high-efficiency measurement of different points on the surface, and its method is simple, and the algorithm is easy to implement.

附图说明：Description of drawings:

图1为曲面法矢量求解方法原理图。其中，L₁′-水平投影激光条纹、L₂′-竖直投影激光条纹、P′-待测点、G₁-第一投影光斑、G₂-第二投影光斑、G₃-第三投影光斑、G₄-第四投影光斑、S-拟合平面、m-拟合平面S的法向量、d-待测点P′到拟合平面S的距离、-曲面在点P′的法矢量、-曲线L₁′在点P′的切向量、-曲线L₂′在点P′的切向量。Figure 1 is a schematic diagram of the surface method vector solution method. Among them, L ₁ ′-horizontal projection laser stripes, L ₂ ′-vertical projection laser stripes, P′-points to be measured, G ₁ -first projection spot, G ₂ -second projection spot, G ₃ -third projection Spot, G ₄ -the fourth projected spot, S-fitting plane, m-the normal vector of the fitting plane S, d-the distance from the point P' to be measured to the fitting plane S, - the normal vector of the surface at point P′, - the tangent vector of the curve L ₁ ' at the point P', - Tangent vector to curve _L2 ' at point P'.

图2为发明的投影图案。其中L₁-水平投影线、L₂-竖直投影线、P-两正交投影线交点、G₁-第一圆形光斑、G₂-第二圆形光斑、G₃-第三圆形光斑、G₄-第四圆形光斑。Figure 2 is the projected pattern of the invention. Among them, L ₁ - horizontal projection line, L ₂ - vertical projection line, P - intersection point of two orthogonal projection lines, G ₁ - first circular spot, G ₂ - second circular spot, G ₃ - third circular Spot, G ₄ - the fourth circular spot.

图3为基于双目视觉测量系统的法矢量求解流程图。Fig. 3 is a flow chart of solving the normal vector based on the binocular vision measurement system.

具体实施方式Detailed ways

本发明结合技术方案和附图，为了更好的说明法矢量求解过程，以飞机蒙皮为实例对其详细叙述。按附图3所示的具体流程如下：（1）利用数控系统将激光投影装置移动到飞机蒙皮待测点P′(x′,y′,z′)处，在保证两激光条纹交点为待测点的基础上，将附图2图案投影在飞机蒙皮待测表面上，投射为四个高亮的光斑G₁′、G₂′、G₃′、G₄′和两条激光条纹L₁′、L₂′，与此同时利用双目视觉系统的左右高速摄像机采集投影图案图像。In order to better illustrate the process of solving the normal vector, the present invention takes the aircraft skin as an example to describe it in detail in combination with the technical scheme and the accompanying drawings. The specific process shown in Figure 3 is as follows: (1) Use the numerical control system to move the laser projection device to the point P'(x', y', z') of the aircraft skin to be measured, and ensure that the intersection point of the two laser stripes is On the basis of the points to be measured, project the pattern shown in Figure 2 on the surface of the aircraft skin to be tested, projecting four bright spots G ₁ ′, G ₂ ′, G ₃ ′, G ₄ ′ and two laser stripes L ₁ ′, L ₂ ′, and at the same time use the left and right high-speed cameras of the binocular vision system to collect projected pattern images.

（2）基于阈值灰度重心法的光斑中心的提取(2) Extraction of the spot center based on the threshold gray-scale center of gravity method

选用Canny算子结合阀值灰度重心法对左右相机采集图像中光斑中心进行提取，完成光斑中心的定位。得左图像中四个光斑中心点的像面坐标(X_i,Y_i)i＝1,2,3,4和右图像的四个光斑中心点的像面坐标为(X_i′,Y_i′)i′＝1,2,3,4。The Canny operator combined with the threshold gray center of gravity method is used to extract the center of the spot in the images collected by the left and right cameras to complete the location of the center of the spot. The image plane coordinates (X _i , Y _i )i=1, 2, 3, 4 of the four spot center points in the left image and the image plane coordinates of the four spot center points in the right image are (X _i′ , Y _{i '} )i'=1,2,3,4.

（3）光斑中心点的匹配和重建(3) Matching and reconstruction of spot center point

将左右图像的光斑中心点的像面坐标带入公式(3)和(4)得到相匹配光斑中心点在世界坐标系下的三维坐标值：Bring the image plane coordinates of the spot center points of the left and right images into formulas (3) and (4) to obtain the three-dimensional coordinate values of the matching spot center points in the world coordinate system:

G₁′(x₁,y₁,z₁)、G₂′(x₂,y₂,z₂)、G₃′(x₃,y₃,z₃)、G₄′(x₄,y₄,z₄)。G ₁ ′(x ₁ ,y ₁ ,z ₁ ), G ₂ ′(x ₂ ,y ₂ ,z ₂ ), G ₃ ′(x ₃ ,y ₃ ,z ₃ ), G ₄ ′(x ₄ ,y ₄ , z ₄ ).

（4）点邻域曲率判定(4) Point neighborhood curvature determination

1）基于光斑中心离散点最小二乘法拟合空间平面1) Fitting the spatial plane based on the least squares method of discrete points in the center of the spot

将重建出的四个光斑中心作为空间点列，利用最小二乘法拟合空间平面S。将四个光斑中心点的三维坐标带入公式(6)，得到平面方程为z＝a₀x+a₁y+a₂，得到平面的法向量为 The reconstructed four spot centers are taken as the spatial point series, and the spatial plane S is fitted by the least square method. Bring the three-dimensional coordinates of the center points of the four spots into formula (6), and the plane equation is z=a ₀ x+a ₁ y+a ₂ , and the normal vector of the plane is

2）求待测点P′到拟合平面的距离2) Find the distance from the point P′ to be measured to the fitting plane

将点P′(x′,y′,z′)代到空间平面的距离公式(7)求取待测点到拟合平面的距离d并将此距离与所设阈值ε进行比较得到d＞ε。此时应选取两空间曲线在待测点P′(x′,y′,z′)的切矢的叉乘积求取法矢量。Substitute the point P'(x',y',z') into the distance formula (7) to the space plane to find the distance d from the point to be measured to the fitting plane, and compare this distance with the set threshold ε to get d> ε. At this time, the cross product of the tangent vectors of the two space curves at the point P′(x′,y′,z′) to be measured should be selected to obtain the normal vector.

（5）曲面任一点法矢量的求解(5) Solving the normal vector of any point on the surface

1）激光条纹中心线点的提取、匹配和重建1) Extraction, matching and reconstruction of laser stripe centerline points

采用基于方向模板的激光条纹中心检测法，完成激光条纹中心线点的提取，如公式(8)。得到左图像中四个光斑中心点的像面坐标(X_i,Y_i)i＝1,2…n和右图像的四个光斑中心点的像面坐标(X_i′,Y_i′)i′＝1,2…n。将提取的左右图像的像面坐标代入公式(3)和(4)进行光斑中心线点的匹配和重建，得光斑中心线点的三维坐标值为(x_i,y_i,z_i)i＝1,2…n。The laser stripe center detection method based on the direction template is used to complete the extraction of the centerline point of the laser stripe, as shown in formula (8). Get the image plane coordinates (X _i , Y _i )i=1, 2...n of the four spot center points in the left image and the image plane coordinates (X _i′ , Y _i′ )i of the four spot center points in the right image '=1,2...n. Substitute the image plane coordinates of the extracted left and right images into formulas (3) and (4) to match and reconstruct the centerline point of the light spot, and obtain the three-dimensional coordinate value of the centerline point of the light spot (x _i , y _i , z _i )i= 1,2...n.

2）基于空间点离散点数据利用三次B样条曲线拟合2) Using cubic B-spline curve fitting based on discrete point data in space

将重建出的激光条纹中心线点的三维坐标值(x_i,y_i,z_i)i＝1,2…n代入公式(10)和加权方程(13)，求取两曲线的方程。在此基础上分别求取两曲线在待测点两个方向的切向量，记为则所求法矢量为： Substitute the reconstructed three-dimensional coordinates ( _xi , _y , _zi ) i=1, 2...n of the centerline point of the laser stripe into formula (10) and weighting formula (13) to obtain the equations of the two curves. On this basis, the tangent vectors of the two curves in the two directions of the point to be measured are respectively obtained, which is recorded as Then the normal vector obtained is:

本发明的测量方法为非接触式测量，在充分考虑曲面一点邻域的曲率情况下，选取不同的测量方案求取自由曲面任一点法矢量的测量。其方法简单，柔性强、实时性高、算法易于实现，在满足测量精度要求的条件下很好的提高了法矢量求取效率。The measurement method of the present invention is a non-contact measurement, under the condition of fully considering the curvature of a point neighborhood of the curved surface, different measurement schemes are selected to obtain the measurement of the normal vector of any point on the free-form surface. The method is simple, flexible, high real-time, and the algorithm is easy to implement, and the efficiency of calculating the normal vector is well improved under the condition of meeting the measurement accuracy requirements.

Claims

1. the free form surface method vector measurement method based on binocular vision, is characterized in that: in measuring method, by laser projection device to the projection pattern of free form surface projection by two orthogonal straight lines L ₁, L ₂form with four circular light spot G that are positioned on two lines; Two straight-line intersections are P, four circular light spot equal and opposite in directions and being distributed on same circumference; Utilize the image of binocular vision system acquired projections pattern; By the threshold determination condition based on distance, estimate the amount of curvature of some small neighbourhood scopes of curved surface, to choose different method vector measurement schemes: in the time of d≤ε, select the normal vector of fit Plane to approach curved surface tested point method vector; In the time of d > ε, utilize two space curves after matching to ask for the method vector at tested point place at the cross product of tested point place tangent vector; The concrete steps of measuring method are as follows:

(1) spot center based on threshold value grey scale centre of gravity method is extracted

The present invention adopts Canny operator, in conjunction with grey scale centre of gravity method, spot center is carried out to extracted with high accuracy, and in gray level image I (i, j), the grey scale centre of gravity of target S is:

\{\begin{matrix} X_{k} = \frac{\underset{(i, j) &Element; S}{Σ} i \times W (i, j)}{\underset{(i, j) &Element; S}{Σ} W (i, j)} \\ Y_{k} = \frac{\underset{(i, j) &Element; S}{Σ} j \times W (i, j)}{\underset{(i, j) &Element;}{Σ} W (i, j)} \end{matrix} - - - (1)

In formula, (X _k, Y _k) be the image coordinate of k spot center point; The weights of W (i, j) for setting; Consider the half-tone information situation between real background and target, the present invention adopts threshold value grey scale centre of gravity method, and its weights W (i, j) are defined as:

\begin{matrix} W (i, j) = \{\begin{matrix} I (i, j) \\ 0 \end{matrix} \\ (I (i, j) > T) \\ (I (i, j) \leq T) \end{matrix} - - - (2)

Wherein, T is the threshold value of distinguishing target and background; Grey scale centre of gravity is got W (i, j)=I (i, j);

(2) coupling of spot center point and reconstruction

After the extraction that completes spot center point, the spot center point on the image of left and right cameras collection is mated and rebuild; Matching process is as follows:

First adopt 8 normalization algorithms of improvement of Hartley proposition to calculate the fundamental matrix F of left and right cameras, then between the two-dimensional digital image gathering by left and right two high-speed cameras, polar curve restriction relation is carried out the first coupling of spot center point, supposes left image spot central point x _i' and right image spot central point x _i'' match, 2 spot center points meet limiting constraint, and limit equation of constraint can be expressed as follows:

x_{i}^{' T} F {x_{i^{'}}}^{'} = 0 - - - (3)

Wherein, x _i' be the image coordinates of the image spot central point of left camera acquisition; x _i'' be and x _i' matching is gathered the image coordinates of image spot central point by right video camera; F is the fundamental matrix between two video cameras;

On this basis all spot center points that meet limiting constraint in the image of left and right are carried out to three-dimensional reconstruction to obtain the D coordinates value of spot center point under world coordinate system, reconstruction formula is as follows:

\{\begin{matrix} x_{i} = \frac{{zX}_{i}^{'}}{f_{1}} \\ y_{i} = \frac{{zY}_{i}^{'}}{f_{1}} \\ z_{i} = \frac{f_{1} (f_{2} t_{y} - {Y_{i^{'}}}^{'} t_{z})}{Y_{1} (r_{7} X_{i}^{'} + r_{8} Y_{i}^{'} + r_{9} f_{1}) - f_{2} (r_{4} X_{i}^{'} + r_{5} Y_{i}^{'} + r_{6} f_{1})} \end{matrix} - - - (4)

Wherein, x _i'=(X _i', Y _i'), X _i', Y _i' be respectively the image spot central point x of left camera acquisition _i' horizontal stroke, ordinate under image coordinates system; x _i''=(X _i'', Y _i''), X _i'', Y _i'' be respectively the image spot central point x of right camera acquisition _{i '}' horizontal stroke, ordinate under image coordinates system; (x _i, y _i, z _i) be by two coupling spot center point x _i', x _{i '}' rebuild the three-dimensional coordinate of free token point out; f ₁, f ₂be respectively the focal length of left and right cameras; for connecting the rotation matrix of left and right cameras relation, [t _xt _yt _z] be the translation matrix of right video camera with respect to left video camera;

(3) curvature is judged

1) least square fitting space plane

Take four spot center points reconstructing in the D coordinates value of world coordinate system as basis, utilize least square fitting space plane, step is as follows:

The general expression of plane equation is:

\{\begin{matrix} Ax + By + Cz + D = 0, (C &NotEqual; 0) \\ z = - \frac{A}{C} x - \frac{B}{C} y - \frac{D}{C} \end{matrix} - - - (5)

The normal vector that wherein (A, B, C) is plane; D is the distance that initial point arrives plane; Note

a_{0} = - \frac{A}{D}, a_{1} = - \frac{B}{D}, a_{2} = - \frac{C}{D};

Za ₀x+a ₁y+a ₂;

Select least square method to utilize n point (n>=3): (x _i, y _i, z _i), i=0,1 ...,, the above-mentioned plane of n-1 matching, makes:

S = Σ_{i = 0}^{n - 1} {(a_{0} x + a_{1} y + a_{2} - z)}^{2}

Minimum;

Wherein S is the quadratic sum that a little arrives the distance of straight line;

Make S obtain minimum value, should meet: k=0,1,2; That is:

\{\begin{matrix} Σ 2 (a_{0} x_{i} + a_{1} y_{i} + a_{2} - z_{i}) x_{i = 0} \\ Σ 2 (a_{1} x_{i} + a_{1} y_{i} + a_{2} - z_{i}) y_{i} = 0 \\ Σ 2 (a_{1} x_{i} + a_{1} y_{i} + a_{2} - z_{i}) = 0 \end{matrix} - - - (6)

Rebuild the 3 d space coordinate (x of spot center by four _i, y _i, z _i), i=0,1,2,3 bring above-mentioned system of equations into tries to achieve a ₀, a ₁, a ₂;

The equation that is fit Plane is: z=a ₀x+a ₁y+a ₂; The normal vector of spatial fit plane is: 2) ask the distance of tested point P ' to fit Plane

Space a bit can be expressed as to the range formula of plane:

Wherein, S _planefor the equation of spatial fit plane; D is the distance that tested point arrives plane; P '=(x ', y ', z ') be the coordinate of tested point under world coordinate system; Q=(x _q, y _q, z _q) be any point in fit Plane; ε is set threshold value; In the time of d≤ε, think that tested point P ' small neighbourhood curved surface scope intrinsic curvature changes little; In the time of d > ε, think that tested point P ' small neighbourhood curved surface scope intrinsic curvature changes greatly;

(4) method vector solves

Based on the method vector measurement Scheme Choice criterion of distance threshold constraint, situation one: if the tested point P ' being positioned on curved surface meets d≤ε to the distance of spatial fit plane, think that tested point P ' small neighbourhood curved surface scope intrinsic curvature changes little, now think the normal vector of plane be exactly the method vector of tested point on curved surface,

\overset{&RightArrow;}{n} = (a_{0} a_{1}, - 1);

Situation two: if tested point P ' is to the distance d > ε of spatial fit plane on curved surface, think that tested point P ' small neighbourhood curved surface scope intrinsic curvature changes greatly, its Proximal surface may be other quadric surfaces such as sphere, parabola, saddle face, now selects two space curves that project on curved surface to carry out solving of method vector; Its step is as follows:

1) extraction of laser stripe centerline points, coupling and reconstruction

The present invention adopts the laser stripe center line detecting method based on direction template, respectively in level, vertical, left bank 45., right bank 45.In direction, arrange the template that big or small fixed-direction is variable, be designated as respectively K ₀, K ₁, K ₂, K ₃, the every a line of two-dimensional digital image is processed respectively by these four templates; With right _irow is treated to example, for K ₀template has:

\begin{matrix} \{\begin{matrix} H_{j} = Σ_{s = 0}^{M - 1} Σ_{t = 0}^{N - 1} K_{0} [s] [t] C [i - \frac{M}{2} + S] [j - \frac{N}{2} + t], j = \frac{N}{2}, \frac{N + 1}{2}, . . ., col - 1 \\ H_{g 0} = \max (H_{\frac{N}{2},} H_{\frac{N}{2} + 1}, . . ., H_{col - 1}), \frac{N}{2} \leq j \leq col - 1 \end{matrix} - - - (8) \end{matrix}

Wherein M is the corresponding line number of template; N is columns corresponding to template; K ₀[s] [] t>=0; represent the gray-scale value of point; Accordingly for template K ₁, K ₂, K ₃there is H _g1, H _g2, H _g3; Ask for H _g=max (H _g0, H _g1, H _g2, H _g3), there is the position of central point of the capable laser stripe of i at a g place; By the method, two-dimensional digital image is carried out can completing by pixel detection line by line the extraction of laser stripe center line; Complete on the basis of laser stripe central line pick-up, adopt with spot center point coupling in the present invention (2) and rebuild identical method and carry out coupling and the reconstruction of laser stripe central point, obtaining the D coordinates value of laser stripe centerline points under world coordinate system;

2) B-spline Curve matching two space curves

The present invention adopts B-spline Curve matching two space laser striped curves, and B-spline curves piecewise function expression formula is:

\{\begin{matrix} c_{1} : p = P_{0} \cdot {N_{10}}^{3} + P_{1} \cdot {N_{11}}^{3} + P_{2} \cdot {N_{12}}^{3} + P_{3} {N_{13}}^{3} & u &Element; [u_{2}, u_{3}] \\ c_{2} : p = P_{1} \cdot {N_{20}}^{3} + P_{2} \cdot {N_{21}}^{3} + P_{3} \cdot {N_{22}}^{3} + P_{4} {N_{23}}^{3} & u &Element; [u_{3}, u_{4}] \\ c_{1} : p = P_{2} \cdot {N_{30}}^{3} + P_{3} \cdot {N_{31}}^{3} + P_{4} \cdot {N_{32}}^{3} + P_{5} {N_{33}}^{3} & u &Element; [u_{4}, u_{5}] \end{matrix} - - - (9)

Wherein P _i(i=0,1 ... 5) represent respectively control vertex; N _ij(i=1 ... 3, j=0,1 ... 4) represent basis function; Being provided with discrete point on the curve that two camera rebuilding go out is b ₁, b ₂..., b _n; Wherein front i point is positioned at c ₁in section, k-i point is positioned at c ₂in section, n-j point is positioned at c ₃in section, above-mentioned some substitution system of equations is obtained:

Making M is the matrix of coefficients on the left side, the vector forming that P is required control vertex, and the laser stripe centerline points that p is three-dimensional reconstruction, above-mentioned equation is abbreviated as:

The normal equation that MP=p (11) can obtain matching is thus:

M'·M·P＝M'·p (12)

For near the fitting precision of curve raising intersection point, above-mentioned equation is introduced to weights; Equation after weighting is:

(M'·H'·M·P)＝(M'·H')·M·p (13)

Can ask for the equation of two curves by this weighted equation.Ask for respectively on this basis the cut arrow of two curves at tested point both direction, be designated as required method vector is:

\overset{&RightArrow;}{n} = \frac{{\overset{&RightArrow;}{P}}_{n} \times {\overset{&RightArrow;}{P}}_{n}}{| {\overset{&RightArrow;}{P}}_{m} \times {\overset{&RightArrow;}{P}}_{n} |} .