CN101858741A

CN101858741A - A method of zoom distance measurement based on single camera

Info

Publication number: CN101858741A
Application number: CN 201010183102
Authority: CN
Inventors: 高宏伟; 陈付国; 于洋; 姜月秋
Original assignee: Shenyang Ligong University
Current assignee: Shenyang Ligong University
Priority date: 2010-05-26
Filing date: 2010-05-26
Publication date: 2010-10-13

Abstract

The invention belongs to the field of computer stereo vision, relates to a zoom distance measurement method based on a single camera, and proposes a new stereo vision model, through which depth information of a two-dimensional image can be obtained. This method only needs to use a digital camera capable of optical zoom to image the same scene once at two different focal lengths, and then the three-dimensional coordinates of the feature points in the scene in the camera coordinate system can be obtained through image processing and analysis technology. The stereoscopic vision model proposed by the present invention has a novel idea, and it can be said that it has opened up a new direction for the research of computer stereoscopic vision theory. Compared with the currently popular binocular stereovision and other visual ranging technologies, it is easier to use and predictable. Its broad application prospects.

Description

A method of zoom distance measurement based on single camera

技术领域technical field

本发明涉及计算机立体视觉领域，具体的说是公开了一种基于单相机的变焦测距方法，提出了一种新的立体视觉模型，通过该模型可从二维图像获得深度信息。The invention relates to the field of computer stereo vision, and specifically discloses a single-camera-based zoom ranging method, and proposes a new stereo vision model, through which depth information can be obtained from two-dimensional images.

背景技术Background technique

获取场景中各点相对于摄像机的距离是计算机视觉系统的重要任务之一。目前比较成熟的视觉测距技术主要基于以下几种模型：双目立体视觉、结构光法、几何光学法。当然还有很多特定环境下的应用模型，这里不再叙述。Obtaining the distance of each point in the scene relative to the camera is one of the important tasks of the computer vision system. At present, the relatively mature visual ranging technology is mainly based on the following models: binocular stereo vision, structured light method, and geometric optics method. Of course, there are many application models in specific environments, which will not be described here.

其中最为重要的就是双目立体视觉模型，该模型由两个完全相同的摄像机构成，两个摄像机在空间上存在着旋转或平移关系。在这个模型中，场景中同一个特征点在两个摄像机图像平面上的成像位置不同，这两个像点称为匹配点对。我们称两个摄像机投影中心之间的距离为基线，两幅图像重叠时匹配点对之间的距离为视差。通过图像处理中的立体匹配技术和视差计算，便可得到场景中物点的深度信息。The most important of these is the binocular stereo vision model, which consists of two identical cameras, and the two cameras have a rotation or translation relationship in space. In this model, the same feature point in the scene has different imaging positions on the two camera image planes, and these two image points are called matching point pairs. We call the distance between the projection centers of the two cameras the baseline, and the distance between matching point pairs when the two images overlap is the disparity. Through stereo matching technology and disparity calculation in image processing, the depth information of object points in the scene can be obtained.

结构光测距成像系统则使用三角测量原理来计算深度。在一个简单的点投影系统中，投影光源仪和摄像机之间相距一个基线距离，通过确定场景点反射光源光后的成像位置和投影角等参数，就可获得场景的深度信息。一般采用激光作为辅助光源。Structured light ranging imaging systems use the principle of triangulation to calculate depth. In a simple point projection system, there is a baseline distance between the projection light source and the camera, and the depth information of the scene can be obtained by determining the imaging position and projection angle after the scene point reflects the light from the light source. A laser is generally used as an auxiliary light source.

几何光学法主要包括聚焦法和离焦法。聚焦法的基本原理是通过调整摄像机的像距，使得成像平面相对于被测点处在聚焦位置，在焦距和像距已知的条件下，可通过透镜成像公式求得物距。聚焦法原理比较简单，但其精度受到硬件的严重限制，不同深度区域需要重新聚焦，因而测量繁琐缓慢，所以应用很少。离焦法避免了寻找精确聚焦位置的操作，它利用物点不聚焦时成像圆斑的大小，即图像的模糊程度，来获取深度信息。但离焦模型的准确标定是其难度所在。Geometric optics mainly includes focusing method and defocusing method. The basic principle of the focusing method is to adjust the image distance of the camera so that the imaging plane is at the focal position relative to the measured point. Under the condition of known focal length and image distance, the object distance can be obtained through the lens imaging formula. The principle of the focusing method is relatively simple, but its accuracy is severely limited by the hardware, and different depth areas need to be refocused, so the measurement is cumbersome and slow, so it is rarely used. The defocus method avoids the operation of finding the precise focus position. It uses the size of the imaging circle spot when the object point is not in focus, that is, the blurring degree of the image, to obtain depth information. But the accurate calibration of the defocus model is the difficulty.

发明内容Contents of the invention

本发明目的在于提出一种新的立体视觉模型，通过该模型可获得二维图像的深度信息。其价值在于该模型为计算机立体视觉理论的研究开拓了新方向。The purpose of the present invention is to propose a new stereo vision model, through which the depth information of the two-dimensional image can be obtained. Its value lies in that the model opens up a new direction for the research of computer stereo vision theory.

本发明提出的基于单相机的变焦测距方法，只需用一个可以光学变焦的数码相机在两个不同的焦距下对同一场景分别成像一次，便可以通过图像处理与分析技术得出场景中的特征点在相机坐标系内的三维坐标。其操作过程主要有以下几步：The single-camera-based zoom distance measurement method proposed by the present invention only needs to use a digital camera capable of optical zoom to image the same scene once at two different focal lengths, and then the distance in the scene can be obtained through image processing and analysis technology. The three-dimensional coordinates of the feature points in the camera coordinate system. Its operation process mainly has the following steps:

步骤一：将相机对准要测距的场景，并固定；Step 1: Aim the camera at the scene to be measured and fix it;

步骤二：在焦距f₁、f₂分别成像一次，并将两次所采集的图像作为一对立体图像保存起来，一般称在较小焦距下的成像为远景图，在较大焦距下的成像为近景图；(不失一般性，可假定焦距f₁＜f₂)Step 2: Imaging once at the focal lengths f ₁ and f ₂ respectively, and saving the images collected twice as a pair of stereoscopic images. Generally, the imaging at a smaller focal length is called a distant view image, and the imaging at a larger focal length is a close-up image; (without loss of generality, it can be assumed that the focal length f ₁ <f ₂ )

步骤三：完成立体图像的匹配；Step 3: complete the matching of stereoscopic images;

步骤四：按要求计算特征点的三维坐标。Step 4: Calculate the three-dimensional coordinates of the feature points as required.

其中最为关键的一步是图像的立体匹配，值得注意的地方有以下几点：The most critical step is the stereo matching of the image. The following points are worth noting:

一、可采用SIFT特征匹配在远景图中标定出两幅图像的公共场景，因为远景图与近景图相比，远景图包括的空间范围更广，而近景图就是两幅图像的公共场景；1. SIFT feature matching can be used to calibrate the common scene of the two images in the distant view image, because the distant view image includes a wider spatial range than the close view image, and the close view image is the common scene of the two images;

二、应先在远景图的公共场景内选取像素点，然后再到近景图中搜索其对应的匹配点。因为近景图中场景的细节描述更为丰富，它应该包含了远景图中公共场景的所有细节，即两幅图像的公共场景内远景图的像点在近景图中都有匹配点，而反过来不一定成立；2. Pixels should be selected in the public scene of the long-range image first, and then the corresponding matching points should be searched in the close-range image. Because the detailed description of the scene in the near view image is richer, it should contain all the details of the common scene in the distant view image, that is, the image points in the distant view image in the common scene of the two images have matching points in the close view image, and vice versa not necessarily established;

三、可在局部范围内搜索匹配点，因为理想模型下，以光心为原点建立的图像坐标系中，相互对应的两个匹配点过光心的斜率应相同，且近景图中匹配点的极半径大于远景图中对应点的极半径。3. You can search for matching points in a local range, because under the ideal model, in the image coordinate system established with the optical center as the origin, the slopes of the two corresponding matching points passing through the optical center should be the same, and the matching points in the close-up image The polar radius is larger than the polar radius of the corresponding point in the perspective map.

附图说明Description of drawings

图1为本发明的基于单相机的变焦测距原理图。FIG. 1 is a principle diagram of zoom distance measurement based on a single camera in the present invention.

图2为本发明的变焦测距模型的空间平面截取图。Fig. 2 is a space plane interception diagram of the zoom ranging model of the present invention.

图3为本发明的一对立体图像。Fig. 3 is a pair of stereoscopic images of the present invention.

具体实施方式Detailed ways

下面结合附图对本发明作进一步描述。The present invention will be further described below in conjunction with the accompanying drawings.

如图1所示的基于单相机的变焦测距原理图，该模型与双目立体视觉模型一样也是基于针孔成像模型进行几何分析。以光心o为原点，建立相机的空间直角坐标系oxyz，其中平面xoy为图像平面。固定相机对同一空间分别在焦距f₁、f₂下各成像一次(这里假定f₁＜f₂)，场景内任意一点s在焦距等于f₁时的成像为s₁在焦距等于f₂时的成像为s₂。将两次成像的模型画在同一坐标系中进行分析。理想模型下像点s₁、s₂都在直线om上，s s₁是过焦距f₁下透镜中心o₁的直线，s s₂是过焦距f₂下透镜中心o₂的直线，其中o o₁＝f₁，o o₂＝f₂。平面som是过z轴的平面，且垂直于像平面xoy。As shown in Figure 1, the principle diagram of zoom distance measurement based on single camera, this model is also based on the pinhole imaging model for geometric analysis like the binocular stereo vision model. With the optical center o as the origin, establish the space Cartesian coordinate system oxyz of the camera, where the plane xoy is the image plane. The fixed camera images the same space once at the focal lengths f ₁ and f ₂ respectively (here it is assumed that f ₁ <f ₂ ), the imaging of any point s in the scene when the focal length is equal to f ₁ is the image of s ₁ when the focal length is equal to f ₂ Imaged as s ₂ . Draw the two-imaging model in the same coordinate system for analysis. Under the ideal model, the image points s ₁ and s ₂ are both on the straight line om, s s ₁ is the straight line passing through the lens center o ₁ at the focal length f ₁ , and s s ₂ is the straight line passing through the lens center o ₂ at the focal length f ₂ , where o o ₁ = f ₁ , o o ₂ =f ₂ . The plane som is a plane passing through the z-axis and perpendicular to the image plane xoy.

截取平面som单独分析，如图2所示。r₁、r₂为像点s₁、s₂到光心的距离，即在平面xoy内的极半径，如图3所示。这里假定f₁＜f₂，图3中abcd表示像平面的大小，a′b′c′d′则表示图像1和图像2的公共场景。因为f₁＜f₂，所以图像1与图像2相比，图像1包含更广的空间，而像2包含更多的细节。我们可以先在图像1的a′b′c′d′内任意取一点s₁，通过特征相关(如彩色值)在图像2中的直线om上找到s₁的匹配点s₂。这样就可以确定r₁、r₂的值了。因此图2中∠α、∠β可解，三角形s o₁ o₂也可解。The intercept plane som is analyzed separately, as shown in Figure 2. r ₁ and r ₂ are the distances from the image points s ₁ and s ₂ to the optical center, that is, the polar radius in the plane xoy, as shown in Fig. 3 . It is assumed here that f ₁ < f ₂ , abcd in FIG. 3 represents the size of the image plane, and a′b′c′d′ represents the common scene of image 1 and image 2 . Because f ₁ < f ₂ , compared with image 2, image 1 contains a wider space, while image 2 contains more details. We can first select a point s ₁ arbitrarily in a'b'c'd' of image 1, and find the matching point s ₂ of s ₁ on the straight line om in image 2 through feature correlation (such as color value). In this way, the values of r ₁ and r ₂ can be determined. Therefore, ∠α and ∠β in Figure 2 can be solved, and the triangle s o ₁ o ₂ can also be solved.

如图2所示，线段os为场景内点s到光心的距离，即在像机坐标系oxyz内的极半径r。由以上分析可知三角形soo₁也可解。设点s在像机坐标系oxyz内的坐标为(x，y，z)，则有x＝rsinγcosθ，y＝rsinγsinθ，z＝rcosγ。As shown in Figure 2, the line segment os is the distance from the point s in the scene to the optical center, that is, the polar radius r in the camera coordinate system oxyz. From the above analysis, it can be known that the triangle soo ₁ can also be solved. Let the coordinates of point s in camera coordinate system oxyz be (x, y, z), then x=rsinγcosθ, y=rsinγsinθ, z=rcosγ.

那么只要求解出r、sinγ、cosγ、sinθ、cosθ，就可以确定点s在相机坐标系xyz内的坐标(x，y，z)，其求解过程如下：Then only by solving r, sinγ, cosγ, sinθ, cosθ, the coordinates (x, y, z) of point s in the camera coordinate system xyz can be determined, and the solution process is as follows:

一、设平面xoy内点s₁坐标为(i，j)，匹配点s₂坐标为(k，n)则有：1. Let the coordinates of point _s1 in the plane xoy be (i, j), and the coordinates of matching point _s2 be (k, n), then:

$sin sin θ θ = = - - \frac{i i}{\sqrt{}} - - - - - - ((11))$

$cos cos θ θ = = - - \frac{j j}{\sqrt{}} - - - - - - ((22))$

${r r}_{11} = = \sqrt{{i i}^{22} + + {j j}^{22}} - - - - - - ((33))$

${r r}_{22} = = \sqrt{{k k}^{22} + + {n no}^{22}} - - - - - - ((44))$

二、先求解三角形s o₁ o₂：2. Solve the triangle s o ₁ o ₂ first:

设d＝o₁ o₂＝f₁-f₂，a＝s o₁，b＝s o₂。Let d=o ₁ o ₂ =f ₁ −f ₂ , a=s o ₁ , b=s o ₂ .

由以上推导可知：It can be seen from the above derivation that:

$sin sin α α = = \frac{{r r}_{11}}{\sqrt{{f f}_{11}^{22} + + {r r}_{11}^{22}}} - - - - - - ((55))$

$cos cos α α = = \frac{{f f}_{11}}{\sqrt{{f f}_{11}^{22} + + {r r}_{11}^{22}}} - - - - - - ((66))$

$tan the tan α α = = \frac{{r r}_{11}}{{f f}_{11}} - - - - - - ((77))$

$cot cot α α = = \frac{{f f}_{11}}{{r r}_{11}} - - - - - - ((88))$

$sin sin β β = = \frac{{r r}_{22}}{\sqrt{{f f}_{22}^{22} + + {r r}_{22}^{22}}} - - - - - - ((99))$

$cos cos β β = = \frac{{f f}_{22}}{\sqrt{{f f}_{22}^{22} + + {r r}_{22}^{22}}} - - - - - - ((1010))$

$tan the tan β β = = \frac{{r r}_{22}}{{f f}_{22}} - - - - - - ((1111))$

$cot cot β β = = \frac{{f f}_{22}}{{r r}_{22}} - - - - - - ((1212))$

有have

d²＝a²+b²-2ab cos(β-α) (13)d ² =a ² +b ² -2ab cos(β-α) (13)

a²＝d²+b²-2db cos(180-β) (14)a ² ＝d ² +b ² -2db cos(180-β) (14)

b²＝d²+a²-2da cosα (15)b ² =d ² +a ² -2da cosα (15)

将(15)代入(13)，(14)代入(15)得Substituting (15) into (13), (14) into (15) to get

a＝b cos(β-α)+d cosα (16)a=b cos(β-α)+d cosα (16)

d＝b cos(180-β)+a cosα (17)d=b cos(180-β)+a cosα (17)

再将(16)代入(17)得Then substitute (16) into (17) to get

$b b = = \frac{d d sin sin α α}{sin sin α α cos cos β β + + cos cos α α sin sin β β}$

$= = \frac{d d}{tan the tan α α cos cos β β + + cot cot α α sin sin β β} - - - - - - ((1818))$

将(18)代入(16)Substitute (18) into (16)

$a a = = \frac{sin sin β β + + 22 sin sin α α cos cos α α cos cos β β}{sin sin α α cos cos β β + + cos cos α α sin sin β β} d d$

$= = \frac{tan the tan β β / / sin sin α α + + 22 cos cos α α}{11 + + cot cot α α tan the tan β β} d d - - - - - - ((1919))$

三、再求解三角形soo₁：3. Solve the triangle soo ₁ again:

$r r = = \sqrt{{f f}_{11}^{22} + + {a a}^{22} + + 22 {f f}_{11} a a cos cos α α} - - - - - - ((2020))$

$cos cos γ γ = = \frac{{f f}_{11}^{22} + + {r r}^{22} - - {a a}^{22}}{22 {f f}_{11} r r}$

$= = \frac{{f f}_{11} + + a a cos cos α α}{r r} - - - - - - ((21 twenty one))$

$sin sin γ γ = = \sqrt{11 - - {cos cos}^{22} γ γ}$

$= = \frac{a a sin sin α α}{r r} - - - - - - ((22 twenty two))$

这样就完成了r、sinγ、cosγ、sinθ、cosθ的求解，由公式In this way, the solution of r, sinγ, cosγ, sinθ, cosθ is completed, and the formula

$\{\begin{matrix} x x = = r r sin sin γ γ cos cos θ θ & ((23 twenty three)) \\ y the y = = r r sin sin γ γ sin sin θ θ & ((24 twenty four)) \\ z z = = r r cos cos γ γ & ((2525)) \end{matrix}$

可得点s在相机坐标系oxyz内的坐标为The coordinates of the available point s in the camera coordinate system oxyz are

$\{\begin{matrix} x x = = j j (({f f}_{11} - - {f f}_{22})) \frac{\sqrt{{k k}^{22} + + {n no}^{22}} + + 22 {f f}_{11} {f f}_{22} [[11 - - {f f}_{11}^{22} / / (({f f}_{11}^{22} + + {i i}^{22} + + {j j}^{22}))]] / / \sqrt{{i i}^{22} + + {j j}^{22}}}{{f f}_{22} \sqrt{{i i}^{22} + + {j j}^{22}} + + {f f}_{11} \sqrt{{k k}^{22} + + {n no}^{22}}} & ((2626)) \\ y the y = = i i (({f f}_{11} - - {f f}_{22})) \frac{\sqrt{{k k}^{22} + + {n no}^{22}} + + 22 {f f}_{11} {f f}_{22} [[11 - - {f f}_{11}^{22} / / (({f f}_{11}^{22} + + {i i}^{22} + + {j j}^{22}))]] / / \sqrt{{i i}^{22} + + {j j}^{22}}}{{f f}_{22} \sqrt{{i i}^{22} + + {j j}^{22}} + + {f f}_{11} \sqrt{{k k}^{22} + + {n no}^{22}}} & ((2727)) \\ z z = = {f f}_{22} + + (({f f}_{22} - - {f f}_{11})) \frac{{22 f f}_{11}^{22} / / (({f f}_{11}^{22} + + {i i}^{22} + + {j j}^{22})) - - {f f}_{22} \sqrt{{i i}^{22} + + {j j}^{22}}}{{f f}_{22} ((\sqrt{{i i}^{22} + + {j j}^{22}} + + {f f}_{11} \sqrt{{k k}^{22} + + {n no}^{22}}))} & ((2828)) \end{matrix}$

其中(i，j)、(k，n)匹配点对s₁、s₂的坐标，f₁、f₂为焦距，且f₁＜f₂。因为是倒立成像，所以空间点s的在平面xoy内的坐标点和成像点总是在相反的区间内。如果将世界坐标系与相机坐标系重合，则测得的三维坐标即为世界坐标系下绝对坐标。Where (i, j), (k, n) match the coordinates of point pair s ₁ , s ₂ , f ₁ , f ₂ are focal lengths, and f ₁ <f ₂ . Because it is inverted imaging, the coordinate point and imaging point of the space point s in the plane xoy are always in the opposite interval. If the world coordinate system coincides with the camera coordinate system, the measured 3D coordinates are the absolute coordinates in the world coordinate system.

Claims

1. A zoom distance measuring method based on stand-alone, it is characterized in that: a digital camera needs to be used to move a certain distance along the optical axis direction, and the same scene is imaged once respectively at two positions before and after moving, just can pass image processing and The analysis technology obtains the three-dimensional coordinates of the feature points in the scene in the camera coordinate system;

Its operation process includes the following steps:

Step 1: Aim the camera and its sliding mechanism at the scene to be measured;

Step 2: Use the sliding mechanism to move the camera along the optical axis for a certain distance, and image the same scene once at the two positions before and after the movement, and save the images collected twice as a pair of stereo images, assuming that the camera moves forward Moving, generally speaking, the imaging before moving is the long-range image, and the imaging before and after the moving is the close-up image;

Step 3: Complete the matching of the stereoscopic image: use SIFT feature matching to calibrate the common scene of the two images in the distant view image, because the distant view image is different from the close view image, the distant view image includes a wider spatial range, and the close view image is two The public scene of the image; then select the pixel points in the public scene of the distant view, and then search for its corresponding matching point in the close view, because the detailed description of the scene in the close view is richer, it should contain the common All the details of the scene, that is, the image points of the distant view image in the common scene of the two images have matching points in the near view image, but the reverse is not necessarily true; finally, the matching point can be searched in a local range, because under the ideal model, the In the image coordinate system established with the optical center as the origin, the slopes of the two matching points corresponding to each other passing through the optical center should be the same, and the polar radius of the matching point in the near view image is larger than the polar radius of the corresponding point in the distant view image.