CN105374039A

CN105374039A - Monocular image depth information estimation method based on contour acuity

Info

Publication number: CN105374039A
Application number: CN201510786727.1A
Authority: CN
Inventors: 马利; 景源; 李鹏; 张玉奇; 胡彬彬; 牛斌
Original assignee: Liaoning University
Current assignee: Liaoning University
Priority date: 2015-11-16
Filing date: 2015-11-16
Publication date: 2016-03-02
Anticipated expiration: 2035-11-16
Also published as: CN105374039B

Abstract

The present invention proposes a monocular image depth information estimation method based on contour sharpness, which uses edge contour sharpness as fuzzy information estimation features, and extracts depth information through low-level clue information. First, edge detection is performed on the image; then the edge energy and contour sharpness are calculated for the edges in the image, and the edge energy and contour sharpness are used as the external energy of the contour, combined with the internal energy of the contour—contour line characteristic energy and contour line distance Energy establishes the contour tracking model, solves the minimum value of the energy function, and searches the image contour; then uses the depth gradient assumption as the prior assumption gradient model, fills the depth value of the area with different contour lines, and calculates the depth distribution; finally uses The original image information and the obtained depth image information are optimized for the obtained depth image to obtain a final disparity map. Experimental results prove that the depth estimation algorithm of the present invention is simple and can quickly and accurately estimate the depth map of a monocular image.

Description

Depth Information Estimation Method of Monocular Image Based on Contour Sharpness

技术领域technical field

本发明涉及一种能够对单目图像进行深度信息估计方法，该方法能够通过图像低层次线索得到单目图像中所示物体的深度信息。The invention relates to a method capable of estimating the depth information of a monocular image, and the method can obtain the depth information of an object shown in the monocular image through low-level clues of the image.

背景技术Background technique

深度信息感知是产生立体视觉的前提，基于深度信息的实际应用在场景重解、三维重建、模式识别、目标跟踪等领域中发挥重要作用。在实际应用中，可采用单目摄像机、多目摄像机或深度摄像机进行深度信息提取。其中，基于单目摄像机的深度信息估计方法由于其实际操作简单、硬件成本低、能够从已有的单目图像素材中直接提取深度信息而具有一定的优势。Depth information perception is the premise of producing stereo vision, and the practical application based on depth information plays an important role in the fields of scene reconstruction, 3D reconstruction, pattern recognition, and object tracking. In practical applications, monocular cameras, multi-cameras or depth cameras can be used for depth information extraction. Among them, the depth information estimation method based on a monocular camera has certain advantages because of its simple operation, low hardware cost, and the ability to directly extract depth information from existing monocular image materials.

单目图像深度信息估计通过单幅图像提取目标的颜色、形状、共面性等二维、三维几何信息，从而利用少量已知条件获取该目标的空间三维信息。目前大多算法都是采用图像高层次、中层次线索实现。如经过学习参考图像得到已知图像的语义标签，利用其实现对目标图像定义语义标签，从而得到相对深度信息。利用对真实视差图像中训练得到的图像结构信息，如颜色、纹理、形状等，对目标图像进行过分割，应用有区分训练的马尔可夫随机场推导出深度信息。而利用低层次线索信息不需要对图像进行内容的分析，仅需要直接应用局部信息即可从图像中恢复深度信息，算法相对简单。Monocular image depth information estimation extracts the two-dimensional and three-dimensional geometric information of the target such as color, shape, and coplanarity through a single image, so as to obtain the spatial three-dimensional information of the target with a small number of known conditions. At present, most algorithms are realized by using high-level and middle-level clues of images. For example, the semantic label of the known image is obtained by learning the reference image, and it is used to define the semantic label of the target image, so as to obtain the relative depth information. The target image is over-segmented by using the image structure information obtained from the training of the real disparity image, such as color, texture, shape, etc., and the depth information is derived by applying the Markov random field with discriminative training. However, the use of low-level clue information does not need to analyze the content of the image, and only needs to directly apply the local information to recover the depth information from the image, and the algorithm is relatively simple.

在单目图像的深度信息估计过程中可以利用模糊信息作为深度估计的一个重要特征。模糊信息多是在成像过程中对焦不准或成像区域内存在不同深度的目标而造成的。根据这一特性，单目图像深度信息估计通过对图像做散焦处理，根据图像的模糊情况确定图像的前景和背景来估计场景的深度。如以边缘位置的模糊值作为初始，应用消光和马尔可夫随机场使模糊扩散到整幅图像，来实现相对深度提取。或者利用热扩离过程实现离焦模糊过程的建模，利用不均匀逆热扩散方程估计边缘位置模糊量来恢复场景深度。而由于模糊扩散方法复杂，效率较低，其实用性较差。Blur information can be used as an important feature of depth estimation in the process of depth information estimation of monocular images. Blurred information is mostly caused by inaccurate focus during imaging or objects with different depths in the imaging area. According to this characteristic, monocular image depth information estimation estimates the depth of the scene by defocusing the image and determining the foreground and background of the image according to the blurring of the image. For example, the blur value of the edge position is used as the initial value, and the extinction and Markov random field are applied to diffuse the blur to the entire image to achieve relative depth extraction. Or use the thermal diffusion process to realize the modeling of the defocus blur process, and use the non-uniform inverse thermal diffusion equation to estimate the amount of blur at the edge position to restore the scene depth. However, due to the complexity and low efficiency of the fuzzy diffusion method, its practicability is poor.

因此，本发明利用图像低层次线索及新的模糊信息特性，对单目图像深度信息估计方法进行了简化。Therefore, the present invention simplifies the monocular image depth information estimation method by utilizing the low-level clues of the image and the new fuzzy information characteristics.

发明内容Contents of the invention

本发明提出一种基于轮廓锐度的单目图像深度信息估计方法，该方法利用图像低层次线索信息，以边缘的轮廓锐度信息作为模糊信息估计特征进行物体轮廓提取，根据物体轮廓与物体深度边缘的联系对不同物体进行深度分配，从而得到图像中不同物体的深度信息。The present invention proposes a monocular image depth information estimation method based on contour sharpness. This method utilizes image low-level clue information, and uses edge contour sharpness information as fuzzy information estimation features to extract object contours. According to the object contour and object depth The connection of the edge assigns the depth of different objects, so as to obtain the depth information of different objects in the image.

本发明的目的是通过下述技术方案实现的：The purpose of the present invention is achieved through the following technical solutions:

基于轮廓锐度的单目图像深度信息估计方法，其特征在于，利用梯度幅度和轮廓锐度信息反映图像中具有不同深度的物体轮廓的特性，在进行深度估计时，不仅仅考虑边缘的梯度幅度，同时加入边缘的空间信息，更充分地体现了物体边缘模糊变化趋势。包括如下步骤：The monocular image depth information estimation method based on the contour sharpness is characterized in that the gradient magnitude and contour sharpness information is used to reflect the characteristics of the contours of objects with different depths in the image, and not only the gradient magnitude of the edge is considered when performing depth estimation , while adding the spatial information of the edge, it can more fully reflect the blurring trend of the edge of the object. Including the following steps:

(1)对于输入图像，进行边缘检测，得到图像中物体的边缘点P＝{p₁,p₂,…p_n}；(1) For the input image, perform edge detection to obtain the edge point P={p ₁ ,p ₂ ,...p _n } of the object in the image;

(2)根据先验深度梯度模型进行初始轮廓线的定义，定义一组互相平行且间距相等的轮廓线作为初始轮廓线V＝{v₀,v₁…v_m}；其中，v(s)＝[x(s),y(s)]，x(s),y(s)为初始轮廓线中的点s的坐标；(2) Define the initial contour line according to the prior depth gradient model, and define a group of contour lines parallel to each other and with equal spacing as the initial contour line V={v ₀ ,v ₁ …v _m }; among them, v(s) =[x(s), y(s)], x(s), y(s) are the coordinates of the point s in the initial contour line;

轮廓线的总能量定义为：The total energy of the contour is defined as:

E_total＝w₁E_edge+w₂E_sharp+w₃E_s+w₄E_d E _total ＝w ₁ E _edge +w ₂ E _sharp +w ₃ E _s +w ₄ E _d

其中E_edge为轮廓边缘能量函数，E_sharp轮廓锐度能量函数，E_s为轮廓线特性能量函数，E_d为轮廓线距离能量函数，w₁、w₂、w₃、w₄为各个函数权重控制参数，可根据具体图像特性赋值。Where E _edge is the contour edge energy function, E _sharp contour sharpness energy function, E _s is the contour line characteristic energy function, E _d is the contour line distance energy function, w ₁ , w ₂ , w ₃ , w ₄ are the weights of each function The control parameters can be assigned according to specific image characteristics.

轮廓边缘能量函数E_edge表示轮廓的梯度幅度，定义为图像I(x,y)沿着梯度方向的梯度幅度大小，表示为：The contour edge energy function E _edge represents the gradient magnitude of the contour, which is defined as the gradient magnitude of the image I(x,y) along the gradient direction, expressed as:

${E E.}_{e e d d g g e e} = = exp exp ((- - \frac{{| | g g ((I I ((x x,, y the y)))) | |}^{22}}{a a}))$

其中， $g (I (x, y)) = \sqrt{{(\partial_{x} I (x, y))}^{2} + {(\partial_{y} I (x, y))}^{2}}$ 是对I(x,y)求梯度幅度；in, $g (I (x, the y)) = \sqrt{{(\partial_{x} I (x, the y))}^{2} + {(\partial_{the y} I (x, the y))}^{2}}$ It is to find the gradient magnitude of I(x,y);

轮廓锐度能量函数E_sharp代表轮廓的模糊程度，通过对梯度轮廓锐度求解得到。定义为：The contour sharpness energy function E _sharp represents the blurring degree of the contour, which is obtained by solving the sharpness of the gradient contour. defined as:

${E E.}_{s the s h h a a r r p p} = = exp exp ((- - \frac{σ σ {((p p (({q q}_{00}))))}^{22}}{b b}));;$

其中，梯度轮廓锐度σ(p(q₀))为梯度轮廓变量方差的均方根。这里梯度轮廓为图像中边缘像素q₀(x₀,y₀)作为起始点，沿梯度方向向边缘的边界追踪，直到边缘的梯度幅度不再发生变化，所得到一维路径p(q₀)形成的梯度幅度曲线。其梯度轮廓锐度定义为Wherein, the gradient profile sharpness σ(p(q ₀ )) is the root mean square of the variance of the gradient profile variable. Here the gradient profile is the edge pixel q ₀ (x ₀ , y ₀ ) in the image as the starting point, and traces along the gradient direction to the boundary of the edge until the gradient amplitude of the edge no longer changes, and the obtained one-dimensional path p(q ₀ ) The resulting gradient magnitude curve. Its gradient profile sharpness is defined as

$σ σ ((p p (({q q}_{00})))) = = \sqrt{\underset{x x &Element; &Element; p p (({q q}_{00}))}{Σ Σ} \frac{g g ((q q))}{G G (({q q}_{00}))} {d d}_{c c}^{22} ((q q,, {q q}_{00}))};; G G (({q q}_{00})) = = \underset{s the s &Element; &Element; p p (({q q}_{00}))}{Σ Σ} g g ((s the s))$

其中，d_c(q,q₀)为梯度轮廓中点q和q₀之间的曲线长度，g(q)为q点处的梯度幅度，G(q₀)为梯度轮廓中所有点的梯度幅度和，s为梯度轮廓中任一点，参数b为权值控制参数。where d _c (q,q ₀ ) is the curve length between points q and q ₀ in the gradient profile, g(q) is the gradient magnitude at point q, and G(q ₀ ) is the gradient of all points in the gradient profile Amplitude sum, s is any point in the gradient profile, and parameter b is the weight control parameter.

轮廓线特性能量函数E_s用于约束轮廓的平滑度；The contour line characteristic energy function E _s is used to constrain the smoothness of the contour;

轮廓线距离能量函数E_d用于控制轮廓跟踪曲线不会超出搜索区域；The contour line distance energy function E _d is used to control the contour tracking curve and will not exceed the search area;

(3)对图像中每一条初始轮廓线从图像的左侧起始点开始，由图像底部向顶部更新每一轮廓点的位置，对每一个轮廓点根据步骤(2)中轮廓线总能量定义计算轮廓跟踪能量；(3) For each initial contour line in the image, start from the left starting point of the image, update the position of each contour point from the bottom of the image to the top, and calculate according to the definition of the total energy of the contour line in step (2) for each contour point Contour tracking energy;

(4)求解能量函数的最小值；对与当前轮廓点相邻的像素列中的像素点进行搜索，搜索满足能量函数定义的边缘点P＝{p₁,p₂,…p_n}中具有最小能量值的点，选择具有最小能量的像素点位置作为新的轮廓点；从图像左侧至右侧，重复搜索最小值，得到最终的轮廓搜索结果，即得到目标轮廓线V；(4) Solve the minimum value of the energy function; search the pixel points in the pixel column adjacent to the current contour point, and search for the edge points P={p ₁ ,p ₂ ,...p _n } that satisfy the definition of the energy function. For the point with the minimum energy value, select the pixel point position with the minimum energy as the new contour point; from the left side of the image to the right side, repeat the search for the minimum value to obtain the final contour search result, that is, the target contour line V;

(5)以深度梯度假设作为先验假设梯度模型，对具有不同的轮廓线的区域进行深度值填充，计算得出深度分布。(5) Using the depth gradient assumption as the prior assumption gradient model, the depth value is filled in the regions with different contour lines, and the depth distribution is calculated.

轮廓线集中轮廓线V＝{v₀,v₁,…,v_m}，对应的分配深度值Depth为：Contour line set contour line V={v ₀ ,v ₁ ,…,v _m }, the corresponding assigned depth value Depth is:

$D D. e e p p t t h h (({v v}_{i i})) = = 255255 \times \times \frac{m m - - i i}{m m},, i i = = 00 ... ... m m$

(6)利用原图像灰度信息和所得深度图像信息对得到的深度图像进行优化处理：(6) Use the original image grayscale information and the obtained depth image information to optimize the obtained depth image:

${Depth Depth}_{f f}^{b b f f} (({x x}_{i i})) = = \frac{11}{W W (({x x}_{i i}))} \underset{{x x}_{j j} &Element; &Element; Ω Ω (({x x}_{i i}))}{Σ Σ} exp exp (({G G}_{{σ σ}_{s the s}} ((| | | | {x x}_{i i} - - {x x}_{j j} | | | |)) {G G}_{{σ σ}_{r r}} ((I I (({x x}_{i i})) - - I I (({x x}_{j j})))))) D D. e e p p t t h h (({x x}_{j j}))$

$W W (({x x}_{i i})) = = \underset{q q &Element; &Element; S S}{Σ Σ} exp exp (({G G}_{{σ σ}_{s the s}} ((| | | | {x x}_{i i} - - {x x}_{j j} | | | |)) {G G}_{{σ σ}_{r r}} ((I I (({x x}_{i i})) - - I I (({x x}_{j j}))))))$

其中Depth(x_i)为输入深度图像，Ω(x_i)是以像素x_i为中心的邻域，I(x_i)为像素x_i的亮度I分量，x_j为像素x_i在邻域Ω(x_i)的邻域像素，W(x_i)是滤波器参数的归一化因子；||x_i-x_j||为两像素的空间欧式距离；I(x_i)-I(x_j)表示两像素的亮度相似性，像素x_i和x_j的空间坐标分别为(x_ix,x_iy)和(x_jx,x_jy)。空间权重系数和色度权重系数定义为：Where Depth( _xi ) is the input depth image, Ω( _xi ) is the neighborhood centered on pixel _xi , I( _xi ) is the brightness I component of pixel _xi , and x _j is the neighborhood of pixel _xi Neighborhood pixels of Ω( _xi ), W( _xi ) is the normalization factor of filter parameters; || _xi -x _j || is the spatial Euclidean distance of two pixels; I( _xi )-I( x _j ) represents the brightness similarity of two pixels, and the spatial coordinates of pixels x _i and x _j are (x _ix , x _iy ) and (x _jx , x _jy ), respectively. Spatial weight coefficient and chroma weighting coefficients defined as:

${G G}_{{σ σ}_{s the s}} ((| | | | {x x}_{i i} - - {x x}_{j j} | | | |)) = = \frac{{[[\sqrt{{(({x x}_{j j x x} - - {x x}_{i i x x}))}^{22} + + {(({x x}_{j j y the y} - - {x x}_{i i y the y}))}^{22}}]]}^{22}}{22 {σ σ}_{s the s}^{22}}$

${G G}_{{σ σ}_{r r}} ((I I (({x x}_{i i})) - - I I (({x x}_{j j})))) = = \frac{{| | I I (({x x}_{j j})) - - I I (({x x}_{i i})) | |}^{22}}{22 {σ σ}_{r r}^{22}};;$

其中σ_s为空间权重的方差、σ_r为色度权重的方差。where σ _s is the variance of the spatial weights, and σ _r is the variance of the chrominance weights.

(7)得到了对输入单目图像进行深度信息估计后的视差图。(7) The disparity map after depth information estimation of the input monocular image is obtained.

本发明的优点是，提出了一种基于轮廓锐度的单目图像深度信息估计方法。传统的单目图像深度信息估计方法需要利用图像高、中层次线索进行学习训练、图像理解等步骤，算法复杂。而本方法则是利用图像低层次线索，计算简单。与之传统利用模糊信息区分物体深度方法不同的是，本发明在计算模糊信息时，利用轮廓锐度信息有效区分物体轮廓，从而得到具有不同深度的物体轮廓，避免了模糊扩散等步骤，提高了方法的实验性。本方法利用梯度幅度和轮廓锐度信息反映图像中具有不同深度的物体轮廓的特性，在进行深度估计时，不仅仅考虑边缘的梯度幅度，同时加入边缘的空间信息，更充分地体现了物体边缘模糊变化趋势。The invention has the advantage of proposing a method for estimating the depth information of a monocular image based on contour sharpness. Traditional monocular image depth information estimation methods need to use image high-level and mid-level clues for learning and training, image understanding and other steps, and the algorithm is complex. However, this method uses the low-level clues of the image, and the calculation is simple. Different from the traditional method of using fuzzy information to distinguish object depth, the present invention uses contour sharpness information to effectively distinguish object contours when calculating fuzzy information, thereby obtaining object contours with different depths, avoiding steps such as blur diffusion, and improving The experimental nature of the method. This method uses the gradient magnitude and contour sharpness information to reflect the characteristics of object contours with different depths in the image. When performing depth estimation, not only the gradient magnitude of the edge is considered, but also the spatial information of the edge is added to fully reflect the edge of the object. Fuzzy changing trends.

附图说明Description of drawings

图1是本方法流程图。Figure 1 is a flow chart of the method.

图2显示了轮廓锐度的定义。Figure 2 shows the definition of contour sharpness.

图3为利用轮廓线所分配的深度估计相对关系示意图。FIG. 3 is a schematic diagram showing the relative relationship of depth estimation assigned by contour lines.

具体实施方式detailed description

下面结合附图及具体实例，对本发明的实施过程给予详细的说明。The implementation process of the present invention will be described in detail below in conjunction with the accompanying drawings and specific examples.

(1)对于输入图像，采用Canny边缘检测算法进行边缘检测，得到图像中物体的边缘点P＝{p₁,p₂,…p_n}；(1) For the input image, use the Canny edge detection algorithm for edge detection, and obtain the edge point P={p ₁ ,p ₂ ,...p _n } of the object in the image;

(2)根据先验深度梯度模型进行初始轮廓线的定义，定义一组互相平行且间距相等的轮廓线作为初始轮廓线V＝{v₀,v₁,...,v_m}。其中，v(s)＝[x(s),y(s)]，x(s),y(s)为初始轮廓线中的点s的坐标；(2) Define the initial contours according to the prior depth gradient model, and define a group of contours parallel to each other with equal spacing as the initial contours V={v ₀ ,v ₁ ,...,v _m }. Wherein, v(s)=[x(s), y(s)], x(s), y(s) are the coordinates of the point s in the initial contour line;

轮廓线的总能量定义为The total energy of the contour is defined as

其中E_edge为轮廓边缘能量函数，E_sharp轮廓锐度能量函数，E_s为轮廓线特性能量函数，E_d为轮廓线距离能量函数，w₁、w₂、w₃、w₄为各个函数权重控制参数，对于通用图像可以设置为w₁＝0.25、w₂＝0.5、w₃＝0.125、w₄＝0.125。Where E _edge is the contour edge energy function, E _sharp contour sharpness energy function, E _s is the contour line characteristic energy function, E _d is the contour line distance energy function, w ₁ , w ₂ , w ₃ , w ₄ are the weights of each function The control parameters can be set as w ₁ =0.25, w ₂ =0.5, w ₃ =0.125, w ₄ =0.125 for common images.

(3)轮廓边缘能量函数E_edge为图像I(x,y)沿着梯度方向的梯度幅度大小，定义为(3) The contour edge energy function E _edge is the gradient magnitude of the image I(x,y) along the gradient direction, defined as

其中参数a为权值控制参数。The parameter a is the weight control parameter.

(4)在本方法中利用边缘的梯度轮廓锐度代表轮廓的模糊程度，所以定义通过梯度轮廓锐度求解得到的轮廓锐度能量函数E_sharp作为轮廓模糊程度的表征值。如图2所示，以图中边缘像素q₀(x₀,y₀)作为起始点，沿梯度方向向边缘的两边追踪，直到边缘的梯度幅度不再发生变化，这样得到路径p(q₀)。沿一维路径p(q₀)形成的梯度幅度曲线被称为梯度轮廓。应用梯度轮廓变量方差的均方根对轮廓锐度进行定义，表示为：(4) In this method, the gradient contour sharpness of the edge is used to represent the blurring degree of the contour, so the contour sharpness energy function E _sharp obtained by solving the gradient contour sharpness is defined as the representative value of the contour blurring degree. As shown in Figure 2, starting from the edge pixel q ₀ (x ₀ , y ₀ ) in the figure, trace along the gradient direction to both sides of the edge until the gradient magnitude of the edge no longer changes, thus obtaining the path p(q ₀ ). The gradient magnitude profile formed along the one-dimensional path p(q ₀ ) is called a gradient profile. The contour sharpness is defined by applying the root mean square of the variance of the gradient contour variable, expressed as:

$σ σ ((p p (({q q}_{00})))) = = \sqrt{\underset{x x &Element; &Element; p p (({q q}_{00}))}{Σ Σ} \frac{g g ((q q))}{G G (({q q}_{00}))} {d d}_{c c}^{22} ((q q,, {q q}_{00}))}$

$G G (({q q}_{00})) = = \underset{s the s &Element; &Element; p p (({q q}_{00}))}{Σ Σ} g g ((s the s))$

这里d_c(q,q₀)为梯度轮廓中点q和q₀之间的曲线长度，g(q)为q点处的梯度幅度，G(q₀)为梯度轮廓中所有点的梯度幅度和，s为梯度轮廓中任一点。Here d _c (q,q ₀ ) is the length of the curve between points q and q ₀ in the gradient profile, g(q) is the gradient magnitude at point q, and G(q ₀ ) is the gradient magnitude at all points in the gradient profile and, s is any point in the gradient profile.

锐度能量函数E_sharp定义为：The sharpness energy function E _sharp is defined as:

${E E.}_{s the s h h a a r r p p} = = exp exp ((- - \frac{σ σ {((p p (({q q}_{00}))))}^{22}}{b b}))$

其中参数b为权值控制参数。Among them, the parameter b is the weight control parameter.

(5)轮廓线特性能量函数E_s作为平滑项约束来控制轮廓跟踪，确保跟踪生成的曲线是平滑，同时防止求解时陷入局部极值中。(5) The characteristic energy function E _s of the contour line is used as a smooth item constraint to control the contour tracking to ensure that the curve generated by the tracking is smooth, and at the same time prevent it from falling into the local extremum when solving.

定义轮廓线中轮廓点为N＝{n₀,n₁,...,n_n}，其中n₀为轮廓起始点，则轮廓线特性能量函数定义为Define the contour points in the contour line as N={n ₀ ,n ₁ ,...,n _n }, where n ₀ is the starting point of the contour, then the characteristic energy function of the contour line is defined as

${E E.}_{s the s} (({n no}_{i i})) = = \frac{{d d}_{s the s} (({n no}_{i i},, {n no}_{i i - - 11}))}{c c} = = \frac{| | {n no}_{i i y the y} - - {n no}_{i i - - 11 y the y} | |}{c c},, i i = = 00,, 11,, ... ... n no$

其中d_s(n_i,n_i-1)表示点n_i与点n_i-1之间的曲线长度，参数c为权值控制参数。Where d _s (n _i ,n _i-1 ) represents the length of the curve between point n _i and point n _i-1 , and parameter c is a weight control parameter.

(6)轮廓线距离能量函数E_d是轮廓跟踪的弹性约束项，它用于约束轮廓跟踪过程的轮廓线中各轮廓点的距离，使轮廓跟踪曲线不会超出搜索区域，保证轮廓线不相互交叉。(6) Contour line distance energy function E _d is the elastic constraint item of contour tracking, which is used to constrain the distance of each contour point in the contour line of the contour tracking process, so that the contour tracking curve will not exceed the search area and ensure that the contour lines do not interact with each other. cross.

${E E.}_{d d} (({n no}_{i i})) = = \frac{{d d}_{e e} (({n no}_{i i},, {n no}_{00}))}{d d} = = \frac{| | {n no}_{i i y the y} - - {n no}_{00 y the y} | |}{d d},, i i = = 00,, 11,, ... ... n no$

其中d_e(n_i,n₀)表示参数表示点n_i与点n₀之间的垂直距离，d为权值控制参数。Among them, d _e (n _i , n ₀ ) indicates that the parameter indicates the vertical distance between point n _i and point n ₀ , and d is a weight control parameter.

(7)对图像中每一条初始轮廓线从图像的左侧起始点开始，由图像底部向顶部更新每一轮廓点的位置，对每一个轮廓点根据步骤(2～6)中轮廓线总能量计算方法计算轮廓跟踪能量；(7) For each initial contour line in the image, start from the left starting point of the image, update the position of each contour point from the bottom of the image to the top, and for each contour point according to the total energy of the contour line in steps (2-6) The calculation method calculates the contour tracking energy;

(8)求解能量函数的最小值。对与当前轮廓点相邻的像素列中的像素点进行搜索，搜索满足能量函数定义的边缘点P＝{p₁,p₂,…p_n}中具有最小能量值的点，选择具有最小能量的像素点位置作为新的轮廓点；从图像左侧至右侧，重复搜索最小值，得到最终的轮廓搜索结果，即得到目标轮廓线V。(8) Find the minimum value of the energy function. Search the pixel points in the pixel column adjacent to the current contour point, search for the point with the minimum energy value among the edge points P={p ₁ ,p ₂ ,…p _n } satisfying the definition of the energy function, and select the point with the minimum energy The pixel position of is used as the new contour point; from the left to the right side of the image, the minimum value is repeatedly searched to obtain the final contour search result, that is, the target contour line V is obtained.

(9)以由下至上逐步加深的深度梯度假设作为先验假设梯度模型，按照图3对具有不同的轮廓线的区域进行深度值填充，计算得出深度分布。(9) Taking the depth gradient assumption gradually deepened from bottom to top as the prior assumption gradient model, filling depth values for areas with different contour lines according to Figure 3, and calculating the depth distribution.

轮廓线集中轮廓线V＝{v₀,v₁,...,v_m}，对应的分配深度值为。Contour line set V={v ₀ ,v ₁ ,...,v _m }, the corresponding assigned depth value.

(10)利用原图像信息和所得深度图像信息对得到的深度图像进行优化处理：(10) Optimizing the obtained depth image by using the original image information and the obtained depth image information:

其中Depth(x_i)为输入深度图像，Ω(x_i)是以像素x_i为中心的邻域，I(x_i)为像素x_i的亮度I分量，像素x_j为像素x_i在邻域Ω(x_i)的邻域像素，W(x_i)是滤波器参数的归一化因子；||x_i-x_j||为两像素的空间欧式距离；I(x_i)-I(x_j)表示两像素的亮度相似性，像素x_i和x_j的空间坐标分别为(x_ix,x_iy)和(x_jx,x_jy)。空间权重系数和色度权重系数定义为：Among them, Depth( _xi ) is the input depth image, Ω( _xi ) is the neighborhood centered on pixel _xi , I( _xi ) is the brightness I component of pixel _xi , and pixel x _j is the neighborhood of pixel _xi. Neighboring pixels of domain Ω( _xi ), W( _xi ) is the normalization factor of filter parameters; || _xi -x _j || is the spatial Euclidean distance between two pixels; I( _xi )-I (x _j ) represents the brightness similarity of two pixels, and the spatial coordinates of pixels x _i and x _j are (x _ix , x _iy ) and (x _jx , x _jy ) respectively. Spatial weight coefficient and chroma weighting coefficients defined as:

${G G}_{{σ σ}_{r r}} ((I I (({x x}_{i i})) - - I I (({x x}_{j j})))) = = \frac{{| | I I (({x x}_{j j})) - - I I (({x x}_{i i})) | |}^{22}}{22 {σ σ}_{r r}^{22}}$

(11)得到了对输入单目图像进行深度信息估计后的视差图。(11) The disparity map after depth information estimation of the input monocular image is obtained.

Claims

1. The monocular image depth information estimation method based on contour sharpness, is characterized in that, comprises the steps:

(1) For the input image, perform edge detection to obtain the edge point P={p ₁ ,p ₂ ,...p _n } of the object in the image;

(2) Define the initial contour line according to the prior depth gradient model, and define a group of contour lines parallel to each other and with equal spacing as the initial contour line V={v ₀ ,v ₁ ...v _m }; where, v( s)=[x(s), y(s)], x(s), y(s) are the coordinates of the point s in the initial contour line;

The total energy of the contour is defined as

E _total ＝w ₁ E _edge +w ₂ E _sharp +w ₃ E _s +w ₄ E _d

Where E _edge is the contour edge energy function, which represents the gradient magnitude of the contour;

E _sharp Contour sharpness energy function, representing the fuzzy degree of the contour;

E _s is the characteristic energy function of the contour line, which is used to constrain the smoothness of the contour;

E _d is the contour line distance energy function, which is used to control the contour tracking curve and will not exceed the search area;

w ₁ , w ₂ , w ₃ , and w ₄ are weight control parameters for each function;

(3) For each initial contour line in the image, start from the left starting point of the image, update the position of each contour point from the bottom of the image to the top, and calculate according to the definition of the total energy of the contour line in step (2) for each contour point Contour tracking energy;

(4) Solve the minimum value of the energy function; search the pixel points in the pixel column adjacent to the current contour point, and search for the edge points P={p ₁ ,p ₂ ,...p _n } that satisfy the definition of the energy function. For the point with the minimum energy value, select the pixel point position with the minimum energy as the new contour point; from the left side of the image to the right side, repeat the search for the minimum value to obtain the final contour search result, that is, the target contour line V;

(5) Using the depth gradient assumption as a priori assumption gradient model, the depth value is filled in areas with different contour lines, and the depth distribution is calculated;

Contour line set contour line V={v ₀ ,v ₁ ,...,v _m }, for i=0...m the corresponding assigned depth value Depth is:

D D. e e p p t t h h (({v v}_{i i})) = = 255255 \times \times \frac{m m - - i i}{m m}

(6) Use the original image grayscale information and the obtained depth image information to optimize the obtained depth image:

{Depth Depth}_{f f}^{b b f f} (({x x}_{i i})) = = \frac{11}{W W (({x x}_{i i}))} \underset{{x x}_{j j} &Element; &Element; Ω Ω (({x x}_{i i}))}{Σ Σ} exp exp (({G G}_{{σ σ}_{s the s}} ((| | | | {x x}_{i i} - - {x x}_{j j} | | | |)) {G G}_{{σ σ}_{r r}} ((I I (({x x}_{i i})) - - I I (({x x}_{j j})))))) D D. e e p p t t h h (({x x}_{j j}))

W W (({x x}_{i i})) = = \underset{q q &Element; &Element; S S}{Σ Σ} exp exp (({G G}_{{σ σ}_{s the s}} ((| | | | {x x}_{i i} - - {x x}_{j j} | | | |)) {G G}_{{σ σ}_{r r}} ((I I (({x x}_{i i})) - - I I (({x x}_{j j}))))))

Where Depth( _xi ) is the input depth image, Ω( _xi ) is the neighborhood centered on pixel _xi , I( _xi ) is the brightness I component of pixel _xi , and x _j is the neighborhood of pixel _xi Neighborhood pixels of Ω( _xi ), W( _xi ) is the normalization factor of filter parameters; || _xi -x _j || is the spatial Euclidean distance of two pixels; I( _xi )-I( x _j ) represents the brightness similarity of two pixels, the spatial coordinates of pixels x _i and x _j are (x _ix , x _iy ) and (x _jx , x _jy ) respectively; the spatial weight coefficient and chroma weighting coefficients defined as:

{G G}_{{σ σ}_{s the s}} ((| | | | {x x}_{i i} - - {x x}_{j j} | | | |)) = = \frac{{[[\sqrt{{(({x x}_{j j x x} - - {x x}_{i i x x}))}^{22} + + {(({x x}_{j j y the y} - - {x x}_{i i y the y}))}^{22}}]]}^{22}}{22 {σ σ}_{s the s}^{22}}

{G G}_{{σ σ}_{r r}} ((I I (({x x}_{i i})) - - I I (({x x}_{j j})))) = = \frac{| | I I (({x x}_{j j})) - - I I (({x x}_{i i})) {| |}^{22}}{22 {σ σ}_{r r}^{22}};;

where σ _s is the variance of the spatial weights, and σ _r is the variance of the chrominance weights.

(7) The disparity map after depth information estimation of the input monocular image is obtained.

2. the monocular image depth information estimation method based on contour sharpness according to claim 1, is characterized in that, described contour edge energy function E _edge is the gradient magnitude of image I (x, y) along gradient direction size, defined as:

{E E.}_{e e d d g g e e} = = exp exp ((- - \frac{| | g g ((I I ((x x,, y the y)))) {| |}^{22}}{a a}))

in,

g (I (x, the y)) = \sqrt{{(\partial_{x} I (x, the y))}^{2} + {(\partial_{the y} I (x, the y))}^{2}}

It is to find the gradient magnitude of I(x,y);

The parameter a is the weight control parameter.

3. the monocular image depth information estimation method based on contour sharpness according to claim 1, is characterized in that, described contour sharpness energy function E _sharp obtains with gradient contour sharpness solution, is defined as:

{E E.}_{s the s h h a a r r p p} = = exp exp ((- - \frac{σ σ {((p p (({q q}_{00}))))}^{22}}{b b}));;

Wherein, the gradient profile sharpness σ(p(q ₀ )) is the root mean square of the variance of the gradient profile variable. Here the gradient profile is the edge pixel q ₀ (x ₀ , y ₀ ) in the image as the starting point, and traces along the gradient direction to the boundary of the edge until the gradient amplitude of the edge no longer changes, and the obtained one-dimensional path p(q ₀ ) The resulting gradient magnitude curve; its gradient profile sharpness is defined as

σ σ ((p p (({q q}_{00})))) = = \sqrt{\underset{x x &Element; &Element; p p (({q q}_{00}))}{Σ Σ} \frac{g g ((q q))}{G G (({q q}_{00}))} {d d}_{c c}^{22} ((q q,, {q q}_{00}))};; G G (({q q}_{00})) = = \underset{s the s &Element; &Element; p p (({q q}_{00}))}{Σ Σ} g g ((s the s))

where d _c (q,q ₀ ) is the curve length between points q and q ₀ in the gradient profile, g(q) is the gradient magnitude at point q, and G(q ₀ ) is the gradient of all points in the gradient profile Amplitude sum, s is any point in the gradient profile, and parameter b is the weight control parameter.