[go: up one dir, main page]

CN112379529B - Transparent object surface reflected light separation method based on polarization characteristics - Google Patents

Transparent object surface reflected light separation method based on polarization characteristics Download PDF

Info

Publication number
CN112379529B
CN112379529B CN202011303630.8A CN202011303630A CN112379529B CN 112379529 B CN112379529 B CN 112379529B CN 202011303630 A CN202011303630 A CN 202011303630A CN 112379529 B CN112379529 B CN 112379529B
Authority
CN
China
Prior art keywords
image
light
reflected light
polarization
correlation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011303630.8A
Other languages
Chinese (zh)
Other versions
CN112379529A (en
Inventor
张焱
张景华
石志广
李飚
杨卫平
张宇
索玉昌
刘荻
凌峰
师晓冉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202011303630.8A priority Critical patent/CN112379529B/en
Publication of CN112379529A publication Critical patent/CN112379529A/en
Application granted granted Critical
Publication of CN112379529B publication Critical patent/CN112379529B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/28—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for polarising
    • G02B27/283—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for polarising used for beam splitting or combining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Algebra (AREA)
  • Pure & Applied Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

The invention belongs to the field of image processing, and relates to a method for separating reflected light on the surface of a transparent object based on polarization characteristics. The method comprises the following steps: (S1) collecting a polarization image and performing a preprocessing; (S2) carrying out polarization calculation on the preprocessed image to obtain light intensity images in the parallel direction and the vertical direction; (S3) separating the reflected images using the parallel-direction and vertical-direction light intensity images; (S4) obtaining a normalized cross-correlation of the separated reflected light image and the transmitted light image; (S5) extracting and summing the reflection dominant pixel and the transmission dominant pixel with the cross-correlation; (S6) solving the minimum value of the normalized cross correlation by using a gradient descent method to obtain the corresponding polarization degree of the transmitted light and the polarization degree of the reflected light, thereby realizing the separation of the reflected light. The method can realize the separation of the reflected light and the transmitted light on the surface of the transparent object, and has good separation effect. By separating the reflected light, the reflection information can be effectively extracted, and clutter suppression and enhancement are performed on the transmission image.

Description

一种基于偏振特征的透明物体表面反射光分离方法A method for separating reflected light from the surface of transparent objects based on polarization characteristics

技术领域technical field

本发明属于光学图像处理领域，涉及可见光偏振图像信息的处理与分析，实现了一种透明物体表面反射光的分离方法。The invention belongs to the field of optical image processing, relates to the processing and analysis of visible light polarization image information, and realizes a method for separating reflected light from the surface of a transparent object.

背景技术Background technique

利用探测器对玻璃等透明材质背后的景物进行成像时，获取的图像由两部分组成：透射光部分和反射光部分。由于透明物体表面反射和透射过程是同时发生的，所以反射光和透射光往往混合在一起，相互影响。随着现代化城市建设及家庭室内装修过程中，玻璃、塑料等具有透光和反光效应的材料被大量采用，透明物体表面透射光和反射光的分离及反射光抑制问题引起国内外学者的广泛关注和重点研究。一方面，反射光会在物体表面呈现虚像，掩盖玻璃背后景物的颜色细节等属性，当入射光光源较强时，还会在物体表面形成高亮区域，影响图像的品质，给计算机处理和人眼识别带来较大问题。另一方面，通过对反射光进行提取和分析，能够获取反射光源强度、位置和周围环境等信息，具有一定的应用价值。因此，透明物体表面反射光和透射光分离是计算机视觉领域中十分具有挑战性的任务，对图像分割、目标识别、立体匹配等应用具有重要的意义。When using a detector to image the scene behind a transparent material such as glass, the acquired image consists of two parts: the transmitted light part and the reflected light part. Since the reflection and transmission process on the surface of transparent objects occur at the same time, the reflected light and the transmitted light are often mixed together and affect each other. In the process of modern urban construction and home interior decoration, materials with light transmission and reflection effects such as glass and plastic are widely used. and key research. On the one hand, the reflected light will present a virtual image on the surface of the object, covering up the color details of the scene behind the glass. Eye recognition poses a bigger problem. On the other hand, by extracting and analyzing the reflected light, information such as the intensity, location and surrounding environment of the reflected light source can be obtained, which has certain application value. Therefore, the separation of reflected light and transmitted light from the surface of transparent objects is a very challenging task in the field of computer vision, which is of great significance for applications such as image segmentation, object recognition, and stereo matching.

现有的反射光分离方法大致可以分为两类：一是基于图像特征的，如相机虚焦引起的反射景物边缘模糊特征、玻璃前后表面同时反射产生的重影特征等。由于反射光图像和透射光图像相互干扰、彼此重叠，单纯的基于图像特征的反射光分离效果并不理想；二是基于物理属性的，如光波反射过程中产生的偏振特征等。由于玻璃表面反射光和透射光具有明显的偏振效应，且反射光和透射光的偏振特征具有明显的差异，因此利用反射光的偏振特征能够实现反射光的分离，且相对于单纯的基于图像特征的反射光分离方法，具有更好的分离效果和稳定性。The existing reflected light separation methods can be roughly divided into two categories: one is based on image features, such as blurred features of reflected scene edges caused by camera defocus, and ghosting features caused by simultaneous reflection of front and rear surfaces of glass. Because the reflected light image and the transmitted light image interfere with each other and overlap each other, the reflected light separation effect based solely on image features is not ideal; the second is based on physical properties, such as the polarization characteristics generated in the process of light wave reflection. Since the reflected light and transmitted light on the glass surface have obvious polarization effects, and the polarization characteristics of reflected light and transmitted light are significantly different, the reflected light can be separated by using the polarization characteristics of reflected light, and compared with pure image-based features The reflected light separation method has better separation effect and stability.

发明内容SUMMARY OF THE INVENTION

针对透明物体表面反射光分离问题，通过对反射区域偏振特征进行分析，结合透射光图像和反射光图像之间的相关性，利用梯度下降法求解透射光图像和反射光图像归一化互相关最小值，得到相关性最小时对应的反射偏振度和透射偏振度。然后根据透射光和反射光在透明物体表面垂直方向上和平行方向上的分布关系，通过偏振正交差分算法，利用反射光偏振度和透射光偏振度实现反射光分离。具体技术方案如下：Aiming at the problem of separation of reflected light on the surface of transparent objects, by analyzing the polarization characteristics of the reflection area, combined with the correlation between the transmitted light image and the reflected light image, the gradient descent method is used to solve the minimum normalized cross-correlation between the transmitted light image and the reflected light image. value to obtain the corresponding reflection polarization degree and transmission polarization degree when the correlation is the smallest. Then, according to the distribution relationship between the transmitted light and the reflected light in the vertical and parallel directions of the surface of the transparent object, the polarization orthogonal difference algorithm is used to realize the separation of the reflected light by using the polarization degree of the reflected light and the polarization degree of the transmitted light. The specific technical solutions are as follows:

一种基于偏振特征的透明物体表面反射光分离方法，包括以下步骤：A method for separating reflected light from the surface of a transparent object based on polarization characteristics, comprising the following steps:

(S1)采集可见光偏振图像，并进行预处理；(S1) collecting visible light polarization images and preprocessing;

(S2)对步骤(S1)中预处理后的图像进行偏振态解算，计算得到平行方向光强图像和垂直方向光强图像；(S2) performing polarization state calculation on the preprocessed image in step (S1), and calculating the light intensity image in the parallel direction and the light intensity image in the vertical direction;

(S3)选取初始的反射光偏振度和透射光偏振度并结合平行方向光强图像和垂直方向光强图像对反射光和透射光混合图像进行分离；(S3) Selecting the initial degree of polarization of reflected light and transmitted light and combining the light intensity image in the parallel direction and the light intensity image in the vertical direction to separate the mixed image of the reflected light and the transmitted light;

(S4)求取初始分离后反射光图像和透射光图像的归一化互相关；(S4) obtaining the normalized cross-correlation of the reflected light image and the transmitted light image after initial separation;

(S5)反射主导像素点和透射主导像素点提取与互相关求和；(S5) Extraction and cross-correlation summation of reflection-dominated pixels and transmission-dominated pixels;

(S6)利用梯度下降法求解归一化互相关最小值，得到最小值对应的透射光偏振度和反射光偏振度，从而实现反射光的分离。(S6) Use the gradient descent method to solve the minimum value of the normalized cross-correlation, and obtain the degree of polarization of the transmitted light and the degree of polarization of the reflected light corresponding to the minimum value, so as to realize the separation of the reflected light.

优选地，所述步骤(S1)中的预处理包括：包括图像校正、滤波、配准和裁剪。Preferably, the preprocessing in the step (S1) includes image correction, filtering, registration and cropping.

优选地，所述步骤(S2)中平行方向光强图像和垂直方向光强图像解算过程为：Preferably, in the step (S2), the calculation process of the light intensity image in the parallel direction and the light intensity image in the vertical direction is:

根据偏振光的表示形式，确定偏振光在不同起偏角下的光强计算公式为：According to the representation of polarized light, the calculation formula for determining the light intensity of polarized light at different polarization angles is:

Figure GDA0003531052670000031
Figure GDA0003531052670000031

其中i、j表示图像中像素点坐标位置，φm为起偏角，φ⊥为反射面垂直方向对应的起偏角。Among them, i and j represent the coordinate position of the pixel point in the image, φ m is the starting angle, and φ ⊥ is the starting angle corresponding to the vertical direction of the reflective surface.

令φ0＝0°，将起偏角φm分别为φ0＝φ0，φ45＝φ0+45°，φ90＝φ0+90°时获取的三通道偏振度图像I0，I45，I90分别代入上式，求得：Let φ 0 = 0°, the polarization angle φ m is respectively φ 0 =φ 0 , φ 45 =φ 0 +45°, and φ 90 =φ 0 +90°The three-channel polarization degree images I 0 , I obtained when 45 and I 90 are respectively substituted into the above formula to obtain:

Figure GDA0003531052670000032
Figure GDA0003531052670000032

分别确定垂直方向光强I⊥和平行方向光强I||如下：The vertical light intensity I ⊥ and the parallel light intensity I || are determined as follows:

Figure GDA0003531052670000033
Figure GDA0003531052670000033

Figure GDA0003531052670000034
Figure GDA0003531052670000034

优选地，所述步骤(S3)的具体过程为：Preferably, the specific process of the step (S3) is:

根据偏振正交分解原理，确定探测器接收到的光强在垂直方向上和平行方向上的分量为：According to the polarization orthogonal decomposition principle, the components of the light intensity received by the detector in the vertical and parallel directions are determined as:

Figure GDA0003531052670000035
Figure GDA0003531052670000035

其中R⊥和R||分别是垂直方向和平行方向反射率，PR是反射光源强度，

Figure GDA0003531052670000036
和
Figure GDA0003531052670000037
分别为反射光垂直方向和平行方向光强分量，ε⊥和ε||分别是垂直方向和平行方向发射率，
Figure GDA0003531052670000038
为透射光垂直方向光强分量，
Figure GDA0003531052670000039
为透射光平行方向光强分量，PT是透射光源强度。where R ⊥ and R || are the vertical and parallel reflectivity, respectively, P R is the reflected light source intensity,
Figure GDA0003531052670000036
and
Figure GDA0003531052670000037
are the vertical and parallel light intensity components of the reflected light, respectively, ε ⊥ and ε || are the vertical and parallel emissivity, respectively,
Figure GDA0003531052670000038
is the light intensity component in the vertical direction of the transmitted light,
Figure GDA0003531052670000039
is the light intensity component in the parallel direction of the transmitted light, and P T is the intensity of the transmitted light source.

玻璃表面反射光和透射光都属于偏振光，令反射光偏振度为γ，透射光偏振度为χ：Both the reflected light and the transmitted light on the glass surface belong to polarized light, let the degree of polarization of the reflected light be γ, and the degree of polarization of the transmitted light to be χ:

Figure GDA0003531052670000041
Figure GDA0003531052670000041

Figure GDA0003531052670000042
Figure GDA0003531052670000042

则：but:

Figure GDA0003531052670000043
Figure GDA0003531052670000043

Figure GDA0003531052670000044
Figure GDA0003531052670000044

将公式(8)、(9)代入公式(5)求解得到透射光在垂直方向上和平行方向上光强分量为：Substitute formulas (8) and (9) into formula (5) to solve the obtained light intensity components of the transmitted light in the vertical and parallel directions are:

Figure GDA0003531052670000045
Figure GDA0003531052670000045

同时确定反射光在垂直方向上和平行方向上光强分量为：At the same time, determine the light intensity components of the reflected light in the vertical and parallel directions as:

Figure GDA0003531052670000046
Figure GDA0003531052670000046

总光强等于垂直方向和平行方向光强之和，因此确定玻璃表面反射光成分和透射光成分如下：The total light intensity is equal to the sum of the light intensity in the vertical and parallel directions, so the components of reflected light and transmitted light on the glass surface are determined as follows:

Figure GDA0003531052670000047
Figure GDA0003531052670000047

优选地，所述步骤(S5)的具体过程为：Preferably, the specific process of the step (S5) is:

将图像中任意一像素点r(i,j)处的归一化自相关

Figure GDA0003531052670000051
表示为：Normalized autocorrelation at any pixel r(i,j) in the image
Figure GDA0003531052670000051
Expressed as:

Figure GDA0003531052670000052
Figure GDA0003531052670000052

其中U、V表示窗口大小，u、v表示像素点在窗口内位置坐标，

Figure GDA0003531052670000053
和
Figure GDA0003531052670000054
分别表示反射光图像和透射光图像窗口内像素灰度平均值。Among them, U and V represent the size of the window, and u and v represent the position coordinates of the pixel in the window.
Figure GDA0003531052670000053
and
Figure GDA0003531052670000054
Represents the average value of pixel grayscale in the reflected light image and the transmitted light image window, respectively.

(S51)反射主导像素点提取(S51) Reflection dominates pixel point extraction

提取反射光为主导像素点的具体步骤如下：首先设置γ＝0.01，χ＝0.2对混合图像进行过分离，得到过分离后的透射光图像Iover-t；然后选取γ＝0.99，χ＝0.2对混合图像进行欠分离，得到欠分离后的透射光图像Iunder-t；紧接着求取Iover-t和Iunder-t两者之间的相关性，得到相关性图像Rt；最后通过将相关性为负的像素点设置为1，相关性为正的像素点设置为0，从而实现反射主导像素点的提取；The concrete steps of extracting reflected light as the dominant pixel point are as follows: first set γ=0.01, χ=0.2 to over-separate the mixed image, and obtain the transmitted light image I over-t after the over-separation; then choose γ=0.99, χ=0.2 The mixed image is under-separated to obtain the under-separated transmitted light image I under-t ; then the correlation between I over-t and I under-t is obtained to obtain the correlation image R t ; Set the pixels with negative correlation to 1, and set the pixels with positive correlation to 0, so as to realize the extraction of reflection-dominant pixels;

Figure GDA0003531052670000055
Figure GDA0003531052670000055

其中Rt表示过分离后的透射光图像Iover-t和欠分离后的透射光图像Iunder-t之间的归一化互相关图像，Mt表示对Rt进行二值化之后的结果。where R t represents the normalized cross-correlation image between the over-separated transmitted light image I over-t and the under-separated transmitted light image I under-t , and M t represents the result of binarizing R t .

(S52)透射主导像素点提取：(S52) Transmission dominant pixel extraction:

通过选取以透射光为主导的像素点求取实际透射偏振度值，首先设置γ＝0.5，χ＝0.01对混合图像进行过分离，得到过分离后的反射光图像Iover-r，然后选取γ＝0.99，χ＝0.5对混合图像进行欠分离，得到欠分离后的透射光图像Iunder-r，求两者之间的互相关得到Rr，并对其进行二值化：The actual transmission polarization value is obtained by selecting the pixel points dominated by the transmitted light. First, set γ=0.5, χ=0.01 to over-separate the mixed image, and obtain the over-separated reflected light image I over-r , and then select γ =0.99, χ=0.5, under-separate the mixed image to obtain the under-separated transmitted light image I under-r , find the cross-correlation between the two to obtain R r , and binarize it:

Figure GDA0003531052670000056
Figure GDA0003531052670000056

其中Rr表示过分离后的反射光图像Iover-r和欠分离后的反射光图像Iunder-r之间的归一化互相关图像，Mr表示对Rr进行二值化之后的结果。where R r represents the normalized cross-correlation image between the over-separated reflected light image I over-r and the under-separated reflected light image I under-r , and Mr r represents the result of binarizing R r .

(S53)反射主导像素点和透射主导像素点互相关求和：(S53) Cross-correlation summation of the reflection dominant pixel point and the transmission dominant pixel point:

以反射为主导的像素点归一化互相关之和fR(γ,χ)为：The sum of the normalized cross-correlation of pixels dominated by reflection f R (γ,χ) is:

Figure GDA0003531052670000061
Figure GDA0003531052670000061

其中row、col分别表示图像的行数和列数。where row and col represent the number of rows and columns of the image, respectively.

以透射为主导的像素点归一化互相关之和fT(γ,χ)为：The sum of normalized cross-correlation of pixels dominated by transmission f T (γ,χ) is:

Figure GDA0003531052670000062
Figure GDA0003531052670000062

优选地，所述步骤(S6)的具体过程为：Preferably, the specific process of the step (S6) is:

将反射光图像和透射光图像归一化互相关fR(γ,χ)和fT(γ,χ)作为反射偏振度γ和透射偏振度χ的函数，然后分别对fR(γ,χ)和fT(γ,χ)求偏导，并让其沿着梯度方向的下降，通过多次迭代后得到fR(γ,χ)和fT(γ,χ)的最小值；The normalized cross-correlation f R (γ,χ) and f T (γ,χ) of the reflected light image and the transmitted light image as a function of the reflection polarization degree γ and the transmission polarization degree χ, and then f R (γ,χ ) and f T (γ, χ) to obtain partial derivatives, and let it descend along the gradient direction, and obtain the minimum value of f R (γ, χ) and f T (γ, χ) after multiple iterations;

Figure GDA0003531052670000063
Figure GDA0003531052670000063

其中η是学习率，n表示当前迭代次数，n+1表示下一次迭代次数。当达到收敛条件时，迭代结束，此时得到归一互相关最小值fR(γm,χm)和fT(γm,χm)，以及他们对应的透射偏振度γm和反射偏振度χm，从而实现反射光和透射光的最优分离。where η is the learning rate, n is the current iteration number, and n+1 is the next iteration number. When the convergence condition is reached, the iteration ends, and the normalized cross-correlation minimum values f R (γ m ,χ m ) and f T (γ m ,χ m ), as well as their corresponding transmission polarization degrees γ m and reflection polarizations, are obtained at this time. degree χ m to achieve optimal separation of reflected and transmitted light.

优选地，所述反射主导像素点提取过程中，反射偏振度γ先后设置为0.01和0.99。Preferably, in the process of extracting the reflection-dominated pixel points, the reflection polarization degree γ is set to 0.01 and 0.99 successively.

优选地，所述透射主导像素点提取过程中，透射偏振度χ先后设置为0.01和0.99。Preferably, in the process of extracting the transmission-dominated pixel points, the transmission polarization degree χ is set to 0.01 and 0.99 successively.

为了充分理解本发明，下面对技术方案中涉及的相关原理进行说明。In order to fully understand the present invention, the relevant principles involved in the technical solutions are described below.

利用探测器对玻璃等透明材质背后的景物进行成像时，获取的图像由两部分组成：透射光部分和反射光部分。由于透明物体表面反射和透射过程是同时发生的，所以反射光和透射光往往混合在一起，相互影响。由于透明物体表面反射光和透射光都是偏振光，且两者之间存在明显差异，其中反射光偏振方向以垂直反射面为主，透射光偏振方向以平行反射面为主，因此利用反射光和透射光的偏振特征能够实现反射光和透射光的分离。When using a detector to image the scene behind a transparent material such as glass, the acquired image consists of two parts: the transmitted light part and the reflected light part. Since the reflection and transmission process on the surface of transparent objects occur at the same time, the reflected light and the transmitted light are often mixed together and affect each other. Since both the reflected light and the transmitted light on the surface of the transparent object are polarized light, and there are obvious differences between the two, the polarization direction of the reflected light is dominated by the vertical reflective surface, and the polarization direction of the transmitted light is dominated by the parallel reflective surface. and the polarization characteristics of the transmitted light enable the separation of reflected and transmitted light.

为了实现反射光的分离，本发明利用透明物体表面反射光和透射光在垂直方向和平行方向上的分布关系，建立了反射光强和透射光强与反射偏振度和透射偏振度之间的方程关系，从而直接通过偏振度实现反射光的分离。由于反射偏振度和透射偏振度无法直接利用探测器测量得到，本发明通过梯度下降法求解反射光图像和透射光图像归一化互相关最小值，从而获取其对应的反射光偏振度和透射光偏振度，最终实现透射光和反射光的最优分离。In order to realize the separation of the reflected light, the present invention establishes the equation relationship between the reflected light intensity and the transmitted light intensity and the reflected polarization degree and the transmitted polarization degree by using the distribution relationship between the reflected light and the transmitted light on the surface of the transparent object in the vertical direction and the parallel direction. , so that the reflected light can be separated directly by the degree of polarization. Since the reflection polarization degree and the transmission polarization degree cannot be directly measured by a detector, the present invention solves the minimum value of the normalized cross-correlation between the reflected light image and the transmitted light image by the gradient descent method, so as to obtain the corresponding reflected light polarization degree and transmitted light. degree of polarization, ultimately achieving optimal separation of transmitted and reflected light.

混合图像中反射主导点和透射主导点提取方法：在实际分解过程中，直接求图像中所有像素点的互相关之和，会存在两个问题：一是计算量大，计算速度慢；二是易受噪声干扰。为了抑制噪声干扰，同时提升计算速度，本发明提出一种提取主导像素点的算法。所谓主导像素点，是指混合图像中以透射光信息或者反射光信息为主导的像素点。由于偏振度大小和透射光与反射光的比值有关，且两者之间相差越大，偏振度越大，相差越小，偏振度越小。对于图像中的反射主导像素点和透射主导像素点，其透射光强度和反射光强度相差都比较大，所以该类像素点偏振度大、抗干扰性强。Extraction method of reflection dominant point and transmission dominant point in mixed image: In the actual decomposition process, there are two problems in directly calculating the sum of the cross-correlation of all pixel points in the image: one is the large amount of calculation and the calculation speed is slow; the other is susceptible to noise interference. In order to suppress noise interference and improve calculation speed at the same time, the present invention proposes an algorithm for extracting dominant pixel points. The so-called dominant pixels refer to pixels dominated by transmitted light information or reflected light information in the mixed image. Since the degree of polarization is related to the ratio of transmitted light to reflected light, and the greater the difference between the two, the greater the degree of polarization, and the smaller the difference, the smaller the degree of polarization. For the reflection-dominant pixels and the transmission-dominant pixels in the image, the difference between the transmitted light intensity and the reflected light intensity is relatively large, so this type of pixel has a large degree of polarization and strong anti-interference.

对于以反射光为主导的像素点，在反射偏振度从小到大变化过程中，反射光分离过程会从过分解变为欠分解，此时透射光图像和反射光图像之间的相关性会从负相关变为正相关，即相关性会发生“正负”变化。而对于以透射光为主导的像素点，当反射偏振度从小到大变化过程中，其相关性变化并不明显，不存在“正负”变化现象。因此通过反射光欠分离和过分离过程中透射光图像的相关性正负变化关系，能够提取出以反射光为主导的像素点；然后利用以反射光为主导的像素点，能够准确、高效的求解出实际反射偏振度大小。For pixels dominated by reflected light, when the degree of reflection polarization changes from small to large, the reflected light separation process will change from over-decomposition to under-decomposition. At this time, the correlation between the transmitted light image and the reflected light image will change from A negative correlation becomes a positive correlation, i.e. the correlation changes "positively and negatively". For the pixels dominated by transmitted light, when the reflected polarization degree changes from small to large, the correlation changes are not obvious, and there is no "positive and negative" change phenomenon. Therefore, through the positive and negative correlations of the transmitted light images during the under-separation and over-separation of the reflected light, the pixels dominated by the reflected light can be extracted; and then the pixels dominated by the reflected light can be used accurately and efficiently. Solve for the actual reflection polarization degree.

同理，对于以透射光为主导的像素点，在透射偏振度从小到大变化过程中，透射光分离过程会从过分解变为欠分解，此时透射光图像和反射光图像之间的相关性会从负相关变为正相关，即相关性会发生“正负”变化。而对于以反射光为主导的像素点，当透射偏振度从小到大变化过程中，其相关性变化并不明显，不存在“正负”变化现象。因此通过透射光图像欠分离和过分离过程中反射光图像的相关性正负变化关系，能够提取出以透射光为主导的像素点；然后利用以透射光为主导的像素点，能够准确、高效的求解出实际透射偏振度大小。In the same way, for pixels dominated by transmitted light, when the degree of transmission polarization changes from small to large, the separation process of transmitted light will change from over-decomposition to under-decomposition. At this time, the correlation between the transmitted light image and the reflected light image is Sex will change from a negative correlation to a positive correlation, i.e. the correlation will change "positively and negatively". For the pixels dominated by reflected light, when the degree of transmission polarization changes from small to large, the correlation changes are not obvious, and there is no "positive and negative" change phenomenon. Therefore, through the positive and negative correlation of the reflected light image during the under-separation and over-separation of the transmitted light image, the pixels dominated by the transmitted light can be extracted; and then the pixels dominated by the transmitted light can be used accurately and efficiently. The solution of the actual transmission polarization degree size.

采用本发明获得的有益效果：The beneficial effects obtained by adopting the present invention:

本发明方法能够有效分离出透明物体表面的反射光和透射光，基于反射光和透射光偏振特征，利用反射光和透射光相互作用的特点，结合反射光和透射光在垂直方向和平行方向上的光强分布关系，采用偏振正交差分分解的方法，通过反射偏振度和透射偏振度求解得到反射光图像和透射光图像。然后基于最优分离时反射光图像和透射光图像相关性最小的原理，利用梯度下降的方法，求解反射光图像和透射光图像归一化互相关最小值对应的反射偏振度和透射偏振度，最终实现透明物体表面反射光和透射的最佳分离。实验结果表明，本文方法能实现不同场景下反射光的分离。通过反射光的分离，一方面能够从反射光中获取物体表面的反射光源位置、反射光源强度等信息，为目标侦察、三维重构等数据基础，另一方面通过对反射光进行分离，实现了透明物体表面反射光的抑制，从而增强透射成分，提升玻璃背后景物的成像质量，更有利于实现复杂反射光环境下目标检测、分割等图像处理。The method of the invention can effectively separate the reflected light and the transmitted light on the surface of the transparent object. Based on the polarization characteristics of the reflected light and the transmitted light, the characteristics of the interaction between the reflected light and the transmitted light are used to combine the reflected light and the transmitted light in the vertical direction and the parallel direction. The light intensity distribution relationship is obtained by using the polarization orthogonal differential decomposition method, and the reflected light image and the transmitted light image are obtained by solving the reflection polarization degree and the transmission polarization degree. Then, based on the principle that the correlation between the reflected light image and the transmitted light image is the smallest in the optimal separation, the gradient descent method is used to solve the reflection polarization degree and the transmission polarization degree corresponding to the minimum normalized cross-correlation of the reflected light image and the transmitted light image, Ultimately, an optimal separation of reflected light and transmitted light from the surface of a transparent object is achieved. The experimental results show that the method in this paper can realize the separation of reflected light in different scenes. Through the separation of reflected light, on the one hand, information such as the position of the reflected light source and the intensity of the reflected light source on the surface of the object can be obtained from the reflected light, which is the data basis for target reconnaissance and 3D reconstruction. The suppression of the reflected light on the surface of the transparent object, thereby enhancing the transmission component, improving the imaging quality of the scene behind the glass, is more conducive to the realization of image processing such as target detection and segmentation in complex reflected light environments.

附图说明Description of drawings

图1为本发明流程图；Fig. 1 is the flow chart of the present invention;

图2为反射光分离过程流程图；Fig. 2 is the flow chart of reflected light separation process;

图3为反射和透射过程示意图；Figure 3 is a schematic diagram of the reflection and transmission process;

图4为反射光过分解和欠分解过程；Fig. 4 is the process of over-decomposition and under-decomposition of reflected light;

图5为透射光过分解和欠分解过程；Fig. 5 is the process of over-decomposition and under-decomposition of transmitted light;

图6为反射数据采集示意图；6 is a schematic diagram of reflection data acquisition;

图7为不同场景下反射光分离结果对比图。Figure 7 is a comparison diagram of reflected light separation results in different scenarios.

具体实施方式Detailed ways

下面结合附图和实施例对本发明作进一步说明。The present invention will be further described below with reference to the accompanying drawings and embodiments.

如图1所示，为本发明的总流程图；为了测试本发明方法的分离效果和可靠性，利用偏振探测器、玻璃等器材获取了三组真实世界中的反射光图像。数据获取过程如图4所示。首先利用玻璃和偏振探测器获取真实世界中的反射光和透射光的混合图像(如图7(a1)、(b1)、(c1)所示)。然后取走玻璃，获取背景图像，即真实透射光图像(如图7(a2)、(b2)、(c2)所示)。图7(a3)、(b3)、(c3)是本发明方法分离后的透射光图像，图5(a4)、(b4)、(c4)是本发明方法分离后的反射光图像。通过将本发明方法分离后的透射光图像(如图7(a3)、(b3)、(c3)所示)与真实透射光背景图像(如7(a2)、(b2)、(c2))进行对比，可以看出，本发明分离后的透射光图像和真实背景图像接近相同，验证本发明方法的有效性，本发明方法具体步骤如下：As shown in FIG. 1, it is the general flow chart of the present invention; in order to test the separation effect and reliability of the method of the present invention, three groups of reflected light images in the real world were obtained by using polarization detectors, glass and other equipment. The data acquisition process is shown in Figure 4. First, the mixed images of reflected and transmitted light in the real world are obtained using glass and polarized detectors (as shown in Fig. 7(a1), (b1), (c1)). Then the glass is removed to obtain a background image, that is, a real transmitted light image (as shown in Figure 7(a2), (b2), (c2)). Figures 7(a3), (b3) and (c3) are the transmitted light images separated by the method of the present invention, and Figures 5(a4), (b4) and (c4) are the reflected light images separated by the method of the present invention. By separating the transmitted light image (as shown in Figure 7(a3), (b3), (c3)) and the real transmitted light background image (such as 7(a2), (b2), (c2)) by the method of the present invention By contrast, it can be seen that the separated transmitted light image and the real background image of the present invention are close to the same, and the validity of the method of the present invention is verified. The specific steps of the method of the present invention are as follows:

(S1)采集可见光偏振图像，并进行预处理；(S1) collecting visible light polarization images and preprocessing;

利用可见光偏振成像探测系统对目标场景采集四个通道(不同的检偏方向，如I0、I45、I90、I135)的可见光偏振图像。图像预处理包括图像校正、滤波、配准、裁剪等，校正用于抵消探测器响应的非均匀性、并补偿光学元器件的非理想性。滤波用于降低图像噪声并去除坏点。配准用于消除同一场景各偏振图像存在的微位移偏差，最后裁剪掉图像配准后的边框。A visible light polarization imaging detection system is used to collect visible light polarization images of four channels (different analysis directions, such as I 0 , I 45 , I 90 , and I 135 ) of the target scene. Image preprocessing includes image correction, filtering, registration, cropping, etc. The correction is used to counteract the non-uniformity of the detector response and compensate for the non-ideality of the optical components. Filtering is used to reduce image noise and remove dead pixels. The registration is used to eliminate the micro-displacement deviation of each polarized image of the same scene, and finally the frame after the registration of the image is cropped.

(S2)对步骤(S1)中预处理后的图像进行偏振态解算，计算得到平行方向光强图像和垂直方向光强图像；(S2) performing polarization state calculation on the preprocessed image in step (S1), and calculating the light intensity image in the parallel direction and the light intensity image in the vertical direction;

根据偏振光的表示形式，确定偏振光在不同起偏角下的光强计算公式为：According to the representation of polarized light, the calculation formula for determining the light intensity of polarized light at different polarization angles is:

Figure GDA0003531052670000101
Figure GDA0003531052670000101

其中i、j表示图像中像素点坐标位置，φm为起偏角，φ⊥为反射面垂直方向对应的起偏角。Among them, i and j represent the coordinate position of the pixel point in the image, φ m is the starting angle, and φ ⊥ is the starting angle corresponding to the vertical direction of the reflective surface.

令φ0＝0°，将起偏角φm分别为φ0＝φ0，φ45＝φ0+45°，φ90＝φ0+90°时获取的三通道偏振度图像I0，I45，I90分别代入上式，求得：Let φ 0 = 0°, the polarization angle φ m is respectively φ 0 =φ 0 , φ 45 =φ 0 +45°, and φ 90 =φ 0 +90°The three-channel polarization degree images I 0 , I obtained when 45 and I 90 are respectively substituted into the above formula to obtain:

Figure GDA0003531052670000102
Figure GDA0003531052670000102

分别确定垂直方向光强I⊥和平行方向光强I||如下：The vertical light intensity I ⊥ and the parallel light intensity I || are determined as follows:

Figure GDA0003531052670000103
Figure GDA0003531052670000103

Figure GDA0003531052670000111
Figure GDA0003531052670000111

(S3)选取初始的反射光偏振度和透射光偏振度并结合平行方向光强图像和垂直方向光强图像对反射光和透射光混合图像进行分离；(S3) Selecting the initial degree of polarization of reflected light and transmitted light and combining the light intensity image in the parallel direction and the light intensity image in the vertical direction to separate the mixed image of the reflected light and the transmitted light;

根据偏振正交分解原理，确定探测器接收到的光强在垂直方向上和平行方向上的分量为：According to the polarization orthogonal decomposition principle, the components of the light intensity received by the detector in the vertical and parallel directions are determined as:

Figure GDA0003531052670000112
Figure GDA0003531052670000112

其中R⊥和R||分别是垂直方向和平行方向反射率，PR是反射光源强度，

Figure GDA0003531052670000113
和
Figure GDA0003531052670000114
分别为反射光垂直方向和平行方向光强分量，ε⊥和ε||分别是垂直方向和平行方向发射率，
Figure GDA0003531052670000115
为透射光垂直方向光强分量，
Figure GDA0003531052670000116
为透射光平行方向光强分量，PT是透射光源强度。where R ⊥ and R || are the vertical and parallel reflectivity, respectively, P R is the reflected light source intensity,
Figure GDA0003531052670000113
and
Figure GDA0003531052670000114
are the vertical and parallel light intensity components of the reflected light, respectively, ε ⊥ and ε || are the vertical and parallel emissivity, respectively,
Figure GDA0003531052670000115
is the light intensity component in the vertical direction of the transmitted light,
Figure GDA0003531052670000116
is the light intensity component in the parallel direction of the transmitted light, and P T is the intensity of the transmitted light source.

玻璃表面反射光和透射光都属于偏振光，令反射光偏振度为γ，透射光偏振度为χ：Both the reflected light and the transmitted light on the glass surface belong to polarized light, let the degree of polarization of the reflected light be γ, and the degree of polarization of the transmitted light to be χ:

Figure GDA0003531052670000117
Figure GDA0003531052670000117

Figure GDA0003531052670000118
Figure GDA0003531052670000118

则：but:

Figure GDA0003531052670000119
Figure GDA0003531052670000119

Figure GDA00035310526700001110
Figure GDA00035310526700001110

将公式(8)、(9)代入公式(5)求解得到透射光在垂直方向上和平行方向上光强分量为：Substitute formulas (8) and (9) into formula (5) to solve the obtained light intensity components of the transmitted light in the vertical and parallel directions are:

Figure GDA0003531052670000121
Figure GDA0003531052670000121

同时确定反射光在垂直方向上和平行方向上光强分量为：At the same time, determine the light intensity components of the reflected light in the vertical and parallel directions as:

Figure GDA0003531052670000122
Figure GDA0003531052670000122

总光强等于垂直方向和平行方向光强之和，因此确定玻璃表面反射光成分和透射光成分如下：The total light intensity is equal to the sum of the light intensity in the vertical and parallel directions, so the components of reflected light and transmitted light on the glass surface are determined as follows:

Figure GDA0003531052670000123
Figure GDA0003531052670000123

(S4)求取初始分离后反射光图像和透射光图像的归一化互相关；(S4) obtaining the normalized cross-correlation of the reflected light image and the transmitted light image after initial separation;

在已知反射光偏振度、透射光偏振度以及垂直方向、平行方向光强的前提下，利用公式(12)可以求得玻璃表面每个像素点处反射光强和透射光强，从而实现反射光的分离。但是玻璃表面反射光和透射光是同时存在、相互叠加的，无法直接利用探测器分别获取反射光偏振度和透射光偏振度。On the premise that the polarization degree of reflected light, transmitted light polarization degree, and the light intensity in the vertical and parallel directions are known, the reflected light intensity and the transmitted light intensity at each pixel point on the glass surface can be obtained by using formula (12), so as to realize the reflection separation of light. However, the reflected light and the transmitted light on the glass surface exist at the same time and are superimposed on each other, and it is impossible to directly obtain the reflected light polarization degree and the transmitted light polarization degree by using the detector.

由于玻璃表面反射光和透射光包含不同的图像内容，在理想的分离情况下，得到的反射光图像和透射光图像具有最小的相关性。因此，尽管玻璃表面反射光偏振度和透射光偏振度无法直接测量，可以通过计算反射光图像和透射光图像的相关性，得到两者相关性最小时对应的反射偏振度和透射偏振度，从而实现反射光的分离。两幅图像的相关性可以用归一化互相关NCC(normalized cross correlation)表示。图像中任意一像素点r(i,j)处的归一化互相关

Figure GDA0003531052670000124
为：Since the reflected and transmitted light from the glass surface contain different image content, under ideal separation, the resulting reflected and transmitted light images have minimal correlation. Therefore, although the degree of polarization of the reflected light and the degree of transmitted light cannot be directly measured on the glass surface, the correlation of the reflected light image and the transmitted light image can be calculated to obtain the corresponding degree of reflected polarization and transmitted polarization when the correlation between the two is the smallest. The separation of reflected light is achieved. The correlation of two images can be represented by normalized cross correlation (NCC). Normalized cross-correlation at any pixel r(i,j) in the image
Figure GDA0003531052670000124
for:

Figure GDA0003531052670000125
Figure GDA0003531052670000125

其中U、V表示窗口大小，u、v表示像素点在窗口内位置坐标，

Figure GDA0003531052670000131
和
Figure GDA0003531052670000132
分别表示反射光图像和透射光图像窗口内像素灰度平均值。Among them, U and V represent the size of the window, and u and v represent the position coordinates of the pixel in the window.
Figure GDA0003531052670000131
and
Figure GDA0003531052670000132
Represents the average value of pixel grayscale in the reflected light image and the transmitted light image window, respectively.

定义f(γ,χ)为分离后反射光图像和透射光图像中所有像素点归一化互相关之和：Define f(γ,χ) as the sum of the normalized cross-correlations of all pixels in the reflected light image and the transmitted light image after separation:

Figure GDA0003531052670000133
Figure GDA0003531052670000133

(S5)反射主导像素点和透射主导像素点提取与互相关求和；(S5) Extraction and cross-correlation summation of reflection-dominated pixels and transmission-dominated pixels;

在实际分解过程中，直接求图像中所有像素点的互相关之和，会存在两个问题：一是计算量大，计算速度慢；二是易受噪声干扰。为了抑制噪声干扰，同时提升计算速度，本发明提出一种提取主导像素点的方法。所谓主导像素点，是指混合图像中以透射光信息或者反射光信息为主导的像素点。In the actual decomposition process, there are two problems in directly calculating the sum of the cross-correlations of all pixels in the image: one is that the calculation amount is large and the calculation speed is slow; the other is that it is susceptible to noise interference. In order to suppress noise interference and improve calculation speed at the same time, the present invention proposes a method for extracting dominant pixel points. The so-called dominant pixels refer to pixels dominated by transmitted light information or reflected light information in the mixed image.

(S51)反射主导像素点提取(S51) Reflection dominates pixel point extraction

提取反射光为主导像素点的具体步骤如下：首先设置γ＝0.01，χ＝0.2对混合图像进行过分离，得到过分离后的透射光图像Iover-t；然后选取γ＝0.99，χ＝0.2对混合图像进行欠分离，得到欠分离后的透射光图像Iunder-t(注：反射偏振度γ较小时，分离过程中会出现反射光过分离现象，如图4(a)所示，因此本发明将反射偏振度设置为0.01进行反射光过分离；反射偏振度γ较大时，分离过程中会出现反射光欠分离现象，如图4(c)所示，因此本发明将反射偏振度设置为0.99进行反射光欠分离；对于透射偏振度χ，大多数透明物体透射偏振度变化范围为0-0.4，因此我们选取中值0.2作为透射偏振度χ的值进行欠分离和过分离求解；通过公式12可以看出，透射光偏振度χ只会影响分离后透射光图像的强弱，并不会改变分离后透射光图像的纹理和细节，无论透射偏振度选取0.1还是0.4，其对欠分离和过分离得到的透视光图像之间的相关性影响并不大，因此设置透射光偏振度χ为0.2是合理的)；紧接着求取Iover-t和Iunder-t两者之间的相关性，得到相关性图像Rt；最后通过将相关性为负的像素点设置为1，相关性为正的像素点设置为0，从而实现反射主导像素点的提取；The concrete steps of extracting reflected light as the dominant pixel point are as follows: first set γ=0.01, χ=0.2 to over-separate the mixed image, and obtain the transmitted light image I over-t after the over-separation; then choose γ=0.99, χ=0.2 Under-separate the mixed image to obtain the under-separated transmitted light image I under-t (Note: when the reflection polarization degree γ is small, the reflected light will be over-separated during the separation process, as shown in Figure 4(a), so In the present invention, the reflection polarization degree is set to 0.01 to carry out over-separation of reflected light; when the reflection polarization degree γ is large, the phenomenon of under-separation of reflected light will occur during the separation process, as shown in FIG. 4( c ). Set it to 0.99 for under-separation of reflected light; for the transmission polarization degree χ, the transmission polarization degree of most transparent objects varies from 0 to 0.4, so we choose the median value of 0.2 as the value of the transmission polarization degree χ for under-separation and over-separation solutions; It can be seen from Equation 12 that the transmitted light polarization degree χ only affects the intensity of the transmitted light image after separation, and does not change the texture and details of the transmitted light image after separation. The correlation between the perspective light images obtained by separation and over-separation has little effect, so it is reasonable to set the transmitted light polarization χ to 0.2); then find the difference between I over-t and I under-t The correlation of , obtains the correlation image R t ; finally, by setting the negative correlation pixel to 1, and the correlation positive pixel to 0, so as to realize the extraction of reflection-dominated pixels;

Figure GDA0003531052670000141
Figure GDA0003531052670000141

其中Rt表示过分离后的透射光图像Iover-t和欠分离后的透射光图像Iunder-t之间的归一化互相关，Mt表示对Rt进行二值化之后的结果。where R t represents the normalized cross-correlation between the over-separated transmitted light image I over-t and the under-separated transmitted light image I under-t , and M t represents the result of binarizing R t .

(S52)透射主导像素点提取：(S52) Transmission dominant pixel extraction:

通过选取以透射光为主导的像素点求取实际透射偏振度值。首先设置γ＝0.5，χ＝0.01对混合图像进行过分离，得到过分离后的反射光图像Iover-r，然后选取γ＝0.99，χ＝0.5对混合图像进行欠分离，得到欠分离后的透射光图像Iunder-r(注：透射偏振度χ较小时，分离过程中会出现透射光过分离现象，如图5(a)所示；透射偏振度χ较大时，分离过程中会出现透射光欠分离现象，如图5(c)所示。因此本发明将透射偏振度分别设置为0.01和0.99进行透射光过分离和欠分离处理；对于反射偏振度γ，反射偏振度变化范围为0-1，因此我们选取中值0.5作为反射偏振度γ的值)，求两者之间的互相关得到Rr，并对其进行二值化：The actual transmission polarization value is obtained by selecting the pixel points dominated by the transmitted light. First, set γ=0.5, χ=0.01 to over-separate the mixed image, and obtain the over-separated reflected light image I over-r , then select γ=0.99, χ=0.5 to under-separate the mixed image, and obtain the under-separated image I over-r . The transmitted light image I under-r (Note: when the transmission polarization degree χ is small, the phenomenon of transmitted light over-separation will appear in the separation process, as shown in Figure 5(a); when the transmission polarization degree χ is large, there will be a phenomenon in the separation process. The phenomenon of under-separation of transmitted light is shown in Fig. 5(c). Therefore, the present invention sets the transmission polarization degrees to 0.01 and 0.99 respectively to carry out over-separation and under-separation of transmitted light; 0-1, so we choose the median value of 0.5 as the value of the reflection polarization degree γ), find the cross-correlation between the two to get R r , and binarize it:

Figure GDA0003531052670000142
Figure GDA0003531052670000142

其中Rr表示过分离后的反射光图像Iover-r和欠分离后的反射光图像Iunde-r之间的归一化互相关图像，Mr表示对Rr进行二值化之后的结果。where R r represents the normalized cross-correlation image between the over-separated reflected light image I over-r and the under-separated reflected light image I unde-r , and Mr r represents the result of binarizing R r .

(S53)反射主导像素点和透射主导像素点互相关求和：(S53) Cross-correlation summation of the reflection dominant pixel point and the transmission dominant pixel point:

Figure GDA0003531052670000151
Figure GDA0003531052670000151

其中row、col分别表示图像的行数和列数。where row and col represent the number of rows and columns of the image, respectively.

以透射为主导的像素点归一化互相关之和fT(γ,χ)为：The sum of normalized cross-correlation of pixels dominated by transmission f T (γ,χ) is:

Figure GDA0003531052670000152
Figure GDA0003531052670000152

(S6)利用梯度下降法求解归一化互相关最小值，得到最小值对应的透射光偏振度和反射光偏振度，从而实现反射光的分离。(S6) Use the gradient descent method to solve the minimum value of the normalized cross-correlation, and obtain the degree of polarization of the transmitted light and the degree of polarization of the reflected light corresponding to the minimum value, so as to realize the separation of the reflected light.

将反射光图像和透射光图像归一化互相关fR(γ,χ)和fT(γ,χ)作为反射偏振度γ和透射偏振度χ的函数。然后分别对fR(γ,χ)和fT(γ,χ)求偏导，并让其沿着梯度方向的下降，通过多次迭代后可得到fR(γ,χ)和fT(γ,χ)的最小值；Normalized cross-correlations f R (γ, χ) and f T (γ, χ) of the reflected and transmitted light images as a function of the reflection polarization degree γ and the transmission polarization degree χ. Then take the partial derivatives of f R (γ,χ) and f T (γ, χ) respectively, and let them descend along the gradient direction, after several iterations, f R (γ, χ) and f T ( γ, χ) minimum value;

Figure GDA0003531052670000153
Figure GDA0003531052670000153

其中η是学习率，n表示当前迭代次数，n+1表示下一次迭代次数。当达到收敛条件时，迭代结束，此时得到归一互相关最小值fR(γm,χm)和fT(γm,χm)，以及他们对应的透射偏振度γm和反射偏振度χm，从而实现反射光和透射光的最优分离。where η is the learning rate, n is the current iteration number, and n+1 is the next iteration number. When the convergence condition is reached, the iteration ends, and the normalized cross-correlation minimum values f R (γ m ,χ m ) and f T (γ m ,χ m ), as well as their corresponding transmission polarization degrees γ m and reflection polarizations, are obtained at this time. degree χ m to achieve optimal separation of reflected and transmitted light.

如图7中(a3)、(b3)、(c3)所示为本发明方法分离后透射光图像结果，通过与真实背景图像进行对比，可以看出两个之间仅存在微小差别，即反射光得到了分离和抑制，验证了本发明方法的有效性。(a3), (b3) and (c3) in Figure 7 are the results of the transmitted light image after separation by the method of the present invention. By comparing with the real background image, it can be seen that there is only a slight difference between the two, that is, the reflection The light is separated and suppressed, verifying the effectiveness of the method of the present invention.

尽管已经示出和描述了本发明的实施例，对于本领域的普通技术人员而言，可以理解在不脱离本发明的原理和精神的情况下可以对这些实施例进行多种变化、修改、替换和变型，本发明的范围由所附权利要求及其等同物限定。Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, and substitutions can be made in these embodiments without departing from the principle and spirit of the invention and modifications, the scope of the present invention is defined by the appended claims and their equivalents.

Claims (9)

1. A method for separating reflected light on the surface of a transparent object based on polarization characteristics is characterized by comprising the following steps:
(S1) acquiring a visible light polarization image and performing preprocessing;
(S2) carrying out polarization state calculation on the image preprocessed in the step (S1) to obtain a light intensity image in the parallel direction and a light intensity image in the vertical direction;
(S3) selecting the initial polarization degree of the reflected light and the initial polarization degree of the transmitted light, and combining the light intensity image in the parallel direction and the light intensity image in the vertical direction to separate the mixed image of the reflected light and the transmitted light;
(S4) obtaining a normalized cross-correlation of the initial separated reflected light image and the transmitted light image;
(S5) extracting and summing the reflection dominant pixel and the transmission dominant pixel with the cross-correlation;
(S6) solving the minimum value of the normalized cross correlation by using a gradient descent method to obtain the polarization degree of the transmitted light and the polarization degree of the reflected light corresponding to the minimum value, thereby realizing the separation of the reflected light.
2. The method for separating the reflected light from the surface of the transparent object based on the polarization characteristics as claimed in claim 1, wherein the preprocessing in the step (S1) comprises: including image correction, filtering, registration, and cropping.
3. The method for separating the reflected light from the surface of the transparent object based on the polarization characteristics as claimed in claim 1, wherein the solving process of the parallel direction light intensity image and the perpendicular direction light intensity image in the step (S2) is as follows:
according to the representation form of the polarized light, the light intensity calculation formula of the polarized light under different polarizing angles is as follows:
Figure FDA0003531052660000011
wherein i and j represent the coordinate position of a pixel point in the image, phimIs the angle of incidence, phi⊥Is the corresponding deflection angle of the vertical direction of the reflecting surface.
Let phi0Angle of departure phi is 0 DEGmAre respectively phi0＝φ0，φ45＝φ0+45°，φ90＝φ0Three-channel polarization degree image I acquired at +90 DEG0，I45，I90Substituting the following formulas into the formulas respectively to obtain:
Figure FDA0003531052660000021
respectively determining the light intensity I in the vertical direction⊥And light intensity in parallel direction I||The following were used:
Figure FDA0003531052660000022
Figure FDA0003531052660000023
4. the method for separating the reflected light from the surface of the transparent object based on the polarization characteristics as claimed in claim 1, wherein the step (S3) of solving the reflected light image and the transmitted light image by combining the polarization degree of the reflected light and the polarization degree of the transmitted light comprises the following steps:
according to the polarization orthogonal decomposition principle, the components of the light intensity received by the detector in the vertical direction and the parallel direction are determined as follows:
Figure FDA0003531052660000024
wherein R is⊥And R||Are respectively in the vertical direction and the horizontal directionLine direction reflectivity, PRIs the intensity of the light source of the reflected light,
Figure FDA0003531052660000025
and
Figure FDA0003531052660000026
light intensity components, epsilon, in the vertical and parallel directions of the reflected light, respectively⊥And ε||Vertical and parallel direction emissivity respectively,
Figure FDA0003531052660000027
a light intensity component in a vertical direction for the transmitted light,
Figure FDA0003531052660000028
for light intensity components parallel to the direction of transmitted light, PTIs the transmitted light source intensity.
The reflected light and the transmitted light on the surface of the glass belong to polarized light, the polarization degree of the reflected light is gamma, and the polarization degree of the transmitted light is chi:
Figure FDA0003531052660000029
Figure FDA0003531052660000031
then:
Figure FDA0003531052660000032
Figure FDA0003531052660000033
substituting the formulas (8) and (9) into the formula (5) to solve the problem that the light intensity components of the transmitted light in the vertical direction and the parallel direction are:
Figure FDA0003531052660000034
and simultaneously determining the light intensity components of the reflected light in the vertical direction and the parallel direction as follows:
Figure FDA0003531052660000035
the total light intensity is equal to the sum of the light intensities in the vertical direction and the parallel direction, and therefore the light components reflected from the glass surface and the light components transmitted therefrom are determined as follows:
Figure FDA0003531052660000036
5. the separation method of the reflected light on the surface of the transparent object based on the polarization characteristics as claimed in claim 1, wherein the specific process of extracting the reflection-dominant pixel points and the transmission-dominant pixel points and the cross correlation summation in the step (S5) is as follows:
(S51) reflection dominant pixel point extraction
The specific steps of extracting the reflected light as the dominant pixel point are as follows: first, the mixed image is set to be 0.01 and 0.2, and the mixed image is subjected to separation to obtain a transmitted light image I after separationover-t(ii) a Then selecting gamma 0.99 and chi 0.2 to under-separate the mixed image and obtain the under-separated transmitted light image Iunder-t(ii) a Then find Iover-tAnd Iunder-tThe correlation between the two is obtained to obtain a correlation image Rt(ii) a Finally, the pixel point with negative correlation is set to be 1, and the pixel point with positive correlation is set to be 0, so that the reflection leading pixel point is extracted;
Figure FDA0003531052660000041
wherein R istRepresenting the transmitted light image I after separationover-tAnd an under-separated transmitted light image Iunder-tNormalized cross-correlation image between, MtRepresents a pair of RtThe result after binarization is performed.
(S52) transmission dominant pixel point extraction:
and calculating the actual transmission polarization degree value by selecting the pixel points taking the transmission light as the leading factor. First, the mixed image is set to be 0.5 and 0.01, and the mixed image is separated to obtain a reflected light image I after separationover-rThen, the mixed image is under-separated by selecting gamma 0.99 and chi 0.5, and the under-separated transmitted light image I is obtainedunder-rObtaining R by cross-correlation between the tworAnd binarizing the image:
Figure FDA0003531052660000042
wherein R isrRepresenting the separated reflected light image Iover-rAnd an under-separated reflected light image Iunder-rNormalized cross-correlation image between, MrRepresents a pair of RrThe result after binarization is performed.
(S53) cross-correlation summing of reflection-dominated pixels and transmission-dominated pixels:
reflection dominated normalized cross correlation sum f of pixelsR(γ, χ) is:
Figure FDA0003531052660000051
where row, col represent the number of rows and columns of the image, respectively.
Transmission dominated normalized cross correlation sum f of pixelsT(γ, χ) is:
Figure FDA0003531052660000052
6. the method for separating the reflected light from the surface of the transparent object based on the polarization characteristics as claimed in claim 5, wherein the step (S6) of solving the minimum value of the normalized cross-correlation by using a gradient descent method to obtain the polarization degree of the transmitted light and the polarization degree of the reflected light corresponding to the minimum value comprises the following specific steps:
normalized cross-correlation f of reflected and transmitted light imagesR(gamma, chi) and fT(gamma, chi) as a function of the degree of reflection polarization gamma and of the degree of transmission polarization chi, and then respectively for fR(gamma, chi) and fT(gamma, chi) calculating partial derivative, making it fall along gradient direction, obtaining f after several iterationsR(gamma, chi) and fT(γ, χ);
Figure FDA0003531052660000053
where η is the learning rate, n represents the current iteration number, and n +1 represents the next iteration number. When the convergence condition is reached, the iteration is ended, and the normalized cross-correlation minimum value f is obtainedR(γm,χm) And fT(γm,χm) And their corresponding transmission polarization degree gammamAnd degree of reflection polarization χm。
7. The method as claimed in claim 6, wherein the separation result is optimized when the cross correlation between the reflected light and the transmitted light is minimized.
8. The method for separating the reflected light on the surface of the transparent object based on the polarization characteristics as claimed in claim 5, wherein the reflection polarization degree γ is set to 0.01 and 0.99 in sequence in the extraction process of the reflection dominant pixel points.
9. The method for separating the reflected light on the surface of the transparent object based on the polarization characteristics as claimed in claim 5, wherein the transmission polarization degree χ is set to 0.01 and 0.99 in sequence in the process of extracting the transmission dominant pixel points.
CN202011303630.8A 2020-11-19 2020-11-19 Transparent object surface reflected light separation method based on polarization characteristics Active CN112379529B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011303630.8A CN112379529B (en) 2020-11-19 2020-11-19 Transparent object surface reflected light separation method based on polarization characteristics

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011303630.8A CN112379529B (en) 2020-11-19 2020-11-19 Transparent object surface reflected light separation method based on polarization characteristics

Publications (2)

Publication Number Publication Date
CN112379529A CN112379529A (en) 2021-02-19
CN112379529B true CN112379529B (en) 2022-04-19

Family

ID=74584620

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011303630.8A Active CN112379529B (en) 2020-11-19 2020-11-19 Transparent object surface reflected light separation method based on polarization characteristics

Country Status (1)

Country Link
CN (1) CN112379529B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101558282A (en) * 2007-02-13 2009-10-14 松下电器产业株式会社 Image processing system, method, device and image format
CN107966412A (en) * 2017-12-05 2018-04-27 天津大学 Based on cross-polarization image stretch and the associated image defogging method of degree of polarization
CN109923855A (en) * 2016-11-15 2019-06-21 索尼公司 Image processing apparatus, image processing method and program
CN111351572A (en) * 2019-11-21 2020-06-30 四川极智朗润科技有限公司 Distributed reflection elimination device and method based on polarization characteristics
CN111539411A (en) * 2020-04-20 2020-08-14 河海大学常州校区 A method of using polarization information to identify objects of different materials

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101558282A (en) * 2007-02-13 2009-10-14 松下电器产业株式会社 Image processing system, method, device and image format
CN109923855A (en) * 2016-11-15 2019-06-21 索尼公司 Image processing apparatus, image processing method and program
CN107966412A (en) * 2017-12-05 2018-04-27 天津大学 Based on cross-polarization image stretch and the associated image defogging method of degree of polarization
CN111351572A (en) * 2019-11-21 2020-06-30 四川极智朗润科技有限公司 Distributed reflection elimination device and method based on polarization characteristics
CN111539411A (en) * 2020-04-20 2020-08-14 河海大学常州校区 A method of using polarization information to identify objects of different materials

Also Published As

Publication number Publication date
CN112379529A (en) 2021-02-19

Similar Documents

Publication Publication Date Title
CN108492274B (en) An image enhancement method for long-wave infrared polarization feature extraction and fusion
Lyu et al. Reflection separation using a pair of unpolarized and polarized images
Wang et al. End-to-end view synthesis for light field imaging with pseudo 4DCNN
Kong et al. A physically-based approach to reflection separation: from physical modeling to constrained optimization
US11398053B2 (en) Multispectral camera external parameter self-calibration algorithm based on edge features
WO2018171008A1 (en) Specular highlight area restoration method based on light field image
CN112837233B (en) Polarization image defogging method for acquiring transmissivity based on differential polarization
US20110043603A1 (en) System And Method For Dehazing
CN118424232B (en) An underwater structure ranging system based on computer vision
Kong et al. High-quality reflection separation using polarized images
CN109886883A (en) Real-time polarized fog-penetrating imaging image enhancement processing method
CN110956661A (en) Method for calculating dynamic pose of visible light and infrared camera based on bidirectional homography matrix
CN107610164A (en) A kind of No. four Image registration methods of high score based on multiple features mixing
WO2018133119A1 (en) Method and system for three-dimensional reconstruction of complete indoor scene based on depth camera
CN110580686B (en) Polarization image restoration device and method based on binocular vision in scattering environment
Kong et al. A physically-based approach to reflection separation
CN110517348A (en) 3D Point Cloud Reconstruction Method of Target Object Based on Image Foreground Segmentation
Hsu et al. Object detection using structure-preserving wavelet pyramid reflection removal network
CN119399344A (en) A satellite 3D reconstruction method in complex lighting environment
Lu et al. Underwater scene enhancement using weighted guided median filter
US20220148137A1 (en) System and method for reflection removal using dual-pixel sensor
Shan et al. Global optimal reconstruction of underwater polarization imaging using low-pass frequency-filtering constraint
Shi et al. Stereo waterdrop removal with row-wise dilated attention
CN112379529B (en) Transparent object surface reflected light separation method based on polarization characteristics
CN118195982B (en) Polarized image edge enhancement calculation method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant