CN114913075A - Polarization direction full-resolution image reconstruction method - Google Patents

Abstract

The invention discloses a polarization direction full-resolution image reconstruction method, and belongs to the technical field of polarization imaging devices and polarization image processing and display. The image reconstruction method of the polarization imaging pixel array decomposes an array image into images in different polarization directions through downsampling, and full-resolution images in all polarization directions are obtained by adopting weight estimation and interpolation calculation to fill up missing pixel values in the polarization directions. Compared with the prior art, the super-pixel unit array with 4 pixels is more compact; and the resolution of the image in the polarization direction is improved to three times of the original resolution by utilizing pixel correlation interpolation calculation.

Description

A full-resolution image reconstruction method in polarization direction

technical field

The invention belongs to the technical field of polarization imaging devices and polarization image processing and display, and in particular relates to a polarization direction full-resolution image reconstruction method.

Background technique

Among polarized imaging sensors, split focal plane polarimeter (DoFP) is a common type. Due to the development of nanofabrication technology and the advantages of time synchronization, DoFP sensors have received extensive attention and applications in acquiring real-time polarization images. As shown in the figure below, the key component of the DoFP sensor is the focal plane array (FPA), in which every four polarized filters collect polarized light with directions of 0°, 45°, 90° and 135°, respectively, forming a 2×2 superpixel unit. These periodic superpixels are arranged on the focal plane, so that the DoFP sensor can simultaneously obtain polarization images in four directions, as shown in Figure 1.

However, since each polarization filter can only collect the intensity information of the corresponding polarization direction, the output image of the DoFP sensor is 4 times that of a single polarization filter, resulting in the loss of spatial resolution. Therefore, when the output image of the DoFP sensor is obtained Afterwards, the missing pixels will be filled in by methods such as interpolation to obtain intensity images of different polarization states at the same time. The purpose of this processing is to obtain full-resolution images representing the four polarization channels.

According to the calculation principle in polarization detection, only the polarization intensity parameters of the target light source in three directions can actually be obtained. We process the focal plane image with three polarization directions proposed in this patent (as shown in Figure 2), and use the existing interpolation method to process it. Since the correlation between pixels cannot be reasonably considered, it is difficult to obtain different polarization directions. full-resolution image.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a focal plane polarization image pixel reconstruction method to solve the problem of focal plane polarization image reconstruction obtained by an existing focal plane polarization imaging pixel array with three polarization directions. The ordinary interpolation method may cause the estimated value inaccurate problem. Aiming at this problem, a scheme that can solve the interpolation error is proposed to improve the resolution of the intensity image in each polarization direction.

In order to achieve the above object, the specific technical scheme of the polarization direction full-resolution image reconstruction method of the present invention is as follows:

A polarization direction full-resolution image reconstruction method specifically includes the following steps, and the following steps are performed in sequence:

Step S1, performing down-sampling processing on the focal plane image, and decomposing it into images with three polarization directions a, b, and c;

Step S2, for the image of each polarization direction, the pixel with the same direction as the specified direction (marked with c) is the pixel with the light intensity value in this direction, which is defined as a valued pixel; and the pixel with a different polarization direction from the specified direction (Fig. A blank pixel unit) is a pixel without a light intensity value in this direction, which is defined as a vacant pixel (as shown in Figure ₄ , the six neighborhoods around pixel c1 are all vacant pixels);

Step S3: Complement the light intensity values of all the vacant pixels in the images of the three polarization directions by performing weight estimation and interpolation calculation on the neighboring pixel intensity of each vacant pixel, thereby obtaining a full-resolution image of each polarization direction.

Further, it specifically includes three steps of downsampling, weight estimation and interpolation calculation:

Taking the calculation of the full-resolution image of the polarization direction c as an example, for a vacant pixel X, the method of estimating its light intensity value I _X is:

On the decomposed image whose polarization direction is c, for each vacancy pixel X, there are three valued images among the 6 pixels in its adjacent six neighborhoods, which are c ₁ pixel, c ₂ pixel and c ₃ pixel respectively;

Weight estimation is performed according to the light intensity values of c ₁ , c ₂ , and c ₃ , that is, the weight values of three valued pixels (c ₁ , c ₂ , and c ₃ ) in the neighborhood of vacant pixels are calculated, as shown in formula 1:

表示不同偏振方向图像中空缺像素邻域有值像素的强度值，w表示各个偏振方向a，b，c；∑I_k为空缺像素六邻域内有值像素光强值之和； _Sk is the influence ratio of each valued pixel in the six-neighborhood pixels of the vacant pixel in estimating the light intensity value of the vacant pixel, k=c ₁ , c ₂ , c ₃ ;

Represents the intensity value of the valued pixels in the neighborhood of the vacant pixel in images with different polarization directions, w represents each polarization direction a, b, c; ∑I _k is the sum of the light intensity values of the valued pixels in the six neighborhoods of the vacant pixel;

After the weight estimation is completed, the interpolation calculation is performed on the vacant pixel, that is, the light intensity value at the vacant pixel X can be expressed as I _X :

A polarization direction full-resolution image reconstruction method of the present invention has the following advantages: a polarization image pixel reconstruction method suitable for the polarization imaging pixel array is invented, the method considers the relationship between adjacent pixels, and can realize the present invention The proposed reconstruction of the focal plane polarization image to generate full-resolution intensity images with different polarization directions can solve the problem of low resolution caused by ordinary interpolation methods, and has certain practical significance.

Description of drawings

FIG. 1 is a focal plane polarization image mentioned in the background art.

FIG. 2 is a schematic diagram of a focal plane image structure with three polarization directions.

FIG. 3 is an overview of a pixel reconstruction method of a defocusing plane polarization image (taking the calculation of a full-resolution image in the c-direction as an example).

FIG. 4 is a schematic diagram of the neighborhood pixel points at the X pixel in the c-direction polarization image.

Description of the symbols in the figure: 201, six neighborhoods of a pixel; 301, downsampling; 302, weight estimation; 303, interpolation calculation; 3011-3013, three images with different polarization directions a, b, c respectively; 3031 , a full-resolution image; 401-403, three pixels with pixel values in the six neighborhoods of the vacant pixel, respectively; 404, a vacant pixel.

Detailed ways

In order to better understand the purpose, structure and function of the present invention, a method for reconstructing a polarization direction full-resolution image of the present invention will be described in further detail below with reference to the accompanying drawings.

As shown in FIG. 2-FIG. 4, the present invention proposes a focal plane polarization image pixel reconstruction method, which solves the problem of reconstruction of the focal plane polarization image obtained by the existing focal plane polarization imaging pixel array with three polarization directions. Ordinary interpolation The method may cause inaccurate estimates to improve the image resolution of the intensity in each polarization direction.

The focal plane image with three polarization directions is shown in Figure 2. For any pixel in its structure, with the pixel as the center and surrounded by six pixels, a hexagonal structure is formed, and this structure is defined as the pixel The Six Neighborhoods 201.

An image pixel reconstruction method using a focal plane polarization imaging pixel array structure includes the following steps, which are performed in sequence. FIG. 3 is an outline of a focal plane polarization image pixel reconstruction method.

The focal plane polarization image pixel reconstruction method is very important for estimating the missing polarization pixel values in each polarization direction.

A focal plane polarization image pixel reconstruction method includes downsampling 301 , weight estimation 302 and interpolation calculation 303 .

In this embodiment, the focal plane polarization image is firstly down-sampled 301 to obtain three images 3011 , 3012 , and 3013 with different polarization directions a, b, and c. On an image of a certain polarization direction, the pixels in the same direction (marked with a, b, c) have known light intensity values, and the pixels (blank) with different polarization directions from this direction have no light intensity values. Pixels without light intensity values in this direction are called vacant pixels. Then, if the light intensity value of a vacant pixel in an image in a polarization direction can be estimated, a full-resolution image in that polarization direction can be obtained.

Taking the calculation of the full-resolution image 3013 to fill the polarization direction c as an example, for a vacant pixel X404, the method for estimating its light intensity value I _X is:

On the decomposed image 3013 whose polarization direction is c, for each vacancy pixel X 404, three of the 6 pixels in its adjacent six neighborhoods have pixel values of c ₁ pixel 401, c ₂ pixel 402 and c ₃ pixel respectively 403. Weight estimation 302 is performed according to the light intensity values of c ₁ 401 , c ₂ 402 and c ₃ 403 . That is, the weight values of the three valued pixels (c ₁ 401 , c ₂ 402 , and c ₃ 403 ) in the neighborhood of the vacant pixel are calculated, as in formula 1.

表示不同偏振方向图像中空缺像素邻域有值像素的强度值，w表示各个偏振方向a，b，c；∑I_k为空缺像素六邻域内有值像素光强值之和。 _Sk is the influence ratio of each valued pixel in the six-neighborhood pixels of the vacant pixel in estimating the light intensity value of the vacant pixel, k=c ₁ , c ₂ , c ₃ ;

Indicates the intensity value of the valued pixels in the neighborhood of the vacant pixel in images with different polarization directions, w represents each polarization direction a, b, c; ΣI _k is the sum of the light intensity values of the valued pixels in the six neighborhoods of the vacant pixel.

After the weight estimation 302 is completed, an interpolation calculation 303 is performed on the vacant pixels. That is, the light intensity value at the vacant pixel X404 can be expressed as I _X :

The calculation is performed successively for each vacant pixel, thereby obtaining a full-resolution image 3031 of the polarization direction.

It can be understood that the present invention is described by some embodiments, and those skilled in the art know that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, in the teachings of this invention, these features and embodiments may be modified to adapt a particular situation and material without departing from the spirit and scope of the invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of the present application fall within the protection scope of the present invention.

Claims (2)

1. a polarization direction full resolution image reconstruction method, is characterized in that, specifically comprises the following steps, and the following steps are carried out in sequence: Step S1, the focal plane image is down-sampled (301), and decomposed into images with three polarization directions a, b, and c (3011, 3012, 3013); Step S2, for the image of each polarization direction (3011, 3012, 3013), the pixel with the same direction as the specified direction is the pixel with the light intensity value in this direction, which is defined as a valued pixel; and the pixel with a different polarization direction from the specified direction. is the pixel without the light intensity value in this direction, which is defined as the vacant pixel; Step S3, by performing weight estimation (302) and interpolation calculation (303) on the neighborhood pixel intensity of each vacancy pixel, to complement the light intensities of all vacant pixels in the images (3011, 3012, 3013) in the three polarization directions value to obtain full-resolution images for each polarization direction. 2. The full-resolution image reconstruction method of the polarization direction according to claim 1, wherein the calculation method of the vacant pixel light intensity value in the step S3 comprises the following steps: For a vacant pixel X (404), the method of estimating its light intensity value I _X is: On the decomposed image (3013) whose polarization direction is c, for each vacancy pixel X (404), there are three valued images among the six pixels in its adjacent six neighborhoods, which are c ₁ pixel (401), c ₂ pixel (402) and c ₃ pixels (403); Calculate the weight values of the three valued pixels c ₁ pixel (401), c ₂ pixel (402), and c ₃ pixel (403) in the neighborhood of the vacant pixel by formula (1):

_Sk is the influence ratio of each valued pixel in the six-neighborhood pixels of the vacant pixel in estimating the light intensity value of the vacant pixel, k=c ₁ , c ₂ , c ₃ ;

Represents the intensity value of the valued pixels in the neighborhood of the vacant pixel in images with different polarization directions, w represents each polarization direction a, b, c; ∑I _k is the sum of the light intensity values of the valued pixels in the six neighborhoods of the vacant pixel; After the weight estimation (302) is completed, an interpolation calculation (303) is performed on the vacant pixel, that is, the light intensity value at the vacant pixel X (404) can be expressed as I _X :

Each vacant pixel is successively calculated to obtain a full-resolution image of the current polarization direction (3031).

Images