CN105554484B - A kind of demosaicing methods for being applied to high photosensitive micro- inclined array image-forming - Google Patents

Abstract

The invention discloses a demosaic method applied to high-sensitivity micro-polarization array imaging. The high-sensitivity micro-polarization array is fixedly arranged on the side of the image sensor close to the incident light, and the image sensor sends the obtained image data to a digital signal processor. , and perform image data demosaic processing according to the following steps: step 1, obtain a low-resolution partial polarization angle image; step 2, obtain a high-resolution unbiased image, and then obtain a low-resolution unbiased image; step 3. Obtain four intermediate images of different polarization angles, and then subtract the low-resolution unbiased images to obtain four low-resolution polarization angle difference images; step 4, obtain a high-resolution angle difference image; step 5, finally Four high-resolution images with different polarization angles are obtained, that is, the demosaicing process of the original image is completed, and the demosaicing problem of the high-sensitivity micro-polarization array is solved.

Description

A Demosaicing Method Applied to High Sensitivity Micro-polarized Array Imaging

technical field

The invention belongs to the technical field of polarization optical imaging and image processing, and relates to a demosaic method applied to high-sensitivity micro-polarization array imaging.

Background technique

Since each pixel position in the image directly acquired by the high-sensitivity micro-polarization array corresponds to only one polarization angle or an unbiased intensity value, in order to obtain the polarization information of the scene, it is necessary to obtain the 0°, 45°, 90° and 135° four Therefore, it is necessary to restore four high-resolution polarization angle images through the demosaic method according to the obtained pixel intensity information, and the existing demosaic method is only suitable for the slightly offset array of the traditional arrangement mode, and cannot It is directly applied on the high-sensitivity micro-polarization array.

Contents of the invention

The object of the present invention is to provide a demosaicing method applied to the imaging of high-sensitivity micro-polarization arrays, so as to solve the demosaicing problem of high-sensitivity micro-polarization arrays.

The technical solution adopted in the present invention is a demosaic method applied to a high-sensitivity micro-polarization array, which is characterized in that the high-sensitivity micro-polarization array is fixedly arranged on the side of the image sensor close to the incident light, and the image sensor will obtain The image data is sent to the digital signal processor, and the image data is demosaiced according to the following steps:

Step 1. For the image data collected by the high-sensitivity micro-polarization array and the image sensor, the polarization angle data of each polarization angle unit are respectively fused into one value to obtain a low-resolution partial polarization angle image;

Step 2. Estimating the unbiased intensity data of the pixel at the polarization angle position in the high-sensitivity micro-polarization array to obtain a high-resolution unbiased image;

Then average the estimated unbiased intensity data in each polarization angle unit of the high-resolution unbiased image to obtain a value as the unbiased intensity value of the corresponding polarization angle unit, and perform the same processing on the entire array range. That is, a low-resolution unbiased image is obtained;

Step 3. Under the guidance of the low-resolution unbiased image obtained through step 2, the low-resolution partial polarization angle image obtained in step 1 is obtained by interpolation to obtain four intermediate images of different polarization angles, and then the obtained Subtract the low-resolution unbiased images from the four low-resolution partial polarization angle images to obtain four low-resolution polarization angle difference images;

Step 4. The four low-resolution polarization angle difference images obtained in step 3 are processed by bilinear interpolation and upsampling to obtain a high-resolution angle difference image;

Step 5. Sum the high-resolution angular difference image obtained in step 4 and the high-resolution unbiased image obtained in step 2, and finally obtain four high-resolution images with different polarization angles, that is, complete the demosaicing of the original image deal with;

Among them, the high-sensitivity micro-polarization array includes several minimum periodic units closely arranged in sequence, and each minimum periodic unit includes four polarization units with different polarization angles, and the four different polarization angles are 0°, 45°, 90° or 135°, the lenses of each polarization unit include several polarizers and non-polarizers.

Further, the non-polarized lenses and polarizers in each polarizing unit are arranged in a staggered arrangement.

Further, the non-polarized lenses or polarizers are respectively arranged continuously along the horizontal, vertical or diagonal directions.

Further, in step 1, the polarization angle data of each polarization angle unit are respectively fused into one value by means of averaging.

Further, in step 2, the size of the low-resolution unbiased image is the same as that of the low-resolution partial polarization angle image obtained in step 1.

Further, in step 3, the method of estimating a certain pixel to be sought in the intermediate image is: to estimate the value of the pixel to be sought according to the known pixel values of four corresponding polarizers of the same angle in the neighborhood of the pixel to be sought , where the pixels to be requested are divided into two categories, the first type of pixels to be requested, whose corresponding polarization angle is the same as that of the adjacent pixels on the main and secondary diagonals, and the second type of pixels to be requested, whose corresponding polarization angle is horizontal or vertical to Neighboring pixels in the same direction.

Further, the method for estimating the second type of pixel to be sought is to first estimate the value of the pixel at the position of the first type of pixel, and use the estimated value of the pixel at the position of the first type of pixel as the known value of the position to estimate the value of the pixel of the second type The value of the pixel at the pixel location.

Further, in step 4, the high-resolution angular difference image has the same size as the high-resolution unbiased image obtained in step 2.

The beneficial effect of the present invention is that the demosaicing method designed for the micro-biased array of the highly photosensitive micro-polarized array solves the problem that the existing demosaicing method is only applicable to the micro-biased array of the traditional arrangement mode, and cannot be directly applied to the highly photosensitive micro-biased array The problem; make full use of the characteristics of the high-sensitivity polarization array, use the data collected by the unpolarized lens to estimate the gradient information required to restore the polarization data, and at the same time have the advantages of the traditional micro-polarization array demosaic, that is, use the neighborhood As much information as possible to estimate missing data. The above two means are combined to ensure the accuracy of the final polarization information as much as possible.

Description of drawings

Fig. 1 is provided with the digital camera system of high-sensitivity micro-polarization array of the present invention;

Among them, 1. Incident light, 2. Lens, 3. High-sensitivity micro-polarization array, 4. Image sensor, 5. Analog signal processor, 6. D/A converter, 7. Digital signal processor, 8. Image display ;

2 is a schematic diagram of functional modules of a digital signal processor in a digital camera system;

In the figure, 11. Low-resolution partial polarization angle module, 12. High-resolution unbiased module, 13. Low-resolution unbiased module, 14. Low-resolution polarization angle difference module, 15. High-resolution polarization angle difference module , 16. High resolution final image module;

Fig. 3 is the minimum periodic unit of the highly sensitive micro-polarization array;

In Fig. 3, A.0 ° polarizing unit, B.45 ° polarizing unit, C.90 ° polarizing unit, D.135 ° polarizing unit, polarizer group in E.0 ° polarizing unit;

Fig. 4 is a partial schematic diagram of the known image information input into the low-resolution polarization angle difference module.

detailed description

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

The present invention provides a demosaic method applied to a high-sensitivity micro-biased array. The high-sensitivity micro-slanted array is fixedly arranged on the side of the image sensor close to the incident light, and the image sensor sends the obtained image data to the digital signal processor 7 , and follow the steps below to demosaic the image data:

Step 1. For the image data collected by the image sensor, the polarization angle data of each polarization angle unit are respectively fused into one value to obtain a low-resolution partial polarization angle image.

Step 2. Estimate the unbiased intensity data of the pixel at the polarization angle position in the high-sensitivity micro-polarization array, and obtain a high-resolution unbiased image with the same size as the original image;

Then average the estimated unbiased intensity data in each polarization angle unit to obtain a value, which is used as the unbiased intensity value of the corresponding polarization angle unit, and the same process is performed on the entire array range to obtain a low-resolution unbiased intensity value. polarized image; the low-resolution unbiased image has the same size as the low-resolution partial polarization angle image obtained through step 1.

Step 3. Under the guidance of the low-resolution unbiased image obtained through step 2, the low-resolution partial polarization angle image obtained in step 1 is obtained by interpolation to obtain four intermediate images of different polarization angles, and then the obtained Subtract the low-resolution unbiased image from the four angle images to obtain the low-resolution polarization angle difference image;

Step 4. The low-resolution polarization angle difference image obtained in step 3 is processed through bilinear interpolation and upsampling to obtain a high-resolution angle difference image. The high-resolution angle difference image and the step The high-resolution unbiased images obtained in 2 have the same size.

Step 5. Sum the high-resolution angular difference image obtained in step 4 and the high-resolution unbiased image obtained in step 2, and finally obtain four high-resolution images with different polarization angles, that is, complete the demosaicing of the original image deal with.

Example:

The high-sensitivity polarization array 3 is arranged on the side of the image sensor 4 close to the incident light, the image sensor 4 sends the obtained image data to the digital signal processor 7, and the image data is demosaiced according to the following steps:

1) Assuming that the minimum periodic unit of the high-sensitivity polarization array 3 is shown in Figure 3, after passing through the 0° polarization angle unit A and the image sensor 4 in the minimum unit, the collected data contains eight unbiased data, and also contains One or more 0° polarization angle data. Wherein, the collection quantity of 0° polarization angle data is determined by the operation mode selected by the image sensor 4 that collects the raw data.

According to the internal structure of the camera shown in Figure 1, the unbiased data and polarization angle data collected above will be processed by the analog signal processor 5 and the D/A converter 6 and then sent to the digital signal processor 7, denoted as For signal a, the entire demosaicing algorithm is performed in the digital signal processor 7, and the functional modules included in the digital signal processor 7 are shown in FIG. 2 .

As shown in Figure 2, the processed signal a is sent to the low-resolution partial polarization angle module 11 and the high-resolution unbiased module 12 at the same time, and the low-resolution partial polarization angle module 11 will be included in each polarization angle unit The n polarization angle data of are fused into one value, which can be obtained by summing and averaging. In this way, in the minimum periodic unit shown in Figure 3, each polarization angle unit corresponds to only one polarization angle value, rather than eight polarization angle values, so it is said that the obtained low-resolution partial polarization angle module 11 is based on Polarization angle unit rather than pixel level images are low resolution partial polarization angle images.

2) In FIG. 2, the high-resolution unbiased module 12 in the digital signal processor 7 is used to estimate the unbiased intensity data of the pixel at the position of the polarizer in the high-sensitivity micro-polarized array.

Taking the 0° polarization angle unit A in Figure 3 as an example, the high-resolution unbiased module 2 can estimate the unbiased data value of each pixel position in the polarizer group E in the 0° polarization unit, so as to estimate the The second column of the second row, numbered as the pixel intensity value I ₂₂ at the position 22 is taken as an example, I ₂₂ =(I ₁₂ +I ₃₂ )/2; the estimation method for other positions is similar. After the high-resolution unbiased module 12 finishes processing, a high-resolution unbiased image with the same size as the high-sensitivity micro-biased array can be obtained.

The low-resolution unbiased module 13 is used to obtain the high-resolution unbiased image output in the high-resolution unbiased module 12, and then respectively sum and average the unbiased data estimated in each polarization angle unit position in the image to obtain A value, as the unbiased intensity data corresponding to the polarization angle unit, so that each polarization angle unit corresponds to an unbiased intensity data, so the image obtained after processing is called a low-resolution unbiased image. The low-resolution unbiased image has the same size as the low-resolution partial polarization angle image obtained by the low-resolution partial polarization angle module 11 .

3) The low-resolution polarization angle difference module 14 is used to receive the output images of the low-resolution partial polarization angle module 11 and the low-resolution unbiased module 13, and then, the low-resolution partial polarization angle image is added to the low-resolution unbiased image Under the guidance of , four intermediate images with different polarization angles were obtained by interpolation.

The estimation method of the intermediate image is as follows: in order to improve the accuracy of the estimation of the intermediate image of the polarization angle, when we estimate the value of a certain pixel in the intermediate image, we need to use four corresponding polarizers with the same angle in the neighborhood of the pixel. According to the arrangement of the high-sensitivity micro-polarization array, the pixels to be obtained can be divided into two categories, one is that the polarization angle corresponding to the pixel to be obtained is the same as that of the adjacent pixels on the main and secondary diagonals, and the other is The category is that the polarization angle corresponding to the pixel to be obtained is the same as that of its adjacent pixels in the horizontal or vertical direction. The first type of situation meets our requirements, while the second type of situation can only be estimated by two pixel values in a single direction. In order to meet our requirements, we have adopted the following solution: first estimate the pixel position of the first type The value of the pixel, after obtaining the estimated value, it is used as the known value of the position to estimate the value of the pixel at the second type of pixel position.

Take the 0° polarized intermediate image as an example, as shown in Figure 4, it is necessary to obtain non-0° polarization angle units such as 22 (representing the position of the second row and the second column), 23 (representing the position of the second row and the third column) position), 24 (representing the position of the fourth column in the second row) corresponds to the 0° value at the position. In Fig. 4, each box represents the pixel at the same position of the two images of the low-resolution unbiased image and the low-resolution partial polarization angle image, and the I in the box represents the unbiased intensity value at this position, I0, I45, I135 respectively represent the polarization angle values of 0 degree, 45 degree and 135 degree at this position.

Step 1, taking the 0° value I0 ₂₂ at number 22 as an example, the specific method is as follows:

1.1) First calculate the gradient (Cgrad) in the main diagonal direction (Lgrad) and the subdiagonal direction (Cgrad), and I represents the unbiased intensity value:

Lgrad=|I ₁₁ -I ₃₃ |+|2*I ₂₂ -I ₁₁ -I ₃₃ |,

_Cgrad ＝|I13- _I31 |+| ₂ *I22- _I13 - _I31 |,

1.2) Calculate the pixel value of the requested pixel position in different diagonal directions:

LI0 ₂₂ = (I0 ₁₁ +I0 ₃₃ )/2+(2*I ₂₂ -I ₁₁ -I ₃₃ )/2,

CI0 ₂₂ = (I0 ₁₃ +I0 ₃₁ )/2+(2*I ₂₂ -I ₁₃ -I ₃₁ )/2,

1.3) Compare the value calculated in the first step with the empirically set threshold T to determine the desired pixel value:

If MAX(Lgrad, Cgrad)<T,

I0 ₂₂ = (LI0 ₂₂ +CI0 ₂₂ )/2,

elseif Lgrad<Cgrad,

I0 ₂₂ = LI0 ₂₂ ,

Else,

I0 ₂₂ =CI0 ₂₂ ,

End.

Then, according to the estimated values in non-diagonal directions, that is, the horizontal and vertical directions, the values of the corresponding missing angles are estimated.

Step 2. Take the 0° value in the second row and third column numbered 23 as an example, the specific method is as follows:

2.1) First calculate the gradients in the horizontal direction (Hgrad) and the vertical direction (Vgrad):

_Hgrad ＝| _I22 -I24|+| ₂ * _I23 -I22- _I24 |,

Vgrad=|I ₁₃ -I ₃₃ |+|2*I ₂₃ -I ₁₃ -I ₃₃ |,

2.2) Calculate the pixel values of the requested pixel positions in the horizontal and vertical directions respectively:

HI0 ₂₃ = (I0 ₂₂ +I0 ₂₄ )/2+(2*I ₂₃ -I ₂₂ -I ₂₄ )/2,

VI0 ₂₃ = (I0 ₁₃ +I0 ₃₃ )/2+(2*I ₂₃ -I ₁₃ -I ₃₃ )/2,

2.3) Compare the value calculated in the first step with the empirically set threshold T to determine the desired pixel value:

If MAX(Hgrad, Vgrad)<T,

I0 ₂₃ ＝(HI0 ₂₃ +VI0 ₂₃ )/2,

elseif Hgrad<Vgrad,

I0 ₂₃ =HI0 ₂₃ ,

Else,

I0 ₂₃ = VI0 ₂₃ ,

End.

In this way, the 0° pixel values of all other pixel positions except the pixel at the outermost position of the array can be obtained. Similarly, we can obtain the pixel values of the other three angles of 45°, 90° and 135°.

Among them, I is the unbiased intensity value, I0, I45, I90 and I135 are the polarization angle values of 0 degrees, 45 degrees, 90 degrees and 135 degrees respectively; the subscript (xy) in the lower right corner of the unbiased intensity value I represents the position of the pixel , x represents the number of rows where the pixel is located, y represents the number of columns where the pixel is located; L and C represent the main diagonal and sub-diagonal directions respectively, H and V represent the horizontal direction and vertical direction respectively, so LI0 ₂₂ represents the second row Estimated 0 degree polarization angle value in the direction of the main diagonal in the second column.

After the low-resolution intermediate images of partial polarization angles are obtained, the low-resolution unbiased images are subtracted from the obtained four intermediate images of different polarization angles, so that the low-resolution polarization angle difference images are obtained.

4) Input the low-resolution polarization angle difference image into the high-resolution polarization angle difference module 15, make it the same size as the high-sensitivity micro-polarization array through bilinear interpolation and up-sampling technology, and obtain the high-resolution image after final processing The size of the angle difference image is the same as the image output from the high-resolution unbiased module 12 .

5) The input of the high-resolution final image module 16 is the image output by the high-resolution unbiased module 12 and the high-resolution polarization angle difference module 15, by summing the high-resolution angle difference image and the high-resolution unbiased image, The final high-resolution image b is obtained, and then the high-resolution image b is sent to the image display 8 for display.

The present invention aims at the demosaicing method designed for the highly photosensitive micro-biased array micro-biased array, which solves the problem that the existing demosaicing method is only applicable to the micro-biased array of the traditional arrangement mode, and cannot be directly applied to the highly photosensitive micro-biased array; fully utilizes The characteristics of the high-sensitivity polarization array are used, and the data collected by the unpolarized lens is used to estimate the gradient information required to restore the polarization data. At the same time, it has the advantages of the traditional micro-polarization array demosaic, that is, using as much information as possible in the neighborhood to estimate missing data. The above two means are combined to ensure the accuracy of the final polarization information as much as possible.

Claims (8)

1. a kind of demosaicing methods for being applied to high photosensitive micro- inclined array, it is characterised in that the photosensitive micro- inclined array (3) of height is solid Surely be arranged at imaging sensor (4) close to incident light (1) side, imaging sensor (4) by obtained view data send to Digital signal processor (7), and follow the steps below to the processing of view data demosaicing：

Step 1, the view data for being collected by high photosensitive micro- inclined array (3) and imaging sensor (4), will wherein each The polarization angle data of polarization angle unit are fused into a value respectively, obtain a width low resolution partial polarization angular image；

In step 2, the high photosensitive micro- inclined array of estimation at polarization angle position pixel unbiased intensity data, obtain high-resolution without Inclined image；

Again by each polarization angle unit of high-resolution unbiased graph picture, it is averaging by the unbiased intensity data estimated A value is obtained, as the unbiased strength values of correspondence polarization angle unit, to carrying out same processing in whole array range, Obtain a width low resolution unbiased graph picture；

Step 3, the low resolution partial polarization angular image that will be obtained through step 1, obtained low resolution is being handled through step 2 Under the guidance of unbiased graph picture, the intermediate image of four width different polarization angles is obtained by interpolation, then again in obtained fourth officer Between subtract low resolution unbiased graph picture in image, that is, obtain fourth officer low resolution polarization angle difference image；

Step 4, the fourth officer low resolution polarization angle difference image that will be obtained through step 3, by bilinear interpolation, the place of up-sampling Reason method, obtains a panel height resolution angle difference image；

Step 5, the high-resolution unbiased graph picture to the high-resolution angle difference image that is obtained through step 4 with being obtained through step 2 are asked With finally give the high-definition picture of different polarization angle, that is, complete the demosaicing processing of original image；

Wherein, the photosensitive micro- inclined array (3) of the height includes several compact arranged minimum period units successively, each described Minimum period unit includes the polarization unit of four different polarization angles, four different polarization angles are respectively 0 °, 45 °, 90 ° or 135 °, the eyeglass of the polarization unit described in each includes several polarizers (I) and without deviation lens (II).

2. demosaicing methods as claimed in claim 1, it is characterised in that in each polarization unit without deviation lens It is the setting that is staggered with polarizer.

3. demosaicing methods as claimed in claim 1, it is characterised in that described without deviation lens or polarizer, respectively along water Flat, vertical or cornerwise direction continuous arrangement.

4. the demosaicing methods as described in any one in claims 1 to 3, it is characterised in that in the step 1, will be each The polarization angle data of individual polarization angle unit, are fused into a value by the method averaged respectively.

5. the demosaicing methods as described in any one in claims 1 to 3, it is characterised in that described in the step 2 The size of low resolution partial polarization angular image of the low resolution unbiased graph picture with being obtained by step 1 is identical.

6. the demosaicing methods as described in any one in claims 1 to 3, it is characterised in that described in the step 3 The method of estimation of a certain pixel to be asked is in intermediate image：It is inclined that equal angular is corresponded to therewith according to four in the neighborhood of pixel to be asked Shake the known pixel values of piece, to estimate the numerical value of the pixel to be asked, wherein, the pixel to be asked is divided into two classes, and the first kind is treated Pixel is sought, its corresponding polarization angle is identical with its major-minor diagonal adjacent pixels, Equations of The Second Kind pixel to be asked, its corresponding polarization Angle is identical with the adjacent pixel in its horizontally or vertically direction.

7. demosaicing methods as claimed in claim 6, it is characterised in that the method for estimation of Equations of The Second Kind pixel to be asked is, first First estimate the value of first kind pixel position pixel to be asked, regard the estimate of first kind pixel position pixel to be asked as the position The given value put, to estimate the value of Equations of The Second Kind pixel position pixel to be asked.

8. the demosaicing methods as described in any one in claims 1 to 3, it is characterised in that described in the step 4 High-resolution angle difference image is identical with the size of the high-resolution unbiased graph picture by being obtained in step 2.