CN110235441A - Multi-camera system for low light - Google Patents

Abstract

A multi-camera system is provided. The multi-camera system includes a color image sensor for generating a color image and a monochromatic filter image sensor for generating a monochromatic image, wherein the monochromatic filter image sensor has a monochromatic filter image sensor, The monochromatic filter image sensor includes a zig-zag pixel array of pixel size x1.4 (ie, the square root of 2), the zigzag pixel array being pixels rotated 45 degrees relative to a square pixel array of pixel size x1 an array; generating an output color image by fusing the color image and the monochrome image.

Description

Multi-camera system for low light

technical field

Embodiments of the present invention relate to a multi-camera system for low illumination.

Background technique

The performance and technology of smartphone cameras are constantly improving. The quality of smartphone cameras is now almost the same as that of a Digital still camera (DSC)/Digital single-lensreflex camera (DSLR). Higher sensitivity is often highly desirable due to the need for further improvements in convenience.

Due to size constraints, smartphones need to use small image sensors and resolutions higher than 8MP or 12MP. To meet these two requirements, the size of a single pixel is getting smaller and smaller, such as 1.4um, 1.12um, 1um or smaller. Due to the small pixel size and the lower sensitivity of the sensor/pixel, smartphone cameras have poorer image quality in low light conditions than DSLR or higher quality still cameras. With the development of smartphone camera image sensors, their efficiency (ie, QE: quantum efficiency) has improved. For image sensors used in smartphone cameras (eg, so-called second cameras using WRGB (or RGBW) arrays or W sensors), a W (white) tinted filter sensor can be used. 4 is a diagram of an exemplary prior art WRGB (or RGBW) sensor.

Patent Citation 1 discloses an image sensor having an RGBW array that includes full wavelength type white (W) pixels in addition to RGB colors. When the pixel density of an image sensor increases and the amount of incident light per pixel decreases, resulting in an increase in relative noise, we propose the RGBW array as a solution to this problem. In FIG. 4 , the part with the diagonal line from the upper left to the lower right shows blue (B), the part with the diagonal line from the lower left to the upper right shows the red (R), and the part with the dotted line shows the green (G).

Patent Citation 2 also discloses that the RGBW arrangement exhibits a reduction in the density of color pixels relative to the RGB (or Bayer) pixel arrangement. 5 is a set of diagrams of examples of both monochrome and color cameras of the related art. FIG. 5 shows that the monochrome (or B/W (black/white)) camera image sensor has the same pixel size as the color camera image sensor. In FIG. 5 , the part with the diagonal line from the upper left to the lower right represents blue (B), the part with the diagonal line from the lower left to the upper right represents red (R), and the part with the dotted line represents green (G).

Citation List

Patent Literature

[Patent Document 1] US9147230B2

[Patent Document 2] US9253459B2

SUMMARY OF THE INVENTION

Problem to be solved by the present invention

The technical problems of the related art can be summarized as follows: (1) The improvement rate of QE is not very large. The QE of state-of-the-art sensors is already high enough. So even if the QE improves, the rate is within a few percent, so the sensitivity doesn't improve much.

(2-1) Although the white pixels of the WRGB array may be 1.3 to 2 times more sensitive than the color pixels, the WRGB array requires a complex interpolation algorithm, and may cause aliasing due to the low color resolution False color appears. Also, the 1.5 to 2x improvement isn't that big for practical use either.

(2-2) Although the second camera using the W sensor may reduce the interpolation and color resolution problems, the sensitivity improvement rate is the same as that of the second camera using the WRGB.

way of solving the problem

Embodiments of the present invention aim to improve low light performance by using relatively large white pixels in a different pixel array on the second camera while maintaining the same resolution as conventional color cameras.

According to an embodiment of the present invention, a multi-camera system is provided, comprising: a color image sensor for generating a color image; a monochromatic filter image sensor for generating a monochromatic image, wherein the monochromatic filter The image sensor has a sensor that includes a zigzag pixel array of pixel size x1.4 (ie, the square root of 2), which is an array of pixels rotated 45 degrees relative to a square pixel array of pixel size x1; by fusion The color image and the monochrome image produce an output color image.

Effects of the present invention

The multi-camera system according to the embodiment of the present invention can employ a second camera with a sensitivity 3 to 4 times that of a color image sensor, and can also provide a fusion output that achieves excellent low-light performance.

Description of drawings

Figure 1 is a diagram of a horizontal/vertical spatial sampling comparison;

Fig. 2 is a set of illustrations of zigzag pixel array processing;

Figure 3 is a diagram of an image processing pipeline;

4 is a diagram of an exemplary prior art WRGB sensor;

5 is a set of illustrations of examples of prior art monochrome and color cameras;

6 is a diagram of a square/zigzag pixel array comparison;

Figure 7 is a diagram of the use of a zigzag pixel array with a pixel size of x1.4;

In Fig. 2-5, the part with the diagonal line from the upper left to the lower right represents blue (B), the part with the diagonal line from the lower left to the upper right represents red (R), and the part with the dotted line represents green (G) .

Detailed ways

Mode 1 for Carrying Out the Invention

According to an embodiment of the present invention, a multi-camera system is provided, comprising: a color image sensor for generating a color image; a monochromatic filter image sensor for generating a monochromatic image, wherein the monochromatic filter The image sensor has a sensor that includes a zigzag pixel array of pixel size x1.4 (ie, the square root of 2), which is an array of pixels rotated 45 degrees relative to a square pixel array of pixel size x1; by fusion The color image and the monochrome image produce an output color image. Figure 1 shows the spatial sampling time interval/frequency of the Bayer colored filter sensor and the proposed monochromatic filter image sensor (an example using BW here (we refer to the monochromatic filter sensor as BW ( black/white) filter sensor)).

The Bayer Color Filter sensor has a checkered pattern with double density green pixels and fewer red and blue pixels because the green information helps to enhance the luminance signal (Y signal). The output of the colored filter sensor is processed by an image signal processor (ISP), and a red/green/blue (RGB) tri-color plane is generated through interpolation (demosaic) processing. See Figure 2.

Image signal processors (ISPs) used in current camera products have good interpolation algorithms and can produce the same resolution in both horizontal and vertical line directions. Figure 6 is an illustration of a square/zigzag pixel array comparison. The BW filter sensor has a zigzag pixel array as shown in Figure 6, which is a pixel array rotated 45 degrees relative to a square pixel array or a standard array. If proper interpolation is applied (if the same interpolation as G (green) interpolation is applied), the horizontal and vertical (H/V) resolution will be the same as that of the colored filter sensor even with 1.4x size pixels (Figure 7). Figure 7 is an illustration of the use of a zigzag pixel array with a pixel size of x1.4. The symbol "x1.4" is relative to the color sensor pixel. When applying the zigzag pixel array (45 degree rotated pixel array) and to obtain the horizontal/vertical resolution of a color sensor with a pixel size of "x1", 1.4 times the size of pixels can be used, as shown in Figure 7. When using 1.4 times the size of the pixel, the pixel area is twice as large, which means twice the sensitivity. Furthermore, when the BW pixel is used, the sensitivity is 1.5 to 2 times.

The overall sensitivity is 3 to 4 times. This is a bigger improvement than other schemes.

An example of the overall image processing pipeline is shown in Figure 3.

Mode 2 for Carrying Out the Invention

According to an embodiment of the present invention, a multi-camera system is provided. The multi-camera system includes a set of multiple cameras including multiple color image sensors, wherein the color image sensors are used to generate color images; the monochromatic filter image sensors are used to generate monochrome images. Monochromatic filter image sensors have sensors that include a zigzag pixel array with a pixel size of x1.4 (ie, the square root of 2). The zigzag pixel array is a pixel array rotated 45 degrees relative to a square pixel array of pixel size x1. An output color image is generated by fusing the plurality of color images and the monochrome image. Apply the same interpolation to the green pixels on the color image and the monochrome image. The sensitivity of the second sensor may have 3 to 4 times.

Claims (8)

1. a kind of multi-camera system characterized by comprising

Color image sensor, for generating color image；

Monochromatic filter imaging sensor, for generating monochrome image, wherein the monochromatic filter imaging sensor has packet The sensor for the zigzag pixel array that pixel size is x1.4 (i.e. 2 square roots) is included, the zigzag pixel array is opposite In 45 degree of square pixels array rotation of the pixel array that pixel size is x1；

Output color image is generated by merging the color image and the monochrome image.

2. a kind of multi-camera system characterized by comprising

Multiple color image sensors, for generating color image；

Monochromatic filter imaging sensor, for generating monochrome image, wherein the monochromatic filter imaging sensor has packet The sensor for the zigzag pixel array that pixel size is x1.4 (i.e. 2 square roots) is included, the zigzag pixel array is opposite In 45 degree of square pixels array rotation of the pixel array that pixel size is x1；

Output color image is generated by merging the color image and the monochrome image；

To on the color image green pixel and the monochrome image apply identical interpolation.

3. a kind of camera module characterized by comprising

Multi-camera system according to claim 1 or 2.

4. a kind of smart phone characterized by comprising

Camera module according to claim 3.

5. a kind of mobile device characterized by comprising

Camera module according to claim 3.

6. a kind of integrated circuit characterized by comprising

Multi-camera system according to claim 1 or 2.

7. a kind of method using multi-camera system and monochromatic filter imaging sensor, which is characterized in that more camera shootings Head system includes color image sensor, wherein it is x1.4 (2 that the monochromatic filter imaging sensor, which has including pixel size, Square root) aliased pixels array sensor, the zigzag pixel array be relative to pixel size be x1 rectangular picture Pixel array rotates 45 degree of pixel array；The described method includes:

The color image sensor generates color image；

The monochromatic filter sensor generates monochrome image；

Output color image is generated by merging the color image and the monochrome image.

8. a kind of method using multi-camera system and monochromatic filter imaging sensor, which is characterized in that more camera shootings Head system includes multiple cameras, including multiple color image sensors；The monochromatic filter imaging sensor has Including pixel size be x1.4 (2 square roots) aliased pixels array sensor, the zigzag pixel array be relative to Pixel size is 45 degree of square pixels array rotation of the pixel array of x1；The described method includes:

The multiple color image sensor generates color image respectively；

The monochromatic filter sensor generates monochrome image；

Output color image is generated by merging the color image and the monochrome image；

To on the color image green pixel and the monochrome image apply identical interpolation.