CN105611258A - Imaging method of image sensor, imaging device and electronic device - Google Patents

Abstract

The invention discloses an imaging method, an imaging device and an electronic device of an image sensor, wherein the image sensor comprises a pixel array and an optical filter array arranged on the pixel array, the optical filter array comprises M optical filter groups arranged in a Bayer mode, each optical filter group covers N pixels, M and N are positive integers, and the imaging method comprises the following steps: reading output pixel values of N pixels covered by each filter set; respectively combining output pixel values of N pixels covered by each filter set in the M filter sets to generate M combined pixel values; and generating an RGB format image according to the M merged pixel values. According to the imaging method provided by the embodiment of the invention, N pixels in the pixel array are logically combined into one merged pixel, the RGB format image can be directly output without interpolation processing, and the generated image still maintains the image detail restoring capability although the number of pixels is reduced, so that the user experience is improved.

Description

Imaging method of image sensor, imaging device and electronic device

technical field

The invention relates to the field of imaging technology, in particular to an imaging method of an image sensor, an imaging device and an electronic device.

Background technique

With the increasing number of pixels of mobile phone image sensors, the ability to restore image details is getting stronger and stronger, but at the same time, because the total area of image sensors has not actually increased, increasing the number of pixels is at the expense of the size of a single photosensitive pixel, resulting in Mobile phones have limited ability to shoot in low light.

In this regard, pixel merging technology is adopted in related technologies, for example, 4 adjacent pixels of the same filter color in a 16MP image sensor are combined into one pixel. Although this technology can improve the signal-to-noise ratio of the image sensor, the disadvantage is that The image resolution is reduced by half, which reduces the ability to restore image details while reducing the number of pixels, thus greatly reducing the user experience.

Contents of the invention

The present invention aims to solve one of the technical problems in the related art at least to a certain extent. For this reason, an object of the present invention is to propose an imaging method for an image sensor, which logically synthesizes N pixels in the pixel array into one merged pixel, and can directly output an image in RGB format, because no need for image processing in the image processing Interpolation processing makes the image generated by this method reduce the number of pixels, but still maintains the ability to restore image details, thereby greatly improving user experience.

A second object of the present invention is to provide an imaging device.

The third objective of the present invention is to provide an electronic device.

In order to achieve the above object, the imaging method of the image sensor according to the embodiment of the first aspect of the present invention, the image sensor includes a pixel array and an optical filter array arranged on the pixel array, and the optical filter array includes a Bayer pattern M filter groups are arranged, and each filter group covers N pixels, wherein M and N are positive integers, and the imaging method includes the following steps: reading the N pixels covered by each filter group The output pixel values of pixels; the output pixel values of the N pixels covered by each filter group in the M filter groups are respectively combined to generate M combined pixel values; and according to the M combination The pixel values generate an image in RGB format.

According to the imaging method of the image sensor in the embodiment of the present invention, the N pixels in the pixel array are logically synthesized into one merged pixel, and the image in RGB format can be directly output. Since there is no need for interpolation processing in the image processing, the generated Although the number of pixels of the image is reduced, it still maintains the ability to restore image details, thereby greatly improving the user experience.

In an embodiment of the present invention, N is equal to four, wherein each filter set includes one red filter, one blue filter and two green filters.

In one embodiment of the present invention, the combining the output pixel values of the four pixels covered by each filter group to generate a combined pixel value includes: combining the output pixel values of the four pixels covered by the two green filters Half of the sum of the output pixel values of the two pixels is respectively used as two green pixel components in the combined pixel value; the output pixel value of the pixel covered by the red filter is used as the red pixel in the combined pixel value component; using the output pixel value of the pixel covered by the blue filter as the blue pixel component in the combined pixel value.

In one embodiment of the present invention, each pixel includes: a photosensitive device, a transfer tube, a source follower, and an analog-to-digital converter, wherein one end electrode of the transfer tube is connected to the cathode electrode of the photosensitive device, The electrode at the other end of the transmission tube is connected to the source follower, and the source follower is connected to the analog-to-digital converter.

In order to achieve the above object, the imaging device of the second embodiment of the present invention includes: an image sensor, the image sensor includes: a pixel array; an optical filter array arranged on the pixel array, and the optical filter array includes M optical filter groups arranged in a Bayer manner, each optical filter group covers N pixels, wherein M and N are positive integers; and an image processing module connected to the image sensor, the image processing module uses Read the output pixel values of the N pixels covered by each filter group, and combine the output pixel values of the N pixels covered by each filter group in the M filter groups respectively to generate M combined pixel values, and generate an RGB format image according to the M combined pixel values.

According to the imaging device of the embodiment of the present invention, the image processing module logically synthesizes N pixels in the pixel array into one merged pixel, and can directly output an image in RGB format. Since there is no need for interpolation processing in the image processing, the generated image Although the number of pixels is reduced, the ability to restore image details is still maintained, thereby greatly improving the user experience.

In an embodiment of the present invention, N is equal to four, wherein each filter set includes one red filter, one blue filter and two green filters.

In one embodiment of the present invention, the image processing module combines the output pixel values of the four pixels covered by each filter group to generate a combined pixel value, wherein,

The image processing module uses half of the sum of the output pixel values of the two pixels covered by the two green filters as the two green pixel components in the combined pixel value, and uses the red filter The output pixel value of the covered pixel is used as the red pixel component in the combined pixel value, and the output pixel value of the pixel covered by the blue filter is used as the blue pixel component in the combined pixel value.

In one embodiment of the present invention, each pixel includes: a photosensitive device, a transfer tube, a source follower, and an analog-to-digital converter, wherein one end electrode of the transfer tube is connected to the cathode electrode of the photosensitive device, The electrode at the other end of the transmission tube is connected to the source follower, and the source follower is connected to the analog-to-digital converter.

In one embodiment of the present invention, the photosensitive device is a photodiode.

In one embodiment of the present invention, the image sensor further includes: a lens array disposed above the filter array, and the lens array is used to converge light to the photosensitive sensor below the filter array. device.

In one embodiment of the present invention, the lens array includes a plurality of microlenses, and each microlens is arranged corresponding to one of the photosensitive devices.

In order to achieve the above purpose, the electronic device according to the embodiment of the third aspect of the present invention includes the imaging device according to the embodiment of the second aspect of the present invention.

According to the electronic device of the embodiment of the present invention, due to the imaging device, N pixels are logically synthesized into one merged pixel during imaging, and an image in RGB format can be directly output without interpolation processing, so that although the generated image is reduced The number of pixels is reduced, but the ability to restore image details is still maintained, which greatly improves the user experience.

In one embodiment of the present invention, the electronic device includes a mobile phone.

Description of drawings

FIG. 1 is a flowchart of an imaging method of an image sensor according to an embodiment of the present invention;

2 is a schematic diagram of an optical filter array according to an embodiment of the present invention;

3 is a schematic diagram of combining output pixel values of four pixels into one combined pixel value according to an embodiment of the present invention;

4 is a schematic circuit diagram of a pixel according to an embodiment of the present invention;

5 is a schematic block diagram of an imaging device according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of an image sensor according to an embodiment of the present invention.

detailed description

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

The imaging method, imaging device and electronic device of the image sensor according to the embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a flowchart of an imaging method of an image sensor according to an embodiment of the present invention.

Firstly, the image sensor used in the method of the embodiment of the present invention will be described.

The image sensor includes a pixel array and a filter array arranged on the pixel array, the filter array includes M filter groups arranged in a Bayer manner, each filter group covers N pixels, and M and N are positive integer.

In one embodiment of the present invention, N is equal to four, wherein each filter set includes one red filter, one blue filter and two green filters.

Specifically, please refer to FIG. 2 for a schematic diagram of the filter array 13 and the filter set 131 , and N=4 is taken as an example in FIG. 2 . Among them, Gr, R, B, and Gb are used to identify the color of the filter, R is used to identify the red filter, B is used to identify the blue filter, and Gr and Gb are used to identify the green filter. piece.

As shown in Figure 1, the imaging method of the image sensor of the embodiment of the present invention includes the following steps:

S1, read the output pixel values of the N pixels covered by each filter group.

Specifically, the output of the image sensor is read, that is, the output pixel values of the N pixels covered by each filter group are read.

S2. Combining the output pixel values of the N pixels covered by each filter group in the M filter groups respectively to generate M combined pixel values.

Specifically, for N pixels corresponding to one filter set, the output pixel values of the N pixels are combined to generate one combined pixel value.

For example, one filter set covers four pixels, and the filters covering the four pixels are respectively a red filter, a blue filter and two green filters. For a 16M pixel image sensor, four adjacent pixels are merged into one merged pixel, then the final output image becomes 4M pixels.

In one embodiment of the present invention, the output pixel values of the four pixels covered by each filter group are combined to generate a combined pixel value, which specifically includes:

Using half of the sum of the output pixel values of the two pixels covered by the two green filters as the two green pixel components in the combined pixel value;

The output pixel value of the pixel covered by the red filter is used as the red pixel component in the combined pixel value;

The output pixel value of the pixel covered by the blue filter as the blue pixel component in the binned pixel value.

Specifically, please refer to Figure 3, the left side of Figure 3 represents four pixels covered by a filter group, these four pixels correspond to the red filter, blue filter and two green filters respectively , combine the output pixel values of these four pixels into a combined pixel value (a combined pixel is shown on the right side of FIG. 3 ).

More specifically, the output pixel value of the pixel covered by the red filter among the four pixels is used as the red pixel component (ie RGB(R)) in the combined pixel value, and the pixel value of the pixel covered by the blue filter The output pixel value is taken as the blue pixel component (i.e. RGB(B)) in the combined pixel value, and half of the sum of the output pixel values of the two pixels covered by the two green filters is used as two of the combined pixel values respectively. A green pixel component (ie RGB(Gr+Gb)/2).

S3, generating an image in RGB format according to the M combined pixel values.

Specifically, since each merged pixel value includes a red pixel component, a blue pixel component and a green pixel component, subsequent image processing does not require interpolation processing, and therefore, a 4M pixel RGB format image can be directly output.

Through the imaging method of the embodiment of the present invention, although the number of pixels of the output image is reduced, since the output of the image sensor pixel array is not interpolated, the output 4M pixel image still has the detail restoration ability of the 16M pixel image .

In addition, since there are two actual G-channel pixels in one merged pixel value after synthesis, the signal-to-noise ratio of the G-channel in the 4M pixel image output by the image sensor will be improved, thereby improving the image quality.

Please refer to FIG. 4 , which is a schematic circuit diagram of a pixel. In one embodiment of the present invention, each pixel 111 includes: a photosensitive device 14, a transmission tube 15, a source follower 16, and an analog-to-digital converter 17, wherein one end electrode of the transmission tube 15 is connected to the cathode of the photosensitive device 14 The other electrode of the transmission tube 15 is connected to the gate electrode of the source follower 16 , and the source follower 16 is connected to the analog-to-digital converter 17 .

Specifically, the photosensitive device 14 is used to convert light into electric charge, and the electric charge generated is proportional to the light intensity. The transmission tube 15 is used to control the circuit to be turned on or off according to the control signal. When the circuit is turned on, the source follower 16 is used to convert the charge signal generated by the photosensitive device 14 through illumination into a voltage signal. The analog-to-digital converter 17 is used to convert the voltage signal into a digital signal for processing by an image processing module connected to the image sensor.

The imaging method of the image sensor in the embodiment of the present invention logically synthesizes N pixels in the pixel array into one merged pixel, and can directly output an image in RGB format. Since there is no need for interpolation processing in image processing, the resulting Although the number of pixels of the image is reduced, it still maintains the ability to restore image details, thereby greatly improving the user experience.

In order to realize the above embodiments, the present invention also proposes an imaging device.

FIG. 5 is a schematic block diagram of an imaging device according to an embodiment of the present invention. As shown in FIG. 5 , the imaging device 100 of the embodiment of the present invention includes: an image sensor 10 and an image processing module 20 .

Wherein, the image sensor 10 comprises: a pixel array 11; a filter array 13 arranged on the pixel array 11, the filter array 13 comprising M filter groups 131 arranged in a Bayer manner, each filter group 131 N pixels 111 are covered, where M and N are positive integers.

In one embodiment of the present invention, N is equal to four, wherein each filter set 131 includes one red filter, one blue filter and two green filters.

Specifically, please refer to FIG. 2 for a schematic diagram of the filter array 13 and the filter set 131 , and N=4 is taken as an example in FIG. 2 . Among them, Gr, R, B, and Gb are used to identify the color of the filter, R is used to identify the red filter, B is used to identify the blue filter, and Gr and Gb are used to identify the green filter. piece.

The image processing module 20 is connected with the image sensor 10, and the image processing module 20 is used to read the output pixel values of the N pixels 111 covered by each filter group 131, and respectively filter each light in the M filter groups The output pixel values of the N pixels 111 covered by the slice group are combined to generate M combined pixel values, and an RGB format image is generated according to the M combined pixel values.

Specifically, the image processing module 20 reads the output of the image sensor, that is, reads the output pixel values of the N pixels covered by each filter set. For N pixels corresponding to one filter set, the output pixel values of the N pixels are combined to generate a combined pixel value, and an RGB format image is directly generated according to the M combined pixel values.

For example, one filter set 131 covers four pixels 11, and the filters covering four pixels are one red filter, one blue filter and two green filters respectively. For an image sensor with 16M pixels, four adjacent pixels 11 are merged into one merged pixel, then the final output image becomes 4M pixels.

In one embodiment of the present invention, the image processing module 20 combines the output pixel values of the four pixels 111 covered by each filter group 131 to generate a combined pixel value, wherein the image processing module 20 combines two green Half of the sum of the output pixel values of the two pixels 11 covered by the optical filter are respectively used as two green pixel components in the combined pixel value; the output pixel value of the pixel 11 covered by the red filter is used as the red color in the combined pixel value Pixel component: the output pixel value of the pixel 11 covered by the blue filter is used as the blue pixel component in the combined pixel value.

Specifically, referring to FIG. 3 , the image processing module 20 combines output pixel values of four adjacent pixels covered by one filter group into one combined pixel value. That is to say, the output pixel value of the pixel covered by the red filter among the four pixels is used as the red pixel component (ie RGB(R)) in the combined pixel value, and the pixel value of the pixel covered by the blue filter The output pixel value is taken as the blue pixel component (i.e. RGB(B)) in the combined pixel value, and half of the sum of the output pixel values of the two pixels covered by the two green filters is used as two of the combined pixel values respectively. A green pixel component (ie RGB(Gr+Gb)/2).

More specifically, since each combined pixel value includes a red pixel component, a blue pixel component and a green pixel component, subsequent image processing does not require interpolation processing. Therefore, the image processing module 20 can directly output a 4M pixel RGB format image without interpolation processing.

Through the imaging device of the embodiment of the present invention, although the number of pixels of the output image is reduced, since the output of the image sensor 10 pixel array 11 is not interpolated, the output 4M pixel image still has the details of the 16M pixel image. ability.

In addition, since there are two actual G-channel pixels in one merged pixel value after synthesis, the signal-to-noise ratio of the G-channel in the output 4M pixel image will be improved, thereby improving the image quality.

Please refer to FIG. 4, in one embodiment of the present invention, each pixel 111 includes: a photosensitive device 14, a transfer tube 15, a source follower 16, and an analog-to-digital converter 13, wherein one end electrode of the transfer tube 15 is connected to The cathode electrode of the photosensitive device 14 and the other end electrode of the transmission tube 15 are connected to a source follower 16 , and the source follower 16 is connected to an analog-to-digital converter 17 .

In one embodiment of the present invention, the photosensitive device 14 is a photodiode.

Please refer to FIG. 6 , in one embodiment of the present invention, the image sensor 10 further includes: a lens array 19 arranged above the filter array 13 , and the lens array 19 is used to converge light to the photosensitive sensor below the filter array 13 . device 14.

Among them, the four filters (R, Gr, Gb, B) in FIG. 6 form a filter set 131 .

In one embodiment of the present invention, the lens array 19 includes a plurality of microlenses 191 , and each microlens 191 is arranged corresponding to one photosensitive device 14 .

In the imaging device of the embodiment of the present invention, the image processing module logically synthesizes N pixels in the pixel array into one combined pixel, and can directly output an image in RGB format. The number of pixels is reduced, but the ability to restore image details is still maintained, thereby greatly improving the user experience.

In order to realize the above embodiments, the present invention also proposes an electronic device 200 . The electronic device 200 includes the imaging device 100 according to the embodiment of the present invention.

In one embodiment of the present invention, the electronic device 200 includes a mobile phone.

The electronic device of the embodiment of the present invention, because of the imaging device, logically synthesizes N pixels into one merged pixel during imaging, and can directly output an image in RGB format without interpolation processing, so that although the generated image has reduced The number of pixels is reduced, but the ability to restore image details is still maintained, which greatly improves the user experience.

In describing the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", " Back", "Left", "Right", "Vertical", "Horizontal", "Top", "Bottom", "Inner", "Outer", "Clockwise", "Counterclockwise", "Axial", The orientation or positional relationship indicated by "radial", "circumferential", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying the referred device or element Must be in a particular orientation, be constructed in a particular orientation, and operate in a particular orientation, and therefore should not be construed as limiting the invention.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

In the present invention, unless otherwise clearly specified and limited, terms such as "installation", "connection", "connection" and "fixation" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection , or integrated; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components or the interaction relationship between two components, unless otherwise specified limit. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention according to specific situations.

In the present invention, unless otherwise clearly specified and limited, the first feature may be in direct contact with the first feature or the first and second feature indirectly through an intermediary. touch. Moreover, "above", "above" and "above" the first feature on the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. "Below", "beneath" and "beneath" the first feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is less horizontally than the second feature.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims (13)

1. a formation method for imageing sensor, is characterized in that, described imageing sensor comprises pel array and is arranged onFilter arrays on described pel array, described filter arrays comprises into M the filter set that Bayer mode is arranged, everyIndividual filter set covers N pixel, and wherein, M and N are positive integer, and described formation method comprises the following steps:

Read the output pixel value of N pixel of described each filter set covering;

Respectively the output pixel value of N the pixel that in a described M filter set, each filter set covers is merged with lifeBecome M to merge pixel value; And

Merge pixel value according to described M and generate rgb format image.

2. the formation method of imageing sensor as claimed in claim 1, is characterized in that, N equals four, wherein, described inEach filter set comprises a Red lightscreening plate, a blue color filter and two green color filters.

3. the formation method of imageing sensor as claimed in claim 2, is characterized in that, described each filter set is coveredThe output pixel value of four pixels of lid merges to generate one and merges pixel value, specifically comprises:

The half of the output pixel value sum of two pixels that described two green color filters are covered is respectively as described merging pictureTwo green pixel components in element value;

The red pixel component of the output pixel value of the pixel that described Red lightscreening plate is covered in described merging pixel value;

The blue pixel component of the output pixel value of the pixel that described blue color filter is covered in described merging pixel value.

4. the formation method of imageing sensor as claimed in claim 1, is characterized in that, each pixel, comprising: sensitizationDevice, transfer tube, source follower and analog-digital converter, wherein,

A described transfer tube wherein termination electrode is connected to the cathode electrode of described sensor devices, and described transfer tube is other end electricity whereinThe utmost point is connected to described source follower, and described source follower is connected to described analog-digital converter.

5. an imaging device, is characterized in that, comprising:

Imageing sensor, described imageing sensor comprises:

Pel array;

Be arranged at the filter arrays on described pel array, described filter arrays comprises into the M that Bayer mode is arrangedIndividual filter set, each filter set covers N pixel, and wherein, M and N are positive integer; And

With the image processing module that described imageing sensor is connected, described image processing module is used for reading described each optical filterThe output pixel value of N the pixel that group covers, and respectively by a N that in a described M filter set, each filter set coversThe output pixel value of pixel merges to generate M and merges pixel value, and merges pixel value generation RGB according to described MFormat-pattern.

6. imaging device as claimed in claim 5, is characterized in that, N equals four, wherein, and described each filter setComprise a Red lightscreening plate, a blue color filter and two green color filters.

7. imaging device as claimed in claim 6, is characterized in that, described image processing module covers each filter setThe output pixel value of four pixels of lid merges to generate one and merges pixel value, wherein,

One half point of the output pixel value sum of two pixels that described image processing module covers described two green color filtersThe output picture of the pixel covering not as two green pixel components in described merging pixel value, and by described Red lightscreening plateElement value is as the red pixel component in described merging pixel value, and the output picture of the pixel that described blue color filter is coveredElement value is as the blue pixel component in described merging pixel value.

8. imaging device as claimed in claim 5, is characterized in that, each pixel, comprising: sensor devices, transfer tube,Source follower and analog-digital converter, wherein,

A described transfer tube wherein termination electrode is connected to the cathode electrode of described sensor devices, and described transfer tube is other end electricity whereinThe utmost point is connected to described source follower, and described source follower is connected to described analog-digital converter.

9. imaging device as claimed in claim 8, is characterized in that, described sensor devices is photoelectric diode.

10. imaging device as claimed in claim 8, is characterized in that, described imageing sensor also comprises:

The lens arra that is arranged on described filter arrays top, described lens arra is for converging to described optical filter by lightOn the described sensor devices of array below.

11. imaging devices as claimed in claim 10, is characterized in that, described lens arra comprises multiple lenticules, everyIndividual lenticule and a corresponding setting of described sensor devices.

12. 1 kinds of electronic installations, is characterized in that, comprise the imaging device as described in any one in claim 5-11.

13. electronic installations as claimed in claim 12, is characterized in that, described electronic installation comprises mobile phone.