CN115278000A - Image sensor, image generation method, camera module and electronic equipment - Google Patents

Abstract

The invention provides an image sensor, an image generation method, a camera module and an electronic device. The image sensor includes: a photosensitive layer, a plurality of color filter layers, and a driving unit connected with the color filter layer; the plurality of color filter layers are superimposed and arranged On the light-receiving surface of the photosensitive layer; the driving unit is used to drive the relative positions between the plurality of color filter layers to change.

Description

Image sensor, image generation method, camera module and electronic device

technical field

Embodiments of the present invention relate to the field of image processing, and in particular, to an image sensor, an image generation method, a camera module, and electronic equipment.

Background technique

With the continuous development of science and technology, people have higher and higher requirements for the quality of photo shooting. How to improve the quality of photo shooting has become a hot issue that various manufacturers are competing to study.

In the related art, in order to improve the shooting quality of photos, multiple cameras with different color filter arrays are usually set on the shooting device, and the color filter arrays are used to adjust the color of light received by the photosensitive element. For example, one color filter array can be set A color camera and a monochrome camera with a monochrome color filter array, the color image with sufficient color information but poor light flux is collected by the color camera, and the monochrome image with insufficient color information but good light flux is collected by the monochrome camera, Then align and match the pixels of the two images, and finally obtain a photo with better quality through image synthesis.

In the process of pixel alignment and matching in the above method, it is difficult to ensure a good pixel alignment effect due to structural and process deviations between different cameras, which may easily lead to poor synthesis of multiple images.

Contents of the invention

Embodiments of the present invention provide an image sensor, an image generation method, a camera module, and an electronic device, which can perform image synthesis based on multiple images of different color filter arrays, and do not need to perform pixel alignment during the synthesis process, avoiding the pixel alignment process Affects the compositing effect, improving the effect and efficiency of image compositing.

In a first aspect, an embodiment of the present invention provides an image sensor, and the image sensor includes:

a photosensitive layer, a plurality of color filter layers, and a driving unit connected to the plurality of color filter layers;

The plurality of color filter layers are stacked on the light-receiving surface of the photosensitive layer;

The driving unit is used for driving the relative positions of the plurality of color filter layers to change.

In a second aspect, an embodiment of the present invention provides an image generation method applied to an electronic device, where the electronic device includes the image sensor of the first aspect, and the method includes:

acquiring a first image collected by the photosensitive layer of the image sensor;

Controlling the driving unit of the image sensor to drive the relative positions between the plurality of color filter layers to change at least once;

After changing the relative positions between the plurality of color filter layers each time, acquiring a second image collected by the photosensitive layer to obtain at least one second image;

A target image is generated based on the first image and the at least one second image.

In a third aspect, an embodiment of the present invention provides an image generating device, the device comprising:

a first acquisition module, configured to acquire a first image collected by the photosensitive layer of the image sensor;

A control module, configured to control the drive unit of the image sensor to drive the relative positions between the multiple color filter layers to change at least once;

The second acquisition module is configured to acquire the second image collected by the photosensitive layer after changing the relative position between the plurality of color filter layers each time, to obtain at least one second image;

A generating module, configured to generate a target image based on the first image and the at least one second image.

In a fourth aspect, an embodiment of the present invention provides a camera module, which includes the image sensor in the first aspect above.

In a fifth aspect, an embodiment of the present invention further provides an electronic device, including the above-mentioned camera module in the fourth aspect.

In a sixth aspect, an embodiment of the present invention provides another image sensor, which includes:

a photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer;

The target color filter layer is arranged on the light-receiving surface of the photosensitive layer;

The driving unit is used to drive the relative position between the target color filter layer and the photosensitive layer to change.

In a seventh aspect, an embodiment of the present invention provides another image generation method, which is applied to an electronic device, where the electronic device includes the image sensor of the sixth aspect, and the method includes:

acquiring a first image collected by the photosensitive layer of the image sensor;

controlling the drive unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once;

After changing the relative position between the target color filter layer and the photosensitive layer each time, acquiring a second image collected by the photosensitive layer to obtain at least one second image;

A target image is generated based on the first image and the at least one second image.

In an eighth aspect, an embodiment of the present invention provides another image generation device, which includes:

a first acquisition module, configured to acquire a first image collected by the photosensitive layer of the image sensor;

A control module, configured to control the drive unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once;

A second acquiring module, configured to acquire a second image collected by the photosensitive layer after changing the relative position between the target color filter layer and the photosensitive layer each time, to obtain at least one second image;

A generating module, configured to generate a target image based on the first image and the at least one second image.

In a ninth aspect, an embodiment of the present invention provides another camera module, which includes the image sensor of the sixth aspect above.

In a tenth aspect, an embodiment of the present invention further provides another electronic device, including the above-mentioned camera module in the ninth aspect.

In the eleventh aspect, the embodiment of the present invention also provides a computer-readable storage medium, which is characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program as provided in the present invention can be realized. The steps of the image generation method.

In the embodiment of the present invention, the image sensor may include: a photosensitive layer, a plurality of color filter layers, and a driving unit connected to the color filter layer; a plurality of color filter layers are stacked on the light-receiving surface of the photosensitive layer; Next, change the relative positional relationship between different color filter layers, or change the relative positional relationship between the color filter layer and the photosensitive layer. The image sensor can work at different positions of the color filter layer, the color filter array of the image sensor can be changed, and the electronic device can capture multiple images based on different color filter arrays through the image sensor, and then perform image synthesis. The color array can be mechanically switched to increase the sensitivity, so that one image sensor can take into account the advantages of different color filter arrays. In addition, because the composited image is taken by the same image sensor, the process of pixel alignment is avoided, and the accuracy of image synthesis is improved. sex and efficiency.

Description of drawings

FIG. 1 is one of the structural diagrams of an image sensor provided by an embodiment of the present invention;

Fig. 2 is a top view of a color filter layer provided by an embodiment of the present invention;

Fig. 3 is one of the schematic diagrams of the color filter layer provided by the embodiment of the present invention;

Fig. 4 is one of the schematic diagrams of the moving direction of the color filter layer provided by the embodiment of the present invention;

Fig. 5 is a top view of a first color filter array provided by an embodiment of the present invention;

Fig. 6 is the second schematic diagram of the moving direction of the color filter layer provided by the embodiment of the present invention;

Fig. 7 is a top view of a second color filter array provided by an embodiment of the present invention;

Fig. 8 is a structural diagram of an electrostatic comb driver provided by an embodiment of the present invention;

FIG. 9 is one of the flow charts of the steps of the image generation method provided by the embodiment of the present invention;

Fig. 10 is a flow chart of image fusion based on multi-scale decomposition provided by an embodiment of the present invention;

Fig. 11 is the second structure diagram of the image sensor provided by the embodiment of the present invention;

Fig. 12 is the second schematic diagram of the color filter layer provided by the embodiment of the present invention;

Fig. 13 is a schematic diagram of a first position relationship provided by an embodiment of the present invention;

Fig. 14 is one of the second schematic diagrams of positional relationship provided by the embodiment of the present invention;

Fig. 15 is the second schematic diagram of the second position relationship provided by the embodiment of the present invention;

Fig. 16 is the second flowchart of the steps of the image generation method provided by the embodiment of the present invention;

Fig. 17 is one of the block diagrams of the image generation device provided by the embodiment of the present invention;

Fig. 18 is the second block diagram of the image generation device provided by the embodiment of the present invention;

FIG. 19 is a schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present invention.

Explanation of reference signs:

10. Photosensitive layer; 11. Photosensitive pixels; 20. Multiple color filter layers; 21. First color filter layer; 22. Second color filter layer; 23. Third color filter layer; 24. Target color filter layer; 30 , drive unit; 31, first drive unit; 32, second drive unit; 33, third drive unit; 34, power supply; 35, fixed frame; 36, first capacitor plate; 37, first comb teeth; 38 , the second capacitive plate; 39, the second comb; 40, the light-gathering layer; 50, the wiring layer; 61, the red color filter pixel; 62, the green color filter pixel; 63, the blue color filter pixel; 64, fully transparent Color filter pixel; 70, color filter pixel array; 81, first color filter group; 82, second color filter group; 83, third color filter group; 84, target color filter group.

Detailed ways

Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

FIG. 1 is one of the structural diagrams of an image sensor provided by an embodiment of the present invention. As shown in FIG. 1 , the image sensor may include: a photosensitive layer, multiple color filter layers, and a driving unit connected to the multiple color filter layers.

The photosensitive layer is also commonly referred to as a photosensor, which can sense the intensity of light and output a corresponding electrical signal according to the intensity of light. In the embodiment of the present invention, the form of the photosensitive layer included in the image sensor is not limited. For example, the photosensitive layer can be It may be a charge coupled device (charge coupled device, CCD), or it may be a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS), or it may be other forms. Wherein, one side of the photosensitive layer is a light-receiving surface, and the light-receiving surface is used for receiving light signals.

Since the photosensitive layer can only sense the intensity of light and cannot distinguish the color of light (that is, the wavelength of light), the image sensor also needs to set a color filter layer on the light-receiving side (light-receiving surface) of the photosensitive layer, and the color filter pixels of the color filter layer Corresponding to the photosensitive pixels of the photosensitive layer, the color filter pixels only allow light of a specific color (wavelength) to pass through, so that the photosensitive pixels of the photosensitive layer receive light of a specific color. Through the correspondence between the color filter pixels and photosensitive pixels, the image sensor is The color information of the light can be obtained, so that a color image can be taken.

In the embodiment of the present invention, as shown in FIG. 1 , multiple color filter layers are superimposed on the light-receiving surface of the photosensitive layer. In order to ensure a good color filtering effect, the multiple color filter layers 20 can completely cover the photosensitive layer 10, and can be combined with The photosensitive layers 10 are arranged parallel to each other. As shown in FIG. 1 , it contains 4 layers of multiple color filter layers 20, all multiple color filter layers 20 can be stacked and arranged in parallel, all multiple color filter layers 20 can completely cover the photosensitive layer, and multiple Each color filter layer 20 can be arranged parallel to the photosensitive layer. In addition, the plurality of color filter layers 20 may also be covered with a light-gathering layer 40. The light-gathering layer 40 is composed of microlenses, and each microlens covers a color filter pixel of the plurality of color filter layers 20. The lens can gather the light so that the light passes through each color filter pixel of the plurality of color filter layers 20 in a more concentrated manner, so as to improve the utilization rate of the light. The backlight surface of the photosensitive layer 10 can also be connected to the wiring layer 50 , and the wiring layer 50 is arranged with signal lines connecting the photosensitive layer 10 and the processor.

The driving unit is used for driving the relative positions among the multiple color filter layers to change. The driving unit is connected with multiple color filter layers, and the relative positional relationship between different color filter layers can be changed under the driving of the driving unit, or the relative positional relationship between multiple color filter layers and the photosensitive layer can be changed. It should be noted that one color filter layer can be connected with one or more corresponding drive units, so that the color filter layer can be independently controlled by the corresponding one or more drive units, and multiple color filter layers can also be connected with one or more drive units at the same time. Corresponding one or more driving units, so as to simultaneously control multiple color filter layers through one or more driving units. It should be noted that one color filter layer may not be provided with a driving unit, and the other color filter layers provided with a driving unit may move relative to the color filter layer without a driving unit under the drive of the driving unit, thereby also being able to The effect of changing the relative positional relationship between the multiple color filter layers and the photosensitive layer is realized.

Referring to FIG. 2, FIG. 2 shows a top view of a color filter layer provided by an embodiment of the present invention. As shown in FIG. 2, the drive unit is connected to the sides of multiple color filter layers, and the drive unit is used to drive multiple color filter layers Layers move in a direction perpendicular to the sides. The driving unit includes a first driving unit connected to the first color filter layer, a second driving unit connected to the second color filter layer, and a third driving unit connected to the third color filter layer. Under the driving of the first driving unit 31, the first color filter layer 21 can move in parallel in the first direction a relative to the second color filter layer 22 and the third color filter layer 23, and under the driving of the second driving unit 32 , the second color filter layer 22 can move parallel to the first color filter layer 21 and the third color filter layer 23 in the first direction a, driven by the third driving unit 33, the third color filter layer 23 can move relatively The first color filter layer 21 and the second color filter layer 22 move in parallel in the second direction b. Wherein, the first direction a and the second direction b are perpendicular to each other. It should be noted that the moving directions of the above-mentioned color filter layers are only exemplary descriptions, and each color filter layer can be set to move in any direction of the plane where it is located according to actual needs, and the distance of each movement is an integer of a color filter pixel distance times, where one color filter pixel distance represents the distance between the center points of two adjacent color filter pixels. In addition, the above-mentioned moving direction and moving distance can also be flexibly adjusted by technicians according to actual needs. This embodiment of the present invention Not specifically limited.

Each color filter layer is composed of several color filter pixels, and these color filter pixels can form a color filter array, which can also be called a color filter array (Color Filter Array, CFA). After the light passes through the color filter array, it can be A color light array composed of light of different colors is projected on the rear photosensitive layer, and the photosensitive layer can output a color image after being sensitive to the color light array.

In the embodiment of the present invention, by adjusting the relative positional relationship between a plurality of different color filter layers, the color filter arrays of different color filter layers can be superimposed in different ways, thereby projecting different patterns of colored light on the photosensitive layer. Arrays can also project color light arrays at different positions on the photosensitive layer by adjusting the relative positional relationship between a single color filter layer and the photosensitive layer. Therefore, by adjusting the position of the color filter layer at the shooting interval, the photosensitive When the same photosensitive pixel on the photosensitive layer captures the same scene, it obtains a variety of different colors of light, which enriches the light information received by the photosensitive layer, and then can obtain an image with richer light information through multiple composite methods, improving the Shooting effect.

Optionally, in an implementation manner of the embodiment of the present invention, the driving unit drives a plurality of color filter layers to switch between the first state and the second state, and in the first state, there are In the first relative positional relationship, multiple color filter layers are stacked on top of each other to form a first color filter array; in the second state, multiple color filter layers have a second relative positional relationship, and multiple color filter layers are stacked on top of each other to form a first color filter array. Second color filter array.

The plurality of color filter layers may have a first relative positional relationship and a second relative positional relationship, wherein, under the first relative positional relationship, the plurality of color filter layers in the image sensor are all at their corresponding first positions, That is, every two color filter layers are in the first relative positional relationship, and in the second relative positional relationship, the plurality of color filter layers in the image sensor are in their corresponding second positions, that is, every two color filter layers The layers are all in the second relative positional relationship. It should be noted that each color filter layer corresponds to its own first position and second position, and the first position and second position corresponding to each color filter layer may be different positions or the same position. The first driving control operation may be an operation of adjusting a plurality of color filter layers to their corresponding first positions, so that every two color filter layers are in a first relative positional relationship, so that the plurality of color filter layers The relative positional relationship is adjusted to the first relative positional relationship, and the second drive control operation may be the operation of adjusting a plurality of color filter layers to their corresponding second positions, so that every two color filter layers are at the first position. A relative positional relationship, for adjusting the relative positional relationship of the plurality of color filter layers to a second relative positional relationship.

In the embodiment of the present invention, different relative positional relationships may correspond to different stacked color filter arrays. When multiple color filter layers are in the first relative positional relationship, each color filter layer is stacked with each other to form the first color filter array; when multiple color filter layers are in the second relative positional relationship, each filter layer The color layers are stacked on each other to form a second color filter array; wherein, the light transmittance of the second color filter array is greater than the light transmittance of the first color filter array. It should be noted that the light transmittance of the color filter array refers to the ratio of the light intensity before passing through the color filter array to the light intensity after passing through the color filter array, and a color filter array with a smaller light transmittance can generally provide a higher light intensity for the photosensitive layer. For accurate color information, a color filter array with a higher light transmittance can generally provide more accurate brightness information for the photosensitive layer. In the embodiment of the present invention, multiple color filter layers can be used to form the Under the circumstances, the first image with accurate color is captured, and the second image with sufficient and accurate brightness is captured when multiple color filter layers form the second color filter array, and then the color and brightness are synthesized through the first image and the second image Accurate target images to achieve the effect of improving the shooting quality.

Optionally, the color filter layer is composed of color filter units, and the plurality of color filter layers include a first color filter layer, a second color filter layer and a third color filter layer. Referring to FIG. 3, FIG. 3 shows one of the schematic diagrams of the color filter layer provided by the embodiment of the present invention. As shown in FIG. Three color filter layers 23 .

In the embodiment of the present invention, each color filter layer may only contain two kinds of color filter pixels, wherein one color filter pixel has a specific color filter pixel, which only allows light of a specific color to pass through, such as red, filter color, Blue, etc. Another color filter pixel is a transparent color filter pixel, which allows light of all colors to pass through. It should be noted that the fully transparent color filter pixel can be realized by not filling any material, or by filling with a transparent material. In the case of filling with a transparent material, the transparent material can allow light of all wavelengths to pass through, or it can only Allow light in the visible light band to pass through to eliminate the influence of infrared light, ultraviolet light and other invisible light on imaging.

As shown in FIG. 3 , the first color filter unit of the first color filter layer 21 includes red color filter pixels 61 and fully transparent color filter pixels 64 , and the second color filter unit of the second color filter layer 22 includes green color filter pixels 62 and the fully transparent filter pixels 64 , the third color filter unit of the third color filter layer 23 includes blue color filter pixels 63 and fully transparent color filter pixels 64 . Specifically, the first color filter layer includes a plurality of 4×4 first color filter units, the first color filter unit includes four 2×2 first color filter groups 81, and the first color filter group includes one red color filter pixel and 3 fully transparent color filter pixels; the second color filter layer includes a plurality of 4×4 second color filter units, and the second color filter unit includes 4 2×2 second color filter groups 82, and the second color filter group includes 2 red color filter pixels arranged diagonally and 2 transparent color filter pixels arranged diagonally; the third color filter layer includes multiple 4×4 third color filter units, and the third color filter unit includes four 2× 2. The third color filter group 83, the first color filter group includes 1 blue filter pixel and 3 transparent filter pixels. Wherein, vertical lines in FIG. 3 represent red color filter pixels, oblique lines represent green color filter pixels, and horizontal lines represent blue color filter pixels.

Since the human eye is naturally sensitive to green, the number of red filter pixels and blue color filter pixels can be set to be equal, and the number of green color filter pixels can be set to be equal to the number of red color filter pixels and blue color filter pixels. and. In order to obtain better green light, it is more suitable for viewing habits of human eyes. Alternatively, set the number of red dropout pixels equal to the sum of the number of green dropout pixels plus the number of blue dropout pixels, or set the number of blue dropout pixels equal to the number of red dropout pixels plus the number of green dropout pixels sum of the quantities. For example, as shown in Figure 3, in the embodiment of the present invention, the color filter unit of the first color filter layer may include 4 red color filter pixels and 12 fully transparent color filter pixels, and the color filter unit of the second color filter layer It may include 8 green color filtering pixels and 8 transparent color filtering pixels, and the color filtering unit of the third color filter layer may include 4 blue color filtering pixels and 12 transparent color filtering pixels.

It should be noted that the color filter unit of the color filter layer shown in Figure 3 is the smallest color filter unit of the color filter layer, and the complete color filter pixel array of each color filter layer can be expanded by the color filter unit shown in Figure 3 For example, the color filter unit in Figure 3 is a 4X4 array with 16 color filter pixels in total, but in the actual production process, the complete color filter pixel array of each color filter layer can be composed of 120,000 4X4 color filter units The spliced 1600X1200 array has a total of 1.92 million color filter pixels.

Referring to FIG. 4, FIG. 4 shows one of the schematic diagrams of the moving direction of the color filter layer provided by the embodiment of the present invention. As shown in FIG. 4, the first color filter layer 21, the second color filter layer 22 and the third color filter layer 23 superimposed on each other.

As shown in Figure 4, when the first color filter layer, the second color filter layer and the third color filter layer overlap each other and their edges are aligned, the first color filter layer can be moved to the direction a by one color filter through the driving unit pixel distance, and then move the third color filter layer to the direction b by a distance of one color filter pixel through the driving unit, so that the first color filter layer, the second color filter layer and the third color filter layer are superimposed on each other to form the first filter layer color array; it is also possible to move the first color filter layer to the opposite direction of direction b by a distance of one color filter pixel, and then move to direction a by a distance of one color filter pixel, and then use the drive unit to move the second color filter layer The distance of one color filter pixel is moved to the opposite direction of the direction b, so that the first color filter layer, the second color filter layer and the third color filter layer are stacked to form the first color filter array; the second color filter array can also be driven by the drive unit The second color filter layer moves the distance of one color filter pixel to the opposite direction of direction a, and then moves the third color filter layer to the opposite direction of direction a by the distance of one color filter pixel through the driving unit, and then moves one color filter layer to direction b The distance between the pixels is such that the first color filter layer, the second color filter layer and the third color filter layer are superimposed on each other to form the first color filter array. It should be noted that the above-mentioned method of moving each color filter layer to form the first color filter array is described as an example, and technicians can also use other moving methods to make the first color filter layer, the second color filter layer and the third color filter layer The first color filter array is formed by stacking each other, which is not specifically limited in this embodiment of the present invention.

Referring to FIG. 5, FIG. 5 shows a top view of a first color filter array provided by an embodiment of the present invention. As shown in FIG. 5, the first color filter layer 21, the second color filter layer 22 and the third color filter layer 23 overlap each other and form the first color filter array, the red color filter pixels of the first color filter layer 21 and half of the transparent color filter pixels of the second color filter layer 22 are aligned, and the other half of the second color filter layer 22 The transparent color filter pixels are aligned with the blue color filter pixels of the third color filter layer 23 , so that the three color filter layers can be stacked to form a Bayer array (such as an RGGB array), so that the sensor can obtain more accurate color information.

Referring to FIG. 6, FIG. 6 shows the second schematic diagram of the moving direction of the color filter layer provided by the embodiment of the present invention. As shown in FIG. 6, in the case where multiple color filter layers are stacked to form the first color filter array, The first color filter layer is moved to the direction b by a distance of one color filter pixel through the driving unit, and then the third color filter layer is moved to the opposite direction of direction b by a distance of one color filter pixel through the driving unit, so that the first color filter layer , the second color filter layer and the third color filter layer overlap each other to form a second color filter array; it is also possible to move the first color filter layer to the direction b by a distance of 2 color filter pixels through the driving unit, and then use the driving unit to move The second color filter layer moves to the direction b by a distance of one color filter pixel, so that the first color filter layer, the second color filter layer and the third color filter layer are overlapped to form a second color filter array; The second color filter layer moves to the opposite direction of direction b by a distance of one color filter pixel, and then the third color filter layer is moved by the driving unit to the opposite direction of direction b by a distance of two color filter pixels, so that the first color filter layer , the second color filter layer and the third color filter layer are stacked to form a second color filter array. It should be noted that the above-mentioned method of moving each color filter layer to form the second color filter array is described as an example, and technicians can also use other moving methods to make the first color filter layer, the second color filter layer and the third color filter layer The second color filter array is formed by stacking each other, which is not specifically limited in the embodiment of the present invention.

Referring to FIG. 7, FIG. 7 shows a top view of a second color filter array provided by an embodiment of the present invention. As shown in FIG. 7, the first color filter layer 21, the second color filter layer 22 and the third color filter layer 23 overlap each other and form the second color filter array, the red filter pixels of the first color filter layer 21 and half of the green color filter pixels of the second color filter layer 22 are aligned, and the blue filter pixels of the third color filter layer 23 The color pixels are aligned with the other half of the green color filter pixels of the second color filter layer 22, and the fully transparent color filter pixels of different color filter layers are aligned with each other, so that half of the second color filter array formed by stacking the three color filter layers The pixel is a fully transparent color filter pixel, which improves the light transmittance compared with the first color filter array, so that the sensor can obtain more accurate brightness information.

In the embodiment of the present invention, if the light transmittance of each green pixel of red, green, and blue is 40%, and the light transmittance of the fully transparent filter pixel is 100%, then by calculation, each 4X4 first The light transmittance of the color filter array is 40%, and the light transmittance of each 4X4 second color filter array is 40%*40%*0.5+100%*0.5=58%, compared with the conventional first color filter The array has a light transmittance improvement of nearly 50%. Therefore, by adopting the multiple color filter layer structures and moving methods proposed by the present invention, multiple color filter layers can be moved to form a second color filter array with high light projection rate, and when taking pictures, the photosensitive layer of the image sensor can be More light is received, improving the effect of taking pictures.

Optionally, generally speaking, the size of the image sensor is relatively small. In order to arrange the driving unit in a small space, the driving unit may include a micro-electro-mechanical system (Micro Electro Mechanical Systems, MEMS), such as an electrostatic comb driver, a micro-electro-mechanical system, etc. The system is a micron-scale mechanical drive system.

Referring to FIG. 8, FIG. 8 shows a structural diagram of an electrostatic comb driver provided by an embodiment of the present invention. As shown in FIG. 8, the electrostatic comb driver may include a power supply 34, a fixed frame 35, a first comb portion and the second comb-tooth portion, wherein the first comb-tooth portion is connected to the fixed frame 35, the second comb-tooth portion is connected to one of the multiple color filter layers, and the power supply 34 is connected to the first comb-tooth portion and the second comb-tooth portion. The two comb teeth are connected, and driven by the power supply 34, the first comb teeth and the second comb teeth move relative to each other, so as to change the relative positional relationship between different color filter layers. The first comb tooth part is made of the first capacitor plate 36 and a plurality of first comb teeth 37 fixed on the first capacitor plate 36, and the second comb tooth part is made of the second capacitor plate 38 and fixed on the second capacitor plate 38. Composed of a plurality of second comb teeth 39, the first capacitive plate 36 and the second capacitive plate 38 are charged with the same polarity charge or different polarity charges under the action of the power supply 34, so that the first capacitive plate 36 and the second capacitive plate 36 A repulsive force or an attractive force is generated between the capacitive plates 38 to change the distance between the first capacitive plate 36 and the second capacitive plate 38 . The first comb teeth 37 and the second comb teeth 39 fit each other, serve as slide rails and play a position-limiting role.

Further, when the first capacitor plate 36 and the second capacitor plate 38 are powered by a pair of positive and negative poles, only positive charges can be charged in one capacitor plate, and negative charges can be charged in the other capacitor plate, so that both Only electrostatic attraction can be produced between the capacitive plates, therefore, an elastic device can also be set between the first capacitive plate 36 and the second capacitive plate 38, and the two capacitive plates are separated from each other by the elastic device between the two capacitive plates. In this way, when the electrostatic attraction between the two capacitor plates is greater than the elastic force, the two capacitor plates are close to each other, and when the electrostatic attraction between the two capacitor plates is smaller than the elastic force, the two capacitor plates are far away from each other. In this way, the electrostatic comb driver can be conveniently driven through a relatively simple control circuit, which reduces research and development and manufacturing costs.

In summary, an image sensor provided by an embodiment of the present invention includes: a photosensitive layer, a plurality of color filter layers, and a drive unit connected to the plurality of color filter layers; the plurality of color filter layers are stacked on the light-receiving surface of the photosensitive layer ; The driving unit is used to drive the relative positions between the multiple color filter layers to change. The image sensor can work at different positions of the color filter layer, the color filter array of the image sensor can be changed, and the electronic device can capture multiple images based on different color filter arrays through the image sensor, and then perform image synthesis. The color array can be mechanically switched to increase the sensitivity, so that one image sensor can take into account the advantages of different color filter arrays. In addition, because the composited image is taken by the same image sensor, the process of pixel alignment is avoided, and the accuracy of image synthesis is improved. sex and efficiency.

FIG. 9 is one of the flow charts of the steps of the image generation method provided by the embodiment of the present invention. The method is applied to an electronic device, and the electronic device includes the above-mentioned image sensor. As shown in FIG. 9, the method may include:

Step 101, acquiring a first image collected by a photosensitive layer of the image sensor.

In the embodiment of the present invention, there may be a first relative positional relationship and a second relative positional relationship among the plurality of color filter layers, wherein, under the first relative positional relationship, the plurality of color filter layers in the image sensor are all in their respective positions. At the corresponding first position, under the second relative positional relationship, the multiple color filter layers in the image sensor are all at their corresponding second positions. It should be noted that each color filter layer corresponds to its own first position and second position, and the first position and second position corresponding to each color filter layer may be different positions or the same position. Different relative positional relationships may correspond to different superimposed color filter arrays. In the case that the multiple color filter layers are in the first relative positional relationship, the multiple color filter layers are stacked on each other to form the first color filter array; in the case that the multiple color filter layers are in the second relative positional relationship, the multiple color filter layers The two color filter layers are stacked to form a second color filter array; wherein, the light transmittance of the second color filter array is greater than the light transmittance of the first color filter array.

When taking a picture through the image sensor, the positions of the multiple color filter layers can be adjusted by the driving unit first, so that the multiple color filter layers are all in the corresponding first positions, so that the multiple color filter layers are in the first relative positional relationship, The light-receiving surfaces of the photosensitive layer are overlapped with each other to form a first color filter array. After the first color filter array is formed, the first image is collected through the photosensitive layer. It is also possible to adjust the position of each color filter layer in the plurality of color filter layers through the driving unit first, so that the plurality of color filter layers are all in the corresponding second position, so that the plurality of color filter layers are in the second relative positional relationship, The light-receiving surfaces of the photosensitive layer are overlapped with each other to form a second color filter array. After forming the second color filter array, the first image is collected through the photosensitive layer.

It should be noted that the multiple color filter layers may also have default positions. In the case of not controlling the driving unit, the multiple color filter layers are kept at the default positions, wherein the default positions corresponding to the multiple color filter layers may be multiple The first position corresponding to one color filter layer can also be the second position corresponding to multiple color filter layers, therefore, when taking a photo through the image sensor, the first image can be collected directly through the photosensitive layer without adjustment by the drive unit The location of multiple color filter layers improves the speed of taking photos.

Step 102, controlling the driving unit of the image sensor to drive the relative positions of the plurality of color filter layers to change at least once.

In the embodiment of the present invention, the plurality of color filter layers include a first color filter layer composed of red color filter pixels and fully transparent color filter pixels, a second filter layer composed of green color filter pixels and fully transparent color filter pixels color layer, and a third color filter layer composed of blue color filter pixels and transparent color filter pixels. In the case where the color filter pixel arrays of the plurality of color filter layers overlap each other to form the first color filter array, all the red color filter pixels of the first color filter layer and half of the second color filter layer are all The transparent color filter pixels overlap with each other, and the other half of the transparent color filter pixels of the second color filter layer overlap with all the blue color filter pixels of the third color filter layer.

In the case where the color filter pixel arrays of the plurality of color filter layers overlap each other to form the first color filter array, the driving unit of the image sensor is controlled to drive the plurality of color filter layers to move, so that the plurality of color filter layers The relative positions between the color filter layers are changed at least once, and the color filter pixel arrays of the plurality of color filter layers are overlapped to form at least one second color filter array; in the first color filter array, the All the red color filter pixels of the first color filter layer overlap with half of the fully transparent color filter pixels of the second color filter layer, and the other half of the fully transparent color filter pixels of the second color filter layer overlap with the third color filter pixels All the blue color filter pixels of the color layer overlap each other; wherein, the light transmittance of the second color filter array is greater than the light transmittance of the first color filter array.

The drive unit may correspond to a first drive control operation and a second drive control operation, wherein the first drive control operation may be an operation of adjusting a plurality of color filter layers to their corresponding first positions, so that the plurality of filter layers The relative positional relationship of the color layers is adjusted to the first relative positional relationship, and the second drive control operation may be an operation of adjusting a plurality of color filter layers to their corresponding second positions, so that the relative position of the plurality of color filter layers The relationship is adjusted to the second relative positional relationship.

Specifically, if the first image is taken under the condition that a plurality of color filter layers form the first color filter array, after obtaining the first image, each color filter layer remains in the first color filter array, therefore, it can be used in different When the color filter pixel arrays of the color filter layers are overlapped to form the first color filter array, the driving unit drives the multiple color filter layers to move, thereby changing the relative positional relationship between different color filter layers, so that different color filter layers The color filter pixel arrays are overlapped to form a second color filter array.

If the first image is taken under the condition that a plurality of color filter layers form the second color filter array, after obtaining the first image, each color filter layer remains in the second color filter array, therefore, different color filter layers can be When the color filter pixel arrays are overlapped to form the second color filter array, the driving unit drives multiple color filter layers to move, thereby changing the relative positional relationship between different color filter layers, so that the color filter layers of different color filter layers The pixel arrays are overlapped to form a first color filter array; wherein, the light transmittance of the second color filter array is greater than that of the first color filter array.

Optionally, the method for controlling the drive unit of the image sensor to drive the plurality of color filter layers to move may include: controlling the drive unit to drive the first color filter layer, the second color filter layer and the The third color filter layer moves so that all the red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap each other, and all the blue color filter pixels of the second color filter layer The color filter pixels are aligned with the other half of the green color filter pixels of the second color filter layer, and the transparent filter pixels of the plurality of color filter layers overlap each other. That is, in the embodiment of the present invention, in the case where the color filter pixel arrays of the plurality of color filter layers are stacked with each other to form a second color filter array, all the red color filter pixels of the first color filter layer and all the red color filter pixels Half of the green color filter pixels of the second color filter layer overlap each other, all the blue color filter pixels of the second color filter layer are aligned with the other half of the green color filter pixels of the second color filter layer, and the multiple The transparent filter pixels of each color filter layer overlap with each other.

In the embodiment of the present invention, by adjusting the relative positional relationship between multiple color filter layers, the same image sensor can be sensitive to light under different color filter arrays, so that different color information and illumination can be obtained for the same shooting scene. Intensive images effectively improve the shooting capability of single-camera electronic equipment.

Step 103 , after changing the relative positions of the plurality of color filter layers each time, acquiring a second image collected by the photosensitive layer to obtain at least one second image.

If the first image is taken when a plurality of color filter layers form the first color filter array, after the plurality of color filter layers change their relative positional relationship once, a type of covering photosensitive layer is formed between the plurality of color filter layers. The second color filter array can collect a second image through the photosensitive layer at this time. It should be noted that if a plurality of color filter layers changes the relative positional relationship multiple times, after changing the relative positional relationship each time, a second color filter array covering the photosensitive layer is formed between the plurality of color filter layers, and the acquisition to multiple second images.

Step 104: Generate a target image based on the first image and the at least one second image.

In the case where at least one second image contains only one second image, the target image can be generated directly according to the first image and the second image, and in the case where at least one second image contains multiple second images, it can be The first image is synthesized with one of the second images to obtain an intermediate image, and then the intermediate image is synthesized with the remaining second image to generate a new intermediate image, and so on until all the second images participate in the synthesis process, thus Get the target image.

In the case where at least one second image contains only one second image, the process of generating the target image based on the first image and the second image is as follows:

In the embodiment of the present invention, since the second color filter array may contain half of the fully transparent filter pixels, the types and light transmittances of the color filter pixels of the first color filter array and the second color filter array are different, and the light transmittance A color filter array with a smaller transmittance can generally provide more accurate color information for the photosensitive layer, and a color filter array with a larger light transmittance can usually provide more accurate brightness information for the photosensitive layer. Therefore, the first image and the second image Among the two images, the color of one image is relatively accurate, and the brightness of the other image is relatively accurate. Therefore, a target image with accurate color and brightness can be synthesized through the first image and the second image, so as to achieve the effect of improving the shooting quality.

It should be noted that, since there are multiple color filter layers in multiple color filter layers, in the color filter array formed by overlapping each color filter layer, there may be merged color filter pixels, such as the blue color filter pixel and the The green color filter pixels are superimposed to form blue-green color filter pixels, and the red and green color filter pixels are superimposed to form red-green color filter pixels. Since the light transmittance of the fusion color filter pixel is lower than that of a single-layer color filter pixel, therefore, for the fusion color filter pixel in the first image or the second image, the fusion filter can be adjusted by the fusion pixel gain. The signal strength of the color pixel is corrected, and then participates in the subsequent image fusion operation process. Among them, the fused pixel gain can be determined by technical personnel through calibration during the research and development stage.

For example, the image sensor can be placed in a flat-field correction (flatfield) light source environment, and an image can be captured in the state of the first color filter array, and the first signal intensity corresponding to the light-sensitive pixel covering the green color filter pixel can be obtained. Take an image in the state of the second color filter array, obtain the second signal strength corresponding to the photosensitive pixel when covering the blue-green fusion filter pixel, and the corresponding third signal strength when covering the red-green fusion pixel, and combine the second signal strength with the The ratio of the first signal intensity is used as the fusion pixel gain of the blue-green fusion filter pixel, and the ratio of the third signal intensity to the first signal intensity is used as the fusion pixel gain of the red-green fusion filter pixel.

In the embodiment of the present invention, the fusion of the first image and the second image may be synthesizing the first image into a new target image, or using one of the images to enhance the other image to obtain the target image.

Specifically, the first image and the second image may be fused based on a multi-scale decomposition method. Referring to FIG. 10, FIG. 10 shows a flow chart of image fusion based on multi-scale decomposition provided by an embodiment of the present invention. As shown in FIG. 10, firstly, the first image 141 and the second image 142 are subjected to multi-scale decomposition to obtain The multi-scale representation 143 of the first image and the multi-scale representation 144 of the second image, and then according to a specific fusion rule, the multi-scale representation 143 of the first image and the multi-scale representation 144 of the second image are fused to obtain the fusion image Multi-scale representation 145 , and multi-scale inverse transformation is performed on the multi-scale representation 145 of the fused image to obtain a target image 146 . Image fusion based on the multi-scale decomposition method can better retain the detail information in the first image and the second image, so that the fused target image can better retain the detail information of the original image, and this method can better retain the original image. The brightness characteristics of the image make the fused target image achieve a better low-light shooting effect.

In the embodiment of this application, the multi-scale decomposition method may include Laplace pyramid transform, contrast pyramid transform, dual-tree complex wavelet transform, curvelet transform, non-subsampled contourlet transform, non-subsampled shearlet transform, etc. . Among them, the Laplace pyramid transform can include four steps of low-pass filtering, down-sampling, interpolation and band-pass filtering on the image; the wavelet transform can decompose the image at two mutually perpendicular Extract the high-frequency edge details of the image in three directions while extracting the high-frequency information; the curvelet transform can well represent the anisotropic edge features in the image, and the combination of the ridgelet transform can well describe the linear features of the image and wavelet Transformation is suitable for expressing the advantages of point features, and the edge information of the image can accurately and precisely represent the curve singularity features of the image with fewer non-zero coefficients; the non-subsampling contourlet transform is a proposed method based on the contourlet transform Improve the transformation method. Compared with the contourlet transformation, the non-subsampling contourlet transformation does not undergo a downsampling operation in the process of image decomposition and reconstruction, so the translation invariance of the transformation is guaranteed, and the pseudo-Gibbs phenomenon is solved; Downsampled shearlet transform is improved on the basis of shearlet transform. Compared with shearlet transform, non-downsampled shearlet transform does not undergo downsampling operation in the process of image decomposition and reconstruction, so non-downsampled shearlet transform The sampled shearlet transform is translation invariant. It should be noted that when the multi-scale decomposition method is used to fuse the first image and the second image, technicians can flexibly adjust the specific transformation method of the multi-scale decomposition according to actual needs, which is not specifically limited in this embodiment of the present application.

In addition, since the first image and the second image are images collected by the same sensor, when the electronic device remains still, the viewing angles of the first image and the second image are completely the same. In the synthesis, there is no need to align the pixels of the first image and the second image one by one, which not only improves the speed of image synthesis, but also avoids the occurrence of poor synthesis results due to alignment errors, and further improves the shooting quality of the target image.

To sum up, an image generation method provided by an embodiment of the present invention includes acquiring a first image collected by a photosensitive layer of an image sensor; controlling a drive unit of the image sensor to drive the relative positions between multiple color filter layers to change at least once ; After changing the relative positions of the plurality of color filter layers each time, acquiring a second image collected by the photosensitive layer to obtain at least one second image; generating a target image based on the first image and at least one second image. When the above scheme of the present invention is used to capture the first image and at least one second image and perform image synthesis, there is no need to align the pixels of the first image and the second image one by one, which not only improves the speed of image synthesis, but also avoids alignment errors. The resultant composite effect is poor, which further improves the shooting quality of the target image.

FIG. 11 is the second structural diagram of an image sensor provided by an embodiment of the present invention. As shown in FIG. 11 , the image sensor may include: a photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer.

The photosensitive layer is also commonly referred to as a photosensor, which can sense the intensity of light and output a corresponding electrical signal according to the intensity of light. In the embodiment of the present invention, the form of the photosensitive layer included in the image sensor is not limited. For example, the photosensitive layer can be It may be a charge coupled device (charge coupled device, CCD), or it may be a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS), or it may be other forms. Wherein, one side of the photosensitive layer is a light-receiving surface, and the light-receiving surface is used for receiving light signals.

Since the photosensitive layer can only sense the intensity of light and cannot distinguish the color of light (that is, the wavelength of light), the image sensor also needs to set a target color filter layer on the light-receiving side (light-receiving surface) of the photosensitive layer. The color pixels correspond to the photosensitive pixels of the photosensitive layer, and the color filter pixels only allow light of a specific color (wavelength) to pass through, so that the photosensitive pixels of the photosensitive layer receive light of a specific color. Through the corresponding relationship between the color filter pixels and photosensitive pixels, the image The sensor can obtain the color information of the light, so as to capture a color image.

In the embodiment of the present invention, as shown in FIG. 11 , the target color filter layer is arranged on the light-receiving surface of the photosensitive layer. In order to ensure a good color filtering effect, the target color filter layer 24 can completely cover the photosensitive layer 10 and be in contact with the photosensitive layer 10. arranged parallel to each other. As shown in FIG. 11 , the image sensor includes a target color filter layer 24 , a drive unit 30 connected to the target color filter layer 24 , and a photosensitive layer 10 covered by the target color filter layer 24 . In addition, the target color filter layer 24 can also be covered with a light-gathering layer 40, the light-gathering layer 40 is composed of microlenses, each microlens covers a color filter pixel of the target color filter layer 24, and the microlenses of the light-gathering layer 40 can be Converge the light, make the light more concentrated through the color filter pixels, and improve the utilization of light. The backlight surface of the photosensitive layer 10 can also be connected to the wiring layer 50 , and the wiring layer 50 is arranged with signal lines connecting the photosensitive layer 10 and the processor.

The driving unit is used for driving the relative position between the target color filter layer and the photosensitive layer to change. The driving unit is connected to the target color filter layer, and the relative positional relationship between the target color filter layer and the photosensitive layer is changed under the drive of the driving unit.

The target color filter layer includes a color filter pixel array, the color filter pixel array includes at least three kinds of color filter pixels, and the photosensitive layer includes a photosensitive pixel array. For example, when the number of types of color filter pixels is 3, the color filter pixels may include red color filter pixels, green color filter pixels and blue green color filter pixels.

The color filter pixels of the color filter pixel array overlap one-to-one with the photosensitive pixels of the photosensitive pixel array. The number of relative positional relationships between the target color filter layer and the photosensitive layer is equal to the number of color filter pixels in the color filter pixel array.

Referring to FIG. 12, FIG. 12 shows the second schematic diagram of the color filter layer provided by the embodiment of the present invention. As shown in FIG. 12, the color filter pixel array of the target color filter layer may include red, green, and blue color filter pixels, The target color filter layer includes a plurality of 4×4 color filter units; the color filter unit includes four 2×2 target color filter groups 84, and each target color filter group 84 includes three color filter pixels of red, green and blue. Wherein, vertical lines in FIG. 12 represent red color filter pixels, oblique lines represent green color filter pixels, and horizontal lines represent blue color filter pixels.

In the embodiment of the present invention, the color filter pixels of the color filter pixel array overlap with the photosensitive pixels of the photosensitive pixel array one-to-one; the number of relative positional relationships between the target color filter layer and the photosensitive layer is equal to The number of color pixels. For example, when the color filter pixels include red, green and blue pixels, the color filter pixel array has a first positional relationship, a second positional relationship and a third positional relationship with respect to the photosensitive layer. In the case of different positional relationships, the types of color filter pixels corresponding to each photosensitive pixel on the photosensitive layer are different.

Referring to FIG. 13, FIG. 13 shows a schematic diagram of a first positional relationship provided by an embodiment of the present invention. As shown in FIG. 13, for the target color filter layer 24 including red, green and blue color filter pixels, the target When the color filter pixel array 70 of the color filter layer 24 is in the first positional relationship with respect to the photosensitive layer 10 , the photosensitive pixels 11 of the photosensitive layer are covered by blue color filter pixels, which receive blue light information.

Referring to FIG. 14, FIG. 14 shows one of the second positional relationship diagrams provided by the embodiment of the present invention. As shown in FIG. 14, the target color filter layer 24 moves along the arrow e direction in FIG. The color filter pixel array 70 is in a second positional relationship with respect to the photosensitive layer. At this time, the photosensitive pixels 11 of the photosensitive layer are covered by red color filter pixels, which receive red light information.

Referring to FIG. 15, FIG. 15 shows the second schematic diagram of the second position relationship provided by the embodiment of the present invention. As shown in FIG. 15, the target color filter layer 24 continues to move along the arrow e direction in FIG. 14, so that the target color filter layer The color filter pixel array 70 at 24 is in another second positional relationship with respect to the photosensitive layer. At this time, the photosensitive pixels 11 of the photosensitive layer are covered by green color filter pixels, which receive green light information.

Optionally, when the number of the target color filter layer is equal to 1, the color filter pixel array of the target color filter layer is composed of an equal number of red color filter pixels, green color filter pixels and blue color filter pixels interlaced to form When the target color filter layer is translated horizontally or vertically relative to the photosensitive layer, the color of the color filter pixels corresponding to all the photosensitive pixels on the photosensitive layer changes once every time the distance of one color filter pixel is moved.

To sum up, an image sensor provided by an embodiment of the present invention includes: a photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer; the target color filter layer is arranged on the light-receiving surface of the photosensitive layer; The relative position between the driving target color filter layer and the photosensitive layer changes. The image sensor can work at different positions of the color filter layer, the color filter array of the image sensor can be changed, and the electronic device can capture multiple images based on different color filter arrays through the image sensor, and then perform image synthesis. The color array can be mechanically switched to increase the sensitivity, so that one image sensor can take into account the advantages of different color filter arrays. In addition, because the composited image is taken by the same image sensor, the process of pixel alignment is avoided, and the accuracy of image synthesis is improved. sex and efficiency.

FIG. 16 is the second flow chart of the steps of the image generation method provided by the embodiment of the present invention. The method is applied to an electronic device, and the electronic device includes the above-mentioned image sensor. As shown in FIG. 16 , the method may include:

Step 201, acquiring a first image collected by a photosensitive layer of the image sensor.

In the embodiment of the present invention, the target color filter layer covered on the photosensitive layer of the image sensor can move in parallel with the photosensitive layer to change the color of light received by each photosensitive pixel in the photosensitive layer. Driven by the driving unit, the target color filter layer can move in parallel with the photosensitive layer, so that there are multiple corresponding relationships between the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer. In the case of different corresponding relationships, The same photosensitive pixel on the photosensitive layer is covered by color filter pixels of different colors.

For example, when the color filter pixels of the target color filter layer are in the first corresponding relationship with the photosensitive pixels of the photosensitive layer, the photosensitive pixel A of the photosensitive layer is covered by the red color filter pixels of the target color filter layer, and in the second corresponding relationship In the case of the relationship, the photosensitive pixel A of the photosensitive layer is covered by the blue color filter pixel of the target color filter layer.

In the embodiment of the present invention, when taking a photo through the image sensor, firstly make the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer in the first corresponding relationship, and then collect the first image through the photosensitive layer.

Step 202, controlling the driving unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once.

After the first image is obtained, the target color filter layer can be controlled to move in parallel with respect to the photosensitive layer through the driving unit, specifically, when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have a first corresponding relationship Drive the target color filter layer to move at least once by the driving unit, thereby changing the relative positional relationship between the target color filter layer and the photosensitive layer at least once, so that the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have at least one Second Correspondence.

In the case that the filter pixels of the target color filter layer have a first relative positional relationship with the photosensitive pixels of the photosensitive layer, control the drive unit of the image sensor to drive the target color filter layer to move, so that the The relative position between the target color filter layer and the photosensitive layer is changed at least once, and the color filter pixels of the target color filter layer have at least one second corresponding relationship with the photosensitive pixels of the photosensitive layer; wherein, in the In the case that the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have the first relative positional relationship, the photosensitive pixels are aligned with one kind of color filter pixels; the filter pixels of the target color filter layer In the case where the color pixel has the second relative positional relationship with the photosensitive pixel of the photosensitive layer, the photosensitive pixel is aligned with another color filter pixel.

For example, if the color filter layer contains three kinds of color filter pixels of red, green and blue, the driving unit of the image sensor is controlled to drive the relative position between the target color filter layer and the photosensitive layer to change twice, so that Each pixel on the photosensitive layer can correspond to color filter pixels of different colors before and after changing the relative position each time.

Step 203 , after each change of the relative position between the target color filter layer and the photosensitive layer, acquire a second image collected by the photosensitive layer to obtain at least one second image.

For example, the target color filter layer is in the first corresponding relationship shown in FIG. 8 with respect to the photosensitive layer. At this time, the first image is acquired through the photosensitive layer, and then the photosensitive layer is moved by one color filter in the direction e shown in FIG. 9 through the driving unit. The pixel distance makes the target color filter layer in a second corresponding relationship with the photosensitive layer as shown in FIG. The direction e shown is moved by a distance of one color filter pixel, so that the target color filter layer is in another second corresponding relationship as shown in FIG. 10 with respect to the photosensitive layer. At this time, the second second image is acquired through the photosensitive layer.

Step 204: Generate a target image based on the first image and the at least one second image.

Generally speaking, in a shooting process, a photosensitive pixel can only receive light of a specific color, and each pixel in the captured image needs at least three color information of red, green and blue, so that the pixels in the image can Displays rich colors. Therefore, it is necessary to calculate the other two color information of the photosensitive pixel through a color restoration algorithm (such as demosaic, remosaic, etc.). Based on the color of light received by several photosensitive pixels, the intensity of the other two colors of the photosensitive pixel is estimated.

However, in the embodiment of the present invention, since each pixel of the first image and the second image corresponds one-to-one, and the colors of light received by the corresponding pixels are different, therefore, for one shot, through the first image and the second image , each photosensitive pixel actually obtains two color signals, and each photosensitive pixel only needs to restore another color signal to generate the final target image, which greatly improves the accuracy of the color restoration process and can significantly improve the final image. The image quality of the output target image.

Further, in the embodiment of the present invention, if the target color filter layer contains three kinds of color filter pixels of red, green, and blue, another first Two correspondences, so as to obtain two second images. For example, when the color filter pixels of the target color filter layer are in the first corresponding relationship with the photosensitive pixels of the photosensitive layer, the photosensitive pixel A of the photosensitive layer is covered by the red color filter pixels of the target color filter layer. In the case of two correspondences, the photosensitive pixel A of the photosensitive layer is covered by the blue color filter pixel of the target color filter layer; in the case of another second correspondence, the photosensitive pixel A of the photosensitive layer is covered by the blue color filter pixel of the target color filter layer Green color dropout pixel overlay.

Correspondingly, after a second image is captured in the case of a second corresponding relationship, the target color filter layer can be further driven to move by the driving unit, thereby changing the relative positional relationship between the target color filter layer and the photosensitive layer , making the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer in another second corresponding relationship, and taking another second image.

In the embodiment of the present invention, for one shot, through the first image and two second images, each photosensitive pixel actually obtains signals of three colors (for example, three colors of red, green, and blue), which can be directly obtained according to each The three color signals corresponding to each photosensitive pixel generate a colored target image pixel, which omits the color restoration process and does not need to estimate the color of the pixel, and can directly generate color pixels based on the real color information, which further improves the final output target image. image quality and effects.

To sum up, another image generation method provided by an embodiment of the present invention includes acquiring a first image collected by the photosensitive layer of the image sensor; controlling the driving unit of the image sensor to drive the target color filter layer and the The relative position between the photosensitive layers is changed at least once; after each change of the relative position between the target color filter layer and the photosensitive layer, the second image collected by the photosensitive layer is acquired to obtain at least one first Two images: generating a target image based on the first image and the at least one second image. When adopting the above-mentioned solution of the present invention to take multiple images and perform image synthesis, each photosensitive pixel can obtain multiple color signals, and color restoration can be performed according to the various color information obtained by the photosensitive pixels, or no color restoration is required at all. , which improves the color accuracy of the final output target image.

Fig. 17 is one of the block diagrams of the image generation device provided by the embodiment of the present invention. As shown in Fig. 17, the image generation device includes:

The first acquisition module 301 is configured to acquire a first image collected by the photosensitive layer of the image sensor;

A control module 302, configured to control the drive unit of the image sensor to drive the relative positions between the multiple color filter layers to change at least once;

The second acquisition module 303 is configured to acquire a second image collected by the photosensitive layer after changing the relative positions between the plurality of color filter layers each time, to obtain at least one second image;

A generating module 304, configured to generate a target image based on the first image and at least one second image.

Optionally, the multiple color filter layers include a first color filter layer composed of red color filter pixels and fully transparent color filter pixels, a second color filter layer composed of green color filter pixels and fully transparent color filter pixels, and The third color filter layer composed of blue color filter pixels and full transparent color filter pixels, the control module includes:

The control sub-module is used to control the driving unit of the image sensor to drive the movement of the multiple color filter layers when the color filter pixel arrays of the multiple color filter layers are overlapped with each other to form the first color filter array, so that the multiple color filter layers The relative positions between the layers are changed at least once, and the color filter pixel arrays of multiple color filter layers are stacked to form at least one second color filter array; in the first color filter array, all the red colors of the first color filter layer The color filter pixels overlap with half of the fully transparent filter pixels of the second color filter layer, and the other half of the fully transparent color filter pixels of the second color filter layer overlap with all the blue color filter pixels of the third color filter layer; wherein, The light transmittance of the second color filter array is greater than the light transmittance of the first color filter array.

Optionally, the control sub-module is also used to control the driving unit to drive the first color filter layer, the second color filter layer and the third color filter layer to move, so that all the red color filter pixels of the first color filter layer and the second color filter Half of the green color filter pixels of the color filter layer overlap each other, all the blue color filter pixels of the second color filter layer are aligned with the other half of the green color filter pixels of the second color filter layer, and the fully transparent color filter pixels of the multiple color filter layers Pixels overlap each other.

To sum up, an image generation device provided by an embodiment of the present invention includes a first acquisition module, configured to acquire a first image captured by a photosensitive layer of an image sensor; a control module, configured to control a drive unit of the image sensor to drive multiple The relative position between the color filter layers is changed at least once; the second acquisition module is configured to obtain the second image collected by the photosensitive layer after each change of the relative position between the multiple color filter layers, and obtain at least one first Two images; a generating module, configured to generate a target image based on the first image and at least one second image. When the above-mentioned device of the present invention is used to capture the first image and at least one second image and perform image synthesis, there is no need to align the pixels of the first image and at least one second image one by one, which not only improves the speed of image synthesis, but also avoids Poor compositing results due to alignment errors occur, further improving the capture quality of the target image.

Fig. 18 is the second block diagram of the image generation device provided by the embodiment of the present invention. As shown in Fig. 18, the image generation device includes:

The first acquisition module 401 is configured to acquire the first image collected by the photosensitive layer of the image sensor;

A control module 402, configured to control the drive unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once;

The second acquiring module 403 is configured to acquire a second image collected by the photosensitive layer after changing the relative position between the target color filter layer and the photosensitive layer each time, to obtain at least one second image;

A generating module 404, configured to generate a target image based on the first image and at least one second image.

Optionally, the control module also includes:

The control submodule is used to control the driving unit of the image sensor to drive the target color filter layer to move when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have a first relative positional relationship, so that the target color filter layer The relative position between the photosensitive layer and the photosensitive layer is changed at least once, and the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have at least one second corresponding relationship; wherein, the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer In the case where the photosensitive pixels of the photosensitive layer have the first relative positional relationship, the photosensitive pixels are aligned with a color filter pixel; In the case of having the second relative positional relationship, the photosensitive pixel is aligned with another color filter pixel.

In summary, an image generation device provided by an embodiment of the present invention includes a first acquisition module, configured to acquire a first image captured by a photosensitive layer of an image sensor; a control module, configured to control a drive unit of an image sensor to drive a target filter The relative position between the color layer and the photosensitive layer is changed at least once; the second acquisition module is configured to obtain the second image collected by the photosensitive layer after changing the relative position between the target color filter layer and the photosensitive layer each time, and obtain At least one second image; a generating module, configured to generate a target image based on the first image and at least one second image. When the above-mentioned device of the present invention is used to capture the first image and at least one second image and perform image synthesis, each photosensitive pixel can obtain multiple color signals, and color restoration can be performed according to the multiple color information obtained by the photosensitive pixel. Or completely eliminate the need for color restoration, improving the color accuracy of the final output target image.

An embodiment of the present invention also provides an electronic device. Referring to FIG. 19 , FIG. 19 shows a schematic diagram of a hardware structure of an electronic device provided by the present invention.

The electronic device 500 includes, but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, a processor 510, and Power supply 511 and other components. Those skilled in the art can understand that the structure of the electronic device shown in Figure 17 does not constitute a limitation on the electronic device, and the electronic device may include more or less components than shown in the illustration, or combine some components, or different components layout. In the embodiment of the present invention, electronic devices include but are not limited to mobile phones, tablet computers, notebook computers, palmtop computers, vehicle-mounted terminals, wearable devices, and pedometers.

Processor 510, configured to perform the following processes:

Acquire the first image collected by the photosensitive layer of the image sensor; control the drive unit of the image sensor to drive the relative position between the multiple color filter layers to change at least once; after changing the relative position between the multiple color filter layers each time, The second image collected by the photosensitive layer is acquired to obtain at least one second image; based on the first image and the at least one second image, a target image is generated.

In the case where the color filter pixel arrays of multiple color filter layers overlap each other to form the first color filter array, control the drive unit of the image sensor to drive the multiple color filter layers to move so that the relative positions between the multiple color filter layers At least one change occurs, and the color filter pixel arrays of multiple color filter layers overlap each other to form at least one second color filter array; in the first color filter array, all the red color filter pixels of the first color filter layer and the second Half of the fully transparent color filter pixels of the color filter layer overlap each other, and the other half of the fully transparent color filter pixels of the second color filter layer overlap with all the blue color filter pixels of the third color filter layer; wherein, the second color filter array The light transmittance is greater than the light transmittance of the first color filter array.

Controlling the driving unit to drive the first color filter layer, the second color filter layer and the third color filter layer to move, so that all the red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap each other , all the blue color filter pixels of the second color filter layer are aligned with the other half of the green color filter pixels of the second color filter layer, and the fully transparent color filter pixels of the multiple color filter layers overlap each other.

Acquire the first image collected by the photosensitive layer of the image sensor; control the drive unit of the image sensor to drive the relative position between the multiple color filter layers to change at least once; after changing the relative position between the multiple color filter layers each time, The second image collected by the photosensitive layer is acquired to obtain at least one second image; based on the first image and the at least one second image, a target image is generated.

In the case that the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have a first relative positional relationship, control the driving unit of the image sensor to drive the target color filter layer to move, so that the target color filter layer and the photosensitive layer The relative position changes at least once, and the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have at least one second corresponding relationship; wherein, the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer When the pixels have the first relative positional relationship, the photosensitive pixels are aligned with a kind of color filter pixels; when the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have the second relative In the case of a positional relationship, the photosensitive pixel is aligned with another color filter pixel.

In the embodiment of the present invention, when the above-mentioned device of the present invention is used to capture the first image and at least one second image and perform image synthesis, there is no need to align the pixels of the first image and at least one second image one by one, which not only improves the The speed of synthesis can also avoid the poor synthesis effect caused by alignment errors, further improve the shooting quality of the target image, or enable each photosensitive pixel to obtain a variety of color signals, which can be obtained according to the photosensitive pixel. A variety of color information is used for color restoration, or no color restoration is required at all, which improves the color accuracy of the final output target image.

It should be understood that, in the embodiment of the present invention, the radio frequency unit 501 can be used for receiving and sending signals during sending and receiving information or during a call. Specifically, after receiving the downlink data from the base station, the processor 510 processes it; Uplink data is sent to the base station. Generally, the radio frequency unit 501 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 501 can also communicate with the network and other devices through a wireless communication system.

The electronic device provides users with wireless broadband Internet access through the network module 502, such as helping users send and receive emails, browse web pages, and access streaming media.

The audio output unit 503 may convert audio data received by the radio frequency unit 501 or the network module 502 or stored in the memory 509 into an audio signal and output as sound. Also, the audio output unit 503 may also provide audio output related to a specific function performed by the electronic device 500 (for example, call signal reception sound, message reception sound, etc.). The audio output unit 503 includes a speaker, a buzzer, a receiver and the like.

The input unit 504 is used for receiving audio or video signals. The input unit 504 may include a graphics processing unit (Graphics Processing Unit, GPU) 5041 and a microphone 5042, and the graphics processor 5041 is used for still pictures or video images obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The data is processed. The processed image frames may be displayed on the display unit 506 . The image frames processed by the graphics processor 5041 may be stored in the memory 509 (or other storage media) or sent via the radio frequency unit 501 or the network module 502 . The microphone 5042 can receive sound, and can process such sound into audio data. The processed audio data can be converted into a format that can be sent to a mobile communication base station via the radio frequency unit 501 for output in the case of a phone call mode.

The electronic device 500 also includes at least one sensor 505, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 5061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 5061 and the / or backlighting. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when it is still, and can be used to identify the posture of electronic equipment (such as horizontal and vertical screen switching, related games) , magnetometer attitude calibration), vibration recognition-related functions (such as pedometer, knocking), etc.; the sensor 505 can also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, Infrared sensors, etc., will not be repeated here.

The display unit 506 is used to display information input by the user or information provided to the user. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an organic light-emitting diode (Organic Light-Emitting Diode, OLED), or the like.

The user input unit 507 can be used to receive input numbers or character information, and generate key signal input related to user settings and function control of the electronic device. Specifically, the user input unit 507 includes a touch panel 5071 and other input devices 5072 . The touch panel 5071, also referred to as a touch screen, can collect touch operations of the user on or near it (for example, the user uses any suitable object or accessory such as a finger or a stylus on the touch panel 5071 or near the touch panel 5071). operate). The touch panel 5071 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, and detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and sends it to the For the processor 510, receive the command sent by the processor 510 and execute it. In addition, the touch panel 5071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 5071 , the user input unit 507 may also include other input devices 5072 . Specifically, other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here.

Furthermore, the touch panel 5071 can be covered on the display panel 5061, and when the touch panel 5071 detects a touch operation on or near it, it will be sent to the processor 510 to determine the type of the touch event, and then the processor 510 can The type of event provides a corresponding visual output on the display panel 5061 . Although in FIG. 17, the touch panel 5071 and the display panel 5061 are used as two independent components to realize the input and output functions of the electronic device, in some embodiments, the touch panel 5071 and the display panel 5061 can be integrated. The implementation of the input and output functions of the electronic device is not specifically limited here.

The interface unit 508 is an interface for connecting an external device to the electronic device 500 . For example, an external device may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, audio input/output (I/O) ports, video I/O ports, headphone ports, and more. The interface unit 508 can be used to receive input from an external device (for example, data information, power, etc.) transfer data between devices.

The memory 509 can be used to store software programs as well as various data. The memory 509 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playback function, an image playback function, etc.); Data created by the use of mobile phones (such as audio data, phonebook, etc.), etc. In addition, the memory 509 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

The processor 510 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device, by running or executing software programs and/or modules stored in the memory 509, and calling data stored in the memory 509 , to perform various functions of the electronic equipment and process data, so as to monitor the electronic equipment as a whole. The processor 510 may include one or more processing units; preferably, the processor 510 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem The processor mainly handles wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 510 .

The electronic device 500 can also include a power supply 511 (such as a battery) for supplying power to various components. Preferably, the power supply 511 can be logically connected to the processor 510 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. and other functions.

The electronic device 500 may further include a camera module, which may be logically connected to the processor 510 and transmit captured images to the processor 510 .

The camera module can also include an image sensor, which can receive light and generate an image signal.

The image sensor may include a photosensitive layer, multiple color filter layers, and a drive unit connected to the multiple color filter layers; multiple color filter layers are stacked on the light-receiving surface of the photosensitive layer; the drive unit is used to drive the multiple color filter layers The relative position between them changes.

The camera module can also include a photosensitive layer, a target color filter layer, and a drive unit connected to the target color filter layer; the target color filter layer is arranged on the light-receiving surface of the photosensitive layer; the drive unit is used to drive the target color filter layer and the photosensitive layer. The relative position between them changes.

Optionally, the drive unit drives the plurality of color filter layers to switch between the first state and the second state;

In the first state, the plurality of color filter layers have a first relative positional relationship, and the plurality of color filter layers are stacked on each other to form a first color filter array;

In the second state, the plurality of color filter layers have a second relative positional relationship, and the plurality of color filter layers are stacked on each other to form a second color filter array;

Wherein, the light transmittance of the second color filter array is greater than the light transmittance of the first color filter array.

Optionally, the plurality of color filter layers include a first color filter layer, a second color filter layer and a third color filter layer;

The first color filter layer includes a plurality of 4×4 first color filter units, the first color filter unit includes four 2×2 first color filter groups, and the first color filter group includes a red filter color pixels and 3 transparent color pixels;

The second color filter layer includes a plurality of 4×4 second color filter units, the second color filter unit includes four 2×2 second color filter groups, and the second color filter group includes diagonally arranged 2 red color filter pixels and 2 fully transparent color filter pixels arranged diagonally;

The third color filter layer includes a plurality of 4×4 third color filter units, the third color filter unit includes four 2×2 third color filter groups, and the first color filter group includes a blue Color filter pixels and 3 transparent color filter pixels.

Optionally, when the plurality of color filter layers are stacked on each other to form a first color filter array, all the red color filter pixels of the first color filter layer and half of the second color filter layer are fully transparent The color filter pixels overlap each other, and the other half of the transparent filter pixels of the second color filter layer overlap with all the blue color filter pixels of the third color filter layer;

In the case where the plurality of color filter layers are stacked to form a second color filter array, all the red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap each other , all the blue color filter pixels of the second color filter layer are aligned with the other half of the green color filter pixels of the second color filter layer, and the fully transparent color filter pixels of the plurality of color filter layers overlap each other .

Optionally, the driving unit is connected to sides of the plurality of color filter layers, and the driving unit is configured to drive the plurality of color filter layers to move in a direction perpendicular to the sides.

Optionally, the drive unit includes a first drive unit connected to the first color filter layer, a second drive unit connected to the second color filter layer, and a drive unit connected to the third color filter layer third drive unit;

The first driving unit is used to drive the first color filter layer to move in a first direction;

The second driving unit is used to drive the second color filter layer to move in the first direction;

The third driving unit is used to drive the third color filter layer to move in a second direction, and the first direction is opposite to the second direction.

Optionally, the driving unit includes an electrostatic comb driver, and the electrostatic comb driver includes a power supply, a first comb part, and a second comb part;

The second comb-tooth portion is connected to the plurality of color filter layers;

The power supply is connected to the first comb-tooth portion and the second comb-tooth portion, and the power supply is used to drive relative movement between the first comb-tooth portion and the second comb-tooth portion.

The image sensor may also include a photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer;

The target color filter layer is arranged on the light-receiving surface of the photosensitive layer;

The driving unit is used to drive the relative position between the target color filter layer and the photosensitive layer to change.

Optionally, the target color filter layer includes a color filter pixel array, the color filter pixel array includes at least three kinds of color filter pixels, and the photosensitive layer includes a photosensitive pixel array;

The color filter pixels of the color filter pixel array overlap one-to-one with the photosensitive pixels of the photosensitive pixel array;

The number of relative positional relationships between the target color filter layer and the photosensitive layer is equal to the number of color filter pixels in the color filter pixel array.

Optionally, the target color filter layer includes a plurality of 4×4 color filter units; the color filter unit includes four 2×2 target color filter groups, and each of the target color filter groups includes red, green, Blue three color filter pixels.

Optionally, the driving unit includes an electrostatic comb driver, and the electrostatic comb driver includes a power supply, a first comb part, and a second comb part;

The second comb tooth part is connected with the target color filter layer;

The power supply is connected to the first comb-tooth portion and the second comb-tooth portion, and the power supply is used to drive relative movement between the first comb-tooth portion and the second comb-tooth portion.

In addition, the electronic device 500 includes some functional modules not shown, which will not be repeated here.

Preferably, the embodiment of the present invention also provides a mobile terminal, including a processor 510, a memory 509, and a computer program stored in the memory 509 and operable on the processor 510. When the computer program is executed by the processor 510, the above-mentioned The various processes in the embodiment of the image generation method can achieve the same technical effect, so in order to avoid repetition, details are not repeated here.

The embodiment of the present invention also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, each process of the above-mentioned image generation method embodiment is realized, and the same technology can be achieved. Effect, in order to avoid repetition, will not repeat them here. Wherein, the computer-readable storage medium is, for example, a read-only memory (Read-Only Memory, ROM for short), a random access memory (Random Access Memory, RAM for short), a magnetic disk or an optical disk, and the like.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods of various embodiments of the present invention.

Embodiments of the present invention have been described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementations, and the above-mentioned specific implementations are only illustrative, rather than restrictive, and those of ordinary skill in the art will Under the enlightenment of the present invention, without departing from the gist of the present invention and the protection scope of the claims, many forms can also be made, all of which belong to the protection of the present invention.

Claims (20)

1. An image sensor, characterized in that, comprising: a photosensitive layer, a plurality of color filter layers, and a driving unit connected to the plurality of color filter layers; The plurality of color filter layers are stacked on the light-receiving surface of the photosensitive layer; The driving unit is used for driving the relative positions of the plurality of color filter layers to change. 2. The image sensor according to claim 1, wherein the driving unit drives the plurality of color filter layers to switch between the first state and the second state; In the first state, the plurality of color filter layers have a first relative positional relationship, and the plurality of color filter layers are stacked on each other to form a first color filter array; In the second state, the plurality of color filter layers have a second relative positional relationship, and the plurality of color filter layers are stacked on each other to form a second color filter array; Wherein, the light transmittance of the second color filter array is greater than the light transmittance of the first color filter array. 3. The image sensor according to claim 2, wherein the plurality of color filter layers comprise a first color filter layer, a second color filter layer and a third color filter layer; The first color filter layer includes a plurality of 4×4 first color filter units, the first color filter unit includes four 2×2 first color filter groups, and the first color filter group includes a red filter color pixels and 3 transparent color pixels; The second color filter layer includes a plurality of 4×4 second color filter units, the second color filter unit includes four 2×2 second color filter groups, and the second color filter group includes diagonally arranged 2 red color filter pixels and 2 fully transparent color filter pixels arranged diagonally; The third color filter layer includes a plurality of 4×4 third color filter units, the third color filter unit includes four 2×2 third color filter groups, and the first color filter group includes a blue Color filter pixels and 3 transparent color filter pixels. 4. The image sensor according to claim 3, wherein when the plurality of color filter layers are stacked on each other to form a first color filter array, all red color filter pixels of the first color filter layer and half of the transparent color filter pixels of the second color filter layer overlap with each other, and the other half of the transparent color filter pixels of the second color filter layer overlap with all the blue color filter pixels of the third color filter layer ; In the case where the plurality of color filter layers are stacked to form a second color filter array, all the red color filter pixels of the first color filter layer and half of the green color filter pixels of the second color filter layer overlap each other , all the blue color filter pixels of the second color filter layer are aligned with the other half of the green color filter pixels of the second color filter layer, and the fully transparent color filter pixels of the plurality of color filter layers overlap each other . 5. The image sensor according to claim 3, wherein the driving unit is connected to the sides of the plurality of color filter layers, and the driving unit is used to drive the plurality of color filter layers perpendicular to The direction of the side moves. 6. The image sensor according to claim 3, wherein the driving unit comprises a first driving unit connected to the first color filter layer, a second driving unit connected to the second color filter layer , and a third drive unit connected to the third color filter layer; The first driving unit is used to drive the first color filter layer to move in a first direction; The second driving unit is used to drive the second color filter layer to move in the first direction; The third driving unit is used to drive the third color filter layer to move in a second direction, and the first direction is opposite to the second direction. 7. The image sensor according to claim 1, wherein the driving unit comprises an electrostatic comb driver, and the electrostatic comb driver includes a power supply, a first comb part and a second comb part; The second comb-tooth portion is connected to the plurality of color filter layers; The power supply is connected to the first comb-tooth portion and the second comb-tooth portion, and the power supply is used to drive relative movement between the first comb-tooth portion and the second comb-tooth portion. 8. A camera module, comprising the image sensor according to any one of claims 1-7. 9. An electronic device, comprising the camera module according to claim 8. 10. An image generation method applied to an electronic device, wherein the electronic device comprises the image sensor according to claims 1 to 7, the method comprising: acquiring a first image collected by the photosensitive layer of the image sensor; Controlling the driving unit of the image sensor to drive the relative positions between the plurality of color filter layers to change at least once; After changing the relative positions between the plurality of color filter layers each time, acquiring a second image collected by the photosensitive layer to obtain at least one second image; A target image is generated based on the first image and the at least one second image. 11. The method according to claim 10, wherein the plurality of color filter layers comprise a first color filter layer composed of red color filter pixels and fully transparent color filter pixels, a first color filter layer composed of green color filter pixels and fully transparent color filter pixels. A second color filter layer composed of color filter pixels, and a third color filter layer composed of blue color filter pixels and fully transparent color filter pixels; The drive unit controlling the image sensor drives the relative positions of the plurality of color filter layers to change at least once, including: In the case where the color filter pixel arrays of the plurality of color filter layers overlap each other to form the first color filter array, the driving unit of the image sensor is controlled to drive the plurality of color filter layers to move, so that the plurality of color filter layers The relative positions between the color filter layers are changed at least once, and the color filter pixel arrays of the plurality of color filter layers are overlapped with each other to form at least one second color filter array; Wherein, in the first color filter array, all the red color filter pixels of the first color filter layer and half of the transparent color filter pixels of the second color filter layer overlap with each other, and the second color filter layer The other half of the transparent color filter pixels overlap with all the blue color filter pixels of the third color filter layer; the light transmittance of the second color filter array is greater than the light transmittance of the first color filter array. 12. The method according to claim 11, wherein the controlling the drive unit of the image sensor to drive the plurality of color filter layers to move comprises: controlling the driving unit to drive the first color filter layer, the second color filter layer and the third color filter layer to move, so that all the red color filter pixels of the first color filter layer and the first color filter layer Half of the green color filter pixels of the two color filter layers overlap each other, all the blue color filter pixels of the second color filter layer are aligned with the other half of the green color filter pixels of the second color filter layer, and the plurality of color filter pixels The transparent color pixels of the color layer overlap each other. 13. An image sensor, characterized in that, comprising: a photosensitive layer, a target color filter layer, and a driving unit connected to the target color filter layer; The target color filter layer is arranged on the light-receiving surface of the photosensitive layer; The driving unit is used to drive the relative position between the target color filter layer and the photosensitive layer to change. 14. The image sensor according to claim 13, wherein the target color filter layer includes a color filter pixel array, the color filter pixel array includes at least three kinds of color filter pixels, and the photosensitive layer includes a photosensitive pixel array ; The color filter pixels of the color filter pixel array overlap one-to-one with the photosensitive pixels of the photosensitive pixel array; The number of relative positional relationships between the target color filter layer and the photosensitive layer is equal to the number of color filter pixels in the color filter pixel array. 15. The image sensor according to claim 13, wherein the target color filter layer comprises a plurality of 4×4 color filter units; the color filter unit comprises four 2×2 target color filter groups, each The target color filter groups each include three color filter pixels of red, green and blue. 16. The image sensor according to claim 13, wherein the driving unit comprises an electrostatic comb driver, and the electrostatic comb driver includes a power supply, a first comb part, and a second comb part; The second comb tooth part is connected with the target color filter layer; The power supply is connected to the first comb-tooth portion and the second comb-tooth portion, and the power supply is used to drive relative movement between the first comb-tooth portion and the second comb-tooth portion. 17. A camera module, comprising the image sensor according to any one of claims 13-16. 18. An electronic device, comprising the camera module according to claim 17. 19. An image generation method applied to an electronic device, wherein the electronic device comprises the image sensor according to claims 13 to 16, the method comprising: acquiring a first image collected by the photosensitive layer of the image sensor; controlling the drive unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once; After changing the relative position between the target color filter layer and the photosensitive layer each time, acquiring a second image collected by the photosensitive layer to obtain at least one second image; A target image is generated based on the first image and the at least one second image. 20. The method according to claim 19, wherein the controlling the driving unit of the image sensor to drive the relative position between the target color filter layer and the photosensitive layer to change at least once comprises: In the case that the filter pixels of the target color filter layer have a first relative positional relationship with the photosensitive pixels of the photosensitive layer, control the drive unit of the image sensor to drive the target color filter layer to move, so that the The relative position between the target color filter layer and the photosensitive layer is changed at least once, and the color filter pixels of the target color filter layer have at least one second corresponding relationship with the photosensitive pixels of the photosensitive layer; Wherein, in the case that the color filter pixels of the target color filter layer and the photosensitive pixels of the photosensitive layer have the first relative positional relationship, the photosensitive pixels are aligned with one kind of color filter pixels; In a case where the color filter pixels of the color layer and the photosensitive pixels of the photosensitive layer have the second relative positional relationship, the photosensitive pixels are aligned with another color filter pixel.

Images