CN111586323A - Image sensor, control method, camera assembly, and mobile terminal - Google Patents

Abstract

The present application discloses an image sensor, a control method, a camera assembly and a mobile terminal. The image sensor includes a two-dimensional pixel array and a lens array. The two-dimensional pixel array includes a plurality of color pixels and a plurality of full-color pixels. The color pixels have a narrower spectral response than the full-color pixels. The two-dimensional pixel array includes a minimum repeating unit. In the minimum repeating unit, the full-color pixels are arranged in a first diagonal direction, and the color pixels are arranged in a second diagonal direction. Each pixel includes at least two sub-pixels. The lens array includes a plurality of lenses, each lens covering a pixel. In an embodiment of the present application, the two-dimensional pixel array includes a plurality of color pixels and a plurality of full-color pixels, which increases the amount of light transmitted compared to a general color sensor, has a better signal-to-noise ratio, has a better focusing performance in dim light, and has a higher sensitivity of the sub-pixels. Each pixel includes at least two sub-pixels, which can improve the resolution of the image sensor while achieving phase focusing.

Description

Image sensor, control method, camera assembly, and mobile terminal

technical field

The present application relates to the field of imaging technologies, and more particularly, to an image sensor, a control method, a camera assembly, and a mobile terminal.

Background technique

With the development of electronic technology, terminals with a camera function have been popularized in people's lives. At present, the focusing methods used in mobile phone photography mainly include contrast focusing and phase detection auto focus (PDAF). Contrast focus is more accurate, but too slow. The phase focusing speed is fast. At present, the phase focusing on the market is realized on the color sensor (Bayer Sensor), and the focusing performance is not good enough in the dark light environment.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image sensor, a control method, a camera assembly, and a mobile terminal.

The image sensor of the embodiment of the present application includes a two-dimensional pixel array and a lens array. The two-dimensional pixel array includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array includes a minimum repeating unit, in which the panchromatic pixels are arranged in a first diagonal direction, and the color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel includes at least two sub-pixels. The lens array includes a plurality of lenses, each of the lenses covering one of the pixels.

The control method of the embodiment of the present application is applied to an image sensor. The image sensor includes a two-dimensional pixel array and a lens array. The two-dimensional pixel array includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array includes a minimum repeating unit, in which the panchromatic pixels are arranged in a first diagonal direction, and the color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel includes at least two sub-pixels. The lens array includes a plurality of lenses, each of the lenses covering one of the pixels. The control method includes: controlling the exposure of the at least two sub-pixels to output at least two sub-pixel information; determining phase information according to the at least two sub-pixel information to perform focusing; and controlling the two-dimensional pixel array in a focus state Exposure to acquire the target image.

The camera assembly of the embodiment of the present application includes a lens and an image sensor. The image sensor is capable of receiving light passing through the lens. The image sensor includes a two-dimensional pixel array and a lens array. The two-dimensional pixel array includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array includes a minimum repeating unit, in which the panchromatic pixels are arranged in a first diagonal direction, and the color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel includes at least two sub-pixels. The lens array includes a plurality of lenses, each of the lenses covering one of the pixels.

The mobile terminal according to the embodiment of the present application includes a casing and a camera assembly. The camera assembly is mounted on the casing. The camera assembly includes a lens and an image sensor. The image sensor is capable of receiving light passing through the lens. The image sensor includes a two-dimensional pixel array and a lens array. The two-dimensional pixel array includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array includes a minimum repeating unit, in which the panchromatic pixels are arranged in a first diagonal direction, and the color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel includes at least two sub-pixels. The lens array includes a plurality of lenses, each of the lenses covering one of the pixels.

The mobile terminal according to the embodiment of the present application includes a casing and a camera assembly. The camera assembly is mounted on the casing. The camera assembly includes a lens and an image sensor. The image sensor is capable of receiving light passing through the lens. The image sensor includes a two-dimensional pixel array and a lens array. The two-dimensional pixel array includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array includes a minimum repeating unit, in which the panchromatic pixels are arranged in a first diagonal direction, and the color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel includes at least two sub-pixels. The lens array includes a plurality of lenses, each of the lenses covering one of the pixels. The mobile terminal further includes a processor for implementing a control method: controlling the exposure of the at least two sub-pixels to output at least two sub-pixel information; determining phase information according to the at least two sub-pixel information to perform focusing; In the in-focus state, the exposure of the two-dimensional pixel array is controlled to obtain a target image.

In the image sensor, control method, camera assembly, and mobile terminal according to the embodiments of the present application, the two-dimensional pixel array includes a plurality of color pixels and a plurality of panchromatic pixels. Better signal-to-noise ratio, better focusing performance in low light, and higher sub-pixel sensitivity. In addition, each pixel includes at least two sub-pixels, which can improve the resolution of the image sensor while realizing phase focusing.

Additional aspects and advantages of embodiments of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of embodiments of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a schematic diagram of an image sensor according to some embodiments of the present application;

2 to 8 are schematic diagrams of the distribution of sub-pixels according to some embodiments of the present application;

9 is a schematic diagram of a pixel circuit according to some embodiments of the present application;

10 is a schematic diagram of exposure saturation time for different color channels;

11 to 20 are schematic diagrams of pixel arrangement and lens coverage of the minimum repeating unit according to some embodiments of the present application;

21 is a schematic flowchart of a control method according to some embodiments of the present application;

22 is a schematic diagram of a camera assembly of some embodiments of the present application;

FIG. 23 is a schematic diagram of the control method of some embodiments of the present application;

24 is a schematic flowchart of a control method according to some embodiments of the present application;

25 to 27 are schematic schematic diagrams of the control method of some embodiments of the present application;

FIG. 28 is a schematic diagram of a mobile terminal according to some embodiments of the present application.

Detailed ways

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, only used to explain the present application, and should not be construed as a limitation on the present application.

Referring to FIG. 1 , the present application provides an image sensor 10 . The image sensor 10 includes a two-dimensional pixel array 11 and a lens array 17 . The two-dimensional pixel array 11 includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array 11 includes a minimum repeating unit in which panchromatic pixels are arranged in a first diagonal direction, and color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel 101 includes at least two sub-pixels 102 . The lens array 17 includes a plurality of lenses 170 , each lens 170 covering one pixel 101 .

Please refer to FIG. 1 and FIG. 21 , the present application also provides a control method. The control method is used for the image sensor 10 . The image sensor 10 includes a two-dimensional pixel array 11 and a lens array 17 . The two-dimensional pixel array 11 includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array 11 includes a minimum repeating unit in which panchromatic pixels are arranged in a first diagonal direction, and color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel 101 includes at least two sub-pixels 102 . The lens array 17 includes a plurality of lenses 170 , each lens 170 covering one pixel 101 . Control methods include:

01: controlling the exposure of at least two sub-pixels 102 to output at least two sub-pixel information;

02: Determine phase information according to at least two sub-pixel information for focusing;

03: In the in-focus state, control the exposure of the two-dimensional pixel array 11 to obtain the target image.

Please refer to FIG. 1 and FIG. 22 , the present application further provides a camera assembly 40 . The camera assembly 40 includes the lens 30 and the image sensor 10 . The image sensor 10 can receive light passing through the lens 30 . The image sensor 10 includes a two-dimensional pixel array 11 and a lens array 17 . The two-dimensional pixel array 11 includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array 11 includes a minimum repeating unit in which panchromatic pixels are arranged in a first diagonal direction, and color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel 101 includes at least two sub-pixels 102 . The lens array 17 includes a plurality of lenses 170 , each lens 170 covering one pixel 101 .

Please refer to FIG. 1 , FIG. 22 and FIG. 28 , the present application further provides a mobile terminal 90 . The mobile terminal 90 includes a casing 80 and a camera assembly 40 . The camera assembly 40 is mounted on the casing 80 . The camera assembly 40 includes the lens 30 and the image sensor 10 . The image sensor 10 can receive light passing through the lens 30 . The image sensor 10 includes a two-dimensional pixel array 11 and a lens array 17 . The two-dimensional pixel array 11 includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array 11 includes a minimum repeating unit in which panchromatic pixels are arranged in a first diagonal direction, and color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel 101 includes at least two sub-pixels 102 . The lens array 17 includes a plurality of lenses 170 , each lens 170 covering one pixel 101 .

Please refer to FIG. 1 , FIG. 21 , FIG. 22 and FIG. 28 , the present application further provides a mobile terminal 90 . The mobile terminal 90 includes a casing 80 and a camera assembly 40 . The camera assembly 40 is mounted on the casing 80 . The camera assembly 40 includes the lens 30 and the image sensor 10 . The image sensor 10 can receive light passing through the lens 30 . The image sensor 10 includes a two-dimensional pixel array 11 and a lens array 17 . The two-dimensional pixel array 11 includes a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels. The two-dimensional pixel array 11 includes a minimum repeating unit in which panchromatic pixels are arranged in a first diagonal direction, and color pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. Wherein, each pixel 101 includes at least two sub-pixels 102 . The lens array 17 includes a plurality of lenses 170 , each lens 170 covering one pixel 101 . The mobile terminal 90 also includes a processor 60, and the processor 60 is used to implement the control method:

01: controlling the exposure of at least two sub-pixels 102 to output at least two sub-pixel information;

02: Determine phase information according to at least two sub-pixel information for focusing;

03: In the in-focus state, control the exposure of the two-dimensional pixel array 11 to obtain the target image.

With the development of electronic technology, terminals with a camera function have been popularized in people's lives. At present, the focusing methods used in mobile phone photography mainly include contrast focusing and phase detection auto focus (PDAF). Contrast focus is more accurate, but too slow. The phase focusing speed is fast. At present, the phase focusing on the market is realized on the color sensor (Bayer Sensor), and the focusing performance is not good enough in the dark light environment.

For the above reasons, the present application provides an image sensor 10 (shown in FIG. 1 ). In the image sensor 10 of the embodiment of the present application, the two-dimensional pixel array 11 includes a plurality of color pixels and a plurality of panchromatic pixels. Compared with a general color sensor, the amount of light passing through is increased, and the signal-to-noise ratio is better. The focusing performance is better in low light, and the sensitivity of the sub-pixel 102 will be higher. In addition, each pixel 101 includes at least two sub-pixels 102, which can improve the resolution of the image sensor 10 while achieving phase focus.

Next, the basic structure of the image sensor 10 will be described. Please refer to FIG. 1 . FIG. 1 is a schematic diagram of an image sensor 10 and a schematic diagram of a pixel 101 according to an embodiment of the present application. The image sensor 10 includes a two-dimensional pixel array 11 , a filter array 16 , and a lens array 17 . Along the light-receiving direction of the image sensor 10 , the lens array 17 , the filter 16 , and the two-dimensional pixel array 11 are arranged in sequence.

The image sensor 10 may use a complementary metal oxide semiconductor (CMOS, Complementary Metal Oxide Semiconductor) photosensitive element or a charge coupled device (CCD, Charge-coupled Device) photosensitive element.

The two-dimensional pixel array 11 includes a plurality of pixels 101 two-dimensionally arranged in an array. Each pixel 101 includes at least two sub-pixels 102 . For example, each pixel 101 includes two sub-pixels 102 , three sub-pixels 102 , four sub-pixels 102 , or more sub-pixels 102 .

The filter array 16 includes a plurality of filters 160 , and each filter 160 covers a corresponding one of the pixels 101 . The spectral response of each pixel 101 (ie, the color of light that the pixel 101 can receive) is determined by the color of the filter 160 corresponding to that pixel 102 .

The lens array 17 includes a plurality of lenses 170 , and each lens 170 covers a corresponding one of the pixels 101 .

2 to 8 show schematic diagrams of the distribution of sub-pixels 102 in various image sensors 10 . In the distribution manners of the sub-pixels 102 shown in FIGS. 2 to 8 , each pixel 101 includes at least two sub-pixels 102 , which can improve the resolution of the image sensor 10 while realizing phase focusing.

For example, FIG. 2 is a schematic diagram of the distribution of sub-pixels 102 according to an embodiment of the present application. Each pixel 101 includes two sub-pixels 102 . The boundary line between the two sub-pixels 102 is parallel to the length direction X of the two-dimensional pixel array 11 . The shape of the cross-section of each sub-pixel 102 is a rectangle, and the areas of the cross-section of the two sub-pixels 102 in each pixel 101 are equal. The cross-section refers to a cross-section taken perpendicular to the light-receiving direction of the image sensor 10 (the same applies hereinafter). The two sub-pixels 102 in each pixel 101 are centrally symmetrically distributed with respect to the center point of the pixel 101 . Phase information can be determined according to the two sub-pixel information output by the two sub-pixels 102 in each pixel 101 exposed to light, so as to realize phase focusing. Meanwhile, each pixel 101 includes a dividing line parallel to the length direction X of the two-dimensional pixel array 11. The two sub-pixels 102 can improve the vertical resolution of the image sensor 10 .

For example, FIG. 3 is a schematic diagram of the distribution of another sub-pixel 102 according to an embodiment of the present application. Each pixel 101 includes two sub-pixels 102 . The boundary line between the two sub-pixels 102 is parallel to the width direction Y of the two-dimensional pixel array 11 . The shape of the cross-section of each sub-pixel 102 is a rectangle, and the areas of the cross-section of the two sub-pixels 102 in each pixel 101 are equal. The two sub-pixels 102 in each pixel 101 are centrally symmetrically distributed with respect to the center point of the pixel 101 . The phase information can be determined according to the two sub-pixel information output by the two sub-pixels 102 in each pixel 101 exposed, so as to realize the phase focusing. Meanwhile, each pixel 101 includes a boundary line parallel to the width direction Y of the two-dimensional pixel array 11. The two sub-pixels 102 can improve the lateral resolution of the image sensor 10 .

For example, FIG. 4 is a schematic diagram of the distribution of still another sub-pixel 102 according to an embodiment of the present application. Each pixel 101 includes two sub-pixels 102 . The boundary line between the two sub-pixels 102 is inclined with respect to the length direction X of the two-dimensional pixel array 11 . The cross-sectional shape of each sub-pixel 102 is a trapezoid, the cross-section of one sub-pixel 102 in the same pixel 101 is a trapezoid with a narrow top and a wide bottom, and the cross-section of another sub-pixel 102 is a trapezoid with a wide top and a narrow bottom. . The two sub-pixels 102 in each pixel 101 are centrally symmetrically distributed with respect to the center point of the pixel 101 . Taking the center point of each pixel 101 as the origin, the length direction parallel to the two-dimensional pixel array 11 is the horizontal axis, and the width direction is the vertical axis, a Cartesian coordinate system is established, and the two sub-pixels 102 are on the positive half axis and the negative half axis of the horizontal axis. The axes are distributed, and the two sub-pixels 102 are distributed on the positive and negative semi-axis of the vertical axis. One sub-pixel 102 in each pixel 101 is distributed in the first quadrant, the second quadrant, and the fourth quadrant at the same time, and the other pixel 102 is distributed in the second quadrant, the third quadrant, and the fourth quadrant at the same time. The two sub-pixels 102 can obtain both the phase information in the horizontal direction and the phase information in the vertical direction, so that the image sensor 10 can be applied not only in a scene containing a large number of solid-color horizontal stripes, but also in a scene containing a large number of solid color horizontal stripes. In the scene of solid color vertical stripes, the image sensor 10 has better scene adaptability and higher accuracy of phase focusing. Meanwhile, each pixel 101 includes two sub-pixels 102 whose boundary lines are inclined with respect to the length direction X of the two-dimensional pixel array 11 , which can improve the lateral resolution or the vertical resolution of the image sensor 10 .

For example, FIG. 5 is a schematic diagram of the distribution of another sub-pixel 102 according to an embodiment of the present application. Each pixel 101 includes two sub-pixels 102 . The boundary line between the two sub-pixels 102 is inclined with respect to the width direction Y of the two-dimensional pixel array 11 . The shape of the cross section of each sub-pixel 102 is a trapezoid, the cross-section of one sub-pixel 102 in the same pixel 101 is a trapezoid that is wide at the top and narrow at the bottom, and the cross-section of the other sub-pixel 102 is a trapezoid that is narrow at the top and wide at the bottom. . The two sub-pixels 102 in each pixel 101 are centrally symmetrically distributed with respect to the center point of the pixel 101 . Taking the center point of each pixel 101 as the origin, the length direction parallel to the two-dimensional pixel array 11 is the horizontal axis, and the width direction is the vertical axis, a Cartesian coordinate system is established, and the two sub-pixels 102 are on the positive half axis and the negative half axis of the horizontal axis. The axes are distributed, and the two sub-pixels 102 are distributed on the positive and negative semi-axis of the vertical axis. One sub-pixel 102 in each pixel 101 is distributed in the first quadrant, the second quadrant, and the third quadrant at the same time, and the other pixel 102 is distributed in the first quadrant, the third quadrant, and the fourth quadrant at the same time. The two sub-pixels 102 can obtain both the phase information in the horizontal direction and the phase information in the vertical direction, so that the image sensor 10 can be applied not only in a scene containing a large number of solid-color horizontal stripes, but also in a scene containing a large number of solid color horizontal stripes. In the scene of solid color vertical stripes, the image sensor 10 has better scene adaptability and higher accuracy of phase focusing. Meanwhile, each pixel 101 includes two sub-pixels 102 whose boundary lines are inclined with respect to the width direction Y of the two-dimensional pixel array 11 , which can improve the lateral resolution or the vertical resolution of the image sensor 10 .

For example, FIG. 6 is a schematic diagram of the distribution of another sub-pixel 102 according to an embodiment of the present application. Each pixel 101 includes two sub-pixels 102. In some of the pixels 101, the boundary between the two sub-pixels 102 is parallel to the length direction X of the two-dimensional pixel array 11; in some of the pixels 101, the boundary between the two sub-pixels 102 is It is parallel to the width direction Y of the two-dimensional pixel array 11 . Further, the dividing line between the two sub-pixels 102 is parallel to the length direction X of the two-dimensional pixel array 11 and the dividing line between the two sub-pixels 102 is parallel to the width direction Y of the two-dimensional pixel array 11. Interlaced distribution or alternate column distribution. In some of the pixels 101, the two sub-pixels 102 can obtain the phase information in the vertical direction; in some of the pixels 101, the two sub-pixels 102 can obtain the phase information in the horizontal direction, so that the image sensor 10 can be applied to a large number of pure colors. In a scene with horizontal stripes, it can also be applied in a scene containing a large number of solid-color vertical stripes. The image sensor 10 has better scene adaptability and higher accuracy of phase focusing. Meanwhile, some of the pixels 101 include two sub-pixels 102 whose boundary lines are parallel to the length direction X of the two-dimensional pixel array 11 , and some of the pixels 101 include two sub-pixels 102 whose boundary lines are parallel to the width direction Y of the two-dimensional pixel array 11 . The lateral resolution and the vertical resolution of the image sensor 10 are improved to a certain extent. In addition, the dividing line between the two sub-pixels 102 and the pixels 101 parallel to the length direction X of the two-dimensional pixel array 11 and the dividing line between the two sub-pixels 102 are interlaced with the pixels 101 parallel to the width direction Y of the two-dimensional pixel array 11 Distribution or distribution in alternate columns will not increase the difficulty of wiring the two-dimensional pixel array 11 too much.

For example, FIG. 7 is a schematic diagram of the distribution of another sub-pixel 102 according to an embodiment of the present application. Each pixel 101 includes two sub-pixels 102. In some of the pixels 101, the boundary between the two sub-pixels 102 is parallel to the length direction X of the two-dimensional pixel array 11; in some of the pixels 101, the boundary between the two sub-pixels 102 is It is parallel to the width direction Y of the two-dimensional pixel array 11 . Further, the dividing line between the two sub-pixels 102 is parallel to the length direction X of the two-dimensional pixel array 11 and the dividing line between the two sub-pixels 102 is parallel to the width direction Y of the two-dimensional pixel array 11. Staggered distribution. In some of the pixels 101, the two sub-pixels 102 can obtain the phase information in the vertical direction; in some of the pixels 101, the two sub-pixels 102 can obtain the phase information in the horizontal direction, so that the image sensor 10 can be applied to a large number of pure colors. In a scene with horizontal stripes, it can also be applied in a scene containing a large number of solid-color vertical stripes. The image sensor 10 has better scene adaptability and higher accuracy of phase focusing. Meanwhile, some of the pixels 101 include two sub-pixels 102 whose boundary lines are parallel to the length direction X of the two-dimensional pixel array 11 , and some of the pixels 101 include two sub-pixels 102 whose boundary lines are parallel to the width direction Y of the two-dimensional pixel array 11 . The lateral resolution and the vertical resolution of the image sensor 10 are improved to a certain extent. In addition, the dividing line between the two sub-pixels 102 and the pixels 101 parallel to the length direction X of the two-dimensional pixel array 11 and the dividing line between the two sub-pixels 102 are staggered with the pixels 101 parallel to the width direction Y of the two-dimensional pixel array 11 distribution to maximize the phase focus effect.

For example, FIG. 8 is a schematic diagram of the distribution of another sub-pixel 102 according to an embodiment of the present application. Each pixel 101 includes four sub-pixels 102, and the four sub-pixels 102 are distributed in a 2*2 matrix. The phase information can be accurately determined according to the information of at least two sub-pixels output by any at least two sub-pixels 102 in each pixel 101 exposed to light, so as to realize phase focusing. Meanwhile, each pixel 101 includes four sub-pixels distributed in a 2*2 matrix. The pixels 102 can simultaneously improve the lateral resolution and the vertical resolution of the image sensor 10 . In other examples, when each pixel 101 includes four sub-pixels 102, the four sub-pixels 102 may also be distributed along the width direction Y of the two-dimensional pixel array 11, and the boundary between two adjacent sub-pixels 102 is the same as the two-dimensional The length direction X of the pixel array 11 is parallel; alternatively, the four sub-pixels 102 may be distributed along the length direction X of the two-dimensional pixel array 11, and the boundary between two adjacent sub-pixels 102 is the width direction Y of the two-dimensional pixel array 11 Parallel or the like, or the boundary line between two adjacent sub-pixels 102 may be inclined with respect to the longitudinal direction X or the width direction Y of the two-dimensional pixel array 11, etc., which are not limited here.

It should be noted that, in addition to the cross-sectional shapes of the sub-pixels 102 shown in FIGS. 2 to 8 , the cross-sectional shapes of the sub-pixels 102 may also be other regular or irregular shapes, which are not limited herein.

In addition, in addition to the case where each pixel 101 includes two sub-pixels 102 and each pixel 101 includes four sub-pixels 102, it may also be that: some pixels 101 include two sub-pixels 102, and some pixels 101 include four sub-pixels 102, Alternatively, any other number of at least two sub-pixels 102 is included, which is not limited here.

FIG. 9 is a schematic diagram of a pixel circuit 110 in an embodiment of the present application. The embodiments of the present application are described by taking each pixel 101 including two sub-pixels 102 as an example, and the case where each pixel 101 includes more than two sub-pixels 102 will be described later. The working principle of the pixel circuit 110 will be described below with reference to FIG. 1 and FIG. 9 .

As shown in FIGS. 1 and 9 , the pixel circuit 110 includes a first photoelectric conversion element 1171 (eg, a photodiode PD), a second photoelectric conversion element 1172 (eg, a photodiode PD), a first exposure control circuit 1161 (eg, a photodiode PD) transfer transistor 112), a second exposure control circuit 1162 (eg, transfer transistor 112), a reset circuit (eg, reset transistor 113), an amplifier circuit (eg, amplifier transistor 114), and a selection circuit (eg, selection transistor 115). At this time, one of the sub-pixels 102 includes the above-mentioned first photoelectric conversion element 1171 , and the other sub-pixel 102 includes the above-mentioned second photoelectric conversion element 1172 . In the embodiment of the present application, the transfer transistor 112 , the reset transistor 113 , the amplifying transistor 114 and the selection transistor 115 are, for example, MOS transistors, but are not limited thereto.

For example, referring to FIGS. 1 and 9 , the gate TG of the transfer transistor 112 is connected to the vertical driving unit (not shown in the figure) of the image sensor 10 through an exposure control line (not shown in the figure); the gate RG of the reset transistor 113 The vertical driving unit is connected through a reset control line (not shown in the figure); the gate SEL of the selection transistor 115 is connected to the vertical driving unit through a selection line (not shown in the figure). The first exposure control circuit 1161 in each pixel circuit 110 is electrically connected to the first photoelectric conversion element 1171 for transferring the potential accumulated by the first photoelectric conversion element 1171 after being illuminated; the second exposure control circuit in each pixel circuit 110 The circuit 1162 is electrically connected to the second photoelectric conversion element 1172 for transferring the electric potential accumulated by the second photoelectric conversion element 1172 after being illuminated. For example, the first photoelectric conversion element 1171 and the second photoelectric conversion element 1172 each include a photodiode PD, the anode of which is connected to the ground, for example. The photodiode PD converts the received light into electric charges. The cathode of the photodiode PD is connected to the floating diffusion unit FD via the first exposure control circuit 1161 or the second exposure control circuit 1162 (eg, the transfer transistor 112 ). The floating diffusion unit FD is connected to the gate of the amplifier transistor 114 and the source of the reset transistor 113 .

For example, both the first exposure control circuit 1161 and the second exposure control circuit 1162 may be the transfer transistor 112 , and the control terminal TG of the first exposure control circuit 1161 and the second exposure control circuit 1162 is the gate of the transfer transistor 112 . When a pulse of an active level (eg, VPIX level) is transmitted to the gate of the transfer transistor 112 through the exposure control line, the transfer transistor 112 is turned on. The transfer transistor 112 transfers the charges photoelectrically converted by the photodiode PD to the floating diffusion unit FD.

For example, the reset circuit is connected to the first exposure control circuit 1161 and the second exposure control circuit 1162 at the same time. The reset circuit may be the reset transistor 113 . The drain of the reset transistor 113 is connected to the pixel power supply VPIX. The source of the reset transistor 113 is connected to the floating diffusion unit FD. Before charges are transferred from the photodiode PD to the floating diffusion unit FD, a pulse of an effective reset level is transmitted to the gate of the reset transistor 113 via the reset line, and the reset transistor 113 is turned on. The reset transistor 113 resets the floating diffusion unit FD to the pixel power supply VPIX.

For example, the gate of the amplification transistor 114 is connected to the floating diffusion unit FD. The drain of the amplification transistor 114 is connected to the pixel power supply VPIX. After the floating diffusion unit FD is reset by the reset transistor 113 , the amplifying transistor 114 outputs the reset level through the output terminal OUT via the selection transistor 115 . After the charge of the photodiode PD is transferred by the transfer transistor 112 , the amplifying transistor 114 outputs the signal level through the output terminal OUT via the selection transistor 115 .

For example, the drain of the selection transistor 115 is connected to the source of the amplification transistor 114 . The source of the selection transistor 115 is connected to a column processing unit (not shown in the figure) in the image sensor 10 through the output terminal OUT. When the pulse of the active level is transmitted to the gate of the selection transistor 115 through the selection line, the selection transistor 115 is turned on. The signal output by the amplification transistor 114 is transmitted to the column processing unit through the selection transistor 115 .

When the two sub-pixels 102 are exposed to output pixel information respectively, or output pixel information and output phase information respectively, the first exposure control circuit 1161 first transfers the charge generated by the first photoelectric conversion element 1171 after receiving the light to output the first sub-pixel information , after the reset circuit is reset, the second exposure control circuit 1162 transfers the charge generated by the second photoelectric conversion element 1172 after receiving the light to output the second sub-pixel information.

When the two sub-pixels 102 are exposed to combine and output pixel information (excluding phase information), the first exposure control circuit 1161 transfers the charge generated by the first photoelectric conversion element 1171 after receiving light, and the second exposure control circuit 1162 can transfer the first The second photoelectric conversion element 1172 receives the charges generated by the light to output the combined pixel information. That is to say, while the first exposure control circuit 1161 transfers the charges generated after the first photoelectric conversion element 1171 receives light, the second exposure control circuit 1162 also transfers the charges generated after the second photoelectric conversion element 1172 receives light, or It may be that the first exposure control circuit 1161 first transfers the charges generated by the first photoelectric conversion element 1171 after receiving the light, and the second exposure control circuit 1162 then transfers the charges generated by the second photoelectric conversion element 1172 after receiving the light.

When the two sub-pixels 102 are exposed to combine the output pixel information and output the phase information, the first exposure control circuit 1161 first transfers the charge generated by the first photoelectric conversion element 1171 after receiving the light to output the first sub-pixel information. After the reset circuit is reset, The second exposure control circuit 1162 then transfers the charges generated by the second photoelectric conversion element 1172 after receiving the light to output the second sub-pixel information, and the first sub-pixel information and the second sub-pixel information are used for combining into combined pixel information. At this time, the image sensor 10 may further include a buffer (not shown) for storing the first sub-pixel information and the second sub-pixel information to output phase information.

In signal transmission, pixel information and phase information can be separately output through the Mobile Industry Processor Interface (MIPI). Since the demand for phase information is small, the pixel circuit 110 can output four lines of pixel information when the pixel circuit 110 outputs four lines of pixel information. , and then output one line of phase information. Of course, in practical applications, each time the pixel circuit 110 outputs one line of pixel information, another line of phase information may be output, which is not limited here.

It should be noted that the pixel structure of the pixel circuit 110 in the embodiment of the present application is not limited to the structure shown in FIG. 9 . For example, the pixel circuit 110 may have a three-transistor pixel structure in which the functions of the amplification transistor 114 and the selection transistor 115 are performed by one transistor. For example, the first exposure control circuit 1161 and the second exposure control circuit 1162 are not limited to the mode of a single transfer transistor 112, and other electronic devices or structures with the function of controlling the conduction of the control terminal can be used as the exposure control in the embodiments of the present application. Circuit, the implementation of a single transfer transistor 112 is simple, low cost, and easy to control.

In addition, when each pixel 101 includes more than two sub-pixels 102 (that is, each pixel 101 includes more than two photoelectric conversion elements), it is only necessary to increase the number of exposure control circuits correspondingly, and each exposure control circuit corresponds to a corresponding The photoelectric conversion elements are connected, and all the exposure control circuits are connected with the reset circuit.

In the embodiment of the present application, the multiple sub-pixels 102 in each pixel 101 share the reset circuit, the amplification circuit and the selection circuit, which is beneficial to reduce the space occupied by the pixel circuit 110 and has a lower cost. In other embodiments, each sub-pixel 102 in each pixel 101 may have a corresponding reset circuit, amplification circuit, selection circuit, etc., which is not limited herein.

In an image sensor that includes pixels of multiple colors, the pixels of different colors receive different amounts of exposure per unit time. After some colors are saturated, some colors are not exposed to the desired state. For example, exposure to 60%-90% of the saturated exposure may have better signal-to-noise ratio and accuracy, but the embodiments of the present application are not limited thereto.

In FIG. 10, RGBW (red, green, blue, full color) is used as an example for illustration. Referring to Figure 10, in Figure 10, the horizontal axis is the exposure time, the vertical axis is the exposure amount, Q is the saturated exposure amount, LW is the exposure curve of the panchromatic pixel W, LG is the exposure curve of the green pixel G, and LR is the red pixel R. , and LB is the exposure curve of the blue pixel.

It can be seen from FIG. 10 that the exposure curve LW of the panchromatic pixel W has the largest slope, that is to say, the panchromatic pixel W can obtain more exposure per unit time, and reaches saturation at time t1. The slope of the exposure curve LG of the green pixel G is second, and the green pixel is saturated at time t2. The slope of the exposure curve LR of the red pixel R is again the same, and the red pixel is saturated at time t3. The slope of the exposure curve LB of the blue pixel B is the smallest, and the blue pixel is saturated at time t4. It can be seen from FIG. 10 that the exposure amount received by the panchromatic pixel W per unit time is greater than the exposure amount received by the color pixel per unit time, that is, the sensitivity of the panchromatic pixel W is higher than that of the color pixel.

If an image sensor that only includes color pixels is used to achieve phase focus, then in a high-brightness environment, the three color pixels of R, G, and B can receive more light, and can output pixels with a high signal-to-noise ratio. At this time, the accuracy of phase focusing is high; but in a low-brightness environment, the three pixels R, G, and B can receive less light, and the signal-to-noise ratio of the output pixel information is low. At this time Phase focus is also less accurate.

Based on the above reasons, the image sensor 10 of the embodiment of the present application can simultaneously arrange full-color pixels and color pixels in the two-dimensional pixel array 11 , which increases the amount of light passing and has better signal-to-noise compared with general color sensors. In contrast, the focusing performance is better in low light, and the sensitivity of the sub-pixel 102 will be higher. In this way, the image sensor 10 of the embodiment of the present application can achieve accurate focusing in scenes with different ambient brightness, which improves the scene adaptability of the image sensor 10 .

It should be noted that the spectral response of each pixel 101 (ie, the color of light that the pixel 101 can receive) is determined by the color of the filter 160 corresponding to the pixel 101 . Color pixels and panchromatic pixels throughout this application refer to pixels 101 that are capable of responding to light of the same color as the corresponding filter 160 .

11 to 20 show examples of the arrangement of pixels 101 in various image sensors 10 (shown in FIG. 1 ). 11 to 20 , the plurality of pixels 101 in the two-dimensional pixel array 11 may simultaneously include a plurality of full-color pixels and a plurality of color pixels (eg, a plurality of first color pixels A, a plurality of second color pixels B and a plurality of third color pixels C), wherein the color pixels and panchromatic pixels are distinguished by the wavelength bands of light that can pass through the filter 160 (shown in FIG. 1 ) covered thereon, and the color pixels have more Narrow spectral response, the response spectrum of a color pixel is, for example, part of the W response spectrum of a panchromatic pixel. Each panchromatic pixel includes at least two sub-pixels 102 , and each color pixel includes at least two sub-pixels 102 . The two-dimensional pixel array 11 is composed of a plurality of minimum repeating units (examples of the minimum repeating units in various image sensors 10 are shown in FIGS. 11 to 20 ), which are replicated and arranged in rows and columns. Each minimum repeating unit includes a plurality of subunits, and each subunit includes a plurality of single-color pixels and a plurality of full-color pixels. For example, each minimum repeating unit includes four subunits, wherein one subunit includes a plurality of single-color pixels A (ie, first color pixels A) and a plurality of full-color pixels W, and two subunits include a plurality of single-color pixels B (ie, the second color pixel B) and a plurality of panchromatic pixels W, and the remaining one subunit includes a plurality of single-color pixels C (ie, the third color pixel C) and a plurality of panchromatic pixels W.

For example, the number of pixels 101 of the rows and columns of the minimum repeating unit is equal. For example, the minimum repeating unit includes, but is not limited to, the minimum repeating unit of 4 rows and 4 columns, 6 rows and 6 columns, 8 rows and 8 columns, and 10 rows and 10 columns. For example, the number of pixels 101 of the rows and columns of the subunit is equal. For example, subunits include, but are not limited to, subunits with 2 rows and 2 columns, 3 rows and 3 columns, 4 rows and 4 columns, and 5 rows and 5 columns. This setting helps to equalize the resolution of the image in the row and column directions and equalize the color performance, improving the display effect.

In an example, in the smallest repeating unit, the panchromatic pixels W are arranged in the first diagonal direction D1, the color pixels are arranged in the second diagonal direction D2, the first diagonal direction D1 and the second diagonal direction The direction D2 is different.

For example, FIG. 11 is a schematic diagram of the arrangement of pixels 101 and the covering method of lenses 170 of a minimum repeating unit in an embodiment of the present application; The arrangement is as follows:

W represents a full-color pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in a plurality of color pixels; C represents a third color pixel in the plurality of color pixels.

As shown in FIG. 11 , the panchromatic pixels W are arranged in the first diagonal direction D1 (that is, the direction connecting the upper left corner and the lower right corner in FIG. 11 ), and the color pixels are arranged in the second diagonal direction D2 (for example, in FIG. 11 ) The direction in which the lower left corner and the upper right corner are connected), the first diagonal direction D1 is different from the second diagonal direction D2. For example, the first diagonal and the second diagonal are perpendicular.

It should be noted that the first diagonal direction D1 and the second diagonal direction D2 are not limited to diagonals, but also include directions parallel to the diagonals. The "direction" here is not a single direction, but can be understood as the concept of a "straight line" indicating the arrangement, and there can be bidirectional directions at both ends of the straight line.

As shown in FIG. 11 , one lens 170 covers one pixel 101 . Each panchromatic pixel and each color pixel includes two sub-pixels 102 .

For example, FIG. 12 is a schematic diagram illustrating a schematic diagram of another arrangement of pixels 101 of a minimum repeating unit and a covering manner of lenses 170 in an embodiment of the present application. The minimum repeating unit is 16 pixels 101 in 4 rows and 4 columns, and the subunit is 4 pixels 101 in 2 rows and 2 columns. The arrangement is as follows:

W represents a full-color pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in a plurality of color pixels; C represents a third color pixel in the plurality of color pixels.

As shown in FIG. 12 , the panchromatic pixels W are arranged in the first diagonal direction D1 (that is, the direction connecting the upper right corner and the lower left corner in FIG. 12 ), and the color pixels are arranged in the second diagonal direction D2 (for example, in FIG. 12 ) the direction in which the upper-left and lower-right corners are connected). The first diagonal direction D1 is different from the second diagonal direction D2. For example, the first diagonal and the second diagonal are perpendicular.

As shown in FIG. 12 , one lens 170 covers one pixel 101 . Each panchromatic pixel and each color pixel includes two sub-pixels 102 .

For example, FIG. 13 is a schematic diagram of the arrangement of the pixels 101 and the covering manner of the lenses 170 of yet another minimum repeating unit in the embodiment of the present application. FIG. 14 is a schematic diagram of another arrangement of pixels 101 of a minimum repeating unit and a covering manner of lenses 170 in an embodiment of the present application. In the embodiments of FIGS. 13 and 14 , corresponding to the arrangement and coverage of FIGS. 11 and 12 respectively, the first color pixel A is the red pixel R; the second color pixel B is the green pixel G; the third color pixel C is the blue pixel Bu.

It should be noted that, in some embodiments, the response band of the panchromatic pixel W is the visible light band (for example, 400 nm-760 nm). For example, the panchromatic pixel W is provided with an infrared filter to filter out infrared light. In some embodiments, the response bands of the panchromatic pixels W are visible light band and near-infrared band (eg, 400nm-1000nm), which match the response band of the photoelectric conversion element (eg, photodiode PD) in the image sensor 10 . For example, the panchromatic pixel W may not be provided with a filter, and the response band of the panchromatic pixel W is determined by the response band of the photodiode, that is, the two match. The embodiments of the present application include, but are not limited to, the above-mentioned waveband ranges.

In some embodiments, in the minimum repeating unit shown in FIG. 11 and FIG. 12 , the first color pixel A may also be a red pixel R, the second color pixel B may also be a yellow pixel Y; the third color pixel C may be Blue pixel Bu.

In some embodiments, in the minimum repeating unit shown in FIG. 11 and FIG. 12 , the first color pixel A may also be a magenta pixel M, the second color pixel B may also be a cyan pixel Cy, and the third color pixel C may also be Can be yellow pixel Y.

For example, FIG. 15 is a schematic diagram illustrating the arrangement of pixels 101 and the covering manner of lenses 170 of yet another minimum repeating unit in the embodiment of the present application. The minimum repeating unit is 36 pixels 101 in 6 rows, 6 columns, and the subunit is 9 pixels 101 in 3 rows, 3 columns, and the arrangement is as follows:

W represents a full-color pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in a plurality of color pixels; C represents a third color pixel in the plurality of color pixels.

As shown in FIG. 15 , the panchromatic pixels W are arranged in the first diagonal direction D1 (that is, the direction connecting the upper left corner and the lower right corner in FIG. 15 ), and the color pixels are arranged in the second diagonal direction D2 (for example, in FIG. 15 ) The direction in which the lower left corner and the upper right corner are connected), the first diagonal direction D1 is different from the second diagonal direction D2. For example, the first diagonal and the second diagonal are perpendicular.

As shown in FIG. 15 , one lens 170 covers one pixel 101 . Each panchromatic pixel and each color pixel includes two sub-pixels 102 .

For example, FIG. 16 is a schematic diagram of the arrangement of the pixels 101 and the covering manner of the lens 170 in yet another minimum repeating unit in the embodiment of the present application. The minimum repeating unit is 36 pixels 101 in 6 rows, 6 columns, and the subunit is 9 pixels 101 in 3 rows, 3 columns, and the arrangement is as follows:

W represents a full-color pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in a plurality of color pixels; C represents a third color pixel in the plurality of color pixels.

As shown in FIG. 16 , the panchromatic pixels W are arranged in the first diagonal direction D1 (that is, the direction connecting the upper right corner and the lower left corner in FIG. 16 ), and the color pixels are arranged in the second diagonal direction D2 (for example, in FIG. 16 ) the direction in which the upper-left and lower-right corners are connected). The first diagonal direction D1 is different from the second diagonal direction D2. For example, the first diagonal and the second diagonal are perpendicular.

As shown in FIG. 16 , one lens 170 covers one pixel 101 . Each panchromatic pixel and each color pixel includes two sub-pixels 102 .

For example, the first color pixel A in the minimum repeating unit shown in FIG. 15 and FIG. 16 may be a red pixel R, the second color pixel B may be a green pixel G, and the third color pixel C may be a blue pixel Bu. Alternatively, the first color pixel A in the minimum repeating unit shown in FIG. 15 and FIG. 16 may be a red pixel R, the second color pixel B may be a yellow pixel Y, and the third color pixel C may be a blue pixel Bu. Alternatively, the first color pixel A in the minimum repeating unit shown in FIG. 15 and FIG. 16 may be a magenta pixel M, the second color pixel B may be a cyan pixel Cy, and the third color pixel C may be a yellow pixel Y.

For example, FIG. 17 is a schematic diagram of the arrangement of the pixels 101 and the covering manner of the lenses 170 of yet another minimum repeating unit in the embodiment of the present application. The minimum repeating unit is 64 pixels 101 in 8 rows and 8 columns, and the subunit is 16 pixels 101 in 4 rows and 4 columns. The arrangement is as follows:

W represents a full-color pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in a plurality of color pixels; C represents a third color pixel in the plurality of color pixels.

As shown in FIG. 17 , the panchromatic pixels W are arranged in the first diagonal direction D1 (that is, the direction connecting the upper left corner and the lower right corner in FIG. 17 ), and the color pixels are arranged in the second diagonal direction D2 (for example, in FIG. 17 ) The direction in which the lower left corner and the upper right corner are connected), the first diagonal direction D1 is different from the second diagonal direction D2. For example, the first diagonal and the second diagonal are perpendicular.

As shown in FIG. 17 , one lens 170 covers one pixel 101 . Each panchromatic pixel and each color pixel includes two sub-pixels 102 .

For example, FIG. 18 is a schematic diagram of the arrangement of the pixels 101 and the covering manner of the lenses 170 of yet another minimum repeating unit in the embodiment of the present application. The minimum repeating unit is 64 pixels 101 in 8 rows and 8 columns, and the subunit is 16 pixels 101 in 4 rows and 4 columns. The arrangement is as follows:

W represents a full-color pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in a plurality of color pixels; C represents a third color pixel in the plurality of color pixels.

As shown in FIG. 18 , the panchromatic pixels W are arranged in the first diagonal direction D1 (that is, the direction connecting the upper right corner and the lower left corner in FIG. 18 ), and the color pixels are arranged in the second diagonal direction D2 (for example, in FIG. 18 ) the direction in which the upper-left and lower-right corners are connected). The first diagonal direction D1 is different from the second diagonal direction D2. For example, the first diagonal and the second diagonal are perpendicular.

As shown in FIG. 18 , one lens 170 covers one pixel 101 . Each panchromatic pixel and each color pixel includes two sub-pixels 102 .

In the examples shown in FIGS. 11 to 18 , in each subunit, adjacent panchromatic pixels W are arranged diagonally, and adjacent color pixels are also arranged diagonally. In another example, in each subunit, adjacent panchromatic pixels are arranged in the horizontal direction, and adjacent color pixels are also arranged in the horizontal direction; or, adjacent panchromatic pixels are arranged in the vertical direction, and adjacent Color pixels are also arranged in the vertical direction. Panchromatic pixels in adjacent subunits may be arranged in a horizontal direction or in a vertical direction, and color pixels in adjacent subunits may also be arranged in a horizontal direction or a vertical direction.

For example, FIG. 19 is a schematic diagram of an arrangement of pixels 101 and a covering manner of lenses 170 of yet another minimum repeating unit in an embodiment of the present application. The minimum repeating unit is 16 pixels 101 in 4 rows and 4 columns, and the subunit is 4 pixels 101 in 2 rows and 2 columns. The arrangement is as follows:

W represents a full-color pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in a plurality of color pixels; C represents a third color pixel in the plurality of color pixels.

As shown in FIG. 19 , in each subunit, adjacent panchromatic pixels W are arranged in the vertical direction, and adjacent color pixels are also arranged in the vertical direction. One lens 170 covers one pixel 101 . Each panchromatic pixel and each color pixel includes two sub-pixels 102 .

For example, FIG. 20 is a schematic diagram of the arrangement of the pixels 101 and the covering manner of the lenses 170 of yet another minimum repeating unit in the embodiment of the present application. The minimum repeating unit is 16 pixels 101 in 4 rows and 4 columns, and the subunit is 4 pixels 101 in 2 rows and 2 columns. The arrangement is as follows:

W represents a full-color pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in a plurality of color pixels; C represents a third color pixel in the plurality of color pixels.

As shown in FIG. 20 , in each subunit, adjacent panchromatic pixels W are arranged in the horizontal direction, and adjacent color pixels are also arranged in the horizontal direction. One lens 170 covers one pixel 101 . Each panchromatic pixel and each color pixel includes two sub-pixels 102 .

19 and 20, the first color pixel A may be a red pixel R, the second color pixel B may be a green pixel G, and the third color pixel C may be a blue pixel Bu. 19 and 20, the first color pixel A may be a red pixel R, the second color pixel B may be a yellow pixel Y, and the third color pixel C may be a blue pixel Bu. 19 and 20, the first color pixel A may be a magenta pixel M, the second color pixel B may be a cyan pixel Cy, and the third color pixel C may be a yellow pixel Y.

In the minimum repeating unit shown in FIGS. 11 to 20 , each full-color pixel and each color pixel includes two sub-pixels 102 . In other embodiments, each panchromatic pixel may include four subpixels 102, and each color pixel may include four subpixels 102; or each panchromatic pixel may include four subpixels 102, and each color pixel may include two subpixels 102, etc., which are not limited here.

In the minimum repeating unit shown in FIGS. 11 to 20 , the boundary between the two sub-pixels 102 is parallel to the width direction Y of the two-dimensional pixel array 11 . In other embodiments, the boundary line between the two sub-pixels 102 may be parallel to the length direction X of the two-dimensional pixel array 11, or any of the foregoing boundary lines or combinations thereof.

A plurality of panchromatic pixels and a plurality of color pixels in the two-dimensional pixel array 11 of any arrangement shown in FIGS. 11 to 20 can be controlled by different exposure control lines, so as to realize the exposure time of the panchromatic pixels. and independent control of exposure time for color pixels. Among them, for the two-dimensional pixel array 11 in any arrangement shown in FIGS. 11 to 18 , the control terminals of the exposure control circuits of at least two panchromatic pixels adjacent to the first diagonal direction are connected to the first exposure control circuit. The wires are electrically connected, and the control terminals of the exposure control circuits of at least two adjacent color pixels in the second diagonal direction are electrically connected to the second exposure control wires. For the two-dimensional pixel array 11 shown in FIG. 19 and FIG. 20 , the control terminals of the exposure control circuits of the full-color pixels in the same row or column are electrically connected to the first exposure control line, and the color pixels in the same row or column are exposed to The control end of the control circuit is electrically connected with the second exposure control line. The first exposure control line may transmit a first exposure signal to control the first exposure time of the full-color pixels, and the second exposure control line may transmit a second exposure signal to control the second exposure time of the color pixels. Wherein, when the panchromatic pixel includes two sub-pixels 102, the two sub-pixels 102 in the panchromatic pixel are electrically connected by the same first exposure control line. When the color pixel includes two sub-pixels 102, the two sub-pixels 102 in the color pixel are electrically connected by the same second exposure control line.

When the exposure time of the panchromatic pixels and the exposure time of the color pixels are independently controlled, the first exposure time of the panchromatic pixels may be shorter than the second exposure time of the color pixels. For example, the ratio of the first exposure time to the second exposure time may be one of 1:2, 1:3, or 1:4. For example, in a dark environment, color pixels are more likely to be underexposed, and the ratio of the first exposure time to the second exposure time can be adjusted to 1:2, 1:3 or 1:4 according to the ambient brightness. Wherein, when the exposure ratio is the above integer ratio or close to the integer ratio, it is beneficial to the setting and control of the timing setting signal.

In some embodiments, the relative relationship between the first exposure time and the second exposure time may be determined according to ambient brightness. For example, when the ambient brightness is less than or equal to the brightness threshold, the panchromatic pixels are exposed at a first exposure time equal to the second exposure time; when the ambient brightness is greater than the brightness threshold, the panchromatic pixels are exposed at a first exposure time less than the second exposure time time to expose. When the ambient brightness is greater than the brightness threshold, the relative relationship between the first exposure time and the second exposure time can be determined according to the brightness difference between the ambient brightness and the brightness threshold. The ratio of the two exposure times is smaller. For example, when the luminance difference is within the first range [a, b), the ratio of the first exposure time to the second exposure time is 1:2; when the luminance difference is within the second range [b, c) , the ratio of the first exposure time to the second exposure time is 1:3; when the luminance difference is greater than or equal to c, the ratio of the first exposure time to the second exposure time is 1:4.

Please refer to FIG. 1 and FIG. 21 , the present application also provides a control method. The control method of the embodiment of the present application can be used for the image sensor 10 described in any of the above-mentioned embodiments. Control methods include:

01: controlling the exposure of at least two sub-pixels 102 to output at least two sub-pixel information;

02: Determine phase information according to at least two sub-pixel information for focusing;

03: In the in-focus state, control the exposure of the two-dimensional pixel array 11 to obtain the target image.

Referring to FIG. 1 and FIG. 22 , the control method of the embodiment of the present application may be implemented by the camera assembly 40 of the embodiment of the present application. The camera assembly 40 includes the lens 30 , the image sensor 10 described in any one of the above embodiments, and the processing chip 20 . The image sensor 10 may receive light incident through the lens 30 and generate electrical signals. The image sensor 10 is electrically connected to the processing chip 20 . The processing chip 20 may be packaged with the image sensor 10 and the lens 30 in the housing of the camera assembly 40; or, the image sensor 10 and the lens 30 may be packaged in the housing of the camera assembly 40, and the processing chip 20 is disposed outside the housing. Step 01 , step 02 and step 03 can all be implemented by the processing chip 20 . That is to say, the processing chip 20 can be used to: control the exposure of at least two sub-pixels 102 to output at least two sub-pixel information; determine the phase information according to the at least two sub-pixel information to focus; in the focus state, control the two-dimensional pixel The array 11 is exposed to acquire the target image.

In the control method and the camera assembly 40 according to the embodiment of the present application, the two-dimensional pixel array 11 includes a plurality of color pixels and a plurality of panchromatic pixels. Compared with a general color sensor, the amount of light passing through is increased, and the information is better. Noise ratio, better focusing performance in low light, and higher sensitivity of sub-pixel 102. In addition, each pixel 101 includes at least two sub-pixels 102, which can improve the resolution of the image sensor 10 while achieving phase focus.

In addition, the control method and the camera assembly 40 of the embodiments of the present application do not require occlusion design for the pixels 101 in the image sensor 10, all the pixels 101 can be used for imaging, and do not need to perform dead pixel compensation, which is beneficial to improve the camera assembly 40 to obtain the quality of the target image.

In addition, all the pixels 101 including at least two sub-pixels 102 in the control method and the camera assembly 40 of the embodiments of the present application can be used for phase focusing, and the accuracy of the phase focusing is higher.

Specifically, the phase information may be a phase difference. Determining the phase information for focusing according to the at least two sub-pixel information includes: (1) calculating the phase difference according to only the panchromatic sub-pixel information for focusing; (2) calculating the phase difference according to only the color sub-pixel information for focusing; (3) At the same time, the phase difference is calculated according to the panchromatic sub-pixel information and the color sub-pixel information for focusing.

The control method and camera assembly 40 of the embodiment of the present application use the image sensor 10 including panchromatic pixels and color pixels to realize phase focusing, so that it can be used in an environment with low brightness (for example, the brightness is less than or equal to the first preset brightness). The panchromatic pixels with higher sensitivity are used for phase focusing, and the color pixels with lower sensitivity are used for phase focusing in an environment with high brightness (for example, the brightness is greater than or equal to the second preset brightness). In an environment where the first preset brightness is lower than the second preset brightness), at least one of panchromatic pixels and color pixels is used to perform phase focusing. In this way, it is possible to avoid the use of color pixels for phase focusing when the ambient brightness is low, and the problem of inaccurate focusing caused by the low signal-to-noise ratio of the color sub-pixel information output by the sub-pixels 102 in the color pixels can also be avoided. When it is high, panchromatic pixels are used for focusing. Due to the oversaturation of sub-pixels 102 in the panchromatic pixels, the focusing is inaccurate. As a result, the accuracy of phase focusing in various application scenarios is relatively high, and the scene of phase focusing is suitable for Sex is better.

Referring to Figures 1 and 11, in some embodiments, a panchromatic pixel includes two panchromatic sub-pixels. The panchromatic subpixel information includes first panchromatic subpixel information and second panchromatic subpixel information. The first panchromatic sub-pixel information and the second panchromatic sub-pixel information are respectively output by the panchromatic sub-pixels located at the first orientation of the lens 170 and the panchromatic sub-pixels located at the second orientation of the lens 170 . A piece of first panchromatic sub-pixel information and a corresponding piece of second panchromatic sub-pixel information are used as a pair of panchromatic sub-pixel information. The step of calculating the phase difference for focusing according to the panchromatic sub-pixel information includes: forming a first curve according to the first panchromatic sub-pixel information in the multiple pairs of panchromatic sub-pixel information; The second panchromatic sub-pixel information forms a second curve; and the phase difference is calculated according to the first curve and the second curve for focusing.

Specifically, referring to FIG. 23 , in an example, the first orientation P1 of each lens 170 is the position corresponding to the left half of the lens 170 , and the second orientation P2 is the position corresponding to the right half of the lens 170 . It should be noted that the first orientation P1 and the second orientation P2 shown in FIG. 23 are determined according to the distribution example of the sub-pixels 102 shown in FIG. 23 . For sub-pixels 102 with other types of distribution, the first orientation P1 and the second orientation P2 will correspondingly change. Corresponding to each panchromatic pixel W of the pixel array 11 in FIG. 23 , one subpixel 102 (ie the panchromatic subpixel W) is located at the first orientation P1 of the lens 170 , and the other subpixel 102 (ie the panchromatic subpixel W) is located at the first orientation P1 of the lens 170 . ) at the second orientation P2 of the lens 170 . The first panchromatic sub-pixel information is output by the panchromatic sub-pixel W located at the first orientation P1 of the lens 170 , and the second panchromatic sub-pixel information is output by the panchromatic sub-pixel W located at the second orientation P2 of the lens 170 . For example, the panchromatic sub-pixels W _11,P1 , W _13,P1 , W _15,P1 , W _17,P1 , W _22,P1 , W _24,P1 , W _26,P1 , W _28,P1 , etc. are located in the first orientation P1, the panchromatic sub-pixels W11 _,P2 , W13 _,P2 , W15,P2, W17, _P2 , W22 _{, P2} , W24,P2, W26, _P2 , W28 _{, P2} , etc. are located in the second position P2. Two panchromatic sub-pixels W in the same panchromatic pixel form a pair of panchromatic sub-pixels, and correspondingly, the panchromatic sub-pixel information of the two panchromatic sub-pixels in the same panchromatic pixel W form a pair of panchromatic sub-pixels. A pair of chromatic sub-pixel information, for example, the panchromatic sub-pixel information of the panchromatic sub-pixel W11 _{, P1} and the panchromatic sub-pixel information of the panchromatic sub-pixel W11 _{, P2} form a pair of panchromatic sub-pixel information, and the panchromatic sub-pixel information. The panchromatic sub-pixel information of the pixels W _{13 and P1} and the pan-chromatic sub-pixel information of the pan-chromatic sub-pixels W _{13 and P2} form a pair of pan-chromatic sub-pixel information, and the pan-chromatic sub-pixel information of the pan-chromatic sub-pixel W _{15 and P1} A pair of panchromatic sub-pixel information is formed with the panchromatic sub-pixel information of the panchromatic sub-pixels W _{15 and P2 .} The chromatic sub-pixel information forms a pair of panchromatic sub-pixel information equivalents, and so on.

After acquiring multiple pairs of panchromatic sub-pixel information pairs, the processing chip 20 forms a first curve according to the first panchromatic sub-pixel information in the multiple pairs of panchromatic sub-pixel information pairs, and according to the The second panchromatic sub-pixel information forms a second curve, and then calculates the phase difference according to the first curve and the second curve. For example, a plurality of first panchromatic sub-pixel information may describe a histogram curve (ie, a first curve), and a plurality of second panchromatic sub-pixel information may describe another histogram curve (ie, a second curve). Subsequently, the processing chip 20 may calculate the phase difference between the two histogram curves according to the positions of the peaks of the two histogram curves. Then, the processing chip 20 can determine the distance that the lens 30 needs to move according to the phase difference and the pre-calibrated parameters. Then, the processing chip 20 can control the lens 30 to move by a required distance so that the lens 30 is in a focus state.

Referring to Figures 1 and 11, in some embodiments, a panchromatic pixel includes two panchromatic sub-pixels. The panchromatic sub-pixel information includes first panchromatic sub-pixel information and second panchromatic sub-pixel information. The first panchromatic sub-pixel information and the second panchromatic sub-pixel information are respectively output by the panchromatic sub-image located at the first orientation of the lens 170 and the panchromatic sub-image located at the second orientation of the lens 170 . The plurality of first panchromatic sub-pixel information and the corresponding plurality of second panchromatic sub-pixel information are used as a pair of panchromatic sub-pixel information. Calculating the phase difference according to the panchromatic sub-pixel information for focusing includes: calculating third panchromatic sub-pixel information according to a plurality of first panchromatic sub-pixel information in each pair of panchromatic sub-pixel information; The fourth panchromatic sub-pixel information is calculated from the plurality of second panchromatic sub-pixel information in the pixel information pair; the first curve is formed according to the plurality of third panchromatic sub-pixel information; the first curve is formed according to the plurality of fourth panchromatic sub-pixel information. two curves; and calculating the phase difference according to the first curve and the second curve for focusing.

Specifically, referring to FIG. 23 , in an example, the first orientation P1 of each lens 170 is the position corresponding to the left half of the lens 170 , and the second orientation P2 is the position corresponding to the right half of the lens 170 . It should be noted that the first orientation P1 and the second orientation P2 shown in FIG. 23 are determined according to the distribution example of the sub-pixels 102 shown in FIG. 23 . For sub-pixels 102 with other types of distribution, the first orientation P1 and the second orientation P2 will correspondingly change. Corresponding to each panchromatic pixel of the pixel array 11 in FIG. 23 , one subpixel 102 (ie the panchromatic subpixel W) is located at the first orientation P1 of the lens 170 , and the other subpixel 102 (ie the panchromatic subpixel W) Located at the second orientation P2 of the lens 170 . The first panchromatic sub-pixel information is output by the panchromatic sub-pixel W located at the first orientation P1 of the lens 170 , and the second panchromatic sub-pixel information is output by the panchromatic sub-pixel W located at the second orientation P2 of the lens 170 . For example, the panchromatic sub-pixels W _11,P1 , W _13,P1 , W _15,P1 , W _17,P1 , W _22,P1 , W _24,P1 , W _26,P1 , W _28,P1 , etc. are located in the first orientation P1, the panchromatic sub-pixels W11 _,P2 , W13 _,P2 , W15,P2, W17, _P2 , W22 _{, P2} , W24,P2, W26, _P2 , W28 _{, P2} , etc. are located in the second position P2. A plurality of panchromatic sub-pixels W located in the first orientation P1 and a plurality of panchromatic sub-pixels W located in the second orientation P2 form a pair of panchromatic sub-pixels. Correspondingly, the information of the plurality of first panchromatic sub-pixels corresponds to The plurality of second panchromatic sub-pixel information is used as a pair of panchromatic sub-pixel information. For example, a plurality of first panchromatic sub-pixel information in the same subunit and a plurality of second panchromatic sub-pixel information in the subunit are used as a pair of panchromatic sub-pixel information, that is, the panchromatic sub-pixel W _{11 , P1} , W _22, the panchromatic sub-pixel information of P1 and the panchromatic sub-pixel information of pan-chromatic pixels W _{11, P2} , W _{22, P2} form a pair of pan-chromatic sub-pixel information pairs, the pan-chromatic sub-pixel W _{13, P1} , the panchromatic sub-pixel information of W _24,P1 and the panchromatic sub-pixel information of pan-chromatic sub-pixels W _13,P2 and W _24,P2 form a pair of pan-chromatic sub-pixel information, and the pan-chromatic sub-pixels W _15,P1 , The panchromatic sub-pixel information of W _26,P1 and the panchromatic sub-pixel information of pan-chromatic sub-pixels W _15,P2 and W _26,P2 form a pair of pan-chromatic sub-pixel information, and the pan-chromatic sub-pixels W _17,P1 and W _{28. The panchromatic sub-pixel information of P1} and the panchromatic sub-pixel information of the panchromatic sub-pixels W _17,P2 and W _28,P2 form a pair of panchromatic sub-pixel information, and so on. For another example, a plurality of first panchromatic sub-pixel information in the same minimum repeating unit and a plurality of second panchromatic sub-pixel information in the minimum repeating unit are used as a pair of panchromatic sub-pixel information, that is, a panchromatic sub-pixel. Panchromatic sub-pixel information of W11,P1, W13 _{, P1} , W22 _{, P1} , W24,P1, W31, _P1 , _W33,P1 , _W42,P1 , _W44,P1 and panchromatic pixel W _{11, P2} , W _{13, P2} , W _{22, P2} , W _{24, P2} , W _{31, P2} , W _{33, P12} , W _{42, P2} , W _{44, P2} The panchromatic sub-pixel information forms a pair of panchromatic sub-pixels Pixel information is equivalent, and so on.

After acquiring multiple pairs of panchromatic sub-pixel information pairs, the processing chip 20 calculates third panchromatic sub-pixel information according to the plurality of first panchromatic sub-pixel information in each pair of panchromatic sub-pixel information, and calculates third panchromatic sub-pixel information according to each pair of panchromatic sub-pixel information. The plurality of second panchromatic subpixel information in the subpixel information pair calculates fourth panchromatic subpixel information. For example, for the panchromatic sub-pixel information pair formed by the panchromatic pixel information of the panchromatic sub-pixels W _11,P1 , W _22,P1 and the panchromatic sub-pixel information of the pan-chromatic sub-pixels W _11,P2 , W _22,P2 , the calculation method of the third panchromatic sub-pixel information can be: LT1=W _11,P1 +W _22,P1 , and the calculation method of the fourth panchromatic sub-pixel information can be: RB1=W _11,P2 +W _22,P2 . Panchromatic sub-pixel information for panchromatic subpixels W11 _,P1 , W13 _,P1 , W22 _{,P1 ,} W24,P1, _W31,P1 , _W33,P1 , _W42,P1 , _W44,P1 It is composed of panchromatic sub-pixel information of panchromatic pixels W11 _,P2 , W13 _,P2 , W22 _{, P2} , W24,P2, W31 _,P2 , _W33,P12 , W42 _,P2 , _W44,P2 For a pair of panchromatic sub-pixel information, the calculation method of the third panchromatic sub-pixel information can be: LT1=(W _11,P1 +W _13,P1 +W _22,P1 +W _24,P1 +W _31,P1 + W _33,P1 +W _42,P1 +W _44,P1 )/8, the calculation method of the fourth panchromatic sub-pixel information can be: RB1=(W _11,P2 +W _13,P2 +W _22,P2 +W _24,P2 +W _31,P2 +W _33,P12 +W _42,P2 +W _44,P2 )/8. The calculation methods of the third panchromatic sub-pixel information and the fourth panchromatic sub-pixel information of the remaining panchromatic sub-pixel information pairs are similar, and will not be repeated here. In this way, the processing chip 20 can obtain a plurality of third panchromatic sub-pixel information and a plurality of fourth sub-panchromatic pixel information. The plurality of third panchromatic sub-pixel information may describe a histogram curve (ie, the first curve), and the plurality of fourth panchromatic sub-pixel information may describe another histogram curve (ie, the second curve). Subsequently, the processing chip 20 can calculate the phase difference according to the two histogram curves. Then, the processing chip 20 can determine the distance that the lens 30 needs to move according to the phase difference and the pre-calibrated parameters. Then, the processing chip 20 can control the lens 30 to move by a required distance so that the lens 30 is in a focus state.

Referring to Figures 1 and 11, in some embodiments, a color pixel includes two color sub-pixels. The color sub-pixel information includes first color sub-pixel information and second color sub-pixel information. The first color sub-pixel information and the second sub-pixel information are output by the color sub-pixels located at the first orientation of the lens 170 and the color sub-pixels located at the second orientation of the lens 170, respectively. A piece of first color sub-pixel information and a corresponding piece of second color sub-pixel information are used as a pair of color sub-pixel information. The step of calculating the phase difference for focusing according to the color sub-pixel information includes: forming a third curve according to the first color sub-pixel information in the multiple pairs of color sub-pixel information; The pixel information forms a fourth curve; and the phase difference is calculated according to the third curve and the fourth curve for focusing. This process is similar to the aforementioned process of only calculating the phase difference according to the panchromatic sub-pixel information for focusing, and will not be described again.

Referring to Figures 1 and 11, in some embodiments, a color pixel includes two color sub-pixels. The color sub-pixel information includes first color sub-pixel information and second color sub-pixel information. The first color sub-pixel information and the second color sub-pixel information are output by the color sub-pixels located at the first orientation of the lens 170 and the color sub-pixels located at the second orientation of the lens 170, respectively. The plurality of first color sub-pixel information and the corresponding plurality of second color sub-pixel information are used as a pair of color sub-pixel information. Calculating the phase difference according to the color sub-pixel information for focusing includes: calculating third color sub-pixel information according to a plurality of first color sub-pixel information in each pair of color sub-pixel information; calculating fourth color sub-pixel information according to the plurality of second color sub-pixel information; forming a third curve according to the plurality of third color sub-pixel information; forming a fourth curve according to the plurality of fourth color sub-pixel information; and according to the third curve and The fourth curve calculates the phase difference for focusing. This process is similar to the aforementioned process of only calculating the phase difference according to the panchromatic sub-pixel information for focusing, and will not be described again.

Referring to Figures 1 and 11, in some embodiments, a panchromatic pixel includes two panchromatic sub-pixels. A color pixel includes two color sub-pixels. The panchromatic subpixel information includes first panchromatic subpixel information and second panchromatic subpixel information, and the color subpixel information includes first color subpixel information and second color subpixel information. The first panchromatic sub-pixel information, the second panchromatic sub-pixel information, the first color sub-pixel information, and the second color sub-pixel information are respectively composed of the panchromatic sub-pixel located at the first orientation of the lens 170 and the The panchromatic sub-pixels in the two orientations, the color sub-pixels in the first orientation of the lens 170 , and the color sub-pixels in the second orientation of the lens 170 output. A first panchromatic sub-pixel information and a corresponding second panchromatic sub-pixel information are regarded as a pair of panchromatic sub-pixel information, and a first color sub-pixel information and a corresponding second color sub-pixel information are regarded as a pair of color Subpixel information pair. Calculating the phase difference for focusing according to the panchromatic sub-pixel information and the color sub-pixel information includes: forming a first curve according to the first panchromatic sub-pixel information in the multiple pairs of panchromatic sub-pixel information; The second full-color sub-pixel information in the information pair forms a second curve; the third curve is formed according to the first color sub-pixel information in the multiple pairs of color sub-pixel information pairs; the second color curve is formed according to the multiple pairs of color sub-pixel information pairs The sub-pixel information forms a fourth curve; and the phase difference is calculated according to the first curve, the second curve, the third curve, and the fourth curve for focusing. This process is similar to the aforementioned process of calculating the phase difference according to the panchromatic sub-pixel information for focusing, and calculating the phase difference according to the color sub-pixel information for focusing, and will not be described again.

Referring to Figures 1 and 11, in some embodiments, a panchromatic pixel includes two panchromatic sub-pixels. A color pixel includes two color sub-pixels. The panchromatic subpixel information includes first panchromatic subpixel information and second panchromatic subpixel information, and the color subpixel information includes first color subpixel information and second color subpixel information. The first panchromatic sub-pixel information, the second panchromatic sub-pixel information, the first color sub-pixel information, and the second color sub-pixel information are respectively composed of the panchromatic sub-pixel located at the first orientation of the lens 170 and the The panchromatic sub-pixels in the two orientations, the color sub-pixels in the first orientation of the lens 170 , and the color sub-pixels in the second orientation of the lens 170 output. The plurality of first panchromatic sub-pixel information and the corresponding plurality of second panchromatic sub-pixel information are regarded as a pair of panchromatic sub-pixel information, and the plurality of first color sub-pixel information and the corresponding plurality of second color sub-pixel information as a pair of color sub-pixel information pairs. Calculating the phase difference for focusing according to the panchromatic sub-pixel information and the color sub-pixel information includes: calculating third panchromatic sub-pixel information according to a plurality of first panchromatic sub-pixel information in each pair of panchromatic sub-pixel information; The fourth panchromatic sub-pixel information is calculated from the plurality of second panchromatic sub-pixel information in each pair of panchromatic sub-pixel information; the third color sub-pixel information is calculated according to the plurality of first color sub-pixel information in each pair of color sub-pixel information. sub-pixel information; calculating fourth color sub-pixel information according to a plurality of second color sub-pixel information in each pair of color sub-pixel information; forming a first curve according to a plurality of third panchromatic sub-pixel information; according to a plurality of fourth color sub-pixel information A second curve is formed according to the full-color sub-pixel information; a third curve is formed according to a plurality of third color sub-pixel information; a fourth curve is formed according to a plurality of fourth color sub-pixel information; The curve, and the fourth curve calculate the phase difference for focusing. This process is similar to the aforementioned process of calculating the phase difference according to the panchromatic sub-pixel information for focusing, and calculating the phase difference according to the color sub-pixel information for focusing, and will not be described again.

Referring to FIG. 1 and FIG. 24, in some embodiments, the control method further includes:

04: Get ambient brightness;

Step 03 In the in-focus state, control the exposure of the two-dimensional pixel array 11 to obtain the target image, including:

031: When the ambient brightness is greater than the first predetermined brightness, in the in-focus state, control the exposure of at least two sub-pixels 102 in each panchromatic pixel to output panchromatic sub-pixel information respectively, and control at least two sub-pixels 102 in each color pixel to be exposed. The sub-pixels 102 are exposed to output color sub-pixel information respectively;

032: Generate a target image according to the panchromatic sub-pixel information and the color sub-pixel information.

Referring to FIG. 1 and FIG. 22 , in some embodiments, step 04 , step 031 , and step 032 may all be implemented by the processing chip 10 . That is to say, the processing chip 20 can be used to: obtain the ambient brightness; when the ambient brightness is greater than the first predetermined brightness, in the in-focus state, control the exposure of at least two sub-pixels 102 in each full-color pixel to output full-color pixels respectively. Color sub-pixel information, controlling at least two sub-pixels 102 in each color pixel to be exposed to output color sub-pixel information respectively; generating a target image according to the panchromatic sub-pixel information and the color sub-pixel information.

Specifically, referring to FIG. 25 , it is taken as an example that each panchromatic pixel includes two panchromatic sub-pixels and each color pixel includes two color sub-pixels. The panchromatic sub-pixel information includes the first pan-sub-color pixel information and the third Two panchromatic sub-pixel information. The first panchromatic sub-pixel information and the second panchromatic sub-pixel information are respectively output by the panchromatic sub-pixel located at the first orientation P1 of the lens 170 and the panchromatic sub-pixel located at the second orientation P2 of the lens 170 (not shown in FIG. 25 ). The first orientation P1 and the second orientation P2 of the lens 170 are marked, which can be combined with the orientation division of the lens 170 in FIG. 23 ). In one example, the first orientation P1 of each lens 170 is the position corresponding to the left half of the lens 170 , and the second orientation P2 is the position corresponding to the right half of the lens 170 . In each panchromatic pixel W, one sub-pixel 102 (ie the panchromatic sub-pixel W) is located at the first orientation P1 of the lens 170 , and the other sub-pixel 102 (ie the panchromatic sub-pixel W) is located at the second orientation P2 of the lens 170 . In each panchromatic pixel W, the panchromatic sub-pixel W located in the first orientation P1 of the lens 170 and the panchromatic sub-pixel W located in the second orientation P2 of the lens 170 respectively output the first panchromatic sub-pixel information and Second panchromatic sub-pixel information. For example, the panchromatic sub-pixels W11, P1 and the panchromatic sub-pixels W11, P2 are exposed to output the first panchromatic sub-pixel information and the second panchromatic sub-pixel information, respectively, the panchromatic sub-pixels W22, P1 and the panchromatic sub-pixels W22, The P2 exposure outputs the first panchromatic sub-pixel information and the second panchromatic sub-pixel information, and so on.

Likewise, the color sub-pixel information includes first color sub-pixel information and second color sub-pixel information. The first color sub-pixel information and the second color sub-pixel information are respectively output by the color sub-pixel located at the first orientation P1 of the lens 170 and the color sub-pixel located at the second orientation P2 of the lens 170 (the lens 170 is not marked in FIG. 25 ). The first azimuth P1 and the second azimuth P2 can be divided in combination with the azimuth of the lens 170 in FIG. 23 ). In one example, the first orientation P1 of each lens 170 is the position corresponding to the left half of the lens 170 , and the second orientation P2 is the position corresponding to the right half of the lens 170 . In each color pixel, one sub-pixel 102 (ie, color sub-pixel A, color sub-pixel B, or color sub-pixel C) is located at the first orientation P1 of the lens 170, and the other sub-pixel 102 (ie, color sub-pixel A, color sub-pixel A, color sub-pixel C) is located at the first orientation P1 of the lens 170 . Pixel B or color sub-pixel C) is located at second orientation P2 of lens 170 . In each color pixel, the color sub-pixel located at the first orientation P1 of the lens 170 and the color sub-pixel located at the second orientation P2 of the lens 170 output first color sub-pixel information and second color sub-pixel information respectively after exposure. For example, color sub-pixels A12, P1 and color sub-pixels A12, P2 are exposed to output the first color sub-pixel information and second color sub-pixel information, respectively, and color sub-pixels B14, P1 and color sub-pixels B14, P2 are exposed to output the first color sub-pixel information, respectively. The color sub-pixel information and the second color sub-pixel information, the color sub-pixels C34, P1 and the color sub-pixels C34, P2 are exposed to output the first color sub-pixel information and the second color sub-pixel information, and so on.

In the embodiment of the present application, when the ambient brightness is greater than the first predetermined brightness (that is, under the condition of sufficient light), each full-color pixel and each color pixel are divided into multiple sub-pixels to output sub-pixel information respectively, which can improve the target The resolution of the image. For example, in Figure 25, each panchromatic pixel and each color pixel are split into two sub-pixels to output sub-pixel information respectively, and the horizontal resolution of the output target image can be doubled (when each panchromatic pixel is and each color pixel is divided into four sub-pixels distributed in a 2*2 matrix to output sub-pixel information respectively, then the horizontal resolution and vertical resolution of the output target image become twice the original, and the phase Focus performance is also better).

Referring to FIG. 1 and FIG. 24, in some embodiments, the control method further includes: 04: obtaining ambient brightness;

Step 03 In the in-focus state, control the exposure of the two-dimensional pixel array 11 to obtain the target image, including:

033: When the ambient brightness is less than the second predetermined brightness, in the focus state, control the exposure of at least two sub-pixels in each panchromatic pixel to output panchromatic sub-pixel information respectively, and control at least two sub-pixels in each color pixel to be exposed. Pixel exposure to merge output color merged pixel information;

034: Generate a target image according to the panchromatic sub-pixel information and the color combined pixel information.

Referring to FIG. 1 and FIG. 22 , in some embodiments, step 04 , step 033 , and step 034 may all be implemented by the processing chip 10 . That is to say, the processing chip 20 can be used to: obtain the ambient brightness; when the ambient brightness is less than the second predetermined brightness, in the in-focus state, control the exposure of at least two sub-pixels in each full-color pixel to output full-color respectively. The sub-pixel information controls the exposure of at least two sub-pixels in each color pixel to combine and output the color-combined pixel information; the target image is generated according to the full-color sub-pixel information and the color-combined pixel information.

Wherein, the second predetermined brightness is less than or equal to the first predetermined brightness.

Specifically, referring to FIG. 26 , taking an example that each panchromatic pixel includes two panchromatic sub-pixels and each color pixel includes two color sub-pixels, the panchromatic sub-pixel information includes the first pan-sub-color pixel information and the third Two panchromatic sub-pixel information. The first panchromatic sub-pixel information and the second panchromatic sub-pixel information are respectively output by the panchromatic sub-pixel located at the first orientation P1 of the lens 170 and the panchromatic sub-pixel located at the second orientation P2 of the lens 170 (not shown in FIG. 26 ). The first orientation P1 and the second orientation P2 of the lens 170 are marked, which can be combined with the orientation division of the lens 170 in FIG. 23 ). In one example, the first orientation P1 of each lens 170 is the position corresponding to the left half of the lens 170 , and the second orientation P2 is the position corresponding to the right half of the lens 170 . In each panchromatic pixel W, one sub-pixel 102 (ie the panchromatic sub-pixel W) is located at the first orientation P1 of the lens 170 , and the other sub-pixel 102 (ie the panchromatic sub-pixel W) is located at the second orientation P2 of the lens 170 . In each panchromatic pixel W, the panchromatic sub-pixel W located in the first orientation P1 of the lens 170 and the panchromatic sub-pixel W located in the second orientation P2 of the lens 170 respectively output the first panchromatic sub-pixel information and Second panchromatic sub-pixel information. For example, the panchromatic sub-pixels W11, P1 and the panchromatic sub-pixels W11, P2 are exposed to output the first panchromatic sub-pixel information and the second panchromatic sub-pixel information, respectively, the panchromatic sub-pixels W22, P1 and the panchromatic sub-pixels W22, The P2 exposure outputs the first panchromatic sub-pixel information and the second panchromatic sub-pixel information, and so on.

The color combination pixel information is combined and output by the color sub-pixels located at the first orientation P1 of the lens 170 and the color sub-pixels located at the second orientation P2 of the lens 170 (the first orientation P1 and the second orientation of the lens 170 are not marked in FIG. 26 ). P2, which can be combined with the azimuthal division of the lens 170 in FIG. 23). In one example, the first orientation P1 of each lens 170 is the position corresponding to the left half of the lens 170 , and the second orientation P2 is the position corresponding to the right half of the lens 170 . In each color pixel, one sub-pixel 102 (ie, color sub-pixel A, color sub-pixel B, or color sub-pixel C) is located at the first orientation P1 of the lens 170, and the other sub-pixel 102 (ie, color sub-pixel A, color sub-pixel A, color sub-pixel C) is located at the first orientation P1 of the lens 170 . Pixel B or color sub-pixel C) is located at second orientation P2 of lens 170 . In each color pixel, the color sub-pixels located at the first orientation P1 of the lens 170 and the color sub-pixels located at the second orientation P2 of the lens 170 are combined to output color combined pixel information after exposure. For example, color sub-pixel A12, P1 and color sub-pixel A12, P2 are exposed and combined to output color combined pixel information, color sub-pixel B14, P1 and color sub-pixel B14, P2 are exposed and combined to output color combined pixel information, color sub-pixel C34, P1 And color sub-pixel C34, P2 exposure combined output color combined pixel information, and so on.

Under the condition of low light, the sensitivity of panchromatic pixels is much higher than that of color pixels. If each color pixel is divided into multiple sub-pixels to output sub-pixel information separately, the signal-to-noise ratio of the image will be seriously reduced. Therefore, in the embodiment of the present application, when the ambient brightness is less than the second predetermined brightness (that is, under the condition of low light), each full-color pixel is divided into a plurality of sub-pixels to output sub-pixel information respectively. Multiple sub-pixels are combined to output combined pixel information, which can improve the resolution of the target image to a certain extent. For example, in FIG. 26, each panchromatic pixel is divided into two sub-pixels to output sub-pixel information respectively, and the lateral resolution of the output target image can be changed to 1.5 times the original.

Referring to FIG. 1 and FIG. 24, in some embodiments, the control method further includes: 04: obtaining ambient brightness;

Step 03 In the in-focus state, control the exposure of the two-dimensional pixel array 11 to obtain the target image, including:

035: When the ambient brightness is lower than the third predetermined brightness, in the focus state, control the exposure of at least two sub-pixels in each panchromatic pixel to combine and output full-color combined pixel information, and control at least two sub-pixels in each color pixel to be exposed. Pixel exposure to merge output color merged pixel information;

036: Generate a target image according to the full-color combined pixel information and the color combined pixel information.

Referring to FIG. 1 and FIG. 22 , in some embodiments, step 04 , step 035 , and step 036 may all be implemented by the processing chip 10 . That is to say, the processing chip 20 can be used to: obtain the ambient brightness; when the ambient brightness is less than the third predetermined brightness, in the in-focus state, control the exposure of at least two sub-pixels in each full-color pixel to combine and output the full-color The pixel information is combined, and the exposure of at least two sub-pixels in each color pixel is controlled to combine and output the color combined pixel information; a target image is generated according to the full-color combined pixel information and the color combined pixel information.

Wherein, the third predetermined brightness is smaller than the second predetermined brightness.

Specifically, referring to FIG. 27 , taking each panchromatic pixel including two panchromatic sub-pixels and each color pixel including two color sub-pixels as an example, the panchromatic combined pixel information is determined by the pixel located at the first orientation P1 of the lens 170 . The combined output of the panchromatic sub-pixels and the panchromatic sub-pixels located at the second orientation P2 of the lens 170 (the first orientation P1 and the second orientation P2 of the lens 170 are not marked in FIG. 27 , can be combined with the orientation division of the lens 170 in FIG. ). In one example, the first orientation P1 of each lens 170 is the position corresponding to the left half of the lens 170 , and the second orientation P2 is the position corresponding to the right half of the lens 170 . In each panchromatic pixel W, one sub-pixel 102 (ie the panchromatic sub-pixel W) is located at the first orientation P1 of the lens 170 , and the other sub-pixel 102 (ie the panchromatic sub-pixel W) is located at the second orientation P2 of the lens 170 . In each panchromatic pixel W, the panchromatic sub-pixel W located in the first orientation P1 of the lens 170 and the panchromatic sub-pixel W located in the second orientation P2 of the lens 170 are combined to output panchromatic combined pixel information after exposure. For example, the panchromatic sub-pixels W11, P1 and the panchromatic sub-pixels W11, P2 are exposed and combined to output the full-color combined pixel information, and the pan-color sub-pixels W22, P1 and the pan-chromatic sub-pixels W22, P2 are exposed and combined to output the full-color combined pixel information, So on and so forth.

The color combination pixel information is combined and output by the color sub-pixels located at the first orientation P1 of the lens 170 and the color sub-pixels located at the second orientation P2 of the lens 170 (the first orientation P1 and the second orientation of the lens 170 are not marked in FIG. 27 ). P2, which can be combined with the azimuthal division of the lens 170 in FIG. 23). In one example, the first orientation P1 of each lens 170 is the position corresponding to the left half of the lens 170 , and the second orientation P2 is the position corresponding to the right half of the lens 170 . In each color pixel, one sub-pixel 102 (ie, color sub-pixel A, color sub-pixel B, or color sub-pixel C) is located at the first orientation P1 of the lens 170, and the other sub-pixel 102 (ie, color sub-pixel A, color sub-pixel A, color sub-pixel C) is located at the first orientation P1 of the lens 170 . Pixel B or color sub-pixel C) is located at second orientation P2 of lens 170 . In each color pixel, the color sub-pixels located at the first orientation P1 of the lens 170 and the color sub-pixels located at the second orientation P2 of the lens 170 are combined to output color combined pixel information after exposure. For example, color sub-pixel A12, P1 and color sub-pixel A12, P2 are exposed and combined to output color combined pixel information, color sub-pixel B14, P1 and color sub-pixel B14, P2 are exposed and combined to output color combined pixel information, color sub-pixel C34, P1 And color sub-pixel C34, P2 exposure combined output color combined pixel information, and so on.

Under extremely low light conditions, the signals of both panchromatic and color pixels are low, and the pursuit of image resolution is of little significance. Therefore, in the embodiment of the present application, when the ambient brightness is less than the third predetermined brightness (that is, under extremely dark conditions), each panchromatic pixel is divided into a plurality of sub-pixels to output sub-pixel information respectively, and each panchromatic pixel and Multiple sub-pixels in each color pixel are combined to output combined pixel information, so as to increase the signal quantity and improve the signal-to-noise ratio.

Expose the two-dimensional pixel array 11 to output an original image including panchromatic sub-pixel information and color combined pixel information (as shown in FIG. 25 ), or output an original image including panchromatic combined pixel information and color combined pixel information (as shown in FIG. 26 ) shown), or after outputting the original image including the full-color combined pixel information and the color combined pixel information (as shown in FIG. 27 ), the processing chip 20 can also correspond to the original image for these three original images of different resolutions The demosaicing algorithm (such as bilinear interpolation algorithm) is processed to complete the pixel information or sub-pixel information of each channel (for example, including red channel, green channel and blue channel), maintain the complete presentation of image color, and finally obtain color target image. In one example, in the process of obtaining the target image from the original image, the processing chip 20 may further perform black level correction processing, lens shadow correction processing, dead pixel compensation processing, color correction processing, global tone mapping processing and Any one or more of the color conversion processes to have a better image.

Please refer to FIG. 1 , FIG. 22 and FIG. 28 , the present application further provides a mobile terminal 90 . The mobile terminal 90 in the embodiment of the present application may be a mobile phone, a tablet computer, a notebook computer, a smart wearable device (such as a smart watch, a smart bracelet, a smart glasses, a smart helmet, etc.), a head-mounted display device, a virtual reality device, etc. No restrictions. The mobile terminal 90 according to the embodiment of the present application includes the camera assembly 40 , the processor 60 , the memory 70 and the casing 80 according to any of the foregoing embodiments. The camera assembly 40 , the processor 60 and the memory 70 are all mounted on the casing 80 . The image sensor 10 in the camera assembly 40 is connected to the processor 60 . The processor 60 can perform the same functions as the processing chip 20 in the camera assembly 40 , in other words, the processor 60 can implement the functions that can be implemented by the processing chip 20 in any of the foregoing embodiments. The memory 70 is connected to the processor 60, and the memory 70 can store data obtained after processing by the processor 60, such as a target image and the like. The processor 60 and the image sensor 10 may be mounted on the same substrate, and in this case, the image sensor 10 and the processor 60 may be regarded as a camera assembly 40 . Of course, the processor 60 may also be mounted on a different substrate from the image sensor 10 .

In the mobile terminal 90 according to the embodiment of the present application, the two-dimensional pixel array 11 includes a plurality of color pixels and a plurality of panchromatic pixels. Compared with a general color sensor, the light throughput is increased, and the signal-to-noise ratio is better. The focusing performance is better in low light, and the sensitivity of the sub-pixel 102 will be higher. In addition, each pixel 101 includes at least two sub-pixels 102, which can improve the resolution of the image sensor 10 while achieving phase focus.

In the description of this specification, reference to the terms "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" or the like is meant to be used in conjunction with the described embodiments. A particular feature, structure, material, or characteristic described in a manner or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

Any description of a process or method in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a specified logical function or step of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and alterations.

Claims (19)

1. An image sensor, comprising:

a two-dimensional array of pixels comprising a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels; the two-dimensional pixel array includes a minimal repeating unit in which the panchromatic pixels are disposed in a first diagonal direction and the color pixels are disposed in a second diagonal direction, the first diagonal direction being different from the second diagonal direction; wherein each pixel comprises at least two sub-pixels; and

a lens array comprising a plurality of lenses, each lens covering one of the pixels.

2. The image sensor of claim 1, wherein each of the pixels comprises two of the sub-pixels, and a boundary between the two sub-pixels is parallel to a length direction of the two-dimensional pixel array; or

A boundary between two of the sub-pixels is parallel to a width direction of the two-dimensional pixel array.

3. The image sensor according to claim 1, wherein each of the pixels includes two of the sub-pixels, and a boundary between the two sub-pixels is inclined with respect to a length direction or a width direction of the two-dimensional pixel array.

4. The image sensor of claim 1, wherein each of the pixels comprises two of the sub-pixels, and a boundary between the two sub-pixels in some of the pixels is parallel to a length direction of the two-dimensional pixel array; in some of the pixels, a boundary between two of the sub-pixels is parallel to a width direction of the two-dimensional pixel array.

5. The image sensor according to claim 4, wherein the pixels whose boundary between two of the sub-pixels is parallel to a longitudinal direction of the two-dimensional pixel array and the pixels whose boundary between two of the sub-pixels is parallel to a width direction of the two-dimensional pixel array are distributed alternately or in columns.

6. The image sensor according to claim 4, wherein the pixels whose boundary between two of the sub-pixels is parallel to a length direction of the two-dimensional pixel array and the pixels whose boundary between two of the sub-pixels is parallel to a width direction of the two-dimensional pixel array are distributed alternately.

7. The image sensor of claim 1, wherein each of the pixels comprises four of the sub-pixels, and the four sub-pixels are arranged in a 2 x 2 matrix.

8. The image sensor of claim 1, further comprising a filter array comprising a plurality of filters, each of the filters covering one of the pixels.

9. The image sensor according to claim 1, wherein each of the pixels includes two of the sub-pixels, one of the sub-pixels includes a first photoelectric conversion element, and the other of the sub-pixels includes a second photoelectric conversion element, each of the pixels further includes a first exposure control circuit connected to the first photoelectric conversion element for transferring electric charges generated by the first photoelectric conversion element receiving light to output pixel information, and a second exposure control circuit connected to the second photoelectric conversion element for transferring electric charges generated by the second photoelectric conversion element receiving light to output pixel information.

10. The image sensor of claim 9, wherein each of the pixels further comprises a reset circuit, the reset circuit being simultaneously connected to the first exposure control circuit and the second exposure control circuit;

when the two sub-pixels are exposed to respectively output pixel information or respectively output the pixel information and output phase information, the first exposure control circuit transfers the electric charge generated after the first photoelectric conversion element receives light to output first sub-pixel information, and after the reset circuit is reset, the second exposure control circuit transfers the electric charge generated after the second photoelectric conversion element receives light to output second sub-pixel information;

when the two sub-pixels are exposed to combine and output pixel information, the first exposure control circuit transfers the charges generated after the first photoelectric conversion element receives light, and the second exposure control circuit can transfer the charges generated after the second photoelectric conversion element receives light to output combined pixel information;

when the two sub-pixels are exposed to combine and output pixel information and output phase information, the first exposure control circuit transfers charges generated after the first photoelectric conversion element receives light to output first sub-pixel information, after the reset circuit is reset, the second exposure control circuit transfers charges generated after the second photoelectric conversion element receives light to output second sub-pixel information, and the first sub-pixel information and the second sub-pixel information are used for being combined into combined pixel information.

11. The image sensor of claim 10, wherein when two of the sub-pixels are exposed to combine output pixel information and output phase information, the image sensor further comprises a buffer for storing the first sub-pixel information and the second sub-pixel information to output phase information.

12. A control method for an image sensor comprising a two-dimensional array of pixels comprising a plurality of color pixels and a plurality of panchromatic pixels, the color pixels having a narrower spectral response than the panchromatic pixels, and a lens array; the two-dimensional pixel array includes a minimal repeating unit in which the panchromatic pixels are disposed in a first diagonal direction and the color pixels are disposed in a second diagonal direction, the first diagonal direction being different from the second diagonal direction; wherein each pixel comprises at least two sub-pixels; the lens array comprises a plurality of lenses, each lens covering one of the pixels; the control method comprises the following steps:

controlling the at least two sub-pixel exposures to output at least two sub-pixel information;

determining phase information according to the at least two pieces of sub-pixel information to carry out focusing;

and under the in-focus state, controlling the two-dimensional pixel array to be exposed so as to acquire a target image.

13. The control method according to claim 12, characterized by further comprising:

obtaining the ambient brightness;

in a focusing state, controlling the exposure of the two-dimensional pixel array to acquire a target image includes:

when the ambient brightness is larger than a first preset brightness, in an in-focus state, controlling the at least two sub-pixel exposures in each panchromatic pixel to respectively output panchromatic sub-pixel information, and controlling the at least two sub-pixel exposures in each color pixel to respectively output color sub-pixel information;

and generating the target image according to the full-color sub-pixel information and the color sub-pixel information.

14. The control method according to claim 12, characterized by further comprising:

obtaining the ambient brightness;

in a focusing state, controlling the exposure of the two-dimensional pixel array to acquire a target image includes:

when the ambient brightness is smaller than a second preset brightness, in an in-focus state, controlling the at least two sub-pixel exposures in each panchromatic pixel to respectively output panchromatic sub-pixel information, and controlling the at least two sub-pixel exposures in each color pixel to jointly output color merged pixel information;

and generating the target image according to the full-color sub-pixel information and the color merging pixel information.

15. The control method according to claim 12, characterized by further comprising:

obtaining the ambient brightness;

in a focusing state, controlling the exposure of the two-dimensional pixel array to acquire a target image includes:

controlling the at least two sub-pixel exposures in each of the panchromatic pixels to merge output panchromatic merged pixel information and controlling the at least two sub-pixel exposures in each of the color pixels to merge output color merged pixel information in an in-focus state when the ambient brightness is less than a third predetermined brightness;

generating the target image from the panchromatic merged pixel information and the color merged pixel information.

16. A camera head assembly, comprising:

a lens; and

the image sensor of any of claims 1-11, said image sensor capable of receiving light passing through said lens.

17. A camera assembly according to claim 16, further comprising a processing chip for implementing the control method of any one of claims 12 to 15.

18. A mobile terminal, comprising:

a housing; and

a camera assembly according to claim 16 or 17, mounted on the housing.

19. A mobile terminal, comprising:

a housing; and

the camera assembly of claim 16, said camera assembly mounted on said housing;

the mobile terminal further comprises a processor for implementing the control method of any one of claims 12-15.

Images