TWI635748B - Image sensing method and device thereof - Google Patents

Abstract

An image sensing method and apparatus are provided. The image sensing device includes an image capturing circuit and an image processor. The image capture circuit has a plurality of pixel photoreceptors. The pixel photoreceptor is divided into at least a plurality of groups of pixels, and each of the pixel photoreceptors includes at least one full spectrum photoreceptor. The image processor is coupled to the image capturing circuit to obtain ambient brightness through the sensing result of the ambient light source, and calculate a plurality of exposure values according to the ambient brightness, wherein each exposure value is different from each other. The image processor sets the exposure time of the full spectrum photoreceptor of the pixel photoreceptor in each group of pixel groups to each exposure value, thereby capturing an image.

Description

Image sensing method and device thereof

The present invention relates to a sensing technique, and more particularly to an image sensing method and apparatus therefor.

In recent years, due to the vigorous development of multimedia applications, the demand for image sensing devices has gradually increased. Image sensing devices such as digital cameras, driving recorders, webcams, mobile phones, and tablets all use photosensitive elements to capture images. At present, the photosensitive elements of the mainstream image sensing device include a charge coupled device (CCD) and a complementary CMOS active pixel sensor. Both of these photosensitive elements capture images by receiving light and converting it into an electronic signal.

In order to meet the needs of the user for the improvement of the pixels and the thinning of the image sensing device, the image capturing circuit of the photosensitive element must also reduce the size of the pixel photoreceptor as the pixel is improved or the device is thinned. However, as the size of the pixel photoreceptor is reduced, the amount of light entering is also reduced. This phenomenon can cause insufficient sensitivity of the image sensing device in a low light source environment. In order to increase the amount of light entering the image sensing device, RWB (red, white, and blue) pixel photoreceptors equipped with a full spectrum photoreceptor (ie, a white photoreceptor) can be used. However, the use of full spectrum photoreceptors also poses some problems. How to improve the problems caused by the full spectrum photoreceptor is one of the goals of various manufacturers' efforts.

The invention provides an image sensing method and a device thereof, which improve the brightness (luminance, or luma) and contrast of an image sensed by an image sensing device with a full spectrum photoreceptor, and obtain High light input, high dynamic contrast and brighter color output images.

The image sensing device of the present invention includes an image capturing circuit and an image signal processor (ISP). The image capture circuit has a plurality of pixel photoreceptors. The pixel photoreceptor is divided into at least a plurality of groups of pixels, and each of the pixel photoreceptors includes at least one full spectrum photoreceptor. The image processor is coupled to the image capturing circuit, and obtains ambient brightness through the sensing result of the ambient light source, and calculates a plurality of exposure values according to the ambient brightness, wherein each exposure value is different from each other. The image processor sets the exposure time of the full spectrum photoreceptor of the pixel photoreceptor in each group of pixel groups to each exposure value, thereby capturing an image.

The image sensing method of the present invention comprises the steps of: sensing an ambient light source using a plurality of pixel photoreceptors, and obtaining an ambient brightness through a sensing result of the ambient light source. The pixel photoreceptors are divided into at least a plurality of groups of pixels, and each of the pixel photoreceptors includes at least one full spectrum photoreceptor. A plurality of exposure values are calculated based on the ambient brightness, wherein each exposure value is different from each other. The exposure time of the full-spectrum photoreceptor of the pixel photoreceptor in each group of pixel groups is set to each exposure value, respectively, to capture an image.

Based on the above, the image sensing method and apparatus of the present invention divides the pixel photoreceptors into different groups and sets different exposure values to the full spectrum photoreceptors in each group of pixel groups. By reducing the exposure time of the full spectrum photoreceptor in some pixel groups, the overall amount of light entering the full spectrum photoreceptor can be reduced, so that the brightness does not reach saturation too quickly (in other words, avoiding too early exposure). The contrast of the image. Thus, even if the RWB photoreceptor sensitive to brightness is used in a strong light source environment, the image captured by the image sensing device is not overexposed as a whole. In addition, the embodiment of the present invention sets the full spectrum photoreceptors in different pixel groups to different exposure values, and can simultaneously retain the high brightness details and low brightness information of the captured images. Therefore, the embodiment of the invention can improve the brightness, saturation, color shift and contrast of the output image of the image sensing device, and obtain an output image with high light input, high dynamic contrast and bright colors.

The above described features and advantages of the invention will be apparent from the following description.

A conventional pixel photoreceptor is a Bayer filter, that is, an RGB filter lens, arranged above a pixel photoreceptor array of an image capture circuit. Each pixel photoreceptor can capture one of the three primary colors in the external light source according to the filter lens corresponding to the primary color, and record the brightness of the primary color. The RWB pixel photoreceptor uses a full-spectrum photoreceptor instead of the green photoreceptor in the RGB pixel photoreceptors (red, green, and blue) to enhance the brightness of the image sensing device in low-light environments. For example, a full-spectrum photoreceptor receives more light than a green photoreceptor, so it is easier to reach the highest brightness that the photoreceptor can sense faster. In other words, the full-spectrum photoreceptor has more light. It is easy to achieve saturation, which makes it difficult for the brightness sensed by the full-spectrum photoreceptor to be accurately evaluated to cause color shift, and the extracted image will lose some contrast. In addition, when the image sensing device determines that the color captured by the photoreceptor has reached saturation, the image sensing device may reduce the compensation gain of the color, resulting in a relatively unsaturated overall color of the output image.

FIG. 1 is a schematic diagram of an image sensing apparatus 100 in accordance with an embodiment of the present invention. The image sensing device 100 can include an image capturing circuit 120 and an image processor 140. The image processor 140 is coupled to the image capturing circuit 120. The image sensing device 100 can be, for example, a device that requires image capture, such as a digital camera, a driving recorder, a web camera, a mobile phone, or a tablet computer. The image capture circuit 120 can be, for example, a charge coupled device (CCD) or a complementary CMOS active pixel sensor. The image processor 140 can be, for example, a central processing unit (CPU), a micro-processor, or an embedded controller having image processing functions.

2A is a schematic diagram of an image capture circuit 120 in accordance with an embodiment of the present invention. The image capturing circuit 120 includes a plurality of pixel photoreceptors 121, which may be, for example, photosensitive pixels on which RWB (red, white, and blue) filter lenses are mounted. In an embodiment of the invention, each pixel photoreceptor comprises at least one full spectrum photoreceptor. For example, in the embodiment of FIG. 2A, the pixel photoreceptor 121 is composed of a red photoreceptor R, a full spectrum photoreceptor W, and a blue photoreceptor B. In the embodiment of FIG. 2B, the pixel photoreceptor 123 is composed of one red photoreceptor R, two full-spectrum photoreceptors W, and one blue photoreceptor B. The arrangement pattern of the pixel photoreceptors can be adjusted by the user according to actual needs, and the present invention is not limited thereto.

The plurality of pixel photoreceptors can be divided into at least a plurality of groups of pixels. For example, in the embodiment of FIG. 2A, a plurality of pixel photoreceptors 121 are divided into three groups of pixels, including T1, T2, and T3. In the embodiment of FIG. 2B, a plurality of pixel photoreceptors 123 are divided into two groups of pixels, including T1 and T2. The number of pixel groups can be adjusted by the user according to actual needs, and the present invention is not limited thereto. The manner of grouping may be, for example, grouping the pixel photoreceptors in units of rows or columns. For example, the embodiment of Figure 2A groups pixel photoreceptors 121 in units of rows. In the image capturing circuit 120, the first row, the second row, and the third row of pixel photoreceptors 121 are respectively divided into groups of T1, T2, and T3. Next, starting from the fourth line, the pixel photoreceptor 121 assigns the groups in the order of T1, T2, and T3. The steps are repeated as such until all of the pixel photoreceptors 121 are assigned to one of the groups. The user can also adjust the manner of grouping according to actual needs, and the present invention is not limited thereto. For example, the embodiment of Figure 2B groups the sub-pixel photoreceptors in pixel photoreceptor 123 in row units. The first line configured with the red photoreceptor R and the full spectrum photoreceptor W is divided into T1 groups, and the second line configured with the blue photoreceptor B and the full spectrum photoreceptor W is divided into T2 groups. Next, starting from the third line, the pixel photoreceptor 123 assigns the groups in the order of T1 and T2. The steps are repeated as such until all of the pixel photoreceptors 123 are assigned to one of the groups.

Please refer to FIG. 1 and FIG. 2A at the same time. Taking the pixel sensor 121 in FIG. 2A as an example, when image sensing is started, the image sensing device 100 first senses the brightness of the ambient light source through the image capturing circuit 120. At this time, in the image capturing circuit 120, each pixel photoreceptor 121 uses the same exposure value, which may be determined, for example, by a preset manner. Then, the image processor 140 obtains the ambient brightness through the sensing result AL of the ambient light source, and calculates a plurality of exposure values according to the ambient brightness. The respective exposure values are different from each other, and the number of exposure values depends on the number of groups of pixel groups. In this embodiment, the image processor 140 calculates three exposure values, a first exposure value V1, a second exposure value V2, and a third exposure value V3, and V1 is greater than V2, and V2 is greater than V3. The method of calculating the exposure value may be, for example, setting the second exposure value V2 to one-half of the first exposure value V1 and setting the third exposure value V3 to one-fourth of the first exposure value V1. Limited to this. Next, the image processor 140 sets the exposure time of the full-spectrum photoreceptor W of the pixel photoreceptor 121 in each group of pixels to the respective exposure values, thereby capturing the image.

When the exposure value is set, each pixel photoreceptor 121 is divided into a first pixel group T1 corresponding to the first exposure value V1, a second pixel group T2 corresponding to the second exposure value V2, and a third exposure value V3. The third pixel group T3. The T1 group has the longest exposure time, the T2 group has a moderate exposure time, and the T3 group has the shortest exposure time. In each of the pixel photoreceptors 121, except that the exposure time of the full-spectrum photoreceptor W needs to be set to a different exposure value in accordance with the grouping, the exposure times of the other primary color photoreceptors can be set to the same exposure value. For example, in this embodiment, the exposure times of the red photoreceptor R and the blue photoreceptor B of each of the pixel photoreceptors 121 are set to the first exposure value V1. Since the full-spectrum photoreceptor absorbs more light than the conventional RGB photoreceptor, the full-spectrum photoreceptor W is sensitive to strong light, and the amount of light entering the saturation is shorter than that of the other primary color photoreceptors. . In this embodiment, group exposure of the full spectrum photoreceptor W can improve the overall exposure level of the image captured by the full spectrum photoreceptor.

3A and 3B are schematic diagrams showing the relationship between brightness and time of each color photoreceptor in the pixel photoreceptor 121 according to an embodiment of the present invention. Referring first to FIG. 3A, the luminance of the vertical axis of FIG. 3A represents the range of luminance that the pixel photoreceptor 121 can sense. Taking the RGB color model as an example, the brightness of each primary color is divided into 256 values (0 to 255) according to 8-bit, and when the brightness of a primary color reaches the highest value of 255, The amount of light entering the primary color has reached the highest brightness that the pixel photoreceptor 121 can sense, that is, saturation. When the amount of light entering the primary color is saturated, the brightness information of the primary color can only be stored in the pixel photoreceptor 121 at the highest value of 255, and thus the brightness of the captured image is distorted. In the traditional RGB photoreceptor, the photoreceptor of each primary color has little difference in brightness. Take Figure 3A as an example. Assume that the exposure time of each primary color photoreceptor is 33 milliseconds (ms), and the conventional RGB photoreceptor is in 33 milliseconds. When the amount of light entering the photoreceptors of each primary color has not yet reached saturation, the brightness information of each primary color is completely captured, and the brightness distortion does not occur. However, when using a RWB photoreceptor with a full-spectrum photoreceptor, the full-spectrum photoreceptor requires a lower exposure level than the red photoreceptor due to the large spectral range of light absorbed by the full-spectrum photoreceptor. The exposure value required for R and blue photoreceptor B when the amount of incoming light reaches saturation. As shown in FIG. 3A, assuming that the full-spectrum photoreceptor W with an exposure time of 33 milliseconds is represented as W1, it can be seen that W1 reaches saturation as early as 25 milliseconds, and thus excessive exposure of luminance distortion occurs at 33 milliseconds. phenomenon.

In order to solve the overexposure phenomenon of the RWB photoreceptor in a high-brightness environment, the present embodiment divides the exposure values of the full-spectrum photoreceptor W into three groups. The exposure value of each full spectrum photoreceptor W1 in the T1 group is set to V1. The exposure value of each full spectrum photoreceptor W2 in the T2 group is set to V2. The exposure value of each full spectrum photoreceptor W3 in the T3 group is set to V3. It is assumed that V1, V2, and V3 are set to 33 milliseconds, 16 milliseconds, and 8 milliseconds, respectively. In the T1 group, the amount of light entering the respective full-spectrum photoreceptors W1 is saturated at 25 msec, so that the brightness distortion occurs and some contrast is lost. However, since the full-spectrum photoreceptor W1 uses the same exposure value V1 as the red photoreceptor R and the blue photoreceptor B, the full-spectrum photoreceptor W1 maintains sufficient brightness compared to the full-spectrum photoreceptors W2 and W3. In the T2 group, each full spectrum photoreceptor W2 maintains only an exposure time of 16 milliseconds. At 16 milliseconds, the full spectrum photoreceptor W2 stops exposure and waits for the remaining primary color photoreceptors to complete the 33 millisecond exposure. The exposure time is shortened so that the brightness of the full-spectrum photoreceptor W2 is reduced, and the relationship between the brightness and time can be equivalent to the W2 curve in Fig. 3A. In this embodiment, the amount of light entering the full spectrum photoreceptor W2 of the T2 group does not reach saturation during the exposure time, so no brightness distortion occurs, but compared to the red photoreceptor R and the blue photoreceptor B. The full spectrum photoreceptor W2 still retains more brightness information, which can make the dark part of the captured image have a brighter effect. By reducing the overall overexposure of the full spectrum photoreceptor W, the T2 group can improve the color cast, contrast, and saturation of the captured image. In the T3 group, each full-spectrum photoreceptor W3 maintains only an exposure time of 8 milliseconds, and its brightness versus time can be equivalent to the W3 curve in FIG. 3A, and W3 is the curve with the lowest luminance response.

When all of the primary color photoreceptors have been exposed, the image processor 140 sums up the image information captured by each of the full spectrum photoreceptors W1, W2, and W3 in the T1, T2, and T3 groups. It should be noted that in order to make the image after the addition can retain the image information when no overexposure occurs, one or more of the exposure values of each group are smaller than the full spectrum photoreceptor W. The exposure value required for saturation. By summing up the image information captured by each full-spectrum photoreceptor, the overall brightness curve of the full-spectrum photoreceptor W as shown in FIG. 3B can be obtained, and the full-spectrum photoreceptor W maintains the RWB photoreceptor. High brightness. At the same time, this embodiment avoids the situation that all the full-spectrum photoreceptors W are overexposed in a high-brightness environment, and the problem of color shift of the output image, insufficient contrast, and low saturation is improved.

FIG. 4 is a flow chart of image sensing by the image sensing device 100 according to an embodiment of the invention. In step S401, the image sensing device 100 determines whether to capture an image using the method proposed by the present invention. To capture an image using the method proposed by the present invention, the process proceeds to step S403. If the image proposed by the present invention is not used to capture an image, the process proceeds to step S411. In step S403, the image capturing circuit 120 senses the ambient light source using the plurality of pixel photoreceptors 121, and the image processor 140 obtains the ambient brightness through the sensing result AL of the ambient light source. In step S405, each pixel photoreceptor 121 is divided into at least a plurality of groups of pixels, and each of the pixel photoreceptors 121 includes at least one full spectrum photoreceptor W. The order of steps S401, S403, and S405 can be adjusted according to the needs of the user. In step S407, the image processor 140 calculates a plurality of exposure values according to the ambient brightness, wherein the respective exposure values are different from each other, and the image processor 140 and the full spectrum photoreceptor W of the pixel photoreceptor 121 in each group of pixel groups The exposure time is set to each exposure value. In step S409, the image capturing device 100 captures an image using the exposure value set in the foregoing step. In step S411, when the image sensing device 100 determines not to use the method proposed by the present invention to capture an image, the image sensing device 100 sets the exposure time of the full-spectrum photoreceptor W of the pixel photoreceptor 121 to An exposure value calculated based on the ambient brightness is set, and then proceeds to step S509 to capture an image.

In summary, the image sensing method and apparatus of the present invention can group pixel photoreceptors and set different exposure values to the full spectrum photoreceptors in each group of pixels, by reducing the full spectrum photoreceptor. The overall exposure time can make the amount of light entering the full spectrum photoreceptor not to be too fast to reach saturation, thus affecting the contrast of the image. Thus, even if the RWB photoreceptor sensitive to brightness is used in a strong light source environment, the image captured by the image sensing device is not overexposed. In addition, the present invention can preserve high-brightness details and low-brightness information of captured images by capturing images at different exposure values. Compared with the conventional image sensing technology, the invention can improve the brightness, saturation, color shift and contrast of the output image of the image sensing device, and obtain high brightness, high dynamic contrast and bright colors. Output image.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

100: image sensing device 120: image capturing circuit 121, 123: pixel photoreceptor 140: image processor V1, V2, V3: exposure value AL: sensing result R: red photoreceptor G: green light sensing B: blue photoreceptor W: full spectrum photoreceptor (white light sensor) T1, T2, T3: pixel photoreceptor group W1, W2, W3: full spectrum photoreceptors S401, S403, S405 corresponding to specific exposure values, S407, S409, S411: steps

1 is a schematic diagram of an image sensing device in accordance with an embodiment of the present invention. 2A and 2B are schematic diagrams showing an image capturing circuit in accordance with an embodiment of the present invention. 3A and 3B are schematic diagrams showing the relationship between brightness and time of each color photoreceptor in a pixel photoreceptor according to an embodiment of the invention. FIG. 4 is a flow chart of image sensing performed by the image sensing apparatus 100 in accordance with an embodiment of the present invention.

Claims (10)

An image sensing device includes: an image capturing circuit having a plurality of pixel photoreceptors, wherein the pixel photoreceptors are at least divided into a plurality of groups of pixels, and each of the pixel photoreceptors includes at least one full The image sensor is coupled to the image capture circuit, wherein the image processor obtains ambient brightness through the sensing result of the ambient light source, and calculates a plurality of exposure values according to the ambient brightness, wherein each exposure value is mutually Differently, the image processor sets the exposure times of the full-spectrum photoreceptors of the pixel photoreceptors in each group of pixel groups to respective exposure values, thereby capturing images, wherein each pixel photoreceptor is further The red photoreceptor and the blue photoreceptor are included, and the exposure value required for the full spectrum photoreceptor to reach saturation is lower than the red photoreceptor and the exposure value required for the blue photoreceptor to reach saturation. . The image sensing device of claim 1, wherein the pixel groups comprise a first pixel group corresponding to a first exposure value, a second pixel group corresponding to a second exposure value, and a corresponding And a third exposure group of the third exposure value, and the first exposure value is greater than the second exposure value, the second exposure value being greater than the third exposure value. The image sensing device of claim 2, wherein one or more of the exposure values are less than an exposure value required by the full spectrum photoreceptor when the amount of incoming light reaches saturation. The image sensing device of claim 1, wherein the pixel photoreceptors are grouped in units of rows or columns. A method for image sensing, the method comprising the steps of: sensing an ambient light source using a plurality of pixel photoreceptors, and obtaining an ambient brightness through a sensing result of the ambient light source; dividing the pixel photoreceptors into at least Grouping pixel groups, and each pixel photoreceptor includes at least one full spectrum photoreceptor; calculating a plurality of exposure values according to the ambient brightness, wherein each exposure value is different from each other; and the pixels in each group of pixel groups The exposure time of the full spectrum photoreceptor of the photoreceptor is set to each exposure value, respectively, to capture an image, wherein each pixel photoreceptor further includes a red photoreceptor and a blue photoreceptor, and the full spectrum photoreceptor is in progress The exposure value required when the amount of light reaches saturation is lower than the exposure value required for the red photoreceptor and the blue photoreceptor when the amount of light entering the saturation. The method of claim 5, wherein the group of pixels comprises a first group of pixels corresponding to a first exposure value, a second group of pixels corresponding to a second exposure value, and a third exposure a third pixel group of values, and the first exposure value is greater than the second exposure value, the second exposure value being greater than the third exposure value. The method of claim 6, wherein one or more of the exposure values are less than an exposure value required by the full spectrum photoreceptor when the amount of incoming light reaches saturation. The method of claim 5, wherein the pixel photoreceptors are grouped in units of rows or columns. An image sensing device includes: an image capturing circuit having a plurality of pixel photoreceptors, wherein the pixel photoreceptors are at least divided into a plurality of groups of pixels, and each of the pixel photoreceptors includes at least one full The image sensor is coupled to the image capture circuit, wherein the image processor obtains ambient brightness through the sensing result of the ambient light source, and calculates a plurality of exposure values according to the ambient brightness, wherein each exposure value is mutually Differentily, the image processor sets an exposure time of the full-spectrum photoreceptor of the pixel photoreceptors in each group of pixel groups to respective exposure values, thereby capturing an image, wherein among the exposure values One or more of them are less than the exposure value required by the full spectrum photoreceptor when the amount of incoming light reaches saturation. A method for image sensing, the method comprising the steps of: sensing an ambient light source using a plurality of pixel photoreceptors, and obtaining an ambient brightness through a sensing result of the ambient light source; dividing the pixel photoreceptors into at least Grouping pixel groups, and each pixel photoreceptor includes at least one full spectrum photoreceptor; calculating a plurality of exposure values according to the ambient brightness, wherein each exposure value is not And setting an exposure time of the full-spectrum photoreceptors of the pixel photoreceptors in each group of pixel groups to respective exposure values, thereby capturing an image, wherein one of the exposure values or one of the exposure values A plurality of exposure values less than that required for the full spectrum photoreceptor when the amount of incoming light reaches saturation.