KR100765252B1 - Apparatus and method for widening the dynamic range of an image sensor - Google Patents

Abstract

The present invention relates to an apparatus and method for widening the dynamic range of a CMOS image sensor. CMOS image sensor to which the present invention is applied is a pixel unit for receiving a subject image; A first readout unit configured to receive a first image signal of the subject image exposed from the pixel unit during an INT1 time period; A second readout unit configured to receive a second image signal of the subject image exposed during the INT2 time from the pixel unit; And receiving the first image signal from the first readout unit and storing the first image signal in a memory, receiving the second image signal from the second readout unit, and synthesizing the first image signal and the second image signal. Afterwards, the data synthesis unit outputs the synthesized value as an output value of the image sensor.

CMOS image sensor, dynamic range

Description

Apparatus and method for expanding dynamic range of image sensor

1 is an equivalent circuit diagram showing a unit pixel of a conventional CMOS image sensor.

2 is a circuit diagram for processing an image signal output from a conventional CMOS image sensor unit pixel.

3 is a block diagram of a CMOS image sensor according to an embodiment of the present invention.

4 is a block diagram of a data synthesizing unit for explaining a process of synthesizing two image signals having different exposure times in a data synthesizing unit according to an exemplary embodiment of the present invention.

Fig. 5 is a graph of the amount of light versus output in a general image sensor according to a preferred embodiment of the invention and in the image sensor according to the invention.

6 is a timing diagram in the case of performing integration and readout on one row through double sampling according to an exemplary embodiment of the present invention.

7 is a timing diagram when an integration is performed in two or more rows at the same time by a rolling shutter method according to an exemplary embodiment of the present invention.

8 is a photograph taken through a general image sensor.

9 is pictures taken with different integration.

FIG. 10 is a photograph obtained by combining two photos in FIG. 9 by a suitable algorithm.

<Explanation of symbols for main parts of the drawings>

301 pixel part

310: first lead-out part

320: second lead-out part

331: data synthesis unit

401: memory

The present invention relates to a method for widening the dynamic range through double sampling in a CMOS image sensor and an image sensor using the same.

The image sensor refers to a semiconductor device that converts an optical image into an electrical signal. Device types of image sensors include CCD (Charge Coupled Device) devices and CMOS (Complementary Metal-Oxide-Silicon) devices. CCD-type device is a device in which charge carriers are stored and transported in capacitors while individual metal-oxide-silicon (MOS) capacitors are located in close proximity to each other, while CMOS-type devices use control circuits and signal processing circuits as peripheral circuits. It is a device that adopts the switching method of making as many MOS transistors as the number of pixels using the technique used and sequentially detecting the output using the same.

CCD (charge coupled device) has several disadvantages such as complicated driving method, high power consumption, high number of mask process steps, complicated process, and difficult to implement signal processing circuit in CCD chip. Recently, in order to overcome such drawbacks, the development of CMOS image sensors using sub-micron CMOS manufacturing techniques has been studied. The CMOS image sensor implements an image by forming a photodiode and a MOS transistor in a unit pixel (Pixel) and sequentially detecting signals in a switching method. In addition, the CMOS image sensor uses CMOS fabrication technology, which makes the process very simple compared to CCD processes requiring 30 to 40 masks with low power consumption and 20 masks. I am in the limelight.

A conventional CMOS image sensor will be described with reference to FIGS. 1 and 2 as follows.

1 is an equivalent circuit diagram illustrating a unit pixel of a conventional CMOS image sensor.

Referring to FIG. 1, a CMOS image sensor may include a photodiode (PD), a transfer transistor (Tx), a reset transistor (Rx; a reset transistor), a select transistor (Sx), and an access transistor (Ax). Access Transistor). The transfer transistor Tx and the reset transistor Rx are connected in series with the photodiode. The applied voltage VDD of the power supply voltage terminal is connected to the drain of the reset transistor Rx. The drain of the transfer transistor Tx, that is, the source of the reset transistor Rx, corresponds to the floating diffusion layer F / D. The floating diffusion layer F / D is connected to the gate Gate of the selection transistor Sx. The selection transistor Sx and the access transistor Ax are connected in series, and an applied voltage VDD is connected to the drain of the selection transistor Sx.

Referring to the operation of the CMOS image sensor having the above configuration is as follows. First, when the reset transistor Rx is turned on, the potential of the floating diffusion layer F / D becomes the applied voltage VDD. When light is incident on the photodiode PD, which is an external light receiving unit, an electron hole pair (EHP) is generated in proportion to the signal, and signal charges are accumulated in the source of the transfer transistor Tx. When the transfer transistor Tx is turned on, the signal charge is transferred to the floating diffusion layer F / D, and the potential of the floating diffusion layer F / D is changed in proportion to the amount of the signal charge transferred. The change in the floating diffusion layer F / D potential corresponds to the change in the gate potential of the selection transistor Sx. At this time, when the access transistor Ax is turned on by the selection signal Row, data is output toward the column. When the reset transistor Rx is turned on again, the potential of the floating diffusion layer F / D becomes the applied voltage VDD, and this process is repeated to output an image signal.

2 is a circuit diagram for processing an image signal output from a conventional CMOS image sensor unit pixel.

Referring to FIG. 2, a unit pixel (Pixel) 201 of a CMOS image sensor is combined with a CDS circuit, where a correlated double sampling (CDS) circuit is used to reset the voltage in a reset state to eliminate FPN (Fixed Pattern Noise). A circuit for outputting a difference in voltage as a signal value when the transfer transistor Tx is turned on. The correlated double sampling (CDS) circuit includes a switch 203 for reset data and a switch 205 for pixel data. When the reset data switch 203 is turned on in the reset state, the voltage value is stored in the capacitor 207 in the reset state. In the state where the transfer transistor Tx is On, the pixel data switch 205 is On, and the voltage value in this state is stored in the capacitor 209. The difference between the two values stored in the capacitors 207 and 209 is amplified by the differential amplifier 211, and the difference between the two amplified voltages is an automatic gain controller (AGC) and an analog-to-digital converter (ADC). , 215).

On the other hand, one of the important criteria for indicating the quality of an image device is the dynamic range. Dynamic range generally represents the maximum range in which a signal can be processed without distorting the input signal. In the case of an image sensor, a wider dynamic range can obtain a good image regardless of a wide range of brightness changes.

However, the conventional image sensor has a disadvantage in that the dynamic range is narrow, so that the dark part and the bright part are not displayed in the dark part or the bright part depending on the exposure time.

An object of the present invention for solving the above problems is to provide a method and image sensor using the same that can widen the dynamic range by synthesizing two image signals having different exposure time through double sampling.

Another object of the present invention is to provide a method capable of widening the dynamic range in which the frame rate does not decrease when operating in accordance with a rolling shutter mechanism, and an image sensor using the same.

In order to achieve the above object, according to an aspect of the present invention, a pixel unit for receiving a subject image; A first readout unit configured to receive a first image signal of the subject image exposed from the pixel unit during an INT1 time, which is a first exposure time; After the first readout unit receives the first image signal, the second image signal of the subject image exposed during the INT2 time, which is a second exposure time having a value different from the first exposure time, from the pixel unit. A second lead out unit for receiving a; And receiving the first image signal from the first readout unit and storing the first image signal in a memory, receiving the second image signal from the second readout unit, and storing the first image signal and the first image signal stored in the memory. After synthesizing two image signals, the CMOS image sensor may include a data synthesizing unit configured to output the synthesized value as an output value of the image sensor.
The first lead-out unit and the second lead-out unit may include a differential amplifier, an auto gain controller, and an analog-digital converter.
In addition, the second exposure time INT2 time may be a value 20 times smaller than the first exposure time INT1 time.
Here, the value synthesized by the data synthesizing unit is a value obtained by multiplying a value of the first image signal by a predetermined value smaller than 1 when the value of the first image signal does not exceed a display limit value. When the value of the first image signal is a display limit value, the value of the first image signal may be a value obtained by adding a value of the second image signal and subtracting an offset and multiplying the value of the first image signal by a predetermined value smaller than 1.
According to another aspect of the invention, the pixel portion for receiving the subject image, the first and second lead-out portion for receiving the image signal of the subject image exposed from the pixel portion for a predetermined time, and the first and second A method of widening a dynamic range of a CMOS image sensor including a data synthesizing unit for synthesizing respective image signals of a subject image transmitted from a readout unit, wherein the first readout unit has a first exposure time from the pixel unit; Receiving and storing a first image signal of the subject image exposed during After the first readout part receives the first image signal from the pixel part, the second readout part is exposed from the pixel part for an INT2 time which is a second exposure time having a value different from the first exposure time. Receiving a second image signal of the subject image; And synthesizing the second image signal transmitted from the second readout unit and the first image signal stored in the memory and outputting a synthesized value. can do.

delete

delete

Next, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

3 is a configuration diagram of a CMOS image sensor according to an exemplary embodiment of the present invention.

Referring to FIG. 3, a CMOS image sensor according to the present invention includes a pixel unit 301, a first readout subcircuit 310, a second readout unit 320, and a data combiner 331. do.

The pixel unit 301 has the same structure as the pixel unit of a conventional CMOS image sensor and transfers two image signals obtained by varying an exposure time to an image of a subject to the readout units 310 and 320. Perform.

The readout parts 310 and 320 are divided into a first leadout part 310 and a second readout circuit 320, and each of the readout parts 310 and 320 is a switch for reset data 311 and 321. ), Switches 312 and 322 for pixel data, capacitors 313 and 323 for storing reset data, capacitors 314 and 324 for storing pixel data, differential amplifiers 315 and 325, and automatic gain controller ( Auto Gain Controller (AGC) and Analog-to-Digital Converter (ADC).

When the reset data switches 311 and 321 are turned on in the reset state, voltage values are stored in the capacitors 313 and 323 in the reset state. In the state where the transfer transistor Tx is On, the pixel data switches 312 and 322 are turned On and the voltage values in this state are stored in the capacitors 314 and 324. The difference between the two values stored in the reset data capacitors 313 and 323 and the pixel data capacitors 314 and 324 are amplified by the differential amplifiers 315 and 325, and the difference between the two amplified voltages is obtained by the auto gain controller AGC. After being amplified by the Automatic Gain Controller (316, 326), it is converted into a digital signal through an analog-to-Digital Converter (317, 327) and delivered to the data synthesizing unit (331).

The conventional image sensor has one lead out portion, but the image sensor according to the present invention has two lead out portions. The two lead-out units 310 and 320 share data lines that are out of the pixel unit 301, and alternately read two image signals having different exposure times out of the pixel unit 301. That is, the first readout unit 310 reads the INT1 data at which the integration is performed, that is, the exposure time is INT1 (first exposure time), and transmits the INT1 data to the data synthesis unit 331, and the second readout unit 320. Reads INT2 data having INT2 (second exposure time) having a value different from the first exposure time, and transmits the INT2 data to the data synthesis unit 331. At this time, INT2 may have a value smaller than INT1 and preferably 20 times smaller than INT1.

The data synthesizing unit 331 synthesizes two data received from the readout units 310 and 320 and outputs the final output value of the image sensor. When double sampling is performed with two read-out units as described above, when the INT1 data and the INT2 data are read out and transferred to the data synthesizing unit 331, there is a time difference between the two data. Therefore, the INT2 data first read out is temporarily stored in the memory, and the data synthesizing unit 331 combines the two data at the moment when the INT2 data is read out. A method of synthesizing the two data in the data synthesizing unit 331 will be described with reference to FIG. 4.

4 is a block diagram illustrating a data synthesizing unit for explaining a process of synthesizing two image signals having different exposure times in the data synthesizing unit according to an exemplary embodiment of the present invention.

Referring to FIG. 4, the data synthesizing unit includes a memory 401 for INT1 data and a synthesizing logic 403. In this case, the INT1 data memory 401 may be included in the first readout unit described with reference to FIG. 3. The memory 401 means a digital memory such as RAM.

Since the INT1 data read out from the first lead-out section precedes the INT2 data read out from the second lead-out section in time, the INT1 data is stored in the INT1 data memory 401 until the INT2 data is read out and available. Temporarily stored. When INT2 data is read out and available, the synthesis logic 403 synthesizes the INT1 data and the INT2 data stored in the memory 401. An example of the synthesis logic 403 is as follows.

If (data (int1) <255)

data_out = ((255-E) / 255) * data (int1)

else

data_out = ((255-E) / 255) * (data (int1) + data (int2) -offset)

In the above example, data (int1) means INT1 data and data (int2) means INT2 data. On the other hand, E is a value indicating how much code is to be applied to an input in which the dynamic range is extended to the extended range. If INT1 data is less than 255 (the largest displayable value), that is, not saturated, then INT1 data multiplied by (255-E) / 255 is the final output value, and if INT1 data is 255, that is, saturated In the state, the value obtained by "INT1 data + INT2 data-offset" multiplied by (255-E) / 255 is the final output value. At this time, (255-E) / 255 is smaller than 1, and the offset depends on the characteristics of the sensor and may be 0 for an ideal image sensor. The offset is stored in a register in a manner that specifies a constant value after testing once the image sensor is manufactured.

In the above example, the meaning of the extended range E and how the dynamic range is widened according to the above example will be described with reference to FIG. 5.

Fig. 5 is a graph of the amount of light vs. output in a general image sensor according to a preferred embodiment of the invention and in the image sensor according to the invention.

Referring to FIG. 5, the graph on the left shows the amount of light vs. the output in the case of a single sampling by a certain time exposure as in a general image sensor, and the graph on the right shows two image signals having different exposure times through double sampling as in the present invention. It shows the amount of light versus output when synthesized. In the two graphs, the horizontal axis represents the amount of light and the unit is lux · sec. The horizontal axis represents the size of the output and is expressed as a code. At this time, the output size may have a value of 0 to 255 in the case of an 8-bit system.

In the two graphs, as shown in the present invention, the conventional image sensor may express an image having a light amount up to M, but when double sampling, the image sensor may display an image having a light amount up to M '. In this case, A section is invisible when using a single sampling, but means a section that is visible when a double sampling is synthesized. That is, in the case of double sampling as in the present invention, it can be seen that the dynamic range is widened by A from M to M '.

In the graph on the right, the extended range (E) is a value that indicates how much code to give to the widening input. In other words, when the extended range E is enlarged, the dynamic range is widened, but a code value that can be expressed in a well-expressed section becomes small.

FIG. 6 is a timing diagram when integration and readout are performed on one row through double sampling according to an exemplary embodiment of the present invention.

Referring to FIG. 6, the signal 601 of the timing diagram represents a time for performing integration for one row. The value obtained by performing the integration for the INT1 time (the first exposure time) in the signal 601 is read out from the first lead-out unit (RO1), and again, the INT2 time (the second exposure) having a value different from the INT1 time. It can be seen that the value obtained by performing the integration during the time) is readout RO2 at the second readout part. The value read out from the first readout unit refers to an image signal exposed for an INT1 time, and the value read out from the second readout unit refers to an image signal exposed for an INT2 time. At this time, the INT2 time may have a value smaller than the INT1 time and preferably 20 times smaller than the INT1 time. Since there is a difference between the time that is read out by the first lead-out unit and the time that is read out by the second lead-out unit, a memory for temporarily storing the first read-out value is required to synthesize the two read-out image signals. need. On the other hand, 1V means the time required to output one frame.

7 is a timing diagram when an integration is performed in two or more rows at the same time by a rolling shutter method according to an exemplary embodiment of the present invention.

Referring to FIG. 7, the rolling shutter method is used because the readout circuit is shared for each column. In the signal 703 for Row1 and the signal 705 for Row2, two types of data can be obtained with a time difference for one row. At this time, the process of obtaining two data has been described with reference to FIG.

In the case of the rolling shutter method, it can be seen that image signals are simultaneously read for two rows. At this time, in the signal 703 for Row1 and the signal 705 for Row2, the two rows have a time difference as much as a delta (delta, Δ). Rows are simultaneously integrated. As described above, the two lines are read out at the same time, but finally, the data synthesis unit synthesizes the INT1 data and the INT2 data and outputs the final output values. At this time, the value of i continues to increase. Delta, on the other hand, represents INT2 time.

Compared to the case where there is only one integration in the existing row1 signal 701, when the double sampling is performed as in the present invention, it can be seen that the time difference between the rows also becomes a delta. As described above, even when the rolling shutter method is applied in the double sampling as in the present invention, the frame rate decrease does not occur.

Referring to FIGS. 8 to 10, a practical example in which the dynamic range is widened will be described.

FIG. 8 is a picture taken through a general image sensor, FIG. 9 is a picture taken with different integration, and FIG. 10 is a picture obtained by combining two pictures in FIG. 9 by a suitable algorithm.

Referring to FIGS. 8 to 10, the photo 801 taken through the conventional image sensor having a narrow dynamic range shows a low contrast portion 803 in an image having a wide contrast, but is well represented but outside a window having a very high illumination. The bright part 805 is hard to see. In other words, the bright part 805 outside the window has an excessively high illuminance and mostly has a maximum output value of the image sensor, and an object cannot be identified. In this case, it is expressed as saturation.

9 are pictures taken at different integration times, ie at different exposure times. The picture 901 taken at the exposure time of the general image sensor does not express a bright part outside the window like the picture 801 taken through the conventional image sensor, but the picture 903 taken at a small exposure time is a part outside the window ( (905) is well expressed, but the rest is dark and indistinguishable from things.

The picture 1001, which is a combination of a picture 901 taken at the exposure time of a general image sensor and a picture 903 taken at a small exposure time, also has a portion 1005 outside the window, which was not well represented in the original picture 801. It is well expressed, and it can be seen that other parts 1003 are well represented. In this case, the synthesis process of the two photos has been described with reference to FIGS. 2 to 7, and it can be seen that the dynamic range is widened by synthesizing two image signals having different exposure times.

The present invention is not limited to the above embodiments, and many variations are possible by those skilled in the art within the spirit of the present invention.

According to the present invention, by synthesizing two image signals having different exposure times through double sampling, the dynamic range can be widened, so that an image having a distinct contrast can be well represented.

In addition, even when operating according to the mechanism of the rolling shutter system according to the present invention there is an effect that the frame rate is not lowered.

Claims (6)

A pixel unit receiving a subject image; A first readout unit configured to receive a first image signal of the subject image exposed from the pixel unit during an INT1 time, which is a first exposure time; After the first readout unit receives the first image signal, the second image signal of the subject image exposed during the INT2 time, which is a second exposure time having a value different from the first exposure time, from the pixel unit. A second lead out unit for receiving a; And The first image signal is received from the first readout unit and stored in a memory, the second image signal is received from the second readout unit, and the first image signal and the second image are stored in the memory. After synthesizing the image signals, the data synthesis unit for outputting the synthesized value as the output value of the image sensor CMOS image sensor comprising a. The method of claim 1, The first lead-out unit and the second lead-out unit include a differential amplifier, an automatic gain controller, and an analog-digital changer. CMOS image sensor. delete The method of claim 1, The second exposure time INT2 time is 20 times smaller than the first exposure time INT1 time CMOS image sensor, characterized in that. The method of claim 1, The value synthesized by the data synthesizing unit is a value obtained by multiplying a value of the first image signal by a predetermined value less than 1 when the value of the first image signal does not exceed a display limit value. Is a display limit value, a value obtained by adding a value of the second image signal to a value of the first image signal and subtracting an offset again and multiplying a value smaller than 1 by a predetermined value. CMOS image sensor, characterized in that. A pixel unit for receiving a subject image, first and second lead-out units for receiving image signals of the subject image exposed from the pixel unit for a predetermined time, and a subject image transmitted from the first and second lead-out units A method of widening the dynamic range of a CMOS image sensor comprising a data synthesizing unit for synthesizing respective image signals of Receiving and storing, by the first readout unit, a first image signal of the subject image exposed during the INT1 time which is a first exposure time from the pixel unit; After the first readout part receives the first image signal from the pixel part, the second readout part is exposed from the pixel part for an INT2 time which is a second exposure time having a value different from the first exposure time. Receiving a second image signal of the subject image; And Synthesizing a second image signal transmitted from the second readout unit with the first image signal stored in the memory and outputting a synthesized value; How to widen the dynamic range in the CMOS image sensor comprising a.

Images