JP5271643B2 - Control method and system for optical time-of-flight range image sensor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent saturation of electric charge accumulation sections of a solid imaging element by reception of a bright background light and a bad effect on distance accuracy of a distance image to be obtained. <P>SOLUTION: A method for controlling a light flying time type distance image sensor which constitutes picture elements by photoelectric conversion elements, a plurality of electric charge accumulation sections provided for each photoelectric conversion element and capable of accumulating an electric charge, distribution means for distributing and accumulating the electric charge in the plurality of electric charge accumulation sections in synchronization with a modulated light, a plurality of capacities conductive to each of the plurality of electric charge accumulation sections, and the solid imaging element having control means for controlling a conductive state between the plurality of the electric charge accumulation sections and the plurality of capacities. In a light accumulation time during which the electric charge is distributed to the plurality of electric charge accumulation sections by the distribution means, the electric charge by lights other than the modulated light is eliminated from the plurality of electric charge accumulation sections by switching the conductive state to be controlled by the control means by any time allocation, while having the electric charge by the modulated light accumulated in the plurality of electric charge accumulation sections. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a control method and system for an optical time-of-flight distance image sensor, and more specifically, to a control method and system for an optical time-of-flight distance image sensor in which one solid-state imaging device is configured as one pixel. Then, by receiving the reflected light of the light irradiated to the target object, the optical flight time is measured using a time of flight (TOF) method, and the target object is measured based on the optical flight time. TECHNICAL FIELD OF THE INVENTION The present invention relates to a method for controlling a time-of-flight distance image sensor that generates a distance image by measuring a distance and an improvement of the system, and in particular, a solid that adopts a charge distribution method that can be used under unknown background light illumination. The present invention relates to a control method and system for a time-of-flight distance image sensor in which pixels are configured by an image sensor.

  Here, as will be described later, the charge distribution method is a method in which two or more charge storage capacitors are connected to one photoelectric conversion element and light modulated in high frequency or light that emits pulses (in this specification, “high frequency The light modulated in the above and the light that emits pulse light are collectively referred to as “modulated light”.) Are synchronized and stored in the two or more charge storage capacitors in synchronization with the modulation and light emission time, It means the principle of increasing the signal-to-noise ratio by reading out and averaging the charges separated and accumulated at regular intervals, and calculating the distance using the signal obtained based on the charges accumulated and separated.

  In general, as a conventional optical time-of-flight distance image sensor using the optical time-of-flight measurement method, for example, a distance image imaging device disclosed in Japanese Patent Application Laid-Open No. 2001-281336 presented as Patent Document 1, Patent Document 2 The solid-state imaging device disclosed in Japanese Patent Laid-Open No. 2003-51988 to be presented, the distance image sensor disclosed in Japanese Patent Laid-Open No. 2004-294420 presented as Patent Document 3, or Japanese Patent Laid-Open No. 2005-235893 presented as Patent Document 4 An optical time-of-flight distance sensor disclosed in the publication is known.

Here, FIG. 1 is a block diagram for explaining a conventional optical time-of-flight distance image sensor, which is well known, and the principle of the optical time-of-flight measurement method will be described with reference to FIG. To do.

  The optical time-of-flight distance image sensor 100 shown in FIG. 1 includes a light source 102, a synchronization control unit 104, a photoelectric conversion element that converts light into electric charge, and a plurality of charge storage units provided for each photoelectric conversion element. The solid-state imaging device 106 according to the charge distribution method configured to include the charge distribution unit that distributes the charges to the plurality of charge storage units, and the distance image generation unit 108 are configured.

  In the optical time-of-flight distance image sensor 100 shown in FIG. 1, in order to simplify the explanation and facilitate understanding, the optical time-of-flight distance image sensor 100 is illustrated as including one solid-state imaging device 106. The mold distance image sensor includes a single solid-state image sensor as one pixel and a plurality of solid-state image sensors arranged in an array on a two-dimensional plane.

  Further, like the solid-state imaging device 106, a photoelectric conversion element that converts light into electric charge, a plurality of charge accumulation units provided for each photoelectric conversion element, and a distribution unit that distributes charges to the plurality of charge accumulation units are provided. Since the technology for configuring the solid-state imaging device is well known, detailed description thereof is omitted.

  In FIG. 1, reference numeral 200 denotes a target object to be measured by the optical time-of-flight distance image sensor 100 that is irradiated with light accompanying light emission of the light source 102.

  The light source 102 of the optical time-of-flight distance image sensor 100 described above is modulated light L1 that is light such as infrared light or visible light that is modulated at a high speed as signal light with a high frequency, for example, a 10 MHz sine wave or rectangular wave. Is irradiated onto the target object 200.

  As such a light source 102, for example, a device capable of high-speed modulation such as an LED can be used.

In the optical time-of-flight distance image sensor 100, when the target object 200 is irradiated with the modulated light L <b> 1 irradiated by the light source 102, the modulated light L <b> 1 irradiated to the target object 200 depends on the reflectance of the target object 200. The light is reflected and incident on the solid-state image sensor 106.

  Here, the incident light L2 that is incident on the solid-state imaging element 106 is light derived from the fact that the modulated light L1 is reflected by the target object 200 (hereinafter referred to as “reflected light derived from the modulated light”) as “modulated light component”. And the light derived from the reflection of the background light from an external light source such as natural light by the target object 200 (hereinafter referred to as “reflected light derived from background light” as “background light component”). As appropriate)).

  When the incident light L2 that is the sum of the modulated light component and the background light component is incident on the solid-state image sensor 106, the solid-state image sensor 106 images the incident light L2.

  Here, the synchronization control unit 104 controls the light source 102 and the solid-state imaging device 106 in synchronization, and the solid-state imaging device 106 modulates the modulated light L1 emitted by the light source 102 according to the synchronization signal of the synchronization control unit 104. In synchronism, a process of distributing charges by incident light L2 at a high speed to a plurality of charge accumulating units provided for each photoelectric conversion element by a distributing unit is performed.

  Then, the distance image generation unit 108 reads out the charges distributed to the plurality of charge storage units provided for each photoelectric conversion element of the solid-state imaging device 106, and modulates the light L1 and the incident light L2 based on the read result. And a distance image is generated based on the calculation result.

Next, FIG. 2 shows a timing chart showing the relationship between the modulated light L1 and the incident light L2, as a timing chart for explaining the operation principle of the above-described optical time-of-flight distance image sensor 100.

  In the timing chart shown in FIG. 2, the vertical axis indicates the light emission intensity of the light source 102 and the light reception intensity of the solid-state image sensor 106, and the horizontal axis indicates the passage of time.

  Here, a curve indicated by reference numeral P1 in the timing chart shown in FIG. 2 indicates a light emission pattern of the modulated light L1 in the light source 102, and an example in which the modulated light L1 is modulated using a sine wave.

  A curve indicated by reference sign P2 in the timing chart shown in FIG. 2 indicates a light reception pattern of the incident light L2 in the solid-state imaging device 106.

  The phase difference between the light emitting pattern P1 and the light receiving pattern P2 is a delay caused by the time of flight until the modulated light L1 is emitted from the light source 102, reflected by the target object 200, and received by the solid-state image sensor 106. .

Here, as described above, the solid-state imaging device 106 performs an operation of distributing charges to the plurality of charge storage units in synchronization with the modulation of the modulated light L1 emitted from the light source 102. The example illustrated in FIG. In this case, one cycle of modulation of the modulated light L1 emitted from the light source 102 is divided into four periods, and charges are distributed to the four charge storage units.

When the four periods are defined as T1, T2, T3, and T4 (see FIG. 2), and the charge amount accumulated in each period is defined as C1, C2, C3, and C4, the phase difference obtained thereby. φ (see FIG. 2) is obtained by the following mathematical formula 1.

Further, the charge amount average A (see FIG. 2) used as general image data is obtained by the following formula 2.

Further, the amplitude amount B (refer to FIG. 2) of the modulated light component that is the light derived from the modulated light L1 reflected by the target object 200 is obtained by the following Equation 3.

  Note that the modulation frequency of a light source generally used as the light source 102 is several tens of MHz, and therefore one modulation period is about several tens of ns. Therefore, in order to obtain a distance image, a charge accumulation time of hundreds to hundreds of thousands of cycles is required.

In an optical time-of-flight distance image sensor such as the optical time-of-flight distance image sensor 100 described above, a distance image and a general luminance image can be obtained simultaneously.

  However, the brightness of the background when the target object is imaged changes in most cases, and the background light from other light sources such as fluorescent lamps, and particularly bright when used outdoors, etc. There will be background light.

  The modulated light emitted from the light emitting source (in the example in FIG. 1, the light source 102) for obtaining the distance image is superimposed on the bright background light, but the pixel of the optical time-of-flight distance image sensor. Because there is a limit to the amount of light that can be converted by the solid-state image sensor that constitutes the light, the charge storage unit that receives these bright background lights will saturate, and the charge distribution operation will be invalidated, and the modulated light and incident light will be incident. There has been a problem that it becomes impossible to calculate the phase difference from light.

For this reason, as a technique for solving these problems, the invention disclosed in Japanese Patent Application Laid-Open No. 2006-0667503 presented as Patent Document 5 and the method disclosed in Japanese Patent Application Laid-Open No. 2006-084430 presented as Patent Document 6 are disclosed. Proposed inventions have been proposed.

  These inventions disclosed in Japanese Patent Application Laid-Open No. 2006-066753 and the invention disclosed in Japanese Patent Application Laid-Open No. 2006-084430 control exposure according to a luminance image captured by an optical time-of-flight distance image sensor. Alternatively, saturation of the charge storage portion of the solid-state imaging device due to reception of bright background light is prevented by controlling the light reception time.

However, in the method disclosed in Japanese Patent Application Laid-Open No. 2006-066753 and the method disclosed in Japanese Patent Application Laid-Open No. 2006-084430, saturation of the charge storage unit of the solid-state image sensor due to reception of bright background light is reduced. Although it is possible to prevent this, since the amount of stored charge itself is reduced, the amount of charge due to the modulated light emitted from the light emitting source to obtain the distance image is also reduced proportionally, and at the same time There is a new problem that the distance accuracy of the calculated distance image is adversely affected.

  In addition, as in the invention disclosed in Japanese Patent Application Laid-Open No. 2006-066753 and the invention disclosed in Japanese Patent Application Laid-Open No. 2006-084430, a structure for selecting charges captured for a plurality of accumulation times for each pixel Is very complicated and causes an increase in manufacturing cost.

JP 2001-281336 A JP 2003-51988 A JP 2004-294420 A JP 2005-235893 A JP 2006-066753 A JP 2006-084430 A

  The present invention has been made in view of the above-described various problems of the prior art, and an object of the present invention is to provide a pixel by a charge distribution type solid-state imaging device used in the optical time-of-flight measurement method. In the configured time-of-flight range image sensor, saturation of the charge storage part of the solid-state image sensor due to reception of bright background light can be prevented, and the distance accuracy of the acquired range image is not adversely affected. It is an object of the present invention to provide a method and system for controlling a time-of-flight range image sensor.

  In addition, an object of the present invention is to provide an optical time-of-flight range image sensor in which pixels are configured by a charge distribution type solid-state imaging device used in the optical time-of-flight measurement method. It is an object of the present invention to provide a control method and system for a time-of-flight distance image sensor capable of preventing saturation of a charge storage unit of a solid-state imaging device due to light reception.

  In order to achieve the above object, according to the present invention, the ratio of the charge due to the background light accumulated in the charge accumulation portion of the solid-state imaging device is controlled.

  In order to achieve the above object, according to the present invention, if there is a pixel having a predetermined luminance value in an image captured at a certain time, the pixel is stored in the charge storage unit of the solid-state image sensor based on the number of pixels. The ratio of the electric charge by the background light to be controlled is controlled.

  In order to achieve the above object, the present invention creates a histogram of a distance image taken at a certain time, and based on this histogram, the ratio of the charge due to background light accumulated in the charge accumulation section of the solid-state imaging device Is controlled.

  Therefore, according to the present invention, in the optical time-of-flight range image sensor in which the pixel is configured by the charge distribution type solid-state imaging device used for the optical time-of-flight measurement method, saturation of light reception of the pixel by bright background light is prevented. And the distance accuracy of the acquired distance image is not adversely affected.

  Further, according to the present invention, in the optical time-of-flight range image sensor in which the pixels are configured by the charge distribution type solid-state imaging device used in the optical time-of-flight measurement method, the bright background light is used without increasing the manufacturing cost. It becomes possible to prevent saturation of light reception of the pixel.

That is, the invention of claim 1 of the present invention includes a photoelectric conversion element that converts light including modulated light to charge, with is provided for each said photoelectric conversion element, an electric charge converted by the photoelectric conversion element A plurality of charge storage units that can be stored, and a distribution that stores the charges in the plurality of charge storage units by distributing the charges converted by the photoelectric conversion elements to the plurality of charge storage units in synchronization with the modulated light. A solid-state imaging device including a plurality of capacitors, a plurality of capacitors each capable of conducting with the plurality of charge storage units, and a control unit for controlling a conduction state between the plurality of charge storage units and the plurality of capacitors. A method for controlling a time-of-flight distance image sensor that counts the number of pixels having a predetermined luminance value from an image captured at a certain time, and based on the counted number of pixels, outputs the modulated light and the like. A rate at which the charge due to the light is removed from the plurality of charge storage units, and a light storage time for distributing the charges to the plurality of charge storage units by the distribution unit, based on the determined ratio, by the control unit By switching the conduction state to be controlled, the charge due to the light other than the modulated light is removed from the plurality of charge accumulation units at the determined ratio while the charge due to the modulated light is accumulated in the plurality of charge accumulation units. It is what you do.

According to a second aspect of the present invention, a photoelectric conversion element that converts light including modulated light into an electric charge, and the electric charge converted by the photoelectric conversion element are provided for each of the photoelectric conversion elements. A plurality of charge storage units that can be stored, and a distribution that stores the charges in the plurality of charge storage units by distributing the charges converted by the photoelectric conversion elements to the plurality of charge storage units in synchronization with the modulated light. A solid-state imaging device including a plurality of capacitors, a plurality of capacitors each capable of conducting with the plurality of charge storage units, and a control unit for controlling a conduction state between the plurality of charge storage units and the plurality of capacitors. A method for controlling the optical time-of-flight range image sensor, wherein a histogram is created from an image captured at a certain time, and based on the created histogram, electric power generated by light other than the modulated light is used. Is determined from the plurality of charge accumulation units, and the light conduction time for distributing the charges to the plurality of charge accumulation units by the distribution unit is controlled by the control unit based on the determined ratio. By switching the state, the charges due to the light other than the modulated light are removed from the plurality of charge accumulation units at the determined ratio while the charges due to the modulated light are accumulated in the plurality of charge accumulation units. Is.

According to a third aspect of the present invention, there is provided a photoelectric conversion element that converts light including modulated light into electric charge, and the electric charge converted by the photoelectric conversion element is provided for each photoelectric conversion element. A plurality of charge storage units that can be stored, and a distribution that stores the charges in the plurality of charge storage units by distributing the charges converted by the photoelectric conversion elements to the plurality of charge storage units in synchronization with the modulated light. A solid-state imaging device including a plurality of capacitors, a plurality of capacitors each capable of conducting with the plurality of charge storage units, and a control unit for controlling a conduction state between the plurality of charge storage units and the plurality of capacitors. A time-of-flight time-distance image sensor control system that counts the number of pixels having a predetermined luminance value from an image captured at a certain time, and the counting means counts Gain adjusting means for determining a rate at which charges due to light other than the modulated light are removed from the plurality of charge storage units based on the number of pixels, and the control means uses the distribution means to By switching the conduction state based on the ratio determined by the gain adjusting means in the light accumulation time for distributing the charge to the charge accumulation unit, the charge due to the modulated light is accumulated in the plurality of charge accumulation units, Charges due to light other than the modulated light are removed from the plurality of charge storage portions at a rate determined by the gain adjusting means.

According to a fourth aspect of the present invention, there is provided a photoelectric conversion element that converts light including modulated light into electric charge, and the electric charge converted by the photoelectric conversion element is provided for each of the photoelectric conversion elements. A plurality of charge storage units that can be stored, and a distribution that stores the charges in the plurality of charge storage units by distributing the charges converted by the photoelectric conversion elements to the plurality of charge storage units in synchronization with the modulated light. A solid-state imaging device including a plurality of capacitors, a plurality of capacitors each capable of conducting with the plurality of charge storage units, and a control unit for controlling a conduction state between the plurality of charge storage units and the plurality of capacitors. A control system for a time-of-flight distance image sensor, comprising: a histogram creation means for creating a histogram from an image captured at a certain time; and the histogram creation means Gain adjusting means for determining a rate at which charges due to light other than the modulated light are removed from the plurality of charge storage sections based on a histogram, and the control means uses the distribution means to control the plurality of charge storage sections. By switching the conduction state based on the ratio determined by the gain adjusting means in the light accumulation time for distributing the charges to each other, the modulated light is accumulated in the plurality of charge accumulation sections while being accumulated in the plurality of charge accumulation units. The charges due to light other than the above are removed from the plurality of charge storage portions at a rate determined by the gain adjusting means.

  Since the present invention is configured as described above, in a light time-of-flight distance image sensor in which pixels are configured by a charge distribution type solid-state imaging device used for a light time-of-flight measurement method, It is possible to prevent saturation of the charge accumulating portion of the image sensor and to achieve an excellent effect that the distance accuracy of the acquired distance image is not adversely affected.

  In addition, since the present invention is configured as described above, in an optical time-of-flight range image sensor in which pixels are configured by a charge distribution type solid-state imaging device used in the optical time-of-flight measurement method, the manufacturing cost is increased. Without bringing about this, there is an excellent effect that it becomes possible to prevent saturation of light reception of a pixel by bright background light.

  Hereinafter, an example of an embodiment of an optical time-of-flight distance image sensor control method and system according to the present invention will be described in detail with reference to the accompanying drawings.

  In the following description, the same or equivalent configuration as the configuration of the conventional optical time-of-flight distance image sensor shown in FIG. 1 is denoted by the same reference numerals as those used in FIG. Detailed description of the configuration and operation will be omitted as appropriate.

First, FIG. 3 shows a block configuration explanatory diagram of an optical time-of-flight distance image sensor that implements the control method of the optical time-of-flight distance image sensor according to the first embodiment of the present invention.

  The optical time-of-flight distance image sensor 10 shown in FIG. 3 includes a light source 102, a synchronization control unit 104, a photoelectric conversion element that converts light into electric charges, and a plurality of charge storage units provided for each photoelectric conversion element. A solid-state imaging device 12 using a charge distribution method configured to include a charge distribution unit that distributes charges to the plurality of charge storage units, a distance image generation unit 108, and a distance image generated by the distance image generation unit 108 are configured. A pixel count unit 14 that counts (counts) pixels having a luminance value equal to or greater than a predetermined threshold among the pixels to be accumulated, and a background that is accumulated in the charge accumulation unit of the solid-state imaging device 12 based on the counting result of the pixel count unit 14 A gain control unit that controls a ratio of charges due to light components (hereinafter, “a ratio of charges due to background light components accumulated in the charge accumulation unit of the solid-state imaging device” is appropriately referred to as “gain”) Is constructed and a 6.

  In addition, in the optical time-of-flight distance image sensor 10 shown in FIG. 3, as in the conventional optical time-of-flight distance image sensor 100 shown in FIG. However, in an actual optical time-of-flight distance image sensor, one solid-state image sensor is used as one pixel, and a plurality of solid-state image sensors are arranged in an array on a two-dimensional plane. Arranged and configured.

Here, FIG. 4 shows a circuit diagram of a basic circuit of the solid-state imaging device 12 used as a pixel of the time-of-flight distance image sensor 10.

  The solid-state imaging device 12 shown in FIG. 4 has the same configuration as the invention disclosed in Japanese Patent Application Laid-Open No. 2008-89346 filed by the same applicant as the present applicant.

  That is, the solid-state imaging device shown in FIG. 4 is a known technique having the same configuration as the solid-state imaging device disclosed in FIG. 6 of Japanese Patent Application Laid-Open No. 2008-89346. The description in Japanese Patent Application Publication No. 2008-89346 is incorporated herein, and an outline thereof will be described in this specification.

  In the solid-state imaging device 12 illustrated in FIG. 4, the symbol PD is a photoelectric conversion device, and light incident on the solid-state imaging device 10 is converted into electric charges by the photoelectric conversion device PD.

  Reference numerals M1 and M2 denote FET switches that are gates for distributing charges according to gate signals indicated by reference numerals Tx1 and Tx2, respectively, and the charge storage units Fd1 and Fd2 are synchronized with the modulated light by driving the FET switches M1 and M2. Distribute charges.

  Further, the symbols M3, M4, M5, M6, M7, M8, M9, and M10 are driven by the gate signals G1 and G2, and are capacitors that store the charges indicated by the symbols C1 and C2 in the charge storage portions Fd1 and Fd2. Sort out.

FIG. 5 shows a timing chart showing the timing for controlling the solid-state imaging device 12 shown in FIG.

  The gate signals Tx1 and Tx2 are ON / OFF controlled according to the synchronization signal of the synchronization control unit 104, and distribute the charge by the incident light L2 in synchronization with the modulation of the modulated light L1 emitted by the light source 102. .

  At this time, the charge of the modulated light component, which is the reflected light derived from the modulated light L1 in the incident light L2, is distributed to the charge storage unit Fd1 and the charge storage unit Fd2 according to the phase, so that the charge storage unit Fd1 The charge accumulation unit Fd2 differs in the amount of accumulated charge, but the background light component of reflected light derived from background light in the incident light L2 is equally distributed to the charge accumulation unit Fd1 and the charge accumulation unit Fd2. Thus, the charge storage units Fd1 and Fd2 have the same amount of stored charge.

  In the present specification, the “time for distributing and storing charges in the charge storage portion Fd1 and the charge storage portion Fd2” is appropriately referred to as “light storage time”.

  Further, when the state in which the gate signal G1 is turned on and the gate signal G2 is turned off is a positive connection period, in the positive connection period in the light accumulation time, the charge in the charge storage portion Fd1 is transferred to the left end of the capacitor C1 in FIG. The charge in the charge storage unit Fd2 moves to the right end of the capacitor C2 in FIG.

  Here, the gate signal G1 is turned off and the gate signal G2 is turned on. When this state is an inversion connection period, in the inversion connection period in this optical storage time, the charge in the charge storage portion Fd1 is to the left end of the capacitor C2 in FIG. 4, and the charge in the charge storage portion Fd2 is in the capacity in FIG. It accumulates at the right end of C1 and functions to cancel out the charges accumulated in the positive connection period.

More specifically, first, all of the gate signals Tx1, Tx2, G1, and G2 are turned on. The time at which all the gate signals Tx1, Tx2, G1, and G2 are turned on first is referred to as a reset period R.

  In the reset period R, since all of the FET switches M1 to M10 are connected, the terminals of the charge storage units Fd1 and Fd2 become equal to the Vdd potential. Further, the potential of the photoelectric conversion element PD is reset to a potential according to the voltages of the gate signals Tx1 and Tx2, and the capacitors C1 and C2 are reset to a discharged state.

  Each reset described above is performed at the beginning of reading the screen of the line to which each pixel of the optical time-of-flight sensor 10 belongs.

  When the above-described reset is completed, the charge of the photoelectrons generated by the photoelectric conversion element PD receiving light is distributed and accumulated in synchronization with the modulated light L1 until the next reset, so that the gate signal Tx1 which is a high-speed pulse signal , Tx2 are applied, and the gate signals G1, G2 are applied so that the positive connection period and the inversion connection period exist alternately.

  Note that the positive connection period and the inversion connection period are the same time, which is an integral multiple of the high-speed pulse by the gate signals Tx1 and Tx2.

FIG. 6 is a diagram showing how the background light is removed while leaving the modulated light component in the above operation.

  Note that ΔQFD1 in FIG. 6 represents the charge flowing into the charge storage unit Fd1 during the positive connection period, and ΔQFD2 in FIG. 6 represents the charge flowed into the charge storage unit Fd2 during the positive connection period.

  When the positive connection period and the inversion connection period are equal, the background light components stored equally in the charge storage unit Fd1 and the charge storage unit Fd2 are canceled out, but the modulated light component is the charge storage unit Fd1 and the charge storage unit Fd2. Therefore, only the difference component between the charge storage unit Fd1 and the charge storage unit Fd2 remains in the capacitors C1 and C2.

In the control method of the time-of-flight distance image sensor according to the present invention, as described above, only the difference component between the charge storage unit Fd1 and the charge storage unit Fd2 remains in the capacitor C1 and the capacitor C2 of the solid-state imaging device 12. In view of the above, by setting the gain, which is the ratio of the background light component accumulated in the charge accumulation unit Fd1 and the charge accumulation unit Fd2, to an arbitrary value, the modulated light component remains and the charge accumulation unit Fd1 and the charge accumulation unit remain. The background light component accumulated in the part Fd2 can be reduced at an arbitrary ratio.

Next, a method for controlling the time-of-flight distance image sensor according to the present invention will be described with reference to FIG.

  In FIG. 7, the period indicated by reference numeral T1 is a positive connection period, and the period indicated by reference numeral T2 is an inverting connection period, but the time interval between the positive connection period T1 and the inverting connection period T2 is as shown in FIG. By applying the gate signals G1 and G2 as described above, the charge due to the background light component accumulated in the charge accumulation portion Fd1 and the charge accumulation portion Fd2 is halved, that is, half as compared with the case shown in FIG. Can be reduced.

  Specifically, in the case of FIG. 7, the first positive connection period T1 and the reverse connection period T2 following the first positive connection period T1 are equal in time interval, but the positive connection period following the reverse connection period T2 T1 is a time interval twice as long as the inversion connection period T2. If the gain when the inversion connection period is not provided is 1, the gain can be set to “0.5”.

  That is, the example shown in FIG. 7 shows how the gate signals G1 and G2 are driven when the gain of the background light component is 0.5, and the background light accumulated during the first positive connection period T1. The component charge is canceled during the inversion connection period T2, and only the background light component charge accumulated during the second positive connection period T1 remains. At this time, “T1: T2 = 3: 1”.

  In this way, the charge of the background light component to be accumulated is halved, but as described above, the charge of the modulated light component remains without being canceled.

  Thereby, the dynamic range of the background light component can be increased without reducing the charge of the modulated light component related to the distance accuracy of the distance image.

  FIG. 8 shows another example of the gain, in which the gate signals G1 and G2 are driven when the gain of the background light component is 0.25. At this time, “T1: T2 = 5: 3”.

Here, the ratio of the positive connection period T1 and the inverting connection period T2 when setting to a certain gain g is obtained by the following Equation 4.

Note that the driving methods shown in FIGS. 7 and 8 are merely examples. If the ratio of T1: T2 does not change, the gate signals G1, G2 can be used at any timing unless the charge storage units Fd1, Fd2 are saturated. May be controlled.

  By switching and controlling the gate signals G1 and G2 at an arbitrary time ratio as described above, the charge of the modulated light component remains in the charge storage units Fd1 and Fd2, and only the charge of the background light component remains in the charge storage unit Fd1, It can be decreased from Fd2.

Then, the optical time-of-flight distance image sensor 10 shown in FIG. 3 corresponds to the distance image generated by the distance image generation unit 108 according to the control method of the optical time-of-flight distance image sensor according to the present invention described above. A control system for automatically varying the gain is provided.

  In the optical time-of-flight distance image sensor 10 shown in FIG. 3, the processing when the distance image generation unit 108 generates a distance image is the optical time-of-flight distance image sensor described in the section “Background Art”. Since it is the same as 100, detailed description thereof is omitted.

  Here, FIG. 9 shows a flowchart of a gain control routine which is a processing routine of control for automatically changing the gain in the optical time-of-flight distance image sensor 10.

  In the gain control routine, when an operation for acquiring a distance image is started by the optical time-of-flight distance image sensor 10, a predetermined time interval (for example, every 1/30 second) is performed during the operation. It is repeatedly activated and executed, and gain adjustment is performed automatically.

  That is, when an operation for acquiring a distance image is started by the optical time-of-flight distance image sensor 10 and a gain control routine is started at a predetermined time interval during the operation, the pixel count unit 14 includes a distance image generation unit. The distance image generated by 108 is acquired (step S902), and a predetermined luminance value set as the first threshold value among the pixels of the acquired distance image (for example, 240 when the luminance value is 0 to 255). .) The number of pixels is counted (counted) (step S904).

  When the process of step S904 is completed, the process proceeds to step S906, and the gain control unit 16 determines that the number of pixels counted in the process of step S904 is equal to or greater than a first predetermined value (for example, 10% or greater of all pixels). It is determined whether or not.

  In the determination process of step S906, when it is determined that the number of pixels counted in the process of step S904 is not equal to or greater than the first predetermined value, the process proceeds to the process of step S907, and among the pixels of the acquired distance image. The number of pixels below a predetermined luminance value set as the second threshold value (for example, 15 when the luminance value is 0 to 255) is counted (counted).

  When the process of step S907 is finished, the process proceeds to step S908, and the gain controller 16 determines that the number of pixels counted in the process of step S907 is equal to or greater than a second predetermined value (for example, 10% or more of all pixels). It is determined whether or not.

  If it is determined in step S908 that the number of pixels counted in step S907 is not equal to or greater than the second predetermined value, the process proceeds to step S910, and the gain is not changed.

  On the other hand, in the determination process of step S908, if it is determined that the number of pixels counted in the process of step S907 is equal to or greater than the second predetermined value, the process proceeds to the process of step S912 to increase the gain. Make changes.

  When increasing the gain in the process of step S912, for example, the gain is increased by 0.05.

  If it is determined in step S906 that the number of pixels counted in step S904 is greater than or equal to the first predetermined value, the process proceeds to step S914 to reduce the gain. Make changes.

  Note that when the gain is lowered in the process of step S914, the gain is lowered by, for example, 0.05.

  When the processing in step S910, step S912, or step S914 described above is completed, the gain adjustment processing is completed thereby (step S916), the gain control routine is terminated, and the next gain control routine is started. wait.

  Therefore, in the optical time-of-flight distance image sensor 10 described above, according to the distance image generated by the distance image generation unit 108 by the control method of the optical time-of-flight distance image sensor according to the present invention described above, The gain obtained can be automatically changed.

Next, FIG. 10 shows a block configuration explanatory diagram of an optical time-of-flight distance image sensor that implements the control method of the optical time-of-flight distance image sensor according to the second embodiment of the present invention.

  The optical time-of-flight distance image sensor 20 shown in FIG. 10 includes a light source 102, a synchronization control unit 104, photoelectric conversion elements that convert light into charges, and a plurality of charge storage units provided for each photoelectric conversion element. A solid-state image pickup device 12 by a charge distribution method configured to have a charge distribution unit that distributes charges to the plurality of charge storage units (the same configuration and operation as the solid-state image pickup device 12 in the above-described optical time-of-flight distance image sensor 10) A distance image generation unit 108, a histogram generation unit 22 that generates a histogram of the distance image generated by the distance image generation unit 108, and a solid-state imaging device based on the histogram generated by the histogram generation unit 22 And a gain control unit 16 that controls a gain as a proportion of the charge of the background light component stored in the 12 charge storage units. It has been.

  In the optical time-of-flight distance image sensor 20 shown in FIG. 10, as in the case of the optical time-of-flight distance image sensor 10 shown in FIG. Although illustrated as including an image sensor 12, in an actual optical time-of-flight distance image sensor, one solid-state image sensor is used as one pixel, and a plurality of solid-state image sensors are arranged in an array on a two-dimensional plane. Configured.

  The histogram is a graph display of the luminance (brightness) distribution in the image, and indicates how many pixels of which brightness exist.

  The optical time-of-flight distance image sensor 20 shown in FIG. 10 corresponds to the distance image generated by the distance image generation unit 108 by the method for controlling the optical time-of-flight distance image sensor according to the present invention described above. And a control system for automatically varying the gain described above.

  In the optical time-of-flight distance image sensor 20 shown in FIG. 10, the processing performed when the distance image generation unit 108 generates a distance image will be described in the section “Background Art”. Since it is the same as 100, detailed description thereof is omitted.

  Here, FIG. 11 shows a flowchart of a gain control routine, which is a control processing routine for automatically varying the gain in the optical time-of-flight distance image sensor 20.

  In the gain control routine, when an operation for acquiring a distance image by the optical time-of-flight distance image sensor 20 is started, a predetermined time interval (for example, every 1/30 second) is performed during the operation. It is repeatedly activated and executed, and gain adjustment is performed automatically.

  That is, when an operation for acquiring a distance image is started by the optical time-of-flight distance image sensor 20 and a gain control routine is started at a predetermined time interval during the operation, the histogram creation unit 22 The distance image generated by 108 is acquired (step S1002), and a histogram of the acquired distance image is created (step S1004).

  When the process of step S1004 is completed, the process proceeds to step S1006, and the gain control unit 24 determines that the histogram acquired in the process of step S1004 is a first specified value (for example, the standard deviation is within 15 and the luminance The average of the values is 200 or more.) It is determined whether or not it is brighter.

  If it is determined in step S1006 that the histogram acquired in step S1004 is not brighter than the first specified value, the process proceeds to step S1008, and the histogram acquired in step S1004 is the first histogram. It is determined whether or not it is darker than a specified value of 2 (for example, the standard deviation is 15 or less and the average luminance value is 55 or less).

  If it is determined in step S1008 that the histogram acquired in step S1004 is not darker than the second specified value, the process proceeds to step S1010, and the gain is not changed.

  On the other hand, in the determination process of step S1008, when it is determined that the histogram acquired in the process of step S1004 is darker than the second specified value, the process proceeds to step S1012 and the gain is changed by increasing the gain. Do.

  Note that when the gain is increased in the process of step S1012, the gain is increased by, for example, 0.05.

  If it is determined in step S1006 that the histogram acquired in step S1004 is brighter than the first specified value, the process proceeds to step S1014, and the gain is changed by decreasing the gain. Do.

  Note that when the gain is lowered in the process of step S1014, the gain is lowered by, for example, 0.05.

  When the processing in step S1010, step S1012 or step S1014 is completed, the gain adjustment processing is completed (step S1016), the gain control routine is terminated, and the next gain control routine is awaited. .

  Therefore, in the optical time-of-flight distance image sensor 20 described above, according to the distance image generated by the distance image generation unit 108 by the method for controlling the optical time-of-flight distance image sensor according to the present invention described above, The gain obtained can be automatically changed.

As described above, according to the first embodiment and the second embodiment of the present invention described above, the background light component is left while leaving the charge due to the modulated light component necessary for obtaining the distance image. Therefore, a luminance image with a wide dynamic range can be obtained simultaneously without reducing the distance accuracy of the distance image.

The embodiment described above can be modified as shown in the following (1) to (4).

  (1) In the first embodiment and the second embodiment described above, the gain control routine is not limited to being repeatedly activated and executed at a predetermined time interval. The user of the image sensors 10 and 20 may be activated and executed at an arbitrary timing.

  (2) In the first embodiment described above, the first predetermined value and the second predetermined value may be different from each other, or may be the same value.

  (3) In the second embodiment described above, the first specified value and the second specified value may be different from each other or the same value.

  (4) You may make it combine suitably the embodiment shown above and the modification shown in said (1) thru | or (3).

  The present invention can be used for an optical time-of-flight distance image sensor that is mounted on an automobile or the like and measures the distance to an obstacle.

FIG. 1 is a block diagram illustrating a general optical time-of-flight distance image sensor that has been conventionally known. FIG. 2 is a timing chart showing the relationship between the modulated light L1 and the incident light L2, which is a timing chart for explaining the operation principle of the optical time-of-flight distance image sensor shown in FIG. FIG. 3 is an explanatory block diagram of the optical time-of-flight distance image sensor that implements the method of controlling the optical time-of-flight distance image sensor according to the first embodiment of the present invention. FIG. 4 is a circuit diagram of a basic circuit of a solid-state imaging device used as a pixel of the time-of-flight distance image sensor shown in FIG. FIG. 5 is a timing chart showing the timing for controlling the solid-state imaging device shown in FIG. FIG. 6 is a timing chart showing how the background light is removed while leaving the modulated light component in the solid-state imaging device shown in FIG. FIG. 7 is an explanatory diagram of the control method of the time-of-flight distance image sensor according to the present invention, and is a timing chart showing how the gate signals G1 and G2 are driven when the gain of the background light component is 0.5. is there. FIG. 8 is an explanatory diagram of the control method of the time-of-flight distance image sensor according to the present invention, and is a timing chart showing how the gate signals G1 and G2 are driven when the gain of the background light component is 0.25. is there. FIG. 9 is a flowchart of a gain control routine which is a processing routine for controlling the gain to be automatically varied in the optical time-of-flight distance image sensor shown in FIG. FIG. 10 is an explanatory diagram of a block configuration of an optical time-of-flight distance image sensor that implements the control method of the optical time-of-flight distance image sensor according to the second embodiment of the present invention. FIG. 11 is a flowchart of a gain control routine, which is a processing routine for automatically changing the gain in the optical time-of-flight distance image sensor shown in FIG.

Explanation of symbols

10, 20, 100 Time-of-flight distance image 12, 106 Solid-state imaging device 14 Pixel counting unit 16, 24 Gain control unit 22 Histogram generation unit 102 Light source 104 Synchronization control unit 108 Distance image generation unit 200 Target object

Claims (4)

A photoelectric conversion element that converts light including modulated light into electric charge;
A plurality of charge storage units provided for each photoelectric conversion element and capable of storing charges converted by the photoelectric conversion element;
A distribution unit configured to distribute the charge converted by the photoelectric conversion element to the plurality of charge storage units in synchronization with the modulated light, and to store the charges in the plurality of charge storage units;
A plurality of capacitors each capable of conducting with the plurality of charge storage units;
A control method of a time-of-flight distance image sensor in which a pixel is formed by a solid-state imaging device having a control unit that controls conduction between the plurality of charge storage units and the plurality of capacitors,
Count the number of pixels with a given luminance value from an image captured at a certain time,
Based on the counted number of pixels, determine a ratio of removing charges from light other than the modulated light from the plurality of charge storage units,
By switching the conduction state controlled by the control unit based on the determined ratio in the light accumulation time for distributing the charge to the plurality of charge storage units by the distribution unit, the charge due to the modulated light is changed to the plurality of charge storage units. A method of controlling a time-of-flight distance image sensor, wherein charges due to light other than the modulated light are removed from the plurality of charge storage units at the determined ratio while being stored in a charge storage unit. A photoelectric conversion element that converts light including modulated light into electric charge;
A plurality of charge storage units provided for each photoelectric conversion element and capable of storing charges converted by the photoelectric conversion element;
A distribution unit configured to distribute the charge converted by the photoelectric conversion element to the plurality of charge storage units in synchronization with the modulated light, and to store the charges in the plurality of charge storage units;
A plurality of capacitors each capable of conducting with the plurality of charge storage units;
A control method of a time-of-flight distance image sensor in which a pixel is formed by a solid-state imaging device having a control unit that controls conduction between the plurality of charge storage units and the plurality of capacitors,
Create a histogram from images taken at a certain time,
Based on the created histogram, determine a ratio of removing charges from light other than the modulated light from the plurality of charge storage units,
By switching the conduction state controlled by the control unit based on the determined ratio in the light accumulation time for distributing the charge to the plurality of charge storage units by the distribution unit, the charge due to the modulated light is changed to the plurality of charge storage units. A method of controlling a time-of-flight distance image sensor , wherein charges due to light other than the modulated light are removed from the plurality of charge storage units at the determined ratio while being stored in a charge storage unit. A photoelectric conversion element that converts light including modulated light into electric charge;
A plurality of charge storage units provided for each photoelectric conversion element and capable of storing charges converted by the photoelectric conversion element;
A distribution unit configured to distribute the charge converted by the photoelectric conversion element to the plurality of charge storage units in synchronization with the modulated light, and to store the charges in the plurality of charge storage units;
A plurality of capacitors each capable of conducting with the plurality of charge storage units;
A control system for a time-of-flight distance image sensor comprising pixels by a solid-state imaging device having control means for controlling a conduction state between the plurality of charge storage units and the plurality of capacitors,
Counting means for counting the number of pixels having a predetermined luminance value from an image captured at a certain time;
Gain adjusting means for determining a ratio of removing charges due to light other than the modulated light from the plurality of charge storage units based on the number of pixels counted by the counting means;
By switching the conduction state based on the ratio determined by the gain adjusting means in the light accumulation time in which the control means distributes the charges to the plurality of charge accumulation units by the distribution means,
The charge due to light other than the modulated light is removed from the plurality of charge accumulation units at a rate determined by the gain adjusting means while the charge due to the modulated light is accumulated in the plurality of charge accumulation units. Control system for optical time-of-flight range image sensor. A photoelectric conversion element that converts light including modulated light into electric charge;
A plurality of charge storage units provided for each photoelectric conversion element and capable of storing charges converted by the photoelectric conversion element;
A distribution unit configured to distribute the charge converted by the photoelectric conversion element to the plurality of charge storage units in synchronization with the modulated light, and to store the charges in the plurality of charge storage units;
A plurality of capacitors each capable of conducting with the plurality of charge storage units;
A control system for a time-of-flight distance image sensor comprising pixels by a solid-state imaging device having control means for controlling a conduction state between the plurality of charge storage units and the plurality of capacitors,
A histogram creation means for creating a histogram from an image captured at a certain time;
Gain adjusting means for determining a ratio of removing charges from light other than the modulated light from the plurality of charge storage units based on the histogram created by the histogram creating means;
By switching the conduction state based on the ratio determined by the gain adjusting means in the light accumulation time in which the control means distributes the charges to the plurality of charge accumulation units by the distribution means,
The charge due to light other than the modulated light is removed from the plurality of charge accumulation units at a rate determined by the gain adjusting means while the charge due to the modulated light is accumulated in the plurality of charge accumulation units. Control system for optical time-of-flight range image sensor .

Images