WO2020137908A1

WO2020137908A1 - Lighting fixture for vehicle, and vehicle

Info

Publication number: WO2020137908A1
Application number: PCT/JP2019/050170
Authority: WO
Inventors: 真太郎杉本; 祐介笠羽; 安男中村; 正人五味; 修己山本; 健人新田; 修廣田; 祐太春瀬; 輝明鳥居; 健佑荒井
Original assignee: 株式会社小糸製作所
Priority date: 2018-12-27
Filing date: 2019-12-20
Publication date: 2020-07-02
Also published as: CN113227838B; JP7408572B2; JPWO2020137908A1; CN113227838A

Abstract

This vehicle lighting fixture 400 comprises a headlamp 410 and a pseudo-heat light source 420. The pseudo-heat light source 420 projects reference light S1 onto an object, varying the intensity distribution of the light in M-different ways. The pseudo-heat light source 420, along with a light detector 120 and an arithmetic processing device 130, constitutes an imaging device 100. At least some constituent elements 430 of the headlamp 410 are shared with the pseudo-heat light source 420.

Description

Vehicle lighting and vehicle

The present invention relates to a vehicle lamp.

An object identification system that senses the position and type of objects existing around the vehicle is used for automatic driving and automatic control of headlamp light distribution. The object identification system includes a sensor and an arithmetic processing unit that analyzes the output of the sensor. The sensor is selected from cameras, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), millimeter wave radar, ultrasonic sonar, etc., in consideration of application, required accuracy and cost.

As one of the imaging devices (sensors), one that uses the principle of ghost imaging is known. Ghost imaging irradiates an object while randomly switching the intensity distribution (pattern) of reference light, and measures the light detection intensity of reflected light for each pattern. The photodetection intensity is an integrated value of energy or intensity over a certain plane, not an intensity distribution. Then, the image of the object is reconstructed by taking the correlation between the corresponding pattern and the light detection intensity.

Japanese Patent No. 6412673

Problem 1. In order to correctly restore the image of the object, it is necessary to control the intensity distribution of the reference light with high accuracy and irradiate the object. When the imaging device is mounted on a vehicle, it is necessary to detect an object several tens of meters ahead, and therefore it is necessary to irradiate the reference light up to several tens of meters while maintaining the intensity distribution.

One of the exemplary purposes of the first aspect of the present invention is to accurately irradiate a distant object with reference light.

Problem 2. The amount of correlation calculation explosively increases according to the number of pixels of the restored image. Specifically, when the number of times of random reference light irradiation is M and the number of pixels is (X×Y), the number of calculations is
M x (X x Y) ²
Becomes

Assuming that one pattern irradiation is required to restore one pixel, M=(X×Y) irradiations are required to restore all pixels (X×Y). Further, the number of times of calculation of correlation calculation per irradiation is (X×Y). Therefore, the number of calculations O per sensing is
O=(X×Y) ²
Becomes

In the case of an imaging device installed in a laboratory, since a high-speed computer or workstation can be used as a processing device, the increase in the number of calculations does not matter so much. However, in an imaging device for in-vehicle use, the performance of the arithmetic processing device is restricted due to the cost and space. Therefore, it is required to reduce the calculation amount of the correlation calculation.

One of the exemplary purposes of the second aspect of the present invention is to provide an imaging apparatus or an imaging method with a reduced amount of calculation.

Problem 3. Patterning devices such as DMD (Digital Micromirror Device) and liquid crystal are used for patterning the reference light. The DMD and the liquid crystal have a plurality of pixels arranged in a matrix, and the reflectance and the transmittance can be controlled for each pixel.

　The in-vehicle imaging device can detect various objects such as cars, people, motorcycles and bicycles, structures, and plants and animals. The situation in which the imaging device is used also changes greatly depending on the traveling environment such as weather, time of day, traveling road, traveling speed, and the like. In addition, the imaging device itself moves, the objects also move, and their relative movement directions are various.

In this way, it is not always optimal to randomly control all pixels of a patterning device in a specific purpose imaging.

One of the exemplary objects of the third aspect of the present invention is to provide an illuminating device suitable for imaging for a specific application.

Problem 4. In a near-field observing microscope, the spread of the luminous flux for each pixel does not matter. However, the in-vehicle imaging device 100 needs to detect an object in the far field. In an imaging apparatus that senses a distant object, it is not always optimal to randomly control all pixels of the patterning device.

One of the exemplary objects of the fourth aspect of the present invention is to provide an illumination device suitable for a distant imaging device.

Problem 5. Conventionally, the number of measurements (irradiation times) required to reconstruct an image of an object has reached several thousand, and it took time to measure data. In addition, the amount of calculation and the calculation time required to reconstruct the original image explosively increase as the number of irradiations increases.

　In an application in which an object is substantially stationary, a large number of irradiations and the amount of calculation do not pose a problem, but in an application in which an object moves, a high frame rate is required, so it is necessary to reduce the number of irradiations.

One of the exemplary purposes of the fifth aspect of the present invention is to provide an imaging device capable of reducing the number of irradiations.

1. The first aspect of the present invention relates to a vehicle lamp. The vehicular lamp includes a headlight and a pseudo heat light source. The pseudo heat light source can irradiate the object while randomly changing the intensity distribution of the reference light. The pseudo thermal light source constitutes an imaging device together with a photodetector that measures reflected light from an object, and an arithmetic processing device that reconstructs a restored image of the object based on the output of the photodetector and the intensity distribution of the reference light. .. At least some components of the headlamp are shared with the pseudo heat source.

　The headlight is designed with a light source and an optical system for the purpose of irradiating light up to several tens of meters. Therefore, by incorporating the pseudo heat light source of the imaging device in the vehicle lamp and diverting some of the components of the headlight to the pseudo heat light source, it is possible to accurately irradiate the distant object with the reference light. .. Also, the overall cost can be reduced.

▽ The pseudo heat light source may share the optical system with the headlight. The headlamp optics may include a patterning device that controls the light distribution. The pseudo heat source and the headlight may share a patterning device.

The headlight optical system may include a reflector that reflects the light emitted from the light source toward the front of the vehicle. The pseudo heat source and the headlight may share a reflector.

The reference light may be infrared or ultraviolet.

▽ The pseudo heat light source may share the light source with the headlight. In this case, the reference light may be white light. Further, the entire headlamp may be operable as a pseudo heat source for the imaging device.

2. The second aspect of the present invention relates to an in-vehicle imaging device. The in-vehicle imaging device divides the measurement range into a plurality of sections, and illuminates a reference light whose intensity distribution is random while switching the sections, a photodetector that measures reflected light from an object, and a plurality of sections. Each of them is provided with an arithmetic processing unit that reconstructs a restored image of a portion included in the section of the object based on the detected intensity based on the output of the photodetector and the intensity distribution of the reference light.

According to this aspect, the calculation time can be reduced.

-The number of multiple partitions may be set so that the amount of decrease in the computation time due to the division is larger than the amount of increase in the measurement time. This enables high-speed sensing.

3. A third aspect of the present invention relates to an illumination device used for an imaging device based on ghost imaging. The lighting device has a plurality of pixels arranged in a matrix, and is configured to be able to modulate the light intensity distribution based on a combination of ON and OFF of the plurality of pixels. The intensity distribution is controlled in units of pixel blocks including at least one pixel, and the pixel blocks are variable.

Another aspect of the present invention is also a lighting device. The illumination device has a plurality of pixels arranged in a matrix, and is configured to be able to modulate the light intensity distribution based on a combination of ON and OFF of the plurality of pixels. The intensity distribution is controlled by a combination of predetermined patterns including two or more ON pixels and OFF pixels.

4. A fourth aspect of the present invention is a lighting device. The lighting device has a plurality of pixels arranged in a matrix, and is configured to be able to modulate the light intensity distribution based on a combination of ON and OFF of the plurality of pixels. On/off of a plurality of pixels is controlled under a predetermined constraint condition.

5. A fifth aspect of the invention relates to an imaging device. The imaging device includes an illumination that illuminates an object while changing the intensity distribution of the reference light in a plurality of M ways, a photodetector that measures reflected light from the object for each of the plurality of intensity distributions I ₁ to I _M , and a plurality of photodetectors. Of the intensity distributions I ₁ to I _M and a plurality of detected intensities b ₁ to b _M based on the output of the photodetector, and an arithmetic processing unit for reconstructing a restored image of the object. The plurality of intensity distributions I ₁ to I _M are (i) modeling the transfer characteristics of the path from the illumination through the object to the photodetector, and (ii) defining the reference object and its corresponding reference image. When the steps of providing an initial value to (iii) a plurality of intensity distributions _{_{I 1 ~ I M, (iv}} ) based on the transfer characteristics, irradiated with the reference light having a plurality of intensity distributions I _{1 ~} I _M to the reference object And a step of calculating estimated values (or calculated values) b^ ₁ to b^ _M of the detected intensities b ₁ to b _M , and (v) a plurality of intensity distributions I ₁ to I _M and a plurality of estimated values b A step of reconstructing the restored image of the reference object by taking the correlation of ^ ₁ to b ^ _M ; and (vi) a plurality of intensity distributions I ₁ to I _M so that the error between the restored image and the reference image becomes small. It is obtained by modifying each of them and (vii) determining a plurality of intensity distributions I ₁ to I _M by repeating steps (iv) to (vi).

1. According to the first aspect of the present invention, it is possible to accurately irradiate the reference light at a distance.
2. According to the second aspect of the present invention, the amount of calculation can be reduced while obtaining high resolution.
3. According to the third aspect of the present invention, it is possible to provide an illumination device suitable for imaging for a specific application.
4. According to the fourth aspect of the present invention, it is possible to provide an illumination device suitable for an imaging device for a distant object.
5. According to the fifth aspect of the present invention, the number of irradiations can be reduced.

It is a figure which shows the vehicle lamp which concerns on Embodiment 1. It is a figure which shows the imaging device of FIG. FIG. 1 is a diagram showing a vehicle lighting device according to a first embodiment. FIG. 6 is a diagram showing a vehicle lamp according to a second embodiment. FIG. 10 is a diagram illustrating first pattern control in the second embodiment. FIG. 9 is a diagram illustrating second pattern control in the second embodiment. FIG. 7 is a diagram showing a vehicle lamp according to a third embodiment. FIG. 9 is a diagram illustrating a first control in the third embodiment. FIG. 10 is a diagram illustrating third control in the third embodiment. It is a block diagram of an object identification system. It is a figure which shows a motor vehicle. It is a block diagram which shows the vehicle lamp provided with an object detection system. FIG. 6 is a diagram showing an imaging device according to a second embodiment. FIG. 7 is a diagram illustrating the intensity distribution of reference light according to the second embodiment. It is a figure explaining the trade-off of calculation time and measurement time. 16(a) and 16(b) are diagrams showing a modified example of a section. FIG. 7 is a diagram showing an imaging device according to a third embodiment. FIGS. 18A to 18C are views for explaining the pixels of the DMD that is the patterning device. 19A to 19D are diagrams showing pixel blocks B having different sizes. 20A and 20B are diagrams showing an example of the pattern signal PTN based on the pixel blocks B having different sizes. It is a figure which shows the modification of pattern control. 22A and 22B are diagrams illustrating the layout of the pixel blocks B having different sizes according to the running scene. It is a figure explaining the dynamic layout of the pixel block B from which size differs. 24A to 24C are diagrams showing a pixel block B according to Modification 3.1. 25A to 25D are diagrams showing a pixel block B according to Modification 3.2. 26A to 26D are diagrams showing pixel blocks B having different shapes. 27A to 27C are diagrams showing an example of the pattern signal PTN based on pixel blocks having different shapes. FIGS. 28A and 28B are views for explaining sensing based on the pattern signal PTN having the pixel block B having a characteristic shape. 29A to 29D are diagrams for explaining the pattern block PB according to the embodiment 3.5. 30A and 30B are diagrams showing examples of pattern signals based on the combination of pattern blocks. FIGS. 31A and 31B are diagrams for explaining the improvement of the spatial incoherence of the reference light. 32A and 32B are diagrams showing examples of intensity distributions that can improve spatial incoherence. FIGS. 33A to 33D are diagrams for explaining the pattern control with the lighting rate as a constraint condition. 34(a) and 34(b) are diagrams for explaining the control of the lighting rate according to the modification. It is a figure which shows the imaging device which concerns on Embodiment 4. 6 is a flowchart showing a method of determining a set of a plurality of intensity distributions I ₁ to I _M. It is a figure explaining the relationship between a reference object and reference image T (x, y). A set _{I 100} ^ shown FIG consisting of 100 kinds of the intensity distribution _I 1 _{~ I 100} obtained for M = 100. It is a figure which shows the restored image when using the set of optimized intensity distribution. It is a figure which shows the restored image when using a set of random intensity distribution.

(Embodiment 1)
Hereinafter, the present invention will be described based on preferred embodiments with reference to the drawings. The same or equivalent constituent elements, members, and processes shown in each drawing will be denoted by the same reference numerals, and duplicative description will be appropriately omitted. Further, the embodiments are merely examples and do not limit the invention, and all features and combinations thereof described in the embodiments are not necessarily essential to the invention.

“The intensity distribution is random” in the present specification does not mean that it is completely random, but may be random as long as an image can be reconstructed in ghost imaging. Therefore, "random" in the present specification can include a certain degree of regularity therein. Also, "random" does not require to be completely unpredictable, but may be predictable and reproducible.

FIG. 1 is a diagram showing a vehicle lamp 400 according to the first embodiment. The vehicle lamp (or lamp system) 400 includes a headlamp 410 and a pseudo heat light source 420. The headlamp 410 and the pseudo heat light source 420 are housed in the housing 402. The front surface of the housing 402 is covered with a transparent cover 404. The headlight 410 includes a low beam, a high beam, or both, and emits a beam Sb for forming a light distribution in front of the vehicle.

The pseudo heat light source 420 constitutes the imaging device 100 together with the photodetector 120 and the arithmetic processing device 130. The photodetector 120 and the arithmetic processing unit 130 may be built in the housing 402 or may be provided outside the housing 402.

2 is a diagram showing the imaging apparatus 100 of FIG. The imaging apparatus 100 is a correlation function image sensor (also referred to as single pixel imaging) that uses the principle of ghost imaging, and includes a pseudo thermal light source 110 (pseudo thermal light source 420 in FIG. 1), a photodetector 120, and an arithmetic processing unit 130. .. The imaging device 100 is also called a quantum radar camera.

The pseudo heat light source 110 generates the reference light S1 having the intensity distribution I(x, y) that can be regarded as substantially random, and irradiates the object OBJ. Irradiation of the reference light S1 onto the object OBJ is performed while changing its intensity distribution according to a plurality of M patterns. The pseudo-thermal light source 110 may include, for example, a light source 112 that generates a light S0 having a uniform intensity distribution, and a patterning device 114 that can spatially modulate the intensity distribution I of the light S0. The light source 112 may use a laser, a light emitting diode, or the like. The wavelength and spectrum of the reference light S1 are not particularly limited, and may be white light having a plurality of or continuous spectra, or monochromatic light having a predetermined wavelength.

A DMD (Digital Micromirror Device) or a liquid crystal device can be used as the patterning device 114. The pattern signal PTN (image data) designating the intensity distribution I is given to the patterning device 114 from the arithmetic processing unit 130. Therefore, the arithmetic processing unit 130 currently receives the reference light S1 irradiated on the object OBJ. Know the intensity distribution I _r .

The photodetector 120 measures the reflected light from the object OBJ and outputs a detection signal D _r . The detection signal D _r is a spatial integral value of light energy (or intensity) incident on the photodetector 120 when the object OBJ is irradiated with the reference light having the intensity distribution I _r . Therefore, as the photodetector 120, a single-pixel photodetector (photodetector) can be used. The photodetector 120 outputs a plurality of detection signals D ₁ to D _M corresponding to a plurality of _M intensity distributions I ₁ to I _M, respectively.

The arithmetic processing unit 130 includes a pattern generator 132 and a reconstruction processing unit 134. The pattern generator 132 generates a pattern signal PTN _r designating the intensity distribution I of the reference light S1 and switches the pattern signal PTN _r with time (r=1, 2,... M).

The arithmetic processing unit 130 can be implemented by a combination of a processor (hardware) such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a microcomputer, and a software program executed by the processor (hardware). The arithmetic processing unit 130 may be a combination of a plurality of processors. Alternatively, the arithmetic processing unit 130 may be composed of only hardware.

The intensity distribution of the reference light S1 generated by the pseudo heat light source 110 may be randomly generated each time. Alternatively, a set of a plurality of intensity distributions I ₁ to I _M may be defined in advance. In this case, a set of the plurality of pattern signals PTN ₁ to PTN _M defining the plurality of intensity distributions I ₁ to I _M may be stored in advance in a memory (pattern memory) inside the pattern generator 132.

The reconstruction processing unit 134 reconstructs the restored image G(x, y) of the object OBJ by correlating the plurality of intensity distributions I ₁ to I _M and the plurality of detected intensities b ₁ to b _M. The detection intensities b ₁ to b _M are based on the detection signals D ₁ to D _M. The relationship between the detection intensity and the detection signal may be determined in consideration of the type and method of the photodetector 120.

It is assumed that the reference light S1 having a certain intensity distribution I _r is irradiated for a certain irradiation period. The detection signal D _r is assumed to represent the amount of light received at a certain time (or a minute time), that is, an instantaneous value. In this case, the detection signal D _r may be sampled multiple times during the irradiation period, and the detection intensity b _r may be an integrated value, an average value, or a maximum value of all sampling values of the detection signal D _r . Alternatively, some of all the sampling values may be selected, and the integrated value, average value, or maximum value of the selected sampling values may be used. In the selection of a plurality of sampling values, for example, the order xth to yth counting from the maximum value may be extracted, sampling values lower than an arbitrary threshold value may be excluded, or the magnitude of signal fluctuation may be excluded. It is also possible to extract a sampling value in a range where is small.

When a device such as a camera whose exposure time can be set is used as the photodetector 120, the output D _r of the photodetector 120 can be directly used as the detection intensity b _r .

The conversion from the detection signal D _r to the detection intensity b _r may be performed by the arithmetic processing device 130 or may be performed outside the arithmetic processing device 130.

The correlation function of Expression (1) is used for the correlation. I _r is the r-th intensity distribution, and b _r is the r-th detected intensity value.

The above is the basic configuration of the entire imaging apparatus 100. Returning to FIG.

In the present embodiment, the pseudo heat light source 420 is built in the vehicle lamp 400. At least a part of the components (common member) 430 of the headlamp 410 is shared with the pseudo heat light source 420.

The above is the configuration of the vehicle lamp 400.

The headlight 410 is designed with a light source and an optical system for the purpose of irradiating light up to several tens of meters. Therefore, by incorporating the pseudo heat light source 420 of the imaging device in the vehicle lamp 400 and diverting some of the components of the headlight 410 to the pseudo heat light source 420, the reference light S1 can be accurately applied to a distant object. Can be irradiated. Moreover, since the number of overlapping members can be reduced, the overall cost can be reduced.

The present invention extends to various devices and methods understood as the block diagram of FIG. 1 or derived from the above description, and is not limited to a specific configuration. Hereinafter, more specific configuration examples and examples will be described in order to help understanding of the essence and operation of the invention and to clarify them, not to narrow the scope of the invention.

(Example 1)
FIG. 3 is a diagram showing a vehicular lamp 400A according to the first embodiment. The pseudo heat light source 420 shares an optical system with the headlight 410. The headlamp 410 includes a light source 412 and a reflector 414. The light source 412 includes a white light emitting diode (or a semiconductor laser) and its lighting circuit. The reflector 414 reflects the light emitted from the light source 412 toward the front of the vehicle.

In the first embodiment, the reflector 414 of the headlight 410 is the common member 430 of FIG. 1 and is shared with the pseudo heat light source 420. Pseudo thermal light source 420 includes light source 422 and patterning device 424. The beam whose intensity distribution is randomized by the patterning device 424 is reflected by the reflector 414 toward the front of the vehicle.

The light S0 generated by the light source 422 may be infrared light or ultraviolet light. In this case, the photodetector 120 is insensitive in the visible wavelength band and may be configured to have sensitivity only to the wavelength of the light S0 (reference light S1). Thereby, the sensing by the imaging device 100 is not affected by the headlight.

The light S0 generated by the light source 422 may include a single wavelength in the visible range or may be white light. In this case, the photodetector 120 is sensitive to both the beam Sb of the headlight 410 and the reference light S1. In this case, the influence of the beam Sb on the output of the photodetector 120 may be reduced by arithmetic processing. For example, when the beam Sb can be regarded as direct current and the reference light S1 can be regarded as alternating current, the influence of the beam Sb may be reduced by a high-pass filter. Alternatively, simply, the offset process of subtracting the estimated value of the component caused by the beam Sb may be performed.

(Example 2)
FIG. 4 is a diagram illustrating a vehicular lamp 400B according to the second embodiment. Also in the second embodiment, the pseudo heat light source 420 shares the optical system with the headlight 410.

In the second embodiment, the headlight 410 is a variable light distribution lamp (ADB: Adaptive Driving Beam), and includes a light source 412, a patterning device 416, and a reflector 414. The light source 412 includes a white LED or LD and its lighting circuit. The patterning device 416 is, for example, a DMD, and spatially modulates the intensity distribution of the light emitted from the light source 412 so as to obtain a desired light distribution pattern. The reflector 414 reflects the light flux corresponding to an on-pixel in the reflected light of the patterning device 416 to the front of the vehicle.

In the second embodiment, the patterning device 416 and the reflector 414 are the common member 430 shared with the pseudo heat light source 420. The emitted light S0 of the light source 422 is incident on the patterning device 416 and is randomly modulated, and the reference light S1 is generated.

In the second embodiment, since the patterning device 416 is shared, it is necessary to suppress mutual influence of the pattern (intensity distribution) of the beam Sb and the pattern of the reference light S1. For that purpose, the light source 422 and the light source 412 may be complementarily turned on. For example, the light source 412 and the light source 422 are alternately turned on in a time-sharing manner, and during the lighting period of the light source 412, image data (light distribution image data) PTNb corresponding to the light distribution pattern is given to the patterning device 416, The random image data (random image data) PTN ₁ to PTN _M may be given to the patterning device 416 (first pattern control). FIG. 5 is a diagram illustrating the first pattern control in the second embodiment.

Alternatively, a case in which it is sufficient to perform the sensing by the imaging device 100 only within the range (irradiation region) where the beam Sb is irradiated is assumed. In this case, the light source 412 and the light source 422 are simultaneously turned on, and the patterning device 416 calculates the light distribution image data PTNb and the random image data PTN _i (i=1, 2,... M) for each pixel to perform patterning. It may be given to the device 416 (second pattern control). FIG. 6 is a diagram illustrating the second pattern control in the second embodiment. ON indicates an irradiation area defined by the light distribution pattern, and OFF indicates a light shielding area defined by the light distribution pattern. If the pixel value corresponding to lighting is 1 and the pixel value corresponding to extinction is 0, the pixel value of the image data Sb is 1 in the irradiation area ON and 0 in the light shielding area OFF. The random image data PTN _i has pixel values in which 1 and 0 are randomly distributed. The pattern of FIG. 6 can be generated by calculating the logical product of the random image data PTN _i and the light distribution image data PTNb.

(Example 3)
FIG. 7 is a diagram illustrating a vehicle lamp 400C according to the third embodiment. In the third embodiment, all the components of the headlight 410 are shared with the pseudo heat light source 420. That is, the headlight 410 has the function of the pseudo heat light source 420. In the third embodiment, the reference light S1 naturally becomes white light.

Control of the patterning device 416 in Example 3 will be described.
First Control In the third embodiment, the normal lighting period Tb and the sensing period Ts may be switched in a time division manner. FIG. 8 is a diagram illustrating the first control in the third embodiment. As shown in FIG. 8, the pattern of the reference light S1 may be switched a plurality of times during one sensing period Ts.

-Second control The second pattern control of the second embodiment may be performed.

-Third control FIG. 9 is a diagram illustrating the third control in the third embodiment. Some patterning devices such as DMDs are capable of gradation control. In this case, corresponding pixel values of the random image data PTN _i and the light distribution image data PTNb may be added together and given to the patterning device 416. In this case, the intensity distribution of the outgoing beam of the vehicular lamp 400 changes randomly with time with the base level determined by the light distribution image data PTNb as a reference.

(Example 4)
In the first to third embodiments, the pseudo thermal light source 420 is composed of the combination of the light source 422 and the patterning device 424, but it is not limited thereto. The pseudo heat light source 420 is composed of an array of a plurality of semiconductor light sources (LED (light emitting diode) or LD (laser diode)) arranged in a matrix, and can control ON/OFF (or brightness) of each semiconductor light source. You may comprise.

In the first embodiment, the illumination device 110 is composed of the combination of the light source 112 and the patterning device 114, but it is not limited thereto. The illuminating device 110 is composed of an array of a plurality of semiconductor light sources (LED (light emitting diode) or LD (laser diode)) arranged in a matrix, and it is possible to control ON/OFF (or brightness) of each semiconductor light source. You may comprise.

Next, the use of the imaging device 100 will be described.

FIG. 10 is a block diagram of the object identification system 10. The object identification system 10 is mounted on a vehicle such as an automobile or a motorcycle and determines the type (category) of an object OBJ existing around the vehicle.

The object identification system 10 includes an imaging device 100 and an arithmetic processing device 40. As described above, the imaging apparatus 100 irradiates the object OBJ with the reference light S1 and measures the reflected light S2 to generate the restored image G of the object OBJ.

The arithmetic processing device 40 processes the output image G of the imaging device 100 and determines the position and type (category) of the object OBJ.

The classifier 42 of the arithmetic processing device 40 receives the image G as an input and determines the position and type of the object OBJ included in the image G. The classifier 42 is implemented based on the model generated by machine learning. The algorithm of the classifier 42 is not particularly limited, but YOLO (You Only Look Once), SSD (Single Shot Multi Box Detector), R-CNN (Region-based Convolutional Neural Network), SPPnet (Spatial Pyramid Pooling), Faster R-CNN , DSSD (Deconvolution-SSD), MaskR-CNN, etc. can be adopted, or an algorithm developed in the future can be adopted.

Information about the object OBJ detected by the arithmetic processing unit 40 may be used for light distribution control of the vehicular lamp 200. Specifically, an appropriate light distribution pattern can be generated based on the information on the type and position of the object OBJ generated by the arithmetic processing device 40.

Information regarding the object OBJ detected by the arithmetic processing unit 40 may be transmitted to the vehicle-side ECU. The vehicle-side ECU may perform automatic driving based on this information.

The above is the configuration of the object identification system 10. By using the imaging device 100 as the sensor of the object identification system 10, noise resistance is significantly improved. For example, it is difficult to recognize the object OBJ with naked eyes when it is raining, snowing, or traveling in fog, but even in such a situation, the restored image G of the object OBJ is not affected by rain, snow, or fog. Can be obtained.

FIG. 11 is a diagram showing an automobile. The automobile 300 includes

vehicle lamps

302L and 302R. As described above, the pseudo heat light source 420 is built in at least one of the

vehicular lamps

302L and 302R in a mode in which a part of the hardware is shared with the headlight.

FIG. 12 is a block diagram showing a vehicle lamp 200 including an object detection system 210. The vehicle lamp 200 constitutes a lamp system 310 together with the vehicle-side ECU 304. The vehicular lamp 200 includes a light source 202, a lighting circuit 204, and an optical system 206. Further, the vehicle lighting device 200 is provided with an object detection system 210. The object detection system 210 corresponds to the object identification system 10 described above, and includes the imaging device 100 and the arithmetic processing device 40.

Information about the object OBJ detected by the arithmetic processing unit 40 may be used for light distribution control of the vehicular lamp 200. Specifically, the lamp-side ECU 208 generates an appropriate light distribution pattern based on the information regarding the type and the position of the object OBJ generated by the arithmetic processing device 40. The lighting circuit 204 and the optical system 206 operate so that the light distribution pattern generated by the lamp-side ECU 208 is obtained.

Information regarding the object OBJ detected by the arithmetic processing unit 40 may be transmitted to the vehicle-side ECU 304. The vehicle-side ECU may perform automatic driving based on this information.

(Embodiment 2)
FIG. 13 is a diagram showing the imaging apparatus 100 according to the second embodiment. The imaging device 100 is a correlation function image sensor that uses the principle of ghost imaging, and includes an illumination device 110, a photodetector 120, and an arithmetic processing device 130. The imaging device 100 is also called a quantum radar camera.

The lighting device 110 is a pseudo heat light source, and generates the reference light S1 having the intensity distribution I that can be regarded as substantially random, and irradiates the object OBJ. FIG. 14 is a diagram illustrating the intensity distribution of the reference light S1 according to the second embodiment. In the figure, the part where the intensity is zero is shown in white, and the part where the intensity is not zero is shown in black.

In the present embodiment, the measurement range 600 is divided into a plurality of sections 602_1 to 602_N. In this example, it is divided into 4 in the vertical direction and 4 in the horizontal direction, and N=16. The illumination device 110 irradiates the reference light S1 in which the intensity distribution I(x, y) in the irradiation section is substantially random, while switching the section (referred to as an irradiation section) 602_i that irradiates light. The intensity in the sections other than the irradiation section (called non-irradiation section) is zero.

Each of the plurality of sections 602_1 to 602_N is irradiated with the reference light S1 having M random intensity distributions. Therefore, the total number of irradiation times per sensing is M×N. When the i-th section (1≦i≦N) is selected, the j-th intensity distribution is I _i,j , and the reference light S1 at that time is shown as S1 _i,j .

Return to FIG. The photodetector 120 measures the reflected light from the object OBJ and outputs a detection signal D. The detection signal D _i,j is a spatial integral value of light energy (or intensity) incident on the photodetector 120 when the object OBJ is irradiated with the reference light having the intensity distribution I _i,j . Therefore, as the photodetector 120, a single-pixel photodetector (photodetector) can be used.

Reference light S1 _i,j (iε1 to N) having M×N intensity distributions I _1,1 to I _1,M , I _2,1 to I _2,M , I _N,1 to I _N,M , Jε1 to M), the photodetector 120 outputs M×N detection signals D _i,j (iε1 to N, jε1 to M). The order of irradiation is not particularly limited.

For example, after setting M intensity distributions for one irradiation section, the next irradiation section may be selected. The order of selecting the irradiation sections is not particularly limited, and the irradiation sections can be selected according to a predetermined rule. For example, the sections in the first row may be selected in order from left to right, and after moving to the rightmost, the next row may be moved. Alternatively, the sections in the first column may be selected in order from top to bottom, and after moving to the bottom, move to the next row.

The illuminator 110 may include, for example, a light source 112 that generates a light S0 having a uniform intensity distribution, and a patterning device 114 that can spatially modulate the intensity distribution of the light S0. The light source 112 may use a laser, a light emitting diode, or the like. The wavelength and spectrum of the reference light S1 are not particularly limited, and may be white light having a plurality of or continuous spectra, or monochromatic light having a predetermined wavelength.

As the patterning device 114, a DMD (Digital Micromirror Device) or a liquid crystal device can be used. In this embodiment, the patterning device 114 covers the entire measurement range 600 and has the ability to simultaneously illuminate the entire measurement range 600, but turns off the pixels corresponding to the non-illuminated sections of the patterning device 114. As a result, it is possible to give a random pattern only to the irradiation section.

A pattern signal PTN _i,j (image data) designating the intensity distribution I _i,j is given to the patterning device 114 from the arithmetic processing unit 130. Therefore, the arithmetic processing unit 130 knows the current position of the irradiation section and the intensity distribution I _i,j of the reference light S1.

The arithmetic processing unit 130 includes a pattern generator 132 and a reconstruction processing unit 134.

The pattern generator 132 may randomly generate the intensity distribution I _i,j of the reference light S1 each time. In this case, the pattern generator 132 may include a pseudo random signal generator.

Alternatively, a set of a plurality of intensity distributions I _i,j may be defined in advance. For example, a plurality (for example, M) of sets of intensity distributions I ₁ to I _M having the same size as the partition 602 may be defined in advance. When the i-th section is the irradiation section, I ₁ to I _M may be assigned to the irradiation section in order or randomly.

In this case, a set of a plurality of pattern signals defining a plurality of intensity distributions I ₁ to I _M may be held in advance in a memory (pattern memory) inside the pattern generator 132.

The reconstruction processing unit 134, for each of the plurality of sections 602_1 to 602_N (602_i), a plurality of detection intensities b _i,1 to b _i,M and an intensity distribution I _{i of the} reference light S1 _i,1 to S1 _i,M _. By taking the correlation of ₁ to I _i,M , the restored image G _{i of the} part of the object OBJ included in the section 602_i is reconstructed.

The detection intensities b _i,1 to b _i,M are based on the detection signals D _i,1 to D _i,M . The relationship between the detection intensity b _i,j and the detection signal D _i,j may be determined in consideration of the type and method of the photodetector 120.

It is assumed that the reference light S1 having a certain intensity distribution I _i,j is irradiated for a certain irradiation period. Further, the detection signal D _i,j is assumed to represent the amount of received light at a certain time (or a minute time), that is, an instantaneous value. In this case, the detection signal D _i,j may be sampled a plurality of times during the irradiation period, and the detection intensity b _i,j may be an integral value, an average value, or a maximum value of all sampling values of the detection signal D _i,j . Alternatively, some of all the sampling values may be selected, and the integrated value, average value, or maximum value of the selected sampling values may be used. In the selection of a plurality of sampling values, for example, the order xth to yth counting from the maximum value may be extracted, sampling values lower than an arbitrary threshold value may be excluded, or the magnitude of signal fluctuation may be excluded. It is also possible to extract a sampling value in a range where is small.

When a device such as a camera whose exposure time can be set is used as the photodetector 120, the output D _i,j of the photodetector 120 can be directly used as the detection intensity b _i,j .

The conversion from the detection signal D _i,j to the detection intensity b _i,j may be performed by the arithmetic processing device 130 or may be performed outside the arithmetic processing device 130.

The correlation function of Expression (2) is used to restore the image G _i of the i-th (1≦i≦N) section 602 — _i . I _i,j is the j-th (1≦j≦M) intensity distribution, and b _i,j is the j-th detected intensity value.

By combining the restored images G ₁ to G _N of all N sections 602_1 to 602_N, an image of the entire measurement range can be obtained.

The above is the configuration of the imaging apparatus 100. Then, the advantage is demonstrated.

The number of calculations in this imaging apparatus 100 is as follows.
The number of pixels in the entire measurement range is X×Y, and the number of pixels in the horizontal and vertical directions in one section is x and y. However, X×Y=(x×y)×N.

Assuming that one pattern irradiation is required to restore one pixel, the irradiation number M′ required to restore x×y pixels is
M′=(x×y)
Becomes It should be noted that the actual number of irradiations may be larger or smaller than M′, but is approximately proportional to (x×y). Therefore, the number of calculations per partition is (x×y) ² . The total number of operations O′ for all N intervals is
O′=N×(x×y) ²
Becomes

The calculation number O of the conventional method of irradiating the entire measurement range is
O=(X×Y) ²
Met. Since the relationship of X×Y=(x×y)×N is established, according to the present embodiment, the number of calculations can be reduced to O′/O=1/N times as compared with the conventional case.

For example, consider the case of X=Y=1024. In this case, when divided into sections of x=y=32, N=32×32=1024, and the number of calculations can be reduced to 1/1024 times.

Further, when the intensity distribution of the reference light is stored in the memory in the pattern generator 132, the intensity distribution of one section (x×y) may be held instead of the entire irradiation area (X×Y). The memory capacity can be reduced.

ㆍReduction in the number of calculations means that the calculation time can be shortened when using the same speed processor. Alternatively, a slower (and thus cheaper) processor can be employed to finish the process in the same amount of time.

Note: For in-vehicle use, it is necessary to sense the measurement range at a frame rate that is somewhat high. In the present embodiment, the number of irradiations M is increased N times (N is the number of sections) in exchange for the reduction in the number of calculations (and thus the calculation time), resulting in an increase in measurement time. FIG. 15 is a diagram for explaining the trade-off between calculation time and measurement time.

The number N of partitions may be set so that the decrease amount δ1 of the calculation time due to the division is larger than the increase amount δ2 of the measurement time. As a result, the frame rate required for in-vehicle use can be realized.

Next, a modified example related to the second embodiment will be described.

(Modification 2.1)
In the embodiment, the number of irradiations M is the same for each section, but the number of irradiations M may be different for each section. When the irradiation number of the i-th section is written as M _i , the calculation number is
O=Σ _i=1:N M _i ×(x×y)
Becomes

The more the irradiation number M _i is, the more accurate the image can be restored, but depending on the position of the section, the accuracy may not be required so much. Therefore, by optimizing the irradiation number for each section, the number of calculations (calculation time) and the measurement time can be adjusted for each section.

(Modification 2.2)
The division method of the section is not limited to the above. 16(a) and 16(b) are diagrams showing a modified example of a section. As shown in FIG. 16A, it may be divided into horizontally long sections. Alternatively, it may be divided into vertically long sections.

In the embodiment, the sizes (number of pixels) of a plurality of partitions are the same, but this is not the case. As shown in FIG. 16B, the number of pixels may be different for each section.

(Modification 2.3)
In the embodiment, the patterning device 114 that covers the entire measurement range 600 is used, but it is not limited thereto. The illuminating device 110 having the irradiation ability for one section may be provided, and the emitted light may be scanned in the horizontal direction or the horizontal direction using the movable mirror.

(Modification 2.4)
Although the illumination device 110 is configured by the combination of the light source 112 and the patterning device 114 in the second embodiment, the present invention is not limited to this. The illuminating device 110 is composed of an array of a plurality of semiconductor light sources (LED (light emitting diode) or LD (laser diode)) arranged in a matrix, and it is possible to control ON/OFF (or brightness) of each semiconductor light source. You may comprise.

(Use)
Next, the application of the imaging device 100 according to the second embodiment will be described. The imaging device 100 can be used for the object identification system 10 of FIG. By using the imaging device 100 described in the second embodiment as the sensor of the object identification system 10, the following advantages can be obtained.

First, the use of the imaging device 100, that is, the quantum radar camera significantly improves noise resistance. For example, when it is raining, snowing, or traveling in fog, it is difficult for the naked eye to recognize the object OBJ, but by using the imaging device 100, the object OBJ is not affected by rain, snow, or fog. The restored image G can be obtained.

Second, the calculation range can be reduced by dividing the measurement range into multiple sections and restoring the image for each section. This makes it possible to increase the frame rate or select an inexpensive processor as the arithmetic processing unit.

Note that in the imaging apparatus 100, the number N of sections may be adaptively changed according to the traveling environment.

The imaging device 100 described in the second embodiment can be mounted on the vehicle shown in FIG. 11 and may be incorporated in the vehicle lighting device shown in FIG.

(Embodiment 3)
FIG. 17 is a diagram showing the imaging apparatus 100 according to the third embodiment. The imaging device 100 is a correlation function image sensor that uses the principle of ghost imaging, and includes an illumination device 110, a photodetector 120, and an arithmetic processing device 130. The imaging device 100 is also called a quantum radar camera.

The lighting device 110 is a pseudo heat light source, and generates the reference light S1 having the intensity distribution I(x, y) that can be regarded as substantially random, and irradiates the object OBJ. Irradiation of the reference light S1 onto the object OBJ is performed while changing its intensity distribution according to a plurality of M patterns.

The lighting device 110 includes a light source 112 and a patterning device 114. The light source 112 generates the light S0 having a uniform intensity distribution. The light source 112 may use a laser, a light emitting diode, or the like. The wavelength and spectrum of the reference light S1 are not particularly limited, and may be white light having a plurality of or continuous spectra, or monochromatic light having a predetermined wavelength. The wavelength of the reference light S1 may be infrared or ultraviolet.

The patterning device 114 has a plurality of pixels arranged in a matrix, and the intensity distribution I of light can be spatially modulated based on the combination of ON and OFF of the plurality of pixels. In this specification, a pixel in an on state is referred to as an on pixel, and a pixel in an off state is referred to as an off pixel. Note that, in the following description, for ease of understanding, each pixel takes only two values (1, 0) of ON and OFF, but the present invention is not limited to this and may take an intermediate gradation.

As the patterning device 114, a reflective DMD (Digital Micromirror Device) or a transmissive liquid crystal device can be used. A pattern signal PTN (image data) generated by the pattern generator 116 is applied to the patterning device 114. The patterning device 114 is assumed to be a DMD.

The pattern generator 116 generates a pattern signal PTN _r designating the intensity distribution I _r of the reference light S1 and switches the pattern signal PTN _r with time (r=1, 2,... M).

The arithmetic processing unit 130 includes a reconstruction processing unit 134. The reconstruction processing unit 134 reconstructs the restored image G(x, y) of the object OBJ by correlating the plurality of intensity distributions I ₁ to I _M and the plurality of detected intensities b ₁ to b _M.

The detection intensities b ₁ to b _M are based on the detection signals D ₁ to D _M. The relationship between the detection intensity and the detection signal may be determined in consideration of the type and method of the photodetector 120.

The correlation function of Expression (3) is used for the correlation. I _r is the r-th intensity distribution, and b _r is the r-th detected intensity value.

The arithmetic processing unit 130 can be implemented by a combination of a processor (hardware) such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a microcomputer, and a software program executed by the processor (hardware). The arithmetic processing unit 130 may be a combination of a plurality of processors. Alternatively, the arithmetic processing unit 130 may be composed of only hardware. The pattern generator 116 may be mounted inside the arithmetic processing unit 130.

The above is the basic configuration of the entire imaging apparatus 100. Subsequently, control of the patterning device 114 will be described based on some examples.

(Example 3.1)
FIGS. 18A to 18C are views for explaining the pixels of the DMD that is the patterning device 114. As shown in FIG. 18A, the DMD is an array of a plurality of pixels PIX arranged in a matrix of m rows and n columns. As shown in FIG. 18B, each pixel PIX is a square mirror, and can be tilted in the ON direction and the OFF direction about a hinge provided diagonally as an axis. The patterning device 114 is configured so that all pixels can be independently turned on and off. In the following description, the shape of the matrix is simplified and shown as shown in FIG.

The pattern generator 116 controls the intensity distribution of the reference light (that is, the pattern signal PTN) in units of the pixel block B including at least one pixel, and the pixel block B is variable. The pixel block B can be understood as a set of continuous (adjacent) ON pixels (or a set of OFF pixels, or a set of ON pixels and OFF pixels). In Example 3.1, the size of the pixel block B is variable.

19A to 19D are diagrams showing pixel blocks B having different sizes. The size can be grasped as the number of pixels (that is, the area) included in the pixel block B. FIGS. 19A to 19D show pixel blocks B _1×1 to B _4×4 of vertical and horizontal 1×1 pixels, 2×2 pixels, 3×3 pixels, and 4×4 pixels, respectively. Pixels included in the same pixel block B are in the same state (on, off).

FIGS. 20A and 20B are diagrams showing examples of pattern signals (image data) PTN based on pixel blocks B having different sizes. In the following, we assume patterning device 114 with m=n=16.

In FIG. 20A, the pixel block B _2×2 of 2×2 pixels in FIG. 19B is applied, and ON/OFF is controlled for each pixel block B _2×2 . The pattern signal PTN changes in M ways per sensing. The hatched pixel PIX is an ON pixel, and the pixel PIX that is not hatched is an OFF pixel.

In FIG. 20B, the pixel block B _4×4 of 4×4 pixels in FIG. 19D is applied, and ON/OFF is controlled for each pixel block B _4×4 . The number M of the pattern signals PTN may be different according to the size of the pixel block. Generally, by increasing the size of the pixel block and decreasing the number of pixel blocks, the pattern per sensing is performed once. It is possible to reduce the number M of

FIG. 21 is a diagram showing a modification of the pattern control. The arrangement of 4×4 pixel pixel blocks B _4×4 is not perfectly aligned in the horizontal direction, but is offset by 2 pixels in the horizontal direction in some rows.

The above is the pattern control according to the embodiment 3.1. It can be understood that this pattern control dynamically changes the effective resolution of the patterning device 114. The calculation amount in the reconstruction processing unit 134 increases according to the resolution. However, in a situation where spatial resolution is not so required, the calculation amount can be reduced by increasing the size of the pixel block B.

Alternatively, if you want to capture an object OBJ that moves at high speed, you can restore a blur-free image by increasing the size of the pixel block B and reducing the number of patterns to reduce the irradiation time. Further, the frame rate can be increased and the movement of the object OBJ can be followed by reducing the calculation amount and the number of irradiations.

(Example 3.2)
In Example 3.1, the same pixel block B was included in one pattern, but this is not the case. In the embodiment 3.2, the layout of pixel blocks having different sizes is defined in advance and selected according to the running scene. 22A and 22B are diagrams illustrating the layout of the pixel blocks B having different sizes according to the running scene.

Referring to FIG. 22A, the pixel block B _S having a smaller size is arranged in the lower region, and the pixel block B _L having a larger size is arranged in the upper region. In a driving scene in the suburbs, the sky (a space in which no vehicle or pedestrian exists) extends above the front of the vehicle, and therefore the size of the pixel block B is increased to reduce the resolution. On the other hand, the lower side corresponds to the road surface and there is a high possibility that an important object (or road surface sign) is present. Therefore, the size of the pixel block B is reduced to improve the resolution.

Referring to FIG. 22B, a pixel block B _S having a smaller size is arranged closer to the center, and a pixel block B _L having a larger size is arranged closer to the outer periphery. In a driving scene such as on a highway, the vanishing point is located near the center of the screen, and a distant oncoming vehicle appears from the vanishing point, the size is small at the beginning, and the size increases as the vehicle approaches. According to the layout of FIG. 22B, it is easy to detect a small object OBJ that looks small in the distance.

(Example 3.3)
In the embodiment 3.2, a plurality of layouts are defined in advance, and one suitable for the running scene is adaptively selected from the plurality of layouts. May be dynamically changed.

FIG. 23 is a diagram illustrating a dynamic layout of pixel blocks B having different sizes. In one frame 1, a pattern of pixel blocks B _{2×2 of} uniform size (2×2 pixels) is used. The position of the object OBJ is estimated from the image restored in this frame 1. Then, in the next frame 2, the pixel block B having a smaller size may be arranged in the area where the object OBJ exists, and the size of the pixel block B may be increased as the distance from the pixel block B increases.

(Modifications related to Examples 3.1 to 3.3)
(Modification 1)
Although the pixel block B has a square shape in the above description, the shape is not limited thereto. 24A to 24C are diagrams showing a pixel block B according to Modification 1. As shown in FIG. The pixel block B is a horizontally long rectangle, and its size dynamically changes.

(Modification 2)
In Examples 3.1 to 3.3, the vertical direction and the horizontal direction are changed on the same scale, but only the number of pixels in the vertical direction or only the number of pixels in the horizontal direction may be changed. 25A to 25D are diagrams showing a pixel block B according to the second modification. In this example, the pixel block B has a variable number of pixels in the horizontal direction.

(Example 3.4)
In Embodiments 3.1 to 3.3, the case where the size of the pixel block B is changed has been described. In Example 3.4, the shape of the pixel block B is adaptively changed.

170(a) to 170(d) are diagrams showing pixel blocks B having different shapes. 170(a) shows a pixel block B _X long in the horizontal direction (X direction), FIG. 170(b) shows a pixel block B _Y long in the vertical direction (Y direction), and FIG. , A diagonally long pixel block B _XY is shown. FIG. 170(d) shows a basic square pixel block B _S. The pixel blocks B in FIGS. 170(a) to 170(d) have the same size.

27A to 27C are diagrams showing an example of the pattern signal PTN based on pixel blocks having different shapes. At least one of the length and the position of the pixel block B is randomly determined. 27A shows an example of the pattern signal PTN _X using the horizontally long pixel block B _X , and FIG. 27B shows an example of the pattern signal PTN _Y using the vertically long pixel block B _Y. (C) shows an example of the pattern signal PTN _XY using the diagonal pixel blocks B _XY .

Consider the relative motion of the object OBJ and the imaging device 100.

When the object OBJ moves in the horizontal direction relative to the imaging apparatus 100, when the pattern signal PTN _X including the horizontally long pixel block B _X is used, the horizontal pixel block B _S is used as compared with the case where the square pixel block B _S is used. Since the effective resolution in the direction decreases, the sharpness of the image in the horizontal direction decreases, or the detection accuracy of the horizontal position decreases, but the capture time (in other words, the exposure time) increases and the detection intensity D Is increased, the S/N ratio is increased to facilitate detection (that is, the sensitivity is increased).

When the object OBJ moving in the vertical direction is sensed using the same pattern signal PTN _X , the detection time is shorter because the capture time is shorter than when the square pixel block B _S is used, and the detection sensitivity is reduced. However, the vertical resolution is improved and the vertical position detection accuracy is increased.

The same idea can be applied to the pattern signal PTN _Y and the pattern signal PTN _Z. As a generalization, the detection sensitivity can be increased for an object moving in the direction in which the pixel block B extends, and a sharp image can be obtained or the position detection accuracy can be increased for an object moving in a direction perpendicular to the detection direction.

FIGS. 28A and 28B are views for explaining sensing based on the pattern signal PTN having the pixel block B having a characteristic shape. First, as shown in FIG. 28A, the pattern signal PTN _S including the pixel block B _S of a certain shape (square of 5×5 pixels) is used for sensing. By this sensing, it is detected that the object OBJ is a car and is moving laterally. In the sensing based on the pattern signal PTN _S , the position of the tip of the object OBJ is determined to be X′, and there is an error from the actual position X of the tip of the object OBJ.

Therefore, in order to accurately detect the lateral position of the object OBJ, switching to the pattern signal PTN _Y comprising elongated pixel block B _Y. As a result, the lateral resolution is increased, so that the position of the tip of the object OBJ is determined to be X″, and it is possible to approach the position X of the tip of the actual object OBJ.

Since the object OBJ does not move in the vertical direction, it can be said that there is almost no demerit of using the vertically long pixel block _BY .

In the examples of FIGS. 28A and 28B, the shape of the pixel block B is switched in order to positively capture a certain object OBJ, but this is not the only option, and in order to erase a specific object OBJ The shape of the block B may be used. For example, it can be said that rain and snow are noise for an in-vehicle sensing device and there is no need to restore an image. Since the moving directions of rain and snow are constant, optimizing the shape of the pixel block B makes it easier to eliminate the influence of rain and snow.

For example, when it is raining in the vertical direction, a pixel block B having a shape that is short in the vertical direction (vertical direction), in other words, long in the horizontal direction, is suitable. In a series of sensing operations for continuously irradiating a plurality of patterns, the light of a specific pixel block reaches the object OBJ on the other side of the rain and returns to the photodetector 120 on the front side of the rain, The probability that the light of a specific pixel block will be significantly affected (blocked) by the raindrops is reduced. That is, since the influence of rain on each pixel is made uniform, the process of removing the influence of rain (noise canceling) becomes easy.

(Example 3.5)
In Examples 3.1 to 3.4 described above, all the pixels included in the same pixel block B are on (or off). On the other hand, in Example 3.5, the same pixel block B includes both ON pixels and OFF pixels. That is, the pixel block B includes two or more ON pixels and OFF pixels which are arranged in a predetermined manner. Such a pixel block is called a pattern block. The illumination device 110 defines the intensity distribution by the combination of pattern blocks.

FIGS. 29A to 29D are diagrams for explaining the pattern block PB according to the embodiment 3.5. 29A to 29D show distributions (patterns) of ON pixels and OFF pixels in the pattern block PB.

In FIG. 29A, the off pixels are arranged along the four sides of the pattern block PB, and in FIG. 29B, the off pixels are arranged along the two adjacent sides of the pattern block PB.

In FIG. 29(c), the ON pixels are arranged so as to cross diagonally. In FIG. 29D, the ON pixels are arranged so as to cross each other vertically and horizontally.

30A and 30B are diagrams showing examples of pattern signals based on the combination of pattern blocks. FIG. 30A is an example of the pattern signal formed by the pattern block of FIG. 29A. FIG. 30B is an example of the pattern signal formed by the pattern block of FIG. 29D. 30A and 30B have the same arrangement of the pattern blocks that are turned on.

By introducing the concept of pattern blocks and optimizing the distribution of on-pixels and off-pixels, it is possible to provide suitable sensing for a specific scene or object.

Further, by defining a plurality of pattern blocks and adaptively selecting them according to the shape and movement of the object OBJ or the traveling scene, it becomes possible to accurately detect the object OBJ, or You can control the frame rate.

In sensing by ghost imaging, the randomness and spatial incoherence of the reference beam S1 have a great influence on image quality. According to the pattern blocks of FIGS. 29A and 29B, the spatial incoherence can be improved as described below.

FIGS. 31A and 31B are diagrams for explaining the improvement of the spatial incoherence of the reference light. Generally, a light flux emitted from a certain light source travels with a certain divergence angle. In a near-field observing microscope, the spread of the luminous flux for each pixel does not matter. However, since the vehicle-mounted imaging apparatus 100 needs to detect an object in the far field, the spread of the light flux causes a problem. Specifically, as shown in FIG. 31A, two light fluxes emitted from two adjacent ON areas (or pixels) A and B of the illumination device 110 are located at the position of the object OBJ far from the illumination device 110. Overlap in. Such overlapping of the light beams reduces the spatial incoherence.

By using the pattern blocks shown in FIGS. 29A and 29B, the number of continuous ON pixels can be limited to two. This means that an OFF pixel is inserted between two adjacent ON regions. As a result, as shown in FIG. 31B, the two adjacent on-regions A and B can be spatially separated, so that the overlapping of the light fluxes can be reduced even for the reference light with which the object OBJ is irradiated. , Spatial incoherence can be improved. Also in the case of using the pattern block B of FIG. 29C, the number of continuous ON pixels can be set to one or two in the horizontal and vertical directions.

For example, when the object OBJ is present at a distance, the pattern block B shown in FIGS. 29A to 29C may be used, and when the object OBJ is close, a normal pattern block (or pixel block) may be used. ..

(Example 3.6)
It can be understood that some of the intensity distribution control based on the pixel block B or the pattern block PB imposes a constraint condition on the on/off control of a plurality of pixels of the illumination device 110.

For example, in the intensity distribution control using the pattern block PB of FIG. 29A, a constraint condition that 2+4×n (n≧0) off pixels are inserted between two adjacent on pixels. I can understand.

Alternatively, in the intensity distribution control using the pattern block PB of FIG. 29B, a constraint condition that 1+4×n (n≧0) off pixels are inserted between three adjacent on pixels. I can understand.

In other words, the intensity distribution that can improve the spatial incoherence may be generated based on a predetermined constraint condition without using the pattern block PB.

FIGS. 32(a) and 32(b) are diagrams showing examples of intensity distributions that can improve spatial incoherence. In FIGS. 32(a) and 32(b), on/off of a plurality of pixels is randomly determined under the constraint that adjacent pixels are not turned on. In FIG. 32A, adjacency of ON pixels in the vertical, horizontal, and diagonal directions is prohibited. In FIG. 32B, adjacency of ON pixels in the diagonal direction is allowed, and adjacency of ON pixels in the vertical direction and the horizontal direction is prohibited.

(Example 3.7)
Another example of controlling the intensity distribution based on the constraint conditions will be described. FIGS. 33A to 33D are diagrams for explaining the pattern control with the lighting rate as a constraint condition. In FIGS. 33A to 33D, the lighting rates (the ratio of the number of ON pixels to the total number of pixels) are different, and the lighting rates are 20%, 40%, 60%, and 80%, respectively. When the pattern generator is composed of a pseudo random signal (PRBS) generator, the lighting rate is associated with the mark rate of the RPBS.

▽ Increasing the lighting rate will increase the amount of light, so it will be possible to sense objects farther away. Alternatively, it is possible to detect an object having a lower reflectance or an object having a smaller reflection area. By increasing the lighting rate, the amount of reflected light can be increased even in a dense fog environment where the light attenuation rate is high, and thus the detection sensitivity can be increased.

On the contrary, when sensing a close object, the lighting rate may be reduced when detecting an object with high reflectance or a large object.

Further, when the reference light S1 is white light, the lighting rate is dynamically controlled according to the traveling environment to improve the visibility from the driver and give attention and warning to other traffic participants. Glare to preceding vehicles, oncoming vehicles, and pedestrians can be reduced.

　For example, if the lighting rate is reduced, the area ahead becomes darker, which can prevent glare to oncoming vehicles and pedestrians. On the contrary, if the lighting rate is increased, the front becomes brighter, and the visibility for the driver can be improved. Further, if the lighting rate is increased or decreased over time, the reference light S1 can be made to blink in a pseudo manner, and the driver of the own vehicle and other traffic participants can be given a warning or an alarm.

In the examples of FIGS. 33A to 33D, the lighting rate for all pixels is defined, but the lighting rate may be defined for each area by dividing into a plurality of areas. 34(a) and 34(b) are diagrams for explaining the control of the lighting rate according to the modification. For example, if the lighting rate is 50% and the PRBS is generated with the mark rate through all pixels set to 50%, ON pixels are concentrated in the upper half and OFF pixels are concentrated in the lower half, resulting in uneven brightness. .. Therefore, as shown in FIG. 34(a), unevenness in brightness can be reduced by generating PRBS with a mark ratio of 50% for each of the upper half region and the lower half region to define the intensity distribution.

As shown in FIG. 34(b), it may be divided into a plurality of areas, and the lighting rate may be independently designated for each area. In this case, control such as lowering the lighting rate can be performed in the area where the oncoming vehicle and the preceding vehicle exist.

Next, a modified example related to the third embodiment will be described.

(Modification 3.1)
In the embodiment, the illuminating device 110 is configured by the combination of the light source 112 and the patterning device 114, but it is not limited thereto. For example, the illuminating device 110 is configured by an array of a plurality of semiconductor light sources (LED (light emitting diode) or LD (laser diode)) arranged in a matrix, and ON/OFF (or brightness) of each semiconductor light source can be controlled. You may comprise.

(Use)
Next, the application of the imaging device 100 according to the third embodiment will be described. This imaging device 100 can be used for the object identification system 10 of FIG. By using the imaging device 100 described in the second embodiment as the sensor of the object identification system 10, the following advantages can be obtained.

By using the imaging device 100, that is, the quantum radar camera, noise resistance is significantly increased. For example, when it is raining, snowing, or traveling in fog, it is difficult for the naked eye to recognize the object OBJ, but by using the imaging device 100, the object OBJ is not affected by rain, snow, or fog. The restored image G can be obtained.

The detection targets of the vehicle-mounted imaging device 100 are various, such as cars, people, motorcycles and bicycles, structures and animals and plants. The situation in which the imaging device is used also changes greatly depending on the traveling environment such as weather, time of day, traveling road, traveling speed, and the like. In addition, the imaging device itself moves, the objects also move, and their relative movement directions are various. By dynamically controlling the intensity distribution of the reference light based on the pixel block and the pattern block, it is possible to perform sensing suitable for in-vehicle use.

The imaging device 100 described in the third embodiment can be mounted on the vehicle shown in FIG. 11 and may be built in the vehicle lighting device shown in FIG.

(Embodiment 4)
(Outline of Embodiment 4)
Embodiment 4 described below relates to an imaging apparatus using the principle of ghost imaging. The imaging device includes an illumination that illuminates an object while changing the intensity distribution of the reference light in a plurality of M ways, a photodetector that measures reflected light from the object for each of the plurality of intensity distributions I ₁ to I _M , and a plurality of photodetectors. And an arithmetic processing unit for reconstructing a restored image of the object by taking a correlation between the intensity distributions I ₁ to I _{M of the above} and a plurality of detection intensities b ₁ to b _M based on the output of the photodetector.

The plurality of intensity distributions I ₁ to I _M can be determined by the following processing.
(I) Model the transfer characteristics of the path from the illumination to the photodetector through the object.
(Ii) Define a reference object and a corresponding reference image.
(Iii) Initial values are given to the plurality of intensity distributions I ₁ to I _M.
(Iv) Estimated values b^ ₁ to b of the detected intensities b ₁ to b _M when the reference object is irradiated with the reference lights having the plurality of intensity distributions I ₁ to I _M based on the modeled transfer characteristics. ^ Calculate _M.
(V) Reconstruct the restored image of the reference object by correlating the plurality of intensity distributions I ₁ to I _M and the plurality of estimated values b ₁ to b _M.
(Vi) Each of the plurality of intensity distributions I ₁ to I _M is modified so that the error between the restored image and the reference image becomes small. By repeating the processes (iv) to (vi), a plurality of intensity distributions I ₁ to I _M can be determined.

According to this embodiment, the number of irradiations can be reduced by defining the reference image according to the assumed subject and optimizing the pattern.

A plurality of N sets (N≧2) of reference objects and reference images may be defined. In this case, since a plurality of subjects can be assumed, versatility can be improved.

The error may be represented by the objective function F(I) of Expression (4).

However, W is the width of the image, H is the height of the image, T _i (x, y) is the i-th reference image, and G _i (x, y, I) is the i-th restored image.

A plurality of sets of a plurality of intensity distributions I ₁ to I _M may be prepared and one set according to the traveling environment may be selectively used. As a result, the image quality can be improved as compared with the case where the same set of intensity distributions is always used in various traveling environments.

(Detailed Description of Embodiment 4)
FIG. 35 is a diagram showing the imaging apparatus 100 according to the fourth embodiment. The imaging device 100 is a correlation function image sensor that uses the principle of ghost imaging, and includes an illumination 110, a photodetector 120, and a calculation processing device 130. The imaging device 100 is also called a quantum radar camera.

The illumination 110 is a pseudo heat light source and generates the reference light S1 having the intensity distribution I(x, y) that can be regarded as substantially random and irradiates the object OBJ. Irradiation of the reference light S1 onto the object OBJ is performed while changing its intensity distribution according to a plurality of M patterns. Illumination 110 may include, for example, a light source 112 that produces light S0 having a uniform intensity distribution, and a patterning device 114 that is capable of spatially modulating the intensity distribution I of this light S0. The light source 112 may use a laser, a light emitting diode, or the like. The wavelength and spectrum of the reference light S1 are not particularly limited, and may be white light having a plurality of or continuous spectra, or monochromatic light having a predetermined wavelength.

A DMD (Digital Micromirror Device) or a liquid crystal device can be used as the patterning device 114. The pattern signal PTN (image data) designating the intensity distribution I is given to the patterning device 114 from the arithmetic processing unit 130. Therefore, the arithmetic processing unit 130 currently outputs the reference light S1 irradiated to the object OBJ. Know the intensity distribution I _r .

The photodetector 120 measures the reflected light from the object OBJ and outputs a detection signal D _r . The detection signal D _r is a spatial integral value of light energy (or intensity) incident on the photodetector 120 when the object OBJ is irradiated with the reference light having the intensity distribution I _r . Therefore, the photodetector 120 can use a single pixel device (photodetector). The photodetector 120 outputs a plurality of detection signals D ₁ to D _M corresponding to a plurality of _M intensity distributions I ₁ to I _M, respectively.

The arithmetic processing unit 130 includes a pattern generator 132 and a reconstruction processing unit 134. The pattern generator 132 generates a pattern signal PTN _r designating the intensity distribution I of the reference light S1 and switches the pattern signal PTN _r with time (r=1, 2,... M). Conventionally, the intensity distribution of the reference light S1 generated by the illumination 110 is randomly generated, but in the present embodiment, a set of a plurality of predetermined intensity distributions I ₁ to I _M is used. Therefore, a set of a plurality of pattern signals PTN ₁ to PTN _M defining a plurality of intensity distributions I ₁ to I _M is held in advance in a memory (pattern memory) inside the pattern generator 132.

The reconstruction processing unit 134 reconstructs the restored image G(x, y) of the object OBJ by correlating the plurality of intensity distributions I ₁ to I _M and the plurality of detected intensities b ₁ to b _M. The detection intensities b ₁ to b _M are based on the detection signals D ₁ to D _M. The relationship between the detection intensity b and the detection signal D may be determined in consideration of the type and method of the photodetector 120.

It is assumed that the reference light S1 having a certain intensity distribution I _r is irradiated for a certain irradiation period. The detection signal D _r is assumed to represent the amount of received light at a certain time (or a minute time), that is, the instantaneous value. In this case, the detection signal D _r may be sampled multiple times during the irradiation period, and the detection intensity b _r may be an integrated value, an average value, or a maximum value of all sampling values of the detection signal D _r . Alternatively, some of all the sampling values may be selected, and the integrated value, average value, or maximum value of the selected sampling values may be used. For selecting a plurality of sampling values, for example, the x-th to the y-th order from the maximum value may be extracted, sampling values lower than an arbitrary threshold value may be excluded, or the magnitude of signal fluctuation may be excluded. It is also possible to extract a sampling value in a range where is small.

The correlation function of Expression (5) is used for the correlation. I _r is the r-th intensity distribution, and b _r is the r-th detected intensity value.

The above is the basic configuration of the entire imaging apparatus 100. Hereinafter, how to determine a plurality of intensity distributions I ₁ to I _M will be described. The plurality of intensities I ₁ to I _M are determined in advance by using a computer.

FIG. 36 is a flowchart showing a method of determining a set of a plurality of intensity distributions I ₁ to I _M. The transfer characteristic of the path from the illumination 110 to the photodetector 120 via the object OBJ is modeled (S100). The transfer characteristics include the transfer characteristics of light from the illumination 110 to the object OBJ, the reflection characteristics of the object OBJ, the propagation characteristics of light from the object OBJ to the photodetector 120, and the conversion characteristics of the photodetector 120. Be done.

Define a reference object and a reference image T(x, y) corresponding to it (S102). The reference image T(x,y) defines the reflection characteristic of the reference object. FIG. 37 is a diagram illustrating the relationship between the reference object and the reference image T(x, y).

　In order to simplify the explanation and facilitate understanding, consider grayscale here. It is assumed that the pixel value of the reference image T(x,y) is normalized with 0 to 1. In this case, the pixel value of each pixel p represents the reflectance of the corresponding portion of the reference object. For example, when the pixel value of a certain pixel p is 1, the reflectance of the corresponding reference object is 1 (that is, 100%), and when the pixel value is 0, the reflectance of the corresponding reference object is 0 ( That is, 0%), and when the pixel value is 0.5, the reflectance of the corresponding reference object can be associated as 0.5 (that is, 50%).

Returning to FIG. Based on the step of giving initial values to a plurality of patterns and (iv) the transfer characteristics, the detected intensities b ₁ to b ₁ when the reference light S1 having a plurality of intensity distributions I ₁ to I _M is irradiated to the reference object OBJ estimate of b _M _b ^ 1 calculates the ~ b _{^ M} (S106).

For example, it is assumed that the light is not attenuated in the optical path from the illumination 110 to the object OBJ, and the reference light S1 is emitted over the entire rectangle including the reference object OBJ (rectangle shown by the broken line on the right side of FIG. 37). Further, it is assumed that the light is not attenuated in the optical path from the object OBJ to the photodetector 120, and all the reflected light from the object OBJ is incident on the photodetector 120. Under this assumption, the estimated value b^ _r of the detected intensity when the reference light whose intensity distribution is I _r (x, y) is applied to the reference object is represented by Expression (6). However, W represents the width of the image and H represents the height of the image.

A combination (or state) of the current intensity distributions I ₁ (x, y) to I _M (x, y) is denoted by I. The restored image G(x, y, I) is reconstructed using the set I of the intensity distribution based on the correlation function of Expression (7) (S108). Expression (7) is obtained by replacing the detection intensity b _r in Expression (5) with the estimated value b̂ _r .

The reference image T(x,y) corresponds to the correct answer of the restored image G(x,y,I). Therefore, the error ε between the restored image G(x,y,I) and the reference image T(x,y) is calculated (S110), and each of the plurality of intensity distributions I ₁ to I _M is reduced so as to reduce the error ε. It is corrected (S114).

This process is repeated while the error ε is larger than the allowable value ε _MAX (Y of S112). When ε<ε _MAX (N in S112), the set I of the plurality of intensity distributions I ₁ to I _M at that time is saved (S116), and the optimization process ends.

It is preferable to prepare a plurality of N sets of the reference image T(x, y) and the reference object and optimize the plurality of intensity distributions I ₁ to I _M so that the total error ε for them becomes small. In this case, since a plurality of subjects can be assumed, versatility can be improved.

The error ε in this case may be represented by the objective function F(I) of Expression (8).

T _i (x, y) represents the i-th set of reference images.

The algorithm for minimizing the error ε is not particularly limited, and a known algorithm can be used. For example, stochastic gradient descent can be used for the minimization. This problem can be formulated by the following equation (9).

I^ is a set of optimal intensity distributions I ₁ to I _M. Since the pixel values of the intensity distributions I ₁ to I _M do not have a negative value, a non-negative constraint condition can be set.

(Verification)
Hereinafter, determination and verification of a specific set of intensity distributions I ₁ to I _M will be described. As the reference image (reference object), a set of image data such as alphabets represented by various fonts called notMNIST was used (https://kaggle.com/lubaroli/notmnist/home). The number of images included in this data set is 529114. In machine learning, since the diversity of data is likely to improve performance, the following processing was added to artificially increase the number of data (bulk). Using these data, 10 epochs were learned with a batch size of 64.
・Randomly move up/down and left/right 10% ・Random 10% zoom ・Random 10% rotation

The number M of the intensity distributions I ₁ to I _M is set to ₁₀₀ , ₅₀₀ , and ₁₀₀₀ , and optimum intensity distribution sets I ₁₀₀ ^, I ₅₀₀ ^, and I ₁₀₀₀ ^ are obtained. Adam (Kingma Diederik, Jimmy Ba, "Adam-: A Method for Stochastic Optimization", arXiv:1412.6980, 2014) was used as the optimization algorithm. Parameters were set to α=0.001, β ₁ =0.9, and β ₂ =0.999 according to the literature.

FIG. 38 is a diagram showing a set I ₁₀₀ ^ composed of ₁₀₀ intensity distributions I ₁ to I ₁₀₀ obtained for M=100. The value of each pixel is normalized in the range of 0 to 255. A similar set can be obtained for M=500 and M=1000, but the illustration is omitted due to space limitations.

FIG. 39 is a diagram showing a restored image when an optimized intensity distribution set is used. The leftmost image is a correct answer image, and photographs of the alphabet K, frog, train, and truck are used in order from the top. Below the restored image, the numerical value of PSNR indicating the error from the correct image is shown. Images of frogs, trains, and trucks are taken from CIFAR-10 (Alex Krizhevsky, “Learning multiple layers of features from tiny images” 2009). The larger the numerical value of PSNR, the smaller the error.

By using the optimized set of intensity distributions, the original object can be restored to some extent even when M=100, more accurately with M=500 and more accurately with M=1000. You can see that it can be restored.

For comparison, we calculated the restored image when using a conventional set of random intensity distributions. The results for the letter K are shown here. FIG. 40 is a diagram showing a restored image when a set of random intensity distributions is used. Using a random intensity distribution, the PSNR is about 9.578 even when irradiation is performed 10,000 times. On the other hand, according to the present embodiment, PSNR=14.20 after 100 irradiations, PSNR=18.63 after 500 irradiations, and PSNR=22.13 after 1000 irradiations are obtained. It can be seen that the PSNR is significantly improved by comparison.

Next, a modified example of the fourth embodiment will be described.

(Modification 4.1)
In the imaging apparatus 100, a plurality of intensity distribution sets may be prepared and switched according to the traveling environment.

In the above description, it is assumed that there is no light attenuation or the like between the imaging device 100 and the object OBJ. This can be associated when the visibility in fine weather is good. Of course, the set of intensity distributions obtained under this assumption is also effective in situations where the vehicle is driving in rainfall, snowfall, or thick fog, but depending on the driving environment such as rainfall, snowfall, or thick fog, the intensity distribution By switching the set of, the error of the restored image G can be further reduced.

For example, in the case of assuming a traveling environment in which rain, snow, and fog exist between the imaging device 100 and the object OBJ, the transfer characteristics (light propagation characteristics) may be modeled in consideration of their influences. In this case, the calculation formula of the estimated value b^ _r of the detection intensity b _r is modified from the formula (6). Then, the set of intensity distributions may be optimized based on the modified estimated value b^ _r .

When optimizing a set of intensity distribution suitable for each driving environment assuming multiple driving environments, a reference object is photographed in each driving environment (that is, rainfall, snowfall, and thick fog), and the obtained image is used as a reference. The above machine learning may be performed as an image. In this case, the modeling of the transfer characteristic (light propagation characteristic) can be simplified.

Alternatively, as a driving environment, in addition to or in addition to the difference of rain, snow, fog, etc., it is suitable for each driving environment in consideration of daytime driving and nighttime driving, low speed driving and high speed driving. A set of intensity distributions may be prepared.

(Modification 4.2)
In the fourth embodiment, the illumination 110 is composed of the combination of the light source 112 and the patterning device 114, but it is not limited thereto. For example, the illumination 110 is composed of an array of a plurality of semiconductor light sources (LED (light emitting diode) or LD (laser diode)) arranged in a matrix, and it is possible to control ON/OFF (or brightness) of each semiconductor light source. You may comprise.

Next, applications of the imaging device 100 according to the fourth embodiment will be described. This imaging device 100 can be used for the object identification system 10 of FIG. By using the imaging device 100 described in the fourth embodiment as the sensor of the object identification system 10, the following advantages can be obtained.

Secondly, by using a set of intensity distributions I ₁ to I _M optimized beforehand by machine learning as the reference light S1, it is possible to restore the image of the object OBJ with a small number of irradiations. In contrast to the above, in the conventional random intensity distribution, the number of irradiations required up to several thousand times is required, but in the present embodiment, the number of irradiations can be reduced to about 100 to 1000 times. As a result, it is possible to increase the time required to obtain one restored image, that is, the frame rate, as compared with the conventional case. As a result, it is possible to achieve the frame rate required for in-vehicle use in which the imaging apparatus 100 and the object OBJ move relatively.

The imaging device 100 described in the fourth embodiment can be mounted on the automobile of FIG. 11 and may be incorporated in the vehicle lamp of FIG.

In Embodiments 1 to 4, the method using correlation calculation has been described as the ghost imaging (or single pixel imaging) method, but the image reconstruction method is not limited to this. In some embodiments, instead of the correlation calculation, an analytical method using Fourier transform or Hadamard inverse transform, a method for solving an optimization problem such as sparse modeling, and an algorithm using AI/machine learning, The image may be reconstructed.

Although the present invention has been described by using specific words and phrases based on the embodiments, the embodiments merely show one aspect of the principle and application of the present invention. Many modifications and changes in arrangement are possible without departing from the spirit of the present invention defined in the range.

The present invention relates to a vehicle lamp.

OBJ... Object, 10... Object identification system, 20... Imaging device, 40... Arithmetic processing device, 42... Classifier, 100... Imaging device, 110... Illumination, 120... Photodetector, 130... Arithmetic processing device, 132... Pattern Generator, 134... Reconstruction processing unit, 200... Vehicle lamp, 202... Light source, 204... Lighting circuit, 206... Optical system, 300... Automotive, 302... Headlight, 310... Lamp system, 304... Vehicle side ECU , 400... Vehicle lamp, 402... Housing, 404... Cover, 410... Headlight, 412... Light source, 414... Reflector, 416... Patterning device, 420... Pseudo thermal light source, 422... Light source, 424... Patterning device, 430... Common member.

Claims

A lamp for a vehicle,
A headlight,
A pseudo heat light source that irradiates an object while randomly switching the intensity distribution of the reference light,
Equipped with
The pseudo thermal light source is a photodetector that measures reflected light from the object, and, based on the output of the detector and the intensity distribution of the reference light, an arithmetic processing unit that reconstructs a restored image of the object. Which constitutes an imaging device,
At least a part of the components of the headlight is shared with the pseudo heat light source.
The vehicular lamp according to claim 1, wherein the pseudo heat light source shares an optical system with the headlight.
The vehicle according to claim 2, wherein the optical system of the headlamp includes a patterning device that controls light distribution, and the pseudo-heat light source and the headlamp share the patterning device. Lamp.
The optical system of the headlamp includes a reflector that reflects light emitted from a light source toward the front of the vehicle, and the pseudo-heat light source and the headlamp share the reflector. Vehicle lighting.
The vehicular lamp according to any one of claims 2 to 4, wherein the reference light is infrared light or ultraviolet light.
The vehicular lamp according to any one of claims 1 to 5, wherein the pseudo heat light source shares a light source with the headlight.
The vehicular lamp according to claim 6, wherein the reference light is white light.
A vehicle comprising the vehicle lamp according to any one of claims 1 to 7.
An illumination device that divides the measurement range into a plurality of sections and irradiates the reference light with a random intensity distribution while switching the sections,
A photodetector that measures the reflected light from the object,
For each of the plurality of sections, based on the detection intensity based on the output of the photodetector and the intensity distribution of the reference light, an arithmetic processing device that reconstructs a restored image of a portion of the object included in the section,
An in-vehicle imaging device comprising:
10. The vehicle-mounted imaging device according to claim 9, wherein the number of the plurality of sections is determined such that the amount of decrease in the calculation time due to the division is larger than the amount of increase in the measurement time.
A vehicle lamp comprising the vehicle-mounted imaging device according to claim 9.
A vehicle comprising the on-vehicle imaging device according to claim 9 or 10.
Dividing the measurement range into a plurality of sections, irradiating a reference light with a random intensity distribution while switching the sections,
Measuring the reflected light from the object with a photodetector,
For each of the plurality of sections, reconstructing a restored image of the portion of the object included in the section based on the detected intensity based on the output of the photodetector and the intensity distribution of the reference light,
An imaging method comprising:
The imaging method according to claim 13, wherein the number of the plurality of sections is determined so that the amount of decrease in calculation time due to the division is larger than the amount of increase in measurement time.
An illumination device used in an imaging device based on ghost imaging,
It has a plurality of pixels arranged in a matrix and is configured to be able to modulate the light intensity distribution based on a combination of ON and OFF of the plurality of pixels,
The illumination device is characterized in that the intensity distribution is controlled in units of a pixel block including at least one pixel, and the pixel block is variable.
The lighting device according to claim 15, wherein the size of the pixel block is variable.
The lighting device according to claim 16, wherein the size of the pixel block is reduced in an area where it is determined that an object exists by past sensing.
The lighting device according to any one of claims 15 to 17, wherein the shape of the pixel block is variable.
The lighting device according to claim 18, wherein the shape of the pixel block changes in at least two types of a horizontally long figure, a vertically long figure, and a figure that extends in an oblique direction.
The lighting device according to claim 18 or 19, wherein the shape of the pixel block is selected based on a moving direction of an object.
The lighting device according to any one of claims 15 to 20, wherein the pixel block includes ON pixels and OFF pixels arranged according to a predetermined pattern.
22. The lighting device according to claim 21, wherein off-pixels are arranged along at least two adjacent sides of the pixel block.
An illumination device used in an imaging device based on ghost imaging,
It has a plurality of pixels arranged in a matrix and is configured to be able to modulate the light intensity distribution based on a combination of ON and OFF of the plurality of pixels,
Equipped with
An illumination device, wherein the intensity distribution is controlled using a predetermined pattern including two or more ON pixels and OFF pixels.
The lighting device according to claim 23, wherein a plurality of the predetermined patterns are defined, and the intensity distribution is controlled based on at least one dynamically selected from the plurality of predetermined patterns.
The illumination device according to any one of claims 15 to 24, which irradiates an object with reference light,
A photodetector for measuring reflected light from the object,
An arithmetic processing unit that reconstructs a restored image of the object based on the detected intensity based on the output of the photodetector and the intensity distribution of the reference light,
An imaging device comprising:
A vehicle lighting device comprising the imaging device according to claim 25.
A vehicle comprising the imaging device according to claim 25.
An illumination device used in an imaging device based on ghost imaging,
It has a plurality of pixels arranged in a matrix and is configured to be able to modulate the light intensity distribution based on a combination of ON and OFF of the plurality of pixels,
An illumination device, wherein on/off of the plurality of pixels is controlled under a predetermined constraint condition.
29. The lighting device according to claim 28, wherein the predetermined constraint condition includes that adjacent pixels are not turned on.
The lighting device according to claim 28 or 29, wherein the predetermined constraint condition is dynamically changed.
The lighting device according to any one of claims 28 to 30, wherein the predetermined constraint condition defines a lighting rate that is a ratio of ON pixels to OFF pixels.
The lighting device according to any one of claims 28 to 31, wherein the plurality of pixels are divided into a plurality of regions, and a ratio of ON and OFF pixels is defined in each region.
The illumination device according to any one of claims 28 to 32, which irradiates an object with reference light,
A photodetector for measuring reflected light from the object,
An arithmetic processing unit that reconstructs a restored image of the object based on the detected intensity based on the output of the photodetector and the intensity distribution of the reference light,
An imaging device comprising:
A vehicle lamp comprising the imaging device according to claim 33.
A vehicle comprising the imaging device according to claim 33.
Illumination for irradiating an object while changing the intensity distribution of reference light in a plurality of M ways;
A photodetector that measures reflected light from the object for each of a plurality of intensity distributions I 1 to I M ;
An arithmetic processing unit that reconstructs a restored image of the object by correlating the plurality of intensity distributions I 1 to I M and the plurality of detected intensities b 1 to b M based on the output of the photodetector;
Equipped with
The imaging apparatus, wherein the plurality of intensity distributions I 1 to I M are generated in advance by machine learning.
38. The plurality of intensity distributions I 1 to I M are prepared in a plurality of sets corresponding to a plurality of traveling environments, and one set according to the traveling environments is selectively used. Imaging equipment.
Illumination for irradiating an object while changing the intensity distribution of reference light in a plurality of M ways;
A photodetector that measures reflected light from the object for each of a plurality of intensity distributions I 1 to I M ;
An arithmetic processing unit for reconstructing a restored image of the object by correlating the plurality of intensity distributions I 1 to I M and the plurality of detected intensities b 1 to b M based on the output of the photodetector;
Equipped with
The plurality of intensity distributions I 1 to I M are
(I) modeling the transfer characteristic of the path from the illumination through the object to the photodetector;
(Ii) defining a reference object and its corresponding reference image;
(Iii) giving initial values to the plurality of intensity distributions I 1 to I M ,
(Iv) An estimated value b^ 1 of the detected intensities b 1 to b M when the reference object having the plurality of intensity distributions I 1 to I M is irradiated to the reference object based on the transfer characteristics calculating b^ M ,
(V) reconstructing a restored image of the reference object by correlating the plurality of intensity distributions I 1 to I M with the plurality of estimated values b 1 to b M
(Vi) modifying each of the plurality of intensity distributions 1 to I M so that an error between the restored image and the reference image becomes small,
(vii) determining the plurality of intensity distributions I 1 to I M by repeating steps (iv) to (vi),
An imaging device characterized by being obtained by.
39. The imaging apparatus according to claim 38, wherein the reference object and the reference image are defined by a plurality of N sets (N≧2).
40. The imaging apparatus according to claim 39, wherein the error is represented by an objective function F(I) of Expression (1).

However, W is the width of the image, H is the height of the image, T i (x, y) is the i-th reference image, and G i (x, y, I) is the i-th restored image.
The imaging device according to any one of claims 38 to 40, characterized in that a stochastic gradient descent method is used in step (vi).
The imaging apparatus according to claim 39, wherein a set of alphabetic image data is used as the reference object and the reference image.
43. The imaging apparatus according to claim 39, wherein a plurality of different environments are assumed, and the plurality of intensity distributions I 1 to I M are prepared for each of the plurality of environments.
An imaging device according to any one of claims 36 to 43,
Based on the image obtained by the imaging device, an arithmetic processing device capable of identifying the type of object,
An object identification system comprising:
A vehicle lamp comprising the object identification system according to claim 44.
A vehicle comprising the object identification system according to claim 44.
A method of determining a set of multiple intensity distributions for use in an imaging device, comprising:
The imaging device,
Illumination for irradiating an object while changing the intensity distribution of reference light in a plurality of M ways;
A photodetector that measures reflected light from the object for each of the plurality of intensity distributions I 1 to I M ;
An arithmetic processing unit for reconstructing a restored image of the object by correlating the plurality of intensity distributions I 1 to I M and the plurality of detected intensities b 1 to b M based on the output of the photodetector;
Equipped with
The method is
(I) modeling the transfer characteristic of the path from the illumination through the object to the photodetector;
(Ii) defining a reference object and its corresponding reference image;
(Iii) giving initial values to the plurality of intensity distributions I 1 to I M ,
(Iv) An estimated value b^ 1 of the detected intensities b 1 to b M when the reference object having the plurality of intensity distributions I 1 to I M is irradiated to the reference object based on the transfer characteristics calculating b^ M ,
(V) reconstructing a restored image of the reference object by correlating the plurality of intensity distributions I 1 to I M with the plurality of estimated values b 1 to b M
(Vi) modifying each of the plurality of intensity distributions I 1 to I M so that an error between the restored image and the reference image becomes small,
And determining the set of the plurality of intensity distributions I 1 to I M by repeating steps (iv) to (vi).
The method according to claim 47, wherein the reference object and the reference image are defined in a plurality of N sets (N≧2).
49. The method of claim 48, wherein the error is represented by the objective function F(I) of equation (1).

However, W is the width of the image, H is the height of the image, T i (x, y) is the i-th reference image, and G i (x, y, I) is the i-th restored image.
The method according to any one of claims 47 to 49, characterized in that a stochastic gradient descent method is used in step (vi).
The method according to any one of claims 47 to 50, wherein a set of alphabetic image data is used as the reference object and the reference image.