JPH07156712A - Driver photographing device - Google Patents

Abstract

PURPOSE:To provide a driver photographing device which can photograph the retina reflected image of a driver easily and clearly and facilitate the extraction of the features of the face by obtaining an image whose contrast is good. CONSTITUTION:A driver 1 in a cabin is illuminated by an illuminating device 20 to photograph a prescribed range including a driver's face with light input means 11, 12. The illuminating light projecting direction 12 of the illuminating means 20, and an optical axis which connects the driver 1 to light inputting means 11, 12 are formed so that they may be on the almost same axis with each other near the driver. The plural light inputting means 11, 12 have different spectral characteristics from each other to obtain the infrared image and visible image of the drive 1.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a driver photographing device used for detecting a driver's behavior such as a dozing state or a looking aside.

[0002]

2. Description of the Related Art FIG. 19 shows, for example, JP-A-60-1583.
It is a block diagram which shows the eye position detection apparatus using the conventional driver | operator imaging device shown by the 03 gazette. In the figure, 1 is a driver in a vehicle compartment, 2 is a driver's seat, 3 is an installation panel, 10a and 10b are light input means, for example, a CCD camera, and 20 is an illumination means for illuminating the driver 1 in the vehicle compartment, for example. Infrared LED, 21 is an LED drive circuit that supplies a current to the infrared LED 20, and 100 is a driver behavior detection circuit.

Next, the operation will be described. Infrared LED
The LED drive circuit 21 is supplied with current from the LED 20, and illuminates the driver 1 in the driver's seat 2. The image of the driver 1 is C installed at a position where a predetermined area including the face of the driver 1 can be photographed.
It is input to the CD cameras 10a and 10b. The input image is input to the driver behavior detection circuit 100, and the position of eyes, the orientation of the face, and the like are detected by image processing. FIG. 20 shows an example of an infrared image obtained by this device. In the figure, 4
Is the iris, 5 is the sclera, 6 is the pupil, and 7 is the face surface. In the image thus obtained, the sclera 5 appears slightly darker than the facial surface 7, the iris 4 appears darker than the sclera 5, and the pupil 6 appears even darker. 21 shows an example of an infrared image when the driver 1 wears the glasses 8. Figure A
As shown in, the reflection of the infrared component and the reflection of the infrared LED 20, which is the illumination light, are reflected on the eyeglasses. Due to this reflection, the facial feature points in the eye region cannot be clearly photographed.

[0004]

Since the conventional driver photographing apparatus is constructed as described above, there is a problem that complicated image processing is required to detect the position of the pupil 6. . That is, for the obtained image, for example, filter calculation is performed, edge detection is performed, and from this edge shape,
The Hough transform process for obtaining the circle corresponding to the pupil 6 is performed for pattern recognition to search for the pupil 6. Further, in the above edge detection, there is also a problem that a circle cannot be obtained unless the edge is properly detected due to noise or the like, and the image processing is complicated and takes time.

On the other hand, as shown in a paper "Trial manufacture of eye-gaze detecting device which allows pupil extraction processing and head movement" (Journal of the Institute of Electronics, Information and Communication Engineers D-II Vol.J76-D-II No.3) In addition, when the face is illuminated by the coaxial epi-illumination device, the retina reflection image can be remarkably captured, and the eye position can be detected by very simple image processing such as binarization processing. The coaxial epi-illumination is an illumination having a structure in which the optical axis of the camera and the irradiation direction of the illumination light are the same. Therefore, it is possible to use this device as a driver photographing device, but when photographing in the infrared region, when extracting feature points of the face such as the outer corners of the eyes and inner corners of the eye other than the retina reflection image, only images with poor contrast are obtained. No
There was a problem that feature points were difficult to extract. further,
When the driver 1 wears the spectacles 8, the reflection of the infrared component, the reflection of the infrared LED 20 which is the illumination light, and the like are reflected on the spectacles 8, so that the feature points of the face in the eye region cannot be clearly photographed. There was a point.

The present invention has been made in order to solve the above problems, and can easily and clearly capture a retina reflection image of a driver and obtain a high-contrast image to identify facial feature points. An object is to obtain a driver photographing device that can be easily extracted and has high LED efficiency.

[0007]

A driver photographing apparatus according to claim 1 of the present invention is installed at a position capable of photographing a predetermined area including a lighting means for illuminating a driver in a vehicle compartment and a driver's face. Is provided with a light input means for inputting the image of the driver, the illumination light irradiation direction of the illumination means, and the optical axis connecting the driver and the light input means, at least near the driver, substantially coaxial, A driver photographing apparatus provided with an illuminating means and a light inputting means is provided with a plurality of light inputting means having different spectral characteristics.

The driver photographing apparatus according to claim 2 is
In addition to the invention of claim 1, at least one optical element having a function of reflecting and transmitting light is installed in the optical path connecting the driver and the light input means.

Further, the driver photographing apparatus according to claim 3 is
An illumination means for illuminating the driver in the vehicle compartment, and an optical input means for inputting an image of the driver installed at a position where a predetermined area including the driver's face can be photographed, and an illumination light irradiation direction of the illumination means is provided. The lighting means and the light input means are installed so that the optical axis connecting the driver and the light input means is substantially coaxial at least in the vicinity of the driver, and the light entering the light input means is identified. The first light limiting means for limiting the light of the wavelength of, and the second light limiting means having a spectral characteristic different from that of the first light limiting means.

[0010]

According to the first aspect of the present invention, an image of the entire face of the driver with good contrast is obtained from a reflection image (visible image) of light having a wavelength in the visible region, and a reflection image (red image of red light having a wavelength in the infrared region is obtained. A clear retinal reflection image of the driver is obtained from the outside image). In addition, since the driver photographing device uses the coaxial incident light, the eye position can be detected by simple image processing.

Further, according to claim 2, the optical axes of a plurality of cameras are made to coincide with each other so that the visible images and the infrared images have the same positional correspondence relationship, and the characteristic points of the images can be easily associated with each other. .

According to the third aspect of the present invention, the whole image of the driver's face having a better contrast than the visible image can be obtained, and the retina reflection image of the driver can be obtained clearly from the infrared image. Further, since the driver photographing device utilizing the coaxial incident light is used, the position of the eyes can be detected by a simple image processing, and since it can be configured by one camera, it can be inexpensive and downsized, and the visible image and infrared The positional correspondences of images can be the same, and feature points of images can be easily associated with each other.

[0013]

【Example】

Example 1. An embodiment of the invention of claim 1 will be described below with reference to the drawings. 1 is a configuration diagram of a driver photographing apparatus according to a first embodiment of the present invention viewed from above the driver, and FIG. 2 is a configuration diagram viewed from the side of the driver. 1 and 2 in FIG.
Reference numerals 1 and 12 denote light input means, for example, cameras installed at positions where a predetermined area including the face of the driver 1 can be photographed, 20
Is an illuminating means for illuminating the driver 1 in the driver's seat 2, and is, for example, an infrared LED having a central wavelength of light emission of 860 nm. In this embodiment, the cameras 11 and 12 are installed side by side, and an infrared LED 20 is installed in the center so that the illumination light can enter both of the cameras 11 and 12. Further, the cameras 11, 12 and the infrared LED 20 are installed close to each other so that the irradiation direction 12 of the infrared LED 20 and the optical axis 13 of the cameras 11, 12 are substantially coaxial in the vicinity of the driver 1.

The two cameras 11 and 12 have different spectral characteristics, and FIG. 3 shows the sensitivity characteristics of the cameras. In the figure, the horizontal axis represents wavelength (nm), the vertical axis represents relative sensitivity, and FIG. 3 (a) represents the characteristics of the camera 11.
(B) shows the characteristics of the camera 12. In this embodiment, the camera 11 is an infrared camera having sensitivity in the infrared region, and the camera 12 is a visible camera having sensitivity in the visible region.

Next, the operation will be described. In FIGS. 1 and 2, the illumination light of the infrared LED 20 is transmitted to the driver 1 through the optical path 12.
Irradiate the face of. The image of the driver 1 reaches the cameras 11, 12 through the optical path 13 and the cameras 11, 12 can capture the image of the driver 1. At this time, the optical axes of the optical paths l2 and l3 are substantially coaxial or within a predetermined angle (2 degrees) as seen from the driver (coaxial epi-illumination). First, the infrared image will be described. When reflected light from the driver 1 is input by the infrared camera 11 to take an image, the image taken by the driver 1 is as shown in FIG. In the coaxial epi-illumination, the light reflected by the retina is observed as if the pupil 6 were shining, and the pupil 6 is remarkably brighter than the surface of the face and other characteristic points. Therefore, from the image thus obtained, only the position of the pupil 6 whose brightness is significantly higher than the others is white as shown in FIG. 5 only by adjusting the binarization threshold depending on the brightness. An image serving as a region is obtained, and the position of the pupil 6 can be easily detected without complicated calculation processing. It should be noted that the position of the pupil 6 can be obtained only by adjusting the threshold value for binarization depending on the magnitude of the brightness.
It can be sufficiently detected even from the infrared image showing the reflection of the infrared component when the glasses 8 as shown in FIG.

Next, the visible image will be described. The visible light from the driver 1 is input by the visible camera 12. Driver 1
If he wears glasses, the captured image is as shown in FIG. Compared with the infrared image shown in FIG. 21, in this visible image, since the reflection of the infrared component is not reflected on the glasses 8, the characteristic points of the eye region such as the outer corners of the eyes and the inner corners of the eyes can be markedly photographed.

As described above, since the whole image of the driver's face with good contrast is obtained by the visible image and the retina reflection image of the driver is clearly obtained by the infrared image, the eye position can be detected by simple image processing. , It is possible to easily and clearly capture the feature points of the driver's face. Furthermore, in this embodiment, since the driver 1 is directly illuminated by the infrared LED 20, a clear image can be obtained with a small amount of light, which is efficient. In addition, the amount of light of the infrared LED 20 can be reduced as compared with the case of indirectly irradiating with a mirror or the like, and the life of the infrared LED 20 can be extended.

Although the infrared LED is used as the illuminating means 20 in the above embodiment, a halogen lamp which emits light having a wavelength in a wide band in order to reduce the influence of temperature,
Alternatively, an illuminating unit including a halogen lamp and a filter that disperses light of a specific wavelength may be used. Also, camera 1
Although 1 and 12 have the characteristics shown in FIG. 3, they are not limited to these, and may be those having different spectral characteristics and capable of obtaining an infrared image and a visible image. For example, the visible camera 12 and the visible camera having sensitivity in the visible region near the infrared region may be used. Similarly, the infrared camera 11
And an infrared camera having sensitivity close to the visible region may be used.

Example 2. An embodiment of the inventions of claims 1 and 2 will be described below with reference to the drawings. FIG. 7 is a configuration diagram of a driver photographing apparatus according to the second embodiment of the present invention as viewed from above the driver. Reference numerals 31 and 32 are optical elements each having a function of reflecting and transmitting light, and are, for example, half mirrors. The half mirrors 31 and 32 are arranged on the optical path connecting the driver 1 and the infrared camera 11 which is the light input means, for example, on the optical axis l3 of the infrared camera 11. The same reference numerals as those in FIG. 1 are the same as or equivalent to those in the first embodiment.

Next, the operation will be described. The illumination light of the infrared LED 20 irradiated on the optical path 11 is the half mirror 3
1 reflects half the light, and the driver's 1 face is illuminated by the optical path l2. The image of driver 1 is half mirror 31 with optical path l3
Half of the light reaches the half mirror 32. The infrared camera 11 inputs the light transmitted through the half mirror 32,
An infrared image of the driver 1 is taken. At this time, the optical paths l2 and l
The optical axis of 3 is almost coaxial or within a predetermined angle (2 degrees) as seen from the driver (coaxial epi-illumination). In this embodiment, it is set to 0 degree, and the half mirror 3
The angle formed by 1 and 32 with the optical axis l3 of the infrared camera 11 is 45.
It is ゜. The obtained infrared image is as shown in FIG. 4 similarly to the first embodiment, and the position of the pupil 6 can be easily detected only by adjusting the binarization threshold depending on the magnitude of the brightness.

Further, the visible camera 12 has a half mirror 32.
The light reflected by is input and a visible image of the driver 1 is captured. The obtained image is an image of the entire face with good contrast, as in the case of Example 1. Even if the driver 1 wears glasses, as shown in FIG.
Infrared reflections are not reflected in the image, and feature points in the eye area such as the outer corners of the eyes and the inner corners of the eyes can be clearly captured.

As described above, in the second embodiment, as in the first embodiment, the retina reflection image can be clearly obtained by utilizing the coaxial epi-illumination.
The position of the eyes can be detected by simple image processing, and the entire image of the driver's face with good contrast is obtained from the visible image, so that the characteristic points of the driver's face can be easily and clearly photographed. Further, by using the half mirrors 31 and 32, the optical axes of the infrared camera 11 and the visible camera 12 can be matched. Therefore, since the face images taken by the two cameras 11 and 12 can be taken at the same position at the same time, the error due to the movement of the driver 1 due to the time difference can be eliminated, and the feature points can be easily associated with each other. .

The positions of the infrared camera 11 and the visible camera 12 are not limited to this, but may be reversed.

Example 3. Hereinafter, other embodiments of the inventions of claims 1 and 2 will be described with reference to the drawings. FIG. 8 is a configuration diagram of a driver photographing apparatus according to a third embodiment of the present invention as viewed from above the driver. Reference numeral 33 denotes an optical element having a function of reflecting and transmitting light, for example, a cold mirror having a wavelength (n
The reflectance (%) for m) is shown in FIG. As can be seen from the figure, the cold mirror 33 has a characteristic of transmitting 90% or more of infrared light and reflecting about 90 to 100% of visible light. The mirrors 31 and 33 are arranged on the optical axis l3 of the infrared camera 11 so that the illumination light irradiation direction l2 of the infrared LED 20 and the optical axis l3 of the infrared camera 11 are substantially coaxial in the vicinity of the driver 1. is doing. The same reference numerals as those in FIG. 1 are the same as or equivalent to those in the first embodiment.

Next, the operation will be described. As in the second embodiment, the half mirror 31 reflects half of the illumination light of the infrared LED 20 that is emitted along the optical path l1 and illuminates the face of the driver 1 through the optical path l2. The image of driver 1 is optical path l3
Then, half of the light passes through the half mirror 31 and reaches the cold mirror 33. The cold mirror 33 is the infrared camera 11
It is arranged so as to form an angle of 45 ° with the optical axis 13 of the cold mirror 33, and most of the visible light is reflected by the cold mirror 33 and most of the infrared light is transmitted. The infrared camera 11 inputs the light transmitted through the cold mirror 33 and captures an infrared image of the driver 1. The obtained infrared image can easily detect the position of the pupil 6 only by adjusting the threshold value for binarization depending on the magnitude of the brightness by coaxial epi-illumination, as in the first embodiment.

Further, the visible camera 12 is the cold mirror 3
The light reflected by 3 is input and a visible image of the driver 1 is captured. The obtained image is a facial image with good contrast, as in the case of Example 1. In particular, even when the driver 1 wears glasses, as shown in FIG. 6, the infrared components are not reflected on the glasses 8 and characteristic points in the eye region such as the outer corners of the eyes and the inner corners of the eyes can be remarkably captured.

Further, by using the half mirror 31 and the cold mirror 33, the optical axes of the infrared camera 11 and the visible camera 12 can be matched with each other, and the face image of the driver 1 can be taken at the same position at the same time. Therefore, the error due to the movement of the driver 1 due to the time difference can be eliminated, and the feature points can be easily associated. Further, in the configuration of the second embodiment, the infrared camera 11 inputs the light transmitted through the half mirrors 31 and 32, and the visible camera 12 receives the half mirror 3.
The light that has passed through 1 and is reflected by the half mirror 32 is input. Therefore, the amount of light is reduced to about 1/4. On the other hand, if the cold mirror 32 is used, the light amount is 1 /
A decrease of 2 is sufficient to prevent a decrease in the amount of light input to the cameras 11 and 12, and a bright image can be obtained, so that the retina reflection image and the facial feature points can be photographed more clearly.

Example 4. Hereinafter, still another embodiment of the present invention will be described with reference to the drawings. 10 is a configuration diagram of a driver photographing apparatus according to a fourth embodiment of the present invention as viewed from above the driver. Reference numeral 34 denotes an optical element having a function of reflecting and transmitting light, for example, a cold filter, and its reflectance (%) with respect to its wavelength (nm) is shown in FIG. As can be seen, the cold filter 34 filters the visible light to 9
It has a property of transmitting 0% or more and reflecting 90% or more of near infrared light. Half mirror 31 and cold filter 3
4 is arranged on the optical axis 13 of the visible camera 12. The same reference numerals as those in FIG. 1 are the same as or equivalent to those in the first embodiment.

Next, the operation will be described. As in the second embodiment, the half mirror 31 reflects half of the illumination light of the infrared LED 20 that is emitted along the optical path l1 and illuminates the face of the driver 1 through the optical path l2. The image of driver 1 is optical path l3
Then, half of the light reaches the cold filter 34 through the half mirror 31. The cold filter 34 is disposed so as to form an angle of 45 ° with the optical axis 13 of the visible camera 12, and most of the near infrared light is reflected by the cold filter 34 and most of the visible light is transmitted. Infrared camera 11
Inputs the light reflected by the cold filter 34 and captures an infrared image of the driver 1. The infrared image obtained is from Example 1.
Similarly to the above, the position of the pupil 6 can be easily detected by adjusting the binarization threshold depending on the magnitude of the brightness by the coaxial incident illumination.

Further, the visible camera 12 inputs the light transmitted through the cold filter 34 and takes a visible image of the driver 1. The obtained image is a facial image with good contrast, as in the case of Example 1. In particular, even when the driver 1 wears the spectacles 8, as shown in FIG. 6, the reflection of the infrared component is not reflected on the spectacles, and the characteristic points of the eye region such as the outer corners of the eyes and the inner corners of the eyes can be markedly photographed.

Further, by using the half mirror 31 and the cold filter 34, the optical axes of the infrared camera 11 and the visible camera 12 can be coincident with each other, and the face image of the driver 1 can be taken at the same position at the same time. . Therefore, the error due to the movement of the driver 1 due to the time difference can be eliminated, and the feature points can be easily associated. In addition, Example 3
Similar to the above, if the cold filter 34 is used, the light amount is 1 /
A decrease of 2 is sufficient to prevent a decrease in the amount of light input to the cameras 11 and 12, and a bright image can be obtained, so that the retina reflection image and the facial feature points can be photographed more clearly.

Example 5. Hereinafter, still another embodiment of the present invention will be described with reference to the drawings. FIG. 12 is a configuration diagram of the driver photographing apparatus according to the fifth embodiment of the present invention as viewed from above the driver. Reference numeral 32 denotes an optical element having a function of reflecting and transmitting light, for example, a half mirror, which is arranged on the optical axis l3 of the infrared camera 11 in this embodiment.
The infrared LED 20 is installed near the optical axis l3. The same reference numerals as those in FIG. 1 are the same as or equivalent to those in the first embodiment.

Next, the operation will be described. The illumination light of the infrared LED 20 driven by the LED drive circuit 21 has an optical path l.
Illuminate driver 1's face with 2. The image of the driver 1 passes through the half mirror 32 in the optical path 13 and half of the light reaches the infrared camera 11. The infrared camera 11 inputs the light transmitted through the half mirror 32 and captures an infrared image of the driver 1.
At this time, the optical axes of the optical paths l2 and l3 are substantially coaxial or within a predetermined angle (2 degrees) as seen from the driver (coaxial epi-illumination). In this embodiment, the angle is approximately 1 degree, and the half mirror 32 is used as the optical axis l of the infrared camera 11.
The angle with 3 is 45 °. The obtained infrared image is as shown in FIG. 4 similarly to the first embodiment, and the position of the pupil 6 can be easily detected only by adjusting the binarization threshold depending on the magnitude of the brightness.

Further, the visible camera 12 has a half mirror 32.
The light reflected by is input and a visible image of the driver 1 is captured. The obtained image is a facial image with good contrast, as in the case of Example 1. In particular, even when the driver 1 wears the glasses, as shown in FIG. 6, the reflection of the infrared component is not reflected on the glasses, and the characteristic points in the eye region such as the outer corners of the eyes and the inner corners of the eyes can be markedly photographed.

In this way, in the fifth embodiment, as in the first embodiment, the retina reflection image is clearly obtained by utilizing the coaxial epi-illumination.
The position of the eyes can be detected by simple image processing, and the entire image of the driver's face with good contrast is obtained from the visible image, so that the characteristic points of the driver's face can be easily and clearly photographed. Furthermore, by using the half mirror 32,
The optical axes of the infrared camera 11 and the visible camera 12 can be matched with each other, and the driver's face image can be taken at the same position at the same time. Therefore, the error due to the movement of the driver 1 due to the time difference can be eliminated, and the feature points can be easily associated. Further, in the second embodiment, the illumination light from the infrared LED 20 irradiates the driver 1 from the optical axis 12 through the half mirror 31, but in this embodiment, the driver 1 is directly illuminated. For this reason, a clear image can be obtained with a small amount of light, and the amount of light of the infrared LED 20 can be reduced.
The life can be extended to 0. Further, the structure is simpler than that of the second embodiment, and the size can be reduced.

The positions of the infrared camera 11 and the visible camera 12 are not limited to this, but may be reversed.

Example 6. Hereinafter, other embodiments of the inventions of claims 1 and 2 will be described with reference to the drawings. 13 is a configuration diagram of a driver photographing apparatus according to a sixth embodiment of the present invention viewed from above the driver. An optical element having a function of reflecting and transmitting 33 light, which is, for example, a cold mirror, has reflectance characteristics similar to those shown in FIG. That is, the cold mirror 33
90% or more of infrared light is transmitted, and visible light is about 90 to 10
It has the property of reflecting 0%. The cold mirror 33 is arranged on the optical axis 13 of the infrared camera 11. The infrared LED 20 is installed near the optical axis l3, and the infrared L
The illumination light irradiation direction 12 of the ED 20 and the optical axis 13 of the infrared camera 11 are configured to be substantially coaxial in the vicinity of the driver 1. The same reference numerals as those in FIG. 1 are the same as or equivalent to those in the first embodiment.

Next, the operation will be described. Similar to the fifth embodiment, the illumination light of the infrared LED 20 illuminates the face of the driver 1 along the optical path l2. The image of the driver 1 reaches the cold mirror 33 along the optical path l3. The cold mirror 33 is arranged so as to form an angle of 45 ° with the optical axis 13 of the infrared camera 11, and most of the visible light is reflected and most of the infrared light is transmitted by the cold mirror 33. The infrared camera 11 inputs the light transmitted through the cold mirror 33 and captures an infrared image of the driver 1. The obtained infrared image can easily detect the position of the pupil 6 only by adjusting the threshold value for binarization depending on the magnitude of the brightness by coaxial epi-illumination, as in the first embodiment.

The visible camera 12 is the cold mirror 3
The light reflected by 3 is input and a visible image of the driver 1 is captured. The obtained image is a facial image with good contrast, as in the case of Example 1. In particular, even when the driver 1 wears the glasses, as shown in FIG. 6, the reflection of the infrared component is not reflected on the glasses, and the characteristic points in the eye region such as the outer corners of the eyes and the inner corners of the eyes can be markedly photographed.

In the configuration of the fifth embodiment, the infrared camera 11 inputs the light transmitted through the half mirror 32, and the visible camera 12
Inputs the light reflected by the half mirror 32. Therefore, the amount of light is reduced to about 1/2. On the other hand, in this embodiment, the cold mirror 32 is used,
The light intensity hardly decreases. Therefore, the cameras 11, 1
It is possible to prevent a decrease in the amount of light input to the light source 2 and obtain a bright image, so that the retina reflection image and the facial feature points can be photographed more clearly. In the third embodiment, the illumination light from the infrared LED 20 is transmitted from the optical axis 12 through the half mirror 31 to the driver 1.
However, in this embodiment, the driver 1 is directly illuminated. Therefore, a clear image can be obtained with a smaller amount of light and the amount of light of the infrared LED 20 can be reduced, so that the life of the infrared LED 20 can be extended. Further, the structure is simpler than that of the third embodiment, and the size can be reduced.

Example 7. Hereinafter, still another embodiment of the present invention will be described with reference to the drawings. 14 is a configuration diagram of a driver photographing apparatus according to a seventh embodiment of the present invention viewed from above the driver. Reference numeral 34 is an optical element having a function of reflecting and transmitting light, for example, a cold filter, the reflectance characteristic of which is It is similar to that shown in FIG. That is, the cold filter 34 has a characteristic of transmitting visible light by 90% or more and reflecting near infrared light by 90% or more. The cold filter 34 is arranged on the optical axis 13 of the visible camera 12. The infrared LED 20 is installed near the optical axis l3, and the illumination light irradiation direction l2 of the infrared LED 20 and the optical axis l3 of the visible camera 12 are arranged to be substantially coaxial in the vicinity of the driver 1. The same reference numerals as those in FIG. 1 are the same as or equivalent to those in the first embodiment.

Next, the operation will be described. Similar to the fifth embodiment, the illumination light of the infrared LED 20 illuminates the face of the driver 1 along the optical path l2. The image of the driver 1 reaches the cold filter 34 in the optical path l3. The cold filter 34 is disposed so as to form an angle of 45 ° with the optical axis 13 of the visible camera 12, and most of the near infrared light is reflected by the cold filter 34 and most of the visible light is transmitted. The infrared camera 11 inputs the light reflected by the cold filter 34,
An infrared image of the driver 1 is taken. The obtained infrared image can easily detect the position of the pupil 6 only by adjusting the threshold value for binarization depending on the magnitude of the brightness by coaxial epi-illumination, as in the first embodiment.

Further, the visible camera 12 inputs the light transmitted through the cold filter 34 and captures a visible image of the driver 1. The obtained image is a facial image with good contrast, as in the case of Example 1. In particular, even when the driver 1 wears the glasses, as shown in FIG. 6, the reflection of the infrared component is not reflected on the glasses, and the characteristic points in the eye region such as the outer corners of the eyes and the inner corners of the eyes can be markedly photographed.

As described above, when the cold filter 34 is used, the light amount is hardly reduced, so that the cameras 11, 1
Since the amount of light input to 2 can be increased and a bright image can be obtained, the retina reflection image and the feature points of the face can be photographed more clearly. Further, in the fourth embodiment, the illumination light from the infrared LED 20 irradiates the driver 1 from the optical axis 12 through the half mirror 31, but in this embodiment, the driver 1 is directly illuminated. Therefore, a clear image can be obtained with a smaller amount of light, and the amount of light of the infrared LED 20 can be reduced.
The life of the ED 20 can be extended. Further, the structure is simpler than that of the fourth embodiment, and the size can be reduced.

Example 8. An embodiment of the invention of claim 3 will be described below with reference to the drawings. FIG. 15 shows the eighth embodiment of the present invention.
FIG. 3 is a configuration diagram of the driver photographing device according to FIG. In the figure, reference numeral 13 is an optical input means arranged at a position where an area including the face of the driver 1 can be photographed, for example, FIG.
As shown in the graph (a) showing the relative sensitivity characteristic with respect to the wavelength (nm), a camera having sensitivity to light having wavelengths in all the infrared and visible regions is used. Reference numeral 31 denotes an optical element having a function of reflecting and transmitting light, which is, for example, a half mirror, and in this embodiment, an optical axis l of the camera 13.
Located on top of 3. 22 is a filter switching circuit,
Reference numerals 35 and 36 denote light limiting means having different spectral characteristics, for example, first and second filters. First filter 3
5 and the second filter 36 are arranged by the filter switching circuit 22 so that the first filter 35 is arranged on the optical axis l3 at one time and the second filter 36 is arranged on the optical axis l3 at another time. The spectral characteristic of the first filter 35 is shown in FIG. 16 (b), and the spectral characteristic of the second filter 36 is shown in FIG. 16 (c). As shown by the transmittance (%) with respect to this wavelength (nm), the first filter 35 has a characteristic of transmitting almost all light in the visible region, and the second filter 36 has a characteristic of transmitting almost all light in the infrared region. The same reference numerals as those in FIG. 1 are the same as those in the first embodiment,
Or a considerable part.

Next, the operation will be described. First, the second filter 36 is placed on the optical axis l3. The illumination light of the infrared LED 20 irradiated on the optical path 11 is the half mirror 3
1 reflects half the light, and the driver's 1 face is illuminated along the optical path l2. The image of driver 1 is half mirror 31 with optical path l3
Half of the light reaches the second filter 36. Second
The filter 36 transmits most of the infrared light and blocks most of the visible light. Therefore, the light input to the camera 13 becomes infrared light, and an infrared image of the driver 1 is captured.
At this time, the optical axes of the optical paths l2 and l3 are substantially coaxial or within a predetermined angle (2 degrees) as seen from the driver (coaxial epi-illumination). In this embodiment, the angle is 0 °, and the angle formed by the half mirror 31 and the optical axis 13 of the camera 13 is 45 °. The obtained infrared image is as shown in FIG. 4 similarly to the first embodiment, and the position of the pupil 6 can be easily detected only by adjusting the binarization threshold depending on the magnitude of the brightness.

After that, the filter switching circuit 22 is operated to arrange the first filter 35 on the optical axis l3. When the infrared LED 20 is driven again by the LED drive circuit 21, the image of the driver 1 passes through the half mirror 31 along the optical path l3 and half of the light is emitted from the first filter 35 in the same manner as above.
To reach. The first filter 35 transmits most of visible light and blocks most of infrared light. Therefore, the light input to the camera 13 becomes visible light, and a visible image of the driver 1 is captured. The obtained image is a facial image with good contrast, as in the case of Example 1. In particular, even when the driver 1 wears the glasses, as shown in FIG. 6, the reflection of the infrared component is not reflected on the glasses, and the characteristic points in the eye region such as the outer corners of the eyes and the inner corners of the eyes can be markedly photographed.

In this way, in the eighth embodiment, as in the first embodiment, the retina reflection image is clearly obtained by utilizing the coaxial epi-illumination.
Eye positions can be detected by simple image processing, and a visible image provides a driver's face image with good contrast over the entire image, so that the feature points of the driver's face can be captured easily and clearly.

Further, since the infrared image and the visible image can be photographed by one camera, the driver's face image can be photographed at exactly the same position, and the feature points can be easily corresponded. Furthermore, it can be made smaller and cheaper. In the above embodiment, the visible image is captured after the infrared image is captured, but the same effect can be obtained regardless of the order.

Example 9. Hereinafter, another embodiment of the invention of claim 3 will be described with reference to the drawings. 17 is a configuration diagram of a driver photographing apparatus according to a ninth embodiment of the present invention as viewed from above the driver. In this embodiment, the infrared LED 20 is the camera 13
It is installed in the vicinity of the optical axis l3 of. The same reference numerals as those in FIG. 15 are the same as or equivalent to those in the eighth embodiment.

Next, the operation will be described. First, the second filter 36 is placed on the optical axis l3. Infrared LED
The illumination light of 20 illuminates the face of the driver 1 along the optical path l2. The image of the driver 1 reaches the second filter 36 in the optical path l3. The second filter 36 transmits most of the infrared light and blocks most of the visible light. Therefore, the camera 1
The light input to 3 becomes infrared light, and an infrared image of the driver 1 is captured. The obtained infrared image is as shown in FIG. 4 by the coaxial epi-illumination as in the case of Example 8, and the position of the pupil 6 can be easily detected only by adjusting the binarization threshold depending on the magnitude of the brightness.

After that, the filter switching circuit 22 is operated to place the first filter 35 on the optical axis l3. When the LED driving circuit 21 drives the infrared LED 20 again, the image of the driver 1 reaches the first filter 35 along the optical path l3 in the same manner as above. The first filter 35 transmits most of visible light and blocks most of infrared light.
Therefore, the light input to the camera 13 becomes visible light,
A visible image of the driver 1 is taken. The obtained image is a facial image with good contrast, as in the case of Example 8. Especially when the driver 1 wears glasses,
As shown in FIG. 6, the infrared components are not reflected on the eyeglasses, and the characteristic points in the eye region such as the outer corners of the eyes and the inner corners of the eyes can be markedly photographed.

As described above, in the ninth embodiment, as in the eighth embodiment, since the retinal reflection image is clearly obtained by utilizing the coaxial epi-illumination,
The position of the eyes can be detected by simple image processing, and the entire image of the driver's face with good contrast is obtained from the visible image, so that the characteristic points of the driver's face can be easily and clearly photographed.

Further, the infrared image and the visible image can be taken by one camera, and the size can be reduced. Further, in the eighth embodiment, the illumination light from the infrared LED 20 irradiates the driver 1 from the optical axis l2 through the half mirror 31, but in this embodiment, the driver 1 is directly illuminated without passing through the half mirror. ing. Therefore, a clear image can be obtained with a small amount of light, and the amount of light of the infrared LED 20 can be reduced, so that the efficiency can be improved,
The life of the infrared LED 20 can be extended.

Example 10. Hereinafter, still another embodiment of the invention of claim 3 will be described with reference to the drawings. 18 is a configuration diagram of a driver photographing apparatus according to a tenth embodiment of the present invention as viewed from above the driver. In the figure, 37 is a total reflection mirror having one hole 37a in the central portion. In this embodiment, the infrared LED 20 is installed on the back surface of the hole 37a of the total reflection mirror 37, and the driver 1 is in the optical path l2 through the hole 37a.
Illuminate. The same reference numerals as those in FIG. 15 are the same as or equivalent to those in the eighth embodiment.

Next, the operation will be described. First, the optical axis l
The second filter 36 is positioned on the incident side of the camera 13 arranged at a place where the light of 3 can be photographed. The illumination light of the infrared LED 20 emitted along the optical path 12 illuminates the face of the driver 1. The image of the driver 1 passes through the optical path 13 and is totally reflected by the total reflection mirror 37 to reach the second filter 36. The second filter 36 transmits most of the infrared light and blocks most of the visible light. Therefore, the light input to the camera 13 becomes infrared light, and an infrared image of the driver 1 is captured. The obtained infrared image was taken from Example 8 by coaxial epi-illumination.
4, the position of the pupil 6 can be easily detected only by adjusting the binarization threshold depending on the magnitude of the brightness.

After that, the filter switching circuit 22 is operated to arrange the first filter 35 on the incident side of the camera 13. The infrared LED 20 is again driven by the LED drive circuit 21.
Is driven, the image of the driver 1 reaches the first filter 35 along the optical path l3 in the same manner as described above. First filter 3
At 5, most of visible light is transmitted and most of infrared light is blocked. Therefore, the light input to the camera 13 becomes visible light, and a visible image of the driver 1 is captured. The obtained image is a facial image with good contrast, as in the case of Example 8. In particular, even when the driver 1 wears the glasses, as shown in FIG. 6, the reflection of the infrared component is not reflected on the glasses, and the characteristic points in the eye region such as the outer corners of the eyes and the inner corners of the eyes can be markedly photographed.

As described above, in the tenth embodiment, as in the eighth embodiment, the retina reflection image is clearly obtained by utilizing the coaxial incident light.
The eye position can be detected by simple image processing, and the driver's face image with a good body contrast can be obtained from the visible image, so that the characteristic points of the driver's face can be easily and clearly photographed.

Further, the infrared image and the visible image can be taken by one camera, and the size can be reduced. Further, in the eighth embodiment, the illumination light from the infrared LED 20 irradiates the driver 1 from the optical axis l2 through the half mirror 31, but in this embodiment, the driver 1 is directly illuminated without passing through the half mirror. ing. For this reason, a loss of light amount is small, a clear image can be obtained with a small light amount, and the light amount of the infrared LED 20 can be reduced, so that efficiency can be improved and the life of the infrared LED 20 can be extended.

The coaxial epi-illumination is not limited to the above-described embodiment. For example, a light-emitting element may be provided at the center of the lens of the camera, and the driver may be directly illuminated to form the coaxial epi-illumination.

[0061]

As described above, according to the first aspect of the present invention, the illumination light irradiation direction of the illumination means and the optical axis connecting the driver and the light input means are substantially coaxial at least in the vicinity of the driver. So that the lighting means and the light input means are installed,
Since a plurality of light input means having different spectral characteristics are provided, there is an effect that a driver photographing apparatus capable of easily and clearly photographing the characteristic points of the face can be obtained.

According to the second aspect, in addition to the effect of the first aspect, at least one optical element having a function of reflecting and transmitting light is installed in the optical path connecting the driver and the light input means. Since the face images of the driver taken by different light input means can be taken at exactly the same position and the feature points can be easily taken in correspondence with each other, the image processing is simpler and an efficient driver photographing device can be obtained. .

According to the third aspect, the illumination means and the light are arranged such that the illumination light irradiation direction of the illumination means and the optical axis connecting the driver and the light input means are substantially coaxial at least in the vicinity of the driver. An input unit is installed, and the first light limiting unit limits the light of a specific wavelength of the light incident on the light input unit, and the second light limiting unit having a spectral characteristic different from that of the first light limiting unit. Therefore, the feature points of the face can be photographed easily and clearly. Furthermore, there is an effect that a driver photographing device in which the feature points of the driver's face image can be easily taken can be obtained.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing a driver photographing apparatus according to a first embodiment of the present invention.

FIG. 2 is a configuration diagram showing a driver photographing apparatus according to the first embodiment.

FIG. 3 is a graph showing the sensitivity characteristic with respect to the wavelength (nm) of the camera of Example 1.

FIG. 4 is an explanatory diagram showing a captured image of a driver captured by a driver capturing apparatus according to the first embodiment.

FIG. 5 is an explanatory diagram showing a photographed image of a driver processed by the driver photographing apparatus according to the first embodiment.

FIG. 6 is an explanatory diagram showing a photographed image of a driver processed by the driver photographing apparatus according to the first embodiment.

FIG. 7 is a configuration diagram showing a driver photographing apparatus according to a second embodiment of the present invention.

FIG. 8 is a configuration diagram showing a driver photographing apparatus according to a third embodiment of the present invention.

9 is a wavelength (nm) of the cold mirror according to Example 3. FIG.
5 is a graph showing a reflectance (%) characteristic with respect to FIG.

FIG. 10 is a configuration diagram showing a driver photographing apparatus according to a fourth embodiment of the present invention.

FIG. 11 shows the wavelength (n of the cold filter according to the fourth embodiment.
It is a graph which shows the reflectance (%) characteristic to m).

FIG. 12 is a configuration diagram showing a driver photographing device according to a fifth embodiment of the present invention.

FIG. 13 is a configuration diagram showing a driver photographing apparatus according to Embodiment 6 of the present invention.

FIG. 14 is a configuration diagram showing a driver photographing apparatus according to Embodiment 7 of the present invention.

FIG. 15 is a configuration diagram showing a driver photographing apparatus according to Embodiment 8 of the present invention.

FIG. 16 shows the wavelength (n of the camera according to the eighth embodiment of the present invention.
3 is a graph showing a sensitivity characteristic with respect to m) and a graph showing a transmittance (%) characteristic with respect to a wavelength (nm) of the filter.

FIG. 17 is a configuration diagram showing a driver photographing apparatus according to Embodiment 9 of the present invention.

FIG. 18 is a configuration diagram showing a driver photographing apparatus according to Embodiment 10 of the present invention.

FIG. 19 is a configuration diagram showing a conventional driver photographing device.

FIG. 20 is an explanatory diagram showing a captured image of a driver captured by a conventional driver capturing device.

FIG. 21 is an explanatory diagram showing a captured image of a driver captured by a conventional driver capturing device.

[Explanation of symbols]

 1 Driver 6 Pupil 11 Infrared Camera 12 Visible Camera 13 Camera 20 Infrared LED 21 LED Drive Circuit 22 Filter Switching Circuit 31 Half Mirror 32 Half Mirror 33 Cold Mirror 34 Cold Filter 35 First Filter 36 Second Filter 37 Total Reflection Mirror 100 Driver behavior detection circuit

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁶ Identification number Office reference number FI Technical indication location H04N 7/18 N (72) Inventor Yoshiharu Morihiro 8-1-1 Tsukaguchihonmachi, Amagasaki Mitsubishi Electric Corporation Company Industrial Systems Research Center

Claims (3)

[Claims] 1. An illumination means for illuminating a driver in a vehicle compartment, and an optical input means for inputting an image of the driver installed at a position where a predetermined area including the driver's face can be photographed. The illumination means and the light input means are installed such that the illumination light irradiation direction of the illumination means and the optical axis connecting the driver and the light input means are substantially coaxial at least in the vicinity of the driver. A driver photographing device, comprising a plurality of light input means having different spectral characteristics as the light input means. 2. At least one optical element having a function of reflecting and transmitting light is provided in an optical path connecting the driver and the light input means.
The driver photographing device according to claim 1, wherein the driver photographing device is installed individually. 3. An illumination means for illuminating a driver in a vehicle compartment, and an optical input means installed at a position where a predetermined area including the face of the driver can be photographed and for inputting an image of the driver, The illumination means and the light input means are installed so that the illumination light irradiation direction of the illumination means and the optical axis connecting the driver and the light input means are substantially coaxial at least in the vicinity of the driver. And further comprising first light limiting means for limiting light of a specific wavelength of light incident on the light input means, and second light limiting means having spectral characteristics different from those of the first light limiting means. Driver imaging device.