JP4771797B2 - Distance measuring device and distance measuring method - Google Patents

Description

  The present invention relates to a distance measuring device and a distance measuring method for measuring the distance of an object to be photographed in an image.

  In recent years, a technique for acquiring a distance from a specific device (hereinafter, also simply referred to as a device) to a photographing object (person) has been developed. Hereinafter, in order to make the explanation easy to understand, the subject to be photographed is explained as a person. Such technology is applied to airbag control, robot control, and the like. Here, ultrasonic waves, laser radars, and cameras are used as methods for acquiring the distance from the apparatus to the person. However, the camera is more suitable considering the cost and the effect on the human body. There are various types of cameras, but using a compound eye camera enables highly accurate distance measurement. Stereo matching is an example of a distance measuring method using a compound eye camera. Stereo matching refers to a technique of searching for the same place of a three-dimensional object automatically copied by area correlation or the like from left and right images constituting a solid and converting it into distance information. However, the processing cost in stereo matching is high. In the actual environment, the amount of reflected light from the object captured by each camera may vary depending on the position of the light source, and it may not be possible to correctly search for corresponding points in stereo matching. The distance accuracy is reduced. Furthermore, in an actual environment, it is difficult to detect a person due to uneven brightness on the surface of a person (face) due to a change in illumination conditions due to light shielding (shadows) by an obstacle. Furthermore, even a human face facing the front of the apparatus may unintentionally rotate around the camera optical axis (in-plane rotation) with respect to the camera, making it difficult to detect the person.

Techniques for solving such problems are shown below. First, for the problem of high processing cost in stereo matching, features such as light and shade edges are used in stereo matching. Such a technique is disclosed in Patent Document 1 below. In the technique disclosed in Patent Document 1, the image is differentiated, and stereo matching is performed using the obtained edge image. Also, in a real environment, it is difficult to detect a person due to uneven brightness on the surface of the person (face) due to a change in illumination conditions due to light shielding (shadows) by an obstacle. Introduces human detection using face-to-face distance sensors such as ultrasound. Such a technique is disclosed in Patent Document 2 below. In the technique disclosed in Patent Document 2, a face-to-face distance sensor is used as a head detection tracking unit that detects the head of a person. In addition, even with the face of a person facing the front of the device, the face may unintentionally rotate around the camera optical axis (in-plane rotation) with respect to the camera, making it difficult to detect the person. On the other hand, a technique for correcting the orientation of the face using the positional relationship between the eyes, nose, and mouth, which is a feature of the face, is used. Such a technique is disclosed in Patent Document 3 below. In the technique disclosed in Patent Document 3, first, the position of the eye, which is a feature of the face, is extracted, the position of the mouth is obtained using the information, and then the face orientation ( (In-plane rotation) is corrected.
JP2003-15816 (paragraph 0035) JP 2002-56388 A (paragraph 0019) JP 2000-163592 A (paragraph 0010)

  However, in the technique disclosed in Patent Document 1, since distance information is obtained only for edges, there is a problem that the distance information becomes sparse. Moreover, in the technique disclosed in Patent Document 2, there is a problem that the cost increases due to the facing distance sensor. Further, the technique disclosed in Patent Document 3 has a problem that it greatly depends on the extraction accuracy of the eye position and the amount of calculation for correcting the orientation of the face is large.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide a distance measuring apparatus and a distance measuring method capable of performing distance measurement with high speed and high accuracy while suppressing cost.

In order to achieve the above object, according to the present invention, a photographing unit that photographs a subject to be photographed by two cameras arranged at a predetermined interval, and the photographing that is photographed by the two cameras of the photographing unit. Among the images of the object, an image quality improvement means for removing the influence of the lighting environment at the time of shooting and performing image quality improvement processing of the image under a predetermined image area, and the image quality improvement means Based on the image whose image quality has been improved, in order to reduce the influence of uneven brightness in the object to be photographed on the image, the image area of the dark portion on the image is defined as a predetermined reference line on the image. The image of the object to be imaged on the image is inverted by performing a reversal / smoothing process for smoothing the image using the luminance value of the image before the reversal and the luminance value of the image after the reversal . in front A detection unit that detects position information; and an image area on the image to be processed is limited based on the position information of the photographing object on the image detected by the detection unit, and the limited In the image area, correction processing means for emphasizing a change in luminance with respect to the two images taken by the two cameras of the photographing means, and processing performed by the correction processing means in the limited image area. And searching for corresponding points between the two images, and generating parallax image information based on a shift in the position of the searched corresponding points, and the parallax image information generating unit A distance calculating unit that obtains a parallax value based on the parallax image information, and calculates a distance between the imaging unit and the photographing object based on the parallax value; Obtaining the distance measuring device is provided. With this configuration, cost can be reduced, and high-speed and high-precision distance measurement can be performed.

  In the present invention, it is a preferable aspect of the present invention that the photographing object is a human face. With this configuration, the characteristics of a person's face can be used.

  In the present invention, when photographing the photographing object by the photographing means, a near-infrared light projecting means for projecting near-infrared light onto the photographing object in order to enable photographing for 24 hours. Providing is a preferred aspect of the present invention. With this configuration, the distance measuring device can be operated for 24 hours.

  Further, in the present invention, when photographing the object to be photographed by the photographing means on at least one of the two cameras, in order to reduce the influence of disturbance light including visible light, and to enable stable images to be photographed. It is a preferable aspect of the present invention to include a specific wavelength transmission filter that receives only the reflected light of the near infrared light projected from the near infrared light projecting means. With this configuration, a stable distance measurement can be performed against disturbance light.

  In the present invention, it is a preferable aspect of the present invention that the parallax image information generating unit uses dynamic programming to search for the corresponding points with high accuracy. With this configuration, distance measurement with high accuracy can be performed.

  Further, in the present invention, the correction processing means reduces a decrease in position measurement accuracy due to a difference in reflected light from the object to be photographed for each camera according to a light source position, and a highly accurate distance in an actual environment. In order to enable measurement, it is a preferred aspect of the present invention to perform processing using a LoG filter. With this configuration, highly accurate distance measurement can be performed in an actual environment.

  Further, in the present invention, the image quality improvement means includes a logarithmic conversion filter that absorbs an influence of an illumination environment at the time of shooting, and a high-pass filter that reduces a decrease in contrast of the image via the logarithmic conversion filter. This is a preferred embodiment of the present invention. With this configuration, it is possible to reduce the influence due to the change in the illumination condition in the actual environment, and it is possible to detect the photographing target that is robust against the illumination change.

  In the present invention, it is a preferred aspect of the present invention that the detection means detects the position information of the object to be photographed on the image using a neural network. With this configuration, identification suitable for the object can be used.

Further, in the present invention, when the detection unit rotates the optical axis with respect to the optical axis in order to enable detection of the photographing object rotating around the optical axis of the camera . The amount of rotation is rotated in the direction opposite to the direction of rotation, and how much the object to be photographed on the image is similar to the object on the previously learned image comprising a plurality of detectors for determining the degree of similarity indicating a, to detect the position information of the shooting target in the detector the image on narrowed on the basis of the similarity obtained by using the present This is a preferred embodiment of the invention. With this configuration, it is possible to detect an object to be photographed that rotates around the optical axis of the camera (in-plane rotation).

Further, in the present invention, the detection means inverts an image area of a dark portion on the image with reference to a predetermined reference line on the image, and the luminance value of the image before the inversion and the after inversion The position information of the object to be photographed on the inverted / smoothed image is smoothed by using a predetermined detector among the plurality of detectors. Preferably, the position information of the object to be photographed on the image that has not been subjected to the inversion / smoothing process is detected by a detector other than the predetermined detector among the plurality of detectors. It is. With this configuration, luminance unevenness on the face surface is reduced, and even when the face is tilted leftward or rightward, a pseudo shadow is not generated on the face surface, and the face can be detected correctly.

  In the present invention, it is a preferred aspect of the present invention that the detection means selects the detector to be used based on past detection results. With this configuration, the processing speed can be increased.

In addition, according to the present invention, one of the step of photographing the photographing object with two cameras arranged at a predetermined interval and the image of the photographing object photographed with the two cameras is selected. On the other hand, under the predetermined image area, the step of removing the influence of the illumination environment at the time of shooting and performing the image quality improvement processing of the image, and the shooting object on the image based on the image with improved image quality In order to reduce the influence of unevenness of brightness in the image, the image area of the dark portion on the image is inverted with reference to a predetermined reference line on the image, and the brightness value of the image before the inversion and after the inversion comprising the steps of by the inverted-smoothing processing for smoothing the image using a luminance value of an image detecting the position information of the shooting target on the image, the shooting on said detected image Based on the position information of the object, the image area on the image to be processed is limited, and in the limited image area, the luminance change is emphasized with respect to the two images taken by the two cameras. And searching for corresponding points between the two processed images in the limited image region, and generating parallax image information based on a positional shift of the searched corresponding points And a step of obtaining a parallax value based on the generated parallax image information, and calculating a distance between an imaging unit including the two cameras and the photographing object based on the parallax value. A distance measuring method is provided. With this configuration, cost can be reduced, and high-speed and high-precision distance measurement can be performed.

  In the present invention, it is a preferable aspect of the present invention that the photographing object is a human face. With this configuration, the characteristics of a person's face can be used.

  In addition, in the present invention, it is preferable that the method further includes a step of projecting near-infrared light onto the imaging object in order to enable 24-hour imaging when imaging the imaging object. It is. With this configuration, the distance measuring device can be operated for 24 hours.

  Further, in the present invention, when photographing the object, the reflected light of the near infrared light that is projected in order to reduce the influence of disturbance light including visible light and enable a stable image to be photographed. It is a preferable aspect of the present invention to further include a step of receiving only light. With this configuration, a stable distance measurement can be performed against disturbance light.

  Moreover, in this invention, it is a preferable aspect of this invention to use a dynamic programming in order to search the said corresponding point with high precision. With this configuration, distance measurement with high accuracy can be performed.

  Further, in the present invention, in order to reduce a decrease in position measurement accuracy due to a difference in reflected light from the object to be photographed captured by each camera according to a light source position, and to enable highly accurate distance measurement in an actual environment. In addition, it is a preferable aspect of the present invention to perform processing by the LoG filter. With this configuration, highly accurate distance measurement can be performed in an actual environment.

Further, in the present invention, the logarithmic conversion filter that absorbs the influence of the illumination environment absorbs the influence of the illumination environment at the time of shooting, and the high-pass filter that reduces the reduction of the contrast of the image reduces the contrast of the absorbed image. Mitigating is a preferred embodiment of the present invention. With this configuration, it is possible to reduce the influence due to the change in the illumination condition in the actual environment, and it is possible to detect the photographing target that is robust against the illumination change.

  In the present invention, it is a preferred aspect of the present invention to detect the position information of the photographing object on the image using a neural network. With this configuration, identification suitable for the object can be used.

In the present invention, in order to enable detection of the object to be photographed that rotates around the optical axis of the camera, the image with improved image quality is rotated when the image is rotated with respect to the optical axis. Rotate the amount of rotation that is rotating in the opposite direction to the direction in which the image is present, and obtain a degree of similarity indicating how similar the imaged object on the image to the object of the previously learned image It is a preferable aspect of the present invention to detect the position information of the photographing object on the image narrowed down based on the similarity obtained using a plurality of detectors. With this configuration, it is possible to detect an object to be photographed that rotates around the optical axis of the camera (in-plane rotation).

Further, in the present invention, the image area of the dark portion on the image is inverted with reference to a predetermined reference line on the image, and the luminance value of the image before the inversion and the luminance value of the image after the inversion The position information of the object to be imaged on the image that has been subjected to inversion / smoothing processing is smoothed by using a predetermined detector of the plurality of detectors, and the inversion / smoothing is performed. It is a preferable aspect of the present invention in which the position information of the object to be photographed on an image that has not been processed is detected by a detector other than a predetermined detector among the plurality of detectors. With this configuration, luminance unevenness on the face surface is reduced, and even when the face is tilted leftward or rightward, a pseudo shadow is not generated on the face surface, and the face can be detected correctly.

  In the present invention, it is a preferred aspect of the present invention to select the detector to be used based on past detection results. With this configuration, the processing speed can be increased.

  The distance measuring device and the distance measuring method of the present invention have the above-described configuration, can reduce the cost, and can perform distance measurement with high speed and high accuracy.

<First Embodiment>
Hereinafter, a first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a configuration diagram showing the configuration of the distance measuring apparatus according to the first embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of the area limited image quality improving unit in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 3 is a block diagram showing the configuration of the face detection unit in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 4 is a diagram for explaining the inversion / smoothing process in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 4A shows an image before inversion. FIG. 4B is a diagram showing the image after inversion. FIG. 4C shows a smoothed image.

  FIG. 5 is a diagram for explaining normalization of density information normalization in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 5A is a diagram showing the luminance value distribution of the input image. FIG. 5B shows a luminance value distribution of the output image. FIG. 6 is a diagram for explaining a mask applied to the background portion of the input image of the face identification unit in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 7 is a diagram for explaining position information output from the post-processing unit in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 8 is a diagram showing past face detection results in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 9 is a configuration diagram showing the configuration of the area limited brightness correction unit in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 10 is a diagram showing a processing target area that is a target of the brightness correction process in the distance measuring apparatus according to the first embodiment of the present invention.

  FIG. 11 is a diagram showing pixel values obtained by converting pixel values after LoG filter processing in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 12 is a configuration diagram showing the configuration of the region-limited stereo matching unit in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 13 is a diagram for explaining the parallax value in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 13A is a diagram for explaining the concept of the parallax value. FIG. 13B is a diagram for explaining the calculation of the parallax value in the dynamic programming. FIG. 14 is a configuration diagram showing the configuration of the person distance calculation unit in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 15 is a diagram for explaining the maximum parallax value in the distance measuring apparatus according to the first embodiment of the present invention. FIG. 15A is a diagram illustrating the parallax value of the parallax image information on the face surface. FIG. 15B is a diagram illustrating the magnitude relationship between the parallax values. FIG. 16 is a flowchart for explaining a processing flow in the distance measuring apparatus according to the first embodiment of the present invention.

  First, the configuration of the distance measuring apparatus according to the first embodiment of the present invention will be described with reference to FIG. As shown in FIG. 1, the distance measuring apparatus 100 includes a stereo image acquisition unit 104, a near-infrared light projector 105, a region limited image quality improvement unit 106, a face detection unit 107, a region limited light / dark correction unit 108, and a region limited stereo matching unit. 109, a person distance calculation unit 110, and history information 111. Here, in the following first and second embodiments, the distance measuring device 100 will be described as a person distance measuring device 100 that measures a distance from a person in particular. The present invention can be used in various fields. Here, description will be given for robot control, game control, and airbag control. First, it will be described as a human distance measuring device for robot control. The person distance measuring apparatus 100 transmits the distance information calculated by the person distance calculating unit 110 to a knowledge / judgment control apparatus (not shown). Based on the transmitted distance information, the knowledge / judgment control device controls the motor for movement. For example, the motor is controlled to approach a human if the distance to the human is far. Next, a human distance measuring device for game control will be described. The person distance measuring device 100 transmits the distance information calculated by the person distance calculating unit 110 to an operation control device (not shown). Based on the transmitted distance information, the operation control device operates the game. Finally, it will be described as a person distance measuring device for air bag control. The person distance measuring device 100 transmits the distance information calculated by the person distance calculating unit 110 to an airbag control device (not shown). Based on the transmitted distance information, the airbag control device controls an airbag deployment method, deployment power, and the like. For example, the deployment power of the airbag is controlled in several steps according to the distance between the passenger's face and the airbag.

  The stereo image acquisition unit 104 further includes a right camera 102a having a specific wavelength transmission filter 101a, a left camera 102b having a specific wavelength transmission filter 101b, and a camera control unit 103. Here, the right camera 102a and the left camera 102b are, for example, a CCD (Charge Coupled Device) camera or the like. When installing the right camera 102a and the left camera 102b, the right camera 102a and the left camera 102b are arranged at a predetermined interval, the optical axes of the right camera 102a and the left camera 102b are parallel to each other, and the optical axes of each other are on the same reference plane. Is desirable. The camera control unit 103 controls the stereo image acquisition unit 104 as a whole. For example, the camera control unit 103 transmits a synchronization signal for photographing to the right camera 102a, the left camera 102b, and the near-infrared light projector 105. . Based on this synchronization signal, the right camera 102a and the left camera 102b capture an object to be imaged, and the near-infrared light projector 105 emits light to the object to be imaged.

  As a result, it is possible to shoot for 24 hours. The right camera 102a and the left camera 102b need not receive light other than the reflected light of the light emitted by the near-infrared light projector 105 in order to capture a stable image. Therefore, the right camera 102a and the left camera 102b are provided with specific wavelength transmission filters 101a and 101b that transmit only light of a specific wavelength so that only the reflected light of the light emitted by the near-infrared light projector 105 is received. . As described above, the near-infrared light projector 105 emits light, but the emitted light is near-infrared light. Since the emitted light is near-infrared light, it does not affect the field of view, especially when a person is taken as an object.

  As shown in FIG. 2, the area limited image quality improving unit 106 includes an attention area setting unit 200, a high-pass filter 201, and a logarithmic conversion filter 202. Here, the logarithmic conversion filter 202 is a filter for absorbing the influence of the illumination environment (illumination conditions) at the time of photographing. The luminance value after logarithmic conversion can be obtained by the following equation (1). Here, Iin (x, y) is the luminance value of the image before logarithmic conversion, Iout (x, y) is the luminance value of the image after logarithmic conversion, and a, b, and c are predetermined parameter values.

  However, the image after passing through the logarithmic conversion filter 202 has a problem that the contrast of the image is lowered, and it is difficult to see facial organs (eyes, nose, mouth, etc.). Here, since the facial organ image is a high-frequency component, a high-contrast image can be obtained while reducing the influence of illumination conditions at the time of photographing by introducing the high-pass filter 201 in order to maintain the facial organ characteristics. It is done. The filter mask HP (i, j) of the high-pass filter 201 is represented by the following formula (2). The mask size is 3 × 3 pixels, for example. In this way, a high contrast image can be obtained by enhancing the center of the filter mask HP.

  Note that, in the processing in the high-pass filter 201 and the logarithmic conversion filter 202 described above, the attention area setting unit 200 selects an area to be processed based on the history information 111 in order to reduce useless processing and increase the processing speed. Set. The high-pass filter 201 and the logarithmic conversion filter 202 need only perform processing for the set region. The area-limited image quality improvement unit 106 receives only the image captured by the right camera 102a from the stereo image acquisition unit 104, which will be described. In order to realize high-speed processing, it is first necessary to find the position of an object to be measured. In order to find the position of the object, an image of only the right camera 102a is used. Although the image of the right camera 102a is used here, the present invention is not limited to this, and the image of the left camera 102b may be used. Further, in the history information 111, position information and the like of a person's face detected in the past are stored.

  As shown in FIG. 3, the face detection unit 107 includes a preprocessing unit 300, face identification units 306 to 308, and a postprocessing unit 316. The pre-processing unit 300 further includes a noise removal unit 301, an image reduction unit 302, an image area acquisition unit 303, an inversion / smoothing processing unit 304, and a density information normalization unit 305. In the noise removing unit 301, a low-pass filter (not shown) is applied to the image input from the area-limited image quality improving unit 106, and noise is removed. In the image reduction unit 302, the input image is converted into a plurality of sizes in order to cope with a change in the size of the face due to a change in the distance from the right camera 102a and the left camera 102b to the face of the person to be photographed. The

  In the image area acquisition unit 303, a plurality of images converted by the image reduction unit 302 are cut out with a predetermined size, for example, h (vertical) × w (width) pixels. Here, the clipped image is referred to as a clipped image. In the inversion / smoothing processing unit 304, all of the cutout images cut out by the image area acquisition unit 303 are subjected to a process of reducing luminance unevenness generated on the face surface due to biased illumination or the like. Here, the process of reducing luminance unevenness is performed by the following equations (3) and (4). Here, I ′ (x, y) is the luminance value after inversion, I (x, y) is the luminance value before inversion, and w is the width of the above-described clipped image. I ″ (x, y) is a smoothed luminance value. However, the domains of x and y are 0 to w−1 and 0 to h−1, respectively.

  In order to make the explanation of the processing in the inversion / smoothing processing unit 304 easy to understand, an explanation is given using an image in which the entire face is projected as a cut-out image. The image before inversion is shown in FIG. 4A, the image after inversion is shown in FIG. 4B, and the image after smoothing is shown in FIG. 4C. FIG. 4A shows an image before inversion, where a dark (high) portion 400 exists. By inverting the image of FIG. 4A, the dark portion 400 is reversed, and the dark portion 401 as shown in FIG. 4B is obtained. By smoothing FIGS. 4 (a) and 4 (b), a smoothed image as shown in FIG. 4 (c) is obtained, and luminance unevenness of the entire cut-out image is reduced.

  The tone information normalization unit 305 uses the following equation (5) for the image processed by the inversion / smoothing processing unit 304 to change the brightness of the entire clipped image, for example, outdoors at night and daytime. In order to cope with this, the shading information of the cut-out image is normalized. h (x) is the relative appearance frequency of the luminance value x, and f (g) is the luminance value after smoothing the histogram. Round [] means rounding off the first decimal place. By this conversion, a new luminance value is assigned so as to fall within the luminance value from 0 to 255 in proportion to the cumulative appearance ratio of the luminance value. FIG. 5A shows the distribution of luminance values of the input image, and FIG. 5B shows the distribution of luminance values of the output image. By substituting the relative appearance frequency of the luminance value of the input image shown in FIG. 5A into Expression (5), the shading of the image is normalized as shown in FIG. 5B.

  The output from the density information normalization unit 305 is input to the face identification units 306 to 308. In the first embodiment of the present invention, face identification units 307 and 308 are provided to enable detection of faces rotating around the optical axes of the right camera 102a and the left camera 102b (detection of face inclination). ing. Details of the face identification units 307 and 308 will be described later. First, the face identification unit 306 will be described. The face identification unit 306 is an identification unit that detects a face that does not rotate around the optical axes of the right camera 102a and the left camera 102b (detection of a face facing the front).

  The face identification unit 306 has a hierarchical neural network 309. The hierarchical neural network 309 includes an input layer, a hidden layer, and an output layer. The output value from the output layer is a numerical value from 0 to 1, and the closer the output value is to 1, the more the input image looks like a face, and the closer the output value is to 0, the less the face looks like the face. Show. Since the hierarchical neural network is a known technique, detailed description thereof is omitted.

  Since the face identification units 307 and 308 have the same configuration, only the face identification unit 307 will be described. The face identification unit 307 includes a hierarchical neural network 310, a mask 312, and a left / right reversing unit 313. Here, since the hierarchical neural network 310 is the same as the above-described hierarchical neural network 309, description thereof is omitted. The left / right reversing unit 313 is configured to detect the face when the person to be photographed is tilted left and right toward the front, with the input image as the axis of symmetry (the person is not tilted to the left and right). Invert. That is, the left / right reversing unit 313 is input when, for example, an image in which a person to be photographed is inclined to the right by α ° toward the front (rotating with respect to the optical axis of the camera) is input to itself. Invert the image to obtain an image tilted α ° to the left. The same applies to the left / right inversion unit 315 of the face identification unit 308. For example, when an image in which a person to be photographed is tilted β ° to the right toward the front is input to itself, the input image is inverted and left Acquire an image that is tilted by β °.

  Here, the background portion is included in the image in which the person to be photographed is tilted to the left and right, and the face portion cannot be detected with high accuracy. It is the mask 312 that excludes the background portion. The mask 312 will be described with reference to FIG. As shown in FIG. 6, the mask 312 masks the background portion of the input image. By applying the mask, the background portion can be eliminated, and more accurate face detection can be performed.

  As described above, the face identification units 306 to 308 obtain the degree of similarity of the face (which indicates how similar the input image is to the learned face image) with respect to the input image, The result is output to the post-processing unit 316. The post-processing unit 316 selects face candidates using a predetermined threshold or the like for the input face similarity, checks whether the plurality of face candidates after selection are the same face, and narrows down the candidates. The post-processing unit 316 outputs the position information of the face on the image, for example, the center coordinates of the detected face, the size of the face, etc., for the narrowed face. The position information will be described with reference to FIG. The face 700 on the narrowed-down image is subjected to a detection window 701 of Hc × Wc with the center coordinate of the face obtained by enlarging the cut-out image cut out by h × w being Oc.

  Note that in order to speed up the face detection process, it is possible to limit the processing range in the face identification units 306 to 308 using the past face detection results shown in FIG. The past face detection result may be stored in the history information 111 or may be stored in a predetermined storage area (not shown). Here, FIG. 8 will be described. For example, in the past time, when the person to be imaged is inclined α ° to the left (when face identification is performed in the face identification unit 307 (left direction) in the second row in FIG. 8), the next time (current At (time), there are two possible states: the person to be imaged further tilts to the left by α ° or more or returns to the front direction. This is because it is difficult to think that it is inclined more than α ° to the right from a state where it is inclined α ° to the left in a short time. Therefore, at the past time, if the person to be imaged is inclined α ° to the left (when face identification is performed in the face identification unit 307 (left direction) in the second row in FIG. 8), the next time (current Only the face identifying unit 308 (left direction), the face identifying unit 307 (left direction), and the face identifying unit 306 (front) that are assumed at (time) may be processed by each face identifying unit. As a result, the processing speed can be increased.

  As shown in FIG. 9, the area limited brightness correction unit 108 includes an area limitation setting unit 900 and a brightness correction processing unit 901. The area limitation setting unit 900 is a process as shown in FIG. 10 that is a target of a light / dark correction process (a process for emphasizing a change in luminance), which will be described later, based on the position information of the face on the image input from the face detection unit 107. The target area 1000 is set, and the area information (coordinates) is output. The brightness correction processing unit 901 acquires images of the right camera 102a and the left camera 102b from the stereo image acquisition unit 104 based on the region information from the region limitation setting unit 900, and applies LoG (Laplacian of Gaussian to the acquired image. ) Brightness / darkness correction processing using a filter is performed, and the corrected left and right camera images are output. Since the LoG filter is a known technique, a detailed description thereof will be omitted. The LoG filter is shown in the following formula (6). Here, G (x, y) and I (x, y) are an input pixel value and a pixel value after LoG filter processing, respectively. The performance of the LoG filter depends on the value of σ (a numerical value representing the degree of emphasizing the luminance change with surrounding pixels). Here, σ = 1. The coefficient of the filter mask LoG (i, j) generated when σ = 1 is shown in the following formula (7). This mask size is 7 × 7 pixels.

  The luminance value after LoG filter processing is a real value, but the pixel (luminance) value is a discrete value. Therefore, it is necessary to convert from a real value to a discrete value. Here, the real value is converted to the discrete value using the following equation (8). Here, J (x, y) and I (x, y) are a discrete pixel value and a pixel value after LoG filter processing, respectively. Further, s is a sample interval used for conversion, and is a predetermined parameter value. In addition, the pixel value after conversion by Formula (8) is shown in FIG.

  As shown in FIG. 12, the area limited stereo matching unit 109 includes an area limited setting unit 1200 and a corresponding point search unit 1201. Based on the face position information on the image input from the face detection unit 107, the region limitation setting unit 1200 sets a search target region to be a corresponding point search target described later, and outputs the region information (coordinates). To do. The corresponding point search unit 1201 is based on the region information from the region limitation setting unit 1200, and the corresponding points between the images for the corrected right camera 102a and left camera 102b input from the region limitation light / darkness correction unit 108. A search is performed and parallax image information is output. In the first embodiment of the present invention, in order to obtain a more accurate corresponding point, a technique based on dynamic programming that enables a corresponding point search in pixel units is used. Since this technique is a known technique, detailed description thereof is omitted.

  The parallax value will be described with reference to FIG. FIG. 13A shows an image 1300 captured by the right camera 102a and an image 1301 captured by the left camera 102b. The parallax value is the difference in x-coordinate between corresponding points in the left and right images. That is, in this case, the parallax value at a certain vertex is X1-Xr. The parallax image information represents parallax values as luminance values. On the other hand, FIG.13 (b) is a figure for demonstrating calculation of the parallax value in a dynamic programming. The horizontal axis of the figure excluding the right image 1300 and the left image 1301 (hereinafter also referred to as a figure showing the position of the area) indicates the position of each area on a certain scanning line in the image 1300 (right image 1300) photographed by the right camera 102a. The axis indicates the position of each region on a certain scanning line in the image 1301 (left image 1301) photographed by the left camera 102b. As each image is scanned from the left side, the right image 1300 enters the area 1303 of the object 1302 and then reaches the target point 1304. On the other hand, the left image 1301 enters the area of the surface 1305 of the object 1302, then enters the surface 1303 of the object 1302, and then reaches the destination point 1304. In the diagram showing the position of the region in FIG. 13B indicating this, a straight line having an inclination is shown at a corresponding point (corresponding region), and a straight line having no inclination is shown at a non-corresponding point (non-corresponding region). In this case, the parallax value at the target point 1304 is X1-Xr.

  The person distance calculation unit 110 includes a specific parallax value search unit 1400 and a distance calculation unit 1401 as shown in FIG. The specific parallax value search unit 1400 obtains a parallax value to be measured based on the parallax image information input from the region-limited stereo matching unit 109. This function depends on the application. In robot control, airbag control, and the like that require safety, the nearest distance is obtained as the distance to the human, and the specific parallax value search unit 1400 obtains the maximum parallax value. Obtaining the maximum parallax value means obtaining the coordinates of the face of the person to be imaged closest to the left and right cameras. By obtaining the maximum parallax value, it is possible to determine whether the robot does not collide with a human or whether to deploy an airbag. Here, the concept of obtaining the maximum parallax value from the parallax image information will be described with reference to FIG. FIG. 15A is a diagram illustrating the parallax value of the parallax image information on the face surface. As shown in FIG. 15A, the nose portion protrudes from the face surface as compared to other eyes and mouth. The parallax value corresponding to the nose portion is larger than the parallax values corresponding to other eyes and mouths. The magnitude relationship between the parallax values is shown in FIG. As shown in FIG. 15B, it is the nose that is closest to the camera, and its parallax value is c. It is assumed that the two cameras described above are installed near the airbag. For game control that does not require accuracy, the specific parallax value search unit 1400 can also obtain an average parallax value.

  The distance calculation unit 1401 calculates a distance from the face surface to the stereo image acquisition unit 104 using a specific parallax value based on the principle of triangulation, and the calculated distance information is transmitted to a robot control device (not shown) or an airbag. The data is output to the control device 1402 or the like. Here, the distance from the face surface to the stereo image acquisition unit 104 is calculated, but the present invention is not limited to this. That is, it may be a distance from the face surface to at least one of the two cameras, may be a distance from the face surface to a predetermined reference point of the stereo image acquisition unit 104, or may be a stereo from the face surface. The distance to a predetermined reference point other than the image acquisition unit 104 may be used. Specifically, in the case of a robot control device, an air bag control device 1402 or the like, the distance calculation unit 1401 calculates the closest distance from the face surface to the camera by the following equation (9). Dmin [mm] is the closest distance between the camera and the face surface, and z is the maximum parallax value on the face surface (corresponding to the parallax value c in the example of FIG. 15). F, h, and s indicate the focal length [mm] of the stereo camera, the distance [mm] between the two cameras, and the pixel size [mm / pixel], respectively. Note that Dmin may be calculated for each camera, and one of the calculated Dmins may be used.

  Since triangulation is a known technique, the description thereof is omitted. Based on this distance information Dmin, the airbag control device 1402 controls the airbag. Here, although the person distance measuring device 100 according to the first embodiment of the present invention is used as the airbag control device, the airbag control device is not limited to the airbag control device, and includes various types such as robot control and game control. It can be applied in various fields. In the first embodiment of the present invention described above, an example of the internal configuration of each component is shown. If processing is possible, a part of the internal configuration of a certain component may be changed. It may be moved into the components.

  Next, a person distance measuring method in the person distance measuring apparatus according to the first embodiment of the present invention will be described with reference to FIG. As shown in FIG. 16, the camera control unit 103 of the stereo image acquisition unit 104 captures the face of the person to be imaged via the right camera 102a and the left camera 102b (step S1601). The area-limited image quality improvement unit 106 applies the image from the right camera 102a input from the stereo image acquisition unit 104 to the high-pass filter 201 and the logarithmic conversion filter 202, and acquires a high contrast image while reducing the influence of illumination conditions at the time of shooting. (Step S1602). At this time, it is possible to limit the area to be processed based on the position information of the face of the person detected in the past accumulated in the history information 111.

  The face detection unit 107 detects the position information of the face on the image based on the image input from the area-limited image quality improvement unit 106 and the position information of the face of the person detected in the past accumulated in the history information 111. Is output (step S1603). The area limited brightness correction unit 108 sets a processing target area for the brightness correction process based on the position information of the face on the image input from the face detection unit 107, and based on the processing target area, the stereo image acquisition unit Images of the right camera 102a and the left camera 102b are acquired from 104, and brightness correction processing using a LoG filter is performed on the acquired images (step S1604). The area-limited stereo matching unit 109 sets a search target area to be searched for corresponding points based on the position information of the face on the image input from the face detection unit 107, and based on the search target area, Corresponding point search between images is performed on the corrected left and right camera images input from the limited brightness correction unit 108, and parallax image information is output (step S1605). The person distance calculation unit 110 obtains a specific parallax value based on the parallax image information input from the region-limited stereo matching unit 109, and based on the principle of triangulation, the distance from the face surface to the camera using the specific parallax value Is calculated (step S1606).

<Second Embodiment>
Hereinafter, a distance measuring device and a distance measuring method according to the second embodiment of the present invention will be described with reference to FIGS. The configuration of the distance measuring device according to the second embodiment of the present invention is basically the same as the configuration of the distance measuring device according to the first embodiment, and the difference is the configuration of the face detection unit. . In the following, the configuration of the face detection unit in the second embodiment of the present invention and the person distance measurement method in the distance measurement apparatus according to the second embodiment of the present invention will be described. It should be noted that when the same components as those of the distance measuring device according to the first embodiment, which are the same as those of the distance measuring device according to the first embodiment, are described. It demonstrates using the code | symbol of the component of the distance measuring device which concerns.

  As shown in FIG. 17, the face detection unit 1071 includes a preprocessing unit 1700, face identification units 1707 to 1709, and a postprocessing unit 1717. The pre-processing unit 1700 further includes a noise removal unit 1701, an image reduction unit 1702, an image region acquisition unit 1703, an inversion / smoothing processing unit 1704, and density information normalization units 1705 and 1706. In the noise removing unit 1701, a low-pass filter (not shown) is applied to the image input from the region-limited image quality improving unit 106, and noise is removed. In the image reduction unit 1702, the input image is converted into a plurality of sizes in order to cope with a change in the size of the face due to a change in the distance from the right camera 102a and the left camera 102b to the face of the person to be photographed. The

  In the image area acquisition unit 1703, a plurality of images converted by the image reduction unit 1702 are cut out with a predetermined size, for example, h (vertical) × w (width) pixels. Here, the clipped image is referred to as a clipped image. The inversion / smoothing processing unit 1704 performs a process of reducing luminance unevenness generated on the face surface due to biased illumination or the like with respect to all the cut-out images cut out by the image region acquisition unit 1703. Here, the process of reducing luminance unevenness is performed by the equations (3) and (4) described in the first embodiment.

  In order to make the description of the processing in the inversion / smoothing processing unit 1704 easier to understand, description will be made using an image in which the entire face is projected as a cut-out image. The image before inversion is shown in FIG. 4A described in the first embodiment, the image after inversion is shown in FIG. 4B described in the first embodiment, and the image after smoothing is shown in FIG. FIG. 4C described in the embodiment of FIG. FIG. 4A shows an image before inversion, and a dark portion 400 exists. By reversing the image of FIG. 4A, the dark portion 400 is reversed, and a dark portion 401 as shown in FIG. 4B is obtained. By smoothing FIGS. 4 (a) and 4 (b), a smoothed image as shown in FIG. 4 (c) is obtained, and luminance unevenness of the entire cut-out image is reduced.

  The density information normalization units 1705 and 1706 apply the expression (5) described in the first embodiment to the image processed by the inversion / smoothing processing unit 1704 and the cut-out image cut out by the image area acquisition unit 1703. ) Is used to normalize the grayscale information of the clipped image in order to deal with the brightness variation of the entire clipped image, for example, night and daytime outdoors. By this conversion, a new luminance value is assigned so as to fall within the luminance value from 0 to 255 in proportion to the cumulative appearance ratio of the luminance value. FIG. 5A shows the distribution of luminance values of the input image, and FIG. 5B shows the distribution of luminance values of the output image. By substituting the relative appearance frequency of the luminance value of the input image shown in FIG. 5A into Expression (5), the shading of the image is normalized as shown in FIG. 5B.

  An output from the density information normalization unit 1705 is input to the face identification unit 1707. The output from the shading information normalization unit 1706 is input to the face identification units 1708 and 1709. The separation of outputs in this way will be described below. For the front face, uneven brightness on the face surface due to direct sunlight, concealment, etc. is reduced, but if the inversion / smoothing process is performed when the face is tilted leftward or rightward, There is a possibility that a pseudo shadow may occur in the face and the face may not be detected correctly. Therefore, by dividing the output as described above, luminance unevenness on the face surface is reduced, and even when the face is tilted leftward or rightward, a pseudo shadow does not occur on the face surface, It can be detected correctly. In the second embodiment of the present invention, face identification units 1708 and 1709 are provided to enable detection of faces rotating around the optical axes of the right camera 102a and the left camera 102b (detection of face inclination). ing. Details of the face identification units 1708 and 1709 will be described later. First, the face identification unit 1707 will be described. The face identification unit 1707 is an identification unit that detects a face that does not rotate around the optical axes of the right camera 102a and the left camera 102b (detection of a face facing the front).

  The face identification unit 1707 has a hierarchical neural network 1710. The hierarchical neural network 1710 includes an input layer, a hidden layer, and an output layer. The output value from the output layer is a numerical value from 0 to 1, and the closer the output value is to 1, the more the input image looks like a face, and the closer the output value is to 0, the less the face looks like the face. Show. Since the hierarchical neural network is a known technique, detailed description thereof is omitted.

  Since both the face identification units 1708 and 1709 have the same configuration, only the face identification unit 1708 will be described. The face identification unit 1708 includes a hierarchical neural network 1711, a mask 1712, and a left / right reversing unit 1713. Here, since the hierarchical neural network 1711 is the same as the above-described hierarchical neural network 1710, description thereof is omitted. The left / right reversing unit 1713 uses the input image as a symmetry axis with the input image (the person is not tilted to the left and right) to enable detection of the face when the person to be photographed is tilted to the left and right. Invert. That is, the left / right reversing unit 1713 is input when an image of the person to be imaged is inclined to the right by α ° toward the front (rotated with respect to the optical axis of the camera) is input to itself. Invert the image to obtain an image tilted α ° to the left. The same applies to the left / right reversing unit 1716 of the face identifying unit 1709. For example, when an image in which a person to be photographed is tilted β ° to the right toward the front is input to itself, the input image is inverted and left Acquire an image that is tilted by β °.

  Here, the background portion is included in the image in which the person to be photographed is tilted to the left and right, and the face portion cannot be detected with high accuracy. A mask 1712 excludes the background portion. For the mask 1712, refer to FIG. 6 described in the first embodiment. As shown in FIG. 6, the mask 1712 masks the background portion of the input image. By applying the mask, the background portion can be eliminated, and more accurate face detection can be performed.

  As described above, the face identification units 1707 to 1709 obtain the degree of similarity of the face with respect to the input image (which indicates how similar the input image is to the learned face image) The result is output to the post-processing unit 1717. The post-processing unit 1717 selects a face candidate using a predetermined threshold or the like for the input face similarity, checks whether or not the plurality of face candidates after selection are the same face, and selects a candidate. Narrow down. The post-processing unit 1717 outputs the position information of the face on the image, for example, the center coordinates of the detected face, the size of the face, etc., for the narrowed face. The position information will be described with reference to FIG. 7 described in the first embodiment. The face 700 on the narrowed-down image is covered with a detection window 701 of 2Hc × 2Wc with the center coordinate of the face obtained by enlarging the cut-out image cut out by h × w being Oc.

  Note that in order to speed up the face detection process, it is possible to limit the processing range in the face identification units 1707 to 1709 using the past face detection results shown in FIG. The past face detection result may be stored in the history information 111 or may be stored in a predetermined storage area (not shown). Here, FIG. 18 will be described. For example, in the past time, when the person to be imaged is inclined α ° to the left (when face identification is performed in the face identification unit 1708 (left direction) in the second row in FIG. 18), the next time (current At (time), there are two possible states: the person to be imaged further tilts to the left by α ° or more or returns to the front direction. This is because it is difficult to think that it is inclined more than α ° to the right from a state where it is inclined α ° to the left in a short time. Therefore, at the past time, if the person to be imaged is inclined α ° to the left (when face identification is performed in the face identification unit 1708 (left direction) in FIG. 18), the next time (current For the (time), only the face identification unit 1709 (left direction), the face identification unit 1708 (left direction), and the face identification unit 1707 (front) may be processed by each face identification unit. As a result, the processing speed can be increased. In addition, since it is the same as that of the component of 1st Embodiment about other components other than the face identification part of the distance measuring device which concerns on 2nd Embodiment, description is abbreviate | omitted.

  Next, a person distance measuring method in the person distance measuring apparatus according to the second embodiment of the present invention will be described with reference to FIG. As shown in FIG. 19, the camera control unit 103 of the stereo image acquisition unit 104 captures the face of the person to be imaged via the right camera 102a and the left camera 102b (step S1901). The area-limited image quality improvement unit 106 applies the image from the right camera 102a input from the stereo image acquisition unit 104 to the high-pass filter 201 and the logarithmic conversion filter 202, and acquires a high contrast image while reducing the influence of illumination conditions at the time of shooting. (Step S1902). At this time, it is possible to limit the area to be processed based on the position information of the face of the person detected in the past accumulated in the history information 111.

  The face detection unit 1071 is based on the image input from the area-limited image quality improvement unit 106 and the position information of the face of the person detected in the past accumulated in the history information 111, and the position information of the face on the image. Is output (step S1903). The area limited brightness correction unit 108 sets a processing target area for the brightness correction process based on the position information of the face on the image input from the face detection unit 1071, and based on the processing target area, the stereo image acquisition unit The images of the right camera 102a and the left camera 102b are acquired from 104, and brightness correction processing using the LoG filter is performed on the acquired images (step S1904). The region-limited stereo matching unit 109 sets a search target region to be searched for corresponding points based on the position information of the face on the image input from the face detection unit 1071, and based on the search target region, A corresponding point search between images is performed on the corrected left and right camera images input from the limited brightness correction unit 108, and parallax image information is output (step S1905). The person distance calculation unit 110 obtains a specific parallax value based on the parallax image information input from the region-limited stereo matching unit 109, and based on the principle of triangulation, the distance from the face surface to the camera using the specific parallax value Is calculated (step S1906).

  The distance measuring apparatus and the distance measuring method according to the present invention can perform cost measurement at high speed and with high accuracy, and therefore the distance measuring apparatus and the distance measuring method for measuring the distance of the object to be photographed in the image. It is useful for such as.

It is a block diagram which shows the structure of the distance measuring device which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the area | region limited image quality improvement part in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the face detection part in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the inversion and smoothing process in the distance measuring device which concerns on the 1st Embodiment of this invention. (A) It is a figure which shows the image before inversion. (B) It is a figure which shows the image after inversion. (C) It is a figure which shows the smoothed image. It is a figure for demonstrating normalization of the shading information normalization in the distance measuring device which concerns on the 1st Embodiment of this invention. (A) It is a figure which shows luminance value distribution of an input image. (B) It is a figure which shows the luminance value distribution of an output image. It is a figure for demonstrating the mask put on the background part of the input image of the face identification part in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the positional information output from the post-processing part in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a figure which shows the past face detection result in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the area | region limited brightness correction part in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a figure which shows the process target area | region used as the object of the brightness correction process in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a figure which shows the pixel value which converted the pixel value after the LoG filter process in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the area | region limited stereo matching part in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the parallax value in the distance measuring device which concerns on the 1st Embodiment of this invention. (A) It is a figure for demonstrating the concept of a parallax value. (B) It is a figure for demonstrating calculation of the parallax value in a dynamic programming. It is a block diagram which shows the structure of the person distance calculation part in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the maximum parallax value in the distance measuring device which concerns on the 1st Embodiment of this invention. (A) It is a figure which shows the parallax value of the parallax image information in the face surface. (B) It is a figure which shows the magnitude relationship of a parallax value. It is a flowchart for demonstrating the processing flow in the distance measuring device which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the face detection part in the distance measuring device which concerns on the 2nd Embodiment of this invention. It is a figure which shows the past face detection result in the distance measuring device which concerns on the 2nd Embodiment of this invention. It is a flowchart for demonstrating the processing flow in the distance measuring device which concerns on the 2nd Embodiment of this invention.

Explanation of symbols

100 person distance measuring device (distance measuring device)
101a Specific wavelength transmission filter of right camera 101b Specific wavelength transmission filter of left camera 102a Right camera 102b Left camera 103 Camera control unit 104 Stereo image acquisition unit (imaging means)
105 Near-infrared light projector 106 Area-limited image quality improvement unit (image quality improvement means)
107, 1071 Face detection unit (detection means)
108 Area limited brightness correction section (correction processing means)
109 Region-limited stereo matching unit (parallax image information generating means)
110 Person distance calculation unit (distance calculation means)
111 history information 200 attention area setting section 201 high pass filter 202 logarithmic transformation filter 300, 1700 preprocessing section 301, 1701 noise removal section 302, 1702 image reduction section 303, 1703 image area acquisition section 304, 1704 inversion / smoothing processing section 305 , 1705, 1706 Gray information normalization unit 306, 307, 308, 1707, 1708, 1709 Face identification unit 309, 310, 311, 1710, 1711, 1714 Hierarchical neural network 312, 314, 1712, 1715 Mask 313, 315, 1713, 1716 Left-right inversion unit 316, 1717 Post-processing unit 400 High-intensity portion of image before inversion 401 High-intensity portion of image after inversion 700 Face on image 701 Detection window 900, 1200 Area limitation setting unit 901 Brightness correction processing unit 1000 Processing target region 1201 Corresponding point search unit 1300 Image captured by right camera (right image)
1301 Image taken with left camera (left image)
1302 Object 1303, 1305 Object surface 1304 Target point 1400 Specific parallax value search unit 1401 Distance calculation unit 1402 Airbag control device

Claims (22)

Photographing means for photographing an object to be photographed by two cameras arranged at a predetermined interval;
The image quality of the image is obtained by removing the influence of the illumination environment at the time of shooting, with respect to one of the images of the shooting target image shot by the two cameras of the shooting unit, under a predetermined image area. Image quality improvement means for performing improvement processing,
Based on the image whose image quality has been improved by the image quality improving means, in order to reduce the influence of uneven brightness on the object to be photographed on the image, an image area of a dark portion on the image is displayed on the image. The image is inverted with respect to a predetermined reference line, and an inversion / smoothing process is performed on the image to smooth the image using the luminance value of the image before the inversion and the luminance value of the image after the inversion . Detecting means for detecting the position information of the photographing object;
Based on the position information of the object to be photographed on the image detected by the detecting means, the image area on the image to be processed is limited, and in the limited image area, the photographing means Correction processing means for performing processing for emphasizing a change in luminance with respect to two images taken by two cameras;
Parallax image information for searching for corresponding points between the two images processed by the correction processing means in the limited image area, and generating parallax image information based on a positional shift of the searched corresponding points Generating means;
A distance calculation unit that obtains a parallax value based on the parallax image information generated by the parallax image information generation unit, and calculates a distance between the imaging unit and the photographing object based on the parallax value; The
A distance measuring device provided.   The distance measuring device according to claim 1, wherein the photographing object is a human face.   3. A near-infrared light projecting unit that projects near-infrared light onto the object to be photographed in order to enable 24-hour imaging when the imaging object is imaged by the imaging unit. The distance measuring device described in 1.   In order to reduce the influence of disturbance light including visible light when photographing the object to be photographed by the photographing means on at least one of the two cameras, the near-infrared light can be captured. The distance measuring device according to claim 3, further comprising a specific wavelength transmission filter that receives only the reflected light of the near infrared light projected from the light projecting unit.   The distance measurement apparatus according to claim 1, wherein the parallax image information generation unit uses dynamic programming to search for the corresponding points with high accuracy.   The correction processing means reduces a decrease in position measurement accuracy due to a difference in reflected light from the object to be photographed for each camera depending on a light source position, and enables highly accurate distance measurement in an actual environment. The distance measuring device according to claim 1, wherein processing by a LoG filter is performed.   The said image quality improvement means is provided with the logarithmic conversion filter which absorbs the influence of the illumination environment at the time of imaging | photography, and the high-pass filter which reduces the fall of the contrast of the said image through the said logarithmic conversion filter. The distance measuring device according to one. It said detecting means, a distance measuring apparatus according to any one of claims 1 to 7 for detecting the position information of the object to be shot on the image using a neural network. The detection means is rotated when the image with improved image quality is rotated with respect to the optical axis to enable detection of the object to be photographed that rotates about the optical axis of the camera . Rotate the amount of rotation that is rotating in a direction opposite to the direction to obtain a similarity indicating how similar the object to be photographed on the image to the object on the image learned in advance. comprising a plurality of detectors, any one of claims 1 to 8 for detecting the position information of the shooting target in the detector used in the similarity the image on narrowed based on the obtained The distance measuring device described in 1. The detection means inverts the image area of the dark portion on the image with reference to a predetermined reference line on the image , and calculates the luminance value of the image before the inversion and the luminance value of the image after the inversion. Using the predetermined detector of the plurality of detectors to detect the position information of the object to be photographed on the image subjected to inversion / smoothing processing, smoothing the image using the inversion / smoothing The distance measuring apparatus according to claim 9 , wherein the position information of the object to be imaged on an unprocessed image is detected by a detector other than a predetermined detector among the plurality of detectors. The distance measuring device according to claim 9 or 10 , wherein the detection unit selects the detector to be used based on a past detection result. Photographing an object to be photographed by two cameras arranged at a predetermined interval;
Of the images of the object to be photographed photographed by the two cameras, the effect of the illumination environment at the time of photographing is removed under a predetermined image area and the image quality improvement processing of the image is performed. Steps,
Based on the image whose image quality has been improved, in order to reduce the influence of uneven brightness in the object to be photographed on the image, the image area of the dark portion on the image is defined as a predetermined reference line on the image. The image of the object to be imaged on the image is inverted by performing a reversal / smoothing process for smoothing the image using the luminance value of the image before the reversal and the luminance value of the image after the reversal . Detecting position information;
Based on the position information of the object to be photographed on the detected image, the image area on the image to be processed is limited, and the two images captured by the two cameras in the limited image area. Performing a process of emphasizing a change in luminance for one image;
Searching for corresponding points between the processed two images in the limited image region, and generating parallax image information based on a shift in the position of the searched corresponding points;
Obtaining a parallax value based on the generated parallax image information, and calculating a distance between the imaging unit including the two cameras and the photographing object based on the parallax value;
A distance measuring method. The distance measuring method according to claim 12 , wherein the photographing object is a human face. The distance measuring method according to claim 12 or 13 , further comprising a step of projecting near-infrared light onto the object to be photographed so as to enable photographing for 24 hours when the object to be photographed is photographed. A step of receiving only the reflected light of the near-infrared light to be projected in order to reduce the influence of disturbance light including visible light and to capture a stable image when photographing the object to be photographed; The distance measurement method according to claim 14 , further comprising: Distance measuring method according to to explore the precise the corresponding points, any one of claims 12 to 15 using a dynamic programming. Processing by LoG filter in order to reduce the degradation of position measurement accuracy due to the difference in reflected light from the object to be photographed for each camera depending on the light source position, and to enable highly accurate distance measurement under real environment The distance measuring method according to any one of claims 12 to 16 , wherein: Absorbing the influence of the illumination environment at the time of photographing by the logarithmic conversion filter that absorbs the impact of lighting environment, by the high-pass filter to reduce the degradation of the contrast of the image from the claims 12 to reduce the lowering of the contrast of the absorbed the image The distance measuring method according to any one of 17 . The distance measuring method according to any one of claims 12 to 18 , wherein the position information of the photographing object on the image is detected using a neural network. In order to enable detection of the object to be photographed rotating around the optical axis of the camera, when the image with improved image quality is rotated with respect to the optical axis, the direction opposite to the direction of rotation wherein the rotation amount which is rotating is rotated, using a plurality of detectors for determining the similarity indicating whether the photographing object on the image is similar extent relative to the object in advance learned image The distance measurement method according to any one of claims 12 to 19 , wherein the position information of the object to be photographed on the image narrowed down based on the similarity obtained in this way is detected. The image area of the dark portion on the image is inverted with reference to a predetermined reference line on the image, and the luminance value of the image before the inversion and the luminance value of the image after the inversion are used. performs detection of the position information of the shooting target on reversed-smoothed image to smoothing by a given detector of the plurality of detectors, not processed the inverted-smoothed image The distance measurement method according to claim 20 , wherein the position information of the object to be photographed is detected by a detector other than a predetermined detector among the plurality of detectors. The distance measuring method according to claim 20 or 21 , wherein the detector to be used is selected based on a past detection result.

Images