JP5280768B2 - Vehicle periphery monitoring device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle vicinity monitoring apparatus capable of highly precisely calculating a parallax offset amount by taking an object for reference in calculating the parallax offset amount as an object to be moved. <P>SOLUTION: The vehicle vicinity monitoring apparatus calculates the parallax offset amount resulting from a deviation of parallelism between optical axes of two imaging devices. When prescribed conditions that only one object exists in an irradiation area in which a distance is detected by a radar apparatus and only one extracted object exists in a local image area for a captured image corresponding to the one irradiation area are satisfied, a distance to the object is detected by the radar apparatus and a parallax between the objects obtained by the two imaging devices is calculated. Thus, the parallax offset amount can be calculated with high precision. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a vehicle periphery monitoring device that monitors the periphery of a vehicle using a captured image obtained by an imaging device mounted on the vehicle.

A technique is known in which the parallax of an object is calculated based on two images obtained by two imaging means mounted on a vehicle, and the distance from the parallax to the object is detected (see, for example, Patent Document 1). ). In the conventional technique, the parallax of the object stationary at the first and second times is calculated, and the movement amount of the vehicle in the period from the first time to the second time is calculated from the vehicle speed sensor. Then, based on the parallax and the amount of movement of the vehicle, a parallax offset amount caused by a deviation in parallelism between the optical axes of the two imaging units is calculated, and the parallax is corrected by the calculated parallax offset amount.
JP 2001-169310 A

  However, in the above-described prior art, when the parallax offset amount is calculated, the target object serving as a reference for the parallax at the first and second times is set as a stationary target object. For this reason, when calculating the parallax offset amount, if the target object for parallax at the first and second times is a moving object, the amount of change in parallax at the first and second times and the vehicle Since the correspondence with the movement amount is not constant, the parallax offset amount calculated by the above-described conventional technique cannot be guaranteed.

  Therefore, an object of the present invention is to provide a vehicle periphery monitoring device that can calculate a parallax offset amount with high accuracy as a target object that moves a target object when calculating the parallax offset amount.

A vehicle environment recognition device according to a first aspect of the invention calculates a parallax of an object based on two captured images obtained by two imaging devices mounted on a vehicle, and detects a distance from the parallax to the object. A vehicle periphery monitoring device that monitors the periphery of the vehicle, and detects the distance to the first object by detecting the intensity of the reflected wave in each of the irradiation areas of the plurality of transmission waves that radiate from the vehicle. Whether the radar apparatus, the parallax calculation means for calculating the parallax of the second object obtained by the two imaging apparatuses, and the first object and the second object are the same object When the same object determination means and the same object determination means determine that the first object and the second object are the same object, the radar device detects The same object from the vehicle The amount of parallax offset caused by the deviation in parallelism between the optical axes of the two imaging devices based on the distance up to and the parallax of the same object obtained by the two imaging devices calculated by the parallax calculation means And a parallax correction unit that corrects the parallax based on the parallax offset amount calculated by the parallax offset amount calculation unit .
Then, the same object determination means includes only one first object in the one irradiation area where the distance is detected by the radar device, and corresponds the captured image to the irradiation area. A predetermined condition that only one second object obtained by the two imaging devices exists in one local image region in the captured image corresponding to the one irradiation region. Whether the first object and the second object are the same object or not is determined according to whether or not is satisfied.

  According to the vehicle periphery monitoring device of the first aspect of the invention, the first object whose distance from the vehicle is detected by the radar device and the second object obtained by the two imaging devices are the same object. If determined, based on the distance from the vehicle to the same object detected by the radar device and the parallax of the same object obtained by the two imaging devices, the parallel deviation between the optical axes of the two imaging devices is detected. A parallax offset amount for calculating the resulting parallax offset amount is calculated. In view of the fact that the distance from the vehicle to the first object is accurately detected by the radar device regardless of the stationary object or the moving object, the parallax offset amount can be accurately determined by the method as described above. Can be calculated.

Then, there is only one object in one irradiation area where the distance is detected by the radar apparatus, and one object extracted in one local image area in the captured image corresponding to one irradiation area. If the predetermined condition that there is only one object is not satisfied, the object whose distance is detected by the radar device in one irradiation area and the object extracted from one corresponding local image area are the same object. It is difficult to determine whether or not, and the amount of parallax offset cannot be calculated with high accuracy. On the other hand, if the predetermined condition is satisfied, there is a possibility that the object whose distance is detected by the radar device in one irradiation area and the object extracted from one corresponding local image area are the same object. high. In view of this point, when the predetermined condition is satisfied, the distance to the target is detected by the radar device and the parallax of the target obtained by the two imaging devices is calculated. As described above, the parallax offset amount can be calculated with high accuracy.

The vehicle periphery monitoring device according to a second aspect is the vehicle periphery monitoring device according to the first aspect, wherein the parallax offset amount calculation means corresponds to the plurality of irradiation regions and the plurality of irradiation regions by the same object determination unit, respectively. When the predetermined condition is satisfied for a plurality of the local image regions, the parallax offset in the plurality of the same objects determined to be the same as the first object and the second object The amount is averaged, and the parallax correction unit corrects the parallax based on the averaged parallax offset amount.

According to the vehicle periphery monitoring device of the second invention , when it is determined that the predetermined condition is satisfied for the plurality of irradiation regions and the plurality of local image regions respectively corresponding to the plurality of irradiation regions, the plurality of conditions that satisfy the predetermined condition are satisfied. The amount of parallax offset in the target object is averaged. Therefore, even when there are variations in measurement distance errors to the object in a plurality of irradiation regions that satisfy a predetermined condition, the influence of measurement error variations can be reduced and the parallax offset amount can be calculated with high accuracy.

A vehicle periphery monitoring device according to a third aspect of the invention calculates a parallax of an object based on two captured images obtained by two imaging devices mounted on the vehicle, and detects a distance from the parallax to the object. A vehicle periphery monitoring device for monitoring the periphery of the vehicle, wherein the radar device detects the distance to the object by detecting the intensity of the reflected wave in each of the irradiation areas of the plurality of transmission waves radiating from the vehicle; The radar device calculates the parallax of the object obtained by the two imaging devices and vehicle determination means for determining whether the object is a forward vehicle based on the distance to the object. a parallax calculating unit, when the object by the vehicle determining means is determined to be a forward vehicle, the distance between the parallax calculating unit from the vehicle detected by the radar device to the front vehicle Parallax offset amount calculating means for calculating a parallax offset amount caused by a parallelism shift between the optical axes of the two imaging devices based on the calculated parallax of the preceding vehicle obtained by the two imaging devices. A parallax correction unit that corrects the parallax based on the parallax offset amount calculated by the parallax offset amount calculation unit, and the parallax offset amount calculation unit determines that the object is a forward vehicle by the vehicle determination unit. When it is determined, the parallax offset amount is calculated based only on the preceding vehicle .

According to the vehicle periphery monitoring device of the third invention , when the radar device determines that the object is a forward vehicle based on the distance to the object, the distance from the vehicle detected by the radar device to the preceding vehicle is Based on the parallax of the preceding vehicle obtained by the two imaging devices, a parallax offset amount caused by the deviation in parallelism between the optical axes of the two imaging devices is calculated. In view of the fact that the distance from the vehicle to the preceding vehicle is accurately detected by the radar device, the parallax offset amount can be calculated with high accuracy by the method as described above.

A vehicle periphery monitoring device according to a fourth aspect of the invention calculates a parallax of an object based on two captured images obtained by two imaging devices mounted on the vehicle, and detects a distance from the parallax to the object. A vehicle periphery monitoring device for monitoring the periphery of the vehicle, wherein the radar device detects the distance to the object by detecting the intensity of the reflected wave in each of the irradiation areas of the plurality of transmission waves radiating from the vehicle; Vehicle determination means for determining whether or not the object obtained by the two image pickup devices is a preceding vehicle, and parallax calculation means for calculating the parallax of the object obtained by the two image pickup devices. , when the object by the vehicle determining means is determined to be a forward vehicle, the calculated by the distance between the parallax calculating unit from the vehicle detected by the radar device to the front vehicle Parallax offset amount calculating means for calculating a parallax offset amount caused by a parallelism shift between the optical axes of the two imaging devices based on the parallax of the preceding vehicle obtained by two imaging devices; and the parallax offset amount calculation Parallax correction means for correcting parallax based on the parallax offset amount calculated by the means, and when the vehicle determination means determines that the object is a forward vehicle, the parallax offset amount calculation means The parallax offset amount is calculated based only on the preceding vehicle .

According to the vehicle periphery monitoring device of the fourth aspect of the present invention , when it is determined that the object obtained by the two imaging devices is a forward vehicle, the distance from the vehicle to the forward vehicle detected by the radar device and the two imagings are detected. Based on the parallax of the preceding vehicle obtained by the apparatus, a parallax offset amount caused by a deviation in parallelism between the optical axes of the two imaging apparatuses is calculated. In view of the fact that the distance from the vehicle to the preceding vehicle is accurately detected by the radar device, the parallax offset amount can be calculated with high accuracy by the method as described above.

[First embodiment]
1st Embodiment of the vehicle periphery monitoring apparatus of this invention is described. First, the structure of the vehicle periphery monitoring apparatus of this embodiment is demonstrated. The vehicle periphery monitoring device includes an image processing unit 1 shown in FIGS. 1 and 2. Connected to the image processing unit 1 are two infrared cameras 2R and 2L as imaging devices that capture an image ahead of the host vehicle 10 and a radar device 3 that detects a distance to an object ahead of the host vehicle 10. At the same time, as sensors for detecting the traveling state of the host vehicle 10, the yaw rate sensor 4 that detects the yaw rate of the host vehicle 10, the vehicle speed sensor 5 that detects the traveling speed (vehicle speed) of the host vehicle 10, and the brake operation of the host vehicle 10. A brake sensor 6 that detects presence or absence is connected. Further, the image processing unit 1 includes a speaker 7 for outputting auditory notification information such as voice, and a display device for displaying captured images and visual notification information captured by the infrared cameras 2R and 2L. 8 is connected. The infrared cameras 2R and 2L correspond to the imaging device in the present invention.

  Although not shown in detail, the image processing unit 1 includes electronic circuits including an A / D conversion circuit, a microcomputer (having a CPU, a RAM, a ROM, and the like), an image memory, and the like. The infrared cameras 2R, 2L, Analog signals output from the radar device 3, the yaw rate sensor 4, the vehicle speed sensor 5, and the brake sensor 6 are converted into digital data by an A / D conversion circuit and input to a microcomputer. The microcomputer detects an object such as a person (pedestrian or person riding a bicycle) based on the input data, and the speaker 7 when the detected object satisfies a predetermined notification requirement. And a process of issuing a report to the driver by the display device 8.

  As shown in FIG. 2, the infrared cameras 2 </ b> R and 2 </ b> L are attached to the front portion (the front grill portion in the figure) of the host vehicle 10 in order to image the front of the host vehicle 10. In this case, the infrared cameras 2R and 2L are respectively disposed at a position on the right side and a position on the left side of the center of the host vehicle 10 in the vehicle width direction. These positions are symmetrical with respect to the center of the vehicle 10 in the vehicle width direction. The infrared cameras 2R and 2L are fixed so that their optical axes extend in parallel in the front-rear direction of the host vehicle 10, and the heights of the respective optical axes from the road surface are equal to each other. The infrared cameras 2R and 2L have sensitivity in the far-infrared region, and have a characteristic that the higher the temperature of an object to be imaged, the higher the level of the output video signal (the higher the luminance of the video signal). doing.

  The radar device 3 is attached to the front side of the host vehicle 10 so as to be positioned above the infrared cameras 2R and 2L. When viewed from above, the radar apparatus 3 transmits a millimeter wave (electromagnetic wave) beam BM spreading in a narrower width than the imaging area of the infrared cameras 2R and 2L to the front of the host vehicle 10 as shown in FIG. The irradiation area in the radar device 3 is divided into a first irradiation area BM1 to a seventh irradiation area BM7. The radar apparatus 3 is a scanning radar apparatus, and a beam BM having a constant intensity is transmitted while shifting its transmission angle by a certain angle so that the front area of the host vehicle 3 is scanned in the left-right direction of the host vehicle 10. . The scanning range of the beam BM in the left-right direction is set so as to include all the imaging areas of the infrared cameras 2R, 2L in the horizontal direction. The vertical width of the beam BM is set so as to include the vertical field of view of the infrared cameras 2R and 2L. The radar apparatus 3 receives this millimeter wave reflected wave, that is, the millimeter wave reflected by the object existing in front of the host vehicle 10 by a receiving antenna (not shown) arranged in the vertical direction. Further, the radar device 3 measures the intensity of the reflected wave for each irradiation area, and uses reflection intensity data representing the correlation between the reflection intensity and the distance from the vehicle 10 to the object causing the reflected wave. Output to the image processing unit 1. As the radar apparatus, a radar apparatus using an elastic vibration wave such as an ultrasonic wave as well as an electromagnetic wave such as a millimeter wave (laser light or the like) may be employed.

  In the present embodiment, the display device 8 includes a head-up display 8a (hereinafter referred to as HUD 8a) that displays image information on the front window of the host vehicle 10. In addition, as a display device 8, instead of the HUD 8a or together with the HUD 8a, a display provided integrally with a meter for displaying a traveling state such as the vehicle speed of the host vehicle 10 or a display provided in an in-vehicle navigation device is provided. It may be used.

  Next, parallax offset amount calculation processing, which is a main function of the vehicle periphery monitoring device of the present embodiment, will be described with reference to the flowchart of FIG.

  First, the distance to the object is detected by the radar device 3 (FIG. 4 / STEP 100). Specifically, the radar device 3 measures the intensity of the reflected wave for each irradiation area, and reflects the correlation between the reflected intensity and the distance from the vehicle 10 to the object that caused the reflected wave. The intensity data is output to the image processing unit 1.

  Next, there is only one target in one irradiation area where the distance is detected by the radar apparatus 3, and extraction is performed by the target extracting unit in one local image area in the captured image corresponding to the one irradiation area. The presence / absence of a set of an irradiation area and a local image area that satisfies a predetermined condition that only one target object exists is determined (FIG. 4 / STEP 102). In the first irradiation area BM1 to the seventh BM7 of the radar apparatus 3, for example, an object is detected as shown in FIG. “◯” in the first irradiation area BM1 to the seventh irradiation area BM7 indicates an object. Further, the local image areas in the captured images of the infrared cameras 2R and 2L corresponding to the irradiation areas BM1 to BM7 in the radar apparatus 3 are, for example, as shown in FIG. The first local image region LA1 to the seventh local image region LA7 obtained by dividing the captured images of the infrared cameras 2R and 2L in the left-right direction so as to correspond to the seven irradiation regions BM7. “◯” in the first local image area LA1 to the seventh local image area LA7 indicates an object. In the first irradiation area, two objects whose distances are detected are overlapped to be detected, and only one object is extracted in the first local image area LA1 corresponding to the first irradiation area BM1. The Therefore, it is determined that the set of the first irradiation region BM1 and the first local image region LA1 does not satisfy the predetermined condition. In the second irradiation area BM2, two objects whose distances are detected are detected apart from each other, and the object is separated in the second local image area LA2 corresponding to the second irradiation area BM2. Two are extracted. Therefore, it is determined that the set of the second irradiation region BM2 and the second local image region LA2 does not satisfy the predetermined condition. In the third irradiation area BM3, only one object whose distance is detected is detected, and in the third local image area LA3 corresponding to the third irradiation area BM3, only one object is detected. Extracted. Therefore, it is determined that the set of the third irradiation region BM3 and the third local image region LA3 satisfies the predetermined condition. Further, in the fourth irradiation area BM4, only one object whose distance is detected is detected, and in the fourth local image area corresponding to the fourth irradiation area BM4, only one object is extracted. . Therefore, it is determined that the set of the fourth irradiation region BM4 and the fourth local image region LA4 satisfies the predetermined condition. Further, in the fifth irradiation area BM5, only one target whose distance is detected is detected, and in the fifth local image area corresponding to the fifth irradiation area BM5, only one target is extracted. Is done. Therefore, it is determined that the set of the fifth irradiation region BM5 and the fifth local image region LA5 satisfies the predetermined condition. In the sixth irradiation area BM6, two objects whose distances are detected are detected, and in the sixth local image area LA6 corresponding to the sixth irradiation area BM6, two objects are extracted. The Therefore, it is determined that the set of the sixth irradiation region BM6 and the sixth local image region LA6 does not satisfy the predetermined condition. In addition, no object is detected in the seventh irradiation area BM7, and no object is extracted in the seventh local image area LA7 corresponding to the seventh irradiation area BM7. Therefore, it is determined that the set of the seventh irradiation region BM7 and the seventh local image region LA7 does not satisfy the predetermined condition. As described above, the irradiation area and the local image area determined to satisfy the predetermined condition are the third irradiation area and the third local image area, the fourth irradiation area and the fourth local image area, and the fifth area. These are the irradiation region and the fifth local image region.

  If there is an irradiation area and a local image area determined to satisfy the predetermined condition (FIG. 4 / STEP 102... YES), the distance to one object existing in one irradiation area is calculated using the radar. Calculation is performed based on the reflection intensity data transmitted from the apparatus 3 to the image processing unit 1 (FIG. 4 / STEP 104).

  Thereafter, the parallax of the object existing in one local image region corresponding to one irradiation region obtained from the captured images of the infrared cameras 2R and 2L is calculated (FIG. 4 / STEP 106). As shown in FIG. 6, the parallax is calculated based on the epipolar constraint that causes the horizontal direction of the object SO of the search image identical to the object BO of the reference image to match.

  Next, the parallax offset amount is calculated based on the distance from the host vehicle 10 calculated in STEP 104 to the target and the parallax of the target calculated in STEP 106 (FIG. 4 / STEP 108). Specifically, the parallax offset amount Offset is calculated by Expression (1). Z is the distance from the host vehicle 10 to the object calculated in STEP 104, Dn is the parallax of the object calculated in STEP 106, D is the baseline length that is the distance between the left and right of the infrared cameras 2R and 2L, And F is a value obtained by dividing the focal length of the lenses of the infrared cameras 2R and 2L by the pixel pitch.

  Then, the parallax offset amount is updated to a value obtained as a result of this parallax offset amount calculation process (parallax correcting unit).

  In addition, as shown in FIG. 5 and FIG. 6, when the presence or absence of the set of the irradiation area | region and local image area | region which satisfy | fills the said predetermined condition is determined about several target objects, and predetermined conditions are satisfy | filled about several sets May average a plurality of parallax offset amounts.

  According to the vehicle periphery monitoring device that exhibits the above function, based on the distance from the own vehicle 10 to the target detected by the radar device 3 and the parallax calculated from the target obtained by the infrared cameras 2R and 2L. Then, the parallax offset amount resulting from the deviation in parallelism between the optical axes of the infrared cameras 2R and 2L is calculated (see FIGS. 4 / S100 to S108, FIGS. 5 and 6). There is only one object in one irradiation area where the distance is detected by the radar apparatus 3, and one extracted object is in one local image area in the captured image corresponding to one irradiation area. If the predetermined condition is not satisfied, the object whose distance is detected by the radar device 3 in one irradiation area and the object extracted from one corresponding local image area are the same object. It is difficult to determine whether or not, and the amount of parallax offset cannot be calculated with high accuracy. On the other hand, if the predetermined condition is satisfied, there is a possibility that an object whose distance is detected by the radar device 3 in one irradiation area and an object extracted from one corresponding local image area are the same object. Is expensive. In view of this point, when the predetermined condition is satisfied, the radar apparatus 3 detects the distance to the object and calculates the parallax of the object obtained by the infrared cameras 2R and 2L. As described above, the parallax offset amount can be calculated with high accuracy.

  In addition, when there are a plurality of objects that satisfy the predetermined condition and the difference in distance from the vehicle 10 to the object in each object calculated in STEP 104 is equal to or less than a predetermined value, the parallax offsets in the plurality of objects The amount may be averaged. For example, as shown in FIG. 7, the object to be used as a reference when calculating the parallax offset amount can be the front vehicle FB obtained from the captured images of the infrared cameras 2R and 2L. The forward vehicle FB is one object that spans a plurality of local image areas LA3 to LA5. In this case, the parallax is calculated based on the epipolar constraint that causes the horizontal position of the object SO of the search image that is the same as the object BO of the reference image to match. Note that whether or not the object extracted from the captured images of the infrared cameras 2R and 2L is a vehicle is determined by, for example, the process described in FIG. 7 of Japanese Patent Application Laid-Open No. 2004-348645 by the present applicant. Can do. According to this example, when the difference in distance from each vehicle 10 to the object in the plurality of irradiation areas is equal to or less than the predetermined value, the parallax offset amounts in the plurality of objects are averaged. Therefore, even when there are variations in measurement distance errors to the object in a plurality of irradiation regions that satisfy a predetermined condition, the influence of measurement error variations is reduced, and the plurality of distances measured in each of the plurality of irradiation regions. It is possible to calculate the parallax offset amount with high accuracy for one object (specifically, the preceding vehicle) having the target object as a part.

The configuration in which the processing of STEP 102 is performed by the image processing unit 1 corresponds to the same object determination unit of the present invention, and the configuration of executing the processing in STEP 106 corresponds to the parallax calculation unit of the present invention. The configuration for executing this corresponds to the parallax offset amount calculation means of the present invention.
[Second Embodiment]
Next, a second embodiment of the vehicle periphery monitoring device of the present invention will be described. In the present embodiment, the parallax offset amount calculation process shown in FIG. 4 in the first embodiment is changed to the parallax offset amount calculation process shown in FIG.

  First, the radar device 3 determines whether or not there is a vehicle ahead of the object based on the distance to the object (FIG. 8 / STEP 200). Specifically, the difference between the distance to the object measured by the radar device 3 at the first time and the distance to the object measured by the radar device 3 at the second time is within a predetermined value set in advance. In this case, it is determined that the object whose distance is measured by the radar device 3 is a forward vehicle.

  Next, when it is determined that the object whose distance has been measured by the radar device 3 is a preceding vehicle (FIG. 8 / STEP 200... YES), the distance to the preceding vehicle is transmitted from the radar device 3 to the image processing unit 1. Calculation is based on the reflection intensity data (FIG. 8 / STEP 202).

  Then, the parallax of the front vehicle obtained from the captured images of the infrared cameras 2R and 2L is calculated (FIG. 8 / STEP 204). As shown in FIG. 7, the parallax is calculated based on epipolar restraint so that the horizontal position of the object SO of the search image identical to the object BO of the reference image matches.

  Thereafter, a parallax offset amount is calculated based on the distance from the host vehicle 10 calculated in STEP 202 to the object and the parallax of the object calculated in STEP 204 (FIG. 8 / STEP 206). Specifically, the parallax offset amount Offset is calculated by Expression (1).

  Then, the parallax offset amount is updated to a value obtained as a result of this parallax offset amount calculation process (parallax correcting unit).

  In addition, when there are a plurality of objects in which it is determined that the object whose distance is measured by the radar apparatus 3 is a forward vehicle, the parallax offset amounts of the plurality of objects may be averaged.

  According to the vehicle periphery monitoring device that exhibits the above function, when the radar device 3 determines that the target object is a forward vehicle based on the distance to the target object, from the vehicle detected by the radar device 3 to the forward vehicle. , And the parallax of the vehicle ahead of the vehicle obtained by the infrared cameras 2R and 2L, the amount of parallax offset due to the deviation in parallelism between the optical axes of the two imaging devices is calculated (FIGS. 8 / S200 to S206, (See FIG. 7). An object moving at the same speed as the host vehicle, the parallax is constant because the relative distance to the host vehicle does not change, and the parallax offset amount is calculated with high accuracy by the method as described above. Can do.

Note that the configuration in which the processing of STEP 200 is performed by the image processing unit 1 corresponds to the vehicle determination unit of the present invention, and the configuration of executing the processing in STEP 204 corresponds to the parallax calculation unit of the present invention and executes the processing of STEP 206. This configuration corresponds to the parallax offset amount calculation means of the present invention.
[Third embodiment]
Next, a third embodiment of the vehicle periphery monitoring device of the present invention will be described. In the present embodiment, the parallax offset amount calculation processing shown in FIG. 4 in the first embodiment is changed to the parallax offset amount calculation processing shown in FIG.

  First, it is determined whether or not the object obtained by the infrared cameras 2R and 2L has a forward vehicle (FIG. 9 / STEP 300). For example, the determination is made by pattern matching with a template stored in advance in a memory.

  Next, when it is determined that the object obtained by the infrared cameras 2R and 2L is a front vehicle (FIG. 9 / STEP 300... YES), the distance to the front vehicle is transmitted from the radar device 3 to the image processing unit 1. It is calculated based on the reflected intensity data (FIG. 9 / STEP 302).

  Then, the parallax of the front vehicle obtained from the captured images of the infrared cameras 2R and 2L is calculated (FIG. 9 / STEP 304). As shown in FIG. 7, the parallax is calculated based on epipolar restraint so that the horizontal position of the object SO of the search image identical to the object BO of the reference image matches.

  Thereafter, a parallax offset amount is calculated based on the distance from the host vehicle 10 calculated in STEP 302 to the object and the parallax of the object calculated in STEP 304 (FIG. 9 / STEP 306). Specifically, the parallax offset amount Offset is calculated by Expression (1).

  Then, the parallax offset amount is updated to a value obtained as a result of this parallax offset amount calculation process (parallax correcting unit).

  According to the vehicle periphery monitoring device that exhibits the above function, when it is determined that the object obtained by the infrared cameras 2R and 2L is a front vehicle, the distance from the vehicle detected by the radar device 3 to the front vehicle is Based on the parallax of the preceding vehicle obtained by the infrared cameras 2R and 2L, the parallax offset amount resulting from the deviation in parallelism between the optical axes of the infrared cameras 2R and 2L is calculated (FIGS. 9 / S300 to 306, FIG. 7). reference). An object moving at the same speed as the host vehicle, the parallax is constant because the relative distance to the host vehicle does not change, and the parallax offset amount is calculated with high accuracy by the method as described above. Can do.

  The configuration in which the processing of STEP 300 is performed by the image processing unit 1 corresponds to the vehicle determination unit of the present invention, and the configuration of executing the processing in STEP 304 corresponds to the parallax calculation unit of the present invention and executes the processing of STEP 306. This configuration corresponds to the parallax offset amount calculation means of the present invention.

The figure which shows the whole structure of one Embodiment of the periphery monitoring apparatus of the vehicle of this invention. The perspective view of the vehicle provided with the periphery monitoring apparatus of FIG. Explanatory drawing about the scanning range of the radar device The flowchart which shows the parallax offset amount calculation process in 1st Embodiment. The figure for demonstrating the process of an image processing unit The figure for demonstrating the process of an image processing unit The figure for demonstrating the process of an image processing unit The flowchart which shows the parallax offset amount calculation process in 2nd Embodiment. The flowchart which shows the parallax offset amount calculation process in 3rd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image processing unit (same object determination means, vehicle determination means, parallax offset amount calculation means, parallax correction means), 2R, 2L ... infrared camera (imaging device), 3 ... radar device

Claims (4)

Vehicle periphery monitoring for calculating the parallax of an object based on two captured images obtained by two imaging devices mounted on the vehicle, and detecting the distance from the parallax to the object to monitor the periphery of the vehicle A device,
A radar device that detects the distance to the first object by detecting the intensity of the reflected wave in each of the irradiation areas of the plurality of transmission waves that radiate from the vehicle;
Parallax calculation means for calculating parallax of the second object obtained by the two imaging devices;
The same object determining means for determining whether or not the first object and the second object are the same object;
When it is determined that the first object and the second object are the same object by the same object determination unit, the vehicle from the vehicle detected by the radar device to the same object Based on the distance and the parallax of the same object obtained by the two imaging devices calculated by the parallax calculation means, a parallax offset amount caused by a deviation in parallelism between the optical axes of the two imaging devices is calculated. Parallax offset amount calculating means for performing,
Parallax correction means for correcting parallax based on the parallax offset amount calculated by the parallax offset amount calculation means ,
The same object determination unit divides the captured image so that only one first object exists in one irradiation area where the distance is detected by the radar apparatus, and the captured image corresponds to the irradiation area. The predetermined condition that only one second object obtained by the two imaging devices is present in one local image region in the captured image corresponding to the one irradiation region is satisfied with respect to the local image region thus obtained. A vehicle periphery monitoring device that determines whether or not the first object and the second object are the same object, depending on whether or not the operation is performed . In the vehicle periphery monitoring device according to claim 1 ,
The parallax offset amount calculating means, when the predetermined condition is satisfied for the plurality of irradiation areas and the plurality of local image areas respectively corresponding to the plurality of irradiation areas by the same object determination means, Average the parallax offset amounts of the plurality of the same objects determined to be the same as the one object and the second object;
The vehicle periphery monitoring apparatus, wherein the parallax correction unit corrects parallax based on the averaged parallax offset amount. Vehicle periphery monitoring for calculating the parallax of an object based on two captured images obtained by two imaging devices mounted on the vehicle, and detecting the distance from the parallax to the object to monitor the periphery of the vehicle A device,
A radar device that detects a distance to an object by detecting an intensity of a reflected wave in each irradiation region of a plurality of transmission waves that radiate from the vehicle;
Vehicle determination means for determining whether or not the object is a forward vehicle based on a distance to the object by the radar device;
Parallax calculating means for calculating parallax of the object obtained by the two imaging devices;
When the vehicle determination means determines that the object is a front vehicle, the distance from the vehicle to the front vehicle detected by the radar device and the two imaging devices calculated by the parallax calculation means Parallax offset amount calculation means for calculating a parallax offset amount caused by a parallelism shift between the optical axes of the two imaging devices based on the obtained parallax of the preceding vehicle;
Parallax correction means for correcting parallax based on the parallax offset amount calculated by the parallax offset amount calculation means ,
The parallax offset amount calculation unit calculates the parallax offset amount based on only the front vehicle when the vehicle determination unit determines that the object is a front vehicle. . Vehicle periphery monitoring for calculating the parallax of an object based on two captured images obtained by two imaging devices mounted on the vehicle, and detecting the distance from the parallax to the object to monitor the periphery of the vehicle A device,
A radar device that detects a distance to an object by detecting an intensity of a reflected wave in each irradiation region of a plurality of transmission waves that radiate from the vehicle;
Vehicle determination means for determining whether or not the object obtained by the two imaging devices is a forward vehicle;
Parallax calculating means for calculating parallax of the object obtained by the two imaging devices;
When the vehicle determination means determines that the object is a front vehicle, the distance from the vehicle to the front vehicle detected by the radar device and the two imaging devices calculated by the parallax calculation means Parallax offset amount calculation means for calculating a parallax offset amount caused by a parallelism shift between the optical axes of the two imaging devices based on the obtained parallax of the preceding vehicle;
Parallax correction means for correcting parallax based on the parallax offset amount calculated by the parallax offset amount calculation means ,
The parallax offset amount calculation unit calculates the parallax offset amount based on only the front vehicle when the vehicle determination unit determines that the object is a front vehicle. .

Images