JP7514139B2 - Outside environment recognition device - Google Patents

Description

The present invention relates to an external environment recognition device that identifies specific objects that exist in the traveling direction of the vehicle.

Conventionally, there is known a technology, such as that described in Patent Document 1, that detects a preceding vehicle located in front of the vehicle and performs tracking control to reduce damage caused by a collision with the preceding vehicle and to maintain a safe distance between the vehicle and the preceding vehicle.

Japanese Patent No. 3349060

In order to reduce the damage caused by a collision between the vehicle and the vehicle ahead, or to achieve tracking control to make the vehicle follow the vehicle ahead, it is first necessary to determine whether a three-dimensional object located around the vehicle is a specific object such as a vehicle. For example, a three-dimensional object is determined to be a leading vehicle by recognizing the shape of the object in a brightness image captured by an imaging unit, or the relative distance and speed of the object in a distance image.

However, when a three-dimensional object is located far away from the vehicle, the area that the object itself occupies in the image becomes smaller, causing fluctuations in the shape in the brightness image and the relative distance in the distance image, resulting in a decrease in the accuracy of identifying specific objects.

In view of these problems, the present invention aims to provide an exterior environment recognition device that can improve the accuracy of determining specific objects.

To solve the above problem, the vehicle exterior environment recognition device of the present invention includes a first width derivation unit that derives a first width of a first three-dimensional object based on a luminance image that can identify the luminance of an imaged object, a second width derivation unit that derives a second width of a second three-dimensional object based on a distance image that can identify the distance of the imaged object, an overlap degree derivation unit that determines an overlap width where the first width and the second width overlap in the horizontal direction and determines the overlap degree as the larger of the overlap width/first width and the overlap width/second width, and a specific object determination unit that determines that the first three-dimensional object and the second three-dimensional object are the same specific object if the overlap degree is equal to or greater than a predetermined threshold value.

The specific object determination unit may lower the threshold for the degree of overlap when determining that the specific objects are the same.

The specific object determination unit may lower the threshold for the degree of overlap, and then raise the threshold for the degree of overlap that is determined not to be the same specific object.

The specific object determination unit may gradually change the threshold for the degree of overlap depending on the number of times the specific object is determined to be the same.

The specific object determination unit may change the threshold value depending on whether the wipers are operating.

The specific object determination unit may change the threshold value depending on the speed of the vehicle.

The present invention makes it possible to improve the accuracy of determining specific objects.

FIG. 1 is a block diagram showing the connections of the exterior environment recognition system. FIG. 2 is a functional block diagram showing a schematic function of the vehicle exterior environment recognition device. FIG. 3 is a flowchart showing the flow of the vehicle exterior environment recognition method. FIG. 4 is an explanatory diagram for explaining a luminance image and a distance image. FIG. 5 is a diagram illustrating the overlapping degree. FIG. 6 is a graph for explaining how to obtain a threshold value referred to by the specific object determination unit. FIG. 7 is an explanatory diagram for explaining a change in the threshold value. FIG. 8 is an explanatory diagram for explaining a change in the threshold value.

The preferred embodiment of the present invention will be described in detail below with reference to the attached drawings. The dimensions, materials, and other specific values shown in the embodiment are merely examples to facilitate understanding of the invention, and do not limit the present invention unless otherwise specified. In this specification and drawings, elements having substantially the same functions and configurations are given the same reference numerals to avoid duplicated explanations, and elements not directly related to the present invention are not illustrated.

(Exterior environment recognition system 100)
1 is a block diagram showing connections in a vehicle exterior environment recognition system 100. The vehicle exterior environment recognition system 100 includes an imaging device 110, an exterior environment recognition device 120, and a vehicle control device 130.

The imaging device 110 is configured to include imaging elements such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and can capture an image of the external environment in front of the vehicle 1 and generate a luminance image (color image or monochrome image) that contains at least luminance information. The imaging devices 110 are arranged in the direction of travel of the vehicle 1, spaced apart in the approximately horizontal direction so that the optical axes of the two imaging devices 110 are approximately parallel. The imaging device 110 continuously generates luminance images of three-dimensional objects present in the detection area in front of the vehicle 1, for example, at every 1/60th of a second frame (60 fps).

In addition, the vehicle exterior environment recognition device 120 recognizes the environment outside the vehicle through the brightness image acquired from the imaging device 110 and a distance image based on the brightness image 2. The vehicle exterior environment recognition device 120 then performs speed control and steering angle control during the traveling of the vehicle 1 based on the recognized vehicle exterior environment and the traveling conditions of the vehicle 1. The vehicle exterior environment recognition device 120 will be described in detail later.

The vehicle control device 130 is composed of an ECU (Electronic Control Unit) and receives operational inputs from the driver via the steering wheel 132, accelerator pedal 134, and brake pedal 136, and controls the host vehicle 1 by referring to a speed sensor 138 mounted on the axle and transmitting the inputs to the steering mechanism 142, drive mechanism 144, and braking mechanism 146. The vehicle control device 130 also controls the steering mechanism 142, drive mechanism 144, and braking mechanism 146 according to instructions from the exterior environment recognition device 120. When driving in the rain, the vehicle control device 130 also operates wipers 148 that wipe away dirt and impurities from the front and rear windows of the host vehicle 1 in response to the driver's operation.

(Exterior environment recognition device 120)
2 is a functional block diagram showing the schematic functions of the vehicle exterior environment recognition device 120. As shown in FIG. 2, the vehicle exterior environment recognition device 120 includes an I/F unit 150, a data storage unit 152, and a central control unit 154.

The I/F unit 150 is an interface for two-way information exchange with the imaging device 110 and the vehicle control device 130. The data storage unit 152 is composed of RAM, flash memory, HDD, etc., and stores various information required for the processing of each functional unit shown below.

The central control unit 154 is composed of a semiconductor integrated circuit including a central processing unit (CPU), a ROM in which programs and the like are stored, a RAM as a work area, and the like, and controls the I/F unit 150, the data storage unit 152, and the like through a system bus 156. In this embodiment, the central control unit 154 also functions as an image acquisition unit 160, a distance image generation unit 162, a three-dimensional object identification unit 163, a first width derivation unit 164, a second width derivation unit 166, an overlap derivation unit 168, and a specific object determination unit 170. Below, the external environment recognition method characteristic of this embodiment, which extracts a three-dimensional object in front of the vehicle 1 and determines a specific object such as a preceding vehicle, will be described in detail, taking into account the operation of each functional unit of the central control unit 154.

(Method of recognizing the external environment)
FIG. 3 is a flowchart showing the flow of the vehicle exterior environment recognition method. The vehicle exterior environment recognition device 120 executes the vehicle exterior environment recognition method at each predetermined interruption time. In the vehicle exterior environment recognition method, first, the image acquisition unit 160 acquires a plurality of luminance images (S200). The distance image generation unit 162 generates a distance image (S202). The three-dimensional object identification unit 163 identifies a three-dimensional object (S203). The first width derivation unit 164 derives a first width of the first three-dimensional object based on a luminance image capable of identifying the luminance of the imaging target (S204). The second width derivation unit 166 derives a second width of the second three-dimensional object based on a distance image capable of identifying the distance of the imaging target (S206). The overlap degree derivation unit 168 calculates an overlap width, which is the width of an area where the first width and the second width overlap in the horizontal direction of the image. Then, the overlapping degree derivation unit 168 determines the overlapping degree to be the larger of the overlapping width/first width and the overlapping width/second width (S208). If the overlapping degree is equal to or greater than a predetermined threshold, the specific object determination unit 170 determines that the first three-dimensional object and the second three-dimensional object are the same specific object (S210). Each process of the vehicle exterior environment recognition method will be described in detail below, and a description of processes unrelated to the features of this embodiment will be omitted. In this embodiment, "/" means division, and for example, overlapping width/first width indicates that the overlapping width is divided by the first width.

(Image acquisition process S200)
4 (FIGS. 4A to 4C) are explanatory diagrams for explaining a luminance image and a distance image. The image acquisition unit 160 acquires a plurality of (here, two) luminance images captured by the imaging device 110 with different optical axes. Here, it is assumed that the image acquisition unit 160 acquires, as the luminance image 180, a first luminance image 180a captured by the imaging device 110 located relatively to the right of the vehicle 1 as shown in FIG. 4A, and a second luminance image 180b captured by the imaging device 110 located relatively to the left of the vehicle 1 as shown in FIG. 4B.

With reference to FIG. 4, it can be seen that the image positions of the three-dimensional objects included in the first luminance image 180a and the second luminance image 180b differ in the horizontal direction due to differences in the imaging position of the imaging device 110. Here, horizontal refers to the horizontal direction of the screen of the captured image, and vertical refers to the vertical direction of the screen of the captured image.

(Distance image generation process S202)
The distance image generation unit 162 generates a distance image 182, as shown in FIG. 4C, capable of identifying the distance to the imaging subject, based on the first luminance image 180a shown in FIG. 4A and the second luminance image 180b shown in FIG. 4B, acquired by the image acquisition unit 160.

Specifically, the distance image generating unit 162 derives disparity information including disparity and an image position indicating the position of an arbitrary block in an image by using so-called pattern matching. Specifically, a block corresponding to a block arbitrarily extracted from one luminance image (here, first luminance image 180a) is searched for in the other luminance image (here, second luminance image 180b). Here, the block is represented, for example, by an array of 4 pixels horizontally by 4 pixels vertically. Also, pattern matching is a technique for searching for a block corresponding to a block arbitrarily extracted from one luminance image in the other luminance image.

For example, functions for evaluating the degree of agreement between blocks in pattern matching include SAD (Sum of Absolute Difference), which takes the difference in luminance, SSD (Sum of Squared Intensity Difference), which uses the squared difference, and NCC (Normalized Cross Correlation), which takes the similarity of the variance value obtained by subtracting the average value from the luminance of each pixel.

The distance image generating unit 162 performs this type of block-based parallax derivation processing for all blocks displayed in a detection area of, for example, 600 pixels x 200 pixels. Here, the blocks are 4 pixels x 4 pixels, but the number of pixels in a block can be set arbitrarily.

However, although the distance image generating unit 162 can derive the parallax for each block, which is the detection resolution unit, it cannot recognize what kind of three-dimensional object the block is a part of. Therefore, the parallax information is derived independently for each detection resolution unit in the detection area, for example, for each block, rather than for each three-dimensional object. Here, for ease of explanation, the blocks from which the parallax is derived are represented by black dots.

The distance image generating unit 162 converts the parallax information for each block in the distance image 182 into three-dimensional position information using a so-called stereo method, and derives the relative distance. Here, the stereo method is a method of deriving the relative distance of a block to the imaging device 110 from the parallax of that block by using a triangulation method.

The vehicle exterior environment recognition device 120 recognizes the vehicle exterior environment through the brightness image 180 and distance image 182 derived in this manner, and for example, determines that a three-dimensional object ahead of the vehicle 1 is a specific object such as a leading vehicle. However, when the three-dimensional object 184 in FIG. 4A and the three-dimensional object 186 in FIG. 4C are located far from the vehicle 1, the area occupied by the three-dimensional object itself in the image is small, and the shape in the brightness image 180 and the relative distance in the distance image 182 fluctuate, resulting in a decrease in the accuracy of determining the specific object. Therefore, the degree of overlap between the three-dimensional objects identified in the brightness image 180 and the distance image 182 is derived, and it is determined whether the two three-dimensional objects are the same specific object according to the degree of overlap.

(Three-dimensional object identification process S203)
The three-dimensional object identification unit 163 first identifies the road surface in front of the vehicle 1. Then, the three-dimensional object identification unit 163 identifies a three-dimensional object having a height vertically above the identified road surface. Specifically, the three-dimensional object identification unit 163 identifies blocks located at a height from the road surface that is a predetermined distance (e.g., 0.3 m) or more as candidates for a three-dimensional object protruding in the height direction from the road surface. Among the multiple blocks that are candidates for a three-dimensional object having a height vertically above the road surface, the three-dimensional object identification unit 163 groups blocks that are at the same relative distance from the vehicle 1, and identifies them as three-dimensional objects.

(First width derivation process S204)
The first width derivation unit 164 derives a first width, which is the horizontal width of a predetermined three-dimensional object 184 (first three-dimensional object) in one of the two luminance images 180, for example, the first luminance image 180a. The first width is expressed, for example, by the number of pixels or a converted value of distance.

Note that machine learning technology is used to extract three-dimensional objects from the brightness image 180. For example, the edge pattern and temporal changes of the brightness image 180 are input, and a three-dimensional object that can be considered to be integrally formed is output. Since various existing technologies can be applied to such machine learning technology, a detailed description thereof will be omitted here.

(Second width derivation process S206)
The second width derivation unit 166 derives a second width, which is the horizontal width of a predetermined three-dimensional object 186 (second three-dimensional object) in the distance image 182. The second width is expressed, for example, by the number of pixels or a converted value of distance.

Here, both the first width derivation unit 164 and the second width derivation unit 166 derive the horizontal width of the three-dimensional objects 184, 186. This is for the following reason. The vehicle 1 may sway in the pitch and roll directions due to unevenness in the road. On the other hand, the yaw direction does not sway as much as the pitch and roll directions. Therefore, by targeting the horizontal width corresponding to the stable yaw direction, it is possible to improve the accuracy of determining a specific object. It goes without saying that on a flat road, the vertical width corresponding to the pitch direction can be targeted instead of or in addition to the horizontal width corresponding to the yaw direction.

(Overlap derivation process S208)
5 (FIGS. 5A to 5F) are diagrams for explaining the overlap degree. The overlap degree derivation unit 168 first obtains the first width 190 derived by the first width derivation unit 164 and the second width 192 derived by the second width derivation unit 166, and derives an overlap width 194, which is the width of the area where the first width and the second width overlap in the horizontal direction of the image.

In Figures 5A to 5C, the first width 190 of the three-dimensional object 184 is longer than the second width 192 of the three-dimensional object 186. In Figure 5A, the three-dimensional object 186 is included in the three-dimensional object 184 in the horizontal direction. Therefore, the overlap width 194 is equal to the second width 192. In Figure 5B, the three-dimensional object 184 and the three-dimensional object 186 each partially overlap in the horizontal direction. Therefore, the overlap width 194 is shorter than the first width 190 and the second width 192. In Figure 5C, the three-dimensional object 184 and the three-dimensional object 186 do not overlap in the horizontal direction. Therefore, the overlap width 194 is 0.

In Figures 5D to 5F, the first width 190 of the three-dimensional object 184 is shorter than the second width 192 of the three-dimensional object 186. In Figure 5D, the three-dimensional object 184 is included in the three-dimensional object 186 in the horizontal direction. Therefore, the overlap width 194 is equal to the first width 190. In Figure 5E, the three-dimensional object 184 and the three-dimensional object 186 each partially overlap in the horizontal direction. Therefore, the overlap width 194 is shorter than the first width 190 and the second width 192. In Figure 5F, the three-dimensional object 184 and the three-dimensional object 186 do not overlap in the horizontal direction. Therefore, the overlap width 194 is 0.

Next, when the overlap width indicates a value other than 0, i.e., when it is determined that there is at least an overlap, the overlap derivation unit 168 derives the overlap width 194/first width 190 and the overlap width 194/second width 192, and determines the overlap width to be the larger of the overlap width 194/first width 190 and the overlap width 194/second width 192.

In the example of FIG. 5A, overlap width 194/first width 190<overlap width 194/second width 192, so the overlap degree is overlap width 194/second width 192 (=1). In the example of FIG. 5B, overlap width 194/first width 190<overlap width 194/second width 192, so the overlap degree is overlap width 194/second width 192. In the example of FIG. 5C, overlap width 194 is 0, meaning there is no overlap, so the overlap degree is also 0.

In the example of FIG. 5D, overlap width 194/first width 190>overlap width 194/second width 192, so the overlap degree is overlap width 194/first width 190 (=1). In the example of FIG. 5E, overlap width 194/first width 190>overlap width 194/second width 192, so the overlap degree is overlap width 194/first width 190. In the example of FIG. 5F, overlap width 194 is 0, meaning there is no overlap, so the overlap degree is also 0.

(Specific object determination process S210)
The specific object determination unit 170 refers to a threshold value to determine whether the three-dimensional object 184 and the three-dimensional object 186 are the same specific object.

Figure 6 is a graph to explain how to determine the threshold value referenced by the specific object determination unit 170. The horizontal axis of the graph in Figure 6 indicates the relative distance, and the vertical axis indicates the degree of overlap. Here, the degree of overlap is expressed in the range from 0 to 1. Such an overlap graph is prepared in advance.

For example, the vehicle 1 travels in advance in a specified area, and among the multiple three-dimensional objects extracted during that time, only the specific object corresponding to the vehicle is identified, and the actual measured values of the relative distance and overlap are plotted. In this way, the point cloud of FIG. 6 can be obtained. It can be seen that the longer the relative distance, the smaller the overlap value. In FIG. 6, a straight line that is equal to or smaller than the lower limit value of the point cloud at all relative distances is generated. This straight line becomes the threshold 196. Therefore, the threshold 196 can be expressed as a linear function of the relative distance. Note that, as long as the threshold 196 is equal to or smaller than the lower limit value of the point cloud at all relative distances, it may be expressed as a curve, that is, a multi-order function, rather than as a straight line. Note that, here, an example in which the vehicle 1 itself plots the actual measured values of the relative distance and overlap is given, but this is not limited to such a case, and the actual measured values of the relative distance and overlap plotted by another vehicle may be reflected on the vehicle 1.

The specific object determination unit 170 refers to the threshold value 196 thus prepared, and if the overlap between the three-dimensional object 184 and the three-dimensional object 186 at the relative distance at which the three-dimensional object 184 and the three-dimensional object 186 are located, derived by the overlap derivation unit 168, is equal to or greater than the threshold value 196, it determines that the three-dimensional object 184 and the three-dimensional object 186 are the same specific object.

Here, a high value is used as the threshold value 196 in order to strictly determine that the three-dimensional object 184 and the three-dimensional object 186 are the same specified object. However, it is not necessary to set the threshold value 196 high until it has been determined that the three-dimensional object 184 and the three-dimensional object 186 are the same specified object. Therefore, a hysteresis function is provided so that once it has been determined that the three-dimensional object is the same specified object, it can continue to be determined that the three-dimensional object is the same specified object even if the degree of overlap decreases slightly.

7 and 8 are explanatory diagrams for explaining the change in threshold value 196. Specifically, when specific object determination unit 170 determines that three-dimensional object 184 and three-dimensional object 186 are the same specific object, threshold value 196 for comparing the overlapping degree between three-dimensional object 184 and three-dimensional object 186 is lowered, for example, from threshold value 196a shown by the dashed line in FIG. 7 to threshold value 196b shown by the dashed line.

Thus, even if the overlap between three-dimensional object 184 and three-dimensional object 186, which have been determined to be the same specified object, fluctuates near threshold value 196a, it will be at least equal to or greater than threshold value 196b, so it will be easier to continue to determine them as the same specified object thereafter. Therefore, it is possible to stably determine a specified object.

The specific object determination unit 170 counts the number of consecutive times that it has determined that the specific object is the same. Then, the specific object determination unit 170 may gradually lower the threshold value 196 for the degree of overlap according to the number of times that it has determined that the specific object is the same, through multiple threshold values that have been prepared in advance, such as threshold value 196a shown by the dashed line in FIG. 7 → threshold value 196b shown by the dashed line → threshold value 196c shown by the solid line.

Specifically, when the specified object is determined to be the same a predetermined number of times (e.g., 10 times) based on threshold value 196a, the specified object determination unit 170 lowers the threshold value 196 for the overlap between three-dimensional object 184 and three-dimensional object 186 determined to be the same specified object from threshold value 196a in FIG. 7 to threshold value 196b, for example. Furthermore, when the specified object is determined to be the same specified object a predetermined number of times (e.g., 10 times) based on threshold value 196b, the specified object determination unit 170 lowers the threshold value 196 for the overlap between three-dimensional object 184 and three-dimensional object 186 determined to be the same specified object from threshold value 196b in FIG. 7 to threshold value 196c, for example.

In this way, when three-dimensional object 184 and three-dimensional object 186 are determined to be the same specified object, threshold value 196 is gradually lowered, making it easier for them to continue to be determined to be the same specified object thereafter. Therefore, it becomes possible to more stably determine the specified object.

Furthermore, even if threshold value 196 has been lowered, if the degree of overlap falls below threshold value 196 and it is determined that three-dimensional object 184 and three-dimensional object 186 are not the same specified object, then threshold value 196 should no longer be maintained. Therefore, even when it is no longer determined that they are the same specified object, a hysteresis function is provided, so that once it is determined that they are not the same specified object, they will not be determined to be the same specified object unless the degree of overlap increases.

Specifically, when the threshold value 196 is threshold value 196c shown by a solid line in FIG. 8, if the specific object determination unit 170 determines that the three-dimensional object 184 and the three-dimensional object 186 are not the same specific object, the threshold value 196 for comparing the overlapping degree between the three-dimensional object 184 and the three-dimensional object 186 is increased, for example, from threshold value 196c to threshold value 196b shown by a dashed line in FIG. 8.

In this way, even if the overlap between three-dimensional object 184 and three-dimensional object 186, which have been determined not to be the same specific object, exceeds threshold value 196c, they are unlikely to be determined to be the same specific object until they exceed threshold value 196b. Therefore, three-dimensional objects that are not specific objects can be stably eliminated.

The specific object determination unit 170 may increase the threshold value 196 for the degree of overlap in stages according to the number of times it has determined that the specific objects are not the same, such as from threshold value 196c shown by the solid line in FIG. 8 to threshold value 196b shown by the dashed line to threshold value 196a shown by the dotted line.

When the three-dimensional object 184 and the three-dimensional object 186 that have been determined to be the same specified object are determined not to be the same specified object a predetermined number of times (e.g., 10 times) by the threshold value 196c, the specified object determination unit 170 increases the threshold value 196 for the overlap between the three-dimensional object 184 and the three-dimensional object 186 that have been determined to be the same specified object, for example, from the threshold value 196c in FIG. 8 to the threshold value 196b. Furthermore, when the three-dimensional object 184 and the three-dimensional object 186 that have been determined to be the same specified object are determined not to be the same specified object a predetermined number of times (e.g., 10 times) by the threshold value 196b, the specified object determination unit 170 increases the threshold value 196 for the overlap between the three-dimensional object 184 and the three-dimensional object 186 that have been determined to be the same specified object, for example, from the threshold value 196b in FIG. 8 to the threshold value 196a.

In this way, when three-dimensional object 184 and three-dimensional object 186, which have been determined to be the same specific object, are determined to not be the same specific object, threshold value 196 is raised in stages, and it becomes more difficult to determine them as the same specific object thereafter. Therefore, three-dimensional objects that are not specific objects can be more stably eliminated.

When driving in the rain, raindrops may adhere to the front of the imaging device 110, for example, on the windshield or lens, causing the brightness image 180 and the distance image 182 to become blurred. In that case, the overlap between the three-dimensional object 184 and the three-dimensional object 186 may become smaller than the original value. In that case, the specific object determination unit 170 may determine that the three-dimensional object 184 and the three-dimensional object 186, which should be determined to be the same specific object, are not the same specific object. Therefore, when raindrops are expected to adhere, for example, when the wiper 148 is operating, the specific object determination unit 170 may lower the threshold value 196 compared to when the wiper 148 is not operating. With this configuration, it becomes possible to appropriately determine a specific object even when driving in the rain.

In addition, when the speed of the host vehicle 1 is high, a preceding vehicle must be more appropriately determined as a specific object located far away, but when the speed of the host vehicle 1 is low, the determination does not need to be made as reliably as when the speed of the host vehicle 1 is high. Therefore, the specific object determination unit 170 changes the threshold value 196 according to the speed of the host vehicle 1. For example, when the speed of the host vehicle 1 detected by the speed sensor 138 is high, the threshold value 196 is increased compared to when the speed is low, and when the speed of the host vehicle 1 is low, the threshold value 196 is decreased compared to when the speed is high. Here, the threshold value 196 may be changed linearly or stepwise in proportion to the speed of the host vehicle 1. In this way, it becomes possible to appropriately determine a specific object regardless of the speed of the host vehicle 1.

Then, the specific object determination unit 170 identifies which of the identical specific objects has an overlapping degree equal to or greater than a predetermined threshold. For example, the specific object determination unit 170 first groups, among a plurality of blocks in the distance image 182 that are located at a height from the road surface equal to or greater than a predetermined distance, blocks that are at the same relative distance from the vehicle 1 and are close to each other in the vertical and horizontal directions, and sets them as candidates for the preceding vehicle, for example. Next, the specific object determination unit 170 identifies a rectangular area in the first luminance image 180a that includes all the three-dimensional objects thus identified as a three-dimensional object area. Here, the rectangle is composed of two straight lines that extend vertically and touch the left and right edges of the three-dimensional object, respectively, and two straight lines that extend horizontally and touch the top and bottom edges of the three-dimensional object, respectively.

The specific object determination unit 170 identifies specific objects contained in the solid object area thus formed as leading vehicles based on various conditions that indicate vehicle-likeness. When a solid object area is thus identified as a leading vehicle, the exterior environment recognition device 120 controls the vehicle to reduce damage caused by a collision with the leading vehicle and to maintain a safe distance from the leading vehicle.

The specific object determination unit 170 also stores the positions and velocities of the three-dimensional objects 184 and 186 determined to be the same specific object in the data storage unit 152, and tracks the three-dimensional objects 184 and 186 based on their positions and velocities. In this way, the specific object determination unit 170 is able to identify the three-dimensional objects 184 and 186 determined to be the same specific object in the previous frame.

In this embodiment, the accuracy of identifying specific objects can be improved by using a brightness image and a distance image to determine whether or not the specific objects are the same depending on the degree of overlap of the three-dimensional objects in each image.

Also provided are a program that causes a computer to function as the vehicle exterior environment recognition device 120, and a storage medium, such as a computer-readable flexible disk, optical magnetic disk, ROM, CD, DVD, or BD, on which the program is recorded. Here, a program refers to a data processing means written in any language or description method.

Although the preferred embodiment of the present invention has been described above with reference to the attached drawings, it goes without saying that the present invention is not limited to such an embodiment. It is clear that a person skilled in the art can come up with various modified or revised examples within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present invention.

For example, in the above embodiment, an example was given of determining a preceding vehicle as a specific object located particularly far ahead of the vehicle 1, but it goes without saying that the present invention is not limited to such a case and can also be applied to specific objects located in the vicinity.

In the above-described embodiment, an example was given in which two luminance images 180 captured by two imaging devices 110 are used, and one of the images, a first luminance image 180a, and a distance image 182 based on both images are used. However, the luminance image 180 and the distance image 182 may be acquired independently of each other. For example, the luminance image 180 may be derived from one imaging device 110 of the three imaging devices 110, and the distance image 182 may be derived from the second imaging device 110.

Note that each step of the vehicle exterior environment recognition method in this specification does not necessarily need to be processed in chronological order according to the order described in the flowchart, and may include parallel or subroutine processing.

The present invention can be used in an exterior environment recognition device that identifies specific objects that exist in the direction of travel of the vehicle.

110 Imaging device 120 Vehicle exterior environment recognition device 130 Vehicle control device 164 First width derivation unit 166 Second width derivation unit 168 Overlap degree derivation unit 170 Specific object determination unit 180 Brightness image 182 Distance image 190 First width 192 Second width 194 Overlap width 196 Threshold

Claims (6)

a first width derivation unit that derives a first width of the first three-dimensional object based on a luminance image that can identify the luminance of an imaging target;
a second width derivation unit that derives a second width of the second three-dimensional object based on a distance image that can specify a distance to an imaging target;
an overlapping degree derivation unit that determines an overlapping width where the first width and the second width overlap in a horizontal direction, and sets the larger of the overlapping width/the first width and the overlapping width/the second width as an overlapping degree;
a specific object determination unit that determines that the first three-dimensional object and the second three-dimensional object are the same specific object if the overlap degree is equal to or greater than a predetermined threshold value;
An exterior environment recognition device comprising: The vehicle exterior environment recognition device according to claim 1, wherein the specific object determination unit lowers the threshold value for the degree of overlap determined to be the same specific object. The vehicle exterior environment recognition device according to claim 2, wherein the specific object determination unit lowers the threshold for the degree of overlap, and then raises the threshold for the degree of overlap that is determined not to be the same specific object. The vehicle exterior environment recognition device according to claim 2 or 3, wherein the specific object determination unit gradually changes the threshold value for the degree of overlap depending on the number of times the specific object is determined to be the same. The vehicle exterior environment recognition device according to any one of claims 1 to 4, wherein the specific object determination unit changes the threshold value depending on whether the wipers are operating. The vehicle exterior environment recognition device according to any one of claims 1 to 5, wherein the specific object determination unit changes the threshold value according to the speed of the vehicle.

Images