JP2010256040A - Vehicle detection device - Google Patents

Abstract

An object of the present invention is to provide a vehicle detection device that detects vehicle information from an image with high accuracy.
A vehicle detection apparatus 1 for detecting vehicle information from an image captured by an imaging unit 4, a surface candidate estimation unit 11 for estimating a vehicle surface candidate, and a predetermined feature from the image of the vehicle. Feature point extracting means 12 for extracting a plurality of feature points having a quantity, three-dimensional position estimating means 13 for estimating the three-dimensional position of the extracted feature points, and surface candidates estimated from images taken at different times, respectively. For each of the combinations, a search is made for a feature point correspondence between the feature points that reduces the difference in feature quantity, and a three-dimensional position is translated between the feature points of the searched correspondence. Vehicle information detection means 14 is provided for obtaining a position error after rotational conversion, extracting a candidate surface of the combination that minimizes the position error, and detecting a surface of the vehicle based on the extracted candidate surface of the combination. And wherein the Rukoto.
[Selection] Figure 1

Description

  The present invention relates to a vehicle detection device.

  In recent years, driving support devices such as collision prevention devices and inter-vehicle control devices have been developed. In these driving assistance devices, it is important to detect a vehicle traveling in front of the host vehicle. Some of such detection devices detect vehicle information based on an image captured by a camera mounted on the host vehicle. In the apparatus described in Patent Document 1, a vertical edge is detected from an image, and a side surface and a rear surface of the vehicle are determined based on the number of vertical edges.

Japanese Patent Laying-Open No. 2005-50089

  When the vehicle to be detected is directed obliquely with respect to the own vehicle, such as when the vehicle is changing from the adjacent lane to the own lane, the side and rear surfaces of the vehicle are simultaneously imaged by the camera. In this case, since only the side surface and the rear surface can be discriminated in the above-described apparatus, it is not possible to cope with an image in which two surfaces are captured, and vehicle information (vehicle end point indicating the boundary between the two surfaces, two surfaces) Cannot be detected).

  Accordingly, an object of the present invention is to provide a vehicle detection device that detects vehicle information from an image with high accuracy.

  A vehicle detection apparatus according to the present invention is a vehicle detection apparatus that detects vehicle information from an image captured by an imaging unit, and that captures a vehicle using a surface candidate estimation unit that estimates a candidate for a surface of the vehicle, and an imaging unit. Feature point extraction means for extracting a plurality of feature points having a predetermined feature amount from an image, three-dimensional position estimation means for estimating the three-dimensional position of the feature points extracted by the feature point extraction means, and imaging means at different times For each combination of surface candidates estimated from each captured image, a search is made for a correspondence relationship between feature points that reduces the difference in feature amount between a plurality of feature points, and a plurality of the corresponding correspondence relationships searched for. A position error is obtained after translational and rotational conversion of a three-dimensional position between feature points, a combination surface candidate that minimizes the position error is extracted, and a vehicle surface is determined based on the extracted combination surface candidate. Characterized in that it comprises a vehicle information detecting means for detecting.

  In this vehicle detection apparatus, a candidate for a surface of the vehicle is estimated by a surface candidate estimation unit. In the vehicle detection device, each time an image is picked up by the image pickup means, a plurality of feature points having a predetermined feature amount (for example, edge, wavelet, SIFT) are extracted from the image by the feature point extraction means, and the three-dimensional position estimation means is used. The three-dimensional position of each feature point is estimated. In the vehicle detection device, the vehicle information detection unit is characterized by a plurality of feature points in the surface candidates between the surface candidates at different times for each combination of surface candidates estimated from the images captured at different times. Corresponding relationships between feature points with a small amount difference are searched, and a position error is obtained by translating and rotating a three-dimensional position between a plurality of feature points of the corresponding relationship. Since changes in the behavior of the vehicle are taken into account by translation and rotation transformation of the three-dimensional positions of a plurality of feature points, in the case of a surface candidate that correctly estimates the surface of the vehicle among the surface candidates, at different times The position errors of the corresponding feature points between the surface candidates are reduced. Therefore, in the vehicle detection device, the vehicle information detection means compares the position error between the surface candidates of each combination, extracts the surface candidate of the combination that minimizes the position error, and the surface of the combination that minimizes the position error. A vehicle surface is detected based on the candidates. Thus, in the vehicle detection device, vehicle information (vehicle surface) can be detected with high accuracy by using position errors of a plurality of feature points between surface candidates at different times.

  The surface of the vehicle (surface candidate) is a surface of the entire vehicle shown in the image, and may be only one surface (front surface, rear surface, side surface) of the vehicle, or may be a plurality of surfaces of the vehicle (for example, the front surface). + Side surface, rear surface + side surface). The translation and rotation conversion of the three-dimensional position includes not only the case of converting the three-dimensional position and the rotation conversion, but also the case of performing only the translation conversion and the case of performing only the rotation conversion.

  In the vehicle detection device of the present invention, the vehicle information detection means may be configured to detect an end point of the vehicle based on the detected surface.

  When the detected surface includes two surfaces of the vehicle, the end point of the vehicle (boundary position of the two surfaces) can be extracted from the intersection of the two surfaces in the detected surface. Therefore, in this vehicle detection device, the end point of the vehicle can be detected with high accuracy based on the surface detected with high accuracy.

  In the vehicle detection device of the present invention, the vehicle information detection means is configured to detect the vehicle based on a region where there is a feature point whose position error of the three-dimensional position between corresponding feature points is within a threshold value between the extracted combination surface candidates. It is good also as a structure which detects the width | variety of a surface.

  Among the plurality of feature points in each surface candidate (the detected vehicle surface) of the extracted combination, there may be a case where feature points other than the vehicle are included. When the corresponding feature points between different times are vehicle feature points, the three-dimensional positions of the feature points are close to each other even between different times. However, when corresponding feature points between different times are feature points of an object other than a vehicle, the three-dimensional positions of the feature points between different times are in a distant positional relationship. Therefore, when the position error of the corresponding feature point between the extracted combination surface candidates is large, it can be regarded as a feature point of an object other than the vehicle, and when the position error is small, the feature point of the vehicle Can be considered. Therefore, in this vehicle detection device, based on the region where there is a feature point whose position error of the three-dimensional position of the corresponding feature point is within a threshold value between the extracted candidate surfaces of the combination (that is, between different times), The width of the surface can be detected with high accuracy.

  According to the present invention, vehicle information (vehicle surface) can be detected with high accuracy by using position errors of a plurality of feature points between surface candidates at different times.

1 is a configuration diagram of a vehicle detection device according to a first embodiment. It is an example of the reflective point by a laser radar. It is an example of the division | segmentation position of a reflective point group. It is an example of a shape model, (a) is model 1, and (b) is model 2. A sobel filter, (a) is for detecting a horizontal edge, and (b) is for detecting a vertical edge. It is an example of the edge image with respect to the image by a camera. It is a figure which shows the relationship between the position on the image of a feature point, and a three-dimensional position, (a) is a case where a vehicle is perspectively viewed, (b) is a case where it is a top view. It is explanatory drawing of the time point correlation of a feature point, (a) is a comparative example between time, (b) is the comparative example 1 with a large positional error, (c) is the comparative example 2 with a small positional error. It is. It is a flowchart which shows the flow of the main process in a vehicle position and surface candidate estimation part. It is a flowchart which shows the flow of the grouping process in the main process in a vehicle position and surface candidate estimation part. It is a flowchart which shows the flow of the shape model estimation process in the main process in a vehicle position and surface candidate estimation part. It is a flowchart which shows the flow of the on-image range estimation process (front surface / rear surface) in the main process in a vehicle position and surface candidate estimation part. It is a flowchart which shows the flow of the on-image range estimation process (side surface) in the main process in a vehicle position and surface candidate estimation part. It is a flowchart which shows the flow of a process in a feature point three-dimensional position candidate estimation part. It is a flowchart (first half part) which shows the flow of the process in a vehicle end point extraction part. It is a flowchart (latter half part) which shows the flow of the process in a vehicle end point extraction part. It is a block diagram of the vehicle detection apparatus which concerns on 2nd Embodiment. It is an example of the position error of the feature point between 2 time, (a) is a figure which shows the relationship between the image position of the feature point at the previous time t1, and a three-dimensional position, (b) is the present time t2. It is a figure which shows the relationship between the position on the image of the feature point in FIG., And a three-dimensional position, (c) is a figure which shows the position error of the three-dimensional position of the feature point between the previous time t1 and the present time t2. It is explanatory drawing of the process in a vehicle width estimation part. It is a flowchart which shows the flow of the process in a vehicle width estimation part.

  Hereinafter, an embodiment of a vehicle detection device according to the present invention will be described with reference to the drawings.

  In the present embodiment, the vehicle detection device according to the present invention is applied to a vehicle detection device mounted on a vehicle. The vehicle detection device according to the present embodiment detects a vehicle existing in front of the host vehicle, and provides the detected vehicle information to a driving support device such as an inter-vehicle distance control device or a collision prevention device. In this embodiment, there are two forms. The first embodiment is a form in which an end portion (vehicle surface) of the vehicle is detected as vehicle information, and the second embodiment is a vehicle in which vehicle information is used. It is a form which detects the width | variety of a vehicle other than the edge part of this.

  With reference to FIGS. 1-8, the vehicle detection apparatus 1 which concerns on 1st Embodiment is demonstrated. FIG. 1 is a configuration diagram of the vehicle detection device according to the first embodiment. FIG. 2 is an example of a reflection point by a laser radar. FIG. 3 is an example of the division position of the reflection point group. FIG. 4 shows an example of a shape model, where (a) is model 1 and (b) is model 2. FIG. 5 shows a sobel filter, in which (a) is for detecting a horizontal edge and (b) is for detecting a vertical edge. FIG. 6 is an example of an edge image with respect to an image by a camera. FIG. 7 is a diagram illustrating a relationship between the position of the feature point on the image and the three-dimensional position, in which (a) is a perspective view of the vehicle and (b) is a top view. FIG. 8 is an explanatory diagram of the time point correspondence of feature points, (a) is a comparative example between times, (b) is a comparative example 1 with a large positional error, and (c) is a positional error. It is a small comparative example 2.

  The vehicle detection device 1 estimates a range on an image where each surface of the vehicle exists using radar information from a radar sensor, and based on edge information in the estimated range on the image using image information from a camera. The end point of the vehicle (the boundary position between the two surfaces) is detected. In particular, the vehicle detection device 1 provides a correspondence relationship between feature points that reduces the difference in feature amount between a plurality of feature points for each combination of surface candidates estimated from each image between two consecutive times. Search, find the minimum position error after translating and rotating the three-dimensional position between the multiple feature points of the correspondence, and the combination surface that minimizes the position error among all combination surface candidates Candidates are extracted, a surface of the vehicle is detected based on the extracted surface candidates for the combination, and an end point of the vehicle is extracted from the surface. For this purpose, the vehicle detection apparatus 1 includes a laser radar 3, a camera 4, and an ECU [Electronic Control Unit] 10. The ECU 10 includes a vehicle position / surface candidate estimation unit 11, a feature point extraction unit 12, a feature point three-dimensional position. A candidate estimation unit 13 and a vehicle end point extraction unit 14 are configured.

  In the first embodiment, the vehicle position / surface candidate estimation unit 11 corresponds to the surface candidate estimation means described in the claims, and the feature point extraction unit 12 extracts the feature points described in the claims. The feature point 3D position candidate estimation unit 13 corresponds to the 3D position estimation unit described in the claims, and the vehicle end point extraction unit 14 corresponds to the vehicle information detection unit described in the claims. To do.

  The laser radar 3 is a scanning radar that detects an object using laser light. The laser radar 3 is attached to the center of the front end of the host vehicle. The laser radar 3 is mounted at a height that can sufficiently detect the vehicle to be detected and substantially parallel to the road surface. In the laser radar 3, the laser beam is emitted from the own vehicle forward while scanning in the horizontal direction at predetermined angles every predetermined time, and the reflected laser beam is received. In the laser radar 3, a radar composed of data (horizontal scanning azimuth angle, emission time, light reception time, light reception intensity (reflection intensity), etc.) about each reflection point (detection point) that can be received at predetermined time intervals. A signal is transmitted to ECU10.

  The camera 4 is composed of a CCD [Charge Coupled Device], a lens, and the like, and is a camera that images the front of the host vehicle. The camera 4 is attached to a predetermined position in the center of the front of the host vehicle, and is arranged in a direction in which the direction of the center axis extending in front of the host vehicle and the optical axis of the camera coincide. The camera 4 captures an image of the front of the host vehicle every predetermined time (for example, every 100 milliseconds), and transmits data of the captured image to the ECU 10 as an image signal. The camera 4 may be a black and white camera or a color camera, as long as it can acquire at least optical point density information about the object.

  The mounting position relationship between the laser radar 3 and the camera 4 is measured in advance, and various parameters necessary for coordinate conversion between the coordinate system of the laser radar 3 and the coordinate system of the camera 4 are acquired. .

  The ECU 10 is an electronic control unit including a CPU [Central Processing Unit], a ROM [Read Only Memory], a RAM [Random Access Memory], and the like, and comprehensively controls the vehicle detection device 1. In the ECU 10, an application program stored in the ROM is loaded onto the RAM and executed by the CPU, whereby the vehicle position / surface candidate estimation unit 11, the feature point extraction unit 12, the feature point three-dimensional position candidate estimation unit 13, the vehicle The end point extraction unit 14 is configured. The ECU 10 receives a radar signal from the laser radar 3 and an image signal from the camera 4 at predetermined time intervals. Then, the ECU 10 performs processing of each of the units 11, 12, 13, and 14 using these signals, and when a vehicle can be detected, transmits the vehicle information as a vehicle information signal to the driving support device.

  The vehicle position / surface candidate estimation unit 11 will be described. The vehicle position / surface candidate estimation unit 11 performs grouping on a large number of reflection points detected by the laser radar 3, applies the vehicle shape model to the reflection point group of the group for each group, and forms the vehicle shape model. If there is a group to which is applied, a vehicle surface candidate (range on the image) is estimated from the reflection point group.

  FIG. 2 shows an example of the reflection points pt1, pt2,..., Pt19 detected by the laser radar 3. In the traveling environment of the vehicle, the vehicle ahead of the host vehicle and other objects such as guardrails around the vehicle are separated by a certain distance. Therefore, the reflection points reflected by the same object are close, but the reflection points reflected by different objects are far. In the case of the example shown in FIG. 2, the distance between adjacent reflection points is small in the reflection point groups pt12 to pt17 in the vehicle, and the reflections adjacent to each other in the reflection point groups pt1 to pt11 and the reflection point groups pt18 to pt19 in the other objects. Although the distance between the points decreases, the distance between the reflection point group pt12 and the adjacent reflection point pt11 in the vehicle increases, and the distance between the reflection point group pt17 and the adjacent reflection point pt18 increases in the vehicle.

The grouping process will be specifically described. In the vehicle position / surface candidate estimation unit 11, for each reflection point pt detected by the laser radar 3, the distance LN _{pt, pt−1} between the reflection point pt and the adjacent reflection point pt−1 is calculated according to the equation (1). Calculate In equation (1), rpx is the x coordinate of the reflection point pt in the three-dimensional laser radar coordinate system, rpy is the y coordinate of the reflection point pt in the three-dimensional laser radar coordinate system, and rpz is the three-dimensional laser. This is the z coordinate of the reflection point pt in the radar coordinate system.

Then, the vehicle position / surface candidate estimation unit 11 determines whether or not the distance LN _{pt, pt-1} between the reflection points is less than the threshold value THR_PTL. The threshold value THR_PTL is a threshold value for determining whether or not two adjacent reflection points are reflection points reflected by the same object based on the distance between the adjacent reflection points. The threshold value THR_PTL is, for example, (lane width−vehicle width) / 2 from the viewpoint of separation of the vehicle and the roadside object.

The vehicle position / surface candidate estimation unit 11 classifies the two reflection points pt and pt-1 into the same group when the distance LN _{pt, pt-1} between the reflection points is less than the threshold value THR_PTL, and the distance LN between the reflection points LN. _{When pt and pt-1} are equal to or greater than the threshold value THR_PTL, the two reflection points pt and pt-1 are classified into different groups. In the vehicle position / surface candidate estimation unit 11, this reflection point distance determination is performed on all reflection points detected by the laser radar 3, and all reflection points are grouped.

The shape model estimation process will be specifically described. The vehicle position / surface candidate estimation unit 11 sequentially sets the division position spt for the group of reflected points included in the group gn for each grouped group gn. Then, in the vehicle position / surface candidate estimation unit 11, the reflection point group included in the group gn is divided into two reflection point groups RPT1 = {rpt _{gn, 1} ,..., Rpt _{gn, spt with the} division position spt as a boundary. }, RPT2 = {rpt _{gn, spt} ,..., Rpt _{gn, Ngn} }. The division position spt is set to an intermediate position between two adjacent reflection points or a position outside the reflection points at both ends in the group. Therefore, when the number of reflection points is n, n + 1 division positions are set. FIG. 3 shows a group of reflection points rpt _{gn, 1} ,..., Rpt _{gn, 6} , and the reflection points rpt _{gn, 3} and rpt _{gn, 4} adjacent to each other as division positions spt. Is set in between and is divided into two reflection point groups RPT1 = {rpt _{gn, 1} , rpt _{gn, 2} , rpt _{gn, 3} }, RPT2 = {rpt _{gn, 4} , rpt _{gn, 5} , rpt _{gn, 6} } The

Each time the division position spt is set, the vehicle position / surface candidate estimation unit 11 uses two reflection point groups RPT1 = {rpt _{gn, 1} ,..., For each vehicle shape model model prepared in advance. rpt _{gn, spt} }, RPT2 = {rpt _{gn, spt} ..., rpt _{gn, Ngn} } and the degree of coincidence SM _{model, spt} of the vehicle shape model model are calculated. The coincidence degree SM is calculated as, for example, the sum of errors between the shape model and each reflection point.

  The vehicle shape model includes a two-surface model of a vehicle including a front / rear model and a side model, and a one-surface model of the vehicle in the front, rear, and side surfaces. The model of each surface is a model composed of lines indicating the outer shape of each surface when the vehicle is cut in a horizontal plane, for example, a line represented by a linear function or a quadratic function as a front / rear model, The model for use is a line represented by a linear function. FIG. 4 shows an example of a vehicle shape model. The shape model model 1 shown in FIG. 4A is composed of a straight side model sm and a curved front / rear model fm, and is shown in FIG. The shape model model2 is composed of a curved front / rear model fm and a straight side model sm. Incidentally, when the division position spt is set between adjacent reflection points, a process with a two-surface model is performed, and when the division position spt is set outside the reflection point group, one surface is processed. Process with the model.

FIG. 4 shows two reflection point groups RPT1 = {rpt _{gn, 1} , rpt _{gn, 2} , rpt _{gn, 3} }, RPT2 = {rpt _{gn, 4} , rpt _gn, divided at the division position spt shown in FIG. ₅ , rpt _{gn, 6} } shows a case where model 1 shown in FIG. 4A is applied and a case where model 2 shown in FIG. 4B is applied. The shape model model 1 shown in FIG. Although applied, the shape model model2 shown in FIG. 4B is not applied.

Each time the degree of coincidence SM _{model, spt} is calculated, the vehicle position / surface candidate estimation unit 11 determines whether the degree of coincidence SM _{model, spt} is less than the threshold value THR_SM. The threshold value THR_SM is a threshold value for determining whether the reflection point group of the group is applicable to the vehicle shape model based on the degree of coincidence.

In the vehicle position / surface candidate estimation unit 11, when the degree of coincidence SM _{model, spt} is less than the threshold value THR_SM, two reflection point groups RPT1 = {rpt _{gn, 1} ,..., Rpt _{gn, spt} }, RPT2 = {rpt _{gn, spt} ,..., rpt _{gn, Ngn} }, the length between the reflection points at both ends (the length in the lateral direction) is calculated, and the longer of the two lengths is the length of the longest part. Let LW _{model, spt} . Then, the vehicle position / surface candidate estimation unit 11 determines whether or not the length LW _{model, spt} of the longest part is within a predetermined range (THR_LW_LOW to THR_LW_HIGH). The predetermined range (THR_LW_LOW to THR_LW_HIGH) is based on the length LW _{model, spt} of the longest portion, and whether or not the horizontal length of the reflection point group of each group is within the range corresponding to the surface of the vehicle (that is, , (Excludes objects that are not vehicles).

In the vehicle position / plane candidate estimation unit 11, when the length LW _{model, spt} of the longest part is within a predetermined range (THR_LW_LOW to THR_LW_HIGH), the shape matching degree SM _{model, spt} is the segment position set so far. It is determined whether the minimum error is less than MIN_SM _spt . The minimum error MIN_SM _spt is a parameter for searching for the minimum shape matching degree SM _{model, spt} , and ∞ is set as an initial value.

In the vehicle position and face candidate estimation unit 11, when the shape matching degree _{SM model, spt} is less than minimum error MIN_SM _spt, shape matching degree _{SM model} as minimal error MIN_SM _spt, it sets a _spt, geometric model of the vehicle at that time Model parameters MP _{model, spt} are stored. The model parameters MP _{model and spt} are parameters indicating a vehicle shape model. For example, the model parameter MP _{model, spt} = (a1, a2, a3, a4, a5, a6) is calculated using the RANSAC [RANdom Sample Consensus] method. However, the front surface x ′ = a1y ′ ² + a2y ′ + a3, the side surface y = a4x + a5, a6 = cosφ, x ′ = cosφx−sinφy, and y ′ = sinφx + cosφy.

Then, the vehicle position / surface candidate estimation unit 11 performs the above process on all the shape models model prepared for each group gn, and finally stores the model parameters MP _{model, spt} ( That is, the matching point SM _{model, spt} is less than the threshold value THR_SM, the longest length LW _{model, spt} is within a predetermined range (THR_LW_LOW to THR_LW_HIGH), and the matching degree SM _{model, spt} is When there is a minimum shape model), it is determined that one vehicle shape model has been applied at the division point spt of the reflection point group of the group gn, and the shape model indicated by the model parameter MP _{model, spt} is determined as the group gn Paired with the split position spt in the reflection point group To establish as a vehicle of the shape model that. Thus, when there is a group gn (reflection point group) that applies to the shape model, the vehicle is detected.

In the vehicle position / surface candidate estimation unit 11, when data is stored in the model parameters MP _{model, spt} (when there is a vehicle shape model that applies at the division position spt of the reflection point group of the group gn), the equation (2) Thus, using the data stored in the model parameters MP _{model, spt} , the vehicle orientation φ _spt (corresponding to a vehicle orientation candidate at the divided position spt) is calculated.

  The on-image range estimation process will be specifically described. When there is a group gn to which the shape model of the vehicle on the image is applied, the range of the vehicle on the image is estimated using the reflection point group of the group gn. The estimation is performed by dividing the range into the side image. Thus, the reason for estimating the range on the image separately for the front surface / rear surface and the side surface is to perform plane fitting when estimating the three-dimensional position of each feature point in the feature point three-dimensional position candidate estimation unit 13. This is to improve the accuracy in the plane fitting.

First, front and rear range estimation will be described. In the vehicle position / surface candidate estimation unit 11, for each reflection point rp _{gn, fr, i} belonging to the front surface / rear surface in the group gn, the reflection point rp _{gn, fr, i} = three-dimensional position according to the equation (3). (X _{gn, fr, i} , y _{gn, fr, i} , z _{gn, fr, i} , 1) The position Pimg _{gn, fr, i} on the image from ^T = two-dimensional position (u _{gn, fr, i} , v _{gn , Fr, i} 1) ^T is calculated. P in Equation (3) is a 3 × 4 conversion matrix for converting from the laser radar coordinate system to the image coordinate system. Internal parameters such as the focal length of the camera 4, and the relationship between the laser radar 3 and the camera 4 It is calculated from the relative position and orientation. Further, s in Expression (3) is a depth parameter considering that there is no depth direction when projecting from a three-dimensional coordinate system to a two-dimensional coordinate system.

Each time the position Pimg _{gn, fr, i} is calculated, the vehicle position / plane candidate estimation unit 11 determines whether the lateral position u _{gn, fr, i} is smaller than the left end point position FRLPt _gn for the front and rear surfaces. . When the vehicle position / plane candidate estimation unit 11 determines that the lateral position u _{gn, fr, i} is smaller than the left end point position FRLPt _gn , the lateral position u _{gn, fr, i} is set as the left end point position FRLPt _gn . Further, the vehicle position / surface candidate estimation unit 11 determines whether or not the lateral position ugn _{, fr, i} is larger than the front end / rear surface right end point position FRRPt _gn . When the vehicle position / plane candidate estimation unit 11 determines that the lateral position u _{gn, fr, i} is greater than the right end point position FRRPt _gn , the lateral position u _{gn, fr, i} is set as the right end point position FRRPt _gn . The left end point position FRLPt _gn is a parameter for searching the left end point on the image of the reflection point group belonging to the front and rear surfaces of the group gn to which the vehicle shape model is applied, and the horizontal width IMG_WIDTH of the image is set as an initial value. The The right end point position FRRPt _gn is a parameter for searching the right end point on the image of the reflection point group belonging to the front surface / rear surface of the group gn to which the vehicle shape model is applied, and 0 is set as an initial value. Incidentally, in the image coordinate system, the horizontal position u at the left end on the image is 0, and the horizontal position u increases toward the right. In the image coordinate system, the vertical position v at the upper end of the image is 0, and the vertical position v increases as it goes downward.

The vehicle position / surface candidate estimation unit 11 performs the above processing on all the reflection points belonging to the front surface / rear surface in the group gn to which the vehicle shape model is applied. The final left end point position FRLPt _gn corresponds to the left end point of the front and rear surfaces of the vehicle on the group gn image, and the right end point position FRRPt _gn corresponds to the right end of the front and rear surfaces of the vehicle on the group gn image. It corresponds to a point.

Next, the vehicle position / surface candidate estimation unit 11 uses the three-dimensional positions of all the reflection points rp _{gn, fr, i} belonging to the front surface / rear surface in the group gn, and calculates the average position rpmean _{gn, fr} = ( xm _{gn, fr} , ym _{gn, fr} , zm _{gn, fr} , 1) ^T is calculated. Then, the vehicle position / surface candidate estimation unit 11 uses the average position rpmean _{gn, fr} to calculate the vehicle front / rear lower end point FRVlow _gn on the image of the group gn according to the equation (4). Further, the vehicle position / surface candidate estimation unit 11 uses the average position rpmean _{gn, fr} to calculate the vehicle front / rear upper end points FRVhigh _gn on the group gn image according to the equation (5). Hv _fr in the expressions (4) and (5) is an amount on the image corresponding to the height of the hood of the vehicle (for example, 0.8 m). In the equations (4) and (5), p _ij is an element in the i-th row and j-th column of the above-mentioned 3-by-4 conversion matrix P.

The vehicle position / surface candidate estimation unit 11 then sets the left end point position FRLPt _gn (= FRLPt _gn −dw finally set by the above processing) to the right end point position FRRPt _gn (= finally set by the above processing). FRRPt _gn + dw) is stored as a horizontal range on the image of the reflection point group belonging to the front surface / rear surface of the group gn. Further, the vehicle position and face candidate estimation unit 11, the calculated lower end point position FRVlow _gn ~ upper end point position FRVhigh _gn in the above process, the reflection points belonging to the front and rear surfaces of the group gn image on longitudinal Store as a range. Note that dw is a parameter for dealing with a case where an error of the laser radar 3 or a point sequence is not reflected, and an amount about the lateral resolution of the laser radar 3 is set.

Next, side surface range estimation will be described. In the vehicle position / surface candidate estimation unit 11, for each reflection point rp _{gn, sd, i} belonging to the side surface in the group gn, the reflection point rp _{gn, sd, i} = three-dimensional position (x _{gn, sd, i} , y _{gn, sd, i} , z _{gn, sd, i} , 1) The position Pimg _{gn, sd, i} on the image from ^T = two-dimensional position (ugn _{, sd, i} , v _{gn, sd , I} 1) ^T is calculated. P in Equation (6) is the same matrix as P in Equation (3). Further, s in equation (6) is the same parameter as s in equation (3).

Each time the position Pimg _{gn, sd, i} is calculated, the vehicle position / surface candidate estimation unit 11 determines whether or not the lateral position u _{gn, sd, i} is smaller than the left end point position SDLPt _gn for the side surface. If the vehicle position / plane candidate estimation unit 11 determines that the lateral position u _{gn, sd, i} is smaller than the left end point position SDLPt _gn , the lateral position u _{gn, sd, i} is set as the left end point position SDLPt _gn . Further, the vehicle position / surface candidate estimation unit 11 determines whether or not the lateral position u _{gn, sd, i} is larger than the right end point position SDRPt _gn for the side surface. If the vehicle position / plane candidate estimation unit 11 determines that the lateral position u _{gn, sd, i} is larger than the right end point position SDRPt _gn , the lateral position u _{gn, sd, i} is set as the right end point position SDRPt _gn . The left end point position SDLPt _gn is a parameter for searching the left end point on the image of the reflection point group belonging to the side surface of the group gn to which the vehicle shape model is applied, and the horizontal width IMG_WIDTH of the image is set as an initial value. The right end point position SDRPt _gn is a parameter for searching the right end point on the image of the reflection point group belonging to the side surface of the group gn to which the vehicle shape model is applied, and 0 is set as an initial value.

The vehicle position / surface candidate estimation unit 11 performs the above processing on all reflection points belonging to the side surface in the group gn to which the vehicle shape model is applied. The finally set left end point position SDLPt _gn corresponds to the left end point of the side surface of the vehicle on the group gn image, and the right end point position SDRPt _gn corresponds to the right end point of the side surface of the vehicle on the group gn image. .

Next, the vehicle position / surface candidate estimation unit 11 uses the three-dimensional positions of all the reflection points rp _{gn, sd, i} belonging to the side surfaces in the group gn, and calculates the average position rpmean _{gn, sd} = (xm _{gn , Sd} , ym _{gn, sd} , zm _{gn, sd} , 1) Calculate ^T. Then, the vehicle position / surface candidate estimation unit 11 calculates the lower end point SDVlow _gn for the side surface of the vehicle on the image of the group gn by using the average position rpmean _{gn, sd} using the equation (7). Further, the vehicle position / surface candidate estimation unit 11 calculates the upper end point SDVhigh _gn for the side surface of the vehicle on the image of the group gn by using the average position rpmean _{gn, sd} using the average position rpmean _{gn, sd} . Hv _sd in Expression (7) and Expression (8) is an amount on the image corresponding to the height of the vehicle (for example, 2.0 m). In formula (7) and formula (8), p _ij is an element in the i-th row and j-th column of the above-mentioned 3-by-4 conversion matrix P.

The vehicle position / surface candidate estimation unit 11 then sets the left end point position SDLPt _gn (= SDLPt _gn −dw finally set by the above processing) to the right end point position SDRPt _gn (= finally set by the above processing). SDRPt _gn + dw) is stored as a horizontal range on the image of the reflection point group belonging to the side surface of the group gn. Further, the vehicle position and face candidate estimation unit 11, the calculated lower end point position SDVlow _gn ~ upper end point position SDVhigh _gn above process, as the vertical direction in the range on the image of the reflection points belonging to the side of the group gn Remember. The range in the horizontal direction and the range in the vertical direction are ranges on the image of the detected side surface of the vehicle.

  When the front and rear image upper ranges and the side image upper ranges are estimated, the vehicle position / surface candidate estimation unit 11 sets surface candidates including the front and rear image upper ranges and the side image upper ranges of the same group gn. To do.

The feature point extraction unit 12 will be described. When the vehicle position / surface candidate estimation unit 11 detects a vehicle and the front / rear surface range and the side surface range on the vehicle image are estimated, the feature point extraction unit 12 Then, an edge (feature point) is extracted from the image captured by the camera 4. As an edge extraction method, a local region is extracted from an image, and a feature amount of the region is calculated using a sobel filter or the like. As a method of extracting the local region, extraction at a constant interval, corner detection such as a Harris operator, or the like may be used. The sobel filter convolves and integrates an image using a horizontal edge detection filter shown in FIG. 5A and a vertical edge detection filter shown in FIG. Specifically, in the feature point extraction unit 12, for each pixel (u, v) in the range estimated by the vehicle position / surface candidate estimation unit 11, the value of each pixel (for example, luminance _{value) I u,} with _v, calculates the vertical edge intensity? Iv _{u, v} by the equation (10) as well as calculating the lateral edge strength ΔIh _{u, v} by the formula (9). Further, the feature point extraction unit 12 calculates the edge strength ΔI _{u, v} by the equation (11) using the edge strengths ΔIh _{u, v} , ΔIv _{u, v} and the edge direction θ _u by the equation (12). _{, V} is calculated. Note that this edge extraction processing may be performed on the entire image captured by the camera 4.

  FIG. 6 shows an example of an edge image (contour image) in which edge processing is performed for each pixel on an image captured by the camera 4 and edge strength is set at each pixel position (portion where a vehicle is shown). In this image, there is a vehicle in which the front and left sides are shown.

  The feature point three-dimensional position candidate estimation unit 13 will be described. In the feature point three-dimensional position candidate estimation unit 13, for each feature point extracted by the feature point extraction unit 12, a three-dimensional position on the front surface / rear surface or side surface of the surface candidate estimated by the vehicle position / surface candidate estimation unit 11 is calculated. calculate. Here, since the direction vector in which each feature point exists is known from the position on the image, the three-dimensional position is estimated by calculating the intersection of the direction vector and the front / back surface or side surface of the estimated surface candidate.

FIG. 7 shows the relationship between the position of the feature point on the image, the three-dimensional position candidate obtained from the position, and the actual three-dimensional position, and FIG. FIG. 4B shows a case where the vehicle is viewed from above. When the position of the feature point CP on the image IP is known, the direction n _f where the feature point CP is located can be found from the camera 4 from the parameters indicating the relationship between the camera 4 and the laser radar 3. Since the vehicle shape model for the surface candidate estimated by the vehicle position / surface candidate estimation unit 11 can be estimated, the three-dimensional feature point CP is determined by the intersection of the front position MP of the shape model and the direction n _f of the feature point CP. The position candidate 3P ′ can be estimated. The three-dimensional position candidate 3P ′ has an error with respect to the actual feature point three-dimensional position 3P due to an error of the shape model. However, if the shape model can be correctly estimated, the error is dependent on the size of the vehicle. Smaller and less affected.

Specifically, the direction n _f = ( _{nx, f} , _{ny, f} , _{nz, f} , 1) where the feature point exists ^T is the position of the feature point f on the image (13) according to equation (13). u _f , v _f , 1) can be calculated using ^T. S in equation (13) is the depth parameter s in equation (3) and the like. The matrix of 3 rows and 4 columns in Equation (13) is the 3 × 4 transformation matrix P in Equation (3) and the like.

Here, since the depth parameter s may be any value, when s = 1, Expression (13) becomes Expression (14). Furthermore, equation (14) can be transformed into equation (15).

Therefore, in the feature point three-dimensional position candidate estimation unit 13, for each surface candidate spt estimated by the vehicle position / surface candidate estimation unit 11, each feature point included in the on-image range of the surface candidate spt is obtained by Expression (15). For f, using the position (u _f , v _f , 1) ^T of the feature point f on the image, the direction n _f = ( _{nx, f} , _{ny, f} , _{nz, f} ) where the feature point CP exists Calculate ^T respectively. Further, the feature point three-dimensional position candidate estimation unit 13 uses the direction n _f = ( _{nx, f} , _{ny, f} , n _{z, f} ) ^T to estimate by the vehicle position / plane candidate estimation unit 11. Three-dimensional position candidates (x _{f, spt} , y _{f, spt} , z _{f, spt} ) of the feature point f are calculated as positions where the surface candidate spt intersects with the front surface model surface or the side model surface. For example, the surface of the shape model is y = ax + y, passes through the optical center position (C _x , C _y , C _z ) of the camera 4, and the direction n _f = ( _{nx, f} , _{ny, f} , _nz, three-dimensional position of the feature point f of ^{_{f) T (x f, spt}} , y f, spt, z f, spt) can be calculated by equation (16).

If there is a case that does not intersect the shape model, the feature point is discarded. In addition, when there can be a plurality of solutions, the one with the shortest distance is defined as a three-dimensional position ( _{xf , spt , yf, spt, zf, spt} ).

  The vehicle end point extraction unit 14 will be described. The vehicle end point extraction unit 14 performs processing between the surface candidates estimated from the respective images captured at two consecutive times t1 and t2. In the vehicle end point extraction unit 14, for each combination of the surface candidate at time t1 and the surface candidate at time t2, the corresponding relationship of the three-dimensional position of each feature point is obtained between the surface candidates, and the feature points of the corresponding relationship The position error is calculated after correcting the feature point of one surface candidate by translational movement conversion and rotation conversion so that the position error between them is minimized. When the position error between the feature points is obtained for all the surface candidate combinations, the vehicle end point extraction unit 14 extracts the surface candidates of the combination that minimizes the position error, and the vehicle end points based on the division positions in the surface candidates. (Boundary position between two surfaces) is extracted. Note that time t2 is the current time, and time t1 is one hour before the imaging interval of the camera 4.

In the example shown in FIG. 8, a plurality of surface candidates spt _t1 (characteristic points are drawn in circles) at the previous time t1 and a plurality of surface candidates spt _t2 (characteristic points are rectangular at the current time t2) in FIG. In other words, each combination of a plurality of surface candidates spt _t1 and a plurality of surface candidates spt _t2 is processed (compared). In each comparison, between surface candidate spt _t2 surface candidate spt _t1 and the current time t2 before time t1, one time after the correspondence is taken between the feature points of the feature point and the current time t2 before time t1 The translation amount and rotation amount for converting the position of each feature point of the surface candidate are estimated, and the position error between corresponding feature points is calculated after rotating with the translation amount and rotation amount. . In the case of the comparative example 1 shown in FIG. 2B, the position of the corresponding feature point at the previous time t1 is far from the position of the feature point at the current time t2, and the position error is large. On the other hand, in the case of the comparative example 2 shown in FIG. 4C, the position of the corresponding feature point at the previous time t1 and the position of the feature point at the current time t2 substantially overlap, and the position error is small.

  The reason why the position error becomes large is that the estimated shape model does not match and the error between the three-dimensional positions of the feature points increases, and the three-dimensional position of the feature points is estimated from the wrong shape model. Therefore, there is a reason that the error between the three-dimensional positions of the feature points becomes large.

Specifically, in the vehicle end point extracting section 14, prior to each combination of the surface candidate _{spt t2} in terms candidate _{spt t1} and the current time t2 at time t1, _{N ft1} feature points contained in the plane candidate _{spt t1} Corresponding feature points are searched for between N _ft2 feature points f _t2 included in f _t1 and surface candidate spt _t2 . As a search method, the equation (17), for each feature point _{f t2} surface candidate _{spt t2, N ft1} pieces included in the face candidate _{spt t1} of the difference [Delta] F 'of the feature quantity of each feature point _{f t1} respectively calculated, will be stored as 'feature point f _t2 corresponding feature point f _t1 which is the smallest in the feature point f _t2' difference ΔF of the feature. In Expression (17), ΔIh _ft1 is the edge strength in the horizontal direction of the feature point f _t1 , ΔIv _ft1 is the edge strength in the vertical direction of the feature point f _t1 , and ΔIh _ft2 is the edge strength in the horizontal direction of the feature point f _t2. in and,? Iv _{ft @ 2} is a vertical edge strength of the feature point _{f t2.}

When searching for a correspondence between the feature points, the vehicle end point extracting section 14, to each combination of surface candidate spt _t1 at the previous time t1 and face candidate spt _t2 at the current time t2, the position error between the corresponding feature points translation amount such that the minimum seek _{(dx spt,} dy _{spt, dz spt)} and the rotation amount [psi _spt, (in this case, the surface candidate _{spt t1)} one surface candidate spt translational movement amounts of each feature point f of ( The position error E between the corresponding feature points (that is, the position error E between the surface candidates spt _t1 and spt _t2 ) is calculated after conversion by _dxspt , _dyspt , _dzspt ) and the rotation amount _ψspt . The position error E can be calculated by equation (18). X in equation (18) can be calculated by equation (19). Translation amount such that the position error is minimized _{(dx spt, dy spt, dz} spt) as a method of determining the the amount of rotation [psi _spt, for example, utilizes a RANSAC method.

When the position error E between all the combination surface candidates is calculated, the vehicle end point extraction unit 14 compares the position error E between all the combination surface candidates, and determines the combination STP1 and SPT2 of the surface candidates that give the minimum position error E. Extract. Then, the vehicle end point extraction unit 14 selects two reflection points rpt _{gn and SPT1} and rpt _{gn and SPT1 + 1} in the vicinity of the division position from the reflection point group of the group gn corresponding to the extracted surface candidate SPT1 at the previous time t1. The average value of the two reflection points is extracted as the end point of the vehicle (boundary position between the two surfaces) at the previous time t1. Further, the vehicle end point extraction unit 14 calculates two reflection points rpt _{gn, SPT2} and rpt _{gn, SPT2 + 1} in the vicinity of the division position from the reflection point group of the group gn corresponding to the extracted surface candidate SPT2 at the current time t2. The average value of the two reflection points is extracted as the end point of the vehicle at the current time t2. Further, the vehicle end point extraction unit 14 calculates the end point (two-dimensional position) of the vehicle on the image using the end point (three-dimensional position) of the vehicle.

  The operation of the vehicle detection device 1 will be described. In particular, the processing of the vehicle position / surface candidate estimation unit 11 in the ECU 10 will be described with reference to the flowcharts of FIGS. 9 to 13, and the processing of the feature point three-dimensional position candidate estimation unit 13 will be described with reference to the flowchart of FIG. The processing of the vehicle end point extraction unit 14 will be described with reference to the flowcharts of FIGS. 15 and 16. FIG. 9 is a flowchart showing a flow of main processing in the vehicle position / surface candidate estimation unit. FIG. 10 is a flowchart showing the flow of the grouping process in the main process in the vehicle position / surface candidate estimation unit. FIG. 11 is a flowchart showing the flow of the shape model estimation process in the main process in the vehicle position / surface candidate estimation unit. FIG. 12 is a flowchart showing the flow of the on-image range estimation process (front / rear) in the main process in the vehicle position / plane candidate estimation unit. FIG. 13 is a flowchart showing the flow of the on-image range estimation process (side surface) in the main process in the vehicle position / surface candidate estimation unit. FIG. 14 is a flowchart showing a flow of processing in the feature point three-dimensional position candidate estimation unit. FIG. 15 is a flowchart (first half) showing a flow of processing in the vehicle end point extraction unit. FIG. 16 is a flowchart (second half) showing the flow of processing in the vehicle end point extraction unit.

  The laser radar 3 emits laser light while scanning in the horizontal direction at predetermined intervals, receives the reflected light, and transmits information about each reflected point received to the ECU 10 as a radar signal. The camera 4 captures the front of the host vehicle and transmits an image signal to the ECU 10 every predetermined time.

  The ECU 10 performs the following processing at regular intervals. First, as processing in the vehicle position / surface candidate estimation unit 11, the ECU 10 performs reflection point grouping processing using information about each reflection point from the laser radar 3 (S1).

When the process proceeds to the grouping process (S1), the ECU 10 first initializes the reflection point pt to 0 in order to perform the process on all the reflection points detected by the laser radar 3 (S10). Further, the ECU 10 initializes GID ₀ , which is the group ID of the reflection point pt = 0, with 0 (S11).

Every time the reflection point pt is updated, the ECU 10 calculates the distances LN _{pt, pt-1} between the current reflection point pt and the previous reflection point pt-1 adjacent thereto by the equation (1) (S12). . Then, the ECU 10 determines whether or not the distance LN _{pt, pt-1} is less than the threshold value THR_PTL (S13). When it is determined in S13 that the distance LN _{pt, pt-1} is less than the threshold value THR_PTL, the ECU 10 sets the group ID of the reflection point pt to make the current reflection point pt the same group as the previous reflection point pt-1. GID _{pt -1} is set to GID _pt (S14). On the other hand, when it is determined in S13 that the distance LN _{pt, pt-1} is equal to or greater than the threshold value THR_PTL, the ECU 10 uses the reflection point pt group in order to make the current reflection point pt different from the previous reflection point pt-1. GID _pt-1 +1 is set to GID _pt of ID (S15).

  Next, the ECU 10 adds 1 to pt in order to update the reflection point pt (S16). Then, the ECU 10 determines whether or not the updated pt is equal to or greater than the number Nrp of reflection points that can be detected by the laser radar 3 (S17). If it is determined in S17 that pt is equal to or greater than Nrp, the current grouping process is terminated. On the other hand, if it is determined in S17 that pt is less than Nrp, the ECU 10 returns to S12 and performs processing for the updated reflection point pt.

  When the grouping process is completed, the ECU 10 first initializes the group gn with 0 in order to perform the process on all the grouped groups (S2).

  Every time the group gn is updated, the ECU 10 performs a shape model estimation process on the group gn (S3).

  When the process proceeds to the shape model estimation process (S3), the ECU 10 first initializes the division position spt with 0 (S20).

Every time the division position spt is updated, the ECU 10 first initializes the minimum error MIN_SM _spt with ∞ (S21). In the ECU 10, the reflection point group of the group gn is divided into two reflection point groups RPT1 = {rpt _{gn, 1} ,..., Rpt _{gn, spt} }, RPT2 = {rpt _{gn, spt} , .., Rpt _{gn, Ngn} } are divided (S22). Further, in order to perform processing on all the vehicle shape models prepared in advance, the ECU 10 first initializes the shape model model with 0 (S23).

Each time the shape model model is updated, the ECU 10 applies the two surfaces of the shape model to the divided reflection point group RPT1 and reflection point group RPT2, and calculates the shape matching degree SM _{model, spt} (S24). Then, the ECU 10 determines whether or not the shape matching degree SM _{model, spt} is less than the threshold value THR_SM (S25). If the shape matching degree SM _{model, spt} is determined to be greater than or equal to the threshold value THR_SM in S25, the ECU 10 proceeds to the process of S31 and adds 1 to model in order to update the shape model model (S31).

When it is determined in S25 that the shape matching degree SM _{model, spt} is less than the threshold value THR_SM, the ECU 10 calculates the length LW _{model, spt} of the longest part among the divided reflection point group RPT1 and reflection point group RPT2 (S26). ). Then, the ECU 10 determines whether or not the length LW _{model, spt} of the longest part is within a predetermined range (THR_LW_LOW to THR_LW_HIGH) (S27). When it is determined that the length LW _{model, spt} of the longest part is outside the predetermined range, the ECU 10 proceeds to the process of S31 and adds 1 to the model in order to update the shape model model (S31).

When it is determined that the length LW _{model, spt} of the longest part is within the predetermined range, the ECU 10 determines whether or not the shape matching degree SM _{model, spt} is less than the minimum error MIN_SM _spt (S28). If it is determined in S28 that the shape matching degree SM _{model, spt} is equal to or greater than the minimum error MIN_SM _spt , the ECU 10 proceeds to the process of S31 and adds 1 to model in order to update the shape model model (S31).

If it is determined in S28 that the shape matching degree SM _{model, spt} is less than the minimum error MIN_SM _spt , the ECU 10 updates the minimum error MIN_SM _spt with the shape matching degree SM _{model, spt} (S29). Further, the ECU 10 stores model parameters MP _{model, spt} corresponding to the current shape model model (S30). Then, the ECU 10 adds 1 to the model in order to update the shape model model (S31).

  When the model is updated in S31, the ECU 10 determines whether or not the shape model model is equal to or more than the number Nm of shape models prepared in advance (S32). If it is determined in S32 that the model is less than Nm, the ECU 10 returns to S24 and performs the process for the updated shape model model.

If it is determined in S32 that the model is Nm or more, the ECU 10 calculates the vehicle orientation φ _spt (corresponding to the orientation candidate) according to the equation (2) only when there is a shape model applicable at the division position spt at that time ( S33). Then, the ECU 10 adds 1 to spt in order to update the division position spt (S34). Then, the ECU 10 determines whether or not spt exceeds the number of reflection points Ngn of the group gn (S35). When it is determined in S35 that spt is equal to or less than Ngn, the ECU 10 returns to S21 and performs processing for the updated division position spt. On the other hand, when it is determined in S35 that spt has exceeded Ngn, the ECU 10 ends the shape model estimation process for the group gn. Here, finally, when the parameter values are stored in the model parameters MP _{model, spt} at each division position spt, it is indicated that there is a vehicle shape model model applicable at the division position spt of the current group gn. The shape model can be specified by the parameter value.

  When the shape model estimation process for the group gn ends, the ECU 10 determines whether or not there is a vehicle shape model applicable to the current group gn (that is, whether or not a vehicle has been detected) (S4). If it is determined in S4 that there is no vehicle shape model applicable to the current group gn, the ECU 10 proceeds to S6 and adds 1 to gn to update the group gn (S6).

  When it is determined in S4 that there is a vehicle shape model that is applicable at a certain division position spt of the current group gn, the ECU 10 performs an on-image range estimation process for the reflection point group of the current group gn (S5).

When the process shifts to the on-image range estimation processing, first, processing is performed on the front and rear surfaces of the group gn. First, the ECU 10 initializes the front and rear left end point positions FRLPt _gn with the image width IMG_WIDTH and initializes the front and rear right end point positions FRRPt _gn with 0 (S40). Further, the ECU 10 initializes the variable i for updating the reflection points in the reflection point group belonging to the front surface / rear surface of the group gn with 0 (S41).

Each time the variable i (reflection point) is updated, the ECU 10 calculates the two-dimensional position Pimg _{gn, fr, i} on the image from the three-dimensional position of the reflection point rp _{gn, fr, i} by the expression (3) ( S42). Then, the ECU 10, the position _{Pimg gn} on the _image, determines _fr, lateral position _{u gn} in the _{i, fr, i} is whether the left end point position FRLPt _gn smaller (S43). When it is determined in S43 that the lateral position u _{gn, fr, i} is smaller than the left end point position FRLPt _gn , the ECU 10 updates the left end point position FRLPt _gn with the lateral position u _{gn, fr, i} (S44). Further, the ECU 10 determines whether or not the lateral position ugn _{, fr, i in} the position Pimg _{gn, sd, i} on the image is larger than the right end point position FRRPt _gn (S45). When it is determined in S45 that the lateral position u _{gn, fr, i} is larger than the right end point position FRRPt _gn , the ECU 10 updates the right end point position FRRPt _gn with the lateral position u _{gn, fr, i} (S46). The ECU 10 adds 1 to i in order to update the reflection point (S47). Then, the ECU 10 determines whether i is equal to or greater than the number Ngn _{fr of} reflection points belonging to the front and rear surfaces of the group gn (S48). When it is determined in S48 that i is less than the number of reflection points Ngn _fr of the group gn, the ECU 10 returns to the process of S42 and performs the process for the updated reflection point.

When it is determined in S48 that i is equal to or greater than the number Ngn _{fr of the} reflection points belonging to the front and rear surfaces of the group gn, the ECU 10 uses the three-dimensional positions of all the reflection points of the front and rear reflection points of the group gn. Then, the average position rpmean _{gn, fr} is calculated (S49). Further, the ECU 10, the formula (4), the equation (5), the average position _{Rpmean gn,} using _fr, calculates the lower end point position on the image of the front and rear surfaces of the group _gn FRVlow gn and the upper end point position FRVhigh _gn (S50).

Then, in the ECU 10, the left end point position FRLPt _gn = FRLPt _gn −dw, the right end point position FRRPt _gn = FRRPt _gn + dw, and the lower end point position as ranges on the front and rear images of the group gn to which the vehicle shape model is applied. FRVlow _gn and upper end point position FRVhigh _gn are stored (S51).

Next, processing is performed on the side surface of the group gn. First, the ECU 10 initializes the left end point position SDLPt _gn of the side surface with the image width IMG_WIDTH, and initializes the right end point position SDRPt _gn of the side surface with 0 (S60). Further, the ECU 10 initializes a variable i for updating the reflection points in the reflection point group belonging to the side surface of the group gn with 0 (S61).

Every time the variable i (reflection point) is updated, the ECU 10 calculates the two-dimensional position Pimg _{gn, sd, i} on the image from the three-dimensional position of the reflection point rp _{gn, sd, i} by the equation (6) ( S62). Then, the ECU 10 determines whether or not the lateral position u _{gn, sd, i in} the position Pimg _{gn, sd, i} on the image is smaller than the left end point position SDLPt _gn (S63). When it is determined in S63 that the lateral position u _{gn, sd, i} is smaller than the left end point position SDLPt _gn , the ECU 10 updates the left end point position SDLPt _gn with the lateral position u _{gn, sd, i} (S64). Further, the ECU 10, determines the position _{Pimg gn} on that _{image, sd,} lateral position _{u gn} in the _{i, sd, i} is whether greater than the rightmost point position SDRPt _gn (S65). When it is determined in S65 that the lateral position u _{gn, sd, i} is larger than the right end point position SDRPt _gn , the ECU 10 updates the right end point position SDRPt _gn with the lateral position u _{gn, sd, i} (S66). In the ECU 10, 1 is added to i in order to update the reflection point (S67). Then, the ECU 10 determines whether i is equal to or greater than the number Ngn _{sd of} reflection points belonging to the side surface of the group gn (S68). When it is determined in S68 that i is less than the number of reflection points Ngn _sd of the group gn, the ECU 10 returns to the process of S62 and performs the process for the updated reflection point.

When it is determined in S68 that i is equal to or more than the number Ngn _{sd of the} reflection points belonging to the side surface of the group gn, the ECU 10 uses the three-dimensional positions of all the reflection points of the reflection point group on the side surface of the group gn, The position rpmean _{gn, sd} is calculated (S69). Further, the ECU 10, the formula (7), the equation (8), the average position _{Rpmean gn,} with _sd, calculates the lower end point position SDVlow _gn and the upper end point position SDVhigh _gn on the image side of the group gn ( S70).

Then, in the ECU 10, the left end point position SDLPt _gn = SDLPt _gn −dw, the right end point position SDRPt _gn = SDRPt _gn + dw, and the lower end point position SDVlow _gn are the ranges on the side image of the group gn to which the vehicle shape model is applied. , The upper end point position SDVhigh _gn is stored (S71).

  When the front and rear image upper ranges and the side image ranges are estimated, the ECU 10 sets surface candidates including the front and rear image upper ranges and the side image upper ranges of the group gn.

  When the on-image determination estimation process ends, the ECU 10 adds 1 to gn in order to update the group gn (S6). Then, the ECU 10 determines whether or not gn is equal to or greater than the number of grouped groups GN (S7). When it is determined in S7 that gn is less than the number GN of groups, the ECU 10 returns to the process of S3 and performs the process for the updated group gn. On the other hand, when it is determined in S7 that gn is equal to or greater than the number of groups GN, the ECU 10 ends the processing in the vehicle position / surface candidate estimation unit 11.

When there is a group gn that applies to the shape model of the vehicle, the ECU 10 uses the image captured by the camera 4 as a process in the feature point extraction unit 12 for each pixel (u, v) within the estimated range on the image. , the vertical edge intensity? Iv _u, a _v calculated lateral edge strength DerutaIh _u, with calculating the _v by the equation (10) by equation (9) using the values _{I u, v} of each pixel, each Using the edge strengths ΔIh _{u, v} and ΔIv _{u, v} , the edge strength ΔI _{u, v} is calculated by the equation (11) and the edge direction θ _{u, v} is calculated by the equation (12). Thus, by generating an edge image, feature points having feature quantities due to edges are extracted.

  When feature points are extracted, as a process of the feature point three-dimensional position candidate estimation unit 13, the ECU 10 first initializes the surface candidate spt with 0 (S80).

  Every time the surface candidate spt is updated, the ECU 10 initializes the feature point f included in the surface candidate spt with 0 (S81).

Every time the feature point f is updated, the ECU 10 calculates the direction n _{f in} which the feature point exists using the position of the feature point f on the image by using the equation (15) (S82). Further, the ECU 10 calculates the intersection point between the direction n _f and the surface of the front / rear model or the surface of the side model, and determines the intersection point as the three-dimensional position (x _{f, spt} , y _{f, spt} , z) of the feature point f. _{f, spt} ) (S83). The ECU 10 adds 1 to f in order to update the feature point f (S84). Then, the ECU 10 determines whether f is equal to or greater than the number of feature points included in the surface candidate spt (S85). If it is determined in S85 that f is less than the number of feature points, the ECU 10 returns to the process of S82 and performs the process for the updated feature point f.

  When it is determined in S85 that f is equal to or greater than the number of feature points, the ECU 10 adds 1 to spt in order to update the surface candidate spt (S86). Then, the ECU 10 determines whether or not spt is equal to or greater than the number of face candidates (S87). When it is determined in S87 that spt is less than the number of face candidates, the ECU 10 returns to the process of S81 and performs the process for the updated face candidate spt.

  If it is determined in S87 that spt is equal to or greater than the number of surface candidates, the ECU 10 ends the process in the feature point three-dimensional position candidate estimation unit 13.

When the three-dimensional position of the feature point is estimated, the ECU 10 first _{initializes the} minimum position errors E _{sptt1 and sptt2} with THR_ERROR (for example, ∞) as a process in the vehicle end point extraction unit 14, and the surface candidate with the minimum error. STP1 and STP2 are initialized with 0 (S90). Further, the ECU 10 initializes the surface candidate spt _t2 at the current time _t2 with 0 (S91).

Every time the surface candidate spt _t2 at the current time t2 is updated, the ECU 10 initializes the surface candidate spt _t1 at the previous time _t1 with 0 (S92).

Every time the surface candidate spt _t1 at the previous time t1 is updated, the ECU 10 initializes the feature point f _t2 at the current time _t2 with 0 (S93).

Each time updating the feature point _{f t2} the current time t2, the ECU 10, is initialized with 0 feature points _{f t2} 'before the time t1 corresponding to the feature point _{f t2,} the minimum feature amount difference _{[Delta] F t1, t2} Initialization is performed with THR_FEATURE (for example, ∞) (S94). Further, the ECU 10 initializes the feature point f _t1 at the previous time _t1 with 0 (S95).

Every time the feature point f _t1 at the previous time t1 is updated, the ECU 10 calculates the difference ΔF ′ in the feature amount between the feature point f _t1 and the feature point f _{t2 according} to the equation (17) (S96). Then, the ECU 10 determines whether or not the feature amount difference ΔF ′ is smaller than the minimum feature amount difference ΔF _{t1, t2} (S97). When it is determined in S97 that the feature amount difference ΔF ′ is equal to or larger than the minimum feature amount difference ΔF _{t1, t2} , the ECU 10 proceeds to the processing of S100 and updates the feature point f _t1 at the previous time _t1 to update the feature point f _t1. 1 is added to (S100).

When it is determined in S97 that the feature amount difference ΔF ′ is smaller than the minimum feature amount difference ΔF _{t1, t2} , the ECU 10 sets the current feature amount difference ΔF ′ as the minimum feature amount difference ΔF _{t1, t2} (S98). ), set the current feature point _{f t1} as feature points _{f t2} 'before the time t1 corresponding to the feature point _{f t2} (S99). Then, the ECU 10 adds 1 to f _t1 in order to update the feature point f _t1 at the previous time t1 (S100). Further, the ECU 10, determines whether the number _{N ft1} or more feature points in the feature point _{f t1} the time t1 (S101). If S101 at _{f t1} is determined as the number _N less than _ft1 feature points at time t1, the ECU 10, returns to the processing of S96, performs processing for the feature point _{f t1} the updated.

When it is determined in S101 that f _t1 is _{equal to} or greater than the number N _ft1 of feature points at time t1, the feature point stored in f _t2 ′ is the feature point at the previous time t1 corresponding to the current feature point f _t2. is there. Then, the ECU 10, to update the feature point _{f t2} the current time _t2, and adds 1 to _{f t2} (S102). Further, the ECU 10, determines whether the number _{N ft @ 2} or more feature points in the feature point _{f t2} the time t2 (S103). If S103 at _{f t2} is determined as the number _N less than _ft2 feature points at time t2, the ECU 10, returns to the processing of S94, performs processing for the feature point _{f t2} the updated.

If S103 at _{f t2} is determined as the number _{N ft @ 2} or more feature points at time t2, since the processing for all the combinations of the feature point _{f t1} and the feature point _{f t2} is the finished, current time t2 The feature point f _t2 ′ (= f _t1 ) at the previous time t1 corresponding to all the feature points f _t2 is determined.

Then, the ECU 10, the RANSAC method, translation amount as the position error is minimized between feature points corresponding in the previous time t1 and the current time _{t2 (dx spt, dy spt,} dz spt) and the rotation amount [psi _spt Is estimated (S104). Further, the ECU 10, the translational movement amount of each feature point _{f t1} face candidate _{spt t1} before time _{t1 (dx spt, dy spt,} dz spt) on converted in the rotation amount [psi _spt, between the corresponding feature points The position error E is calculated (S105). Then, the ECU 10 determines whether or not the position error E is smaller than the minimum position errors E _{sptt1 and sptt2} (S106). If the position error E is determined to be _{equal to} or greater than the minimum position error E _{sptt1, sptt2 in} S106, the ECU 10 _{proceeds to} the processing of S109 and adds 1 to spt _t1 in order to update the surface candidate spt _f1 at the previous time t1. (S109).

When it is determined in S106 that the position error E is smaller than the minimum position errors E _{sptt1 and sptt2} , the ECU 10 _sets the current position error E as the minimum position errors E _{sptt1 and sptt2} (S107), and sets the current position candidate STP1 as the current position error STP1. The surface candidate stp _t1 is set, and the current surface candidate stp _t2 is set as the surface candidate STP2 (S108). Then, the ECU 10 adds 1 to spt _t1 in order to update the surface candidate spt _t1 at the previous time t1 (S109). Further, the ECU 10 determines whether or not the surface candidate spt _t1 is _{equal to} or greater than the number N of surface candidates at time t1 (S110). If it is determined in S110 that the surface candidate spt _t1 is less than the number of surface candidates N _sptt1 at time t1, the ECU 10 returns to the process of S93 and performs the process for the updated surface candidate spt _t1 .

When it is determined in S110 that the surface candidate spt _t1 is _{equal to} or _larger than the number of surface candidates N _sptt1 at time t1, the ECU 10 adds 1 to spt _t2 in order to update the surface candidate spt _t2 at the current time t2 (S111). ). Further, the ECU 10 determines whether or not the surface candidate spt _t2 is _{equal to} or greater than the number N of surface candidates at time t2 (S112). When it is determined in S112 that the surface candidate spt _t2 is less than the number of surface candidates N _sptt2 at time t2, the ECU 10 returns to the process of S92 and performs the process for the updated surface candidate spt _t2 .

If it is determined in S112 that the surface candidate spt _t2 is _{equal to} or greater than the number of surface candidates N _sptt2 at time t2, the processing for all combinations of the surface candidate spt _t2 and the surface candidate spt _t1 is completed, and is stored in STP1. The surface candidate is determined to be the surface of the vehicle at the previous time t1, and the surface candidate stored in the STP2 is determined to be the surface of the vehicle at the current time t2. Therefore, the ECU 10 extracts two reflection points rpt _{gn, SPT1} and rpt _{gn, SPT1 + 1} in the vicinity of the division position from the reflection point group of the group gn corresponding to the surface candidate SPT1 at the previous time t1, and the two reflections. The average value of the points is set as the end point (boundary position between the two surfaces) of the vehicle at time t1 (S113). Further, the ECU 10 extracts two reflection points rpt _{gn, SPT2} and rpt _{gn, SPT2 + 1} near the division position from the reflection point group of the group gn corresponding to the surface candidate SPT2 at the current time t2, and the two reflections. The average value of the points is set as the end point of the vehicle at time t2 (S113). Then, the ECU 10 transmits vehicle information including at least vehicle end point information as a vehicle information signal to the driving support device. Here, for example, only the end point of the vehicle at the current time t2 may be output, or the end points of the vehicle at the previous time t1 and the current time t2 may be output.

  According to the vehicle detection device 1, the minimum position error is obtained by translating and rotating the three-dimensional position between feature points corresponding to each combination of surface candidates between two consecutive times, and all combinations are obtained. By extracting a combination of surface candidates that minimizes the position error from the surface candidates, the surface of the vehicle can be detected with high accuracy. Furthermore, the end of the vehicle can be detected with high accuracy based on the surface of the vehicle.

  With reference to FIGS. 17-19, the vehicle detection apparatus 2 which concerns on 2nd Embodiment is demonstrated. FIG. 17 is a configuration diagram of a vehicle detection device according to the second embodiment. FIG. 18 is an example of the position error of the feature point between two times, (a) is a diagram showing the relationship between the position of the feature point on the image and the three-dimensional position at the previous time t1, (b) FIG. 6 is a diagram showing the relationship between the position of the feature point on the image and the three-dimensional position at the current time t2, and (c) is a diagram showing the position error of the three-dimensional position of the feature point between the previous time t1 and the current time t2. It is. FIG. 19 is an explanatory diagram of processing in the vehicle width estimation unit.

  Compared with the vehicle detection device 1 according to the first embodiment, the vehicle detection device 2 estimates the end points of both ends of the vehicle, and uses the end points of the vehicle extracted by the respective end points and the vehicle end point extraction unit 14. The only difference is the addition of a configuration for estimating the width of. For this purpose, the vehicle detection device 2 includes a laser radar 3, a camera 4, and an ECU 20. The ECU 20 includes a vehicle position / surface candidate estimation unit 11, a feature point extraction unit 12, a feature point three-dimensional position candidate estimation unit 13, and a vehicle. An end point extraction unit 14 and a vehicle width estimation unit 25 are configured. In the second embodiment, the vehicle end point extraction unit 14 and the vehicle width estimation unit 25 correspond to vehicle information detection means described in the claims.

  The ECU 20 is an electronic control unit including a CPU, a ROM, a RAM, and the like, and comprehensively controls the vehicle detection device 2. In the ECU 20, an application program stored in the ROM is loaded onto the RAM and executed by the CPU, whereby the vehicle position / surface candidate estimation unit 11, the feature point extraction unit 12, the feature point three-dimensional position candidate estimation unit 13, the vehicle The end point extraction unit 14 and the vehicle width estimation unit 25 are configured. The ECU 20 receives a radar signal from the laser radar 3 and an image signal from the camera 4 at predetermined time intervals. Then, the ECU 20 performs processing of each of the units 11, 12, 13, 14, 25 using these signals, and when the vehicle can be detected, transmits the vehicle information as a vehicle information signal to the driving support device. Here, only the process in the vehicle width estimation unit 25 will be described in detail.

  The vehicle width estimation unit 25 will be described. When the vehicle end point extraction unit 14 extracts the end point of the vehicle (the boundary position between the two surfaces), the vehicle width estimation unit 25 calculates the distance between corresponding feature points on the respective planes STP1 and STP2 of the vehicle estimated by the vehicle end point extraction unit 14. The position error of each of the surfaces of the feature points farthest from the vehicle end point (boundary position of the two surfaces) extracted by the vehicle end point extraction unit 14 among the feature points having a small position error is obtained. It is extracted as an end point, and the width of the vehicle is estimated based on the end point on the other side of each surface of the end point of the vehicle.

FIG. 18 shows an example of a position error at a feature point on the front surface of the vehicle and a position error at a feature point on an object other than the vehicle between the previous time t1 and the current time t2. FIG. 6A shows the case of the previous time t1, CP1 _t1 as a feature point of the sign B is a feature point on the image IP, and 3P1 _t1 ′ is a vehicle point estimated by the vehicle end point extraction unit 14 of the feature point CP1 _t1 . It is a three-dimensional position candidate on the plane STP1, 3P1 _t1 is an actual three-dimensional position of the feature point CP1 _t1 , CP2 _t1 is a feature point on the image IP as a feature point on the front surface of the vehicle V, and 3P2 _t1 'Is a three-dimensional position candidate on the vehicle surface STP1 of the feature point CP2 _t1 , and 3P2 _t1 is an actual three-dimensional position of the feature point CP2 _t1 . FIG. 6B shows the case of the current time t2, CP1 _t2 is a feature point on the image IP as a feature point of the sign B, and 3P1 _t2 ′ is a vehicle point estimated by the vehicle end point extraction unit 14 of the feature point CP1 _t2 . It is a three-dimensional position candidate on the plane STP2, 3P1 _t2 is an actual three-dimensional position of the feature point CP1 _t2 , CP2 _t2 is a feature point on the image IP as a feature point on the front surface of the vehicle V, and 3P2 _t2 'Is a three-dimensional position candidate of the feature point CP2 _{t2 on} the vehicle surface STP2, and 3P2 _t2 is an actual three-dimensional position of the feature point CP2 _t2 . Further, FIG. 3C shows the position error of the three-dimensional position candidate between the previous time t1 and the current time t2, and the position between the three-dimensional position candidates 3P1 _t1 ′ and 3P1 _t2 ′ for the feature point of the marker B An error L1 and a position error L2 between the three-dimensional position candidates 3P2 _t1 ′ and 3P2 _t2 ′ for the feature points of the vehicle V are shown. As can be seen from this example, in the case of the feature point of the vehicle, the difference in the position of each three-dimensional position candidate on the plane STP1, STP2 of the feature point on the image between two times t1, t2 is small. Correspondence of feature points is established between two times t1 and t2. However, in the case of an object other than the vehicle V (sign B), the difference in position of each three-dimensional position candidate on the vehicle planes STP1 and STP2 with respect to the feature points on the image between two times t1 and t2 is large. There is no correspondence between feature points between times t1 and t2. Therefore, by removing the feature points that increase the positional error between the two times t1 and t2 from the feature point group, only the feature points of the vehicle remain.

In FIG. 19, the outline | summary of the process (thing with respect to the front surface of a vehicle) in the vehicle width estimation part 25 is shown. Feature points extracted from the image I of the current time t2 (3-dimensional position _{candidates) 3P1 t2 ', 3P2 t2'} , ···, 3P12 t2 ', shows an example of ..., in this example, feature Points 3P1 _t2 ′, 3P9 _t2 ′, and 3P12 _t2 ′ are feature points of an object other than the vehicle. For each feature point, the position error E of the three-dimensional position candidate of the corresponding feature point between the previous time t1 and the current time t2 is calculated, and the position error E is set to the horizontal position u on the image as the horizontal axis. This is shown in the graph G1. The As can be seen from the Gulag G1, feature points of an object other than a vehicle _{3P1 t2} ', _{3P9 t2',} since the position error is greater than a predetermined threshold value THR_ERROR_FP for 3P12 _t2 ', to eliminate this feature point. For only the remaining feature points, the number of feature points is counted for each horizontal position u (in units of several pixels, not in units of one pixel), and the count value (frequency EHist _u ) is counted in the horizontal position u (several pixels). A graph G2 is shown with the unit) as the horizontal axis. As can be seen from this graph G2, in the vehicle end U _edge extracted by the vehicle end point extraction unit 14, the frequency EHist _u exceeds the predetermined threshold value THR_ERROR_FPCNT, and otherwise, the lateral position u where the frequency EHist _u exceeds the threshold value THR_ERROR_FPCNT. the lateral position u _L farthest from the end U _edge of the vehicle and the end of the other side of the front of the vehicle of.

Specifically, in the vehicle width estimation unit 25, for each feature point f _t2 ′, f _t2 corresponding to the surface candidate STP2 at the previous time t1 and the surface candidate STP2 at the current time t2, the feature points f _t2 ′, f _t2 translation amount as the position error is minimized between _{(dx spt, dy spt, dz} spt) and using a rotation amount [psi _spt, feature points _{f t2} 'translation amount _{(= f t1) (dx spt} , dy _spt, on converted by _{dz spt)} and the rotation amount [psi _spt, corresponding feature points _{f t2 ',} calculates the position error _{E ft @ 2} between _{f t2.} The position error E _ft2 can be calculated by the equation (20). X in equation (20) can be calculated by equation (21). Note that the translation amount (dx _spt , dy _spt , dz _spt ) and the rotation amount ψ _spt between the corresponding feature points f _t2 ′ and f _t2 in Expression (21) have already been obtained by the vehicle end point extraction unit 14. ing.

Determining for each to calculate the position error _{E ft @ 2,} in the vehicle width estimator 25, _{f t2} 'of the feature _point, position error _{E ft @ 2} between _{f t2} is whether the threshold THR_ERROR_FP smaller. The threshold value THR_ERROR_FP is a threshold value for determining whether the feature point is a feature point of the vehicle or a feature point other than the vehicle based on the positional error E _ft2 between the feature points. When the position error E _ft2 is smaller than the threshold value THR_ERROR_FP, the vehicle width estimation unit 25 counts up the frequency EHist _u of the lateral position u (lateral position every several pixels) of the feature point.

When the processing between all corresponding feature points f _t2 ′ and f _t2 is completed, the vehicle width estimation unit 25 determines whether or not the threshold value THR_ERROR_FPCNT is exceeded from the frequency EHist _u at each lateral position u. Threshold THR_ERROR_FPCNT is based on the frequency EHist _u, is a threshold for determining whether there is a predetermined number or more of characteristic points as end points of the vehicle.

The vehicle width estimation unit 25 extracts a lateral position u farthest from the vehicle end point (boundary position of the two surfaces) extracted by the vehicle end point extraction unit 14 from the lateral positions u whose frequency EHist _u exceeds the threshold value THR_ERROR_FPCNT. The lateral position u is defined as the end point on the other side of the vehicle surface.

  In the vehicle width estimation unit 25, the above processing is performed separately for the front surface / rear surface and the side surface of the vehicle surfaces STP1 and STP2 extracted by the vehicle end point extraction unit 14. Each end point is estimated. The vehicle width estimation unit 25 calculates the vehicle width based on the distance between the endpoints using the estimated endpoints on the other side of the front and rear surfaces, the endpoints on the other side of the side surface, and the vehicle endpoints extracted by the vehicle endpoint extraction unit 14. calculate. In the case of the distance between the end point on the other side of the front and rear surfaces and the end point on the other side of the side surface, it is the width of the entire vehicle shown in the image. In the case of the distance between the end points on the other side of the front surface / rear surface and the end points of the vehicle extracted by the vehicle end point extraction unit 14, this is the width of the front surface of the vehicle shown in the image. In the case of the distance between the end point on the other side of the side surface and the end point of the vehicle extracted by the vehicle end point extracting unit 14, it is the length of the side surface of the vehicle shown in the image.

  The operation of the vehicle detection device 2 will be described. Since the operation of the vehicle detection device 2 is different from the operation of the vehicle detection device 1 according to the first embodiment, only the processing in the vehicle width estimation unit 25 is performed after the vehicle end point extraction unit 14 ends. Only the process in the vehicle width estimation unit 25 in the ECU 20 will be described. The processing of the vehicle width estimation unit 25 in the ECU 20 will be described along the flowchart of FIG. FIG. 20 is a flowchart showing the flow of processing in the vehicle width estimation unit.

As the processing of the vehicle-width estimating section 25, the ECU 20, first, the feature points _{f t2} the current time t2 initialized to 0 (S120), is initialized to zero frequency EHist _u at each lateral position.

Every time the feature point f _t2 is updated, the ECU 20 translates the translation amount (dx) for the feature point f _t2 ′ in the surface candidate SPT1 at the previous time t1 corresponding to the feature point f _t2 in the surface candidate STP2 at the current time t2. The position error E _ft2 between the feature points f _t2 and f _t2 ′ is calculated after conversion using _spt , dy _spt , dz _spt ) and the rotation amount ψ _spt dy _spt , dz _spt ) (S121). Then, the ECU 20 determines whether or not the position error E _ft2 is smaller than the threshold value THR_ERROR_FP (S122). Position if the error _{E ft @ 2} determines the threshold THR_ERROR_FP smaller at S122, the ECU 20, the frequency EHist _u with respect to the lateral position u on the image feature points _{f t2} counted up by 1 (S123).

In ECU 20, to update the feature point _{f t2} the current time _t2, and adds 1 to _{f t2} (S124). Further determines the ECU 20, whether the number _{N ft @ 2} or more feature points in the feature point _{f t2} the time t2 (S125). If S125 at _{f t2} is determined as the number _N less than _ft2 feature points at time t2, the ECU 20, the process returns to the S121, performs processing for the feature point _{f t2} the updated.

If S125 at _{f t2} is determined as the number _{N ft @ 2} or more feature points at time t2, so that the processing for all of the corresponding feature point _{f t2, f t2} 'between the ends. Therefore, the ECU 20 sets the lateral position u farthest from the vehicle end point (boundary position of the two surfaces) in the lateral position u where the frequency EHist _u exceeds the threshold value THR_ERROR_FPCNT as the end point on the other side of the vehicle surface (S126). ). Then, the ECU 20 estimates the width of the vehicle based on the difference between the end point on the other side of the surface of the vehicle and the end point (boundary position of the two surfaces) of the vehicle (S127).

  The ECU 20 performs the above processing separately for the front / rear and side surfaces, estimates the other end point of the front / rear surface and the other end point of the side surface, and estimates the width of the vehicle. Then, the ECU 20 transmits vehicle information including at least information on the surface of the vehicle, each end point of the vehicle, and the width of the vehicle as a vehicle information signal to the driving support device.

  According to this vehicle detection device 2, in addition to the same effects as the vehicle detection device 1 according to the first embodiment, the following effects are also obtained. According to the vehicle detection device 2, the remaining end point on the other side of each surface of the vehicle can be estimated with high accuracy by using the position error between corresponding feature points between times t1 and t2. The width of the vehicle can be estimated with high accuracy from the end point of the vehicle.

  As mentioned above, although embodiment which concerns on this invention was described, this invention is implemented in various forms, without being limited to the said embodiment.

  For example, in the present embodiment, the present invention is applied to a vehicle detection device mounted on a vehicle, but the present invention can also be applied to other devices such as a driving support device such as a collision prevention device and an inter-vehicle distance control device, and a periphery monitoring device.

  In the present embodiment, the present invention is applied to the case of detecting a vehicle in front of the host vehicle.

  In the present embodiment, the radar information of the laser radar is used to detect the vehicle and estimate the surface candidate of the vehicle and the range on the image of the vehicle. However, the radar of another radar such as a millimeter wave radar is used. Information may be used, or estimation may be performed by other methods such as pattern recognition using an image from a camera.

  In this embodiment, an edge is applied as a predetermined feature amount (feature point). However, other feature amounts such as a wavelet and SIFT may be used.

  In the present embodiment, the configuration is such that the vehicle end point is obtained based on the estimated vehicle surface, but only the vehicle surface may be estimated.

  Further, in the present embodiment, processing is performed for two consecutive times, but it may be performed for three or four times, or may not be performed between consecutive times when the period between the times is short.

  DESCRIPTION OF SYMBOLS 1, 2 ... Vehicle detection apparatus, 3 ... Laser radar, 4 ... Camera, 10, 20 ... ECU, 11 ... Vehicle position and surface candidate estimation part, 12 ... Feature point extraction part, 13 ... Feature point three-dimensional position candidate estimation part , 14 ... Vehicle end point extraction unit, 25 ... Vehicle width estimation unit

Claims (3)

A vehicle detection device that detects vehicle information from an image captured by an imaging means,
A surface candidate estimation means for estimating a vehicle surface candidate;
Feature point extraction means for extracting a plurality of feature points having a predetermined feature amount from an image of the vehicle imaged by the imaging means;
3D position estimation means for estimating a 3D position of the feature points extracted by the feature point extraction means;
For each combination of surface candidates estimated from images captured at different times by the imaging means, search for a correspondence relationship between feature points for which a feature amount difference between the plurality of feature points is small, and the search A translational and rotational transformation of a three-dimensional position between a plurality of feature points of the corresponding correspondence, a position error is obtained, a surface candidate of a combination that minimizes the position error is extracted, and a surface candidate of the extracted combination Vehicle information detecting means for detecting the surface of the vehicle based on the vehicle detection device.   The vehicle detection device according to claim 1, wherein the vehicle information detection unit detects an end point of the vehicle based on the detected surface.   The vehicle information detection means detects the width of the surface of the vehicle based on a region where a feature point having a positional error of a three-dimensional position between corresponding feature points within a threshold is present among the extracted combination surface candidates. The vehicle detection device according to claim 1, wherein the vehicle detection device is characterized by the following.

Images