JPH08285534A - Vehicle-mounted image processing device - Google Patents

Abstract

PURPOSE: To improve the reliability of an image processing result by detecting, for example, the deviation, etc., of the installation direction of an image pick-up means during driving. CONSTITUTION: A vehicle-mounted image processing device is provided with a plurality of image pick-up means 52 and 54 for picking up the images of the front of a vehicle, and an image processing means 10 for performing each kind of image processing such as the recognition processing of a preceding vehicle based on each image data from a plurality of image pick-up means 52 and 54. The image processing means 10 is provided with a part 12 for calculating the distance between white lines for outputting the information on distance between white lines by calculating the distance between a plurality of white lines when a plurality of lateral white lines are recognized by a white line recognition part 68, thus comparing the calculated information on distance between white line with the predetermined information on distance between white lines. It is also provided with a part 18 for judging that the image pick-up direction of the image pick-up means 52 and 54 is abnormal when the comparison result differs from a fixed value, and a part 20 for alarming about abnormal image pick-up direction for outputting an alarm signal for reporting a failure to a display means 58.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle-mounted image processing apparatus, and more particularly to a vehicle-mounted image processing apparatus for determining an abnormality in the installation direction of image pickup means.

[0002]

2. Description of the Related Art Conventionally, there is an on-vehicle image processing apparatus which calculates a distance between a preceding vehicle and the like on the basis of the principle of triangulation using two cameras. Such an in-vehicle image processing device, as shown in FIG. 7, includes a first camera 52, a second camera 54, and image data 6 from the plurality of cameras.
An image processing unit 66 that performs image processing on the second and the second vehicles 64 and performs a recognition process of the preceding vehicle and the like, and a display unit 58 that outputs the determination result of the image processing unit 66 are displayed.

The image processing means 66 issues a preceding vehicle approach warning by outputting a warning about approaching a preceding vehicle (see FIG. 8) by various vehicle signals 56 from a brake sensor and a vehicle speed sensor. Therefore, the image processing means 66 uses the first camera 5
The white line recognition unit 68 for extracting the white line portion in the image data 62 from the second camera and the image data 64 from the second camera 54.
And a distance measuring means 70 for measuring the distance to the object based on the image data 62 and 64.

Distance measurement using the principle of triangulation is disclosed, for example, in Japanese Patent Laid-Open No. 3-245011. The white line recognition process is disclosed in, for example, Japanese Patent Application Laid-Open No. 4-137014.

Next, distance measurement using the principle of triangulation will be described. As shown in the positional relationship in FIGS. 9 and 10, the camera 5 placed in parallel with a distance d.
The image data 62 and 64 obtained from 2, 54 are shown in FIG.
As shown in (a) and (b). Further, when the preceding vehicle portions 52a and 54a in the image data 62 and 64 are combined, the image shown in FIG. 11C is obtained. FIG. 11 (c)
As shown in FIG.
The amount of deviation (Δx ₁ [dots]) between the positions of a and the preceding vehicle 54a by the second camera 54 on the image is obtained. The distance L from this deviation amount Δx ₁ to the preceding vehicle is expressed by the following equation (1).

[0006]

[Equation 1]

Here, Sx is a conversion coefficient of dots in the x-axis direction of the image and meters (m) and represents the number of dots per 1 [m]. f is the focal length of the camera lens, τ is the dip angle with respect to the horizontal axis of the camera, and H is the height of the camera from the ground.

This distance measuring principle is based on the premise that the positions of the cameras are parallel to each other at a predetermined distance d as shown in the figure.

If this condition is not satisfied, an incorrect distance will be calculated. As shown in FIG. 12, the first camera 52
13 is not parallel to the second camera 54 and is tilted to the left, the image data 62 of the first camera 52 is
As shown in FIG. 13A, the image data 64 obtained by the second camera is as shown in FIG. 13B.

An image obtained by combining the image data 62 and 64 shown in FIGS. 13A and 13C is shown in FIG.
As shown in (c), the positional deviation Δx ₂ on the image data of the preceding vehicle 60 becomes larger than that in the case shown in FIG. 11 (c). Therefore, the distance L to the preceding vehicle 60 is as shown by the following equation (2).

[0011]

[Equation 2]

The result of calculation by the equation (2) is shorter than the actual distance. f = 25 [mm], Sx = 8 × 10
Assuming that ⁴ [dots / m], d = 1 [m], and τ = 0, the case of L = 50 [m] will be described as an example.

When L = 50 [m], Δx ₁ = 40 [dots] when the camera is in a normal position.

Here, when the camera 1 is displaced once to the left, if the same L = 50 [m], Δx2≈75 [dots], and if the distance L is calculated from this value, L≈26.
It becomes 7 [m]. That is, one camera is displaced once, so that the distance of 50 [m] is actually about 26.7 [m].
Will be calculated.

[0015]

Therefore, in the conventional example,
When the installation direction of the camera is deviated, there is an inconvenience that the distance to the preceding vehicle cannot be accurately calculated.

Further, there is no method for checking whether or not this camera position is at the correct position. In particular, there is a problem in that when a deviation in the installation direction of the camera occurs while the vehicle is traveling, it cannot be detected.

[0017]

It is an object of the present invention to improve the problems of the conventional example, and more particularly, to improve the reliability of the image processing result by detecting the deviation of the installation direction of the image pickup means during traveling. It is an object of the present invention to provide an image processing device.

[0018]

In view of the above, according to the present invention, a plurality of image pickup means for picking up images of the front of the vehicle and various image processing such as a preceding vehicle recognition processing based on the image data from the plurality of image pickup means. And an image processing unit for displaying the processing result of the image processing unit, and the image processing unit includes a white line recognition unit for extracting a white line portion from the image data, and a target based on each image data. Distance measuring means for measuring the distance to the object.

Moreover, the image processing means calculates the distance between the plurality of white lines when the white line recognition section recognizes a plurality of horizontal white lines and outputs the white line distance calculation section. And comparing the calculated distance information between white lines with predetermined distance information between white lines, and if they differ by a certain value or more, an image pickup direction abnormality determination unit that determines that the image pickup direction of the image pickup means is abnormal, A configuration is provided in which an imaging direction abnormality alarm unit that outputs an alarm signal indicating an abnormality to a display unit is provided. This aims to achieve the above-mentioned object.

[0020]

During operation of the vehicle-mounted image processing apparatus, the plurality of image pickup means respectively image the front of the vehicle. Next, the image processing means performs various image processing such as recognition processing of the preceding vehicle based on the image data from the plurality of image pickup means. Further, the display means displays the processing result of this image processing means.

In the image processing means, the white line recognition section extracts the white line portion from each image data. At this time, the white line distance calculation unit calculates the distance between the white lines when a plurality of horizontal white lines are recognized, and outputs the white line distance information. Upon receiving this distance information between white lines, the image capturing direction abnormality determination unit compares the calculated distance information between white lines with predetermined distance information between white lines, and if they differ by a predetermined value or more, the image capturing means captures an image. It is determined that the direction is abnormal. Further, the imaging direction abnormality alarm unit outputs an alarm signal notifying the abnormality to the display means.

In addition, when the image processing means has a traveling road determination unit for determining the type of traveling road, when the traveling road determination unit determines that the vehicle is traveling on a highway. The white line recognition process in the horizontal direction and the inter-white line distance calculation process are performed. Then, on the highway, the white line in the lateral direction is almost only the inter-vehicle distance confirmation line, so this inter-vehicle distance confirmation line is recognized. In this case, since the distance between the inter-vehicle distance confirmation lines is 50 [m], by comparing the inter-white line distance calculated by the inter-white line distance calculating means with this 50 [m], the installation direction of the imaging means can be allowed. It is judged whether or not it exceeds a certain range.

[0023]

An embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an in-vehicle image processing apparatus according to the present invention. The in-vehicle image processing device includes a plurality of image pickup means 52 for picking up images of the front of the vehicle.
54, an image processing means 10 for performing various image processing such as recognition processing of a preceding vehicle based on the image data 62, 64 from the plurality of image pickup means 52, 54, and a processing result of the image processing means 10. Display means 58 for

Moreover, the image processing means 10 extracts the white line portion w from the image data 52 and 54, and the white line recognition unit 68.
And a distance measuring means 70 for measuring the distance to the object based on each image data.

Furthermore, the image processing means 10 calculates the distance between the plurality of white lines w1 and w2 when the white line recognition section 68 recognizes a plurality of horizontal white lines w, and the inter-white line distance information 14 is calculated. The distance between white lines calculating unit 12 that outputs the distance between the white lines and the calculated distance information between white lines 14 and the predetermined distance information 16 between white lines are compared, and if they differ by a predetermined value or more, the image pickup means 52 and 54 capture images. An imaging direction abnormality determination unit 18 that determines that the direction is abnormal, and an imaging direction abnormality alarm unit 20 that outputs an alarm signal that notifies the abnormality to the display unit 58 are provided.

This will be described in detail. Here, the same configurations as those of the related art will be denoted by the same reference numerals, and the description will be applied correspondingly.

The distance measuring means 70 obtains the distances l ₁ and l ₂ from the vehicle 50 for the _two lateral white lines w ₁ and w ₂ . The distance between white lines calculation unit 14 obtains the distance between the white lines w ₁ and w _{2 from} the respective distances l ₁ and l ₂ .

The image pickup direction abnormality determining unit 18 compares the actual distance 16 between the _two horizontal white lines w ₁ and w _{2 with} the distance calculated by the white line distance calculating unit 12 to obtain an image. A defect in the installation position of the camera 52, 54, which is a means, is detected. Therefore, the distance between the white lines w ₁ and w ₂ is stored in the imaging direction abnormality determination unit 18 in advance. This white line w
₁ and w ₂ may be provided, for example, on a course that the vehicle travels at the time of shipping, or a white line in the lateral direction provided on a general road whose distance is known is used. You may do it.

Since this can be carried out if the distance between two points on the road in the traveling direction of the vehicle is determined, the distance in the traveling direction is known if the accuracy of the white line recognition is very high. A vertical white line may be used.

FIG. 2 shows a white line w on the highway for checking the inter-vehicle distance. The white line w is composed of three lines that are parallel to the road in the lateral direction, and the interval is 50 [m]. Also, on expressways, the white line drawn in the lateral direction is only about this inter-vehicle distance confirmation line. In other words, if the white line in the lateral direction is recognized, it is called the inter-vehicle distance confirmation line. Therefore, in this embodiment, the line for checking the inter-vehicle distance is imaged and the image processing is performed to determine the abnormality in the imaging direction.

When the white line recognition section 68 recognizes the white lines w ₁ and w ₂ on the expressway, the distance measuring means 70 triangulates the distances (l ₁ , l ₂ ) between the first and second white lines. The white line distance calculation unit 12 calculates the difference (l
₂ −l ₁ ) is calculated. The distance between these lines is 50.
Since it is already determined to be [m], it is the difference in the calculated distance (l
_2- l ₁ ) should be 50 [m]. Therefore, the imaging direction abnormality determination unit 18 determines that the distance obtained here is 50 [m].
If not, it is determined that some trouble such as displacement of the cameras 52 and 54 has occurred. In this case, in this embodiment, the distance, which is the original purpose, cannot be measured to properly issue an alarm or some control. Therefore, an alarm is output to notify the driver of the occurrence of the defect.

FIG. 3 shows a method for measuring the distance of the white line based on the principle of triangulation. FIG. 3A shows image data 62 extracted by recognizing the white line w by the first camera 52, and FIG. 3B shows similar image data 64 of the second camera 54. In the image data 62 and 64, the first camera 52 corresponds to the distance d between the cameras 52 and 54.
The white line 32 of the second camera and the white line 34 of the second camera are different.

FIG. 4 is a composite of the image data 62 and 64 shown in FIGS. 3A and 3B, and it can be seen that the same white line w is at a slightly displaced position. In this 1
The deviation amount Δx between the white line 32 formed by the first camera 52 and the white line 34 formed by the second camera may be calculated for each of the two portions.

FIG. 5 is an enlarged view of a part of the combined image data shown in FIG. Here, the distance measuring means 70 measures the reference for measuring the amount of deviation at the center of gravity of the white line portion. By this method, the amount of deviation Δx ₁ .. of the images of the cameras 52 and 54 on the _first white line w ₁ and the second white line w ₂ . We are seeking Δx ₂ . Furthermore, the above-mentioned formula (1)
To each white line (l
₁ , l ₂ ).

Further, the distance between white lines calculation unit 12 is the distance from this vehicle to the white lines w ₁ and w ₂ (l ₁ , l ₂ ).
From this, the distance (l ₂ −l ₁ ) 14 between the white lines is calculated. Next, the imaging direction abnormality determination unit 18 determines the distance 14 between the white lines.
And the predetermined distance 16 between the white lines are compared to detect an abnormality in the imaging direction.

Since the distance 14 between the white lines is calculated by measuring the deviation on the image, the accuracy is in pixel units. As can be seen from the equation (1), the distance 1 and the shift amount Δx are inversely proportional, and in the process of checking whether (l ₂ −l ₁ ) is 50 [m], (l ₂ , l ₁ ) is (30 [m] , 80
The accuracy differs between [m]) and (100 [m], 150 [m]). For example, in the example shown in FIGS. 9 and 10, about 3.2 per pixel in the vicinity of L = 80 [m].
[M], which is about 12.9 [m] per pixel near L = 150 [m].

[0038] Accordingly, where it said that because either (l ₂ -l ₁₎ is 50 [m], in its _{(l 2 -l 1), (} l 2 -l 1) is 50 [m ] Is the number of pixels corresponding to that. Therefore, the white line distance calculating means 12 calculates the number of pixels corresponding to 50 [m] determined by l ₁ . Furthermore, the imaging direction abnormality determination unit 18
Determines the abnormality in the imaging direction by comparing the number of pixels corresponding to 50 [m] determined by l ₁ with the number of pixels at the distance 14 between white lines.

Further, in the present embodiment, this abnormality detection processing of the image pickup direction is performed only during traveling on the highway. this is,
This is because it is difficult to know the distance between the white lines in the lateral direction other than the expressway in advance. Therefore, the image processing means 10 includes a traveling road determination unit 22 that determines whether or not the traveling road is an expressway.

It is possible to judge that the vehicle is traveling on an expressway when the vehicle speed above a certain level continues for a certain period of time, or the current position of the own vehicle and the map information by the navigation system or the like. The high-speed traveling determination signal 22a may be output when the position is determined to be on the highway. Alternatively, it may be determined that the vehicle is traveling on an expressway based on a road beacon or a signal from a tollgate.

Further, the image processing means 10 is provided with a white line distance calculation start-up unit 24 for starting the white line distance calculation unit 12 when receiving the high speed running judgment signal 22a from the running road judging unit 22. That is, except when the vehicle is traveling on a highway, the process of calculating the distance of the horizontal white line w and the process of detecting an abnormality in the imaging direction, which are the characteristic processes of this embodiment, are not performed.
However, when the vehicle travels on a specific course or the like and the distance between the lateral white lines is predetermined, this abnormality detection processing is performed.

Next, the operation of this embodiment will be described. FIG. 6 is a flowchart showing an example of the processing steps of this embodiment.

First, the traveling road determination unit 22 determines whether or not the road currently traveling is an expressway (step S
1). If you are driving on a highway, the distance between white lines calculation unit 1
Start up 2. On the other hand, if you are not driving on the highway,
The originally intended control is performed (step S8).

The white line recognition section 68 recognizes the image data 62 and 64 obtained by the first camera 52 and the second camera 54, respectively (step S2). Then, it is confirmed whether or not a plurality of lateral white lines w are recognized by this processing (step S3), and when they are recognized, the distance measuring means 70 determines that the distance l _{1 of the} recognized first white line w ₁ Second white line w ₂
And the distance l ₂ are calculated (steps S4 and S5). If not recognized (step S3), the originally intended control is performed (step S8).

Further, the white line distance calculating section 12 calculates the distance 14 from the white line w ₂ to the white line w ₁ . further,
The imaging direction abnormality determination unit 18 checks whether this distance is near 50 [m]. If it is in the vicinity of 50 [m], the image processing apparatus is in a normal state, and therefore the process proceeds to step S8. On the other hand, if the allowable range from 50 [m] is exceeded, there is some defect such as a defective camera position.
0 is activated and an alarm is output to the display means 58. Furthermore, the control intended as the original purpose is stopped (step S
9).

As described above, according to this embodiment, in the distance measuring system based on the principle of triangulation using two cameras, it is difficult to confirm the system state (camera position, etc.) during traveling. However, this method makes it possible to check even while driving.

The optical axis of the camera can be checked by image processing without actually measuring the position of the camera with a precise measuring device.

In addition, in a system that performs image processing under the control of the original purpose, no special hardware is required to add the check operation of the present proposal, and a part for obtaining a distance by recognizing a white line is obtained. This can be done by adding software.

[0049]

Since the present invention is constructed and functions as described above, according to this, when the plurality of white lines in the horizontal direction are recognized, the distance between the white lines is calculated. Then, the distance information between white lines is output, and the imaging direction abnormality determination unit compares the calculated distance information between white lines with predetermined distance information between white lines, and if the difference is equal to or more than a predetermined value, the image pickup means. Since it is determined that the image capturing direction is abnormal, the image capturing direction abnormality can be detected even while the vehicle is traveling. Further, since the imaging direction abnormality warning unit outputs an alarm signal notifying the abnormality to the display means, it is possible to inform the user of the image processing result that the image processing result is based on inaccurate image data. As described above, it is possible to provide an unprecedented in-vehicle image processing device capable of enhancing the reliability of the image processing result by detecting the deviation of the installation direction of the image pickup means during traveling.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing an example of an inter-vehicle distance confirmation line on a highway.

FIG. 3 is a diagram showing an example of image data when an inter-vehicle distance confirmation line shown in FIG. 2 is imaged by the image capturing means shown in FIG. 1, and FIG. 3 (a) shows an image captured by the first camera. FIG. 3B is a diagram showing an image captured by the second camera.

FIG. 4 is an explanatory diagram showing an example of an image obtained by combining the images shown in FIG.

FIG. 5 is an explanatory diagram in which the white line of the first camera and the white line of the second camera shown in FIG. 4 are partially enlarged.

FIG. 6 is a flow chart showing processing steps by the configuration shown in FIG.

FIG. 7 is an explanatory diagram showing a configuration of a conventional vehicle-mounted image processing device.

8 is an explanatory diagram showing an example of a preceding vehicle recognition process with the configuration shown in FIG. 7. FIG.

9 is an explanatory diagram showing a positional relationship between the two cameras shown in FIG. 7. FIG.

10 is an explanatory diagram showing an imaging range based on the positional relationship of the cameras shown in FIG.

11 is an explanatory diagram showing an image in which a preceding vehicle is imaged in the positional relationship of the cameras shown in FIG. 10, and FIG. 11A is a diagram showing image data obtained by the first camera.
11B is a diagram showing image data obtained by the second camera, and FIG. 11C is an explanatory diagram showing an image obtained by combining the image data obtained by the two cameras.

FIG. 12 is an explanatory diagram showing an imaging range when the first camera is displaced leftward.

13 is an explanatory diagram showing an image in which a preceding vehicle is imaged in the positional relationship of the cameras shown in FIG. 12, and FIG. 13 (a) is a diagram showing image data obtained by the first camera.
FIG. 13B is a diagram showing image data obtained by the second camera, and FIG. 13C is an explanatory diagram showing an image obtained by combining the image data obtained by the two cameras.

[Explanation of symbols]

 10 Image Processing Means 12 White Line Distance Calculation Unit 14 White Line Distance Information 16 Predetermined White Line Distance 18 Imaging Direction Abnormality Determination Unit 20 Imaging Direction Abnormality Warning Unit 22 Running Road Determination Unit 50 Own Vehicle 52 First Camera 54 2 camera 58 warning means (display means) 68 white line recognition section 70 distance measuring means

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G06T 7/00 H04N 5/225 C G08B 21/00 7/18 J H04N 5/225 G06F 15/62 380 7/18 415 15/64 M

Claims (3)

[Claims] 1. A plurality of image pickup means for picking up images in front of a vehicle, an image processing means for performing various image processing such as recognition processing of a preceding vehicle based on image data from the plurality of image pickup means, and the image processing. And a display unit for displaying the processing result of the unit, the image processing unit, the white line recognition unit for extracting the white line portion from the image data, and the measurement for measuring the distance to the object based on each image data. In the in-vehicle image processing device including a distance unit, the image processing unit calculates a distance between the plurality of white lines when the white line recognition unit recognizes a plurality of horizontal white lines, and calculates the distance between the white lines. The distance between white lines calculation unit that outputs distance information, and the calculated distance information between white lines and predetermined distance information between white lines are compared, and if they differ by a certain value or more, the image pickup direction of the image pickup means is changed. The imaging direction abnormality determining unit determines that the normal, in-vehicle image processing apparatus characterized by comprising an imaging direction alarming unit for outputting an alarm signal indicating the abnormality on the display means. 2. A highway traveling section, wherein the image processing means determines whether or not the road on which the vehicle is traveling is a highway, determines that the highway is traveling, and outputs a highway traveling determination signal. The in-vehicle image processing device according to claim 1, further comprising: a white line distance calculation start unit that starts the white line distance calculation unit when a determination signal is received. 3. The image processing means has a function of stopping various image processing such as the preceding vehicle recognition processing when an alarm signal is output from the imaging direction abnormality warning unit. Item 1. The vehicle-mounted image processing device according to Item 1 or 2.