JPH0248706A - automatic driving device - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION TECHNICAL FIELD The present invention relates to an automatic driving device that allows a vehicle to automatically travel while searching for a driving route.

Conventionally, in this type of automatic driving device, dedicated guidelines are set in advance to guide the vehicle's movement on the road surface, and the device is mounted on the vehicle so that it can image the road surface in front of the vehicle. While following the guidelines with a video camera,
The steering control of the vehicle is made to drive the vehicle in accordance with the guidelines (Special Publications Publication No. 58-
42482 and JP-A-62-140109).

However, such conventional automatic driving devices cannot guide the running of a vehicle on a road surface where guidelines are not provided.

Conventionally, for example, in a lawn mowing vehicle, a video camera mounted on the vehicle captures an image of the area in the vehicle's direction of travel, emphasizing the difference in brightness between the working area and the unworked area, and providing continuous edge information. The boundary between the work area and the unworked area is detected by appropriately processing the image information of the image taken to obtain the information, and the steering control between the vehicles is performed so that the vehicle travels along the detected boundary. (Refer to Japanese Unexamined Patent Publication No. 70916/1983).

However, unlike the above-mentioned automatic driving devices, although they have the function of searching for driving guidelines on their own and driving along those lines, they are essentially the same as the above-mentioned ones. The guidance is simply made to follow a single guideline.

1”y The present invention has been made in consideration of the above points.

While searching for a road by capturing an image in front of the vehicle using an imaging device, the vehicle is controlled to travel on the searched road, and the vehicle can be driven at intersections, straight roads, winding roads, etc. To provide an automatic driving device which can appropriately switch an imaging device between wide-angle and telephoto when the vehicle approaches a road, and appropriately image the road in front of the vehicle according to the road conditions.

(2) Bottom In order to achieve the objective, the automatic driving device according to the present invention includes a means for recognizing the road in front of the vehicle by image information processing of an image obtained by capturing an image in front of the vehicle with an imaging device attached to the vehicle, and a means for recognizing the road in front of the vehicle. means for controlling the running of a vehicle so that the vehicle travels on a road that has been A means for detecting that the vehicle is approaching a road having predetermined road characteristics and outputting the information, and a means for appropriately switching the imaging characteristics of the imaging device to wide-angle, telephoto, or vertical in accordance with the output information. That's what I do.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 shows the basic configuration of an automatic driving system according to the present invention, and shows a video camera 1 attached to a vehicle so as to be able to image an area in the direction of travel between vehicles, and an image taken by the video camera 1. The driveable area recognition unit 2 recognizes the driveable area, such as a road, in the traveling direction of the vehicle, and the vehicle is located within the driveable area recognized by the driveable area recognition unit 2. Target route setting section 3 that sets a target route for travel
and a vehicle speed sensor 5 for detecting the traveling speed V of the vehicle. Yaw rate (angular velocity of change in one direction) T as the vehicle travels
The current running state of the vehicle is recognized based on the sensor outputs of the yaw rate sensor 6, which detects the tire angle δ, and the tire angle sensor 7, which detects the tire angle δ caused by steering the vehicle. a control unit 4 that calculates a control target amount necessary for the vehicle to travel on a target route by a predetermined calculation process; a steering control unit 8 and a steering drive that steer the vehicle in accordance with the determined control target amount; 9. Actually, the driveable area recognition unit 2. The target route setting section 3 and the control section 4 are replaced by a computer control device. It is also possible to include the steering control section 8 within the computer control device.

FIG. 2 shows a specific example of the configuration of the travelable area recognition section 2.

Here, under the control of the CPU 211 or the CPU 212, one input image each sequentially sent from the video camera 1 is once alternately stored in the buffer memory 231 or 232 through the image input section 22. Thereafter, each input image stored in the buffer memory 231 or 232 is sequentially read out, and the image processing unit 24 extracts, for example, road edges as described later to determine the vehicle driveable area. Image processing for recognition is performed. Next, the image information of the driveable area recognized through the image processing is stored in a plurality of memories 251.
- 253 or the image memory group 26 including a plurality of memories 261 to 263 are sequentially updated and stored in the image memory group 25 or the image memory group 26. Image information of continuous driveable areas is accumulated as the vehicle travels a certain distance. Note that the image information of the drivable area stored in the image memory group 25 or the image memory group 26 is read out and given to the target route setting section 3.

Here, by providing two CPUs 211 and 212 and two buffer memories 231 and 232 in parallel, for example, the input image stored earlier is read out from the buffer memory 231 under the control of one CPU 211. In the meantime, the next input image is stored in the papier memory 232 under the control of the other CPU 212, so that input processing of the input image can be performed in real time.

Similarly, by providing two image memory groups 25, 26 in parallel, it is possible to perform the process of writing or reading image information alternately in each image memory group 25, 26 in real time. ing.

In FIG. 2, 27 indicates each memory select section, and 28 indicates each decoding section. In addition, 5-BUS is the system bus, and D-BtJS is the high-speed data bus.
S indicates a high-speed memory access bus, respectively.

In the vehicle configured as described above, recognition of the drivable area of the vehicle by the drivable area recognition section 2 is performed as follows.

First, an input image sent from a video camera 1 that captures an image of the road surface in front of the vehicle is subjected to differential processing to detect edges in the image. An automatic threshold setting circuit automatically sets an optimum threshold according to the degree of shading of the input image at that time, and binarizes the edge image.

In this case, the input image may be binarized first, and then differential processing for edge detection may be performed. Alternatively, instead of binarizing, the shading of one image may be It may also be possible to perform multivalue representation.

Next, based on the edge-detected, binarized or multivalued processed image, the line segment on the X-Y coordinates is expressed as ρ-θ.
By performing Hough transformation processing, which is a known method of performing coordinate transformation on points on the coordinates, continuous point sequences can be combined and isolated points with no continuity can be removed.

For example, information on continuous road edges as shown in FIG. 3 is obtained.

Here, θ is the angle of a perpendicular drawn from a straight line on the XY coordinate to the origin of the coordinate, and ρ is the length of the perpendicular. For example, a line segment on the X-Y coordinates shown in FIG. 12 appears as point 01 on the ρ-θ coordinates as shown in FIG. 13.

At this time, edge tracking processing may be performed based on the binarized processed image to determine continuous edge portions of the image.

In addition, if multiple processes such as Hough conversion processing and edge tracking processing to determine the continuity of image edges are performed in parallel, and a comprehensive judgment is made from the results of each process, more accurate road You will be able to request information about Etsuji.

Furthermore, by performing the above-mentioned processing for extracting continuous image edges while growing the region of the input image as the vehicle travels, it is possible to extract road edge information with higher accuracy. Become.

Finally, since the image captured by the video camera 1 is based on perspective projection, the image of the road edge using perspective projection as shown in Figure 3 is changed to the image of the road edge as shown in Figure 4, which eliminates the effects of perspective projection. Projective transformation processing, which is a known method for converting images of road edges, is performed, and the area between consecutive road edges is recognized as a driveable area for a vehicle.

Note that the projective transformation characteristics are set in advance in the driveable area recognition unit 2 according to the perspective projection characteristics of the video camera l.

Next, when the drivable area recognition unit 2 recognizes a road that is a drivable area in the direction of travel of the vehicle, the target route setting unit 3 determines a target road that is the optimum driving route for the vehicle within the recognized road. The route is set.

As will be explained later, it is desirable that the target route be set in a way that is suitable for the current inter-vehicle driving conditions, taking into consideration the road shape and vehicle speed. It is uniformly set as follows depending on whether it is narrow or wide.

That is, when the target route setting unit 3 determines that the road is a wide gauge road with a width of a certain level or more, for example, as shown in FIG. For example, a target route ○C along the reference edge is set with a predetermined separation width W of about 1.5 m.

In addition, if the target route setting unit 3 determines that the road width is narrow gauge with less than a certain level, although not particularly shown in the figure,
Set a target route in the center of that road.

Then, the position on the x-y coordinates of the set target route is stored in the internal memory of the target route setting section 3 while being updated successively as the vehicle travels. The scale on the +xy coordinates is then determined by the magnification of the video camera 1.

In FIG. 5, point P indicates the current position of the vehicle. For example, the mounting position of the video camera 1 in the vehicle is set in advance so that the lower center of the imaging surface of the video camera 1 is the point P. ing. In the figure, as will be described later, the trajectory from point P to point 0 is determined by steering control of the vehicle under the control of the control unit 4 until the vehicle at point P merges into the target amount 1oc. The driving route is shown. The 0 point is the merging position of the vehicle to the target route ○C at that time.

Further, in the present invention, it is also possible to detect the driving state of the vehicle and set an optimal target route for the vehicle on the road in accordance with the detected driving state as described below.

That is, the target route setting unit 3 reads the traveling speed of the vehicle detected by the vehicle speed sensor 5, and if the vehicle speed at that time is within a low speed range below a preset threshold value, the target route setting unit 3 reads the traveling speed of the vehicle detected by the vehicle speed sensor 5, and if the vehicle speed at that time is within a low speed range below a preset threshold value, the vehicle speed is determined as shown in FIG. As shown in a), the target route ○C is set so as to follow the shape of the road.

Similarly, if the vehicle speed at that time is within a high speed range exceeding a preset threshold, as shown in Figure 6(b), when driving on a winding road, the lateral force acting on the vehicle will be A target route OC with a gentle curvature that reduces acceleration as much as possible is set within the road.

Next, once the target route on the road is set, the control unit 4 calculates a control target amount for causing the vehicle to join the target route by the following calculation process.

Now, for example, as shown in FIG. 7, the vehicle 13 at point P
Target amount! ! Let's consider the case of merging into 80C.

First, based on the current vehicle speed v (m/s) of the vehicle detected by the vehicle speed sensor 5, the distance L (m) (L=vXT) of the vehicle at point P on the X axis after T seconds is determined. , the deviation yQ between the target route Oc and six points on the X-axis that are separated by a distance of 111tL from the P point, that is, the target route after T seconds.

A value proportional to the position on C is calculated.

Similarly, the predicted route AC of the vehicle is calculated based on the yaw rate T (rad/s) of the vehicle detected by the yaw rate sensor 6, and the deviation ym of the predicted route AC from the 6 points on the X axis, that is, after T seconds. Predicted path A in
A value proportional to the position on C is obtained according to the following formula.

ym=(-vXT''/2)XT-(1) Note that the sign of the yaw rate T is 1, for example, when the predicted route AC curves to the left, it is positive.

Then, the yaw rate T' of the vehicle to be corrected is determined according to the difference between the determined deviations yΩ and yIll according to the following formula.

T' = (ym-y(riX2/ (vXT2)-(2
)Finally, the tire angle δ of the vehicle at point P detected by the tire angle sensor 7 is taken in, and the target tire angle control amount δ' for merging the vehicle onto the target route oC is determined according to the following formula. be done.

δ'=δ+(T'/v)XW・(1+Kv2)-(3
) Here, W is the wheelbase, and K is a constant coefficient determined by vehicle characteristics such as tire characteristics and wheelbase.

Then, the steering control section 8 issues a drive command to the steering drive section 9 according to the control target amount δ' given from the control section 4, so that the steering drive section 9 appropriately drives the steering wheel to move the vehicle along the target path. Perform steering to merge into OC.

In addition, in the relationship shown in FIG. 7, when setting the distance L on the X axis (target merging distance), under the control of the control unit 4,
The set distance can be made variable depending on the traveling speed V of the vehicle detected by the vehicle speed sensor 5. In other words, the lower the traveling speed V is, the shorter the distance is set, so that the traveling route for the vehicle at point P to merge with the target route OC becomes shorter, and the vehicle merges onto the target route ○C. be carried out promptly. In addition, the higher the traveling speed V, the longer the distance is set, so that the traveling route for the vehicle at point P to merge with the target route OC becomes longer, so that the vehicle merges onto the target route ○C. Do it slowly.

Furthermore, when driving on a winding road, the smaller the curvature, the shorter the distance, and the target route oC of the vehicle is set.
It is also possible to have the vehicle join quickly.

Also, from point P when merging the vehicle to the target route oC,
Using the curve model of the travel route up to the point, a travel route with a predetermined curve pattern is determined by approximate analysis according to the route curvature determined by the control unit 4 according to the target merging distance and vehicle speed V. It is also possible to use the determined travel route pattern to smoothly merge into the target route OC.As a curve model of the travel route, for example, the function y=
x-sinx or the function y=x3, etc. are used.

FIG. 8 shows a travel route pattern using a travel route curve model based on the function y=x-sinx.

The above processing is repeated at predetermined time intervals on the order of several seconds, thereby making it possible for the vehicle to automatically travel along the target route oC.

In this way, the automatic driving device according to the present invention searches for a drivable area by itself by recognizing road edges based on an image of the road surface in front of the vehicle taken by an imaging device, and then locates an appropriate target within the drivable area. Set the route,
The optimum control target quantity for merging the vehicle onto the target route can be determined according to the vehicle running condition at that time, and the vehicle running control can be performed with high precision according to the control target quantity. ing.

At this time, a two-step route generation process is used, in which a target route is set within the drivable area of the vehicle, and then a travel route is devised for the vehicle to join the target route. Automatic driving can be performed with high precision.

In the automatic traveling device configured as described above, in particular, in the present invention, a telephoto camera 11 and a wide-angle camera 12 are provided as the video camera 1, and the video camera 1 is provided with a telephoto camera 11 and a wide-angle camera 12. intersection,
The telephoto camera 11 and the wide-angle camera 12 can be selectively switched under the control of the drivable area recognition unit 2 in accordance with information RD indicating road characteristics such as a straight road or a winding road. It is characterized by

As will be described later, the navigation system detects changes in the vehicle's travel distance and direction of travel while sequentially determining the position on the X-Y coordinates through arithmetic processing. It is designed to display the current location. The navigation system then sets a planned driving route on the road map displayed on the screen in advance, and uses information RD to indicate intersections, straight roads, winding roads, etc. on the planned driving route.
When it is detected that the vehicle is approaching the designated intersection etc. from the vehicle's current position displayed on the road map on one screen, the predetermined information RD is inputted in advance. It is now output.

At that time, once the starting point and destination point on the map are input and specified, the computer processing in the navigation system automatically sets, for example, the shortest travel route from the starting point to the destination point. You can also do it.

For example, when the drivable area recognition unit 2 receives information RD indicating an intersection from the navigation system, it switches to the wide-angle camera 12 in response to the information RD, and performs a wide search for the specified intersection. Further, when the drivable area recognition unit 2 receives the information RD indicating a straight road, it switches to the telephoto camera 11 to optimally capture an image of the straight road looking ahead. Furthermore, when the drivable area recognition unit 2 receives the information RD indicating a winding road, it switches to the wide-angle camera 12 to optimally image the winding road.

Note that it is possible to provide the telephoto camera 11 with a zoom function, so that the zoom function can be activated as appropriate depending on the length of a straight line on a straight road, for example.

FIG. 9 shows a configuration example of a navigation system, for example, a photoelectric type that outputs a pulse signal for each unit travel distance according to the rotation of the vehicle's tires.

A distance sensor 91 using an electromagnetic type or a mechanical contact type,
For example, the distance traveled by the vehicle is measured by counting the number of pulse signals from a yaw rate sensor 92, such as a gyroscope, which outputs a signal proportional to the amount of change in angular velocity in one direction as the vehicle travels, and a distance sensor 91. At the same time, the change in the traveling direction of the vehicle is determined according to the output signal of the yaw rate sensor 92, and the position on the X-Y coordinates for each unit travel distance of the vehicle is determined by sequential calculation.
In addition, there is a signal processing device (computer control device) 93 consisting of a CPU, program ROM, control RAM, etc. that performs centralized control of the entire system, and information on the ever-changing X-Y coordinates determined by the signal processing device 93. A travel trajectory storage device 94 that sequentially stores position data and holds it as finite continuous position information corresponding to the current position of the vehicle, and map information that stores a plurality of pre-digitized map information in file units. a storage medium 95;
A storage medium playback device 96 selectively reads out necessary map information from the storage medium 95, displays a map surface image on the screen according to the read map information, and displays a map image stored in the travel trajectory storage device 94. The current position of the vehicle based on the position data! .

and a display device 97 that updates and displays the travel trajectory up to that point, the current direction of travel, etc. on the same screen every moment.

In addition to giving operation commands to the signal processing device 93, the display device 97 selects and specifies the map to be displayed, sets the starting point of the vehicle on the displayed map, and also specifies the planned travel route on the displayed map. Along with setting,
It is comprised of an operating device 98 into which information RD indicating road characteristics such as intersections, straight roads, winding roads, etc. on the planned travel route can be input in advance.

With this configuration, as shown in FIG. 10, the map selectively read out is displayed on the screen of the display device 97, and the vehicle can be seen from the starting point set on the map. Signal processing device as you drive! 93
A display mark Ml indicating the current position of the vehicle on the X-Y coordinates according to a map scale rate preset by , a display mark M2 indicating the direction of travel of the vehicle at the current position, and a display mark M2 indicating the direction of travel of the vehicle at the current position, and a starting point S to the current position. The driving trajectory display mark M3 is displayed in a simulative manner following the driving state of the vehicle.

Also, when setting the planned travel route on the map displayed on the 5 screen by inputting the operation device 98.

For example, as shown in FIG. 11, a target point a is located at a major intersection or branch point on the planned travel route on the map.

bl cl... are set, and instructions such as turning right or left can be given at each target point.

Then, the signal processing device 93 1, for example, when the display mark Ml of the vehicle's current position is a certain distance before the target point a [D], the signal processing device 93 performs a message such as "Go ahead at the Y-junction to the right". Navigation information is emitted to the driver through, for example, a voice generator (not shown), and information RD indicating a Y-junction is provided to the drivable area recognition unit 2.

Furthermore, as described above, map information can be obtained without specifying an intersection on the planned travel route by inputting the operating device 98 and inputting the designation information RD of the road characteristics of the specified intersection or the like. Designation information RD of the location of an intersection, etc. on the map and its road characteristics is registered in the storage medium 95 in advance together with map information, and under the control of the signal processing device 93, a predetermined intersection on the map displayed on the screen is registered. For example, when a mark indicating the current position of the vehicle arrives, instruction information RD of road characteristics registered in advance according to the position may be provided to the drivable area recognition unit 2.

In addition, in the navigation system, the distance sensor 9
1, the travel distance of the vehicle can be determined based on the output of the vehicle speed sensor 5 in the automatic traveling system shown in FIG. Furthermore, the yaw rate sensor 6 in the automatic traveling system shown in FIG. 1 can be used in combination without providing a separate yaw rate sensor 92.

As described above, the automatic driving device according to the present invention can search for the road itself to drive on based on an image taken by the imaging device of the area in the direction of travel of the vehicle, and in this case, in particular, intersections, straight roads,
When approaching a winding road, use a wide-angle or telephoto camera.

By switching the standard as appropriate to capture images of intersections, straight roads, winding roads, etc., it is possible to optimize road imaging according to the road conditions, and use the captured images to image intersections, straight roads, winding roads, etc. It has the excellent advantage of being able to appropriately recognize the following.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment of an automatic traveling device according to the present invention, FIG. 2 is a block diagram showing a specific configuration example of a travelable area recognition section in the same embodiment, and FIG. 3 is a video camera. Figure 4 is a diagram showing an example of a road recognized from an image taken by A diagram showing an example of a route. Figures 6 (a) and (b) are diagrams showing each target route set on the road when the vehicle is running at low speed and at high speed, respectively. Figure 7 is a diagram showing the target route and the predicted route of the vehicle. FIG. 8 is a diagram showing an example of the vehicle travel route when merging with the target route, FIG. 9 is a block diagram showing an example of the configuration of the navigation system, and FIG. FIG. 11 is a diagram showing an example of a display screen, FIG. 11 is a diagram showing the input state of navigation information on the planned travel route in the navigation system, FIG. 12 is a diagram showing line segments on the X-Y coordinates, and FIG. 13 is a diagram showing points on the ρ-θ coordinates when the line segment in FIG. 12 is roughly transformed. FIG.

Claims (1)

[Scope of Claims] 1. A means for recognizing a road in front of a vehicle by image information processing of an image obtained by capturing an image in front of the vehicle with an imaging device attached to the vehicle, and a means for driving on the recognized road. means for controlling the running of the vehicle; means for setting instruction information of road characteristics on the driving route; and means for detecting that the vehicle approaches a road having predetermined road characteristics on the driving route and transmitting the information. An automatic traveling device comprising: means for outputting the information; and means for switching the imaging characteristics of the imaging device according to the outputted information. 2. The first road characteristic is characterized in that the road characteristic is an intersection, a straight road, a winding road, etc. on the driving route.
Automated traveling device as described in Section. 3. The automatic driving device according to item 1 above, wherein the imaging characteristics are wide-angle, telephoto, and standard.