JPH03273315A - Picture processor for moving vehicle - Google Patents

Abstract

PURPOSE:To exhibit respective advantages of an ocellus viewing part and a compound eyes viewing part by providing these view parts and selectively switching one of the input pictures due to the ocellus view and that due to the compound eyes view. CONSTITUTION:The processor 3b of a stereo viewing part 3 detects positions and feature quantities of two pictures inputted from cameras 1L and 1R and sends them as the information of a mobile object to the processor 4b of an ocellus viewing part (ocellus view and path planning) 4. The processor 4b of the ocellus viewing part 4 processes the picture from the camera 1R. When the processor 3b sends the information relating to the mobile object to the processor 4b, the processor 4b discriminates the mobile object as an obstacle if this mobile object recognized by the processor 3b is present on the running course recognized by the processor 4b itself. A running plan to avoid the obsta cle is prepared, and a controller 5 is controlled in accordance with this plan.

Description

DETAILED DESCRIPTION OF THE INVENTION (Industrial Application Field) The present invention relates to an image processing device for a mobile vehicle that performs autonomous driving control, and in particular, it is equipped with an image input system having a compound eye and a monocular, The present invention relates to an image processing device for a moving vehicle that recognizes obstacles and the like.

(Conventional technology) Obstacle recognition in conventional autonomous vehicle travel control
Distance sensors such as ultrasonic sensor lasers are used.

These sensors generally have the disadvantage that although they can measure distances in a certain direction, they cannot measure over a wide range. In order to supplement such a distance sensor, an obstacle recognition device equipped with a stereo camera and inputting images to perform environment recognition has been proposed, for example, in Japanese Patent Application Laid-Open No. 1983-1999.
No. 26913. This patent application uses two TV cameras to measure distances over a wide field of view based on the principle of triangulation.

(Problem to be solved by the invention) However, in image processing using stereo viewing, a huge amount of processing is required to match left and right images.
This has been an obstacle to practical application when performing environmental recognition in real time.

Therefore, the present invention has been proposed in order to eliminate the drawbacks of the above-mentioned conventional examples, and its purpose is to use either an input image obtained by binocular vision or an input image obtained by monocular vision, so as to bring out the advantages of each. An image processing device for a moving vehicle that performs image processing by selectively switching is proposed.

(Means and operations for achieving the object) The configuration of the present invention for achieving the above object is a mobile vehicle equipped with an image input means having a plurality of image pickup means for inputting images for external world recognition. In the image processing device, the plurality of imaging means include a monocular viewing section that performs imaging by encircling and a compound eye viewing section using compound eyes, and a recognition means that performs external world recognition based on the image input by the image inputting means. The present invention is characterized by comprising a switching selection means for switching and selecting one of the image inputted by the monocular viewing section and the image inputted by the compound eye viewing section, and outputting the selected input to the recognition means.

That is, the selection means selectively switches between an input image obtained by binocular vision and an input image obtained by monocular vision so as to bring out the advantages of each.

(Embodiment) An embodiment in which the image processing device of the present invention is applied to autonomous driving of a moving vehicle will be described below with reference to the accompanying drawings.

FIG. 1 is a block diagram showing the configuration of this embodiment. This embodiment device includes two left and right cameras IL and IR, a stereo naked section 3 that recognizes obstacles through stereo vision,
The image from the right camera 1R is input to recognize the driving route, and the monocular vision unit 4 creates a travel plan based on the vehicle's own weight based on information on obstacles recognized by the stereo naked unit 3, and controls the steering and brakes. It consists of a vehicle control controller 5.

The stereo bare section 3 consists of an image memory 3a and a processor 3b. Two images input from the camera 1L and IR are input to the image memory 3a. The processor 3b analyzes these two images using a stereoscopic technique,
A distance image in the moving direction of the moving object is generated. Furthermore, from this distance image, we estimate the area that has a height relative to the road surface, and
The position and features I (eg, color, size) of the area are calculated. These positions and feature amounts represent a moving object.

The processor 3b sends these positions and feature amounts to the processor 4b of the monocular viewing unit as information about the moving object.

On the other hand, the monocular viewing unit 4 includes an image memory 4a and a processor 4b. The image from the right camera IR is stored in the image memory 4a.
and processed by the processor 4b. That is,
The processor 4b performs binarization processing, labeling processing, etc. on the image of the image memo J4a independently of the processor 3b of the stereo naked section 3, detects a tangled edge, and recognizes a running route.

When the processor 3b of the stereo naked section 3 sends the above information regarding the moving object to the processor 4b of the monocular viewing section 4, the processor 4b determines whether there is a moving object recognized by the processor 3b within the travel route recognized by itself. If so, it is determined that it is an obstacle. Then, a travel plan is formulated to avoid the obstacle, and the controller 5 is controlled according to the plan.

The control cycle of the processor 3b is 1. Each time interval. That is, recognition of a moving object is performed every t2 interval. On the other hand, the control cycle of the processor 4b is every t2 time interval. That is, the travel plan is created at intervals of t2.

Here, it is set that t + > t 2 . In other words, image processing for recognizing a moving object is performed in long time cycles, and image processing for driving control is performed in short time cycles. This was done for the following reasons. In other words, it is not necessary to recognize that an area is an obstacle, that is, to recognize whether or not a certain area has a vertical component in a three-dimensional space, for all the images that are inputted moment by moment. , may be performed at a certain long sampling interval, since it is unlikely that an obstacle will suddenly disappear. This is because if the obstacle is tracked with a certain degree of roughness, it is unlikely that the obstacle will suddenly disappear from the field of view. In other words, this obstacle tracking processing does not need to be performed with the precision and frequency required for obstacle recognition, and low-level image processing is sufficient. In other words, once the location of an obstacle is recognized through high-level image processing, the vehicle can then track the obstacle using low-level image processing and control the vehicle to avoid the obstacle. Bye.

FIG. 2A and FIG. 2B show control procedures for the processors 3b and 4b, respectively. The control procedure of the processor 3b shown in FIG. 2A is executed every time interval t1. That is, in step S2, the images of the cameras IR and IL are input to the memory 3a, and in step S4, a passing process is performed to extract information regarding obstacles. This operation is repeated every 51 hours.

In the control procedure of the processor 4b shown in FIG. 2B, in step SIO, the passage of time tz (<t+) is checked. If this time has not elapsed, step S
In step 12, it is checked whether information regarding the mobile object has been input from the processor 3b. If not, the process returns to step S10 and waits for t2 to elapse.

In step SL2, if it is determined that information regarding the mobile object has been input from the processor 3b, the mobile object information stored in the memory (not shown) of the processor 4b is updated with the newly sent information, In step S16, the image from the right camera IR is input into the image memory 4a. on the other hand,
If it is determined in step S10 that 12 hours have passed, the image from the right camera IR is input into the image memory 4a in step S16. That is, the image from the right camera IR is input to the image memory 4a either after 12 hours have elapsed or when the moving object is newly recognized by the processor 3b.

In step S18, low-level image processing is performed according to the contents of the image memory 4a to recognize the travel route. In step S20, a bus plan is created to avoid obstacles,
In step S22, vehicle travel control is performed.

According to the image processing device of the embodiment described above, when performing autonomous driving control, there is no need for stereo viewing processing for all input stereo images, and the system as a whole You can continue to recognize the driving route.

The present invention can be modified in various ways without departing from the spirit thereof.

In the above embodiment, the stereo wedge section is configured to inform the monocular viewing section of the area where the obstacle is present, but the following modification is proposed. That is, by estimating the area where an obstacle is likely to be located (for example, an edge image found in a place other than the white line) from the recognition result of the monocular viewing unit and transmitting it to the stereo wedge unit, the stereo wedge unit can It is possible to narrow down the locations where there is a high probability of an obstacle being present, and the recognition of obstacles can be made faster.

In addition, although the above embodiments have been explained with emphasis on recognizing obstacles, the present invention is not limited thereto. For example, the present invention can be applied to tracking a preceding vehicle when the vehicle is following the preceding vehicle. It is possible by applying

Further, in the above embodiment, parallel control is performed by two processors, but this can be replaced by processing by one image processing processor.

(Effects of the Invention) As explained above, the configuration of the image processing device of the present invention is applicable to an image processing device for a moving vehicle equipped with an image input means having a plurality of image pickup means for inputting images for external world recognition. , the plurality of imaging means include a monocular viewing unit that performs imaging in a single shot and a compound eye viewing unit that uses compound eyes, and a recognition unit that performs external world recognition based on the image input by the image inputting unit; The present invention is characterized by comprising a switching selection means for switching and selecting one of the image inputted by the section and the image inputted by the double silver bare section and outputting the selected input to the recognition means.

That is, the selection means selectively switches between an input image obtained by binocular vision and an input image obtained by monocular vision so as to bring out the advantages of each. For example, when three-dimensional recognition is required, images obtained with binocular vision are processed, and when such precision is not required, images obtained with monocular vision are processed. It has become. Therefore, as a whole, the throughput of the system increases by speeding up image processing using monocular vision.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the configuration of a preferred embodiment to which the present invention is applied, and FIGS. 2A and 2B are flowcharts showing the control procedure of this embodiment. Part, 4a... Image memory, 4b... Processor, 5
...Vehicle control controller. In the figure, IL, IR...camera, 3...stereo wedge part, 3a
...Image memory, 3b...Processor, 4...Monocular vision Fig. 2A, Fig. 2B

Claims (1)

[Claims] (1) In an image processing device for a moving vehicle equipped with an image input means having a plurality of image pickup means for inputting images for external world recognition, the plurality of image pickup means include a monocular viewing unit that performs image pickup with a single eye; a recognition means comprising a compound eye viewing section using compound eyes and performing external world recognition based on the image input by the image input means; An image processing device for a mobile vehicle, comprising a switching selection means for switching and selecting an input and outputting the selected input to the recognition means.