JP2012156903A - Vehicle periphery monitoring device - Google Patents

Abstract

A vehicle periphery monitoring device capable of detecting an obstacle or the like suitable for the situation of the vehicle in a short time is provided.
In a vehicle periphery monitoring device that detects an obstacle or the like around a vehicle, the vehicle behavior information of the vehicle is acquired from an imaging unit 4 that is mounted on the vehicle and captures an image of the periphery of the vehicle, and a sensor included in the vehicle. An information input unit 7, an image processing unit 6 that processes an image signal output from the imaging unit 4 to detect an obstacle or the like, and a memory that stores behavior information and a preset target area in association with each other. The image processing unit 6 selects a target area in the captured image corresponding to the behavior information acquired by the information input unit 7 and detects an obstacle or the like in the target area. The configuration is to be performed.
[Selection] Figure 1

Description

  The present invention relates to a vehicle periphery monitoring device that monitors a vehicle periphery, and more specifically, a vehicle periphery that detects an obstacle or the like using a captured image of the vehicle periphery captured by an imaging module mounted on the vehicle. It relates to a monitoring device.

Conventionally, in order to monitor the surroundings of a vehicle, for example, to grasp an obstacle such as another vehicle or a person, the driver has confirmed visually or directly through a mirror. However, at positions that are difficult to see from the driver, such as on the side of the vehicle or behind the vehicle, there is a high possibility that an obstacle will be misidentified or overlooked, and it has been difficult to ensure safety by visual observation alone.
Therefore, in recent years, an apparatus that detects obstacles around the vehicle by various sensors and notifies the driver is proposed and put into practical use.

  As a related technique, Patent Document 1 (Japanese Patent No. 3891537) discloses a vehicle side surface monitoring device. This device includes a monitoring camera device that is mounted in the vicinity of a driver seat and captures a peripheral image including a part of a vehicle side surface, a radar device that detects the distance of an obstacle, and a display device disposed in the vicinity of the driver seat And an alarm device, and displays an image captured by the monitoring camera device on the display device and issues a warning by the alarm device according to the distance to the obstacle detected by the radar device. As a result, obstacles such as side-by-side vehicles on large vehicles and the like are monitored, and it is possible to reduce the burden of safety confirmation of dryness and improve safety when driving, changing lanes and turning.

  Japanese Patent Laid-Open No. 2008-165393 discloses an in-vehicle obstacle detection device that detects an obstacle around a vehicle using a captured image. This apparatus includes an imaging unit that captures an image around the vehicle, and a detection unit that performs scanning to detect an obstacle by pattern matching processing from the near side to the far side of the vehicle. Thereby, it becomes possible to detect obstacles, such as a pedestrian with high possibility of collision with a vehicle, in a shorter time.

Japanese Patent No. 3891537 JP 2008-165393 A

  However, in the device disclosed in Patent Document 1, since the radar device has strong directivity, it is difficult to detect an obstacle or the like in an urban area or the like, and there is a possibility that erroneous detection or oversight occurs. Also, obstacle detection by a radar device is difficult to detect an object having a low relative speed such as a parallel vehicle. Furthermore, having both the surveillance camera device and the radar device has a problem that the device is increased in size and cost.

On the other hand, as disclosed in Patent Document 2 and the like, when detecting an obstacle or the like from a captured image using image recognition technology, it takes a long time to scan if the entire range of the captured image is set as a detection target area. In addition, it is difficult to notify the driver of the presence of an obstacle or the like in real time. In addition, scanning an image from near to far in detecting an obstacle is suitable for detecting a pedestrian, but for example, there are no pedestrians on an expressway, so it is inappropriate. In addition, it was not possible to detect obstacles according to the scene flexibly corresponding to the situation of the vehicle.
Therefore, an object of the present invention is to provide a vehicle periphery monitoring apparatus that can detect an appropriate obstacle or the like according to the situation of the vehicle in a short time.

  In order to solve the above problems, a vehicle periphery monitoring device according to the present invention is a vehicle periphery monitoring device that detects an obstacle or the like around the vehicle, and an imaging unit that is mounted on the vehicle and captures an image of the vehicle periphery, An information input unit that acquires behavior information of the vehicle from a sensor included in the vehicle, an image processing unit that detects an obstacle or the like by processing an image signal output from the imaging unit, and the behavior information A storage unit associated with a preset target area, and the image processing unit based on the behavior information acquired by the information input unit, the target in a captured image corresponding thereto An area is selected, and the obstacle or the like is detected in the target area.

According to the present invention, the obstacle is not detected in the entire area of the captured image, and the obstacle is detected only in a partial target area in the captured image. It is possible to reduce the time required for image processing for detecting the obstacle and to detect an obstacle or the like at high speed.
In addition, since the target area is selected based on the behavior information of the vehicle, it is possible to appropriately detect an obstacle or the like according to the situation of the vehicle.
Obstacles are obstacles that should avoid collisions such as dynamic obstacles such as other vehicles and people, static obstacles such as guide rails and telephone poles, etc. It includes objects (including lines displayed on the road surface) that are required when traveling such as lines and when parking.

The behavior information is preferably information related to at least one of start, stop, travel direction, travel speed, acceleration / deceleration, and bending of the vehicle.
Since all the behavior information described above is information that can be detected from sensors included in the vehicle, it is possible to accurately determine the situation of the vehicle in real time and to select a more appropriate target area. Become.

Further, it is preferable that a plurality of notification means for notifying the detection result of the obstacle or the like is provided, and the detection result notification means is selected based on the target area selected by the image processing unit.
In this way, by adopting a configuration in which the notification means is selected from the plurality of notification means based on the target area, it is possible to provide necessary information for the driver with the necessary means.

Furthermore, it is preferable that the notifying unit includes at least one of an image superimposing unit that superimposes and displays the vehicle course prediction guideline on the captured image, and a warning output unit that outputs a warning.
Thus, the driver can easily grasp the detection result by notifying the driver of the detection result of the obstacle or the like using at least one of the image superimposing means and the warning output means.

Furthermore, the image processing unit includes warning output means for outputting a warning, calculates a predicted course of the vehicle from the behavior information of the vehicle, and uses the predicted path to contact the obstacle or the like. It is preferable to estimate approach and output a warning signal by the warning output means.
As a result, it is possible to warn the driver of the possibility of contact with or approaching an obstacle at an early stage, and to further improve driving safety.

According to the present invention, the obstacle is not detected in the entire area of the captured image, and the obstacle is detected only in a partial target area in the captured image. It is possible to reduce the time required for image processing for detecting the obstacle and to detect an obstacle or the like at high speed.
In addition, since the target area is selected based on the behavior information of the vehicle, it is possible to appropriately detect an obstacle or the like according to the situation of the vehicle.

It is a block diagram which shows the structure of the vehicle periphery monitoring apparatus which concerns on embodiment of this invention, and its peripheral device. It is a figure explaining the arrangement configuration of an imaging module, (A) is a front view of a vehicle showing a field angle of a perpendicular direction, and (B) is a top view of a vehicle showing a field angle of a horizontal direction. It is a flowchart which shows an example of a process of the vehicle periphery monitoring apparatus which concerns on embodiment of this invention. It is a figure which shows the captured image imaged with the imaging module shown in FIG. It is a figure explaining a subsequent area, (A) is a figure which shows the subsequent area on real space, (B) is a figure which shows the subsequent area on a captured image. It is a figure explaining a back area, (A) is a figure which shows the back area on real space, (B) is a figure which shows the back area on a captured image. It is a figure explaining a winding area, (A) is a figure which shows the winding area in real space, (B) is a figure which shows the winding area on a captured image. It is a figure explaining an adjacent area and a parking area, (A) is a figure which shows the adjacent area and parking area in real space, (B) is a figure which shows the adjacent area on a captured image. It is a figure which shows the parking area on a captured image.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the shape of the component described in this example is not intended to limit the scope of the present invention, but is merely an illustrative example.

First, the configuration of the vehicle periphery monitoring apparatus according to the embodiment of the present invention will be described with reference to FIG. Here, FIG. 1 is a block diagram showing the configuration of the vehicle periphery monitoring device and its peripheral devices according to the embodiment of the present invention.
The vehicle periphery monitoring device 1 according to the embodiment of the present invention mainly includes an imaging module 2 including an imaging optical system 3 and an imaging unit 4, an image processing unit 6, an information input unit 7, and a storage unit 9. .

The imaging optical system 3 is an optical member that is mounted on the vehicle and causes light (that is, a surrounding image of the vehicle) to enter the imaging unit 4. Specifically, the imaging optical system includes one or a plurality of lenses. The lens is preferably a wide-angle lens in order to acquire a wide area image. The imaging optical system 3 may include other optical members such as a filter in addition to the lens, and the plurality of optical members are arranged so that their optical axes coincide with each other.
The image pickup unit 4 picks up an image around the vehicle via the image pickup optical system 3, and is used by an image pickup element, a CCD sensor, a CMOS sensor, or the like that converts an optical image formed by the image pickup optical system 3 into an electric signal. It is done.

Further, a signal processing unit 5 that performs signal processing such as A / D conversion for converting the analog output of the imaging unit 4 into a digital signal is provided between the imaging unit 4 and the image processing unit 6. The image processed by the signal processing unit 5 may be displayed on the display means 25 via the line 15 as a video signal. Here, the display means is, for example, an in-vehicle monitor.
Here, the imaging optical system 3, the imaging unit 4, and the signal processing unit 5 are referred to as an imaging module 2.

The image processing unit 6 processes an image signal output from the imaging unit 4 and detects an obstacle or the like by image recognition. The image processing unit 6 includes, for example, an MPU (Micro Processing Unit), an LSI (Large Scale Integration), or the like.
Obstacles detected by image recognition are obstacles that should avoid collisions such as dynamic obstacles such as other vehicles and people, or static obstacles such as guide rails and telephone poles, or parking frame lines and wheels. It includes objects (including lines displayed on the road surface) that are required when traveling or parking, such as stops and central lines.
Image recognition used for detecting an obstacle or the like in the image processing unit 6 is at least one of shape recognition by detecting a contour line of a subject, template matching, feature point extraction, difference extraction by temporally continuous images, and optical flow. Or it is preferable to be carried out by a plurality of combinations.

Further, the image processing unit 6 detects an obstacle or the like by image recognition, and then notifies the driver of the detection result by a notification means. The notification means performs warning processing 11 for outputting a warning according to the detection result, performs image processing for superimposing another image (for example, a course prediction guideline) on the captured image, and displays the superimposed image via the line 16. For example, image superimposing means to be displayed is used. Here, the warning output means 11 may be configured to issue a warning by sound using an alarm device, or may be configured to display a warning image on the in-vehicle monitor, or blink a lighting lamp attached in the vehicle. It is good also as a structure.
As a specific example, the image processing unit 6 detects an obstacle such as another vehicle or a person, and outputs a warning signal by the warning output unit 11 when the obstacle or the like approaches the own vehicle.
Furthermore, the image processing unit 6 detects an obstacle or the like by image recognition, calculates a predicted trajectory of the vehicle, estimates a collision or approach to the obstacle from the detected obstacle or the like and the predicted trajectory, The warning output means 11 may output a warning signal.

The information input unit 7 acquires vehicle behavior information from various sensors included in the vehicle.
Here, the sensors included in the vehicle are, for example, a shift position detection sensor, a brake pedal sensor, an accelerator depression amount detection sensor, a vehicle speed sensor, an acceleration sensor, a rudder angle sensor, a direction indicator (blinker) detection sensor, and the like. The shift position detection sensor detects the position of the shift lever and detects whether the shift lever is set to move forward or backward. The brake pedal sensor is a sensor that detects depression of the brake pedal and detects a brake operation. The accelerator depression amount detection sensor is a sensor that detects an accelerator depression amount. The vehicle speed sensor or the acceleration sensor is a sensor that detects the speed or acceleration of the vehicle. The steering angle sensor is a sensor that detects the steering angle of the steering wheel. The direction indicator detection sensor is a sensor that detects the lighting state of the right turn or the left turn. For example, when the direction indicator is in the right turn lighting state, the current flowing in the right turn indicator is detected to electrically detect the right turn lighting state. .

A first sensor 21 shown in the figure is a sensor that detects a change in a detection target over time. Among the sensors described above, a brake pedal sensor, an accelerator depression amount detection sensor, a vehicle speed sensor, an acceleration sensor, and a steering angle sensor Etc. The detection signal of the first sensor 21 is once introduced into the filter circuit 8 and necessary information is calculated, and then input to the information input unit 7 as vehicle behavior information.
On the other hand, the second sensor 22 is a sensor that detects an instantaneous state of a detection target, and among the sensors described above, a shift position detection sensor, a direction indicator detection sensor, and the like can be given. In addition, although the specific example of each sensor is shown here as an example, you may use the other kind of sensor which has the same function. For example, instead of the brake pedal sensor, a hydraulic pressure sensor that detects brake hydraulic pressure and acquires brake operation information, a brake displacement amount detection sensor, or the like may be used.

The vehicle behavior information is preferably information related to at least one of start, stop, travel direction, travel speed, acceleration / deceleration, and turn (right turn or left turn) of the vehicle. For example, information on start, stop, or travel speed of the vehicle can be acquired from a vehicle speed sensor or a brake pedal sensor, information on the travel direction of the vehicle can be acquired from a shift position detection sensor, and information on the acceleration of the vehicle can be acquired from an acceleration sensor. The information regarding the turning of the vehicle can be acquired from the rudder angle sensor or the direction indicator detection sensor.
The vehicle behavior information acquired from the sensor may be acquired via an in-vehicle network 17 such as a CAN (Controller Area Network).
The storage unit 9 stores vehicle behavior information and a preset target area in the captured image in association with each other.

The arrangement configuration of the imaging module 2 will be described with reference to FIG. Here, FIG. 2A is a front view of the vehicle representing the angle of view in the vertical direction, and FIG. 2B is a plan view of the vehicle representing the angle of view in the horizontal direction. Note that the arrangement configuration of the imaging module 2 illustrated in FIG. 2 is a suitable example, and the number of installed imaging modules 2, the mounting position, the mounting direction, and the like are not limited.
The imaging module 2 is attached to each side surface of the vehicle 30 one by one. The imaging module 2 is arranged in a direction that enables imaging of the side of the vehicle 30. At this time, the imaging optical system of the imaging module 2 is a wide-angle lens, and the wide-angle lens has a vertical angle of view α1 of 120 degrees or more upward from the road surface as shown in FIG. 2A, and FIG. As shown in the figure, it is preferable that the vehicle 30 has a horizontal angle of view α2 of 180 degrees or more in the front-rear direction. Thereby, a wide-area captured image can be obtained with a small number of modules.

  Here, the configuration in which the imaging module 2 that is integrally arranged is arranged in the vehicle as an in-vehicle camera has been described, but in the present embodiment, the image processing amount can be reduced and a small image processing unit 6 can be used. In addition, all the components in the vehicle periphery monitoring device 1 can be arranged integrally. In that case, you may use the vehicle periphery monitoring apparatus 1 as a vehicle-mounted camera, and may install in the position of the imaging module 2 shown in FIG.

Next, an example of processing of the vehicle periphery monitoring device according to the embodiment of the present invention will be described with reference to FIG.
First, a captured image is acquired by capturing a surrounding image of the vehicle 30 by the imaging module 2. This captured image is accumulated in, for example, an image memory of the storage unit 9. Here, as an example, a case in which the side monitoring of the vehicle is performed using the imaging module 2 having the arrangement configuration illustrated in FIG. 2 will be described. At this time, the captured image is as shown in FIG.
On the other hand, in the figure, at the information input processing step, the information input unit 7 acquires information on the behavior of the vehicle from the first sensor 21 via the filter circuit 8 or directly relates to the behavior of the vehicle from the second sensor 22. Get information. Of course, information may be acquired from both sensors 21 and 22.

Then, the image processing unit 6 determines whether or not the side monitoring condition is satisfied. The side monitoring condition is stored in advance in the storage unit 9 and refers to a case where the behavior information associated with the target area is satisfied.
The side monitoring conditions are, for example, (1) when the vehicle is at high speed and constant speed or acceleration, (2) when the vehicle is at high speed and decelerating, (3) the vehicle is at low speed and is decelerated or right For example, when turning left, (4) when the vehicle is parked at low speed, (5) when the vehicle is parked back.
From the vehicle behavior information acquired from the information input unit 7, when the side monitoring condition described above is not satisfied, a step of determining the traveling state is performed, and then the process returns to the information input processing step.

Next, each condition and its target area will be described.
(1) When it is determined from the vehicle behavior information that the vehicle speed is high and constant or accelerating, a target area for detecting an obstacle or the like is set as a subsequent area.
5A and 5B are diagrams for explaining the subsequent area. FIG. 5A is a diagram illustrating the subsequent area in the real space, and FIG. 5B is a diagram illustrating the subsequent area on the captured image.
As shown in FIG. 5A, when the vehicle 30 is at a high speed and is at a constant speed or accelerating, it is estimated that the vehicle 30 normally travels (forwards) on the road. The target object is the succeeding vehicle 41a in the same lane. This is for monitoring the overtaking of the following vehicle 41a, the inter-vehicle distance and the like. Therefore, in the above vehicle situation, the target area for detecting an obstacle or the like is the subsequent area 51a behind the vehicle 30. Here, the subsequent area 51a is an area where a subsequent vehicle traveling in the same lane is located.
Since the subsequent area 51a in the real space becomes the subsequent area 51b in the captured image 40A shown in FIG. 5B, an obstacle or the like is detected using the subsequent area 51b as a target area.

(2) When it is determined from the vehicle behavior information that the vehicle is decelerating at high speed, the target area for detecting an obstacle or the like is set as the rear area.
6A and 6B are diagrams illustrating the rear area. FIG. 6A is a diagram illustrating the rear area in the real space, and FIG. 6B is a diagram illustrating the rear area on the captured image.
As shown in FIG. 6 (A), when the vehicle 30 is decelerating at a high speed, the vehicle 30 is normally traveling (advancing) on the road, and is preparing to change lanes or is congested. Therefore, the object that the vehicle 30 should pay particular attention to is the rear vehicle 42a. Therefore, in the above-described vehicle situation, the target area for detecting an obstacle or the like is the rear area 52a behind the vehicle 30. Here, the rear area 52a is an area that travels in the adjacent lane and the vehicle 42a behind the host vehicle is located.
Since the rear area 52a in the real space is the rear area 52b in the captured image 40B shown in FIG. 5B, an obstacle or the like is detected using the rear area 52b as a target area. In the figure, 42b is an image of the rear vehicle 42a displayed in the captured image 40B.

And after satisfy | filling the conditions of (1) or (2) and detecting an obstruction etc. in a corresponding target area, when an approach of a following vehicle or a back vehicle is recognized as a detection result, warning output means 11 may output a warning.
When these processes are completed, the process returns to the information input process step described above.

(3) When it is determined from the vehicle behavior information that the vehicle is slow and decelerates or turns left or right, the target area for detecting an obstacle or the like is set as the entrainment area.
FIGS. 7A and 7B are diagrams for explaining a winding area. FIG. 7A is a diagram illustrating a winding area in real space, and FIG. 7B is a diagram illustrating a winding area on a captured image.
As shown in FIG. 7A, when the vehicle 30 is slow and decelerates or turns left or right, the object that the vehicle 30 should pay particular attention to is an accident involving another vehicle 43a or a person. Therefore, in the above-described vehicle situation, the target area for detecting an obstacle or the like is a rear-side entrainment area 53 a in the bending direction of the vehicle 30. Here, the entrainment area 53a is the rear side in the bending direction.
In the captured image 40C shown in FIG. 7B, for example, a winding area 53b is set, and obstacles and the like are detected using the winding area 53b as a target area. 7 (A) and 7 (B) show the entrainment area in different vehicle situations, (A) shows the entrainment area 53a when turning left at the intersection, and FIG. 7 (B) is when turning right. The winding area 53b is shown. Further, in the figure, 43b is an image of another vehicle located in the vicinity of the entrainment area 53b and projected in the captured image 40C.

(4) When it is determined from the vehicle behavior information that the vehicle is at a low speed and starts to park, a target area for detecting an obstacle or the like is set as an adjacent area.
8A and 8B are diagrams illustrating the adjacent area and the parking area. FIG. 8A is a diagram illustrating the adjacent area and the parking area in the real space, and FIG. 8B is a diagram illustrating the adjacent area on the captured image.
As shown in FIG. 8A, when the vehicle 30 starts at a low speed and starts parking, an object that the vehicle 30 should pay particular attention to is a contact accident with an adjacent other vehicle 44a or a person. Therefore, in the above vehicle situation, the target area for detecting an obstacle or the like is the adjacent area 54 a next to the vehicle 30.
Since the adjacent area 54a in the real space is the adjacent area 54b in the captured image 40D shown in FIG. 8B, an obstacle or the like is detected using the adjacent area 54b as a target area. In the figure, 44b is an image of the adjacent vehicle 44a displayed in the captured image 40D.

When the condition (3) or (4) is satisfied, obstacles and the like are detected in the corresponding target areas, and the course prediction trajectory of the vehicle rear wheel is calculated and obtained. May be displayed as a guideline, or the involvement or approach of the obstacle may be estimated from the detected obstacle or the like and the predicted trajectory, and the warning output means 11 may output a warning.
When these processes are completed, the process returns to the information input process step described above.

(5) When it is determined from the vehicle behavior information that the vehicle is parked back, the target area for detecting an obstacle or the like is set as the parking area.
Referring to FIG. 8A again, when the vehicle 30 performs back parking, the object that the vehicle 30 should pay particular attention to is contact or approach with another vehicle 45a or a person who has already parked. Therefore, in the vehicle situation described above, the target area for detection of obstacles and the like is the parking area 55a laterally rearward of the vehicle 30.
Here, FIG. 9 is a diagram showing a parking area on the captured image.
Since the parking area 55a in the real space described above becomes the parking area 55b in the captured image 40E shown in FIG. 9, an obstacle or the like is detected using the parking area 55b as a target area.

When the condition (5) is satisfied, obstacles and the like are detected in the corresponding target area, a guideline corresponding to the parking frame line is displayed on the captured image, and obstacles are detected from the detected obstacles. It is also possible to estimate the entrainment or approach and output a warning by the warning output means 11.
When these processes are completed, the process returns to the information input process step described above.

According to the present embodiment, the obstacle is not detected in the entire area of the captured image, and the obstacle is detected only in a partial target area in the captured image. It is possible to reduce the time required for image processing for detecting the obstacles and the like, and to detect obstacles and the like at high speed.
In addition, since the target area is selected based on the behavior information of the vehicle, it is possible to appropriately detect an obstacle or the like according to the situation of the vehicle.

DESCRIPTION OF SYMBOLS 1 Vehicle periphery monitoring apparatus 2 Imaging module 3 Imaging optical system 4 Imaging part 5 Signal processing part 6 Image processing part 7 Information input part 8 Filter circuit 9 Memory | storage part 21 1st sensor 22 2nd sensor 25 Display means

Claims (5)

In the vehicle periphery monitoring device that detects obstacles around the vehicle,
An imaging unit mounted on the vehicle and capturing an image around the vehicle;
An information input unit for acquiring behavior information of the vehicle from a sensor included in the vehicle;
An image processing unit that processes the image signal output from the imaging unit and detects the obstacle, and the like;
A storage unit in which the behavior information and a preset target area are associated and stored;
The image processing unit selects the target area in a captured image corresponding to the behavior information acquired by the information input unit, and detects the obstacle or the like in the target area. A vehicle periphery monitoring device.   The vehicle periphery monitoring device according to claim 1, wherein the behavior information is information related to at least one of start, stop, travel direction, travel speed, acceleration / deceleration, and bending of the vehicle. A plurality of notification means for notifying the detection results of the obstacles,
The vehicle periphery monitoring device according to claim 1, wherein the detection result notifying unit is selected based on the target area selected by the image processing unit.   4. The vehicle periphery monitoring according to claim 3, wherein the notifying unit includes at least one of an image superimposing unit that superimposes and displays a course prediction guideline of the vehicle on the captured image, and a warning output unit that outputs a warning. apparatus.   The image processing unit includes warning output means for outputting a warning, calculates a predicted course of the vehicle from the behavior information of the vehicle, and estimates contact or approach with the obstacle using the predicted path 4. The vehicle periphery monitoring device according to claim 1, wherein the warning output means outputs a warning signal.

Images