JP5464324B2 - Driving support apparatus and method for parking - Google Patents

Description

  The present invention relates to a driving support apparatus and method at the time of parking in which the attitude of the host vehicle with respect to the left and right parking frame lines of a parking space can be detected based on a captured image of the rear of the vehicle imaged by an imaging device mounted on the vehicle. Is.

  As one of the technologies for supporting driving of a vehicle, there is a technology that images the outside of the vehicle with an in-vehicle camera and displays the captured image on a monitor in the vehicle, or presents necessary information to the driver by image processing. Are known. In particular, when parking, it is difficult to visually recognize the rear of the vehicle.For example, when parking in parallel in a parking space, it is difficult for the driver to recognize the left and right parking frame lines. It was very difficult to make it back.

Therefore, driving support devices that install an in-vehicle camera at the rear of the vehicle and display the image captured by the in-vehicle camera on the monitor in the vehicle, letting the driver know the state behind the vehicle without looking at the mirror, have become widespread. Have begun to do.
In recent years, in addition to this, a driving support device has also been proposed in which predetermined image processing is performed on an image captured by an in-vehicle camera so that necessary information is easily displayed to the driver.
For example, in Patent Document 1 (Japanese Patent Laid-Open No. 10-83500), a plurality of image sensors are attached to the bottom surface of a vehicle body to photograph a road surface, and the relative positional relationship between a white line and the vehicle is calculated and displayed on a display inside the vehicle. Techniques to do this are disclosed.
Further, Patent Document 2 (Japanese Patent Laid-Open No. 11-334470) displays a predicted tire trajectory according to the steering angle of the steering wheel on an image captured by an in-vehicle camera, and informs the driver of the destination of the vehicle. A driving support device that enables this is disclosed.

JP 10-83500 A JP-A-11-334470

However, in the prior art, in order to recognize the rear parking frame line, it is necessary to newly install an in-vehicle camera on the bottom surface of the vehicle body or increase the number of in-vehicle cameras, which increases the cost. In addition, when the camera is installed on the bottom surface of the vehicle body, there is a problem that since the distance from the bottom surface of the vehicle body to the road surface is short, the shooting range becomes narrow and only the white line directly under the vehicle is recognized. Further, with respect to the mounting position of the in-vehicle camera, the light receiving window of the image pickup apparatus may be soiled by mud splashing on the bottom of the vehicle body, and the image quality may be deteriorated. Furthermore, in the prior art, the relative positional relationship between the white line and the host vehicle is calculated and obtained, but there is a problem that the calculation processing becomes complicated.
Further, in the technology for detecting the attitude of the host vehicle based on the change in the steering angle of the steering wheel, the steering angle sensor is expensive, and a processing unit that calculates a predicted trajectory of the tire traveling direction using sensor information of the steering angle sensor There is also a problem that the cost is increased.
Therefore, an object of the present invention is to provide a driving support apparatus and method at the time of parking in which the posture of the host vehicle with respect to the parking frame line can be detected with a simple arithmetic processing and apparatus configuration.

In order to solve the above-mentioned problem, a driving assistance device at the time of parking is used to detect the posture of the vehicle with respect to the left and right parking frame lines of a parking space using a wide-angle captured image of the rear of the vehicle imaged by an imaging device mounted on the vehicle. And
Compression means for compressing the captured image in the vertical direction;
A parking frame line detection means for extracting two linear images corresponding to the left and right parking frame lines from the captured image compressed in the vertical direction ;
Inclination θ1 and θ2 of the two linear images with respect to the horizontal direction or the vertical direction of the captured image are calculated, and vehicle parallelism for detecting vehicle parallelism with respect to the left and right parking frame lines based on a relative relationship between the inclinations θ1 and θ2. And a degree detecting means.

According to the present invention, without directly detecting the relative positional relationship between the left and right parking frame lines and the host vehicle, the vehicle parking frame is based on the relative relationship between the inclinations θ1 and θ2 of the two left and right parking frame lines. Since the parallelism of the vehicle body with respect to the line is detected, the posture of the vehicle can be detected easily and accurately. The vehicle body parallelism refers to the degree of parallelism between the left and right parking frame lines and the direction of the vehicle (direction perpendicular to the axle). The relative relationship between the inclinations θ1 and θ2 is represented by the value of the inclination θ1 + θ2, for example, when the inclinations θ1 and θ2 are represented by numerical values having plus and minus. In this case, the parallelism of the vehicle body becomes closer to parallel as θ1 + θ2 approaches 0. However, the relative relationship between the inclinations θ1 and θ2 is not limited to the above. As another example, it may be expressed by a difference between absolute values of the inclinations θ1 and θ2. In this case, the parallelism of the vehicle body becomes closer to parallel as the difference between the absolute values of the inclinations θ1 and θ2 approaches zero. Note that the inclinations θ1 and θ2 of the two linear images with respect to the horizontal direction or the vertical direction may be the inclinations of the perpendicular lines of the two linear images with respect to the horizontal direction or the vertical direction.
In addition, since the parallelism of the vehicle body can be detected only by installing one imaging device at the rear of the vehicle, it is not necessary to newly install a plurality of imaging devices and can be implemented only by adding an image processing unit, and is inexpensive. It can be a device. Furthermore, an existing back camera (rear view camera) can be used, and the cost can be further reduced.

In addition, when the optical axis of the imaging device and the orientation of the vehicle have an angle θ3, an angle θ3 is set in advance in the vehicle body parallelism detecting means, and the relative relationship between the inclinations θ1 and θ2 and the angle θ3 are set. Based on this, the vehicle body parallelism with respect to the left and right parking frame lines is detected.
Thus, by setting in advance the angle θ3 between the optical axis of the imaging device and the vehicle direction, the parallelism of the vehicle body can be detected with high accuracy even when the optical axis is inclined with respect to the vehicle direction. .

Further, a processing target area having a shape in which a portion other than a parking space including a reflected portion of the vehicle is excluded from the captured image, and a width on the lower side with respect to the vertical direction is wide in the left-right width and narrowed toward the upper side. The vehicle body parallelism is detected by the vehicle body parallelism detection means within the processing target area acquired by the area read device.
In this way, by setting the processing target area according to the parking space displayed in the wide-angle captured image and detecting the vehicle body parallelism only within this processing target area, only the necessary area is image-processed. Time is shortened.

The compression unit compresses the captured image in the vertical direction to amplify the angle of the linear image.
When the vehicle and the left and right parking frame lines approach each other in parallel, the inclinations θ1 and θ2 of the parking frame lines become close to each other. Therefore, high sensitivity is obtained by compressing the captured image in the vertical direction and amplifying the angle of the linear image. Thus, it becomes possible to detect the parallelism of the vehicle body.

Furthermore, the parking frame line detection means deletes the two extracted linear images by a predetermined number of pixels from the upper and lower ends in the vertical direction, and leaves only the linear image extending in the vertical direction. Features.
In this way, by deleting the two extracted linear images by predetermined pixels from the upper and lower ends in the vertical direction, only the left and right parking frame lines extending in the substantially vertical direction can be left, and the horizontal parking frame It can be distinguished from the line and erroneous recognition can be prevented.

Further, the parking frame line detection means selects two linear images that are close to the center of the captured image when three or more linear images are extracted from the captured image. It is characterized by.
In this way, when a plurality of parking frame lines are detected, the left and right parking frame lines of the parking space to be parked are reliably detected by selecting two linear images that are close to the center of the captured image. It becomes possible to do. The means for selecting a linear image corresponding to the left and right parking frame lines when a plurality of parking frame lines is detected is not limited to the above-described means. Other means include, for example, means for selecting left and right parking frame lines based on the areas of a plurality of extracted linear images, and providing threshold values for the inclinations θ1 and θ2, and θ1 and θ2 deviate from the threshold values, respectively. And a means for selecting the remaining linear image as the left and right parking frame lines.

In addition, the vehicle body parallelism detection means includes a first imaginary line that is an average straight line of the two linear images extracted by the parking frame line detection means, and a second imaginary line that is an extension of the center line of the vehicle. A deviation from a virtual line is detected, and a deviation in the vehicle width direction of the vehicle with respect to the parking space is detected.
As a result, not only the parallelism of the vehicle body but also the displacement in the vehicle width direction of the vehicle with respect to the parking space can be detected, and the vehicle can be parked at an appropriate position.

Furthermore, a signal for prompting the steering operation of the vehicle is output based on the vehicle body parallelism detected by the vehicle body parallelism detecting means.
As a result, the driver of the vehicle can grasp the posture of the host vehicle in real time and can perform an appropriate driving operation. Examples of the signal include marks and characters displayed on the in-vehicle monitor, audio output from the in-vehicle speaker, and the like.

In addition, a driving support method at the time of parking that detects the vehicle posture with respect to the left and right parking frame lines of the parking space using a wide-angle captured image of the rear of the vehicle imaged by an imaging device mounted on the vehicle,
The captured image is input to an image processing unit,
After the image processing unit performs image processing including binarization processing on the captured image, two linear images corresponding to the left and right parking frame lines are extracted, and the captured image is compressed in the vertical direction. , wherein the horizontal direction or vertical direction of the captured image is compressed in the vertical direction to calculate the two linear images of inclination θ1 and .theta.2, body parallel to said left and right parking frame lines on the basis of the relative relationship of the inclined-out θ1 and .theta.2 It is characterized in that the degree is detected.

As described above, according to the present invention, the relative positional relationship between the left and right parking frame lines and the host vehicle is not directly detected, but based on the relative relationship between the inclinations θ1 and θ2 of the left and right parking frame lines. Since the vehicle body parallelism with respect to the parking frame line of the vehicle is detected, the posture of the vehicle can be detected easily and accurately.
Further, since the parallelism of the vehicle body can be detected only by installing one image pickup device at the rear of the vehicle, it is not necessary to newly install a plurality of image pickup devices, and the device can be made inexpensive. Furthermore, an existing back camera can be used, and the cost can be further reduced.

In addition, by setting the angle θ3 between the optical axis of the imaging device and the vehicle in advance, the parallelism of the vehicle body can be detected with high accuracy even when the optical axis is inclined with respect to the vehicle direction.
In addition, by setting the processing target area according to the parking space projected in the wide-angle captured image, and detecting the vehicle body parallelism only within this processing target area, only the necessary area is image-processed, and the processing time It is shortened.
Furthermore, the parallelism of the vehicle body can be detected with high sensitivity by compressing the captured image in the vertical direction and amplifying the angle of the linear image.
Furthermore, by deleting the two extracted linear images by a predetermined number of pixels from the upper and lower ends in the vertical direction, it is possible to leave only the left and right parking frame lines extending in the substantially vertical direction. And can prevent misrecognition.

Further, by detecting a deviation between the first imaginary line that is an average straight line of two linear images and the second imaginary line that is an extension of the center line of the vehicle, not only the parallelism of the vehicle body but also the parking space It is possible to detect a shift in the vehicle width direction of the vehicle with respect to the vehicle and to park the vehicle at an appropriate position.
Furthermore, by outputting a signal that prompts the steering operation of the vehicle based on the parallelism of the vehicle body, the driver of the vehicle can grasp the posture of the host vehicle in real time and can perform an appropriate driving operation. .

It is a block block diagram of the driving assistance device which concerns on embodiment of this invention. It is a figure which shows the process target area in a captured image. It is a figure explaining the detection of vehicle body parallelism, (a) is a figure which shows the positional relationship of a parking frame line and a vehicle, (b) is a figure which shows the captured image of the imaging device in the state of (a). It is a figure explaining the detection of the vehicle body parallelism when an optical axis and a vehicle are not parallel, (a) is a figure which shows the positional relationship of a parking frame line and a vehicle, (b) is an imaging of the imaging device in the state of (a) It is a figure which shows an image. It is a figure explaining a compression process, (a) is a figure which shows the image before compression, (b) is a figure which shows the image after compression. 5A and 5B are diagrams illustrating a compression process different from FIG. 5, in which FIG. 5A is a diagram illustrating an image before compression, and FIG. 5B is a diagram illustrating an image after compression. 7A and 6B are diagrams illustrating the compression processing of FIG. 6, where FIG. 7A is a pixel image diagram before compression, and FIG. 7B is a pixel image diagram after compression. It is a flowchart explaining the compression process of FIG. It is a figure which shows the captured image which combined the display mark. It is a figure explaining the shift detection of a vehicle and a parking space, (a) is a figure which shows the positional relationship of a parking frame line and a vehicle, (b) is a figure which shows the captured image of the imaging device in the state of (a). It is a flowchart explaining the process of the shift | offset | difference detection of a vehicle and a parking space. It is a flowchart which shows the process of the driving assistance method which concerns on embodiment of this invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the shape of the component described in this embodiment is not intended to limit the scope of the present invention, but merely an illustrative example.
The driving support device according to the embodiment of the present invention detects the posture of the host vehicle with respect to a parking frame line (for example, a white line) drawn on a road surface. It detects the vehicle angle (the vehicle body angle perpendicular to the axle).

FIG. 1 is a block diagram of a driving support apparatus according to an embodiment of the present invention.
The imaging device 1 is a device that captures an image of the outside rear of the vehicle. The image pickup apparatus 1 includes a lens 1a or an optical element composed of a combination of a lens and a prism or a mirror, and a solid-state image pickup element such as a CCD 1b or CMOS, and incident light entering the lens 1a or the like is separated. You may comprise so that it may be transmitted to the image pick-up element in a position. The imaging device 1 is a color or monochrome digital camera (on-vehicle camera), for example, a rear view camera installed at the rear of the vehicle. Preferably, the imaging device 1 includes a wide-angle lens and has a wide-angle field of view. One or a plurality of imaging devices 1 are installed, but preferably a single device is used. In addition, the imaging device 1 includes a CPU and a frame memory (not shown), and the captured image is stored in the memory and transmitted to the image processing unit 10 as appropriate.
The AFE (analog front end) unit 2 includes a CDS (Correlated Double Sampling) circuit 3, an AD conversion means 4, and the like. The image signal output from the solid-state imaging device of the imaging device 1 is subjected to A / D conversion by the AD conversion means 4 after removing a part of noise by the CDS circuit 3, and is converted into a digital signal by a series of processes including these. It is converted into some image data.

The image processing unit 10 includes a luminance signal processing unit 11, a color signal processing unit 12, an image storage unit (memory) 13, a delay circuit 14, an image composition unit 15, an area unit reading unit 16, a parking frame line. A detection unit 17, a vehicle body parallelism detection unit 18, a display mark selection unit 19, and a display mark reading unit (memory) 20 are provided.
The luminance signal processing means 11 performs luminance signal processing such as offset and gain adjustment, gamma conversion, and sharpening processing using the luminance signal among the image data obtained by separating the color signal and the luminance signal, and the high frequency component of the luminance signal. Is extracted to emphasize the contour of the image.
The color signal processing means 12 performs color signal processing such as offset and gain adjustment, gamma conversion, white balance processing (AWB processing, etc.) using the color signal among the image data obtained by separating the color signal and the luminance signal. is there.

The image storage unit 13 is a unit that stores image data obtained by combining the luminance signal processed by the luminance signal processing unit 11 and the color signal processed by the color signal processing unit 12.
The delay circuit 14 is a means for delaying the image data from the image storage means 13 by a time corresponding to the processing time from the area portion reading means 16 to the display mark reading means 20.
The image synthesizing unit 15 is a unit that synthesizes the image data stored in the image storing unit 13 and the display mark obtained by performing the processing from the area part reading unit 16 to the display mark reading unit 20.

The area reading unit 16 excludes the area where the bumper 38 is projected and the area which is not related to the parking frame line such as sky or far away from the image data stored in the image storage unit 13, Is a means for reading out only the processing target area A (see FIG. 2) corresponding to. As shown in FIG. 2, it is preferable that the processing target area A has a shape in which the width on the lower side with respect to the vertical direction is wide and the width on the left and right is narrowed toward the upper side.
The parking frame line detection unit 17 performs binarization processing and filter processing on the image data stored in the image storage unit 13 or the image data read out of the processing target area A by the area reading unit 16. , A labeling process, a Hough transform process, a compression process, and the like are appropriately selected to detect two linear images corresponding to the parking frame line.
The vehicle body parallelism detection means 18 calculates the relative relationship between the inclinations of the two linear images detected by the parking frame line detection means 17, and based on this relative relationship, the vehicle (direction perpendicular to the axle) And means for detecting the parallelism of the left and right parking frame lines.

The display mark selection means 19 is a means for selecting a display mark to be presented to the driver based on the parallelism between the vehicle and the left and right parking frame lines detected by the vehicle body parallelism detection means 18.
The display mark reading means (memory) 20 is a means in which a plurality of display marks (including characters) are stored and read out as appropriate.
The monitor 21 is installed at a position that can be visually recognized by the driver in the vehicle. For example, a liquid crystal screen is used, but other display devices such as a plasma screen can also be used. On the monitor 21, a captured image stored in the image storage unit 13 and an image in which a display mark is combined with the image by the image combining unit 15 are displayed. The monitor 21 may be newly attached, but preferably an existing vehicle-mounted GPS monitor or the like may be shared.

  FIG. 3 is a diagram illustrating a driving support method according to the embodiment of the present invention. As shown in FIG. 3A, for example, a parking space 30 is arranged in a direction perpendicular to the road 40. When parallel parking is performed, the vehicle body is operated while changing the direction of the vehicle 100 by operating a handle. The vehicle is parked in the parking space 30 by driving so that the angle is parallel to the left and right parking frame lines L1 and L2. At that time, the front portion 101 of the vehicle 100 does not face the parking space 30, and the rear portion 102 faces the parking space 30. The imaging device 1 is installed in the rear portion 102 of the vehicle 100.

FIG. 3A shows a state in the middle of parking the vehicle 100 in the parking space 30. The vehicle body angle with respect to the left and right parking frame lines L1 and L2 is slanted, and at this time, the captured image as shown in FIG.
The captured image of FIG. 3B obtained by the imaging device 1 is input to the image processing unit 10 via the AFE unit 2. In the image processing unit 10, a predetermined processing is performed in the luminance signal processing unit 11 and the color signal processing unit 12, and the parking frame line detection unit 17 corresponds to the parking frame line from the captured image stored in the image storage unit 13. The two linear images L1 and L2 to be recognized are recognized. Then, the vehicle body parallelism detecting means 18 sets an arbitrary point in the captured image, the perpendicular line dropped from the arbitrary point to the two recognized linear images L1 and L2, and the horizontal direction of the captured image or The inclinations θ1 and θ2 with respect to the vertical direction are detected, and the attitude of the vehicle 100 with respect to the parking frame lines L1 and L2 is detected based on the relative relationship between the inclinations θ1 and θ2.

In the present embodiment shown in FIG. 3, the relative relationship between the inclinations θ1 and θ2 is expressed by an equation of inclination θ1 + θ2 when the inclinations θ1 and θ2 are expressed by numerical values having plus and minus. In this case, the parallelism of the vehicle body approaches parallel as the value of θ1 + θ2 approaches 0, and the angle of the vehicle parallelism increases as the value of θ1 + θ2 increases.
That is, when the two inclinations are θ1 + θ2 = 0, it is determined that the vehicle 100 is parallel to the parking frame lines L1 and L2. The arbitrary point is a corner of the screen as shown in the figure, for example. θ1 is the angle (inclination) between the vertical line of the left parking frame line L1 and the screen edge, and θ2 is the angle (inclination) between the vertical line of the right parking frame line L2 and the screen edge. The screen edge is a horizontal line of the screen upper frame. Since θ1 and θ2 have plus and minus values, the relative relationship between the inclinations θ1 and θ2 of the parking frame lines L1 and L2 can be obtained by the equation θ1 + θ2. When θ1 + θ2 = 0 does not hold, it is determined that the vehicle 100 and the parking frame lines L1, L2 have an angle, and the angle of the vehicle 100 with respect to the parking frame lines L1, L2 is detected based on the value of θ1 + θ2.
The relative relationship between the inclinations θ1 and θ2 is not limited to the above. As another example, the relative relationship between the inclinations θ1 and θ2 may be represented by the difference between the absolute values of the inclinations θ1 and θ2. In this case, the parallelism of the vehicle body becomes closer to parallel as the difference between the absolute values of the inclinations θ1 and θ2 approaches zero. In addition, the inclinations θ1 and θ2 may be inclinations of the parking frame lines L1 and L2 with respect to the horizontal direction or the vertical direction, or may be inclinations of perpendicular lines of the parking frame lines L1 and L2 with respect to the horizontal direction or the vertical direction.

The above-described process is based on the premise that the optical axis of the imaging device 1 and the direction of the vehicle 100 are parallel.
As shown in FIG. 4A, when the optical axis L3 of the imaging device 1 and the orientation L6 of the vehicle 100 are not parallel, as shown in FIG. The angle θ3 of the orientation of 100 is registered in the image processing unit 10, and the difference between the inclination θ1 of the left parking frame line L1 and the inclination θ2 of the right parking frame line L2 is the optical axis L3 of the imaging device 1 and the vehicle When the angle θ3 is 100, it is determined that the vehicle 100 is parallel to the parking frame lines L1 and L2. That is, when θ1 + θ2 = θ3, the vehicle 100 is assumed to be parallel to the parking frame lines L1 and L2. Therefore, when θ1 + θ2 = θ3 does not hold, the vehicle 100 is not parallel to the parking frame lines L1 and L2, and the vehicle 100 with respect to the parking frame lines L1 and L2 depends on how far the values of θ1 + θ2 and θ3 are separated. Detect the posture (tilt).
As another method, when the optical axis L3 of the imaging device 1 and the direction of the vehicle 100 are not parallel, the left and right parking frame lines L1, L2 and the vehicle 100 are installed in parallel, and the imaging device 1 uses the imaging device 1 to set the left and right parking frames. You may make it discriminate | determine the optical axis inclination of the imaging device 1 by imaging the lines L1 and L2.

FIG. 5 is a diagram for explaining the compression processing in the parking frame line detection means 18.
As shown in FIG. 5A, when the vehicle 100 and the parking frame lines L1 and L2 approach each other in parallel, the vertical inclinations θ1 and θ2 of the parking frame lines L1 and L2 with respect to the horizontal or vertical direction of the captured image are close to each other. Thus, the degree of difficulty in detecting the angle between the vehicle 100 and the parking frame lines L1 and L2 becomes higher. Therefore, as shown in FIG. 5B, the left and right parking frame lines L1 ′ and L2 ′ are detected based on the image compressed in the vertical direction when the left and right parking frame lines L1 and L2 are detected. In this compression process, the image stored in the image storage unit 13 may be compressed as it is, or may be compressed after the parking frame line detection unit 17 performs binarization processing. Also, when the compression processing is performed, if the captured image is vertically compressed to 1 / N, writing to the memory is performed every N lines.
As is clear when comparing the case where the image is not compressed in the vertical direction as shown in FIG. 5A and the case where the image is compressed in the vertical direction as shown in FIG. The vertical inclinations θ1 ′ and θ2 ′ of the parking frame lines L1 and L2 with respect to the horizontal or vertical direction of the captured image are amplified and calculated as large values. Thus, the parallelism between the parking frame lines L1 and L2 and the vehicle 100 can be detected with higher accuracy by compressing the captured image in the vertical direction.

6 to 8 are diagrams for explaining another compression process in the parking frame line detection means 18.
The binarized image obtained by extracting only the processing area A by the area reading unit 16 and binarizing by the parking frame line detecting unit 18 is as shown in FIG. In this image, there may be a horizontal parking frame line L4 and left and right vertical parking frame lines L1 and L2. Therefore, in order to distinguish between the horizontal parking frame line L4 and the left and right parking frame lines L1 and L2 and to prevent erroneous recognition, only the vertical direction of the screen of the parking frame line is compressed from the vertical direction so that the vertical distance is reduced. The short substantially horizontal parking frame line L4 disappears, and it becomes possible to leave only the parking frame lines L1 and L2 extending in the substantially vertical direction as shown in FIG. 6B. The compression referred to here means deleting predetermined pixels from the upper and lower ends of the parking frame lines L1, L2, and L4 in the vertical direction.

Specific processing will be described with reference to the flow of compression processing shown in FIG.
First, the picked-up image picked up by the image pickup device 1 is binarized by the parking frame line detection means 18 (S11). At this time, the image captured by the imaging device 1 and stored in the image storage unit 13 of the image processing unit 10 may be binarized as it is, or only the processing target area A is extracted by the area unit reading unit 16. The binarized image may be processed.
The binarized image is shown in FIG. In this image, scanning is performed in the vertical direction for each column of pixels (S12), and white pixels that change from white to black or from black to white are detected (S13). And the process which changes the detected white pixel into a black pixel is performed (S14). At this time, the number of pixels to be changed from white pixels to black pixels is appropriately determined. However, the number of pixels corresponding to the vertical distance of the parking frame line L4 in the horizontal direction is acquired in advance, and the number of pixels corresponding to this number of pixels is used. It is preferable to perform a process of changing to black pixels. The image after the compression processing is shown in FIG.

FIG. 9 shows a monitor screen in which display marks 51 and 52 are displayed on the monitor 21.
When the above-described parking frame line detection means 18 and vehicle body parallelism detection means 19 detect the posture of the vehicle 100 with respect to the left and right parking frame lines L1, L2, a display mark that alerts the driver according to the detected posture Is displayed on the monitor 21. This is because a plurality of display marks are registered in advance in the display mark reading means 20 corresponding to the posture of the vehicle 100, and the display marks corresponding to the display marks selected by the display mark selecting means 19 are displayed as display marks. A selection is made from the reading means 20, and the selected display mark is sent to the image composition means 15. The image synthesizing unit 15 synthesizes the image stored in the image storing unit 13 and the display mark, and displays them on the monitor 21. For example, as shown in FIG. 9A, a mark 51 that prompts the user to turn the handle to the right and a character “to the right of the handle” are displayed on the captured image, or the handle is moved as shown in FIG. A mark 52 that prompts the user to keep the state as it is and a character “handle as it is” are displayed on the captured image. This may be either a display mark or a character, or may be configured to output by voice using a vehicle speaker instead of the monitor 21.

FIG. 11 is a flowchart illustrating a process for detecting a relative position (deviation) in the vehicle width direction of the vehicle with respect to the left and right parking frame lines L1 and L2. The flow will be described below with reference to FIG.
Even if it is detected that the vehicle 100 and the parking frame lines L1 and L2 are parallel as shown in FIG. 10A, the distance between the center line (optical axis) L3 of the vehicle 100 and the center line 35 of the parking space 30 If there is a deviation, the vehicle 100 is parked at the end of the parking space 30.
Therefore, first, an arbitrary measurement point P1 of the optical axis (second imaginary line) L3 of the vehicle 100 is calculated (S21), and then an average straight line (first imaginary line) L5 of the left and right parking frame lines L1 and L2 is calculated. Then, an arbitrary measurement point P2 of the average straight line L5 is calculated (S22), and the relative position in the vehicle width direction of the vehicle 100 with respect to the parking space 30 is detected by calculating the distance between the measurement points P1 and P2 (S23). ). The detected relative position may be notified to the driver by displaying on the monitor 21 display marks and characters indicating the presence / absence of the deviation, the magnitude of the deviation, etc. by the display mark selection means 19 and the display mark reading means 20. preferable. In the above-described processing, it is assumed that the optical axis L3 of the imaging device 1 is aligned with the center of the vehicle 100. However, when the optical axis L3 is deviated from the center of the vehicle 100, It is preferable to acquire and register the deviation amount, and calculate the distance between the measurement points P1 and P2 in consideration of the deviation amount.

Next, with reference to FIG. 12, the flow of the driving support method according to the embodiment of the present invention will be described.
First, the imaging device 1 captures an image of the rear side of the vehicle 100 and captures the captured image (S1). The captured image is subjected to A / D conversion after a part of noise is removed by the AFE unit 2 and converted into digital signal image data (hereinafter referred to as an image) by a series of processes including these.
This image is input to the image processing unit 10, and the image is separated into a luminance signal and a color signal and processed by the luminance signal processing means 11 and the color signal processing means 12, respectively, and then the luminance signal and the color signal are combined. And stored in the image storage unit 13.

As shown in FIG. 2, the image stored in the image storage unit 13 excludes the bumper 38 as shown in FIG. 2, and the vehicle 100 has a wide left and right width on the front side (bottom of the screen). Only an area having a shape in which the lateral width becomes narrower toward the back (on the screen) side is acquired as the processing target area A (S2). The area portion reading means 16 is not an essential configuration, and the following processing may be performed using all images.
The parking frame line detecting means 17 performs binarization processing and various filter processes on the processing target area A, and extracts only the left and right parking frame lines L1 and L2 (S3). Here, the binarization process is a well-known process for converting a grayscale image into two gradations of white and black based on a predetermined luminance threshold value set in advance, and as shown in FIG. An image like this is obtained. Various filter processes include the compression process shown in FIG. 5 and the compression processes shown in FIGS.

A labeling process is performed on the image subjected to the binarization process and the compression process, and the left parking frame line L1 and the right parking frame line L2 are classified (S4).
Here, it is determined whether or not two or more parking frame lines are detected (S5). If only one parking frame line is detected or not detected at all, the process returns to image capture (S1). When two or more parking frame lines are detected, the Hough transform is performed to calculate approximate straight lines for the left and right parking frame lines L1 and L2 (S6).
When a plurality of parking frame lines are detected, two linear images that are close to the center of the captured image may be selected. The method of selecting a linear image corresponding to the left and right parking frame lines when a plurality of parking frame lines is detected is not limited to the above-described method. Based on the method of selecting the left and right parking frame lines L1, L2, the thresholds are provided for the inclinations θ1, θ2 of the left and right parking frame lines L1, L2, and those in which θ1, θ2 deviate from the threshold values are excluded and remain. A method of selecting the linear images as the left and right parking frame lines L1 and L2 is used.

Then, the vehicle body parallelism detecting means 18 calculates inclinations θ1 and θ2 between the vertical lines of the approximate straight lines on the left and right parking frame lines L1 and L2 and the horizontal direction of the image. The relative relationship between the inclinations θ1 and θ2 of the left and right parking frame lines L1 and L2 is calculated, and the parallelism between the vehicle 100 and the parking frame lines L1 and L2 is detected based on this relative relationship (S7).
At this time, when the optical axis L3 of the image pickup apparatus 1 is parallel to the vehicle 100, the parallelism of the vehicle 100 is detected based only on the relative relationship between the inclinations θ1 and θ2, and the image pickup apparatus with respect to the vehicle 100 is detected. When one optical axis L3 is inclined, the angle θ3 of the direction of the optical axis L3 and the vehicle 100 is registered in the image processing unit 10 in advance, and the vehicle body parallelism is detected in consideration of the angle θ3.

A parallelism threshold value is set in advance in the vehicle body parallelism detecting means 18, and the detected parallelism is compared with the parallelism threshold value (S8), and the detected parallelism is the parallelism threshold. If it is equal to or smaller than the value, it is determined that the vehicle 100 is substantially parallel to the parking space 30, the handle operation completion display is selected by the display mark selection means 19, and the corresponding display mark ( The captured image stored in the image storage unit 13 and the display mark are combined and displayed on the monitor 21 (S10).
On the other hand, when the detected parallelism is larger than the parallelism threshold value (S8), it is determined that the vehicle 100 is inclined with respect to the parking space 30, and the display mark selection means 19 displays the steering direction. The display mark reading means 20 reads the corresponding display mark (including characters), and the image combining means 15 combines the captured image stored in the image storage means 13 with the display mark. The information is displayed on the monitor 21 to alert the driver to operate the steering wheel.

According to the embodiment of the present invention, without directly detecting the relative positional relationship between the left and right parking frame lines L1 and L2 and the host vehicle 100, the horizontal direction of the screen of the left and right parking frame lines L1 and L2 is determined. Alternatively, since the vehicle body parallelism with respect to the parking frame lines L1 and L2 of the vehicle 100 is detected based on the relative relationship between the inclinations θ1 and θ2 with respect to the vertical direction, the posture of the vehicle can be detected easily and accurately. It becomes.
In addition, since the parallelism of the vehicle body can be detected simply by installing one imaging device 1 at the rear of the vehicle, it is not necessary to newly install a plurality of imaging devices, and can be implemented only by adding the image processing unit 10. An inexpensive apparatus can be obtained. Furthermore, an existing back camera (rear view camera) can be used, and the cost can be further reduced.

Further, by setting in advance the angle θ3 between the optical axis L3 of the imaging device 1 and the vehicle 100, even when the optical axis L3 is inclined with respect to the direction of the vehicle 100, the parallelism of the vehicle body can be detected with high accuracy. Is possible.
In addition, the processing area A is set according to the parking space 30 displayed in the wide-angle captured image, and the parallelism of the vehicle body is detected only in the processing area A, so that only the necessary area is image-processed. Processing time is shortened.

Further, when the vehicle 100 and the left and right parking frame lines L1 and L2 approach each other, the inclinations θ1 and θ2 become close to each other. Therefore, the captured image is compressed in the vertical direction to amplify the angle of the linear image. This makes it possible to detect the parallelism of the vehicle body with high sensitivity.
Furthermore, by deleting the two extracted parking frame lines L1 and L2 by predetermined pixels from the upper and lower ends in the vertical direction, only the left and right parking frame lines L1 and L2 extending in the substantially vertical direction may be left. It can be distinguished from the parking frame line L4 in the horizontal direction, and erroneous recognition can be prevented.

Further, by detecting a deviation between the first imaginary line L5 that is an average straight line of the parking frame lines L1 and L2 and the second imaginary line (optical axis) L3 that is an extension of the center line of the vehicle, A shift in the vehicle width direction of the vehicle 100 with respect to the parking space 30 as well as the parallelism can be detected, and the vehicle 100 can be parked at an appropriate position.
Furthermore, by outputting signals 51 and 52 that prompt the steering operation of the vehicle 100 based on the parallelism of the vehicle body, the driver of the vehicle 100 can grasp the posture of the host vehicle in real time and perform an appropriate driving operation. It becomes possible.

  Since the present invention can detect the posture of the host vehicle with respect to the parking frame line with a simple arithmetic processing and device configuration, the present invention can be suitably applied to all vehicles in which an in-vehicle camera is installed at the rear of the vehicle.

DESCRIPTION OF SYMBOLS 1 Imaging device 2 AFE part 10 Image processing part 11 Luminance signal processing means 12 Color signal processing means 13 Image storage means 14 Delay circuit 15 Image composition means 16 Area part reading means 17 Parking frame line detection means 18 Car body parallelism detection means 19 Display Mark selection means 20 Display mark reading means 21 Monitor 30 Parking space 100 Vehicle L1 Left parking frame line L2 Right parking frame line L3 Optical axis

Claims (8)

A driving assistance device at the time of parking that detects the vehicle posture with respect to the left and right parking frame lines of the parking space using a wide-angle captured image of the rear of the vehicle imaged by an imaging device mounted on the vehicle,
Compression means for compressing the captured image in the vertical direction;
A parking frame line detection means for extracting two linear images corresponding to the left and right parking frame lines from the captured image compressed in the vertical direction ;
Inclination θ1 and θ2 of the two linear images with respect to the horizontal direction or the vertical direction of the captured image are calculated, and vehicle parallelism for detecting vehicle parallelism with respect to the left and right parking frame lines based on a relative relationship between the inclinations θ1 and θ2. A driving support device during parking, comprising: a degree detecting means;   When the optical axis of the image pickup device and the orientation of the vehicle have an angle θ3, an angle θ3 is set in advance in the vehicle body parallelism detecting means, and based on the relative relationship between the inclinations θ1 and θ2 and the angle θ3. The driving assist device for parking according to claim 1, wherein a parallelism of the vehicle body with respect to the left and right parking frame lines is detected.   A portion other than the parking space including the reflected portion of the vehicle is excluded from the captured image, and a processing target area having a shape in which the lateral width is wide on the lower side with respect to the vertical direction and the lateral width is narrowed toward the upper side is acquired. 3. The driving during parking according to claim 1, wherein the vehicle body parallelism is detected by the vehicle body parallelism detecting means within the processing target area acquired by the area portion reading means. Support device. The parking frame line detecting means deletes the two extracted linear images by a predetermined number of pixels from the upper and lower ends in the vertical direction, and leaves only the linear image extending in the vertical direction. The driving assistance device at the time of parking according to any one of claims 1 to 3 . The parking frame line detection means selects two linear images that are close to the center of the captured image when three or more linear images are extracted from the captured image. The driving assistance device at the time of parking according to any one of claims 1 to 4 . The vehicle body parallelism detection means includes a first imaginary line that is an average straight line of two linear images extracted by the parking frame line detection means, and a second imaginary line that is an extension of the center line of the vehicle. deviation detecting a driving support apparatus during parking according to any one of claims 1 to 5, characterized in that to detect the deviation in the vehicle width direction of the vehicle relative to the parking space with. Based on the vehicle body parallelism detected by the vehicle parallelism detection means, the driving support apparatus during parking according to any one of claims 1 to 6, characterized in that a signal for prompting the steering operation of the vehicle . A driving support method at the time of parking that detects the vehicle attitude relative to the left and right parking frame lines of the parking space using a wide-angle captured image of the rear of the vehicle imaged by an imaging device mounted on the vehicle,
The captured image is input to an image processing unit,
After the image processing unit performs image processing including binarization processing on the captured image, two linear images corresponding to the left and right parking frame lines are extracted, and the captured image is compressed in the vertical direction. , wherein the horizontal direction or vertical direction of the captured image is compressed in the vertical direction to calculate the two linear images of inclination θ1 and .theta.2, body parallel to said left and right parking frame lines on the basis of the relative relationship of the inclined-out θ1 and .theta.2 A driving support method at the time of parking, characterized in that the degree is detected.

Images