JP2006178667A - Video correcting apparatus and method for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To intuitively grasp positional relation between a user's vehicle and a peripheral object when the periphery of the vehicle is imaged by a camera mounted on the vehicle. <P>SOLUTION: A video correcting apparatus for a vehicle is provided with an imaging part 1 for imaging the video around the vehicle, an image processing part 2 for inputting the video imaged by the imaging part 1 and executing image correction processing for the video, and a display part 3 for inputting the video image correction processing of which is carried out by the image processing part 2 and displaying the video. In the video correcting apparatus for the vehicle, the image processing part 2 sets a specific position which is a reference for image correction processing in the video imaged by the imaging part 1 and shifts the video part of a horizontal direction in the vertical direction with respect to the specific position to linearly correct distortion in the video in the width direction of the vehicle. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a vehicular video correction apparatus and a vehicular video correction method for capturing a video around a vehicle, performing correction processing on the video, and presenting the video to a driver.

  2. Description of the Related Art Conventionally, a vehicle rear monitoring system including a camera and a monitor is widely known as a technique for imaging a vehicle periphery and presenting it to a driver. In this vehicle rear monitoring system, when the camera mounting position and the angle of view are deviated from the center position in the left-right direction of the vehicle, even if the image is captured by the camera and displayed on the monitor, the image is not displayed for the driver. There is a drawback that it is difficult to grasp the traveling direction of the vehicle.

On the other hand, conventionally, a technique described in Patent Document 1 below is known as a technique for correcting an image captured by a camera by image processing. The technique described in Patent Literature 1 divides an image captured by a camera into pixel rows, and performs image processing to generate an image in which each row is shifted to the left and right by a predetermined number of pixels. It is converted into an image that makes it easy to grasp the surroundings of the vehicle.
JP 2002-374523 A

  By the way, in recent years, it is composed of a camera that captures the surroundings of the host vehicle and a monitor that displays images captured by the camera to the driver, depending on the blind spot information that cannot be seen directly by the driver and the steering angle of the vehicle. Devices that support driving are beginning to be widely used by presenting the positional relationship between the determined predicted trajectory and the position of an obstacle to the driver.

  In such a driving assistance device, a wide-angle camera is often used to display a wide-field image. However, when the wide-angle camera is used, distortion in the peripheral portion of the image becomes large. Specifically, for example, when a wide-angle camera is attached to the vehicle, a distortion occurs in which the bumper of the own vehicle is deformed into a U shape, and the driver intuitively grasps the surroundings of the vehicle in the video. There was a problem that it was difficult.

  Therefore, the present invention has been proposed in view of the above-described situation, and when the surroundings of a vehicle are imaged with a camera attached to the vehicle, it is possible to intuitively grasp the positional relationship between the host vehicle and surrounding objects. An object of the present invention is to provide a vehicle image correction device and a vehicle image correction method.

  The present invention relates to an image pickup means for picking up an image around a vehicle, an image processing means for inputting an image picked up by the image pickup means and performing an image correction process on the image, and an image correction by the image processing means. Display means for inputting the processed video and displaying the video.

  In the present invention, the image processing unit sets a specific position serving as a reference for the image correction process in the image captured by the image capturing unit, and the horizontal video portion is vertically aligned with respect to the specific position. And the above-mentioned problem is solved by correcting the distortion in the image in the width direction of the vehicle to a straight line.

  According to the present invention, even if there is distortion in the image when the periphery of the vehicle is imaged, the image portion in the left-right direction can be shifted in the up-down direction with respect to the specific position. It is possible to correct the internal distortion in a straight line, and to intuitively grasp the positional relationship between the host vehicle and surrounding objects.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Configuration of vehicle surroundings monitoring system]
The present invention is applied to, for example, a vehicle surrounding monitoring system configured as shown in FIG.

  This vehicle surroundings monitoring system receives an image of a vehicle surroundings and outputs image data, and image processing that performs image correction processing on the image data. Unit 2 and a display unit 3 for displaying video data on which image correction processing has been performed by the image processing unit 2.

  In such a vehicle surrounding monitoring system, the image processing unit 2 alleviates distortion of an image obtained by capturing an approximate straight line parallel to the left and right direction of the vehicle, so that the image in the left and right direction at a specific position in the image can be obtained. Image processing for correcting and displaying so as to eliminate distortion is performed. Here, the specific position in the video is, for example, a driver's attention position, and the image processing unit 2 performs image processing for reducing distortion at the attention position.

  The imaging unit 1 includes a CCD (Charge Coupled Device) camera or the like, generates video data including luminance information and color information, and outputs the video data to the image processing unit 2. The imaging unit 1 is not limited to a CCD camera, but may be an infrared camera used at night. The imaging unit 1 is mounted with the rear side of the host vehicle 10 as the imaging direction as shown in FIG. 2A, or is mounted with the front side of the host vehicle 10 as the imaging direction, as shown in FIG. 2B. In addition, the imaging unit 1 is a wide-angle camera in which optical components are selected in order to make the imaging range as wide as possible around the host vehicle.

  As shown in FIGS. 2A and 2B, the image processing unit 2 includes a sensor 11 such as a vehicle speed sensor or a steering angle sensor, a laser radar device that detects an obstacle behind the vehicle or in front of the vehicle, or the like. The sensor 12 is connected. The image processing unit 2 inputs sensor information from the sensors 11 and 12 and performs image correction processing according to the sensor information.

  The display unit 3 is composed of a liquid crystal display or the like provided at a position visible to the driver, and is supplied with video data that has been subjected to image correction processing by the image processing unit 2. The vehicle surrounding monitoring system includes an operation mechanism that instructs to display an image of the surroundings of the host vehicle around the display unit 3, and the video data generated by the imaging unit 1 in response to the operation of the operation mechanism. Is subjected to image correction processing by the image processing unit 2 and displayed on the display unit 3.

  In such a vehicle surrounding monitoring system, as shown in FIG. 2A, when the host vehicle 10 moves backward, the imaging unit 1 generates video data, the image processing unit 2 performs image correction processing, and the display unit 3 As shown in FIG. 2 (b), the image data is generated by the imaging unit 1 when the host vehicle 10 moves forward, the image correction process is performed by the image processing unit 2, and the display unit 3 presents the image data. It may be. In addition, as a situation where an image is displayed when the host vehicle 10 moves forward, for example, when an obstacle is avoided while moving forward on a narrow road or the like. Moreover, as a case where the periphery of the vehicle is imaged and displayed, it is conceivable that the lateral part of the vehicle is imaged by the imaging unit 1 incorporated in a door mirror or the like. Even in this case, an image correction process described later is performed. Of course, it may be. In the following description, as shown in FIG. 2B, a case where the front of the host vehicle 10 is set as the imaging direction of the imaging unit 1 will be described.

  In the vehicle surrounding monitoring system configured as described above, the imaging unit 1 is mounted at a position above the badge of the grill unit at the front of the vehicle and with the imaging direction set slightly downward, and the imaging unit 1 is configured with a wide-angle camera. If it is, the video data generated by the imaging unit 1 is as shown in FIG. According to FIG. 3, the video data from the imaging unit 1 includes the badge of the host vehicle 10 that is chrome-plated from the host vehicle 10 to the front side of the host vehicle 10 and the front bumper 20 of the host vehicle that is distorted into a U-shape. The white line 21 parallel to the host vehicle 10 is imaged approximately 30 centimeters from the width of the host vehicle 10, the white line 22 30 centimeters forward from the host vehicle 10, and 130 centimeters forward from the host vehicle 10. The white line 23, the white line 24 ahead of the vehicle 10 by 230 centimeters, and the like are captured.

  Here, the white lines 21, 22, and 23 are drawn for comparison with video data (FIG. 4) after image correction processing, which will be described later. Usually, white lines are not always drawn on the ground. Further, for example, when the vehicle is about 30 centimeters forward from the own vehicle 10 and there is an obstacle in one white line 22 portion and in the case where an obstacle exists in the other white line 22 portion in FIG. For a person, there is a possibility that it is recognized that the obstacle exists at a different distance from the host vehicle 10.

  That is, an obstacle present at a substantially central position in the width direction of the host vehicle 10 appears at a position in the image near the host vehicle 10, and an obstacle present near the end in the width direction of the host vehicle 10 It appears at a position in the image that exists far from the vehicle 10.

  On the other hand, the image processing unit 2 converts the video data as shown in FIG. 3 into the video data as shown in FIG. 4 when the video data as shown in FIG. 3 is input. I do. At this time, the image processing unit 2 sets the vicinity of the white line 22 as a specific position that is noticed by the driver, and increases the distortion correction amount from the specific position toward the width direction.

  That is, when the white line 22 that exists in the left-right direction with respect to the white line 22 at a specific position is moved downward in the image to make the plurality of white lines 22 straight, and similarly, when the plurality of white lines 22 are made straight. The image at the same position in the width direction is also moved downward in the image. Thereby, for example, the image correction process is also performed for distortion near the end in the width direction of the front bumper 20 of the host vehicle 10.

  In the video data shown in FIG. 4, the video distortion is corrected so that the white line 22 approximately 30 centimeters ahead of the host vehicle 10 is a straight line equidistant from the host vehicle 10. The image processing unit 2 extends the image of the front bumper 20 portion of the host vehicle 10 by about 30 centimeters in the width direction of the host vehicle 10. 4 makes it easy to intuitively grasp the positional relationship between the position in the image of the front bumper 20 of the host vehicle 10 and the white line 22.

"Specific examples of image correction processing"
Next, a specific example of a process for performing the image correction process on the video data shown in FIG. 3 to generate the video data shown in FIG. 4 will be described.

  In this image correction process, first, pixel data (pixel data) of video data input from the imaging unit 1 is divided into columns, and each column or each column is divided into a plurality of sections, and a predetermined number of pixels is obtained. By performing a process of shifting in the vertical direction and a process of expanding or reducing in the vertical direction at a predetermined magnification, the image in the horizontal direction at the specific position (region) is reconfigured so as to be a substantially straight line.

  In this image correction processing, the image processing unit 2 inputs video data as shown in FIG. In response to this, the image processing unit 2 converts the video data shown in FIG. 5A into a set of columns arranged in a grid as shown in FIG. 5B, thereby creating a raster image. Then, it is stored in a buffer (not shown). Here, each video area constituting the raster image may be a single pixel or a combination of a plurality of pixels. In this example, an image of 640 × 480 pixels is input from the imaging unit 1 to the image processing unit 2, and a single grid is formed by a plurality of pixels as shown in FIG. 5B.

  Next, the image processing unit 2 shifts in the vertical direction for each column having the same X-axis coordinate and arranged in the vertical direction in the video. More specifically, the driver's position of interest in the raster image shown in FIG. 5B is a specific position (region), and a white line having the same characteristic amount as the characteristic amount (for example, edge information) of the white line 22 at the specific position. The 22 feature values (for example, edge information) are shifted to the same X coordinate value in the width direction. This shift amount may be calculated from, for example, an arithmetic expression or read from a table.

In FIG. 5, when a video of 640 × 480 pixels is stored in the buffer and the shift amount of each vertical column is 0th in the leftmost vertical column in the video, the nth from the leftmost in the image. As shown in Equation 1 below,
((N−320) / 320) ² * (120−V) (Formula 1)
The pixel (region) calculated by the following equation is shifted downward. Here, V in Formula 1 is the speed of the host vehicle 10. At this time, the shift distance may be enlarged / reduced simultaneously with the shift, or the line may be divided vertically at the center of the video, and the respective divided sections may be enlarged / reduced at different magnifications.

  Next, the image processing unit 2 extracts an image area for one screen to be displayed on the display unit 3 as shown in FIG. 5D from the shifted raster image shown in FIG. As shown in (e), an image of only the extracted image area is created. That is, the image processing unit 2 shifts and discards video data that is not included in one screen, and extracts only the video data of the portion to be displayed on the display unit 3.

  According to such a vehicle surrounding monitoring system, the image processing unit 2 can alleviate an unnatural distortion of a video that makes it difficult to determine the distance between the host vehicle 10 and an obstacle or a white line. By displaying the image of the bumper of the host vehicle 10 distorted in a more natural shape, it is possible to provide an intuitively easy-to-understand image to the driver.

  As a result, according to the vehicle surrounding monitoring system, distortion can be corrected with almost no deterioration in image quality by a simple image processing in which a video portion having the same characteristics in the width direction of a specific position (region) is moved up and down. Can be provided to the driver.

  Next, a case where the image processing unit 2 inputs speed information of the host vehicle 10 and changes the specific position (region) to perform image correction processing will be described.

  The image processing unit 2 receives a vehicle speed signal from the sensors 11 and 12, determines whether the vehicle speed of the host vehicle 10 is stopped or low (for example, 5 km / h to 20 km / h), and stops or slows down. In the case of, the specific position (area) is set below the video, that is, near the host vehicle 10, and when the vehicle is not stopped or slow, the specific position (area) is set above the video as the vehicle speed increases. Set far from the host vehicle 10.

  In such an image correction process, when the image shown in FIG. 6A is input to the image processing unit 2, it is converted into the image shown in FIGS. 6B to 6G according to the vehicle speed. Then, when the vehicle is stopped (vehicle speed: 0 km / h), the image processing unit 2 sets the white line 22 to a specific position (region) and performs image correction processing, thereby lowering the lower part of the video. The video data shown in FIG. 6G with the corrected distortion is created.

  Further, as the vehicle speed becomes higher, the image correction process is performed on the image shown in FIGS. 6F to 6B with respect to FIG. That is, in FIG. 6G, the white line 22 close to the host vehicle 10 is corrected to a straight line in the width direction. However, as shown in FIGS. By moving inward and upward, the horizon away from the host vehicle 10 is gradually converted into a straight line.

  At this time, the image processing unit 2 is the case where the host vehicle 10 is stopped with respect to FIG. 6G, and the parameter “120−V” in the above-described equation 1 is set to “120”. f) is the case where the host vehicle 10 is traveling at 60 km / h, and the parameter “120-V” in the above equation 1 is set to “60”. In the case where the vehicle travels at 90 km / h, the parameter “120-V” in the above-described equation 1 is set to “30”, and the vehicle 10 travels at 120 km / h in FIG. In this case, the parameter “120-V” in the above equation 1 is set to “0”, and FIG. 6C shows the case where the host vehicle 10 is traveling at 150 km / h. The parameter “120-V” in FIG. And 0 ", the vehicle 10 for FIG 6 (b) is a case where running at 180 km / h, the" 120-V "parameter in Equation 1 above is" -60 ".

  Here, by setting the parameter to a positive value, it is possible to shift the vertical column away from the specific position in the width direction downward as described in FIG. 5, and by setting the parameter to a negative value, the specific position It is possible to shift the vertical column away from the width direction upward.

  As another process for shifting the vertical column in the vertical direction, for example, when the current speed is maintained, the distance traveled in a certain time such as 1 second is set as a specific position where the driver pays attention, and the distance is automatically The parameter may be set so that the distortion at the place where the vehicle 10 travels forward is most relaxed.

  According to the image processing unit 2 that performs such image correction processing, when the host vehicle 10 stops or travels at a low speed, it is possible to easily determine the distance from the host vehicle 10 to an obstacle that is located near the front. At the same time, when the vehicle is traveling at high speed and the vehicle speed is high, the specific position is moved upward in the image so as to reduce the distortion of the distant place that the driver pays attention to, and the image in the width direction at the specific position is distorted. Can be presented as an easy-to-understand video.

  Therefore, according to this vehicle surrounding monitoring system, for example, when stopping or traveling at a low speed, an image for avoiding contact with an obstacle near the host vehicle 10 that becomes a blind spot of the driver is presented to the driver. When traveling, it is possible to detect the distance between the preceding vehicle and the distance between the preceding vehicle and the driver by superimposing the distance between the preceding vehicle and the sensor 12 such as a reference line indicating the distance between the preceding vehicle and an infrared radar.

  That is, according to this vehicle surrounding monitoring system, the distance from the front of the host vehicle 10 to which the driver pays attention is estimated based on the speed of the host vehicle 10 based on the speed of the host vehicle 10, and the specific position is moved in the vertical direction. It is possible to correct the distortion of the vehicle and intuitively display the image of the place where the driver wants to see.

  Next, a case where the image processing unit 2 inputs the steering angle information of the host vehicle 10 and changes the specific position (region) to perform image correction processing will be described.

  The image processing unit 2 inputs not only a speed signal from the sensors 11 and 12 but also a steering angle signal, and sets the traveling direction of the host vehicle 10 as a specific position (region). That is, the image processing unit 2 obtains the video portion as a place where distortion should be most mitigated from the current steering angle, and performs image processing according to the location of the video.

  Here, when the host vehicle 10 stops or travels at a low speed (for example, 5 km / h to 20 km / h), the driver pays attention when there is no obstacle immediately in front of the host vehicle 10 from the host vehicle 10. It is a place about 10 meters away. Therefore, as shown in FIG. 7, the image processing unit 2 calculates the predicted travel locus of the host vehicle 10 estimated from the steering angle read from the steering angle sensors that are the sensors 11 and 12, and displays the predicted travel locus on the predicted travel locus. A point that has been advanced by about 10 meters is a point of interest. Then, a distance is calculated as to how far the point of interest on the predicted travel locus is away from the host vehicle 10.

  In addition, as a calculation method for obtaining the predicted travel locus, a known process in which an expected travel miracle is displayed superimposed on the image of the imaging unit 1 can be used. Based on the correspondence relationship, it can be determined how much the host vehicle 10 has traveled on the predicted travel locus and to what position in the image the host vehicle 10 has traveled.

  Therefore, according to the image processing unit 2, the driver's point of interest is estimated based on the current steering angle of the host vehicle 10, the position in the image of the point of interest is set as the specific position, and the specific position is set. On the other hand, image distortion can be corrected. This makes it possible to intuitively and accurately grasp the obstacle at the target position that is expected to be reached during steering.

  Further, in the image processing unit 2, when it is determined that the speed of the host vehicle 10 is stopped to low speed traveling (for example, 0 to 5 km / h), as illustrated in FIG. 8, the points of interest are a1, a2, and a3. As the steering angle increases, the specific position that is the place where the distortion should be most relaxed may be moved to a1 ′, a2 ′, a3 ′ in front of the host vehicle 10. Further, as another process, the image processing unit 2 assumes, for example, that the place where the driver pays attention is the position of the host vehicle 10 after 2 seconds when the traveling speed is maintained. The position of interest may be calculated using both, and the specific position in the video may be changed as needed.

  That is, when the host vehicle 10 is close to the stop state, the position that the driver wants to watch is immediately in front of the host vehicle 10. Therefore, it is possible to correct the distortion at the position in the video closer to the point of interest, and to grasp the obstacle more intuitively and accurately.

  Next, a case where the image processing unit 2 inputs obstacle information around the host vehicle 10 and changes the specific position (region) to perform image correction processing will be described.

  The image processing unit 2 inputs the direction and distance of the obstacle from the sensors 11 and 12, and sets the direction and distance of the obstacle as a specific position (region). That is, the image processing unit 2 obtains the video part that the driver should pay attention to from the direction and distance of the obstacle as the video part that should most relieve the distortion, and sets the video part as a specific position. Perform correction processing. Here, ultrasonic sensors can be used as the sensors 11 and 12 for detecting the direction and distance of the obstacle.

  The image processing unit 2 detects the distance from the obstacle using an ultrasonic sensor, sets a point away from the vehicle by that distance as a point of interest, and sets a position in the image corresponding to the point of interest as a specific position. Then, as described above, the image distortion in the width direction is most mitigated by shifting the image in the width direction in the vertical direction from the specific position.

  As a result, when there is an obstacle in the vicinity of the host vehicle 10, the driver can intuitively and accurately grasp the distance between the host vehicle 10 and the obstacle in order to reduce the distortion of the image around the obstacle most. And can be operated so as not to come into contact with obstacles easily.

  The above-described embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and various modifications can be made depending on the design and the like as long as the technical idea according to the present invention is not deviated from this embodiment. Of course, it is possible to change.

It is a block diagram which shows the structure of the vehicle periphery monitoring system to which this invention is applied. It is the example of attachment to the own vehicle of the vehicle periphery monitoring system to which this invention is applied, (a) is the structure for monitoring back, (b) is the structure for monitoring the front. It is a figure which shows the image | video imaged by the imaging part. It is a figure which shows the image | video after correction | amendment. It is a figure for demonstrating the image correction process of an image process part, Comprising: (a) Before correction | amendment, (b) Column division | segmentation process, (c) Vertical column movement process, (d) Video extraction process for one screen (E) After correction. It is a figure for demonstrating moving a specific position to an up-down direction according to the speed of the own vehicle, (a) Before correction | amendment and (b)-(g) after correction | amendment. It is a figure for demonstrating moving a specific position with the advancing direction according to the steering angle of the own vehicle. It is a figure for demonstrating moving a specific position to an up-down direction with the advancing direction according to the steering angle of the own vehicle.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging part 2 Image processing part 3 Display part 10 Own vehicle

Claims (8)

Image pickup means for picking up images around the vehicle;
Image processing means for inputting video captured by the imaging means and performing image correction processing on the video;
A display unit configured to input a video that has been subjected to image correction processing by the image processing unit and display the video;
The image processing means sets a specific position as a reference for image correction processing in the video imaged by the imaging means, shifts a video portion in the left-right direction up and down with respect to the specific position, and An image correction apparatus for a vehicle, wherein distortion in the image in the width direction of the vehicle is corrected linearly.   The image processing means sets a position in the video corresponding to a position of interest of the driver of the vehicle as the specific position, and shifts a video portion in a horizontal direction with respect to the position of interest in a vertical direction. The vehicular image correction apparatus according to claim 1, wherein   The image processing unit divides the video imaged by the imaging unit into a plurality of columns in which pixels are arranged in the vertical direction, and the vertical direction of the column increases as the position in the width direction of each column with respect to the specific position increases. The vehicular image correction apparatus according to claim 1, wherein the shift amount is increased.   The image processing means inputs the speed of the vehicle, sets the specific position below in the image when the vehicle is stopped, and increases the position above the image as the vehicle speed increases. The vehicular image correction apparatus according to claim 1, wherein a specific position is set.   2. The image processing unit according to claim 1, wherein the image processing unit inputs a steering angle of the vehicle to obtain a traveling direction, and sets the specific position by moving the specific position in the left-right direction in the video according to the traveling direction. The vehicle image correction apparatus according to the description.   The image processing means inputs a steering angle of the vehicle to obtain a traveling direction, moves the specific position in the left-right direction in the image according to the traveling direction, and the larger the steering angle of the vehicle, The vehicular image correction apparatus according to claim 1, wherein the specific position is moved upward.   The image processing means inputs the direction and distance of the obstacle with respect to the vehicle, and moves and sets the specific position in the left-right direction in the video according to the direction of the obstacle, and sets the distance of the obstacle. The vehicular image correction apparatus according to claim 1, wherein the specific position is set by moving the image vertically in response. Take a picture of the surroundings of the vehicle,
Input the captured video, perform image correction processing on the video,
An image correction method for a vehicle that inputs an image subjected to the image correction process and displays the image,
In the image processing, a specific position serving as a reference for image correction processing is set in the captured video, and a video portion in a horizontal direction is shifted in the vertical direction with respect to the specific position, so that An image correction method for a vehicle, wherein distortion in the image is corrected linearly.