JP2013129264A - Calibration method for on-vehicle camera and calibration device for on-vehicle camera - Google Patents

Abstract

A method and apparatus for calibrating an in-vehicle camera by a simple process that does not use a marker placed on a floor surface and without a special operation by a user.
An image capturing step for capturing an image with a camera while setting an exposure time to be a predetermined time or more during which a background image flows while the vehicle is moving, and an image of a bumper from a processing target image based on the captured image captured by the image capturing step. A specific image part extracting unit 14 for extracting a specific image part, a processing target image is rotated within a predetermined rotation angle range, and the specific image part is bisected by a line segment in the vertical coordinate direction passing through an intermediate point in the horizontal coordinate direction; A roll that calculates a rotation angle at which the degree of correlation between the mirror inverted image and the first image portion with respect to the line segment of the second image portion becomes the highest as the roll angle correction value for one image portion and the second image portion. An angle correction value calculating step.
[Selection] Figure 2

Description

  The present invention relates to a method for calibrating a camera mounted on a vehicle and an apparatus for calibrating a camera mounted on the vehicle.

  When detecting lane marks or other vehicles laid on the road from the image taken by the camera mounted on the vehicle and performing steering control of the own vehicle or judgment of possibility of contact with the object, the detected object It is necessary to accurately calculate the relative position between the vehicle and the host vehicle. Therefore, conventionally, calibration for accurately detecting the mounting position of the camera in the vehicle has been performed (see, for example, Patent Documents 1 and 2).

  In Patent Document 1, after a vehicle on which a camera is mounted is accurately aligned with a marker (calibration target) placed on the floor, the marker is imaged with the camera, and the image portion of the marker in the captured image is captured. A method for calculating parameters for camera calibration based on the position (two-dimensional coordinates) is described.

  Also, in Cited Document 2, an image in which an adjustment marker is superimposed on a captured image including a rear bumper of the host vehicle is displayed on a monitor, and the user changes the position of the adjustment marker to match the image portion of the rear bumper. A method for calculating parameters for camera calibration based on the amount of change is described.

JP 2010-107348 A JP 2007-256030 A

  In the method described in Patent Document 1, it is necessary to secure a certain amount of space in the vehicle production line and install the marker. In addition, periodic marker maintenance is required to prevent the feature points of the marker from being extracted from the captured image due to marker contamination or the like. Furthermore, the vehicle must be stopped with the position accurately aligned with the marker, and the vehicle must be kept stationary, which increases the number of work steps on the production line. .

  On the other hand, in the method described in Patent Document 2, it is not necessary to prepare a marker on the floor surface. However, in the case where a vehicle having a different bumper shape or mounting position is targeted, the shape and mounting position of each bumper are used. It is necessary to prepare adjustment marker data. In addition, in order for the user to perform a calibration operation, it is necessary for the user to learn knowledge and techniques relating to calibration.

  The present invention has been made in view of such a background, and provides a method and apparatus for calibrating an in-vehicle camera by a simple process that does not use a marker placed on a floor surface and without a special operation by a user. The purpose is to provide.

The present invention has been made in view of the above background, and the calibration method for an in-vehicle camera of the present invention is as follows.
A method of calibrating a camera mounted on a vehicle so that a specific part of the vehicle having horizontal symmetry is included in an imaging range,
An imaging step in which the camera captures the exposure time as a predetermined time or more during which the background image of the specific portion flows during movement of the vehicle;
A specific image portion extraction step of extracting a specific image portion which is an image of the specific portion from a processing target image based on the captured image captured by the imaging step;
The first image portion and the second image portion obtained by rotating the processing target image within a predetermined rotation angle range and bisecting the specific image portion by a line segment in the vertical coordinate direction passing through an intermediate point in the horizontal coordinate direction. A roll angle correction value calculating step of calculating, as a roll angle correction value, a rotation angle at which the degree of correlation between the mirror inverted image with respect to the line segment of the image portion and the first image portion is highest or equal to or greater than a predetermined threshold; ,
A calibration step of performing calibration processing of the camera using the roll angle correction value (first invention).

  According to the first aspect of the present invention, in the imaging step, while the vehicle is moving, imaging is performed by the camera with the exposure time being a predetermined time during which the background image of the specific portion flows. In this case, since the specific portion is always captured by the camera, the image hardly flows. However, since the background image other than the specific portion of the captured image flows due to the movement of the vehicle, the contrast is lowered. Therefore, in the specific image portion extraction step, the specific image portion can be extracted with high accuracy from other image regions of the processing target image.

  Since the specific image portion has horizontal symmetry, when the processing target image is rotated within a predetermined rotation angle range in the roll angle correction value calculating step, the specific image portion is For the first image portion and the second image portion that are bisected by a line segment that passes through an intermediate point in the horizontal coordinate direction, the degree of correlation between the mirror inverted image and the first image portion with respect to the line segment of the second image portion is When the specific image portion becomes horizontal, the height is highest. Therefore, the rotation angle of the image to be processed when the correlation degree becomes the highest or exceeds the predetermined threshold value is calculated as a correction value for the roll angle (rotation angle around the optical axis of the camera). In the calibration step, the camera calibration process can be performed using the roll angle correction value.

  Thus, according to the first invention, by using the specific image portion that is a captured image of the specific portion having horizontal symmetry, the vehicle can be easily processed without using a marker placed on the floor. The in-vehicle camera can be calibrated without any special operation by the user.

Further, a method for calibrating a camera mounted on a vehicle so that a specific part of the vehicle having symmetry in the horizontal direction is included in the imaging range,
An imaging step of performing imaging a plurality of times with the camera during movement of the vehicle;
A specification for generating a processing target image that has been processed to remove different image portions between a plurality of captured images captured by the imaging step, and extracting a specific image portion that is an image of the specific portion from the processing target image Image partial extraction process;
The first image portion and the second image portion obtained by rotating the processing target image within a predetermined rotation angle range and bisecting the specific image portion by a line segment in the vertical coordinate direction passing through an intermediate point in the horizontal coordinate direction. A roll angle correction value calculating step of calculating, as a roll angle correction value, a rotation angle at which the degree of correlation between the mirror inverted image with respect to the line segment of the image portion and the first image portion is highest or equal to or greater than a predetermined threshold; ,
A calibration step of performing calibration processing of the camera using the roll angle correction value (second invention).

  According to the second invention, in the imaging step, imaging is performed a plurality of times by the camera while the vehicle is moving. In this case, although the specific part is always imaged by the camera, the imaging object and its position change in accordance with the movement of the vehicle in the area other than the image of the specific part of the captured image.

  Therefore, in the specific image portion extraction step, processing (such as AND, EX-OR, etc.) for removing the changed image portion is performed between the plurality of captured images captured in the imaging step, and the specific image portion It is possible to generate the processing target image excluding the image portion other than as much as possible. Then, the specific image portion can be accurately extracted from the processing target image.

  As in the first invention, the roll angle correction value is calculated in the roll angle correction value calculation step, and the camera is calibrated using the roll angle correction value in the calibration step. Can do.

  Thus, according to the second invention, by using the specific image portion that is a captured image of the specific portion having horizontal symmetry, by a simple process that does not use a marker or the like placed on the floor, The in-vehicle camera can be calibrated without any special operation by the user.

In the first invention or the second invention,
Yaw angle correction for calculating a yaw angle correction value for correcting the deviation amount based on the deviation amount between the intermediate position in the horizontal coordinate direction of the specific image portion in the processing target image and a predetermined horizontal reference position. Including a value calculation step,
In the calibration step, the camera is calibrated using the yaw angle correction value (third invention).

  According to the third invention, in the calibration step, the yaw angle correction value for correcting the yaw angle (horizontal shake angle) of the camera is calculated by the yaw angle correction value calculation step. The camera can be calibrated.

In any one of the first to third inventions,
A pitch angle correction value for calculating a pitch angle correction value for correcting the shift amount based on a shift amount between the intermediate position in the vertical coordinate direction of the specific image portion in the captured image and a predetermined vertical reference position. Including the calculation process,
In the calibration step, the camera calibration process is performed using the pitch angle correction value (fourth invention).

  According to the fourth invention, the pitch angle correction value for correcting the pitch angle (vertical shake angle) of the camera is calculated by the pitch angle correction value calculating step, and the pitch angle correction value is calculated in the calibration step. The camera can be calibrated using the angle correction value.

Next, the in-vehicle camera calibration device of the present invention is
A camera mounted on the vehicle such that a specific portion of the vehicle having horizontal symmetry is included in the imaging range;
An imaging unit that captures the exposure time as a predetermined time during which the background image of the specific part flows during movement of the vehicle;
A specific image portion extraction unit that extracts a specific image portion that is an image of the specific portion from a processing target image based on a captured image captured by the imaging unit;
The first image portion and the second image portion obtained by rotating the processing target image within a predetermined rotation angle range and bisecting the specific image portion by a line segment in the vertical coordinate direction passing through an intermediate point in the horizontal coordinate direction. A roll angle correction value calculation unit that calculates a rotation angle at which the degree of correlation between the mirror inverted image and the first image portion with respect to the line segment of the image portion is the highest or equal to or greater than a predetermined threshold, as a roll angle correction value; ,
A calibration unit that performs camera calibration processing using the roll angle correction value is provided (fifth invention).

  According to the fifth invention, the imaging by the camera is performed by the imaging unit with the exposure time being a predetermined time during which the background image of the specific part flows during the movement of the vehicle. In this case, since the specific portion is always captured by the camera, the image hardly flows. However, since the background image other than the specific portion of the captured image flows due to the movement of the vehicle, the contrast is lowered. Therefore, the specific image portion extraction unit can accurately extract and extract the specific image portion from other image regions of the processing target image.

  Since the specific image portion has horizontal symmetry, when the roll angle correction value calculation unit rotates the processing target image within a predetermined rotation angle range, the specific image portion For the first image portion and the second image portion that are bisected by a line segment that passes through an intermediate point in the horizontal coordinate direction, the degree of correlation between the mirror inverted image and the first image portion with respect to the line segment of the second image portion is When the specific image portion becomes horizontal, the height is highest. Therefore, the rotation angle of the image to be processed when the correlation degree becomes the highest or becomes the predetermined threshold value or more can be calculated as a roll angle correction value, and the calibration unit can correct the roll angle correction. The camera can be calibrated using the values.

  Thus, according to the fifth aspect, by using the specific image portion that is a captured image of the specific portion having symmetry in the horizontal direction, the vehicle can be easily processed without using the marker placed on the floor. The in-vehicle camera can be calibrated without any special operation by the user.

In addition, a camera mounted on the vehicle so that a specific portion of the vehicle having horizontal symmetry is included in the imaging range;
An imaging unit that performs imaging multiple times with the camera during movement of the vehicle;
A specification for generating a processing target image that has been processed to remove different image portions between a plurality of captured images captured by the imaging unit, and extracting a specific image portion that is an image of the specific portion from the processing target image An image partial extractor;
The first image portion and the second image portion obtained by rotating the processing target image within a predetermined rotation angle range and bisecting the specific image portion by a line segment in the vertical coordinate direction passing through an intermediate point in the horizontal coordinate direction. A roll angle correction value calculation unit that calculates a rotation angle at which the degree of correlation between the mirror inverted image and the first image portion with respect to the line segment of the image portion is the highest or equal to or greater than a predetermined threshold, as a roll angle correction value; ,
A calibration unit that performs camera calibration processing using the roll angle correction value is provided (sixth invention).

  According to the sixth aspect of the present invention, the imaging unit performs imaging a plurality of times while the vehicle is moving. In this case, although the specific part is always imaged by the camera, the imaging object and its position change in accordance with the movement of the vehicle in the area other than the image of the specific part of the captured image.

  Therefore, the specific image portion extraction unit performs processing (AND, EX-OR, etc.) of removing the changed image portion between the plurality of captured images captured by the imaging unit, and the specific image portion It is possible to generate the processing target image excluding the image portion other than as much as possible. Then, the specific image portion can be accurately extracted from the processing target image.

  Then, similarly to the fifth invention, the roll angle correction value is calculated by the roll angle correction value extraction unit, and the calibration processing of the camera is performed by using the roll angle correction value by the calibration unit. it can.

  Thus, according to the sixth invention, by using the specific image portion that is a captured image of the specific portion having horizontal symmetry, by a simple process that does not use a marker or the like placed on the floor, The in-vehicle camera can be calibrated without any special operation by the user.

In the fifth invention or the sixth invention,
Yaw angle correction for calculating a yaw angle correction value for correcting the deviation amount based on the deviation amount between the intermediate position in the horizontal coordinate direction of the specific image portion in the processing target image and a predetermined horizontal reference position. With a value calculator,
The calibration unit performs calibration processing of the camera using the yaw angle correction value (seventh invention).

  According to the seventh invention, the yaw angle correction value calculation unit calculates the yaw angle correction value for correcting the yaw angle (horizontal shake angle) of the camera, and the calibration unit calculates the yaw angle correction value. The camera can be calibrated using the angle correction value.

In any one of the fifth to seventh inventions,
A pitch angle correction value for calculating a pitch angle correction value for correcting the shift amount based on a shift amount between a vertical intermediate position of the specific image portion in the processing target image and a predetermined vertical reference position. With a calculator
The calibration unit performs calibration processing of the camera using the pitch angle correction value (eighth invention).

  According to an eighth aspect of the invention, the pitch angle correction value calculating unit calculates the pitch angle correction value for correcting the pitch angle (vertical shake angle) of the camera, and the calibration unit calculates the pitch angle. The camera can be calibrated using the angle correction value.

Explanatory drawing of the usage condition of the calibration apparatus of a vehicle-mounted camera. The block diagram of the calibration apparatus of a vehicle-mounted camera. 6 is a flowchart of camera parameter calculation processing. Explanatory drawing of the picked-up image in the state which the vehicle has stopped, and its edge extraction image. Explanatory drawing of the captured image in the state which the vehicle is moving, and its edge extraction image. Explanatory drawing of a calculation process of a roll angle correction value. Explanatory drawing of the extraction process of the specific image part in 2nd Embodiment.

  An embodiment of a calibration device for an in-vehicle camera according to the present invention will be described with reference to FIGS. With reference to FIG. 1, the on-vehicle camera calibration device of the present embodiment is provided as a part of the function of the controller 10 mounted on the vehicle 1 (own vehicle). The controller 10 displays the captured image of the camera 21 mounted on the vehicle 1 on the display 23 in the passenger compartment, and assists the driver.

  The camera 21 is attached at a position where the rear bumper 25 of the vehicle 1 enters the imaging range, and images the rear of the vehicle 1. The camera 21 captures an image in accordance with a control signal from the controller 10 and outputs captured image data to the controller 10.

  The controller 10 is an electronic circuit unit including a CPU, a memory, and the like (not shown). By causing the CPU to execute a camera parameter calculation program for the camera 21, the vehicle movement determination unit 11, the imaging unit 12, and the processing region setting are performed. 13, specific image part extraction unit 14, body-dependent amount correction unit 15, and camera parameter correction unit 16 (functions of roll angle correction value calculation unit, yaw angle correction value calculation unit, pitch angle correction value calculation unit of the present invention) Function).

  Further, the controller 10 includes camera parameters (roll angle correction value, yaw angle correction value, pitch angle correction value, which will be described later) calculated by the camera parameter correction unit 16, lens distortion parameters of the camera 21, and attachment positions of the camera 21. A parameter storage memory 17 is provided for storing an offset value or the like from the center of the vehicle 1 in the vehicle width direction.

[First Embodiment]
Next, a first embodiment of camera parameter calculation processing by the controller 10 will be described according to the flowchart shown in FIG.

  STEP 1 in FIG. 3 is processing by the vehicle movement determination unit 11. The vehicle movement determination unit 11 determines whether or not the vehicle 1 is moving from the detection signal of the movement speed of the vehicle 1 input from the vehicle speed sensor 22. In the manufacturing process of the vehicle 1 or the like, when the vehicle 1 is being transported by the transport device, it may be determined whether or not the vehicle 1 is moving based on a signal from the transport device.

  If the vehicle 1 is not moving, the process branches to STEP13. In this case, the camera parameter is not calculated. On the other hand, when the vehicle 1 is moving, the process proceeds to STEP2. STEP 2 is processing performed by the imaging unit 12, and the imaging unit 12 performs imaging with a camera 21 with a longer exposure time. Note that the step of performing the processing according to STEP 2 corresponds to the “imaging step” of the present invention.

  Here, FIG. 4A shows a captured image Im1 with a normal exposure time by the camera 21 (exposure time at which the background image does not flow), and an image of the bumper 25 of the vehicle 1 (see FIG. 1). 30 are included. FIG. 4B is an edge extraction image Im2 obtained by extracting edge points from the captured image Im1 of FIG.

  In the edge extracted image Im2, a pixel (edge point) whose luminance changes by a predetermined level or more is “1” (black in FIG. 4B), and a pixel other than the edge point is “0” (white in FIG. 4B). In addition to the edge portion of the image portion 31 extracted from the image of the bumper 25, a number of edge portions are also extracted from the images of surrounding objects.

  On the other hand, FIG. 5A shows a captured image Im3 with a long exposure time by the camera 21 (exposure time through which a background image flows, which corresponds to an exposure time longer than a predetermined time of the present invention). . In the captured image Im3, the image 40 of the bumper 25 of the vehicle 1 is captured with a relatively high contrast, whereas the other objects in the background flow and the contrast is low.

  FIG. 5B is an edge extraction image Im4 obtained by extracting an edge from the captured image Im3 of FIG. 5A, and is extracted in addition to the edge portion of the image portion 41 extracted from the image 40 of the bumper 25. The portion is greatly reduced as compared with FIG. Therefore, the image portion of the bumper 25 can be detected with high accuracy from the edge extracted image Im4 of FIG.

  The subsequent STEP 3 is processing by the processing area setting unit 13. The processing area setting unit 13 is an area used for the calibration process of the camera 21 in the captured image of the camera 21 based on the relationship between the mounting position and orientation of the camera 21 and the mounting position of the bumper 25 in the design of the vehicle 1. (Processing area) is set. Note that data about the relationship between the mounting position and orientation of the camera 21 and the mounting position of the bumper 25 in the design of the vehicle 1 is stored in the parameter storage memory 17 in advance.

  The next STEP 4 to STEP 6 are processes performed by the specific image portion extraction unit 14. In STEP 4, the specific image portion extraction unit 14 performs edge extraction processing on an image captured by the camera 21 with long exposure using an edge operator such as Canny. In STEP 5, the specific image portion extraction unit 14 performs noise removal that removes edge points whose edge strength is a predetermined level or less and edge points whose run length is a predetermined length or less.

  In STEP 6, the specific image portion extraction unit 14 extracts a specific image portion that is an image of the bumper 25 from the image subjected to edge extraction and noise removal. The process of performing STEP 4 to STEP 6 corresponds to the “specific image partial extraction process” of the present invention.

  Subsequent STEP 7 to STEP 8 are processes performed by the vehicle body dependence amount correction unit 15. In STEP 7, the vehicle body dependence correction unit 15 determines whether or not the mounting position of the camera 21 is offset from the center of the vehicle 1 in the vehicle width direction (left-right direction). In the present embodiment, as shown in FIG. 1, the mounting position of the camera 21 is offset from the intermediate position in the vehicle width direction of the vehicle 1. Therefore, go to STEP8.

  When the camera 21 is offset from the intermediate position in the vehicle width direction of the vehicle 1, in the captured image of the camera 21, the intermediate position in the horizontal coordinate direction (x direction) of the image portion of the bumper 25 is the intermediate of the horizontal coordinate of the image. It will deviate from the point. Therefore, the vehicle body dependence amount correction unit 15 generates a processing target image by correcting the captured image so that the intermediate point in the horizontal direction (x direction) of the image portion 41 of the bumper 25 is the intermediate position of the horizontal coordinate of the image. To do.

  In the processing target image, the intermediate position of the image portion of the bumper 25 in the horizontal coordinate direction substantially coincides with the intermediate point of the horizontal coordinate of the image. On the other hand, when the camera is not offset from the intermediate position in the vehicle width direction of the vehicle, the process branches from STEP 7 to STEP 9, and the process of STEP 8 by the vehicle body dependent amount correction unit 15 is not performed.

  Subsequent STEP 9 to STEP 12 are processes performed by the camera parameter correction unit 16. Hereinafter, the edge extraction image Im4 in FIG. 5B will be described as a processing target image. The camera parameter correction unit 16 calculates a roll angle correction value in STEP9. The roll angle correction value is calculated by rotating the processing target image Im4 within a rotation angle range (for example, a rotation angle range of ± 4 degrees) determined in consideration of the assumed mounting error width of the camera 21. 25 image portions 41 (corresponding to specific image portions of the present invention; hereinafter referred to as specific image portions 41) are bisected by a line segment Lc in the vertical coordinate (y coordinate) direction passing through an intermediate position in the horizontal coordinate (x coordinate) direction. This is done by calculating the degree of correlation between the first image portion 42a and the second image portion 42b.

  Here, with reference to FIG. 6A, when the specific image portion 41 is bisected by a line segment Lc in the y-axis direction passing through the midpoint of the x direction (the midpoint of x1 and x3) x2, FIG. As shown in a), as the inclination of the specific image portion 41 (inclination with respect to the x-axis and y-axis) increases, the correlation between the mirror inverted image 42bm and the first image portion 42a with respect to the line segment Lc of the second image portion 42b. The degree becomes lower. Further, when the inclination of the specific image portion 41 is 0 degree, the degree of correlation between the mirror inverted image 42bm with respect to the line segment Lc of the second image portion 42b and the first image portion 42a becomes high.

  Therefore, the camera parameter correction unit 16 rotates the processing target image Im4 by a predetermined angle and calculates the degree of correlation between the first image portion 42a and the mirror inverted image 42bm at each rotation angle. Then, the rotation angle at which the correlation degree is equal to or higher than the predetermined threshold and the correlation degree is the highest is calculated as the roll angle correction value. The step of performing STEP 9 corresponds to the “roll angle correction value calculating step” of the in-vehicle camera calibration method of the present invention. Further, the rotation angle at which the correlation degree is the highest, or the rotation angle at which the correlation degree is equal to or greater than a predetermined threshold value may be calculated as the roll angle correction value.

  In subsequent STEP 10, as shown in FIG. 6B, the camera parameter correction unit 16 sets the x-axis component bx of the reference position B on the processing target image set in advance (corresponding to the horizontal reference position of the present invention). A deviation amount dx between the specific image portion 41 and the intermediate position x2 in the x-axis direction is calculated as a yaw angle correction value. The step of performing STEP 10 corresponds to the “yaw angle correction value calculating step” of the present invention.

  The degree of correlation between the mirror inverted image 42bm of the second image part 42b and the first image part 42a is calculated by shifting the position of Lc within a predetermined range in the x-axis direction. In addition, the difference between the position of Lc when the degree of correlation becomes the highest and the position of Lc before shifting may be calculated as the yaw angle correction value.

  In subsequent STEP 11, the camera parameter correction unit 16 sets the y-axis component by (corresponding to the vertical reference position of the present invention) of the reference position B on the preset processing target image and the y-axis direction of the specific image portion 41. A deviation dy from the intermediate position y2 is calculated as a pitch angle correction value. The step of performing STEP 11 corresponds to the “pitch angle correction value calculation step” of the in-vehicle camera calibration method of the present invention.

  In the next STEP 12, the camera parameter correction unit 16 writes the data of the roll angle correction value, the yaw angle correction value, and the pitch angle correction value calculated in STEP 9 to STEP 11 in the parameter storage memory 17 and stores them in STEP 13. Proceed and finish the process.

  The calibration unit 18 performs a calibration process for rotating and shifting the captured image of the camera 21 using the roll angle correction value, the yaw angle correction value, and the pitch angle correction value based on the data stored in the parameter storage memory 17. The rear image as viewed from the center of the vehicle 1 in the vehicle width direction is displayed on the display 23. The process of performing the calibration process corresponds to the “calibration process” of the present invention.

  Further, the calibration process of the camera 21 may be performed in the manufacturing configuration of the vehicle 1 or may be appropriately performed while the vehicle 1 is being used (running).

[Second Embodiment]
Next, a second embodiment of the camera parameter calculation process by the controller 10 will be described with reference to FIG. In the first embodiment described above, the controller 10 extracts the specific image portion (bumper image) from the captured image obtained by the long-time exposure in STEP2 to STEP6 in FIG. 3, but in the second embodiment, the movement of the vehicle 1 is performed. A specific image portion is extracted from the two images Im10 and Im20 captured by the imaging unit 12 at different points in time.

  The specific image portion extraction unit 14 generates an edge extraction image Im11 in which an edge point is extracted from Im10, and also generates an edge extraction image Im21 in which an edge point is extracted from Im20. The edge extraction images Im11 and Im21 are binary images in which the pixel at the edge point is 1 (black in FIG. 7) and the pixels other than the edge point are 0 (white in FIG. 7).

  Then, the specific image portion extraction unit 14 performs an AND process between the corresponding pixels at the same coordinate position in the edge extraction images Im11 and Im21 to generate an image Im30. In this case, since the picked-up images Im10 and Im20 have different pick-up points, the background images other than the image portions 50 and 60 of the bumper are changed.

  Therefore, in the edge extraction images Im11 and Im21, the bumper edge images 51 and 61 are extracted with substantially the same shape, but different edge points are extracted for the background. Accordingly, in the image Im30 obtained by performing AND processing between the corresponding pixels at the same coordinate position in the edge extracted images Im11 and Im21, most of the edge portions other than the edge portion of the image portion 70 of the bumper 25 are deleted.

  Then, the specific image portion extraction unit 14 extracts a specific image portion 70 that is an image portion of the bumper 25 after performing noise removal that removes an edge point or the like whose edge point is a predetermined length or less from the image Im30. The processing after extracting the specific image portion 70 is the same as STEP 7 to STEP 12 in FIG.

  In the second embodiment, the AND process is performed between the corresponding pixels of the edge extraction images Im11 and Im21, but an EX-OR (exclusive OR) process may be performed. Further, for example, by performing processing between the captured images Im10 and Im20 before edge extraction, setting a pixel whose luminance change is equal to or higher than a predetermined level to 0, and setting a pixel whose luminance change is lower than the predetermined level to 1, The background image other than the bumper image may be set to zero.

  In the present embodiment, the bumper 25 at the rear portion of the vehicle 1 is used as the specific portion having the symmetry in the horizontal direction of the present invention in the vehicle 1, but other portions having the symmetry in the horizontal direction are used. It may be used as a specific part.

  Further, in the present embodiment, an example in which the camera is attached to the rear part of the vehicle and the rear of the vehicle is imaged is shown, but the present invention can be applied even when the camera is attached to the front part or the side surface of the vehicle. It is.

  In the present embodiment, the roll angle correction value, the yaw angle correction value, and the pitch angle correction value are calculated as camera parameters. However, if at least the roll angle correction value is calculated, the present invention can be applied. Is possible.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle, 10 ... Controller, 11 ... Vehicle movement determination part, 12 ... Imaging part, 13 ... Processing area setting part, 14 ... Specific image part extraction part, 15 ... Car body dependence amount correction | amendment part, 16 ... Camera parameter correction part, 17 ... Parameter storage memory, 18 ... Calibration unit, 21 ... Camera, 23 ... Display, 25 ... Bumper.

Claims (8)

A method of calibrating a camera mounted on a vehicle so that a specific part of the vehicle having horizontal symmetry is included in an imaging range,
An imaging step in which the camera captures an exposure time as a predetermined time during which a background image of the image of the specific part flows during movement of the vehicle;
A specific image portion extraction step of extracting a specific image portion which is an image of the specific portion from a processing target image based on the captured image captured by the imaging step;
The first image portion and the second image portion obtained by rotating the processing target image within a predetermined rotation angle range and bisecting the specific image portion by a line segment in the vertical coordinate direction passing through an intermediate point in the horizontal coordinate direction. A roll angle correction value calculating step of calculating, as a roll angle correction value, a rotation angle at which the degree of correlation between the mirror inverted image with respect to the line segment of the image portion and the first image portion is highest or equal to or greater than a predetermined threshold; ,
And a calibration step of performing calibration processing of the camera using the roll angle correction value. A method of calibrating a camera mounted on a vehicle so that a specific part of the vehicle having horizontal symmetry is included in an imaging range,
An imaging step of performing imaging a plurality of times with the camera during movement of the vehicle;
A specification for generating a processing target image that has been processed to remove different image portions between a plurality of captured images captured by the imaging step, and extracting a specific image portion that is an image of the specific portion from the processing target image Image partial extraction process;
The first image portion and the second image portion obtained by rotating the processing target image within a predetermined rotation angle range and bisecting the specific image portion by a line segment in the vertical coordinate direction passing through an intermediate point in the horizontal coordinate direction. A roll angle correction value calculating step of calculating, as a roll angle correction value, a rotation angle at which the degree of correlation between the mirror inverted image with respect to the line segment of the image portion and the first image portion is highest or equal to or greater than a predetermined threshold; ,
And a calibration step of performing calibration processing of the camera using the roll angle correction value. In the in-vehicle camera calibration method according to claim 1 or 2,
Yaw angle correction for calculating a yaw angle correction value for correcting the deviation amount based on the deviation amount between the intermediate position in the horizontal coordinate direction of the specific image portion in the processing target image and a predetermined horizontal reference position. Including a value calculation step,
A calibration method for an in-vehicle camera, wherein the camera calibration process is performed using the yaw angle correction value in the calibration step. In the in-vehicle camera calibration method according to any one of claims 1 to 3,
A pitch angle correction value for calculating a pitch angle correction value for correcting the shift amount based on a shift amount between the intermediate position in the vertical coordinate direction of the specific image portion in the captured image and a predetermined vertical reference position. Including the calculation process,
A calibration method for an in-vehicle camera, wherein the camera calibration process is performed using the pitch angle correction value in the calibration step. A camera mounted on the vehicle such that a specific portion of the vehicle having horizontal symmetry is included in the imaging range;
An imaging unit that captures the exposure time as a predetermined time during which the background image of the specific part flows during movement of the vehicle,
A specific image portion extraction unit that extracts a specific image portion that is an image of the specific portion from a processing target image based on a captured image captured by the imaging unit;
The first image portion and the second image portion obtained by rotating the processing target image within a predetermined rotation angle range and bisecting the specific image portion by a line segment in the vertical coordinate direction passing through an intermediate point in the horizontal coordinate direction. A roll angle correction value calculation unit that calculates a rotation angle at which the degree of correlation between the mirror inverted image and the first image portion with respect to the line segment of the image portion is the highest or equal to or greater than a predetermined threshold, as a roll angle correction value; ,
An in-vehicle camera calibration apparatus comprising: a calibration unit that performs camera calibration processing using the roll angle correction value. A camera mounted on the vehicle such that a specific portion of the vehicle having horizontal symmetry is included in the imaging range;
An imaging unit that performs imaging multiple times with the camera during movement of the vehicle;
A specification for generating a processing target image that has been processed to remove different image portions between a plurality of captured images captured by the imaging unit, and extracting a specific image portion that is an image of the specific portion from the processing target image An image partial extractor;
The first image portion and the second image portion obtained by rotating the processing target image within a predetermined rotation angle range and bisecting the specific image portion by a line segment in the vertical coordinate direction passing through an intermediate point in the horizontal coordinate direction. A roll angle correction value calculation unit that calculates a rotation angle at which the degree of correlation between the mirror inverted image and the first image portion with respect to the line segment of the image portion is the highest or equal to or greater than a predetermined threshold, as a roll angle correction value; ,
An in-vehicle camera calibration apparatus comprising: a calibration unit that performs camera calibration processing using the roll angle correction value. In the in-vehicle camera calibration device according to claim 5 or 6,
Yaw angle correction for calculating a yaw angle correction value for correcting the deviation amount based on the deviation amount between the intermediate position in the horizontal coordinate direction of the specific image portion in the processing target image and a predetermined horizontal reference position. With a value calculator,
The in-vehicle camera calibration apparatus, wherein the calibration unit performs calibration processing of the camera using the yaw angle correction value. In the in-vehicle camera calibration device according to any one of claims 5 to 7,
A pitch angle correction value for calculating a pitch angle correction value for correcting the shift amount based on a shift amount between a vertical intermediate position of the specific image portion in the processing target image and a predetermined vertical reference position. With a calculator
The in-vehicle camera calibration apparatus, wherein the calibration unit performs a calibration process of the camera using the pitch angle correction value.

Images