JP6677141B2 - Parking frame recognition device - Google Patents

Description

  The present disclosure relates to a technology for recognizing a parking frame that indicates a parking frame in which a host vehicle can park.

  Patent Literature 1 below discloses, as a technique for recognizing a parking available frame, among white lines constituting a parking frame observable from the host vehicle, a white line farther from the host vehicle than a white line closer to the host vehicle is used. A technique has been proposed for determining that the frame is a parking available frame when the frame is long.

JP-A-2006-016681

  According to the technology disclosed in Patent Document 1, when a vehicle such as a minicar having a short vehicle length is parked in a parking frame, an observation is made such that a white line on a side farther to the host vehicle is longer than a white line on a side closer to the host vehicle. Therefore, there is a problem that a parking frame in which another vehicle is parked is easily erroneously recognized as a parking available frame.

  The present disclosure provides a technology that enables a parking frame recognition device that recognizes a parking frame indicating a parking frame in which the host vehicle can park to more accurately recognize the parking frame.

  The parking frame recognition device (20) of the present disclosure includes an image acquisition unit (S110), a line extraction unit (S120, S150), and a possible frame recognition unit (S210, S220). The image acquisition unit is configured to acquire a plurality of captured images representing respective captured images captured by a plurality of imaging devices arranged at different positions of the host vehicle.

  The line extracting unit is configured to extract a parking frame from the plurality of captured images, and to extract a target line representing a line defining an end of the parking frame on a side farther from the host vehicle. The possible frame recognition unit determines whether the shape of the target line is the same for the same parking frame extracted from the plurality of captured images, and can park the parking frame determined to be the same in the shape of the target line. It is configured to be recognized as a frame.

  That is, in the parking frame recognition device of the present disclosure, when a parked vehicle is present in the parking frame, the shape of the same target line imaged at different positions is different because the target line is hidden by the parked vehicle. Utilize the property of being observed. The shape of the target line can be determined based on how many comparison parameters such as the number of corners of the line, the length of the line, the aspect ratio, and the curvature of the edge match. The degree of coincidence can be determined, for example, by comparing the ratio of the comparison parameter with a threshold.

  According to such a parking frame recognition device, it is determined whether or not the same target line imaged at a different position is the same as the shape of the same target line. Can be recognized.

  Note that reference numerals in parentheses described in this column and in the claims indicate a correspondence relationship with specific means described in the embodiment described below as one aspect, and the technical scope of the present disclosure will be described. It is not limited.

It is a block diagram showing the composition of an imaging system. It is a flowchart which shows a parking available frame recognition process. FIG. 3 is an image diagram illustrating an example of an image captured by a front camera. It is an image figure showing an example of the picture picked up by the left camera. It is a bird's-eye view which shows an example of how a target line looks.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
[1. Embodiment]
[1-1. Constitution]
As shown in FIG. 1, the imaging system 1 of the present embodiment is a system mounted on a vehicle such as a passenger car, and includes a control unit 10. Further, a front camera 4F, a rear camera 4B, a right camera 4R, a left camera 4L, a display unit 30, a vehicle control unit 32, and the like may be provided. Note that a vehicle on which the imaging system 1 is mounted is also referred to as a host vehicle.

  The front camera 4F and the rear camera 4B are for capturing images of a road ahead and behind the host vehicle, respectively, and are attached to a front part and a rear part of the host vehicle. The right camera 4R and the left camera 4L are for capturing images of the right and left roads of the own vehicle, respectively, and are attached to the right and left surfaces of the own vehicle. That is, the front camera 4F, the rear camera 4B, the right camera 4R, and the left camera 4L are arranged at different positions of the own vehicle. Here, different positions indicate that the imaging devices are not arranged on the same vertical line but are arranged on different vertical lines.

  The control unit 10 generates a bird's-eye view image of the road around the vehicle from a vertical direction from the images captured by the cameras 4F, 4B, 4R, and 4L. Then, the generated bird's-eye image is displayed on the display unit 30 which is configured by a liquid crystal display or the like and is arranged in the vehicle interior.

  Further, the control unit 10 recognizes the parkable frame from the images captured by the cameras 4F, 4B, 4R, and 4L. Note that a parking frame is an object having at least one line having a width within a preset range and a side or line parallel to the line at a position separated from the line by a distance corresponding to the width of the vehicle. Represents the area between.

  In addition, the parkable frame indicates a parking frame in which the host vehicle can park. The term "parallel" used herein includes substantially parallel. In addition, the “object having sides or lines parallel to the line” here includes, for example, curbs, other lines, walls, trees, guardrails, and the like.

The control unit 10 includes an imaging signal input unit 12, a detection signal input unit 14, a memory 16, a display control unit 18, and an image processing unit 20.
The imaging signal input unit 12 is for taking in imaging signals from the front camera 4F, the rear camera 4B, the right camera 4R, and the left camera 4L, and inputting them to the image processing unit 20 as captured image data.

  The detection signal input unit 14 takes in detection signals from a wheel speed sensor 6 for detecting the rotational speed of each wheel of the own vehicle and a steering angle sensor 8 for detecting a steering angle of the steering, respectively, and outputs wheel speed data and steering angle data. And to input the result to the image processing unit 20.

  The image processing unit 20 is mainly configured by a well-known microcomputer including a CPU 18 and a semiconductor memory (hereinafter, memory 16) such as a RAM, a ROM, and a flash memory. Various functions of the image processing unit 20 are realized by the CPU 18 executing a program stored in a non-transitional substantial recording medium. In this example, the memory 16 corresponds to a non-transitional substantial recording medium storing a program.

  When this program is executed, a method corresponding to the program is executed. The non-transitional substantive recording medium means that the recording medium excludes electromagnetic waves. Further, the number of microcomputers constituting the image processing unit 20 may be one or more.

  As illustrated in FIG. 1, the image processing unit 20 includes a line detection unit 22, a position correspondence unit 24, a frame estimation unit 26, a tracking unit 28 And. The method of realizing these elements constituting the image processing unit 20 is not limited to software, and some or all of the elements may be realized using one or a plurality of hardware. For example, when the above function is implemented by an electronic circuit that is hardware, the electronic circuit may be implemented by a digital circuit including a large number of logic circuits, an analog circuit, or a combination thereof.

The function as the line detection unit 22 performs a process of detecting a line such as a white line or a yellow line by performing well-known image processing such as Hough transform on each captured image.
In the function as the position correspondence unit 24, it is determined whether or not the lines existing in the common imaging range with another camera are the same line for each detected line, and a process of associating the same line is performed. Note that correspondence information in which an imaging range common to other cameras and corresponding coordinates are set for each camera is stored in the memory 16 in advance, and whether or not the lines are the same is determined using the correspondence information.

The function as the frame estimating unit 26 performs a process of estimating one or a plurality of parking frames existing in the captured image and estimating a parkable frame from these parking frames.
The function as the tracking unit 28 performs a process of tracking the object in the captured image by associating the amount of movement of the vehicle with the amount of movement of the object in the captured image. In particular, in the present embodiment, the parking frame is tracked and the position of the parking frame is recognized. Information on the position of the parkable frame is sent to the vehicle control unit 32.

  In the function as the tracking unit 28, an image indicating the parking frame is generated while taking into account the amount of movement of the host vehicle, and this image is output to the display control unit 18. The display control unit 18 converts the image sent from the image processing unit into a video signal that can be displayed on the display unit 30 and sends the video signal to the display unit 30.

  The vehicle control unit 32 receives the position of the parking available frame, generates a trajectory of the own vehicle for parking in the parking available frame, accelerates / decelerates the own vehicle, and moves the own vehicle to move the own vehicle along the trajectory. The steering angle of the vehicle is controlled.

[1-2. processing]
Next, the parking available frame recognition process executed by the image processing unit 20 will be described with reference to the flowchart of FIG. The parking available frame recognition process is started, for example, when the vehicle is powered on, and is a process that is repeatedly performed thereafter. The parking available frame recognition process may be started for each imaging cycle of the camera.

  In the parking available frame recognition process, as shown in FIG. 2, first, in S110, an image captured by a camera is acquired. In this process, the respective captured images obtained by the front camera 4F, the right camera 4R, and the left camera 4L at substantially the same time are acquired.

  For example, a captured image as shown in FIG. 3 is obtained as a captured image captured by the front camera 4F, and a captured image as shown in FIG. 4 is obtained as a captured image captured by the left camera 4L. Can be In the following processing, two or more captured images obtained by the cameras 4F, 4B, 4R, and 4L mounted on the vehicle can be used in any combination. However, in the present embodiment, the description is simplified. Only an example in which images taken by the front camera 4F and the left camera 4L are used in combination will be described.

  Subsequently, in S120, a line such as a white line or a yellow line is detected for each acquired captured image. In this process, an edge which is a boundary between luminance and color is detected in a large number of pixels constituting the captured image, and a well-known process such as a Hough transform is performed on the edge. Line is detected. The line here has a width and includes, for example, road paint. Note that the detected line is called a detection line. In the examples shown in FIGS. 3 and 4, among the white lines 40 indicating all the parking frames, all the lines 41, 42, and 43 are the detection lines.

  Subsequently, in S130, it is determined whether or not a plurality of detection lines exist in each captured image. In this processing, since the area between the two lines is set as the parking frame, if no two or more lines are detected, it is recognized that there is no parking frame. Therefore, in S130, it can be said that it is determined whether there is a possibility that a parking frame exists.

  If a plurality of detection lines do not exist, the fact that there is no usable parking frame is recorded in the memory 16 in S240, and the parking available frame recognition process ends. If there are a plurality of detection lines, the whole of the captured image obtained in S140 or at least the detection lines of the captured image is converted into a plane coordinate system. This process may be realized by a well-known process of generating a bird's-eye image in which the captured image is viewed from the vertical direction.

  Subsequently, parking frame estimation is performed in S150. In this process, a parking frame is extracted from a plurality of captured images. As the parking frame, an area that is interposed between at least two lines and in which the interval between the two lines falls within a preset range set based on the width of the vehicle is extracted. At this time, all existing parking frames in the captured image are extracted. In the examples shown in FIGS. 3 and 4, the area between the white lines 41 and 42 and the area between the white lines 42 and 43 are extracted as the parking frames. The area between the white lines 41 and 43 can also be extracted as a parking frame, but in this processing, only the area where the distance between the white lines is within a preset range is set as the parking frame. Is excluded from the parking slots.

  When extracting a parking frame, only a specific parking frame such as a parking frame closest to the own vehicle and a parking frame located only on the right or left side of the own vehicle may be extracted. In this process, a target line representing a line defining an end of the parking frame on the farther side from the host vehicle is extracted. In the example shown in FIGS. 3 and 4, the white line 42 is the target line in the parking frame between the white lines 41 and 42, and the white line 43 is the target line in the parking frame between the white lines 42 and 43.

  Subsequently, in S160, it is determined whether or not there is a parking frame candidate. Here, the parking frame candidate includes a parking frame, but is not limited to a parking frame, and indicates a general parking frame including a parking frame in which another vehicle is already parked.

  If there is no parking frame candidate, the process proceeds to S240 described above. If there is a parking frame candidate, it is determined in S210 whether or not the shape of the target line is the same in an imaging region common to a plurality of captured images.

  In this process, first, the same parking frame observed from different positions is extracted from a plurality of captured images, and the shapes of the same target lines constituting the parking frame are compared. The same parking frame specifies a common area where the same object is captured in a plurality of captured images by conditions under which a plurality of captured images are obtained, for example, an offset of arrangement in a plurality of cameras, and a parking frame in a captured image It is specified by extracting a parking frame within the common area.

When comparing the shapes of the target lines, for example, the lengths of the target lines are compared. Here, the length of the line indicates the length of the line in the longitudinal direction. The longitudinal direction is a direction orthogonal to the width direction of the line. When comparing the length of the subject line, for example, as shown in FIGS. 3 and 4, comparing the length L ₁ and L ₂ of the same white line 42. The length of the white line is compared with the length when the coordinates are converted to the plane coordinate system.

  As shown in FIG. 5, when a vehicle is parked in the parking frame, a part of the white line 42 is hidden by the vehicle, and the hidden part of the white line 42 differs depending on the difference in the imaging position of the camera. Therefore, a difference occurs in the length of the white line 42 observed by the camera.

  On the other hand, if the vehicle is not parked in the parking frame, for example, the target line such as the white line 43 is observed in a relatively wide range without being hidden by the vehicle. In this case, the difference between the lengths of the target lines becomes smaller.

  Therefore, in the present embodiment, it is assumed that the shapes are the same if the difference between the lengths of the target lines is within a preset threshold. If the shape of the target line is the same in S210, the parking frame determined to have the same shape in S220 is recognized as the parking available frame. If the shapes of the target lines are not the same, the parking frame determined to have the same shape in S230 is recognized as not being a parkable frame.

When such processing ends, the parking available frame recognition processing ends.
[1-3. effect]
According to the embodiment described in detail above, the following effects can be obtained.

  (1a) In the above-described imaging system 1, the image processing unit 20 acquires a plurality of captured images representing the respective captured images captured by the plurality of imaging devices arranged at different positions of the host vehicle. Then, the image processing unit 20 extracts a parking frame from the plurality of captured images, and extracts a target line representing a line defining an end of the parking frame on the farther side from the own vehicle. In addition, the image processing unit 20 determines whether or not the shape of the target line is the same for the same parking frame extracted from the plurality of captured images, and determines whether the shape of the target line is the same. Recognize as a parking available frame.

  According to such an imaging system 1, since it is determined whether or not the frame is a parkable frame based on whether or not the shape of the same target line imaged at a different position is the same, the parkable frame is accurately recognized. it can.

(1b) In the above-described imaging system 1, the image processing unit 20 determines whether the length of the target line is the same as the shape of the target line.
According to such an imaging system 1, since the shape of the target line is determined by comparing the lengths of the target lines, the parking available frame can be formed with a simpler configuration than in the configuration that is determined based on the shape of the entire line. Can be recognized.

(1c) In the above-described imaging system 1, the image processing unit 20 extracts a region between at least two lines as a parking frame.
According to such an imaging system 1, since a region sandwiched between at least two lines is extracted as a parking frame, the parking frame can be extracted using a known white line recognition technique. Therefore, the parking available frame can be recognized with a simpler configuration.

  (1d) In the above-described imaging system 1, the image processing unit 20 converts each of the captured images into a plane coordinate system, and sets a target line for the same parking frame extracted from each of the captured images converted into the plane coordinate system. It is determined whether or not the shape of the target line is the same, and the parking frame whose shape of the target line is determined to be the same is recognized as the parkable frame.

  According to such an imaging system 1, it is determined whether or not the shape of the target line is the same in the plane coordinate system. Therefore, even if the target line is deformed due to a difference in the position where the captured image is obtained, it is preferable. Can be determined.

[2. Other Embodiments]
Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above embodiments, and can be implemented with various modifications.

  (2a) In the above embodiment, whether or not the shape of the target line is the same is determined based on the length of the target line. However, the present invention is not limited to this. For example, it may be determined whether or not the shape of the target line is the same as the overall shape of the target line, the shape of the end of the target line that is farther from the host vehicle, or the like.

  Specifically, when it is determined whether or not the shape of the end portion of the target line that is farther from the vehicle is the same, as shown in FIGS. The shapes, that is, the shapes of the white lines 42 in the region [A] and the region [B] are compared. When a vehicle is present in the parking frame, the far end of the white line 42 away from the vehicle is hidden by the vehicle, and depending on the position of the camera, which part of the vehicle is hidden by the white line 42 changes. The shape of the end far from the vehicle changes. On the other hand, when the white line 42 is not hidden, the shape of the end of the white line 42 far from the vehicle does not change. Therefore, whether or not the parking frame is a parkable frame can be determined based on whether or not the shape of the end portion of the target line that is farther from the host vehicle is the same.

  According to such an imaging system, since the shape of the target line is determined by comparing the shape of the end portion of the target line on the far side from the host vehicle, the configuration is simpler than the configuration in which the shape is determined based on the shape of the entire line. With such a configuration, the parking available frame can be recognized.

  (2b) In the imaging system 1, when determining whether or not the parking frame is the parking available frame, the determination may be made using only the above described parking available frame recognition process. The determination may be made by combining the process with a known process. For example, the parking available frame may be temporarily extracted by a known process, and the parking available frame recognition process may be used for final determination. Further, the parking frame may be determined to be a parking frame when it is determined that the parking frame is a parking frame by at least one of a known process and a parking frame recognition process, or a majority process.

  (2c) A plurality of functions of one component in the above embodiment may be realized by a plurality of components, or one function of one component may be realized by a plurality of components. . Also, a plurality of functions of a plurality of components may be realized by one component, or one function realized by a plurality of components may be realized by one component. Further, a part of the configuration of the above embodiment may be omitted. Further, at least a part of the configuration of the above-described embodiment may be added to or replaced with the configuration of another above-described embodiment. Note that all aspects included in the technical idea specified by the language described in the claims are embodiments of the present disclosure.

  (2d) In addition to the above-described imaging system 1, various devices such as a parking frame recognition device that are components of the imaging system 1, a program for causing a computer to function as the imaging system 1, a semiconductor memory that stores the program, and the like The present disclosure can be realized in various forms such as a non-transitional actual recording medium, a parking frame recognition method, and the like.

[3. Correspondence between Configuration of Embodiment and Configuration of the Present Disclosure]
In the above embodiment, the image processing unit 20 of the imaging system 1 corresponds to a parking frame recognition device according to the present disclosure. Further, among the processes executed by the image processing unit 20, the process of S110 corresponds to the image acquisition unit according to the present disclosure, and the processes of S120 and S150 in the embodiment correspond to the line extracting unit according to the present disclosure. Further, in the above embodiment, the process of S140 corresponds to a coordinate conversion unit according to the present disclosure, and in the above embodiment, the processes of S210 and S220 correspond to a possible frame recognition unit according to the present disclosure.

  REFERENCE SIGNS LIST 1 imaging system 4B rear camera 4F front camera 4L left camera 4R right camera 6 wheel speed sensor 8 steering angle sensor 10 control unit 12 imaging signal input unit 14 Detection signal input unit, 16 memory, 18 display control unit, 18 CPU, 20 image processing unit, 22 line detection unit, 24 position correspondence unit, 26 frame estimation unit, 28 tracking unit, 30 unit Display unit, 32 ... vehicle control unit, 40, 41, 42, 43 ... white line.

Claims (6)

A parking frame recognition device (20) configured to recognize a parking frame indicating a parking frame in which the vehicle can park, and
An image acquisition unit (S110) configured to acquire a plurality of captured images representing the respective captured images captured by the plurality of imaging devices arranged at different positions of the host vehicle;
A line extraction unit (S120, S150) configured to extract a parking frame from the plurality of captured images and to extract a target line representing a line defining an end of the parking frame farther from the host vehicle; ,
For the same parking frame extracted from the plurality of captured images, it is determined whether or not the shape of the target line is the same, and the parking frame determined to have the same shape of the target line is set as the parking available frame. A possible frame recognition unit (S210, S220) configured to recognize;
Parking frame recognition device equipped with It is a parking frame recognition device of Claim 1, Comprising:
The parking frame recognition device configured to determine whether or not the length of the target line is the same as the shape of the target line. It is a parking frame recognition apparatus of Claim 1 or Claim 2, Comprising:
The parking frame recognizing device is configured to determine whether or not the shape of the target line is the same as the shape of the target line at the end of the target line that is farther from the host vehicle. The parking frame recognition device according to any one of claims 1 to 3,
The parking frame recognition device, wherein the line extracting unit is configured to extract, as the parking frame, an area sandwiched by at least two lines. The parking frame recognition device according to any one of claims 1 to 4,
A coordinate conversion unit (S140) configured to convert each of the captured images into a plane coordinate system;
Further comprising
The possible frame recognition unit determines whether or not the shape of the target line is the same for the same parking frame extracted from each captured image converted into the plane coordinate system, and the shape of the target line is determined. A parking frame recognition device configured to recognize a parking frame determined to be the same as the parking available frame.   The parking frame recognition device according to any one of claims 1 to 5,
  The image acquisition unit acquires a plurality of captured images captured substantially simultaneously as the plurality of captured images.
  Frame recognition device configured as described above.

Images