KR101815840B1 - Around view monitoring apparatus based on video estimation process - Google Patents

Abstract

The present invention relates to an apparatus and method for monitoring an ambient view based on image estimation, in which a left and right image of a current position of a moving object is estimated using only front and rear cameras installed in a moving object such as a vehicle or an airplane, .

Description

Around view monitoring apparatus based on video estimation process

The present invention relates to an ambient view monitoring (AVM) technique, and more particularly to an image estimation based surround view monitoring apparatus.

Implementing the Around View Monitoring (AVM) function using four cameras installed on a moving object such as a vehicle or an aircraft is costly.

There may be a way to minimize the use of the camera as a way to implement the low-cost AV view (AVM) function on vehicles such as vehicles and aircraft.

Korean Patent Laid-Open No. 10-2003-0024857 (Mar. 23, 2003) discloses a method for estimating a scene out of the view of a camera installed behind a vehicle and estimating an image of a rear region of the vehicle including the estimated portion, Image processing technology.

Korean Patent Publication No. 10-2003-0024857 (March 23, 2003)

The object of the present invention is to provide an image estimation based surround view monitoring apparatus capable of estimating left and right images of a current position of a moving object by using only front and rear cameras mounted on a moving object such as a vehicle or an aircraft to obtain an eye view image do.

According to an aspect of the present invention, there is provided an apparatus for monitoring an ambient view based on an image, the apparatus comprising: an image buffer unit for temporarily storing images captured by a forward camera and a rear camera of a moving object; An image estimator for analyzing past images stored in the image buffer unit and estimating left and right images at a current position of the moving object; An image matching unit for matching the current image photographed by the forward camera and the backward camera of the moving object with the left and right images estimated by the image estimating unit to generate an overview image; An image display unit for displaying the surround view image generated by the image matching unit on the display screen of the moving object; And the like.

According to a further aspect of the present invention, the image estimating unit analyzes the current image photographed by each of the front camera and the rear camera of the moving object and the previous images stored in the image buffer unit, determines the similarity between the images, And estimates the at least one image as a left and right image at the current position of the moving object.

According to a further aspect of the present invention, the image estimating unit estimates the left and right images at the current position of the moving object from the previous forward images captured by the forward camera of the moving object and stored in the image buffer unit, .

According to a further aspect of the present invention, the image estimating unit estimates the left and right images at the current position of the moving object from the previous backward images captured by the rear camera of the moving object and stored in the image buffer unit when the moving object moves backward .

According to a further aspect of the present invention, the image matching unit removes overlapping portions of the adjacent two previous images stored in the image buffer unit from one image.

According to a further aspect of the present invention, the above-mentioned image estimation based surround-view monitoring apparatus may further include a motion estimation unit for estimating a motion of the moving image based on the difference of brightness between the current image captured by each of the forward camera and the backward camera of the moving object, An image correction unit for improving the image quality; And further comprising:

According to a further aspect of the present invention, when the image correction unit senses a headlight or an on-off signal, the brightness of the current image photographed by the front camera or the rear camera of the moving object is converted into the estimated left and right image brightness levels .

Since the present invention can obtain a surrounding view image by estimating the left and right images with respect to the current position of the moving object by using only the front and rear cameras mounted on a moving object such as a vehicle or an aircraft, it is possible to implement a low- have.

1 is a block diagram showing a configuration of an embodiment of an apparatus for monitoring an ambient view based on image estimation according to the present invention.
2 is a conceptual diagram of right and left image estimation at the time of vehicle straight backward movement.
3 is a conceptual diagram of right and left image estimation at the time of vehicle backward rotation.
4 is a view showing an area of an overview view generated by an apparatus for monitoring the surrounding view based on the image estimation according to the present invention.
FIG. 5 is a flowchart illustrating a procedure for generating an overview image of an apparatus for monitoring an ambient view based on an image estimation according to the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout.

In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

The terms used throughout the specification of the present invention have been defined in consideration of the functions of the embodiments of the present invention and can be sufficiently modified according to the intentions and customs of the user or operator. It should be based on the contents of.

1 is a block diagram showing a configuration of an embodiment of an apparatus for monitoring an ambient view based on image estimation according to the present invention. As shown in FIG. 1, the apparatus for monitoring an ambient view based on an image estimation according to the present embodiment includes an image buffer unit 110, an image estimating unit 120, an image matching unit 130, (140).

The image buffer unit 110 temporarily stores images captured by the front camera 200 and the rear camera 300 of the moving object. In this case, the image buffer unit 110 may be implemented as a plurality of buffers, and temporarily store the images captured by the front camera 200 and the rear camera 300 in the two buffers, respectively . Meanwhile, the image buffer unit 110 may use a circular buffer.

The image estimating unit 120 analyzes past images stored in the image buffer unit 110 and estimates left and right images at the current position of the moving object. At this time, the image estimating unit 120 analyzes the current image photographed by each of the front camera 200 and the rear camera 300 of the mobile body and the previous images stored in the image buffer unit 110, Judging similarity, and selecting at least one image from the previous images to estimate the left and right images at the current position of the moving object.

For example, the image estimating unit 120 estimates the left and right images at the current position of the moving object from the previous forward images captured by the forward camera 200 of the moving object and stored in the image buffer unit 110, . ≪ / RTI >

Alternatively, the image estimating unit 120 may calculate the left and right images at the current position of the moving object from the previous backward images captured by the rear camera 300 of the moving object and stored in the image buffer unit 110, . ≪ / RTI >

2 and 3, a right and left image estimation operation at the time of vehicle backward movement will be described. FIG. 2 is a conceptual diagram of right and left image estimation at the time of vehicle straight-ahead backward, and FIG. 3 is a conceptual diagram of right and left image estimation at the time of vehicle backward rotation.

2 and 3, the vehicle position at the current time t is displayed on the rightmost side. 2 and FIG. 3, the four regions from the right to the uppermost are the past forward image captured by the forward camera at the time t-3, t-2, and t-1 and stored in the image buffer unit 110, Is a front image photographed by a front camera.

2 and 3, the four regions from the right to the bottom are the past backward image photographed by the rear camera at the time t-3, t-2, and t-1 and stored in the image buffer unit 110, Is a rear image photographed by a front camera.

2 and 3, since the past backward image captured by the rear camera and stored in the image buffer unit 110 is used to estimate the left and right images at the current position of the moving object, , t-2, and t-1 are selected as the left and right images at the current position of the moving object.

The image matching unit 130 matches the current image photographed by the forward camera 200 and the backward camera 300 of the moving object with the left and right images estimated by the image estimating unit 120, .

For example, when the image matching unit 130 removes the overlapping portions of the adjacent two images stored in the image buffer unit 110 from one image and then synthesizes the adjacent previous images, The current image captured by the front camera 200 and the rear camera 300 may be combined to generate the surround view image.

The image display unit 140 displays the surrounding view image generated by the image matching unit 130 on the screen of the display 400 of the moving object. For example, the image display unit 140 may be configured to display the surround view image generated by the image matching unit 130 as a forward running image or a backward parked image.

4 is a view showing an area of an overview view generated by an apparatus for monitoring the surrounding view based on the image estimation according to the present invention. 4, the upper region and the lower region are actual images respectively captured by the front camera 200 and the rear camera 300 of the moving object at the current time, and the central region is the actual image captured from the past images stored in the image buffer unit 110 The estimated left and right images.

According to the present invention, since the left and right images of the current position of the moving object can be estimated using only the front and rear cameras mounted on the moving object such as the vehicle or the airplane to obtain the surrounding view image, Implementation is possible.

Meanwhile, when the image estimating unit 120 estimates the left and right images at the current position of the moving object from the past images stored in the image buffer unit 110, the number of left and right images selected according to the speed of the moving object may be varied . This is because the number of images stored in the image buffer unit 110 per unit time is smaller than that in the case where the moving object is at a high speed.

According to a further aspect of the present invention, the image estimation based surround view monitoring apparatus 100 may further include an image correction unit 150. The image correcting unit 150 corrects for the difference in brightness between the current image photographed by each of the front camera 200 and the rear camera 300 of the moving object and the left and right images estimated by the image estimating unit 120 Improve.

For example, when the image correcting unit 150 senses a headlight or an on-off signal, the brightness of the current image photographed by each of the front camera 200 and the rear camera 300 of the moving object is estimated To be adjusted to the image brightness level.

An image photographed by the front camera 200 at the current time included in the surrounding view image generated by the image estimation-based surrounding view monitoring apparatus 100 during the nighttime driving of the vehicle is bright by the headlight, The image photographed by the rear camera 300 at the current time is dark by the taillight.

Since the left and right images estimated from the image buffer unit 110 are estimated from past images captured by the front camera 200 and stored in the image buffer unit 110 at the time of moving forward, When the brightness of the surround view image generated by the image estimation based surround view monitoring apparatus 100 is not adjusted because it is estimated from the past image captured by the image sensing unit 300 and stored in the image buffer unit 110, Lt; / RTI >

The brightness of the current image photographed by each of the front camera 200 and the rear camera 300 of the moving object is estimated based on the estimated brightness when the headlamp or the non-turned on signal is sensed through the image corrector 150, To the left and right image brightness levels.

The process of generating the surround view image of the surrounding view monitoring apparatus based on the image estimation according to the present invention will be described with reference to FIG. FIG. 5 is a flowchart illustrating a procedure for generating an overview image of an apparatus for monitoring an ambient view based on an image estimation according to the present invention.

First, in step 510, an image estimation based surround view monitoring apparatus temporarily stores images captured by the front camera and the rear camera of the moving object in real time. In this case, it is possible to temporarily store images captured in real time by the front camera and the rear camera in each of the plurality of buffers.

Next, in step 520, the image estimation based surround view monitoring apparatus analyzes the past images temporarily stored by the step 510 to estimate the left and right images at the current position of the moving object.

At this time, by analyzing the current image photographed by each of the forward camera and the backward camera of the moving object and the previous images temporarily stored by the step 510, it is determined whether the similarity of the boundary between the images is at least one, It can be implemented to estimate the left and right images at the current position.

For example, in step 520, when the moving object is moved forward, the left and right images at the current position of the moving object may be estimated from the previous forward images captured by the forward camera of the moving object and temporarily stored by the step 510.

Alternatively, in step 520, the left and right images at the current position of the moving object may be estimated from the previous backward images captured by the rear camera of the moving object and temporarily stored in step 510, when the moving object is backward.

Next, in step 530, the image estimation based surround view monitoring apparatus matches the current image captured by the front camera and the rear camera of the moving object with the left and right images estimated by the above step 520, thereby generating the surround view image.

For example, in step 530, the overlapped portions of the adjacent two previous images temporarily stored by the 510 are removed from any one of the images, and the adjacent previous images are synthesized, and the front and rear cameras of the moving object And generates the surround view image.

Next, in step 540, the brightness estimation difference between the current image photographed by each of the forward camera and the backward camera of the moving object included in the generated surround view image based on the image estimation based surround view monitoring apparatus, Thereby improving the sense of heterogeneity caused by the above.

For example, when the headlight or the non-turn-on signal is detected in step 540, the brightness of the current image captured by the front camera or the rear camera of the moving object may be adjusted to the estimated left and right image brightness levels .

Then, in step 550, the image estimation based surround view monitoring apparatus displays the surround view image with improved sense of heterogeneity due to the brightness difference on the display screen of the moving object.

According to the present invention, since the left and right images of the current position of the moving object can be estimated using only the front and rear cameras mounted on the moving object such as the vehicle or the airplane to obtain the surrounding view image, So that the object of the present invention can be achieved.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. .

INDUSTRIAL APPLICABILITY The present invention is industrially applicable in the field of AVM (Around View Monitoring) technology and its application technology.

100: Around view monitoring device
110:
120:
130:
140:
150:
200: Front camera
300: rear camera
400: Display

Claims (7)

An image buffer unit for temporarily storing images photographed in real time by the front camera and the rear camera of the moving body;
An image estimator for analyzing past images stored in the image buffer unit and estimating right and left images at a current position of the moving object by selecting at least one image from previous images, wherein the number of left and right images selected according to the speed of the moving object is variable;
An image matching unit for matching the current image photographed by the forward camera and the backward camera of the moving object with the left and right images estimated by the image estimating unit to generate an overview image;
An image display unit for displaying the surround view image generated by the image matching unit on the display screen of the moving object;
Based on the estimated image information. The method according to claim 1,
The image estimating unit:
The current image captured by the forward camera and the backward camera of the moving object and the previous images stored in the image buffer unit are analyzed to determine the similarity of the boundary between the images to select at least one image from the previous images, And estimates the image based on the left and right images. 3. The method of claim 2,
The image estimating unit:
And estimates the left and right images based on the current position of the moving object from the previous forward images captured by the forward camera of the moving object and stored in the image buffer unit when the moving object is advanced. 3. The method of claim 2,
The image estimating unit:
And estimates the left and right images at the current position of the moving object from the previous backward images captured by the rear camera of the moving object and stored in the image buffer unit when the moving object moves backward. 3. The method of claim 2,
Wherein the image matching unit comprises:
Wherein the overlapping part of the boundary between the two adjacent images stored in the image buffer unit is removed from any one of the images. 6. The method according to any one of claims 1 to 5,
Wherein the image estimation based surround view monitoring apparatus comprises:
An image correcting unit for improving a sense of heterogeneity due to a difference in brightness between a current image photographed by each of a front camera and a rear camera of the moving object and a right and left image estimated by the image estimator;
Based on the estimated information. The method according to claim 6,
The image correcting unit:
And adjusting the brightness of the current image photographed by each of the front camera or the rear camera of the moving object to the estimated left and right image brightness levels when the headlight or the on-off signal is sensed. Device.

Images