KR101371925B1 - Restoration methods of image focusing - Google Patents

Abstract

The present invention relates to a method for restoring the focus on the entire image captured by the imaging device.

According to one or more exemplary embodiments, a method for restoring an image focus may include: dividing an input image into a predetermined area; Calculating a degree of focus by analyzing an outline for each of the divided regions; Differentially blurring the respective areas based on the calculated degree of focus; And deblurring the entire image including each of the differentially blurred regions.

In accordance with another aspect of the present invention, there is provided a method for reconstructing an image focus of a person center, comprising: detecting a face of a person in an input image and measuring a size of the detected face; Calculating a distance to the person based on the measured size of the face; Calculating a point spread function (PSF) based on the calculated distance from the person; And deconvolving the image.

The present invention relates to a method of restoring an image focus to enable a general lens to be restored to an image with a uniform focus without using a special blurring lens. The present invention also relates to a method for restoring the focus from the person-centered image to the person center.

Imager, Focus, Outline, Blur, Deblur, Restore

Description

How to restore image focus {RESTORATION METHODS OF IMAGE FOCUSING}

The present invention relates to a method for restoring the focus on the entire image captured by the imaging device.

In general, focus adjustment of a lens may be divided into a method of measuring infrared rays by using a distance from a subject and an image processing of input image data, and a method of adjusting a focal length of a lens based on the result.

Recently, as the use of Charge Coupled Device (CCD) or Complementary Metal Oxide Semiconductor (CMOS) devices has increased, the price and performance of image chips have decreased. It is used.

For example, as an auto focus (AF) method of adjusting the focus of a digital camera, an image element is photoelectrically converted by an imaging element such as a CCD to generate an image signal, and a high frequency component is obtained from the image signal of a predetermined AF area in the captured image. It is known to calculate an AF evaluation value which is a contrast value of an image by extracting the value, and to detect the combined position of a photographing lens based on the AF calculated value.

In this method, the AF evaluation value is calculated at each position of the focus lens while moving the focus lens in the optical axis direction, and the position at which the AF evaluation value is maximized is detected as the focal position.

Recently, in order to acquire a digital image, various mobile devices such as an imaging device such as a mobile phone, a PDA, an MP3, a PMP, a UMPC, and the like are equipped with a camera equipped with the above-mentioned focusing lens, and have gained great response from users. Such mobile devices have many advantages in terms of mobility and convergence, but due to limitations in unit cost and technical problems of the products, some degree of image quality difference occurs.

In order to solve this problem, conventionally, various post-processing techniques for improving image quality at low illumination, removing noise, and correcting the focus of an image have been studied.

1A is a schematic exploded view of a general camera lens. As shown in Fig. 1A, a general camera lens is configured to make an accurate image on a sensor using several lenses. However, such a configuration has a problem with the above problem, and a new lens structure having a special structure has been proposed.

FIG. 1B is a schematic diagram showing a structural change to a conventional lens. When the lens combination shown in FIG. 1B is used, the sensor does not form an accurate image and intentionally blurs the image, thereby performing signal processing later. Restore the focus by restoring the focus.

Since the lens combination shown in FIG. 1B is configured to widen the depth of field, the lens combination can focus on a wider area than the focus area that can be expressed by a general lens.

Figure 1c shows a schematic flowchart of a conventional image focus reconstruction method. When an input image passes through a lens that has been structurally modified (hereinafter, referred to as a blurring lens), the input image has an even blurring degree overall, and the image is formed on the sensor as it is.

The entire uniform output image can be obtained by signal processing (ISP) and deblurring the image having the above uniform blurring degree.

According to the method shown in FIG. 1C, the blurring lens can be uniformly focused on the entire image to obtain a uniform image.

However, the method of using a blurring lens for expanding the focal region requires full real computation because of the complete algorithmic characteristics and computational speed of the algorithm used to improve the blurred image through post-processing.

However, performing a full real number calculation has a limitation and a limitation of a calculation speed, and thus deterioration of image quality occurs compared to an image photographed using another lens. In addition, there is a problem in that it is not possible to apply a shooting technique such as intentional outfocusing.

In addition, there is a burden of designing / attaching a lens of a new structure called a blurring lens, which makes it difficult to produce a small quantity and causes a price increase of the camera module. In addition, it is difficult to freely change the focus, it can not produce a variety of effects of the user.

Also, generally, as a method for restoring focus on an image, the photographed image is formed to have a fixed point spread function (PSF), and the focus is restored by performing an inverse process of the point spread function.

However, in the image photographed by the lens, since the point diffusion function is different from the distance between the object and the lens, the degree of restoration of the focus varies for each object according to the distance, and thus deterioration of image quality occurs in a specific region.

In particular, in a person-centered image, when the image of the person to be captured by the photographer is deteriorated, serious problems may occur.

Accordingly, the present invention has been made to solve the above problems, and an object of the present invention is to compensate for the deterioration of image quality, the limitation of lens selection, and the like, which are problems caused by the use of a blurring lens for expanding a focal region of an image.

In addition, an object of the present invention is to propose a method for restoring the focus to the center of the person in the person image.

Instead of using a conventionally modified lens, a general lens is used, and the focus of the captured image is analyzed to restore the focus of the input image through differential blurring for each region.

In addition, a focal reconstruction image centered on the person is realized by selecting / applying a focus restoration function that is pre-calculated and stored according to the distance from the person.

According to the present invention, local or selective use is possible through selective focus restoration for each region, and the user can achieve various effects.

In addition, it is possible to produce a uniform effect while using a conventional general lens, thereby contributing to cost reduction of the camera module.

In addition, when photographing a person-centered image, it is possible to improve the quality of the image by focusing and restoring to the center of the face of the person.

In addition, the present invention is a core technology for improving the image quality of the camera used in the next-generation mobile devices, and overcomes the shortcomings and shortcomings in the short-caliber optical device, such as conventional camera phones and mobile devices, which cannot produce a clear image as a whole. Can be.

The present invention relates to a method for restoring the focus on the entire image captured by the imaging device.

According to one or more exemplary embodiments, a method for restoring an image focus may include: dividing an input image into a predetermined area; Calculating a degree of focus by analyzing an outline for each of the divided regions; Differentially blurring the respective areas based on the calculated degree of focus; And deblurring the entire image including each of the differentially blurred regions.

Preferably, the step of differentially blurring the respective areas based on the calculated degree of focusing, blurring of the remaining areas of the lowest focusing area based on the lowest focusing area among the areas. And differentially blurring to have a blurring, and blur the respective regions to make the image have a uniform degree of blurring as a whole.

Preferably, the step of dividing the input image into a predetermined region comprises: dividing the input image into a matrix form, or dividing the input image into a pixel unit, or by an object captured by the image, The calculating of the focusing degree by analyzing the outline for each region may further include classifying the image into a predetermined group according to the calculated focusing degree.

In accordance with another aspect of the present invention, there is provided a method for reconstructing an image focus of a person center, comprising: detecting a face of a person in an input image and measuring a size of the detected face; Calculating a distance to the person based on the measured size of the face; Calculating a point spread function (PSF) based on the calculated distance from the person; And deconvolving the image.

Preferably, the distance to the person is pre-calculated and stored according to the number of lenses and pixels of the optical system, and calculating the distance to the person based on the measured size of the face may include determining the size of the measured face. If greater than the threshold of the distance to the person is calculated.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 and 3 are flowcharts illustrating a method of restoring an image focus according to an exemplary embodiment of the present invention.

In the image focus reconstruction method according to an embodiment of the present invention, the focus of the image is reconstructed by using an ordinary general lens, and by analyzing the boundary of the image, the degree of focusing for each region is calculated. By differentially blurring, after forming an image having a uniform degree of blurring, it is characterized in that the overall blurring.

As shown in Fig. 2, the image inputted to the imaging device forms an image on the sensor through a general lens as shown in Fig. 1A. Then, an output image is obtained through predetermined signal processing.

However, since the output image is not in the entire focus restoration, the focus restoration method according to the present invention is applied.

Using the outline analysis technique, the degree of focus of each image area is quantified. That is, the image focus reconstruction method according to an embodiment of the present invention divides an input image into a predetermined area and calculates a degree of focus for each area.

The division method may be divided into a matrix, divided into pixels, or divided into objects captured in the image.

In general, the deterioration of the focus in the input image is the area where the different gray levels are adjacent to each other, that is, the outline portion of the image, in which the focus is particularly blurred, which degrades the overall image quality. .

Therefore, it is common to determine whether the focus is well focused using an outline extraction algorithm, and the outline extraction algorithm is a well-known technical area described in many patents and publications, and thus, detailed description thereof will be omitted.

When the degree of focus is calculated by analyzing the outline of each divided area, the respective areas are differentially blurred based on the calculated degree of focus. Preferably, the remaining area is differentially blurred to have the blurring degree of the lowest focusing area based on the lowest focusing area among the divided areas.

Thus, the image is formed to have a uniform degree of blurring as a whole. If the image having the blurring of the overall uniform degree is de-blurred, a focus reconstruction image having an improved image quality can be obtained.

The deblurring refers to a process of eliminating blurring present in an image, and can be generally implemented using a deblurring filter.

The deblurring filter may be formed by combining a filter used for blurring and a filter for correcting other focus (eg, a sharpen filter). The image that is blurred due to poor focus is generally corrected so that the image is properly focused through the deblurring process.

The deblurring principle and method are well known to those skilled in the art to which the present invention pertains and correspond to known techniques, and thus detailed descriptions thereof will be omitted.

3 is a flowchart illustrating a method of reconstructing an image focus according to an exemplary embodiment of the present invention in more detail.

When an external image is input to the imaging apparatus by the imaging apparatus (S300), the input image is divided into a predetermined region (S305). In this case, as described above, the input image may be divided into a matrix having a predetermined size, that is, an N × M matrix, or divided into pixels, or divided by an object captured in the image. can do.

Next, an edge of each segmented image is detected / analyzed to calculate a degree of focus for each region (S310). In each of the divided areas, there may be an area in which no outline exists, and in this case, the step of calculating the focusing degree may be omitted.

The fact that an edge is not detected in a predetermined region means that the region is one continuous image, and therefore, the focus deterioration phenomenon generally does not occur or occurs weakly in the region. Therefore, even if the step of calculating the focus level is omitted, such an area may not significantly affect the overall image focus restoration.

Next, each area is differentially blurred based on the calculated degree of focus. Different ways of blurring the divided regions may be various. In FIG. 3, a method of grouping divided regions according to the calculated degree of focus (S315) is illustrated.

That is, according to the degree of focus, it can be classified into a well-focused region, a poorly-focused region, and an intermediate region thereof. The classification scheme is exemplary, and it is apparent that the number and extent of groupings may vary depending on the embodiment.

When grouping is performed, strong blurring is performed in the well-focused area (S320), medium blurring is performed in the middle area (S325), and weak blurring is performed in the poorly-focused area (S330).

After performing the above steps, an image having an overall uniform blurring degree is generated (S335). Next, as described above, when the image is deblured using the deblurring technique (S340), an image in which the focus is restored may be obtained (S345).

4 is a graph schematically illustrating a change in blurring degree for each divided image when the image focus reconstruction method according to an embodiment of the present invention is applied.

As shown in the graph, the image photographed by the image capturing apparatus has a different degree of blurring for each region. In general, the blurring degree of a region including a lot of outlines is relatively larger than that of other regions, and the blurring degree is weaker in a continuous image without an outline region.

According to an embodiment of the present invention, when differential blurring is applied to each region, as shown in (b) grat, the entire image has a uniform blurring degree. Therefore, by debluring this image to remove blurring, it is possible to obtain an image with a very low blurring degree, preferably almost no blurring, as shown in the graph (c).

It is apparent to those skilled in the art that the graphs of (a) to (c) are merely exemplary graphs for easily explaining the present invention, and are not intended to limit the present invention, and other types of graphs may be used.

5 is a flowchart illustrating a method of restoring an image focus of a person center according to another exemplary embodiment of the present invention.

The face of the person is detected from the image input to the imaging apparatus (S500). Algorithms for detecting parts of the face, eyes, etc. in specific images are very advanced. When the face of the person is detected using this algorithm, the distance of the person is measured by measuring the size of the detected face (S505). The distance to the person may be calculated in advance according to the lens angle of view and the number of pixels of the optical system and stored in the system. Alternatively, the distance may be calculated by quantifying the recorded size according to the face size of the captured image or by comparing the recorded face size with the recorded face size.

At this time, the distance to the person can be calculated only when the measured face size is larger than the predetermined threshold. If the face size is smaller than the predetermined threshold, it is difficult to see the image as a person image or the photographer is inherently inclined. This is because they are not likely to be intended.

When the distance to the person is calculated, a point spread function (PSF) according to the distance is calculated (S510). The point diffusion function changes according to the distance between the object and the lens. Therefore, in designing an image pickup apparatus, a focus restoration coefficient according to a distance may be measured in advance and stored in a system, and when the distance is calculated, a previously calculated focus restoration coefficient may be calculated and used to restore the image focus.

When the point spread function is calculated, the input image is deconvolutiond (S515). That is, inverse filtering of the point spread function on the image may remove the influence of the point spread function. Such operation may be performed in the frequency domain and the spatial domain.

Through the above process, the focus of an image whose focus is deteriorated may be restored.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. There will be.

The scope of the present invention is defined by the appended claims, and all differences within the scope of the claims are to be construed as being included in the present invention.

1A is an exploded view schematically showing the configuration of a general camera lens;

1B and 1C are schematic exploded views and flowcharts of a lens used in a conventional method for restoring image focus.

2 and 3 are flowcharts illustrating a method of restoring an image focus according to an exemplary embodiment of the present invention.

4 is a graph schematically illustrating a change in blurring degree for each divided image when the image focus reconstruction method according to an embodiment of the present invention is applied.

5 is a flowchart illustrating a method for restoring an image focus of a person center according to another embodiment of the present invention.

Claims (8)

In the method of restoring the focus of the image, Dividing an input image into a predetermined area; Calculating a degree of focus by analyzing an outline for each of the divided regions; Differentially blurring the respective areas based on the calculated degree of focus; And And deblurring the entire image including each of the differentially blurred regions. The method of claim 1, Differential blurring of the respective areas based on the calculated focus degree may include: And differentially blurring the remaining area to have the blurring degree of the lowest focusing area based on the lowest focusing area of the area. 3. The method of claim 2, And blurring the respective areas differentially so that the image has an overall uniform blur. The method of claim 1, The dividing of the input image into a predetermined area may include: And dividing the input image into a matrix, dividing pixel by pixel, or dividing the input image into objects captured by the image. The method of claim 1, The step of calculating the degree of focus by analyzing the outline for each divided region, And classifying the image into a predetermined group according to the calculated degree of focus. In fixed focus optical systems, Detecting a face of a person from an input image and measuring a size of the detected face; Calculating a distance to the person based on the lens angle of view and number of pixels of the optical system and the measured size of the face; Calculating a point spread function (PSF) based on a lens angle of view of the optical system and the number of pixels and a distance between the person calculated based on the measured size of the face; And And deconvolution the image. The method according to claim 6, And a distance from the person is calculated and stored according to the lens angle of view and the number of pixels of the optical system. The method according to claim 6, Computing the distance to the person based on the measured size of the face, And calculating a distance to the person when the measured face size is larger than a predetermined threshold.

Images