KR101942198B1 - Terminal devise, Method and Recording medium for enhancing image based on Retinex model - Google Patents

Abstract

Disclosed are a terminal device and a method for improving an image based on a retinex model, and a recording medium for performing the method. The terminal device comprises: a memory unit which stores a computer readable command; and a processor unit which is implemented to execute the command. The processor unit calculates a lighting element and a reflectance element which minimize a value of an energy function in an inputted target image, and calculates a restoration image by removing the calculated lighting element from the target image, and the energy function is formed of linear combination a first data fidelity term between a modeling image, which is formed by combination of the lighting element and the reflectance element, and the target image, a smoothing constraint term of the lighting element, a smoothing constraint term of the reflectance element, and a second fidelity term between the lighting element and a bright channel, wherein the smoothing constraint term of the reflectance element corresponds to l1-norm of the gradient of the lighting element.

Description

Technical Field [0001] The present invention relates to a terminal device and method for improving an image based on a Retinex model, and a recording medium for performing the method.

Embodiments of the present invention are directed to a terminal apparatus and method capable of preserving an edge while simultaneously performing brightness enhancement and noise cancellation on a low-illuminance image based on a Retinex model, and a recording medium performing the same .

The Human Vision System (HVS) recognizes the color of an object, and considers the illumination of the surrounding environment together with the color of the object, so it can recognize the color of the object without being greatly affected by the lighting environment. On the other hand, the machine vision system (MVS) has a large difference in the signal values acquired according to the illumination environment. As a result, the color of the object recognized by the machine's visual system and the color perceived by the human perception system (HVS) are different.

Particularly, in a low-illuminance environment, it is difficult to obtain a high-quality input image because a sensor included in a machine vision system (MVS) can not sufficiently respond to very small incident light. In addition, interference of light between the reduced pixels results in chrominance noise, which results in difficulty in post-processing steps such as object recognition, detection and tracking.

To solve this problem, various image brightness enhancement methods have been proposed

Among the conventional methods, the histogram-based method improves the contrast of the input image by redistributing the intensity bins or modifying the cumulative distribution function (CDF) at a low computational cost. However, since the low-illuminance image provides a narrow histogram distribution, there is a disadvantage that the CDF is rapidly changed, resulting in brightness saturation and color distortion.

In addition, the haze removal method of the conventional method improves the input image based on statistical prior knowledge of the haze-free image called DCP (dark channel prior). However, this method can preserve a much lighter area than the histogram-based method, but has the disadvantage that the noise is amplified and the color is distorted in the resulting image.

Among the conventional methods, Retinex-based image enhancement methods are based on a human perception system (HVS). By using low-pass filtering and logarithmic transformation, Improve the image. However, this method has a disadvantage that a halo effect appears at the edge portion in the resultant image.

In order to solve the problems of the related art as described above, in the present invention, an image is generated based on a Retinex model, for example, by performing brightness enhancement and noise cancellation on a low- Apparatus and method, and a recording medium for performing the same.

Other objects of the invention will be apparent to those skilled in the art from the following examples.

According to an aspect of the present invention, there is provided a terminal device for improving an image based on a Retinex model, the terminal device comprising: a memory unit for storing a command readable by a computer; And a processor unit configured to execute the command, wherein the processor unit calculates an illumination component and a reflectance component that minimize a value of an energy function in an input target image, Wherein the energy function includes a first data fidelity term between a modeling image composed of a combination of the illumination component and the reflectance component and the target image, a smoothing constraint term of the illumination component, And a second data fidelity term between the illumination component and the bright channel, wherein the smoothing term of the reflectance component corresponds to a l 1-norm of the gradient of the illumination component. A terminal device is provided.

The bright channel may be calculated in pixel units of the target image.

The energy function can be expressed by the following equation.

는 상기 에너지 함수,

는 상기 제1 피델리티 항,

는 상기 모델링 영상,

는 상기 조명 성분, 상기

는 반사율 성분,

는 상기 대상 영상,

는 상기 조명 성분의 평활화 제약 항,

는 상기 조명 성분의 그래디언트,

는 상기 반사율 성분의 평활화 제약 항,

는 상기 반사율 성분의 그래디언트,

는 상기 제2 피델리티 항,

는 상기 브라이트 채널,

,

,

각각은 정규화 매개 변수를 각각 의미함. here,

Is the energy function,

The first fidelity term,

The modeling image,

The illumination component,

Reflectance component,

The target image,

Is a smoothing constraint term of the illumination component,

The gradient of the illumination component,

Is a smoothing constraint term of the reflectance component,

Is a gradient of the reflectance component,

The second fidelity term,

The bright channel,

,

,

Each represents a normalization parameter, respectively.

The brightness channel can be expressed by the following equation.

는 픽셀

에 대한 상기 브라이트 채널,

는 상기 픽셀

에 중심이 위치하는 대상 영상 내의 패치,

는 상기 패치 내의 i번째 픽셀,

는 RGB 채널의 상기 대상 채널(

),

는 범위 필터(range filter),

는 공간 필터(spatial filter)를 각각 의미함.here,

Gt;

The bright channel,

Lt; RTI ID =

A patch in the target image,

Is an i-th pixel in the patch,

Lt; RTI ID = 0.0 > (RGB) <

),

Is a range filter,

Quot; means a spatial filter.

According to another embodiment of the present invention, there is provided a Retinex model-based image enhancement method performed in an apparatus including a processor, the method comprising the steps of: determining a minimum value of an energy function for extracting an illumination component and a reflectance component from an input image; Calculating the illumination component and the reflectance component so that the illumination component and the reflectance component become equal to each other; And a step of calculating a reconstructed image by removing the calculated illumination component from the object image, wherein the energy function is a function of extracting a first modeling image, which is a combination of the illumination component and the reflectance component, A smoothing constraint term of the reflectance component, and a second data fidelity term between the illumination component and the bright channel, wherein the smoothing constraint term of the reflectance component is a sum of the smoothed constraint term of the illumination component, And the l 1-norm of the gradient of the image.

According to the present invention, there is an advantage that edges can be preserved while simultaneously performing brightness enhancement and noise reduction on a low-illuminance image based on a Retinex model.

It should be understood that the effects of the present invention are not limited to the above effects and include all effects that can be deduced from the detailed description of the present invention or the configuration of the invention described in the claims.

1 is a diagram showing a schematic configuration of a terminal device according to an embodiment of the present invention.
FIG. 2 is a flowchart illustrating a method of improving an image based on a Retinex model according to an exemplary embodiment of the present invention. Referring to FIG.
FIGS. 3 and 4 are views for explaining effects according to the present invention.

As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. In this specification, the terms "comprising ", or" comprising "and the like should not be construed as necessarily including the various elements or steps described in the specification, Or may be further comprised of additional components or steps. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

1 is a diagram showing a schematic configuration of a terminal device 100 according to an embodiment of the present invention.

Referring to FIG. 1, a terminal device 100 according to an embodiment of the present invention includes an input unit 110, a memory unit 120, a processor unit 130, and a display unit 140.

The input unit 110 is a component for receiving images obtained from a user in a low-illuminance environment. For example, the input unit 110 may receive a low-illuminance image obtained from a camera provided in the terminal device 100, or may receive a low-illuminance image transmitted through a wired communication or a wireless communication.

The memory unit 120 may be a volatile and / or non-volatile memory, and may store instructions or data related to at least one other component of the terminal device 100. In particular, the memory unit 120 may store a command or data related to a computer program or a recording medium that outputs a reconstructed image by improving a low-illuminance image.

The processor unit 130 may include one or more of a central processing unit, an application processor, or a communications processor. For example, the processor unit 130 may perform operations and data processing relating to control and / or communication of at least one other component of the terminal device 100. [ In particular, the processor unit 130 may execute an instruction related to the execution of the computer program.

The display unit 140 may include a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, and the like. In particular, the display unit 140 may output an execution screen of a computer program executed by the processor 130. [

Hereinafter, operations performed in the terminal device 100, and in particular, the processor 130 will be described in detail with reference to FIGS. 2 to 4. FIG.

FIG. 2 is a flowchart illustrating a method of improving an image based on a Retinex model, which is an operation of the terminal device 100 according to an embodiment of the present invention.

Hereinafter, a process performed in each step will be described in detail.

First, in step 202, the input unit 110 receives a target image. In this case, the target image may be an image of an RGB channel obtained in a low-illuminance environment, that is, a low-illuminance image.

Thereafter, in step 204, the processor unit 130 calculates a modeling image based on the input target image.

In this case, the modeling image is an image corresponding to a combination of an illumination component and a reflectance component included in a target image. The illumination image means a signal component corresponding to the surrounding illumination environment, Means a signal component according to the original color of the object.

According to an embodiment of the present invention, a modeling image can be expressed by Equation 1 below.

는 대상 영상,

는 모델링 영상,

는 조명 성분,

는 반사율 성분을 각각 의미한다. 이 때, 조명 성분은 대상 영상에서 저주파수 성분으로 간주되고, 반사율 성분은 대상 영상에서 고주파수 성분으로 간주된다. here,

However,

However,

Lt; / RTI >

Reflectance component " At this time, the illumination component is regarded as a low-frequency component in the target image, and the reflectance component is regarded as a high-frequency component in the target image.

Subsequently, in step 206, the processor unit 130 calculates the illumination component and the reflectance component in the target image. At this time, the processor unit 130 calculates the illumination component and the reflectance component in the target image using the energy function. This will be described in more detail below.

Finally, in step 208, the processor unit 130 removes the illumination component calculated in the target image to calculate a reconstructed image. That is, the processor unit 130 may calculate the reconstructed image using the extracted reflectance component.

Hereinafter, the process performed in step 206 will be described in more detail.

The energy function is a function used to calculate the illumination component and the reflectance component.

More specifically, the conventional Retinax model uses a Gaussian low-pass filter to calculate an illumination component, but in this case, a halo effect occurs near the edge. On the other hand, the present invention does not use a Gaussian low-pass filter, and can simultaneously extract an illumination component and a reflectance component based on an energy function based on norm minimization.

According to the present invention, the energy function includes a first data fidelity term between the modeling image and the target image, a smoothness constraint term of the illumination element, a smoothing constraint term of the reflectance factor, And a second data fidelity term between the bright channels, and the processor unit 130 calculates an illumination component and a reflectance component that minimize the value of the energy function.

Here, the first data fidelity term means a term that reduces the difference between the restored image and the target image. That is, when the reconstructed image is made dark again, it can be said that the reconstructed image has been properly calculated in the same image as the target image (that is, the input low-illuminance image). Also, the second data fidelity term means a term that reduces the difference of the illumination components estimated during the improvement process of the bright channel and the low-illuminance image. That is, it functions to suppress the halo phenomenon occurring in the reflectance component.

The smoothing constraint term of the reflectance component means a term that reduces the high frequency component of the noise included in the target image. Also, the bright channel corresponds to the maximum brightness value among the pixel brightness values of the RGB channel of the target image.

According to one embodiment of the present invention, the energy function can be expressed as Equation (2) below.

는 에너지 함수,

는 제1 피델리티 항,

는 모델링 영상,

는 조명 성분,

는 반사율 성분,

는 대상 영상,

는 조명 성분의 평활화 제약 항,

는 조명 성분의 그래디언트,

는 반사율 성분의 평활화 제약 항,

는 반사율 성분의 그래디언트,

는 제2 피델리티 항,

는 브라이트 채널,

,

,

각각은 정규화 매개 변수를 각각 의미한다. here,

Is an energy function,

The first Fidelity term,

However,

Lt; / RTI >

Reflectance component,

However,

A smoothing constraint term of the illumination component,

The gradient of the illumination component,

Is a smoothing constraint term of the reflectance component,

The gradient of the reflectance component,

The second Fidelity term,

A bright channel,

,

,

Each representing a normalization parameter, respectively.

In this case, as described below, the illumination component and the reflectance component using the energy function are repeatedly updated and calculated, and the gradient corresponds to the difference value for the iterative calculation.

Referring to Equation (2), the processor unit 130 calculates an illumination component and a reflectance component that minimize a value of an energy function, that is, a sum of the four terms becomes a minimum value, Can be calculated.

At this time, the first data fidelity, wherein the smoothing constraint, wherein the lighting element, wherein the second data fidelity is l 2-norm is applied, and is a smoothing constraint, wherein the reflectance is subject to the l 1-norm. In particular, the present invention uses the l 1-norm of the gradient of the illumination component as the smoothing term of the reflectance component. Accordingly, the present invention can simultaneously perform brightness enhancement and noise cancellation, and it is possible to save the edge when noise is removed, and to produce a restored image with improved contrast.

Meanwhile, according to an embodiment of the present invention, the processor unit 130 may calculate a bright channel in units of pixels. That is, the present invention can calculate the bright channel in pixel units instead of patch units. This will be described in more detail as follows.

Conventional Retinex models use a Gaussian low-pass filter to estimate the illumination component to extract a spatially smoothness component. However, since the estimated illumination component is continuous near the edge, it does not coincide with the human visual system (HVS). In this case, a halo effect occurs due to a wrongly estimated illumination component. Therefore, the present invention uses a bright channel to suppress the halo effect, and in particular, it can calculate a bright channel on a pixel-by-pixel basis. As a result, edge information can be preserved while noise is minimized.

According to an embodiment of the present invention, the processor unit 130 may calculate a bright channel using a bilateral filter as shown in Equation (3) below.

는 픽셀

에 대한 브라이트 채널,

는 픽셀

에 중심이 위치하는 대상 영상 내의 패치,

는 상기 패치 내의 i번째 픽셀,

는 RGB 채널의 대상 채널(

),

는 범위 필터(range filter),

는 공간 필터(spatial filter)를 각각 의미한다. here,

Gt;

For Bright Channel,

Gt;

A patch in the target image,

Is an i-th pixel in the patch,

Is the target channel of the RGB channel (

),

Is a range filter,

Respectively denote a spatial filter.

Referring to Equation (3), the bi-letter filter can effectively remove noise while preserving the edge, and the bright channel makes it possible to estimate the optimal illumination component with the restriction of the normalized minimization.

FIG. 3 is a view comparing a reconstructed image according to the present invention and a conventional reconstructed image.

3B shows a bright channel calculated by a conventional patch method, FIG. 3C shows a bright channel of a pixel unit of the present invention, FIG. 3D shows a conventional restored image using FIG. 3B, FIG. And FIG. 3C, respectively.

Referring to FIG. 3, the backlight effect is generated near the edge area of the patch-type bright channel, but in the case of the present invention, the backlight effect can be reduced.

An example of a process of calculating the restored image by calculating the illumination component and the reflectance component in steps 206 and 208 will be described.

First, when the illumination component is set to a constant value (for example, an initial value is set using a Gaussian low-pass filter), the energy function for the reflectance component is expressed by the following equation (4).

At this time, as described above, the present invention estimates a reflection component including a high-frequency component by using l 1-norm minimization for the reflectance component. At this time, the energy function of Equation (4) is applied to the Euler-Lagrange equation, which is expressed as Equation (5) below.

At this time, the solution for the reflectance component can be derived by using a gradient descent technique that updates the solution repeatedly as shown in Equation (6) below.

는 수렴 속도를 제어하는 매개 변수를 의미하며, 각각의 반복에서

는 영역 [0,1]의 범위에 존재한다. here,

Means the parameter controlling the convergence rate, and in each iteration

Is in the range of the area [0, 1].

Next, when the previously estimated reflectance component is used, the energy function of the illumination component can be expressed by Equation (7) below.

In this case, the optimal condition of the energy function of Equation (7) can be calculated using Equation (8) related to the linear equation.

In this case, a fast Fourier transform can be used, which is expressed by Equation (9) below.

는 고속 푸리에 변환 연산자,

는 역 고속 푸리에 변환 연산자를 의미한다. 이 때, 반사율 성분 역시 영역 [0,1]에 존재하여야 하고, 조명 성분은 매 반복마다 대상 영상보다 더 크거나 같은 값이 되어야 한다.

Is a fast Fourier transform operator,

Denotes an inverse fast Fourier transform operator. At this time, the reflectance component should also be in the area [0, 1], and the illumination component should be greater or equal to the target image at every repetition.

At this time, brightness saturation may occur in the estimated reflectance component. Thus, the present invention improves the contrast of the estimated illumination components using a sigmoid function and locally adaptive histogram equalization.

Finally, the reconstructed image is calculated by multiplying the enhanced illumination and the estimated reflectance, which can be expressed as Equation (10).

는 복원 영상,

은 상기한 수학식 9에 의해 향상된 조명 성분을 나타낸다. here,

The restored image,

≪ / RTI > represents the illumination enhancement by equation (9) above.

FIG. 4 shows a step-by-step image for reconstructed image calculation according to the present invention. 4A, 4B, 4C, 4E, 4E, 4E, 4E, 4E, 4E, 4E, 4E, and 4E, respectively. .

In addition, embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Examples of program instructions, such as magneto-optical and ROM, RAM, flash memory and the like, can be executed by a computer using an interpreter or the like, as well as machine code, Includes a high-level language code. The hardware devices described above may be configured to operate as one or more software modules to perform operations of one embodiment of the present invention, and vice versa.

As described above, the present invention has been described with reference to particular embodiments, such as specific elements, and limited embodiments and drawings. However, it should be understood that the present invention is not limited to the above- Various modifications and variations may be made thereto by those skilled in the art to which the present invention pertains. Accordingly, the spirit of the present invention should not be construed as being limited to the embodiments described, and all of the equivalents or equivalents of the claims, as well as the following claims, belong to the scope of the present invention .

Claims (6)

A terminal apparatus for performing an operation of improving an image based on a Retinex model,
A memory unit for storing instructions readable by a computer; And
And a processor unit configured to execute the instruction,
Wherein the processor unit calculates an illumination component and a reflectance component that minimize a value of an energy function in an input target image and removes the calculated illumination component from the target image to calculate a reconstructed image,
Wherein the energy function includes a first data fidelity term between a modeling image composed of a combination of the illumination component and the reflectance component, a smoothing constraint term of the illumination component, a smoothing constraint term of the reflectivity component, And a second data fidelity term between the bright channels, wherein the l 1-norm of the gradient of the illumination component is used as the smoothing term of the reflectance component. The method according to claim 1,
Wherein the brightness channel is calculated in units of pixels of the target image. 3. The method of claim 2,
Wherein the energy function is expressed by the following equation.

here,

Is the energy function,

The first fidelity term,

The modeling image,

The illumination component,

Reflectance component,

The target image,

Is a smoothing constraint term of the illumination component,

The gradient of the illumination component,

Is a smoothing constraint term of the reflectance component,

Is a gradient of the reflectance component,

The second fidelity term,

The bright channel,

,

,

Each represents a normalization parameter, respectively. The method of claim 3,
Wherein the brightness channel is expressed by the following equation.

here,

Gt;

The bright channel,

Lt; RTI ID =

A patch in the target image,

Is an i-th pixel in the patch,

Is the target channel of the RGB channel (

),

Is a range filter,

Quot; means a spatial filter. A Retinex model-based image enhancement method performed in an apparatus including a processor,
Calculating the illumination component and the reflectance component such that a value of an energy function for extracting an illumination component and a reflectance component is minimized in an input target image; And
And calculating a restored image by removing the calculated illumination component from the target image,
Wherein the energy function includes a first data fidelity term between a modeling image composed of a combination of the illumination component and the reflectance component, a smoothing constraint term of the illumination component, a smoothing constraint term of the reflectivity component, And a second data fidelity term between the bright channels, wherein the l 1-norm of the gradient of the illumination component is used as a smoothing term of the reflectance component. A computer-readable recording medium recording a program for performing the method of claim 5.

Images