CN102265305A - Image processing device, method and image display device - Google Patents

Abstract

A first intermediate image generating means (1) generates an intermediate image (D1) obtained by extracting the components of a particular frequency band of an input image (DIN). A second intermediate image generating means (2) generates an intermediate image (D2) having higher frequency components than the intermediate image (D1). An intermediate image processing means (3M) generates an intermediate image (D3M) obtained by amplifying the pixel values of the intermediate image (D1). An intermediate image processing means (3H) generates an intermediate image (D3H) obtained by amplifying the pixel values of the intermediate image (D2). An adding means (4) adds together the input image (DIN), intermediate image (D3M) and intermediate image (D3H) to provide an output image (DOUT). A first amplification factor (D3MA) and a second amplification factor (D3HA) are decided in accordance with the pixel values of the input image (DIN). Even if the input image includes any folding components on the high frequency component side or includes no sufficient high frequency components, an enhanced image can be obtained without occurrence of any overshoots.

Description

Image processing device, method and image display device

technical field

The present invention relates to an image processing device and method for enhancing an input image, and an image display device using the same. For example, when an enlarged image obtained by enlarging an original image is input as an input image, by generating and added to perform image enhancement processing to obtain an output image with a high sense of clarity.

Background technique

Generally, an image is reproduced and displayed after appropriate image processing is applied to an image signal representing the image.

Furthermore, when image enhancement processing is performed on a color image, image enhancement processing is performed on a luminance signal.

For example, in the image processing device described in Patent Document 1, for a detailed image converted into multiple resolutions, an enhancement coefficient for a detailed image in a desired frequency band is set based on a signal of a detailed image whose frequency band is lower than the desired frequency band, thereby enhancing expected frequency band.

Furthermore, the sharpness enhancement circuit described in Patent Document 2 includes a first enhancement circuit and a second enhancement circuit, and the first enhancement circuit uses a frequency band including the highest high-frequency component of the brightness component for the brightness component of the input video signal. The second enhancement circuit enhances the luminance component of the video signal based on the lower center frequency than the first enhancement circuit.

prior art literature

patent documents

Patent Document 1: Japanese Patent Application Laid-Open No. 9-44651

Patent Document 2: Japanese Patent Laid-Open No. 2000-115582

Contents of the invention

The problem to be solved by the invention

However, in an image processing device that appropriately sets an enhancement factor for a detailed image in a desired frequency band for a detailed image converted into multiple resolutions, the enhancement process may be inappropriate or insufficient depending on the input image, and appropriate image quality cannot be obtained. output image.

For example, when an enlarged image is input as an input image, a part of the spectrum of the image before the enlargement is superimposed (superimposed component) appears on the high-frequency component side of the spectrum of the input image. Therefore, when the high-frequency component is simply enhanced, the superimposed component is also enhanced, which is inappropriate processing. Furthermore, when the frequency band is limited and only the frequency band not including the superimposed component is enhanced, the enhancement on the side of the high-frequency component is avoided in consideration of the frequency spectrum, resulting in insufficient enhancement processing.

Furthermore, when a noise-processed image is input as an input image, the frequency spectrum on the high-frequency component side disappears due to noise processing. Therefore, even if a high-frequency component is extracted, it may not be able to be extracted, and image enhancement processing cannot be performed sufficiently.

Also, when enhancing the luminance component of an input video signal, sometimes the color becomes white (or light) near the colored edge, or becomes black near the edge, or in the colored part (the chromaticity is relatively low). The color shade changes around the edge of the high part).

It is an object of the present invention to provide an image processing apparatus and method that can sufficiently enhance an image both when the input image contains superimposed components on the side of high-frequency components in its frequency spectrum and when the high-frequency components are not sufficiently included. deal with.

means to solve the problem

The image processing device of the present invention has:

a first intermediate image generating unit that generates a first intermediate image obtained by extracting a component of a specific frequency band of an input image;

a second intermediate image generation unit, which generates a second intermediate image based on the first intermediate image;

a first intermediate image processing unit, which generates a third intermediate image based on the first intermediate image;

a second intermediate image processing unit that generates a fourth intermediate image based on the second intermediate image; and

an adding unit that adds the input image, the third intermediate image, and the fourth intermediate image,

It is characterized in that,

The first intermediate image processing unit amplifies the pixel values of the first intermediate image using a first magnification factor obtained from the pixel values of the input image, or,

The second intermediate image processing unit amplifies the pixel values of the second intermediate image using a second magnification factor obtained from the pixel values of the input image.

The effect of the invention

According to the present invention, both when the input image contains superimposed components on the high-frequency component side in its frequency spectrum and when the high-frequency components do not sufficiently contain high-frequency components, image enhancement processing can be sufficiently performed while preventing overshoot from occurring.

Description of drawings

FIG. 1 is a block diagram showing the configuration of an image processing device according to Embodiment 1 of the present invention.

FIG. 2 is a block diagram showing a configuration example of the first intermediate image generation unit 1 in FIG. 1 .

FIG. 3 is a block diagram showing a configuration example of the second intermediate image generation unit 2 in FIG. 1 .

FIG. 4 is a block diagram showing a configuration example of the first intermediate image processing unit 3M in FIG. 1 .

FIG. 5 is a block diagram showing a configuration example of the second intermediate image processing unit 3H in FIG. 1 .

FIG. 6 is a block diagram showing a configuration example of the horizontal non-linear processing unit 2Ah in FIG. 3 .

FIG. 7 is a block diagram showing a configuration example of the vertical nonlinear processing unit 2Av in FIG. 3 .

(A) to (C) of FIG. 8 are diagrams showing pixel arrangements of the input image DIN and the images D1h and D1v.

9 is a block diagram showing a configuration example of an image display device using the image processing device according to Embodiment 1. FIG.

FIG. 10 is a block diagram showing a configuration example of the image enlarging unit U1 of FIG. 9 .

(A) to (E) of FIG. 11 are pixel arrangement diagrams showing the operation of the image enlarging unit U1 of FIG. 10 .

(A) to (D) of FIG. 12 are diagrams showing a frequency response and a frequency spectrum for explaining the operation of the image enlarging unit U1 of FIG. 10 .

(A) to (E) of FIG. 13 are diagrams showing a frequency response and a frequency spectrum for explaining the operation of the first intermediate image generating unit 1 in FIG. 1 .

(A) to (C) of FIG. 14 are diagrams showing a frequency response and a frequency spectrum for explaining the operation of the second intermediate image generating unit 2 in FIG. 1 .

(A) to (C) of FIG. 15 are diagrams showing a step edge and consecutive pixel signal values obtained when the step edge is sampled at the sampling interval S1.

(A) to (C) of FIG. 16 are diagrams showing a step edge and continuous pixel signal values obtained when the step edge is sampled at the sampling interval S2.

(A) to (F) of FIG. 17 are diagrams showing consecutive pixel signal values for explaining the operations of the first intermediate image generating unit 1 and the second intermediate image generating unit 2 in FIG. 1 .

(A) and (B) of FIG. 18 show that in the case where image sharpness is increased by adding high-frequency components moderately, and when high-frequency component addition is excessively performed, a picture is caused. A graph of consecutive pixel signal values in the case of quality degradation.

FIG. 19 is a diagram showing the relationship between the pixel value of the input image DIN and the magnification ratio in the first intermediate image processing unit 3M and the second intermediate image processing unit 3H.

20 is a block diagram showing the configuration of an image processing device according to Embodiment 2 of the present invention.

FIG. 21 is a block diagram showing a configuration example of the first intermediate image processing unit 103M in FIG. 20 .

FIG. 22 is a block diagram showing a configuration example of the second intermediate image processing unit 103H in FIG. 20 .

FIG. 23 is a block diagram showing a configuration example of the horizontal magnification determining unit 103MAh in FIG. 21 .

FIG. 24 is a block diagram showing a configuration example of vertical magnification determining unit 103MAv in FIG. 21 .

(A) and (B) of FIG. 25 are diagrams showing the relationship between the pixel value of the input image DIN and the magnification ratio in the first intermediate image processing section 103M and the second intermediate image processing section 103H.

FIG. 26 is a block diagram showing another configuration example of the horizontal magnification determining unit 103MAh in FIG. 21 .

FIG. 27 is a block diagram showing the configuration of an image processing device according to Embodiment 3 of the present invention.

FIG. 28 is a block diagram showing a configuration example of the first intermediate image processing unit 203M in FIG. 27 .

FIG. 29 is a block diagram showing a configuration example of the second intermediate image processing unit 203H in FIG. 27 .

(A) to (C) of FIG. 30 are diagrams showing pixel arrangements of the luminance and color difference addition image YC, image D1h, and image D1v.

31 is a block diagram showing a configuration example of an image display device using the image processing device according to Embodiment 3.

(A) to (E) of FIG. 32 are diagrams showing a frequency response and a frequency spectrum for explaining the operation of first intermediate image generating section 201 .

FIG. 33 is a diagram showing the relationship between the pixel values of the luminance and color difference added image YC and the magnification ratios in the first intermediate image processing section 203M and the second intermediate image processing section 203H.

34 is a block diagram showing the configuration of an image processing device according to Embodiment 4 of the present invention.

FIG. 35 is a block diagram showing a configuration example of the first intermediate image processing unit 303M in FIG. 34 .

FIG. 36 is a block diagram showing a configuration example of the second intermediate image processing unit 303H in FIG. 34 .

(A) to (E) of FIG. 37 are diagrams showing the pixel arrangement of the luminance-color-difference added image YC, the image D1h, and the image D1v and the arrangement of information indicating the sign of each pixel.

(A) and (B) of FIG. 38 are diagrams showing the relationship between the pixel values of the luminance and color difference added image YC and the magnification ratios in the first intermediate image processing section 303M and the second intermediate image processing section 303H.

39 is a block diagram showing the configuration of an image processing device according to Embodiment 5 of the present invention.

FIG. 40 is a block diagram showing a configuration example of the color difference increase/decrease unit 405 in FIG. 39 .

(A) to (C) of FIG. 41 are diagrams showing pixel arrangements of the high-frequency component addition image D404 , the input CR image CRIN , and the input CB image CBIN.

FIG. 42 is a diagram showing an example of the relationship between the pixel value L of the high-frequency component added image D404 and the magnification factor GAIN determined by the magnification factor determination unit 405A.

FIG. 43 is a diagram showing another example of the relationship between the pixel value L of the high-frequency component added image D404 and the magnification factor GAIN determined by the magnification factor determination unit 405A.

44 is a block diagram showing the configuration of an image processing device according to Embodiment 6 of the present invention.

FIG. 45 is a flowchart showing a processing procedure in the image processing method according to the sixth embodiment.

FIG. 46 is a flowchart showing processing in the first intermediate image generating step ST1 in FIG. 45 .

FIG. 47 is a flowchart showing processing in the second intermediate image generating step ST2 in FIG. 45 .

FIG. 48 is a flowchart showing processing in the horizontal non-linear processing step ST2Ah of FIG. 47 .

FIG. 49 is a flowchart showing processing in the vertical non-linear processing step ST2Av of FIG. 47 .

FIG. 50 is a flowchart showing processing in the first intermediate image processing step ST3M in FIG. 45 .

FIG. 51 is a flowchart showing processing in the second intermediate image processing step ST3H in FIG. 45 .

FIG. 52 is a flowchart showing a processing procedure in Embodiment 7 of the present invention.

FIG. 53 is a flowchart showing processing in the first intermediate image processing step ST103M in FIG. 52 .

FIG. 54 is a flowchart showing processing in the second intermediate image processing step ST103H in FIG. 52 .

FIG. 55 is a flowchart showing processing in the horizontal magnification factor determination step ST103MAh in FIG. 53 .

FIG. 56 is a flowchart showing processing in the vertical magnification determining step ST103MAv in FIG. 53 .

FIG. 57 is a flowchart showing a processing procedure in an image processing method according to Embodiment 8 of the present invention.

FIG. 58 is a flowchart showing processing in the first intermediate image processing step ST203M in FIG. 57 .

FIG. 59 is a flowchart showing processing in the second intermediate image processing step ST203H in FIG. 57 .

FIG. 60 is a flowchart showing a processing procedure in the image processing device according to Embodiment 9 of the present invention.

FIG. 61 is a flowchart showing processing in the first intermediate image processing step ST303M in FIG. 60 .

FIG. 62 is a flowchart showing processing in the second intermediate image processing step ST303H in FIG. 60 .

FIG. 63 is a flowchart showing a processing procedure in the image processing method according to Embodiment 10 of the present invention.

FIG. 64 is a flowchart showing processing in the magnification factor determining step ST405 of FIG. 63 .

Detailed ways

The image processing device according to the embodiment of the present invention described below can be used, for example, as a part of an image display device.

Embodiment 1

FIG. 1 is a diagram showing a configuration example of an image processing device according to Embodiment 1 of the present invention.

The illustrated image processing device includes a first intermediate image generation unit 1 , a second intermediate image generation unit 2 , a first intermediate image processing unit 3M, a second intermediate image processing unit 3H, and an addition unit 4 .

The first intermediate image generation unit 1 generates an intermediate image (first intermediate image) D1.

The second intermediate image generating unit 2 generates an intermediate image (second intermediate image) D2 obtained by performing processing described later on the intermediate image D1.

The first intermediate image processing unit 3M generates an intermediate image (third intermediate image) D2M obtained by processing the intermediate image D1 to be described later.

The second intermediate image processing unit 3H generates an intermediate image (fourth intermediate image) D3H obtained by processing the intermediate image D2 to be described later.

The adding unit 4 adds the input image DIN, the intermediate image D3M, and the intermediate image D3H. The image obtained after the addition by the adding unit 4 is output as the final output image DOUT.

FIG. 2 is a diagram showing a configuration example of the first intermediate image generating unit 1. The illustrated first intermediate image generating unit 1 includes: a high-frequency component image generating unit 1A that generates and extracts only the first frequency or higher from the input image DIN. and a low-frequency component image generation unit 1B that generates an image D1B obtained by extracting only low-frequency components below the second frequency of the image D1A. A band-pass filter unit that extracts specific frequency band components is constituted by the high-frequency component image generating unit 1A and the low-frequency component image generating unit 1B. Image D1B is output from first intermediate image generation unit 1 as intermediate image D1.

FIG. 3 is a diagram showing a configuration example of the second intermediate image generating unit 2. The illustrated second intermediate image generating unit 2 includes a nonlinear processing unit 2A whose output is subjected to nonlinear processing described later on the intermediate image D1. The resulting image D2A; and a high-frequency component image generating unit 2B that outputs an image D2B obtained by extracting only high-frequency components of the third frequency (third predetermined frequency) or higher of the image D2A. Image D2B is output from second intermediate image generating means 2 as intermediate image D2.

FIG. 4 is a diagram showing a configuration example of the first intermediate image processing unit 3M, and the illustrated first intermediate image processing unit 3M includes a magnification factor determining unit 3MA and a pixel value amplifying unit 3MB. The magnification factor determination unit 3MA determines the magnification factor D3MA based on the pixel values of the input image DIN. Pixel value enlarging means 3MB amplifies the pixel values of intermediate image D1 using enlarging factor D3MA determined by enlarging factor determining means 3MA, and outputs the result as intermediate image D3MB. The intermediate image D3MB is output from the first intermediate image processing unit 3M as an intermediate image D3M.

The magnification determination unit 3MA has a horizontal magnification determination unit 3MAh and a vertical magnification determination unit 3MAv, and the pixel value amplifying unit 3MB has a horizontal pixel value amplifying unit 3MBh and a vertical pixel value amplifying unit 3MBv. A first horizontal intermediate image processing unit 3Mh is formed by the horizontal magnification determining unit 3MAh and a horizontal pixel value amplifying unit 3MBh, and a first vertical intermediate image is formed by the vertical magnifying factor determining unit 3MAv and the vertical pixel value amplifying unit 3MBv. Processing unit 3Mv.

FIG. 5 is a diagram showing a configuration example of the second intermediate image post-processing unit 3H, and the illustrated second intermediate image post-processing unit 3H includes a magnification factor determining unit 3HA and a pixel value magnifying unit 3HB. The magnification factor determining unit 3HA determines the magnification factor D3HA based on the pixel value of the input image DIN. Pixel value enlarging means 3HB amplifies the pixel values of intermediate image D2 using enlarging factor D3HA determined by enlarging factor determining means 3HA, and outputs the result as intermediate image D3HB. The intermediate image D3HB is output from the first intermediate image post-processing unit 3H as an intermediate image D3H.

The magnification factor determining unit 3HA has a horizontal magnification factor determining unit 3HAh and a vertical magnification factor determining unit 3HAv, and the pixel value enlarging unit 3HB has a horizontal pixel value enlarging unit 3HBh and a vertical pixel value enlarging unit 3HBv. A second horizontal intermediate image processing unit 3Hh is formed by the horizontal magnification determining unit 3HAh and the horizontal pixel value enlarging unit 3HBh, and a second vertical intermediate image is formed by the vertical enlarging factor determining unit 3HAv and the vertical pixel value enlarging unit 3HBv Handling unit 3Hv.

The adding unit 4 adds the intermediate image D3M and the intermediate image D3H to the input image DIN to generate the final output image DOUT.

Hereinafter, detailed operations of the image processing device according to Embodiment 1 will be described.

First, the detailed operation of the first intermediate image generation unit 1 will be described.

The first intermediate image generation unit 1 generates an image D1A obtained by extracting only the high-frequency components of the input image DIN having a first frequency or higher in the high-frequency component image generation unit 1A. High-frequency components can be extracted by performing high-pass filter processing. The extraction of high-frequency components is carried out for the horizontal direction and vertical direction of the image respectively. That is, the high-frequency component image generating unit 1A includes a horizontal high-frequency component image generating unit 1Ah that performs horizontal high-pass filter processing on the input image DIN to generate high-frequency components that extract only the first horizontal frequency or higher in the horizontal direction. The resulting image D1Ah; and a vertical high-frequency component image generation unit 1Av, which performs high-pass filtering processing in the vertical direction to generate an image D1Av obtained by extracting only high-frequency components above the first vertical frequency for the vertical direction, image D1A Consists of image D1Ah and image D1Av.

Then, the first intermediate image generating unit 1 generates an image D1B obtained by extracting only low-frequency components equal to or less than the second frequency of the image D1A in the low-frequency component image generating unit 1B. Low-frequency components can be extracted by performing low-pass filter processing. The extraction of the low-frequency components is performed separately for the horizontal direction and the vertical direction. That is, the low-frequency component image generating unit 1B includes a horizontal low-frequency component image generating unit 1B that performs low-pass filtering processing in the horizontal direction on the image D1Ah to generate a low-frequency component obtained by extracting only low-frequency components equal to or lower than the second horizontal frequency in the horizontal direction. Image D1Bh; and a vertical direction low-frequency component image generation unit 1Bv, which performs a vertical low-pass filter process on the image D1Av to generate an image D1Bv obtained by extracting only low-frequency components below the second vertical frequency for the vertical direction, and the image D1B is obtained by Image D1Bh and image D1Bv are constituted. Image D1B is output from first intermediate image generation unit 1 as intermediate image D1. In addition, the intermediate image D1 is composed of an image D1h corresponding to the image D1Bh, and an image D1v corresponding to the image D1Bv.

Next, the detailed operation of the second intermediate image generation unit 2 will be described.

First, the second intermediate image generation unit 2 generates an image D2A obtained by performing nonlinear processing described later on the intermediate image D1 in the nonlinear processing unit 2A. Non-linear processing is performed separately for the horizontal and vertical directions. That is, the non-linear processing unit 2A has: a horizontal non-linear processing unit 2Ah that performs the non-linear processing described later on the image D1h to generate an image D2Ah; and a vertical non-linear processing unit 2Av that performs the non-linear processing described below on the image D1v. Non-linear processing is performed to generate an image D2Av, and the image D2A is composed of an image D2Ah and an image D2Av.

The operation of the nonlinear processing unit 2A will be described in more detail. The non-linear processing unit 2A has a horizontal non-linear processing unit 2Ah and a vertical non-linear processing unit 2Av having the same configuration as each other. The horizontal non-linear processing unit 2Ah performs horizontal processing, and the vertical non-linear processing unit 2Av performs vertical processing.

FIG. 6 is a diagram showing a configuration example of the horizontal non-linear processing unit 2Ah. The illustrated horizontal non-linear processing unit 2Ah has a zero-crossing determination unit 311h and a signal amplification unit 312h. The image D1h is input to the nonlinear processing unit 2Ah as the input image DIN311h.

The zero-cross determination unit 311h confirms changes in pixel values in the input image DIN311h in the horizontal direction. Then, the position where the pixel value changes from positive to negative or from negative to positive is captured as a zero-crossing point, and the position of the pixels before and after the zero-crossing point (adjacent pixels) is conveyed to the signal amplifying unit 312h through the signal D311h . Here, "front and rear" means front and rear in the order of signal supply, "left and right" when pixel signals are supplied from left to right in the horizontal direction, and "up and down" when pixel signals are supplied from top to bottom in the vertical direction. In the zero cross determination unit 311h in the horizontal non-linear processing unit 2Ah, pixels located around the zero cross point are recognized as pixels located before and after the zero cross point.

The signal amplifying unit 312h determines the pixels located before and after the zero crossing point (adjacent pixels) according to the signal D311h, and generates a non-linearly processed image D312h after only amplifying the pixel values (increasing the absolute value) of the pixels located before and after the zero crossing point. That is, the magnification factor is set to a value greater than 1 for the pixel values of pixels located before and after the zero cross point, and the magnification factor is set to 1 for the pixel values of other pixels.

The non-linearly processed image D312h is output from the horizontal direction non-linear processing unit 2Ah as the image D2Ah.

FIG. 7 is a diagram showing a configuration example of the vertical nonlinear processing unit 2Av. The illustrated vertical non-linear processing unit 2Av has a zero-cross determination unit 311v and a signal amplification unit 312v. The image D1v is input to the nonlinear processing unit 2Av as an input image DIN311v.

The zero-cross determination unit 311v confirms changes in pixel values in the input image DIN311v along the vertical direction. Then, the position where the pixel value changes from positive value to negative value or from negative value to positive value is captured as a zero-crossing point, and the position of the pixels before and after the zero-crossing point (adjacent pixels) is conveyed to the signal amplifying unit 312v through the signal D311v . In the zero-cross determination section 311v in the vertical non-linear processing section 2Av, pixels located above and below the zero-cross point are recognized as pixels located before and after the zero-cross point.

The signal amplifying unit 312v determines the pixels located before and after the zero cross point (adjacent pixels) according to the signal D311v, and generates a non-linearly processed image D312v in which only the pixel values of the pixels located before and after the zero cross point are amplified (increased in absolute value). That is, the magnification factor is set to a value greater than 1 for the pixel values of pixels located before and after the zero cross point, and the magnification factor is set to 1 for the pixel values of other pixels.

The above is the operation of the nonlinear processing unit 2A.

Then, the second intermediate image generation unit 2 generates the image D2B obtained by extracting only the high-frequency components of the third frequency or higher of the image D2A in the high-frequency component image generation unit 2B. High-frequency components can be extracted by performing high-pass filter processing. The high-frequency components are extracted for the horizontal direction and the vertical direction of the image respectively. That is, the high-frequency component image generating unit 2B has a horizontal high-frequency component image generating unit 2Bh that performs horizontal high-pass filter processing on the image D2Ah to generate high-frequency components that extract only the third horizontal frequency or higher in the horizontal direction. The obtained image D2Bh; and a vertical high-frequency component image generation unit 2Bv, which performs a vertical high-pass filter process on the image D2Av to generate an image D2Bv obtained by extracting only high-frequency components above the third vertical frequency for the vertical direction, Image D2B is composed of image D2Bh and image D2Bv. Image D2B is output from second intermediate image generating means 2 as intermediate image D2. Intermediate image D2 is composed of image D2h corresponding to image D2Bh and image D2v corresponding to image D2Bv.

Next, detailed operations of the first intermediate image processing unit 3M will be described.

In the first intermediate image processing unit 3M, the magnification factor D3MA is determined by the magnification factor determining unit 3MA based on the pixel values of the input image DIN. As described above, the pixel values of the first intermediate image D1 are amplified according to the magnification factor D3MA. However, since the first intermediate image D1 is composed of the image D1h and the image D1v, the magnification factor D3MAh for the image D1h and the magnification factor for the image D3MA are determined as the magnification factor D3MA. Magnification D3MAv of image D1v. That is, the magnification factor determining unit 3MA has a horizontal magnification factor determining unit 3MAh and a vertical magnification factor determining unit 3MAv. In the horizontal magnification factor determining unit 3MAh, the magnification factor D3MAh is determined based on the pixel value of the input image DIN, and the magnification rate in the vertical direction is enlarged. In the ratio determination unit 3MAv, the amplification ratio D3MAv is determined based on the pixel value of the input image DIN, and the amplification ratio D3MAh and the amplification ratio D3MAv are output from the amplification ratio determination unit 3MA as the amplification ratio D3MA.

The operations of the horizontal magnification determining unit 3MAh and the vertical magnifying factor 3MAv will be described in more detail.

(A) to (C) of FIG. 8 are diagrams showing the input image DIN and images D1h and D1v. FIG. 8(A) shows the input image DIN, FIG. 8(B) shows the image D1h, and FIG. 8(C) Denotes image D1v. In addition, in (A) to (C) of FIG. 8 , horizontal coordinates, vertical coordinates, and respective coordinate values are shown for the horizontal direction and the vertical direction of the image. In addition, for the input image DIN, the pixel value of the pixel at the position of the horizontal coordinate x and the vertical coordinate y is represented by a symbol such as DIN(xy), and for the image D1h, the pixel value of the pixel at the position of the horizontal coordinate x and the vertical coordinate y The pixel value of the image D1h (xy) is represented by the symbol D1h (xy), and the pixel value of the pixel at the position of the horizontal coordinate x and the vertical coordinate y for the image D1v is represented by the symbol D1v (xy).

The horizontal magnification determining means 3MAh determines the magnification of each pixel of the image D1h based on the pixel values at the same coordinates of the input image DIN. That is, the magnification factor for the pixel value D1h(11) is determined according to the pixel value DIN(11), and the magnification factor for the pixel value D1h(12) is determined according to the pixel value DIN(12). Generally, for example, according to the pixel value DIN( xy) determines the magnification factor for the pixel value D1h(xy), the magnification factor is determined from the pixel value at the same coordinates of the input image DIN, and the result is output as the magnification factor D3MAh.

Further, the vertical magnification determining means 3MAv determines the magnification of each pixel of the image D1v based on the pixel values of the same coordinates of the input image DIN. That is, the magnification factor for the pixel value D1v(11) is determined according to the pixel value DIN(11), and the magnification factor for the pixel value D1v(12) is determined according to the pixel value DIN(12). Generally, for example, according to the pixel value DIN( xy) determines the magnification factor for the pixel value D1v(xy), the magnification factor is determined from the pixel value at the same coordinates of the input image DIN, and the result is output as the magnification factor D3MAv.

Then, the pixel value enlarging means 3MB amplifies the pixel values of the first intermediate image D1 according to the enlarging factor D3MA. Since the first intermediate image D1 is composed of an image D1h and an image D1v, the pixel value enlarging unit 3MB has: a horizontal pixel value enlarging unit 3MBh for enlarging the pixel value of the image D1h; and a vertical enlarging unit 3MBh for enlarging the pixel value of the image D1v. Directional pixel value amplification unit 3MBv.

That is, the horizontal pixel value enlarging unit 3MBh outputs an image D3MBh in which the pixel values of the image D1h are enlarged by the magnification factor D3MAh, and the vertical pixel value enlarging unit 3MBv outputs an image D3MBv in which the pixel values of the image D1v are amplified by the magnification factor D3MAv. Then, image D3MBh and image D3MBv are output from pixel value enlarging unit 3MB as image D3MB.

Image D3MB is output from first intermediate image processing unit 3M as intermediate image D3M. Intermediate image D3M is composed of image D3Mh equivalent to image D3MBh and image D3Mv equivalent to image D3MBv.

The above is the operation of the first intermediate image processing unit 3M.

Next, the operation of the second intermediate image processing unit 3H will be described. Comparing Fig. 4 with Fig. 5, the second intermediate image processing unit has the same structure as the first intermediate image processing unit except that the input signal is the input image DIN and intermediate image D2, and the second intermediate image processing unit 3H outputs a pair of intermediate image D2 Intermediate image D3H obtained by performing the same processing as that performed on intermediate image D1 by first intermediate image processing unit 3M. Since the detailed operation of the second intermediate image processing unit 3H is also clear from the detailed operation description of the first intermediate image processing unit 3M, the detailed operation description of the second intermediate image processing unit 3H is omitted.

Finally, the operation of the adding unit 4 will be described. Addition unit 4 generates output image DOUT obtained by adding input image DIN, intermediate image D3M, and intermediate image D3H. The output image DOUT of the adding unit 4 is output from the image processing device as a final output image.

The intermediate image D3M is composed of the image D3Mh and the image D3Mv, and the intermediate image D3H is composed of the image D3Hh and the image D3Hv, thus adding the input image DIN, the intermediate image D3M and the intermediate image D3H means that all the images D3Mh, D3Mv, D3Hh and D3Hv are combined with The input images DIN are summed.

Here, the addition process in the addition unit 4 is not limited to simple addition, but weighted addition may be performed. That is, each of the images D3Mh, D3Mv, D3Hh, and D3Hv may be enlarged at different magnification factors and added to the input image DIN.

Hereinafter, an example in which the image processing device in this embodiment is used as a part of the image display device will be described. The operation and effects of the image processing device in this embodiment will also be clarified through this description. In the following description, unless otherwise specified, the symbol Fn represents the Nyquist frequency of the input image DIN.

FIG. 9 shows an image display device using the image processing device in this embodiment. In the illustrated image display device, an image corresponding to the original image DORG is displayed on the monitor U3.

Image enlarging unit U1 outputs image DU1 in which original image DORG is enlarged when the image size of original image DORG is smaller than the image size of monitor U3. Here, the enlargement of the image can be performed using, for example, a bicubic method or the like.

The image processing device U2 in this embodiment outputs the image DU2 obtained by performing the processing described above on the image DU1. Then, the image DU2 is displayed on the monitor U3.

Hereinafter, assuming that the pixel values of the original image DORG are half of the pixel values of the monitor U3 in the horizontal and vertical directions, the operation and function of the image enlarging unit U1 will be described first.

10 is a diagram showing the structure and operation of the image enlarging unit U1. The image enlarging unit U1 has a horizontal direction zero insertion unit U1A, a horizontal low frequency component passing unit U1B, a vertical zero insertion unit U1C, and a vertical low frequency component passing unit U1D. .

The horizontal direction zero insertion unit U1A generates a pixel with a pixel value 0 that is appropriately inserted in the horizontal direction of the original image DORG (one row of pixels with a pixel value of 0 is inserted between adjacent pixel columns in the horizontal direction of the original image DORG) pixel column) of image DU1A.

The horizontal low-frequency component passing unit U1B generates an image DU1B obtained by extracting only the low-frequency component of the image DU1A through low-pass filtering.

The vertical direction zero insertion unit U1C generates a pixel with a pixel value of 0 that is appropriately inserted with respect to the vertical direction of the image DU1B (one row of pixels with a pixel value of 0 is inserted between adjacent pixel rows in the vertical direction of the image DU1B) After the rows of pixels that make up the image DU1C.

The vertical low-frequency component passing unit DU1D generates an image DU1D obtained by extracting only the low-frequency component of the image DU1C.

The image DU1D is output from the image enlarging unit U1 as the image DU1 that is twice the size of the original image DORG in the horizontal direction and the vertical direction.

(A) to (E) of FIG. 11 are diagrams for explaining the operation of the image enlarging unit U1 in detail. (A) of FIG. 11 shows the original image DORG, (B) of FIG. 11 shows the image DU1A, and (C ) represents the image DU1B, (D) of FIG. 11 represents the image DU1C, and (E) of FIG. 11 represents the image DU1D. Regarding (A) to (E) of FIG. 11 , squares (each square) represent pixels, and symbols or numerical values written therein represent pixel values of each pixel.

The horizontal direction zero insertion unit U1A inserts a pixel with a pixel value of 0 every other pixel in the horizontal direction for the original image DORG shown in (A) of FIG. One pixel column consisting of pixels with a pixel value of 0 is inserted between the columns), and an image DU1A shown in (B) of FIG. 11 is generated. The horizontal direction low-frequency component passing unit U1B performs low-pass filter processing on the image DU1A shown in (B) of FIG. 11 to generate the image DU1B shown in (C) of FIG. 11 .

The vertical direction zero insertion unit U1C inserts a pixel with a pixel value of 0 every other pixel in the vertical direction for the image DU1B shown in (C) of FIG. One pixel row consisting of pixels with a pixel value of 0 is inserted between each other), and an image DU1C shown in (D) of FIG. 11 is generated. The vertical low-frequency component passing unit U1D performs low-pass filter processing on the image DU1C shown in (D) of FIG. 11 to generate the image DU1D shown in (E) of FIG. 11 . An image DU1D in which the original image DORG is doubled in both the horizontal direction and the vertical direction is generated through the above processing.

(A) to (D) of FIG. 12 are diagrams showing the operation of the processing of the image enlargement unit U1 in frequency space, (A) of FIG. 12 shows the frequency spectrum of the original image DORG, and (B) of FIG. 12 shows the frequency spectrum of the image DU1A. Spectrum, (C) of FIG. 12 shows the frequency response of the low-frequency component in the horizontal direction passing through the unit U1B, and (D) of FIG. 12 shows the spectrum of the image DU1B. In addition, in (A) to (D) of FIG. 12 , the horizontal axis represents the frequency axis representing the spatial frequency in the horizontal direction, and the vertical axis represents the intensity of the frequency spectrum or frequency response.

The number of pixels of the original image DORG is half of that of the input image DIN, in other words, the sampling interval of the original image DORG is twice the sampling interval of the input image DIN. Therefore, the Nyquist frequency of the original image DORG is half of the Nyquist frequency of the input image DIN, ie Fn/2.

In addition, in (A) to (D) of FIG. 12 , only one frequency axis is used for simple representation. Usually, however, image data is composed of pixel values given on a pixel array arranged in a two-dimensional plane, and its spectrum is also a spectrum given on a plane developed with a frequency axis in the horizontal direction and a frequency axis in the vertical direction. Therefore, in order to accurately represent the spectrum of the original image DORG or the like, it is necessary to describe both the frequency axis in the horizontal direction and the frequency axis in the vertical direction. However, the spectral shape of the original image DORG is usually a shape that is isotropically expanded around the origin on the frequency axis, and those skilled in the art can easily understand it by showing the spatial spectrum expanded by one frequency axis. Expand to the space expanded by two frequency axes for investigation. Therefore, in the following description, unless otherwise specified, the frequency space will be described using a space expanded on one frequency axis.

First, the frequency spectrum of the original image DORG will be described. Normally, a natural image is input as the original image DORG, but in this case, the spectral intensity of the original image DORG is concentrated around the origin of the frequency space. Therefore, the spectrum of the original image DORG becomes the spectrum SPO shown in (A) of FIG. 12 .

Next, the spectral intensity of image DU1A will be described. For the original image DORG, a pixel having a pixel value of 0 is inserted every other pixel in the horizontal direction to generate an image DU1A. When this processing is performed, a superposition centered on the Nyquist frequency of the original image DORG is generated on the frequency spectrum. That is, since the spectrum SPM obtained by superimposing the spectrum SPO around the frequency ±Fn/2 is generated, the spectrum of the image DU1A is as shown in (B) of FIG. 12 .

Next, the frequency response of the horizontal low-frequency component passing unit U1B will be described. Since the horizontal low-frequency component passing unit is realized by a low-pass filter, as shown in (C) of FIG. 12 , the higher the frequency, the lower the frequency response of the horizontal low-frequency component passing unit U1B.

Finally, the frequency spectrum of image DU1B will be described. Image DU1B shown in FIG. 12(D) is obtained by performing low-pass filter processing having a frequency response shown in FIG. 12(C) on image DU1A having a frequency spectrum shown in FIG. 12(B) . Therefore, the spectrum of the image DU1B is composed of the spectrum SP2 in which the intensity of the spectrum SPM is decreased to some extent, and the spectrum SP1 in which the intensity of the spectrum SPO is decreased to some extent, as shown in the image DU1B. Also, generally, the higher the frequency, the lower the frequency response of the low-pass filter. Therefore, when the intensity of the spectrum SP1 is compared with the spectrum SPO, the horizontal low-frequency component passing unit U1B passes through the spectrum in which the intensity of the spectrum on the high-frequency side, that is, at frequencies around ±Fn/2, decreases.

Also, in the processing of the image enlarging unit U1, the processing of the vertical zero insertion unit U1C and the vertical low-frequency component passing unit U1D is omitted, but it can be easily understood from the content of the processing. , has the same effect as that described with reference to (A) to (D) of FIG. 12 with respect to the axial direction indicating the spatial frequency in the vertical direction. That is, the spectrum of image DU1D is obtained by two-dimensionally expanding the spectrum shown in (D) of FIG. 12 .

Also, in the following description, the spectrum SP2 will be referred to as a superposition component. On the image, this superimposed component appears as noise or artifacts with higher frequency components. Such noise or glitches include overshoot, sawtooth, ringing, and the like.

The operation and effect of the image processing device in this embodiment will be described below.

(A) to (E) of FIG. 13 schematically show how the intermediate image D1 is generated from the input image DIN when the image DU1D obtained by enlarging the original image DORG is input as the input image DIN (or image DU1 ). Figure 13(A) shows the frequency spectrum of the input image DIN, FIG. 13(B) shows the frequency response of the high-frequency component image generating unit 1A, and FIG. 13(C) shows the low-frequency component image generating unit 1B. 13 (D) shows the frequency response of the first intermediate image generation unit 1, and (E) of FIG. 13 shows the frequency spectrum of the intermediate image D1. In addition, in (A) to (E) of FIG. 13 , only one frequency axis is used for the same reason as in (A) to (D) of FIG. 12 .

Furthermore, in (A) to (E) of FIG. 13 , the strength of the frequency spectrum or frequency response is shown only in the range where the spatial frequency is equal to or greater than 0, but the frequency spectrum or frequency response in the following description is based on the origin on the frequency axis Centrosymmetric shape. Therefore, it is sufficient that the diagrams used in the description only show the range in which the spatial frequency is equal to or greater than 0.

First, the frequency spectrum of the input image DIN will be described. Since the image DU1D generated by the enlargement processing in the image enlargement unit U1 is input as the input image DIN, as shown in (A) of FIG. 13 , the frequency spectrum of the input image DIN has They have the same shape, and are composed of a spectrum SP1 in which the intensity of the spectrum SPO of the original image DORG is reduced to some extent, and a spectrum SP2 that becomes a superimposed component.

Next, the frequency response of the high-frequency component image generating unit 1A will be described. Since the high-frequency component image generating unit 1A is composed of a high-pass filter, as shown in (B) of FIG. 13 , the lower the frequency, the lower its frequency response.

Next, the frequency response of the low-frequency component image generating unit 1B will be described. Since the low-frequency component image generating unit 1B is constituted by a low-pass filter, as shown in (C) of FIG. 13 , the higher the frequency, the lower its frequency response.

Next, the frequency response of the first intermediate image generation unit 1 will be described. The high-frequency component image generating unit 1A in the first intermediate image generating unit 1 attenuates the region on the low-frequency component side shown in (D) of FIG. "Low frequency band) the frequency component of RL1. On the other hand, by the low-frequency component image generating unit 1B in the first intermediate image generating unit 1, the frequency of the region on the high-frequency component side (a frequency band higher than the second frequency FL2) RH1 shown in FIG. 13(D) is attenuated. portion. Therefore, as shown in (D) of FIG. 13 , the frequency response of the first intermediate image generation unit 1 is within the middle region (specific frequency band) RM1 in which the frequency band is limited by the region RL1 on the low-frequency component side and the region RH1 on the high-frequency component side. has a peak.

Next, the frequency spectrum of the intermediate image D1 will be described. The input image DIN having the frequency spectrum shown in (A) of FIG. 13 passes through the first intermediate image generation unit 1 having the frequency response shown in (D) of FIG. 13 to obtain the intermediate image shown in (E) of FIG. 13 D1. Then, the frequency response of the first intermediate image generation unit 1 has a peak in the middle region RM1 where the frequency band is limited by the region RL1 on the low-frequency component side and the region RH1 on the high-frequency component side, so that the frequency spectrum of the intermediate image D1 becomes a weakened input image. In the DIN spectrum, the intensity spectrum of the parts included in the region RL1 on the low-frequency component side and the region RH1 on the high-frequency component side is obtained. Therefore, the intermediate image D1 is an image obtained by removing the spectrum SP2 serving as the superimposed component from the high-frequency components of the input image DIN. That is, the first intermediate image generating unit 1 has an effect of generating an intermediate image D1 obtained by removing the spectrum SP1 serving as the superimposed component from the high-frequency components of the input image DIN.

(A) to (C) of FIG. 14 are diagrams showing the operation and effect of the second intermediate image generation unit 2, (A) of FIG. 14 shows the frequency spectrum of the nonlinear processing unit D2A, and (B) of FIG. The frequency response of the component image generating unit 2B, (C) of FIG. 14 shows the frequency spectrum of the image D2B. In addition, in (A) to (C) of FIG. 14 , for the same reason as in (A) to (E) of FIG. 13 , the spectrum or frequency response is shown only in the range where the spatial frequency is 0 or higher.

As will be described later, in the nonlinear processed image D2A, a high-frequency component corresponding to a region RH2 on the high-frequency component side is generated. (A) of FIG. 14 is a diagram schematically showing this situation. Nonlinearly processed image D2A passes through high-frequency component image generating means 2B, thereby generating image D2B shown in (C) of FIG. 14 . The high-frequency component image generator 2B is composed of a high-pass filter that passes components of the third frequency FL3 or higher, and as shown in (B) of FIG. 14 , the higher the frequency, the higher the frequency response. Therefore, as shown in (C) of FIG. 14 , the spectrum of the image D2B is obtained by removing the component corresponding to the region RL2 on the low-frequency component side (frequency components lower than the third frequency FL3 ) from the spectrum of the nonlinearly processed image D2A. of. In other words, the nonlinear processing unit 2A has the effect of generating a high-frequency component corresponding to the region RH2 on the high-frequency component side, and the high-frequency component image generating unit 2B has the effect of extracting only the high-frequency component generated by the nonlinear processing unit 2A. In addition, in the illustrated example, the third frequency FL3 is substantially equal to Fn/2.

The operation and effect described above will be described in more detail.

(A) to (C) of FIG. 15 and (A) to (C) of FIG. 16 are diagrams showing signals obtained when a step edge is sampled.

(A) of FIG. 15 represents a step edge and sampling interval S1, (B) of FIG. 15 represents a signal obtained when the step edge is sampled with sampling interval S1, and (C) of FIG. 15 represents (B) of FIG. 15 ) shows the high-frequency component of the signal. On the other hand, (A) of FIG. 16 shows a step edge and a sampling interval S2 wider than the sampling interval S1, and (B) of FIG. 16 shows a signal obtained when the step edge is sampled at the sampling interval S2. FIG. (C) of 16 shows a high-frequency component of the signal shown in (B) of FIG. 16 . In the following description, it is assumed that the length of the sampling interval S2 is twice the length of the sampling interval S1.

As shown in (C) of FIG. 15 and (C) of FIG. 16 , the center of the step edge appears as a zero-crossing point Z in a signal representing a high-frequency component. Moreover, the shorter the sampling interval, the steeper the slope of the signal representing the high-frequency component near the zero-crossing point Z, and the shorter the sampling interval, the position of the point that gives the local maximum and minimum near the zero-crossing point Z will be The closer to the zero crossing point Z.

That is, even if the sampling interval changes, the zero-crossing position of the signal representing the high-frequency component near the edge does not change, but the smaller the sampling interval (or the higher the resolution), the steeper the slope of the high-frequency component near the edge, The closer the position of the point giving the local maximum and minimum is to the zero-crossing point.

(A) to (F) of FIG. 17 show the operation and effect when the signal obtained by sampling the step edge at the sampling interval S2 is amplified by a factor of 2 and then input to the image processing device in this embodiment. In particular, the figure shows the operations and effects of the first intermediate image generation unit 1 and the second intermediate image generation unit 2 . In addition, since the internal processing of the first intermediate image generation unit 1 and the second intermediate image generation unit 2 is performed separately for the horizontal direction and the vertical direction as described above, the processing is performed one-dimensionally. Therefore, in (A) to (F) of FIG. 17 , the processing content is expressed using a one-dimensional signal.

Like (B) of FIG. 16 , (A) of FIG. 17 is a signal obtained by sampling step edges at the sampling interval S2 . (B) of FIG. 17 shows a signal obtained by amplifying the signal shown in (A) of FIG. 17 twice. That is, when the edge shown in (A) of FIG. 17 is included in the original image DORG, the signal shown in (B) of FIG. 17 is input as the input image DIN. In addition, when the signal is amplified to 2 times, the sampling interval is half of that before amplification, so the sampling interval of the signal shown in (B) of FIG. 17 is the same as the sampling interval S1 in (A) to (C) of FIG. 14 . Also, in (A) of FIG. 17 , the position indicated by the coordinate P3 is the boundary portion on the low luminance side (low level side) of the edge signal, and the position indicated by the coordinate P4 is the high luminance side (high level side) of the edge signal. flat side) of the boundary portion.

(C) of FIG. 17 is a signal representing a high-frequency component of the signal shown in (B) of FIG. 17 , that is, a signal corresponding to image D1A output from high-frequency component image generating unit 1A. In addition, since the image D1A is obtained by extracting the high-frequency components of the input image DIN, it also includes superimposed components.

(D) of FIG. 17 is a signal representing the low-frequency component of the signal shown in (C) of FIG. 17 , that is, a signal corresponding to the image D1B output from the low-frequency component image generating unit 1B. In addition, since image D1B is output as intermediate image D1 as mentioned above, (D) of FIG. 17 also corresponds to intermediate image D1. As shown in (D) of FIG. 17 , in the intermediate image D1, the local minimum value near the zero-crossing point Z appears at the coordinate P3, and the local maximum value appears at the coordinate P4. This situation is similar to that shown in (C) of FIG. 16 , The high-frequency components extracted from the signal obtained by sampling the step edge at the sampling interval S2 agree. Also, superimposed components contained in the image D1A are removed by low-pass filter processing performed in the low-frequency component image generating unit 1B.

(E) of FIG. 17 shows an output signal when the signal shown in (D) of FIG. 17 is input to the nonlinear processing unit 2A, that is, an image output from the nonlinear processing unit 2A when the intermediate image D1 is input. D2A. In the nonlinear processing unit 2A, the signal values of the coordinates P1 , P2 before and after the zero crossing point Z (adjacent before and after) are amplified. Therefore, as shown in (E) of Figure 17, the signal value of image D2A at coordinates P1 and P2 is larger than other values, and near the zero-crossing point Z, the position where the local minimum appears changes from coordinate P3 to closer to The coordinate P1 of the zero-crossing point Z, the position where the local maximum occurs changes from the coordinate P4 to the coordinate P2 closer to the zero-crossing point Z. This means that a high-frequency component is generated by the non-linear processing of amplifying pixel values before and after the zero-crossing point Z in the non-linear processing unit 2A. In this way, high-frequency components can be generated by appropriately changing the amplification factor for each pixel, or by appropriately changing the content of processing for each pixel. That is, nonlinear processing unit 2A has an effect of generating high-frequency components not included in intermediate image D1 , that is, high-frequency components corresponding to region RH2 on the high-frequency component side shown in FIG. 14(A) .

(F) of FIG. 17 is a signal representing the high-frequency component of the signal shown in (E) of FIG. 17 , that is, a signal corresponding to the image D2B output from the high-frequency component image generator 2B. The more accurate shape of image D2B will be described later. As shown in (F) of FIG. The peak on the positive side) appears at the coordinate P2, which corresponds to the high-frequency component extracted from the signal obtained by sampling the step edge at the sampling interval S1 shown in (C) of FIG. 15 . This means that the high-frequency components generated in the nonlinear processing unit 2A are taken out by the high-frequency component image generating unit 2B and output as an image D2B.

Furthermore, it can be said that the extracted image D2B is a signal including frequency components corresponding to the sampling interval S1. In other words, the high-frequency component image generating unit 2B has an effect of taking out only the high-frequency components generated in the nonlinear processing unit 2A.

The above is the effect of the second intermediate image generation processing unit 2. In summary, the nonlinear processing unit 2A in the second intermediate image generation processing unit 2 has the effect of generating a high-frequency component corresponding to the region RH2 on the high-frequency component side. 2. The high-frequency component image generating unit 2B in the intermediate image generation processing unit 2 has the effect of extracting only the high-frequency components corresponding to those generated by the nonlinear processing unit 2A. Then, since the image D2B is output as the intermediate image D2, the second intermediate image generation unit 2 can output the intermediate image D2 having high-frequency components corresponding to the sampling interval S1.

Here, image enhancement processing can be performed by adding intermediate image D1 and intermediate image D2 to input image DIN.

In this embodiment, instead of adding the first intermediate image D1 and the second intermediate image D2 to the input image DIN, the effect obtained when the first intermediate image and the second intermediate image are added is assumed below. Next, the effect of adding the third intermediate image D3M and the fourth intermediate image D3H instead of the first intermediate image D1 and the second intermediate image D2 will be described.

First, the effect of adding the intermediate image D1 will be described. As mentioned above, the intermediate image D1 is obtained by extracting superimposed components from the high-frequency components of the input image DIN. As shown in (E) of FIG. portion. As explained in (D) of FIG. 12 , the spectral intensity near the Nyquist frequency of the original image DORG is weakened by the enlarging process in the image enlarging unit U1. Spectral intensity attenuated by processing. Also, since the superimposed component is removed from the intermediate image D1, artifacts such as overshoot, jaggies, or ringing are not enhanced.

Next, the effect of adding the intermediate image D2 will be described. As mentioned above, the intermediate image D2 is a high-frequency component corresponding to the sampling interval S1. Therefore, by adding the intermediate image D2, high-frequency components in the frequency band above the Nyquist frequency of the original image DORG can be given, thereby increasing the sharpness of the image.

To sum up, by adding the intermediate image D1 and the intermediate image D2 to the input image DIN, high-frequency components can be added without enhancing the superimposed components, and the clarity of the image can be improved.

In addition, by adding the generated high-frequency components to the input image as described above, the sharpness of the image can be increased, and the image quality can be improved. However, excessive addition of high-frequency components may actually lead to a decrease in image quality.

(A) and (B) of FIG. 18 are diagrams for explaining image quality degradation caused by addition of high-frequency components, and (A) of FIG. 18 shows that image quality is increased by adding high-frequency components appropriately. In the case of sharpness, (B) of FIG. 18 shows a case where high-frequency components are excessively added and image quality deteriorates.

(A) of FIG. 18 shows that the intermediate image D1 shown in (D) of FIG. 17 and the intermediate image D2 shown in (F) of FIG. 17 are added to the input image DIN shown in (B) of FIG. 17 In the figure of the result after that, the boundary portion on the low-brightness side of the step edge represented by the coordinate P3 in (A) of FIG. The boundary portion on the low-brightness side of the step edge indicated by the coordinate P3 in (A) of (A) is corrected to the position indicated by the coordinate P1 in (A) of FIG. The boundary portion on the high-brightness side of the step edge is corrected to the position indicated by the coordinate P2 in (A) of FIG. 18. As a result, comparing (A) of FIG. 17 with (B) of FIG. (A) of 18 approaches the step edge shown in (A) of FIG. 16 . This indicates that the sharpness of the image increases by moderately adding high-frequency components.

On the other hand, (B) of FIG. 18 also shows that the intermediate image D1 shown in (D) of FIG. 17 and the intermediate image D2 shown in (F) of FIG. 17 and the input shown in (B) of FIG. The graph of the result of adding the image DIN is different from the case of FIG. 18(A), and FIG. 18(B) shows the case where the high-frequency component is excessively added. Comparing with (A) of FIG. 18 , it can be seen that the luminance of the position indicated by the coordinates P1 and P3 is unnaturally low (undershoot) compared with its surroundings, or the luminance of the position indicated by the coordinates P2 and P4 is high compared with its surroundings. The result is unnatural (overshoot), and the image quality is degraded.

When the magnitude of luminance added or subtracted by intermediate image D1 and intermediate image D2 (hereinafter referred to as correction amount) is excessively large, undershoot and overshoot, which are factors for these image quality degradations, tend to occur. Therefore, it is considered that it is only necessary to adjust so that the correction amounts of the intermediate image D1 and the intermediate image D2 do not become excessively large.

As this method, for example, it is considered that there is a method of detecting a local maximum value of the correction amount given by intermediate image D1 and intermediate image D2, and when the detected maximum value exceeds a predetermined value, by appropriately applying a gain so that the intermediate The correction amounts of the image D1 and the intermediate image D2 are reduced so that the correction amounts do not increase excessively.

However, when such a method is used, in order to determine a local maximum value, it is necessary to refer to data of several pixels, which leads to an increase in circuit scale. Furthermore, when it is necessary to refer to the data of several pixels in the vertical direction, the addition of line memory also becomes a factor of cost increase.

Therefore, in this embodiment, by changing the magnification applied to intermediate image D1 and intermediate image D2 according to the pixel value of input image DIN, the correction amount added by intermediate image D1 and intermediate image D2 will not be excessively increased. Especially to prevent the occurrence of overshoot. In the image processing device according to Embodiment 1, in the first intermediate image processing unit 3M and the second intermediate image processing unit 3H, different magnification factors are appropriately applied to the intermediate image D1 and the intermediate image D2 according to the pixel values of the input image DIN. , to adjust the correction amount.

19 shows the magnification to be applied to the intermediate image D1 and D2 in the first intermediate image processing unit 3M and the second intermediate image processing unit 3H, or the magnification to be determined by the magnification determining unit 3MA. A map of D3MA and the amplification factor D3HA to be determined by the amplification factor determination unit 3HA. Preferably, the larger the pixel values of the input image DIN, the smaller these magnifications. For example, such a form can be considered: when the pixel value of the input image DIN is 0, a certain predetermined value Gb is taken, and gradually decreases with a slope k1 between the pixel value from 0 to a certain value A1, and when the pixel value changes from Between A1 and a certain value A2, it gradually decreases with a slope k2, and when the pixel value is above A1, it gradually decreases with a slope k3. In addition, since it is clear that the amplification factor is preferably 0 or more, when the amplification factor is negative in the above determination, the value is set to 0.

When denoting the magnification as G and the pixel value of the input image DIN as L, the relationship is expressed as:

[Equation 1]

where ...(1)

The reason why the above-mentioned magnification is appropriate is described below.

Intermediate image D1 is generated by performing low-pass filter processing on input image DIN after high-pass filter processing. Here, the high-pass filtering process corresponds to subtracting a local average value from each pixel value of the input image DIN. Therefore, when the pixel value of the pixel of interest in the input image DIN is large, there is a high possibility that the output value after the high-pass filtering process given to the pixel will also be a large positive value.

On the other hand, the low-pass filtering process is the same as calculating the local average value of the input data. Therefore, when the output value of the high-pass filter process is a large positive value, there is a high possibility that the output value of the low-pass filter process will also be a large positive value.

Furthermore, the intermediate image D2 is obtained by performing a high-pass filter process in the high-frequency component image generating unit 2B after performing nonlinear processing on the intermediate image D1 in the nonlinear processing unit 2A. Since the intermediate image D1 is enlarged only near the zero-crossing point in the nonlinear processing unit 2A, it is basically considered that when the intermediate image D1 has a large positive value, the image D2A output by the nonlinear processing unit 2A also has a large positive value . When image D2A has a large positive value, there is a high possibility that intermediate image D2 that is the result of the high-pass filtering process on image D2A also has a large positive value.

From the above, when the pixel value of the input image DIN is large, there is a high possibility that the pixel values of the intermediate image D1 and intermediate image D2 are also large positive values. In other words, if the correction amount is excessively added by intermediate image D1 and intermediate image D2 , overshooting tends to occur.

Therefore, as the pixel value of the input image DIN increases, the magnification factor applied to the intermediate image D1 and intermediate image D2 decreases, and it is expected that the correction amount can be controlled so that the correction amount does not increase excessively. In other words, it is expected that the overshoot can be controlled so that it is difficult to occur.

That is, by determining the magnification D3MA or the magnification D3HA according to a monotonically decreasing function such that the larger the pixel value of the input image DIN is, the smaller the magnification shown in FIG. Overshoot occurred) processing.

As described above, in the image processing device according to Embodiment 1, there is an effect that image enhancement processing can be performed while suppressing the occurrence of overshoot. When an overshoot occurs excessively in an image, only a part of the image appears abnormally, which is unpleasant in terms of visual characteristics. Therefore, the image processing device according to Embodiment 1 is also very preferable in terms of visual characteristics.

Furthermore, in the image processing device according to Embodiment 1, in order to suppress the occurrence of overshoot, the first intermediate image processing unit 3M and the second intermediate image processing unit 3H determine the magnification ratios for the intermediate image D1 and the intermediate image D2, however, The only information required at this time is the pixel value of the input image DIN. Therefore, the magnification can be determined with a simple circuit, and the increase in circuit size accompanying the addition of the first intermediate image processing unit 3M and the second intermediate image processing unit 3H is also reduced.

In addition, the relationship between the magnification factor determined in the first intermediate image processing unit 3M and the second intermediate image processing unit 3H and the pixel value of the input image DIN is not limited to the relationship described in this embodiment, as long as the pixel value of the input image DIN The larger the value, the smaller the magnification.

Embodiment 2

20 is a diagram showing a configuration example of an image processing device according to Embodiment 2 of the present invention.

The illustrated image processing device includes a first intermediate image generation unit 1 , a second intermediate image generation unit 2 , a first intermediate image processing unit 103M, a second intermediate image processing unit 103H, and an addition unit 4 . In addition, since the configurations and operations of the first intermediate image generation unit 1 , the second intermediate image generation unit 2 , and the addition unit 4 are the same as those in the first embodiment, description thereof will be omitted.

FIG. 21 is a diagram showing a configuration example of the first intermediate image processing unit 103M, and the illustrated first intermediate image processing unit 103M includes a magnification factor determining unit 103MA and a pixel value amplifying unit 103MB. The magnification factor determining unit 103MA determines the magnification factor D103MA based on the pixel values of the input image DIN and the intermediate image D1. Pixel value enlarging section 103MB amplifies the pixel values of intermediate image D1 using enlarging factor D103MA determined by enlarging factor determining section 103MA, and outputs the result as intermediate image D103MB. Intermediate image D103MB is output from first intermediate image processing unit 103M as intermediate image D103M.

The magnification factor determining unit 103MA has a horizontal magnification factor determining unit 103MAh and a vertical magnification factor determining unit 103MAv, and the pixel value amplifying unit 103MB has a horizontal pixel value magnifying unit 103MBh and a vertical pixel value magnifying unit 103MBv. The first horizontal intermediate image processing unit 103Mh is formed by the horizontal magnification determining unit 103MAh and the horizontal pixel value amplifying unit 103MBh, and the first vertical intermediate image is formed by the vertical magnifying factor determining unit 103MAv and the vertical pixel value amplifying unit 103MBv. Processing unit 103Mv.

FIG. 22 is a diagram showing a configuration example of the second intermediate image processing unit 103H, and the illustrated second intermediate image processing unit 103H includes a magnification factor determining unit 103HA and a pixel value magnifying unit 103HB. The magnification factor determining unit 103HA determines the magnification factor D103HA based on the pixel values of the input image DIN and the intermediate image D2. Pixel value enlarging section 103HB amplifies the pixel values of intermediate image D2 using enlarging factor D103HA determined by enlarging factor determining section 103HA, and outputs the result as intermediate image D103HB. Intermediate image D103HB is output from first intermediate image processing unit 103H as intermediate image D103H.

The magnification factor determining unit 103HA has a horizontal magnification factor determining unit 103HAh and a vertical magnification factor determining unit 103HAv, and the pixel value amplifying unit 103HB has a horizontal pixel value magnifying unit 103HBh and a vertical pixel value magnifying unit 103HBv. The second horizontal intermediate image processing unit 103Hh is constituted by the horizontal enlargement rate determining unit 103HAh and the horizontal pixel value enlarging unit 103HBh, and the second vertical intermediate image is constituted by the vertical enlargement rate determining unit 103HAv and the vertical pixel value enlarging unit 103HBv. Processing unit 103Hv.

First, the detailed operation of the first intermediate image processing unit 103M will be described.

In the first intermediate image processing unit 103M, the magnification factor D103MA is determined in the magnification factor determining unit 103MA based on the pixel values of the input image DIN and the intermediate image D1. As described above, the pixel values of the first intermediate image D1 are amplified by the magnification factor D103MA. However, since the first intermediate image D1 is composed of the image D1h and the image D1v, as the magnification factor D103MA, the magnification factor D103MAh for the image D1h and the magnification factor for the image D1h are determined. Magnification D103MAv of image D1v. That is, the magnification factor determining unit 103MA includes a horizontal magnification factor determining unit 103MAh and a vertical magnification factor determining unit 103MAv. In the horizontal magnification determining unit 103MAh, the magnification D103MAh is determined based on the pixel values of the input image DIN and the image D1h. Vertical magnification determining unit 103MAv determines magnification D103MAv based on the pixel values of input image DIN and image D1v, and magnification D103MAh and magnification D103MAv are output from magnification determining unit 103MA as magnification D103MA.

The operations of the horizontal magnification factor determining unit 103MAh and the vertical magnification factor determining unit 103MAv will be described in more detail.

In addition, the input image DIN and the images D1h and D1v are the same as those described with reference to FIG. 8 in the first embodiment.

The horizontal magnification determining unit 103MAh determines the magnification of each pixel of the image D1h based on the pixel values of the same coordinates of the input image DIN and the image D1h. That is, the magnification factor for pixel value D1h(11) is determined based on pixel value DIN(11) and pixel value D1h(11), and the magnification factor for pixel value D1h(12) is determined based on pixel value DIN(12) and pixel value D1h(12). In general, it is determined from the pixel values of the same coordinates of the input image DIN and the image D1h as the magnification ratio for the pixel value D1h(xy) is determined from the pixel value DIN(xy) and the pixel value D1h(xy). magnification, and output the result as magnification D103MAh.

Further, the vertical magnification determining unit 103MAv determines the magnification of each pixel of the image D1v based on the pixel values of the same coordinates of the input image DIN and the image D1v. That is, the magnification factor for pixel value D1v(11) is determined based on pixel value DIN(11) and pixel value D1v(11), and the amplification factor for pixel value D1v(12) is determined based on pixel value DIN(12) and pixel value D1v(12). In general, the magnification rate for the pixel value D1v(xy) is determined based on the pixel value of the same coordinates of the input image DIN and the image D1v as the magnification rate for the pixel value D1v(xy) is determined based on the pixel value DIN(xy) and the pixel value D1v(xy). magnification, and output the result as magnification D103MAv.

FIG. 23 is a diagram showing a configuration example of the horizontal magnification determining means 103MAh. The illustrated horizontal magnification determination unit 103MAh includes a first magnification determination unit 511h, a second magnification determination unit 512h, a sign determination unit 52h, and a selection unit 53h. The input image DIN and the image D1h are input to the horizontal magnification determining unit 103MAh. The first magnification factor determining means 511h outputs a magnification factor D511h determined by a first characteristic described later based on the pixel value of the input image DIN. The second magnification factor determining means 512h outputs a magnification factor D512h determined by a second characteristic described later based on the pixel value of the input image DIN. The sign determination unit 52h determines the sign (positive or negative) of the pixel value of the image D1h, and transmits the determination result to the selection unit 53h as a signal D52h. Based on the output D52h of the sign determination unit 52h, when the sign of the pixel value of the image D1h is positive, the selection unit 53h outputs the magnification D511h as the horizontal magnification D103MAh, and when the sign of the pixel value of the image D1h is negative, Output the amplification factor D512h as the horizontal direction amplification factor D103MAh.

FIG. 24 is a diagram showing a configuration example of the vertical magnification determining unit 103MAv. The illustrated vertical magnification determination unit 103MAv includes a first magnification determination unit 511v, a second magnification determination unit 512v, a sign determination unit 52v, and a selection unit 53v. The input image DIN and the image D1v are input to the vertical magnification determination unit 103MAv. The first magnification factor determining unit 511v outputs a magnification factor D511v determined by a first characteristic described later based on the pixel values of the input image DIN. The second magnification factor determining unit 512v outputs a magnification factor D512v determined by a second characteristic described later based on the pixel value of the input image DIN. The first magnification factor determining unit 511v and the second magnification factor determining unit 512v operate in the same way as the first magnification factor determining unit 511h and the second magnification factor determining unit 512h shown in FIG. The units are shared (the amplification factor determination units 511h and 512h are also used as the amplification factor determination units 511v and 512v). The sign determination unit 52v determines the sign (positive or negative) of the pixel value of the image D1v, and transmits the determination result to the selection unit 53v as a signal D52v. Based on the output D52v of the sign determination unit 52v, when the sign of the pixel value of the image D1v is positive, the selection unit 53v outputs the magnification D511v as the vertical magnification D103MAv, and when the sign of the pixel value of the image D1v is negative, Output magnification D512v as vertical magnification D103MAv.

Then, the pixel value enlarging unit 103MB amplifies the pixel values of the first intermediate image D1 according to the enlarging factor D103MA. Since the first intermediate image D1 is composed of an image D1h and an image D1v, the pixel value enlarging unit 103MB has: a horizontal pixel value enlarging unit 103MBh for enlarging the pixel value of the image D1h; and a vertical enlarging unit 103MBh for enlarging the pixel value of the image D1v. Directional pixel value amplification unit 103MBv. That is, the horizontal pixel value enlarging unit 103MBh outputs an image D103MBh in which the pixel values of the image D1h are enlarged by the magnification factor D103MAh, and the vertical pixel value enlarging unit 103MBv outputs an image D103MBv in which the pixel values of the image D1v are amplified by the magnification factor D103MAv. Then, the image D103MBh and the image D103MBv are output from the pixel value enlarging unit 103MB as the image D103MB.

Image D103MB is output from first intermediate image processing unit 103M as intermediate image D103M. The intermediate image D103M is composed of an image D103Mh corresponding to the image D103MBh and an image D103Mv corresponding to the image D103MBv.

The above is the operation of the first intermediate image processing unit 103M.

Next, the operation of the second intermediate image processing unit 103H will be described. Comparing Fig. 21 and Fig. 22, the second intermediate image processing unit has the same structure as the first intermediate image processing unit except that the input signal is the input image DIN and the intermediate image D2, and the second intermediate image processing unit 103H outputs a pair of intermediate image D2 Intermediate image D103H obtained by performing the same processing as that performed on intermediate image D1 by first intermediate image processing unit 103M. Since the detailed operation of the second intermediate image processing unit 103H is also clear from the detailed operation description of the first intermediate image processing unit 103M, the detailed operation description of the second intermediate image processing unit 103H is omitted.

The image processing device in this embodiment can also be used as a part of the image display device shown in FIG. 9 like the image processing device in Embodiment 1. Hereinafter, an example in which the image processing device in this embodiment is used as a part of the image display device will be described. Through this description, the functions and effects of the image processing device in this embodiment will also be clarified.

As described in Embodiment 1 with reference to (A) and (B) of FIG. The addition of high-frequency components sometimes leads to a decrease in image quality.

Therefore, in this embodiment, by changing the magnification applied to intermediate image D1 and intermediate image D2 according to the pixel value of input image DIN and the pixel value of intermediate image D1 or intermediate image D2, intermediate image D1 and intermediate image D2 The correction amount added or subtracted will not increase excessively, preventing the occurrence of overshoot and undershoot.

In the illustrated image processing device, in the first intermediate image processing unit 103M and the second intermediate image processing unit 103H, the sign (positive Negative) Appropriately apply different magnifications to the intermediate image D1 and intermediate image D2, and adjust the correction amount.

(A) of FIG. 25 shows, in order to prevent the occurrence of overshoot, in the first intermediate image processing unit 103M and the second intermediate image processing unit 103H, the magnification ratio to be applied to the intermediate image D1 and the intermediate image D2, or the magnification ratio to be applied to the intermediate image D1 and the intermediate image D2. It is a graph showing the characteristic (first characteristic) of the amplification factor D103MA to be determined by the factor determining unit 103MA and the factor D103HA to be determined by the factor determining unit 103HA. In order to prevent overshooting from occurring, these magnifications are preferably smaller the larger the pixel values of the input image DIN are. For example, such a form can be considered: when the pixel value of the input image DIN is 0, a certain predetermined value B1 is taken, and the pixel value gradually decreases from 0 to a certain value A11 with a slope k11, and when the pixel value changes from Between A11 and a certain value A12, it gradually decreases with a slope k12, and when the pixel value is greater than A11, it gradually decreases with a slope k13. In addition, since it is obvious that the amplification factor is 0 or more, when the amplification factor has a negative value in the above determination, its value is set to 0.

When denoting the magnification as G and the pixel value of the input image DIN as L, the relationship is expressed as:

[Equation 2]

where …(2)

In the first horizontal magnification determination unit 511h and the first vertical magnification determination unit 511v shown in FIG. 23 and FIG. Rate.

On the other hand, (B) of FIG. 25 shows the magnification to be applied to the intermediate image D1 and the intermediate image D2 in the first intermediate image processing unit 103M and the second intermediate image processing unit 103H in order to prevent undershoot. , or a graph of the characteristics (second characteristics) of the amplification factor D103MA to be determined by the amplification factor determination unit 103MA and the amplification factor D103HA to be determined by the amplification factor determination unit 103HA. In order to prevent undershooting, it is preferable that the smaller the pixel values of the input image DIN, the smaller these magnifications are, in other words, the larger the pixel values of the input image DIN, the larger these magnifications. For example, such a form can be considered: when the pixel value of the input image DIN is 0, a certain predetermined value B2 is taken, and gradually increases with a slope k21 between the pixel value from 0 to a certain value A21, and when the pixel value changes from A21 to It gradually increases with slope k22 until a certain value A22, and gradually increases with slope k23 when the pixel value is A22 or more.

When denoting the magnification as G and the pixel value of the input image DIN as L, the relationship is expressed as:

[Equation 3]

where …(3)

In the second horizontal magnification determination unit 512h and the second vertical magnification determination unit 512v shown in FIG. 23 and FIG. Rate.

The reason why the above-mentioned magnification is appropriate is described below.

Intermediate image D1 is generated by performing high-pass filter processing on input image DIN and then performing low-pass filter processing. Here, the high-pass filtering process corresponds to subtracting a local average value from each pixel value of the input image DIN. Therefore, when the pixel value of the pixel of interest in the input image DIN is large, there is a high possibility that the output value after the high-pass filtering process given to the pixel will also be a large positive value.

On the other hand, the low-pass filtering process is the same as calculating the local average value of the input data. Therefore, when the output value of the high-pass filter process is a large positive value, there is a high possibility that the output value of the low-pass filter process will also be a large positive value.

Furthermore, the intermediate image D2 is obtained by performing a high-pass filter process in the high-frequency component image generating unit 2B after performing nonlinear processing on the intermediate image D1 in the nonlinear processing unit 2A. Since the intermediate image D1 is enlarged only near the zero-crossing point in the nonlinear processing unit 2A, it is basically considered that when the intermediate image D1 has a large positive value, the image D2A output by the nonlinear processing unit 2A also has a large positive value . When image D2A has a large positive value, there is a high possibility that intermediate image D2 that is the result of the high-pass filtering process on image D2A also has a large positive value.

From the above, when the pixel value of the input image DIN is large, there is a high possibility that the pixel values of the intermediate image D1 and intermediate image D2 are also large positive values. In other words, if the correction amount is excessively added by intermediate image D1 and intermediate image D2 , overshooting tends to occur.

Conversely, when the pixel values of the input image DIN are small, the pixel values of the intermediate images D1 and D2 are likely to be large negative values. In other words, the correction amount is excessively subtracted with intermediate image D1 and intermediate image D2, and undershoot is likely to occur.

Therefore, when the pixel values of the intermediate image D1 and the intermediate image D2 are positive values, the larger the pixel value of the input image DIN is, the smaller the magnification applied to the intermediate image D1 and the intermediate image D2 is, and when the intermediate image D1 and When the pixel value of the intermediate image D2 is a negative value, the smaller the pixel value of the input image DIN is, the smaller the magnification applied to the intermediate image D1 and intermediate image D2 is, so that it can be expected to be controlled so that The subtracted correction amount does not increase excessively. In other words, it can be expected that overshoot and undershoot can be controlled so that it is difficult to occur.

That is, the sign of the intermediate image D1 or the intermediate image D2 is discriminated, and if the sign is positive, the larger the pixel value of the input image DIN, the larger the magnification factor, as shown in (A) of FIG. 25 or equation (2). If the magnification factor D103MA or magnification factor D103HA is determined by such a monotonically decreasing function, if the sign is negative, the smaller the pixel value of the input image DIN as shown in (B) of FIG. 25 or equation (3), then A monotonically increasing function such that the smaller the amplification factor is, determines the amplification factor D103MA or the amplification factor D103HA, so that such processing (which makes overshooting and undershooting difficult to occur) can be performed.

As described above, the image processing device according to Embodiment 2 has the effect that image enhancement processing can be performed while suppressing the occurrence of overshoot and undershoot. When excessive overshooting and undershooting occur in an image, only a part of the image appears abnormally, which is unpleasant in terms of visual characteristics. Therefore, the image processing device according to Embodiment 2 is also very preferable in terms of visual characteristics.

In addition, in the image processing device according to Embodiment 2, in order to suppress the occurrence of overshoot and undershoot, in the first intermediate image processing section 103M and the second intermediate image processing section 103H, the enlargement of intermediate image D1 and intermediate image D2 is determined. However, the only information required at this time is the sign of the pixel value of the input image DIN and the pixel value of the intermediate image D1 or intermediate image D2 (self image). Therefore, the magnification can be determined with a simple circuit, and the increase in circuit size accompanying the addition of the first intermediate image processing unit 103M and the second intermediate image processing unit 103H is also reduced.

In addition, the relationship between the magnification determined in the first intermediate image processing unit 103M and the second intermediate image processing unit 103H and the pixel value of the input image DIN is not limited to the relationship described in this embodiment, as long as it is used to prevent the occurrence of overshoot The first characteristic is that the larger the pixel value of the input image DIN is, the smaller the magnification factor is, and the second characteristic for preventing undershoot is that the smaller the pixel value of the input image DIN is, the smaller the magnification factor is. And, as long as the above characteristics are satisfied, the formula (2), the formula (2) and the formula can be changed between the horizontal magnification determination unit 103MAh, the vertical magnification determination unit 103MAv, the horizontal magnification determination unit 103HAh, and the vertical magnification determination unit 103HAh. The coefficients of (3) and the functional form itself.

Furthermore, the horizontal magnification determining means 103MAh is not limited to the horizontal magnifying factor determining means shown in FIG. 23 , and for example, a horizontal magnifying factor determining means configured as shown in FIG. 26 may be used. Horizontal magnification determining unit 103MAh shown in FIG. 26 includes sign determining unit 52h, coefficient determining unit 54h, and magnification determining unit 55h. The sign determination unit 52h determines the sign (positive or negative) of the pixel value of the image D1h, and transmits the determination result to the coefficient determination unit 54h as a signal D52h. The coefficient determining unit 54h determines a predetermined coefficient group D54h (k1, k2, k3, A1, A2, B ).

[Equation 4]

k1=-k11

k2=-k12

k3=-k13 ...(4)

A1=A11

A2=A12

B=B1

And when the sign of the pixel value of image D1h is negative, predetermined coefficient group D54h (k1, k2, k3, A1, A2, B) is determined according to the following formula (5).

[Equation 5]

k1=k21

k2=k22

k3=k23 ...(5)

A1=A21

A2=A22

B=B2

The magnification factor determination unit 55 calculates the horizontal magnification factor D103MAh using the following equation (6) from the coefficient group D54h.

[Equation 6]

where …(6)

Substituting the coefficients, it can be seen that when the sign of the pixel value of image D1h is positive, formula (6) is equivalent to formula (2), and when the sign of the pixel value of image D1h is negative, formula (6) and Equation (3) is equivalent. Therefore, the output horizontal magnification D103MAh is the same as that of the configuration explained in FIG. 23 . On the other hand, by adopting the structure shown in FIG. 26, the calculation in the amplification factor determining unit 55h can be described using a single relational expression, and the characteristic can be switched only by changing the coefficient according to the sign of the image D1h, thereby reducing the circuit scale of the calculation. Furthermore, the same configuration can be employed for the vertical magnification determining means 103MAv, the horizontal magnification determining means 103HAh, and the vertical magnifying factor determining means 103HAv, and the same effect can be obtained.

In addition, in this embodiment, an example is shown in which the magnification factor is calculated by calculation based on the pixel value of each pixel of the input image DIN. The value of the magnification. In the case of using such a LUT, there is no need to perform the calculation of formula (2), formula (3) or formula (6), so the horizontal direction amplification ratio determination unit 103MAh, the vertical direction amplification ratio determination unit 103MAv, the horizontal direction amplification ratio determination unit 103MAh, and the horizontal direction amplification ratio can be realized. The processing in the ratio determination unit 103HAh and the vertical amplification ratio determination unit 103HAh is simplified.

In addition, in the above-mentioned embodiment, as shown in FIGS. 23 and 24 , both of the first horizontal magnification determining unit 103MAh and the first vertical magnification determining unit 103MAv of the first intermediate image processing unit 103M have: 1 Amplification ratio determining means (511h, 511v) which output the amplification ratio according to the first characteristic that the larger the pixel value of the input image is, the smaller the amplification ratio is; The smaller the pixel value, the smaller the second characteristic output magnification; the sign determination unit (52h, 52v), which determines the sign of the pixel value of the first intermediate image D1; and the selection unit (53h, 53v), It selects and outputs the amplification factor output by the first amplification factor determination unit (511h, 511v) and the amplification factor output by the second amplification factor determination unit (512h, 512v) according to the determination result of the sign determination unit, however, only Either one of the first horizontal magnification determining unit 103MAh and the first vertical magnification determining unit 103MAv has the above-mentioned structure, and the other has a different structure.

26, both of the first horizontal magnification determination unit 103MAh and the first vertical magnification determination unit 103MAv may include a sign determination unit (52h) that determines the sign of the pixel value of the first intermediate image. a coefficient determining unit (54h), which outputs a predetermined coefficient according to the determination result of the sign determining unit (52h); and an amplification factor determining unit (55h), which uses the pixel value of the input image and the coefficient output by the coefficient determining unit (52h) to determine To determine the magnification, however, only one of the first horizontal magnification determining unit 103MAh and the first vertical magnification determining unit 103MAv may have the above structure, and the other may have a different structure.

The same applies to the second horizontal magnification determining unit 103HAh and the second vertical magnification determining unit 103HAv of the second intermediate image processing unit 103H.

Embodiment 3

27 is a diagram showing a configuration example of an image processing device according to Embodiment 3 of the present invention.

An input image IMGIN is input to the illustrated image processing device, and the illustrated image processing device outputs an output image IMGOUT. The input image IMGIN is a color image and is composed of a signal YIN representing a luminance component (hereinafter referred to as an input luminance image YIN) and signals CRIN and CBIN representing color difference components. A signal CRIN (hereinafter referred to as an input CR image CRIN) represents a Cr component among the color difference components, and a signal CBIN (hereinafter referred to as an input CB image CBIN) represents a Cb component among the color difference components. The output image IMGOUT is also a color image, and is composed of a signal YOUT representing a luminance component (hereinafter referred to as output luminance image YOUT) and signals CROUT and CBOUT representing color difference components. The signal CROUT (hereinafter referred to as the output CR image CROUT) represents the Cr component of the color difference components, and the signal CBOUT (hereinafter referred to as the output CB image CBOUT) represents the Cb component of the color difference components.

The image processing device according to Embodiment 3 includes a first intermediate image generation unit 201, a second intermediate image generation unit 2, a luminance and color difference addition unit 205, a first intermediate image post-processing unit 203M, a second intermediate image post-processing unit 203H, and an addition Unit 204.

The first intermediate image generating section 201 generates an intermediate image (first intermediate image) obtained by extracting a specific frequency band component (that is, a component from a first frequency (first predetermined frequency) to a second frequency (second predetermined frequency)) from the input luminance image YIN. middle image) D1.

The second intermediate image generating unit 2 generates an intermediate image (second intermediate image) D2 obtained by performing processing described later on the intermediate image D1.

The luminance and color difference addition section 205 generates a luminance and color difference added image YC obtained by performing weighted addition of the input luminance image YIN, the input CR image CRIN, and the input CB image CBIN as will be described later.

The first intermediate image post-processing unit 203M generates an intermediate image (third intermediate image) D203M obtained by processing the intermediate image D1 to be described later.

The second intermediate image post-processing unit 203H generates an intermediate image (fourth intermediate image) D203H obtained by performing processing described later on the intermediate image D2.

The adding unit 204 adds the input luminance image YIN, the intermediate image D203M, and the intermediate image D203H.

In the image processing device of the illustrated embodiment, only the luminance component is processed. That is, the output image of the adding unit 204 is output as the output luminance image YOUT, while the input CR image CRIN is directly output as the output CR image CROUT, and the input CB image CBIN is directly output as the output CB image CBOUT.

The first intermediate image generating section 201 performs the same processing as that performed on the input image DIN by the first intermediate image generating section 1 in Embodiment 1 on the input luminance image YIN. Therefore, its configuration may be the same as that of the first intermediate image generation unit 1 in the first embodiment.

The operation and configuration of the second intermediate image generation unit 2 are the same as those of the first intermediate image generation unit 2 in Embodiment 1, and thus description thereof will be omitted.

FIG. 28 is a diagram showing a configuration example of the first intermediate image processing unit 203M, and the illustrated first intermediate image processing unit 203M includes a magnification factor determining unit 203MA and a pixel value magnifying unit 203MB. The magnification determining unit 203MA determines the magnification D203MA based on the pixel values of the luminance and color difference added image YC described later.

Pixel value enlarging section 203MB amplifies the pixel values of intermediate image D1 using enlarging factor D203MA determined by enlarging factor determining section 203MA, and outputs the result as intermediate image D203MB. Then, intermediate image D203MB is output from first intermediate image processing unit 203M as intermediate image D203M.

The magnification determining unit 203MA has a horizontal magnifying factor determining unit 203MAh and a vertical magnifying factor determining unit 203MAv, and the pixel value enlarging unit 203MB has a horizontal pixel value enlarging unit 203MBh and a vertical pixel value enlarging unit 203MBv. The first horizontal intermediate image processing unit 203Mh is formed by the horizontal magnification determining unit 203MAh and the horizontal pixel value amplifying unit 203MBh, and the first vertical intermediate image is formed by the vertical magnifying factor determining unit 203MAv and the vertical pixel value amplifying unit 203MBv. Processing unit 203Mv.

FIG. 29 is a diagram showing a configuration example of the second intermediate image processing unit 203H, and the illustrated second intermediate image processing unit 203H includes a magnification factor determining unit 203HA and a pixel value magnifying unit 203HB. The magnification factor determining unit 203HA determines the magnification factor D203HA based on the pixel values of the luminance and color difference added image YC described later.

Pixel value enlarging section 203HB amplifies the pixel values of intermediate image D2 using enlarging factor D203HA determined by enlarging factor determining section 203HA, and outputs the result as intermediate image D203HB. Then, intermediate image D203HB is output from first intermediate image processing unit 203H as intermediate image D203H.

The magnification factor determining unit 203HA has a horizontal magnification factor determining unit 203HAh and a vertical magnification factor determining unit 203HAv, and the pixel value amplifying unit 203HB has a horizontal pixel value magnifying unit 203HBh and a vertical pixel value magnifying unit 203HBv. The second horizontal intermediate image processing unit 203Hh is constituted by the horizontal enlargement factor determining unit 203HAh and the horizontal pixel value enlarging unit 203HBh, and the second vertical intermediate image is formed by the vertical enlargement factor determining unit 203HAv and the vertical pixel value enlarging unit 203HBv. Processing unit 203Hv.

Addition unit 204 adds intermediate image D203M and intermediate image D203H to input luminance image YIN to generate output luminance image YOUT.

Hereinafter, detailed operations of the image processing device according to Embodiment 3 will be described. However, as described above, the operations of the first intermediate image generating unit 201 and the second intermediate image generating unit 2 are the same as those in the first embodiment, and therefore description thereof will be omitted.

First, the detailed operation of luminance and color difference adding section 205 will be described. The luminance and color difference adding section 205 performs weighted addition of the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN for each pixel (only by weighted addition). , to generate the luminance and color difference addition image YC. That is, using the input luminance image YIN, the input CR image CRIN, and the input CB image CBIN, the luminance-color-difference added image YC is obtained according to the following equation.

YC＝Ky·YIN+Kcr·|CRIN|+Kcb·|CBIN|...(7)

In the formula, Ky, Kcr, and Kcb are coefficients used for weighting.

Next, detailed operations of the first intermediate image processing unit 203M will be described.

In the first intermediate image processing unit 203M, in the magnification factor determining unit 203MA, the magnification factor D203MA is determined based on the pixel values of the luminance and color difference added image YC. As described above, the pixel values of the first intermediate image D1 are amplified by the magnification factor D203MA. However, since the first intermediate image D1 is composed of the image D1h and the image D1v, the magnification factor D203MAh for the image D1h and the magnification factor for the image D1h are determined as the magnification factor D203MA. Magnification D203MAv of image D1v. That is, the magnification factor determining unit 203MA has a horizontal magnification factor determining unit 203MAh and a vertical magnification factor determining unit 203MAv. In the horizontal magnification factor determining unit 203MAh, the magnification factor D203MAh is determined based on the pixel values of the luminance and color difference added image YC. The vertical magnification determination unit 203MAv determines the magnification D203MAv based on the pixel values of the luminance and color difference added image YC, and outputs the magnification D203MAh and the magnification D203MAv from the magnification determination unit 203MA as the magnification D203MA.

The operations of the horizontal magnification factor determining unit 203MAh and the vertical magnification factor determining unit 203MAv will be described in more detail.

(A) to (C) of FIG. 30 are diagrams showing the luminance and color difference added image YC and images D1h and D1v, FIG. 30(A) shows the luminance and color difference added image YC, and FIG. 30(B) shows the image D1h, (C) of FIG. 30 shows an image D1v. In addition, in (A) to (C) of FIG. 30 , horizontal coordinates, vertical coordinates, and respective coordinate values are shown for the horizontal direction and the vertical direction of the image. In addition, for the luminance and color difference addition image YC, the pixel value of the pixel at the position of the horizontal coordinate x and the vertical coordinate y is represented by a symbol such as YC(xy), and for the image D1h, at the position of the horizontal coordinate x and the vertical coordinate y The pixel value of a pixel above is represented by a symbol D1h(xy), and for the image D1v, the pixel value of a pixel at a position of horizontal coordinate x and vertical coordinate y is represented by a symbol D1v(xy).

The horizontal magnification determining unit 203MAh determines the magnification for each pixel of the image D1h based on the pixel values at the same coordinates of the luminance-color-difference added image YC. That is, the magnification factor for the pixel value D1h(11) is determined based on the pixel value YC(11), and the magnification factor for the pixel value D1h(12) is determined based on the pixel value YC(12). Generally speaking, for example, based on the pixel value YC( xy) determines the magnification factor for the pixel value D1h(xy), the magnification factor is determined from the pixel value at the same coordinates of the luminance-chromatic-difference addition image YC, and the result is output as the magnification factor D203MAh.

Further, the vertical magnification determining unit 203MAv determines the magnification of each pixel of the image D1v based on the pixel values at the same coordinates of the luminance-chromatic-difference addition image YC. That is, the magnification factor for the pixel value D1v(11) is determined according to the pixel value YC(11), and the magnification factor for the pixel value D1v(12) is determined according to the pixel value YC(12). Generally speaking, for example, according to the pixel value YC( xy) determines the magnification factor for the pixel value D1v(xy), the magnification factor is determined from the pixel value at the same coordinates of the luminance-chromatic-difference addition image YC, and the result is output as the magnification factor D203MAv.

Then, the pixel value enlarging unit 203MB amplifies the pixel values of the first intermediate image D1 according to the enlarging factor D203MA. Since the first intermediate image D1 is composed of an image D1h and an image D1v, the pixel value enlarging unit 203MB has: a horizontal pixel value enlarging unit 203MBh for enlarging the pixel value of the image D1h; and a vertical enlarging unit 203MBh for enlarging the pixel value of the image D1v. Directional pixel value amplification unit 203MBv. That is, the horizontal pixel value enlarging unit 203MBh outputs an image D203MBh in which the pixel values of the image D1h are enlarged by the magnification factor D203MAh, and the vertical pixel value enlarging unit 203MBv outputs an image D203MBv in which the pixel values of the image D1v are amplified by the magnification factor D203MAv. Then, an image D203MBh and an image D203MBv are output from the pixel value enlarging unit 203MB as an image D203MB.

Image D203MB is output from first intermediate image processing unit 203M as intermediate image D203M. Intermediate image D203M is composed of image D203Mh corresponding to image D203MBh and image D203Mv corresponding to image D203MBv.

The above is the operation of the first intermediate image processing unit 203M.

Next, the operation of the second intermediate image processing unit 203H will be described. Comparing Fig. 28 with Fig. 29, the structure of the second intermediate image processing unit is the same as that of the first intermediate image processing unit except that the input signal is the luminance and color difference addition image YC and the intermediate image D2, and the output of the second intermediate image processing unit 203H is The intermediate image D2 is an intermediate image D203H obtained by performing the same processing as that performed on the intermediate image D1 by the first intermediate image processing unit 203M. Since the detailed operation of the second intermediate image processing unit 203H is also clear from the detailed operation description of the first intermediate image processing unit 203M, the detailed operation description of the second intermediate image processing unit 203H is omitted.

Finally, the operation of adding unit 204 will be described. Addition unit 204 generates output luminance image YOUT obtained by adding input luminance image YIN, intermediate image D203M, and intermediate image D203H. The output image IMGOUT of the adding unit 204, which is composed of the output luminance image YOUT, the output CR image CROUT, and the output CB image CBOUT, is output from the image processing device as a final output image.

The intermediate image D203M is composed of the image D203Mh and the image D203Mv, and the intermediate image D203H is composed of the image D203Hh and the image D203Hv, so adding the input luminance image YIN, the intermediate image D203M, and the intermediate image D203H means adding all the images D203Mh, D203Mv, D203Hh, and D203Hv Added to the input luminance image YIN.

Here, the addition processing in the addition unit 204 is not limited to simple addition, and weighted addition may be performed. That is, each of the images D203Mh, D203Mv, D203Hh, and D203Hv may be enlarged at different magnification factors and added to the input luminance image YIN.

Hereinafter, an example in which the image processing device in this embodiment is used as a part of the image display device will be described. The operation and effects of the image processing device in this embodiment will also be clarified through this description.

FIG. 31 shows an image display device using the image processing device in this embodiment. In the illustrated image display device, an image corresponding to the original image IMGORG is displayed on the monitor U203.

When the image size of the original image IMGORG is smaller than the image size of the monitor U203, the color image enlarging unit U201 outputs an image IMGU201 in which the original image IMGORG is enlarged. Here, the enlargement of the image can be performed using, for example, a bicubic method or the like.

The image processing device U202 in this embodiment uses the image IMGU201 as an input image IMGIN, and outputs an image DU202 obtained by performing the processing described above on the input image IMGIN. Then, the image DU202 is displayed on the monitor U203.

In addition, since the image DU202 is divided into a luminance signal (Y) and a color difference signal (Cr, Cb) (hereinafter sometimes referred to as a YCbCr format), it is usually converted into red (R) green ( G) Color signal of blue (B) (hereinafter sometimes referred to as RGB format). The conversion between the YCbCr format and the RGB format is described, for example, in Recommendation ITU-R.BT601 of the International Telecommunication Union, and the conversion from the RGB format to the YCbCr format is performed according to the following formula.

Y＝0.299R+0.587G+0.114B

Cr＝0.500R-0.419G-0.081B

Cb=-0.169R-0.331G+0.500B

…(8)

The conversion from the YCbCr form to the RGB form is performed according to the following formula.

R=1.000Y+1.402Cr+0.000Cb

G=1.000Y-0.714Cr-0.344Cb

B=1.000Y+0.000Cr+1.772Cb

…(9)

In addition, the coefficient shown by Formula (8) and Formula (9) is an example, and this embodiment is not limited to this. In addition, when the input image is 8-bit data, the values of Cr and Cb are usually rounded to the range of -127 or more and 128 or less.

Hereinafter, assuming that the pixel values of the original image IMGORG are half of the pixel values of the monitor U203 in the horizontal and vertical directions, the operation and function of the color image enlarging unit U201 will be described first.

The color image enlargement unit U201 has: an image enlargement unit U201Y that generates an image DU201Y that enlarges an image YORG representing a luminance component of an original image IMGORG; an image enlargement unit U201CR that generates an image DU201CR that enlarges an image CRORG representing a Cr component; The image enlarging unit U201CB generates an image DU201CB in which the image CBORG representing the Cb component is enlarged. The configuration and operation of image enlargement unit U201Y, image enlargement unit U201CR, and image enlargement unit U201CB may be the same as those of image enlargement unit U1 described with reference to FIG. 10 in Embodiment 1, and thus detailed description thereof will be omitted.

As described for Embodiment 1, image enhancement processing can be performed by adding intermediate image D1 and intermediate image D2 to input image DIN.

In this embodiment, instead of adding the first intermediate image D1 and the second intermediate image D2 to the input image YIN, the effect obtained when the first intermediate image and the second intermediate image are added is assumed to be described below. Next, the effect of adding the third intermediate image D203M and the fourth intermediate image D203H instead of the first intermediate image D1 and the second intermediate image D2 will be described.

First, the effect of adding the intermediate image D1 will be described. From the comparison with Embodiment 1, it can be seen that the intermediate image D1 is obtained by extracting superimposed components from the high-frequency components of the input brightness image YIN, as shown in (E) of Figure 32, corresponding to the Nygui of the original image DORG High-frequency components around the Stern frequency. As explained in (D) of FIG. 32 , the spectral intensity near the Nyquist frequency of the original image DORG is weakened by the enlargement process in the image enlargement unit U201. Spectral intensity attenuated by processing. Also, since the superimposed component is removed from the intermediate image D1, artifacts such as overshoot, jaggies, or ringing are not enhanced.

Next, the effect of adding the intermediate image D2 will be described. It can be seen from the comparison with Embodiment 1 that the intermediate image D2 is a high-frequency component corresponding to the sampling interval S1. Therefore, by adding the intermediate image D2, high-frequency components in the frequency band above the Nyquist frequency of the original image DORG can be given, thereby increasing the sharpness of the image.

To summarize, by adding intermediate image D1 and intermediate image D2 to input luminance image YIN, high frequency components can be added without enhancing superimposed components. In addition, in terms of human visual characteristics, information obtained from the luminance component is dominant for the sharpness of an image, so the above-mentioned processing can improve the sharpness of an image.

In addition, by adding the generated high-frequency components to the input luminance image YIN as described above, the sharpness of the image can be increased, and the image quality can be improved. However, excessive addition of high-frequency components may cause deterioration of the image quality. decline. That is, as can be seen from the comparison with Embodiment 1, when the input luminance image YIN has a step-edge luminance change, overshoot and undershoot may occur.

In particular, when overshoot occurs in the input luminance image YIN, the luminance signal increases excessively. It can be known from formula (9) that when the value of the luminance signal (Y) increases and the value of the luminance signal (Y) is converted to the RGB format, the first item on the right side of each formula representing R, G, and B increases. Therefore, R, G, B is a large value.

R, G, and B are all large values, which means that they are close to white. In other words, approaching white means that the color becomes lighter. Originally, even if the color is lightened in the near-achromatic part, it is relatively inconspicuous. However, when the color becomes light near the colored edge, only the color around the edge becomes light, giving an unnatural feeling.

In other words, when the magnitude of the luminance added by intermediate image D1 and intermediate image D2 (hereinafter referred to as correction amount) is excessively increased in a colored portion, it is considered that these image quality degradations tend to occur. Therefore, it is considered that it is only necessary to adjust so that the correction amount of the intermediate image D1 and the intermediate image D2 does not increase excessively in the colored portion.

As this method, for example, it is considered that there is a method of detecting a colored portion where the correction amount provided by intermediate image D1 and intermediate image D2 increases, and at the detected portion, by appropriately applying a gain to make intermediate image D1 and intermediate image The correction amount of D2 is reduced so that the correction amount does not increase excessively.

Whether it is colored can be judged according to the chroma (which can be expressed by the square root of the square sum of Cr and Cb). That is, for a chromatic color, the saturation is a large value. Also, the square root of the sum of the squares of Cr and Cb can be approximated using the sum of the absolute values of Cr and Cb. This is because, when the absolute value of Cr or Cb increases, the square root of the sum of the squares of Cr and Cb also increases. Also, since calculating the sum of absolute values is simpler than calculating the square root of the sum of squares, the circuit scale can also be reduced.

Whether or not the amount of correction provided by the intermediate image D1 and intermediate image D2 has increased can be determined to some extent from the pixel values of the input luminance image YIN. The reason is described below.

The intermediate image D1 is generated by performing high-pass filter processing on the input luminance image YIN and then performing low-pass filter processing. Here, the high-pass filtering process corresponds to subtracting a local average value from each pixel value of the input luminance image YIN. Therefore, when the pixel value of the pixel of interest in the input luminance image YIN is large, there is a high possibility that the output value after the high-pass filtering process given to the pixel will also be a large positive value.

On the other hand, the low-pass filtering process is the same as calculating the local average value of the input data. Therefore, when the output value of the high-pass filter process is a large positive value, there is a high possibility that the output value of the low-pass filter process will also be a large positive value.

Furthermore, the intermediate image D2 is obtained by performing a high-pass filter process in the high-frequency component image generating unit 2B after performing nonlinear processing on the intermediate image D1 in the nonlinear processing unit 2A. Since the intermediate image D1 is enlarged only near the zero-crossing point in the nonlinear processing unit 2A, it is basically considered that when the intermediate image D1 has a large positive value, the image D2A output by the nonlinear processing unit 2A also has a large positive value . When image D2A has a large positive value, there is a high possibility that intermediate image D2 that is the result of the high-pass filtering process on image D2A also has a large positive value.

As described above, when the pixel value of the input luminance image YIN is large, there is a high possibility that the pixel values of the intermediate image D1 and intermediate image D2 are also large positive values. In other words, when the pixel value of the input luminance image YIN is large, it can be determined that the correction amount provided by the intermediate image D1 and the intermediate image D2 is increased to some extent.

From the above reasons, it is considered that the correction amount increases for a pixel in which any one of the absolute value of the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN is larger, and the correction amount becomes larger. There is a disadvantage that the color fades when changing to RGB form.

In addition, when any one of the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN is large, the weighted addition value is also a large value. .

Therefore, in this embodiment, the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN are weighted and added according to the formula (7) to generate the luminance The color difference addition image YC, the larger the pixel value of the brightness color difference addition image YC, the smaller the magnification applied to the intermediate image D1 and intermediate image D2, thereby avoiding the unfavorable situation of color fading near the colored edges.

The relationship between such a magnification factor (GAIN) and the pixel value (L) of the luminance-chromatic-difference added image YC is expressed as follows, for example:

[Equation 7]

where ...(10)

FIG. 33 is a graph showing the relationship between the magnification (GAIN) represented by the equation (10) and the pixel value (L) of the luminance-color-difference added image YC. When the pixel value of the luminance and color difference added image YC is 0, a certain predetermined value B is taken, and gradually decreases with a slope k1 between the pixel value from 0 to a certain value A1, and when the pixel value is from A1 to a certain value A2 gradually decreases with a slope k2, and gradually decreases with a slope k3 when the pixel value is greater than or equal to A1.

In short, the magnification D203MA or the magnification D203HA is determined according to a monotonically decreasing function such that the larger the pixel value of the luminance-chromatic-difference added image YC is, the smaller the magnification is, as shown in FIG. 33 or Equation (10). A treatment that prevents the adverse condition of color fading near colored edges.

As described above, in the image processing device according to Embodiment 3, it is possible to perform image enhancement processing while suppressing the occurrence of disadvantages such as color fading near colored edges. When the color fades near a colored edge, it feels unnatural in terms of visual properties, so the image processing device according to Embodiment 3 is also very preferable in terms of visual properties.

In addition, in the image processing device according to Embodiment 3, in order to suppress the disadvantage that the color fades near the colored edge, in the first intermediate image processing unit 203M and the second intermediate image processing unit 203H, it is determined that for the intermediate image The magnification ratios of D1 and intermediate image D2, however, the only information required at this time is the weighted phase of the pixel values of the input luminance image YIN, the absolute values of the pixel values of the input CR image CRIN, and the absolute values of the pixel values of the input CB image CBIN value added. Therefore, the amplification factor can be determined in a simple circuit, and an increase in circuit scale can also be reduced.

Furthermore, the relationship between the magnification determined in the first intermediate image processing section 203M and the second intermediate image processing section 203H and the pixel value of the luminance-chromatic-difference added image YC is not limited to the relationship described in this embodiment, as long as the luminance-chromatic difference is similar The larger the pixel value of the added image YC, the smaller the magnification ratio.

Embodiment 4

34 is a diagram showing a configuration example of an image processing device according to Embodiment 4 of the present invention.

As described for the third embodiment, the input image IMGIN is input to the illustrated image processing device, and the illustrated image processing device outputs the output image IMGOUT. The input image IMGIN is a color image composed of an input luminance image YIN, an input CR image CRIN, and an input CB image CBIN. The output image IMGOUT is also a color image, and is composed of an output luminance image YOUT, an output CR image CROUT, and an output CB image CBOUT.

The image processing device according to Embodiment 4 includes a first intermediate image generation unit 201 , a second intermediate image generation unit 2 , a luminance and color difference addition unit 205 , a first intermediate image processing unit 303M, a second intermediate image processing unit 303H, and an addition unit 204 .

The configurations and operations of first intermediate image generating section 201 , second intermediate image generating section 2 , luminance and color difference adding section 205 , and adding section 204 are the same as those of the same reference numerals in Embodiment 3, respectively.

The first intermediate image processing unit 303M generates an intermediate image (third intermediate image) D303M obtained by processing the intermediate image D1 to be described later.

The second intermediate image processing unit 303H generates an intermediate image (fourth intermediate image) D303H obtained by processing the intermediate image D2 to be described later.

In the image processing device of the illustrated embodiment, only the luminance component is processed. That is, the output image of the adding unit 204 is output as the output luminance image YOUT, while the input CR image CRIN is directly output as the output CR image CROUT, and the input CB image CBIN is directly output as the output CB image CBOUT.

FIG. 35 is a diagram showing a configuration example of the first intermediate image processing unit 303M, and the illustrated first intermediate image processing unit 303M includes a magnification factor determining unit 303MA and a pixel value magnifying unit 303MB. The magnification determining unit 303MA determines the magnification D303MA based on the pixel values of the luminance and color difference added image YC described later. Pixel value enlarging section 303MB amplifies the pixel values of intermediate image D1 using enlarging factor D303MA determined by enlarging factor determining section 303MA, and outputs the result as intermediate image D303MB. Then, intermediate image D303MB is output from first intermediate image processing unit 303M as intermediate image D303M.

The magnification determination unit 303MA has a horizontal magnification determination unit 303MAh and a vertical magnification determination unit 303MAv, and the pixel value amplifying unit 303MB has a horizontal pixel value amplifying unit 303MBh and a vertical pixel value amplifying unit 303MBv. The first horizontal intermediate image processing unit 303Mh is composed of the horizontal magnification factor determining unit 303MAh and the horizontal pixel value enlarging unit 303MBh, and the first vertical intermediate image is formed by the vertical magnification factor determining unit 303MAv and the vertical pixel value enlarging unit 303MBv. Processing unit 303Mv.

FIG. 36 is a diagram showing a configuration example of the second intermediate image processing unit 303H, and the illustrated second intermediate image processing unit 303H includes an enlargement factor determining unit 303HA and a pixel value enlarging unit 303HB. The magnification determining means 303HA determines the magnification D303HA based on the pixel values of the luminance and color difference added image YC described later.

Pixel value enlarging section 303HB amplifies the pixel values of intermediate image D2 using enlarging factor D303HA determined by enlarging factor determining section 303HA, and outputs the result as intermediate image D303HB. Then, intermediate image D303HB is output from first intermediate image processing unit 303H as intermediate image D303H.

The magnification determination unit 303HA has a horizontal magnification determination unit 303HAh and a vertical magnification determination unit 303HAv, and the pixel value amplifying unit 303HB has a horizontal pixel value amplifying unit 303HBh and a vertical pixel value amplifying unit 303HBv. The second horizontal intermediate image processing unit 303Hh is constituted by the horizontal enlargement ratio determination unit 303HAh and the horizontal pixel value enlargement unit 303HBh, and the second vertical intermediate image is constituted by the vertical enlargement ratio determination unit 303HAv and the vertical pixel value enlargement unit 303HBv. Processing unit 303Hv.

Hereinafter, detailed operations of the image processing device according to Embodiment 4 will be described.

Note that the detailed operations of first intermediate image generation section 201 , second intermediate image generation section 2 , luminance and color difference addition section 205 , and addition section 204 are the same as those described in Embodiment 3, and thus description thereof will be omitted.

First, the detailed operation of the first intermediate image processing unit 303M will be described.

In the first intermediate image processing unit 303M, in the magnification factor determining unit 303MA, the magnification factor D303MA is determined based on the pixel value of the luminance and color difference added image YC and the symbol sD1 of the pixel value of the intermediate image D1. Here, since the first intermediate image D1 is composed of the image D1h and the image D1v, the symbol sD1 is composed of the symbol sD1h representing the symbol of the image D1h and the symbol sD1v representing the symbol of the image D1v. As described above, the pixel values of the first intermediate image D1 are amplified by the magnification factor D303MA. However, since the first intermediate image D1 is composed of the image D1h and the image D1v, the magnification factor D303MAh for the image D1h and the magnification factor for the image D1h are determined as the magnification factor D303MA. Magnification D303MAv of image D1v. That is, the magnification determination unit 303MA has a horizontal magnification determination unit 303MAh and a vertical magnification determination unit 303MAv. In the horizontal magnification determination unit 303MAh, the pixel value of the image YC and the pixel value of the image D1h are added according to the luminance and color difference. The magnification factor D303MAh is determined by the sign sD1h of the magnification factor. In the vertical direction magnification factor determination unit 303MAv, the magnification factor D303MAv is determined based on the sign sD1v of the pixel value of the luminance and color difference added image YC and the pixel value of the image D1v. The magnification factor D303MAv is output from the magnification factor determination unit 303MA. The ratio D303MAh and the amplification ratio D303MAv are used as the amplification ratio D303MA.

The operations of the horizontal magnification determination unit 303MAh and the vertical magnification determination unit 303MAv will be described in more detail.

(A) to (E) of FIG. 37 are diagrams showing the luminance and color difference added image YC and images D1h and D1v, the symbol sD1h of the pixel value of the image D1h, and the symbol sD1v of the pixel value of the image D1v. (A) of FIG. 37 37(B) shows image D1h, FIG. 37(C) shows image D1v, FIG. 37(D) shows symbol sD1h, and FIG. 37(E) shows symbol sD1v. In addition, in (A) to (E) of FIG. 37 , horizontal coordinates, vertical coordinates, and respective coordinate values are shown for the horizontal direction and the vertical direction of the image. In addition, for the luminance and color difference addition image YC, the pixel value of the pixel at the position of the horizontal coordinate x and the vertical coordinate y is represented by a symbol such as YC(xy), and for the image D1h, at the position of the horizontal coordinate x and the vertical coordinate y The pixel value of a pixel above is represented by a symbol D1h(xy), and for the image D1v, the pixel value of a pixel at a position of horizontal coordinate x and vertical coordinate y is represented by a symbol D1v(xy). For the symbol sD1h of the pixel value of the image D1h, the symbol of the pixel at the position of the horizontal coordinate x and the vertical coordinate y is represented by a symbol such as sD1h(xy), and for the symbol sD1v of the pixel value of the image D1v, at the horizontal coordinate x, The sign of the pixel at the position of the vertical coordinate v is represented by a sign such as sD1v(xy).

The horizontal magnification determining unit 303MAh determines the magnification for each pixel of the image D1h based on the signs of the pixel values at the same coordinates in the luminance-chromatic-difference added image YC and the pixel values at the same coordinates in the image D1h. That is, the magnification factor for the pixel value D1h(11) is determined based on the pixel value YC(11) and the symbol sD1h(11), and the magnification factor for the pixel value D1h(12) is determined based on the pixel value YC(12) and the symbol sD1h(12) Generally speaking, as the magnification rate for the pixel value D1h(xy) is determined from the pixel value YC(xy) and the symbol sD1h(xy), the pixel value of the same coordinates of the image YC and the pixel value of the image D1h are added according to the brightness and color difference The signs of the pixel values at the same coordinates determine the magnification, and the result is output as the magnification D303MAh.

Further, the vertical magnification determining unit 303MAv determines the magnification of each pixel of the image D1v based on the signs of the pixel values at the same coordinates of the luminance and color difference added image YC and the pixel values of the same coordinates of the image D1v. That is, the magnification factor for the pixel value D1v(11) is determined based on the pixel value YC(11) and the symbol sD1v(11), and the magnification factor for the pixel value D1v(12) is determined based on the pixel value YC(12) and the symbol sD1v(12) Generally speaking, as the magnification rate for the pixel value D1v(xy) is determined from the pixel value YC(xy) and the symbol sD1v(xy), the pixel value of the same coordinates of the image YC and the value of the image D1v are added according to the luminance and color difference The signs of the pixel values at the same coordinates determine the magnification, and the result is output as the magnification D303MAv.

Then, the pixel value enlarging unit 303MB amplifies the pixel values of the first intermediate image D1 according to the enlarging factor D303MA. Since the first intermediate image D1 is composed of an image D1h and an image D1v, the pixel value enlarging unit 303MB has: a horizontal pixel value enlarging unit 303MBh for enlarging the pixel value of the image D1h; and a vertical enlarging unit 303MBh for enlarging the pixel value of the image D1v. Directional pixel value amplification unit 303MBv. That is, the horizontal pixel value enlarging unit 303MBh outputs an image D303MBh in which the pixel values of the image D1h are enlarged by the magnification factor D303MAh, and the vertical pixel value enlarging unit 303MBv outputs an image D303MBv in which the pixel values of the image D1v are amplified by the magnification factor D303MAv. Then, an image D303MBh and an image D303MBv are output from the pixel value enlarging unit 303MB as an image D303MB.

Image D303MB is output from first intermediate image processing unit 303M as intermediate image D303M. Intermediate image D303M is composed of image D303Mh equivalent to image D303MBh and image D303Mv equivalent to image D303MBv.

The above is the operation of the first intermediate image processing unit 303M.

Next, the operation of the second intermediate image processing unit 303H will be described. Comparing Fig. 35 with Fig. 36, the structure of the second intermediate image processing unit is the same as that of the first intermediate image processing unit except that the input signal is the luminance and color difference addition image YC and the intermediate image D2, and the output of the second intermediate image processing unit 303H is The intermediate image D2 is an intermediate image D303H obtained by performing the same processing as that performed on the intermediate image D1 by the first intermediate image processing unit 303M. Since the detailed operation of the second intermediate image processing unit 303H is also clear from the detailed operation description of the first intermediate image processing unit 303M, the detailed operation description of the second intermediate image processing unit 303H is omitted.

Hereinafter, an example in which the image processing device in this embodiment is used as a part of the image display device will be described. In addition, the image processing device in this embodiment can be used, for example, as a part of the image display device shown in FIG. 31 .

As described in Embodiment 3, by adding intermediate image D1 and intermediate image D2 to input luminance image YIN, high-frequency components can be added without enhancing superimposed components, and image clarity can be improved.

By adding the generated high-frequency components to the input image as described above, the sharpness of the image can be increased, and the image quality can be improved. However, when the high-frequency components are added excessively, there will be a step in the input luminance image YIN Overshooting or undershooting occurs at edge-like changes, which sometimes leads to a decrease in image quality.

Overshoot occurs when the sign of the pixel values of intermediate image D1 and intermediate image D2 is positive, and undershoot occurs when the sign of the pixel value of intermediate image D1 and intermediate image D2 is negative.

Hereinafter, disadvantageous situations in the case of occurrence of overshoot and disadvantageous situations in the case of occurrence of undershoot are respectively considered, and how these disadvantageous situations are prevented by the present embodiment will be described.

First, a description will be given of overshoot.

When overshoot occurs in the input luminance image YIN, the luminance signal increases excessively. It can be known from formula (9) that when the value of the luminance signal (Y) increases and the value of the luminance signal (Y) is converted to the RGB format, the first item on the right side of each formula representing R, G, and B increases. Therefore, R, G, B is a large value.

R, G, and B are all large values, which means that they are close to white. In other words, approaching white means that the color becomes lighter. Originally, even if the color is lightened in the near-achromatic part, it is relatively inconspicuous. However, when the color becomes light near the colored edge, only the color around the edge becomes light, giving an unnatural feeling.

In other words, when the signs of the pixel values of the intermediate image D1 and the intermediate image D2 are positive, when the magnitude of the luminance added by the intermediate image D1 and the intermediate image D2 in the colored portion (hereinafter referred to as the correction amount) is excessively increased , these image quality degradations are considered to occur easily. Therefore, it is considered that it is only necessary to adjust so that the correction amount of the intermediate image D1 and the intermediate image D2 does not increase excessively in the colored portion.

As this method, for example, it is considered that there is a method in which, when the sign of the pixel value of the intermediate image D1 and the intermediate image D2 is positive, a part that is colored and the correction amount provided by the intermediate image D1 and the intermediate image D2 is increased, In the detected part, the correction amount of the intermediate image D1 and the intermediate image D2 is reduced by appropriately applying a gain so that the correction amount does not increase excessively.

Whether it is colored can be judged according to the chroma (which can be expressed by the square root of the square sum of Cr and Cb). That is, for a chromatic color, the saturation is a large value. Also, the square root of the sum of the squares of Cr and Cb can be approximated using the sum of the absolute values of Cr and Cb. This is because, when the absolute value of Cr or Cb increases, the square root of the sum of the squares of Cr and Cb also increases. Also, since calculating the sum of absolute values is simpler than calculating the square root of the sum of squares, the circuit scale can also be reduced.

Whether or not the amount of correction provided by the intermediate image D1 and intermediate image D2 has increased can be determined to some extent from the pixel values of the input luminance image YIN. The reason is described below.

The intermediate image D1 is generated by performing high-pass filter processing on the input luminance image YIN and then performing low-pass filter processing. The low-pass filtering process is the same as calculating the local average value of the input data. Therefore, when the output value of the high-pass filter process is a large positive value, there is a high possibility that the output value of the low-pass filter process will also be a large positive value, and the correction amount provided by the intermediate image D1 tends to be a large value.

On the other hand, the high-pass filtering process corresponds to subtracting a local average value from each pixel value of the input luminance image YIN. Therefore, when the pixel value of the pixel of interest in the input luminance image YIN has a large value and the pixel values of its surrounding pixels are small, the output value after the high-pass filter process given to the pixel is also a large positive value.

Conversely, when the pixel value of the pixel of interest in the input luminance image YIN has a small value, the output value after the high-pass filtering process will not be a large positive value.

Therefore, when the pixel value of the pixel of interest in the input luminance image YIN is large, there is a high possibility that the output value after the high-pass filtering process given to the pixel will also be a large positive value.

On the other hand, the low-pass filtering process is the same as calculating the local average value of the input data. Therefore, when the output value of the high-pass filter process is a large positive value, there is a high possibility that the output value of the low-pass filter process will also be a large positive value.

As described above, when the pixel value of the pixel of interest in the input luminance image YIN is large, the correction amount provided by the intermediate image D1 tends to be large.

Furthermore, the intermediate image D2 is obtained by performing a high-pass filter process in the high-frequency component image generating unit 2B after performing nonlinear processing on the intermediate image D1 in the nonlinear processing unit 2A. Since the intermediate image D1 is enlarged only near the zero-crossing point in the nonlinear processing unit 2A, it is basically considered that when the intermediate image D1 has a large positive value, the image D2A output by the nonlinear processing unit 2A also has a large positive value . When image D2A has a large positive value, there is a high possibility that intermediate image D2 that is the result of the high-pass filtering process on image D2A also has a large positive value.

As described above, when the pixel value of the input luminance image YIN is large, there is a high possibility that the pixel values of the intermediate image D1 and intermediate image D2 are also large positive values. In other words, when the pixel value of the input luminance image YIN is large, it can be determined that the correction amount provided by the intermediate image D1 and the intermediate image D2 is increased to some extent.

From the above reasons, it is considered that the correction amount increases for a pixel in which any one of the absolute value of the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN is larger, and the correction amount becomes larger. There is a disadvantage that the color fades when changing to RGB form.

In addition, when any one of the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN is large, the weighted addition value is also a large value. .

Therefore, in this embodiment, the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN are weighted and added according to the formula (7) to generate the luminance In the color difference added image YC, when the sign of the pixel value of the intermediate image D1 and the intermediate image D2 is positive, the larger the pixel value of the brightness and color difference added image YC, the smaller the value applied to the intermediate image D1 and the intermediate image D2. Magnification, thus avoiding the unfavorable situation of color fading near colored edges.

The relationship between such a magnification factor (GAIN) and the pixel value (L) of the luminance-chromatic-difference added image YC is expressed as follows, for example:

[Equation 8]

where …(11)

(A) of FIG. 38 is a diagram showing the relationship between the magnification factor (GAIN) represented by Equation (11) and the pixel value (L) of the added luminance and color difference image YC. When the pixel value of the luminance and color difference added image YC is 0, a certain predetermined value pB is taken, and gradually decreases with a slope pk1 between the pixel value from 0 to a certain value pA1, and when the pixel value is from pA1 to a certain value pA2 gradually decreases with slope pk2, and when the pixel value is pA2 or more, gradually decreases with slope pk3.

In short, the magnification D303MA or the magnification D303HA is determined based on a monotonically decreasing function such that the larger the pixel value of the luminance-color-difference added image YC is, the smaller the magnification shown in (A) of FIG. 38 or the formula (11). , so that processing can be performed to prevent the disadvantageous situation of fading in the vicinity of colored edges.

Then, description will be given for the undershoot.

When undershooting occurs in the input luminance image YIN, the luminance signal decreases excessively. It can be seen from formula (9) that when the value of the luminance signal (Y) is reduced and converted to the RGB format, the first term on the right side of each formula representing R, G, and B decreases, and therefore, R, G, and B are all small values.

All of R, G, and B being small values mean that they are close to black. When the color becomes black near the edge, a black false outline bordering the edge appears, giving an unnatural feeling.

In other words, when the sign of the pixel value of intermediate image D1 and intermediate image D2 is negative, when the magnitude of the brightness subtracted by intermediate image D1 and intermediate image D2 (hereinafter referred to as correction amount) is an excessively large value, It is thought that these image quality drops are easy to occur. Therefore, it is considered that when the signs of the pixel values of intermediate image D1 and intermediate image D2 are negative, it is only necessary to adjust so that the correction amount does not increase excessively.

As this method, for example, there is considered a method in which, when the signs of the pixel values of intermediate image D1 and intermediate image D2 are negative, a portion where the correction amount increases is detected, and a gain is appropriately applied to the detected portion so that The correction amounts of intermediate image D1 and intermediate image D2 are reduced so that the correction amount does not increase excessively.

Whether or not the correction amount provided by intermediate image D1 and intermediate image D2 increases (is a negative value with a large absolute value) can be determined to some extent from the pixel value of the input luminance image YIN. The reason is described below.

The intermediate image D1 is generated by performing high-pass filter processing on the input luminance image YIN and then performing low-pass filter processing. Here, the high-pass filtering process corresponds to subtracting a local average value from each pixel value of the input luminance image YIN. Therefore, when the pixel value of a pixel of interest in the input luminance image YIN is small, there is a high possibility that the output value after the high-pass filtering process applied to the pixel will also be a negative value with a large absolute value.

On the other hand, the low-pass filtering process is the same as calculating the local average value of the input data. Therefore, when the output value of the high-pass filter process is a negative value with a large absolute value, there is a high possibility that the output value of the low-pass filter process will also be a negative value with a large absolute value.

Furthermore, the intermediate image D2 is obtained by performing a high-pass filter process in the high-frequency component image generating unit 2B after performing nonlinear processing on the intermediate image D1 in the nonlinear processing unit 2A. Since the intermediate image D1 is enlarged only near the zero-crossing point in the nonlinear processing unit 2A, it is basically considered that when the intermediate image D1 has a negative value with a large absolute value, the image D2A output by the nonlinear processing unit 2A also has an absolute value large negative value. When image D2A has a large negative value in absolute value, there is a high possibility that intermediate image D2 , which is the result of the high-pass filtering process on image D2A, also has a large negative value in absolute value.

As described above, when the pixel values of the input luminance image YIN are small, there is a high possibility that the pixel values of the intermediate images D1 and D2 are also small negative values. In other words, when the pixel value of the input luminance image YIN is small, it can be determined that the correction amount provided by the intermediate image D1 and the intermediate image D2 is a large value to some extent.

For the above reasons, it is considered that the correction amount is large for pixels with small pixel values of the input luminance image YIN, and there is a disadvantage that the pixels become black when the RGB format is changed.

In this embodiment, when the sign of the pixel value of the intermediate image D1 and the intermediate image D2 is negative, instead of the pixel value of the input luminance image YIN, the smaller the pixel value of the luminance-color-difference addition image YC is, the smaller the sign is. The magnification applied to the intermediate image D1 and the intermediate image D2 is small so as to avoid the disadvantage of false contours near the edges. (The reason why the luminance-color-difference addition image YC can be used instead of the input luminance image YIN will be described later.)

The relationship between such a magnification factor (GAIN) and the pixel value (L) of the luminance-chromatic-difference added image YC is expressed as follows, for example:

[Equation 9]

In the formula ...(12)

(B) of FIG. 38 is a graph showing the relationship between the magnification factor (GAIN) represented by Equation (12) and the pixel value (L) of the added luminance and color difference image YC. When the pixel value of the luminance and color difference added image YC is 0, a certain predetermined value mB is taken, and the pixel value gradually increases from 0 to a certain value mA1 with a slope mk1, and when the pixel value is from mA1 to a certain value mA2 Gradually increase with slope mk2, and gradually increase with slope mk3 when the pixel value is mA2 or more.

By performing the above processing, it is possible to perform processing to prevent the disadvantage that the color near the edge is close to black near the edge of the achromatic color. This is because, in the case of achromatic color, the absolute value of the pixel value of the input CR image CRIN and the absolute value of the pixel value of the input CB image CBIN are close to zero, so it is considered that the magnitude relationship of the pixel values of the luminance-added image YC directly expresses The magnitude relationship of the pixel values of the input brightness image YIN. Therefore, it is considered that there is no problem in using the luminance-color-difference added image YC instead of the input luminance image YIN near the edge of the achromatic color.

On the other hand, in the vicinity of colored edges, since the absolute values of the pixel values of the input CR image CRIN and the absolute values of the pixel values of the input CB image CBIN can take large values, the magnitude relationship of the pixel values of the luminance-added image YC is different. It must directly represent the magnitude relationship of the pixel values of the input luminance image YIN. For example, when the pixel value of the input luminance image YIN is small but the absolute value of the pixel value of the input CR image CRIN or the pixel value of the input CB image CBIN is large, the pixel value of the input luminance image YIN is small while that of the luminance-chromatic-difference addition image YC is The pixel value is a large value.

However, when the luminance in a natural image is small, the absolute value of the color difference is often small, so the pixel value of the input luminance image YIN is small but the absolute value of the pixel value of the input CR image CRIN or the pixel value of the input CB image CBIN is large Situations are rare. Therefore, controlling the magnification ratios for intermediate images D1 and D2 using the pixel values of the luminance-color-difference added image YC instead of the pixel values of the input luminance image YIN as described above is considered to be practically less problematic.

In short, the magnification ratio D303MA or the magnification ratio D303HA is determined based on a monotonically increasing function such that the larger the pixel value of the luminance-chromatic-difference added image YC is, the larger the magnification ratio is shown in FIG. 38(B) or Equation (12). , so that processing can be performed to prevent the disadvantage that the color is close to black in the vicinity of the edge.

As described above, in the image processing device according to Embodiment 4, it is possible to perform image enhancement processing while suppressing occurrence of disadvantages such as fading or approaching black in the vicinity of edges. If the color near the edge becomes light or close to black, it will feel unnatural in terms of visual properties, so in the image processing device according to Embodiment 4, it is also very preferable in terms of visual properties.

Furthermore, in the image processing device according to Embodiment 4, the magnification ratios for the intermediate image D1 and the intermediate image D2 are determined in the first intermediate image processing unit 303M and the second intermediate image processing unit 303H, but the information required at this time is only is the sign of the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, the absolute value of the pixel value of the input CB image CBIN, and the pixel value of the intermediate image D1 or intermediate image D2. Therefore, the amplification factor can be determined in a simple circuit, and an increase in circuit scale can also be reduced.

For example, comparing formula (11) and formula (12), the two differ only in parameters such as coefficients and thresholds used therein. That is, the magnification determining unit 303MA or the magnifying factor determining unit 303MB can be constituted using an arithmetic unit and a coefficient switching unit that can perform the following calculation expressed using coefficients k1, k2, k3, thresholds A1, A2, and value B:

[Equation 10]

In the formula ...(13)

The coefficient switching unit changes the values of the coefficients k1, k2, k3, thresholds A1, A2 and value B given to the arithmetic unit according to the sign of the pixel value of the intermediate image D1 or intermediate image D2.

That is, the switching unit can be used to select -pk1, -pk2, -pk3, pA1, pA2, pB when the sign of the pixel value is positive, and select mk1, mk2, mk3, mA1, mA2 when the sign of the pixel value is negative , mB, can be used as k1, k2, k3, A1, A2, B to carry out the operation of formula (13) respectively.

In addition, in the above-mentioned embodiment, in the first intermediate image processing section 303M, the magnification ratio D303MA is determined based on the output YC of the luminance and color difference adding section 205 and the symbol sD1 of the pixel value of the first intermediate image D1, and the magnification ratio D303MA is determined in the second intermediate image D1. In the image processing unit 303H, the magnification D303HA is determined based on the output YC of the luminance and color difference adding unit 205 and the sign sD2 of the pixel value of the second intermediate image D2. However, only the first intermediate image processing unit 303M and the second intermediate image may One of the processing units 303H determines the magnification using the method described above, and the other determines the magnification using another method.

Furthermore, the relationship between the magnification determined in the first intermediate image processing unit 303M and the second intermediate image processing unit 303H and the pixel value of the luminance and color difference added image YC is not limited to the relationship described in this embodiment. In the intermediate image D1 Or when the sign of D2 is positive, the larger the pixel value of the added luminance and color difference image YC, the smaller the magnification. The smaller the pixel value of YC, the smaller the magnification ratio.

Embodiment 5

39 is a diagram showing a configuration example of an image processing device according to Embodiment 5 of the present invention.

As described for the third embodiment, the input image IMGIN is input to the illustrated image processing device, and the illustrated image processing device outputs the output image IMGOUT. The input image IMGIN is a color image composed of an input luminance image YIN, an input CR image CRIN, and an input CB image CBIN. The output image IMGOUT is also a color image, and is composed of an output luminance image YOUT, an output CR image CROUT, and an output CB image CBOUT.

The illustrated image processing device includes a first intermediate image generation unit 201 , a second intermediate image generation unit 2 , an addition unit 404 , and a color difference increase/decrease unit 405 .

The operation and configuration of the first intermediate image generation unit 201 and the second intermediate image generation unit 2 are the same as those of the same reference numerals in the third embodiment.

The color difference increase/decrease section 405 performs processing described later on the input CR image CRIN and the input CB image CBIN, and outputs the output CR image CROUT and the output CB image CBOUT.

The addition unit 404 adds the input luminance image YIN, the intermediate image D1 and the intermediate image D2, outputs the addition result as the output luminance image YOUT, and outputs the result of adding the intermediate image D1 and the intermediate image D2 as the high-frequency component addition Image D404.

40 is a diagram showing a configuration example of the color difference increase/decrease unit 405. The illustrated color difference increase/decrease unit 405 includes a magnification factor determination unit 405A, a color difference Cr multiplication unit (first color difference multiplication unit) 405B1, and a color difference Cb multiplication unit (first color difference multiplication unit). 2 color difference multiplication units) 405B2.

The amplification factor determining section 405A determines the amplification factor D405A based on the high-frequency component added image D404.

The color difference Cr multiplier 405B1 increases or decreases each pixel value of the input CR image CRIN according to the value of the magnification factor D405A, and outputs the result as an image D405B1.

The color difference Cb multiplier 405B2 increases or decreases each pixel value of the input CB image CBIN according to the value of the magnification factor D405A, and outputs the result as an image D405B2.

Hereinafter, detailed operations of the image processing device according to Embodiment 5 will be described.

However, the detailed operations of the first intermediate image generation unit 201 and the second intermediate image generation unit 2 are the same as those described in Embodiment 3, and thus description thereof will be omitted.

Next, the operation of adding unit 404 will be described. Addition unit 404 outputs the result of adding intermediate image D1 and intermediate image D2 as high-frequency component added image D404. Then, the result of adding the high-frequency component added image D404 to the input luminance image YIN (that is, the result of adding the intermediate image D1 and the intermediate image D2 ) is output as the output luminance image YOUT. The output luminance image YOUT is output from the image processing device as part of the final output image IMGOUT.

Since the intermediate image D1 is composed of the image D1h and the image D1v, and the intermediate image D2 is composed of the image D2h and the image D2v, adding the input image D1 and the intermediate image D2 refers to adding all the images D1h, D1v, D2h and D2v.

Here, the addition processing in adding section 404 is not limited to simple addition, and weighted addition may be performed. That is, each of the images D1h, D1v, D2h, and D2v may be enlarged and added at different magnification factors as the high-frequency component added image D404.

Next, the detailed operation of the color difference increase/decrease unit 405 will be described. First, the color difference increasing/decreasing section 405 determines the amplification factor D405A from the high-frequency component added image D404 in the amplification factor determining section 405A. Here, the magnification D405A is determined for each pixel.

(A) to (C) of FIG. 41 are diagrams showing the pixel arrangement of the high-frequency component added image D404, the input CR image CRIN, and the input CB image CBIN. FIG. 41(A) shows the high-frequency component added image D404, (B) of FIG. 41 shows the input CR image CRIN, and (C) of FIG. 41 shows the input CB image CBIN. In addition, in (A) to (C) of FIG. 41 , horizontal coordinates and vertical coordinates are shown for the horizontal direction and vertical direction of the image. Furthermore, for the high-frequency component added image D404, the pixel value of the pixel at the position of the horizontal coordinate x and the vertical coordinate y is represented by a symbol such as L(xy), and for the input CR image CRIN, the pixel value of the pixel at the horizontal coordinate x and the vertical coordinate The pixel value of the pixel at the position of y is represented by a symbol such as Cr(xy), and for the input CB image CBIN, the pixel value of the pixel at the position of horizontal coordinate x and vertical coordinate y is represented by a symbol such as Cb(xy) .

The magnification factor determining section 405A determines the magnification factor for each pixel of the input CR image CRIN and the input CB image CBIN based on the pixel values at the same coordinates of the high-frequency component added image D404 . That is, the magnification factors for the pixel values Cr(11) and Cb(11) are determined based on the pixel value L(11), and the magnification factors for the pixel values Cr(12) and Cb(12) are determined based on the pixel value L(12), In general, as the amplification factors for the pixel values Cr(xy) and Cb(xy) are determined based on the pixel value L(xy), the amplification factors are determined based on the pixel values at the same coordinates of the high-frequency component addition image D404, and the resulting values are output. The result is taken as magnification D405A.

FIG. 42 is a diagram showing the relationship between the pixel value (hereinafter denoted as L) of the high-frequency component added image D404 and the magnification factor (hereinafter denoted as GAIN) determined by the magnification factor determining section 405A.

As shown in the figure, when L is zero, GAIN is 1, when L is a positive value, GAIN is a value greater than 1, and when L is a negative value, GAIN is a positive value smaller than 1. Such a relationship between L and GAIN can be represented by the following formula, for example:

[Equation 11]

In the formula, kp and km are predetermined positive coefficients, kp represents the slope in the region of L>0 of the curve in FIG. 42 , and km represents the slope in the region of L<0 of the curve in FIG. 42 .

In addition, GAIN must take a positive value. In the case of calculating GAIN using formula (14), as long as the value of km is sufficiently small relative to the possible value of L, the value of GAIN can always be a positive value. For example, when L is a signed 8-bit integer value, the possible value of L is -128 or more and 127 or less. Therefore, it is sufficient as long as km is a value smaller than 1/128. In general, when the possible value of L is -ML or more (ML is a positive value), the value of km only needs to be 1/ML or less, so that GAIN does not become a negative value. In this way, the limitation on km can be easily considered according to the minimum value of the possible value of L.

The color difference Cr multiplier 405B1 outputs the result obtained by applying the amplification factor D405A to the pixel value of the input CR image CRIN as an image D405B1. When the magnification ratio D405A is larger than 1, the pixel value of the input CR image CRIN is enlarged, and when the magnification ratio D405A is smaller than 1, the pixel value of the input CR image CRIN is reduced, and when the magnification ratio D405A is 1 , the pixel values of the input CR image CRIN are maintained. In addition, the value of the amplification factor D405A is larger than 1 when the pixel value of the high-frequency component added image D404 is positive, smaller than 1 when negative, and 1 when zero. Therefore, the pixel value of the input CR image CRIN When the pixel value of the high-frequency component addition image D404 is positive, it is amplified, when it is negative, it is decreased, and when it is zero, it is maintained.

Similarly, color difference Cb multiplier 405B2 outputs the result obtained by adding amplification factor D405A to the pixel value of input CB image CBIN as image D405B2. In the case where the magnification ratio D405A is larger than 1, the pixel value of the input CB image CBIN is enlarged, and in the case where the magnification ratio D405A is smaller than 1, the pixel value of the input CB image CBIN is reduced, and in the case where the magnification ratio D405A is 1 , the pixel values of the input CB image CBIN are maintained. In addition, the value of the amplification factor D405A is greater than 1 when the pixel value of the high-frequency component added image D404 is positive, less than 1 when negative, and 1 when zero. Therefore, the pixel value of the input CB image CBIN When the pixel value of the high-frequency component addition image D404 is positive, it is enlarged, when it is negative, it is reduced, and when it is zero, it is maintained.

Then, image D405B1 is output as output CR image CROUT, and image D405B2 is output as output CB image CBOUT. Then, the output CR image CROUT and the output CB image CBOUT are output from the image processing device as part of the final output image IMGOUT.

The above is the operation of the color difference increase/decrease unit 405 .

Hereinafter, an example in which the image processing device in this embodiment is used as a part of the image display device will be described. In addition, the image processing device in this embodiment can be used, for example, as a part of the image display device shown in FIG. 31 .

In the image processing device of this embodiment, intermediate image D1 and intermediate image D2 are added to input luminance image YIN. As mentioned above, by adding the intermediate image D1 and intermediate image D2 to the input luminance image YIN, high-frequency components can be added without enhancing the superimposed components, and the clarity of the image can be improved.

However, by adding the generated high-frequency components to the input image as described above, the sharpness of the image can be increased and the image quality can be improved. However, if the high-frequency components are added too much, the input luminance image YIN appears Overshooting or undershooting occurs at the step-edge-like change, which sometimes leads to a decrease in image quality.

In particular, when overshoot occurs in the input luminance image YIN, the luminance signal increases excessively. It can be seen from formula (9) that when the value of the luminance signal (Y) increases and the value of the luminance signal (Y) is converted to the RGB format, the first item on the right side of each formula representing R, G, and B increases, and therefore, R, G, B are all large values.

R, G, and B are all large values, which means that they are close to white. In other words, approaching white means that the color becomes lighter. Originally, even if the color is lightened in the part close to achromatic color, it is relatively inconspicuous. However, when the color becomes light near the edge of the colored part (a part with relatively high saturation), only the color around the edge becomes light, resulting in an unnatural appearance. Feel.

In other words, when the magnitude of the luminance (hereinafter referred to as the correction amount) added by the intermediate image D1 and the intermediate image D2 (or the high-frequency component added image D404) is excessively increased in the colored portion, the luminance is too large relative to the color difference , there will be a problem of color fading. Also, from the opposite point of view, when the correction amount is an excessively small negative value, the luminance is too small relative to the color difference, resulting in a problem that the color is too dark. That is, there is a problem that the shade of the color changes near the edge of the colored portion.

The reason for the above problem is that, in the case where the correction amount is a positive value, the chromatic aberration decreases with respect to the luminance, and in the case where the correction amount is a negative value, the chromatic aberration increases with respect to the luminance.

Therefore, in the present embodiment, by appropriately increasing or decreasing the color difference signal according to the correction amount, the color difference is prevented from decreasing or increasing with respect to the luminance.

That is, when the correction amount is a positive value, the color difference signal is amplified to prevent the color difference from being relatively reduced, and when the correction amount is a negative value, the color difference signal is reduced to prevent the color difference from being relatively increased.

In the image processing device of the present embodiment, the value of the color difference signal is increased or decreased in the color difference increasing/decreasing section 405 according to the pixel value of the high-frequency component added image D404. That is, when the pixel value of the high-frequency component added image D404 is positive, a value larger than 1 is output from the amplification factor determination unit 405A as the amplification factor D405A, and the color difference signal is amplified in the color difference Cr multiplication unit 405B1 and the color difference Cb multiplication unit 405B2. . And, when the pixel value of the high-frequency component added image D404 is negative, a value smaller than 1 is output from the amplification factor determination unit 405A as the amplification factor D405A, and subtracted in the color difference Cr multiplication unit 405B1 and the color difference Cb multiplication unit 405B2. Small color difference signal. Therefore, the above-mentioned problems can be prevented in advance.

As described above, in the image processing device according to Embodiment 5, it is possible to perform image enhancement processing while suppressing the occurrence of disadvantageous changes in color shading near the edges of colored portions. When the color shading changes near the edge of the colored portion, the visual characteristics are unnatural. Therefore, the image processing device according to the fifth embodiment can obtain an effect which is very preferable also in terms of visual characteristics.

Also, by applying the same amplification factor D405A to each pixel value of the input CR image CRIN and the input CB image CBIN, the color shade (or chroma) changes, but the ratio of each pixel value of the input CR image CRIN and the input CB image CBIN does not change. change, so the hue does not change. Therefore, in the image processing device according to Embodiment 5, it is possible to correct color shading occurring near the edge without changing the hue.

Furthermore, the relationship between the magnification factor D405A determined in the magnification factor determining unit 405A and the pixel value of the high-frequency component added image D404 is not limited to the relationship shown in equation (14), as long as it is such a relationship: The pixel value of the added image D404 takes a value larger than 1 when it is positive, and takes a value smaller than 1 when it is negative. However, in order to correct the color difference signal more effectively, it is preferable that the larger the positive value of the pixel value of the high-frequency component added image D404 is, the larger the value of the amplification factor D405A is. The smaller the negative value of the pixel value of the added image D404 is, the smaller the positive value smaller than 1 is.

However, even if this is not the case, it is only necessary that the magnification ratio D405A increases monotonously with respect to the pixel value of the high-frequency component added image D404.

Furthermore, in order to prevent excessive correction of the color difference signal, a limit may also be set on the range of the amplification factor D405A. That is, setting thresholds TH1 and TH2 (assuming TH1>1, 1>TH2>0), the relationship between the magnification (GAIN) and the pixel value (L) of the luminance and color difference added image YC can be defined as follows, for example:

[Equation 12]

In the formula ...(15)

kp and km in Equation (15) are the same as those described for Equation (14). The relationship between the magnification factor GAIN expressed by the formula (15) and the pixel value L is shown in FIG. 43 .

By thus setting upper and lower limits on the value of the magnification ratio (GAIN) using the threshold, excessive correction can be prevented from being applied to the color difference components in the color difference increase/decrease unit 405 .

In addition, various modification examples of the relationship between GAIN and L can be considered other than Expression (14) and Expression (15). Also, in the above description, the same amplification factor is used in the color difference Cr amplification unit 405B1 and the color difference Cb amplification unit 405B2, but different amplification factors may be used in the color difference Cr multiplication unit 405B1 and the color difference Cb multiplication unit 405B2.

Furthermore, in the image processing apparatus according to Embodiment 1 to Embodiment 5, in the first intermediate image generation unit 1 or 201 and the second intermediate image generation unit 2, processing related to the horizontal direction of the image and processing related to the vertical direction are performed in parallel. The related processing is thus not limited to only the horizontal direction or only the vertical direction of the image, the above-mentioned effect can be obtained for any direction.

Furthermore, in the image processing apparatuses according to Embodiments 1 to 5, considering the frequency space, in the frequency band transitioning from the origin to Fn, the Nyquist frequency of the original image DORG around ±Fn/2 (or specified The frequency band of the input image DIN or the input luminance image YIN has a component, and an image D2B corresponding to a high frequency component near the Nyquist frequency ±Fn of the input image DIN or the input luminance image YIN is generated. Therefore, even if the frequency components near the Nyquist frequency ±Fn are lost in the input image DIN or the input luminance image YIN (or the input image IMGIN) for some reason, the Nyquist frequency ±Fn can be provided by the image D2B nearby frequency components. In other words, since the input image DIN or the input luminance image YIN is provided with frequency components on the side of higher frequency components, the sharpness of the output image DOUT or the output luminance image YOUT (or the output image IMGOUT) can be increased.

Also, the portion used as a specific frequency band is not limited to the vicinity of ±Fn/2. That is, by appropriately changing the frequency responses of the high-frequency component image generating unit 1A and the low-frequency component image generating unit 1B, the frequency band to be used can be changed.

Furthermore, in the above-mentioned descriptions of Embodiments 1 to 5, image enlargement processing was cited as an example of losing frequency components near the Nyquist frequency Fn. The cause of the frequency component near the Guister frequency Fn is not limited to this, but noise removal processing and the like may also be considered. Therefore, the use of the image processing device in the present invention is not limited to after image enlargement processing.

Embodiment 6

In Embodiments 1 to 5, it has been described that the present invention is implemented using hardware, but it is also possible to implement a part or all of the configuration shown in FIG. 1 using software. Processing in this case will be described with reference to FIG. 44 and FIGS. 45 to 64 .

FIG. 44 shows an image processing device used in the sixth to tenth embodiments. The illustrated image processing device has a CPU 11, a program memory 12, a data memory 13, a first interface 14, a second interface 15, and a bus 16 connecting these components. The CPU 11 operates according to a program stored in the program memory 12. In the process, the data memory 13 is made to store various data, which can replace the image processing device shown in Fig. 1 or Fig. 20, and be used as the image processing device U2 in the display device shown in Fig. 9, or replace Fig. 27, Fig. 34 Or the image processing device shown in FIG. 39 is used, for example, as the image processing device U202 in the display device shown in FIG. 31 .

First, a case where the image processing device of FIG. 44 is used instead of the image processing device of FIG. 1 will be described. In this case, the image DU1 output from the image enlargement unit U1 shown in FIG. The image DOUT is supplied to, for example, the monitor U3 in the image display device shown in FIG. 9 via the interface 15 as the image DU2 for display on the monitor U3.

45 is a diagram showing the flow of an image processing method according to Embodiment 6 of the present invention implemented by making the image processing device in FIG. 44 have the same functions as the image processing device according to Embodiment 1, and the illustrated image processing method It includes a first intermediate image generation step ST1, a second intermediate image generation step ST2, a first intermediate image processing step ST3M, a second intermediate image processing step ST3H, and an addition step ST4.

As shown in FIG. 46 , the first intermediate image generation step ST1 includes a high-frequency component image generation step ST1A and a low-frequency component image generation step ST1B.

The high-frequency component image generating step ST1A includes a horizontal high-frequency component image generating step ST1Ah and a vertical high-frequency component image generating step ST1Av, and the low-frequency component image generating step ST1B includes a horizontal low-frequency component image generating step ST1Bh and a vertical low-frequency component image generating step ST1Bh. Step ST1Bv.

As shown in FIG. 47, the second intermediate image generation step ST2 includes a nonlinear processing step ST2A and a high-frequency component image generation step ST2B.

The nonlinear processing step ST2A includes a horizontal nonlinear processing step ST2Ah and a vertical nonlinear processing step ST2Av, and the high-frequency component image generation step ST2B includes a horizontal high-frequency component passing step ST2Bh and a vertical high-frequency component passing step ST2Bv.

As shown in FIG. 48, the horizontal nonlinear processing step ST2Ah includes a zero-crossing determination step ST311h and a signal amplification step ST312h. As shown in FIG. 49, the vertical nonlinear processing step ST2Av includes a zero-crossing determination step ST311v and a signal amplification step ST312v.

As shown in FIG. 50 , the first intermediate image processing step ST3M includes a magnification factor determining step ST3MA and a pixel value changing step ST3MB.

As shown in FIG. 51 , the second intermediate image processing step ST3H includes an enlargement factor determining step ST3HA and a pixel value changing step ST3HB.

First, the operation of the first intermediate image generation step ST1 will be described in accordance with the flow shown in FIG. 46 .

In the high-frequency component image generation step ST1A, the following processing is performed on the input image DIN input in the image input step (not shown).

First, in the horizontal high-frequency component image generating step ST1Ah, an image D1Ah obtained by extracting a horizontal high-frequency component from the input image DIN is generated by horizontal high-pass filtering.

In the vertical high-frequency component image generation step ST1Av, an image D1Av obtained by extracting a vertical high-frequency component from the input image DIN is generated by a vertical high-pass filter process.

That is, the high-frequency component image generating step ST1A performs the same processing as the high-frequency component image generating means 1A, and generates an image D1A composed of an image D1Ah and an image D1Av from the input image DIN. This operation is the same as that of the high-frequency component image generating unit 1A.

In the low-frequency component image generation step ST1B, the following processing is performed on the image D1A. First, in the horizontal low-frequency component image generating step ST1Bh, an image D1Bh obtained by extracting the horizontal low-frequency component from the image D1Ah is generated by horizontal low-pass filtering processing.

In the vertical low-frequency component image generation step ST1Bv, an image D1Bv obtained by extracting a vertical low-frequency component from the image D1Av is generated by a vertical low-pass filter process.

That is, the low-frequency component image generating step ST1B performs the same processing as that of the low-frequency component image generating means 1B, and generates an image D1B composed of an image D1Bh and an image D1Bv from the image D1A. This operation is the same as that of the low-frequency component image generating unit 1B.

The above is the operation of the first intermediate image generating step ST1. The first intermediate image generating step ST1 sets image D1Bh as image D1h and image D1Bv as image D1v, and outputs intermediate image D1 composed of image D1h and image D1v. The above operation is the same as that of the first intermediate image generation unit 1 .

Next, the operation of the second intermediate image generation step ST2 will be described in accordance with the flow of FIGS. 47 to 49 .

First, the intermediate image D1 is processed as follows in the nonlinear processing step ST2A.

First, in the horizontal non-linear processing step ST2Ah, an image D2Ah is generated from the image D1h by processing according to the flow shown in FIG. 48 . The processing of the flow shown in FIG. 48 is as follows. First, in the zero-cross determination step ST311h, changes in pixel values in the image D1h are checked in the horizontal direction. Then, the position where the pixel value changes from positive value to negative value or from negative value to positive value is captured as a zero crossing point, and the pixels located around the zero crossing point are notified to the signal amplification step ST312h. In the signal amplification step ST312h, with respect to the image D1h, the pixel value of the notified pixel positioned around the zero cross point is amplified, and the image is output as the image D2Ah. That is, the non-linear processing step ST2Ah performs the same processing as the horizontal non-linear processing means 2Ah on the image D1h to generate the image D2Ah.

Then, in the vertical non-linear processing step ST2Av, an image D2Av is generated from the image D1v by processing according to the flow shown in FIG. 49 . The processing of the flow shown in FIG. 49 is as follows. First, in the zero-cross determination step ST311v, changes in pixel values in the image D1v are checked in the vertical direction. Then, the position where the pixel value changes from positive to negative or from negative to positive is captured as a zero-crossing point, and the pixels located above and below the zero-crossing point are notified to the signal amplification step ST312v. In the signal amplification step ST312v, with respect to the image D1v, the pixel values of the pixels notified to be located above and below the zero cross point are amplified, and this image is output as the image D2Av. That is, the non-linear processing step ST2Av performs the same processing as the vertical non-linear processing means 2Av on the image D1v to generate the image D2Av.

The above is the operation of the nonlinear processing step ST2A. The nonlinear processing step ST2A generates the image D2A composed of the image D2Ah and the image D2Av. This operation is the same as that of the nonlinear processing unit 2A.

Then, in the high-frequency component image generation step ST2B, the following processing is performed on the image D2A.

First, in the horizontal high-frequency component image generating step ST2Bh, image D2Bh obtained by performing horizontal high-pass filter processing on image D2Ah is generated. That is, the horizontal high-frequency component image generation step ST2Bh performs the same processing as that of the horizontal high-frequency component image generation unit 2Bh.

Next, in the vertical high-frequency component image generation step ST2Bv, image D2Bv in which image D2Av has been subjected to vertical high-pass filter processing is generated. That is, the vertical high-frequency component image generation step ST2Bv performs the same processing as the vertical high-frequency component image generation unit 2Bv.

The above is the operation of the high-frequency component image generating step ST2B. The high-frequency component image generating step ST2B generates the image D2B composed of the image D2Bh and the image D2Bv. This operation is the same as that of the high-frequency component image generating unit 2B.

The above is the operation of the second intermediate image generation step ST2, and the second intermediate image generation step ST2 outputs the image D2B as the intermediate image D2. That is, an intermediate image D2 in which the image D2Bh is the image D2h and the image D2Bv is the image D2v is output. This operation is the same as that of the second intermediate image generation unit 2 .

Next, the operation of the first intermediate image processing step ST3M will be described in accordance with the flow shown in FIG. 50 .

First, in the first intermediate image processing step ST3M, in the magnification factor determination step ST3MA, the magnification factor for the pixel value of each pixel of the intermediate image D1 is determined. Here, since the intermediate image D1 is composed of the image D1h and the image D1v, the enlargement factor is determined for each pixel of the image D1h and the image D1v. That is, for the image D1h, the magnification for each pixel is determined in the horizontal magnification determination step ST3MAh, and for the image D1v, the magnification for each pixel is determined in the vertical magnification determination step ST3MAv. Here, the operation of the horizontal magnification determination step ST3MAh is the same as that of the horizontal magnification determination unit 3MAh, and the operation of the vertical magnification determination step ST3MAv is the same as that of the vertical magnification determination unit 3MAv, so description thereof is omitted.

Then, in the pixel value changing step ST3MB, the pixel value of each pixel of the intermediate image D1 is amplified according to the magnification factor determined in the magnification factor determination step ST3MA. Here, since the intermediate image D1 is composed of the image D1h and the image D1v, the pixel values of each of the image D1h and the image D1v are enlarged. That is, the image D3MBh is generated by enlarging each pixel value of the magnification factor of the image D1h determined in the horizontal magnification factor determination step ST3MAh. Then, an image D3MBv is generated based on each pixel value of the magnification enlarged image D1v determined in the vertical magnification determination step ST3MAv. This operation is the same as that of the pixel value changing unit 3MB.

Then, an intermediate image D3M composed of an image D3Mh corresponding to the image D3MBh and an image D3Mv corresponding to the image D3MBv is generated by the first intermediate image processing step ST3M. The above is the operation of the first intermediate image processing step ST3M, which is the same as that of the first intermediate image processing unit 3M.

Next, the operation of the second intermediate image processing step ST3H will be described in accordance with the flow shown in FIG. 51 .

First, in the second intermediate image processing step ST3H, in the magnification factor determination step ST3HA, the magnification factor for the pixel value of each pixel of the intermediate image D2 is determined. Here, since the intermediate image D2 is composed of the image D2h and the image D2v, the enlargement factor is determined for each pixel of the image D2h and the image D2v. That is, for the image D2h, the magnification for each pixel is determined in the horizontal magnification determination step ST3HAh, and for the image D2v, the magnification for each pixel is determined in the vertical magnification determination step ST3HAv. Here, the operation of the horizontal magnification determination step ST3HAh is the same as that of the horizontal magnification determination unit 3HAh, and the operation of the vertical magnification determination step ST3HAv is the same as that of the vertical magnification determination unit 3HAv, so description thereof is omitted.

Then, in the pixel value changing step ST3HB, the pixel value of each pixel of the intermediate image D2 is amplified according to the magnification factor determined in the magnification factor determination step ST3HA. Here, since the intermediate image D2 is composed of the image D2h and the image D2v, the pixel values are enlarged for each of the image D2h and the image D2v. That is, the image D3HBh is generated by enlarging each pixel value of the image D2h at the magnification determined in the horizontal magnification determination step ST3HAh. Then, the image D3HBv is generated by enlarging each pixel value of the image D2v with the magnification determined in the vertical magnification determination step ST3HAv. This operation is the same as that of the pixel value changing unit 3HB.

Then, an intermediate image D3H composed of an image D3Hh corresponding to the image D3HBh and an image D3Hv corresponding to the image D3HBv is generated by the second intermediate image processing step ST3H. The above is the operation of the second intermediate image processing step ST3H, which is the same as that of the second intermediate image processing unit 3H.

The addition step ST4 adds the input image DIN, the intermediate image D3M, and the intermediate image D3H to generate the output image DOUT. Since the intermediate image D3M is composed of the image D3Mh and the image D3Mv, and the intermediate image D3H is composed of the image D3Hh and the image D3Hv, in the adding step ST4, all the images D3Mh, D3Hv, D3Mh, and D3Hv are added to the input image DIN. At this time, the images D3Mh, D3Hv, D3Mh, and D3Hv may be simply added to the input image DIN, or weighted may be added. The output image DOUT is output as the final output image of the image processing method in this embodiment. The above is the operation of the adding step ST4 , which is the same as that of the adding unit 4 .

The above is the operation of the image processing method in this embodiment.

As can be seen from the description, the operation of the image processing method in the present embodiment is the same as that of the image processing device in the first embodiment. Therefore, the image processing method in this embodiment has the same effect as the image processing device in Embodiment 1. Moreover, in the image display device shown in FIG. 9 , for example, by implementing the above-mentioned image processing method inside the image processing device U2, the image processed by the above-mentioned image processing method can be displayed on the image display shown in FIG. 9 on the device.

Embodiment 7

In Embodiment 2, it was described that the present invention is implemented using hardware, but it is also possible to implement a part or all of the configuration shown in FIG. 20 using software. The configuration of the image processing device used in this case is the same as that shown in FIG. 44 .

The image processing device in this case is used as the image processing device U2 forming part of the image display device in FIG. 20, the output image DOUT generated after the processing is provided to the monitor U3 of FIG. 9 via the interface 15 as image DU2 for display on the monitor U3.

FIG. 52 is a diagram showing the flow of an image processing method according to Embodiment 7 of the present invention implemented by making the image processing device of FIG. 44 have the same functions as the image processing device according to Embodiment 2. The illustrated image processing method It includes a first intermediate image generation step ST1, a second intermediate image generation step ST2, a first intermediate image processing step ST103M, a second intermediate image processing step ST103H, and an addition step ST4. In addition, the configuration and operation of the first intermediate image generation step ST1 , the second intermediate image generation step ST2 , and the addition step ST4 are the same as those in Embodiment 6, and therefore description thereof will be omitted.

As shown in FIG. 53 , the first intermediate image processing step ST103M includes a magnification factor determining step ST103MA and a pixel value changing step ST103MB.

As shown in FIG. 54 , the second intermediate image processing step ST103H includes an enlargement factor determining step ST103HA and a pixel value changing step ST103HB.

First, the operation of the first intermediate image processing step ST103M will be described in accordance with the flow shown in FIG. 53 .

First, in the first intermediate image processing step ST103M, in the magnification factor determination step ST103MA, the magnification factor for the pixel value of each pixel of the intermediate image D1 is determined. Here, since the intermediate image D1 is composed of the image D1h and the image D1v, the enlargement factor is determined for each pixel of the image D1h and the image D1v. That is, for the image D1h, the magnification for each pixel is determined in the horizontal magnification determination step ST103MAh, and for the image D1v, the magnification for each pixel is determined in the vertical magnification determination step ST103MAv.

FIG. 55 is a flowchart showing the operation of the horizontal magnification determining step ST103MAh. First, in the sign determination step ST52h, the sign (positive or negative) of the pixel value of each pixel of the image D1h is determined. When the sign of the image D1h is positive, the magnification is determined in the first magnification determining step ST511h, and when the sign of the image D1h is negative, the magnification is determined in the second magnification determining step ST512h. The first magnification factor determination step ST511h determines the magnification factor based on the first characteristic described in Embodiment 2 with respect to the pixel value of each pixel of the input image DIN. In the second magnification factor determination step ST512h, the magnification factor is determined based on the second characteristic described in Embodiment 2 with respect to the pixel value of each pixel of the input image DIN. The operation of the horizontal magnification determining step ST103MAh described above is the same as that of the horizontal magnification determining means 103MAh.

FIG. 56 is a flowchart showing the operation of the vertical magnification determining step ST103MAv. First, in the sign determination step ST52v, the sign (positive or negative) of the pixel value of each pixel of the image D1v is determined. When the sign of the image D1v is positive, the magnification is determined in the first magnification determining step ST511v, and when the sign of the image D1h is negative, the magnification is determined in the second magnification determining step ST512v. The first magnification factor determination step ST511v determines a magnification factor based on the first characteristic described in Embodiment 2 with respect to the pixel value of each pixel of the input image DIN. In the second magnification factor determination step ST512v, the magnification factor is determined based on the second characteristic described in Embodiment 2 with respect to the pixel value of each pixel of the input image DIN. The operation of the above vertical magnification determining step ST103MAv is the same as that of the vertical magnification determining means 103MAv.

Then, in the pixel value changing step ST103MB, the pixel value of each pixel of the intermediate image D1 is amplified according to the magnification factor determined in the magnification factor determination step ST103MA. Here, since the intermediate image D1 is composed of the image D1h and the image D1v, the pixel values of each of the image D1h and the image D1v are enlarged. That is, the image D103MBh is generated by enlarging each pixel value of the image D1h at the magnification determined in the horizontal magnification determination step ST103MAh. Then, an image D103MBv is generated based on each pixel value of the magnification enlarged image D1v determined in the vertical magnification determination step ST103MAv. This operation is the same as that of the pixel value changing unit 103MB.

Then, an intermediate image D103M composed of an image D103Mh corresponding to the image D103MBh and an image D103Mv corresponding to the image D103MBv is generated by the first intermediate image processing step ST103M. The above is the operation of the first intermediate image processing step ST103M, and this operation is the same as that of the first intermediate image processing unit 103M.

Next, the operation of the second intermediate image processing step ST103H will be described in accordance with the flow shown in FIG. 54 .

First, in the second intermediate image processing step ST103H, in the magnification factor determination step ST103HA, the magnification factor for the pixel value of each pixel of the intermediate image D2 is determined. Here, since the intermediate image D2 is composed of the image D2h and the image D2v, the enlargement factor is determined for each pixel of the image D2h and the image D2v. That is, for the image D2h, the magnification for each pixel is determined in the horizontal magnification determining step ST103HAh, and for the image D2v, the magnification for each pixel is determined in the vertical magnification determining step ST103HAv. Here, the operation of the horizontal magnification determination step ST103HAh is the same as that of the horizontal magnification determination unit 103HAh, and the operation of the vertical magnification determination step ST103HAv is the same as that of the vertical magnification determination unit 103HAv, and thus their description is omitted.

Then, in the pixel value changing step ST103HB, the pixel value of each pixel of the intermediate image D2 is amplified according to the magnification factor determined in the magnification factor determination step ST103HA. Here, since the intermediate image D2 is composed of the image D2h and the image D2v, the pixel values are enlarged for each of the image D2h and the image D2v. That is, the image D103HBh is generated by enlarging each pixel value of the magnification factor of the image D2h determined in the horizontal magnification factor determination step ST103HAh. Then, the image D103HBv is generated by enlarging each pixel value of the magnification factor of the image D2v determined in the vertical magnification factor determination step ST103HAv. This operation is the same as that of the pixel value changing unit 103HB.

Then, an intermediate image D103H composed of an image D103Hh corresponding to the image D103HBh and an image D103Hv corresponding to the image D103HBv is generated by the second intermediate image processing step ST103H. The above is the operation of the second intermediate image processing step ST103H, which is the same as that of the second intermediate image processing unit 103H.

The above is the operation of the image processing method in this embodiment.

As can be seen from the description, the operation of the image processing method in the present embodiment is the same as that of the image processing device in the second embodiment. Therefore, the image processing method in this embodiment has the same effect as the image processing device in Embodiment 2. Moreover, in the image display device shown in FIG. 9 , for example, by implementing the above-mentioned image processing method inside the image processing device U2, the image processed by the above-mentioned image processing method can be displayed on the image display shown in FIG. 9 on the device.

Embodiment 8

In Embodiment 3, it was described that the present invention is implemented using hardware, but it is also possible to implement a part or all of the configuration shown in FIG. 27 using software. The configuration of the image processing device used in this case is the same as that shown in FIG. 44 .

However, the image processing device in this case is used as the image processing device U202 forming a part of the image display device of FIG. 27, the output image IMGOUT generated after processing is provided to, for example, the monitor U203 of FIG. 31 via the interface 15 as the image DU202 for display on the monitor U203.

FIG. 57 is a diagram showing the flow of an image processing method according to Embodiment 8 of the present invention implemented by making the image processing device of FIG. 44 have the same functions as the image processing device according to Embodiment 3. The image processing method according to Embodiment 8 The method includes a luminance and color difference signal addition step ST200, a first intermediate image generation step ST201, a second intermediate image generation step ST2, a first intermediate image processing step ST203M, a second intermediate image processing step ST203H, and an addition step ST204.

Also, in the image processing method of the eighth embodiment, as in the third embodiment, image processing is performed on the input image IMGIN input in the YCbCr format. That is, the input image IMGIN input in the image input step (not shown) is a color image composed of a signal YIN representing a luminance component (hereinafter referred to as an input luminance image YIN) and signals CRIN and CBIN representing color difference components. A signal CRIN (hereinafter referred to as an input CR image CRIN) represents a Cr component among the color difference components, and a signal CBIN (hereinafter referred to as an input CB image CBIN) represents a Cb component among the color difference components.

The first intermediate image generation step ST201 performs the same processing as that performed on the input image DIN in the first intermediate image generation step ST1 in Embodiment 6 on the input luminance image YIN. In addition, its configuration may be the same as that of the first intermediate image generation step ST1 described in Embodiment 6.

The operation and configuration of the second intermediate image generating step ST2 may be the same as the second intermediate image generating step ST2 described in the sixth embodiment.

As shown in FIG. 58 , the first intermediate image processing step ST203M includes a magnification factor determining step ST203MA and a pixel value changing step ST203MB.

As shown in FIG. 59 , the second intermediate image processing step ST203H includes an enlargement factor determining step ST203HA and a pixel value changing step ST203HB.

First, the operation of the luminance and color difference signal adding step ST200 will be described. The luminance and color difference signal addition step ST200 performs weighted addition of the pixel value of the input luminance image YIN, the absolute value of the pixel value of the input CR image CRIN, and the absolute value of the pixel value of the input CB image CBIN for each pixel to generate a luminance and color difference summation Image YC. The relationship between the luminance and color difference addition image YC, the input luminance image YIN, the input CR image CRIN, and the input CB image CBIN is expressed by Equation (7). This operation is the same as that of luminance and color difference adding section 205 .

The first intermediate image generation step ST201 performs the same processing as that performed on the input image DIN in the first intermediate image generation step ST1 in Embodiment 6 on the input luminance image YIN.

Then, the operation of the second intermediate image generation step ST2 is the same as that of the second intermediate image generation step ST2 described in the sixth embodiment.

Next, the operation of the first intermediate image processing step ST203M will be described in accordance with the flow shown in FIG. 58 .

First, in the first intermediate image processing step ST203M, in the magnification factor determination step ST203MA, the magnification factor for the pixel value of each pixel of the intermediate image D1 is determined. Here, since the intermediate image D1 is composed of the image D1h and the image D1v, the enlargement factor is determined for each pixel of the image D1h and the image D1v. That is, for the image D1h, the magnification for each pixel is determined in the horizontal magnification determination step ST203MAh, and for the image D1v, the magnification for each pixel is determined in the vertical magnification determination step ST203MAv. Here, the operation of the horizontal magnification determination step ST203MAh is the same as that of the horizontal magnification determination unit 203MAh, and the operation of the vertical magnification determination step ST203MAv is the same as that of the vertical magnification determination unit 203MAv, and thus their description is omitted.

Then, in the pixel value changing step ST203MB, the pixel value of each pixel of the intermediate image D1 is amplified according to the magnification factor determined in the magnification factor determination step ST203MA. Here, since the intermediate image D1 is composed of the image D1h and the image D1v, the pixel values of each of the image D1h and the image D1v are enlarged. That is, the image D203MBh is generated by enlarging each pixel value of the magnification factor of the image D1h determined in the horizontal magnification factor determination step ST203MAh. Then, an image D203MBv is generated based on each pixel value of the magnification enlarged image D1v determined in the vertical magnification determination step ST203MAv. This operation is the same as that of the pixel value changing unit 203MB.

Then, an intermediate image D203M composed of an image D203Mh corresponding to the image D203MBh and an image D203Mv corresponding to the image D203MBv is generated by the first intermediate image processing step ST203M. The above is the operation of the first intermediate image processing step ST203M, which is the same as that of the first intermediate image processing unit 203M.

Next, the operation of the second intermediate image processing step ST203H will be described in accordance with the flow shown in FIG. 59 .

First, in the second intermediate image processing step ST203H, in the magnification factor determination step ST203HA, the magnification factor for the pixel value of each pixel of the intermediate image D2 is determined. Here, since the intermediate image D2 is composed of the image D2h and the image D2v, the enlargement factor is determined for each pixel of the image D2h and the image D2v. That is, for the image D2h, the magnification for each pixel is determined in the horizontal magnification determination step ST203HAh, and for the image D2v, the magnification for each pixel is determined in the vertical magnification determination step ST203HAv. Here, the operation of the horizontal magnification determination step ST203HAh is the same as that of the horizontal magnification determination unit 203HAh, and the operation of the vertical magnification determination step ST203HAv is the same as that of the vertical magnification determination unit 203HAv, and thus their description is omitted.

Then, in the pixel value changing step ST203HB, the pixel value of each pixel of the intermediate image D2 is amplified according to the magnification factor determined in the magnification factor determination step ST203HA. Here, since the intermediate image D2 is composed of the image D2h and the image D2v, the pixel values are enlarged for each of the image D2h and the image D2v. That is, the image D203HBh is generated by enlarging each pixel value of the magnification factor of the image D2h determined in the horizontal magnification factor determination step ST203HAh. Then, the image D203HBv is generated by enlarging each pixel value of the image D2v with the magnification determined in the vertical magnification determination step ST203HAv. This operation is the same as that of the pixel value changing section 203HB.

Then, an intermediate image D203H composed of an image D203Hh corresponding to the image D203HBh and an image D203Hv corresponding to the image D203HBv is generated by the second intermediate image processing step ST203H. The above is the operation of the second intermediate image processing step ST203H, which is the same as that of the second intermediate image processing unit 203H.

The addition step ST204 adds the input luminance image YIN, intermediate image D203M, and intermediate image D203H to generate an output luminance image YOUT. Since the intermediate image D203M is composed of the image D203Mh and the image D203Mv, and the intermediate image D203H is composed of the image D203Hh and the image D203Hv, in the addition step ST204, all the images D203Mh, D203Hv, D203Mh, and D203Hv are added to the input luminance image YIN. At this time, the images D203Mh, D203Hv, D203Mh, and D203Hv may be simply added to the input luminance image YIN, or weighted may be added. Then, the output luminance image YOUT is output as the final output image of the image processing method in this embodiment. The above is the operation of the addition step ST204 , which is the same as the operation of the addition unit 204 .

The above is the operation of the image processing method in this embodiment.

As can be seen from the description, the operation of the image processing method in the present embodiment is the same as that of the image processing device in the third embodiment. Therefore, the image processing method in this embodiment has the same effect as the image processing device in Embodiment 3. Moreover, in the image display device shown in FIG. 31, for example, by implementing the above-mentioned image processing method inside the image processing device U202, the image processed by the above-mentioned image processing method can be displayed on the image display shown in FIG. on the device.

Embodiment 9

In Embodiment 4, it was described that the present invention is implemented using hardware, but it is also possible to implement a part or all of the configuration shown in FIG. 34 using software. The configuration of the image processing device used in this case is the same as that shown in FIG. 44 .

However, the image processing device in this case is used as the image processing device U202 forming a part of the image display device of FIG. 34, the output image IMGOUT generated after processing is provided to, for example, the monitor U203 of FIG. 31 via the interface 15 as the image DU202 for display on the monitor U203.

FIG. 60 is a diagram showing the flow of an image processing method according to Embodiment 9 of the present invention implemented by making the image processing device of FIG. 44 have the same functions as the image processing device according to Embodiment 4. The illustrated image processing method It includes a luminance and color difference signal addition step ST200, a first intermediate image generation step ST201, a second intermediate image generation step ST2, a first intermediate image processing step ST303M, a second intermediate image processing step ST303H, and an addition step ST204.

Also, in the image processing method of the ninth embodiment, as in the fourth embodiment, image processing is performed on the input image IMGIN input in the YCbCr format. That is, the input image IMGIN input in the image input step (not shown) is a color image composed of a signal YIN representing a luminance component (hereinafter referred to as an input luminance image YIN) and signals CRIN and CBIN representing color difference components. A signal CRIN (hereinafter referred to as an input CR image CRIN) represents a Cr component among the color difference components, and a signal CBIN (hereinafter referred to as an input CB image CBIN) represents a Cb component among the color difference components.

The operation and configuration of the first intermediate image generation step ST201 are the same as those of the first intermediate image generation step ST201 in the eighth embodiment.

The operation and configuration of the second intermediate image generation step ST2 are the same as those of the second intermediate image generation step ST2 in the eighth embodiment.

As shown in FIG. 61 , the first intermediate image processing step ST303M includes a magnification factor determining step ST303MA and a pixel value changing step ST303MB.

As shown in FIG. 62 , the second intermediate image processing step ST303H includes an enlargement factor determining step ST303HA and a pixel value changing step ST303HB.

Hereinafter, the image processing method of this embodiment will be described. However, the operations of the luminance and color difference signal adding step ST200 , first intermediate image generating step ST201 , second intermediate image generating step ST2 , and adding step ST204 are the same as those described in Embodiment 8, and therefore description thereof will be omitted.

First, the operation of the first intermediate image processing step ST303M will be described in accordance with the flow shown in FIG. 61 .

First, in the first intermediate image processing step ST303M, in the magnification factor determination step ST303MA, the magnification factor for the pixel value of each pixel of the intermediate image D1 is determined. Here, since the intermediate image D1 is composed of the image D1h and the image D1v, the enlargement factor is determined for each pixel of the image D1h and the image D1v. That is, for the image D1h, the magnification for each pixel is determined in the horizontal magnification determination step ST303MAh, and for the image D1v, the magnification for each pixel is determined in the vertical magnification determination step ST303MAv. Here, the operation of the horizontal magnification determination step ST303MAh is the same as that of the horizontal magnification determination unit 303MAh, and the operation of the vertical magnification determination step ST303MAv is the same as that of the vertical magnification determination unit 303MAv, so description thereof is omitted.

Then, in the pixel value changing step ST303MB, the pixel value of each pixel of the intermediate image D1 is amplified according to the magnification factor determined in the magnification factor determination step ST303MA. Here, since the intermediate image D1 is composed of the image D1h and the image D1v, the pixel values of each of the image D1h and the image D1v are enlarged. That is, the image D303MBh is generated by enlarging each pixel value of the image D1h at the magnification determined in the horizontal magnification determination step ST303MAh. Then, an image D303MBv is generated based on each pixel value of the magnification enlarged image D1v determined in the vertical magnification determination step ST303MAv. This operation is the same as that of the pixel value changing unit 303MB.

Then, an intermediate image D303M composed of an image D303Mh corresponding to the image D303MBh and an image D303Mv corresponding to the image D303MBv is generated by the first intermediate image processing step ST303M. The above is the operation of the first intermediate image processing step ST303M, which is the same as that of the first intermediate image processing unit 303M.

Next, the operation of the second intermediate image processing step ST303H will be described in accordance with the flow shown in FIG. 62 .

First, in the second intermediate image processing step ST303H, in the magnification factor determination step ST303HA, the magnification factor for the pixel value of each pixel of the intermediate image D2 is determined. Here, since the intermediate image D2 is composed of the image D2h and the image D2v, the enlargement factor is determined for each pixel of the image D2h and the image D2v. That is, for the image D2h, the magnification for each pixel is determined in the horizontal magnification determination step ST303HAh, and for the image D2v, the magnification for each pixel is determined in the vertical magnification determination step ST303HAv. Here, the operation of the horizontal magnification determination step ST303HAh is the same as that of the horizontal magnification determination unit 303HAh, and the operation of the vertical magnification determination step ST303HAv is the same as that of the vertical magnification determination unit 303HAv, and thus their description is omitted.

Then, in the pixel value changing step ST303HB, the pixel value of each pixel of the intermediate image D2 is amplified according to the magnification factor determined in the magnification factor determination step ST303HA. Here, since the intermediate image D2 is composed of the image D2h and the image D2v, the pixel values are enlarged for each of the image D2h and the image D2v. That is, the image D303HBh is generated by enlarging each pixel value of the image D2h with the magnification determined in the horizontal magnification determination step ST303HAh. Then, the image D303HBv is generated by enlarging each pixel value of the magnification factor of the image D2v determined in the vertical magnification factor determination step ST303HAv. This operation is the same as that of the pixel value changing section 303HB.

Then, an intermediate image D303H composed of an image D303Hh corresponding to the image D303HBh and an image D303Hv corresponding to the image D303HBv is generated by the second intermediate image processing step ST303H. The above is the operation of the second intermediate image processing step ST303H, which is the same as that of the second intermediate image processing unit 303H.

The above is the operation of the image processing method in this embodiment.

As can be seen from the description, the operation of the image processing method in the present embodiment is the same as that of the image processing device in the fourth embodiment. Therefore, the image processing method in this embodiment has the same effect as the image processing device in Embodiment 4. Moreover, in the image display device shown in FIG. 31, for example, by implementing the above-mentioned image processing method inside the image processing device U202, the image processed by the above-mentioned image processing method can be displayed on the image display shown in FIG. on the device.

Embodiment 10

In Embodiment 5, it was described that the present invention is implemented using hardware, but it is also possible to implement a part or all of the configuration shown in FIG. 39 using software. The configuration of the image processing device used in this case is the same as that shown in FIG. 44 .

However, the image processing device in this case is used as the image processing device U202 forming a part of the image display device of FIG. 39, the same processing as the image processing device in FIG. 39 is performed, and the output image IMGOUT generated after processing is provided to the monitor U203 in FIG.

FIG. 63 is a diagram showing the flow of an image processing method according to Embodiment 10 of the present invention implemented by making the image processing apparatus of FIG. 44 have the same functions as the image processing apparatus according to Embodiment 5. The image processing method according to Embodiment 10 The method includes a first intermediate image generation step ST201, a second intermediate image generation step ST2, an addition step ST404, and a color difference increase/decrease step ST405.

Also, in the image processing method of the tenth embodiment, as in the fifth embodiment, image processing is performed on the input image IMGIN input in the YCbCr format. That is, the input image IMGIN input in the image input step (not shown) is a color image composed of a signal YIN representing a luminance component (hereinafter referred to as an input luminance image YIN) and signals CRIN and CBIN representing color difference components. A signal CRIN (hereinafter referred to as an input CR image CRIN) represents a Cr component among the color difference components, and a signal CBIN (hereinafter referred to as an input CB image CBIN) represents a Cb component among the color difference components.

An output image IMGOUT composed of an output luminance image YOUT, an output CR image CROUT, and an output CB image CBOUT, which is generated by processing the input image YIN according to the description shown below, passes through a not-shown image output step as the final output image. output.

The operation and configuration of the first intermediate image generation step ST201 are the same as those of the first intermediate image generation step ST201 in the eighth embodiment.

The operation and configuration of the second intermediate image generation step ST2 are the same as those of the second intermediate image generation step ST2 in the eighth embodiment.

Next, the operation of the addition step ST404 will be described.

The adding step ST404 outputs the result of adding the input luminance image YIN, intermediate image D1, and intermediate image D2 as a high-frequency component added image D404. Then, the result of adding the high-frequency component added image D404 to the input luminance image YIN (that is, the result of adding the intermediate image D1 and the intermediate image D2 ) is output as the output luminance image YOUT.

In addition, since the intermediate image D1 is composed of the image D1h and the image D1v, and the intermediate image D2 is composed of the image D2h and the image D2v, in the addition step ST404, all the images D1h, D1v, D2h, and D2v are added to generate a high-frequency component phase Add image D404. At this time, the images D1h, D1v, D2h, and D2v may be simply added or weighted. Then, the output luminance image YOUT is output as a part of the final output image of the image processing method in this embodiment. The above is the operation of the adding step ST404, which is the same as the operation of the adding means 404.

Next, the operation of the luminance and color difference addition step ST405 will be described in accordance with the flowchart in FIG. 64 .

In the luminance and color difference adding step ST405, first, in the magnification factor determining step ST405A, the magnification factor for each pixel of the input CR image CRIN and the input CB image CBIN is determined from the pixel values at the same coordinates of the high-frequency component added image D404. . The relationship between the pixel value of the high-frequency component added image D404 and the amplification factor determined in the amplification factor determining section 405A is the same as that in the fifth embodiment.

Then, the color difference Cr multiplication step ST405B1 outputs, as an image D405B1, a result obtained by applying a magnification rate given by the magnification rate D405A to the pixel values of the input CR image CRIN.

Then, the color difference Cb multiplication step ST405B2 outputs, as an image D405B2, a result obtained by applying a magnification rate given by the magnification rate D405A to the pixel values of the input CB image CBIN.

Then, the image D405B1 is output as the output CR image CROUT, and the image D405B2 is output as the output CB image CBOUT. The output CR image CROUT and the output CB image CBOUT are used as part of the final output image IMGOUT.

The above is the operation of the color difference amplifying step ST405 , which is the same as that of the color difference amplifying section 405 .

The above is the operation of the image processing method in this embodiment.

As can be seen from the description, the operation of the image processing method in the present embodiment is the same as that of the image processing device in the fifth embodiment. Therefore, the image processing method in this embodiment has the same effect as the image processing device in Embodiment 5. Moreover, in the image display device shown in FIG. 31, for example, by implementing the above-mentioned image processing method inside the image processing device U202, the image processed by the above-mentioned image processing method can be displayed on the image display shown in FIG. on the device.

Label description

1: the first intermediate image generation unit; 2: the second intermediate image generation unit; 3M: the first intermediate image processing unit; 3H: the second intermediate image processing unit; 4: the addition unit; 103M: the first intermediate image processing unit; 103H: the second intermediate image processing unit; 201: the first intermediate image generating unit; 203M: the first intermediate image processing unit; 203H: the second intermediate image processing unit; 204: adding unit; 205: brightness and color difference adding unit; 303M: 1st intermediate image processing unit; 303H: 2nd intermediate image processing unit; 404: addition unit; 405: color difference increase and decrease unit; DIN: input image; D1: intermediate image; D2: intermediate image; D3M: intermediate image; D3H: DOUT: output image; D103M: intermediate image; D103H: intermediate image; IMGIN: input image; YIN: input brightness image; CRIN: input CR image; CBIN: input CB image; D203M: intermediate image; D203H: intermediate image ; IMGOUT: output image; YOUT: output brightness image; CROUT: output CR image; CBOUT: output CB image; D303M: intermediate image; D303: intermediate image; D404: high-frequency component addition image.

Claims (32)

1. An image processing device, the image processing device has: a first intermediate image generating unit that generates a first intermediate image obtained by extracting a component of a specific frequency band of an input image; a second intermediate image generation unit, which generates a second intermediate image based on the first intermediate image; a first intermediate image processing unit, which generates a third intermediate image based on the first intermediate image; a second intermediate image processing unit that generates a fourth intermediate image based on the second intermediate image; and an adding unit that adds the input image, the third intermediate image, and the fourth intermediate image, It is characterized in that, The first intermediate image processing unit amplifies the pixel values of the first intermediate image using a first magnification factor obtained from the pixel values of the input image, or, The second intermediate image processing unit amplifies the pixel values of the second intermediate image using a second magnification factor obtained from the pixel values of the input image. 2. The image processing device according to claim 1, wherein at least one of the first magnification factor and the second magnification factor is determined to be smaller as the pixel value of the input image is larger. 3. The image processing device according to claim 1 or 2, wherein: The first intermediate image generating unit has: a first high-frequency component image generating unit that generates a first high-frequency component image obtained by extracting only high-frequency components above a first frequency of the input image; and A low-frequency component image generating unit extracts only low-frequency components below a second frequency of the first high-frequency component image. 4. The image processing device according to any one of claims 1 to 3, wherein the second intermediate image generating unit has a non-linear processing unit that generates The enlargement factor is a non-linearly processed image obtained by enlarging the pixel values of the first intermediate image. 5. The image processing device according to claim 4, wherein the second intermediate image generation unit has a second high-frequency component image generation unit, and the second high-frequency component image generation unit generates only the A second high-frequency component image obtained by linearly processing high-frequency components above the third frequency of the image. 6. The image processing device according to any one of claims 1 to 5, wherein the first intermediate image processing unit has: a first magnification determining unit that determines the first magnification; and A first pixel value enlarging unit that amplifies the pixel value of the first intermediate image according to the first enlarging factor. 7. The image processing device according to any one of claims 1 to 6, wherein the second intermediate image processing unit has: a second magnification determining unit that determines the second magnification; and A second pixel value enlarging unit that amplifies the pixel value of the second intermediate image according to the second enlarging factor. 8. The image processing device according to claim 3, wherein: The first high-frequency component image generation unit includes a first horizontal high-frequency component image generation unit that uses pixels existing in the horizontal vicinity of each pixel of the input image, generating a first horizontal high frequency component image obtained by extracting high frequency components above the first horizontal frequency, The low-frequency component image generation unit includes a horizontal low-frequency component image generation unit for generating low-frequency components obtained by extracting only low-frequency components below a second horizontal frequency of the first horizontal high-frequency component image. The first horizontal intermediate image. 9. The image processing device according to claim 8, wherein: The first intermediate image includes the first horizontal intermediate image, The first intermediate image processing unit has: a first horizontal magnification determining unit that determines the first horizontal magnification; and A first horizontal pixel value enlarging unit for enlarging the pixel values of the first horizontal intermediate image according to the first horizontal enlarging factor. 10. The image processing device according to claim 9, wherein: The first magnification includes a first horizontal magnification for the first horizontal intermediate image, The larger the pixel value of the first horizontal intermediate image is, the smaller the first horizontal enlargement factor is determined. 11. The image processing device according to claim 8, wherein: The first intermediate image includes the first horizontal intermediate image, The non-linear processing unit includes a horizontal non-linear processing unit that generates a horizontal non-linear image obtained by enlarging each pixel value of the first horizontal intermediate image at a magnification factor that varies depending on the pixel. process images, The second high-frequency component image generation unit includes a second horizontal high-frequency component image generation unit for generating a third horizontal frequency component from which only the horizontal non-linear processing image is extracted. The second horizontal intermediate image obtained from the above high-frequency components. 12. The image processing device according to claim 9, wherein: The horizontal non-linear processing unit has: a horizontal zero-crossing judging unit, which judges a position where the pixel value of the first horizontal intermediate image changes from positive to negative or from negative to positive as a zero-crossing point; and a horizontal direction signal amplification unit that determines an amplification factor for each pixel of the first horizontal direction intermediate image based on a determination result of the horizontal direction zero-crossing determination unit, and amplifies the first horizontal direction using the determined amplification factor. The pixel value of the intermediate image. 13. The image processing device according to claim 12, wherein: The second intermediate image processing unit has: a second horizontal magnification determining unit that determines the second horizontal magnification; and A second horizontal pixel value enlarging unit that amplifies the pixel value of the second horizontal intermediate image based on the second horizontal enlarging factor. 14. The image processing device according to claim 13, wherein: The second magnification includes a second horizontal magnification for the second horizontal intermediate image, The second horizontal magnification factor is determined to be smaller as the pixel value of the second horizontal intermediate image is larger. 15. The image processing device according to claim 3, wherein: The first high-frequency component image generation unit includes a first vertical high-frequency component image generation unit that uses pixels that exist in the vertical direction of each pixel of the input image, generating a first vertical high frequency component image obtained by extracting high frequency components above the first vertical frequency, The low-frequency component image generation unit includes a vertical low-frequency component image generation unit for generating a low-frequency component obtained by extracting only a low-frequency component equal to or lower than a second vertical frequency of the first vertical high-frequency component image. The first vertical intermediate image. 16. The image processing device according to claim 15, wherein: The first intermediate image includes the first vertical intermediate image, The first intermediate image processing unit has: a first vertical magnification determining unit that determines the first vertical magnification; and A first vertical pixel value enlarging unit that amplifies the pixel values of the first vertical intermediate image according to the first vertical enlarging factor. 17. The image processing device according to claim 16, wherein: The first magnification includes a first vertical magnification for the first vertical intermediate image, The first vertical magnification is determined to be smaller as the pixel value of the first vertical intermediate image is larger. 18. The image processing device according to claim 15, wherein: The first intermediate image includes the first vertical intermediate image, The non-linear processing unit includes a vertical non-linear processing unit that generates a vertical non-linear image obtained by enlarging each pixel value of the first vertical intermediate image at a magnification factor that varies depending on the pixel. process images, The second high-frequency component image generation unit includes a second vertical high-frequency component image generation unit that generates a third vertical frequency component that extracts only the non-linearly processed image in the vertical direction. The second vertical intermediate image obtained from the above high-frequency components. 19. The image processing device according to claim 18, wherein: The non-linear processing unit in the vertical direction has: a vertical zero-crossing judging unit, which judges a position where the pixel value of the first vertical intermediate image changes from positive to negative or from negative to positive as a zero-crossing point; and a vertical direction signal amplifying unit, which determines an amplification factor for each pixel of the first vertical direction intermediate image based on the determination result of the vertical direction zero-crossing determination unit, and amplifies the first vertical direction using the determined amplification factor. The pixel value of the intermediate image. 20. The image processing device according to claim 19, wherein: The second intermediate image processing unit has: a second vertical magnification determining unit that determines the second vertical magnification; and A first vertical pixel value enlarging unit that amplifies the pixel values of the second vertical intermediate image according to the second vertical enlarging factor. 21. The image processing device according to claim 20, wherein: the second magnification includes a second vertical magnification for the second vertical intermediate image, The second vertical magnification is determined to be smaller as the pixel value of the second vertical intermediate image is larger. 22. An image processing device, the image processing device having: a first intermediate image generating unit that generates a first intermediate image obtained by extracting a component of a specific frequency band of an input image; a second intermediate image generation unit, which generates a second intermediate image based on the first intermediate image; a first intermediate image processing unit, which processes the first intermediate image and outputs a third intermediate image; a second intermediate image processing unit that processes the second intermediate image to output a fourth intermediate image; and an adding unit that adds the input image, the third intermediate image, and the fourth intermediate image, It is characterized in that, The first intermediate image processing unit has: a first magnification determining unit that determines a first magnification based on pixel values of the input image and pixel values of the first intermediate image; and a first pixel value enlarging unit that amplifies the pixel value of the first intermediate image according to the first enlarging factor, or, The second intermediate image processing unit has: a second magnification determining unit that determines a second magnification based on pixel values of the input image and pixel values of the second intermediate image; and A second pixel value enlarging unit that amplifies the pixel value of the second intermediate image according to the second enlarging factor. 23. An image processing device comprising: a first intermediate image generation unit that generates a first intermediate image obtained by extracting a component of a specific frequency band from an input image composed of a luminance signal and a color difference signal; a second intermediate image generation unit, which generates a second intermediate image based on the first intermediate image; A brightness and color difference adding unit, which synthesizes the brightness signal and the color difference signal of the input image; a first intermediate image processing unit, which generates a third intermediate image based on the first intermediate image; a second intermediate image processing unit that generates a fourth intermediate image based on the second intermediate image; and an adding unit that adds the luminance signal of the input image, the third intermediate image, and the fourth intermediate image, It is characterized in that, The first intermediate image processing unit amplifies the pixel value of the first intermediate image according to a first magnification factor determined according to the output of the luminance-color difference adding unit, or, The second intermediate image processing unit amplifies the pixel value of the second intermediate image according to a second magnification factor determined according to the output of the luminance-color difference adding unit. 24. An image processing device comprising: a first intermediate image generation unit that generates a first intermediate image obtained by extracting a component of a specific frequency band from an input image composed of a luminance signal and a color difference signal; a second intermediate image generation unit, which generates a second intermediate image based on the first intermediate image; A brightness and color difference adding unit, which synthesizes the brightness signal and the color difference signal of the input image; a first intermediate image processing unit, which generates a third intermediate image based on the first intermediate image; a second intermediate image processing unit that generates a fourth intermediate image based on the second intermediate image; and an adding unit that adds the luminance signal of the input image, the third intermediate image, and the fourth intermediate image, It is characterized in that, The first intermediate image processing unit amplifies the pixel value of the first intermediate image according to a first magnification factor determined according to the output of the luminance color difference adding unit and the sign of the pixel value of the first intermediate image, or , The second intermediate image processing unit amplifies the pixel value of the second intermediate image according to a second amplification factor determined according to the output of the luminance-color difference adding unit and the sign of the pixel value of the first intermediate image. 25. A kind of image processing device, this image processing device is input color image, it is characterized in that, this image processing device has: a first intermediate image generating unit that generates a first intermediate image obtained by extracting a component of a specific frequency band from a luminance image representing a luminance signal of the color image; a second intermediate image generation unit, which generates a second intermediate image based on the first intermediate image; an adding unit for generating a high-frequency component addition image obtained by adding the first intermediate image and the second intermediate image, and adding the high-frequency component addition image to the luminance image The output brightness image of ; and The color difference increase/decrease unit increases or decreases each pixel value of a color difference image representing a color difference signal of the color image based on each pixel value of the high frequency component added image. 26. An image display device comprising the image processing device according to claims 1-25. 27. An image processing method, characterized in that the image processing method has: A first intermediate image generation step, generating a first intermediate image obtained by extracting components of a specific frequency band of the input image; A second intermediate image generation step, generating a second intermediate image based on the first intermediate image; A first intermediate image processing step, generating a third intermediate image according to the first intermediate image; A second intermediate image processing step of generating a fourth intermediate image based on the second intermediate image; and an addition step, adding the input image, the third intermediate image and the fourth intermediate image, It is characterized in that, The first intermediate image processing step amplifies the pixel values of the first intermediate image using a first magnification factor obtained from the pixel values of the input image, or, The second intermediate image processing step amplifies the pixel values of the second intermediate image using a second magnification factor obtained from the pixel values of the input image. 28. An image processing method, the image processing method having: A first intermediate image generation step, generating a first intermediate image obtained by extracting components of a specific frequency band of the input image; A second intermediate image generation step, generating a second intermediate image based on the first intermediate image; A first intermediate image processing step, processing the first intermediate image to output a third intermediate image; A second intermediate image processing step, processing the second intermediate image to output a fourth intermediate image; and an addition step, adding the input image, the third intermediate image and the fourth intermediate image, It is characterized in that, The first intermediate image processing step has: a first magnification determining step of determining a first magnification based on the pixel values of the input image and the pixel values of the first intermediate image; and A first pixel value changing step of enlarging the pixel value of the first intermediate image according to the first magnification factor, or, The 2nd intermediate image processing step has: In a second magnification determining step, a second magnification is determined based on the pixel values of the input image and the pixel values of the second intermediate image; and The second pixel value changing step is to enlarge the pixel value of the second intermediate image according to the second enlargement factor. 29. An image processing method, the image processing method having: a first intermediate image generation step, generating a first intermediate image obtained by extracting a component of a specific frequency band from an input image composed of a luminance signal and a color difference signal; A second intermediate image generation step, generating a second intermediate image based on the first intermediate image; Luminance and color difference adding step, synthesizing the luminance signal and the color difference signal of the input image; A first intermediate image processing step, generating a third intermediate image according to the first intermediate image; A second intermediate image processing step of generating a fourth intermediate image based on the second intermediate image; and an addition step of adding the luminance signal of the input image, the third intermediate image, and the fourth intermediate image, It is characterized in that, The first intermediate image processing step amplifies the pixel values of the first intermediate image according to a first magnification factor determined according to the output of the luminance-chromatic-difference adding step, or, The second intermediate image processing step amplifies the pixel value of the second intermediate image based on a second magnification factor determined according to the output of the luminance-color difference adding step. 30. An image processing method, the image processing method having: a first intermediate image generating step of generating a first intermediate image obtained by extracting a component of a specific frequency band from an input image composed of a luminance signal and a color difference signal; A second intermediate image generation step, generating a second intermediate image based on the first intermediate image; Luminance and color difference adding step, synthesizing the luminance signal and the color difference signal of the input image; A first intermediate image processing step, generating a third intermediate image according to the first intermediate image; A second intermediate image processing step of generating a fourth intermediate image based on the second intermediate image; and an addition step of adding the luminance signal of the input image, the third intermediate image, and the fourth intermediate image, It is characterized in that, The first intermediate image processing step amplifies the pixel value of the first intermediate image based on a first magnification factor determined according to the sign of the pixel value of the first intermediate image and the output of the luminance-color difference adding step, or , The second intermediate image processing step amplifies the pixel value of the second intermediate image according to a second magnification factor determined according to the sign of the pixel value of the second intermediate image and the output of the luminance-color difference adding step. 31. An image processing method, which processes color images, is characterized in that, the image processing method has: a first intermediate image generating step of generating a first intermediate image obtained by extracting a component of a specific frequency band from a luminance image representing a luminance signal of the color image; A second intermediate image generation step, generating a second intermediate image based on the first intermediate image; an adding step of generating a high-frequency component addition image obtained by adding the first intermediate image and the second intermediate image, and an output luminance obtained by adding the high-frequency component addition image to the brightness image images; and The color difference increasing/decreasing step increases or decreases each pixel value of a color difference image representing a color difference signal of the color image based on each pixel value of the high frequency component added image. 32. An image display device, characterized in that the image display device displays the image processed by the image processing method according to claims 27-31.

Images