JP5566500B2 - Resolution conversion apparatus and control method thereof - Google Patents

Description

  The present invention relates to a resolution conversion apparatus and a control method thereof.

2. Description of the Related Art Conventionally, there is a technique for performing resolution conversion of digital image data by generating an interpolation pixel by an interpolation method such as a nearest neighbor method, a linear interpolation method, or a cubic convolution interpolation method (Bi-Cubic method). However, in these methods, when an interpolation pixel is generated at an edge portion such as a diagonal line in an image, a jagged line may occur in the diagonal line.
As a conventional technique for solving such a problem, there is a method of determining a pixel value of an interpolation pixel using an interpolation coefficient corresponding to the edge direction when generating an interpolation pixel in an edge portion (for example, (See Patent Documents 1 and 2).

JP 2000-182039 A JP 2000-242774 A

However, in the techniques disclosed in Patent Document 1 and Patent Document 2 described above, an edge detector is used to determine whether or not an interpolation pixel generation position is an edge portion. The thin line portion cannot be accurately detected (for example, erroneous detection, omission of detection, etc.). As a result, the jaggy in the thin line portion cannot be eliminated and the image quality is deteriorated.
In the techniques disclosed in Patent Document 1 and Patent Document 2, an inappropriate direction is detected as an edge direction in an image portion including a noisy portion or a complicated pattern in which diagonal lines intersect. There is a fear.
If a wrong direction is detected as such, the interpolation coefficient is frequently switched in time or space, and thus it may be recognized as a sense of disturbance.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a resolution conversion apparatus capable of performing high-quality resolution conversion on a thin line portion or the like and a control method therefor.

A resolution conversion apparatus according to the present invention is a resolution conversion apparatus that performs resolution conversion by generating an interpolation pixel of an input image, and includes a two-dimensional filter for detecting an edge, and peripheral pixels existing around the interpolation pixel. A pixel value is used to obtain an edge direction and a first reliability representing the certainty of the edge direction, and a two-dimensional filter for detecting a thin line, and pixels of peripheral pixels existing around the interpolation pixel The edge direction corresponding to the higher one of the first reliability and the second reliability is obtained by using the value to obtain the thin line direction and the second reliability representing the certainty of the thin line direction. or the direction determining means for selecting a fine line direction, and interpolation function corresponding to the selected edge direction or fine line direction by the direction determination means, based on the pixel values of the peripheral pixels existing around the interpolation pixel Resolution conversion apparatus, characterized by chromatic and a determining means for determining a pixel value of the interpolation pixel.

A method for controlling a resolution conversion apparatus according to the present invention is a method for controlling a resolution conversion apparatus that performs resolution conversion by generating interpolation pixels of an input image, and includes a two-dimensional filter for detecting an edge, and surroundings of the interpolation pixels. 2 is used to obtain the edge direction and the first reliability representing the certainty of the edge direction using the pixel values of the peripheral pixels existing in the pixel, and a two-dimensional filter for detecting a thin line; Using the pixel values of existing peripheral pixels, the fine line direction and the second reliability representing the certainty of the fine line direction are obtained, and the higher reliability of the first reliability and the second reliability is obtained. and direction determination step of selecting an edge direction or fine line direction corresponding to the time, an interpolation function corresponding to the selected edge direction or fine line direction by the direction determination step, existing around the interpolation pixel Based on the pixel value of the edge pixel, characterized by chromatic and a determining step of determining a pixel value of the interpolation pixel.

  ADVANTAGE OF THE INVENTION According to this invention, the resolution converter which can perform high quality resolution conversion with respect to a thin line part etc., and its control method can be provided.

1 is a block diagram illustrating an example of a functional configuration of a resolution conversion apparatus according to Embodiment 1. FIG. The figure which shows an example of an interpolation pixel and a surrounding pixel. The figure explaining the calculation method of the 1st two-dimensional coefficient. FIG. 3 is a block diagram illustrating an example of a functional configuration of an angle calculation unit according to the first embodiment. The flowchart which shows an example of a process of an edge angle calculation part. The flowchart which shows an example of a process of a thin wire angle calculation part. The figure which shows an example of the two-dimensional filter for detecting an edge and a thin line. The figure which shows the conversion characteristic of Formula 5. The figure which shows the conversion characteristic of Formula 6. The figure which shows an example of a surrounding pixel. FIG. 9 is a block diagram illustrating an example of a functional configuration of an angle calculation unit according to the second embodiment. The flowchart which shows an example of a process of a complicated pattern detection part. The figure which shows an example of the two-dimensional filter for detecting a complicated pattern. The figure which shows the conversion characteristic of Formula 8. The figure which shows an example of a surrounding pixel.

<Example 1>
Hereinafter, a resolution conversion apparatus and a control method thereof according to Embodiment 1 of the present invention will be described with reference to the drawings. The resolution conversion apparatus according to the present embodiment converts the resolution of an input image (input image) by generating interpolation pixels.

FIG. 1 is a block diagram illustrating an example of a functional configuration of the resolution conversion apparatus 100 according to the present embodiment. The resolution conversion apparatus 100 includes a memory 201, a coordinate calculation unit 202, an image cutout unit 203, an angle calculation unit 204, a coefficient calculation unit 205, a coefficient blend unit 206, a convolution operation unit 207,
An image writing unit 208, a memory 209, and the like are included.

  It is assumed that image data (image data of a frame at a certain time when a moving image is a target) is stored in the memory 201. As the memory 201 and the memory 209 (described later), a magnetic recording medium such as a hard disk, a dielectric memory such as a nonvolatile memory, or the like may be applied. One memory may serve as both the memory 201 and the memory 209.

  The image cutout unit 203 accesses the memory 201 and refers to a plurality of pixels (peripheral pixels) existing around the interpolation pixel (specifically, the generation position of the interpolation pixel to be generated). In this embodiment, 16 pixels ((vertical) 4 × (horizontal) 4 pixels; 4 × 4 tap pixels) existing around the interpolation pixel are referred to as peripheral pixels. Interpolated pixels are pixels generated for resolution conversion. In this embodiment, as shown in FIG. 2, the interpolation pixels are four pixels (coordinates (1, 1), (1, 2), (2, 1), (2, 2) inside the 16 pixels. ). The peripheral pixels are not limited to 4 × 4 tap pixels. For example, it may be a 2 × 2 or 8 × 8 pixel, or a 2 × 4 pixel.

  The angle calculation unit 204 calculates the interpolation direction (interpolation angle r) and the reliability z of the interpolation direction (interpolation pixel direction calculation means). The interpolation direction is the direction of the edge or thin line included in the area around the interpolation pixel (peripheral area), and the reliability z is a value representing the probability that the peripheral area includes the edge or thin line in the interpolation direction. . In this embodiment, interpolation is performed using a plurality of two-dimensional filters and pixel values (luminance signals and color signals such as R, G, and B) of 4 × 4 tap pixels cut out (referenced) by the image cutout unit 203. The angle r is calculated. A detailed calculation method of the interpolation angle r and the reliability z will be described later.

  The coefficient calculation unit 205 calculates the first two-dimensional coefficient w [j] [i] using an interpolation function corresponding to the interpolation direction (interpolation angle r) (first coefficient calculation means). Further, in this embodiment, the coefficient calculation unit 205 uses the interpolation function indicating an isotropic distribution (for example, an interpolation function used in the Bi-Cubic method or the like) to generate the second two-dimensional coefficient u [j] [ i] is calculated. j represents the vertical direction and i represents the horizontal coordinate. Note that the second two-dimensional coefficient may be determined in advance. The first two-dimensional coefficient w [j] [i] is calculated using, for example, a three-dimensional interpolation function as shown in FIGS. By using the first two-dimensional coefficient w [j] [i], the weight of the pixel in the interpolation direction (edge direction) is increased, so that the occurrence of jaggy can be suppressed. Specifically, FIG. 3A shows an interpolation function used when generating an interpolation pixel in a region including a 45-degree diagonal line, and each contour line connects coefficients of the same size. Is a line. By using the two-dimensional coefficient obtained from the interpolation function of FIG. 3A, it is possible to generate an interpolated pixel having an appropriate pixel value in a region including a 45-degree oblique line. FIG. 3B shows an interpolation function used when generating an interpolation pixel in an area including a diagonal line of 135 degrees. By using the two-dimensional coefficient obtained from the interpolation function shown in FIG. 3B, it is possible to generate an interpolated pixel having an appropriate pixel value in an area including a 135-degree oblique line.

For example, when the first two-dimensional coefficient w [j] [i] is calculated from the interpolation function shown in FIG. 3A, the position (iy, ix) of the interpolation pixel as shown in FIG. To match the origin of the interpolation function. The value of the function at the position [j] [i] of each input pixel (input image pixel) is defined as a first two-dimensional coefficient w [j] [i].
The coordinate value (ii, ix) is a coordinate value in the input coordinate system when the interpolation pixel is mapped to the input coordinate system, and is calculated by the coordinate calculation unit 202. The input coordinate system is a coordinate system of a space in which an input image is configured.

The coefficient blending unit 206 includes a first two-dimensional coefficient w [j] [i] and a second two-dimensional coefficient u [j
] [I] with a weight corresponding to the reliability z of the interpolation direction calculated by the angle calculation unit 204, thereby calculating a third two-dimensional coefficient t [j] [i] (third Coefficient calculation means).
For example, if the weight is α (0 ≦ α ≦ 1), t [j] [i]

t [j] [i] = w [j] [i] × α + u [j] [i] × (1−α) (Equation 1)

It can be written as In this embodiment, since the peripheral pixels are (vertical) 4 × (horizontal) 4 pixels, 0 ≦ j <4 and 0 ≦ i <4.

In this embodiment, the reliability z in the interpolation direction is expressed with 256 gradations (8 bits accuracy). In that case, the expression 1 can be realized by the following expression 2, for example. Note that the reliability z in the interpolation direction is not limited to 8 bits accuracy.

t [j] [i] = {w [j] [i] × z
+ U [j] [i] × (256-z) +128} >> 8 (Formula 2)

Note that “>> 8” in Equation 2 means 8-bit shift, that is, division by 256.

The convolution operation unit 207 performs a product-sum operation on the third two-dimensional coefficient t [j] [i] output from the coefficient blending unit 206 and the pixel value x [j] [i] of the peripheral pixel (input pixel). . The calculation result is set as the pixel value of the interpolation pixel (interpolation pixel value y) (interpolation pixel value calculation means). The processing of the convolution operation unit 207 can be expressed as, for example, Equation 3.

y = Σ (t [j] [i] × x [j] [i]) (Formula 3)

  The image writing unit 208 records the interpolation pixel value y in the memory 209. The interpolated pixel value y is recorded in the memory 209 in association with the coordinate value (ly, lx) of the output coordinate system. The output coordinate system is a coordinate system of a space in which an image after resolution conversion is configured, and the coordinate value (ly, lx) of the output coordinate system is calculated by the coordinate calculation unit 202. By outputting (reading out) input pixels from the memory 201 and interpolation pixels from the memory 209 with the coordinate values in the output coordinate system, an image after resolution conversion can be displayed on a display device or the like. The coordinate value of the input pixel in the output coordinate system is calculated by the coordinate calculation unit 202 and stored in the memory 201 in association with the input pixel.

  FIG. 4 is a block diagram illustrating an example of a functional configuration of the angle calculation unit 204. The angle calculation unit 204 includes an edge angle calculation unit 501, a fine line angle calculation unit 502, a reliability comparison unit 503, and an angle selection unit 504.

  The edge angle calculation unit 501 determines whether or not the peripheral region includes an edge using the plurality of two-dimensional filters for detecting the edge and the pixel values of the peripheral pixels described above. Further, when the peripheral region includes an edge, the direction of the edge (angle r1) and the first reliability z1 that is the reliability of the direction are calculated. In this embodiment, the edge angle calculation unit 501 uses an F1 filter and an F2 filter shown in FIG. 7 as a plurality of two-dimensional filters. An edge is a portion where there is a large difference in pixel values between regions.

The fine line angle calculation unit 502 determines whether or not the peripheral area includes a fine line by using the plurality of two-dimensional filters for detecting the fine line and the pixel values of the peripheral pixels described above. Further, when the peripheral region includes a thin line, the direction of the thin line (angle r2) and the second reliability z2 that is the reliability of the direction are calculated. In the present embodiment, the fine line angle calculation unit 502 uses an F3 filter, an F4 filter, an F5 filter, and an F6 filter shown in FIG. 7 as a plurality of two-dimensional filters.

  Note that the two-dimensional filter used in the edge angle calculation unit 501 and the thin line angle calculation unit 502 is not limited to the configuration shown in FIG. For example, in this embodiment, the edge angle calculation unit 501 uses two two-dimensional filters for detecting horizontal and vertical edges, but the two-dimensional filter for detecting diagonal edges is used. It may be used.

  The reliability comparison unit 503 determines the reliability of the edge direction (first reliability z1) and the reliability of the direction of the thin line (second reliability z2) output from the edge angle calculation unit 501 and the fine line angle calculation unit 502. Compare. Specifically, when z1> z2, the reliability comparison unit 503 outputs the first reliability z1 as the interpolation direction reliability z and outputs 1 as the interpolation angle selection signal m. In the case of z1 ≦ z2, the reliability comparison unit 503 outputs the second reliability z2 as the reliability z in the interpolation direction and outputs 0 as the interpolation angle selection signal m.

  The angle selection unit 504 determines an interpolation direction according to the interpolation angle selection signal m output from the reliability comparison unit 503. Specifically, when the interpolation angle selection signal m is 1, the angle selection unit 504 selects the edge direction (angle r1) as the interpolation direction (interpolation angle r). When the interpolation angle selection signal m is 0, the angle selection unit 504 selects the direction of the thin line (angle r2) as the interpolation direction.

  That is, in this embodiment, the higher reliability and the direction corresponding to the first reliability z1 and the second reliability z2 are the reliability of the interpolation direction and the interpolation direction, respectively. In the present embodiment, when z1 = z2, the second reliability z2 is set as the reliability of the interpolation direction and the direction corresponding thereto is set as the interpolation direction. Either the reliability z1 or the second reliability z2 may be used as the reliability in the interpolation direction. Either the edge direction or the thin line direction may be used as the interpolation direction.

Hereinafter, the processing flow of the edge angle calculation unit 501 will be described in detail. FIG. 5 is a flowchart illustrating an example of processing of the edge angle calculation unit 501.
First, the edge angle calculation unit 501 performs a product-sum operation on the pixel values of the surrounding pixels (4 × 4 tap pixels) and the F1 filter, and a product-sum operation on the pixel values of the surrounding pixels and the F2 filter (steps S600 and S601). .

  The F1 filter is a two-dimensional filter for detecting lateral edges. When the peripheral area includes a horizontal edge, the degree of matching between the pixel value of the peripheral pixel and the F1 filter, that is, the absolute value of the calculation result (F1out) of the product-sum operation of the pixel value of the peripheral pixel and the F1 filter Is a large value. The F2 filter is a two-dimensional filter for detecting a vertical edge. When the peripheral region includes a vertical edge, the pixel value of the peripheral pixel and the calculation result (F2out) of the product-sum operation of the F2 filter are obtained. The absolute value is a large value. When the peripheral area includes an oblique edge, | F1out | and | F2out | are both intermediate values. When the peripheral area does not include an edge, | F1out | and | F2out | Is also a small value. Therefore, when the peripheral region includes an edge, the sum of | F1out | and | F2out | is a large value. When the peripheral region does not include an edge, the sum of | F1out | and | F2out | . Therefore, after step S601, the edge angle calculation unit 501 calculates the sum of the absolute values of the calculation results in steps S600 and S601 (step S602), and compares the calculated value with the threshold value a (step S603). ). The threshold value a is, for example, the minimum value that can be recognized that the peripheral area includes an edge.

When the value calculated in step S602 is equal to or smaller than the threshold value a (step S604: NO), the edge angle calculation unit 501 determines that the peripheral region does not include an edge (step S60).
5). When the value calculated in step S602 is larger than the threshold value a (step S604: YES), the edge angle calculation unit 501 determines that the peripheral area includes an edge, and calculates the edge direction (angle) (step). S606). Since the F1 filter is a two-dimensional filter for detecting the edge in the horizontal direction, and the F2 filter is a two-dimensional filter for detecting the edge in the vertical direction, F1out and F2out are respectively the horizontal component of the edge, It can be interpreted as a vertical component. Therefore, the direction of the edge can be obtained by the following formula 4. It should be noted that the calculation method of the edge direction is naturally different if the configuration of the two-dimensional filter to be used is different. In this embodiment, the angle θ obtained from the following expression is not used as the edge direction, and the edge direction (angle r1) is selected from 0, 45, 90, and 135 degrees according to the value of the angle θ. Select.

θ = tan ⁻¹ (F2out / F1out) (Formula 4)

  Specifically, when the angle θ is greater than 26 degrees and smaller than 63 degrees (step S607: YES), the edge angle calculation unit 501 determines that the edge direction (angle r1) is 45 degrees (step S608). When the angle θ is greater than 116 degrees and smaller than 153 degrees (step S609: YES), the edge angle calculation unit 501 determines that the edge direction (angle r1) is 135 degrees (step S610). If the angle θ is outside the above range (outside the range studied in steps S607 and S609), the edge angle calculation unit 501 determines that the edge direction (angle r1) is 0 degree or 90 degrees. Specifically, it is determined that the angle r1 is 90 degrees when the angle θ is 63 degrees or more and 116 degrees or less, and the angle r1 is 0 degrees when the angle θ is 26 degrees or less or 153 degrees or more.

Further, after the sum of | F1out | and | F2out | is calculated (that is, after the processing of step S602), the edge angle calculation unit 501 performs the processing of step S612 separately from the above processing. In step S612, the edge angle calculation unit 501 calculates the reliability of the edge direction (first reliability z1). When the lower threshold of the sum of | F1out | and | F2out | is the threshold a (threshold used in step S603) and the upper threshold is the threshold d, the first reliability z1 is

z1 = 256 × (| F1out | + | F2out | −a) / (d−a) (Formula 5)

It can be expressed as. The threshold value d is, for example, the minimum value at which it is recognized that the peripheral area surely includes an edge.

  FIG. 8 shows the conversion characteristics of Expression 5 (the relationship between the first reliability z1 (vertical axis), the sum of | F1out |, and | F2out | (horizontal axis)). As the sum of | F1out | and | F2out | Therefore, by calculating the first reliability z1 as shown in Equation 5, as the sum of | F1out | and | F2out | increases as shown in FIG. 8, the higher first reliability z1 can be obtained.

Next, the processing flow of the fine line angle calculation unit 502 will be described in detail. FIG. 6 is a flowchart showing an example of processing of the fine line angle calculation unit 502.
First, the thin line angle calculation unit 502 performs a product-sum operation on the pixel values of the surrounding pixels and the F3 filter, and a product-sum operation on the pixel values of the surrounding pixels and the F4 filter (steps S700 and S701). Similarly, the product-sum operation of the pixel values of the peripheral pixels and the F5 filter, and the product-sum operation of the pixel values of the peripheral pixels and the F6 filter are performed (steps S702 and S703).

The F3 filter is a two-dimensional filter for detecting a fine line in the direction of 45 degrees. When the peripheral region includes a 45-degree thin line, the absolute value of the pixel value of the peripheral pixel and the calculation result (F3out) of the product-sum operation of the F3 filter is a large value. The F4 filter is a two-dimensional filter for detecting a fine line in the direction of 135 degrees, and the calculation result of the product-sum operation of the pixel values of the surrounding pixels and the F4 filter is F4out. The F5 filter is a two-dimensional filter for detecting a thin line in the direction of 90 degrees, and the calculation result of the product-sum operation of the peripheral pixel and the F5 filter is F5out. The F6 filter is a two-dimensional filter for detecting a fine line in the direction of 0 degree, and the calculation result of the product-sum operation of the pixel values of the surrounding pixels and the F6 filter is F6out.

  Following step S703, the fine line angle calculation unit 502 compares | F3out |, | F4out |, | F5out |, | F6out |, and the threshold value b (step S704). The threshold value b is, for example, the minimum value that can be recognized that the peripheral area includes a thin line. If all the absolute values (all of | F3out |, | F4out |, | F5out |, | F6out |) are equal to or less than the threshold value b (step S705: NO), the process proceeds to step S706. In step S706, the fine line angle calculation unit 502 determines that the peripheral area does not include a fine line. If at least one of | F3out |, | F4out |, | F5out |, | F6out | is greater than the threshold value b, the fine line angle calculation unit 502 determines that the peripheral region includes a fine line, and proceeds to step S707. . In the present embodiment, all absolute values are compared with one threshold value, but may be compared with a different threshold value for each absolute value (each used two-dimensional filter).

  In step S707, the fine line angle calculation unit 502 detects the absolute value having the largest value among the absolute values larger than the threshold value b (the pixel value of the surrounding pixels and the absolute value of the calculation result of the product-sum operation of the two-dimensional filter). . When | F3out | is the maximum (step S708: YES), the fine line angle calculation unit 502 calculates the reliability of the direction of the fine line (second reliability z2) using | F3out | (step S709). ). Then, it is determined that the direction of the fine line (angle r2) is 45 degrees (step S710). If | F4out | is the maximum (step S711: YES), the fine line angle calculation unit 502 calculates the second reliability z2 using | F4out | (step S712), and the direction of the fine line (angle r2). ) Is determined to be 135 degrees (step S713). When | F5out | is the maximum (step S714: YES), the fine line angle calculation unit 502 calculates the second reliability z2 using | F5out | (step S715), and the direction of the fine line (angle r2). ) Is determined to be 90 degrees (step S716). When | F6out | is the maximum (step S714: NO), the fine line angle calculation unit 502 calculates the second reliability z2 using | F6out | (step S717), and the direction of the fine line (angle r2). ) Is determined to be 0 degrees (step S718).

An example of the second reliability calculation method in steps S709, S712, S715, and S717 will be specifically described. When the maximum absolute value (the absolute value detected in step S707) is | F3-6 |, the lower threshold is the threshold b (threshold used in step S704), and the upper threshold is the threshold e, the second reliability z2 is

z2 = 256 × (| F3-6 | −b) / (eb) (Formula 6)

It can be expressed as. The threshold value e is, for example, a minimum value that is recognized to ensure that the peripheral area includes a thin line.

  FIG. 9 shows the conversion characteristics of Equation 6 (the relationship between the second reliability z2 (vertical axis) and the maximum absolute value | F3-6 | (horizontal axis)). The larger the absolute value, the greater the degree of matching between the two-dimensional filter and the pixel values of the peripheral pixels, and there is a high possibility that the peripheral pixels constitute a thin line in that direction. Therefore, by calculating the second reliability z2 as shown in Equation 6, the higher the | F3-6 |, the higher the second reliability z2 can be obtained as shown in FIG.

A specific example of the calculation process of the interpolation direction (interpolation angle r) and the reliability z of the interpolation direction will be described with reference to FIG. FIG. 10 is a diagram illustrating an example of peripheral pixels (4 × 4 tap pixels). In FIG. 10, the numerical values described in the frames represent the pixel values of the respective peripheral pixels. In the following description, the threshold values a and b are 0, the threshold value d is 8184, and the threshold value e is 12276.

  In the example of FIG. 10A, the edge direction (angle r1) is determined to be 0 degrees from the pixel value of the surrounding pixels and the result of the product-sum operation of the two-dimensional filter, and the direction of the thin line (angle r2) is 0. It is judged as a degree. Since | F1out | + | F2out | = 8184, the first reliability z1 (reliability in the edge direction) = 256 is obtained from Equation 5. Since the maximum value | F3-6 | = 12276, the second reliability z2 (reliability in the direction of the thin line) = 256 is obtained from Equation 6. Therefore, in the example of FIG. 10A, the interpolation direction (interpolation angle r) is 0 degree, and the reliability z of the interpolation direction is 256.

  In the example of FIG. 10B, the angles r1 and r2 are both determined to be 45 degrees. Since | F1out | + | F2out | = 8184, the first reliability z1 = 256 is obtained from Equation 5. Since the maximum value | F3-6 | = 6138, the second reliability z2 = 128 from Equation 6. Therefore, in the example of FIG. 10B, the interpolation angle r is 45 degrees, and the reliability z in the interpolation direction is 256.

  In the example of FIG. 10C, the angles r1 and r2 are both determined to be 45 degrees. Since | F1out | + | F2out | = 2046, the first reliability z1 = 64 is obtained from Equation 5. Since the maximum value | F3-6 | = 6138, the second reliability z2 = 128 from Equation 6. Therefore, in the example of FIG. 10C, the interpolation angle r is 45 degrees, and the reliability z in the interpolation direction is 128.

  In the example of FIG. 10D, it is determined that the peripheral region does not include an edge, and the angle r2 is determined to be 45 degrees. Since | F1out | + | F2out | = 0, the first reliability z1 = 0 is obtained from Equation 5. Since the maximum value | F3-6 | = 12276, the second reliability z2 = 256 from Equation 6. Therefore, in the example of FIG. 10D, the interpolation angle r is 45 degrees, and the reliability z in the interpolation direction is 256.

  As described above, according to the present embodiment, the first two-dimensional coefficient obtained from the interpolation function corresponding to the interpolation direction and the second two-dimensional coefficient obtained from the interpolation function used in the Bi-Cubic method or the like. Are combined with a weight according to the reliability of the interpolation direction. Then, the pixel value of the interpolation pixel is calculated using the obtained third two-dimensional coefficient. Thereby, jaggy can be suppressed. Furthermore, according to the present embodiment, it is determined not only whether or not the peripheral area includes an edge, but also whether or not the peripheral area includes a thin line, and the interpolation direction and its reliability are determined from the determination results. Therefore, jaggy in the thin line portion can be suitably suppressed, and high-quality resolution conversion without interfering with erroneous direction detection can be performed.

<Example 2>
Hereinafter, a resolution conversion apparatus and a control method thereof according to Embodiment 2 of the present invention will be described with reference to the drawings. When an interpolation pixel is generated by the conventional method or the method according to the first embodiment, an incorrect direction is an interpolation direction in a region including a noisy portion or a complicated pattern (complex pattern) where diagonal lines intersect. There is a risk of being. Such erroneous calculation of the interpolation direction causes deterioration in image quality. Therefore, in the present embodiment, a configuration will be described in which the pixel value of the interpolation pixel is determined in consideration of the case where the surrounding area includes a complex pattern.

The configuration of the resolution conversion apparatus according to the present embodiment is the same as that of the first embodiment, and only the configuration of the angle calculation unit is different (in order to distinguish from the first embodiment, the angle calculation unit according to the present embodiment is denoted by reference numeral 304. ). Therefore, in the following, the angle calculation unit 304 will be described in detail, and description of other configurations will be omitted.

  FIG. 11 is a block diagram illustrating an example of a functional configuration of the angle calculation unit 304. The angle calculation unit 304 includes an edge angle calculation unit 501, a fine line angle calculation unit 502, a reliability comparison unit 503, an angle selection unit 504, a complex pattern detection unit 1201, and a reliability correction unit 1202. In addition, the same code | symbol is attached | subjected about the same function as Example 1, and description is abbreviate | omitted.

The complex pattern detection unit 1201 determines whether or not the peripheral area includes a complex pattern using a plurality of two-dimensional filters for detecting the complex pattern and the pixel values of the peripheral pixels. Further, when the peripheral area includes a complex pattern, a third reliability z3 that is a probability that the peripheral area includes the complex pattern is calculated. In this embodiment, the complex pattern detection unit 1201 uses an F7 filter, an F8 filter, an F9 filter, an F10 filter, and an F11 filter shown in FIG. 13 as a plurality of two-dimensional filters. The complex patterns that can be detected by the F7 filter to F11 filter are different. Further, the configuration of the two-dimensional filter to be used is not limited to these.
Then, in accordance with the degree of matching between the two-dimensional filter and the pixel values of the surrounding pixels, a complex presence / absence signal n indicating whether or not the surrounding area includes a complex pattern and a third reliability z3 are output.

The reliability correction unit 1202 corrects the reliability z output from the reliability comparison unit 503. Specifically, the reliability correction unit 1202 determines whether or not the surrounding area includes a complex pattern based on the complexity presence / absence signal n output from the complex pattern detection unit 1201. When a complicated pattern is not included, the reliability z output from the reliability comparison unit 503 is output as it is as the corrected reliability (interpolation direction reliability) zh. When a complex pattern is included, the reliability z4 corresponding to the third reliability z3 is output as the corrected reliability zh. In this case, the higher the possibility that the peripheral area includes a complex pattern, the more unreliable the interpolation direction is. Therefore, it is preferable to lower the corrected reliability zh as the third reliability z3 increases. . For example, a value obtained by subtracting the third reliability z3 from the maximum value that the reliability can take may be set as the reliability z4 (Formula 7).

z4 = 256-z3 (Formula 7)

Hereinafter, the processing flow of the complex pattern detection unit 1201 will be described in detail. FIG. 12 is a flowchart illustrating an example of processing of the complex pattern detection unit 1201.
First, the complex pattern detection unit 1201 performs a product-sum operation on the pixel values of the surrounding pixels and the F7 filter, and a product-sum operation on the pixel values of the surrounding pixels and the F8 filter (steps S1300 and S1301). Similarly, the product-sum operation of the pixel value of the peripheral pixel and the F9 filter, the product-sum operation of the pixel value of the peripheral pixel and the F10 filter, and the product-sum operation of the pixel value of the peripheral pixel and the F11 filter are performed (steps S1302, S1303, S1304). ).

The large absolute value of the result of the product-sum operation of the two-dimensional filter (F7 filter to F11 filter) for detecting the complex pattern and the pixel values of the peripheral pixels is that the complex pattern that can be detected by the two-dimensional filter is It means that the area is likely to contain.
Therefore, after step S1304, the complex pattern detection unit 1201 compares the absolute value of the calculation result in steps S1300 to S1304 with the threshold c (step S1305). The threshold value c is, for example, the minimum value that can be recognized that the peripheral area includes a complex pattern. In the present embodiment, all absolute values are compared with one threshold value, but may be compared with a different threshold value for each absolute value (each used two-dimensional filter).

All absolute values (| F7out |, | F8out |, | F9out |, | F10
When all of out | and | F11out |) is equal to or less than the threshold value c (step S1306: NO), the process proceeds to step S1307. In step S1307, the complex pattern detection unit 1201 determines that the peripheral region does not include a complex pattern, and sets the complexity presence / absence signal n to zero. If at least one of | F7out |, | F8out |, | F9out |, | F10out |, | F11out | is greater than the threshold c, the process proceeds to step S1308.

In step 1308, the complex pattern detection unit 1201 detects the absolute value having the largest value. When | F7out | is maximum (step S1309: YES), the complex pattern detection unit 1201 uses the | F7out | to determine the reliability (third reliability z3) that the peripheral area includes the complex pattern. Calculate (step S1310). If | F8out | is the maximum (step S1312: YES), the third reliability z3 is calculated from | F8out | (step S1313). If | F9out | is the maximum (step S1314: YES), the third reliability z3 is calculated from | F9out | (step S1315). If | F10out | is the maximum (step S1316: YES), the third reliability z3 is calculated from | F10out | (step S1317). If | F11out | is the maximum (step S1316: NO), the third reliability z3 is calculated from | F11out | (step S1318). After calculating the third reliability z3, the process proceeds to step S1311.
In step S1311, the complex pattern detection unit 1201 determines that the surrounding area includes a complex pattern, and sets the complexity presence / absence signal n to 1.

An example of the third reliability calculation method in steps S1310, S1313, S1315, S1317, and S1318 will be specifically described. If the maximum absolute value (the absolute value detected in step S1308) is | F7-11 |, the lower threshold is the threshold c (threshold used in step S1305), and the upper threshold is the threshold g, the third reliability z3 is

z3 = 256 × (| F7-11 | −c) / (g−c) (Formula 8)

It can be expressed as. The threshold value g is, for example, a minimum value at which the peripheral area is recognized as surely including a complex pattern.

  FIG. 14 shows the conversion characteristic of Equation 8 (the relationship between the third reliability (vertical axis) and the maximum absolute value | F7-11 | (horizontal axis)). The greater the absolute value, the greater the degree of matching between the two-dimensional filter and the pixel values of the peripheral pixels, and there is a high possibility that the peripheral region includes a complex pattern. Therefore, by calculating the third reliability z3 as shown in Equation 8, the higher third reliability z3 can be obtained as | F7-11 | increases as shown in FIG.

  A specific example of the calculation process of the interpolation direction (interpolation angle r) and the corrected reliability zh will be described with reference to FIG. FIG. 15 is a diagram illustrating an example of peripheral pixels (4 × 4 tap pixels). In FIG. 15, the numerical values described in the frames represent the pixel values of each peripheral pixel. In the following description, the threshold values a, b, and c are set to 0, the threshold values d and g are set to 8184, and the threshold value e is set to 12276. 15 (a) to 15 (d) are the same as FIGS. 10 (a) to 10 (d), description of the interpolation angle r calculation process is omitted.

  In the example of FIG. 15A, since | F7out |, | F8out |, | F9out |, | F10out |, and | F11out | are all equal to or less than the threshold value c, the complex presence / absence signal n is 0. Therefore, in the example of FIG. 15A, the interpolation angle r is 0 degree and the reliability in the interpolation direction is 256, as in the first embodiment.

In the example of FIG. 15B, the maximum value | F7-11 |
Becomes 1, and the third reliability z3 = 64 from Equation 8. Therefore, in the example of FIG. 15B, the interpolation angle r is 45 degrees as in the first embodiment, and the reliability z in the interpolation direction is 256-64 = 192.

  In the example of FIG. 15C, since the maximum value | F7-11 | = 1023, the complexity presence / absence signal n is 1, and the third reliability z3 = 32 from Equation 8. Therefore, in the example of FIG. 15C, the interpolation angle r is 45 degrees as in the first embodiment, and the reliability z in the interpolation direction is 256−32 = 224.

  In the example of FIG. 15D, since the maximum value | F7-11 | = 4092 is obtained, the complexity presence / absence signal n is 1, and the third reliability z3 = 128 from Equation 8. Therefore, in the example of FIG. 15D, the interpolation angle r is 45 degrees as in the first embodiment, and the reliability z in the interpolation direction is 256-128 = 128.

  In the example of FIG. 15E, it is determined that the peripheral area does not include an edge. Since | F3out | and | F4out | both have the maximum value | F3-6 |, the angle r2 is determined to be 45 degrees or 135 degrees. Whether the angle r2 is set to 45 degrees or 135 degrees depends on the order of processing in steps S708 and S711 in FIG. 6 (in the example in FIG. 6, the processing in step S708 precedes the processing in step S711). The angle r2 is 45 degrees). Since | F1out | + | F2out | = 0, the first reliability z1 = 0 is obtained from Equation 5. Since the maximum value | F3-6 | = 12276, the second reliability z2 = 256 from Equation 6. As a result, the reliability comparison unit 503 outputs 45 degrees or 135 degrees as the interpolation angle r, and outputs 256 as the reliability z in the interpolation direction.

  In the example of FIG. 15E, the surrounding area includes a complex pattern. If such reliability z is used as it is, an incorrect value is calculated as the pixel value of the interpolation pixel. However, in the example of FIG. 15E, since the maximum value | F7-11 | = 8184, the complexity presence / absence signal n is 1, and the third reliability z3 from equation 8 is z3 = 256. Then, the value of reliability z in the interpolation direction is corrected. Specifically, the value of reliability z4 (256-256 = 0) corresponding to the third reliability z3 is set as the corrected reliability zh. As a result, in the third two-dimensional coefficient t [j] [i] used for calculating the pixel value of the interpolation pixel, the first two-dimensional coefficient w calculated using the interpolation function corresponding to the interpolation angle r. [J] [i] is not taken into consideration, and the erroneous calculation as described above can be suppressed.

  In the example of FIG. 15F, it is determined that the peripheral region does not include an edge, and the angle r2 is determined to be 135 degrees. Since | F1out | + | F2out | = 0, the first reliability z1 = 0 is obtained from Equation 5. Since the maximum value | F3-6 | = 12276, the second reliability z2 = 256 from Equation 6. As a result, the reliability comparison unit 503 outputs 135 degrees as the interpolation angle r, and outputs 256 as the reliability z in the interpolation direction.

  In the example of FIG. 15F, the surrounding area includes a complex pattern. If such reliability z is used as it is, an incorrect value is calculated as the pixel value of the interpolation pixel. However, in the example of FIG. 15F, since the maximum value | F7-11 | = 6138, the complexity presence / absence signal n is 1, and from Equation 8, the third reliability z3 is z3 = 192. And the value (256-192 = 64) of the reliability z4 according to the 3rd reliability z3 is made into the reliability zh after correction | amendment. As a result, a third two-dimensional coefficient t [j] [i] obtained by combining the two-dimensional coefficient w [j] [i] and the two-dimensional coefficient u [j] [i] with a weight of 1: 3 is obtained. Thus, the erroneous calculation as described above can be suppressed.

As described above, according to the present embodiment, the reliability of the final interpolation direction is determined after considering whether or not the surrounding area includes a complex pattern. As a result, in addition to the effects described in the first embodiment, jaggies can be suitably suppressed even for complex pattern portions, and high-quality resolution conversion can be performed without a sense of interference due to erroneous detection of directions.

DESCRIPTION OF SYMBOLS 100 Resolution converter 204 Angle calculation part 205 Coefficient calculation part 206 Coefficient blend part 207 Convolution operation part 501 Edge angle calculation part 502 Fine line angle calculation part 503 Reliability comparison part 504 Angle selection part

Claims (10)

A resolution conversion device that converts resolution by generating interpolation pixels of an input image,
Using the two-dimensional filter for detecting the edge and the pixel values of the peripheral pixels existing around the interpolation pixel, the edge direction and the first reliability representing the probability of the edge direction are obtained, and the thin line Using the two-dimensional filter for detecting the pixel value and the pixel values of the peripheral pixels existing around the interpolated pixel, a fine line direction and a second reliability representing the probability of the fine line direction are obtained, Direction determining means for selecting an edge direction or a thin line direction corresponding to the higher one of the first reliability and the second reliability ;
Determining means for determining a pixel value of the interpolated pixel based on an interpolation function corresponding to the edge direction or thin line direction selected by the direction determining means and a pixel value of surrounding pixels existing around the interpolated pixel; ,
Resolution conversion apparatus, characterized by have a. The determining means includes a first coefficient relating to an interpolation function corresponding to the selected edge direction or thin line direction according to the higher one of the first reliability and the second reliability, and the like. and a second coefficient of the isotropic interpolation function weighted synthesis, the resolution conversion apparatus according to claim 1, wherein the determining the pixel value of the interpolation pixel. The direction determination means includes
Using a two-dimensional filter for detecting a complex pattern and the pixel values of the surrounding pixels, a third reliability representing the probability that the surrounding area includes a complex pattern is obtained ,
The determining means includes
When the surrounding area includes a complex pattern, the isotropic interpolation function and the first coefficient related to the interpolation function corresponding to the selected edge direction or thin line direction are selected according to the third reliability. The resolution conversion apparatus according to claim 2 , wherein the second coefficient is weighted and synthesized to determine a pixel value of the interpolation pixel. In the case where the surrounding area includes a complex pattern, the determining unit corresponds to the selected edge direction or thin line direction according to a value obtained by subtracting the third reliability from the maximum value that can be taken by the reliability. 4. The resolution according to claim 3 , wherein a pixel value of the interpolation pixel is determined by weighting and combining the first coefficient related to the interpolation function and the second coefficient corresponding to the isotropic interpolation function. Conversion device. The second factor, the resolution conversion apparatus according to any one of claims 2 to 4, characterized in that a two-dimensional coefficient calculated using the interpolation function used in Bi-Cubic method. A method for controlling a resolution conversion apparatus that converts resolution by generating interpolation pixels of an input image,
Using the two-dimensional filter for detecting the edge and the pixel values of the peripheral pixels existing around the interpolation pixel, the edge direction and the first reliability representing the probability of the edge direction are obtained, and the thin line Using the two-dimensional filter for detecting the pixel value and the pixel values of the peripheral pixels existing around the interpolated pixel, a fine line direction and a second reliability representing the probability of the fine line direction are obtained, A direction determining step of selecting an edge direction or a thin line direction corresponding to the higher one of the first reliability and the second reliability ;
A determination step of determining a pixel value of the interpolation pixel based on an interpolation function corresponding to the edge direction or the thin line direction selected in the direction determination step and a pixel value of a peripheral pixel existing around the interpolation pixel; ,
The method of resolution conversion apparatus characterized by have a. In the determining step, according to a higher reliability of the first reliability and the second reliability, a first coefficient relating to an interpolation function corresponding to the selected edge direction or thin line direction ; The method of controlling a resolution conversion apparatus according to claim 6 , wherein a pixel value of the interpolated pixel is determined by weighting and combining a second coefficient related to an isotropic interpolation function. In the direction determination step,
Using a two-dimensional filter for detecting a complex pattern and the pixel values of the surrounding pixels, a third reliability representing the certainty that the surrounding area includes a complex pattern is obtained ,
In the determination step,
When the surrounding area includes a complex pattern, the isotropic interpolation function and the first coefficient related to the interpolation function corresponding to the selected edge direction or thin line direction are selected according to the third reliability. 8. The method of controlling a resolution conversion apparatus according to claim 7 , wherein the second coefficient is weighted and synthesized to determine a pixel value of the interpolation pixel. In the determining step, when the peripheral region includes a complex pattern, the selected edge direction or the fine line direction corresponds to a value obtained by subtracting the third reliability from the maximum value that the reliability can take. 9. The resolution according to claim 8 , wherein the first coefficient relating to the interpolation function and the second coefficient relating to the isotropic interpolation function are weighted and synthesized to determine a pixel value of the interpolation pixel. Control method of conversion device. The second factor is the control method of resolution conversion apparatus according to any one of claims 7-9, characterized in that a two-dimensional coefficient calculated using the interpolation function used in Bi-Cubic method .

Images