CN109510920B

CN109510920B - Pixel interpolation circuit and pixel interpolation method

Info

Publication number: CN109510920B
Application number: CN201710826909.6A
Authority: CN
Inventors: 李国豪; 杨得炜
Original assignee: Himax Technologies Ltd
Current assignee: Himax Technologies Ltd
Priority date: 2017-09-14
Filing date: 2017-09-14
Publication date: 2021-03-23
Anticipated expiration: 2037-09-14
Also published as: CN109510920A

Abstract

A pixel interpolation circuit comprises a pixel extraction circuit, a weight processing circuit, a compensation circuit and an output circuit, wherein the pixel extraction circuit is used to extract multiple input pixels from an input pixel array; the weight processing circuit is coupled to the pixel extraction circuit and is used to assign at least one weight value to each of the multiple input pixels respectively; the compensation circuit is used to perform a compensation operation on at least one of the multiple weight values corresponding to the multiple input pixels; and the output circuit is coupled to the compensation circuit and is used to perform an interpolation operation according to the multiple weight values corresponding to the multiple input pixels after the compensation operation to output an output pixel in an output pixel array, wherein the resolution of the output pixel array is higher than that of the input pixel array.

Description

Pixel interpolation circuit and pixel interpolation method

Technical Field

The present invention relates to a pixel interpolation circuit, and more particularly, to a pixel interpolation circuit having a double-edge (double) filtering function and related method.

Background

In image processing, a Bilateral Filter (Bilateral Filter) is a nonlinear Filter for smoothing an image, and is different from a traditional image smoothing algorithm, the Bilateral Filter not only uses geometric closeness (represented by coordinates) between pixels, but also considers brightness/color difference (represented by energy value) between pixels, so that the Bilateral Filter can effectively remove noise on the image under the condition of considering two different weight values, and simultaneously stores edge information on the image.

Disclosure of Invention

It is therefore one of the objectives of the claimed invention to provide a pixel interpolation circuit with bilateral filtering function and related method to solve the above-mentioned problems.

According to an embodiment of the present invention, a pixel interpolation circuit is disclosed, which comprises a pixel extraction circuit, a weight processing circuit, a compensation circuit and an output circuit, wherein the pixel extraction circuit is used for extracting a plurality of input pixels in an input pixel array; the weight processing circuit is coupled to the pixel extraction circuit and is used for respectively distributing at least one weight value to each pixel in the plurality of input pixels; the compensation circuit is used for performing compensation operation on at least one of a plurality of weighted values corresponding to the plurality of input pixels; and the output circuit is coupled to the compensation circuit and is used for executing interpolation operation according to the weight values corresponding to the input pixels after the compensation operation so as to output an output pixel in an output pixel array, wherein the resolution of the output pixel array is higher than that of the input pixel array.

According to an embodiment of the present invention, a pixel interpolation method is disclosed, comprising: extracting a plurality of input pixels in an input pixel array; respectively allocating at least one weight value to each pixel in the plurality of input pixels; performing a compensation operation on at least one of a plurality of weighted values corresponding to the plurality of input pixels; and after the compensation operation, performing an interpolation operation according to the weight values corresponding to the input pixels to output an output pixel in an output pixel array, wherein the resolution of the output pixel array is higher than that of the input pixel array.

Drawings

FIG. 1 is a diagram of a pixel interpolation circuit according to an embodiment of the invention.

FIG. 2 is a diagram illustrating an energy intensity weight value and a coordinate weight value according to an embodiment of the invention.

FIG. 3 is a diagram illustrating a compensation operation performed according to an energy intensity weight value and a coordinate weight value according to a first embodiment of the present invention.

FIG. 4 is a diagram illustrating a compensation operation performed according to an energy intensity weight value and a coordinate weight value according to a second embodiment of the present invention.

Fig. 5 is a schematic diagram of an output circuit according to an embodiment of the invention.

Detailed Description

Certain terms are used throughout the description and following claims to refer to particular components. As one of ordinary skill in the art will appreciate, manufacturers may refer to a component by different names. This specification and the claims that follow do not intend to distinguish between components that differ in name but not function. In the following description and in the claims, the terms "include" and "comprise" are used in an open-ended fashion, and thus should be interpreted to mean "include, but not limited to. Furthermore, the term "coupled" is used herein to encompass any direct or indirect electrical connection, such that if a first device is coupled to a second device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.

Fig. 1 is a schematic diagram of a pixel interpolation circuit 100 according to an embodiment of the invention, as shown IN fig. 1, the pixel interpolation circuit 100 receives an input pixel array IN and outputs an output pixel array OUT through operation of its internal circuit, wherein the resolution of the output pixel array OUT is higher than that of the input pixel array IN, it should be noted that the invention is not limited to the position of the pixel interpolation circuit 100 IN image processing, that is, the input pixel array IN is not limited to raw image data extracted by a camera, a scanner, etc., the input pixel array IN may be a pixel array that has undergone known image processing, such as gamma processing and YUV processing, and the output pixel array OUT is not limited to the final result of image processing, that is, the output pixel array OUT may be further processed by other image processing procedures, and is not limited to direct display on a display. As shown in fig. 1, the pixel interpolation circuit 100 includes a pixel extraction circuit 110, a weight processing circuit 120, a compensation circuit 130, and an output circuit 140. The pixel extraction circuit 110 is used for extracting a plurality of pixels IN the input pixel array IN for interpolation, for example, the pixel extraction circuit 110 extracts the pixel P after receiving the input pixel array IN₁₁、P₁₂、P₂₁、P₂₂And (2) an energy value (or pixel value) of (c), and (d) a pixel P₁₁、P₁₂、P₂₁、P₂₂The weight processing circuit 120, the compensation circuit 130 and the output circuit 140 are sequentially used to process the pixel OP of the output pixel array OUT₁₁It should be noted that the present invention is not limited to the pixel interpolation implementation, i.e., the present invention is not limited to the extraction of only the pixel P₁₁、P₁₂、P₂₁、P₂₂To generate a pixel OP₁₁Although various pixel interpolation methods may be used to generate the pixel OP11, those skilled in the art should readily understand that other embodiments of pixel interpolation can be used, and the present invention will focus on the description of bilateral filtering for brevity. The pixel extraction circuit 110 is further used for extracting the pixel P₁₁、P₁₂、P₂₁、P₂₂A center pixel is extracted for subsequent bilateral filtering, and in the present embodiment, a pixel P is set₂₂As the center pixel for subsequent bilateral filtering operations. The weight processing circuit 120 processes the weight according to the center pixel (pixel P in this embodiment)₂₂) Position with other pixels and pixel P₁₁、P₁₂、P₂₁、P₂₂To the pixel P by the pixel value of₁₁、P₁₂、P₂₁、P₂₂Each pixel in (1) is assigned a coordinate weight COR and an energy intensity weight INT for bilateral filtering. The compensation circuit 130 is used for compensating the pixel P₁₁、P₁₂、P₂₁、P₂₂The pixel interpolation circuit 100 of the present invention can save the divider used in the conventional bilateral filtering to greatly reduce the production cost and power consumption, and details about the distribution of the coordinate weight value COR and the energy intensity weight value INT and the compensation operation will be discussed in the following paragraphs. The output circuit 140 is used for receiving the pixel P after the compensation operation₁₁、P₁₂、P₂₁、P₂₂And according to the pixel P₁₁、P₁₂、P₂₁、P₂₂The corresponding coordinate weight COR, energy intensity weight INT and pixel P₁₁、P₁₂、P₂₁、P₂₂To perform an interpolation operation to output a pixel OP₁₁. It should be noted that the weight processing circuit 120 is used for the pixel P in the present embodiment₁₁、P₁₂、P₂₁、P₂₂Each pixel in the image is assigned a coordinate weight value COR and an energy intensity weight value INT, however, the invention is not limited to the assignment of coordinate weight value COR and energy intensity weight value INTWhile only one circuit can be used to perform the weight processing, in other embodiments, the weight processing circuit 120 can be two independent processing circuits, which respectively assign the coordinate weight COR and the energy intensity INT to the P₁₁、P₁₂、P₂₁、P₂₂. In addition, in fig. 1, the weight processing circuit 120 and the compensation circuit 130 are two independent circuit blocks, however, in other embodiments, the weight processing circuit 120 and the compensation circuit 130 may be integrated into a single circuit block to perform the distribution of the coordinate weight value COR and the energy intensity weight value INT and complete the compensation operation at the same time.

FIG. 2 is a diagram illustrating distribution of energy intensity weight INT and coordinate weight COR according to an embodiment of the present invention, as described in the embodiment of FIG. 1, for extracting a pixel P in a pixel extraction circuit 110₁₁、P₁₂、P₂₁、P₂₂And will pixel P₂₂After setting as the center pixel, the weight processing circuit 120 calculates the pixel P₁₁、P₁₂、P₂₁、P₂₂And a central pixel P₂₂And according to the energy difference with the central pixel P₂₂The energy difference of the pixel P is distributed by a specific step distance₁₁、P₁₂、P₂₁、P₂₂Corresponding energy intensity weight INT, e.g. for each pixel P₁₁、P₁₂、P₂₁、P₂₂The energy value of the pixel is determined by the gray scale intensity of the pixel, and the digital unit of the specific gradation pattern also represents the gray scale intensity. As shown in FIG. 2, when the pixel P is known₁₁、P₁₂、P₂₁、P₂₂And a central pixel P₂₂After the energy differences of (1) are 239, 112 and 0, the pixels P are assigned according to the assignment of the specific pitches₁₁、P₁₂、P₂₁、P₂₂The energy intensity weight INT of (1) is 0, 1792, 1024, 1792, wherein the specific pitch is defined by an algorithm in the image processing process, which is not a key point of the present invention and is not a limitation of the present invention, and the specific pitch can be allocated to the pixels P according to different allocation manners according to different algorithms₁₁、P₁₂、P₂₁、P₂₂Is high in energyA degree weight value INT; in addition, the weight processing circuit 120 is based on the pixel P₁₁、P₁₂、P₂₁、P₂₂And a central pixel P₂₂To assign a coordinate weight value COR, as shown in fig. 2, pixel P₁₁、P₁₂、P₂₁、P₂₂The coordinate weight values COR of (1) are 576, 1728 and 5184, obviously, the closer the pixel P22 is to the center, the higher the coordinate weight value COR, and the algorithm used for assigning the coordinate weight value COR is not a limitation of the present invention, that is, the pixel P is₁₁、P₁₂、P₂₁、P₂₂The coordinate weight value COR and the energy intensity weight value INT of (a) are only illustrated as an example in this embodiment. The weight processing circuit 120 is distributing the pixel P₁₁、P₁₂、P₂₁、P₂₂After the coordinate weight COR and the energy intensity weight INT are calculated, the corresponding pixel P can be calculated separately₁₁、P₁₂、P₂₁、P₂₂The sum of the two weight values of (2), pixel P₁₁、P₁₂、P₂₁、P₂₂The sum of the two

weights

576, 3520, 2752, and 6976, and the sum of 576+3520+2752+6976 is 13824, and referring to the following bilateral filtering formula, the parameters included therein will not be described in detail since the bilateral filtering formula should be well known in the prior art:

as can be seen from the formula, the bilateral filtering includes a normalization (normalization) operation, which is expressed by the parameter WP in the formula, and it is obvious from the formula that in order to achieve the normalization effect, the circuit implementation must include a divider, and the implementation of the divider always involves multiple transistors, which will increase the circuit area and power consumption, and in order to save the production cost and power consumption, the compensation circuit 130 is directed to the pixel P₁₁、P₁₂、P₂₁、P₂₂The sum of the two weight values is compensated to obtain the pixel P₁₁、P₁₂、P₂₁、P₂₂To the nearest power of 2, for example, in this embodiment, pixel P₁₁、P₁₂、P₂₁、P₂₂The sum of the two weight values of (a) is 13824, which is closest to a power of 2 of 16384, where the difference is 2560, so that the compensation circuit 130 will add the difference 2560 to the pixel P₁₁、P₁₂、P₂₁、P₂₂Among the corresponding weight values. FIG. 3 is a diagram illustrating the compensation operation performed according to the energy intensity weight INT and coordinate weight COR according to the first embodiment of the present invention, wherein the compensation circuit 130 adds the difference 2560 to the pixel P as described above₁₁、P₁₂、P₂₁、P₂₂The compensation circuit 130 adds 2560 to the pixel P in the embodiment shown in FIG. 3₂₂In the corresponding energy intensity weighted value INT, make the pixel P₂₂The corresponding energy intensity weight INT is 1792+2560 ═ 4352, so that the pixel P₁₁、P₁₂、P₂₁、P₂₂The sum of the two corresponding weight values is 16384, which is a power of 2, to achieve normalization and save the circuit cost of bilateral filtering, note that the pixel P is required to be matched₁₁、P₁₂、P₂₁、P₂₂The compensation circuit 130 is not limited to adding the difference to the pixel P₂₂In the corresponding energy intensity weight INT, FIG. 4 is a schematic diagram illustrating a compensation operation performed according to the energy intensity weight INT and the coordinate weight COR according to a second embodiment of the present invention, in which the compensation circuit 130 averagely adds the difference 2560 to the pixel P₁₁、P₁₂、P₂₁、P₂₂Corresponding energy intensity weighted value INT, so that the pixel P₁₁、P₁₂、P₂₁、P₂₂The corresponding energy intensity weight INT values are 640, 2432, 1664, and 2432, respectively, such that the pixel P is₁₁、P₁₂、P₂₁、P₂₂The sum of the two corresponding weight values is still 16384, which is a power of 2, so that the normalization effect can be achieved, and the circuit cost required by bilateral filtering can be saved. Need toIt is noted that the difference 2560 is not limited to be added to the energy intensity weight INT, the description of the compensation operation of the embodiments of fig. 3 and 4 is only an example, the invention is not limited to the way the compensation circuit 130 adds the difference, obviously, the difference 2560 can be added to the pixel P singly by taking the embodiments of fig. 3 and 4 as examples₁₁、P₁₂、P₂₁、P₂₂One of the corresponding energy intensity weight values INT or pixel P₁₁、P₁₂、P₂₁、P₂₂The difference 2560 can be divided into two, three or even more compensation values to be added to the pixel P₁₁、P₁₂、P₂₁、P₂₂Any corresponding weight values.

The compensation circuit 130 is for the pixel P₁₁、P₁₂、P₂₁、P₂₂After the compensation operation is performed on the corresponding energy intensity weight value INT and coordinate weight value COR to make the sum of the weight values be a power of 2, the output circuit 140 performs an interpolation operation according to the compensated energy intensity weight value INT and coordinate weight value COR, fig. 5 is a schematic diagram of the output circuit 140 according to an embodiment of the present invention, and returning to the embodiment of fig. 3, the compensation circuit 130 adds the difference 2560 to the pixel P₂₂After the corresponding energy intensity weight INT, the pixel P₁₁、P₁₂、P₂₁、P₂₂The corresponding compensated energy intensity weight INT and coordinate weight COR are input to the output circuit 140, as shown in fig. 5, the output circuit 140 includes

multipliers

141 and 142 and an adder 143, the

multipliers

141 and 142 respectively multiply the coordinate weight COR and the compensated energy intensity weight INT by the corresponding pixel value to output a first output pixel value (e.g., P) of each pixel_11,first、P_12,first、P_21,first、P_22,first) And a second output pixel value (P as shown)_11,second、P_12,second、P_21,second、P_22,second) The adder 143 adds all the first output pixel values and the second output pixel values to obtain the output pixel OP in the output array OUT₁₁Thereby completing the interpolation operation and outputting the pixel OP₁₁I.e. with a bilateral filtering effect.

In the invention, the compensation circuit is used for compensating the energy intensity weighted value and the coordinate weighted value corresponding to the pixel, so that the sum of the energy intensity weighted value and the coordinate weighted value corresponding to the pixel can be a power of 2, the normalization step can be simplified, the circuit cost and the power loss can be saved, and the interpolation operation of the output circuit is matched to ensure that the output pixel has the bilateral filtering effect.

The above-mentioned embodiments are merely preferred embodiments of the present invention, and all equivalent changes and modifications made by the claims of the present invention should be covered by the scope of the present invention.

List of reference numerals

100-pixel interpolation circuit

110 pixel extraction circuit

120 weight processing circuit

130 compensation circuit

140 output circuit

IN input pixel array

OUT output pixel array

P₁₁、P₁₂、P₂₁、P₂₂Pixel

OP₁₁Output pixel

P_11,first、P_12,first、P_21,first、P_22,firstFirst output pixel value

P_11,second、P_12,second、P_21,second、P_22,secondSecond output pixel value

141. 142 multiplier

143 adder

Claims

1. A pixel interpolation circuit, comprising:

a pixel extraction circuit for extracting a plurality of input pixels in an input pixel array;

a weight processing circuit coupled to the pixel extraction circuit, wherein the weight processing circuit is configured to assign at least one weight value to each of the plurality of input pixels;

a compensation circuit for performing a compensation operation on at least one of the plurality of weight values corresponding to the plurality of input pixels; and

an output circuit coupled to the compensation circuit, wherein the output circuit is used for performing an interpolation operation according to the weight values corresponding to the input pixels after the compensation operation to output an output pixel in an output pixel array, wherein the resolution of the output pixel array is higher than that of the input pixel array,

wherein a sum of the plurality of weight values is not a power of 2 before the compensation operation, and the compensation operation comprises:

adjusting at least one of the weighted values corresponding to the input pixels so that the sum of the weighted values is a power of 2 after the compensation operation.

2. The pixel interpolation circuit of claim 1, wherein the at least one weight value comprises at least one of a coordinate weight value and an energy intensity weight value.

3. The pixel interpolation circuit of claim 2, wherein the at least one weighting value comprises the energy intensity weighting value, and the weighting processing circuit assigns the energy intensity weighting value to each of the plurality of input pixels at a specific pitch according to pixel values of the plurality of input pixels, respectively, wherein the specific pitch is generated according to gray-scale intensity.

4. A method of pixel interpolation, comprising:

extracting a plurality of input pixels in an input pixel array;

respectively allocating at least one weight value to each pixel in the plurality of input pixels;

performing a compensation operation on at least one of a plurality of weighted values corresponding to the plurality of input pixels; and

after the compensation operation, an interpolation operation is performed according to the weight values corresponding to the input pixels to output an output pixel in an output pixel array, wherein the resolution of the output pixel array is higher than that of the input pixel array,

5. The method of claim 4, wherein the at least one weight value comprises at least one of a coordinate weight value and an energy intensity weight value.

6. The method of claim 5, wherein the at least one weight value comprises the energy intensity weight value, and the step of separately assigning at least one weight value to each of the plurality of input pixels comprises:

the energy intensity weight values are respectively allocated to the input pixels in a specific gradation mode according to the pixel values of the input pixels, wherein the specific gradation mode is generated according to the gray scale intensity.