BR112013007203B1 - MOTION COMPENSATION METHOD - Google Patents

Abstract

METHOD AND APPARATUS FOR INTERPOLATING AN IMAGE AGAINST THE SMOOTHING AND COMPUTER READABLE RECORDING MEDIA A method for interpolating an image by determining interpolation filter coefficients is provided. A method for interpolating an image in contrast to smoothing includes differently selecting an interpolation filter based on a sub-pel unit interpolation location and a smoothness among interpolation filters to generate at least one localized sub-pel unit pixel value. between pixels of entire pel unit; and generating the at least one sub-pel unit pixel value by interpolating pixel values of the sub-pel unit pixels using the selected interpolation filter.

Description

TECHNICAL FIELD

[0001] The present invention relates to predictive coding using motion compensation.

FUNDAMENTALS OF THE TECHNIQUE

[0002] In typical image encoding and decoding methods, to encode an image, an illustration is divided into macro blocks. After that, prediction coding is performed in each macro block using inter prediction, or intra prediction.

[0003] Inter prediction refers to a method of compressing an image by removing temporal redundancy between images and its representative example is motion estimation coding. In motion estimation coding, each block of a current illustration is predicted using at least one reference illustration. A reference block that is most similar to a current block is found in a predetermined search range using a predetermined evaluation function.

[0004] A current block is predicted based on a reference block, and a residual block obtained by subtracting from the current block a prediction block generated as a prediction result is encoded. In this case, to more accurately perform prediction, interpolation is performed on a reference illustration search range, sub-pel unit pixels smaller than integer pel unit pixels are generated, and inter prediction is performed in the generated sub-pel unit pixels.

DISCLOSURE OF THE INVENTION TECHNICAL PROBLEM

[0005] The present invention provides a method and apparatus for determining interpolation filter coefficients, appropriate in response to image characteristics in order to generate a sub-pel unit pixel by interpolating entire pel unit pixels.

TECHNICAL SOLUTION

[0006] In accordance with one aspect of the present invention, there is provided a method of interpolating an image in return for smoothing, the method including differently selecting an interpolation filter based on a sub-pel unit interpolation location and a smoothing among the interpolation filters to generate at least one sub-pel unit pixel value between whole pel unit pixels; and generating the at least one sub-pel unit pixel value by interpolating pixel values of the entire pel unit pixels using the selected interpolation filter.

ADVANTAGEOUS EFFECTS

[0007] When a video is encoded and decoded, as a high-quality image is generated by interpolating a reference frame and motion estimation and compensation are performed based on the high-quality image, the accuracy of inter prediction can be increased . Furthermore, as a smoothing interpolation filter is used to reduce high-frequency components in an interpolation result and thus a smoother interpolation result is obtained, high-frequency components can be removed and coding efficiency image and image decoding can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram of an image interpolation apparatus in accordance with an embodiment of the present invention;

[0009] Figure 2 is a diagram for describing a relationship between an entire pel unit and a sub-pel unit;

[00010] Figure 3 is a diagram illustrating adjacent whole pel unit pixels to be referenced in order to determine a sub-pel unit pixel value, in accordance with an embodiment of the present invention;

[00011] Figures 4A to 4C are diagrams illustrating examples of whole pel unit pixels to be referred to in order to determine a sub-pel unit pixel value, in accordance with an embodiment of the present invention;

[00012] Figure 5 is a graph of a smoothing parameter of a smoothing interpolation filter, in accordance with an embodiment of the present invention;

[00013] Figure 6 is a graph of a spline function usable by a smoothing interpolation filter, in accordance with an embodiment of the present invention;

[00014] Figure 7 is a flowchart of an image interpolation method in accordance with an embodiment of the present invention;

[00015] Figures 8A to 8C are tables showing filter coefficients of 12-lead interpolation filters determined based on a smoothing parameter and an interpolation location, in accordance with embodiments of the present invention;

[00016] Figures 9A to 9C are tables showing filter coefficients of 6-lead interpolation filters determined based on a smoothing parameter and an interpolation location, in accordance with embodiments of the present invention;

[00017] Figure 10 is a table showing the filter coefficients of 6-lead interpolation filters determined for chroma pixels based on a smoothing parameter and an interpolation location, in accordance with an embodiment of the present invention;

[00018] Figure 11 is a table showing filter coefficients of the smoothing interpolation filters determined differently based on a color component and an image interpolation location, in accordance with an embodiment of the present invention;

[00019] Figures 12A to 12C are tables showing the filter coefficients of smoothing interpolation filters based on an image interpolation location and a scale factor, in accordance with embodiments of the present invention;

[00020] Figure 13A is a block diagram of a video coding apparatus using a smoothing interpolation filter, in accordance with an embodiment of the present invention;

[00021] Figure 13B is a block diagram of a video coding apparatus using a smoothing interpolation filter, in accordance with an embodiment of the present invention;

[00022] Figure 14A is a flowchart of an image coding method using a smoothing interpolation filter, in accordance with an embodiment of the present invention;

[00023] Figure 14B is a flowchart of an image decoding method using a smoothing interpolation filter, in accordance with an embodiment of the present invention;

[00024] Figure 15 is a diagram for describing a concept of coding units according to an embodiment of the present invention;

[00025] Figure 16 is a block diagram of an image encoder based on coding units, according to an embodiment of the present invention;

[00026] Figure 17 is a block diagram of an image decoder based on coding units, according to an embodiment of the present invention;

[00027] Figure 18 is a diagram illustrating deeper coding units according to depths, and partitions, according to an embodiment of the present invention;

[00028] Figure 19 is a diagram for describing a relationship between a coding unit and transformation units, in accordance with an embodiment of the present invention;

[00029] Figure 20 is a diagram for describing coding information of coding units corresponding to a coded depth, in accordance with an embodiment of the present invention;

[00030] Figure 21 is a diagram of deeper coding units according to depths, according to an embodiment of the present invention;

[00031] Figures 22 to 24 are diagrams for describing a relationship between coding units, prediction units, and transformation units, in accordance with an embodiment of the present invention;

[00032] Figure 25 is a diagram for describing a relationship between a coding unit, a prediction unit or a partition, and a transformation unit, according to a coding mode information from Table 1;

[00033] Figure 26 is a flowchart of a video coding method using a smoothing interpolation filter based on coding units having a tree structure, in accordance with an embodiment of the present invention; It is

[00034] Figure 27 is a flowchart of a video decoding method using a smoothing interpolation filter based on coding units having a tree structure, in accordance with an embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

[00035] In accordance with one aspect of the present invention, there is provided a method of interpolating an image in return for smoothing, the method including differently selecting an interpolation filter based on a sub-pel unit interpolation location and a smoothing between the interpolation filters to generate a sub-pel unit pixel value located between whole pel unit pixels; and generating the at least one sub-pel unit pixel value by interpolating pixel values of the entire pel unit pixels using the selected interpolation filter.

[00036] The interpolation filter may include filter coefficients for transforming the entire pel unit pixels based on a plurality of basic functions and inversely transforming a plurality of generating coefficients as a result of the transformation.

[00037] The interpolation filter may include filter coefficients having the smoothness determined based on a distance between the interpolation location and the entire pel unit pixels. The interpolation filters may include filter coefficients having smoothness determined based on a distance between the interpolation location and the entire pel unit pixels adjacent to the interpolation location.

[00038] To interpolate the entire pel unit pixels in a spatial domain, the interpolation filter may include filter coefficients obtained by combining a filter for performing transformation and inverse transformation using the plurality of basic functions, and a window function , and the window function can be symmetric with respect to the interpolation location.

[00039] To interpolate the entire pel unit pixels in a spatial domain, the interpolation filter may include filter coefficients obtained by combining a filter for performing transformation and inverse transformation using a plurality of basic functions, and a smoothing parameter , and the smoothing parameter can control at least one of a smoothing speed and a smoothing range.

[00040] The interpolation filter may include filter coefficients based on a spline function. The interpolation filter may include filter coefficients to maximize a low frequency response of the interpolation filter based on a polynomial function.

[00041] Selection of the interpolation filter may include selecting an interpolation filter including filter coefficients scaled to integers, from among the interpolation filters, and generating the at least one sub-pel unit pixel value may include normalizing the at least one sub-pel unit pixel value generated using the selected interpolation filter, based on a scale factor.

[00042] Selection of the interpolation filter may include differently selecting an interpolation filter based on pixel characteristics among the interpolation filters, and generating the at least one sub-pel unit pixel value may include generating at least one sub-pel unit pixel value using the differently selected interpolation filter based on the pixel characteristics.

[00043] In accordance with another aspect of the present invention, there is provided an apparatus for interpolating an image in return for smoothing, the apparatus including a filter selector for differently selecting an interpolation filter based on a unit interpolation location. sub-pel and a smoothness among the interpolation filters to generate at least one sub-pel unit pixel value located between whole pel unit pixels; and an interpolator for generating the at least one sub-pel unit pixel value by interpolating pixel values of the entire pel unit pixels using the selected interpolation filter.

[00044] According to another aspect of the present invention, there is provided a method of interpolating an image in return for a color component, the method including differently selecting an interpolation filter based on a sub-unit interpolation location. pel and a color component of a current pixel among the interpolation filters to generate at least one sub-pel unit pixel value located between the whole pel unit pixels; and generating the at least one sub-pel unit pixel value by interpolating pixel values of the entire pel unit pixels using the selected interpolation filter.

[00045] Selection of the interpolation filter may include, to interpolate a chroma pixel, selecting an interpolation filter having a stronger smoothness than that of an interpolation filter for a luma pixel, among the interpolation filters.

[00046] The interpolation filter having stronger smoothness than that of the interpolation filter for a luma pixel may be one of a filter including filter coefficients for smoothing the entire pel unit pixels, transforming the entire pel unit pixels. pel smoothed by using a plurality of basic functions, and inversely transforming various coefficients generated as a result of the transformation; a filter obtained by combining filter coefficients to perform transformation and inverse transformation by using the plurality of basic functions, and window function coefficients to perform low-pass filtering; a filter including filter coefficients for further smoothing the boundary whole pel unit pixels based on a boundary condition of a spline function; and a filter including filter coefficients for maximizing a low frequency response of an interpolation filter based on a polynomial function.

[00047] In accordance with another aspect of the present invention, there is provided an apparatus for interpolating an image in return for a color component, the apparatus including a filter selector for differentially selecting an interpolation filter based on an interpolation location of sub-pel unit and a color component of a current pixel among the interpolation filters to generate at least one sub-pel unit pixel value located between the entire pel unit pixels; and an interpolator for generating the at least one sub-pel unit pixel value by interpolating the pixel values of the entire pel unit pixels using the selected interpolation filter.

[00048] According to another aspect of the present invention, there is provided a video encoder utilizing an image interpolation filter, the video encoder including an encoder for differentially selecting an interpolation filter based on a subunit interpolation location. -pel and a smoothness among the interpolation filters stored in the video encoder; with respect to each block of an input illustration, perform predictive coding to generate at least one sub-pel unit pixel value by interpolating pixel values of the entire pel unit pixels using the selected interpolation filter, and perform transformation and quantization on a prediction result of the prediction coding; an output unit for producing a bit stream generated by performing entropy coding on quantized transformation coefficients and coding information; and a storage means for storing the filter coefficients of the interpolation filters.

[00049] According to another aspect of the present invention, there is provided a video decoder using an image interpolation filter, the video decoder including a receiver and extractor for receiving an encoded bit stream from a video and extracting coding information. and encoded data from a video image by performing entropy decoding and bit stream analysis; a decoder to perform inverse quantization and inverse transformation on the quantized transformation coefficients of the encoded data of a current block of the images, differently selecting an interpolation filter based on a sub-pel unit interpolation location and a smoothness among the interpolation filters stored in the video decoder; performing predictive decoding to generate at least one sub-pel unit pixel value by interpolating pixel values of the entire pel unit pixels using the selected interpolation filter, and reconstructing the illustration; and a storage means for storing the filter coefficients of the interpolation filters.

[00050] According to another aspect of the present invention, a computer readable recording medium is provided having recorded thereon a computer program for carrying out the above method.

MODE FOR INVENTION

[00051] In the following description, an “image” can broadly refer to a moving image such as a video, as well as a still image.

[00052] Image interpolation considering smoothing, according to an embodiment of the present invention, is disclosed with reference to Figures 1 to 12C. Furthermore, encoding and decoding video using a smoothing interpolation filter, in accordance with an embodiment of the present invention, is disclosed with reference to Figures 13A to 27. Specifically, encoding and decoding video using a smoothing interpolation filter based in coding units having a tree structure, in accordance with an embodiment of the present invention, are disclosed with reference to Figures 15 to 27.

[00053] Image interpolation considering smoothing, and a smoothing interpolation filter; in accordance with embodiments of the present invention; will now be described in detail with reference to Figures 1 to 12C.

[00054] Figure 1 is a block diagram of an image interpolation apparatus 10 in accordance with an embodiment of the present invention.

[00055] The image interpolation apparatus 10 considering smoothing includes a filter selector 12 and an interpolator 14. Operations of the filter selector 12 and the interpolator 14 of the image interpolation apparatus 10 can be controlled cooperatively by means of a processing processor. video encoding, a central processing unit (CPU), and a graphics processor.

[00056] The image interpolation apparatus 10 may receive an input image and may generate sub-pel unit pixel values by interpolating entire pel unit pixels. The input image can be a sequence of illustrations, an illustration, a frame, or blocks of a video.

[00057] Filter selector 12 may differentially select an interpolation filter to generate at least one sub-pel unit pixel value located between whole pel units, based on a sub-pel unit interpolation location and a smoothness .

[00058] The interpolator 14 may interpolate entire pel unit pixels adjacent to the sub-pel unit interpolation location using the interpolation filter selected by filter selector 12, thereby generating sub-pel unit pixel values. Interpolation filtering of pel integer unit pixels to generate sub-pel unit pixel values may include interpolation filtering of pel integer unit reference pixel values including pel integer unit pixels adjacent to the unit interpolation location sub-pel in a region supported by the interpolation filter.

[00059] An interpolation filter according to one embodiment may include filter coefficients for transforming entire pel unit reference pixels based on a plurality of basic functions, and for inversely transforming a plurality of coefficients generated as a result of transformation .

[00060] The interpolation filter can be a one-dimensional filter or a two-dimensional filter. If the selected interpolation filter is a one-dimensional filter, the interpolator 14 may sequentially perform filtering using one-dimensional interpolation filters in two or more directions, thereby generating a current sub-pel unit pixel value.

[00061] A smoothing interpolation filter according to an embodiment may have a smoothness determined based on a distance between an interpolation location and entire pel unit pixels.

[00062] An interpolation filter according to one embodiment may include different filter coefficients based on a sub-pel unit interpolation location and a smoothness. Next, an interpolation filter determined in contrast to a sub-pel unit interpolation location and a smoothness is referred to as a smoothing interpolation filter.

[00063] A smoothing interpolation filter according to one embodiment may have a smoothness determined based on a distance between an interpolation location and whole pel unit pixels adjacent to the interpolation location.

[00064] Additionally, the smoothing interpolation filter may include filter coefficients to more heavily smooth entire pel unit reference pixels away from the interpolation location.

[00065] To interpolate the entire pel unit pixels in a spatial domain, the smoothing interpolation filter can be obtained by combining filter coefficients to perform transformation and inverse transformation by using a plurality of basic functions, and coefficients of window function to perform low-pass filtration.

[00066] A window function according to an embodiment may be symmetric with respect to an interpolation location. The smoothing interpolation filter obtained by combining the filter coefficients for performing transformation and inverse transformation and window function coefficients for performing low-pass filtering may include filter coefficients for providing a large weight to an integer unit reference pixel. pel near the interpolation location and provide a small weight to an entire pel unit reference pixel away from the interpolation location.

[00067] The smoothing interpolation filter may include filter coefficients for smoothing whole pel unit reference pixels by transforming the smoothed whole pel unit reference pixels using a plurality of basic functions, and inversely transforming a plurality of coefficients generated as a result of the transformation.

[00068] The smoothing interpolation filter is an interpolation filter in a spatial domain, and may include filter coefficients obtained by combining an interpolation filter to perform transformation and inverse transformation, and a smoothing parameter. The smoothing parameter can control at least one of a smoothing speed and a smoothing range.

[00069] The smoothing interpolation filter may include filter coefficients based on a spline function. That is, a base transformation and inverse transformation function for determining interpolation filter coefficients can be a spline function. To obtain a smoother interpolation result, the smoothing interpolation filter can include filter coefficients determined using a spline function.

[00070] According to one embodiment, a smoothing interpolation filter based on a spline function may include filter coefficients to more intensely smooth boundary entire pel unit reference pixels based on a boundary condition of the spline function.

[00071] According to another embodiment, if a transform basis and inverse transform function is a polynomial function, a smoothing interpolation filter may include filter coefficients to maximize a low frequency response of an interpolation filter based on the polynomial function.

[00072] A smoothing interpolation filter according to one embodiment may include different filter coefficients based on a filter length as well as a sub-pel unit interpolation location and a smoothness.

[00073] Furthermore, the smoothing interpolation filter may include different filter coefficients based on a scale factor of an interpolation result as well as a sub-pel unit interpolation location, a smoothness, and a filter length. The filter selector 12 can select a smoothing interpolation filter including filter coefficients scaled to integers. The interpolator 14 normalizes the pixel values generated using the smoothing interpolation filter selected by the filter selector 12.

[00074] Additionally, the filter selector 12 can differently select an interpolation filter based on pixel characteristics. The interpolator 14 may generate sub-pel unit pixel values using the interpolation filter selected differently based on the pixel characteristics.

[00075] The interpolation filter selectable by filter selector 12 may include a smoothing interpolation filter and a general interpolation filter that does not consider smoothing. Thus, based on image characteristics, the filter selector 12 can select a general interpolation filter that does not consider smoothing at all.

[00076] For example, according to another embodiment, the image interpolation apparatus 10 can perform image interpolation using different interpolation filters according to the color components.

[00077] According to another embodiment, the filter selector 12 may differentially select an interpolation filter based on the sub-pel unit interpolation location and a color component of a current pixel. According to another embodiment, the interpolator 14 may interpolate entire pel unit pixels using the selected interpolation filter, thereby generating at least one sub-pel unit pixel value.

[00078] For example, the filter selector 12 may differently determine an interpolation filter for a luma component and an interpolation filter for a chroma component.

[00079] To interpolate a chroma pixel, the filter selector 12 may select a smoothing interpolation filter having a more intense smoothness than that of an interpolation filter for a luma pixel.

[00080] For example, to interpolate a chroma pixel, an interpolation filter including filter coefficients determined based on a spline function or an interpolation filter including filter coefficients determined based on a polynomial function can be selected. Filter coefficients determined based on a spline function can more heavily smooth bordering whole pel unit pixels based on a boundary condition of the spline function. The interpolation filter determined based on a polynomial function may include filter coefficients to maximize a low frequency response.

[00081] Furthermore, to interpolate a chroma pixel, an interpolation filter including filter coefficients determined based on a smoothing parameter having greater smoothness than that of an interpolation filter for a luma pixel, or a smoothing filter interpolation including filter coefficients combined with a window function to remove more high-frequency components than an interpolation filter for a luma pixel can be selected.

[00082] To obtain a smooth interpolation result of a chroma component, a smoothing interpolation filter obtained by combining filter coefficients to perform transformation and inverse transformation based on a plurality of basis functions, and window function coefficients to perform low-pass filtration can be selected.

[00083] Image interpolation is used to transform a low-quality image into a high-quality image, to transform an interlaced image into a progressive image, or to up-sample a low-quality image to a high-quality image. Furthermore, when a video coding apparatus encodes an image, a motion estimator and compensator may perform interprediction using an interpolated reference frame. Inter prediction accuracy can be increased by interpolating a reference frame to generate a high-quality image, and performing motion estimation and compensation based on the high-quality image. Similarly, when an image decoding apparatus decodes an image, a motion compensator can perform motion compensation using an interpolated reference frame, thereby increasing interprediction accuracy.

[00084] Furthermore, the smoothing interpolation filter used by the image interpolation apparatus 10 can obtain a smooth interpolation result by reducing high-frequency components in an interpolation result using an interpolation filter. As high-frequency components reduce image compression efficiency, image coding and decoding efficiency can also be improved by performing adjustable smoothness image interpolation.

[00085] Figure 2 is a diagram for describing a relationship between an entire pel unit and a sub-pel unit.

[00086] Referring to Figure 2, the image interpolation apparatus 10 generates pixel values from locations “X” by interpolating entire pel unit pixel values from locations “0” of a predetermined block 20 in a spatial domain. . The pixel values of the “X” locations are sub-pel unit pixel values of interpolation locations determined by ax and ay. Although Figure 2 illustrates that the predetermined block 20 is a 4x4 block, it will be readily understood by those of ordinary skill in the art that the block size is not limited to 4x4 and may be larger or smaller than 4x4.

[00087] In video processing, a motion vector is used to perform motion compensation and prediction on a current image. Based on predictive coding, a previously decoded image is referred to as predicting a current image, and a motion vector indicates a predetermined point of a reference image. Therefore, a motion vector indicates an entire unit of pel from a reference image.

[00088] However, a pixel to be referred to by a current image may be located between entire pel unit pixels of a reference image. Such a location is referred to as a sub-pel unit location. Since a pixel does not exist at a sub-pel unit location, a sub-pel unit pixel value is merely predicted using entire pel unit pixel values. In other words, a sub-pel unit pixel value is estimated by interpolating entire pel unit pixels.

[00089] A method of interpolating entire pel unit pixels will now be described in detail with reference to Figures 3, and 4A to 4C.

[00090] Figure 3 is a diagram illustrating adjacent full-pel unit pixels to be referenced in order to determine a sub-pel unit pixel value, in accordance with an embodiment of the present invention.

[00091] Referring to Figure 3, the image interpolation apparatus 10 generates a sub-pel unit pixel value 35 from an interpolation location by interpolating integer pel unit pixel values 31 and 33 in a spatial domain. . The interpolation location is determined by a.

[00092] Figures 4A to 4C are diagrams illustrating examples of whole pel unit pixels to be referred to in order to determine a sub-pel unit pixel value, in accordance with an embodiment of the present invention.

[00093] Referring to Figure 4A, to generate the sub-pel unit pixel value 35 by interpolating the two whole-pel unit pixel values 31 and 33, several adjacent whole-pel unit pixel values 37 and 39 including pel integer unit pixel values 31 and 33, are used. In other words, pixels of order 0 and the 1st pixel can be interpolated by performing one-dimensional interpolation filtering on 2M pixel values from a pixel value -(M-1)° to a pixel value M° .

[00094] Furthermore, although Figure 4A illustrates that pixel values in a horizontal direction are interpolated, one-dimensional interpolation filtering can be performed by using pixel values in a vertical or diagonal direction.

[00095] Referring to Figure 4B, a pixel value P(a) of an interpolation location a can be generated by interpolating pixels P0 41 and P1 43 that are adjacent to each other in a vertical direction. When Figures 4A and 4B are compared, their interpolation filtering methods are similar and the only difference between them is that pixel values 47 and 49 aligned in a vertical direction are interpolated in Figure 4B while pixel values 37 and 39 aligned in a horizontal direction are interpolated in Figure 4A.

[00096] Referring to Figure 4C, similarly, a pixel value 44 of the interpolation location a is generated by interpolating two adjacent pixel values 40 and 42. The only difference from Figure 4A is that the pixel values 46 and 48 aligned in a diagonal direction are used instead of pixel values 37 and 39 aligned in a horizontal direction.

[00097] In addition to the directions shown in Figures 4A to 4C, one-dimensional interpolation filtering can be performed in various directions.

[00098] Interpolation filtering can be performed to interpolate entire pel unit pixels to generate a sub-pel unit pixel value. Interpolation filtering can be represented by the following equation.

[00099] A pixel value p(x) is generated by performing interpolation based on a dot product of a vector P of integer unit reference pixels of pel 2M {pm}={p- M+1, p -M+2, ..., p0, p1, ..., pM} and a vector f(x) of filter coefficients {fm}={f-M+1, f-M+2, ... , f0, f1, ..., fM}. Since a filter coefficient f(a) varies based on the interpolation location a and a pixel value p(a) obtained by performing interpolation is determined based on the filter coefficient f(a), a selected interpolation filter, i.e. is, the determined filter coefficient f(x) greatly influences the performance of interpolation filtration.

[000100] Image interpolation using transformation and inverse transformation based on basis functions, and a method of determining an interpolation filter will now be described in detail.

[000101] An interpolation filter using transformation and inverse transformation initially transforms pixel values using a plurality of basis functions having different frequency components. The transformation may include all types of transformation from pixel values in a spatial domain to coefficients in a transformation domain, and may be discrete cosine transformation (DCT). Pel integer unit pixel values are transformed using a plurality of basis functions. A pixel value can be a luma pixel value or a chroma pixel value. Basis functions are not limited to specific basis functions and can include all basis functions for transforming pixel values in a spatial domain to pixel values in a transform domain. For example, a basis function can be a cosine or sine function to perform DCT and inverse DCT (IDCT). Alternatively, several basis functions such as a spline function and a polynomial function can be used. Furthermore, DCT can be modified DCT (MDCT) or windowed MDCT.

[000102] The interpolation filter using transformation and inverse transformation changes the phases of the basis functions used to perform transformation and inversely transforms the values of a plurality of coefficients generated using the phase-shifted basis functions, that is, values into a transformation domain. As a result of inverse transformation, pixel values in a spatial domain are produced, and the produced values may be pixel values of an interpolation location.

Filter Coefficients using Orthogonal Transformation and Inverse Transformation based on Orthogonal Basis Functions

[000103] A case when the interpolator 14 performs interpolation filtering using transformation and inverse transformation based on orthogonal basis functions will now be described in detail. Specifically, DCT is described as an example of the transformation.

[000104] For example, with reference to Figure 4A, to generate the sub-pel unit pixel value 35 by interpolating two entire pel unit pixel values 31 and 33, by using a plurality of adjacent pixel values of pel integer units 37 and 39 including the pel integer unit pixel values 31 and 33, the order 0 pixels and the first pixels can be interpolated by performing one-dimensional DCT on 2M pixel values from a value of pixel -(M-1)° up to a pixel value M°, and performing one-dimensional IDCT based on phase-shifted basis functions.

[000105] Interpolator 14 initially performs one-dimensional DCT on whole pel unit pixel values. One-dimensional DCT can be performed as represented in Equation 1. p(l) represents pixel values 37 and 39 from a pixel value -(M-1)° to a pixel value M°; and Ck represents a plurality of coefficients in a frequency domain, which are generated by performing one-dimensional DCT on pixel values 37 and 39. In this case, k is a positive integer that satisfies the above condition of Equation 1.

[000106] After the one-dimensional DCT is performed on pixel values 37 and 39 using Equation 1, interpolator 14 performs inverse transformation on the coefficients as represented in Equation 2. a represents an interpolation location between two pixel values as illustrated in Figure 13, and can have several fractional values such as 1/2, 1/4, 3/4, 1/8, 3/8, 5/8, 7 /8, 1/16, etc. Fractional value is not limited to a specific value, and can be an actual value rather than a fractional value. P(a) represents the sub-pel 35 unit pixel value of the interpolation location a, which is generated as a one-dimensional IDCT result.

[000107] When Equation 2 is compared with Equation 1, the phase of a cosine function that is a basis function used to perform IDCT is determined based on a fractional number a instead of an integer l, and so is different from the phase of a basis function used to perform one-dimensional DCT. In other words, the phase of each basis function used to perform inverse transformation, that is, a cosine function, is changed based on 2a. If the interpolator 14 performs IDCT based on the phase-shifted cosine functions according to Equation 2, the sub-pel unit pixel value 35 of the interpolation location a, i.e., P(a), is generated.

[000108] DCT according to Equation 1 is expressed by a matrix equation represented in Equation 3.

[000109] Here, C is a 2Mx1 matrix of the 2M coefficients described above in relation to Equation 1, and REF is a 2Mx1 matrix of the pel integer unit pixel values, that is, P-(M-1), .. PM pixel values, described above in relation to Equation 1. The number of pel integer unit pixel values used to perform interpolation, that is, 2M, refers to the number of taps of a one-dimensional interpolation filter. D is a square matrix for performing one-dimensional DCT and can be defined as represented in Equation 4. kel are integers that satisfy the above conditions, and Dkl refers to a row k and a column l of the square matrix D to perform DCT in Equation 3. M is the same as that in Equation 3.

[000110] IDCT using a plurality of phase-shifted basis functions according to Equation 2 is expressed by a matrix equation represented in Equation 5.

[000111] Here, P(a) is the same as that of Equation 2, and W(a) is a 1x2M matrix for performing one-dimensional IDCT by using a plurality of phase-shifted basis functions and can be defined as represented in Equation 6. k is an integer that satisfies the above condition, and Wk(a) refers to a column k of the matrix W(a) described above with respect to Equation 5. A filter F(a) to perform one-dimensional DCT and one-dimensional IDCT using a plurality of phase basis functions changed according to Equations 3 and 5 can be defined as represented in Equation 7. kel are integers that satisfy the above conditions, Fl(a) refers to a column l of F(a), and W(a) and D are identical to those in Equation 3.

[000112] To generate intensely smoothed sub-pel unit pixel values, the interpolator 14 may change an interpolation filter used to perform transformation and inverse transformation based on a basis function.

[000113] A case when a window function is used; a case when several smoothing parameters are used; a case when a spline function is used as a base function; and a case when a polynomial function is used as a basis function to determine various smoothing interpolation filters; will now be described sequentially in detail.

Smoothing Interpolation Filter using Window Function

[000114] A method of smoothing interpolation filter coefficients using a window function will now be described in detail.

[000115] A window function may include a hamming window function, a cosine window function, an exponential window function, a hamming window function, a Blackman window function, and a triangle window function. Although the cases when transform-based and inverse-transform interpolation filters are smoothed through use of certain window functions are described below for convenience of explanation, it will be readily understood by those of ordinary skill in the art that, in addition to the described window functions, other window functions having similar frequency responses can also be used.

[000116] Window coefficients based on a hamming window function satisfy Equation 24.

[000117] In various window functions including the hamming window function, an input n is symmetric with reference to N/2 and a frequency response is similar to that of a low-pass filter. Among the inputs of a window function, only one input covered by a window formed by the window function can be produced. A window size N can be set as a positive integer greater than the length of an original interpolation filter. For example, to apply a window function to an interpolation filter to generate a sub-pel unit pixel such as 1/2 or 1/4 pixel, the center location of the window function can be moved by 1/2 or 1/ 4 pixels. That is, because the center location of the window function is moved to an interpolation location, the window function can be symmetric with respect to the interpolation location.

[000118] For example, Equations 9 and 10 respectively show window coefficients of hamming window functions for 1/2 pel unit and 1/4 pel unit interpolation filters, respectively.

[000119] Equation 11 sequentially shows the window coefficients of a hamming window function, a cosine window function and an exponential window function as window functions for interpolation filters, which are generalized based on a unit interpolation location sub-pel a.

[000120] By combining the window coefficients according to Equation 11 with an original interpolation filter fk(a)r smoothing interpolation filter coefficients can be determined according to Equation 12.

[000121] As a smoothing interpolation filter is determined using a window function, a weight of an interpolation filter coefficient can be adjusted based on the distance between an entire pel unit reference pixel and an interpolation location . For example, a smoothing interpolation filter may be determined such that, through a window function, among the filter coefficients of an interpolation filter, a filter coefficient for an entire pel unit reference pixel located far from an interpolation location is changed much, and a filter coefficient for an entire pel unit reference pixel located close to the interpolation location is not changed much.

[000122] Furthermore, if a smoothing interpolation filter is determined using a window function, the interpolation filtering can be performed after the integer and pel unit reference pixels are smoothed. Introduced pel whole unit reference pixels Ref={p-M+1, p-M+2, ..., p0, p1, ..., pM} may include noise or may be damaged due to such an error as a quantization error. As such, if the entire pel unit reference pixels are smoothed before interpolation filtering is performed, the image interpolation apparatus 10 can enhance an interpolation effect.

Smoothing Interpolation Filter using Two Parameters

[000123] A smoothing interpolation filter can determine the smoothness of filter coefficients based on two parameters. The sub-pel unit smoothing interpolation filter coefficients obtained by combining a smoothing matrix S and transform-based and inverse-transform interpolation filter coefficients satisfy Equation 13.

[000124] Equation 14 shows an example of the smoothing matrix S.

[000125] The smoothing matrix S according to Equation 14 is a diagonal-3 matrix. In other words, among the components of the smoothing matrix S, components except the components on a central diagonal line and on two diagonal lines corresponding to each other and adjacent to the central diagonal line are all 0.

[000126] In the smoothing matrix S, a smoothness ai can be determined independently of the distance (i-a) from the entire pel unit pixels to be interpolated. In this case, smoothing based on the smoothing matrix S can be referred to as uniform smoothing.

[000127] Furthermore, in the matrix of S, the smoothing ai may vary based on an index l of an entire pel unit pixel location. In this case, smoothing based on the smoothing matrix S can be referred to as non-uniform smoothing. For example, the smoothness ai may satisfy Equation 15.

[000128] A positive index l can increase a smoothing effect if the distance between an interpolation location and an entire pel unit reference pixel is large. Consequently, the positive index l can control the speed of smoothing based on the distance between an interpolation location and an entire pel unit reference pixel. A smoothing parameter ß can control the range of smoothing around an interpolation location.

[000129] If the smoothing parameter ß is less than 0, the smoothing matrix S according to Equation 13 can be changed to a sharpening filter. Consequently, if the smoothing matrix S, which is less than 0, is combined with an interpolation filter using transformation and inverse transformation, a filter for amplifying high-frequency components can be generated.

[000130] To perform sub-pel unit prediction, the image interpolation apparatus 10 may use smoothing interpolation filter coefficient data previously stored in memory.

[000131] Figure 5 is a graph 50 of a smoothing factor based on a smoothing parameter ß of a smoothing interpolation filter, in accordance with an embodiment of the present invention.

[000132] First and second curves, 52 and 54, show a smoothing factor for smoothing a discrete transformation-based interpolation filter. If m is large, that is, if the distance from the integer pel unit pixels to be interpolated is increased, the smoothing factor is close to 0.

[000133] Here, compared to the second curve 54 in a case when the smoothing parameter ß is large, the first curve 52 in a case when the smoothing parameter ß is small has a relatively large width of the smoothing factor. In other words, if the smoothing parameter ß of the smoothing interpolation filter is large, low-frequency components can be mostly filtered out and thus relatively heavily smoothed sub-pel unit pixel values can be generated. If the smoothing parameter ß of the smoothing interpolation filter is relatively small, relatively high frequency components can remain and be interpolated and thus sub-pel unit pixel values can be generated.

[000134] To determine the filter coefficients of a smoothing interpolation filter, the image interpolation apparatus 10 may use a spline function or a polynomial function as a basis function as well as an orthogonal basis function.

Smoothing Interpolation Filter based on Spline Function

[000135] The image interpolation apparatus 10 can determine the filter coefficients of a smoothing interpolation filter based on a spline function.

[000136] Furthermore, to smooth an interpolation result, the image interpolation apparatus 10 may use a spline function having a boundary condition. In more detail, for example, if polynomial spline interpolation having a variable p is used to form an interpolation filter using m integer unit pixels pel pm (M is an integer equal to or greater than 2), to allow the variable p has a maximum smoothness in a range of 3<p<M+1 and to allow a spline value, that is, an interpolation result value to be infinitely smooth in one pixel (-M+2)° and one pixel ( M-1)°, (p-1) additional conditions may be established. These additional conditions are referred to as not-a-knot boundary conditions or Boor boundary conditions.

[000137] An interpolation result using interpolation filter coefficients based on a spline function can be represented as a weighted sum calculated using Equation 16.

[000138] Input pixels pm are pel integer unit reference pixels, and a set {pm} of input pixels in which the range of m is [-M+1, M] (i.e. -M+1 <m<M) is entered. A spline function S(x) corresponds to the pixel values generated as a result of interpolation. fm(x) is a means of cardinal spline interpolation and corresponds to filter coefficients based on a cardinal spline function. fm(x) can be cardinal spline function values having the same boundary condition and having values only at pel integer unit reference pixel locations (i.e. -M+1<m<M, m is an integer) .

[000139] The filter coefficient fm(x) can be determined using equation 17.

[000140] When k is an integer in a range of 0 < k < 2M-2, the spline filter coefficient fm(x) can be determined at each integer m in a range of [-M+1+k, -M+k+2], that is, from (-M+1+k) to (-M+k+2). In Equation 17, an ok coefficient m can be determined based on Equation 18.

[000141] For sub-pel unit interpolation a finite impulse response (FIR) filter including spline filter coefficients fm(a) according to an interpolation location a can be calculated in advance and stored, and a pixel value of sub-pel unit at an interpolation location a between a pixel of order 0° and a first pixel can be generated by performing interpolation filtering using the FIR filter including the spline filter coefficients fm(a) in the unit reference pixel entire pel pm.

[000142] Figure 6 is a graph of a spline function 60 usable by a smoothing interpolation filter, in accordance with an embodiment of the present invention.

[000143] With reference to Figure 6, based on a spline function having a variable p of 3, three spline interpolation medium curves f-2(x) 61, f-1(x) 62, and f0(x) 63 for 2M=6, i.e. a 6-lead interpolation filter, are illustrated. For example, the interpolation filter coefficients for generating sub-pel unit pixel values that satisfy a=1/4 can be determined as f-2(1/4) 64, f-1(1/4) 65, and f0(1/4) 66 in the spline interpolation means curves f-2(x) 61, f-1(x) 62, and f0(x) 63.

[000144] Smoothing Interpolation Filter based on Polynomial Function

[000145] The image interpolation apparatus 10 can determine filter coefficients of a smoothing interpolation filter based on a polynomial function.

[000146] A polynomial interpolation function including interpolation filter coefficients {fk} based on a polynomial function can be represented based on a polynomial function as a basis function using Equation 19. The integer k is defined in from a range of -M+1 < k < M.

[000147] Furthermore, to smooth an interpolation result, the image interpolation apparatus 10 may determine optimized filter coefficients for a low frequency range from among the interpolation filter coefficients {fk} based on a polynomial function. For example, if a frequency is 0, the filter coefficients {fk} determined when a function value of a polynomial interpolation function and function values of derivatives of the polynomial interpolation function are identical can be determined as filter coefficients interpolation systems optimized for a low frequency range. As such, as represented in Equation 20, as a function for the integer k, 2M linear functions for 2M filter coefficients {fk} (2M is an unknown) can be obtained.

[000148] Solutions of the linear functions of Equation 20 can be calculated using a Newton polynomial function. Equation 21 represents 2M filter coefficients {fk} calculated as solutions of the linear functions of Equation 20.

[000149] An interpolation filter including the filter coefficients {fk} determined based on the Newton polynomial function of Equations 20 and 21 has a maximum response in a low frequency range, a more intensely smoothed interpolation result can be obtained by use of pixel values using this interpolation filter. Consequently, an interpolation filter including filter coefficients determined based on a polynomial function as a basis function can be selected as a smoothing interpolation filter.

[000150] As such, the image interpolation apparatus 10 can generate more intensely smoothed interpolation pixels by selecting a smoothing interpolation filter including interpolation filter coefficients based on a polynomial function. Particularly, as chroma pixels have strong high-frequency components, to generate sub-pel unit pixel values from whole pel unit chroma pixels, a smoothing interpolation filter including interpolation filter coefficients based on a polynomial function can be used.

Interpolation Filter Coefficients for Scaled Interpolation

[000151] Various smoothing interpolation filter generation methods in accordance with embodiments of the present invention rely on an arithmetic pressure to generate a floating point number rather than an integer, and absolute values of filter coefficients are not typically greater than 1. Specifically, a calculation result of a real number rather than an integer can be generated by a local sub-pel unit interpolation.

[000152] The efficiency of integer-based calculation is greater than that of floating-point-based calculation. As such, the image interpolation apparatus 10 can improve the interpolation filtering calculation efficiency by scaling the filter coefficients to integers using a scaling factor. Furthermore, as a bit depth of pixel values is increased, the accuracy of interpolation filtering can also be improved.

[000153] The image interpolation apparatus 10 can multiply the filter coefficients fm(a) by a predetermined value, and can perform image interpolation using large filter coefficients Fm(a). For example, the filter coefficients Fm(a) can be scaled from the filter coefficients fm(a) as represented in Equation 22.

[000154] For calculation efficiency, the scale factor can be in the form of 2n. n can be 0 or a positive integer. The result of interpolation filtering using filter coefficients scaled by 2n may have a bit depth scaled by n bits compared to a result obtained using original filter coefficients.

[000155] Integer calculation interpolation filtering using the scaled filter coefficients Fm(a) can satisfy Equation 23. In other words, after interpolation filtering is performed using the scaled filter coefficients Fm(a), the scaled bit depth has to be rebuilt to an original bit depth. In this case, an offset could be 2n-1.

[000156] In other words, as a step filtering result using a step smoothing interpolation filter has to be reduced by a scale factor, i.e., 2n in order to be reconstructed to original bits, a bit depth of the result Stepped filtering can be reduced by n bits.

[000157] If two-step interpolation filtering is performed by performing one-dimensional interpolation filtering in a horizontal direction and performing one-dimensional interpolation filtering in a vertical direction, a reduction can be made by a total of 2n bits. Consequently, if a first one-dimensional interpolation filter is scaled by n1 bits and a second one-dimensional interpolation filter is scaled by n2 bits, after two-step interpolation filtering is performed using the first and second one-dimensional interpolation filters, a reduction can be done through a sum of n1 and n2, that is, 2n bits. The first one-dimensional interpolation filter may be an interpolation filter that is not scaled.

[000158] As a sum of the smoothing filter coefficients fm(a) is 1, scaled smoothing filter Fm(a) of the scaled interpolation filter must satisfy Equation 25.

[000159] However, the normalization condition according to Equation 25 may cause a rounding error. The image interpolation apparatus 10 may round the scaled filter coefficients Fm(a) based on the normalization condition according to Equation 19. For normalization, some of the scaled filter coefficients Fm(a) may be adjusted within a predetermined range of original values. For example, some of the scaled filter coefficients Fm(a) can be adjusted within a range of ±1 to correct for a rounding error.

[000160] Various smoothing interpolation filters and filter coefficients are described above. Specifically, as a function to determine the filter coefficients of a smoothing interpolation filter, a window function, a spline function, a polynomial function, etc. can be used. For a smoothing interpolation filter, a frequency response of a function may vary based on a frequency, but a filter gain of the function's frequency response may be close to 1. Consequently, the image interpolation apparatus 10 may determine the filter coefficients by using a function having a filter gain of which a frequency response is closest to 1 even when a frequency varies, and can select a smoothing interpolation filter including the filter coefficients.

[000161] Figure 7 is a flowchart of an image interpolation method in accordance with an embodiment of the present invention.

[000162] In operation 71, an interpolation filter is selected differently based on a sub-pel unit interpolation location and a smoothness among the interpolation filters to generate at least one sub-pel unit pixel value located between the entire pel unit pixels of an image. An interpolation filter smoothness can be determined based on a distance between an interpolation location and entire pel units.

[000163] An interpolation filter according to one embodiment may be a filter including filter coefficients for performing transformation and inverse transformation based on a plurality of basis functions. A smoothing interpolation filter according to an embodiment may include at least one of an interpolation filter combined with a window function, an interpolation filter based on a plurality of smoothing parameters, an interpolation filter based on a smoothing parameter , a spline interpolation filter, and a polynomial function interpolation filter.

[000164] To perform filtering using a smoothing interpolation filter, filter coefficients can be determined to more intensely smooth entire pel unit reference pixels away from an interpolation location.

[000165] In operation 73, at least one pixel value of sub-pel units is generated by interpolating pixel values of the entire pel unit pixels using the interpolation filter selected in operation 71.

[000166] Among the interpolation filters, if an interpolation filter including filter coefficients scaled to integers is selected, the pixel values generated using the interpolation filter can be normalized based on a scaling factor.

[000167] According to one embodiment, an interpolation filter can be selected differently based on the characteristics of the pixels to be interpolated, and sub-pel unit pixel values can be generated using the selected interpolation filter.

[000168] Various examples of filter coefficients of an interpolation filter determined in consideration of a sub-pel unit interpolation location and a smoothness will now be described with reference to Figures 8A to 12C.

[000169] Figures 8A to 8C are tables showing filter coefficients of 12-lead interpolation filters determined based on a smoothing parameter and an interpolation location, in accordance with embodiments of the present invention.

[000170] With reference to Figures 9A to 8C, among the interpolation filters described above based on orthogonal transformation, to perform orthogonal transformation and inverse transformation after smoothing, the whole pel unit reference pixels as described above in relation to Figure 5, filter coefficients of a smoothing interpolation filter obtained by combining a smoothing matrix and an interpolation filter based on an orthogonal transformation are shown.

[000171] Figures 8A to 8C show various interpolation filters including different filter coefficients as a smoothing parameter ß varies as 0, 0.002, 0.004, 0.006, 0.008, 0.010, 0.012, 0.014, 0.016, 0.018 and 0.020, and an interpolation location varies it as 1/8, 1/4, 3/8, 1/2, 5/8, 3/4, and 7/8.

[000172] For example, in the table of Figure 8A, if the smoothing parameter ß is 0.002 and the interpolation location a is 1/8, a filter including the filter coefficients {fm}, for example, {f-11, f-10, f-9, f-8, f-7, f-6, f-5, f-4, f-3, f-2,f-1, f0, f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12}, determined as {-1, 4, -7, 12, -24, 246, 37, -16, 9, -5, 3, -1} can be selected as an interpolation filter.

[000173] Figures 9A to 9C are tables showing filter coefficients of 6-lead interpolation filters determined based on a smoothing parameter and an interpolation location, in accordance with embodiments of the present invention.

[000174] Although the filter coefficients of Figures 8A to 8C are 12 filter coefficients of a 12-lead interpolation filter among the smoothing interpolation filters obtained by combining a smoothing matrix and an orthogonal transformation interpolation filter, Figures 9A to 9C show 6 filter coefficients of a 6-lead interpolation filter. In Figures 8A to 8C, and 9A to 9C, various smoothing interpolation filters including different filter coefficients based on the smoothing parameter ß and the interpolation location a can be shown.

[000175] The filter coefficients shown in Figures 9A to 8C, and 9A to 9C are coefficients scaled by a scale factor of 256(=28) based on scaled interpolation filtering and then rounded.

[000176] In Figures 8A to 8C and 9A to 9C, when the interpolation location a is constant and the smoothing parameter ß is increased, a filter coefficient fm can be relatively small.

[000177] Furthermore, when the smoothing parameter ß is constant and the interpolation location a is away from 1/2, if m of the filter coefficient fm is away from m=0 towards m=-M+1 or m=M, the filter coefficient fm may be relatively small compared to f0. The filter coefficient fm near m=0 can be relatively large.

[000178] Consequently, when the smoothing parameter ß is increased, if the interpolation location a is away from 1/2, that is, close to the whole pel unit pixel, a sharper interpolation filter, that is , a less smoothing interpolation filter, can be selected.

[000179] As an interpolation filter according to one embodiment is an inversely reflective symmetric filter, a filter coefficient fm(a) of an interpolation location (1-a) can be determined using the filter coefficient fm(a ) from the interpolation location a. For example, in Figure 9A, among the filter coefficients {fm} having the smoothing parameter ß=0.002, the filter coefficients {fm(3/8)} of the interpolation location a=3/8 and the filter coefficients {fm(5/8)} from the interpolation location a=1-3/8=5/8 are compared as shown below. {fm(3/8)} = { 11, -42, 196, 117, -35, 10 }, {fm(5/8)} = { 10, -35, 117, 196, -42, 11 }

[000180] That is, it is shown that the filter coefficients {fm(3/8)} when m = -2, -1, 0 are identical to the filter coefficients {fm(5/8)} when m = 3, 2, 1, and the filter coefficients {fm(3/8)} when m = 3, 2, 1 are identical to the filter coefficients {fm(5/8)} when m = -2, -1, 0. Consequently, in the tables of Figures 10 to 12C, although only the interpolation filter coefficients fm(a) in a case when the interpolation location is less than or equal to 1/2 are shown, it will be readily understood by those of knowledge It is common in the art that the interpolation filter coefficients fm(a) in a case when the interpolation location is greater than 1/2 can also be determined.

[000181] Figure 10 is a table showing the filter coefficients of 6-lead interpolation filters determined for chroma pixels based on a smoothing parameter and an interpolation location, in accordance with an embodiment of the present invention.

[000182] The image interpolation apparatus 10 may differently select an interpolation filter based on image characteristics. For example, if a smoothing interpolation filter obtained by combining a smoothing matrix and an orthogonal transform interpolation filter is determined, a smoothing parameter may vary based on image characteristics.

[000183] For example, because chroma pixels are down-sampled based on a 4:2:0 color format, chroma pixels have fewer low-frequency components compared to luma pixels. In this case, with reference to Figure 10, regardless of an interpolation filter for luma pixels, only an interpolation filter for chroma pixels can be additionally selected. Various filter coefficients of interpolation filters selected differently based on a color component will now be described with reference to Figure 11.

[000184] Figure 11 is a table showing the filter coefficients of the smoothing interpolation filters determined differently based on a color component and an image interpolation location, in accordance with an embodiment of the present invention.

[000185] Referring to Figure 11, various smoothing interpolation filters including different filter coefficients such as a number of filter taps 2M, an interpolation location a, and a color component L(luma)/C(chroma) varies . The filter coefficients in Figure 11 are coefficients scaled to a scale factor of 256(=28) and rounded. As described above, based on the inverse reflective characteristics of interpolation filter coefficients, only one case when the interpolation location a is less than or equal to 1/2 is shown.

[000186] Similarly to Figures 8A to 10, a result of comparing filter coefficients for a chroma component and filter coefficients for a luma component is similar to a result of comparing filter coefficients in a case when a smoothing parameter ß is large and the filter coefficients in a case when the smoothing parameter ß is small.

[000187] Figures 12A to 12C are tables showing filter coefficients of smoothing interpolation filters based on an image interpolation location and a scale factor, in accordance with embodiments of the present invention.

[000188] Figures 12A to 12C show several modified examples of the filter coefficients of the smoothing interpolation filters, which are scaled, rounded, and normalized as a scale factor of 2n varies such as 512, 256, 128, and 64 , and the number of filter taps of an interpolation filter and an interpolation location vary a.

[000189] Specifically, in Figure 12C, interpolation filter coefficients for interpolating 1/8 pixel units may be useful for performing motion compensation on chroma pixels. However, as the image quality of chroma pixels, which is visually recognized by people, is less crucial compared to luma pixels due to a relatively short filter lead, e.g. 4 leads, and a low bit depth, a smoothing interpolation filter having a scale factor of 25 can also be used.

[000190] The filter coefficients shown in Figures 9A to 12C are only parts of several examples, and it will be readily understood by those of ordinary skill in the art that the filter coefficients of interpolation filters considering smoothing, in accordance with the embodiments of the present invention, can be modified based on various factors including an interpolation location, a smoothing parameter, a window function, a spline function, a polynomial function, a scale factor and rounding.

[000191] Video coding and decoding using a smoothing interpolation filter, in accordance with embodiments of the present invention, are described below with reference to Figures 13A to 27. Video coding and decoding based on coding units having a tree structure, in accordance with embodiments of the present invention, are described below with reference to Figures 15 to 25. Methods of encoding and decoding video using a smoothing interpolation filter, in accordance with embodiments of the present invention , are described below with reference to Figures 26 and 27.

[000192] When various operations are performed on image data, the image data can be divided into data groups and the same operation can be performed on data in the same data group. In the following description, a group of data formed according to a predetermined pattern is referred to as a “data unit”, and an operation is performed on each “data unit” using data included in the data unit. Video Encoding and Decoding using Smoothing Interpolation Filter

[000193] Figure 13A is a block diagram of a video coding apparatus 100 using a smoothing interpolation filter, in accordance with an embodiment of the present invention.

[000194] The operations of an encoder 120 and an output unit 130 of the video coding apparatus 100 can be cooperatively controlled by a video coding processor, a CPU, and a graphics processor.

[000195] To encode a current image from an input video, the video encoding apparatus 100 divides the current image into data units having a predetermined size and encodes each data unit.

[000196] For example, the current image includes pixels in a spatial domain. To encode spatially adjacent pixels of the current image at the same time, the current image can be divided into groups of pixels having a predetermined size such that adjacent pixels within a predetermined range form a group. By performing a series of encoding operations on the pixels of the divided pixel groups, the current image can be encoded.

[000197] Since the initial data of an image to be encoded are pixel values in the spatial domain, each group of pixels can be used as a data unit to be encoded. Furthermore, when transformation coefficients in a transformation domain are generated by performing transformation for video coding on pixel values of the pixel group in the spatial domain, the transformation coefficients are included in groups of coefficients having the same size as the groups of pixels in the spatial domain. Consequently, a group of coefficients of the transformation coefficients in the transformation domain can also be used as a data unit to encode an image.

[000198] Consequently, in the spatial domain and the transformation domain, a group of data having a predetermined size can be used as a data unit to be encoded. In this case, the size of a data unit can be defined as the number of pieces of data included in the data unit. For example, the number of pixels in the spatial domain or the number of transformation coefficients in the transformation domain can represent the size of a data unit.

[000199] An encoding method or encoding characteristics of a current data unit may be determined with respect to each group of data of any data level among a data unit, a slice, an image, and a sequence of images of a video that must be currently encoded.

[000200] The video coding apparatus 100 can encode a current image by performing predictive coding including inter-prediction and intra-prediction, transformation, quantization, and entropy coding on each data unit.

[000201] Based on inter prediction, to estimate a current pixel value with reference to a pixel value of a temporally previous or subsequent image, residual data between a pixel value of a reference region of a reference image and a pixel value of a current image and reference data indicating said pixel value can be determined.

[000202] To perform inter prediction more accurately, the video coding apparatus 100 can determine residual data and reference data using a sub-pel unit pixel value. To perform inter-sub-pel unit prediction, the video coding apparatus 100 may determine a sub-pel unit pixel value located between adjacent whole-pel unit pixels by interpolating the adjacent whole-pel unit pixels.

[000203] Furthermore, the sub-pel unit pixel value may be generated by performing interpolation filtering on two or more whole-pel unit reference pixels including the adjacent whole-pel unit pixels. The reference pixels for performing interpolation filtering can be pixels from a reference image.

[000204] To efficiently perform image interpolation, the video coding apparatus 100 can selectively determine the interpolation filter coefficients. The encoder 120 may include the image interpolation apparatus 10 illustrated in Figure 1. In other words, to perform inter-sub-pel unit prediction, the encoder 120 may generate a sub-pel unit pixel value through use of a filter. of interpolation including the filter coefficients determined by the image interpolation apparatus 10 based on transformation and inverse transformation.

[000205] To efficiently perform interpolation filtering, the video coding apparatus 100 may previously store interpolation filter coefficients in memory. According to an interpolation location, a smoothness, the number of filter taps, a bit depth, a scale factor, and a transformation-based interpolation filtering base function can be stored in the memory of the encoding apparatus. of video 100.

[000206] For example, at least one of (i) 8-lead 1/4 pel unit filter coefficients {-1, 4, -10, 57, 19, -7, 3, -1} having a factor of 26, (ii) 8-lead 1/2 pel unit filter coefficients {-1, 4, -11, 40, 40, -11, 4, -1} having a scaling factor of 26, (iii) 4-lead 1/8 pel unit filter coefficients {-3, 60, 8, -1} having a scaling factor of 26, (iv) 1/4 pel unit filter coefficients of 4 leads {-4, 54, 16, -2} having a scale factor of 26, (v) 4-lead 3/8 pel unit filter coefficients {-5, 46, 27, -4} having a factor scale factor of 26, and (vi) 4-lead 1/2 pel unit filter coefficients {-4, 36, 36, -4} having a scale factor of 26 may be stored in memory and may be used to perform smoothing interpolation filtering.

[000207] In addition to the filter coefficients mentioned above, smoothing interpolation filter coefficients that can be modified based on various basis functions and window functions as shown in Figures 9A to 9C can be used to perform interpolation filtering.

[000208] If interpolation filtering is performed using the filter coefficients stored in memory, an inter-prediction calculation speed can be improved.

[000209] Among a plurality of interpolation filters, the encoder 120 may select and use a desired smoothing interpolation filter to perform inter-prediction based on a sub-pel unit interpolation location and a smoothness. Additionally, an appropriate smoothing interpolation filter for a current pixel can be determined based on the number of filter taps, a bit depth, a scaling factor, etc.

[000210] Encoder 20 may determine an interpolation filter based on image characteristics. For example, the encoder 120 may determine different interpolation filters based on the color components of the pixels. For example, an interpolation filter for luma pixels and an interpolation filter for chroma pixels can be selected separately and thus sub-pel unit pixel values can be individually generated by performing interpolation filtering.

[000211] A video can be encoded by performing inter prediction based on sub-pel unit interpolation, intra prediction, transformation, and quantization.

[000212] The output unit 130 can encode and output coding information and can output encoded image data. Like the encoding information, information about the selected interpolation filter can be additionally encoded. In other words, information about an interpolation filter used to perform sub-pel unit prediction coding may be encoded. For example, a decoder must know about an interpolation filter used to encode an image in order to decode the image using the same interpolation filter used in the encoding process. To do this, information indicating the interpolation filter used can be encoded together with the image. However, if a filter is selected based on a previous encoding result, i.e., context, information about the selected filter may not be additionally encoded.

[000213] The output unit 130 can perform entropy coding on the coding information and coded image data and can produce a bit stream.

[000214] Figure 13B is a block diagram of a video decoding apparatus 200 using a smoothing interpolation filter, in accordance with an embodiment of the present invention.

[000215] The video decoding apparatus 200 includes a receiver and extractor 220 and a decoder 230. The operations of the receiver and extractor 220 and the decoder 230 of the video decoding apparatus 200 can be controlled cooperatively through a processor video decoding system, a graphics processor, and a CPU.

[000216] To reconstruct an image from a bit stream, the video decoding apparatus 200 may decode the encoded image data from the bit stream by performing operations including entropy decoding, inverse quantization, inverse transformation, inter prediction. /compensation, and intra/compensation prediction.

[000217] The receiver and extractor 220 receives, and analyzes, a bit stream of an encoded video. The receiver and extractor 220 can extract the encoded data from each data unit of a current image, and encode information including information about an encoding method to be used to decode the encoded data, from the analyzed bit stream.

[000218] If the coding information includes interpolation filter information, the decoder 230 may read information about an interpolation filter used to perform intra-subunit prediction from the interpolation filter information, and may perform interpolation compensation. movement through the use of the interpolation filter used in a coding process.

[000219] The decoder 230 can decode the encoded image data by performing various decoding operations such as entropy decoding, inverse quantization, inverse transformation, inter/compensation prediction, and intra/compensation prediction on a coded image according to the various decoding methods determined based on information about an encoding mode.

[000220] To perform motion compensation, a reference region of a reference image that is temporally prior or subsequent to a current image may be determined using reference data, and a pixel value of the reference region and the Residual data can be combined to reconstruct a current pixel value.

[000221] If residual data and reference data are determined based on the interpolated pixels in a subpixel unit in an encoding process, the decoder 230 may also perform motion compensation based on the interpolated pixels in a subpixel unit. To perform subpixel unit motion compensation, decoder 230 may generate a subpixel unit pixel value by interpolating adjacent whole pel unit pixels of the reference image. The sub-pel unit pixel value may be generated by performing interpolation filtering on two or more whole-pel unit reference pixels including the adjacent whole-pel unit pixels.

[000222] To efficiently realize image information, the video decoding apparatus 200 can selectively determine the interpolation filter coefficients. Decoder 230 may include the image interpolation apparatus 10 illustrated in Figure 1. In other words, to perform sub-pel unit motion compensation, decoder 230 may generate the sub-pel unit pixel value through use of a filter. interpolation based on transformation.

[000223] To efficiently perform interpolation filtering, the video decoding apparatus 200 may pre-store variously selectable interpolation filter coefficients in memory according to an interpolation location, a smoothness, the number of filter taps, a bit depth, a scale factor, and a transformation-based interpolation filtering base function.

[000224] As described above, for example, at least one of (i) 8-lead 1/4 pel unit filter coefficients {-1, 4, -10, 57, 19, -7, 3, -1 } having a scaling factor of 26, (ii) 8-lead 1/2 pel unit filter coefficients {-1, 4, -11, 40, 40, -11, 4, -1} having a scaling factor scale of 26, (iii) 4-lead 1/8 pel unit filter coefficients {-3, 60, 8, -1} having a scaling factor of 26, (iv) pel unit filter coefficients of 1 /4 4-lead {-4, 54, 16, -2} having a scale factor of 26, (v) 4-lead 3/8 pel unit filter coefficients {-5, 46, 27, -4 } having a scale factor of 26, and (vi) 4-lead 1/2 pel unit filter coefficients {-4, 36, 36, -4} having a scale factor of 26 may be stored in memory and can be used to perform smoothing interpolation filtering. In addition to the filter coefficients mentioned above, smoothing interpolation filter coefficients that can be modified according to various basis functions and window functions as shown in Figures 8A to 12C can be used to perform smoothing interpolation filtering.

[000225] Among a plurality of interpolation filters, the decoder 230 may select and use an interpolation filter appropriate for a current pixel to perform sub-pel unit motion compensation according to a sub-pel unit interpolation location. , the number of filter taps, a bit depth, a scaling factor, etc.

[000226] Additionally, decoder 230 may determine a smoothing interpolation filter based on image characteristics. For example, different interpolation filters can be determined according to the color components of pixels, interpolation filtering for luma pixels and interpolation filtering for chroma pixels can be performed separately, and thus the interpolated sub-pel unit pixel values can be individually generated.

[000227] Accordingly, the decoder 230 can reconstruct the data in a spatial domain by performing inverse quantization/inverse transformation, and can reconstruct the pixel values in a current image by performing intra prediction and motion compensation based on interpolation of sub-pel unit as well as whole pel unit interpolation. If images are reconstructed, a video can be decoded.

[000228] Figure 14A is a flowchart of an image coding method using a smoothing interpolation filter, in accordance with an embodiment of the present invention.

[000229] In operation 1410, to encode a current image from an input video, predictive coding is performed using sub-pel unit interpolation. An interpolation filter is selected differently based on a sub-pel unit interpolation location and a smoothness among the interpolation filters to generate a sub-pel unit pixel value. The smoothness of the interpolation filter can be determined based on the distance between an interpolation location and entire pel units.

[000230] The sub-pel unit pixel value can be generated by performing interpolation filtering on two or more entire pel unit reference pixels of a reference image. Residual data and reference data are determined using the generated sub-pel unit pixel value, thereby realizing prediction coding.

[000231] To efficiently perform image interpolation, the interpolation filter coefficients can be determined selectively. Among the interpolation filter coefficients previously stored in memory, a desired interpolation filter can be selected based on a sub-pel unit interpolation location, a smoothness, a number of filter taps, a bit depth, a factor scale, a base interpolation filtering function based on the transformation, and a color component, and interpolation can be performed to generate the sub-pel unit pixel value.

[000232] In operation 1420, transformation and quantization are performed on an inter prediction result based on sub-pel unit interpolation, and intra prediction.

[000233] In operation 1430, a bit stream can be produced by performing entropy coding on the coding information and image data encoded in the form of quantized transformation coefficients. The coding information may include information about an interpolation filter used to perform sub-pel unit prediction coding.

[000234] Figure 14B is a flowchart of an image decoding method using a smoothing interpolation filter, in accordance with an embodiment of the present invention.

[000235] In operation 1450, a bitstream of an encoded video is received, entropy decoded, and analyzed to extract quantized transformation coefficients and encode information of a current image from the bitstream.

[000236] If the coding information includes information about an interpolation filter, the type of an interpolation filter required can be read from the information.

[000237] In operation 1460, according to various decoding methods determined based on a coding mode read from the coding information, inverse quantization and inverse transformation are performed on the quantized transformation coefficients, and residual data is additional, thereby mode by reconstructing the data in a spatial domain.

[000238] In operation 1470, encoded image data can be decoded by performing various decoding operations such as motion compensation and intra prediction based on the encoding mode.

[000239] Specifically, if encoded residual data and reference data are extracted based on the interpolated pixels in a sub-pel unit, motion compensation can be performed based on the interpolated pixels in a sub-pel unit. Among the interpolation filters to generate a sub-pel unit pixel value, an interpolation filter is selected differently based on a sub-pel unit interpolation location and a smoothness.

[000240] To efficiently perform image interpolation, interpolation filter coefficients can be determined selectively. Among the interpolation filter coefficients previously stored in memory, a desired interpolation filter can be selected according to a sub-pel unit interpolation location, a smoothness, a number of filter taps, a bit depth, a factor scale, an interpolation filtering base function based on the transformation, and a color component, and interpolation can be performed to generate the sub-pel unit pixel value. Since motion compensation is performed on the interpolated pixels using the interpolated filter coefficients previously stored in memory, the calculation speed can be increased.

[000241] A reference image and a reference region are determined using the reference data, and the sub-pel unit pixel value can be generated by performing interpolation filtering on two or more whole unit reference pixels of the reference structure. Motion compensation can be performed by combining the generated sub-pel unit pixel value and the residual data, and thus prediction decoding can be performed.

[000242] In operation 1480, a current image is reconstructed using the pixel values obtained by performing predictive decoding, and thus a video is decoded.

Video Coding and Decoding using Smoothing Interpolation based on Coding Units having Tree Structures

[000243] Video encoding and decoding apparatus using a smoothing interpolation filter based on coding units having a tree structure, and video encoding and decoding methods corresponding to the video encoding and decoding apparatus, in accordance with the Embodiments of the present invention will now be described in detail with reference to Figures 13 to 27.

[000244] The video coding apparatus 100 can encode a video based on coding units and transformation units having a tree structure.

[000245] A current image of a video can be divided based on a maximum coding unit for the current image. If the current image is larger than the maximum coding unit, image data of the current image may be divided into at least one maximum coding unit. The maximum coding unit may be a data unit having a size of 32x32, 64x64, 128x128, 256x256, etc., wherein a format of the data unit is a square having a width in length in squares of 2. The encoder 120 may encode the image data of each of at least one maximum encoding unit.

[000246] A coding unit according to an embodiment of the present invention can be characterized by a maximum size and depth. Depth denotes the number of times the coding unit is spatially divided from the maximum coding unit, and as the depth increases, the deeper coding units according to the depths can be divided from the maximum coding unit. from maximum encoding to a minimum encoding unit. A maximum coding unit depth is a higher depth and a minimum coding unit depth is a lower depth. As a size of a coding unit corresponding to each depth decreases as the depth of the maximum coding unit deepens, a coding unit corresponding to a higher depth may include a plurality of coding units corresponding to lower depths.

[000247] As described above, the image data of the current image is divided into maximum coding units according to the maximum size of the coding unit, and each of the maximum coding units may include deeper coding units that are divided according to the depths. As the maximum coding unit according to an embodiment of the present invention is divided according to depths, image data of a spatial domain included in the maximum coding unit can be hierarchically classified according to depths.

[000248] A maximum depth and a maximum size of a coding unit, which limit the total number of times a height and a width of the maximum coding unit are hierarchically divided, can be predetermined.

[000249] Encoder 120 encodes at least one split region obtained by dividing a region of the maximum coding unit according to depths, and determines the depth to produce image data finally encoded according to at least one split region. In other embodiments, the encoder 120 determines a coded depth by encoding the image data into the deepest coding units according to the depths, according to the maximum coding unit of the current image, and selecting a depth having the smallest error of codification.

[000250] The encoder 120 may output the encoded image data from the encoding unit corresponding to the determined encoded depth. Furthermore, the encoder 120 can transmit information about the determined encoded depth to the output unit 130 such that the encoded depth information can be encoded as coding information.

[000251] The image data in the maximum coding unit is encoded based on the deepest coding units corresponding to at least a depth equal to or below the maximum depth, and the results of encoding the image data are compared based on each one of the deepest coding units. A depth having the minimum coding error can be selected after comparing the coding errors of the deeper coding units. At least one encoded depth can be selected for each maximum encoding unit.

[000252] The size of the maximum coding unit is divided as a coding unit is divided hierarchically according to depths, and as the number of coding units increases. Furthermore, even if the coding units correspond to the same depth at a maximum coding unit, it is determined whether to divide each of the coding units corresponding to the same depth to a lower depth by measuring a coding error of the coding data. image of each coding unit separately. Consequently, even when image data is included in a maximum coding unit, the image data is divided into regions according to depths, and coding errors may differ according to the regions in a maximum coding unit, and thus the encoded depths may differ according to regions in the image data. Thus, one or more coded depths can be determined in a maximum coding unit, and the image data of the maximum coding unit can be divided according to the coding units of at least one coded depth.

[000253] Accordingly, encoder 120 can determine coding units having a tree structure included in the maximum coding unit. "Coding units having a tree structure" according to an embodiment of the present invention include coding units corresponding to a depth determined to be the coded depth, among all the deepest coding units included in the maximum coding unit. A coding unit of a coded depth can be determined hierarchically according to the depths in the same region of the maximum coding unit, and can be determined independently in different regions. Similarly, a depth encoded in a current region can be determined independently from a depth encoded in another region.

[000254] A maximum depth according to an embodiment of the present invention is an index related to the number of times the division is performed from a maximum coding unit to a minimum coding unit. A first maximum depth according to an embodiment of the present invention may denote the total number of times the division is performed from the maximum coding unit to the minimum coding unit. A second maximum depth according to an embodiment of the present invention may denote the total number of depth levels from the maximum coding unit to the minimum coding unit. For example, when a maximum coding unit depth is 0, a depth of a coding unit, in which the maximum coding unit is divided once, can be set to 1, and a depth of a coding unit, in which the maximum coding unit is divided twice, can be set to 2. Here, if the minimum coding unit is a coding unit in which the maximum coding unit is divided four times, 5 levels deep, from depth 0 , 1, 2, 3 and 4 exist, and so the first maximum depth can be set to 4, and the second maximum depth can be set to 5.

[000255] Prediction coding and transformation can be performed according to the maximum coding unit. Predictive coding and transformation are also performed based on the deepest coding units according to a depth equal to or a depth less than the maximum depth according to the maximum coding unit.

[000256] As the number of deeper coding units increases whenever the maximum coding unit is divided according to depths, coding including prediction coding and transformation is performed on all deeper coding units generated at as the depth increases. For convenience of description, prediction coding and transformation will now be described based on the coding unit of a current depth, in a maximum coding unit.

[000257] The video encoding apparatus 100 may variously select a size or format of a data unit for encoding the image data. To encode the image data, operations such as prediction coding, transformation, and entropy coding are performed, and this time, the same data unit can be used for all operations or different data units can be used for each. operation.

[000258] For example, the video coding apparatus 100 may select not only a coding unit to encode the image data, but also a data unit other than the coding unit in order to perform predictive coding on the image data. image in the encoding unit.

[000259] To perform prediction coding at the maximum coding unit, prediction coding can be performed based on a coding unit corresponding to a coded depth, that is, based on a coding unit that is no longer divided into coding units corresponding to a lower depth. Then, the coding unit that is no longer divided and becomes a basic unit for prediction coding will now be referred to as a “prediction unit”. A partition obtained by dividing the prediction unit may include a prediction unit or a data unit obtained by dividing at least one of a height and a width of the prediction unit.

[000260] For example, when a 2Nx2N coding unit (where N is a positive integer) is no longer divided and becomes a 2Nx2N prediction unit, a partition size may be 2Nx2N, 2NxN, Nx2N, or NxN. Examples of a partition type include symmetric partitions that are obtained by symmetrically dividing a height or width of the prediction unit, partitions obtained by asymmetric division of the height or width of the prediction unit, such as 1:n or n:1, partitions which are obtained by geometric division of the prediction unit, and partitions having arbitrary formats.

[000261] A prediction mode of the prediction unit can be at least one of an intra mode, an inter mode, and a skip mode. For example, intra mode or inter mode can be realized in the partition of 2Nx2N, 2NxN, Nx2N or NxN. Furthermore, jump mode can be performed only on the 2Nx2N partition. Coding is performed independently in a prediction unit in a coding unit, thereby selecting a prediction mode having a minimum coding error.

[000262] The video encoding apparatus 100 may also perform transformation on the image data into a coding unit based not only on the coding unit for encoding the image data, but also based on a data unit that is different from the encoding unit.

[000263] To perform the transformation in the coding unit, the transformation can be performed based on a transformation unit having a size less than or equal to the coding unit. For example, the transformation unit for the transformation may include a data unit for an intra mode and a data unit for an inter mode.

[000264] Similar to the coding unit, the transformation unit in the coding unit can be recursively divided into regions of smaller sizes, so that the transformation unit can be independently determined into region units. Thus, the residual data in the coding unit can be divided according to the transformation units having the tree structure according to the transformation depths.

[000265] A transformation depth indicating the number of times division is performed to reach the transformation unit by dividing the height and width of the coding unit can also be established in the transformation unit. For example, in a current encoding unit of 2Nx2N, a transformation depth may be 0 when the size of a transformation unit is also 2Nx2N, may be 1 when the size of the transformation unit is NxN, and may be of 2 when the size of the transformation unit is N/2xN/2. In other words, transformation units having a tree structure can be established according to transformation depths.

[000266] Coding information according to a coded depth requires not only information about the coded depth, but also information about prediction and transformation coding. Accordingly, the encoder 120 not only determines a coded depth having a minimum coding error, but also determines a partition type in a prediction unit, a prediction mode according to the prediction units, and a size of a prediction unit. transformation to transformation. For inter prediction, the coding information according to a coded depth may include information related to interpolation filtering to interpolate sub-pel units.

[000267] Furthermore, the encoder 120 can perform transformation using transformation units having a tree structure for encoding the coding units, based on a maximum division level of the transformation units, which is adjusted in advance and accordingly. restrictively in each maximum coding unit or in a current coding unit.

[000268] In each of the deeper coding units according to the depths, a basic transformation unit having a size less than or equal to a coding unit can be hierarchically divided into lower transformation units or transformation depths. Transformation units having a tree structure may include a basic transformation unit having a maximum size that is currently allowed, and lower level transformation units with respect to a maximum division level that is allowed for the encoding units.

[000269] After performing the transformation at each level according to a transformation depth in a current coding unit, the encoder 120 can determine transformation units having a tree structure, which are independent of the transformation units of adjacent regions and based on a hierarchical structure between transformation units in the same region according to the transformation depths.

[000270] In other words, transformation units having a tree structure can be determined by performing transformation on each coding unit using transformation units of different sizes and then comparing the results of the transformation. Although a coding unit is being determined, a transformation unit to transform the coding unit can be determined. Whenever coding units according to each one or more depths are encoded, transformation units according to each one or more transformation depths can be used to perform the transformation.

[000271] A transformation unit having a minimum coding error must be determined for each coding unit. To determine a transformation depth having a minimum coding error in a transformation unit, coding errors can be measured and compared across all deeper transformation units according to the depths. A transformation unit can be determined as a unit to minimize a transformation error of a coding unit.

[000272] Consequently, as a combination of a deeper coding unit and a deeper transformation unit according to the depths, which has a minimum coding error, is determined individually in each region of a maximum coding unit, coding units having a tree structure and transformation units having a tree structure can be determined.

[000273] Methods of determining coding units having a tree structure, partitions, and transformation units having a tree structure into a maximum coding unit, in accordance with embodiments of the present invention, will be described in detail later with reference to Figures 15 to 25.

[000274] Encoder 120 can measure a coding error of deeper coding units according to depths using rate distortion optimization based on Lagrangian multipliers.

[000275] The video coding apparatus 100 can output the image data of the maximum coding unit, which is coded based on at least one coded depth determined by the coder 120, and information about a coding mode in accordance with the encoded depth, which is encoded by the output unit 130, in the form of a bit stream.

[000276] Information about the coding mode of deeper coding units according to depths, which is determined how an image is coded based on coding units, prediction units, and transformation units having a tree structure , can be included in a header, a sequence parameter set (SPS), or a picture parameter set (PPS) of a bitstream.

[000277] Encoded image data can be obtained by encoding residual data of an image.

[000278] Information about the coding mode according to the coded depth may include information about the coded depth, about the type of partition in the prediction unit, the prediction mode, and the size of the transformation unit.

[000279] The encoded depth information can be defined by using information divided according to depths, which represents whether coding is performed in coding units of a lower depth rather than a current depth. If the current depth of the current coding unit is the coded depth, the image data in the current coding unit is coded and output, and thus the split information can be set not to split the current coding unit to a smaller depth. Alternatively, if the current depth of the current coding unit is not the coded depth, coding is performed at the coding unit of the lower depth, and thus the split information can be set to divide the current coding unit to obtain the coding units from the lower depth.

[000280] If the current depth is not the encoded depth, encoding is performed in the encoding unit that is divided to the encoding unit of the lower depth. Since there is at least one bottom-depth coding unit in a current depth coding unit, coding is performed repeatedly in each bottom-depth coding unit, and thus coding can be performed recursively for the coding units having the same depth.

[000281] As coding units having a tree structure are determined to a maximum coding unit, and information about at least one coding mode is determined to a coding unit of a coded depth, information about at least one coding mode coding unit can be determined for a maximum coding unit. Furthermore, a coded depth of the image data of the maximum coding unit may be different according to the locations, since the image data is divided hierarchically according to the depths, and thus the information about the coded path and the mode encoding can be adjusted for the image data.

[000282] Accordingly, the output unit 130 can assign coding information about a corresponding coded path and a coding mode to at least one of the coding unit, prediction unit, and a coding mode to at least one of the coding, prediction unit, and a minimum unit included in the maximum coding unit.

[000283] The minimum unit according to an embodiment of the present invention is a rectangular data unit obtained by dividing the minimum coding unit constituting the lowest depth by 4. Alternatively, the minimum unit may be a maximum rectangular data unit that can be included in all coding units, prediction units, partition units, and transformation units included in the maximum coding unit.

[000284] For example, the coding information produced through the output unit 130 can be classified into coding information according to coding units, and coding information according to prediction units. The coding information according to the coding units may include information about the prediction mode and the size of the partitions. The coding information according to the prediction units may include information about an estimated direction of an inter-mode, relative to an inter-mode reference image index, relative to a motion vector, relative to a chroma component. in an intra mode, and in relation to an intra mode interpolation method.

[000285] Information about a maximum coding unit size defined according to images, slices, or GOPs, and information about a maximum depth can be inserted into a header, an SPS, or a PPS of a bit stream.

[000286] In the video coding apparatus 100, the deepest coding unit may be a coding unit obtained by dividing a height or width of a coding unit of a higher depth, which is one layer above, by two. In other words, when the coding unit size of the current depth is 2Nx2N, the coding unit size of the lower depth is NxN. Furthermore, the coding unit of the current depth having the size of 2Nx2N can include a maximum of 4 of the coding units of the lower depth.

[000287] Accordingly, the video coding apparatus 100 can form the coding units having the tree structure by determining the coding units having an optimal format and an optimal size for each maximum coding unit, based on the size of the unit maximum encoding and maximum depth determined considering the characteristics of the current image. Furthermore, since coding can be performed on each maximum coding unit using any of several prediction modes and transformations, an optimal coding mode can be determined by considering the coding unit characteristics of various image sizes.

[000288] Thus, if an image having high resolution or large amount of data is encoded in a conventional macro block, a number of macro blocks per image increases excessively. Consequently, a number of pieces of compressed information generated for each macro block increases, and thus it is difficult to transmit the compressed information and the efficiency of data compression decreases. However, by using video coding apparatus 100, image compression efficiency can be increased once a coding unit is adjusted while considering the characteristics of an image while increasing the maximum size of a coding unit while considering a image size.

[000289] The output unit 130 can encode and output coding information indicating a coding method used to encode a video based on coding units having a tree structure and transformation units having a tree structure. The coding information may include information about various coding modes of the coding units corresponding to a coded depth, and information about the coded depth.

[000290] Definitions of various terms, such as a coding unit, a depth, a prediction unit, a transformation unit, and information about various coding modes, for various operations of the video decoding apparatus 200, are identical to those described with reference to video coding apparatus 100.

[000291] Receiver 210 receives a bit stream of an encoded video. The receiver and extractor 220 analyze the received bit stream. The receiver and extractor 220 extract the encoded image data for each coding unit from the analyzed bit stream, wherein the coding units have a tree structure according to each maximum coding unit, and output the image data. image extracted to the decoder 230. The receiver and extractor 220 can extract information about a maximum size of a coding unit of a current image, from a header, an SPS, or a PPS about the current image.

[000292] Furthermore, the receiver and extractor 220 can extract coding information about the coding units having a tree structure according to each maximum coding unit from the analyzed bit stream. Information about a coded depth and a coding mode is extracted from the coding information. The extracted information about the encoded depth and encoding mode is output to the decoder 230. In other words, the image data in a bit stream can be divided into maximum coding unit so that the decoder 230 can decode the data of image for each maximum coding unit.

[000293] Information about the encoded depth and the encoding mode according to the maximum coding unit can be set to information about at least one coding unit corresponding to the encoded depth, and information about a coding mode can include information about a partition type of a corresponding coding unit that corresponds to the coded depth, relative to the prediction mode, and a size of a transformation unit. For inter prediction, information related to interpolation filtering to interpolate sub-pel units can be extracted from the coding information according to a coded depth. Furthermore, the division information according to depths can be extracted as the encoded depth information.

[000294] Information about the coded depth and the coding mode according to each maximum coding unit extracted by the receiver and extractor 220 is information about a coded depth and a coding mode determined to generate a minimum coding error when an encoder , such as video coding apparatus 100, repeatedly performs coding for each deepest coding unit according to the depths in accordance with each maximum coding unit. Accordingly, the video decoding apparatus 200 can reconstruct an image by decoding the image data according to a coded depth and a coding mode that generates the minimum coding error.

[000295] As the coding information about the coded depth, and the coding mode, can be assigned to a predetermined data unit among a corresponding coding unit, a prediction unit, and a minimum unit, the receiver and extractor 220 can extract the information about the encoded depth and encoding mode according to the predetermined data units. The predetermined data units to which the same information about the coded depth and the coding mode are assigned can be deduced to be the data units included in the same maximum coding unit.

[000296] Decoder 230 can determine at least one encoded depth of a maximum coding unit using information divided according to depths. If the split information represents that image data is no longer split at the current depth, the current depth is the encoded depth. Accordingly, decoder 230 may decode image data encoded from at least one coding unit corresponding to each depth encoded in the current maximum coding unit by using information about the partition type of the prediction unit, the prediction mode, and the size of the transformation unit for each coding unit corresponding to the coded depth, and output the image data of the current maximum coding unit.

[000297] In other words, data units containing coding information including the same division information can be grouped by observing the set of coding information assigned to the predetermined data unit among the coding unit, the prediction unit and the minimum unit, and the grouped data units can be considered to be one data unit to be decoded by decoder 230 in the same encoding mode.

[000298] The decoder 230 can reconstruct the current image by decoding the image data encoded in each maximum coding unit based on information about the encoded depth and the coding mode according to the maximum coding units. The partition type, prediction mode, and transformation unit can be read as the coding mode for each coding unit among the coding units having the tree structure included in each maximum coding unit. A decoding process may include a prediction including intra prediction and motion compensation and an inverse transformation.

[000299] The decoder 230 may perform intra prediction or motion compensation according to a partition and a prediction mode of each coding unit based on information about the partition type and the prediction mode of the prediction unit of the coding units. encoding having a tree structure.

[000300] Furthermore, the decoder 230 can read the structure of the transformation units having a tree structure and can perform inverse transformation on each coding unit based on the transformation units.

[000301] The video decoding equipment 90 can obtain information about at least one coding unit that generates the minimum coding error when coding is performed recursively for each maximum coding unit, and can use the information to decode the current image. In other words, coding units having the tree structure determined to be the optimal coding unit in each maximum coding unit can be decoded. Furthermore, the maximum size of the encoding unit is determined by considering the resolution and an amount of image data.

[000302] Consequently, even if the image data has high resolution and a large amount of data, the image data can be efficiently decoded and restored using a size of a coding unit and a coding mode, which are determined adaptively according to the characteristics of the image data, using information about an optimal coding mode received from an encoder.

[000303] Figure 15 is a diagram for describing a concept of hierarchical coding units according to an exemplary embodiment.

[000304] A size of a coding unit can be expressed in width x height, and can be 64x64, 32x32, 16x16 and 8x8. A 64x64 encoding unit can be divided into 64x64, 64x32, 32x64, or 32x32 partitions, and a 32x32 encoding unit can be divided into 32x32, 32x16, 16x32, or 16x16 partitions, a 16x16 encoding unit can be divided into 16x16, 16x8, 8x16, or 8x8 partitions, and an 8x8 encoding unit can be divided into 8x8, 8x4, 4x8, or 4x4 partitions.

[000305] In video data 310, a resolution is 1920x1080, a maximum size of a coding unit is 64, and a maximum depth is 2. In video data 320, a resolution is 1920x1080, a maximum size of encoding is 64, and a maximum depth is 3. In 330 video data, a resolution is 352x288, a maximum size of an encoding unit is 16, and a maximum depth is 1. The maximum depth shown in Figure 15 denotes a total number of divisions from a maximum encoding unit to a minimum decoding unit.

[000306] If a resolution is high or if an amount of data is large, a maximum size of a coding unit can be so large as to not only increase coding efficiency, but also accurately reflect the characteristics of a image. Accordingly, the maximum coding unit size of video data 310 and 320 having higher resolution than video data 330 may be 64.

[000307] Since the maximum depth of video data 310 is 2, coding units 315 of video data 310 may include a maximum coding unit having a long axis size of 64, and coding units having long axis sizes longs of 32 and 16 since the depths are deepened to two layers by dividing the maximum coding unit twice. However, because the maximum depth of the video data 330 is 1, the coding units 335 of the video data 330 may include a maximum coding unit having a long axis size of 16, and coding units having a long axis size long of 8 since the depths are deepened to a layer by dividing once from the maximum coding unit.

[000308] Since the maximum depth of video data 320 is 3, coding units 325 of video data 320 may include a maximum coding unit having a long axis size of 64, and coding units having long axis sizes long of 32, 16 and 8 as the depths are deepened to 3 layers by dividing the maximum coding unit three times. As the depth is deepened, detailed information can be expressed accurately.

[000309] Figure 16 is a block diagram of an image encoder 400 based on coding units, according to an exemplary embodiment.

[000310] The image encoder 400 performs operations of the coding unit determiner 120 of the video coding equipment 100 to encode the image data. In other words, an intra predictor 410 performs intra prediction on the coding units in an intra mode, within a current frame 405, and a motion estimator 420 and a motion compensator 425 perform inter estimation and motion compensation on the coding units in an intra mode. an inter mode within the current frame 405 using the current frame 405, and a reference frame 495.

[000311] To accurately perform motion estimation using reference pixels in sub-pel units, the motion estimator 420 and motion compensator 425 can generate pixels in sub-pel units by interpolating pixels in whole pel units. . An interpolation filter for generating pixels in sub-pel units may be the smoothing interpolation filter described above in relation to Figures 1 and 13A.

[000312] Data output from the intra predictor 410, the motion estimator 420, and the motion compensator 425 is output as a quantized transformation coefficient through the transformer 430 and a quantizer 440. The quantized transformation coefficient is restored as data in a spatial domain through an inverse quantizer 460 and an inverse transformer 470, and the restored data in the spatial domain is output as the reference frame 495 after being post-processed through a deblocking unit 480 and a filtering unit. loop 490. The quantized transformation coefficient may be output as a bit stream 455 through an entropy encoder 450.

[000313] In order for the image encoder 400 to be applied to the video coding equipment 100, all elements of the image encoder 400, i.e., the intra predictor 410, the motion estimator 420, the motion compensator 425, the transformer 430, the quantizer 440, the entropy encoder 450, the inverse quantizer 460, the inverse transformer 470, the deblocking unit 480, and the loop filtering unit 490 perform operations based on each coding unit among coding units having a tree structure while considering the maximum depth of each maximum coding unit.

[000314] Specifically, the intra predictor 410, the motion estimator 420, and the motion compensator 425 determine partitions and a prediction mode of each coding unit among the coding units having a tree structure while considering the maximum size and the maximum depth of a current maximum coding unit, and the transformer 430 determines the size of the transformation unit in each coding unit among the coding units having a tree structure.

[000315] Figure 17 is a block diagram of an image decoder 500 based on coding units, according to an exemplary embodiment.

[000316] An analyzer 510 analyzes the encoded image data to be decoded and information about the encoding required for decoding from a bit stream 505. The encoded image data is output as inverse quantized data through an entropy decoder 520 and an inverse quantizer 530, and the inverse quantized data is restored to image data in a spatial domain through an inverse transformer 540.

[000317] An intra predictor 550 performs intra prediction on the coding units in an intra mode with respect to image data in the spatial domain, and a motion compensator 560 performs motion compensation on the coding units in an inter mode, using a 585 reference frame.

[000318] To accurately perform motion estimation using reference pixels in sub-pel units, the motion compensator 560 can generate pixels in sub-pel units by interpolating pixels in whole pel units. An interpolation filter for generating pixels in sub-pel units may be the smoothing interpolation filter described above in relation to Figures 2 and 13B.

[000319] The image data in the spatial domain, which has passed through the intra predictor 550 and the motion compensator 560, can be output as a restored frame 595 after being post-processed through an unblocking unit 570 and a deblocking unit 570. loop filtering 580. Furthermore, image data that is post-processed through the deblocking unit 570 and the loop filtering unit 580 can be output as the reference frame 585.

[000320] To decode the image data in the decoder 95 of the video decoding equipment 90, the image decoder 500 may perform the operations that are performed after the analyzer 510.

[000321] For the image decoder 500 to be applied to the video decoding equipment 200, all elements of the image decoder 500, i.e., the analyzer 510, the entropy decoder 520, the inverse quantizer 530, the inverse transformer 540, the intra predictor 550, the motion compensator 560, the deblocking unit 570, and the loop filtering unit 580 perform operations based on the coding units having a tree structure for each maximum coding unit.

[000322] Specifically, the intra prediction 550 and the motion compensator 560 perform operations based on the partitions and a prediction mode for each of the coding units having a tree structure, and the inverse transformer 540 performs operations based on a size of one transformation unit for each encoding unit.

[000323] Figure 18 is a diagram illustrating deeper coding units according to depths, and partitions, according to an exemplary embodiment.

[000324] The video coding equipment 100 and the video decoding equipment 200 use hierarchical coding units in order to consider the characteristics of an image. A maximum height, a maximum width, and a maximum depth of the coding units can be determined adaptively according to the characteristics of the image, or can be adjusted differently by a user. Deeper coding unit sizes, depending on the depths, can be determined according to the predetermined maximum coding unit size.

[000325] In a hierarchical structure 600 of coding units, according to an exemplary embodiment, the maximum height and maximum width of the coding units are individually 64, and the maximum depth is 4. As a depth is deepened to Along a vertical axis of the hierarchical structure 600, a height and a width of the deepest coding unit are individually divided. Furthermore, a prediction unit and partitions, which are the bases for prediction coding of each deeper coding unit, are shown along a horizontal axis of the hierarchical structure 600.

[000326] In other words, a coding unit 610 is a maximum coding unit in the hierarchical structure 600, where a depth is 0 and a size, that is, a height by width, is 64x64. The depth is deepened along the vertical axis, and there is an encoding unit 620, having a size of 32x32, and a depth of 1; a coding unit 630, having a size of 16x16, and a depth of 2; a coding unit 640, having a size of 8x8, and a depth of 3; and a coding unit 650 having a size of 4x4 and a depth of 4. The coding unit 650 having a size of 4x4 and a depth of 4 is a minimum coding unit.

[000327] The prediction unit and partitions of a coding unit are arranged along the horizontal axis according to each depth. In other words, if the coding unit 610, having the size of 64x64, and the depth of 0, is a prediction unit, the prediction unit can be divided into partitions included in the coding unit 610, that is, a partition 610 partitions having a size of 64x64, 612 partitions having a size of 64x32, 614 partitions having a size of 32x64, or 616 partitions having a size of 32x32.

[000328] Similarly, a prediction unit of the coding unit 620 having the size of 32x32 and the depth of 1 can be divided into partitions included in the coding unit 620, that is, a partition 620 having a size of 32x32, partitions 622 having a size of 32x16, 624 partitions having a size of 16x32, and 626 partitions having a size of 16x16.

[000329] Similarly, a prediction unit of the coding unit 630 having the size of 16x16 and the depth of 2 can be divided into partitions included in the coding unit 630, that is, a partition having a size of 16x16 included in the coding unit 630. encoding 630, partitions 632 having a size of 16x8, partitions 634 having a size of 8x16, and partitions 636 having a size of 8x8.

[000330] Similarly, a prediction unit of the coding unit 640 having the size of 8x8 and the depth of 3 can be divided into partitions included in the coding unit 640, that is, a partition having a size of 8x8 included in the coding unit 640. encoding 640, partitions 642 having a size of 8x4, partitions 644 having a size of 4x8, and partitions 646 having a size of 4x4.

[000331] To determine the at least one coded depth of the coding units constituting the maximum coding unit 610, the decoding unit 11 of the video coding apparatus 10 performs coding for the coding units corresponding to each depth included in the coding unit. maximum encoding 610.

[000332] A number of deeper coding units, according to the depths including data in the same range and the same size, increases as the depth is deepened. For example, four coding units corresponding to a depth of 2 are required to cover the data that is included in one coding unit corresponding to a depth of 1. Consequently, to compare the results of coding the same data according to the depths , the coding unit corresponding to depth 1; and four coding units corresponding to depth 2; are individually coded.

[000333] To perform coding for each depth, a minimum coding error can be selected for the current depth by performing coding for each prediction unit in the coding units corresponding to the current depth, along the horizontal axis of the hierarchical structure 600 Alternatively, the minimum coding error can be searched for by comparing the minimum coding errors according to the depths, by performing coding for each depth as the depth is deepened along the vertical axis of the hierarchical structure 600. A depth and a partition having the minimum coding error in the coding unit 610 can be selected as the coded depth and a partition type of the coding unit 610.

[000334] Figure 19 is a diagram for describing a relationship between a coding unit 710 and a transformation unit 720, in accordance with an embodiment of the present invention.

[000335] The video encoding apparatus 100 or the video decoding apparatus 200 in accordance with one embodiment may encode or decode an image according to the encoding units having sizes less than or equal to a maximum encoding unit for each maximum coding unit. Transformation unit sizes for transformation during encoding can be selected based on data units that are not larger than a corresponding encoding unit.

[000336] For example, in the video coding apparatus 100 or 200, or the video decoding apparatus 200, if a size of the coding unit 710 is 64x64, the transformation can be carried out using the transformation units 720 having a size of 32x32.

[000337] Furthermore, data from the encoding unit 710 having the size of 64x64 can be encoded by performing transformation in each of the transformation units having the size of 32x32, 16x16, 8x8, and 4x4, which are smaller than 64x64, and then a transformation unit having the minimum coding error can be selected.

[000338] Figure 20 is a diagram for describing coding information of coding units corresponding to a coded depth, in accordance with an embodiment of the present invention.

[000339] The output unit 130 of the video encoding apparatus 100 can encode and transmit information 800 about a partition type, information 810 about a prediction mode, and information 820 about a size of a transformation unit for each processing unit. encoding corresponding to an encoded depth, as information about an encoding mode.

[000340] Information 800 indicates information about a format of a partition obtained by dividing a prediction unit from a current coding unit, wherein the partition is a data unit for prediction coding of the current coding unit. For example, a current coding unit CU_0 having a size of 2Nx2N can be divided into any of a partition 802 having a size of 2Nx2N, a partition 804 having a size of 2NxN, an 806 partition having a size of Nx2N, and an 808 partition having a size of NxN. Here, information 800 about a partition type is established to indicate one of the partition 804 having a size of 2NxN, the partition 806 having a size of Nx2N, and the partition 808 having a size of NxN.

[000341] Information 810 indicates a prediction mode for each partition. For example, information 810 may indicate a prediction coding mode performed on a partition indicated by information 800, that is, an intra mode 812, an inter mode 814, or a skip mode 816.

[000342] Information 820 indicates a transformation unit to be based on when the transformation is performed on a current coding unit. For example, the transformation unit may be a first intra transformation unit 822, a second intra transformation unit 824, a first inter transformation unit 826, or a second intra transformation unit 828.

[000343] The receiver and extractor 220 of the video decoding apparatus 200 can extract and use information 800, 810 and 820 for decoding, according to each deeper coding unit.

[000344] Figure 21 is a diagram of deeper coding units according to depths, according to an exemplary embodiment.

[000345] Division information can be used to indicate a change in depth. The division information indicates whether a coding unit of a current depth is divided into coding units of a lower depth.

[000346] A prediction unit 910 for predictive coding of a coding unit 900 having a depth of 0 and a size of 2N_0x2N_0 may include partitions of a partition type 912 having a size of 2N_0x2N_0, a partition type 914 having a size of 2N_0xN_0, a partition type 916 having a size of N_0x2N_0, and a partition type 918 having a size of N_0xN_0. Figure 9 illustrates only the partition types 912 to 918 that are obtained by symmetric division of the prediction unit 910, but a partition type is not limited thereto, and the partitions of the prediction unit 910 may include asymmetric partitions, partitions having a predetermined shape and partitions having a geometric shape.

[000347] Predictive coding is performed repeatedly on one partition having a size of 2N_0x2N_0, two partitions having a size of 2N_0xN_0, two partitions having a size of N_0x2N_0, and four partitions having a size of N_0xN_0, according to each partition type. Predictive coding in an intra mode and an inter mode can be performed on partitions having the sizes of 2N_0x2N_0, N_0x2N_0, 2N_0xN_0 and N_0xN_0. Predictive coding in a skip mode is performed only on the partition having the size of 2N_0x2N_0.

[000348] Coding errors including prediction coding in partition types 912 to 918 are compared, and the minimum coding error is determined across partition types. If a coding error is minor in one of the partition types 912 to 916, the prediction unit 910 may not be partitioned to a lower depth.

[000349] If the encoding error is the smallest in partition type 918, a depth is changed from 0 to 1 to divide partition type 918 in operation 920, and encoding is performed repeatedly in encoding units 930 having a depth of 2 and a size of N_0xN_0 to look for a minimum coding error.

[000350] A prediction unit 940 for prediction coding of the coding unit 930 having a depth of 1 and a size of 2N_1x2N_1(=N_0xN_0) may include partitions of a partition type 942 having a size of 2N_1x2N_1, a partition type 944 having a size of 2N_1xN_1, a partition type 946 having a size of N_1x2N_1, and a partition type 948 having a size of N_1xN_1.

[000351] If a coding error is the smallest in partition type 948, a depth is changed from 1 to 2 for dividing partition type 948 in operation 950, and coding is performed repeatedly in coding units 960, which have a depth of 2 and a size of N_2xN_2 to look for a minimum coding error.

[000352] When a maximum depth is d, the division operation according to each depth can be carried out until when a depth becomes d-1; and division information can be encoded even when a depth is one of 0 to d-2. In other words, when coding is performed up to when the depth is d-1 after a coding unit corresponding to a depth of d-2 is divided in operation 970, a prediction unit 990 for prediction coding of a coding unit 980 having a depth of d-1 and a size of 2N_(d-1)x2N_(d-1) may include partitions of a partition type 992 having a size of 2N_(d-1)x2N_(d-1), a partition type 994 having a size of 2N_(d-1)xN_(d-1), a partition type 996 having a size of N_(d-1)x2N_(d-1), a partition type 998 having a size of N_(d-1)xN_(d-1).

[000353] Predictive coding can be performed repeatedly on a partition having a size of 2N_(d- 1)x2N_(d-1), two partitions having a size of 2N_(d-1)xN_(d- 1), two partitions having a size of N_(d-1)x2N_(d-1), four partitions having a size of N_(d-1)xN_(d-1) among partition types 992 to 998 to search for a partition type having minimal coding error.

[000354] Even when partition type 998 has the minimum coding error, as a maximum depth is d, a CU_(d-1) coding unit having a depth of d-1 is no longer partitioned to a lower depth , and a coded depth for the coding units constituting a current maximum coding unit 900 is determined to be d-1 and a partition type of the current maximum coding unit 900 may be determined to be N_(d-1)xN_( d-1). Furthermore, since the maximum depth is d and a minimum coding unit 980 having the lowest depth of d-1 is no longer split to a lower depth, splitting information for the minimum coding unit 980 is not established.

[000355] Data unit 999 may be a minimum unit for the current maximum coding unit. A minimum unit according to an exemplary embodiment may be a rectangular data unit obtained by dividing a minimum coding unit 980 by 4. By performing coding repeatedly, video coding equipment 100 may select a depth having the error of minimum coding by comparing the coding errors according to the depths of the coding unit 900 to determine a coded depth, and establishing a corresponding partition type and a prediction mode as a coding mode of the coded depth.

[000356] As such, the minimum coding errors according to the depths are compared at all depths from 1 to d, and the depth having the smallest coding error can be determined as a coded depth. The encoded depth, the prediction unit partition type, and the prediction mode can be encoded and transmitted as information about a coding mode. Furthermore, as a coding unit is divided from a depth of 0 to a coded depth, only the division information of the coded depth is set to 0, and the division information of depths excluding the coded depth is set to 1. .

[000357] The receiver and extractor 220 of the video decoding apparatus 200 may extract and use information about the encoded depth and prediction unit of the coding unit 900 to decode the partition 912. The video decoding apparatus 200 may determining a depth, in which the divided information is 0, as an encoded depth using the information divided according to the depths, and using the information about an encoding mode of the corresponding depth for decoding.

[000358] Figures 22 to 24 are diagrams for describing a relationship between coding units 1010, prediction units 1060, and transformation units 1070, according to an exemplary embodiment.

[000359] The coding units 1010 are coding units that have a tree structure, corresponding to the coded depths determined by the video coding apparatus 100, in a maximum coding unit. The prediction units 1060 are partitions of the prediction units of each of the coding units 1010, and the transformation units 1070 are transformation units of each of the coding units 1010.

[000360] When a depth of a maximum coding unit is 0 in coding units 1010, the depths of coding units 1012 and 1054 are 1, the depths of coding units 1014, 1016, 1018, 1028, 1050, and 1052 are 2, the depths of the decoding units 1020, 1022, 1024, 1026, 1030, 1032, and 1048 are 3, and the depths of the encoding units 1040, 1042, 1044, and 1046 are 4.

[000361] In prediction units 1060, some coding units 1014, 1016, 1022, 1032, 1048, 1050, 1052, 1054 are obtained by dividing the coding units into coding units 1010. In other words, partition types in the coding units 1014, 1022, 1050, and 1054 have a size of 2NxN, the partition types in coding units 1016, 1048, and 1052 have a size of Nx2N, and a partition type of coding unit 1032 has a size of NxN. The prediction units and partitions of the coding units 1010 are less than or equal to each coding unit.

[000362] Transformation or inverse transformation is performed on the image data of the coding unit 1052 in the transformation units 1070 into a data unit that is smaller than the coding unit 1052. Additionally, the coding units 1014, 1016, 1022, 1032, 1048, 1050 and 1052 in the transformation units 1070 are different from those in the prediction units 1060 in terms of sizes and shapes. In other words, video encoding and decoding apparatus 100 and 200 can perform intra prediction, motion estimation, motion compensation, transformation, and inverse transformation individually on a data unit in the same coding unit.

[000363] Consequently, coding is performed recursively in each of the coding units having a hierarchical structure in each region of a maximum coding unit to determine an optimal coding unit, and thus coding units having a structure of recursive tree can be obtained. The coding information may include partition information about a coding unit, information about a partition type, information about a prediction mode, and information about a size of a transformation unit. Table 1 shows the coding information that can be established by video encoding and video decoding apparatus 100 and 200. Table 1

[000364] The output unit 130 of the video coding apparatus 100 can output the coding information about the coding units having a tree structure, and the receiver and extractor 220 of the video decoding apparatus 200 can extract the coding information about the coding units having a tree structure from a received bit stream.

[000365] Division information indicates whether a current coding unit is divided into coding units of a lower depth. If the split information of a current depth d is 0, a depth, in which a current coding unit is no longer split at a lower depth, is an encoded depth, and thus the information about a partition type, the prediction mode , and a size of one transformation unit, can be set to the encoded depth. If the current coding unit is further divided according to the split information, coding is carried out independently into four split coding units of a lower depth.

[000366] A prediction mode can be one of: an intra mode, an inter mode, and a jump mode. Intra-mode and inter-mode can be set on all partition types, and skip mode is only set on a partition type having a size of 2Nx2N.

[000367] The partition type information may indicate symmetric partition types having sizes of 2Nx2N, 2NxN, Nx2N, and NxN, which are obtained by symmetrically dividing a height or a width of a prediction unit, and asymmetric partition types having sizes of 2NxnU, 2NxnD, nLx2N and nRx2N, which are obtained by asymmetrically dividing the height or width of the prediction unit. The asymmetric partition types having the sizes of 2NxnU and 2NxnD can be obtained respectively by dividing the height of the prediction unit into 1:3 and 3:1, and the asymmetric partition types having the sizes of nLx2N and nRx2N can be obtained respectively by dividing the width of the prediction unit into 1:3 and 3:1.

[000368] The size of the transformation unit can be adjusted to be of two types in intra mode and two types in inter mode. In other words, if the split information of the transformation unit is 0, the size of the transformation unit can be 2Nx2N, which is the size of the current coding unit. If split information of the transformation unit is 1, the transformation units can be obtained by dividing the current coding unit. Furthermore, if a division type of the current coding unit having the size of 2Nx2N is an asymmetric partition type, a size of a transformation unit may be NxN, and if the partition type of the current coding unit is a type of asymmetric partition, the size of the transformation unit can be N/2xN/2.

[000369] The coding information about the coding units having a tree structure may include at least one of a coding unit corresponding to a coded depth, a prediction unit, and a minimum unit. The coding unit corresponding to the coded depth may include at least one of a prediction unit and a minimum unit containing the same coding information.

[000370] Consequently, it is determined whether adjacent data units are included in the same coding unit corresponding to the coded depth by comparing the coding information of adjacent data units. Furthermore, a corresponding coding unit corresponding to a coded depth is determined using the coding information of a data unit, and thus a distribution of the coded depths in a maximum coding unit can be determined.

[000371] Consequently, if a current coding unit is predicted based on the coding information of adjacent data units, coding information of data units in deeper coding units adjacent to the current coding unit can be directly queried and used .

[000372] Alternatively, if a current coding unit is predicted based on the coding information of adjacent data units, the data units adjacent to the current coding unit are searched using the coding information of the data units, and the searched adjacent coding units can be referred to for prediction of the current coding unit.

[000373] Figure 25 is a diagram for describing a relationship between a coding unit, a prediction unit or a partition, and a transformation unit, in accordance with the coding mode information of Table 1.

[000374] The maximum coding unit 1300 includes coding units 1302 1304, 1306, 1312, 1314, 1316 and 1318 of coded depths. Here, as the coding unit 1318 is a coding unit of a coded depth, the split information can be set to 0. Information about a partition type of the coding unit 1318 having a size of 2Nx2N can be set to be a of: a partition type 1322 having a size of 2Nx2N, a partition type 1324 having a size of 2NxN, a partition type 1326 having a size of Nx2N, a partition type 328 having a size of NxN, a partition type 1332 having a size of 2NxnU, a partition type 1334 having a size of 2NxnD, a partition type 1336 having a size of nLx2N, and a partition type 1338 having a size of nRx2N.

[000375] A transformation unit division information (TU size flag) is a type of transformation index; a size of a transformation unit corresponding to a transformation index can be modified according to a prediction unit type or a partition type of a coding unit.

[000376] For example, when the partition type is set to be symmetric, that is, the partition type 1322 (2Nx2N), 1324(2NxN), 1326 (Nx2N) or 1328 (NxN), a transformation unit 1342 having a size of 2Nx2N is established if split information (TU size flag) of a transformation unit is 0, and a transformation unit 1344 having a size of NxN is established if split information (TU size flag) of a transformation is 1.

[000377] When the partition type is established as being asymmetric, that is, the partition type 1332 (2NxnU), 1334 (2NxnD), 1336 (nLx2N) or 1338 (nRx2N), a transformation unit 1352 having a size of 2Nx2N is established if a TU size flag is 0, and a transformation unit 1354 having a size of N/2xN/2 is established if a TU size flag is 1.

[000378] Referring to Figure 21, the TU size flag is a flag having a value of 0 or 1, but the TU size flag is not limited to 1 bit, and the transformation unit can be divided hierarchically having a tree structure while the TU size flag increases from 0. The TU size flag can be used as an example of a transformation index.

[000379] In this case, the size of a transformation unit that was actually used can be expressed using a size indicator TU of a transformation unit, according to an exemplary embodiment, together with a maximum size and a size minimum of the transformation unit. According to an exemplary embodiment, video coding equipment 80 is capable of encoding maximum transformation unit size information, minimum transformation unit size information, and a maximum TU size indicator. The result of encoding the maximum transformation unit size information, minimum transformation unit size information, and the maximum TU size indicator can be input into an SPS. According to an exemplary embodiment, video coding equipment 200 can decode video using the maximum transformation unit size information, the minimum transformation unit size information, and the maximum TU size indicator.

[000380] For example, (a) if the size of a current encoding unit is 64x64 and a maximum transformation unit size is 32x32, then the size of a transformation unit can be 32x32 when a TU size indicator is 0 , can be 16x16 when the TU size indicator is 1, and can be 8x8 when the TU size indicator is 2.

[000381] As another example, (b) if the current encoding unit size is 32x32 and a minimum transformation unit size is 32x32, then the transformation unit size can be 32x32 when the TU size indicator is 0. Here, the TU size indicator cannot be set to a value other than 0, since the size of the transformation unit cannot be smaller than 32x32.

[000382] As another example, (c) if the current coding unit size is 64x64 and a maximum TU size indicator is 1, then the TU size indicator can be 0 or 1. Here, the TU size indicator does not can be set to a value other than 0 or 1.

[000383] Thus, if it is defined that the maximum TU size indicator is "MaxTransformSizeIndex", a minimum transformation unit size is "MinTransformSize", and a transformation unit size is "RootTuSize" when the TU size indicator is 0, then a current minimum transformation unit size "CurrMinTuSize" that can be determined in a current coding unit, can be defined by Equation (1):

[000384] Compared to the current minimum transformation unit size "CurrMinTuSize" that can be determined in the current coding unit, a transformation unit size "RootTuSize" when the TU size indicator is 0, can denote a size of maximum transformation unit that can be selected in the system. In Equation (1), "RootTuSize(2AMaxTransformSizeIndex)" denotes a transformation unit size when the transformation unit size "RootTuSize", when the TU size indicator is 0, is divided the number of times corresponding to the size indicator Maximum TU, and "MintransformSize" denotes a minimum transform size. Thus, a smaller value between "RootTuSize/(2AMaxTransformSizeIndex)" and "MinTransformSize" can be the current minimum transformation unit size "CurrMinTuSize" that can be determined in the current encoding unit.

[000385] According to an exemplary embodiment, the maximum RootTuSize transformation unit size may vary according to the type of prediction mode.

[000386] For example, if a current prediction mode is an inter mode, then RootTuSize can be determined using Equation (2) below. In Equation (2), "MaxTransformSize" denotes a maximum transformation unit size, and "PUSize" denotes a current prediction unit size.

[000387] That is, if the current prediction mode is inter mode, the "RootTuSize" transformation unit size when the TU size indicator is 0 may be a smaller value between the maximum transformation unit size and the current prediction unit size.

[000388] If a prediction mode of a current partition unit is an intra mode, "RootTuSize" can be determined using Equation (3) below. In Equation (3), "PartitionSize" denotes the size of the current partition unit.

[000389] That is, if the current prediction mode is intra mode, the "RootTuSize" transformation unit size when the TU size indicator is 0 may be a smaller value between the maximum transformation unit size and the size of the current partition drive.

[000390] However, the current maximum transformation unit size that varies depending on the type of a prediction mode in a partition unit, the root transformation unit size "RootTuSize", is only an example and the present invention is not limited to it.

[000391] Figure 26 is a flowchart of a video coding method using a smoothing interpolation filter based on coding units having a tree structure, in accordance with an embodiment of the present invention.

[000392] In operation 2610, to encode a current image from an input video, the current image is divided into at least one maximum coding unit. Each of the at least one division region, which is obtained by dividing a region of each maximum coding unit according to the depths, can be encoded. To encode each division region according to the depths, transformation and quantization are performed on a result of inter prediction, based on sub-pel unit interpolation, and intra prediction.

[000393] Here, a splitting depth for producing a final coding result according to the at least one splitting region can be determined by comparing the coding results of the splitting regions according to the depths, and coding units included. in a current maximum coding unit and having a tree structure can be determined. As coding units having a tree structure, transformation units having a tree structure can be determined. In other words, as a coding result of an image, such as the determined coding units having a tree structure, a coding result of the transformation units having a tree structure can be produced as coded image data.

[000394] Inter prediction can be performed on each prediction unit or partition of the coding unit. Movement of a current prediction unit or partition can be predicted with reference to pixels generated by performing sub-pel unit interpolation. Among the interpolation filters to generate a sub-pel unit pixel value, an interpolation filter is selected differently based on a sub-pel unit interpolation location and a smoothness. In order to efficiently perform image interpolation, interpolation filter coefficients can be determined selectively.

[000395] Among the interpolation filter coefficients previously stored in memory, a desired interpolation filter can be selected according to a sub-pel unit interpolation location, a smoothness, the number of filter taps, a bit depth , a scale factor, a transformation-based interpolation filtering base function, and a color component, and interpolation can be performed to generate the pixel value of the sub-pel unit.

[000396] In operation 2620, image data obtained as the final coding result according to at least one division region of each maximum coding unit, and information about the coded depth and coding mode are output as a stream of bits.

[000397] Coding mode information may include information about encoded depth or division information, information about a partition type of a prediction unit, information about a prediction mode, and information about a tree structure of prediction units. transformation. The coding information may include information about an interpolation filter used to perform sub-pel unit prediction coding. The encoded information about the encoding mode can be transmitted to a decoding apparatus together with the encoded image data.

[000398] Figure 27 is a flowchart of a video decoding method using a smoothing interpolation filter based on coding units having a tree structure, in accordance with an embodiment of the present invention.

[000399] In operation 2710, a bit stream of an encoded video is received and analyzed.

[000400] In operation 2720, encoded image data of a current image assigned to a maximum coding unit, and information about a coded depth and a coding mode according to the maximum coding units are extracted from the bit stream analyzed. Information about an interpolation filter required to perform sub-pel unit motion compensation can be extracted from the coding information.

[000401] Information about the encoded depth and the encoding mode can be extracted from the encoding information. According to the information about the coding depth and coding mode, a maximum coding unit can be divided into coding units having a tree structure. Furthermore, according to the information about a tree structure of the transformation units included in the extracted information, transformation units having a tree structure according to the transformation depths in the coding units can be determined.

[000402] In operation 2730, by using information about the coded depth and the coding mode according to each maximum coding unit, image data of each maximum coding unit can be decoded based on the coding units having a structure tree structure, prediction units, and transformation units having a tree structure. As a current coding unit is decoded based on information about the encoded depth and coding mode, a current coding unit can be transformed inversely by using a transformation unit determined from among the transformation units having a tree structure.

[000403] Coded image data can be decoded by performing various decoding operations such as motion compensation and intra prediction on each prediction unit or partition of the coding unit based on the coding mode.

[000404] Specifically, if coded residual data and reference data are extracted based on the interpolated pixels in a sub-pel unit, motion compensation in a current prediction unit or a current partition can be performed based on the interpolated pixels in the sub-pel units. Among the interpolation filters for generating a sub-pel unit pixel value, an interpolation filter can be selected differently based on a sub-pel unit interpolation location and a smoothness.

[000405] To efficiently perform image interpolation, the interpolation filter coefficients can be determined selectively. Among the interpolation filter coefficients previously stored in memory, a desired interpolation filter can be selected according to a sub-pel unit interpolation location, a smoothness, the number of filter taps, a bit depth, a factor scale, an interpolation filtering base function based on the transformation, and a color component, and interpolation can be performed to generate the sub-pel unit pixel value.

[000406] A reference image and a reference region are determined using the reference data, and the sub-pel unit pixel value can be generated by performing interpolation filtering on two or more whole unit reference pixels of the reference image. Motion compensation can be performed on the current prediction unit or current partition by combining the generated sub-pel unit pixel value and the residual data, and thus prediction decoding can be performed.

[000407] As each maximum coding unit is decoded, image data in a spatial domain can be reconstructed, and an image and a video that is a sequence of images can be reconstructed. The reconstructed video can be played back through a playback device, can be stored on a storage medium, or can be transmitted over a network.

[000408] Exemplary embodiments of the present invention can be recorded as computer programs and can be implemented on commonly used digital computers that execute programs using a computer-readable recording medium. Examples of computer-readable recording media include magnetic storage media (e.g., ROM, floppy disks, hard drives, etc.) and optical recording media (e.g., CD-ROMs or DVDs).

[000409] Although exemplary embodiments have been particularly shown and described above, it will be understood by those of ordinary skill in the art that various changes in form and details can be made thereto without departing from the essence and scope of the inventive concept as defined by the appended claims. . Exemplary embodiments should be considered only in a descriptive sense and not for limiting purposes. Therefore, the scope of the invention is defined not by the detailed description of the exemplary embodiments, but by the appended claims, and all differences within the scope will be considered to be included in the present inventive concept.

Claims (2)

1. MOTION COMPENSATION METHOD, the method characterized by comprising: determining, in a luma reference image, a luma reference block for predicting a current block, using a luma movement vector of the current block; generating a luma sample from a 1/2 pixel location in the luma reference block by applying an 8-lead interpolation filter to the luma samples from an integral pixel location of the luma reference image; determining, in a chroma reference image, a chroma reference block for predicting a current block, using a chroma motion vector of the current block; and generating a chroma sample from a 1/2 pixel location in the chroma reference block by applying an interpolation filter to chroma samples from an integral pixel location of the chroma reference image; wherein the 8-lead interpolation filter comprises eight filter coefficients to generate the luma sample from the 1/2 pixel location which are {-1, 4, -11, 40, 40, -11, 4, -1}. 2. METHOD according to claim 1, characterized in that generating the luma sample comprises scaling the luma sample generated by applying the 8-lead filter using a luma scaling factor of 64 in order to reconstruct a bit depth of the luma sample .