CN100542262C - Inverse Quantization Method Image Decoding Method and Device and Processor - Google Patents

Abstract

Picture decoding method according to the present invention is a kind of coding/decoding method that coded image is decoded of coming by re-quantization and inverse orthogonal transformation, wherein, handle as re-quantization, the quantization matrix of pantograph ratio of each element quantization step-length of definition be multiply by a certain multiplier as frequency translation or quantization step coefficient, and multiplication result be multiply by quantized value.

Description

Inverse Quantization Method Image Decoding Method and Device and Processor

Cross References to Related Applications

This application claims the benefit of the following U.S. Provisional Applications: Nos. 60/540,636 filed January 30, 2004; 60/551,690 filed March 9, 2004; 60/551,690 filed March 12, 2004 552,907; and No. 60/561,351, filed April 12, 2004, the contents of which are hereby incorporated by reference in their entirety.

technical field

The present invention relates to an image encoding method for efficiently compressing a moving image, an image decoding method for correctly decoding such a compressed moving image, an image encoding device, an image decoding device, and a program thereof.

Background technique

Recently, with the advent of the multimedia era that comprehensively processes audio, video, and pixel values, existing information media, namely newspapers, magazines, television, radios, and telephones, as well as other devices for conveying information to people, are included in the scope of multimedia . In general, multimedia refers to a representation: not only characters, but also graphic symbols, audio, and especially images, etc., are related to each other. However, in order to include the above-mentioned existing information media within the scope of multimedia, it is clear that a prerequisite is to represent such information in digital form.

However, when estimating the amount of information contained in each of the above-mentioned digital form information media, the amount of information per character requires 1-2 bytes, and audio requires more than 64 kilobits per second (telephone quality), and when considering When moving pictures, it needs more than 100 megabits per second (current TV reception quality). Therefore, it is unrealistic for the above-mentioned information medium to handle such a large amount of information as it is in digital form. For example, video telephony has been put into practical use through the Integrated Services Digital Network (ISDN), with a transmission rate of 64 kilobits to 1.5 megabits. Image.

Therefore, this requires information compression techniques, for example in the case of video telephony, using video compression techniques conforming to the H.261 and H.263 standards that are internationally standardized by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T). Image information as well as audio information can be stored on a general music compact disc (CD) according to an information compression technology conforming to the MPEG-1 standard.

Here, the Moving Picture Experts Group (MPEG) is an international standard for compressing moving picture signals, and MPEG-1 is a standard for compressing video signals down to 1.5 megabits (bits) per second, which means The information contained in a television signal is compressed down to approximately one percent. The target quality of the MPEG-1 standard is medium quality in order to achieve a transmission rate of around 1.5 megabits. Therefore, MPEG-2, which is standardized for the purpose of meeting higher quality image requirements, achieves television broadcasting quality at 2 megabits to 15 megabits bit rate to transmit moving image signals.

In the current environment, the working group (ISO/IEC/JTC1/SC29/WG11) that was previously responsible for the standardization of MPEG-1/MPEG-2 further standardized MPEG-4, and MPEG-4 achieved better performance than MPEG-1/MPEG-2. The achieved compression ratio allows encoding/decoding operations on a per-object basis, and enables new functions required in the multimedia era. First, during the MPEG-4 standardization process, the goal was to standardize low-bit-rate encoding, however, the current goal extends to more general encoding, including high-bit-rate encoding for interlaced images and the like. Also, the standardization of MPEG-4 AVC (Advanced Video Coding) and ITU H.264, which are the next-generation encoding methods jointly developed by ITU-T and the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC), is in progress, They have a higher compression ratio. Since August 2002, the Next Generation Coding Method has been published under the name of Committee Draft (CD).

In moving image coding, information amount compression is generally performed by eliminating redundancy in the spatial and temporal directions. Therefore, the purpose of inter-picture predictive encoding to reduce temporal redundancy is to estimate the motion, and refer to the forward and backward images to generate a predictive image block by block, and then compare the obtained predictive The difference between images is encoded. Here, "image" is a term representing an image on a screen, and means a frame when used for a progressive image, and a frame or a field when used for an interlaced image. Here, an interlaced image is an image in which a single frame consists of two fields, each having a different capture time. For encoding and decoding interlaced images, three ways of processing a single frame are possible: as a frame; as two fields; and as a frame/field structure, depending on the blocks in the frame.

A picture on which intra-picture prediction coding is performed without a reference picture is called an "I picture". A picture for which inter picture predictive coding is performed with reference to a single picture is called a "P picture". A picture for which inter picture predictive encoding is performed by referring to two pictures at the same time is called a "B picture". A B picture may refer to two pictures arbitrarily selected from the forward or backward pictures in display order. A reference image (ie, a reference image) can be designated for each block used as a basic encoding/decoding unit. Such reference pictures will be distinguished by referring to a reference picture to be described first in the encoded bitstream as a first reference picture, and a reference picture to be described later in the encoded bitstream as a second reference picture. Note that as a condition for encoding and decoding these types of pictures, pictures used for reference must have already been encoded and decoded.

Motion-compensated inter-picture predictive coding is used to code P-pictures or B-pictures. Coding using motion-compensated inter-picture prediction is a coding method that uses motion compensation in inter-picture predictive coding. Unlike a method that performs prediction based only on pixel values in a reference image, motion compensation is a method that can perform prediction by estimating the amount of motion (hereinafter referred to as "motion vector") for each part within an image, and further taking this amount of motion into consideration, A technique to improve prediction accuracy and reduce the amount of data. For example, it is possible to perform motion by estimating the motion vector of the image currently to be encoded, and then encoding the prediction residual between the predicted value obtained by shifting only the amount of each motion vector and the image currently to be encoded Compensation, thereby reducing the amount of data. In this technique, motion vectors are also recorded or transmitted in encoded form because motion vector information is required at the time of decoding.

Motion vectors are estimated on a per macroblock basis. More specifically, a macroblock should be fixed in advance in the current picture to be coded, so that the motion vector can be estimated by finding the most similar reference block of this fixed macroblock within a search area in the reference picture.

Figure 1 shows an example data structure for a bitstream. As shown in FIG. 1, the bit stream has a hierarchical structure as follows. A bit stream (stream) is composed of a plurality of groups of pictures (GOP). By using a GOP as a basic coding unit, it is possible to edit moving pictures as well as perform random access. Each GOP is composed of a plurality of pictures, and each picture is an I picture, a P picture or a B picture. Each image is further composed of a number of slices. Each slice, which is a stripe within each picture, is composed of a plurality of macroblocks. Also, each stream, GOP, picture, and slice includes a synchronization signal (sync) indicating the end point of each unit, and a header (header) which is a data block common to such units.

In the case of transmitting data not in a bit stream as a stream sequence but in packets as a unit of piecemeal data, the data of the header and the portion other than the header may be separately transmitted. In this case, the header and data sections should not be merged into the same bitstream, as shown in Figure 1. However, in the case of packets, the headers and corresponding data parts must not be transmitted sequentially, but are simply transmitted in different packets. Therefore, the same concept of the bitstream as described with reference to FIG. 1 applies even in cases where the header and data parts are not incorporated into the same bitstream.

In general, the human visual system is characterized by its sensitivity to low frequency components in images as compared to high frequency components. Furthermore, since the energy of the low-frequency components in the image signal is greater than the energy of the high-frequency components, image encoding is performed in order from the low-frequency components to the high-frequency components. As a result, the number of bits required for low-frequency component encoding is larger than the number of bits required for high-frequency component encoding.

In view of the above points, when quantizing transform coefficients of each frequency obtained by orthogonal transform, existing encoding methods use a larger quantization step size for high frequency components than for low frequency components. This technique makes it possible for conventional coding methods to achieve a substantial increase in the compression ratio with negligible degradation in the subjective quality of the image.

Since the quantization step size of the high-frequency component in contrast to the quantization step size of the low-frequency component depends on the image signal, a technique of changing the quantization step size of each frequency component from image to image is conventionally used. A quantization matrix (also called a "weighting matrix") is used to derive the quantization step size for each frequency component. Fig. 2 shows an example of a quantization matrix. In this figure, the upper left component is the DC component, while the rightward component is the horizontal high frequency component, and the downward component is the vertical high frequency component. The quantization matrix in FIG. 2 also indicates that the quantization step size becomes larger as the value becomes larger. In general, it is possible to use a different quantization matrix for each image. The value indicating the quantization step size of each frequency component is fixed-length coded. Note that in general the values of each component of the quantization matrix and each quantization step are approximately proportional to each other, but this relationship need not be enforced as long as the correspondence between them is clearly defined.

FIG. 3 shows a flowchart of inverse quantization as proposed in MPEG-2 and MPEG-4, performed by a conventional image encoding device or image decoding device.

As shown in the figure, the conventional image coding device or image decoding device obtains the weight matrix Wi, j and the quantization parameter QP (S11 and S12), calculates the quantization step size QStep, and obtains the quantization value (ie, the quantized frequency coefficient) fi, j( S14). Then, the image encoding device derives an inverse quantization value by calculating fi,j×QStep×Wi,j (S15-S17).

In the quantization process performed by the image encoding device, the frequency coefficient obtained as a result of the orthogonal transformation is multiplied by the reciprocal of the value resulting from the calculation of QStep×Wi,j.

However, the problem is that the conventional quantization and inverse quantization process imposes a lot of computational effort since a large number of divisions and multiplications need to be performed in the process.

Contents of the invention

An object of the present invention is to provide an image decoding method, an image encoding method, an image decoding device, an image encoding device and a program for reducing the workload required for quantization and inverse quantization calculations.

One aspect of the present invention provides an inverse quantization method for obtaining inverse quantized orthogonal transform coefficients by dequantizing quantized orthogonal transform coefficients. The method includes: obtaining a weighting matrix; obtaining quantization parameters; multiplying the value calculated from the component at the i-th row and j-column position in the weighting matrix by a quantization step size corresponding to the i-th row and j-column in the weighting matrix to calculate the grade scale value The position of the component of and the remainder obtained by dividing the quantization parameter by an integer; multiplying the quantized orthogonal transform coefficient by the level scale value; and shifting the product obtained by the multiplication according to the quantization The number of bits of the parameter to obtain the inverse quantized orthogonal transform coefficients.

Another aspect of the present invention provides an image decoding method for performing inverse quantization and inverse orthogonal transformation on quantized orthogonal transformation coefficients to obtain a block image. The method includes: obtaining a weighting matrix; obtaining quantization parameters; The grade scale value is calculated by multiplying the value calculated from the component at the i-th row j-th column position in the weighting matrix by the quantization step size corresponding to the i-th row j-th in the weighting matrix The position of the component of the column and the remainder obtained by dividing the quantization parameter by an integer; multiplying the quantized orthogonal transform coefficient by the level scale value; shifting the product obtained by the multiplication according to the quantization The number of bits of the parameter to obtain the inverse quantized orthogonal transform coefficient; perform inverse orthogonal transform on the obtained inverse quantized orthogonal transform coefficient through addition/subtraction operation and bit shift operation to obtain a block image.

Another aspect of the present invention provides an image decoding device for decoding encoded image data on a block basis to obtain a block image, the device comprising: an obtaining unit operable to obtain a weighting matrix and a quantization parameter , and calculate the grade scale value by multiplying the value calculated from the component at the i-th row and j-th column position in the weighting matrix with the quantization step size corresponding to the i-th row in the weighting matrix The position of the component of the j-th column and the remainder obtained by dividing the quantization parameter by an integer; the multiplication unit is operable to multiply the quantized orthogonal transform coefficient by the level scale value; the shifter is the the product obtained by the multiplication is shifted by the number of bits according to the quantization parameter to obtain an inverse-quantized orthogonal transform coefficient; The inverse quantized orthogonal transform coefficients are subjected to inverse orthogonal transform to obtain a block image.

Another aspect of the present invention provides a processor used in a decoding device for decoding moving images, the processor includes: an integrated circuit; wherein the processor i) uses the integrated circuit to obtain a weighting matrix and Quantization parameters; ii) calculating the grade scale value by multiplying the value calculated from the component at the i-th row and j-th column position in the weighting matrix by a quantization step size corresponding to The position of the component of the i-th row and the j-th column and the remainder obtained by dividing the quantization parameter by an integer; iii) multiplying the quantized orthogonal transform coefficient by the level scale value; iv) multiplying the multiplied shifting the obtained product by the number of bits according to the quantization parameter to obtain inverse quantized orthogonal transform coefficients; and v) performing inverse orthogonal transform on the shifted result.

In order to achieve the above object, the image encoding method of the present invention is an image decoding method for decoding an encoded image by inverse quantization and inverse orthogonal transformation performed block by block. The method includes the following steps as an inverse quantization process: multiplying a quantization matrix indicating the scaling ratio of the quantization step for each frequency component and a multiplier being the orthogonal transform or quantization step and multiplying the product resulting from said multiplication by the quantized value.

The multiplier may be related to a normalization factor used in the inverse orthogonal transform process.

According to the above structure, the workload required for calculation can be reduced because multiplication required for frequency coefficients need not be performed in inverse orthogonal transform processing. That is, by precomputing the multiplications required for deriving the quantization step size, it is possible to reduce the multiplications required for inverse orthogonal transform so as not to increase the workload required for quantization calculations.

The multiplication between the quantization matrix and the multiplier may be performed for each predetermined unit of encoded data that includes the encoded block, and the multiplication between the product and the quantized value may be performed block by block. in total.

The product resulting from the multiplication between the quantization matrix and the multiplier may be stored in memory, and the multiplication between the product and the quantization value may involve accessing memory.

The encoded data of a predetermined unit may be data corresponding to an image.

According to the above structure, it is possible to reduce the number of operations to further reduce the calculation workload by dividing the processing into two parts: multiplication on a per-image basis; and multiplication on a per-block basis.

The image encoding method, image decoding apparatus, image encoding apparatus, program, and semiconductor device according to the present invention have the same structure, and obtain the same effects as above.

Description of drawings

These and other objects, advantages and features of the invention will become apparent from the following description of the invention considered in conjunction with the accompanying drawings illustrating specific embodiments of the invention. In the attached picture:

Figure 1 shows an example data structure of a bitstream;

Figure 2 shows an example quantization matrix;

FIG. 3 shows a flowchart of inverse quantization as proposed in MPEG-2 and MPEG-4, performed by a conventional image coding device;

FIG. 4 shows a structural block diagram of an image coding device according to a first embodiment of the present invention;

Fig. 5 shows the block structure and the orthogonal transformation to be performed on the block in the case of performing 16×16 intra image predictive coding on the macroblock luma block;

6 shows a block structure in the case of performing 4×4 intra-frame image predictive coding or 4×4 inter-frame image predictive coding on a luma block of a macroblock and an orthogonal transformation to be performed on the block;

Figure 7 shows the structure of a macroblock chrominance block and the orthogonal transformation to be performed on the block;

Figure 8 shows the equations used in the Hadamard transform;

Figure 9A shows the equations used in the integer precision DCT;

Figure 9B shows the equations used in the integer precision inverse DCT;

Figures 10A-10D show examples of encoding order in quantization matrices, respectively;

Figure 11A shows an array of weighted components in a quantization matrix, where each weighted component is defined for an orthogonal transform;

11B and 11C respectively show how each data obtained by encoding each component in the quantization matrix is placed in the header;

FIG. 12 shows quantized input-output characteristic curves;

Fig. 13 shows the quantization step size characteristic curve as a function of the quantization parameter;

Fig. 14 shows the signal-to-noise ratio characteristic curve as a function of the quantization parameter;

15A to 15C are for explaining normalization and inverse quantization processing;

FIG. 16A shows a block diagram of a first example structure of a quantization unit;

Fig. 16B shows a block diagram of a first example structure of an inverse quantization unit;

FIG. 17A shows a block diagram of a second example structure of a quantization unit suitable for a weighting matrix;

17B shows a block diagram of a second example structure of an inverse quantization unit using a weighting matrix;

FIG. 18A shows a block diagram of a third example structure of a quantization unit using a weighting matrix;

18B shows a block diagram of a third example structure of an inverse quantization unit using a weighting matrix;

FIG. 19 shows a flow chart of inverse quantization processing in the case where each quantization step calculation including frequency transform multiplication is performed on each (i, j) component based on the quantization parameter QP;

FIG. 20 shows a quantization flowchart in the case where each quantization step calculation including frequency transform multiplication is performed based on the quantization parameter QP in advance;

FIG. 21 shows a quantization flowchart in a case where each quantization step calculation including frequency transform multiplication is performed based on the quantization parameter QP as needed;

Fig. 22 shows a structural block diagram of an image decoding device;

23A-23C are illustrations of storage media storing programs;

Fig. 24 shows a block diagram of the general configuration of the content supply system;

FIG. 25 shows a specific example of a cellular phone using image encoding and decoding methods;

Figure 26 shows a block diagram of a cellular telephone;

Figure 27 shows an example of a digital broadcasting system;

28-31 show examples of deriving quantization matrices based on 8×8 weighting matrices according to the second embodiment;

Figures 32-35 show examples of deriving quantization matrices based on 4x4 weighting matrices;

Fig. 36 shows a block diagram of an inverse quantization unit according to a third embodiment;

Figure 37 shows an example of a weighting matrix;

Figure 38 shows the inverse quantization process;

Figures 39 and 40 illustrate the inverse quantization process;

Figures 41 and 42 show examples of tables, respectively; and

Fig. 43 shows inverse quantization performed on a 4x4 chroma DC block.

Detailed ways

(first embodiment)

FIG. 4 shows a structural block diagram of an image encoding device according to a first embodiment of the present invention.

The image encoding apparatus 1 is an apparatus that outputs an encoded image signal Str obtained by performing compression encoding on an input image signal Vin, and then converting the encoded image signal into a bit stream such as a variable-length code. Such an image coding apparatus 1 includes a motion estimation unit ME, a motion compensation unit MC, a subtractor Sub, an orthogonal transformation unit T, a quantization unit Q, an inverse quantization unit IQ, an inverse orthogonal transformation unit IT, an adder Add, an image memory PicMem , a switch SW and a variable length coding unit VLC.

The image signal Vin is input to the subtractor Sub and the motion estimation unit ME. The subtractor Sub calculates a residual image between each image in the input image signal Vin and each predicted image, and outputs the calculated residual image to the orthogonal transformation unit T.

The orthogonal transform unit T performs orthogonal transform on the residual image to transform it into orthogonal transform coefficients or frequency coefficients, and outputs them to the quantization unit Q.

The quantization unit Q quantizes the frequency coefficient of each block input from the orthogonal transform unit T using a quantization step derived with reference to the quantization matrix WM input from the outside, and outputs the obtained quantized value Qcoef to Variable length coding unit VLC.

The inverse quantization unit IQ performs inverse quantization on the quantization values Qcoef using a quantization step derived with reference to the quantization matrix WM to transform them into frequency coefficients, and outputs them to the inverse orthogonal transform unit IT. The inverse quantization unit IQ according to this embodiment performs inverse quantization in two steps: the first step is to multiply a quantization matrix indicating each quantization step scaling ratio of each frequency component by a The coefficient or the multiplier of the quantization step size, and store the multiplication result in the memory; the second step is to multiply the result stored in the memory by each quantization value. The first step operates on a per-image basis, while the second step operates on a per-block basis. The multipliers used for the frequency transform include inverse orthogonal transform normalization factors. In this case, the multiplication result stored in the memory is a value obtained by multiplying each quantization step size and an inverse orthogonal transform normalization factor.

The inverse orthogonal transform unit IT performs inverse frequency transform on the frequency coefficients to transform them into a residual image, and outputs the residual image to the adder Add. The adder Add adds each residual image and each predicted image output from the motion compensation unit MC to obtain a decoded image. In a case where it is indicated that such a decoded image should be stored, the switch SW is turned on, thereby storing the decoded image in the image memory PicMem.

The motion estimation unit ME to which the image signal Vin is inputted on a per-macroblock basis detects an image area most similar to the input image signal Vin in the decoded image stored in the image memory PicMem, and determines a motion vector indicating the position of such an image area MV. Motion vector estimation is performed for each block obtained by further dividing the macroblock.

The motion compensation unit MC uses the motion vectors detected in the above process to extract the most suitable image area from the decoded image stored in the image memory PicMem as a predictive image.

The variable-length coding unit VLC performs variable-length coding on each of the quantization matrix WM, quantization value Qcoef, and motion vector MV to obtain a bit stream Str.

5 and 6 respectively show the orthogonal transformation performed by the orthogonal transformation unit T according to MPEG-4 AVC. For a luminance macroblock composed of 16x16 pixels, orthogonal transformation and block division by means of orthogonal transformation are performed differently for the case of 16x16 macroblock-based intra image predictive encoding and the case of other types of encoding.

FIG. 5 shows the structure and orthogonal transformation of a macroblock luma block in the case where 16×16 intra image predictive coding is to be performed on the luma block. In this case, the orthogonal transform unit T performs orthogonal transform as in (1) to (4) below. (1) Divide the luminance block of 16×16 pixels into 16 4×4 pixel blocks. (2) Orthogonal transform based on integer precision 4x4 discrete cosine transform (DCT) is performed on each 4x4 block resulting from division. Here, the integer precision DCT does not maintain the same properties as the DCT because the values are rounded, but it can still be used as a DCT-like transform. (3) Generate a 4×4 DC block composed of a direct current (DC) component in each orthogonal transform block. (4) Hadamard transform is performed on the 4x4 DC block. The Hadamard transform, properly called the "Discrete Hadamard Transform (DHT)", is a simple orthogonal transform that performs only addition and subtraction.

Fig. 6 shows the structure of the luminance block and the luminance block in the case of coding other than 16×16 intra-frame predictive coding, such as 4×4 intra-frame predictive coding and 4×4 inter-frame predictive coding Orthogonal transformation to perform. In this case, the orthogonal transform unit T performs orthogonal transform as in (1) and (2) below. (1) Divide the luminance block of 16×16 pixels into 16 4×4 pixel blocks. (2) An integer-precision DCT-based orthogonal transform is performed on each 4×4 block resulting from division.

Fig. 7 shows the structure of a macroblock chroma block and the orthogonal transform to be performed on the chroma block. In this case, the orthogonal transform unit T performs orthogonal transform as in (1) to (4) below. (1) Divide the chrominance macroblock composed of 8*8 pixels into four 4*4 pixel blocks. (2) Perform an orthogonal transform based on integer precision 4×4 DCT for each 4×4 block generated by the partition. (3) Generate a 2×2 pixel DC block composed of DC components in each orthogonal transform block. (4) Perform Hadamard transform on each 2×2 DC block.

Thus, the orthogonal transform unit T will use the Hadamard transform on the DC block, which is one of the simplest orthogonal transforms, which can be realized only by performing addition and subtraction.

Fig. 8 shows the equations and waveforms used in the Hadamard transform, each waveform representing a corresponding equation. In the figure, "h0"-"h3" denote four input signals, and "H0"-"H3" denote components on which Hadamard transform is performed, respectively. "H0" is the DC component on which Hadamard transformation is performed, and "H3" is the highest frequency component on which Hadamard transformation is performed. The inverse of the Hadamard transform is the Hadamard transform. That is, "h0"-"h3" can be obtained by performing Hadamard transformation on "H0"-"H3" again.

Note that the Hadamard transform performed on a 2×2 DC luma block can be obtained using the following equation. That is, the following equations are used once for each row and column in a 2 × 2 DC block.

$\mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="h"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">h</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="0"\; label="h"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">h</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; h</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\sqrt{\mathrm{<span\; class="notranslate">\; 2</span>}}$

$\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="h"\; filenames="C20058000303500441.png,C20058000303500531.png"\; state="\{\{state\}\}">h</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; h</span>}\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; h</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\sqrt{\mathrm{<span\; class="notranslate">\; 2</span>}}$

FIG. 9A shows equations and waveform diagrams used by the orthogonal transform unit T for integer precision DCT, each waveform diagram representing a corresponding equation. "d0"-"d3" denote four input signals, and "D0"-"D3" denote components on which integer precision DCT is performed. In integer precision 4x4 DCT, the 4-input DCT shown in Figure 9A is applied once for each row and column in a 4x4 pixel block.

"D0" is the DC component, and "D3" is the highest frequency component. Between the frequency components H1 and D1, the difference between the waveform diagram of the integer-precision DCT and that of the Hadamard transform shown in FIG. 8A is apparent. That is, the frequency component D1 (current component of the lowest frequency) expresses more smoothly (ie, gradually changes from the maximum value to the minimum value) than the frequency component H1 .

倍或8/5倍。FIG. 9B shows the equations used by the orthogonal transform unit T for integer-precision inverse DCT. In the figure, actual operations are required to obtain D1' and D3' using frequency components D1 and D3. In this embodiment, the quantization step size in the inverse quantization process is multiplied by some value in advance (the first step size taken by the inverse quantization unit IQ as described above), in order to avoid the actual operation in the integer precision inverse DCT . Consequently, the number of multiplications performed for the integer-precision inverse DCT is reduced, and the amount of work required for the calculation is also reduced. As a result, in the inverse quantization process, the quantization step size of odd-order frequency components is equal to the quantization step size of even-number frequency components.

times or 8/5 times.

10A-10D each show an example of the coding order in the quantization matrix. This order is used to encode or decode the quantization matrix, and the quantization matrix is rearranged in the order of the components actually to be operated on during quantization and inverse quantization. In the orthogonal transformation performed in image coding, 4×4 pixels and 8×8 pixels are the two most frequently used units. 10A and 10C show examples using 4×4 pixel units, and FIGS. 10B and 10D show examples using 8×8 pixel units. Compression efficiency is high in encoding from low-frequency components toward high-frequency components, as shown in FIGS. 10A and 10B , but in some cases, encoding in horizontal order may be used, as shown in FIGS. 10C and 10D .

11A-11C show quantization matrices, and data structures of quantization matrices (ie, weighting matrices) that are coded when streaming is formed. In the figure, "header" means the head of "GOP" or "picture" shown in FIG. 1, or equivalent information. Fig. 11A shows an array of frequency components in a quantization matrix. "Wi,j" denotes a component in row "i" and column "j" of the quantization matrix. 11B and 11C respectively show examples of how the encoded data of each component in the quantization matrix is placed in the header. "WeightingMatrix" indicates a bit stream obtained by encoding a quantization matrix. FIG. 11B shows a stream obtained by encoding quantization matrices in the order shown in FIG. 10B , and FIG. 11C shows a stream obtained by encoding quantization matrices in the order shown in FIG. 10D . Note that "Wi,j" in the streams shown in FIGS. 11B and 11C indicates the encoding variable-length code at the position indicated by "Wi,j" in the quantization matrix.

FIG. 12 shows input-output characteristics of quantization (or inverse quantization) performed by quantization unit Q and inverse quantization unit IQ. Quantization means that the result of dividing the orthogonally transformed frequency components (values to be quantized in FIG. 12 ) by the quantization step is rounded so that the frequency components become integers. The rounded integer is called the quantized value. Conversely, converting the value to be quantized back into frequency components is called "inverse quantization". By changing the size of the quantization step, it is possible to increase or decrease the number of bits produced by encoding. Thus, by changing the size of the quantization step, it is possible to maintain the same compression ratio (ie, the amount of codes per unit time).

Fig. 13 shows a characteristic curve of quantization step size versus quantization parameter. The quantization parameter is used to derive the quantization step size and is adjusted so that the code size corresponds to the bit rate. The quantization parameters to be coded are used by the inverse quantization unit IQ and quantization unit Q to derive the quantization step size, since the quantization step size is not coded directly. In the example shown in FIG. 13, when the quantization parameter QP is increased by 6, the quantization step size is doubled. As a result, in the case where the quantization parameter QP is changed, the quantization parameter QP becomes proportional to the signal-to-noise (SN) ratio, and the amount of change in the SN ratio is maintained at the same level, as shown in FIG. 14 , regardless of the value of the quantization parameter QP .

Note that when the quantization parameter indicates a value ranging from 0 to 51, the maximum value of the quantization step size indicates a value equal to 256 times the minimum value.

这意谓分量乘以8/5。基于此，对于在逆量化处理中预先进行的归一化，应该预先乘以量化步长，如以下的(a)至(c)。15A to 15C are for explaining normalization and inverse quantization. The following describes a simplified inverse quantization process that takes The method of multiplication for normalization in the orthogonal transform shown in Fig. 9B. As shown in FIG. 9B, the components D1 and D3 are located at the Nth position (N is an odd number). multiply the component at the Nth position both horizontally and vertically twice

This means that the components are multiplied by 8/5. Based on this, for the normalization performed in advance in the inverse quantization process, the quantization step size should be multiplied in advance, as in (a) to (c) below.

(a) In the case where a component is located at the Mth (M is an even number) position in both the horizontal and vertical directions, normalization multiplication is not performed.

(b) In the case where the component is at the Nth (N is an odd number) position in the horizontal or vertical direction, multiply the component by

(c) In the case where the component is located at the Nth (N is an odd number) position in both the horizontal and vertical directions, the component is multiplied by 8/5.

now, given $\mathrm{\&beta;}=\mathrm{\&alpha;}\&times;\sqrt{8}/\sqrt{5},$ γ=α×8/5, (a) to (c) become the following (A) to (C).

(A) In the case where the component is located at the Mth (M is an even number) position in both the horizontal and vertical directions, the component is multiplied by α.

(B) In the case where the component is located at the Nth (N is an odd number) position in the horizontal or vertical direction, the component is multiplied by β.

(C) In the case where the component is located at the Nth (N is an odd number) position both in the horizontal and vertical directions, the component is multiplied by γ.

With this simple rule, it is possible to achieve normalization together in the inverse quantization process. In inverse DCT and quantization processing, dedicated normalization multiplication becomes unnecessary.

When the quantization parameter QP is increased by 6, the quantization step size is doubled. Therefore, the relationship between the quantization parameter QP obtained by calculation including normalization multiplication and the quantization step size is represented by the following equation.

Quantization step size = (quantization step size of QP%6) × (2 ^(QP/6) )

Based on this, as shown in FIG. 15B, by shifting the "quantization step size of QP%6" to the left by "QP/6" bits, the quantization step size involved in normalization can be easily obtained as the same as any quantization parameter QP The corresponding quantization step size (ie the quantization step size involving normalized multiplication). This can be achieved by reserving a total of 18 quantization steps only for "quantization steps of QP%6" corresponding to α, β, and γ, respectively, as shown in FIG. 15C. In this embodiment, the ratio of the quantization parameter size of the luma signal to the quantization parameter size of the chrominance signal will vary on a slice-by-slice basis. Since the degradation of color (especially red) is visually more obvious than the degradation of brightness, it is preferable that the quantization parameter QP of the chrominance signal is smaller than that of the brightness signal.

Obviously, since the quantization step size can be expressed by shifting the quantization step size by QP/6 bits, quantization and inverse quantization can be expressed by a combination of multiplication and shift operations. 16A and 16B are block diagrams showing a first example structure of the quantization unit Q and the inverse quantization unit IQ. In the first example, only multiplication and shift operations are used without using a weighting matrix. The quantization unit Q1 includes a multiplication unit Q11 and a right shifter Q12. The multiplication unit Q11 multiplies the orthogonal transform coefficient by Q1. "Q1" is a multiple of the reciprocal of the quantization step size (quantization step size of QP%6). The quantization step size is usually used for the division performed in quantization, however, the inverse of the quantization step size is precomputed for multiplication because division operations are more complex than multiplication operations. The inverse quantization unit IQ1 includes a multiplication unit IQ11 and a left shifter IQ12.

The quantization unit Q1 operates as follows. The right shifter Q12 shifts the result of the multiplication performed by the multiplication unit Q11 to the right by S1 bits. That is, the right shifter Q12 divides the multiplication result performed by the multiplication unit Q11 by 2 ^S1 . The S1 value varies proportionally to QP/6. The inverse quantization unit IQ1 operates as follows. The multiplication unit IQ11 multiplies the orthogonal transform coefficient by Q2. The "Q2" value changes in proportion to the quantization step size of QP%6. The left shifter IQ12 shifts the result of the multiplication performed by the multiplication unit IQ11 to the left by S1 bits. That is, the left shifter IQ12 multiplies the multiplication result performed by the multiplication unit IQ11 by 2 ^S2 . The "S2" value varies proportionally to QP/6.

Here, "S1" and "S2" are values fixed for all frequency coefficients, each of which varies with the quantization parameter QP. "Q1" and "Q2" have values depending on the quantization parameter QP and the frequency coefficient position. In this case, it is required to satisfy the relationship represented by Q1×(2 ^−S1 )×Q2×(2 ^S2 )=1. In this case, Q1×Q2 results in a binary exponent, and S2-S1 is obtained as a fixed value.

17A and 17B are block diagrams showing a second example structure of a quantization unit Q and an inverse quantization unit IQ in the case of using a weighting matrix. The quantization unit Q2 includes: a multiplication unit Q21 that multiplies the frequency coefficient by Q1; a multiplication unit Q22 that multiplies the result of the multiplication performed by the multiplication unit Q21 by Qa; a right shifter Q23 that right shifts a bit shifter Q23 that shifts the result of the multiplication performed by the multiplication unit Q22 to the right by S1 bits; and a right shifter Q24 that shifts the result of the shift performed by the right shifter Q23 to the right by Sa bits . Inverse quantization unit IQ2 comprises: multiplication unit IQ21, this multiplication unit IQ21 makes quantization frequency coefficient multiplied by Q2; A unit IQ23 multiplies the result of the shift performed by the left shifter Q22 by Qb; and a right shifter IQ24 that shifts the result of the shift performed by the left shifter Q22 to the right by Sb bits.

Here, "Qa" and "Sa" correspond to weighting components Wi,j in a given weighting matrix, and the following relationship can be established: Qa×2 ^−Sa ×Qb×2 ^−Sb =1.

18A and 18B are block diagrams showing a third example structure of a quantization unit Q and an inverse quantization unit IQ, in which the structures shown in FIGS. 17A and 17B are simplified so as to perform multiplication and shift operations together. The quantization unit Q3 in the figure includes: a multiplication unit Q31 that multiplies the orthogonal transform coefficient by Q1a; and a right shifter Q32 that shifts the result of the multiplication performed by the multiplication unit Q31 to the right Bit "S1+Sa" bit. The inverse quantization matrix IQ3 includes: a multiplication unit IQ31 that multiplies the quantized orthogonal transform coefficient by Q2a; and a right shifter IQ32 that shifts the result of the multiplication performed by the multiplication unit IQ31 to the right Bit "Sb-S2" bit. The multiplying unit IQ31 performs two multiplications by Q1 and Qa respectively shown in FIG. 17A by performing one multiplication by Q1a. That is, execution is done using the following equation: Q1b=Q1*Qb. The right shifter Q32 performs two right shifts to the right by S1 bits and Sa bits, respectively, by performing one shift. The multiplication unit IQ31 performs two multiplications with Q2 and Qb, respectively, by performing one multiplication with Q2b. That is, execution is done using the following equation: Q2b=Q2*Qb. The right shifter IQ32 performs a left shift of S2 bits and a right shift of Sb bits by performing one shift.

FIG. 19 shows a flowchart of inverse quantization in the case where quantization step size calculation including frequency transform multiplication is performed for each (i, j) component based on the quantization parameter QP. For example, inverse quantization is performed on a block basis by the inverse quantization unit IQ3 (or IQ2).

First, the inverse quantization unit IQ3 obtains a weight matrix {Wi, j} and a quantization parameter QP (S31, S32). Then, the inverse quantization unit IQ3 further derives {Q2i, j} and S2 from the quantization parameter QP (S33), as a quantization step size obtained by calculation including frequency transform multiplication, and obtains a quantization value (quantized frequency coefficient) {fi , j} (S34). Here, {Q2i, j} is obtained as the quantization step size of QP%6. Use QP/6 to obtain S2b.

Next, the inverse quantization unit IQ3 performs quantization on each frequency coefficient in the block in loop 1 (S35-S40). That is, the inverse quantization unit IQ3 derives {Qbi,j} and Sb according to the weighting matrix {Wi,j} (S36), and obtains the level scale (level scale) LSi,j by multiplying Qbi,j and Q2i,j (S37). The inverse quantization unit IQ3 further obtains the number of bits S2b to be shifted by subtracting S2 from Sb (S38) so as to be simultaneously shifted, and obtains by multiplying the quantization value fi,j and the level scale (level scale) LSi,j The inverse quantization value is calculated, and then the multiplication result is left shifted by S2b bits (S40).

Thus, inverse quantization can be performed by a simple method of performing inverse quantization on a quantization step size obtained by calculation including frequency transform multiplication while calculating the quantization step size using the quantization parameter QP.

FIG. 20 shows a flow chart of quantization processing in the case where the quantization step size obtained by calculation including frequency transform multiplication is calculated in advance using the quantization parameter QP. For example, inverse quantization is performed by the inverse quantization unit IQ3 (or IQ2). The difference between FIG. 20 and FIG. 19 is that all quantization step sizes LSi,j, each obtained by calculation including frequency transform multiplication, are stored in the memory as a table on a per-image basis (S43c in Loop 1 ), and read out LSi,j from the table (S49a in cycle 2). Other operations are almost the same as in Fig. 19, so that their descriptions are omitted. According to the inverse quantization of this embodiment, in cycle 1, all quantization step sizes LSi,j each obtained by calculation including frequency transform multiplication are stored in memory as a table, which is suitable for high-speed Calculating the inverse quantization value.

FIG. 21 shows a flow chart of quantization processing in the case where a quantization step size obtained by calculation including frequency transform multiplication is calculated based on the quantization parameter QP as necessary. For example, inverse quantization is performed by the inverse quantization unit IQ3 (or IQ2). The difference between FIG. 19 and FIG. 21 is that a free area (free area) is allocated for the table in the memory at the time of initialization, and whether the quantization step size LSi,j obtained by calculation including frequency transform multiplication is stored in the table (S56). In judging that the quantization step LSi, j and the number of bits S2b to be shifted are not stored, calculate LSi, j and S2b (S57a-S57c), and the obtained LSi, j and add to the table (S57d ), and when it is judged that the quantization step size LSi,j is stored, LSi,j and S2b are read from the table (S57e). According to the inverse quantization of this embodiment, compared with Fig. 20, it is possible to reduce the amount of calculation to the minimum amount of calculation required by subsequent blocks that jointly use the quantization parameter QP, although for the first block, the amount of calculation is more or less Varies with serving size.

Fig. 22 shows a block diagram of the structure of an image decoding device according to this embodiment. In the drawing, the same reference numerals are assigned to units that operate in the same manner as those included in the image encoding device shown in the block diagram of FIG. 4 , and their descriptions are omitted. The inverse quantization unit IQ and the inverse orthogonal transform unit IT are the same as those shown in Fig. 4, where the operations have been described.

Furthermore, by recording a program for realizing the moving picture encoding/decoding method described in each of the above embodiments on a storage medium such as a floppy disk or the like, the programs shown in the above embodiments can be easily executed in a stand-alone computer system. deal with.

23A, 23B, and 23C are diagrams for realizing the moving image encoding/decoding method described in the above embodiments using a program stored in a storage medium such as a floppy disk in a computer system.

Fig. 23B shows the complete appearance of the floppy disk, its cross-sectional structure, and the floppy disk itself, while Fig. 23A shows the physical format of the floppy disk as the main body of the storage medium. A floppy disk FD is housed in a case F, a plurality of tracks Tr are concentrically formed on the surface of the disk from outside to inside, and each track is divided into 16 sectors Se in the angular direction. Thus, the moving picture encoding method and the moving picture decoding method are recorded as programs in the area allocated thereto on the floppy disk FD.

FIG. 23C shows a configuration for recording the program on the floppy disk FD and reproducing the program on the floppy disk FD. When the program is recorded on the floppy disk FD, the computer system Cs writes the moving picture encoding and decoding method as a program through the floppy disk drive FDD. When a moving picture encoding and decoding method is constructed in a computer system using a program on a floppy disk, the program is read from the floppy disk through the floppy disk drive FDD and then transferred to the computer system.

The above description has been made on the assumption that the storage medium is a floppy disk, but the same processing can also be performed using an optical disk. In addition, the storage medium is not limited to a floppy disk and an optical disk, but any other medium capable of recording a program, such as an integrated circuit (IC) card and a read only memory (ROM) cassette may be used.

The application of the moving image encoding and decoding method explained in the above-mentioned embodiments and a system using the moving image encoding and decoding method are described below.

FIG. 24 is a block diagram showing an overall configuration of a content provision system ex100 for realizing content delivery services. An area for providing communication services is divided into cells having a desired size, and cell sites ex107~ex110 as fixed radio stations are located in the respective cells.

This content supply system ex100 is connected to various devices such as a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a cellular phone ex114 and A cellular phone ex115 with a camera.

However, the content supply system ex100 is not limited to the configuration shown in FIG. 24 but can be connected to any combination of devices. Also, each device can be directly connected to the telephone network ex104 without going through the cell sites ex107~ex110.

The camera ex113 is a device capable of taking video, such as a digital video camera. The cellular phone ex114 may be a cellular phone of any of the following systems: Personal Digital Communications (PDC) system, Code Division Multiple Access (CDMA) system, Wideband Code Division Multiple Access (W-CDMA) system or Global System for Mobile Communications (GMS), Personal Handyphone System (PHS), etc.

The streaming server ex103 is connected to the camera ex113 through the telephone network ex104 and the cell site ex109, and uses the camera ex113 to realize live broadcasting and distribution according to coded data sent from the user. Any one of the camera ex113, the server that transmits the data, and the like can encode the data. The moving image data shot by the camera ex116 can be sent to the streaming server ex103 through the computer ex111. In this case, the camera ex116 or the computer ex111 can encode moving image data. A large scale integration (LSI) ex117 and a camera ex116 included in the computer ex111 perform encoding processing. Software for encoding and decoding moving pictures can be incorporated into any type of storage medium such as compact disc read-only (CD-ROM), floppy disk, and hard disk that can be read by a computer ex111 or the like. Also, a cellular phone having a camera ex115 can transmit moving image data. This moving image data is data encoded by the LSI included in the cellular phone ex115.

The content supply system ex100 encodes the content (such as music live broadcast video) captured by the user with the camera ex113, camera ex116, etc., and transmits them to the streaming server ex103 in the same manner as in the above-mentioned embodiment, At the same time, once the client requests, the streaming server ex103 transmits the content data to the client in a streaming manner. The client includes a computer ex111, a PDA ex112, a camera ex113, a cellular phone ex114, etc. capable of decoding the above coded data. Thus, in the content supply system ex100, the client can receive and reproduce coded data, and further can receive, decode and reproduce data in real time, so as to realize personal broadcasting.

When each device in the system performs encoding or decoding, the moving image encoding device and moving image decoding device shown in the above-described embodiments can be used.

A cellular phone will be described as an example of such a device.

FIG. 25 shows a cellular phone ex115 using the moving image encoding and decoding method explained in the above-mentioned embodiments. The cellular phone ex115 has: an antenna ex201 for communicating with a cell site ex110 by radio waves; a camera unit ex203 capable of taking moving and still images such as a charge-coupled device (CCD) camera; a display unit ex202 such as a liquid crystal display for displaying Data captured by the camera unit ex203 or received by the antenna ex201, such as decoded images, etc.; a main body unit including a set of operation keys ex204; a voice output unit ex208, such as a speaker, for outputting voice; a voice input unit ex205, such as a microphone , for inputting voice; the storage medium ex207, for recording encoded or decoded data, such as moving or still image data taken by a camera, received e-mail data, and moving or still image data; and the slot unit ex206, for To attach the storage medium ex207 to the cellular phone ex115. The storage medium ex207 is a fast storage element housed in a plastic case such as a secure digital memory card (SD card), which is an electrically erasable and rewritable nonvolatile memory Erase Programmable Read-Only Memory (EEPROM).

Next, the cellular phone ex115 will be explained with reference to FIG. 26 . In the cellular phone ex115, the main control unit ex311 designed to comprehensively control each unit of the main body including the display unit ex202 and the operation keys ex204 is interconnected to the power supply circuit unit ex310, the operation input control unit ex304, the image Encoding unit ex312, camera interface unit ex303, liquid crystal display (LCD) control unit ex302, image decoding unit ex309, multiplexing/demultiplexing unit ex308, read/write unit ex307, modem circuit unit ex306, and voice processing unit ex305.

When a call-end key or a power key is turned on by user's operation, the power circuit unit ex310 supplies power from the battery pack to each unit to activate the digital cellular phone with camera ex115 to a ready state.

In the cellular phone ex115, the voice processing unit ex305, under the control of the main control unit ex311 including a central processing unit (CPU), a read-only memory (ROM) and a random access memory (RAM), transfers the voice input unit ex205 in conversation mode to The received voice signal is converted into digital voice data, the modem circuit unit ex306 performs spread spectrum processing on the digital voice data, and the communication circuit unit ex301 performs digital-to-analog conversion and frequency conversion on the data to transmit the data through the antenna ex201. Also, in the cellular phone ex115, the communication circuit unit ex301 amplifies data received through the antenna ex201 in conversation mode, and performs frequency conversion and analog-to-digital conversion on the data, the modem circuit unit ex306 performs inverse spread spectrum processing on the data, and The voice processing unit ex305 converts the data into analog voice data to output the analog voice data through the voice output unit ex208.

Furthermore, when sending an e-mail in the data communication mode, e-mail text data input through the operation key ex204 of the operating body is sent to the main control unit ex311 through the operation input control unit ex304. In the main control unit ex311, after the modem circuit unit ex306 performs spread spectrum processing on the text data and the communication circuit unit ex301 performs digital-to-analog conversion and frequency conversion on the text data, the data is transmitted to the cell site ex110 through the antenna ex201.

When image data is transmitted in the data communication mode, image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 through the camera interface unit ex303. When the image data is not transmitted, it is also possible to directly display the image data captured by the camera unit ex203 on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

The image encoding unit ex312 including the moving image encoding device according to the present invention compresses and encodes the image data supplied from the camera unit ex203 using the encoding method adopted by the moving image encoding device shown in the above-mentioned embodiments, so that It is converted into coded image data and sent to the multiplexing/demultiplexing unit ex308. At this time, during shooting with the camera unit ex203, the cellular phone ex115 sends the voice received through the voice input unit ex205 to the multiplexing/demultiplexing unit ex308 through the voice processing unit ex305 as digital voice data.

The multiplexing/demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the voice data supplied from the voice processing unit ex305 using a predetermined method, and then the modem circuit unit ex306 multiplexes the data obtained as a result of the multiplexing The multiplexed data performs spread spectrum processing, and finally the communication circuit unit ex301 performs digital-to-analog conversion and frequency conversion on the data for transmission through the antenna ex201.

Regarding reception of moving image file data linked to a web page or the like in the data communication mode, the modem circuit unit ex306 performs inverse spread spectrum processing on data received from the cell site ex110 through the antenna ex201, and transmits the result obtained as a result of the inverse spread spectrum processing. The obtained multiplexing data.

In order to decode the multiplexed data received through the antenna ex201, the multiplexing/demultiplexing unit ex308 demultiplexes the multiplexed data into encoded image data streams and encoded voice data streams, and encodes the encoded data streams through the synchronous bus ex313 respectively. The image data is supplied to the image decoding unit ex309, and the audio data is supplied to the audio processing unit ex305.

Next, the image decoding unit ex309 including the moving image decoding device according to the present invention decodes the encoded image data stream to generate reproduced moving image data using the encoding method corresponding to the encoding method shown in the above embodiments , and supply the data to the display unit ex202 through the LCD control unit ex302, so that, for example, image data included in a moving image file linked to a web page is displayed. Meanwhile, the voice processing unit ex305 converts the voice data into analog voice data and supplies the data to the voice output unit ex208 so that voice data included in, for example, a moving image file linked to a web page is reproduced.

The present invention is not limited to the above system because ground-based or satellite digital broadcasting has been reported recently, and at least the moving picture encoding device or moving picture decoding device described in the above embodiments can be integrated into a digital broadcasting system as shown in FIG. 27 . More specifically, the coded video stream is transmitted from the broadcasting station ex409 to the broadcasting satellite ex410 or communicates with the broadcasting satellite ex410 via radio waves. Once the encoded video stream is received, the broadcast satellite ex410 transmits radio waves for broadcast. Then, a home antenna ex406 having a satellite broadcast receiving function receives radio waves, and a television (receiver) ex401 or a set-top box (STB) ex407 decodes the coded bit stream for reproduction. The moving picture decoding device as shown in the above-mentioned embodiments can be implemented in the reproducing device ex403 to read out and decode encoded streams recorded on the storage medium ex402 such as CD and Digital Versatile Disc (DVD). In this case, the reproduced moving image signal is displayed on the monitor ex404. It is also conceivable to realize moving picture decoding equipment in STB ex407 connected to cable ex405 for cable TV or antenna ex406 for satellite and/or ground-based broadcasting so as to reproduce them on monitor ex408 of television set ex401. It is also possible to incorporate a moving picture decoding device into a television set instead of a set-top box. Also, a car ex412 having an antenna ex411 can receive a signal from a satellite ex410 or a cell site ex107 to play back a moving image on a display device such as a car navigation system ex413 provided in the car ex412.

In addition, the moving image coding apparatus as shown in the above-mentioned embodiments can encode image signals and record them on a storage medium. As specific examples, a recorder ex420 such as a DVD recorder for recording image signals on a DVD disk ex421, and a disk recorder for recording image signals on a hard disk can be cited. They can be recorded on SD card ex422. When the recorder ex420 includes the moving image decoding device as described in the above-mentioned embodiments, image signals recorded on a DVD disk ex421 or SD card ex422 can be reproduced to display them on the monitor ex408.

As for the structure of the car navigation system ex413, a structure without the camera unit ex203, camera interface unit ex303, and image encoding unit ex312 among the components shown in FIG. 26 is conceivable. The above also applies to the computer ex111, the television (receiver) ex401, and the like.

In addition, for a terminal such as a cellular phone ex114, three types of realization are conceivable: a transmitting/receiving terminal realized with an encoder and a decoder, a transmitting terminal realized with only an encoder, and a receiving terminal realized only with a decoder.

Note that each functional block in the block diagrams shown in FIGS. 4 , 16A, 16B, 17A, 17B, 18A, 18B, and 22 can be realized as an LSI as an integrated circuit device. Such an LSI can be integrated in the form of one or more chips (for example, functional blocks other than memory can be integrated into a single chip). Here, LSI is taken as an example, however, depending on the degree of integration, it may be called "IC", "system LSI", "super LSI", and "ultra LSI".

The integrated circuit integration method is not limited to LSI, and it can be realized by a dedicated line or a general processor. After the LSI is manufactured, a programmable field programmable gate array (FPGA), or a reconfigurable processor capable of reconfiguring connections and settings for circuit cells in the LSI may be utilized.

Furthermore, with the advent of integrated circuit integration technology that replaces LSI due to the advancement of semiconductor technology or another technology different from semiconductor technology, integration of functional blocks can be implemented using newly emerging technology. Biotechnology applications can be cited as one example.

Among the functional blocks, only a unit for storing data can be separately configured as the storage medium 115 described in this embodiment without being integrated in a chip form.

Note that the functional blocks shown in FIGS. 4 and 22 or main parts in the flowcharts shown in FIGS. 19 to 21 can be realized by a processor or a program.

As described above, it is possible to use the image encoding method and the image decoding method proposed in the above embodiments in any of the above devices or systems. Therefore, it is possible to obtain the effects described in the above embodiments.

(second embodiment)

A second embodiment of the present invention is described below.

<Integer-based division-free quantization scheme>

In order to reduce computational complexity, it is desirable to use only multiplication and shift operations for quantization with the q-matrix. In other words, computationally expensive division operations should actually be avoided. The proposed integer-based division-free non-uniform high-efficiency quantization method can generally be applied to block transformation and quantization of any size in video coding systems.

In FIGS. 16A and 16B , quantization and de-quantization (de-quantization) operations of transform coefficients in the case where only multiplication and shift operations are used are illustrated. For a given QP, the values of S1 and S2 are fixed for all coefficients, while the values of Q1 and Q2 depend on the QP value and coefficient position (see Ref.1: Joint Video Team of ISO/IEC MPEG & ITU-T VCEG ( JVT), "Draft of ITU-T Recommendation and Final Draft International Standard of Joint Video Specification (ITU-T Recommendation and Final Draft International Standard Draft of Joint Video Specification) (ITU-T Rec.H.264|ISO/IEC14496-10 AVC )", JVT-G050r1.doc, Geneva, Switzerland, May 2003). Note that this is not a weighted quantization mechanism, but just a normalization problem, because in video coding standards the rows of integer transforms are orthogonal but do not have the same norm (see Ref.1).

Integrating the q-matrix into division-free quantization and dequantization can first be seen as introducing another pair of multiplications and shifts in these two processes. This is illustrated in Figures 17A and 17B. Note that the values of Qa and Qb depend on the specified q-matrix entries as well as the specified value of Sa.

In order to reduce computational complexity, the division operation can be approximated using only multiplication and shift operations. For a given q-matrix entry Mq and a given (or agreed upon by both encoder and decoder) Sa value, the value of the integer Qa is defined as (1<<(Sa+Bn))/Mq. Accordingly, the value of the integer Qb is defined as Mq<<(Sa-Bn). For example, Bn can be set to 4. Obviously, in this case quantization with a q-matrix with all entries equal to 16 will be the same as uniform quantization in the video coding standard (see Ref. 1). Note that Sa>Bn, and usually Sa>=8. This design is used to maintain a certain level of precision in integer calculations, since both Qa and Qb are integers. Considering this design with quantization of the q-matrix, it can be shown that all operations and memory accesses can be computed in 16 bits. Note that intermediate results are allowed to exceed 16 bits if the data is scaled/shifted before storage. Larger values of Sa allow better integer quantization calculation accuracy, however, the value should be constrained with respect to the dynamic range of the intermediate results with respect to the hardware design.

<Single multiplication of dequantization and normalized inverse transform>

From Figures 17A and 17B, it is straightforward to combine multiply and shift operations accordingly, thereby reducing computational complexity. Figures 18A and 18B show further integration levels using q-matrix quantization. Specifically, the integer Q1a is defined as Q1*(1<<(Sa+Bn))/Mq, and the integer Q2a is equal to Q2*Mq<<(Sa−Bn). Since Q1 and Q2 are obtained from a lookup table, the integration of the q-matrix basically results in an update of the lookup table during execution.

During the initialization phase, the table can be easily changed according to the specified q-matrix. Note that the value of the integer S2 is defined as QP/6, and it is always less than 8, because QP must be in the range [0, 51] in video coding standards (see Ref.1). Importantly, the proposed ensemble helps to make Q1a values differentiable for different values of each q-matrix term (e.g. typically [1, 255]). From an encoder design point of view, this is a desirable feature as it allows the flexibility of finer tuning in terms of quantization.

<Example of new lookup table based on q matrix derivation>

We get a quantization formula of the following general form:

Quantification:

C _ij ＝sign(x _ij )*(abs(x _ij )*Q _q (QP%6, i, j)+(1<<n)*f)>>n

Dequantization:

y _ij ＝sign(c _ij )*((abs(c _ij )*Q _d (QP%6, i, j))>>m)

Here, x _ij , c _ij , and y _ij represent original, quantized, and dequantized coefficients, respectively. _Qq and _Qd are matrices used in quantization and dequantization, respectively. Considering the normalization of integer transformations, Q _q and Q _d are different. Note that in the case of uniform quantization, _Qq and _Qd will be flat matrices. sign(x) equals 1 if x is positive, 0 if x=0, or -1 if x is negative. abs(x) is the absolute value of x. The values of m and n depend on QP, block transform size and Sa value. The f-value usually depends on the block coding type; for example, it is equal to 3/8 for an intra-coded block and 1/6 for an inter-coded block.

<new lookup table/matrix associated with 8×8q matrix>

Consider the case where Bn=4, Sa=8, and matrices Q _q and Q _d are derived as shown in Figures 30 and 31 . For 8×8 integer transforms whose rows are orthogonal and have the same norm, it is straightforward to derive a lookup table using the q matrix. The 8x8q matrix shown in Fig. 28 is used.

Based on the 8×8 integer transformation, quantization and dequantization scaling factor (scaling factor) tables as shown in FIG. 29 are respectively obtained.

In the case of QP=20, the scaling factor (used as a multiplier) for quantization will be equal to 1979 and the scaling factor for dequantization will be equal to 19. Note that in case of uniform quantization these values are the same for all transform coefficients. When using the q-matrix, the scaling factors essentially become the matrices shown in Figures 30 and 31, respectively. Each scaling factor can be initialized when the q matrix is specified.

<new lookup table/matrix associated with 4×4q matrix>

Consider the case where Bn=4, Sa=8, and matrices Q _q and Q _d are derived as shown in FIGS. 34 and 35 . For 4×4 integer transforms with row-orthogonal but different norms (see Ref. 1), deriving the lookup table with the q-matrix requires taking into account the position of the coefficients in the matrix. A 4x4q matrix shown in Fig. 32 is used. The quantization and dequantization scaling factor tables are determined as shown in FIG. 33 (considering normalization of integer transforms).

In the case of QP=20, the matrix of scaling factors (used as multipliers) for quantization and dequantization will be as shown in Figures 34 and 35, respectively.

When specifying 4x4q matrices, these matrices can be initialized with respect to each row of matrices w and v listed in FIG. 33 .

<Separately process DC block quantization>

In case the Hadamard transform is used, the proposed quantization with the q-matrix is not suitable for the second level transform, ie the DC component transform of luma and chrominance. Here are some reasons. In the encoder, quantization follows the transform, while in the decoder, dequantization follows the inverse transform, not before. This is to preserve the possible dynamic range (integer computation precision) during the inverse transform. As a result, quantization and dequantization are actually in different domains. This is not a problem for uniform quantization, but will cause a misalignment of the scaling factors of the coefficients in the process of using the weighted q-matrix. In addition, the Hadamard transform itself does not have to maintain the same DCT (or approximate integer transform) properties as in the first stage. Weighted quantization in the Hadamard transform domain may not be equally meaningful.

This innovation extends uniform quantization schemes to weighted quantization schemes using q matrices while maintaining integer operations and keeping complexity to a minimum. As an important example, this solution allows the integration of quantization matrix schemes into current video coding system (see Ref. 1) implementations with negligible complexity increase and minimal syntax changes.

(third embodiment)

A third embodiment of the present invention is described below.

In video coding systems, a set of quantization matrices is defined by default such that a decoder implements the quantization scheme and matrices. They are used for decoding when the encoded bitstream uses a quantization weighting scheme. Users can define their own quantization matrices and send them to the decoder. This innovation will dictate how the quantization matrix is sent to the decoder.

The innovation also defines the transform selection among various coding mode selections.

In the quantization weighting scheme, we describe the following main features:

1. Specification of a division-free quantization weighting scheme based entirely on integers, requiring only 16-bit memory operations at the decoder, which introduces no increase in complexity compared to uniform quantization schemes.

2. The proposed non-uniform quantization scheme is based on an 8×8 transform to luminance, because we believe that this is the transform that better preserves image texture, which is one of the most important content elements affecting the subjective impression of high-quality images.

Non-uniform quantization weighting is applied to residuals in both intra prediction and inter prediction. This innovation provides a new set of encoding tools and extends uniform quantization schemes to weighted quantization schemes using q matrices while maintaining integer operations and keeping complexity to a minimum. This makes video encoding particularly efficient in high-quality and high-bit-rate encoding applications.

AVC's Professional Extension Profiles (Professional Extension Profiles (Fidelity Range Profiles)) are designed to encode high-resolution images, including high-definition (HD) images. Also, in HD image representation, a high visual fidelity is largely realized. When using AVC Professional Extension Profiles, it is natural to want to allow AVC's excellent encoding efficiency to maximize its direct benefit to visual quality. To improve HD subjective quality, here we propose a quantization weighting scheme that enables non-uniform quantization weighting for block transform coefficients. We believe non-uniform quantization tools are critical for the following reasons:

1. Improve the visual fidelity of encoded images.

2. Non-uniform quantization makes it possible to perform quantization adjustment in proportion to human visual sensitivity, which improves coding efficiency for image fidelity.

3. Provide flexible options to control the final image quality, which is strongly demanded by the high-quality content generation industry.

The quantitative weighting scheme proposed here includes the following main features:

1. Specification of a division-free quantization weighting scheme based entirely on integers, requiring only 16-bit memory operations at the decoder, which introduces no increase in complexity compared to uniform quantization schemes.

2. The proposed non-uniform quantization scheme is based on an 8×8 transform to luminance, because we believe that this is the transform that better preserves image texture, which is one of the most important content elements affecting the subjective impression of high-quality images.

3. Non-uniform quantization weighting is applied to the residual in intra prediction and inter prediction.

Our simulations and viewings in various HD display devices have shown improvements in the subjective quality of all video sequences as well as substantial improvements in many test videos, including some of our JVT sequences and film content from film studios.

<Recommended range for 8×8 transform and quantization weighting matrices>

Based on the many JVT submissions previously submitted to JVT showing good coding efficiency, we will propose to include the 8×8 transform. More importantly, it has been shown that the subjective quality obtained by using the 8x8 transform provides better image texture preservation. Because we have a well-established 8×8 transform on which many previous contributions and previous AVC committee drafts were based (see Ref. 2: S. Gordon, D. Marpe, T. Wiegand, “Simplified use of 8×8 transform(8 ×8 transform), "ISO/IEC JTC1/SC29/WG11 and ITU-TQ6/SG16, document JVT-J029, December 2003..), so we use that transform in our proposal. However, we don't think the results are much different if other 8x8 integer transformation matrix choices are used.

<8×8 luma intra prediction>

Here, in addition to the existing modes Intra 16×16 and Intra 4×4, a new macroblock mode mb_type I_8×8 is proposed for luma 8×8 intra prediction. There are 9 intra 8×8 prediction modes. They are specified in Ref.2. Apply low-pass filtering to reference pixels in order to improve prediction effectiveness. Filtering is also specified in the ABT 8×8 Intra Prediction section in Ref.2.

<chroma intra prediction>

Depending on the chroma sampling format, different quantization weights should be used. For 4:2:0 and 4:2:2 formats, 4x4 quantization weights are used, where the quantization scheme is defined below. For the 4:4:4 format, the same transformation and quantization scheme is applied to the chroma samples.

<8×8 Inter Prediction>

The 8x8 transform is used for all 16x16, 16x8 and 8x16 P and B macroblock types. Also, for P slices, the 8x8 transform is used for any 8x8 submacroblock with sub_mb_type equal to P_L0_8x8, or for B slices, for B_Direct_8x8, B_L0_8x8, B_L1_8x8, or B_Bi_8x8.

<Syntax Elements of Quantized Weighting Matrix>

Additional syntax elements for quantization weighting matrices include defining user-defined weighting matrices at the beginning of the bitstream. Quantization weighting matrices are referenced by a matrix identifier (ID) in the image parameter set.

<quantization weighting matrix>

The quantization weight matrix is applied to the quantization step size just before the inverse transform in the decoder. The weighting at each coefficient index can be different to provide uniform quantization. The weights are appended to the QP defined in the syntax, so that the quantization applied is actually a combination of the quantization weights and the QP (Fig. 36). The weighting matrix can reduce and increase the amount of quantization relative to QP.

In the AVC specification, dequantization is performed by dequantization scaled multiplication followed by shifting, where dequantization is computed by QP modulo 6. There is one integer multiplication per dequantization operation. Quantization is defined similarly when quantization weighting matrices are used.

In the quantization weighting matrix, in order to maintain the weight in a range from greater than 1 to less than 1, the value of the quantization weighting matrix is actually a rounded integer value multiplied by 16 of the actual weighting value. For example, a quantization weight value of 1.2 corresponds to a quantization weight matrix value of 19. Fig. 37 shows an example of a quantization weight matrix.

<Quantization weighting for 8×8 luminance>

For 8x8 luma, d _ij represents the quantized transform coefficient. W(i, j) represents a quantization weight matrix. Thus, we get the dequantization operation as shown in FIG. 38 .

<Quantization Weighting for 4×4 Block Transformation>

For 4:2:0 and 4:2:2 chroma formats, the chroma is transformed to the 4x4DCT domain. The 4x4 quantization weighting matrix is only applied to the AC coefficients. For each 8x8 chroma block, a 2x2 DC block is formed and a transform is further applied in which the coefficients are uniformly quantized. This is the same as the current specification.

Ref.3: ITU-T recommends H.264 and ISO/IEC international standard 14496-10AVC, document JVT-J010d7, the chroma QP derivation process in October 2003 remains the same here. However, the dequantization for the 2×2 chroma DC part is defined as follows,

dcC _ij ＝(f _ij *M(QP _C ％6,0,0)<<QP _C /6-5, for QP _C /6≥5 (5)

dcC _ij = (f _ij *M(QP _C %6,0,0)+1<<(4-QP _C /6))>>5-QP _C /6, for QP _C /6<5(6)

in

M(QP _C %6,0,0)=W(0,0)=W(0,0)*LevelScale(QP _C %6,0,0) (7)

The weighting matrix is only applied to the AC part of the quantized coefficients. Similarly, dequantization is defined by equations (1) and (2) in FIG. 38 , except that the LevelScale function is defined as in 8.5.8 in Ref.3 as shown in FIGS. 39 and 40 .

The rest of the decoding process is as specified in Ref.3. In the case of 4:4:4 chroma sampling, each 8×8 chroma block is transformed and quantized in the same way as luma.

In field coding mode, the field macroblocks in a macroblock pair use the same set of quantization matrices. In the case of B_Direct_8x8 mode in field coding mode, when the motion search block mode is smaller than 8x8 blocks, we would also like to propose to allow the use of 4x4 quantization weights for residuals of 4x4 transform blocks.

In applying the quantization weighting scheme, care must be taken that special care must be taken when designing the quantization weighting matrix in conjunction with QP. It should be ensured that the quantization weights do not extend the bit length of any coefficients. It is most desirable to maintain some balance between the matrix coefficients, although this is the responsibility of the encoder.

<Derivation of Dequantization Scaling Table Based on Quantization Weighting Matrix>

When the user-defined quantization weight matrix is sent to the decoder, the decoder needs to construct the dequantization scaling table according to the quantization weight and QP%6. Each QP%6 corresponds to a scaling table. The table can be calculated by multiplying each entry of the uniform dequantization and scaling table in the current AVC specification by the quantization weighting matrix. An example of this is the dequantization table derivation shown in Fig. 41 based on the transform proposed in Ref.2. If we get a quantization matrix with the values shown in Fig. 41, followed by an additional left shift of 4 times.

The dequantization coefficient table when QP%6=0 will be as shown in FIG. 42 .

<complexity>

Once the quantization scaling table M(QP%6, 0, 0) is generated from the quantization weighting matrix and QP/6, there is no additional complexity to introduce a quantization matrix compared to the current uniform quantization. There are 64 integer multiplications per 8x8 quantization and 16 integer multiplications per 4x4 quantization matrix to generate the scaling table M(QP%6,0,0). Each 8×8 quantization matrix has a maximum total storage capacity of 768 bytes (64*2*6), and each 4×4 quantization matrix has a maximum total storage capacity of 192 bytes.

<Bitstream syntax for quantization weight matrix>

Encoders should be able to choose whether to use quantization weighting. To allow this, the use_weighting_matrix flag needs to be set in the sequence parameter set that uses the quantized weighting matrix.

As in MPEG-2, the user can define quantization weighting matrices in addition to the default set of matrices. A user-supplied quantization weighting matrix can be loaded into the decoder using pic_parameter_ser. Subsequently, the loaded quantization weight matrix can be referenced by other pic_parameter_sets. The loaded matrix will be used to generate the dequantization table which will be stored in the decoder. The loaded matrices are numbered by ID. The weighting matrices can be stored in memory within the current image (available for slices in the current frame), or long-term. The weighting matrices that will only be kept in memory within the current image are local weighting matrices. They can be distinguished from long-term matrices by weighting matrix ID=0-1. Weighting matrices of size 4x4 and 8x4 can have the same ID.

For cases where there is no user-defined weighting matrix, a default set of quantized weighting matrices can be utilized. This set of default quantization weight matrices is known to the decoder. All quantization scaling tables are pre-defined for this set of default weighting matrices, so no precomputation is required. The set of default quantization weighting matrices includes two luma quantization matrices (intra prediction and inter prediction) and two chroma quantization matrices (4×4).

The weighting matrix can then be referenced by an identification number. We propose, for each image parameter set, at most one luma quantization matrix per inter and per intra prediction method. Similarly, there is at most one 4x4 quantization matrix per inter and intra prediction mode.

<syntax>

The following describes the proposed pic_parameter_set_rbsp with embedded entries for defining quantization matrices for 8x8 or 4x4 inter-predicted, intra-predicted macroblocks.

pic_parameter_set_rbsp(){

...

new_quantization_matrices_defined

if(new_quantization_matrices_defined)

def_quant_weighting_matrix

...

intra_quant_mat8_update

if(intra_quant_mat8_update)

Quant_mat8_id

inter_quant_mat8_update

if(inter_quant_mat8_update)

Quant_mat8_id

intra_quant_mat4_update

if(intra_quant_mat4_update)

Quant_mat4_id

inter_quant_mat4_update

if(inter_quant_mat4_update)

Quant_mat4_id

...

}

def_quant_weighting_matrix(){

load_quant_mat8

if(load_quant_mat8){

num_quant_mat8

     for(k=0; k<num_quant_mat8; k++){

Quant_mat8_id

for(i=0; i<8; ++i)

for(j=0;<8;++j)

Quant_mat8[i][j]

}

}

load_quant_mat4

if(load_quant_mat4)

num_quant_mat4

  for (k=0; k<num_quant_mat4; k++){

Quant_mat4_id

for(i=0; i<4; ++i)

for(j=0;<4;++j)

Quant_mat4[i][j]

}

}

}

(fourth embodiment)

A fourth embodiment of the present invention is described below.

<Scaling and transform processing for Intra_16×16 (intra 16×16) macroblock type luminance DC transform coefficients>

Note that the following formula can be used to dequantize scaling function operations.

If QP' _Y is greater than or equal to 36, the scaled result will be deduced as

dcY _ij = (f _ij *LevelScale _{4×4L, intra} ( _QP'Y %6,0,0))<<(QP'Y/6-6 ₎ , i,j=0...3

Otherwise (QP' _Y is less than 36), the scaling result will be deduced as

$\mathrm{<span\; class="notranslate">\; dc</span>}{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="4"\; label="LevelScal\; e"\; filenames="C20058000303500703.png"\; state="\{\{state\}\}">LevelScal</figure-callout></span><figure-callout\; id="4"\; label="LevelScal\; e"\; filenames="C20058000303500703.png"\; state="\{\{state\}\}">\; </figure-callout>}{\mathrm{<span\; class="notranslate">\; </span>}}_{}{\mathrm{<span\; class="notranslate">e</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 4</span>}\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; intra</span>}}<span\; class="notranslate">\; (</span>{{\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="6"\; label="Y\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="Y\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">Y</figure-callout></figure-callout></span><figure-callout\; id="6"\; label="Y\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="Y\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">\; </figure-callout></figure-callout>}}<span\; class="notranslate">\; </span><span\; class="notranslate"></span><span\; class="notranslate">\%</span>\mathrm{<span\; class="notranslate">\; 6,0,0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; 2</span>}}^{{{\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; Y</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; -</span>{{\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; Y</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; 3</span>}$

<Scaling and transformation processing for 4:2:0 chroma format chroma DC transform coefficients>

Note that the following formula can be used to dequantize scaling function operations.

If QP' _C is greater than or equal to 30, the scaled result will be derived as

dcC _ij = (f _ij *LevelScale _{4×4C, Intra} (QP' _C %6, 0, 0)) <<(QP' _C /6-5), i, j = 0, 1

Otherwise (QP' _C is less than 30), the scaling result will be derived as

$\mathrm{<span\; class="notranslate">\; dc</span>}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="4"\; label="LevelScal\; e"\; filenames="C20058000303500703.png"\; state="\{\{state\}\}">LevelScal</figure-callout></span><figure-callout\; id="4"\; label="LevelScal\; e"\; filenames="C20058000303500703.png"\; state="\{\{state\}\}">\; </figure-callout>}{\mathrm{<span\; class="notranslate">\; </span>}}_{}{\mathrm{<span\; class="notranslate">e</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 4</span>}\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Intra</span>}}<span\; class="notranslate">\; (</span>{{\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="6"\; label="C\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="C\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">C</figure-callout></figure-callout></span><figure-callout\; id="6"\; label="C\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="C\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">\; </figure-callout></figure-callout>}}<span\; class="notranslate">\; </span><span\; class="notranslate"></span><span\; class="notranslate">\%</span>\mathrm{<span\; class="notranslate">\; 6,0,0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; 2</span>}}^{{{\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span>{{\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0,1</span>}$

<Scaling and transformation processing for 4:2:2 chroma format chroma DC transform coefficients>

The input to this process is the transform coefficient level values of the chroma DC transform coefficients of one chroma component of the macroblock as a 2x4 array C with elements _Cij , where i and j form a two-dimensional frequency index.

The output of this process is the 8 scaled DC values as a 2x4 array dcC with elements dcC _ij .

The inverse transform of the 2x4 chroma DC transform coefficients is specified by:

The bitstream shall contain no data that would cause any element f _ij of f to exceed the range of integer values [-2 ¹⁵ , 2 ¹⁵ -1].

Variable QP' _{C, DC} = QP' _C +3

After inverse transformation, scaling is performed as follows.

If QP' _C,DC is greater than or equal to 36, the scaled result will be derived as

dcC _ij = (f*LevelScale _{4×4C, Intra} (QP'C _{, DC} %6,0,0,0))<<(QP'C _,DC /6-6), i=0...3, j=0,1

Otherwise (QP' _{C, DC} less than 36), the scaling result will be derived as

$\mathrm{<span\; class="notranslate">\; dc</span>}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="4"\; label="LevelScal\; e"\; filenames="C20058000303500703.png"\; state="\{\{state\}\}">LevelScal</figure-callout></span><figure-callout\; id="4"\; label="LevelScal\; e"\; filenames="C20058000303500703.png"\; state="\{\{state\}\}">\; </figure-callout>}{\mathrm{<span\; class="notranslate">\; </span>}}_{}{\mathrm{<span\; class="notranslate">e</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 4</span>}\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Intra</span>}}<span\; class="notranslate">\; (</span>{{\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="6"\; label="DC\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="DC\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="DC\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">DC</figure-callout></figure-callout></figure-callout></span><figure-callout\; id="6"\; label="DC\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="DC\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="DC\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">\; </figure-callout></figure-callout></figure-callout>}}<span\; class="notranslate">\; </span><span\; class="notranslate"></span><span\; class="notranslate"></span><span\; class="notranslate">\%</span>\mathrm{<span\; class="notranslate">\; 6,0,0,0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; 2</span>}}^{{{\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; DC</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; -</span>{{\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; DC</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0,1</span>}$

The bitstream shall contain no data that would cause any element dcC _ij of dcC to exceed the integer value range [-2 ¹⁵ , 2 ¹⁵ -1].

<Scaling and transformation processing for 4:4:4 chroma format chroma DC transform coefficients>

The input to this process is the transform coefficient level values of the chroma DC transform coefficients of one chroma component of the macroblock as a 4x4 array C with elements _Cij , where i and j form a two-dimensional frequency index.

The output of this process is the 16 scaled DC values as a 4x4 array dcC with elements dcC _ij .

The inverse transform of the 4×4 chroma DC transform coefficients is specified as shown in FIG. 43 .

The bitstream shall contain no data that would cause any element f _ij of f to exceed the range of integer values [-2 ¹⁵ , 2 ¹⁵ -1].

After inverse transformation, scaling is performed as follows.

If QP' _C is greater than or equal to 36, the scaled result will be derived as

dcC _ij = (f*LevelScale _{4×4C, Intra} (QP' _C %6, 0, 0, 0)) <<(QP' _C /6-6), i=0...3, j=0, 1

Otherwise (QP' _C is less than 36), the scaling result will be derived as

$\mathrm{<span\; class="notranslate">\; dc</span>}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="4"\; label="LevelScal\; e"\; filenames="C20058000303500703.png"\; state="\{\{state\}\}">LevelScal</figure-callout></span><figure-callout\; id="4"\; label="LevelScal\; e"\; filenames="C20058000303500703.png"\; state="\{\{state\}\}">\; </figure-callout>}{\mathrm{<span\; class="notranslate">\; </span>}}_{}{\mathrm{<span\; class="notranslate">e</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 4</span>}\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Intra</span>}}<span\; class="notranslate">\; (</span>{{\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="6"\; label="C\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="C\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="C\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">C</figure-callout></figure-callout></figure-callout></span><figure-callout\; id="6"\; label="C\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="C\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}"><figure-callout\; id="0"\; label="C\; \%"\; filenames="C20058000303500553.png"\; state="\{\{state\}\}">\; </figure-callout></figure-callout></figure-callout>}}<span\; class="notranslate">\; </span><span\; class="notranslate"></span><span\; class="notranslate"></span><span\; class="notranslate">\%</span>\mathrm{<span\; class="notranslate">\; 6,0,0,0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; 2</span>}}^{{{\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; DC</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; -</span>{{\mathrm{<span\; class="notranslate">\; QP</span>}}^{<span\; class="notranslate">\; ,</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0,1</span>}$

The bitstream shall contain no data that would cause any element dcC _ij of dcC to exceed the integer value range [-2 ¹⁵ , 2 ¹⁵ -1].

<Scaling and transformation processing for residual 8×8 block>

The input to this process is an 8x8 array c with elements _cij , which is an array of 8x8 residual blocks with respect to the luma component.

The output of this process is residual sampled values as an 8x8 array r with elements rij.

The function LevelScale64 is derived as follows:

- If the macroblock prediction mode is intra prediction mode, and the input is 8×8 luma residual, then LevelScale64()=LevelScale _{8×8, Intra} ().

- If the macroblock prediction mode is inter prediction mode, and the input is an 8×8 luma residual block, then LevelScale64()=LevelScale _{8×8, Inter} ().

Scaling of 8x8 block transform coefficient levels c _ij is performed as follows.

- If QP _Y is greater than or equal to 36, the scaling of the 8×8 block transform coefficient levels c _ij will be performed as

d _ij =(c _ij *LevelScale64(QP _Y %6, i, j))<<(QP _Y /6-6), i, j=0...7

- Otherwise (QP _Y is less than 36), the scaling of the 8x8 block transform coefficient levels c _ij will be performed as

${\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; Level\; Scale</span>}\mathrm{<span\; class="notranslate">\; 64</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="6"\; label="QP\; Y\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}">QP</figure-callout></span><figure-callout\; id="6"\; label="QP\; Y\; \%"\; filenames="C20058000303500531.png,C20058000303500553.png"\; state="\{\{state\}\}">\; </figure-callout>}}_{\mathrm{<span\; class="notranslate">\; Y</span>}}<span\; class="notranslate">\; \%</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; 2</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; QP</span>}}_{\mathrm{<span\; class="notranslate">\; Y</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; ></span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; QP</span>}}_{\mathrm{<span\; class="notranslate">\; Y</span>}}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span>\mathrm{<span\; class="notranslate">\; 7</span>}$

The above formula reflects the additional right shift required to incorporate the scaling coefficients from the q matrix, since the dequantization/quantization weights are defined as w(i,j)=QuantizationMatrix(i,j)/16. After incorporating QuantizationMatrix(i,j) into the LevelScale function, we need to do an additional 4-bit right shift to reflect the division by 16.

The transform process will convert the block of scaled transform coefficients into a block of output samples in a manner that is mathematically equivalent to the following.

First, each (horizontal) row of scaling transform coefficients is transformed using a one-dimensional inverse transform as follows.

-Deduce a set of intermediate values e _ij by

e _i0 =d _i0 +d _i4 , i=0...7

e _i1 =-d _i3 +d _i5 -d _i7 -(d _i7 >>1), i=0...7

e _i2 =d _i0 -d _i4 , i=0...7

e _i3 ＝d _i1 +d _i7 -d _i3 -(d _i3 >>1), i=0...7

e _i4 =(d _i2 >>1)-d _i6 , i=0...7

e _i5 ＝-d _i1 +d _i7 +d _i5 +(d _i5 >>1), i=0...7

e _i6 =d _i2 +(d _i6 >>1), i=0...7

e _i7 ＝d _i3 +d _i5 +d _i1 -(d _i1 >>1), i=0...7

- Calculate the second set of intermediate results f _ij from the intermediate values e _ij as

f _i0 ＝e _i0 +e _i6 , i=0...7

f _i1 ＝e _i1 +(e _i7 ＞＞2), i＝0...7

f _i2 =e _i2 +e _i4 , i=0...7

f _i3 =e _i3 +(e _i5 >>2), i=0...7

f _i4 =e _i2 -e _i4 , i=0...7

f _i5 ＝(e _i3 >>2)-ei5, i=0...7

f _i6 =e _i0 -e _i6 , i=0...7

f _i7 ＝e _i7 -(e _i1 >>2), i=0...7

Then, from these intermediate values f _ij the transformation result g _ij is calculated as

g _i0 =f _i0 +f _i7 , i=0...7

g _i1 =f _i2 +f _i5 , i=0...7

g _i2 =f _i4 +f _i3 , i=0...7

g _i3 =f _i6 -f _i1 , i=0...7

g _i4 =f _i6 -f _i1 , i=0...7

g _i5 =f _i4 -f _i3 , i=0...7

g _i6 =f _i2 -f _i5 , i=0...7

g _i7 =f _i0 -f _i7 , i=0...7

Each (vertical) column of the resulting matrix is then transformed using the same one-dimensional inverse transform as follows.

- Calculate a set of intermediate values h _ij from the horizontal transformation values g _ij as

h _i0 =g _i0 +g _i4 , i=0...7

h _i1 =-g _i3 +g _i5 -g _i7 -(g _i7 >>1), i=0...7

h _i2 =g _i0 -d _i4 , i=0...7

h _i3 ＝g _i1 +g _i7 -g _i3 -(g _i3 >>1), i=0...7

h _i4 ＝(g _i2 >>1)-g _i6 , i=0...7

h _i5 ＝-g _i1 +g _i7 +g _i5 +(g _i5 >>1), i=0...7

h _i6 =g _i2 +(g _i6 >>1), i=0...7

h _i7 ＝g _i3 +g _i5 +g _i1 +(g _i1 >>1), i=0...7

- Calculate the second set of intermediate results k _ij from the intermediate values h _ij as

k _i0 =h _i0 +h _i6 , i=0...7

k _i1 =h _i1 +(h _i7 >>2), i=0...7

k _i2 =h _i2 +h _i4 , i=0...7

k _i3 =h _i3 +(h _i5 >>2), i=0...7

k _i4 =h _i2 -h _i4 , i=0...7

k _i5 =(h _i3 >>2)-h _i5 , i=0...7

k _i6 =h _i0 -h _i6 , i=0...7

k _i7 =h _i7 -(h _i1 >>2), i=0...7

- Calculate the transformation result m _ij from the intermediate value k _ij as

m _i0 =k _i0 +k _i7 , i=0...7

m _i1 =k _i2 +k _i5 , i=0...7

m _i2 =k _i4 +k _i3 , i=0...7

m _i3 =k _i6 +k _i1 , i=0...7

m _i4 =k _i6 -k _i1 , i=0...7

m _i5 =k _i4 -k _i3 , i=0...7

m _i6 =k _i2 -k _i5 , i=0...7

m _i7 =k _i0 -k _i7 , i=0...7

After performing a 1D horizontal and 1D vertical inverse transform to produce an array of transformed samples, the final constructed residual sample values are derived as

r _i7 =(m _ij +2 ⁵ )>>6, i, j=0...7

From the invention thus described, it will be obvious that the embodiments of the invention may be varied in many ways. Such changes should not be regarded as departing from the spirit and scope of the present invention, and it is obvious to those skilled in the art that all such changes should be included within the scope of the following claims.

Industrial Applicability

The present invention is applicable to encoding devices that encode or decode images, and is also applicable to network servers that distribute moving images, network terminals that receive moving images, and even digital cameras capable of recording and reproducing moving images, camera-equipped cellular phones, DVD recorders/players, PDAs and personal computers.

Claims (5)

1. an inverse quantization method is used for obtaining the re-quantization orthogonal transform coefficient by the orthogonal transform coefficient that has quantized is carried out re-quantization, and described method comprises:

Obtain weighting matrix;

Obtain quantization parameter;

By will be from described weighting matrix the value calculated of the component of the capable j column position of i multiply each other with quantization step and calculate the point scale value, the position of the component that described quantization step is listed as corresponding to the capable j of i in the described weighting matrix and the remainder that described quantization parameter is obtained by an integer division;

The orthogonal transform coefficient and the described point scale value that have quantized are multiplied each other; And

The described product that is obtained that multiplies each other is shifted according to the bit number of described quantization parameter, to obtain the re-quantization orthogonal transform coefficient.

2. inverse quantization method according to claim 1, wherein said quantization step are for vertical and horizontal level and definite value according to the position of the component in the described weighting matrix.

3. picture decoding method is used for the orthogonal transform coefficient that has quantized is carried out re-quantization and inverse orthogonal transformation obtains the piece image, and described method comprises:

Obtain weighting matrix;

Obtain quantization parameter;

By will be from described weighting matrix the value calculated of the component of the capable j column position of i multiply each other with quantization step and calculate the point scale value, the position of the component that described quantization step is listed as corresponding to the capable j of i in the described weighting matrix and the remainder that described quantization parameter is obtained by an integer division;

The orthogonal transform coefficient and the described point scale value that have quantized are multiplied each other;

The described product that is obtained that multiplies each other is shifted according to the bit number of described quantization parameter, to obtain the orthogonal transform coefficient of re-quantization; And

By addition/subtraction computing and bit shift operation the re-quantization orthogonal transform coefficient that is obtained is carried out inverse orthogonal transformation and obtain the piece image.

4. picture decoding apparatus is used for the piece that to be the basis to image encoded data decode obtains the piece image, and described device comprises:

Obtain the unit, operation is used to obtain weighting matrix and quantization parameter, and by will be from described weighting matrix the value calculated of the component of the capable j column position of i multiply each other with quantization step and calculate the point scale value, the position of the component that described quantization step is listed as corresponding to the capable j of i in the described weighting matrix and the remainder that described quantization parameter is obtained by an integer division;

Multiplication unit, orthogonal transform coefficient and described point scale value that operation is used for having quantized multiply each other;

Shift unit is shifted the described product that is obtained that multiplies each other according to the bit number of described quantization parameter, to obtain the orthogonal transform coefficient of re-quantization; And

Inverse orthogonal transformation unit, operation is used for by addition/subtraction computing and bit shift operation the re-quantization orthogonal transform coefficient that is obtained being carried out inverse orthogonal transformation and obtains the piece image.

5. a processor is used in the decoding device that moving image is decoded, and described processor comprises:

Integrated circuit;

Wherein said processor

I) use described integrated circuit to obtain weighting matrix and quantization parameter;

Ii) by will be from described weighting matrix the value calculated of the component of the capable j column position of i multiply each other with quantization step and calculate the point scale value, the position of the component that described quantization step is listed as corresponding to the capable j of i in the described weighting matrix and the remainder that described quantization parameter is obtained by an integer division;

Iii) orthogonal transform coefficient that will quantize and described point scale value multiply each other;

Iv) the described product that is obtained that multiplies each other is shifted according to the bit number of described quantization parameter, to obtain the orthogonal transform coefficient of re-quantization; And

V) the result to described displacement carries out inverse orthogonal transformation.

Images