JP4819856B2 - Moving picture coding method, moving picture coding apparatus, moving picture coding program, and computer-readable recording medium recording the program - Google Patents

Description

  INDUSTRIAL APPLICABILITY The present invention is a moving image coding method and apparatus for coding a moving image by performing information compression by transform coding and quantization on a prediction error signal, and the moving image coding method. The present invention relates to a moving image encoding program and a computer-readable recording medium on which the program is recorded.

[Encoding method using cost function of square error criterion]
H. In H.264, with the introduction of intra prediction and variable shape motion compensation, the types of prediction modes are increasing compared to the conventional standardized method. For this reason, in order to reduce the amount of codes while maintaining a constant subjective image quality, it is necessary to select an appropriate prediction mode. H. In the H.264 reference software JM (see Non-Patent Document 1), the following prediction mode that minimizes the RD cost is selected. In the following notation, the symbol ^ in “^ X” (where X is a letter) indicates a symbol attached to “X”.

Here, S is an original signal, q is a quantization parameter, m is a number representing a prediction mode, ^ S _{m, q} is predicted for the original signal S using the prediction mode m, and the quantization parameter q is It is a decoded signal when quantized by using. Further, λ is a Lagrange's undetermined multiplier used for mode selection. Further, D (S, ^ S _{m, q} ) is a sum of square errors shown in the following equation.

Here, S ^Y , S ^U , and S ^V are Y, U, and V components of the original signal, respectively, and ＾ S ^Y _{m, q} , ＳS ^U _{m, q} , and ＳS ^V _{m, q} are respectively the decoded signals. Y, U and V components. Further, x ₀ and y ₀ are coordinate values at positions closest to the origin in the block.

  H. The calculation method of the decoded signal in H.264 is shown below. The symbols used for the explanation are summarized in the following table.

Let P _m be the prediction signal when the prediction method of mode number m is used for the original signal S. H. In the H.264 encoding process, the orthogonal transformation using the transformation matrix to Φ is applied to the prediction error signal R _m (= S−P _m ) when the prediction method of the mode number m is used as follows: Is done. In the following notation, the symbol “˜X” (where X is a letter) indicates a symbol attached to “X”.

Here, ~ Φ ^t represents a transposed matrix for the transformation matrix ~ Φ. Note that the transformation matrix to Φ is an orthogonal matrix of integer elements expressed by the following equation.

  Next, since the matrices Φ are non-normal matrices, processing corresponding to matrix normalization as shown below is performed.

  This is equivalent to using Φ of the following expression instead of ˜Φ in the expression (3).

Further, the quantization using the quantization parameter q is performed on C _m as follows. H. In the H.264 reference software JM, normalization is built into quantization.

On the other hand, H. In the H.264 decoding process, V is inversely quantized as in the following equation to obtain a decoded coefficient ^ C _{m, q} of the transform coefficient.

Next, inverse transform is applied to this ^ C _{m, q} as shown in the following equation to obtain a prediction error decoded signal.

  Finally, a decoded signal of the encoding target image is obtained by the following equation.

[Weighting distortion amount considering subjective image quality]
As described above, H.P. The measure of subjective image quality used in the H.264 reference software JM is a square error. However, this square error is not necessarily a distortion amount reflecting subjective image quality degradation. For example, a change in a high frequency component is less visually detected than a change in a low frequency component.

  However, an encoder (for example, JM) that does not use such visual characteristics still has room for improvement in terms of efficient code amount reduction.

  Therefore, studies have been made to use the difference in visual sensitivity with respect to spatio-temporal frequency components. For example, there is a study of adaptively setting the quantization step width according to the amount of shift (see Patent Document 1).

  Further, there is a study of using a distortion amount weighted on the basis of spatio-temporal visual sensitivity in a cost function for selecting an encoding parameter (see Non-Patent Document 2).

  In the method described in Non-Patent Document 2, the amount of distortion corresponding to the subjective image quality is defined by weighting the amount of distortion for each spatial frequency component according to visual sensitivity with respect to the orthogonal transform coefficient. Further, in consideration of the visual sensitivity in the time direction, the above-described weighted distortion amount is further weighted according to the shift amount. Thus, the weighted distortion amount based on the spatio-temporal visual sensitivity is used in the cost function for selecting the encoding parameter.

  The weighting based on the visual sensitivity for the quantization error signal will be described below. In the following method (the same applies to the present invention), it is assumed that the RD cost of the following equation is used.

Here, C _m is a transform coefficient for the prediction error signal R _m when the mode number m is used, and ^ C _{m, q} is a transform coefficient obtained by quantizing and dequantizing C _m with the quantization parameter q. Is the decoded value. The following weighted distortion amount is used as the distortion amount used for calculating the RD cost.

Here, C _m ^{s (i)} [k, l] (s = Y, U, V) is an element of C _m , and a macroblock (16 × 16 [pixel], U, V in the case of Y component) In the case of components, it is a conversion coefficient included in the i-th sub-block scanned in the raster scanning among the sub-blocks (N × N [pixels]) in 8 × 8 [pixels]). ^ C _{m, q} ^{s (i)} [k, l] (s = Y, U, V) is an element of ^ C _{m, q} , and a macroblock (16 × 16 [pixels in the case of Y component) ], U, and V components, the decoding transform coefficients included in the i-th sub-block scanned in the raster scan among the sub-blocks (N × N [pixels]) in 8 × 8 [pixels]). is there.

Further, W _{k, l} ^s (s = Y, U, V) is a weighting coefficient set to 1 or less, and is hereinafter referred to as a sensitivity coefficient. In Expression (12), setting W _{k, l} ^s to a small value corresponds to estimating the quantization distortion to D (C _m , ^ C _{m, q} ) to be small.

From the normality of orthogonal transformation, if W _{k, l} ^s = 1 (∀k, l; s = Y, U, V), the above-described weighted distortion amount is equivalent to the square error sum.

W _{k, l} ^s (s = Y, U, V) takes a smaller value as the spatial frequency and the temporal frequency are higher. A specific calculation method is discussed in Non-Patent Document 2.
KPLim and G. Sullivan and T. Wiegand, Text Description of Joint Model Reference Encoding Methods and Decoding Concealment Methods.Joint Video Team (JVT) of ISO / IEC MPEG and ITU-T VCEG, JVT-R95, Jan., 2006. http : //ftp3.itu.ch/av-arch/jvt-site/2006_01_Bangkok/JVT-R095.zip JP-A-5-236444 Yukihiro Bando, Kazuya Hayase, Masayuki Takamura, Hitoshi Uekura, Yoshiyuki Yajima, H.264 / AVC mode selection method based on spatio-temporal visual sensitivity characteristics. The Institute of Image Information and Television Engineers Annual Conference, 7-4, 2007

  In the above-described method of weighting the distortion amount based on the contrast sensitivity function (the method described in Non-Patent Document 2), the sensitivity function is weighted in units of macroblocks. Continuity (block distortion) is not reflected.

  In a block-based coding scheme (for example, H.264) that combines inter-frame prediction based on motion compensation and orthogonal transform, block distortion is a characteristic coding distortion. For this reason, when the block distortion is not taken into consideration, a case where the obtained weighted distortion amount cannot correctly reflect the subjective image quality occurs.

  The present invention has been made in view of the above circumstances, and has a subjective image quality including block distortion as compared with a block-based encoding method (for example, H.264) that combines inter-frame prediction based on motion compensation and orthogonal transform. By introducing a measure of coding distortion that appropriately evaluates the video, a new video that enables the implementation of a cost function used to select coding parameters that reflects subjective image quality including block distortion An object is to provide an image encoding technique.

  The reason why the block distortion is not reflected in the coding distortion scale of the conventional method described in Non-Patent Document 2 will be considered below.

  In the conventional method, the DCT coefficient of each macroblock is weighted based on the contrast sensitivity function. For this reason, the contrast sensitivity with respect to the waveform in each macroblock was reflected, but the discontinuity between adjacent blocks was not considered.

  Taking the one-dimensional signal shown in FIG. 10 as an example, in a process closed to blocks that weight the DCT coefficients of each block (block k-1, block k, block k + 1), discontinuity between blocks ( The discontinuity between block k-1 and block k, or the discontinuity between block k and block k + 1) is not known.

  Therefore, in the present invention, frequency analysis is performed in consideration of discontinuity between adjacent blocks together with the waveform in the block, and the distortion amount is weighted based on the contrast sensitivity function.

[Calculation of sensitivity coefficient]
In the present invention, the distortion amount in each block is weighted based on the spatiotemporal visual sensitivity function. In the calculation of the weighting coefficient, input is a conversion coefficient and a shift amount (indicating temporal movement of an image signal such as a motion vector). Here, also for a frame for which intra prediction is performed, a shift amount can be obtained by obtaining temporal movement of an image signal between frames.

  In the following, a case where an image having a vertical width H is observed at a viewing distance rH is considered. r is called a viewing distance parameter. Also, in the following, in order to simplify the expression, the subscript indicating the distinction between Y, U, and V is omitted, and the Y component will be discussed. The U and V components can be discussed in the same manner as described below.

The transformation of a two-dimensional signal by orthogonal transformation is to express a signal using a base image of the orthogonal transformation. When the k-th column vector (N-dimensional vector) of the transformation matrix Φ (N × N matrix) is φ _k , the base image for the matrix is
f _{k, l} (x, y) = φ _k [y] φ _l [x] ^t (0 ≦ x, y ≦ N−1)
It can be obtained from the formula H. In the case of H.264, the possible value for N is either 4 or 8. Here, φ _l ^t is a transposed vector of φ _l .

Predictive error signal R _m [x, y] (Ni _x ≦ x ≦ Ni _x + N−1, Ni _y ≦ y ≦ Ni _y + N−1) in the region of N × N [pixel] is ^expressed as R _m ^{(ix, iy).} And abbreviated as “reference block” hereinafter. Here, i _{x and} i _y are integer values indicating the position of the reference block. Further, assuming that the corresponding orthogonal transform coefficient is C _m ^{(ix, iy)} [k, l] (0 ≦ k, l ≦ N−1), the prediction error signal R _m ^{(ix, iy)} is given by It can be expressed as

M _x × M _y-number of reference block ^{_{(R m (ix, iy)}} ) composed of M _x N × M _y prediction including N pixel error signal _{R m [x, y] (} Ni x0 ≦ x ≦ N ( For i _x0 + M _x ) −1, Ni _y0 ≦ y ≦ N (i _y0 + M _y ) −1) (referred to as an analysis target block), the subjective image quality considering block distortion is considered.

Here, since the analyzed block is intended to be composed of M _x × M _y-number of reference blocks, when Fourier transform the analyte block, discontinuity between adjacent reference blocks not only the reference block It becomes possible to evaluate the property (block distortion).

To represent the relationship between the analyzed block and the reference block, as shown in the following ^formula, R _m ^{(ix, iy)} with respect to, the zero padded, obtaining a signal of M _x N × M _y N pixels .

Similarly, as shown in the following equation, the base image f _k, with respect to l (x, y) (0 ≦ x, y ≦ N-1), the zero padded, the M _x N × M _y N pixels Get a signal.

Zero padding the resulting ^{_{~f k, l (p) (}} x, y) (x = 0, ...., M x N-1; y = 0, ...., M y N-1; p = 0, ...., referred to as a M _x M _y -1) a modified base image. For example, in the case of _{N = 4, M x = 2} , M y = 2 , as shown in FIG. 1, is arranged the base image in 4 × 4 pixels shaded portion, are the zero value to the other position Padded.

Here, FIG. 1A corresponds to i _x = i _x0 +1, i _y = i _y0 , and FIG. 1B corresponds to i _x = i _x0 , i _y = i _y0 , and FIG. ) Corresponds to i _x = i _x0 +1, i _y = i _y0 +1, and FIG. 1D corresponds to i _x = i _x0 , i _y = i _y0 +1.

  At this time, the analysis target block can be expressed as follows using the corrected base image.

  The Fourier transform (represented by F [•]) on both sides of the equation (16) can be expressed as the following equation due to the linearity of the Fourier transform.

  Here, this equation (17) is obtained by applying a Fourier transform coefficient (frequency component due to discontinuity between adjacent reference blocks as well as frequency components in the reference block) obtained by performing Fourier transform on the analysis target block. This means that the corrected base image is represented by a linear sum of Fourier transform coefficients obtained by performing Fourier transform.

^{_{F [~f k, l (ix}} , iy)] is a complex vector of M _x N × M _y N-dimensional, the first (u _x, u _y) elements Fourier transform is expressed by the following equation It is a coefficient. In the following, it is assumed that N = 2 ^m .

Here, j is an imaginary unit. U _{x and} u _y are referred to as spatial frequency indexes.

In this way, the frequency component due to the discontinuity of the block is also taken into account by setting the target of the Fourier transform to ˜f _{k, l} (x, y) instead of f _{k, l} (x, y). Frequency analysis can be performed.

Fourier transform coefficients of the F _k, relative to _{l (u x, u y)} (0 ≦ u x ≦ M x N-1,0 ≦ u y ≦ M y N-1), performs the weighting of the following. Note that (d _{x, dy} ) is a transition amount estimated for the analysis target block.

Hereinafter, ~ F _{k, l} (ux _, u _y ) will be described. Here, {circumflex over (g)} (η, d) is a function set based on visual system characteristics such as contrast sensitivity, and is called a visual sensitivity function. As for the setting of the visual sensitivity function, for example, there is a method shown in [Visual sensitivity function setting 1] or [Visual sensitivity function setting 2] described later.

The sensitivity coefficient W _{k, l} ^s (d _x, d _y ) for the (k, l) basis component of the prediction error signal R _m is defined as the power ratio of the following equation.

  At this time, it is also possible to change the model parameter with the luminance component and the color difference component.

If explained the meaning of the formula (20), since the analyzed block is intended to be composed of M _x × M _y-number of reference blocks, if the analyte block Fourier transform, not only in the block It also becomes possible to evaluate discontinuity between adjacent blocks. On the other hand, as shown in Expression (17), Fourier transform of the analysis target block means that a linear sum of Fourier transform coefficients obtained by Fourier transform of the corrected base image (linear sum using orthogonal transform coefficients as coefficients). Is equivalent to calculating.

Therefore, in the present invention, in order to be able to evaluate not only the block but also the discontinuity between adjacent blocks, the sum of squares of the part described in {•} on the right side of the equation (17) (formula (Corresponding to the denominator of (20)) and calculating the sum of squares corresponding to the weight weighted by equation (19) (corresponding to the numerator of equation (20)), The sensitivity coefficient W _{k, l} ^s (d _x, d _y ) for the (k, l) basis component of the prediction error signal R _m is calculated according to the ratio (20).

  Incidentally, the sum of the calculated values of the denominator of Equation (20) with respect to k and l corresponds to the sum of squares of Equation (16), and from this, the denominator of Equation (20) corresponds to the power of the prediction error signal. To be.

Since the sensitivity coefficient W _{k, l} ^s (d _{x, dy} ) determined in this way indicates a value corresponding to the spatial frequency and the temporal frequency, the amount of distortion corresponding to the subjective image quality is defined. Will be able to. Moreover, since it is determined by performing a frequency analysis taking into account the discontinuity between adjacent blocks together with the waveform in the block, it is possible to define a distortion amount corresponding to subjective image quality including block distortion. become.

That is, the distortion amount of the spatio-temporal frequency component that is difficult to detect visually is set to a relatively small value because the sensitivity coefficient W _{k, l} ^s (d _{x, dy} ) is relatively small. On the other hand, the distortion amount of the spatio-temporal frequency component that is easy to detect visually is relatively high because the value of the sensitivity coefficient W _{k, l} ^s (d _{x, dy} ) becomes relatively large. Since the evaluation is performed as a large value, the distortion amount corresponding to the subjective image quality can be defined. Moreover, since the value of the sensitivity coefficient W _{k, l} ^s (d _{x, dy} ) is determined by performing frequency analysis in consideration of discontinuity between adjacent blocks together with the waveform in the block, The distortion amount corresponding to the subjective image quality including the block distortion can be defined.

  Next, [visual sensitivity function setting 1] and [visual sensitivity function setting 2], which are examples of the visual sensitivity function setting method, will be described.

[Visual Sensitivity Function Setting 1]
Consider a contrast sensitivity function such as:

Here, a ₁ , a ₂ , a ₃ , and a ₄ are parameters (hereinafter referred to as model parameters) that define the function form of the visual sensitivity function.
_{(A 1, a 2, a} 3, a 4) = (6.1, 7.31, 2, 45.9)
Such a value is used.

  Also, η is a spatial frequency [cycle / degree] representing the number of light-dark pairs within the unit viewing angle. Note that η is a one-dimensional spatial frequency.

At this time, between η and the spatial frequency index (either u _x or u _y ),
η (u, r) = θ (r, H) u / 2MN Equation (22)
There is a relationship.

In the case of u = u _x, a M = M _x, the case of u = u _y, is M = M _y. Θ (r, H) is an angle [degrees / pixel] per pixel when an image having a vertical width H is observed at a viewing distance rH.
θ (r, H) = (1 / H) × arctan (1 / r) × (180 / π)
... Formula (23)
Is given by the expression

ω is the angle change per unit time [degrees / sec]. At this time, between the ω the displacement amount d (either d _x or d _y) is
^{ω (d) = tan -1 (} f r d / rH) ··· formula (24)
There is a relationship. Here, _fr is a frame rate.

By substituting Equation (22) and Equation (24) into Equation (21) and expressing the contrast sensitivity function g (η, ω) as a function of u and d, ^ g (u, d)
^ G (u, d) = g (η (u), ω (d)) Equation (25)
Is a visual sensitivity function.

[Visual sensitivity function setting 2]
Consider the following contrast sensitivity function.

Here, b ₁ , b ₂ , b ₃ , and b ₄ are parameters (hereinafter referred to as model parameters) that define the function form of the visual sensitivity function.
_{(B 1, b 2, b} 3, b 4) = (0.4992, 0.2964, -0.114, 1.1)
_{(B 1, b 2, b} 3, b 4) = (0.2, 0.45, -0.18, 1)
_{(B 1, b 2, b} 3, b 4) = (0.31, 0.69, -0.29, 1)
_{(B 1, b 2, b} 3, b 4) = (0.246, 0.615, -0.25, 1)
It takes such a value.

  Also, η is a spatial frequency [cycle / degree] representing the number of light-dark pairs within the unit viewing angle. Note that η is a one-dimensional spatial frequency.

At this time, between η and the spatial frequency index (either u _x or u _y ),
η (u, r) = θ (r, H) u / 2MN Equation (27)
There is a relationship.

In the case of u = u _x, a M = M _x, the case of u = u _y, is M = M _y. Θ (r, H) is an angle [degrees / pixel] per pixel when an image having a vertical width H is observed at a viewing distance rH.
θ (r, H) = (1 / H) × arctan (1 / r) × (180 / π)
... Formula (28)
Is given by the expression

Further, r is adaptively changed according to the magnitude of the shift amount d. For example, the setting method is as follows. Here, A is a threshold value, and r ₁ > r ₂ .

By substituting Equation (27) and Equation (28) into Equation (26) and expressing the contrast sensitivity function g (η) as a function of u and d, ^ g (u, d)
^ G (u, d) = g (η (u, r (d)) (30)
Is a visual sensitivity function.

  In accordance with the configuration described above, according to the present invention, it is possible to introduce a measure of coding distortion that appropriately evaluates subjective image quality including block distortion, thereby reducing the cost used to select coding parameters. By realizing a function that reflects subjective image quality including block distortion as a function, it is possible to realize highly efficient coding and to reduce the amount of codes.

  Next, the configuration of the moving picture coding apparatus constructed according to the present invention will be described.

The moving image encoding apparatus of the present invention encodes a moving image by performing information compression by transform encoding and quantization on an image signal or a prediction error signal obtained by intra-frame prediction and inter-frame prediction. (A) is defined in association with an analysis target block composed of a plurality of blocks to be converted matrix, and a base image of the conversion matrix is arranged in one block, Storage means for storing spatial frequency components calculated for the number of corrected base images corresponding to the number of blocks configured by embedding zero values in the block, and (b) showing temporal movement of the image signal of the analysis target block (C) reading out the spatial frequency component of the modified base image from the storage means, and estimating the spatial frequency index of the spatial frequency component and the estimation means The visual sensitivity value assigned to the spatial frequency component is calculated based on the determined shift amount, the weighting means for weighting the spatial frequency component, (d) the spatial frequency component weighted by the weighting means, A calculation means for calculating the importance of each base component of the prediction error signal based on the spatial frequency component without weighting and the transform coefficient of the block constituting the analysis target block; and (e) the importance calculated by the calculation means Determining means for determining an encoding parameter by evaluating an encoding cost using an amount of encoding distortion weighted using a degree , and (f) the calculating means includes a square sum of transform coefficients, Find the product of the squared norm sum of the weighted spatial frequency components and the product of the square sum of the transform coefficients and the squared norm sum of the unweighted spatial frequency components. , Configured to calculate the importance according quotient of the two multiplied values.

  Here, the storage means for storing the spatial frequency component calculated for the corrected base image is provided because the spatial frequency component of the corrected base image can be obtained regardless of the image to be encoded. This is because it is not necessary to calculate each time.

  When adopting this configuration, the weighting means calculates the visual sensitivity value in the horizontal direction based on the horizontal spatial frequency index and the horizontal component of the displacement estimated by the estimating means, and the vertical spatial frequency index and The visual sensitivity value assigned to the spatial frequency component of the corrected base image read from the storage means may be calculated by calculating the visual sensitivity value in the vertical direction based on the vertical component of the shift amount estimated by the estimation means. .

  The reason why the spatial frequency component and the direction dependency of the motion are considered in this way is because the detection mechanism of the visual system in the spatio-temporal region depends on the spatial edge direction and the direction of motion. For example, when the vertical stripe moves, only the horizontal component of the movement can be recognized as the movement. That is, when evaluating the visual sensitivity with respect to the spatio-temporal frequency, it is necessary to consider the edge direction and the transition direction of the spatial frequency component. In consideration of this, the weighting means may calculate the visual sensitivity value in consideration of the spatial frequency component and the direction dependency of motion.

  The moving image encoding method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program, and this computer program is provided by being recorded on a suitable computer-readable recording medium. Alternatively, the present invention is realized by being provided via a network, installed when the present invention is carried out, and operating on a control means such as a CPU.

  Thus, in the present invention, a moving image is encoded by performing information compression by transform coding and quantization on an image signal or a prediction error signal obtained by intra-frame prediction and inter-frame prediction. In moving picture coding, based on Lagrangian cost functions consisting of distortion, coding quantity, and undetermined multiplier, motion compensation block size, inter prediction mode, quantization parameter in moving picture coding, and still picture coding When determining coding parameters such as intra prediction mode and quantization parameter, the spatial frequency component in the block and the spatial frequency component related to discontinuity between adjacent blocks are measured, and the obtained spatial frequency component and motion are also measured. Estimate temporal frequency components based on the motion vector obtained by estimation, and The cost function is calculated based on the visual sensitivity function, and the cost function is set as a weighted sum with the code amount using the distortion amount obtained as a square error weighted for each frequency based on the importance. The encoding parameter is selected by selecting a mode that minimizes.

  In the present invention, for a block-based coding scheme that combines inter-frame prediction by motion compensation and orthogonal transform, the spatio-temporal frequency is estimated by considering the direction of the spatial frequency component and the direction of the shift amount, We introduce a measure of coding distortion that appropriately evaluates subjective image quality including distortion.

  As a result, according to the present invention, it is possible to realize a cost function used for selecting an encoding parameter that reflects subjective image quality including block distortion, thereby realizing highly efficient encoding. At the same time, the amount of code can be reduced.

  Hereinafter, the present invention will be described in detail according to embodiments.

  FIG. 2 illustrates a device configuration of a moving image encoding device 1 to which the present invention is applied.

  The moving image encoding apparatus 1 to which the present invention is applied is described in H.264. H.264 is used to perform encoding of a moving image. As shown in this figure, an encoding parameter selection unit 10 that selects an encoding parameter used for encoding, and a code selected by the encoding parameter selection unit 10 And an encoding unit 11 that performs encoding of a moving image using the encoding parameters.

  3 to 5 show examples of flowcharts executed by the encoding parameter selection unit 10 to realize the present invention. Here, in this flowchart, it is assumed that the encoding parameter selection unit 10 selects an optimal prediction mode as an encoding parameter.

  When selecting the optimal prediction mode used for encoding according to the present invention, the encoding parameter selection unit 10 firstly, as shown in the flowchart of FIG. Set the prediction mode).

  Subsequently, in step S102, an initial cost indicating a large value is stored in a register for storing a minimum cost (hereinafter also referred to as a minimum cost register), and a register for storing an optimal prediction mode (hereinafter, referred to as a minimum cost register). These registers are initialized by storing meaningless values for (sometimes referred to as optimal mode registers).

Subsequently, in step S103, the shift amount ((d _x, d _y ) described above) is estimated, and the prediction error of each candidate vector is stored in a table. The method for estimating the amount of displacement is given from the outside. For example, H.M. It is also possible to use a motion vector calculated by the H.264 reference software JM as an estimated value of the shift amount used below.

  Subsequently, in step S104, the set prediction mode of the processing target, the prediction vector based on the prediction mode, the quantization parameter, the encoding target frame signal, and the reference frame signal are input, and encoding is performed using the prediction mode. The code amount in the case is calculated. The specific calculation method is as follows. According to the H.264 reference software JM method.

  Subsequently, in step S105, first, a weight considering the spatiotemporal visual sensitivity is determined based on the amount of transition estimated in step S103, and then, the prediction mode to be processed and the prediction based on the prediction mode are determined. Calculates the weighted distortion amount when encoding using the prediction mode based on the input signal and the determined weight based on the vector, quantization parameter, encoding target frame signal, and reference frame signal. To do. A specific calculation method will be described with reference to the flowcharts of FIGS.

  Subsequently, in step S106, the prediction mode to be processed and the quantization parameter that have been set are input, and an undetermined multiplier for encoding using the prediction mode is calculated.

  Subsequently, in step S107, based on the code amount calculated in step S104, the weighted distortion amount calculated in step S105, and the undetermined multiplier calculated in step S106, the RD cost represented by Expression (11) is used. Is calculated.

  Subsequently, in step S108, the calculated RD cost is compared with the cost stored in the minimum cost register, and the calculated RD cost is smaller than the cost stored in the minimum cost register. When judging this, the process proceeds to step S109, where the calculated RD cost is stored in the minimum cost register, and in the subsequent step S110, the set identification information of the prediction mode to be processed is stored in the optimum mode register. .

  On the other hand, when it is determined in step S108 that the calculated RD cost is higher than the cost stored in the minimum cost register, the processes in steps S109 and 110 are omitted.

  Subsequently, in step S111, it is determined whether or not processing has been performed for all prediction modes, and when it is determined that processing has not been performed for all prediction modes, the process proceeds to step S112, and unprocessed according to a predetermined order. After the prediction mode to be processed is updated by selecting one prediction mode from among the prediction modes, the process returns to step S104.

  On the other hand, when it is determined in step S111 that all prediction modes have been processed, the process proceeds to step S113, and the prediction mode stored in the optimum mode register is output to the encoding unit 11 as the optimum prediction mode. The process ends.

  Next, the calculation processing of the weighted distortion amount executed in step S105 of the flowchart of FIG. 3 will be described according to the flowchart of FIG.

  The calculation processing of the weighted distortion amount is executed by calculating the calculation formula of Formula (12). In the following flowchart, for the sake of convenience of explanation, the processing using a one-dimensional index i (index indicating k, l) is described.

  When entering the process of step S105 in the flowchart of FIG. 3, the encoding parameter selection unit 10 first normalizes the transform coefficient in step S201 as shown in the flowchart of FIG.

Subsequently, in step S202, a decoded value of each transform coefficient indicated by ^ _{Cm, qs} ⁽ⁱ⁾ [k, l] (s = Y, U, V) in the equation (12) is obtained.

Subsequently, in step S203, the shift amount (the above-described (d _x, d _y )) estimated in step S103 of the flowchart of FIG. 3 is read. This amount of change is necessary for executing the flowchart of FIG. 5, and is read at this stage.

Subsequently, in step S204, the sensitivity coefficient W _{k, l} ^s (s = Y, U, V) described in the equation (12) is set. A specific setting method will be described with reference to the flowchart of FIG. The sensitivity coefficient set at this time corresponds to the weight described in step S105 of the flowchart of FIG.

  In step S205, a counter i designating i is initialized to 0, and a register S is initialized to 0.

Subsequently, in step S206, | C _m ^{s (i)} [k, l] − ^ C _{m, q} ^{s (i)} [k, l] | ² described in equation (12)
However, s = Y, U, V
The amount of coding distortion for the i-th component of the transform coefficient is calculated according to the following equation.

Subsequently, in step S207, the transform coefficient is obtained by multiplying the distortion amount of the coding distortion calculated in step S206 by the sensitivity coefficient W _{k, l} ^s (s = Y, U, V) set in step S204. The weighted distortion amount for the i-th component is calculated.

  Subsequently, in step S208, the multiplication value calculated in step S207 is added to the register S.

  Subsequently, in step S209, it is determined whether or not all components of the transform coefficient have been processed, and when it is determined that all components of the transform coefficient have not been processed, the process proceeds to step S210 and all components of the transform coefficient are processed. 1 is added to the counter i to return to step S206.

  On the other hand, when it is determined in step S209 that all components of the transform coefficient have been processed, it is determined that the processing for calculating the weighted distortion amount has been completed, and the process proceeds to step S211 where step S105 in the flowchart of FIG. As a calculation result, the weighted distortion amount stored in the register S is output, and the process ends.

Next, the sensitivity coefficient W _{k, l} ^s setting process executed in step S204 of the flowchart of FIG. 4 will be described with reference to the flowchart of FIG.

The setting process of the sensitivity coefficient W _{k, l} ^s is executed by calculating the calculation formula of Expression (20).

  When the encoding parameter selection unit 10 enters the process of step S204 in the flowchart of FIG. 4, first, as shown in the flowchart of FIG. "Process 4" is executed to perform a process of calculating and outputting.

  Since the DFT coefficient of the corrected base image calculated at this time is reused, it is stored in a table (a corrected base image DFT coefficient storage unit 2102 shown in FIG. 7 described later).

  Subsequently, in step S302, the DFT coefficient and DCT coefficient of the modified base image obtained in “Process 4” are input, and the prediction error power (the denominator of Expression (20)) for the analysis target block is calculated and output. "Process 2" is executed to perform the process.

  Subsequently, in step S303, the DFT coefficient of the modified base image obtained in “Process 4”, the DCT coefficient, the motion vector of the analysis target block, and the contrast function are input, and the prediction error power (formula “Process 3” is executed to perform the process of calculating and outputting the numerator (20).

  Subsequently, in step S304, the prediction error power obtained in “Process 2” (the denominator of Expression (20)) and the prediction error power obtained in “Process 3” (the numerator of Expression (20)) are input. “Process 1” is executed to perform a process of calculating and outputting a sensitivity coefficient (weight) for the distortion amount in the cost function of mode selection.

  Next, detailed flowcharts of “Process 1”, “Process 2”, “Process 3”, and “Process 4” will be described. In the flowchart of “Process 1” described below, for the sake of convenience of explanation, the processing up to step S107 (cost function calculation processing) of the flowchart of FIG. 3 is executed based on the set sensitivity coefficient. .

[1] “Process 4”
Input: k-th, l-th base image (k, l = 0,..., N−1)
Output: DFT coefficient for modified base image
(1) Reading indexes i _{x and} i _y indicating position information (2) l = 0
(3) k = 0
(4) Read the kth, l-th base image f _{k, l} ^{(ix, iy)} (5) For the base image of (4), zero padding according to the position information of (1) gives M _x N × M _x N images ~ f _{k, l} ^{(ix, iy)} are generated. The image obtained here is called the modified base image. The specific generation method is the equation (15).
(6) DFT is performed on the modified base image of (5), and the distribution of frequency components in the modified base image is calculated. The specific calculation method is formula (18).
(7) k = k + 1
(8) If k = N, go to (9), otherwise go to (4) (9) l = l + 1
(10) If l = N, end, otherwise go to (3) According to this flowchart, the encoding parameter selection unit 10 receives the base image f _{k, l} ^{(ix, iy)} as input in “Process 4”. Then, the process of calculating the DFT coefficient F _{k, l} ^{(ix, iy)} ( ^ux _, u _y ) of the modified base image is performed.

The DFT coefficients F _{k, l} ^{(ix, iy)} ( ^ux _, u _y ) of the corrected base image calculated in this way are stored in a table (a corrected base image DFT coefficient storage unit 2102 in FIG. 7 described later). Will be.

[2] “Process 2”
Input: DFT coefficient of modified base image (k, l = 0,..., N−1,
i _x = 0, ..., M _x -1, i _y = 0, ..., M _y -1)
: DCT coefficient Output: Prediction error power E [k, l] for the analysis target block (denominator of equation (20))
Where E [k, l] is an array process:
(1) l = 0
(2) k = 0
(3) E [k, l] = 0
(4) i _y = 0
(5) i _x = 0
(6) S = 0
(7) u _y = 0
(8) u _x = 0
(9) Read the u _x, u _y components of the DFT coefficients of the kth, lth modified base image of the position index i _x, i _y from the above table. (10) Complex square norm F _{k, l} read immediately before _{^{(ix, iy) (u x}} , u y) 2 to calculate the (11) S = S + F k, l (ix, iy) (u x, u y) 2
(12) u _x = u _x +1
(13) If u _x = NM _x , go to the next step, otherwise go to (9) (14) u _y = u _y +1
(15) If u _y = NM _y , go to the next, otherwise go to (8) (16) Read the DCT coefficient C ^{(ix, iy)} [k, l] of the kth and ^lth basis (17) E [K, l] = E [k, l] + C ^{(ix, iy)} [k, l] ² S
(18) i _x = i _x +1
(19) If i _x = M _x , go to the next step, otherwise go to (6) (20) i _y = i _y +1
(21) i _y = M _y if to the next, if not to (5) (22) k = k + 1
(23) If k = N, go to the next, otherwise go to (3) (24) l = l + 1
(25) If l = N, end; otherwise, go to (2) According to this flowchart, the encoding parameter selection unit 10 performs the DFT coefficient F of the modified base image obtained in “Process 4” in “Process 2”. _{k, l} ^{(ix, iy)} ( ^ux _, u _y ) and DCT coefficient C ^{(ix, iy)} [k, l] as inputs, and prediction error power E [k, l] (formula The process of calculating the denominator of (20) is performed.

[3] “Process 3”
Input: DFT coefficient of modified base image (k, l = 0,..., N−1,
i _x = 0, ..., M _x -1, i _y = 0, ..., M _y -1)
: DCT coefficient: Motion vector (d _{x, dy} ) of the analysis target block
: Contrast sensitivity function ^ g (η, ω)
Output: Prediction error power ^ E (d _{x, dy} ) [k, l] (numerator of equation (20)) for the analysis target block
Here, ^ E (d _x, d _y ) [k, l] is an array process:
(0) The motion vector (d _{x, dy} ) is read from the transition amount storage unit 208. (1) l = 0
(2) k = 0
(3) ^ E (d _x, d _y ) [k, l] = 0
(4) i _y = 0
(5) i _x = 0
(6) ^ S = 0
(7) u _y = 0
(8) u _x = 0
(9) Read the u _x, u _y components of the DFT coefficients of the kth, lth modified base image of the position index i _x, i _y from the above table. (10) Complex square norm F _{k, l} read immediately before _{^{(ix, iy) (u x}} , u y) 2 to calculate the _{(11) ^ g (η x} , d x) and _{^ g (η y, d y} ) is calculated. Concrete calculation is obtained, for example, by equation (21) (12) ^ S = ^ S + F k, l (ix, iy) (u x, u y) 2 ^ g (η x, d x) 2 ^ g ( η _y, d _y ) ²
(13) u _x = u _x +1
(14) If u _x = NM _x , go to the next step, otherwise go to (9) (15) u _y = u _y +1
(16) If u _y = NM _y , go to the next step, otherwise go to (8) (17) Read the DCT coefficient C ^{(ix, iy)} [k, l] of the kth and lth basis (18) ^ E (Dx _{, dy} ) [k, l] = ^ E (dx _{, dy} ) [k, l] + C ^{(ix, iy)} [k, l] ² ^ S
(19) i _x = i _x +1
(20) If i _x = M _x , go to the next step, otherwise go to (6) (21) i _y = i _y +1
(22) If i _y = M _y , go to the next step, otherwise go to (5) (23) k = k + 1
(24) If k = N, go to the next step, otherwise go to (3) (25) l = l + 1
(26) If l = N, end; otherwise, go to (2). According to this flowchart, the encoding parameter selection unit 10 in “Process 3”, the DFT coefficient F of the modified base image obtained in “Process 4”. ^{_{k, l (ix, iy)}} (u x, u y) and, DCT coefficients C and ^{(ix, iy) [k,} l], the motion vector of the analysis target block (d _x, d _y) and the contrast sensitivity function The process of calculating the prediction error power ^ E (d _{x, dy} ) [k, l] (numerator of equation (20)) for the analysis target block is performed with ^ g (η, ω) as an input. .

[4] “Process 1”
Input: E [k, l] obtained in “Process 2” (k, l = 0,..., N−1)
: ^ E (d _x, d _y ) [k, l] (k, l = 0,..., N−1) obtained in “Process 3”
Output: Weight for the amount of distortion in the cost function of mode selection (sensitivity coefficient)
processing:
(1) l = 0, D ′ = 0
(2) k = 0
(3) D = 0
(4) i _y = 0
(5) i _x = 0
(6) position index i _x, reads the k of the DCT in the reference block of the i _y, square value e of the coefficients of l component ^{[k, l] (ix,} iy) read (7) E [k, l] and (8) Read {circumflex over (E) _} (d _x, d _y ) [k, l] (9) Calculate the weighting coefficient W (d _x, d _y ) [k, l] = ^ E (d _x, d _y ) [K, l] / E [k, l]
(10) D = D + e [k, l] ^{(ix, iy)}
(11) i _x = i _x +1
(12) If i _x = M _x , go to the next step, otherwise go to (6) (13) i _y = i _y +1
(14) i _y = M _y if to the next, otherwise (5) to (15) D '= D' + D * W [k, l]
(16) k = k + 1
(17) If k = N, go to the next, otherwise go to (3) (18) l = l + 1
(19) l = N if finished, calculates otherwise (2) to (20) position index i _x, the estimated value of the code amount in the reference block of the i _y alpha ^{(ix, iy),} the analysis target block (21) The cost function J is calculated. (21) Calculate the estimated value A = Σ _{ix, iy} α ^{(ix, iy)} . J = D '+ λA
According to this flowchart, the encoding parameter selection unit 10 in “Process 1”, the prediction error power E [k, l] (the denominator of Expression (20)) for the analysis target block obtained in “Process 2”, and “Process As an input, the prediction error power ^ E (d _{x, dy} ) [k, l] (numerator of equation (20)) for the analysis target block obtained in consideration of the visual sensitivity of space-time in 3 ″,
W (dx _{, dy} ) [k, l] = ^ E (dx _{, dy} ) [k, l] / E [k, l]
According to the calculation formula (formula (20)), the weight W (d _x, d _y ) [k, l] (the aforementioned sensitivity coefficient W _{k, l} ^s (d _{x ,} d _y )) is calculated.

  In this way, the encoding parameter selection unit 10 executes the flowcharts of FIGS. 3 to 5 to select a prediction mode used for encoding a moving image, as a cost function used for selecting a prediction mode. Thus, it is possible to realize what reflects subjective image quality including block distortion, and thus the encoding unit 11 can realize high-efficiency encoding and reduce the amount of codes.

  6 and 7 illustrate the device configuration of the encoding parameter selection unit 10 configured to execute the flowcharts of FIGS. 3 to 5.

  Next, the apparatus configuration of the encoding parameter selection unit 10 will be described with reference to FIGS.

  In order to execute the flowchart of FIG. 3, the encoding parameter selection unit 10, as shown in FIG. 6, (1) a transition amount storage unit 101 that stores an estimation value of a transition amount of a block to be encoded; 2) a prediction vector calculation unit 102 that calculates a prediction vector of a block to be encoded; (3) a prediction vector storage unit 103 that stores a prediction vector calculated by the prediction vector calculation unit 102; and (4) an initial prediction mode. An initial mode setting unit 104 for setting a value; (5) a mode setting unit 105 for setting a prediction mode to be processed; and (6) a prediction mode set by the initial mode setting unit 104 or the mode setting unit 105. A mode storage unit 106; (7) a code amount calculation unit 107 that calculates a code amount when encoding is performed in the prediction mode stored in the mode storage unit 106; and (8) a code amount calculation. A code amount storage unit 108 that stores the calculated code amount 107, (9) a weighted distortion amount calculation unit 109 that calculates a weighted distortion amount when encoded in the prediction mode stored in the mode storage unit 106; (10) A weighted distortion amount storage unit 110 that stores the weighted distortion amount calculated by the weighted distortion amount calculation unit 109, and (11) an undetermined multiplier when encoded in the prediction mode stored in the mode storage unit 106. Based on the undetermined multiplier calculation unit 111 to calculate, (12) the undetermined multiplier storage unit 112 that stores the undetermined multiplier calculated by the undetermined multiplier calculation unit 111, and (13) the code amount, the weighted distortion amount, and the undetermined multiplier, A cost calculation unit 113 for calculating an encoding cost when encoding is performed in a prediction mode stored in the mode storage unit 106; and (14) an encoding cost calculated by the cost calculation unit 113 is stored. The cost calculation unit 113 calculates, referring to the cost storage unit 114, (15) the minimum cost storage unit 115 that stores the minimum cost obtained so far, and (16) the minimum cost stored in the minimum cost storage unit 115. A minimum cost determination unit 116 that determines whether the encoded cost is the minimum cost obtained so far, (17) an optimal mode storage unit 117 that stores an optimal prediction mode, and (18) a minimum cost determination. An optimal mode update unit 118 that updates the prediction mode stored in the optimal mode storage unit 117 according to the prediction mode stored in the mode storage unit 106 when the unit 116 determines that the cost is the minimum cost; and (19) minimum cost determination. When the unit 116 determines that the cost is not the minimum cost, it immediately determines whether all the prediction modes have been processed, while the minimum cost determination unit 1 When it is determined that 16 is the minimum cost, it is determined whether or not all of the prediction modes have been processed in response to an instruction from the optimum mode update unit 118. A final mode determination unit 119 that instructs the setting unit 105 to set the next prediction mode; and (20) when the final mode determination unit 119 determines that the final prediction mode is set, the optimum mode storage unit 117 And an optimal mode output unit 120 that outputs the stored prediction mode as the optimal prediction mode.

  The encoding parameter selection unit 10 executes the flowchart of FIG. 3 according to this apparatus configuration.

  FIG. 7 illustrates a device configuration of the weighted distortion amount calculation unit 109 that executes the flowcharts of FIGS. 4 and 5.

In order to execute the flowcharts of FIGS. 4 and 5, the weighted distortion amount calculation unit 109 includes (1) a conversion coefficient normalization unit 201 that normalizes conversion coefficients, and (2) conversion. A normalized transform coefficient storage unit 202 that stores the normalized transform coefficient converted by the coefficient normalization unit 201; (3) a transform coefficient decoding unit 203 that decodes the transform coefficient; and (4) a decryption performed by the transform coefficient decoding unit 203. Based on the decoded transform coefficient storage unit 204 that stores the decoded transform coefficient, (5) the transform coefficient stored in the normalized transform coefficient storage unit 202, and the transform coefficient stored in the decoded transform coefficient storage unit 204, A distortion amount calculation unit 205 that calculates the distortion amount of the encoded distortion of the i-th component (i is updated); and (6) a distortion amount storage unit 206 that stores the distortion amount calculated by the distortion amount calculation unit 205. (7) Currently processing A conversion coefficient index storage unit 207 that stores the i-th component value i of the conversion coefficient, (8) a transition amount storage unit 208 that stores an estimated value of the transition amount of the analysis target block, and (9) a base image. A base image storage unit 209, (10) a shift amount stored in the shift amount storage unit 208, a base image stored in the base image storage unit 209, a conversion coefficient stored in the normalized conversion coefficient storage unit 202, and a contrast A sensitivity coefficient calculation unit 210 that calculates the sensitivity coefficient W _{k, l} based on the sensitivity function; and (11) a sensitivity coefficient storage unit 211 that stores the sensitivity coefficient W _{k, l} calculated by the sensitivity coefficient calculation unit 210; (12) a sensitivity coefficient multiplication unit 212 that calculates a weighting distortion amount by multiplying the distortion amount stored in the distortion amount storage unit 206 by the sensitivity coefficient W _{k, l} stored in the sensitivity coefficient storage unit 211; (13) Sensitivity staff A distortion amount storage unit 213 that stores the distortion amount calculated by the multiplication unit 212, and (14) a distortion amount that calculates a weighted distortion amount by calculating a sum of distortion amounts sequentially stored in the distortion amount storage unit 213. A sum calculation unit 214.

  Then, the sensitivity coefficient calculation unit 210 stores (1) the process 4 execution unit 2101 that executes the process of “process 4” described above, and (2) the DFT coefficient of the corrected base image calculated by the process 4 execution unit 2101. The modified base image DFT coefficient storage unit 2102 to be processed, (3) the modified base image DFT coefficient stored in the modified base image DFT coefficient storage unit 2102 and the transform coefficient stored in the normalized transform coefficient storage unit 202 are input. The processing 2 execution unit 2103 that executes the processing of “processing 2”, (4) the modified base image DFT coefficient stored in the modified base image DFT coefficient storage unit 2102, and the transform coefficient stored in the normalized transform coefficient storage unit 202 And a process 3 execution unit 2104 that executes the process of “process 3” described above by using the shift amount stored in the shift amount storage unit 208 and the contrast function as inputs. The process related to the calculation of the sensitivity coefficient in the process of “process 1” described above is input with the prediction error power calculated by the process 2 execution unit 2103 and the prediction error power calculated by the process 3 execution unit 2104 as input. And a process 1 execution unit 2105 to be executed.

  The weighted distortion amount calculation unit 109 executes the flowcharts of FIGS. 4 and 5 according to this apparatus configuration.

  In the embodiment described above, the encoding parameter selection unit 10 has been described as selecting the optimum prediction mode as the encoding parameter. However, the present invention is also applicable to the case of selecting an encoding parameter other than the prediction mode. It can be applied as it is.

  For example, when the encoding parameter selection unit 10 selects an optimal quantization parameter as the encoding parameter, the encoding parameter selection unit 10 executes the flowchart of FIG. 8 instead of the flowchart of FIG. Become.

  That is, when determining the optimal quantization parameter used for encoding the macroblock to be encoded, the encoding parameter selection unit 10 first determines the quantization parameter in step S401 as shown in the flowchart of FIG. The initial value of the quantization parameter (quantization parameter as the initial value) is set.

  Subsequently, in step S402, an initial cost indicating a large value is stored with respect to a register (minimum cost register) for storing a minimum cost, and a register for storing an optimal quantization parameter (hereinafter referred to as an optimal quantization parameter register). These registers are initialized by storing meaningless values.

Subsequently, in step S403, the shift amount ((d _x, d _y ) described above) is estimated, and the prediction error of each candidate vector is stored in a table. The method for estimating the amount of displacement is given from the outside. For example, H.M. It is also possible to use a motion vector calculated by the H.264 reference software JM as an estimated value of the shift amount used below.

  Subsequently, in step S404, the set quantization parameter, the prediction mode used for encoding, the prediction vector based on the prediction mode, the encoding target frame signal, and the reference frame signal are input, and the encoding is performed using the quantization parameter. The amount of code for conversion is calculated.

  Subsequently, in step S405, first, a weight considering the spatiotemporal visual sensitivity is determined based on the amount of transition estimated in step S403, and then the set quantization parameter, the prediction mode used for encoding, Weighted distortion amount when encoding using the prediction parameter, the encoding target frame signal, and the reference frame signal in the prediction mode, and encoding using the quantization parameter based on the input signal and the determined weight Is calculated. A specific calculation method is as described in the flowcharts of FIGS.

  Subsequently, in step S406, the set quantization parameter and the prediction mode used for encoding are input, and an undetermined multiplier when encoding using the quantization parameter is calculated.

  Subsequently, in step S407, based on the code amount calculated in step S404, the weighted distortion amount calculated in step S405, and the undetermined multiplier calculated in step S406, the RD cost represented by Expression (11) is used. Is calculated.

  Subsequently, in step S408, the calculated RD cost is compared with the cost stored in the minimum cost register, and the calculated RD cost is smaller than the cost stored in the minimum cost register. When this is determined, the process proceeds to step S409, where the calculated RD cost is stored in the minimum cost register, and in step S410, the set quantization parameter is stored in the optimum quantization parameter register. On the other hand, when it is determined in step S408 that the calculated RD cost is higher than the cost stored in the minimum cost register, the processes in steps S409 and 410 are omitted.

  Subsequently, in step S411, it is determined whether or not all the quantization parameters have been processed, and when it is determined that all the quantization parameters have not been processed, the process proceeds to step S412 in accordance with a predetermined order. After the quantization parameter to be processed is updated by selecting one quantization parameter from the unprocessed quantization parameters, the process returns to step S404.

  On the other hand, when it is determined in step S411 that all the quantization parameters have been processed, the process proceeds to step S413, and the quantization parameter stored in the optimum quantization parameter register is output as the optimum quantization parameter. The process ends.

  FIG. 9 illustrates a device configuration of the encoding parameter selection unit 10 configured to execute the flowchart of FIG. Here, the same components as those shown in FIG. 7 are indicated by the same symbols.

  As shown in FIG. 9, the encoding parameter selection unit 10 includes an initial quantization parameter setting unit 304 in place of the above-described initial mode setting unit 104 in order to execute the flowchart of FIG. A quantization parameter setting unit 305 instead of the unit 105, a quantization parameter storage unit 306 instead of the mode storage unit 106, and an optimal quantization parameter storage unit 317 instead of the optimal mode storage unit 117. An optimal quantization parameter update unit 318 instead of the optimal mode update unit 118 described above, a final quantization parameter determination unit 319 instead of the final mode determination unit 119, and the optimal mode output unit 120 described above. Instead, an optimum quantization parameter output unit 320 is provided.

  Finally, an experiment conducted to verify the effectiveness of the present invention will be described.

  This experiment was performed by implementing the present invention in the reference software JSVM (version 8.0. [3]) and comparing it with an unmodified JSVM.

The following table shows the experimental conditions. The encoding target sequence has a size of 352 × 288 [pixels] and a frame rate of 30 [fps]. The encoding process was performed on the first 120 frames. The parameters in equation (30) were r ₁ = 8, r ₂ = 6, A = 5. Further, the parameter that gives the size of the reference block is N = 4, and the parameter that gives the size of the analysis target block is M _x = M _y = 4.

The following table shows the comparison results of the code amount. In any sequence and QP value, it can be confirmed that the code amount is reduced by the present invention. It has been confirmed that there is no subjective difference in image quality between the decoded images of both methods. Further, in order to evaluate the relative code amount reduction rate of the present invention with respect to JSVM, the code amount of JSVM and the code amount of the present invention are respectively represented as R _JSVM and R _Ours .
{(R _Ours -R _JSVM ) / R _JSVM } × 100%
The code amount reduction rate represented by the formula is shown in parentheses in the third column of the following table. As a result, it has been confirmed that the present invention achieves an average code amount reduction of 5.3% with respect to JSVM.

  The following table shows the ratio of the encoding mode. Here, the SKIP column indicates the skip mode ratio, the INTER column indicates the inter prediction ratio excluding the skip mode, and the INTRA column indicates the selected ratio of all intra prediction modes. Is shown. As shown in this table, it can be seen that the reduction of the bit rate according to the present invention is realized by selecting many skip modes with a small amount of generated codes.

Furthermore, the evaluation result about block distortion is shown. The decoded pixel value at the position (x, y) of each frame (X × Y [pixel]) is S (x, y), and the inter-pixel difference values in the horizontal and vertical directions are respectively δ _h (x, y) = S. (X + 1, y) -S (x, y)
δ _v (x, y) = S (x, y + 1) −S (x, y)
And At this time,
Δ = {Σ ₁ Σ ₂ | δ _h (Ni _x, i _y ) |} / {2Y (X / N−1)}
+ {Σ ₃ Σ ₄ | δ _v (ix _, Ni _y ) |} / {2X (Y / N−1)}
Where Σ ₁ is the sum of i _x = 1 to (X / N−1)
Σ ₂ is the sum of i _y = 0 to (Y−1)
Σ ₃ is the sum of i _y = 1 to (Y / N−1)
Σ ₄ evaluates the discontinuity between adjacent blocks by using the average value Δ of the inter-pixel difference value at the block boundary expressed by the formula of the sum of i _x = 0 to (X−1).

The table below shows the total frame average of Δ for JSVM and the present invention. In addition, the value in the second column and the value in the third column of each row are set as Δ _JSVM and Δ _Ours respectively.
{(Δ _Ours −Δ _JSVM ) / Δ _JSVM } × 100%
The block distortion reduction rate represented by the formula is shown in parentheses in the third column of the following table. As a result, it was confirmed that the present invention achieved an average 1.2% block distortion reduction with respect to JSVM.

  From this experiment, it was confirmed that according to the present invention, the code amount can be reduced with respect to JSVM, and further, block distortion can be reduced. In the present invention, frequency analysis is performed in consideration of the dependency between adjacent blocks. For this reason, the discontinuity between adjacent blocks is analyzed as a frequency component to which the contrast sensitivity function gives a large weight as horizontal and vertical edges. That is, a large sensitivity coefficient value is set. Since a distortion measure weighted by this sensitivity coefficient is used in the cost function, distortion of components that cause discontinuity between blocks is avoided, and it is considered that block distortion is reduced as a result.

  The present invention can be applied to a case where a moving image is encoded by performing information compression by transform coding and quantization on a prediction error signal obtained by intra-frame prediction or inter-frame prediction. By applying the invention, it becomes possible to realize a cost function used for selecting an encoding parameter that reflects subjective image quality including block distortion, thereby realizing highly efficient encoding. The amount of code can be reduced.

It is explanatory drawing of a correction base image. It is an apparatus block diagram of the moving image encoding apparatus with which this invention is applied. It is a flowchart which an encoding parameter selection part performs. It is a flowchart which an encoding parameter selection part performs. It is a flowchart which an encoding parameter selection part performs. It is an apparatus block diagram of an encoding parameter selection part. It is an apparatus block diagram of an encoding parameter selection part. It is a flowchart which an encoding parameter selection part performs. It is an apparatus block diagram of an encoding parameter selection part. It is explanatory drawing of the discontinuity between blocks.

Explanation of symbols

DESCRIPTION OF SYMBOLS 201 Transformation coefficient normalization part 202 Normalization transformation coefficient memory | storage part 203 Transformation coefficient decoding part 204 Decoding transformation coefficient memory | storage part 205 Distortion amount calculation part 206 Distortion amount memory | storage part 207 Transformation coefficient index memory | storage part 208 Transition amount memory | storage part 209 Base image memory | storage part 210 sensitivity coefficient calculation unit 211 sensitivity coefficient storage unit 212 sensitivity coefficient multiplication unit 213 distortion amount storage unit 214 distortion amount sum calculation unit 2101 processing 4 execution unit 2102 modified base image DFT coefficient storage unit 2103 processing 2 execution unit 2104 processing 3 execution unit 2105 Process 1 execution part

Claims (6)

A moving picture coding method for coding a moving picture by performing information compression by transform coding and quantization on an image signal or a prediction error signal obtained by intra-frame prediction and inter-frame prediction. ,
It is defined by associating with the analysis target block consisting of multiple blocks that are the target of the transformation matrix, and is configured by placing the base image of the transformation matrix in one block and embedding zero values in other blocks A process of reading the spatial frequency components of the corrected base image from the storage means for storing the spatial frequency components calculated for the corrected base images for the number of blocks
Estimating a shift amount indicating temporal movement of the image signal of the analysis target block;
Calculating a visual sensitivity value assigned to the spatial frequency component based on the spatial frequency index of the read spatial frequency component and the shift amount, and weighting the spatial frequency component;
Calculating importance for each base component of the prediction error signal based on the weighted spatial frequency component, the non-weighted spatial frequency component, and the transform coefficient of the block constituting the analysis target block; ,
Determining a coding parameter by evaluating a coding cost using a coding distortion weighted using the importance , and
In the process of calculating, a multiplication value of the square sum of the transform coefficients and the square norm sum of the weighted spatial frequency components, and a square sum of the square sum of the transform coefficients and the square frequency norm sum of the non-weighted spatial frequency components. Obtaining a multiplication value and calculating the importance according to a division value of the two multiplication values ;
A moving image encoding method as a feature. The moving image encoding method according to claim 1,
In the weighting process, a visual sensitivity value in the horizontal direction is calculated based on the horizontal spatial frequency index and the horizontal component of the shift amount, and based on the vertical spatial frequency index and the vertical component of the shift amount. Calculating a visual sensitivity value assigned to the read spatial frequency component by calculating a vertical visual sensitivity value,
A moving image encoding method as a feature. A moving image encoding apparatus that encodes a moving image by performing information compression by transform coding and quantization on an image signal or a prediction error signal obtained by intra-frame prediction and inter-frame prediction. ,
It is defined by associating with the analysis target block consisting of multiple blocks that are the target of the transformation matrix, and is configured by placing the base image of the transformation matrix in one block and embedding zero values in other blocks Storage means for storing spatial frequency components calculated for the corrected base images for the number of blocks
Estimating means for estimating a shift amount indicating temporal movement of the image signal of the analysis target block;
The spatial frequency component of the corrected base image is read from the storage means, and the visual sensitivity value assigned to the spatial frequency component is calculated based on the spatial frequency index of the spatial frequency component and the shift amount, and the spatial frequency is calculated. A weighting means for weighting the components;
Calculation means for calculating importance for each base component of the prediction error signal based on the weighted spatial frequency component, the non-weighted spatial frequency component, and the transform coefficient of the block constituting the analysis target block When,
To assess the coding cost by using the distortion amount of the weighted coded using the importance degree, e Bei and determining means for determining the encoding parameters,
The calculating means multiplies a multiplication value of a square sum of the transform coefficients and a square norm sum of the weighted spatial frequency components, and a multiplication of a square sum of the transform coefficients and a square norm sum of the non-weighted spatial frequency components. And calculating the importance according to a division value of the two multiplication values ,
A moving image encoding device. In the moving image encoding device according to claim 3 ,
The weighting means calculates a visual sensitivity value in the horizontal direction based on a horizontal spatial frequency index and a horizontal component of the shift amount, and based on a vertical spatial frequency index and the vertical component of the shift amount. By calculating the visual sensitivity value in the vertical direction, calculating the visual sensitivity value assigned to the read spatial frequency component,
A moving image encoding device. Video encoding program for executing a video encoding method according to the computer to claim 1 or 2. A computer-readable recording medium on which a moving image encoding program for causing a computer to execute the moving image encoding method according to claim 1 or 2 is recorded.

Images