JP4490351B2 - Inter-layer prediction processing method, inter-layer prediction processing apparatus, inter-layer prediction processing program, and recording medium therefor - Google Patents

Description

  The present invention relates to a high-efficiency image signal encoding method, and more particularly to an inter-layer prediction processing method that can suppress additional information given to each layer in inter-layer prediction encoding.

  In recent years, scalable coding to deal with diversifying network environments and terminal environments has attracted attention. In scalable coding, an image signal is divided hierarchically and coding is performed for each layer. Hierarchical division methods include (i) band division related to spatial frequency and (ii) band division related to time frequency. Typical examples of (i) are wavelet division (see Non-Patent Document 1), and (ii) is Motion Compensation Temporal Fitering (MCTF) (see Non-Patent Document 2).

  For the purpose of improving the coding efficiency for each layer signal, a method of dividing the frame of each layer according to the local property and performing the coding independently for each divided region has been studied. In this case, an encoding parameter is given for each region. An example of giving individual coding parameters for each region is luminance compensation. Luminance compensation is a technique that improves the inter-frame prediction part of motion compensation as described below.

  In motion compensation, first, an encoding target frame f (p, t) (pε {(x, y) | 0 ≦ x ≦ X−1, 0 ≦ y ≦ Y−1}) (screen size X × Y). Is divided into several regions, and the following inter-frame prediction using motion compensation (MC) is performed for each region.

ここで，Ｂ_i は第ｉ番目の領域であり，ｖ_i は次式を満たす動きベクトルである。

Here, B _i is the i-th region, and v _i is a motion vector that satisfies the following equation.

ここで，

here,

は，次に続く関数を最小化するｖを返す。すなわち，探索範囲Ｒ_i において，領域Ｂ_i 内での予測誤差を最小化するｖが動きベクトルｖ_i として選ばれる。次に，予測誤差に直交変換，量子化が施され，最後に，得られた変換係数がエントロピ符号化される。

Returns v which minimizes the next function. That is, in the search range R _i, v that minimizes the prediction error in the area B _i is selected as the motion vector v _i. Next, orthogonal transformation and quantization are performed on the prediction error, and finally, the obtained transform coefficient is entropy coded.

  Since reducing the prediction error leads to a reduction in the code amount of the transform coefficient, the following improvement of inter-frame prediction has been studied for the purpose of reducing the prediction error. This is called luminance compensation (see Non-Patent Document 3 and Non-Patent Document 4).

ここで，γ_i は，

Where γ _i is

であり，式（１）のｖ_i に対して，

And for v _i in equation (1),

を最小化する値として求めたものである。
S.G.Mallat：“A theory for multiresolution signal decomposition: the wavelet representation ”，IEEE Trans. Pattern Analysis and Machine Intelligence ，Vol.11，No.7，pp.674-693，July，1989 J.R.Ohm ：“Three-dimensional subband coding with motion compensation ”，IEEE Trans. Image Processing，Vol.3 ，No.5，pp.559-571，Sept. ，1994 K.H.Tzou，T.R.Hsing ，and N.A.Daly: “Block-recursive matching algorithm（BRMA）for displacement estimation of video images ”，IEEE Int. Conf. Accost. ，Speech，Signal Process. ，pp.359-362，1985 上倉一人，渡辺裕，小林直樹，一之瀬進，安田浩：“演算量低減を考慮したグローバル動き・輝度変化補償動画像符号化”，信学論，Vol.J82-B ，No.9，pp.1676-1688，1999

Is obtained as a value that minimizes.
SGMallat: “A theory for multiresolution signal decomposition: the wavelet representation”, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.11, No.7, pp.674-693, July, 1989 JROhm: “Three-dimensional subband coding with motion compensation”, IEEE Trans. Image Processing, Vol.3, No.5, pp.559-571, Sept., 1994 KHTzou, TRHsing, and NADaly: “Block-recursive matching algorithm (BRMA) for displacement estimation of video images”, IEEE Int. Conf. Accost., Speech, Signal Process., Pp.359-362, 1985 Hitoshi Uekura, Hiroshi Watanabe, Naoki Kobayashi, Susumu Ichinose, Hiroshi Yasuda: “Global Motion / Brightness Change Compensated Video Coding Considering Computational Complexity”, IEICE, Vol.J82-B, No.9, pp. 1676-1688, 1999

  The luminance compensation described above also contributes to improving the coding efficiency of each layer in scalable coding. However, if each layer is encoded independently, improvement in encoding efficiency cannot be expected. Therefore, for the purpose of improving the coding efficiency, a coding method using the correlation between layers is examined. Specifically, inter-layer prediction for predicting a high spatial resolution image signal from a low spatial resolution image signal is performed for two layers having different spatial resolutions. In this case, how to suppress the additional information by prediction is the key to improving the coding efficiency.

  The present invention has been made in view of such circumstances, and an object thereof is to establish an efficient encoding method for additional information given to each layer in scalable encoding.

In the following, coordinates in time t th frame j hierarchical (x, y) and the pixel value at f _j (x, y, t) and, f _j (x, y, t) of the decoded signal for g _j (x , Y, t). f _{j + 1} (x, y, t) has a spatial resolution twice that of f _j (x, y, t). For example, if f ₀ (x, y, t) is a QCIF size, f ₁ (x, y, t) is a CIF size.

The frame at time t in the j-th layer is divided into M _j regions, and each region has an independent encoding parameter (for example, luminance compensation parameter). Here, an area having the same coding parameter is called a segment. The minimum unit of the segment is an L × L pixel area, and the same area is called a unit segment. Also, information representing area division by segments in a frame is called segment information.

  The most important feature of the present invention is that the segment information of the (j + 1) -th layer is predicted from the segment information of the j-th layer and the prediction error is encoded.

  Specifically, the present invention divides an image signal into layers having different spatial resolutions, obtains encoding parameters for each of the divided regions, and performs image encoding processing in which low spatial resolution signals (lower layer signals) are obtained. ) Region division information of a high spatial resolution signal (upper layer signal) is predicted, and prediction error information is expressed using direction information for each unit segment.

In the present invention, when the direction information for the prediction error information is encoded, the direction information to be encoded is
(1) When correction for the predicted shape is unnecessary,
(2) When the predicted shape needs to be corrected and the number of segments to which the unit segment that needs correction belongs is limited to one,
(3) When the prediction shape needs to be corrected and the segment to which the unit segment that needs correction has two types of compensation,
It is classified into three types consisting of and encoded.

In the present invention, when the direction information for the prediction error information is encoded, the direction information to be encoded is
(1) When correction for the predicted shape is unnecessary,
(2) When the predicted shape needs to be corrected and the number of segments to which the unit segment that needs correction belongs is limited to one,
(3) When the prediction shape needs to be corrected and the segment to which the unit segment that needs correction has two candidates,
(4) When the predicted shape needs to be corrected and the unit segment that needs correction does not belong to the adjacent segment,
It is also possible to classify and encode into 4 types.

  Furthermore, in the above invention, when encoding two types of segment candidates to which a unit segment that needs correction belongs, an encoded symbol is set according to a rule that determines the order of the direction information.

  Further, in the above invention, when encoding two candidate segments to which a unit segment that needs correction belongs, an encoding symbol can be set according to the number of pixels in the candidate segment.

  Further, in the above invention, when encoding two candidate segments to which a unit segment requiring correction belongs, the encoding is performed according to the quantization step width in the lower layer segment corresponding to the candidate segment. You can also set a symbol.

  Although the segment is an area having the same coding parameter, the feature of the present invention is that the area division information of the upper layer signal is predicted from the area division information of the lower layer signal between layers having different spatial resolutions. Because the prediction error information is encoded / decoded, a segment does not necessarily have to have the same encoding parameter, but a segment having a specific encoding parameter that can be used as a basis for segmentation. There may be.

  In scalable coding, overhead information is suppressed for segment information given to each layer with different spatial resolution, and coding efficiency is improved. This leads to a reduction in division loss in scalable coding.

The number of segments in each layer shall be transmitted separately as additional information. For this number of segments,
・ When the number of segments is the same in each layer (M ₀ = M ₁ = ... = M _J ),
When the number of segments increases in the upper layer (M ₀ ≦ M ₁ ≦… ≦ M _J )
Is shown below.

[When the number of segments is the same in each layer]
FIG. 1 shows an example of image block division. An example in which the number of segments is the same in each layer will be described with reference to FIG. Here, a square indicated by a broken line represents a unit segment. FIG. 1A shows region division in a frame at time t in the j-th layer, and FIG. 1B shows region division in a frame at time t in the j + 1-th layer.

  When each segment in FIG. 1A is doubled in the vertical / horizontal direction, the area division shown in FIG. 1C is obtained. This corresponds to the process of predicting the segment information of the (j + 1) th layer from the segment information of the jth layer in the present invention.

  The difference between the area division (predicted shape) in FIG. 1C and the area division (true shape) in FIG. 1B is shown in FIG. Here, different unit segments in FIG. 1B and FIG. A unit segment included in a segment different from the segment predicted from the lower hierarchy (predicted segment) is referred to as a corrected unit segment. On the other hand, the remaining unit segments that are not correction unit segments are called non-correction unit segments. An arrow on the correction unit segment indicates the direction of the segment to which the correction unit segment belongs.

  Information indicating the segment to which each unit segment belongs is called attribute information, and the method of giving this attribute information is shown below. First, a correction unit segment and a non-correction unit segment are classified. Furthermore, for the correction unit segment, one of up, down, left and right directions is designated as attribute information. However, the unit segments in the vicinity of the four adjacent neighbors are limited to at most two other than their own predicted segments.

  For example, in the case of a correction unit segment to which a left-pointing arrow is given in FIG. 1D, the up / right direction is in its own prediction segment according to FIG. 1C, and can be taken as attribute information of the correction unit segment. Is limited to two directions, down / left. Further, in the case of a correction unit segment to which a downward arrow is given in FIG. 1D, what can be taken as attribute information of the correction unit segment is limited only in the upward direction. Therefore, attribute information can be represented by at most three types of symbols. For example, there are the following methods for giving the three symbols.

<< Example 1 of attribute information >>
(1-i) Uncorrected unit segment: Attribute information is represented by 0.
(1-ii) In the case of a correction unit segment in which the possible attribute information is limited to one: The attribute information is represented by 1.
(1-iii) In the case of a correction unit segment having two possible attribute information: The two directions corresponding to the possible attribute information are arranged in the order of upward, rightward, downward, and leftward. Symbols 1 and 2 are assigned, and attribute information is represented by 1 or 2. This ordering of directions is an example, and other methods can be used in the same manner.

<< Example 2 of attribute information >>
(2-i) Same as (1-i) above (2-ii) Same as (1-ii) above (2-iii) Correction unit segment with two possible attribute information: Possible attribute information The numbers of pixels in the corresponding two types of segments are arranged in descending order, symbols 1 and 2 are assigned in this order, and attribute information is represented by 1 or 2. When the number of pixels in the segment indicated by each attribute information is equal, the same as (1-i) described above.

<< Example 3 of attribute information >>
(3-i) Same as (1-i) above (3-ii) Same as (1-ii) above (3-iii) In the case of a correction unit segment having two possible attribute information: corresponding to attribute information In each prediction segment, an average quantization step width per unit segment is obtained, the average quantization step widths are arranged in ascending order, symbols 1 and 2 are assigned in this order, and attribute information is represented by 1 or 2. In addition, when the above-mentioned prediction error is equal, it is the same as the above-mentioned (1-i).

[When the number of segments increases in the upper layer]
FIG. 2 shows another example of image block division. An example in which the number of segments increases in the upper layer will be described with reference to FIG. Here, an example is shown in which the j-th layer is divided into three segments, while the j + 1-th layer is divided into four segments. FIGS. 2A and 2C are the same as FIGS. 1A and 1C. FIG. 2B shows a case where a fourth segment is added. Corresponding to this, in FIG. 2 (d), a correction segment represented by dark shading is added. This segment does not belong to any adjacent segment. Such a correction unit segment is called a singular unit segment. When the number of segments increases in the upper layer, a symbol indicating a unique unit segment is required in addition to the above three types of symbols as attribute information. That is, attribute information is represented by a maximum of four types of symbols.

[Inter-layer prediction process]
FIG. 3 shows the flow of the encoding process according to the embodiment of the present invention. An embodiment in which the number of segments is the same in each layer will be described with reference to FIG.

  Step S1: Using a j-layer signal as input, perform processing for obtaining a segment, and write segment information to a register.

  Step S2: The segment information of the j-th layer is input, a process of enlarging each in the vertical / horizontal direction is performed twice, and the expanded segment information (predicted segment) is written to the register.

  Step S3: On the other hand, with the j + 1 layer signal as input, a process for obtaining a segment is performed and the segment information is written to the register.

  Step S4: The prediction segment and the segment information of the j + 1 layer are input, a process for obtaining the difference between the two is performed, and the difference information is written to the register. Thereafter, the following processing is performed for each unit segment.

  Step S5: Using the difference information written in step S4 for the unit segment to be processed as an input, a determination process is performed to determine whether or not the unit segment to be processed is an uncorrected unit segment. If the result is false, the process proceeds to step S7.

  Step S6: The attribute information for the unit segment to be processed is set to 0 and written to the register.

  Step S7: Using the difference information written in step S4 for the unit segment to be processed as an input, determine whether or not the unit segment to be processed has only one complement of attribute information, If so, the process proceeds to step S8. If not, the process proceeds to step S9.

  Step S8: The attribute information for the unit segment to be processed is set to 1 and written to the register.

  Step S9: The prediction segment and the segment information of the j + 1 layer are input, a process for determining the segment to which the correction unit segment to be processed belongs is performed, and direction information indicating the segment is written to the register.

  Step S10: Using the direction information written in step S9 as an input, a process for converting the same direction information into attribute information is performed, and the attribute information is written to the register. A specific conversion process uses (1-iii), (2-iii), or (3-iii).

  Step S11: It is determined whether or not the above processing has been completed for all the unit segments in the frame. If true, the processing is terminated, and if false, the processing returns to step S5 and similarly The process for the unit segment is performed.

  FIG. 4 shows the flow of encoding processing according to another embodiment of the present invention. An embodiment in which the number of segments increases in the upper layer will be described with reference to FIG.

  Step S21: The j-layer signal is input, the process for obtaining the segment is performed, and the segment information is written to the register.

  Step S22: The segment information of the j hierarchy is input, a process of enlarging the vertical / horizontal direction twice is performed, and the expanded segment information (predicted segment) is written to the register.

  Step S23: On the other hand, with the j + 1 layer signal as an input, the process for obtaining the segment is performed, and the segment information is written to the register.

  Step S24: The prediction segment and the segment information of the j + 1 layer are input, a process for obtaining the difference between the two is performed, and the difference information is written to the register. Thereafter, the following processing is performed for each unit segment.

  Step S25: Using the difference information written in step S24 for the unit segment to be processed as an input, a determination process is performed to determine whether or not the unit segment to be processed is an uncorrected unit segment. If the result is false, the process proceeds to step S27.

  Step S26: The attribute information for the unit segment to be processed is set to 0 and written to the register.

  Step S27: Using the difference information written in step S24 for the processing target unit segment as an input, a determination process is performed to determine whether the processing target unit segment is a singular unit segment. If true, the processing of step S28 is performed. If NO, the process proceeds to step S29.

  Step S28: The attribute information for the unit segment to be processed is set to 3 and written to the register.

  Step S29: The difference information written in step S24 with respect to the unit segment to be processed is input, and it is determined whether or not the unit segment to be processed has only one attribute information compensation. If so, the process proceeds to step S30. If not, the process proceeds to step S31.

  Step S30: The attribute information for the unit segment to be processed is set to 1 and written to the register.

  Step S31: The prediction segment and the segment information of the j + 1 layer are input, a process for determining the segment to which the processing target unit segment belongs is performed, and the direction information indicating the segment is written to the register.

  Step S32: Using the direction information written in step S31 as input, a process for converting the same direction information into attribute information is performed, and the same attribute information is written into the register. A specific conversion process uses (1-iii), (2-iii), or (3-iii).

  Step S33: It is determined whether or not the above processing has been completed for all the unit segments in the frame. If true, the processing is terminated. If false, the processing returns to step S25 and the same as described above. The process for the unit segment is performed.

[Inter-layer prediction processor]
FIG. 5 shows a configuration example of the inter-layer prediction processing apparatus according to the embodiment when the number of segments is the same in each layer. Hereinafter, a description will be given with reference to FIG.

  The encoding target signal is written to the j + 1 layer signal storage unit 101 and the j layer signal storage unit 104 for each layer.

  The segment calculation unit 102 receives the j + 1 layer signal read from the j + 1 layer signal storage unit 101 as input, performs a process for obtaining a segment, and writes the obtained segment information to the segment storage unit 103.

  Further, the j-layer segment calculation unit 105 receives the j-layer signal read from the j-layer signal storage unit 104, performs a process for obtaining a segment, and writes the obtained segment information to the segment storage unit 106. The segment enlargement processing unit 107 receives the segment information of the j hierarchy read from the segment storage unit 106, performs a process of enlarging twice in the vertical / horizontal directions, and stores the enlarged segment information (predicted segment) in the segment Write to part 108.

  The segment difference calculation unit 109 receives the segment information of the j + 1 layer read from the segment storage unit 103 and the segment information of the predicted segment read from the segment storage unit 108, performs a process for obtaining a difference between them, and converts the difference information into the segment difference Write to the storage unit 110. Thereafter, the following processing is performed for each unit segment.

  The non-correction unit segment determination unit 111 receives the difference information read from the segment difference storage unit 110 and performs a determination process as to whether or not the unit segment to be processed is an uncorrected unit segment. The process proceeds to the calculation unit 112. If false, the process proceeds to the correction unit segment candidate number determination unit 114.

  The attribute information calculation unit 112 writes the attribute information for the unit segment to be processed as 0 to the attribute information storage unit 113.

  The correction unit segment candidate number determination unit 114 receives the difference information read from the segment difference storage unit 110, performs a determination process as to whether or not the unit segment to be processed has only one attribute information compensation, If true, the process proceeds to the attribute information calculation unit 115. If false, the process proceeds to the direction information calculation unit 117.

  The attribute information calculation unit 115 writes the attribute information for the current processing target unit segment as 1 in the attribute information storage unit 116.

  The direction information calculation unit 117 receives the segment information of the j + 1 layer read from the segment storage unit 103 and the predicted segment read from the segment storage unit 108 and performs a process of determining a segment to which the unit segment to be processed belongs. Is written in the direction information storage unit 118.

  The attribute information calculation unit 119 receives the direction information read from the direction information storage unit 118, performs a process of converting the same direction information into attribute information, and writes the attribute information to the attribute information storage unit 120. A specific conversion process uses (1-iii), (2-iii), or (3-iii).

  FIG. 6 shows a configuration example of the inter-layer prediction processing apparatus according to the embodiment when the number of segments increases in the upper hierarchy. The difference from the inter-layer prediction processing apparatus shown in FIG. 5 is that a singular unit segment determination unit 221, an attribute information calculation unit 222, and an attribute information storage unit 223 are added. Other portions of the j + 1 layer signal storage unit 201 to the attribute information storage unit 220 are the same as the portions of the j + 1 layer signal storage unit 101 to the attribute information storage unit 120 shown in FIG. Hereinafter, a different part from the inter-layer prediction processing apparatus of FIG. 5 is demonstrated.

  The non-correction unit segment determination unit 211 receives the difference information read from the segment difference storage unit 210 and performs a determination process as to whether or not the unit segment to be processed is an uncorrected unit segment. The process proceeds to the calculation unit 212. If false, the process proceeds to the singular unit segment determination unit 221.

  The singular unit segment determination unit 221 receives the difference information read from the segment difference storage unit 210 and performs a determination process as to whether or not the unit segment to be processed is a singular unit segment. If true, the attribute information calculation unit The process proceeds to 222, and if false, the process proceeds to the correction unit segment candidate number determination unit 214.

  The attribute information calculation unit 222 writes the attribute information for the unit segment to be processed as 3 to the attribute information storage unit 223.

  By using the attribute information calculated as described above, for example, it is not necessary to separately encode encoding parameters such as luminance compensation parameters in the lower layer and the upper layer, and the lower layer encoding parameter is not used in the upper layer. By using it in the hierarchy, the amount of codes can be greatly reduced. Of course, not only the encoding parameters such as the luminance compensation parameter, but also the attribute information of the region according to the present invention is used for encoding information that is similarly related to the region of the lower layer signal and the region of the upper layer signal. Obviously you can.

  Further, the example in which the segment information of the (j + 1) -th layer is predicted from the segment information of the j-th layer and the prediction error is encoded has been described, but the same is true when decoding the encoded information encoded thereby. It is obvious that the segment information of the (j + 1) -th layer can be predicted from the segment information of the j-th layer by the process, and the prediction error can be decoded.

  The above-described inter-layer prediction process can be realized by a computer and a software program, and can be provided by recording the program on a computer-readable recording medium or via a network.

  FIG. 7 is a diagram illustrating an example of the overall configuration of a scalable encoding device to which the present invention is applied. The inter-layer prediction processing apparatus described with reference to FIGS. 5 and 6 is used as a part of the scalable encoding apparatus illustrated in FIG. 7, for example. In FIG. 7, 30 is a base layer encoding unit, and 31 is an enhancement layer encoding unit.

  First, the layer separator 300 receives an encoding target frame of an input image as an input, separates the layers into layers having different spatial resolutions, and writes the signals of each layer to the basic layer signal storage unit 301 and the extended layer signal storage unit 302, respectively. . The conversion unit 303 receives the basic layer signal read from the basic layer signal storage unit 301, performs conversion processing such as discrete cosine conversion processing, and writes the calculated conversion coefficient to the conversion coefficient storage unit 304. The quantization unit 305 receives the transform coefficient read from the transform coefficient storage unit 304, performs a quantization process, and writes the quantized value to the quantized value storage unit 306.

  The inverse quantization unit 307 receives the quantization value read from the quantization value storage unit 306, performs an inverse quantization process, and writes the result to the inverse quantization value storage unit 308. The inverse transform unit 309 receives the transform coefficient read from the inverse quantized value storage unit 308 as input, performs inverse transform processing, and writes it to the local decoded signal storage unit 310.

  The prediction processing unit 311 receives the addition value of the local decoded image read from the local decoded signal storage unit 310 and the output of the delay unit 313, performs prediction processing, and writes it to the prediction signal storage unit 312. The processing in the prediction processing unit 311 is prediction processing that requires segment information such as block-based motion compensation and luminance compensation. The prediction signal read from the prediction signal storage unit 312 is input to the delay unit 313 and delayed by one frame.

  The entropy encoding unit 314 receives the quantized value read from the quantized value storage unit 306, performs entropy encoding processing, and writes the encoded result to the encoded stream storage unit 315.

  The inter-layer prediction processing unit 316 performs inter-layer prediction processing according to the embodiment of the present invention described with reference to FIGS.

  The processing from the conversion unit 317 to the delay unit 327 in the enhancement layer encoding unit 31 is the same as the processing from the conversion unit 303 to the delay unit 313 in the base layer encoding unit 30.

  The multiplexer 330 performs a process of multiplexing the encoded streams read from the encoded stream storage unit 315 and the encoded stream storage unit 329, and outputs the result as a scalable encoding result.

  FIG. 8 is a diagram illustrating an example of the overall configuration of a scalable decoding device to which the present invention is applied. The inter-layer prediction processing device described with reference to FIGS. 5 and 6 is also used as a part of the scalable decoding device shown in FIG. 8, for example. In FIG. 8, 40 is a base layer decoding unit, and 41 is an enhancement layer decoding unit.

  The separator 401 receives the encoded stream output from the scalable encoding device shown in FIG. 7, performs a process of separating the stream into a base layer encoded stream and an enhancement layer encoded stream, and performs base layer encoding The stream and the enhancement layer coded stream are written in the base layer coded stream storage unit 402 and the enhancement layer coded stream storage unit 414, respectively.

  The entropy decoding unit 403 receives the encoded stream read from the base layer encoded stream storage unit 402, performs entropy decoding processing, and writes the decoded quantized value to the quantized value storage unit 404. The inverse quantization unit 405 receives the quantization value read from the quantization value storage unit 404, performs an inverse quantization process, and writes it to the transform coefficient storage unit 406. The inverse transform unit 407 receives the transform coefficient read from the transform coefficient storage unit 406 as input, performs an inverse transform process, and writes it to the decoded signal storage unit 408.

  The prediction processing unit 411 writes the addition value of the decoded signal read from the decoded signal storage unit 408 and the prediction value output from the prediction signal storage unit 412 to the base layer signal storage unit 409. Further, a signal obtained by delaying the base layer signal read from the base layer signal storage unit 409 by one frame by the delay unit 410 is written into the prediction signal storage unit 412 as a prediction signal.

  The inter-layer prediction processing unit 413 performs the inter-layer prediction process according to the embodiment of the present invention described with reference to FIGS.

  The processing from the entropy decoding unit 415 to the prediction processing unit 423 in the enhancement layer decoding unit 41 is the same as the processing from the entropy decoding unit 403 to the prediction processing unit 411 in the base layer decoding unit 40.

It is a figure which shows the example of the block division of an image. It is a figure which shows the other example of the block division of an image. It is a figure which shows the flow of the prediction process between layers based on embodiment in case the number of segments is the same in each hierarchy. It is a figure which shows the flow of the inter-layer prediction process which concerns on embodiment when the number of segments increases in an upper hierarchy. It is a figure which shows the structure of the inter-layer prediction processing apparatus which concerns on embodiment when the number of segments is the same in each hierarchy. It is a figure which shows the structure of the inter-layer prediction processing apparatus which concerns on embodiment when the number of segments increases in a higher hierarchy. It is a figure which shows the example of the whole structure of the scalable encoding apparatus to which this invention is applied. It is a figure which shows the example of the whole structure of the scalable decoding apparatus to which this invention is applied.

Explanation of symbols

101, 201 j + 1 layer signal storage unit 102, 105, 202, 205 Segment calculation unit 103, 106, 108, 203, 206, 208 Segment storage unit 104, 204 j layer signal storage unit 107, 207 Segment expansion processing unit 109, 209 Segment difference calculation unit 110, 210 Segment difference storage unit 111, 211 Uncorrected unit segment determination unit 112, 115, 119, 212, 215, 219, 222 Attribute information calculation unit 113, 116, 120, 213, 216, 220, 223 Attribute information storage unit 114, 214 Correction unit segment candidate number determination unit 117, 217 Direction information calculation unit 118, 218 Direction information storage unit

Claims (7)

This is an inter-layer prediction processing method in an inter-layer prediction encoding / decoding device that divides an image signal into a j-th layer having a low spatial resolution and a j + 1-th layer having a high spatial resolution and performs encoding / decoding for each layer. And
Calculating a segment which is an area having the same or specific coding parameter in the signal of the j-th layer, and obtaining segment information representing area division by the segment;
Calculating a segment which is an area having the same or specific coding parameter in the signal of the (j + 1) th layer, and obtaining segment information representing area division by the segment;
Predicting the segment information of the (j + 1) th layer from the segment information obtained in the signal of the jth layer;
Calculating difference information between the segment information obtained from the j + 1 layer signal and the j + 1 layer segment information predicted from the j layer signal;
And a process of encoding or decoding prediction error information of the calculated difference information. In the inter-layer prediction processing method according to claim 1,
In the process of encoding or decoding the prediction error information,
For each unit segment , which is the minimum unit of segments obtained from the signal of the j + 1th layer, whether each unit segment is included in a segment different from the predicted segment of the j + 1th layer predicted from the signal of the jth layer, Decide whether the prediction shape that is the region shape of the prediction segment needs to be corrected , and express the prediction error information using the direction information indicating the vertical and horizontal directions to the segment to which the unit segment belongs when correction is required , When encoding or decoding the direction information for the prediction error information,
If correction of the same direction information to the predicted shape is unnecessary,
When the predicted shape needs to be corrected and the segment to which the unit segment that needs correction belongs is limited to one,
When the predicted shape needs to be corrected and the segment to which the unit segment that needs correction has two types of compensation,
An inter-layer prediction processing method characterized in that it is classified into three types and encoded or decoded. In the inter-layer prediction processing method according to claim 1,
In the process of encoding or decoding the prediction error information,
For each unit segment , which is the minimum unit of segments obtained from the signal of the j + 1th layer, whether each unit segment is included in a segment different from the predicted segment of the j + 1th layer predicted from the signal of the jth layer, Determine whether or not the prediction shape that is the region shape of the prediction segment needs to be corrected , and express prediction error information using direction information that indicates the vertical and horizontal directions to the segment to which the unit segment belongs when correction is required , When encoding or decoding the direction information for the prediction error information,
If correction of the same direction information to the predicted shape is unnecessary,
When the predicted shape needs to be corrected and the segment to which the unit segment that needs correction belongs is limited to one,
If the predicted shape needs to be corrected and the segment to which the unit segment that needs correction has two candidates,
If the predicted shape needs to be corrected and the unit segment that needs to be corrected does not belong to the adjacent segment,
An inter-layer prediction processing method characterized by classifying and encoding or decoding into four types. In the inter-layer prediction processing method according to claim 2 or claim 3,
In the process of encoding or decoding the prediction error information,
When encoding two candidate segments to which a unit segment that needs correction belongs, set an encoded symbol according to a rule that determines the order of the direction information, or according to the number of pixels in the candidate segment A coding symbol is set according to a quantization step width in a segment in a lower layer corresponding to a candidate segment in the candidate segment. An inter-layer prediction processing device in an inter-layer prediction encoding / decoding device that divides an image signal into a j-th layer having a low spatial resolution and a j + 1-th layer having a high spatial resolution and performs encoding / decoding for each layer. And
Means for calculating a segment which is an area having the same or specific encoding parameter in the signal of the j-th layer, and obtaining segment information representing area division by the segment;
Means for calculating a segment which is an area having the same or specific encoding parameter in the signal of the j + 1th layer, and obtaining segment information representing area division by the segment;
Means for predicting the segment information of the (j + 1) th layer from the segment information obtained in the signal of the jth layer;
Means for calculating difference information between the segment information obtained from the j + 1 layer signal and the segment information of the j + 1 layer predicted from the j layer signal;
An inter-layer prediction processing apparatus comprising: means for encoding or decoding prediction error information of the calculated difference information.   An inter-layer prediction processing program for causing a computer to execute the inter-layer prediction processing method according to any one of claims 1 to 4.   A computer-readable recording medium recording an inter-layer prediction processing program for causing a computer to execute the inter-layer prediction processing method according to any one of claims 1 to 4.

Images