CN104094604A - Method and apparatus for encoding and decoding video using temporal motion vector prediction - Google Patents

Abstract

A method of encoding a video into a coded video bitstream with temporal motion vector prediction, the method comprising: determining a value of a flag for indicating whether temporal motion vector prediction is used or not used for the inter-picture prediction of a sub-picture unit of a picture; and writing the flag having said value into a header of the sub-picture unit or a header of the picture; wherein if the flag indicates that temporal motion vector prediction is used, the method further comprises: creating a first list of motion vector predictors comprising a plurality of motion vector predictors including at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture; selecting a motion vector predictor out of the first list for a prediction unit in the sub-picture unit; and writing a first parameter into the coded video bitstream for indicating the selected motion vector predictor out of the first list, wherein if the flag indicates that temporal motion vector prediction is not used, the method further comprises: creating a second list of motion vector predictors comprising a plurality of motion vector predictors without any temporal motion vector predictors; and selecting a motion vector predictor out of the first list for a prediction unit in the sub-picture unit. writing a second parameter into the coded video bitstream for indicating the selected motion vector predictor out of the second list. In addition, there is provided a method of decoding an encoded video and corresponding apparatuses for encoding and decoding a video.

Description

Method and apparatus for encoding and decoding video using temporal motion vector prediction

technical field

The present invention relates to a method of encoding a video and a method of decoding a video using temporal motion vector prediction, and an apparatus therefor. The invention can be applied to any multimedia data encoding, more specifically, the invention can be applied to encoding image and video content with temporal motion vector prediction for inter-picture prediction.

Background technique

Video coding schemes such as H.264/MPEG-4 AVC and the upcoming HEVC (High Efficiency Video Coding) use inter-picture (or "inter" for short) prediction to perform coding of image/video content based on previously coded/decoded reference pictures /decode in order to exploit the redundancy of information across temporally consecutive pictures.

In an encoded video bitstream, reference pictures used for inter-picture prediction processing of a prediction unit (eg, an M×N block of samples) are identified or referenced by using a reference index. A reference index is an index that includes an ordered list of one or more reference pictures (referred to as a reference picture list). Each reference index is uniquely associated with a reference picture in the reference picture list. That is, the reference index is a value for distinguishing a plurality of reference pictures from each other.

The coding schemes described above support temporal prediction of motion vectors (i.e., motion vector prediction or MVP), whereby the motion vector of a sampled target block is derived from the motion vectors of one or more previously coded sampled blocks in a co-located reference picture predicted. Temporal motion vector prediction further reduces the bit rate associated with motion vectors by exploiting the information redundancy between temporally adjacent motion vectors. The co-located reference picture is selected among available reference pictures using a predetermined scheme, eg, the first reference picture is selected in a predetermined reference picture list (eg, reference picture list 0) as the co-located reference picture.

In applications that need to transmit video across lossy environments, temporal motion vector prediction is susceptible to misprediction of motion vectors when co-located reference pictures are missing or contain errors. In the HEVC standard under development, a technique for disabling temporal motion vector prediction of a certain sub-picture unit (eg, slice) is disclosed. JCTVC-G398, "High-level Syntax: Marking process for non-TMVP pictures", Joint Collaborative Team on Video Coding (JCT-VC) of the seventh meeting of ITU-T SG16WP3 and ISO/IECJTC1/SC29/WG11, Geneva, CH, November 2011. In this technique, it is necessary to introduce a marking flag for marking a picture in the decoder picture buffer (DPB) as "not used for temporal motion vector prediction" in the picture parameter set (PPS). This marking process is performed by the decoder when the sub-picture unit refers to a PPS with a marking flag equal to "TRUE".

Reference list

non-patent literature

NPL1: ISO/IEC14496-10,"MPEG-4Part10Advanced Video Coding"

NPL2: JCTVC-G398, "High-level Syntax: Marking process for non-TMVP pictures", ITU-T SG16WP3 and ISO/IEC JTC1/SC29/WG11 7th Joint Collaborative Team on Video Coding (JCT-VC), Geneva , CH, November 2011.

Contents of the invention

technical problem

As mentioned in the background, in the disclosed technique for disabling temporal motion vector prediction for certain slices, it is necessary to introduce a flag flag in the picture parameter set (PPS) for the decoder picture buffer (DPB) The picture is marked as "not used for temporal motion vector prediction". A major problem associated with this technique is that the decoder cannot perform the intended marking process when the slice calling the marking process is missing or contains errors. Consequently, synchronization between subsequent encoders and decoders is lost. The above-described techniques for disabling temporal motion vector prediction are therefore not robust.

problem solution

The present invention seeks to provide methods and apparatus for encoding and decoding video with temporal motion vector prediction with improved error tolerance. In particular, temporal motion vector prediction for sub-picture units (eg, slices) is enabled/disabled in a manner that is not susceptible to errors. For example, according to an embodiment of the present invention, the above-mentioned marking process performed by the decoder (ie for marking a reference picture as "not used for temporal motion vector prediction") is eliminated.

According to a first aspect of the present invention, there is provided a method of encoding video into an encoded video bitstream using temporal motion vector prediction, the method comprising:

determining the value of a flag indicating whether temporal motion vector prediction is used or not used for inter-picture prediction of a sub-picture unit of a picture;

writing the flag into the header of the sub-picture unit or the header of the picture; and

Wherein, if the flag indicates that temporal motion vector prediction is used, the method further includes:

Creating a first list of motion vector predictors comprising a plurality of motion vector predictors comprising: at least one temporal motion derived from at least one motion vector from a collocated reference picture vector predictor;

selecting a motion vector predictor from the first list for a prediction unit in the sub-picture unit; and

A first parameter is written to the encoded video bitstream to indicate a motion vector predictor selected from the first list.

Preferably, if the flag indicates that temporal motion vector prediction is not used, the method further comprises:

creating a second list of motion vector predictors comprising a plurality of motion vector predictors without any temporal motion vector predictors;

selecting a motion vector predictor from the second list for a prediction unit in the sub-picture unit; and

A second parameter is written to the encoded video bitstream to indicate a motion vector predictor selected from the second list.

In one embodiment, the value of the flag is determined based on the temporal layer of the picture.

Preferably, if it is determined that the temporal layer of the picture is the lowest layer or base layer, the value of the flag is set to indicate that temporal motion vector prediction is not used, otherwise, the value of the flag is set to indicate that temporal motion vector prediction is used predict.

In another embodiment, the value of the flag is determined based on a picture order count (POC) value of the picture.

Preferably, if it is determined that said POC value of said picture is greater than any POC value of a reference picture in a decoder picture buffer (DPB), then the value of said flag is set to indicate that temporal motion vector prediction is not used, otherwise, set The value of the flag to indicate that temporal motion vector prediction is used.

In yet another embodiment, the value of the flag is determined based on a sub-picture unit type of an inter-picture sub-picture unit in the picture.

Preferably, if the sub-picture unit type is a predictive (P) type, then set the value of the flag to indicate that temporal motion vector prediction is not used, otherwise, set the value of the flag to indicate that temporal motion vector prediction is used .

In yet another embodiment, the value of the flag is determined based on whether the picture containing the sub-picture unit is a Random Access Point (RAP) picture.

Preferably, if the picture is a RAP picture and the sub-picture unit belongs to the non-base layer of the picture, then set the value of the flag to indicate that temporal motion vector prediction is not used, otherwise, set the value of the flag to indicate Temporal motion vector prediction is used.

Preferably, the flag is written into the header of the sub-picture unit.

Advantageously, the method further comprises: writing one or more parameters into the header of the sub-picture unit to specify reference pictures in one or more reference picture lists for inter-picture prediction of the sub-picture unit Order.

Preferably, the method also includes:

performing motion-compensated inter-picture prediction using the selected motion vector predictor to generate the prediction unit;

subtracting the prediction unit from the original sample block to produce a residual sample block; and

The remaining sample blocks corresponding to prediction units are encoded into the encoded video bitstream.

In one embodiment, said second list includes one less motion vector predictor than said first list, and in addition to said temporal motion vector predictors, said motion vector predictors of said first and second lists characters are the same.

Advantageously, said first and second parameters are represented in said encoded video bitstream using different predetermined bit representations.

In another embodiment, said first and second lists include the same predetermined number of motion vector predictors, and said second list includes A motion vector predictor derived from the presence of motion vectors from any reference picture.

Preferably, the flag is used to indicate whether temporal motion vector prediction is used or not used for the inter-picture prediction of the sub-picture unit independent of other sub-picture units in the picture.

Preferably, the sub-picture unit is a picture slice.

According to a second aspect of the present invention, there is provided a method of decoding an encoded video bitstream using temporal motion vector prediction, the method comprising:

parsing flags from headers of sub-picture units or headers of pictures of the encoded video; and

determining whether the flag indicates whether temporal motion vector prediction is used or not used;

Wherein, if the flag indicates that temporal motion vector prediction is used, the method further includes:

creating a first list of motion vector predictors comprising a plurality of motion vector predictors comprising: at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture ;

A first parameter from the encoded video bitstream is parsed, the first parameter indicating a motion vector predictor selected from the first list for a prediction unit in the sub-picture unit.

Preferably, if the flag indicates that temporal motion vector prediction is not used, the method further comprises:

creating a second list of motion vector predictors comprising a plurality of motion vector predictors without any temporal motion vector predictors; and

A second parameter from the encoded video bitstream is parsed, the second parameter indicating a motion vector predictor selected from the second list for a prediction unit in the sub-picture unit.

According to a third aspect of the present invention, there is provided an apparatus for encoding video into an encoded video bitstream using temporal motion vector prediction, the apparatus comprising:

a control unit operable to: determine a value of a flag indicating whether temporal motion vector prediction is used or not used for inter-picture prediction of a sub-picture unit of a picture;

a writing unit operable to: write a flag having said value into a header of said sub-picture unit or a header of said picture;

motion vector prediction unit; and

an inter-picture prediction unit configured to: perform inter-picture prediction based on a motion vector predictor selected from the motion vector prediction unit,

Wherein the motion vector prediction unit is configured to: receive the flag, and based on the flag being a first value, the motion vector prediction unit is operable to: create a motion vector prediction comprising a plurality of motion vector predictors A first list of motion vector predictors comprising: at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture, and for a prediction unit in the sub-picture unit , select a motion vector predictor from said first list; and

The writing unit is further operable to: write a first parameter to the encoded video bitstream indicating the selected motion vector predictor from the first list.

Advantageously, when said flag is a second value, said motion vector prediction unit is operable to: create a second list of motion vector predictors comprising a plurality of motion vector predictors without any temporal motion vector predictors; and selecting a motion vector predictor from the first list for a prediction unit in the sub-picture unit; and

The writing unit is further operable to write a second parameter to the encoded video bitstream indicating the selected motion vector predictor from the second list.

According to a fourth aspect of the present invention, an apparatus for decoding an encoded video bitstream using temporal motion vector prediction is provided, the apparatus comprising:

a parsing unit operable to: parse a flag from a header of a sub-picture unit or a header of a picture of the encoded video; and determine whether the flag indicates whether temporal motion vector prediction is used or not used;

motion vector prediction unit; and

an inter-picture prediction unit configured to: perform inter-picture prediction based on a motion vector predictor selected from the motion vector prediction unit;

Wherein the motion vector prediction unit is configured to: receive the flag, and based on the flag being a first value, the motion vector prediction unit is operable to: create a motion vector prediction comprising a plurality of motion vector predictors a first list of symbols, the plurality of motion vector predictors comprising: at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture; and

The parsing unit is further operable to: parse a first parameter from the encoded video bitstream, the first parameter indicating a prediction unit from the first list for a prediction unit in the sub-picture unit The selected motion vector predictor.

Advantageously, when said flag is a second value, said motion vector prediction unit is operable to: create a second list of motion vector predictors comprising a plurality of motion vector predictors without any temporal motion vector predictors; as well as

The parsing unit is further operable to parse a second parameter from the encoded video bitstream, the second parameter indicating a prediction unit from the second list for a prediction unit in the sub-picture unit. The selected motion vector predictor.

Advantageous effect of the present invention

Embodiments of the present invention provide methods and apparatus for encoding and decoding video using temporal motion vector prediction with improved error tolerance of inter-picture prediction. For example, these embodiments may also result in improved flexibility and coding efficiency of inter-picture prediction, since temporal motion vector prediction may be enabled and disabled independently for multiple sub-picture units in the same picture.

Description of drawings

Figure 1 depicts an exploded schematic diagram of an exemplary encoded video bitstream according to an embodiment of the present invention;

FIG. 2 depicts a flowchart illustrating a method of encoding video according to an embodiment of the present invention;

Figure 3 depicts a schematic block diagram of an exemplary apparatus for encoding an input video/image bitstream;

FIG. 4 depicts a flowchart illustrating a method of decoding encoded video according to an embodiment of the present invention;

5 depicts a schematic block diagram of an exemplary apparatus for decoding an input encoded bitstream;

Figure 6 depicts a diagram showing different temporal layers of an exemplary set of pictures;

FIG. 7 depicts a flowchart illustrating a method of determining the value of a temporal motion vector prediction usage flag according to a first embodiment;

FIG. 8 depicts a flowchart illustrating a method of determining the value of a temporal motion vector prediction usage flag according to a second embodiment;

FIG. 9 depicts a flowchart illustrating a method of determining the value of a temporal motion vector prediction usage flag according to a third embodiment;

Figure 10 depicts a graph representation of a NAL unit stream, i.e., a series of NAL units for a coded video bitstream;

Figure 11 depicts a graph representation of an exemplary RAP picture comprising multiple views/layers with multiple slices;

FIG. 12 depicts a flowchart illustrating a method of determining the value of a temporal motion vector prediction usage flag according to a fourth embodiment;

FIG. 13 shows an overall configuration of a content providing system for realizing a content distribution service;

FIG. 14 shows an overall configuration of a digital broadcasting system;

FIG. 15 shows a block diagram illustrating a configuration example of a television.

16 shows a block diagram illustrating a configuration example of an information reproduction/recording unit that reads information from and writes information on a recording medium that is an optical disc;

FIG. 17 shows an example of the configuration of a recording medium as an optical disc;

Figure 18A shows an example of a cellular phone;

FIG. 18B is a block diagram showing a configuration example of a cellular phone;

Figure 19 shows the structure of multiplexed data;

Fig. 20 schematically shows how each stream is multiplexed in multiplexed data;

Figure 21 shows in more detail how a video stream is stored in a stream of PES packets;

Figure 22 shows the structure of TS packets and source packets in multiplexed data;

Figure 23 shows the data structure of PMT;

Figure 24 shows the internal structure of the multiplexing data information;

Figure 25 shows the internal structure of stream attribute information;

Figure 26 shows the steps for identifying video data;

FIG. 27 shows an example of the configuration of an integrated circuit for realizing a moving picture encoding method and a moving picture decoding method according to each embodiment;

Figure 28 shows an arrangement for switching between drive frequencies;

Figure 29 shows steps for identifying video data and switching between drive frequencies;

FIG. 30 shows an example of a lookup table in which video data standards are associated with drive frequencies;

31A is a diagram showing an example of a configuration for sharing modules of a signal processing unit;

FIG. 31B is a diagram showing another example of a configuration for sharing modules of a signal processing unit.

Detailed ways

According to exemplary embodiments of the present invention, a method of encoding a video using Temporal Motion Vector Prediction (TMVP) and a method of decoding a video, and apparatuses thereof are provided. In particular, temporal motion vector prediction for sub-picture units (eg, slices) is enabled/disabled in a manner that is not susceptible to errors. In order to achieve this goal, according to a preferred embodiment of the present invention, a flag is introduced into the header of the picture or more preferably into the header of the sub-picture unit to indicate the inter-picture (or simply is "inter") whether temporal motion vector prediction is used for prediction. This flag may also be referred to as a temporal motion vector prediction use flag. In a further aspect of the invention, preferred techniques for determining/determining the value of the flag are disclosed in various embodiments.

Exemplary embodiments of the present invention will now be described in further detail for clarity and simplicity whereby a sub-picture unit is a slice of a picture. Those skilled in the art will appreciate that slice partitioning is only one possible method for dividing a picture into sub-picture partitions. Therefore, embodiments of the present invention described hereinafter are not limited to sub-picture units being slices. For example, other sub-picture segmentation methods such as tiling, entropy slices, and wavefront segmentation units are within the scope of the present invention.

FIG. 1 is an exploded schematic diagram of an exemplary encoded video bitstream 100 according to an embodiment of the present invention. The encoded video bitstream 100 includes a header 110 and a plurality of pictures 112 associated with the header 110 . A picture 112 is typically partitioned into a plurality of sub-picture units (eg, slices) 114 . Each slice 114 includes a slice header 116 and slice data 118 associated with the slice header 116 . The slice data 118 includes a plurality of prediction units 120 of inter-picture prediction type.

In the exemplary embodiment as shown in FIG. 1 , a flag 122 for indicating whether temporal motion vector prediction is used for the inter-picture prediction of the slice 114 is preferably located in the slice header 116 . Thus, temporal motion vector prediction for each slice 114 can be enabled and disabled independently of other slices 114 in the same picture 112 . The slice header 116 also includes a reference picture list ordering parameter 124 for specifying the order of the reference pictures in the one or more reference picture lists. These parameters 124 determine the effective or final order of the reference pictures in the reference picture list used for inter-picture prediction of the slice 114 associated with or corresponding to the slice header 116 . These parameters 124 may specify a reordering process to be performed on one or more of the original reference picture lists, or may specify that the original reference picture lists be used without reordering. As shown in FIG. 1 , flags 122 are preferably located in the same slice header 116 as reference picture list ordering parameters 124 . A motion vector predictor selection parameter 126 is located in each prediction unit 120 for selecting a motion vector predictor among a plurality of motion vector predictors available for inter-picture prediction of the prediction unit 120 .

In another embodiment, the reference picture list ordering parameter 124 and the temporal motion vector prediction usage flag 122 are located in a header (not shown) that is shared between multiple slices 114 in the same picture 112 . For example, the picture level header 110 may be an adaptation parameter set (APS) or a common slice segment header in the HEVC coding scheme.

As explained above, slice partitioning is only one possible method for dividing a picture into sub-picture partitions. Other possible sub-picture segmentation methods can be used, such as tiling, entropy slices, and wavefront segmentation units. In these other sub-picture segmentation methods, the parameters 124 and flags 122 located in the slice header 116 may instead be located in the header of the sub-picture unit as described above.

FIG. 2 depicts a flowchart illustrating a method 200 of encoding video according to an embodiment of the present invention. In step S202, one or more parameters (that is, reference picture list sorting parameters) 124 are written into the header 116 of the slice 114 to specify one or more reference picture lists for inter-picture prediction of the slice 124 The order of the reference pictures. A predetermined position (eg first picture) in one of these reference picture lists (eg reference picture list 0) indicates a co-located reference picture. In step S204, the value of the flag 122 indicating whether temporal motion vectors are used for the inter-picture prediction for the slice 124 is determined. Various techniques for determining the value of flag 122 will be described later in accordance with various embodiments of the invention. Then in step S206 , the flag 122 is written into the header 116 of the slice 114 . In step S208, the value of the flag 122 is analyzed or judged to determine whether the flag 122 indicates whether temporal motion vector prediction is used or not used. For example, a flag 122 with a value of "0" may indicate that temporal motion vector prediction is not used, while a flag 122 with a value of "1" may indicate that temporal motion vector prediction is used, or vice versa.

If the flag 122 indicates that temporal motion vector prediction is used, then in step S210, a list of motion vector predictors (the first list) is created, which includes a plurality of motion vector predictors including at least A motion vector is derived from at least one temporal motion vector predictor. By way of example only, the plurality of motion vectors may include at least one temporal motion vector predictor, one or more motion vectors derived from spatially adjacent prediction units/blocks (i.e., spatial motion vector predictors), and Zero motion vectors. In step S212 , a motion vector predictor is selected from the list of motion vector predictors for the sampled target block (ie prediction unit) 120 in the slice 124 . In step 214, a parameter (i.e., a motion vector predictor selection parameter) (e.g., a first parameter) 126 is written to the encoded video bitstream 100 (i.e., to the prediction unit 120 of the slice 114) for Indicates a motion vector predictor selected from the list of motion vector predictors.

On the other hand, if the flag 122 indicates that temporal motion vector prediction is not used, then in step S216, a list of motion vector predictors including a plurality of motion vector predictors without any temporal motion vector predictor is created (for example, a second list ). In step S218 , a motion vector predictor is selected from the list of motion vector predictors for the sampled target block (ie, prediction unit) in the slice 124 . In step S220, a parameter (i.e., a motion vector predictor selection parameter) (e.g., a second parameter) is written into the encoded video bitstream 100 (i.e., into the slice data 118 associated with the slice header 116 Each prediction unit 120) is used to indicate a motion vector predictor selected from the list of motion vector predictors.

After step S214 or step S220, motion-compensated inter-picture prediction is performed for the slice 214 using the selected motion vector predictor to generate a prediction sample block. Subsequently, in step S226, the predicted sample block is subtracted from the original sample block to generate a residual sample block. Therefore, in step S226 , the remaining sample blocks corresponding to the target block are encoded into the encoded video bitstream 100 .

Therefore, in the above-described embodiments of the present invention, the flag 122 for indicating whether temporal motion vector prediction is used can control one slice 114 independently of other slices 114 in the same picture 112 . Therefore, a flag 122 corresponding to a first slice 114 in a second or other slice in the same picture 112 does not determine whether temporal motion vector prediction is used. In addition, in the above-described embodiments, the marking process for reference pictures in the decoder picture buffer (DPB) as described in the background art is eliminated. This results in improved flexibility and coding efficiency for inter-picture prediction.

In an embodiment of the invention, the first and second lists of motion vector predictors comprise different numbers of motion vector predictors. Preferably, the second list includes one less motion vector predictor than the first list. Motion vector predictors other than temporal motion vector predictors may be the same or equivalent in both the first and second lists. This can increase coding efficiency because the encoder has more options to choose the best candidate from the list comprising temporal motion vector predictors (ie, the first list). The second list may provide better error tolerance because temporal motion vector prediction is not used. In the encoded video bitstream 100, the first and second parameters representing the selected motion vector predictors may be represented using different bits, e.g. using binary or variable length codes with different maximum values The truncated unary representation of .

In another embodiment of the invention, the first and second lists include the same number of motion vector predictors. The second list includes another unique predetermined motion vector predictor not present in the first list, instead of the temporal motion vector predictor. This can increase coding efficiency, since the encoder has more options to select the best candidate from the list (ie, the second list) comprising unique predetermined motion vector predictors. Since the maximum number of candidate temporal motion vector predictors is the same for the first and second lists, this reduces the complexity of the parsing process for the index parameter indicating the selected motion vector predictor. A unique motion vector predictor is derived without temporal dependence (ie, without using motion vectors from any reference pictures). Merely by way of example, the only motion vector predictor may be a spatial motion vector predictor from a predetermined neighboring position. As another example, the only motion vector predictor may be a zero motion vector predictor.

An exemplary apparatus 300 for encoding video according to an embodiment of the present invention will now be described below.

FIG. 3 depicts a schematic block diagram of an exemplary apparatus 300 for encoding an input video/image bitstream 302 on a block-by-block basis to generate an encoded video bitstream 304 . The apparatus 300 comprises: a transformation unit 306 operable to transform input data into frequency coefficients; a quantization unit 308 operable to quantize the input data; an inverse quantization unit 310 operable to dequantize the input data; an inverse transform unit 312 operable to inverse frequency transform input data; a block memory 314 and a picture memory 316 operable to store data such as video and images; an intra-picture prediction unit operable to perform intra-picture prediction 318; an inter-picture prediction unit 320 operable to perform inter-picture prediction; an entropy encoding unit 322 operable to encode input data into the encoded video bitstream 304; operable to decide an inter-picture prediction for a target slice A control unit 324 that predicts whether to use temporal motion vector prediction; a motion vector prediction unit 330 ; and a write unit 328 operable to write data into the encoded video bitstream 304 .

For clarity, an exemplary data flow through apparatus 300 as shown in FIG. 3 will now be described. The input video 302 is input to the adder and the added value 305 is output to the transform unit 306 . The transform unit 306 transforms the added value 305 into a frequency coefficient, and outputs the resulting frequency coefficient 307 to the quantization unit 308 . The quantization unit 308 quantizes the input frequency coefficient 307 and outputs the resulting quantized value 309 to the inverse quantization unit 310 and the entropy encoding unit 322 . The entropy encoding unit 322 encodes the quantized value 309 output from the quantization unit 308 , and outputs an encoded video bitstream 304 .

The dequantization unit 310 dequantizes the quantized value 309 output from the quantization unit 308 , and outputs the frequency coefficient 311 to the inverse transformation unit 312 . The inverse transform unit 312 performs inverse frequency transform on the frequency coefficient 311 to transform the frequency coefficient into a sample value of the bit stream, and outputs the resulting sample value 313 to the adder. The adder adds the sample value 313 of the bitstream output from the inverse transform unit 314 to the predicted video/image value 319 output from the intra-picture prediction unit 318 or the inter-picture prediction unit 320, and outputs to the block memory 105 or the picture memory 106 The resulting added value 315 is used for further prediction. The intra-picture prediction unit 318 or the inter-picture prediction unit 320 searches in the reconstructed video/image stored in the block memory 314 or the picture memory 316 and estimates, for example, the video/image region most similar to the input video/image for predict.

The control unit 324 makes a decision as to whether the inter-picture prediction for the target slice uses temporal motion vector prediction, and outputs a signal 325 indicating the decision to the motion vector prediction unit 330 and the writing unit 322 . Various techniques for deciding/determining whether temporal motion vector prediction is used (ie, determining the value of flag 122 ) will be described later in accordance with various embodiments of the invention. Based on this decision, inter-picture prediction unit 320 performs inter-picture prediction with or without using a temporal motion vector predictor. Specifically, the motion vector prediction unit 330 is configured to receive the flag 122, and if the flag is a first value (eg, "1"), the motion vector prediction unit 330 is operable to create a first list comprising a plurality of motion vector predictors, including at least one temporal motion vector predictor derived from at least one motion vector from a co-located reference picture, and for a prediction unit in a sub-picture unit from the first list A motion vector predictor is selected. The writing unit 328 is further operable to: write a first parameter into the encoded video bitstream indicating the motion vector predictor 331 selected from the first list. On the other hand, if flag 122 is a second value (eg, "0"), motion vector prediction unit 330 is operable to: create a motion vector prediction that includes multiple motion vector predictors without any temporal motion vector predictors a second list of predictors; and for the prediction unit in the sub-picture unit, select a motion vector predictor from the second list. In this case, the writing unit 328 is further operable to: write a second parameter to the encoded video bitstream 304 indicating the motion vector predictor 331 selected from the second list. The writing unit 328 is also operable to: write data 326 representing a flag 122 having a first value or a second value (for example, "0" or "1") indicating whether temporal motion vector prediction is used into the encoded Video bitstream 304 (eg, sub-picture unit headers or picture headers).

FIG. 4 depicts a flowchart illustrating a method 400 of decoding encoded video according to an embodiment of the present invention. Specifically, the method 400 is operable to: decode the encoded video bitstream 100 encoded according to the above-mentioned method for encoding video as shown in FIG. 2 . In step S402, one or more parameters from the header 116 of the slice 114 (that is, the reference picture list sorting parameters) are parsed to specify one or more reference picture lists for inter-picture prediction of the slice 114. The order of the reference pictures. As mentioned above, a predetermined position (eg first picture) in one of the reference picture lists (eg reference picture list 0) indicates a co-located reference picture. In step S404, the flag (ie, temporal motion vector prediction flag) 122 from the header 116 is analyzed, and the flag 122 indicates whether temporal motion vector prediction is used for the inter-picture prediction of the slice 118 . In step S406, the value of the flag 122 is analyzed or judged to determine whether the flag 122 indicates whether temporal motion vector prediction is used or not used.

If the flag 122 indicates that temporal motion vector prediction is used, then in step S408, a list of motion vector predictors (the first list) is created, which includes a plurality of motion vector predictors including at least A motion vector is derived from at least one temporal motion vector predictor. By way of example only, the plurality of motion vectors may include at least one temporal motion vector predictor, one or more motion vectors derived from spatially adjacent prediction units/blocks (i.e., spatial motion vector predictors), and Zero motion vectors. In step S410, parameters (ie, motion vector predictor selection parameters) (eg, first parameters) 126 from the encoded video bitstream 100 (ie, prediction units 120 according to slices 114 ) are parsed, which indicate A motion vector predictor selected from the list of motion vector predictors for the sampled target block in slice 114 (ie, prediction unit 120).

On the other hand, if the flag 122 indicates that temporal motion vector prediction is not used, then in step S412, a list of motion vector predictors including a plurality of motion vector predictors without any temporal motion vector predictor (for example, a second list ). In step S414, a parameter (ie, motion vector predictor selection parameter) (eg, a second parameter) from the encoded video bitstream 100 (ie, prediction unit 120 according to slice 114 ) is parsed, which indicates for The motion vector predictor selected by the sampled target block in slice 114 (ie, prediction unit 120 ) from the list of motion vector predictors.

After step S410 or step S414, in step S416, motion compensated inter-picture prediction is performed using the selected motion vector predictor to generate a prediction sample block. Subsequently, in step S418 , the remaining sample blocks are decoded from the encoded video bitstream 100 . Thereafter, in step S420, the predicted sample block and the remaining sample block are added together to generate a reconstructed sample block corresponding to the target block.

An exemplary apparatus 500 for decoding encoded video according to an embodiment of the present invention will now be described below.

Fig. 5 depicts a schematic block diagram of an exemplary apparatus 500 for decoding an input encoded bitstream 502 on a block-by-block basis and outputting a video/image 504, for example to a display. The apparatus 500 comprises: an entropy decoding unit 506 operable to decode an input encoded bitstream 502; an inverse quantization unit 508 operable to inverse quantize the input data; Inverse transform unit 510 for transform; block memory 512 and picture memory 514 operable to store data such as video and images; intra-picture prediction unit 516 for performing intra-picture prediction; inter-picture prediction for performing inter-picture prediction unit 518; a motion vector prediction unit 522; and a parsing unit 503 operable to parse the input encoded bitstream 502 and output respective parameters 520,521.

For clarity, an exemplary data flow through apparatus 500 as shown in FIG. 5 will now be described. The input encoded bitstream 502 is input to an entropy decoding unit 506 . After the encoded bitstream 502 is input to the entropy decoding unit 506 , the entropy decoding unit 506 decodes the input encoded bitstream 502 and outputs the decoded value 507 to the inverse quantization unit 508 . The dequantization unit 508 dequantizes the decoded value 507 and outputs the frequency coefficient 509 to the inverse transformation unit 510 . The inverse transform unit 510 performs inverse frequency transform on the frequency coefficient 509 to transform the frequency coefficient 509 into a sample value 511, and outputs the resulting sample value 511 to an adder. The adder adds the resulting sampled value 511 to the predicted video/image value 519 output from the intra-picture prediction unit 516 or the inter-picture prediction unit 518 and outputs the resulting sampled value 519 to, for example, a display and to a block memory 512 or a picture memory 514 Value 504 for further prediction. Furthermore, the intra-picture prediction unit 516 or the inter-picture prediction unit 518 searches the video/image stored in the block memory 512 or the picture memory 514 and estimates, for example, the video/image area most similar to the decoded video/image with in forecasting.

In addition, the parsing unit 506 parses the flag 122 from the header of the slice or picture indicating whether the inter-picture prediction for the target slice uses temporal motion vector prediction, and outputs the parsed data 520 to the motion vector predicting unit 522 . The inter-picture prediction unit 518 is operable to perform inter-picture prediction with or without using a temporal motion vector predictor based on the value of the flag 122 and the selected motion vector predictor from the motion vector prediction unit 522 . In particular, motion vector prediction unit 522 is configured to receive data 520 containing flag 122, and if flag is a first value (eg, "1"), motion vector prediction unit 522 is operable to create a motion vector prediction A first list of symbols comprising a plurality of motion vector predictors including at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture. If the flag is a second value (eg, "0"), motion vector unit 522 is operable to: create a second list of motion vector predictors comprising a plurality of motion vector predictors without any temporal motion vector predictors. The parsing unit 503 is further operable to: parse a first or a second parameter from the encoded video bitstream 502, the first or second parameter indicating that the prediction unit in the sub-picture unit is selected from the second list. The extracted motion vector predictor and the parsed data 521 are output to the motion vector prediction unit 522.

As mentioned above, various techniques for deciding/determining whether temporal motion vector prediction is used (ie, determining the value of flag 122 ) will now be described in accordance with various embodiments of the invention.

According to the first embodiment, the value of the flag 122 is determined based on the temporal layer of the current picture. FIG. 6 depicts a diagram showing different temporal layers of a group of pictures, eg, when the group size/structure is configured as four. In this example, there are three temporal strata, namely, temporal stratum “0” 602 , temporal stratum “1” 604 , and temporal stratum “2” 606 . Pictures with picture order count (POC) values of 0, 4, and 8 are in temporal layer "0" 602, pictures with POC values of 2 and 6 are in temporal layer "1" 604, and pictures with POC values of 1, The pictures for POC values of 3, 5 and 7 are located in temporal layer "2" 606 . Time layers "0", "1" and "2" are associated with or represented by time ID0, 1 and 2, respectively. Accordingly, pictures in temporal layer "0" 602 have temporal ID "0" associated therewith, pictures in temporal layer "1" 604 have temporal ID1 associated therewith, and pictures in temporal layer "2" 606 has a time ID2 associated with it.

FIG. 7 depicts a flowchart illustrating a method 700 for determining the value of the flag 122 according to the first embodiment. In step S702, the temporal layer of the current picture is determined based on the temporal ID associated with the current picture. Subsequently, in step S704, analyze or judge whether the determined temporal layer is the lowest layer or the base layer (ie, whether temporal ID=0). If the temporal layer is the lowest layer, in step S706, the flag 122 is set to a value indicating that temporal motion vector prediction is not used (for example, "0"). On the other hand, if the temporal layer is not the lowest layer, then in step S708, the flag 122 is set to a value indicating that temporal motion vector prediction is used (for example, "1"). This is because, in typical coding structures, higher temporal ID pictures usually refer to pictures with temporal ID=0. In the case when a picture with temporal ID=0 is missing or contains an error, the error will be propagated to any picture that references the picture with temporal ID=0. This error propagation may continue and affect the reconstruction of all subsequent pictures using the temporal motion vector picture with temporal ID=0. Therefore, this embodiment improves error tolerance by not using temporal motion vector pictures with temporal ID=0.

According to the second embodiment, the value of the flag 122 is determined based on the POC value of the current picture. FIG. 8 depicts a flowchart illustrating a method 800 for determining the value of the flag 122 according to the second embodiment. In step S802, the POC value of the current picture and the POC values of all reference pictures in the DPB are obtained or determined. In step S804, analyze and judge whether the POC value of the current picture is greater than any POC value of the reference picture in the DPB. If so, then in step S806, flag 122 is set to a value indicating that temporal motion vector prediction is not used (eg, "0"). Otherwise, in step S808, the flag 122 is set to a value indicating that temporal motion vector prediction is used (for example, "1"). This is because higher quality pictures (eg, temporal layer 0 pictures) only reference pictures of the same or higher quality. In this embodiment, a higher quality picture is identified in view of a POC value of a reference picture contained in a decoded picture buffer storing a plurality of reference pictures. Subsequent pictures generally refer to higher quality pictures for reasons similar to the first embodiment described above. Therefore, to prevent or minimize error propagation, and improve fault tolerance, flag 122 is disabled for higher quality pictures.

According to the third embodiment, the value of the flag 122 is determined based on the slice type of the inter slice in the current picture. An inter slice is a slice encoded or decoded using inter-picture prediction. FIG. 9 depicts a flowchart illustrating a method 900 for determining the value of the flag 122 according to the third embodiment. In step 902, the slice type of the inter slice in the current picture is determined. Subsequently, it is analyzed and judged whether the slice type is a P slice (ie, a predictive slice). If so, then in step S906, flag 122 is set to a value indicating that temporal motion vector prediction is not used (eg, "0"). On the other hand, if the determined slice type is not a P slice (for example, it is a bidirectional predictive type or a B slice), then in step S908, the flag 122 is set to a value indicating that temporal motion vector prediction is used (for example, "1 "). The reason for this is because P slices use one-way forward prediction. Therefore, to prevent or minimize error propagation, and improve fault tolerance, flag 122 is disabled for P slices.

According to the fourth embodiment, the value of the flag 122 is determined based on whether the picture is a random access point (RAP) picture. A RAP picture is a picture that itself and subsequent pictures in decoding order can be correctly decoded without having to perform the decoding process of any picture preceding the RAP picture in decoding order. For example, the HEVC specification specifies a RAP picture as a coded picture with a NAL unit type (ie, nal_unit_type) in the range of 7 to 12 (inclusive) for each slice segment. Figure 10 depicts a graph representation of a NAL unit stream, ie a sequence of NAL units 102 for an encoded video bitstream. As known to those skilled in the art, the NAL (Network Abstraction Layer) formats the Video Coding Layer (VCL) representation of encoded video and provides headers in a manner suitable for transmission over various transport layers or storage media. department information. Each NAL unit 102 includes a header 104 followed by a data segment 106 . Header 104 includes parameters to indicate the type of data in NAL unit 102 , and data segment 106 contains the data indicated by header 104 . For example, FIG. 10 shows three NAL units: a first NAL unit containing a parameter set (as indicated by NAL unit type 108), a second NAL unit containing a base view/layer (as indicated by NAL unit type 110). Two NAL units, and a third NAL unit comprising a non-base view/layer (as indicated by NAL unit type 112). The header 104 of each NAL unit also includes: the time ID described in the first embodiment as shown in FIG. 7 .

FIG. 11 depicts a graph representation of an exemplary RAP picture 1100 containing multiple views/layers using multiple slices. As shown, a RAP picture 1100 includes a number of slices 1102 in a base layer (intra-picture view) 1104 and a number of slices 1106 in a non-base layer (inter-picture view) 1110 .

FIG. 12 depicts a flowchart illustrating a method 1200 for determining the value of the flag 122 according to the fourth embodiment. In step S1202, the picture is analyzed to determine or obtain parameters of each slice of the picture of the NAL unit type of the specified slice. Subsequently, in step S1204, it is determined or judged based on the obtained parameters whether the picture including the current slice is a RAP picture, and whether the current slice belongs to a non-base view/layer of the picture. Whether a picture is a RAP picture 1100 may be determined by analyzing the value of the NAL unit type 1008, 1010, 1012 in the header 1004 of each NAL unit or slice 1002 in the picture. As mentioned above, a RAP picture 1100 is a picture that itself and subsequent pictures in decoding order can be correctly decoded without performing the decoding process of any picture preceding the RAP picture 1100 in decoding order. For example, the HEVC specification specifies a RAP picture as a coded picture with NAL unit types ranging from 7 to 12 (boundary) for each slice segment. Thus, in this example, a picture is determined to be a RAP picture 1100 if the NAL unit type 1008, 1010, 1012 of each NAL unit 1002 in the picture is in the range 7 to 12 inclusive. Whether the current slice is a non-base layer of a picture may be determined by checking the NAL unit type 1008, 1010, 1012 of the current slice. For example, NAL unit type 1012 indicates that the associated slice 1006 belongs to a non-base layer, and NAL unit type 1010 indicates that the associated slice 1006 belongs to a base layer. However, it will be apparent to those skilled in the art that non-base layers may be identified based on other parameters depending on the video coding scheme. For example, in the current HEVC multi-view HEVC working draft, whether the current slice is a non-base layer of a picture is determined by a layer ID. If the picture is the RAP picture 1100 and the current slice belongs to the non-base layer of the picture, then in step S1206, the flag 122 is set to a value indicating that temporal motion vector prediction is not used (for example, "0"). Otherwise, in step S1208, the flag 122 is set to a value indicating that temporal motion vector prediction is used (for example, "1"). The reason for this is because the benefit of using temporal motion vector prediction is to temporally improve the motion vector prediction, that is, to make predictions from other pictures that are temporally different. However, it is not beneficial to use temporal motion vector prediction if the intra-picture and inter-picture are within the same time of the current picture. Therefore, to improve encoding/decoding efficiency, the flag 122 is disabled for slices 1106 belonging to non-base (or inter-picture view) layers of the RAP picture 1100 .

(Example A)

The processing described in each embodiment can be performed by recording a program for realizing the configuration of the moving picture encoding method (image encoding method) and moving picture decoding method (image decoding method) described in each embodiment in a recording medium, Simple implementation in a stand-alone computer system. The recording medium may be any recording medium such as a magnetic disk, an optical disk, a magneto-optical disk, an IC card, and a semiconductor memory as long as the program can be recorded.

Hereinafter, applications of the moving picture encoding method (image encoding method) and moving picture decoding method (image decoding method) described in the respective embodiments and a system using them will be described. The system is characterized by having an image encoding and decoding device including an image encoding device using an image encoding method and an image decoding device using an image decoding method. Other configurations in the system can be changed appropriately according to the situation.

Fig. 13 shows an overall configuration of a content providing system ex100 for realizing content distribution services. An area for providing communication services is divided into cells of a desired size, and base stations ex106, ex107, ex108, ex109, and ex110, which are fixed wireless stations, are placed in the respective cells.

The content providing system ex100 is connected to various devices such as a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a cellular phone ex114, and a game machine ex115 via the Internet ex101, an Internet service provider ex102, a telephone network ex104, and base stations ex106 to ex110, respectively. equipment.

However, the configuration of the content providing system ex100 is not limited to the configuration shown in FIG. 13 , and a combination in which any of these elements are connected is acceptable. In addition, each device can be directly connected to the telephone network ex104 instead of via the base stations ex106 to ex110 (which are fixed wireless stations). In addition, these devices may be interconnected with each other via short-range wireless communication or the like.

The video camera ex113 (such as a digital video camera) is capable of taking video. The camera ex116 (eg, a digital camera) is capable of capturing still images and video. In addition, the cellular phone ex114 may be a device that meets requirements such as Global System for Mobile Communications (GSM) (registered trademark), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), and High Speed Packet Access. (HSPA) standards for cellular telephones of any standard. Alternatively, the cellular telephone ex114 may be a Personal Handyphone System (PHS).

In the content providing system ex100, a streaming server ex103 is connected to a camera ex113 and other devices via a telephone network ex104 and a base station ex109, which can distribute images of live programs and the like. In such a distribution, as described above in various embodiments, the content captured by the user using the camera ex113 (for example, a video of a live music program) is encoded (that is, the camera is used as an image according to an aspect of the present invention) encoding device), and transmits the encoded content to the streaming server ex103. On the other hand, when a client makes a request, the streaming server ex103 performs streaming distribution of the transmitted content data to the client. The clients include a computer ex111, a PDA exX112, a video camera ex113, a cellular phone ex114, and a game machine ex115 capable of decoding the above-mentioned encoded data. Each device that has received the distributed data decodes and reproduces the coded data (ie, functions as an image decoding device according to an aspect of the present invention).

Captured data may be encoded by the camera ex113 or the streaming server ex103 sending the data, or the encoding process may be shared between the camera ex113 and the streaming server ex103. Similarly, distributed data may be decoded by the client or the streaming server ex103, or the decoding process may be shared between the client and the streaming server ex103. In addition, data of still images and videos captured not only by the camera ex113 but also by the camera ex116 can be sent to the streaming server ex103 through the computer ex111. The encoding process may be performed by the camera ex116, the computer ex111, or the streaming server ex103, or shared among them.

In addition, encoding and decoding processes can be performed by LSI ex500 generally included in each computer ex111 and equipment. The LSI ex500 can be configured with a single chip or multiple chips. Software for encoding and decoding video may be integrated into some type of recording medium such as CD-ROM, floppy disk, and hard disk readable by a computer ex111, and the encoding and decoding process may be performed using the software. Also, when the cellular phone ex114 is equipped with a camera, video data obtained by the camera can be transmitted. Video data is data encoded by the LSI ex500 included in the cellular phone ex114.

In addition, the streaming server ex103 may include servers and computers, and may disperse data and process, record, or distribute data on the dispersed data.

As described above, the client can receive and reproduce encoded data in the content providing system ex100. In other words, the client can receive and decode information sent by the user, and reproduce the decoded data in real time in the content providing system ex100, thereby enabling a user who does not have any specific rights and equipment to realize personal broadcasting.

In addition to the example of the content providing system ex100, at least one of the moving picture encoding device (image encoding device) and the moving picture decoding device (image decoding device) described in the respective embodiments may be in the digital broadcasting system shown in FIG. 14 Implemented in ex200. More specifically, the broadcasting station ex201 transmits or sends multiplexed data obtained by multiplexing audio data and the like onto video data to the broadcasting satellite ex202 via radio waves. Video data is data encoded by the moving picture encoding method described in each embodiment (ie, data encoded by an image encoding device according to an aspect of the present invention). When multiplexed data is received, the broadcast satellite ex202 transmits radio waves for broadcasting. Then, the home antenna ex204 having a satellite broadcast receiving function receives the radio waves. Next, devices such as a television (receiver) ex300 and a set-top box (STB) ex217 decode the received multiplexed data, and reproduce the decoded data (i.e., used as image decoding according to an aspect of the present invention) device).

In addition, the reader/recorder ex218 (i) reads and decodes the multiplexed data recorded on the recording medium ex215 such as DVD and BD, or (i) encodes the video signal in the recording medium ex215, and In some cases, data obtained by multiplexing an audio signal onto encoded data is written. The reader/recorder ex218 may include a moving picture decoding device or a moving picture encoding device as shown in various embodiments. In this case, the reproduced video signal is displayed on the monitor ex219, and can be reproduced by another device or system using the recording medium ex215 on which the multiplexed data is recorded. It is also possible to realize the motion picture decoding means in the set-top box ex217 connected to the cable ex203 of cable TV or the antenna ex204 of satellite and/or terrestrial broadcasting to display the video signal on the monitor ex219 of the television set ex300. The moving picture decoding device may be realized in the TV ex300 instead of the set-top box.

FIG. 15 shows a television (receiver) ex300 using the moving picture encoding method and the moving picture decoding method described in the respective embodiments. The television ex300 includes: a tuner ex301, which obtains or provides multiplexed data obtained by multiplexing audio data onto video data through an antenna ex204 for receiving broadcasts, a cable ex203, etc.; a modulation/demodulation unit ex302, which converts The received multiplexed data is demodulated or the data is modulated into multiplexed data to be supplied to the outside; and a multiplexing/demultiplexing unit ex303 which demultiplexes the modulated multiplexed data into video data and audio data , or multiplex the video data and audio data encoded by the signal processing unit ex306 into data.

The television set ex300 also includes: a signal processing unit ex306, which includes an audio signal processing unit ex304 and a video signal processing unit ex305 (which are used as a signal processing unit according to the present invention) for encoding audio data and video data and decoding the audio data and video data respectively. and an output unit ex309 including a speaker ex307 providing a decoded audio signal and a display unit ex308 (such as a display) displaying a decoded video signal. In addition, the television ex300 includes an interface unit ex317 including an operation input unit ex312 that receives an input of a user's operation. In addition, the TV ex300 includes: a control unit ex310 for controlling the overall components of the TV ex300, and a power circuit unit ex311 for supplying power to each component. In addition to the operation input unit ex312, the interface unit ex317 may include: a bridge ex313 connected to an external device such as a reader/writer ex218; a slot unit ex314 for attaching a recording medium ex216 such as an SD card; a drive ex315 for an external recording medium of a hard disk; and a modem ex316 connected to a telephone network. Herein, the recording medium ex216 may be electrically recorded using a nonvolatile/volatile semiconductor memory element for storage. The constituent elements of the television ex300 are connected to each other through a sync bus.

First, a configuration in which the television ex300 decodes multiplexed data obtained from the outside through the antenna ex204 and the like and reproduces the decoded data will be described. In the television ex300, when the user operates the remote controller ex220, etc., the multiplexing/demultiplexing unit ex303 performs multiplexing on the multiplexed data demodulated by the modulation/demodulation unit ex302 under the control of the control unit ex310 including the CPU. Demultiplexing. Also, using the decoding methods described in the respective embodiments, in the television ex300, the audio signal processing unit ex304 decodes the demultiplexed audio data, and the video signal processing unit ex305 decodes the demultiplexed video data. The output unit ex309 supplies decoded video signals and audio signals to the outside, respectively. When the output unit ex309 supplies video signals and audio signals, the signals can be temporarily stored in the buffers ex318 and ex319 and others so that the signals are reproduced in synchronization with each other. In addition, the television ex300 can read the multiplexed data not by broadcasting or the like, but from recording media ex215 and ex216 such as a magnetic disk, an optical disk, and an SD card. Next, a configuration in which the television ex300 encodes audio signals and video signals, and transmits data externally or writes data on a recording medium will be described. In the television ex300, when the user operates through the remote controller ex220 or the like, under the control of the control unit ex310 using the encoding method described in each embodiment, the audio signal processing unit ex304 encodes the audio signal, and the video signal processes A unit ex305 encodes a video signal. The multiplexing/demultiplexing unit ex303 multiplexes encoded video signals and audio signals, and supplies the resulting signals to the outside. When the multiplexing/demultiplexing unit ex303 multiplexes video signals and audio signals, the signals can be temporarily stored in the buffers ex320 and ex321 and others so that the signals are reproduced in synchronization with each other. Herein, there may be multiple buffers ex318, ex319, ex320 and ex321 as shown in the figure, or at least one buffer may be shared in the TV set ex300. In addition, data can be stored in a buffer, thereby avoiding, for example, system overflow and underflow between the modulation/demodulation unit ex302 and the multiplexing/demultiplexing unit ex303.

In addition, the television ex300 may include a configuration for receiving AV input from a microphone or a video camera (different from a configuration for obtaining audio and video data from a broadcast or a recording medium), and may encode the obtained data. Although in this specification, the television ex300 can encode, multiplex, and provide data to the outside, it may only be able to receive, decode, and provide data to the outside, but not be able to encode, multiplex, and provide data to the outside.

Also, when the reader/writer ex218 reads data from or writes data on the recording medium, one of the television ex300 and the reader/writer ex218 can decode or encode the multiplexed data, And the TV ex300 and the reader/recorder ex218 can share decoding or encoding.

As an example, FIG. 16 shows the configuration of the information reproducing/recording unit ex400 when reading data from or writing data on an optical disc. The information reproducing/recording unit ex400 includes constituent elements ex401, ex402, ex403, ex404, ex405, ex406, and ex407 to be described below. The optical head ex401 irradiates a laser spot in the recording surface of the recording medium ex215 which is an optical disc for writing information, and detects reflected light from the recording surface of the recording medium ex215 to read information. The modulation recording unit ex402 electrically drives the semiconductor laser included in the optical head ex401, and modulates the laser light according to recorded data. The reproduction demodulation unit ex403 amplifies a reproduction signal obtained by electrically detecting reflected light from the recording surface using a photodetector included in the optical head ex401, and separates the signal components recorded on the recording medium ex215 to demodulate the reproduced signal in order to reproduce the necessary information. The buffer ex404 temporarily holds information to be recorded on the recording medium ex215 and information reproduced from the recording medium ex215. The disk motor ex405 rotates the recording medium ex215. The servo control unit ex406 moves the optical head ex401 to a predetermined information track while controlling the rotational drive of the disc motor ex405 so as to follow the laser spot. The system control unit ex407 controls the entire information reproduction/recording unit ex400. The reading and writing process can be realized by the system control unit ex407 which uses various information stored in the buffer ex404 and generates and adds new information as necessary, and by modulating the recording unit ex402, reproducing the demodulating unit ex403 , and a servo control unit ex406 that operates in a coordinated manner while recording and reproducing information through the optical head ex401. For example, the system control unit ex407 includes a microprocessor, and executes processing by causing a computer to execute programs for reading and writing.

Although the optical head ex401 irradiates a laser spot in this specification, it can perform high-density recording using near-field light.

FIG. 17 shows a recording medium ex215 which is an optical disk. On the recording surface of the recording medium ex215, a guide groove is formed spirally, and the information track ex230 records address information indicating an absolute position on the disc in advance according to a change in the shape of the guide groove. The address information includes information for specifying the position of the recording block ex231 which is a unit for recording data. Reproducing the information track ex230 and reading the address information in the apparatus for recording and reproducing data can lead to determination of the position of the recording block. In addition, the recording medium ex215 includes a data recording area ex233, an inner peripheral area ex232, and an outer peripheral area ex234. The data recording area ex233 is an area for recording user data. The inner circumference area ex232 and the outer circumference area ex234 inside and outside the data recording area ex233 are used for specific purposes other than recording user data, respectively. The information reproducing/recording unit 400 reads and writes encoded audio, encoded video data from and on the data recording area ex233 of the recording medium ex215, or is obtained by multiplexing encoded audio and video data. of multiplexing data.

Although an optical disc having layers such as DVD and BD is described as an example in this specification, the optical disc is not limited thereto and may be an optical disc having a multi-layer structure and capable of being recorded on a portion other than the surface. In addition, the optical disc may have a structure for multi-dimensional recording/reproduction such as recording information using colors of light having different wavelengths in the same portion of the optical disc, and for recording information with different layers from various angles.

Also, in the digital broadcasting system ex200, a car ex210 having an antenna ex205 can receive data from a satellite ex202 etc., and reproduce video on a display device such as a car navigation system ex211 provided in the car ex210. Here, the configuration of the car navigation system ex211 will be, for example, a configuration including a GPS receiving unit from the configuration shown in FIG. 15 . The same is true for the configuration of the computer ex111, the cellular phone ex114, and the like.

FIG. 18A shows a cellular phone ex114 using the moving picture encoding method and moving picture decoding method described in the embodiment. The cellular phone ex114 includes: an antenna ex350 for transmitting and receiving radio waves through the base station ex110; a camera unit ex365 capable of capturing moving and still images; and a display data such as captured by the camera unit ex365 or received by the antenna ex350 decoded video) of the display unit ex358 (such as a liquid crystal display). The cellular phone ex114 also includes: a main body unit including an operation key unit ex366; an audio output unit ex357 (such as a speaker) for audio output; an audio input unit ex356 (such as a microphone) for audio input; or a memory unit ex367 of coded or decoded data of a still image, recorded audio, received video, still picture, e-mail, etc.; and a slot unit ex364 for storing data in the same manner as the memory unit ex367 interface unit for recording media.

Next, an example of the configuration of the cellular phone ex114 will be described with reference to FIG. 18B. In the cellular phone ex114, the main control unit ex360 designed to overall control the respective units of the main body including the display unit ex358 and the operation key unit ex366 is connected to the power supply circuit unit ex361, the operation input control unit ex362, Video signal processing unit ex355, camera interface unit ex363, liquid crystal display (LCD) control unit ex359, modulation/demodulation unit ex352, multiplexing/demultiplexing unit ex353, audio signal processing unit ex354, slot unit ex364, and memory unit ex367.

When the call end key or the power key is turned on by the user's operation, the power circuit unit ex361 supplies power from the battery pack to each unit, thereby activating the mobile phone ex114.

In the cellular phone ex114, under the control of the main control unit ex360 including CPU, ROM, and RAM, the audio signal processing unit ex354 converts audio signals collected by the audio input unit ex356 in voice call mode into digital audio signals. Then, the modulation/demodulation unit ex352 performs spread spectrum processing on the digital audio signal, and the transmission and reception unit ex351 performs digital-to-analog conversion and frequency conversion on the data to transmit the resultant data via the antenna ex350. Also, in the cellular phone ex114, the transmission and reception unit ex351 amplifies data received by the antenna ex350 in voice call mode, and performs frequency conversion and analog-to-digital conversion on the data. Then, the modulation/demodulation unit ex352 performs inverse spread spectrum processing on the data, and the audio signal processing unit ex354 converts it into an analog audio signal to output it via the audio output unit ex357.

Also, when sending an e-mail in the data communication mode, the text data of the e-mail input by operating the operation key unit ex366 of the main body is sent to the main control unit ex360 via the operation input control unit ex362. The main control unit ex360 causes the modulation/demodulation unit ex352 to perform spread spectrum processing on the text data, and the transmission and reception unit ex351 performs digital-to-analog conversion and frequency conversion on the generated data to transmit the data to the base station ex110 via the antenna ex350. When e-mail is received, processing substantially inverse to that for sending e-mail is performed on the received data, and the resulting data is supplied to the display unit ex358.

When transmitting video, still images, or video and audio in the data communication mode, the video signal processing unit ex355 compresses and encodes the video signal supplied from the camera unit ex365 using the moving picture encoding method shown in the respective embodiments (that is, function as an image coding device according to the scheme of the present invention), and sends the coded video data to the multiplexing/demultiplexing unit ex353. In contrast, the audio signal processing unit ex354 encodes audio signals collected by the audio input unit ex356 and sends the encoded audio data to the multiplexing/demultiplexing unit ex353 while the camera unit ex365 captures video, still images, etc.

The multiplexing/demultiplexing unit ex353 multiplexes the encoded video data supplied from the video signal processing unit ex355 and the encoded audio data supplied from the audio signal processing unit ex354 using a predetermined method. Then, the modulation/demodulation unit (modulation/demodulation circuit unit) ex352 performs spread spectrum processing on the multiplexed data, and the transmission and reception unit ex351 performs digital-to-analog conversion and frequency conversion on the data to transmit the generated data via the antenna ex350. data.

When receiving a video file etc. linked to a web page in data communication mode, or when receiving an e-mail with video and/or audio attached, in order to decode multiplexed data received via the antenna ex350, multiplexing/demultiplexing The multiplexed data is demultiplexed into a video data bit stream and an audio data bit stream by the unit ex353, and the encoded video data is supplied to the video signal processing unit ex355 and the encoded audio is supplied to the audio signal processing unit ex354 through the synchronous bus ex370 data. The video signal processing unit ex355 decodes the video signal using the moving picture decoding method corresponding to the moving picture encoding method shown in the respective embodiments (that is, serves as the image decoding means according to the aspect of the present invention), and then the display unit ex358 For example, video and still images included in video files linked to web pages are displayed via the LCD control unit ex359. In addition, the audio signal processing unit ex354 decodes audio signals, and the audio output unit ex357 supplies audio.

In addition, similar to the television ex300, a terminal such as a cellular phone ex114 may have 3 types of implementation configurations including not only (i) a transmitting and receiving terminal including both encoding means and decoding means, but also (ii) including only A transmitting terminal of encoding means and (iii) a receiving terminal comprising only decoding means. Although in this specification, the digital broadcasting system ex200 receives and transmits multiplexed data obtained by multiplexing audio data onto video data, the multiplexed data may be obtained not by multiplexing audio data but by multiplexing video-related The character data is obtained by multiplexing the video data, and may not be the multiplexed data but the video data itself.

Therefore, the moving picture encoding method and the moving picture decoding method in each embodiment can be used in any of the described devices and systems. Therefore, the advantages described in each embodiment can be obtained.

In addition, the present invention is not limited to these embodiments, and various modifications and revisions are possible without departing from the scope of the present invention.

(Example B)

Video data can be generated by switching as needed between (i) the moving picture encoding method or moving picture encoding apparatus shown in the respective embodiments and (ii) conforming to different standards such as MPEG-2, MPEG-4 AVC and VC-1) moving picture encoding method or moving picture encoding apparatus.

Herein, when a plurality of video data conforming to different standards is generated and then decoded, it is necessary to select a decoding method so as to conform to the different standards. However, since it is impossible to detect which standard each of a plurality of video data to be decoded conforms to, there is a problem that an appropriate decoding method cannot be selected.

In order to solve this problem, multiplexed data obtained by multiplexing audio data or the like onto video data has a structure including identification information indicating a standard to which the video data complies. Such a specific structure of multiplexed data of video data included in the moving picture encoding methods shown in the respective embodiments and generated by the moving picture encoding apparatus will be described below. The multiplexed data is a digital stream in the MPEG-2 transport stream format.

Fig. 19 shows the structure of multiplexed data. As shown in FIG. 19, the multiplexed data may be obtained by multiplexing at least one of a video stream, an audio stream, a presentation graphics stream (PG), and an interactive graphics stream. The video stream represents the main video and the secondary video of the movie, the audio stream (IG) represents the main audio part and the secondary audio part to be mixed with the main audio part, and the presentation graphics stream represents the subtitle of the movie. In this article, the primary video is the normal video to be displayed on the screen, and the secondary video is the video to be displayed on a smaller window within the primary video. In addition, the interactive graphics flow represents an interactive screen generated by arranging GUI components on the screen. The video stream is encoded in the moving picture encoding method shown in the various embodiments or by a moving picture encoding device, or by a moving picture encoding method conforming to conventional standards such as MPEG-2, MPEG-4 AVC, and VC-1 or encoded by a motion picture encoding device. Audio streams are encoded according to standards such as Dolby AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD, and Linear PCM.

Each stream included in the multiplexed data is identified by a PID. For example, 0x1011 is assigned to the video stream for video for movies, 0x1100 to 0x111F are assigned to audio streams, 0x1200 to 0x121F are assigned to presentation graphics streams, 0x1400 to 0x141F are assigned to interactive graphics streams, and 0x1B00 to 0x1B1F are assigned to A video stream for secondary video of a movie, and 0x1A00 to 0x1A1F are allocated to an audio stream for secondary audio to be mixed with primary audio.

Fig. 20 schematically shows how data is multiplexed. First, the video stream ex235 composed of video frames and the audio stream ex238 composed of audio frames are respectively converted into a stream of PES packets ex236 and a stream of PES packets ex239, and further converted into TS packets ex237 and TS packets ex240. Similarly, the data of the presentation graphics stream ex241 and the data of the interactive graphics stream ex244 are converted into a stream of PES packets ex242 and a stream of PES packets ex245, respectively, and further converted into TS packets ex243 and TS packets ex246. These TS packets are multiplexed into a stream to obtain multiplexed data ex247.

Figure 21 shows in more detail how a video stream is stored in a stream of PES packets. The first column in Fig. 21 shows the flow of video frames in the video stream. The second column shows the flow of PES packets. As indicated by the arrows labeled yy1, yy2, yy3, and yy4 in Figure 21, the video stream is divided into pictures as I-pictures, B-pictures, and P-pictures, each of which is a video presentation unit , and these pictures are stored in the payload in each of the PES packets. Each PES packet has a PES header, and the PES header stores a presentation time stamp (PTS) indicating a display time of a picture, and a decoding time stamp (DTS) indicating a decoding time of a picture.

Fig. 22 shows the format of TS packets finally written on the multiplexed data. Each TS packet is a 188-byte fixed-length packet including a 4-byte TS header with information such as a PID for identifying a stream and a 184-byte TS payload for storing data. PES packets are divided and stored in TS payloads respectively. When using a BD ROM, each TS packet is given a TP_Extra_Header of 4 bytes, resulting in a source packet of 192 bytes. Write source packets to multiplexed data. TP_Extra_Header stores information such as Arrival_Time_Stamp (ATS). ATS shows the transmission start time of transmitting each TS packet to the PID filter. Source packets are arranged in the multiplexed data as shown at the bottom of FIG. 22 . The number incremented from the header of the multiplexed data is called a source packet number (SPN).

Each TS packet included in the multiplexed data includes not only streams of audio, video, subtitles, etc., but also a Program Association Table (PAT), Program Map Table (PMT), and Program Clock Reference (PCR). The PAT shows what is indicated by the PID in the PMT used in the multiplexed data, and the PID itself of the PAT is registered as zero. The PMT stores PIDs of streams of audio, video, subtitles, etc. included in the multiplexed data and attribute information of these streams corresponding to the PIDs. The PMT also has various descriptors related to the multiplexed data. These descriptors have information such as copy control information showing whether copying of multiplexed data is permitted or not. The PCR stores STC time information corresponding to the ATS that shows when the PCR packet is transmitted to the decoder, so that the Arrival Time Clock (ATC) (which is the time axis of the ATS) and the System Time Clock (STC) (which is the PTS and DTS time axis).

Fig. 23 shows the data structure of PMT in detail. The PMT head is arranged on top of the PMT. The PMT header describes the length and the like of data included in the PMT. A plurality of descriptors related to multiplexed data are arranged after the PMT header. Information such as copy control information is described in the descriptor. After the descriptor, pieces of stream information related to the stream included in the multiplexed data are arranged. Each piece of stream information includes stream descriptors respectively describing information such as a stream type for identifying a compression codec of the stream, a stream PID, and stream attribute information such as frame rate or aspect ratio. The number of stream descriptors is equal to the number of streams in the multiplexed data.

When multiplexed data is recorded on a recording medium or the like, it is recorded together with the multiplexed data information file.

Each multiplexed data information file is management information of multiplexed data as shown in FIG. 24 . Multiplexing data information files correspond to multiplexing data one by one, and each file includes multiplexing data information, stream attribute information and entry mapping.

As shown in FIG. 24, the multiplexed data information includes a system rate, a reproduction start time, and a reproduction end time. The system rate indicates the maximum transmission rate at which the system target decoder to be described below transmits the multiplexed data to the PID filter. The interval of ATS included in the multiplexed data is set not higher than the system rate. The reproduction start time indicates the PTS in the video frame at the head of the multiplexed data. An interval of one frame is added to the PTS in the video frame at the end of the multiplexed data, and the PTS is set as the reproduction end time.

As shown in FIG. 25, for each PID of each stream included in the multiplexed data, a piece of attribute information is registered in the stream attribute information. Each piece of attribute information has different information depending on whether the corresponding stream is a video stream, an audio stream, a display graphics stream, or an interactive graphics stream. Each piece of video stream attribute information carries information including which compression codec is used to compress the video stream, and the resolution, aspect ratio, and frame rate of each piece of picture data included in the video stream. Each piece of audio stream attribute information carries information including which compression codec is used to compress the audio stream, how many channels are included in the audio stream, which language the audio stream supports, and how high the sampling frequency is. The video stream attribute information and the audio stream attribute information are used to initialize the decoder before the player plays back the information.

In the embodiments herein, the multiplexed data to be used is of the stream type included in the PMT. In addition, when the multiplexed data is recorded on the recording medium, the video stream attribute information included in the multiplexed data information is used. More specifically, the moving picture encoding method or moving picture encoding apparatus described in each embodiment includes: for assigning an indication to the stream type or video stream attribute information included in the PMT by the moving picture encoding method or motion picture encoding method in each embodiment A step or unit of unique information for video data generated by a picture encoding device. With this configuration, video data generated by the moving picture encoding method or moving picture encoding device described in the respective embodiments can be distinguished from video data conforming to another standard.

In addition, FIG. 26 shows the steps of the motion picture decoding method according to the embodiment of this document. In step exS100, the stream type included in the PMT or the video stream attribute information included in the multiplexed data information is obtained from the multiplexed data. Next, in step exS101, it is determined whether stream type or video stream attribute information indicates that multiplexed data is generated by the moving picture encoding method or moving picture encoding apparatus in each embodiment. When it is determined that the stream type or video stream attribute information indicates that the multiplexed data is generated by the moving picture encoding method or moving picture encoding device in each embodiment, then in step exS102, decoding is performed by the moving picture decoding method in each embodiment . Also, when the stream type or video stream attribute information indicates compliance with conventional standards such as MPEG-2, MPEG-4 AVC, and VC-1, then in step exS103, decoding is performed by a moving picture decoding method conforming to conventional standards.

Thus, assigning a new unique value to stream type or video stream attribute information enables determination of whether the moving picture decoding method or moving picture decoding apparatus described in each embodiment can perform decoding. Even when multiplexed data conforming to different standards is input, an appropriate decoding method or device can be selected. Therefore, the information can be decoded without any errors. In addition, the moving picture encoding method or device, or the moving picture decoding method or device in the embodiments herein can be used in the above-mentioned devices and systems.

(Example C)

Each of the moving picture encoding method, moving picture encoding device, moving picture decoding method, and moving picture decoding device in the respective embodiments is generally implemented in the form of an integrated circuit or a large scale integration (LSI) circuit. As an example of LSI, FIG. 27 shows a configuration of an LSI ex500 made into one chip. The LSI ex500 includes elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509 to be described below, and these elements are connected to each other through a bus ex510. When the power supply circuit unit ex505 is turned on, the power supply circuit unit ex505 is activated by supplying power to each element.

For example, when performing encoding, under the control of the control unit ex501 including the CPU ex502, the memory controller ex503, the stream controller ex504, and the drive frequency control unit ex512, the LSI ex500 receives AV signals from the microphone ex117, the video camera ex113, etc. through the AV IO ex509 . The received AV signal is temporarily stored in the external memory ex511 (such as SDRAM). Under the control of the control unit ex501, the stored data is divided into data sections according to the amount of processing and the speed to be transmitted to the signal processing unit ex507. Then, the signal processing unit ex507 encodes the audio signal and/or the video signal. Herein, the encoding of the video signal is the encoding described in the respective embodiments. In addition, the signal processing unit ex507 sometimes multiplexes encoded audio data and encoded video data, and the stream IO ex506 supplies the multiplexed data to the outside. The provided multiplexed data is sent to the base station ex107 or written on the recording medium ex215. When the data sets are multiplexed, the data should be temporarily stored in the buffer ex508 so that the data sets are synchronized with each other.

Although the memory ex511 is an element external to the LSI ex500, it may be included in the LSI ex500. The buffer ex508 is not limited to one buffer but may be composed of a plurality of buffers. In addition, LSI ex500 can be made into one chip or multiple chips.

In addition, although the control unit ex501 includes a CPU ex502, a memory controller ex503, a stream controller ex504, and a driving frequency control unit ex512, the configuration of the control unit ex501 is not limited thereto. For example, the signal processing unit ex507 may also include a CPU. Including another CPU in the signal processing unit ex507 can increase the processing speed. Also, as another example, the CPU ex502 may function as or be a part of the signal processing unit ex507, and may include an audio signal processing unit, for example. In this case, the control unit ex501 includes a signal processing unit ex507 or a CPU ex502 including a part of the signal processing unit ex507.

The name used in this article is LSI, but it can also be called IC, system LSI, super LSI, and ultra-large-scale LSI depending on the degree of integration.

In addition, the method of achieving integration is not limited to LSI, and integration can also be achieved by special circuits or general-purpose processors or the like. A field programmable gate array (FPGA) that can be programmed after the LSI is manufactured, or a reconfigurable processor that allows reconfiguration of connection or configuration of the LSI can be used for the same purpose.

In the future, with the advancement of semiconductor technology, completely new technologies may replace LSI. Functional blocks can be integrated using such techniques. The invention has potential application in biotechnology.

(Example D)

When decoding video data generated in the moving picture encoding method described in the respective embodiments or by the moving picture encoding apparatus, video data conforming to conventional standards such as MPEG-2, MPEG-4 AVC, and VC-1 Compared to decoding, it is likely to increase the amount of processing. Therefore, it is necessary to set the LSI ex500 to a higher drive frequency than that of the CPU ex502 used for decoding video data conforming to the conventional standard. However, when the driving frequency is set high, there is a problem of increased power consumption.

In order to solve this problem, moving picture decoding apparatuses such as the television ex300 and LSI ex500 are configured to determine a standard to which video data complies, and switch between driving frequencies according to the determined standard. Fig. 28 shows a configuration ex800 in this embodiment. The driving frequency switching unit ex803 sets the driving frequency to a higher driving frequency when video data is generated by the moving picture coding method or moving picture coding device described in the respective embodiments. Then, the driving frequency switching unit ex803 instructs the decoding processing unit ex801 executing the moving picture decoding method described in each embodiment to decode video data. When the video data conforms to the conventional standard, the driving frequency switching unit ex803 sets the driving frequency to a lower driving frequency than that of the video data generated by the moving picture encoding method or moving picture encoding apparatus described in the respective embodiments. Then, the driving frequency switching unit ex803 instructs the decoding processing unit ex802 conforming to the conventional standard to decode the video data.

More specifically, the driving frequency switching unit ex803 includes the CPU ex502 and the driving frequency control unit ex512 in FIG. 27 . Herein, each of the decoding processing unit ex801 executing the moving picture decoding method described in the respective embodiments and the decoding processing unit ex802 conforming to the conventional standard corresponds to the signal processing unit ex507 in FIG. 27 . The CPU ex502 determines which standards the video data conforms to. Then, the driving frequency control unit ex512 determines the driving frequency based on the signal from the CPU ex502. Also, the signal processing unit ex507 decodes video data based on a signal from the CPU ex502. For example, the identification information described in Embodiment B is likely to be used to identify video data. The identification information is not limited to that described in Embodiment B, but may be arbitrary information as long as the information indicates the standard to which the video data complies. For example, when the standard to which the video data complies can be determined based on an external signal used to determine that the video data is for use in a television set or a magnetic disk, etc., the determination may be made based on such an external signal. In addition, the CPU ex502 selects the driving frequency based on, for example, a lookup table in which the standard of video data is associated with the driving frequency as shown in FIG. 30 . The drive frequency can be selected by storing a lookup table in the buffer ex508 and in the internal memory of the LSI, and referring to the lookup table by the CPU ex502.

Figure 29 illustrates steps for performing the methods in the embodiments herein. First, in step exS200, the signal processing unit ex507 obtains identification information from the multiplexed data. Next, in step exS201, the CPU ex502 determines based on the identification information whether the video data is generated by the encoding method and encoding device described in the respective embodiments. When the video data is generated by the moving picture coding method and the moving picture coding apparatus described in the respective embodiments, in step exS202, the CPU ex502 sends to the driving frequency control unit ex512 a signal for setting the driving frequency to a higher driving frequency. Signal. Then, the driving frequency control unit ex512 sets the driving frequency to a higher driving frequency. On the other hand, when the identification information indicates that the video data conforms to conventional standards such as MPEG-2, MPEG-4 AVC, and VC-1, in step exS203, the CPU ex502 sends a signal for setting the driving frequency to the driving frequency control unit ex512. Signals with lower drive frequencies. Then, the drive frequency control unit ex512 sets the drive frequency to a lower drive frequency than the case where video data is generated by the moving picture encoding method and the moving picture encoding apparatus described in the respective embodiments.

In addition, together with the switching of the driving frequency, it is possible to enhance the power saving effect by changing the voltage applied to the LSI ex500 or a device including the LSI ex500. For example, when the driving frequency is set low, the voltage applied to LSI ex500 or a device including LSI ex500 is likely to be set to a lower voltage than the voltage in the case where the driving frequency is set high.

Also, as for the method for setting the driving frequency, when the processing amount for decoding is large, the driving frequency can be set high, and when the processing amount for decoding is small, the driving frequency can be set low . Therefore, setting methods are not limited to those described above. For example, when compared with the processing amount for decoding video data generated by the moving picture encoding method and moving picture encoding apparatus described in the respective embodiments, the amount of processing for decoding video data conforming to MPEG-4 AVC When larger, the drive frequency is likely to be set in the reverse order of the above settings.

In addition, the method for setting the driving frequency is not limited to the method for setting the driving frequency lower. For example, when the identification information indicates that video data is generated by the moving picture coding method and moving picture coding device described in the respective embodiments, the voltage applied to LSI ex500 or a device including LSI ex500 is likely to be set high. When the identification information indicates that video data conforms to conventional standards such as MPEG-2, MPEG-4 AVC, and VC-1, the voltage applied to LSI ex500 or a device including LSI ex500 is likely to be set lower. As another example, when the identification information indicates that the video data is generated by the moving picture encoding method and moving picture encoding apparatus described in the various embodiments, then the driving of the CPU ex502 probably does not need to be suspended. When the identification information indicates that video data conforms to conventional standards such as MPEG-2, MPEG-4AVC, and VC-1, the drive of the CPU ex502 is likely to be suspended at a given time because the CPU ex502 has extra processing capacity. Even when the identification information indicates that the video data is generated by the moving picture encoding method and moving picture encoding apparatus described in the respective embodiments, in the case where the CPU ex502 has an extra processing capacity, the driving of the CPU ex502 is likely to be performed at a given time pause. In such a case, the pause time is likely to be set shorter than in the case where the identification information indicates that video data conforms to conventional standards such as MPEG-2, MPEG-4 AVC, and VC-1.

Therefore, power saving can be improved by switching between driving frequencies according to the standards to which the video data complies. In addition, when the LSI ex500 or a device including the LSI ex500 is driven with a battery, the battery life can be extended with a power saving effect.

(Embodiment E)

There are cases where a plurality of video data conforming to different standards are supplied to devices and systems such as televisions and cellular phones. In order to be able to decode multiple video data conforming to different standards, the signal processing unit ex507 of LSI ex500 needs to conform to different standards. However, the problems of circuit scale increase and cost increase of the LSI ex500 arise with corresponding use of the signal processing unit ex507 conforming to each standard.

In order to solve this problem, the following configuration is conceived: a decoding processing unit for realizing the moving picture decoding method described in each embodiment and a decoding processing unit part conforming to conventional standards such as MPEG-2, MPEG-4 AVC, and VC-1 shared. ex900 in Fig. 31A shows an example of this configuration. For example, the motion picture decoding methods described in the various embodiments share some processing details in common with MPEG-4 AVC compliant motion picture decoding methods, such as entropy coding, inverse quantization, deblocking filtering, and motion compensated prediction. Processing details to be shared likely include use of the MPEG-4 AVC-compliant decoding processing unit ex902. Instead, the dedicated decoding processing unit ex901 is likely to be used for other processing specific to the scheme of the present invention. For example, since the solution of the present invention is particularly characterized by inverse quantization, a dedicated decoding processing unit ex901 is used for inverse quantization. Otherwise, the decoding processing unit is likely to be shared for one or all of entropy decoding, deblocking filtering and motion compensation. A decoding processing unit for implementing the moving picture decoding method described in each embodiment may be shared for processing to be shared, and a dedicated decoding processing unit may be used for processing specific to a dedicated decoding processing unit for MPEG-4 AVC.

In addition, ex1000 in FIG. 31B shows another example of partial sharing processing. This example uses a configuration including: a dedicated decoding processing unit ex1001 supporting processing specific to a certain scheme of the present invention, a dedicated decoding processing unit ex1002 supporting processing specific to another conventional standard, and a dedicated decoding processing unit ex1002 supporting processing in a scheme according to the present invention The decoding processing unit ex1003 of processing shared between the moving picture decoding method and the conventional moving picture decoding method. Herein, the dedicated decoding processing units ex1001 and ex1002 are not necessarily dedicated to processing according to the scheme of the present invention and processing of conventional standards, respectively, and may be decoding processing units capable of realizing general processing. In addition, the configuration of the embodiments herein can be realized by LSI ex500.

Therefore, by sharing a decoding processing unit for processing to be shared between the moving picture decoding method according to the scheme of the present invention and the moving picture decoding method conforming to the conventional standard, it is possible to reduce the circuit scale and cost of the LSI.

It will be apparent to those skilled in the art that various changes and/or modifications may be made to the invention shown in the specific embodiments without departing from the spirit or scope of the invention as broadly described. Accordingly, the embodiments herein should be considered in all respects as illustrative rather than restrictive.

Industrial Applicability

The present invention is applicable to an encoding device that encodes audio, still images, and video, and a decoding device that decodes data encoded by the encoding device. For example, the present invention is applicable to various audiovisual equipment such as audio equipment, cellular phones, digital video cameras, BD recorders, and digital televisions.

Claims (24)

1. A method of encoding video into an encoded video bitstream using temporal motion vector prediction, the method comprising: determining the value of a flag indicating whether temporal motion vector prediction is used or not used for inter-picture prediction of a sub-picture unit of a picture; writing a flag with the value to the header of the sub-picture unit or the header of the picture; and Wherein, if the flag indicates that temporal motion vector prediction is used, the method further includes: creating a first list of motion vector predictors comprising a plurality of motion vector predictors comprising at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture; selecting a motion vector predictor from the first list for a prediction unit in the sub-picture unit; and A first parameter is written to the encoded video bitstream to indicate a motion vector predictor selected from the first list. 2. The method of claim 1, wherein if the flag indicates that temporal motion vector prediction is not used, the method further comprises: creating a second list of motion vector predictors comprising a plurality of motion vector predictors without any temporal motion vector predictors; selecting a motion vector predictor from the second list for a prediction unit in the sub-picture unit; and A second parameter is written to the encoded video bitstream to indicate a motion vector predictor selected from the second list. 3. The method according to claim 1 or 2, wherein the value of the flag is determined based on the temporal layer of the picture. 4. The method of claim 3, wherein if it is determined that the temporal layer of the picture is the lowest layer or base layer, the value of the flag is set to indicate that temporal motion vector prediction is not used, otherwise, the value of the flag is set The value of the flag to indicate that temporal motion vector prediction is used. 5. The method of claim 1 or 2, wherein the value of the flag is determined based on a picture order count (POC) value of the picture. 6. The method of claim 5, wherein if it is determined that the POC value of the picture is greater than any POC value of a reference picture in a decoder picture buffer (DPB), the value of the flag is set to indicate Temporal motion vector prediction is not used, otherwise the value of the flag is set to indicate that temporal motion vector prediction is used. 7. The method according to claim 1 or 2, wherein the value of the flag is determined based on a sub-picture unit type of an inter-picture sub-picture unit in the picture. 8. The method of claim 7, wherein if the sub-picture unit type is a predictive (P) type, the value of the flag is set to indicate that temporal motion vector prediction is not used, otherwise, the flag is set value to indicate that temporal motion vector prediction is used. 9. The method of claim 1 or 2, wherein the value of the flag is determined based on whether the picture containing the sub-picture unit is a Random Access Point (RAP) picture. 10. The method of claim 9, wherein if the picture is a RAP picture and the sub-picture unit belongs to a non-base layer of the picture, setting the value of the flag to indicate that temporal motion vector prediction is not used, Otherwise, the value of the flag is set to indicate that temporal motion vector prediction is used. 11. The method according to any one of claims 1 to 10, wherein the flag is written in a header of the sub-picture unit. 12. The method according to any one of claims 1 to 11, wherein the method further comprises: writing one or more parameters into the header of the sub-picture unit, so as to specify The order of the reference pictures in one or more reference picture lists for inter-picture prediction. 13. The method according to any one of claims 1 to 12, wherein the method further comprises: performing motion-compensated inter-picture prediction using the selected motion vector predictor to generate the prediction unit; subtracting the prediction unit from the original sample block to produce a residual sample block; and The remaining sample blocks corresponding to prediction units are encoded into the encoded video bitstream. 14. A method according to any one of claims 1 to 13, wherein the second list includes one less motion vector predictor than the first list, and in addition to the temporal motion vector predictors, all The motion vector predictors of the first list and the second list are the same. 15. The method according to any one of claims 1 to 14, wherein the first parameter and the second parameter are represented in the encoded video bitstream using different predetermined bit representations . 16. A method according to any one of claims 1 to 13, wherein the first list and the second list comprise the same predetermined number of motion vector predictors, and the second list comprises not are present in the first list and are motion vector predictors that were derived without using motion vectors from any reference pictures. 17. The method according to any one of claims 1 to 16, wherein the flag is used to indicate that for the inter-picture prediction for a sub-picture unit independent of other sub-picture units in the picture, use Again temporal motion vector prediction is not used. 18. The method of any one of claims 1 to 17, wherein the sub-picture unit is a slice of a picture. 19. A method of decoding an encoded video bitstream using temporal motion vector prediction, the method comprising: parsing flags from headers of sub-picture units or headers of pictures of the encoded video; and determining whether the flag indicates whether temporal motion vector prediction is used or not used; Wherein, if the flag indicates that temporal motion vector prediction is used, the method further includes: creating a first list of motion vector predictors comprising a plurality of motion vector predictors comprising at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture; A first parameter from the encoded video bitstream is parsed, the first parameter indicating a motion vector predictor selected from the first list for a prediction unit in the sub-picture unit. 20. The method of claim 19, wherein if the flag indicates that temporal motion vector prediction is not used, the method further comprises: creating a second list of motion vector predictors comprising a plurality of motion vector predictors without any temporal motion vector predictors; and A second parameter from the encoded video bitstream is parsed, the second parameter indicating a motion vector predictor selected from the second list for a prediction unit in the sub-picture unit. 21. An apparatus for encoding video into an encoded video bitstream using temporal motion vector prediction, the apparatus comprising: a control unit operable to: determine a value of a flag indicating whether temporal motion vector prediction is used or not used for inter-picture prediction of a sub-picture unit of a picture; a writing unit operable to: write a flag having said value into a header of said sub-picture unit or a header of said picture; motion vector prediction unit; and an inter-picture prediction unit configured to: perform inter-picture prediction based on a motion vector predictor selected from the motion vector prediction unit, Wherein the motion vector prediction unit is configured to: receive the flag, and based on the flag being a first value, the motion vector prediction unit is operable to: create a motion vector prediction comprising a plurality of motion vector predictors A first list of motion vector predictors comprising: at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture, and for a prediction unit in the sub-picture unit , select a motion vector predictor from said first list; and The writing unit is further operable to: write a first parameter to the encoded video bitstream indicating the selected motion vector predictor from the first list. 22. The apparatus of claim 21 , when the flag is a second value, the motion vector prediction unit is operable to: create a motion vector predictor comprising a plurality of motion vector predictors without any temporal motion vector predictors a second list of vector predictors; and for a prediction unit in the sub-picture unit, select a motion vector predictor from the first list; and The writing unit is further operable to write a second parameter to the encoded video bitstream indicating the selected motion vector predictor from the second list. 23. An apparatus for decoding an encoded video bitstream using temporal motion vector prediction, the apparatus comprising: a parsing unit operable to: parse a flag from a header of a sub-picture unit or a header of a picture of the encoded video; and determine whether the flag indicates whether temporal motion vector prediction is used or not used; motion vector prediction unit; and an inter-picture prediction unit configured to: perform inter-picture prediction based on a motion vector predictor selected from the motion vector prediction unit; Wherein the motion vector prediction unit is configured to: receive the flag, and based on the flag being a first value, the motion vector prediction unit is operable to: create a motion vector prediction comprising a plurality of motion vector predictors a first list of symbols, the plurality of motion vector predictors comprising: at least one temporal motion vector predictor derived from at least one motion vector from a collocated reference picture; and The parsing unit is further operable to: parse a first parameter from the encoded video bitstream, the first parameter indicating a prediction unit from the first list for a prediction unit in the sub-picture unit The selected motion vector predictor. 24. The apparatus of claim 23, wherein, when the flag is a second value, the motion vector prediction unit is operable to: create a motion vector predictor comprising a plurality of motion vector predictors without any temporal motion vector predictors a second list of motion vector predictors; and The parsing unit is further operable to parse a second parameter from the encoded video bitstream, the second parameter indicating a prediction unit from the second list for a prediction unit in the sub-picture unit. The selected motion vector predictor.