CN101888555B - Method and apparatus for decoding/encoding a video signal - Google Patents

Abstract

The present invention relates to methods and devices for decoding/encoding video signals. A method for decoding a video signal is disclosed, and the invention includes: decoding a bit stream of a first layer; obtaining flag information indicating whether to perform inter-layer prediction on a current block of a second layer; obtaining at least one offset based on the flag information information, the offset information represents the position difference between the up-sampled image of the first layer used for the inter-layer prediction and the current image of the second layer; and by using the at least one offset information, to The reference image of the first layer is upsampled.

Description

Method and device for decoding/encoding video signal

This application is a divisional application of the patent application with the application date of September 8, 2008 and the application number of 200780008172.1 entitled "Method and Device for Decoding/Encoding Video Signal".

technical field

The invention relates to a scheme for encoding/decoding video signals.

Background technique

Generally speaking, compression encoding/decoding refers to a series of signal processing techniques that transmit digitized information through communication lines, or store it in a storage medium in a suitable format. The objects of compression coding include audio, video, text, etc., especially the scheme of performing compression coding with video as the object is called video sequence compression. In general, video sequences are characterized by spatial redundancy and temporal redundancy.

In particular, the Scalable Video Coded bitstream can be partially and selectively decoded. For example, a low-complexity decoder can decode the base layer and extract a low data rate bitstream for transmission over a network with limited capacity. In order to further gradually generate high-resolution images, it is necessary to improve the image quality in stages.

Contents of the invention

technical problem

In particular, the Scalable Video Coded bitstream can be partially and selectively decoded. For example, a low-complexity decoder can decode the base layer and extract a low data rate bitstream for transmission over a network with limited capacity. In order to further gradually generate high-resolution images, it is necessary to improve the image quality in stages.

technical means

Accordingly, the present invention is directed to a scheme for encoding/decoding video signals that substantially obviates one or more problems due to limitations and disadvantages of the related art.

An object of the present invention is to provide a method for improving encoding/decoding efficiency when encoding/decoding video signals.

Another object of the present invention is to provide a method for minimizing the transmission of information related to inter-layer prediction in case a region in an enhancement layer does not correspond to a reference layer.

Another object of the present invention is to provide a method for minimizing transmission of information related to inter-layer prediction by confirming configuration information of a scalable video-coded bitstream.

Another object of the present invention is to provide a method of minimizing transmission of information related to inter-layer prediction by confirming information indicating whether inter-layer prediction is performed.

An object of the present invention is to provide a method of minimizing transmission of information related to inter-layer prediction by confirming quality identification information.

Another object of the present invention is to provide a method for improving encoding/decoding efficiency of a video signal by defining information representing slice boundary processing.

A further object of the present invention is to provide a method for improving encoding/decoding efficiency of a video signal by confirming configuration information of a scalable video encoded bitstream at an appropriate position.

Beneficial effect

Accordingly, the present invention provides the following effects or advantages.

First, the present invention checks whether a current block of an enhancement layer can be predicted using inter-layer prediction. In case the current block of the above-mentioned enhancement layer is not predicted by using inter-layer prediction, there is no need to transmit encoding/decoding information for inter-layer prediction. Therefore, the present invention can improve encoding/decoding efficiency. Second, the transmission of information related to inter-layer prediction is minimized by confirming the configuration information of the scalable video coded bitstream at an appropriate position. For example, by identifying information indicating whether to perform inter-layer prediction and/or quality identification information, transmission of information related to inter-layer prediction can be minimized. Furthermore, the present invention can enable parallel processing by defining information representing processing of slice boundaries. Encoding/decoding efficiency of video signals can be significantly improved by applying the various methods explained above.

Description of drawings

The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention principle.

In the attached picture:

1 is a schematic block diagram of a scalable video coding system according to the present invention;

FIG. 2 and FIG. 3 are respectively a structural diagram of configuration information of a scalable sequence that can be added to a scalable video coding bitstream and an image for illustrating the configuration information according to an embodiment of the present invention;

4 is a diagram of a cropping relationship between a sampled base layer and an enhancement layer according to an embodiment of the present invention;

5 and 6 are diagrams of syntax related to macroblock prediction and sub-macroblock prediction through inter-layer prediction according to an embodiment of the present invention;

7 is a diagram of syntax related to residual prediction through inter-layer prediction according to one embodiment of the present invention;

FIG. 8 is a structural diagram of syntax for performing deblocking filtering according to whether inter-layer prediction is performed according to an embodiment of the present invention;

FIG. 9 is a structural diagram of the syntax of the offset information indicating the position difference between the upsampled reference image and the current image according to whether inter-layer prediction is performed according to an embodiment of the present invention;

FIG. 10 is a structural diagram of a syntax for obtaining flag information indicating whether to restrict the use of intra-blocks (intra-blocks) in a reference layer according to whether inter-layer prediction is performed according to an embodiment of the present invention;

FIG. 11 is a structural diagram of syntax for obtaining adaptive prediction information according to whether to perform inter-layer prediction according to an embodiment of the present invention.

best practice

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be learned from the description, or may be obtained by practice of the invention. The objectives and other advantages of the invention will be realized and obtained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

To achieve these and other advantages and in accordance with the objects of the present invention as included and broadly described, a method of decoding a video signal according to the present invention comprises: decoding a bitstream of a first layer; obtaining an indication whether a current block of a second layer flag information for performing inter-layer prediction; at least one offset information is obtained based on the flag information, and the offset information represents the up-sampled image of the first layer and the current image of the second layer used for the inter-layer prediction a position difference between images; and upsampling the reference image of the first layer by using the at least one offset information.

Preferably, the method further comprises obtaining information indicating whether the parameter used for the upsampling exists in a corresponding region of the second layer, wherein based on the indicating whether the parameter used for the upsampling exists in a region of the second layer The information of the corresponding area is obtained by obtaining the at least one offset information.

More preferably, the method further comprises obtaining information about the phase shift of the chrominance signal based on the information indicating whether the parameter for the upsampling exists in the corresponding area of the second layer, wherein, by using the information about Information about the phase shift of the chroma signal, which the first layer is upsampled.

Here, the information on the phase shift of the chroma signal is obtained based on the information representing the chroma format. Also, the information on the phase shift of the chrominance signal includes phase shift information in the horizontal direction and phase shift information in the vertical direction.

More preferably, the method further includes obtaining quality identification information for identifying the quality of the current block of the second layer, wherein the offset information is obtained based on the quality identification information.

Preferably, the second layer and the first layer have different screen ratios or spatial resolutions, and the first layer comes from the same video signal of the second layer.

In order to further achieve these and other advantages and according to the purpose of the present invention, an apparatus for decoding a video signal includes: a base layer decoding unit for decoding a bitstream of the first layer; a first header information obtaining unit for obtaining flag information indicating whether to perform inter-layer prediction on the current block of the second layer; the second header information obtaining unit is configured to obtain at least one offset information based on the flag information, and the offset information indicates that the inter-layer prediction is used for the inter-layer prediction The position difference between the up-sampled image of the first layer and the current image of the second layer; and an up-sampling unit, configured to perform the reference image of the first layer by using the at least one offset information upsampling.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

Embodiment of the invention

Reference will now be made in detail with reference to the preferred embodiments of the invention, examples of which are illustrated in the accompanying drawings.

First, the compression encoding/decoding of video signal data considers spatial redundancy, temporal redundancy, scalable redundancy, and inter-view redundancy. Compression encoding/decoding considering scalable redundancy is only one embodiment of the present invention. However, the technical concept of the present invention is applicable to temporal redundancy, spatial redundancy, inter-view redundancy, and the like. Furthermore, the "encoding/decoding" referred to in this specification includes two concepts of encoding and decoding, which can be flexibly interpreted according to the technical concept and technical scope of the present invention.

In the bit sequence configuration of the video signal, there is an independent layer structure called NAL (Network Abstraction Layer, Network Abstraction Layer), which is located between the VCL (Video Code Layer, Video Coding Layer) and the transmission and Between subordinate systems that store coded information. The output generated by the encoding process is VCL data, which is mapped into NAL units before transmission or storage. Each NAL unit contains compressed video data or data RBSP (Raw Byte Sequence Payload, original byte sequence payload, result data of motion image compression) corresponding to header information.

The NAL unit basically includes two parts, the NAL header and the RBSP. The NAL header includes flag information (nal_ref_idc) indicating whether a slice serving as a reference picture of the NAL unit is included, and information indicating the type of the NAL unit (nal_unit_type). Store compressed raw data in RBSP. And, in order to express the length of RBSP as a multiple of 8 bits, RBSP trailing bits (RBSP trailing bits) are added at the end of RBSP. The types of NAL units include IDR (Instantaneous Decoding Refresh, instant decoding refresh) image, SPS (SequenceParameter Set, sequence parameter set), PPS (Picture Parameter Set, image parameter set) and SEI (Supplemental Enhancement Information, supplementary enhancement information), etc.

Therefore, if the information (nal_unit_type) indicating the type of the NAL unit is expressed as a scalable video coding slice, encoding/decoding efficiency can be improved by adding various configuration information related to the above-mentioned scalable encoding/decoding. For example, flag information indicating whether the current access unit is an instant decoding refresh (hereinafter referred to as IDR) access unit, dependency identification information indicating spatial scalability, quality identification information, and whether to use inter-layer prediction can be added The flag information (no_inter_layer_pred_flag), priority identification information, etc. It will be described in detail with reference to FIG. 2 .

In the standard, in order to be able to purchase the target product at an appropriate cost, requirements for various profiles and levels are specified. In this case, the decoder must meet the requirements defined in the corresponding profile and class. Similarly, two concepts of "profile" and "level" are defined to represent functions or parameters, which are used to represent the size of the range of compressed sequences that a decoder can handle. A bitstream based on a predetermined profile can be identified by a profile identifier (profile_idc). The profile identifier refers to a flag indicating the profile on which the bitstream is based. For example, in H.264/AVC, if the profile identifier is 66, it means that the bitstream is based on the reference profile; if the profile identifier is 77, it means that the bitstream is based on the main profile; if the profile identifier is 88, which indicates that the bitstream is based on an extended profile. In addition, the above-mentioned profile identifier may be included in the sequence parameter set.

Therefore, in order to process scalable sequences, it is necessary to identify whether the input bitstream is a profile of scalable sequences, and if identified as a profile of scalable sequences, it is necessary to add syntax such that at least one additional information for scalable sequences can be transmitted. Here, the profile of the scalable sequence, as an additional scheme of H.264/AVC, means a profile mode for processing scalable video. Since SVC is an additional scheme for conventional AVC technology, it is more effective to add syntax as additional information for the SVC mode than to unconditionally add syntax. For example, when the profile identifier of AVC is expressed as a profile of a scalable sequence, encoding/decoding efficiency can be improved if information about the scalable sequence is added.

Various embodiments for providing an efficient video signal decoding method will be described below.

FIG. 1 is a schematic block diagram of a scalable video encoding/decoding system according to the present invention.

In order to provide optimized sequences for various communication environments and various terminals, the sequences provided to the terminals should be diversified. Indicates that a single sequence source is prepared for combined values of various parameters, including transmission frames per second, resolution, bits per pixel, if the sequence optimized for each terminal is provided to the corresponding terminal wait. Therefore, provision of an optimized sequence imposes a burden on content providers. Therefore, the content provider encodes the original sequence into high bit rate compressed sequence data. Upon receiving a sequence request made by a terminal, the content provider decodes the original sequence, encodes it into sequence data suitable for the sequence processing capability of the terminal, and then provides the encoded data to the terminal. Since this transcoding is accompanied by an encoding-decoding-encoding process, a time delay cannot be avoided in providing the sequence. Therefore, complex hardware devices and algorithms are additionally required.

On the other hand, Scalable Video Coding (SVC) is a coding scheme for coding a video signal with optimum image quality such that a partial sequence of the resulting image sequence is represented as a sequence by decoding. Here, the partial sequence may mean a sequence composed of frames intermittently picked out from the entire sequence. For image sequences encoded by SVC, the sequence size can be reduced for low bit rates by using spatial scalability. Also, the image quality of the sequence can be reduced using quality scalability. Here, an image sequence with a small screen size and/or a low frame rate per second may be referred to as a base layer, and a sequence with a relatively large screen size and/or a relatively high frame rate per second may be referred to as an enhanced or enhancement layer.

Image sequences encoded by the above mentioned scalable schemes achieve low image quality sequence representations in such a way that only part of the sequence is received and processed. However, if the bit rate becomes lower, the image quality degrades a lot. In order to solve the problem of degraded image quality, a separate auxiliary image sequence may be provided for low bitrates, for example, with small size screens and/or low frames per second. Such auxiliary sequences may be referred to as base layers, and main image sequences may be referred to as enhanced or enhancement layers.

In describing various embodiments for inter-layer prediction, the present invention uses the concept of including a base layer and an enhancement layer. For example, the enhancement layer may have a different spatial resolution or screen scale than the base layer. Also, the enhancement layer may have a different image quality than the base layer. For example in detail, the base layer may be a reference layer, and the enhancement layer may be a current layer. The base layer and the enhancement layer explained below are only exemplary, and do not constitute a limit to the interpretation of the present invention.

The scalable video encoding/decoding system is described in detail below. First, the scalable encoding/decoding system includes an encoder 102 and a decoder 110 . The encoder 102 includes a base layer encoding unit 104 , an enhancement layer encoding unit 106 , and a multiplexing unit 108 . And, the decoder may include a demultiplexing unit 112 , a base layer decoding unit 114 , and an enhancement layer decoding unit 116 . The base layer encoding unit 104 is capable of generating an elementary bit stream by compressing the input sequence signal X(n). The enhancement layer encoding unit 106 can generate an enhancement layer bitstream by using the input sequence signal X(n) and information generated by the base layer encoding unit 104 . And, the multiplexing unit 108 can generate a scalable bitstream by using the base layer bitstream and the enhancement layer bitstream.

The generated scalable bit stream is transmitted to the decoder 110 through a specific channel, and the transmitted scalable bit stream can be divided into an enhancement layer bit stream and a base layer bit stream by the demultiplexing unit 112 of the decoder 110 . The base layer decoding unit 114 receives the base layer bitstream and decodes the base layer bitstream into a sequence signal between macroblocks and residual and motion information between blocks. Here, corresponding decoding may be performed based on a single-loop decoding method.

The enhancement layer decoding unit 116 receives the enhancement layer bitstream, and decodes the output sequence signal Xe(n) with reference to the base layer reconstructed by the base layer decoding unit 114 . Here, the output sequence signal Xb(n) will be a sequence signal having a lower image quality or resolution than the subsequent output sequence signal Xe(n).

Accordingly, each of the enhancement layer encoding unit 106 and the enhancement layer decoding unit 116 performs encoding by using inter-layer prediction. Inter-layer prediction means predicting a sequence signal of an enhancement layer by using motion information and/or texture information of a base layer. Here, the texture information may represent image data or pixel values belonging to a macroblock. For example, in the inter-layer prediction method, there is an intra base prediction mode or a residual prediction mode. The intra base prediction mode may represent a mode for predicting a block of an enhancement layer based on a corresponding region in the base layer. Here, a corresponding region in the base layer may represent a region coded in an inter-layer mode. At the same time, the residual prediction mode can use the corresponding region with residual data, which is the difference value of the image in the base layer. In both cases, the corresponding regions in the aforementioned base layer can be upscaled or downscaled by sampling for inter-layer prediction. Sampling means changing the image resolution. Also, sampling may include resampling, downsampling, upsampling, and the like. For example, intra-sample resampling can be performed to perform inter-layer prediction. Also, image resolution may be reduced by regenerating pixel data using a downsampling filter, which may be referred to as downsampling. Also, some additional pixel data may be generated to increase image resolution by using an upsampling filter, which may be referred to as upsampling. Resampling can include two concepts of downsampling and upsampling. In the present invention, the term "sampling" can be correctly interpreted according to the scope and technical idea of the corresponding embodiments of the present invention.

Meanwhile, for the same sequence content, the base layer and the enhancement layer are generated for different uses or purposes, and are different from each other in terms of spatial resolution, frame rate, bit rate, and the like. When encoding a video signal by inter-layer prediction, the non-second-order case, ie the ratio of the enhancement layer to the base layer in spatial resolution other than an integer of 2, may be referred to as Extended Spatial Scalability (ESS). For example, when the enhancement layer is encoded by inter-layer prediction as a video signal with a 16:9 (horizontal:vertical) ratio, it may happen that the base layer is encoded as a picture with a 4:3 ratio. In this case, since the base layer is encoded in a cropping state in which the original video signal is partially cropped, even if the base layer is enlarged for inter-layer prediction, it cannot cover the entire area of the enhancement layer. Therefore, since a partial region of the enhancement layer has no corresponding region in the upsampled base layer, the partial region cannot use the information of the upsampled base layer for inter-layer prediction. That is, it means that inter-layer prediction is not applicable for this part of the area. In this case, encoding information used for inter-layer prediction may not be transmitted. Specific embodiments will be described in detail below with reference to FIG. 5 to FIG. 11 .

FIG. 2 and FIG. 3 are respectively a structural diagram of configuration information that can be added to a scalable sequence of a scalable video coding bitstream according to an embodiment of the present invention, and images used to describe the configuration information.

FIG. 2 shows a structural example of a NAL unit that has configuration information on scalable sequences added thereto. First, a NAL unit may mainly include a NAL unit header and RBSP (Raw Byte Sequence Payload: result data of moving image compression). The NAL unit header may include identification information (nal_ref_idc) indicating whether the NAL unit includes a slice of a reference image and information (nal_unit_type) indicating a NAL unit type. And, an extension area of a NAL unit header may also be limitedly included. For example, if the information indicating the NAL unit type is associated with scalable video coding or indicates a prefix NAL unit, the NAL unit may include the extension area of the NAL unit header. Specifically, if nal_unit_type is equal to 20 or 14, the NAL unit may contain the extension area of the NAL unit header. Also, the configuration information about the scalable sequence can be added to the extension area of the NAL unit header according to the flag information (svc_mvc_flag) that can identify whether it is an SVC bit stream.

In another example, if the information representing the NAL unit type is the information representing the subset sequence parameter set, the RBSP may include information on the subset sequence parameter set. Specifically, if nal_unit_type is equal to 15, the RBSP may include information on a subset sequence parameter set, information on a slice layer, and the like. In this case, according to the profile information, the subset sequence parameter set may include an extended area of the sequence parameter set. For example, if the profile information (profile_idc) is a profile related to scalable coding, then the subset sequence parameter set may include the extension area of the sequence parameter set. Alternatively, according to the profile information, the sequence parameter set may include an extended area of the sequence parameter set. The extension area of the sequence parameter set may include information for controlling characteristics of a deblocking filter for inter-layer prediction, information related to parameters for an upsampling process, and the like. Various configuration information about the scalable sequence, for example, the configuration information that may be included in the extension area of the NAL unit header, the extension area of the sequence parameter set, and the slice layer will be described in detail below.

First, flag information (inter_layer_deblocking_filter_control_present_flag) indicating whether there is information for controlling the characteristics of the deblocking filter used for inter-layer prediction can be obtained from the extension area of the sequence parameter set. Also, information (extended_spatial_scalability) indicating the location of parameter-related information used for the upsampling process can be obtained from the extended area of the sequence parameter set. Specifically, for example, if the extended_spatial_scalability is equal to 0, it may indicate that there is no parameter for the upsampling process in the sequence parameter set or the slice header. If the extended_spatial_scalability is equal to 1, it may indicate that there are parameters for the upsampling process in the sequence parameter set. If the extended_spatial_scalability is equal to 2, it may indicate that there are parameters for the upsampling process in the slice header. Parameters for the upsampling process will be described in detail below with reference to FIG. 9 .

The information ④ indicating whether inter-layer prediction is used may refer to flag information indicating whether inter-layer prediction is used in decoding the encoded slice. Flag information can be obtained from the extension area of the NAL header. For example, if the flag information is set to 1, it may indicate that inter-layer prediction is not used. If the flag information is set to 0, inter-layer prediction may or may not be used according to a coding scheme in a macroblock. This is because inter-layer prediction in a macroblock may or may not be used.

The quality identification information ③ indicates information for identifying the quality of the NAL unit. Configuration information will be described with reference to FIG. 3 . For example, a single image can be coded into layers of mutually different quality. In FIG. 3, the layers on Spa_Layer0 and Spa_Layer1 can be coded as layers with different qualities from each other. Specifically, assuming that information identifying the quality of a NAL unit is named quality_id, layers B1, B2, . . . , B10 may be set to have quality_id equal to 0. Also, layers Q1, Q2, . . . , Q10 may be set to have quality_id equal to 1. In other words, layers B1, B2, . . . , B10 may represent layers with the lowest image quality. These are called base images. Layers Q1, Q2, . . . , Q10 correspond to layers including layers B1, B2, . And, quality identification information may be defined in various ways. For example, quality identification information can be expressed as 16 levels.

The information representing spatial scalability refers to information representing identification of dependencies on NAL units. Configuration information is described with reference to FIG. 3 . For example, dependencies vary according to spatial resolution. In Figure 3, the layers in Spa_Layer0 and Spa_Layer1 have the same resolution. The layers in Spa_Layer0 may include images obtained by performing downsampling on the layers in Spa_Layer1. Specifically, for example, assuming that information identifying dependency on NAL units is expressed as dependency_id, dependency_id between layers in Spa_Layer0 is equal to 0. And, the dependency_id between the layers in Spa_Layer1 is equal to 1. Dependency identification information may be defined in various ways. Therefore, NAL units that identify dependency information with the same value can be represented by a dependency representation.

Meanwhile, a single layer may be defined based on information identifying dependency relationships and quality identifying information. In this case, NAL units having the same value of identifying dependency information and quality identifying information can be represented by layer representation.

Identification information indicating temporal scalability refers to information identifying temporal levels with respect to NAL units. Temporal levels can be described in a hierarchical B-picture structure. For example, layers (B1, Q1) and layers (B3, Q3) in Spa_Layer0 may have the same temporal level Tem_Layer0. If layer(B5,Q5) references layer(B1,Q1) and layer(B3,Q3), then layer(B5,Q5) can have higher temporal level Tem_Layer0 than layer(B1,Q1) and layer(B3,Q3) The time level Tem_Layer1. Likewise, if layer (B7, Q7) references layer (B1, Q1) and layer (B5, Q5), then layer (B7, Q7) has a higher temporal level Tem_Layer2 than layer (B5, Q5)'s temporal level Tem_Layer1 . All NAL units in a single access unit may have the same temporal level value. In case of an IDR access unit, the temporal level value may be changed to 0.

The flag information indicating whether the reference base picture is used as the reference picture indicates whether the reference base picture is used as the reference picture in the inter-layer prediction process or whether the decoded picture is used as the reference picture in the inter-layer prediction process. For NAL units in the same layer, that is, NAL units having the same information for identifying dependencies, the flag information may have the same value.

The priority identification information refers to information identifying the priority of the NAL unit. It is possible to provide inter-layer scalability or inter-image scalability by using priority identification information. For example, it is possible to provide the user with sequences of various temporal and spatial levels by using priority identification information. Thus, the user is able to watch only the sequence in a certain time and space, or only watch the sequence according to different constraints. Priority information can be configured in different ways according to its reference conditions. Priority information can be randomly configured without being based on a specific reference. And, the priority information can be decided by the decoder.

Also, the configuration information in the extension area of the NAL unit header may include flag information indicating whether the current access unit is an IDR access unit.

Various information for inter-layer prediction may be included in the slice layer. For example, it may include information ⑤ indicating the processing of the slice boundary in the upsampling process, information ⑥ related to the operation of the deblocking filter, information ⑦ related to the phase shift of the chroma signal, indicating the base layer and Offset information ⑧ of the position difference between enhancement layers, and information indicating whether there is adaptive prediction ⑨. The above information can be obtained from the slice header.

As examples of the information ⑥ related to the operation of the deblocking filter, there may be information (disable_deblocking_filter_idc) indicating the method of the deblocking filter, offset information (inter_layer_slice_alpha_c0_offset_div2, inter_layer_slice_beta_offset_div2) necessary for deblocking filtering.

As an example of information ⑦ related to a phase shift of a chroma signal, there may be information on horizontal and vertical phase shifts (scaled_ref_layer_left_offset, scaled_ref_layer_top_offset, scaled_ref_layer_right_offset, scaled_ref_layer_bottom_offset) of a chroma component of an image used for inter-layer prediction.

As an example of the offset information ⑧ representing the position difference between layers, there may be information representing the top, bottom, left and right position differences between the up-sampled reference image used for inter-layer prediction and the current image (scaled_ref_layer_left_offset, scaled_ref_layer_top_offset , scaled_ref_layer_right_offset, scaled_ref_layer_bottom_offset).

As an example of information ⑤ representing the processing of macroblocks at the slice boundary during base layer upsampling, there may be at least two In the case of slices, whether the current macroblock cannot be predicted by using the corresponding intra-coded block in the base layer (constrained_intra_resampling_flag).

And, the information ⑨ indicating whether adaptive prediction exists may indicate whether information associated with prediction exists in the slice header and the macroblock layer. Based on the information indicating whether adaptive prediction exists, it can be decided which type of adaptive prediction method will be used. This will be described in detail later with reference to FIG. 11 .

FIG. 4 is a diagram about the cropping relation between the sampled base layer and the enhancement layer 1 .

In scalable video coding, it is possible to check whether the current block of the enhancement layer can use inter-layer prediction. For example, it may be checked whether an area corresponding to all pixels in the current block exists in the base layer. As a result of the checking process, if the current block of the enhancement layer is not used for inter-layer prediction, then no coding information for inter-layer prediction has to be transmitted. Therefore, coding efficiency can be improved.

Therefore, a function can be defined that is able to check whether the current block of the enhancement layer uses inter-layer prediction. For example, a function in_crop_window() may be defined to check whether a region corresponding to all pixels in the current block exists in the base layer. Assuming that the macroblock index in the horizontal direction on the enhancement layer is set to mbIdxX, and the macroblock index in the vertical direction is set to mbIdxY, the function in_crop_window() can return the value "TRUE" (or " 1").

mbIdxX≥(ScaledBaseLeftOffset+15)/16

mbIdxX≤(ScaledBaseLeftOffset+ScaledBaseWidth-1)/16

mbIdxY≥(ScaledBaseTopOffset+15)/16

mbIdxY≤(ScaledBaseTopOffset+ScaledBaseHeight-1)/16

mbIdxX can be derived by using the macroblock address and the macroblock number in the horizontal direction. mbIdxY can be derived by different methods depending on whether a macroblock adaptive frame-field (macroblock adaptive frame-field) is applied or not. For example, mbIdxY can be derived by considering macroblock pairs if macroblock adaptive frame-field is applied. When considering a pair of macroblocks, it is assumed that the index of the upper macroblock is set to mbIdxY0 and the index of the bottom macroblock is set to mbIdxY1. mbIdxY0 can be derived from offset information representing the upper position difference between the upsampled image used for inter-layer prediction and the current image, and macroblock number information in the horizontal direction. In this case, the value of the horizontal macroblock number information may differ depending on whether the current picture is a frame picture or a field picture. mbIdxY1 can be derived from offset information representing the upper position difference between the upsampled image used for inter-layer prediction and the current image, and macroblock number information in the vertical direction. Meanwhile, mbIdxY0 and mbIdxY1 may be set to the same value if macroblock adaptive frame-field is not applied.

ScaledBaseLeftOffset represents offset information representing a position difference of the left side between the up-sampled image used for inter-layer prediction and the current image. ScaledBaseTopOffset represents the upper position difference between the up-sampled image used for inter-layer prediction and the current image. ScaledBaseWidth represents the horizontal width of the upsampled image. And, ScaledBaseHeight represents the vertical height of the up-sampled image.

If any one of the above conditions is not satisfied, the function in_crop_window() will return a "FALSE" (or "0") value.

Information associated with inter-layer prediction is not used when the pixel corresponding to at least one pixel in the current block is not in the upsampled base layer, that is, when the function in_crop_window(CurrMbAddr) returns a "FALSE" value in the current block, and this information will not be transmitted. Therefore, according to an embodiment of the present invention, if it is recognized through in_crop_window(CurrMbAddr) that a corresponding region in the base layer does not exist, transmission of information related to inter-layer prediction for a current block may be omitted.

According to an embodiment of the present invention, a case where encoding is performed by using the function in_crop_window() is described below.

First, when it is recognized through in_crop_window(CurrMbAddr) that a region corresponding to a current block exists in the base layer, the enhancement layer encoding unit 106 performs inter-layer prediction by using texture and/or motion information of the base layer. In this case, the motion information may include reference index information, motion vector information, partition information, and the like.

When the texture and/or motion information of the current block is set as the texture and/or motion information of the corresponding block, or when the texture and/or motion information of the current block is derived from the texture and/or motion information of the corresponding block, The enhancement layer encoding unit 106 adds indication information indicating complete or derived information to the data stream of the enhancement layer, and notifies the decoder 110 of the addition. However, when it is recognized through in_crop_window(CurrMbAddr) that the region corresponding to the current block does not exist in the base layer, the enhancement layer encoding unit 106 may generate the enhancement layer without performing inter-layer prediction. Meanwhile, if the decoder 110 confirms that the area corresponding to the current block does not exist in the base layer through in_crop_window(CurrMbAddr), the decoder 110 decides not to transmit the indication information.

FIG. 5 and FIG. 6 are respectively syntax diagrams related to macroblock and sub-macroblock prediction through inter-layer prediction according to an embodiment of the present invention.

When inter-layer prediction is performed, information related to inter-layer prediction of slice data of the current NAL is transmitted to the decoder. For example, in the prediction of the motion vector of the current block in the enhancement layer, a flag (motion_prediction_flag_lx) indicating whether to use the motion vector of the base layer can be obtained from the macroblock layer. According to one embodiment of the present invention, the decoder knows whether the information associated with the inter-layer prediction is transmitted by the encoder by checking in_crop_window(CurrMbAddr) (510, 610). For example, according to in_crop_window(CurrMbAddr), if the region corresponding to the current block does not exist in the base layer, the flag motion_prediction_flag_10/11 will not be transmitted in the bitstream (520/530, 620/630).

And, a flag adaptive_motion_prediction_flag indicating whether information associated with motion vector prediction exists in the macroblock layer may be obtained from slice data of the current NAL. According to an embodiment of the present invention, by checking adaptive_motion_prediction_flag and in_crop_window(CurrMbAddr), the encoder may not transmit information associated with inter-layer prediction (510). For example, according to in_crop_window(CurrMbAddr), the flag motion_prediction_flag_10/ 11 (520/530, 620/630). The above technical ideas are also applicable to the sub-macroblock prediction shown in FIG. 6 .

Therefore, if the above two conditions are satisfied after the above two kinds of information are identified, the information associated with the inter-layer prediction is transmitted. Therefore, coding efficiency can be improved.

FIG. 7 is a syntax diagram related to residual prediction through inter-layer prediction according to one embodiment of the present invention.

In case of performing inter-layer prediction, information related to inter-layer prediction in the slice data of the current NAL is transmitted to the decoder. For example, in the case of predicting the residual signal of the current block, a flag residual_prediction_flag indicating whether to use the residual signal of the base layer may be obtained from the macroblock layer (740). In this case, the base layer can be known through layer representation information. According to an embodiment of the present invention, by confirming in_crop_window (CurrMbAddr), the encoder may not transmit information related to inter-layer prediction.

For example, the above residual_prediction_flag ( 710 ) may be obtained according to the information adaptive_residual_prediction_flag indicating the existence of the information related to the prediction of the residual signal in the macroblock and the slice type information of the current block. The above residual_prediction_flag can also be obtained according to base_mode_flag. The above-mentioned base_mode_flag indicates whether the type (mb_type) of the current macroblock is derived from the corresponding area of the base layer (720). The residual_prediction_flag can also be obtained according to the type of the current macroblock and the function in_crop_window(CurrMbAddr). For example, when the type of macroblock and sub-macroblock is not intra mode (MbPartPredType(mb_type, 0) != Intra_16x16(8x8 and 4x4)), and the value of in_crop_window(CurrMbAddr) is "true" (which means corresponding to The region of the current macroblock exists in the base layer), residual_prediction_flag can be obtained (730). If the type of the current macroblock is not intra mode or the region corresponding to the current macroblock does not exist in the base layer (in_crop_window(CurrMbAddr)=0), residual prediction is not performed. Also, the aforementioned encoder 102 generates an enhancement layer without including the residual_prediction_flag.

If the above residual_prediction_flag is set to 1, the residual signal of the current block is predicted from the residual signal of the base layer. If residual_prediction_flag is set to 0, the residual signal is coded without inter-layer prediction. If the residual_prediction_flag does not exist in the macroblock layer, it may be derived as follows. For example, the residual_prediction_flag can be derived as a preset value (default_residual_prediction_flag) only when the following conditions are fully met. First, base_mode_flag should be set to 1 or the type of the current macroblock should not be intra mode. Second, in_crop_window(CurrMbAddr) should be set to 1. Third, a flag no_inter_layer_pred_flag indicating whether to use inter-layer prediction should be set to 0. Fourth, the stripe type should not be an EI stripe. Otherwise, 0 is derived.

By in_crop_window(CurrMbAddr), when it is confirmed that the region corresponding to the block of the current sequence does not exist in the base layer, the enhancement layer decoding unit 116 decides that the motion prediction flag (motion_prediction_flag) information does not exist in the macroblock or sub-macroblock, and only passes The data bitstream of the enhancement layer is used to reconstruct the video signal without inter-layer prediction. The enhancement layer decoding unit 116 may derive a residual prediction flag residual_prediction_flag if the syntax elements for residual prediction are not contained in the data bitstream of the enhancement layer. In doing so, it may be considered by in_crop_window(CurrMbAddr) whether an area corresponding to the current block exists in the base layer. If in_crop_window(CurrMbAddr) is set to 0, the enhancement layer decoding unit 116 may confirm that the region corresponding to the current sequence block does not exist in the base layer. In this case, the residual_prediction_flag is derived to be 0, and then the video signal can be reconstructed only by using the data of the enhancement layer without performing residual prediction by using the residual signal of the base layer.

FIG. 8 is a syntax structure diagram of performing deblocking filtering according to whether there is inter-layer prediction according to an embodiment of the present invention.

First, according to an embodiment of the present invention, the encoder may not transmit information related to inter-layer prediction by checking the configuration information of the scalable video coded bitstream. The configuration information of the scalable video coded bitstream can be obtained from the extension area of the NAL header. For example, the information related to the deblocking filter ( 810 ) may be obtained according to information indicating whether to use inter-layer prediction (no_inter_layer_pred_flag) and quality identification information (quality_id). As examples of information related to the operation of the deblocking filter, there may be information indicating an operation method of the deblocking filter (disable_deblocking_filter_idc), offset information (slice_alpha_c0_offset_div2, slice_beta_offset_div2) required for deblocking filtering, and the like.

First, information representing the operation of the deblocking filter can be obtained based on the information for controlling the characteristics of the deblocking filter. In this case, as mentioned in the description of FIG. 2 , the information for controlling the characteristics of the deblocking filter can be obtained from the extended area of the sequence parameter set. For example, as information for controlling the characteristics of the deblocking filter, there may be flag information (inter_layer_deblocking_filter_control_present_flag) indicating whether information controlling the characteristics of the deblocking filter for inter-layer prediction exists (820). Accordingly, information representing an operation method of the deblocking filter can be obtained from the above-mentioned flag information (830).

Specifically, if disable_deblocking_filter_idc is equal to 0, then filtering may be performed for all block edges of the luma and chroma signals of the current picture. If disable_deblocking_filter_idc is equal to 1, filtering may not be performed on all block edges of the current picture. disable_deblocking_filter_idc equal to 2, performs filtering on all block edges except block edges with overlapping slice boundaries. If disable_deblocking_filter_idc is equal to 3, block edges with non-overlapping slice boundaries are filtered, and then block edges with overlapping slice boundaries are filtered. If disable_deblocking_filter_idc is equal to 4, filtering is only performed on block edges of luma signals and may not be performed on block edges of chroma signals. If disable_deblocking_filter_idc is equal to 5, all block edges of luma signals other than block edges with overlapping slice boundaries are filtered, and no filtering may be performed on block edges of chroma signals. If disable_deblocking_filter_idc is equal to 6, the block edge of the chrominance signal may not be filtered, but only the block edge of the luma signal may be filtered. After filtering block edges of luma signals with non-overlapping slice boundaries, block edges of luma signals with overlapping slice boundaries may be filtered.

Based on the information indicating the operation method of the deblocking filter, offset information required for deblocking filtering can be obtained. For example, if disable_deblocking_filter_idc is equal to 1, deblocking filtering is not performed on all block edges. Therefore, only when the value of disable_deblocking_filtor_idc is not set to 1, the offset information ( 840 ) necessary for deblocking filtering can be obtained. For example, inter_layer_slice_alpha_c0_offset_div2 and inter_layer_slice_beta_offset_div2 may refer to offset information ( 850 ) used when accessing a DF table in a macroblock during inter-layer prediction. Therefore, deblocking filtering can be performed by using the obtained offset information.

Fig. 9 is a syntax structure diagram for obtaining offset information representing a position difference between an upsampled reference image and a current image according to whether there is inter-layer prediction according to an embodiment of the present invention.

According to an embodiment of the present invention, the encoder can check the configuration information of the scalable video coding bit stream without transmitting the information related to the inter-layer prediction. The configuration information of the Scalable Video Coding bitstream can be obtained from the extension area of the NAL header. For example, information related to parameters for an upsampling process may be obtained from information (no_inter_layer_pred_flag) indicating whether to use inter-layer prediction and quality identification information (quality_id) ( 910 ). As examples of information related to parameters used for the upsampling process, there are information (930) on a phase shift of a chroma signal, offset information (940) indicating a positional difference between images, and the like. And, information related to parameters used for the upsampling process can be obtained from the extension area of the sequence parameter set and the slice header.

As an example of the information (930) on the phase shift of the chroma signal, there is information on the horizontal phase shift (ref_layer_chroma_phase_x_plus1) of the chroma component of an image used for inter-layer prediction, information on its vertical phase change (ref_layer_chroma_phase_y_plus1). As an example of the offset information (940) indicating the position difference between images, there is offset information (scaled_ref_layer_left_offset, scaled_ref_layer_top_offset, scaled_ref_layer_right_offset, scaled_ref_layer_bottom_offset) indicating the distance between the up-sampled image used for inter-layer prediction and the current image. The left, top, right, and bottom position differences of .

The information related to the parameters for the up-sampling process may be obtained based on the information (extended_spatial_scalability) representing the location of the information about the parameters used for the up-sampling process. For example, if the above-mentioned extended_spatial_scalability is set to 0, it may indicate that information related to parameters used for the upsampling process does not exist in either the sequence parameter set or the slice header. If extended_spatial_scalability is set to 1, it may indicate that information related to parameters for the upsampling process does not exist in the slice header but exists in the sequence parameter set. If extended_spatial_scalability is set to 2, it may indicate that information related to parameters used for the upsampling process does not exist in the sequence parameter set but exists in the slice header. Therefore, if extended_spatial_scalability is set to 2, information related to parameters for the upsampling process in the slice header can be controlled ( 920 ). Also, if extended_spatial_scalability is set to 1, information related to parameters used for the upsampling process in the sequence parameter set can be controlled.

The information (930) on the phase shift of the chrominance signal, and the offset information (940) representing the positional difference between the reference picture and the current picture are used in the upsampling process.

FIG. 10 is a syntax structure diagram for obtaining flag information indicating whether to restrict the use of an intra-block in a base layer according to whether there is inter-layer prediction according to an embodiment of the present invention.

According to an embodiment of the present invention, by checking the configuration information of the scalable video coding bitstream, the encoder may not transmit information related to inter-layer prediction. The above configuration information of the scalable video coding bitstream can be obtained from the extension area of the NAL header. For example, the information indicating the processing of the slice boundary in the upsampling process may be obtained according to the information (no_inter_layer_pred_flag) indicating whether to use inter-layer prediction and the quality identification information (quality_id) (1010). As an example of information indicating processing of a slice boundary, there is information (constrained_intra_resampling_flag) indicating whether to restrict use of an intra block in a base layer for a current block in an enhancement layer. By defining information indicating whether to limit the use of intra blocks, it is possible to increase the decoding speed when parallel processing is performed. Information indicating whether to limit the use of intra blocks can be obtained from the slice header.

Since the information indicating whether to restrict the usage of the intra block can be obtained from the slice header, even if its value is set to 1, it is necessary to check whether the reference block of the base layer corresponding to the current block is contained in a specific slice of the base layer. Accordingly, when constrained_intra_resampling_flag is set to 1, it may be confirmed whether a reference block of a base layer corresponding to a current block is included in a specific slice of the base layer. For example, when a reference block of the base layer overlaps with at least two slices in the base layer, the current block is marked as not using an intra block in the base layer. Specifically, the current block cannot be encoded by using an intra-base prediction mode. The intra base prediction mode may mean a mode of predicting a current block of an enhancement layer based on a corresponding region of the base layer. In this case, the corresponding region of the base layer represents a block coded in intra mode. When a corresponding region of the base layer is included in a specific slice of the base layer, the current block may be decoded by using an intra-block of the base layer. In this case, the current block can be marked as using intra-base prediction mode.

If the above constrained_intra_resampling_flag is set to 1, the information (disable_deblocking_filter_idc) indicating the operation method of the deblocking filter described with reference to FIG. 8 will be restricted. For example, disable_deblocking_filter_idc can only be set to 1, 2 or 4.

If the above constrained_intra_resampling_flag is set to 0, a current block of an enhancement layer may be decoded using an intra-block of the base layer even if a corresponding block in the base layer overlaps with at least two slices in the base layer.

The embodiments described above can be applied to chrominance signals, and also to luminance signals in the same way.

FIG. 11 is a syntax diagram for obtaining adaptive prediction information according to whether there is inter-layer prediction according to one embodiment of the present invention.

According to an embodiment of the present invention, by confirming the configuration information of the scalable video coding bitstream, the encoder may not transmit information related to inter-layer prediction. Configuration information of a scalable video coding bitstream can be obtained from an extension area of a NAL header. For example, adaptive prediction information may be obtained based on information no_inter_layer_pred_flag indicating whether inter-layer prediction is used (1110). The adaptive prediction information may indicate whether syntax related to prediction exists in a corresponding position. For example, there is information adaptive_prediction_flag indicating whether the syntax related to prediction exists in the slice header and the macroblock layer, information adaptive_motion_prediction_flag indicating whether the syntax related to motion prediction exists in the macroblock layer, and whether the syntax related to residual prediction exists in The information adaptive_residual_prediction_flag of the macroblock layer, etc.

When inter-layer prediction is performed according to information indicating whether inter-layer prediction is used, first flag information slice_skip_flag indicating whether slice data exists may be obtained (1120). By confirming information indicating that slice data exists, it can be determined whether to derive information within a macroblock in order to perform inter-layer prediction. According to the information indicating the existence of the above-mentioned slice data, if the slice data exists in the slice (1130), an adaptive prediction flag adaptive_prediction_flag may be obtained (1140). Also, information adaptive_residual_prediction_flag indicating whether syntax related to residual prediction exists in the macroblock layer can be obtained (1180). According to the adaptive prediction flag described above, information default_base_mode_flag indicating how to derive information indicating whether to predict motion information or the like can be obtained from a corresponding block of the base layer (1150). When motion information and the like are not predicted from the corresponding block of the base layer (1155), information adaptive_motion_prediction_flag indicating whether syntax related to motion prediction exists in the macroblock layer may be obtained (1160). If syntax related to motion prediction does not exist in the macroblock layer (1165), information default_motion_prediction_flag indicating how to infer motion prediction flag information can be obtained (1170).

Information adaptive_motion_prediction_flag indicating whether syntax related to motion prediction exists in the macroblock layer and information adaptive_residual_prediction_flag indicating whether syntax related to residual prediction exists in the macroblock layer may be used at the macroblock layer. For example, the flag motion_prediction_flag_lx indicating whether to use the motion vector of the base layer can be obtained based on the above-mentioned adaptive_motion_prediction_flag. And, based on the above adaptive_residual_prediction_flag, a flag residual_prediction_flag indicating whether to use the residual signal of the base layer can be obtained.

As described above, the decoder/encoder applicable to the present invention is provided to a broadcast transmitter/receiver for multimedia broadcasting such as DMB (Digital Multimedia Broadcasting) for decoding video signals, data signals, etc. . And, the above-mentioned multimedia broadcast transmitter/receiver may include a mobile communication terminal.

A decoding/encoding method to which the present invention is applied is provided as a program for computer execution and stored in a computer-readable recording medium. And, multimedia data having the data structure of the present invention can be stored in a computer readable recording medium. The computer-readable recording medium includes all types of storage devices for storing data readable by a computer system. The computer-readable recording medium includes ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk storage devices, etc., and also includes devices implemented by carrier waves (for example, transmission via Internet). And, the bit stream generated by the encoding method is stored in a computer-readable medium or transmitted through a wired/wireless communication network.

Industrial Applicability

Although the present invention has been described and illustrated with reference to preferred embodiments thereof, it will be apparent to those skilled in the art that various modifications and changes can be made therein without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the claims and their equivalents.

Claims (4)

1. A method of decoding an enhancement layer of a bitstream for scalable video coding using inter-layer prediction, comprising: extracting inter-layer prediction information from the scalable video coded bitstream, the inter-layer prediction information indicating whether the inter-layer prediction is used to decode a current slice in the enhancement layer; extracting quality identification information from a bitstream of scalable video coding, the quality identification information identifying the image quality of the current slice; When the inter-layer prediction information indicates that the inter-layer prediction is used to decode the current slice and the quality identification information is set to zero, extracting prediction restriction information from a header area of the current slice, the The prediction restriction information indicates whether intra-coded samples of a reference block in the base layer are used to derive a prediction of the current block in the enhancement layer when the reference block overlaps with at least two slices in the base layer value, the reference block is referenced by the current block in the current slice; checking whether the reference block overlaps at least two slices within the base layer; using a prediction mode other than the intra basic prediction mode to derive the predicted value of the current block, when the intra-coded samples of the reference block are not used to derive the predicted value of the current block according to the prediction information and When the reference block overlaps with at least two slices in the base layer, the intra-frame basic prediction mode represents a prediction mode for deriving a prediction value of the current block from intra-coded samples of the reference block, wherein the predictor of the current block is derived from the reference block in the base layer based on offset information comprising at least one left pixel representing the reference block and a left pixel of the current block The left offset information of the position difference between, the top offset information indicating the position difference between the top pixel of the reference block and the top pixel of the current block, and the top offset information of the position difference between the right pixel of the reference block and the right pixel of the current block Right offset information of the position difference of and bottom offset information representing the position difference between the bottom pixel of the reference block and the bottom pixel of the current block; and The current block is reconstructed using the predicted value of the current block. 2. The method according to claim 1, further comprising extracting parameter identification information indicating whether a parameter to be used for the upsampling process of the base layer is present in at least one sequence parameter set and a slice header , wherein at least one piece of offset information is acquired based on the parameter identification information. 3. The method according to claim 2, further comprising obtaining phase information related to a phase shift of a chrominance signal based on the inter-layer prediction information and the parameter identification information, Wherein the predicted value of the current block is obtained based on the phase information and the offset information, and the phase information includes horizontal phase shift information and vertical phase shift information. 4. The method of claim 1, wherein the base layer has a different screen scale or spatial resolution than that of the enhancement layer, the enhancement layer being derived from the same video signal as the base layer.

Images