CN101455084A - Method and apparatus for decoding/encoding video signal - Google Patents

Abstract

The present invention provides a method of decoding a video signal. The method includes the steps of checking an encoding scheme of the video signal, obtaining configuration information for the video signal according to the encoding scheme, recognizing a total number of views using the configuration information, recognizing inter-view reference information based on the total number of the views, and decoding the video signal based on the inter- view reference information, wherein the configuration information includes at least view information for identifying a view of the video signal.

Description

Method and apparatus for decoding/encoding video signal

technical field

The present invention relates to a method and device for decoding/encoding video signals.

Background technique

Compression coding refers to a series of signal processing techniques for transmitting digitized information via a communication circuit or storing digitized information in a form suitable for a storage medium. Compression coded objects include audio, video, text, etc. In particular, techniques for performing compression encoding on sequences are referred to as video sequence compression. Video sequences are often characterized by spatial redundancy and temporal redundancy.

Contents of the invention

technical purpose

The object of the present invention is to improve the coding efficiency of video signals.

Technical solutions

It is an object of the present invention to efficiently encode a video signal by defining view information for identifying a view of a picture.

Another object of the present invention is to improve coding efficiency by coding video signals based on inter-view reference information.

Another object of the present invention is to provide view scalability of video signals by defining view level information.

Another object of the present invention is to efficiently encode a video signal by defining an inter-view synthesis prediction identifier indicating whether a picture of a virtual view is obtained or not.

Beneficial effect

In encoding a video signal, the present invention enables encoding to be performed more efficiently by performing inter-view prediction using view information for identifying a view of a picture. And, the present invention can provide a view sequence suitable for a user by newly defining level information indicating information for a hierarchical structure to provide view scalability. Also, the present invention provides compatibility with legacy decoders by defining a view corresponding to the lowest level as a reference view. In addition, the present invention improves coding efficiency by deciding whether to predict a picture of a virtual view during performing inter-view prediction. In the case of predicting pictures of a virtual view, the invention enables more accurate prediction, thereby reducing the number of bits to be transferred.

Description of drawings

Fig. 1 is a schematic block diagram of an apparatus for decoding a video signal according to the present invention.

FIG. 2 is a diagram of configuration information on multi-view video that can be added to a multi-view video encoding bitstream according to an embodiment of the present invention.

FIG. 3 is an internal block diagram of the reference picture list construction unit 620 according to an embodiment of the present invention.

FIG. 4 is a diagram of a hierarchical structure for providing level information of view scalability of a video signal according to an embodiment of the present invention.

FIG. 5 is a diagram of a NAL unit configuration including level information in an extension area of a NAL header according to one embodiment of the present invention.

FIG. 6 is a diagram of an overall predictive structure of a multi-view video signal for explaining the concept of an inter-view picture group according to an embodiment of the present invention.

FIG. 7 is a diagram of a predictive structure for explaining the concept of a newly defined inter-view picture group according to an embodiment of the present invention.

FIG. 8 is a schematic block diagram of an apparatus for decoding multi-view video using inter-view group of picture identification information according to an embodiment of the present invention.

FIG. 9 is a flowchart of a process for constructing a reference picture list according to an embodiment of the present invention.

FIG. 10 is a diagram for explaining a method of initializing a reference picture list when a current slice (slice) is a P-slice (P-slice) according to one embodiment of the present invention.

FIG. 11 is a diagram for explaining a method of initializing a reference picture list when a current slice is a B-slice (B-slice) according to one embodiment of the present invention.

FIG. 12 is an internal block diagram of the reference picture list rearrangement unit 630 according to an embodiment of the present invention.

FIG. 13 is an internal block diagram of the reference index assignment changing unit 643B or 645B according to one embodiment of the present invention.

FIG. 14 is a diagram for explaining a process of rearranging a reference picture list using view information according to one embodiment of the present invention.

FIG. 15 is an internal block diagram of the reference picture list rearrangement unit 630 according to another embodiment of the present invention.

FIG. 16 is an internal block diagram of the reference picture list rearrangement unit 970 for inter-view prediction according to an embodiment of the present invention.

17 and 18 are diagrams of syntax for reference picture list rearrangement according to one embodiment of the present invention.

FIG. 19 is a diagram of syntax for reference picture list rearrangement according to another embodiment of the present invention.

FIG. 20 is a diagram of a process for obtaining a luminance difference value of a current block according to one embodiment of the present invention.

FIG. 21 is a flowchart of a process for performing brightness compensation of a current block according to an embodiment of the present invention.

FIG. 22 is a diagram of a process for obtaining a luma difference prediction value of a current block using information on neighboring blocks according to one embodiment of the present invention.

FIG. 23 is a flowchart of a process for performing brightness compensation using information on neighboring blocks according to one embodiment of the present invention.

FIG. 24 is a flowchart of a process for performing brightness compensation using information on neighboring blocks according to another embodiment of the present invention.

25 is a diagram of a process for predicting a current picture using pictures in a virtual view, according to one embodiment of the invention.

26 is a flowchart of a process for synthesizing pictures in a virtual view during performing inter-view prediction in MVC according to an embodiment of the present invention.

FIG. 27 is a flowchart of a method for performing weighted prediction according to slice types in video signal encoding according to the present invention.

FIG. 28 is a chart of allowable macroblock types among slice types in video signal encoding according to the present invention.

29 and 30 are diagrams of syntax for performing weighted prediction according to newly defined segment types according to one embodiment of the present invention.

31 is a flowchart of a method of performing weighted prediction using flag information indicating whether to perform inter-view weighted prediction in video signal encoding according to the present invention.

32 is a diagram for explaining a weighted prediction method according to flag information indicating whether to perform weighted prediction using information about a picture in a view different from that of the current picture according to one embodiment of the present invention.

FIG. 33 is a diagram of syntax for performing weighted prediction according to newly defined flag information according to one embodiment of the present invention.

FIG. 34 is a flowchart of a method of performing weighted prediction according to NAL (Network Abstraction Layer) unit types according to an embodiment of the present invention.

35 and 36 are diagrams of syntax for performing weighted prediction in a case where the NAL unit type is related to multi-view video coding according to one embodiment of the present invention.

FIG. 37 is a partial block diagram of a video signal decoding apparatus according to a newly defined slice type according to an embodiment of the present invention.

FIG. 38 is a flowchart for explaining a method of decoding a video signal in the apparatus shown in FIG. 37 according to the present invention.

FIG. 39 is a diagram of macroblock prediction modes according to one embodiment of the present invention.

40 and 41 are diagrams of syntax to which a slice type and a macroblock mode are applied according to the present invention.

FIG. 42 is a diagram of an embodiment to which the segment types in FIG. 41 are applied.

FIG. 43 is a diagram of various embodiments of segment types included in the segment types shown in FIG. 41 .

FIG. 44 is a diagram of allowable macroblocks for mixed slice types according to prediction of two mixed predictions according to one embodiment of the present invention.

45 to 47 are diagrams of macroblock types of macroblocks present in a predicted hybrid slice according to two hybrid predictions according to one embodiment of the present invention.

FIG. 48 is a partial block diagram of a video signal encoding apparatus according to a newly defined slice type according to an embodiment of the present invention.

FIG. 49 is a flowchart of a method of encoding a video signal in the apparatus shown in FIG. 48 according to the present invention.

Detailed ways

To achieve these and other advantages as embodied and broadly described herein and in accordance with the object of the present invention, a method of decoding a video signal comprises the steps of: checking the encoding scheme of the video signal, obtaining configuration information for the video signal according to the encoding scheme , identifying a total number of views using configuration information, identifying inter-view reference information based on the total number of views, and decoding the video signal based on the inter-view reference information, wherein the configuration information includes at least view information for identifying a view of the video signal.

To further achieve these and other advantages and in accordance with the object of the present invention, a method of decoding a video signal comprises the steps of: checking the coding scheme of the video signal, obtaining configuration information for the video signal according to the coding scheme, checking the video signal from the configuration information A level for view scalability, using configuration information to identify inter-view reference information, and decoding a video signal based on the level and the inter-view reference information, wherein the configuration information includes view information for identifying a view of a picture.

Patterns for the invention

Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

Techniques for compressing and encoding video signal data consider spatial redundancy, temporal redundancy, scalable redundancy, and inter-view redundancy. And, it is also possible to perform compression encoding by considering mutual redundancy between views in the compression encoding process. A technique for compression encoding that considers inter-view redundancy is only one embodiment of the present invention. Also, the technical idea of the present invention can be applied to temporal redundancy, scalable redundancy, and the like.

Studying the configuration of the bit stream in H.264/AVC, there is a separate layer called NAL (Network Abstraction Layer) between the VCL (Video Coding Layer) that handles the moving picture coding process itself and the lower layer system that transmits and stores the coding information. layer structure. The output from the encoding process is VCL data and it is mapped by NAL units before being transmitted or stored. Each NAL unit includes compressed video data or RBSP (Raw Byte Sequence Payload: result data of motion picture compression), which is data corresponding to header information.

A NAL unit basically includes a NAL header and RBSP. The NAL header includes flag information (nal_ref_idc) indicating whether to include a slice as a reference picture of the NAL unit and an identifier (nal_unit_type) indicating the type of the NAL unit. The compressed raw data is stored in RBSP. And, the last bit of RBSP is added to the last part of RBSP to indicate that the length of RBSP is 8-bit multiplication. As the type of NAL unit, there are IDR (instant decoding refresh: instantaneous decoding refresh) pictures, SPS (sequence parameter set: sequence parameter set), PPS (picture parameter set: picture parameter set), SEI (additional enhancement information: supplemental enhancement information) wait.

In standardization, restrictions on various profiles and levels are set so that a target product can be realized at an appropriate cost. In this case, the decoder should satisfy the constraints determined according to the corresponding class and level. Therefore, two concepts "class" and "level" are defined to indicate functions or parameters for expressing to what extent a decoder can handle the range of compressed sequences. And, the profile identifier (profile_idc) can identify that the bit stream is based on a prescribed profile. A class identifier refers to a flag indicating a class on which a bit stream is based. For example, in H.264/AVC, if the class identifier is 66, this means that the bitstream is based on the baseline profile. If the class identifier is 77, this means that the bit stream is based on the main class. If the class identifier is 88, this indicates that the bit stream is based on the extended class. And, a class identifier can be included in a sequence parameter set.

Therefore, in order to process multi-view video, it is necessary to identify whether the class of the input bit stream is the multi-view class. If the class of the input bitstream is a multi-view class, it is necessary to add syntax to enable transmission of at least one additional information about multi-view. In this case, the multi-view class indicates a class mode of processing multi-view video which is a revised technology of H.264/AVC. In MVC, it may be more efficient to add syntax that is additional information about the MVC pattern than to add unconditional syntax. For example, when the class identifier of AVC indicates a multi-view class, if information on multi-view video is added, encoding efficiency can be improved.

A sequence parameter set indicates header information containing encoded information (eg, class, level, etc.) covering the overall sequence. The entire compressed motion picture, ie sequence, should start with a sequence header. Therefore, the sequence parameter set corresponding to the header information should reach the decoder before the data referring to the parameter set arrives. That is, the sequence parameter set RBSP plays the role of header information for the result data of moving picture compression. Once the bitstream is input, the class identifier preferably identifies which of the plurality of classes the input bitstream is based on. Therefore, it is decided whether the input bitstream relates to the multi-view profile by adding a part to the syntax for determining whether the input bitstream relates to the multi-view profile (eg, "If (profile_idc==MULTI_VIEW_PROFILE)"). Various configuration information can be added only when it is determined that the input bit stream relates to the multi-view class. For example, it is possible to add the number of all views, the number of inter-view reference pictures (List0/1) in the case of an inter-view picture group, and the number of inter-view reference pictures (List0/1) in the case of a non-inter-view picture group wait. And, various information on a view can be used to generate and manage a reference picture list in a decoded picture buffer.

Fig. 1 is a schematic block diagram of an apparatus for decoding a video signal according to the present invention.

Referring to FIG. 1, the device for decoding video signals according to the present invention includes a NAL parser 100, an entropy decoding unit 200, an inverse quantization/inverse transformation unit 300, an intra prediction unit 400, a deblocking filtering unit 500, a decoded picture The buffer unit 600, the inter prediction unit 700, and the like.

The decoded picture buffer unit 600 includes a reference picture storage unit 610, a reference picture list construction unit 620, a reference picture management unit 650, and the like. Also, the reference picture list construction unit 620 includes a variable derivation unit 625 , a reference picture list initialization unit 630 and a reference picture list rearrangement unit 640 .

And, the inter prediction unit 700 includes a motion compensation unit 710, a brightness compensation unit 720, a brightness difference prediction unit 730, a view synthesis prediction unit 740, and the like.

The NAL parser 100 performs parsing by NAL units to decode a received video sequence. Typically, at least one sequence parameter set and at least one picture parameter set are passed to the decoder prior to decoding the slice header and slice data. In this case, various configuration information can be included in the NAL header area or the extension area of the NAL header. Since MVC is a modified technology to the conventional AVC technology, it may be more efficient to just add configuration information instead of adding unconditionally in the case of an MVC bitstream. For example, flag information for identifying the existence or non-existence of the MVC bit stream may be added in the NAL header area or the extension area of the NAL header. Configuration information on multi-view video can be added only when the input bit stream is a multi-view video encoded bit stream according to flag information. For example, the configuration information may include time level information, view level information, inter-view GOP identification information, view identification information, and the like. This is explained in detail with reference to FIG. 2 as follows.

FIG. 2 is a diagram of configuration information on multi-view video that can be added to a multi-view video coding bitstream according to one embodiment of the present invention. Details about the configuration information of the multi-view video are explained in the following description.

First, the temporal level information indicates information on a hierarchical structure for providing temporal scalability according to video signals (①). Through the time level information, sequences in various time zones (zones) can be provided to the user.

The view level information indicates information on a hierarchical structure for providing view scalability according to video signals (②). In multi-view video, it is necessary to define levels with respect to time as well as levels with respect to views, so as to provide users with various temporal and view sequences. In case of defining the above level information, time scalability and view scalability can be used. Thus, the user can select a sequence at a particular time and view, or the selected sequence can be limited by conditions.

The level information can be set in various ways according to specific conditions. For example, level information can be set differently according to camera position or camera calibration. And, level information can be determined by considering view dependencies. For example, the level for a view with an I-picture in an inter-view picture group is set to 0, the level for a view with a P-picture in an inter-view picture group is set to 1, and the level for a view in an inter-view picture group The level of a view with a B-picture is set to 2. Also, level information can be set randomly without being based on specific conditions. The view level information will be explained in detail later with reference to FIGS. 4 and 5 .

The inter-view picture group identification information indicates information for identifying whether the coded picture of the current NAL unit is an inter-view picture group (③). In this case, an inter-view picture group refers to a coded picture in which all slices refer only to slices having the same picture sequence number. For example, an inter-view picture group refers to a coded picture that only refers to a slice in a different view and does not refer to a slice in the current view. During decoding of multi-view video, inter-view random access may be required. Inter-view group-of-picture identification information may be necessary to enable efficient random access. Also, inter-view reference information may be necessary for inter-view prediction. Therefore, the inter-view picture group identification information can be used to obtain inter-view reference information. Also, the inter-view picture group identification information may be used to add reference pictures for inter-view prediction during construction of a reference picture list. Also, inter-view picture group identification information may be used to manage added reference pictures for inter-view prediction. For example, reference pictures can be classified into inter-view picture groups and non-inter-view picture groups, and the classified reference pictures can then be marked so that reference pictures that cannot be used for inter-view prediction will not be used. Meanwhile, the inter-view picture group identification information may be applied to a hypothetical reference decoder. Details of the inter-view picture group identification information will be explained later with reference to FIG. 6 .

View identification information refers to information used to distinguish a picture in the current view from pictures in a different view (④). During encoding of a video signal, POC (picture order number) or "frame_num (frame number)" may be used to identify each picture. In case of a multi-view video sequence, inter-view prediction may be performed. Therefore, identification information for distinguishing a picture in a current view from a picture in another view is required. Therefore, it is necessary to define view identification information for identifying a view of a picture. This view identification information can be obtained from the header area of the video signal. For example, the header area may be a NAL header area, an extension area of a NAL header, or a segment header area. The view identification information is used to obtain information on a picture in a view different from that of the current picture, and a video signal can be decoded using the information on the picture in the different view. The view identification information can be applied to the overall encoding/decoding process of the video signal. And, view identification information can be applied to multi-view video encoding using 'frame_num' which considers a view instead of a specific view identifier.

Meanwhile, the entropy decoding unit 200 performs entropy decoding on the parsed bitstream, and then extracts coefficients, motion vectors, and the like of each macroblock. The inverse quantization/inverse transformation unit 300 obtains a converted coefficient value by multiplying the received quantization value by a constant, and then inverse transforms the coefficient value to reconstruct a pixel value. Using the reconstructed pixel values, the intra prediction unit 400 performs intra prediction from decoded samples in the current picture. Meanwhile, the deblocking filtering unit 500 is applied to each coded macroblock to reduce block distortion. Filters smooth block edges to improve the image quality of decoded frames. The choice of filtering process depends on the boundary strength and gradient of the image samples around the edge. The filtered picture is output or stored in the decoded picture buffer unit 600 to be used as a reference picture.

The decoded picture buffer unit 600 plays a role of storing or opening a previously encoded picture to perform inter prediction. In this case, in order to store a picture in the decoded picture buffer unit 600 or to open a picture, "frame_num" and POC (picture order number) of each picture are used. Therefore, since a picture in a view different from that of the current picture exists in a previously encoded picture, view information for identifying a view of a picture may be used together with 'frame_num' and the POC. The decoded picture buffer unit 600 includes a reference picture storage unit 610 , a reference picture list construction unit 620 , and a reference picture management unit 650 . The reference picture storage unit 610 stores pictures to be referenced for encoding a current picture. The reference picture list construction unit 620 constructs a list of reference pictures for inter-picture prediction. In multi-view video coding, inter-view prediction may be required. So, if the current picture references a picture in another view, it may be necessary to construct a reference picture list for inter-view prediction. In this case, the reference picture list construction unit 620 can use information on views during generation of a reference picture list for inter-view prediction. Details of the reference picture list construction unit 620 will be explained later with reference to FIG. 3 .

FIG. 3 is an internal block diagram of the reference picture list construction unit 620 according to an embodiment of the present invention.

The reference picture list construction unit 620 includes a variable derivation unit 625 , a reference picture list initialization unit 630 and a reference list rearrangement unit 640 .

The variable derivation unit 625 derives variables used for reference picture list initialization. For example, a variable can be derived using "frame_num" indicating a picture identification number. Specifically, the variables FrameNum (frame number) and FrameNumWrap (frame number wrap) can be used for each short-term reference picture. First, the variable FrameNum is equal to the value of the syntax element frame_num. The variable FrameNumWrap can be used in the decoded picture buffer unit 600 to assign a smaller number to each reference picture. Also, the variable FrameNumWrap can be derived from the variable FrameNum. Therefore, the variable PicNum (picture number) can be derived using the derived variable FrameNumWrap. In this case, the variable PicNum may refer to the identification number of the picture used by the decoding picture buffer unit 600 . In the case of indicating a long-term reference picture, the variable LongTermPicNum (long-term picture number) can be used.

To construct a reference picture list for inter-view prediction, a first variable (eg, ViewNum) may be derived to construct a reference picture list for inter-view prediction. For example, the second variable (for example, ViewId (view identifier)) can be derived using "view_id (view identifier)" for identifying a view of a picture. First, the second variable may be equal to the value of the syntax element "view_id". Also, a third variable (eg, ViewIdWrap (view identifier wrap)) may be used in the decoded picture buffer unit 600 to assign a smaller view ID to each reference picture, and may be derived from the second variable. In this case, the first variable ViewNum may refer to a view identification number of a picture used by the decoding picture buffer unit 600 . However, since the number of reference pictures used for inter-view prediction may be relatively smaller than the number of reference pictures used for temporal prediction in multi-view video coding, another variable indicating the view identification number of a long-term reference picture may not be defined .

The reference picture list initialization unit 630 initializes the reference picture list using the variables described above. In this case, initialization procedures for the reference picture list may differ according to slice types. For example, in case of decoding P-slices, reference indices may be assigned based on decoding order. In case of decoding B-slices, reference indexes may be allocated based on picture output order. In case of initializing a reference picture list for inter-view prediction, an index may be assigned to a reference picture based on a first variable, ie, a variable derived from view information.

The reference picture list rearrangement unit 640 plays a role of improving compression efficiency by assigning smaller indexes to frequently referenced pictures in the initialized reference picture list. This is because smaller bits are allocated if the reference index used for encoding becomes smaller.

Furthermore, the reference picture list rearrangement unit 640 includes a segment type checking unit 642 , a reference picture list 0 rearrangement unit 643 and a reference picture list 1 rearrangement unit 645 . If the initialized reference picture list is input, the slice type checking unit 642 checks the type of the slice to be decoded and then decides whether to rearrange the reference picture list 0 or the reference picture list 1 . So, reference picture list 0/1 rearrangement units 643, 645 perform rearrangement of reference picture list 0 if the slice type is not I-slice, and additionally also perform reference picture list 1 if the slice type is B-slice rearrangement. Therefore, after the rearrangement process ends, the reference picture list is constructed.

Reference picture list 0/1 rearrangement units 643, 645 include identification information obtaining units 643A, 645A and reference index assignment changing units 643B, 645B, respectively. If the reordering of the reference picture list is performed according to the flag information indicating whether to perform the reordering of the reference picture list, the identification information obtaining unit 643A, 645A receives the identification information (reordering_of_pic_nums_idc) indicating the allocation method of the reference index. And, the reference index allocation changing unit 643B, 645B rearranges the reference picture list by changing the allocation of the reference index according to the identification information.

And, another method may be used to operate the reference picture list rearranging unit 640 . For example, rearrangement may be performed by checking the NAL unit type passed before passing through the segment type checking unit 642 and then classifying the NAL unit type into MVC NAL cases and non-MVC NAL cases.

The reference picture management unit 650 manages reference pictures to more flexibly perform inter prediction. For example, a memory management control operation method and a sliding window method may be used. This is to realize efficient memory management by unifying the memory into one memory to manage the reference picture memory and the non-reference picture memory and utilize a smaller memory. During multi-view video coding, since pictures in a view direction have the same picture order number, information for identifying the view of each picture can be used to label pictures along the view direction. And, the inter prediction unit 700 may use the reference picture managed in the above manner.

The inter prediction unit 700 performs inter prediction using reference pictures stored in the decoded picture buffer unit 600 . Inter-coded macroblocks can be divided into macroblock partitions. Also, each macroblock partition can be predicted from one or two reference pictures. The inter prediction unit 700 includes a motion compensation unit 710, a luma compensation unit 720, a luma difference prediction unit 730, a view synthesis prediction unit 740, a weighted prediction unit 750, and the like.

The motion compensation unit 710 compensates the motion of the current block using the information delivered from the entropy decoding unit 200 . The motion vectors of the neighboring blocks of the current block are extracted from the video signal, and then the motion vector prediction of the current block is derived from the motion vectors of the neighboring blocks. And, the motion of the current block is compensated using the derived motion vector prediction and the differential motion vector extracted from the video signal. And, motion compensation can be performed using one reference picture or a plurality of pictures. During multi-view video encoding, in case a current picture refers to a picture in a different view, motion compensation may be performed using reference picture list information for inter-view prediction stored in the decoded picture buffer unit 600 . And, it is also able to perform motion compensation using view information for identifying a view of a reference picture. The direct mode is an encoding mode for predicting motion information of a current block from motion information used to encode the block. Compression efficiency is improved because this approach can save the number of bits needed to encode motion information. For example, the time direction mode predicts motion information about a current block using correlation of motion information in a time direction. Using a method similar to this method, the present invention can predict motion information about a current block using correlation of motion information in a view direction.

Meanwhile, in a case where an input bit stream corresponds to a multi-view video, since each view sequence is obtained by a different camera, a luminance difference occurs due to internal and external factors of the camera. In order to prevent this, the brightness compensation unit 720 compensates the brightness difference. During performing luminance compensation, flag information indicating whether to perform luminance compensation on a specific layer of a video signal may be used. For example, luminance compensation may be performed using flag information indicating whether to perform luminance compensation for a corresponding slice or macroblock. During performing luma compensation using flag information, luma compensation can be applied to various macroblock types (eg, inter 16x16 mode, B-skip mode, direct mode, etc.).

During performing luminance compensation, the current block can be reconstructed using information on neighboring blocks or information on blocks in a view different from that of the current block. And, it is also possible to use the luminance difference value of the current block. In this case, if the current block refers to a block in a different view, brightness compensation can be performed using reference picture list information for inter-view prediction stored in the decoded picture buffer unit 600 . In this case, the luminance difference value of the current block indicates a difference between an average pixel value of the current block and an average pixel value of a reference block corresponding to the current block. As an example of using the luminance difference value, a luminance difference predicted value of the current block is obtained using neighboring blocks of the current block, and a difference between the luminance difference value and the luminance difference predicted value (luminance difference residual value) is used. Therefore, the decoding unit can reconstruct the luma difference value of the current block using the luma difference residual value and the luma difference predictive value. During obtaining the luma difference predictor for the current block, information about neighboring blocks may be used. For example, the brightness difference value of the adjacent block may be used to predict the brightness difference value of the current block. Before prediction, it is checked whether the reference index of the current block is equal to the reference index of the neighboring block. Based on the result of the check, it is then decided which neighbor or value will be used.

The view synthesis prediction unit 740 is used to synthesize pictures in the virtual view using pictures in views adjacent to the view of the current picture, and to predict the current picture using the synthesized pictures in the virtual view. The decoding unit can decide whether to synthesize pictures in the virtual view according to the inter-view synthesis prediction identifier passed from the encoding unit. For example, if view_synthesize_pred_flag=1 or view_syn_pred_flag=1, the slice or macroblock in the virtual view is synthesized. In this case, when the inter-view synthesis prediction identifier informs that a virtual view will be generated, the picture in the virtual view may be generated using view information for identifying a view of the picture. And, during prediction of a current picture from a synthesized picture in the virtual view, view information may be used to use a picture in the virtual view as a reference picture.

The weighted prediction unit 750 is used to compensate for the phenomenon that the picture quality of a sequence is significantly reduced when coding a sequence whose brightness changes temporarily. In MVC, weighted prediction can be performed to compensate for differences in luminance with sequences in different views, as well as for sequences whose luminance temporarily changes. For example, weighted prediction methods can be classified into explicit weighted prediction methods and implicit weighted prediction methods.

Specifically, the explicit weighted prediction method can use one reference picture or two reference pictures. In the case of using one reference picture, a prediction signal is generated by multiplying a prediction signal corresponding to motion compensation by a weighting coefficient. In the case of using two reference pictures, a prediction signal is generated by adding an offset value to a value obtained by multiplying a prediction signal corresponding to motion compensation by a weighting coefficient.

And, implicit weighted prediction performs weighted prediction using a distance from a reference picture. As a method of obtaining the distance from the reference picture, for example, POC (Picture Order Number) indicating the output order of pictures can be used. In this case, the POC can be obtained by considering the identity of the view of each picture. In obtaining weighting coefficients about pictures in different views, view information for identifying views of pictures may be used to obtain distances between views of respective pictures.

During encoding of a video signal, depth information may be used for a specific application or for another purpose. In this case, the depth information may refer to information capable of indicating a disparity difference between views. For example, a disparity vector can be obtained by inter-view prediction. And, the obtained disparity vector should be delivered to the decoding device for disparity compensation of the current block. However, if a depth map is obtained and then passed to a decoding device, it is possible to derive a disparity vector from the depth map (or a disparity map) without passing the disparity vector to a decoding device. In this case, there is an advantage in that the number of bits of depth information to be delivered to the decoding device can be reduced. Therefore, by deriving the disparity vector from the depth map, a new method of disparity compensation can be provided. Therefore, in case pictures in different views are used during derivation of the disparity vector from the depth map, view information for identifying the view of the picture may be used.

An inter-predicted or intra-predicted picture through the process explained above is selected according to the prediction mode to reconstruct the current picture. In the following description, various embodiments of a decoding method providing an efficient video signal are explained.

FIG. 4 is a diagram of a hierarchical structure for providing level information of view scalability of a video signal according to one embodiment of the present invention.

Referring to FIG. 4 , level information on each view may be decided by considering inter-view reference information. For example, since it is impossible to decode P-pictures and B-pictures without I-pictures, "level=0" may be assigned to a base view whose inter-view picture group is an I-picture, and to a base view whose inter-view picture group is a P-picture. The base view of a picture is assigned "level=1", and "level=2" is assigned to the base view whose inter-view picture group is a B-picture. However, it is also possible to randomly decide the level information according to certain criteria.

The level information may be randomly decided according to a certain standard or without a standard. For example, in the case of determining level information based on views, view V0 as a base view can be set to view level 0, a view of a picture predicted using a picture in one view can be set to view level 1, and multiple views will be used The view of the picture in the predicted picture is set to view level 2. In this case, at least one view sequence with compatibility with legacy decoders (eg, H.264/AVC, MPEG-2, MPEG-4, etc.) may be required. This base view becomes the basis for multi-view coding, which can correspond to a reference view used to predict another view. A sequence corresponding to a base view in MVC (Multi-View Video Coding) can be configured into an independent bitstream by encoding using a conventional sequence encoding scheme (MPEG-2, MPEG-4, H.263, H.264, etc.). The sequence corresponding to the base view can be compatible with H.264/AVC or may not be compatible. However, sequences in H.264/AVC compatible views correspond to base views.

As can be seen in FIG. 4 , view V2 of pictures predicted using pictures in view V0 , view V4 of pictures predicted using pictures in view V2 , view V6 of pictures predicted using pictures in view V4 And view V7 of the picture predicted using the picture in view V6 is set to view level 1 . And, the view V1 of the picture predicted using the pictures in the views V0 and V2 , the view V3 predicted in the same way, and the view V5 predicted in the same way may be set as view level 2 . So, in case a user's decoder cannot watch a multi-view video sequence, it only decodes the sequence in the view corresponding to view level 0. In case the user's decoder is restricted by the class information, only the restricted view level information can be decoded. In this case, the class means that technical elements of algorithms used in the video encoding/decoding process are standardized. Specifically, a class is a set of technical elements required for decoding a bit sequence of a compressed sequence and may be a substandard.

According to another embodiment of the present invention, the level information can be changed according to the position of the camera. For example, assume that views V0 and V1 are sequences obtained by cameras located at the front, views V2 and V3 are sequences obtained by cameras located at the rear, views V4 and V5 are sequences obtained by cameras located at the left, and views V6 and V7 is the sequence acquired by the camera on the right, then you can set views V0 and V1 to view level 0, views V2 and V3 to view level 1, views V4 and V5 to view level 2, and view V6 and V7 are set to view level 3. Alternatively, the level information may change according to camera calibration. Alternatively, the class information may be randomly decided without being based on a specific standard.

FIG. 5 is a diagram of a NAL unit configuration including level information in an extension area of a NAL header according to one embodiment of the present invention.

Referring to FIG. 5, a NAL unit basically includes a NAL header and an RBSP. The NAL header includes flag information (nal_ref_idc) indicating whether a slice becoming a reference picture of the NAL unit is included and an identifier (nal_unit_type) indicating the type of the NAL unit. And, the NAL header may further include level information (view_level) indicating information on a hierarchical structure to provide view scalability.

The compressed raw data is stored in the RBSP, and the last bit of the RBSP is added to the last part of the RBSP to indicate that the length of the RBSP is an 8-bit multiplication number. As the type of NAL unit, there are IDR (instant decoding refresh: instantaneous decoding refresh), SPS (sequence parameter set: sequence parameter set), PPS (picture parameter set: picture parameter set), SEI (additional enhancement information: supplemental enhancement information), etc.

The NAL header includes information on the view identifier. And, during decoding according to the view level, the video sequence of the corresponding view level is decoded with reference to the view identifier.

A NAL unit includes a NAL header 51 and a slice layer 53 . The NAL header 51 includes a NAL header extension 52 . And, the segment layer 53 includes a segment header 54 and segment data 55 .

The NAL header 51 includes an identifier (nal_unit_type) indicating the type of the NAL unit. For example, the identifier indicating the NAL unit type may be an identifier on scalable coding and multi-view video coding. In this case, the NAL header extension 52 may include flag information for distinguishing whether the current NAL is a NAL for scalable video coding or a NAL for multi-view video coding. And, according to the flag information, the NAL header extension 52 may include extension information on the current NAL. For example, in case a current NAL is a NAL for multi-view video coding according to flag information, the NAL header extension 52 may include level information (view_level) indicating information on a hierarchical structure to provide view scalability.

FIG. 6 is a diagram of an overall prediction structure of a multi-view video signal according to an embodiment of the present invention, for explaining the concept of an inter-view picture group.

Referring to FIG. 6 , T0 to T100 on the horizontal axis indicate frames according to time, and S0 to S7 on the vertical axis indicate frames according to views. For example, a picture at T0 refers to frames captured by different cameras at the same time zone T0, while a picture at S0 refers to a sequence captured by a single camera at different time zones. And, arrows in the figure indicate prediction directions and prediction orders of respective pictures. For example, picture P0 in view S2 on time zone T0 is a picture predicted from I0, which becomes a reference picture for picture P0 in view S4 on time zone T0. And, it becomes a reference picture for pictures B1 and B2 on time zones T4 and T2 in view S2, respectively.

During multi-view video decoding, inter-view random access may be required. So, by reducing decoding effort, access to random views should be possible. In this case, implementing efficient access may require the concept of an inter-view picture group. An inter-view picture group refers to a coded picture in which all slices refer only to slices with the same picture order number. For example, an inter-view picture group refers to a coded picture that only refers to a slice in a different view and does not refer to a slice in the current view. In FIG. 6 , if the picture I0 in the view S0 in the time zone T0 is an inter-view picture group, then all the pictures in different views in the same time zone, that is, in the time zone T0, become an inter-view picture group. As another example, if picture I0 in view S0 in time zone T8 is an inter-view picture group, all pictures in different views in the same time zone, that is, time zone T8, are inter-view picture groups. Likewise, all pictures in T16, ..., T96, and T100 also become inter-view picture groups.

FIG. 7 is a diagram of a prediction structure for explaining the concept of a newly defined inter-view picture group according to an embodiment of the present invention.

In the overall prediction structure of MVC, a GOP may start with an I-picture. Also, the I-picture is compatible with H.264/AVC. Therefore, all inter-view picture groups compatible with H.264/AVC can always be I-pictures. However, more efficient encoding is possible in the case where P-pictures are used instead of I-pictures. Specifically, using a prediction structure that enables a GOP to start with a P-picture compatible with H.264/AVC enables more efficient encoding.

In this case, if the inter-view picture group is redefined, all slices become coded pictures capable of referring not only slices in frames on the same time zone but also slices in the same view on different time zones. However, in the case of referring to slices on different time zones in the same view, it may be limited to only inter-view picture groups compatible with H.264/AVC. For example, the P-picture at the timing point T8 in view S0 in FIG. 6 may become a newly defined inter-view picture group. Likewise, the P-picture at the timing point T96 in the view S0 or the P-picture at the timing point T100 in the view S0 can become a newly defined inter-view picture group. And, it is able to define an inter-view picture group only when it is a base view.

After an inter-view picture group has been decoded, all sequentially coded pictures are decoded from the pictures decoded before the inter-view picture group in output order without inter prediction.

Considering the overall coding structure of multi-view video shown in Figure 6 and Figure 7, because the inter-view reference information of the inter-view picture group is different from the inter-view reference information of the non-inter-view picture group, it is necessary to identify the information according to the inter-view picture group Inter-view picture groups and non-inter-view picture groups are distinguished from each other.

The inter-view reference information refers to information capable of identifying a prediction structure between inter-view pictures. This can be obtained from the data field of the video signal. For example, available from the Sequence Parameter Sets area. And, inter-view reference information may be identified using the number of reference pictures and view information on the reference pictures. For example, the number of all views is obtained and then view information for identifying each view may be obtained based on the number of all views. And, the number of reference pictures of the reference direction for each view may be obtained. According to the number of reference pictures, view information on each reference picture can be obtained. In this way, inter-view reference information can be obtained. And, inter-view reference information can be identified by distinguishing an inter-view picture group from a non-inter-view picture group. This can be obtained using inter-view picture group identification information indicating whether a coded slice in the current NAL is an inter-view picture group. Details of the inter-view picture group identification information are explained with reference to FIG. 8 as follows.

Fig. 8 is a schematic block diagram of an apparatus for decoding multi-view video using inter-view group of picture identification information according to an embodiment of the present invention.

Referring to FIG. 8 , a decoding device according to an embodiment of the present invention includes a bitstream decision unit 81 , an inter-view picture group identification information acquisition unit 82 , and a multi-view video decoding unit 83 .

If a bit stream is input, the bit stream decision unit 81 decides whether the input bit stream is an encoded bit stream for scalable video encoding or an encoded bit stream for multiview video encoding. This can be determined by flag information included in the bitstream.

If, as a result of the decision, the input bit stream is a bit stream for multi-view video encoding, the inter-view picture group identification information obtaining unit 82 can obtain the inter-view picture group identification information. If the obtained inter-view picture group identification information is "true", it means that the current NAL coded segment is an inter-view picture group. If the obtained inter-view picture group identification information is "false (false)", it means that the current NAL coded segment is a non-inter-view picture group. The inter-view picture group identification information can be obtained from the extension area or the slice layer area of the NAL header.

The multi-view video decoding unit 83 decodes the multi-view video according to the inter-view picture group identification information. According to the overall coding structure of a multi-view video sequence, the inter-view reference information of an inter-view picture group is different from the inter-view reference information of a non-inter-view picture group. So, for example, during adding reference pictures for inter-view prediction, the inter-view picture group identification information can be used to generate the reference picture list. And, it is also possible to manage reference pictures used for inter-view prediction using inter-view picture group identification information. Also, the inter-view picture group identification information may be applied to a hypothetical reference decoder.

As another example of using inter-view picture group identification information, in the case of using information in different views for each decoding process, inter-view reference information included in a sequence parameter set may be used. In this case, information for distinguishing whether the current picture is an inter-view picture group or a non-inter-view picture group, ie, inter-view picture group identification information, may be required. Therefore, different inter-view reference information can be used for each decoding process.

FIG. 9 is a flowchart of a process for generating a reference picture list according to an embodiment of the present invention.

Referring to FIG. 9 , the decoded picture buffer unit 600 plays a role of storing or opening a previously encoded picture to perform inter-picture prediction.

First, a picture encoded prior to a current picture is stored in the reference picture storage unit 610 to be used as a reference picture (S91).

During multi-view video coding, because some of previously coded pictures are located in different views than the view of the current picture, view information for identifying the view of a picture can be used to utilize these pictures as reference pictures. Therefore, the decoder should obtain view information for identifying a view of a picture (S92). For example, the view information may include "view_id" for identifying a view of a picture.

The decoded picture buffer unit 600 needs to derive the variables used therein to generate the reference picture list. Because inter-view prediction may be required for multi-view video coding, it may be necessary to generate a reference picture list for inter-view prediction if the current picture references pictures in different views. In this case, the decoded picture buffer unit 600 needs to use the obtained view information to derive variables used to generate a reference picture list for inter-view prediction (S93).

According to the segment type of the current segment, a reference picture list for temporal prediction or a reference picture list for inter-view prediction may be generated by different methods ( S94 ). For example, if the slice type is a P/SP slice, reference picture list 0 is generated (S95). In case the slice type is a B-slice, reference picture list 0 and reference picture list 1 are generated (S96). In this case, reference picture list 0 or 1 may include only a reference picture list for temporal prediction or both a reference picture list for temporal prediction and a reference picture list for inter-view prediction. This will be explained in detail later with reference to FIGS. 8 and 9 .

The initialized reference picture list undergoes a process for assigning smaller numbers to frequently referenced pictures to further increase the compression rate (S97). And, this may be called a rearrangement process for a reference picture list, which will be explained in detail later with reference to FIGS. 12 to 19 . The current picture is decoded using the rearranged reference picture list, and the decoded picture buffer unit 600 needs to manage the decoded reference pictures to operate the buffer more efficiently (S98). The reference picture managed through the above process is read out by the inter prediction unit 700 to be used for inter prediction. In multi-view video coding, inter prediction may include inter-view prediction. In this case, a reference picture list for inter-view prediction can be used.

A detailed example of a method for generating a reference picture list according to slice types is explained with reference to FIGS. 10 and 11 as follows.

FIG. 10 is a diagram for explaining a method of initializing a reference picture list when a current slice is a P-slice according to one embodiment of the present invention.

Referring to FIG. 10, times are indicated by T0, T1, ... TN, and views are indicated by V0, V1, ..., V4. For example, the current picture indicates the picture at time T3 in view V4. And, the slice type of the current picture is a P-slice. "PN" is an abbreviation of a variable PicNum, "LPN" is an abbreviation of a variable LongTermPicNum, and "VN" is an abbreviation of a variable ViewNum. A number appended to the end part of each variable indicates the time (for PN or LPN) of each picture or the index of the view (for VN) of each picture. This can be applied to FIG. 11 in the same manner.

A reference picture list for temporal prediction or a reference picture list for inter-view prediction may be generated differently according to a slice type of a current slice. For example, the segment type in FIG. 12 is a P/SP segment. In this case, reference picture list 0 is generated. Specifically, reference picture list 0 may include a reference picture list for temporal prediction and/or a reference picture list for inter-view prediction. In this embodiment, it is assumed that the reference picture list includes both the reference picture list for temporal prediction and the reference picture list for inter-view prediction.

Various methods exist to order reference pictures. For example, reference pictures may be arranged according to the order of decoding or picture output. Alternatively, reference pictures may be arranged based on variables derived using view information. Alternatively, reference pictures may be arranged according to inter-view reference information indicating an inter-view prediction structure.

In case of a reference picture list for temporal prediction, short-term reference pictures and long-term reference pictures may be arranged based on a decoding order. For example, they may be arranged according to the value of the variable PicNum or LongTermPicNum derived from a value indicating a picture identification number (for example, frame_num or Longtermframeidx). First, short-term reference pictures can be initialized before long-term reference pictures. The arrangement order of the short-term reference pictures may be set from a reference picture having the highest value of variable PicNum to a reference picture having the lowest value of the variable. For example, among PN0 to PN2, short-term reference pictures may be arranged in the order of PN1 having the highest variance, PN2 having an intermediate variance, and PN0 having the lowest variance. The arrangement order of the long-term reference pictures may be set from a reference picture having the lowest value of the variable LongTermPicNum to a reference picture having the highest value of the variable. For example, the long-term reference pictures may be arranged in the order of LPN0 having the highest variance and LPN1 having the lowest variance.

In case of a reference picture list for inter-view prediction, the reference pictures may be arranged based on the first variable ViewNum derived using view information. Specifically, the reference pictures may be arranged in the order from the reference picture with the highest value of the first variable (ViewNum) to the reference picture with the lowest value of the first variable (ViewNum). For example, among VN0, VN1, VN2, and VN3, reference pictures may be arranged in the order of VN3, VN2, VN1 having the highest variance and VN0 having the lowest variance.

Therefore, both the reference picture list for temporal prediction and the reference picture list for inter-view prediction can be managed as one reference picture list. Alternatively, the reference picture list for temporal prediction and the reference picture list for inter-view prediction may be managed separately as separate reference picture lists. In the case where both the reference picture list for temporal prediction and the reference picture list for inter-view prediction are managed as one reference picture list, they may be initialized sequentially or simultaneously. For example, in the case where both the reference picture list for temporal prediction and the reference picture list for inter-view prediction are initialized according to the order, the reference picture list for temporal prediction is preferentially initialized, and the reference picture list for inter-view prediction The reference picture list is then additionally initialized. This concept can also be applied to Figure 11.

A case where a slice type of a current picture is a B-slice is explained with reference to FIG. 11 as follows.

FIG. 11 is a diagram for explaining a method of initializing a reference picture list when a current slice is a B-slice according to one embodiment of the present invention.

Referring to FIG. 9 , in case the slice type is a B-slice, reference picture list 0 and reference picture list 1 are generated. In this case, reference picture list 0 or reference picture list 1 may include only a reference picture list for temporal prediction or both a reference picture list for temporal prediction and a reference picture list for inter-view prediction.

In the case of a reference picture list for temporal prediction, the short-term reference picture arrangement method may be different from the long-term reference picture arrangement method. For example, in the case of short-term reference pictures, the reference pictures may be arranged according to picture order numbers (abbreviated as POC hereinafter). In the case of long-term reference pictures, the reference pictures may be arranged according to the variable (LongtermPicNum) value. Also, short-term reference pictures may be initialized before long-term reference pictures.

In the order of arranging the short-term reference pictures of the reference picture list 0, among the reference pictures having a POC value smaller than the current picture, the reference pictures are preferentially arranged from the reference picture having the highest POC value to the reference picture having the lowest POC value, and Then among the reference pictures with POC values larger than the current picture, the order is arranged from the reference picture with the lowest POC value to the reference picture with the highest POC value. For example, reference pictures may be preferentially arranged from PN1 to PN0 having the highest POC value among reference pictures PN0 and PN1 having a POC value smaller than the current picture, and then from reference pictures PN3 and PN4 having a POC value smaller than the current picture The PN3 to PN4 ranks with the lowest POC values in .

In the order in which the long-term reference pictures of the reference picture list 0 are arranged, the reference pictures are arranged from the reference picture with the lowest variable LongtermPicNum to the reference picture with the highest variable. For example, reference pictures are arranged from LPN0 having the lowest value among LPNO and LPN1 to LPN1 having the second lowest variable.

In case of a reference picture list for inter-view prediction, the reference pictures may be arranged based on the first variable ViewNum derived using view information. For example, in the case of reference picture list 0 for inter-view prediction, it is possible to go from a reference picture having the highest first variable value among reference pictures having a lower first variable value than the current picture to a reference picture having the lowest first variable value. A reference picture of a variable value to rank the reference pictures. The reference pictures are then arranged from a reference picture having the lowest first variable value to a reference picture having the highest first variable value among the reference pictures having the first variable value greater than the current picture. For example, the reference pictures are preferentially arranged from VN1 having the highest first variable value among VN0 and VN1 having the first variable value smaller than the current picture to VN0 having the lowest first variable value, and then from among VN0 having the first variable value smaller than the current picture Among the first variable values VN3 and VN4 of the picture, VN3 having the lowest first variable value is arranged to VN4 having the highest first variable value.

In the case of reference picture list 1, the method of arranging reference list 0 explained above can be similarly applied.

First, in the case of the reference picture list for temporal prediction, in the order of arranging the short-term reference pictures of the reference picture list 1, the reference picture with the lowest POC value is preferentially selected from the reference picture with the POC value greater than that of the current picture. picture to the reference picture with the highest POC value, and then arrange from the reference picture with the highest POC value to the reference picture with the lowest POC value among the reference pictures with POC values smaller than the current picture. For example, reference pictures may be preferentially arranged from PN3 to PN4 having the lowest POC value among reference pictures PN3 and PN4 having a POC value greater than the current picture, and then from reference pictures PN0 and PN4 having POC values greater than the current picture. PN1 to PN0 having the highest POC value among PN1 are arranged.

In the order in which the long-term reference pictures of the reference picture list 1 are arranged, the reference pictures are arranged from the reference picture with the lowest variable LongtermPicNum to the reference picture with the highest variable. For example, reference pictures are arranged from LPN0 having the lowest value among LPN0 and LPN1 to LPN1 having the lowest variable.

In case of a reference picture list for inter-view prediction, the reference pictures may be arranged based on the first variable ViewNum derived using view information. For example, in the case of reference picture list 1 for inter-view prediction, it is possible to go from the reference picture with the lowest first variable value to the one with the highest first variable value among the reference pictures with the first variable value larger than the current picture The reference picture of the value to arrange the reference picture. The reference pictures are then arranged from the reference picture with the highest first variable value to the reference picture with the lowest first variable value among the reference pictures with the first variable value smaller than the current picture. For example, the reference pictures are preferentially arranged from VN3 having the lowest first variable value to VN4 having the highest first variable value among VN3 and VN4 having the first variable value larger than the current picture, and then among VN3 and VN4 having the first variable value smaller than the current picture VN0 and VN1 of the first variable value of the picture are arranged from VN1 having the highest first variable value to VN0 having the lowest first variable value.

The reference picture list initialized through the above process is passed to the reference picture list rearrangement unit 640 . The initialized reference picture list is then rearranged for more efficient coding. The rearrangement process is used to reduce the bit rate by assigning a small number to the reference picture with the highest probability of being selected as a reference picture by manipulating the decoded picture buffer. Various methods of rearranging the reference picture list are explained with reference to FIGS. 12 to 19 as follows.

FIG. 12 is an internal block diagram of the reference picture list rearrangement unit 640 according to an embodiment of the present invention.

Referring to FIG. 12 , the reference picture list rearrangement unit 640 basically includes a slice type check unit 642 , a reference picture list 0 rearrangement unit 643 , and a reference picture list 1 rearrangement unit 645 .

Specifically, the reference picture list 0 rearrangement unit 643 includes a first identification information obtaining unit 643A and a first reference index allocation changing unit 643B. And, the reference picture list 1 rearrangement unit 645 includes a second identification obtaining unit 645A and a second reference index assignment changing unit 645B.

The segment type checking unit 642 checks the segment type of the current segment. Then decide whether to rearrange reference picture list 0 and/or reference picture list 1 according to the segment type. For example, if the slice type of the current slice is an I-slice, neither reference picture list 0 nor reference picture list 1 is rearranged. If the slice type of the current slice is a P-slice, only reference picture list 0 is rearranged. If the slice type of the current slice is a B-slice, both reference picture list 0 and reference picture list 1 are rearranged.

If the flag information for performing rearrangement of reference picture list 0 is "true" and if the slice type of the current slice is not an I-slice, the reference picture list 0 rearrangement unit 643 is activated. The first identification information obtaining unit 643A obtains identification information indicating a reference index assignment method. The first reference index assignment changing unit 643B changes the reference index assigned to each reference picture of the reference picture list 0 according to the identification information.

Likewise, if the flag information for performing rearrangement of reference picture list 1 is “true” and if the slice type of the current slice is a B-slice, the reference picture list 1 rearrangement unit 645 is activated. The second identification information obtaining unit 645A obtains identification information indicating a reference index assignment method. The second reference index assignment changing unit 645B changes the reference index assigned to each reference picture of the reference picture list 1 according to the identification information.

Therefore, reference picture list information for actual inter prediction is generated by the reference picture list 0 rearrangement unit 643 and the reference picture list 1 rearrangement unit 645 .

A method of changing a reference index assigned to each reference picture by the first or second reference index assignment changing unit 643B or 645B is explained with reference to FIG. 13 as follows.

FIG. 13 is an internal block diagram of the reference index assignment changing unit 643B or 645B according to one embodiment of the present invention. In the following description, the reference picture list 0 rearrangement unit 643 and the reference picture list 1 rearrangement unit 645 shown in FIG. 12 are explained together.

Referring to FIG. 13 , each of the first and second reference index allocation changing units 643B and 645B includes a reference index allocation changing unit 644A for temporal prediction, a reference index allocation changing unit 644B for long-term reference pictures, and a reference index allocation changing unit 644B for inter-view Predicted reference index allocation change unit 644C and reference index allocation change termination unit 644D. Based on the identification information obtained by the first and second identification information obtaining units 643A and 645A, the components in the first and second reference index assignment changing units 643B and 645B are activated, respectively. And, the rearrangement process is kept being performed until identification information for terminating the reference index assignment change is input.

For example, if identification information for changing allocation of reference indexes for temporal prediction is received from the first or second identification information obtaining unit 643A or 645A, the reference index allocation changing unit for temporal prediction 644A is activated. The reference index assignment changing unit 644A for temporal prediction obtains the picture number difference from the received identification information. In this case, the picture number difference refers to the difference between the picture number of the current picture and the predicted picture number. Also, the predicted picture number may refer to the number of the reference picture allocated just before. Therefore, the assignment of reference indexes can be changed using the obtained picture number difference. In this case, depending on the identification information, the picture number difference may be added/subtracted to/from the predicted picture number.

As another example, if identification information for changing the assignment of the reference index to a designated long-term reference picture is received, the reference index assignment changing unit 644B for the long-term reference picture is activated. The reference index assignment changing unit 644B for a long-term reference picture obtains the long-term reference picture number of the specified picture from the identification number.

As another example, if the identification information for changing the allocation of the reference index for inter-view prediction is received, the reference index allocation changing unit 644C for inter-view prediction is activated. The reference index assignment changing unit 644C for inter-view prediction obtains the view information difference from the identification information. In this case, the view information difference refers to the difference between the view number of the current picture and the predicted view number. And, the predicted view number may indicate the view number of the reference picture allocated just before. Therefore, assignment of reference indexes can be changed using the obtained view information difference. In this case, the view information difference may be added/subtracted to/from the predicted view number according to the identification information.

For another example, if identification information for terminating a reference index allocation change is received, the reference index allocation change termination unit 644D is activated. The reference index allocation change terminating unit 644D terminates the allocation change of the reference index according to the received identification information. Therefore, the reference picture list rearrangement unit 640 generates reference picture list information.

Therefore, reference pictures for inter-view prediction can be managed together with reference pictures for temporal prediction. Alternatively, reference pictures for inter-view prediction can be managed independently of reference pictures for temporal prediction. For this, new information for managing reference pictures for inter-view prediction may be required. This will be explained later with reference to FIGS. 15 to 19 .

Details of the reference index allocation changing unit 644C for inter-view prediction are explained with reference to FIG. 14 as follows.

FIG. 14 is a diagram for explaining a process of rearranging a reference picture list using view information according to one embodiment of the present invention.

Referring to FIG. 14, if the view number VN of the current picture is 3, if the size of the decoded picture buffer DPBsize is 4, and if the slice type of the current slice is a P-slice, the rearrangement process for reference picture list 0 is explained as follows .

First, the initially predicted view number is "3", which is the view number of the current picture. And, the initial arrangement of the reference picture list 0 used for inter-view prediction is "4, 5, 6, 2" (①). In this case, if identification information for changing allocation of a reference index for inter-view prediction by subtracting the view information difference is received, '1' is obtained as the view information difference from the received identification information. A new predicted view number (=2) is calculated by subtracting the view information difference (=1) from the predicted view number (=3). Specifically, the first index of reference picture list 0 for inter-view prediction is assigned to the reference picture with view number 2. And, a picture previously assigned to the first index may be moved to the last part of the reference picture list 0. Referring to FIG. Therefore, the rearranged reference picture list 0 is "2, 5, 6, 4" (②). Subsequently, if identification information for changing allocation of a reference index for inter-view prediction by subtracting the view information difference is received, "-2" is obtained as the view information difference from the identification information. A new predicted view number (=4) is then calculated by subtracting the view information difference (=-2) from the predicted view number (=2). Specifically, the second index of reference picture list 0 for inter-view prediction is assigned to the reference picture with view number 4. Therefore, the rearranged reference picture list 0 is "2, 4, 6, 5" (③). Subsequently, if the identification information for terminating the reference index allocation change is received, the reference picture list 0 finally having the rearranged reference picture list 0 is generated according to the received identification information (④). Therefore, the order of the finally generated reference picture list 0 for inter-view prediction is "2, 4, 6, 5".

For another example of rearranging the remaining pictures after the first index of reference picture list 0 for inter-view prediction has been allocated, the picture allocated to each index may be moved to a position just behind the corresponding picture. Specifically, the second index is assigned to the picture with view number 4, the third index is assigned to the picture (view number 5) to which the second index is assigned, and the fourth index is assigned to the picture to which the third index is assigned. Picture (view number 6). Therefore, the rearranged reference picture list 0 becomes "2, 4, 5, 6". And, the subsequent rearrangement process can be performed in the same manner.

The reference picture list generated through the process explained above is used for inter prediction. Both the reference picture list for inter-view prediction and the reference picture list for temporal prediction can be managed as one reference picture list. Alternatively, each of the reference picture list for inter-view prediction and the reference picture list for temporal prediction may be managed as a separate reference picture list. This is explained with reference to FIGS. 15 to 19 as follows.

FIG. 15 is an internal block diagram of the reference picture list rearrangement unit 640 according to another embodiment of the present invention.

Referring to FIG. 15 , in order to manage a reference picture list for inter-view prediction as a separate reference picture list, new information may be required. For example, in some cases, the reference picture list used for temporal prediction is rearranged, and then the reference picture list used for inter-view prediction is rearranged.

The reference picture list rearrangement unit 640 basically includes a reference picture list rearrangement unit for temporal prediction 910 , a NAL type check unit 960 and a reference picture list rearrangement unit for inter-view prediction 970 .

The reference picture list rearrangement unit 910 for temporal prediction includes a segment type checking unit 642, a third identification information obtaining unit 920, a third reference index allocation changing unit 930, a fourth identification information obtaining unit 940, and a fourth reference index allocation changing unit 940. Unit 950. The third reference index allocation changing unit 930 includes a reference index allocation changing unit for temporal prediction 930A, a reference index allocation changing unit for long-term reference pictures 930B, and a reference index allocation changing termination unit 930C. Likewise, the fourth reference index allocation changing unit 950 includes a reference index allocation changing unit for temporal prediction 950A, a reference index allocation changing unit for long-term reference pictures 950B, and a reference index allocation changing termination unit 950C.

The reference picture list rearrangement for temporal prediction unit 910 rearranges reference pictures for temporal prediction. The operation of the reference picture list rearrangement unit 910 for temporal prediction is the same as that of the aforementioned reference picture list rearrangement unit 640 shown in FIG. 10 except for the information of the reference picture used for inter-view prediction. Therefore, details of the reference picture list rearranging unit 910 for temporal prediction are omitted in the following description.

The NAL type checking unit 960 checks the NAL type of the received bitstream. If the NAL type is NAL for multi-view video coding, reference pictures for inter-view prediction are rearranged by the reference picture list rearrangement unit 970 for temporal prediction. The generated reference picture list for inter-view prediction is used for inter prediction together with the reference picture list generated by the reference picture list rearrangement unit 910 for temporal prediction. However, if the NAL type is not NAL for multi-view video coding, the reference picture list for inter-view prediction is not rearranged. In this case, only reference picture lists for temporal prediction are generated. And, the inter-view prediction reference picture list rearrangement unit 970 rearranges reference pictures used for inter-view prediction. This is explained in detail with reference to FIG. 16 as follows.

FIG. 16 is an internal block diagram of the reference picture list rearrangement unit 970 for inter-view prediction according to an embodiment of the present invention.

Referring to FIG. 16 , the reference picture list rearrangement unit 970 for inter-view prediction includes a segment type checking unit 642, a fifth identification information obtaining unit 971, a fifth reference index assignment changing unit 972, a sixth identification information obtaining unit 973, and a fifth identification information obtaining unit 973. Six reference indexes are assigned to change unit 974 .

The segment type checking unit 642 checks the segment type of the current segment. If so, then it is decided according to the slice type whether to perform the rearrangement of reference picture list 0 and/or reference picture list 1 . Details of the segment type checking unit 642 can be deduced from FIG. 10 , which are omitted in the following description.

Each of the fifth and sixth identification information obtaining units 971 and 973 obtains identification information indicating a reference index assignment method. And, each of the fifth and sixth reference index assignment changing units 972 and 974 changes a reference index assigned to each reference picture of the reference picture lists 0 and/or 1 . In this case, the reference index may simply refer to the view number of the reference picture. And, identification information indicating a reference index allocation method may be flag information. For example, if the flag information is true, the allocation of view numbers is changed. If the flag information is false, the view number rearrangement process may be terminated. If the flag information is true, each of the fifth and sixth reference index allocation changing units 972 and 974 may obtain the view number difference according to the flag information. In this case, the view number difference refers to the difference between the view number of the current picture and the view number of the predicted picture. Also, the view number of the predicted picture may refer to the view number of the reference picture allocated just before. The view number difference can then be used to change the view number assignment. In this case, the view number difference may be added/subtracted to/from the view number of the predicted picture according to the identification information.

Therefore, in order to manage a reference picture list for inter-view prediction as a separate reference picture list, it is necessary to newly define a syntax structure. As one example of the contents explained in FIGS. 15 and 16 , the syntax is explained with reference to FIGS. 17 , 18 and 19 as follows.

17 and 18 are diagrams of syntax for reference picture list rearrangement according to one embodiment of the present invention.

Referring to FIG. 17 , operations of the reference picture list rearrangement unit 910 for temporal prediction shown in FIG. 15 are represented as syntax. Compared with the blocks shown in FIG. 15 , the segment type checking unit 642 corresponds to S1 and S6 , and the fourth identification information obtaining unit 940 corresponds to S7 . Internal blocks of the third reference index assignment changing unit 930 correspond to S3, S4, and S5, respectively. And, internal blocks of the fourth reference index allocation changing unit 950 correspond to S8, S9, and S10, respectively.

Referring to FIG. 18 , operations of the NAL type checking unit 960 and the inter-view reference picture list rearranging unit 970 are represented as syntax. Compared with the respective blocks shown in FIGS. 15 and 16 , the NAL type checking unit 960 corresponds to S11, the segment type checking unit 642 corresponds to S13 and S16, the fifth identification information obtaining unit 971 corresponds to S14, and the sixth identification information The obtaining unit 973 corresponds to S17. The fifth reference index assignment changing unit 972 corresponds to S15, and the sixth reference index assignment changing unit 974 corresponds to S18.

FIG. 19 is a diagram of syntax for reference picture list rearrangement according to another embodiment of the present invention.

Referring to FIG. 19 , operations of the NAL type checking unit 960 and the inter-view reference picture list rearranging unit 970 are represented as syntax. Compared with the respective blocks shown in FIGS. 15 and 16 , the NAL type checking unit 960 corresponds to S21, the segment type checking unit 642 corresponds to S22 and S25, the fifth identification information obtaining unit 971 corresponds to S23, and the sixth identification information The obtaining unit 973 corresponds to S26. The fifth reference index assignment changing unit 972 corresponds to S24, and the sixth reference index assignment changing unit 974 corresponds to S27.

As described in the foregoing description, the reference picture list used for inter-view prediction may be used by the inter prediction unit 700 and may also be used to perform brightness compensation. Luminance compensation may be applied during performance of motion estimation/motion compensation. In a case where a current picture uses a reference picture in a different view, brightness compensation can be performed more efficiently using a reference picture list for inter-view prediction. Brightness compensation according to an embodiment of the present invention is explained as follows.

FIG. 20 is a diagram of a process for obtaining a luminance difference value of a current block according to one embodiment of the present invention.

Luminance compensation refers to the process used to decode adaptive motion compensated video signals based on changes in luminance. And, it can be applied to the prediction structure of the video signal, such as inter-view prediction, intra-view prediction and so on.

Luma compensation refers to a process for decoding a video signal using a luma difference residual and a luma difference prediction value corresponding to a block to be decoded. In this case, a luma difference prediction value may be obtained from neighboring blocks of the current block. A process for obtaining a luma difference predictor from a neighboring block may be decided using reference information for the neighboring block, and sequence and direction can be considered during searching the neighboring block. A neighboring block refers to a block that has already been decoded and also refers to a block decoded by considering redundancy with respect to a view or time in the same picture or a sequence decoded by considering redundancy in different pictures.

When comparing the similarity between a current block and a candidate reference block, the brightness difference between the two blocks should be taken into consideration. To compensate for luminance differences, a new motion estimation/compensation is performed. New SADs can be discovered using Equation 1.

[Formula 1]

$\begin{array}{c}{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; cur</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; T</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; m</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Tgr;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; no</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; ref</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; T</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; p</span>}}{\overset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; q</span>}}{\overset{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>\end{array}$

[Formula 2]

$\mathrm{<span\; class="notranslate">\; New\; SAD</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; m</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; no</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\text{<span class="notranslate"> \&Sigma;</span>}}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; cur</span>}}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; ref</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; |</span>$

In this case, "Mcurr" indicates the average pixel value of the current block, and "Mref" indicates the average pixel value of the reference block. "f(i, j)" indicates a pixel value of a current block, and "r(i+x, j+y)" indicates a pixel value of a reference block. By performing motion estimation based on the new SAD according to Equation 2, an average pixel difference value between the current block and the reference block can be obtained. And, the obtained average pixel difference value may be referred to as a luminance difference value (IC_offset).

In the case of performing motion estimation to which luminance compensation is applied, a luminance difference value and a motion vector are generated. And, brightness compensation is performed according to Formula 3 using the brightness difference value and the motion vector.

[Formula 3]

NewR(i, j)={f(i, j)-M _cur }-{r(i+x', j+y')-M _ruf (m+x', n+y')}

={f(i,j)-r(i+x',j+y')}-{M _cur -M _ref (m+x',n+y')}

＝{f(i, j)-r(i+x', j+y')}-IC_offsei

In this case, NewR(i, j) indicates a brightness compensation error value (residual value), and (x', y') indicates a motion vector.

The luminance difference value (Mcurr-Mref) should be passed to the decoding unit. The decoding unit performs brightness compensation in the following manner.

[Formula 4]

$\left.\begin{array}{c}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; NewR</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; cur</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; ref</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span>\\ <span\; class="notranslate">\; =</span><span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; NewR</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; IC</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; offset</span>}\end{array}\right|$

In Equation 4, NewR"(i,j) indicates the reconstructed luminance compensation error value (residual value), and f'(i,j) indicates the reconstructed pixel value of the current block.

In order to reconstruct the current block, the luma difference should be passed to the decoding unit. And, a luminance difference value may be predicted from information of neighboring blocks. In order to further reduce the number of bits to encode the luminance difference, only the difference (RIC_offset) between the luminance difference (IC_offset) of the current block and the luminance difference (predIC_offset) of the neighboring block may be sent. This is expressed as Equation 5.

[Formula 5]

RIC_offset=IC_offset-predIC_offset

FIG. 21 is a flowchart of a process for performing brightness compensation of a current block according to an embodiment of the present invention.

Referring to FIG. 21, first, a brightness difference value of an adjacent block indicating an average pixel difference value between adjacent blocks of a current block and a block referenced by the adjacent block is extracted from a video signal (S2110).

Subsequently, a luminance difference prediction value for luminance compensation of the current block is obtained using the luminance difference value (S2120). Therefore, the obtained luma difference prediction value can be used to reconstruct the luma difference value of the current block.

In obtaining the luminance difference predicted value, various methods can be used. For example, before predicting the luminance difference value of the current block from the luminance difference value of the neighboring block, it is checked whether the reference index of the current block is equal to the reference index of the neighboring block. It is then possible to decide which neighbor or value to use based on the result of the check. For another example, in obtaining the luma difference prediction value, flag information (IC_flag) indicating whether to perform luma compensation of the current block may be used. And, the flag information for the current block may also be predicted using the information of the neighboring blocks. For another example, both the reference index checking method and the flag information prediction method can be used to obtain the luma difference prediction value. This is explained in detail with reference to FIGS. 22 to 24 as follows.

FIG. 22 is a block diagram of a process for obtaining a luma difference prediction value of a current block using information on neighboring blocks according to one embodiment of the present invention. Referring to FIG.

Referring to FIG. 22 , during obtaining a luma difference prediction value of a current block, information on neighboring blocks may be used. In the present disclosure, a block may include a macroblock or a sub-macroblock. For example, the brightness difference value of the current block can be predicted using the brightness difference value of the neighboring blocks. Before doing this, it is checked whether the reference index of the current block is equal to the reference index of the neighboring block. Depending on the result of the check, it can then be decided which neighbor or value will be used. In FIG. 22 , "refIdxLX" indicates the reference index of the current block, and "refIdxLXN" indicates the reference index of the block-N. In this case, 'N' is a flag of a block adjacent to the current block and indicates A, B, or C. And, 'PredIC_offsetN' indicates a luminance difference value for luminance compensation of the adjacent block-N. If Block-C located at the upper right end of the current block cannot be used, Block-D can be used instead of Block-C. Specifically, information on Block-D can be used as information on Block-C. If both Block-B and Block-C cannot be used, Block-A can be used instead. That is, the information on Block-A can be used as the information on Block-B or Block-C.

For another example, during obtaining the luma difference prediction value, flag information (IC_flag) indicating whether to perform luma compensation of the current block may be used. Alternatively, both the reference index checking method and the flag information prediction method may be used during obtaining the luminance difference prediction value. In this case, if the flag information on the adjacent block indicates that luminance compensation is not performed, that is, if IC_falg==0, the luminance difference value "PredIC_offsetN" of the adjacent block is set to 0.

FIG. 23 is a flowchart of a process for performing brightness compensation using information on neighboring blocks according to one embodiment of the present invention.

Referring to FIG. 23 , the decoding unit extracts an average pixel value of a reference block, a reference index of a current block, a reference index of a reference block, etc. from a video signal, and can then obtain a luma difference prediction value of a current block using the extracted information. The decoding unit obtains the difference between the luminance difference value and the luminance difference prediction value of the current block (luminance difference residual value), and then can reconstruct the luminance of the current block using the obtained luminance difference residual value and the luminance difference prediction value difference. In this case, it is able to use information on neighboring blocks to obtain a luminance difference prediction value of the current block. For example, the luminance difference value of the current block can be predicted using the luminance difference value of the neighboring blocks. Before doing this, it is checked whether the reference index of the current block is equal to the reference index of the neighboring block. Depending on the result of the check, it can then be decided which neighbor or value will be used.

Specifically, a luminance difference value of an adjacent block indicating an average pixel difference value between an adjacent block of the current block and a block referenced by the adjacent block is extracted from the video signal (S2310).

Subsequently, it is checked whether the reference index of the current block is equal to the reference index of one of the neighboring blocks (S2320).

As a result of the checking step S2320, if there is at least one neighboring block having the same reference index as the current block, it is checked whether there is a corresponding neighboring block (S2325).

As a result of checking step S2325, if there is only one neighboring block with the same reference index as the current block, assign the luma difference value of the neighboring block with the same reference index as the current block to the luma difference predictor of the current block (S2330). Specifically, it is "PredIC_offset=PredIC_offsetN".

If there is no neighboring block with the same reference index as the current block as a result of checking step S2320, or if there are at least two neighboring blocks with the same reference index as the current block as a result of checking step S2325, the adjacent The median of the luma difference values (PredIC_offsetN, N=A, B, or C) of the block is assigned to the luma difference prediction value of the current block (S650). Specifically, it is "PredIC_offset=Median(PredIC_offsetA, PredIC_offsetB, PredIC_offsetC)".

FIG. 24 is a flowchart of a process for performing brightness compensation using information on neighboring blocks according to another embodiment of the present invention.

Referring to FIG. 24 , the decoding unit must reconstruct a luminance difference value of a current block to perform luminance compensation. In this case, the luminance difference prediction value of the current block may be obtained using information on neighboring blocks. For example, the brightness difference value of the adjacent block may be used to predict the brightness difference value of the current block. Before doing this, it is checked whether the reference index of the current block is equal to the reference index of the neighboring block. Depending on the result of the check, it can then be decided which neighbor or value will be used.

Specifically, a luminance difference value of an adjacent block indicating an average pixel difference value between an adjacent block of the current block and a block referenced by the adjacent block is extracted from the video signal (S2410).

Subsequently, it is checked whether the reference index of the current block is equal to the reference index of one of the neighboring blocks (S2420).

As a result of the checking step S720, if there is at least one neighboring block having the same reference index as the current block, it is checked whether there is a corresponding neighboring block (S2430).

As a result of checking step S2430, if there is only one neighboring block with the same reference index as the current block, assign the luma difference value of the neighboring block with the same reference index as the current block to the luma difference predictor of the current block (S2440). Specifically, it is "PredIC_offset=PredIC_offsetN".

If there is no neighboring block having the same reference index as the current block as a result of the checking step S2420, the luminance difference prediction value of the current block is set to 0 (S2460). Specifically, it is "PredIC_offset=0".

If there are at least two neighboring blocks with the same reference index as the current block as a result of the checking step S2430, set the neighboring blocks with a different reference index than the current block to 0, and will include the value set to 0 The median value of the luminance difference values of the adjacent blocks is assigned to the luminance difference prediction value of the current block (S2450). Specifically, it is "PredIC_offset=Median(PredIC_offsetA, PredIC_offsetB, PredIC_offsetC)". However, in case there is a neighboring block having a reference index different from the current block, a value of '0' can be included in PredIC_offsetA, PredIC_offsetB, or PredIC_offsetC.

Meanwhile, view information for identifying a view of a picture and a reference picture list for inter-view prediction may be applied to synthesize pictures in a virtual view. In the process for compositing pictures in a virtual view, pictures in different views may be referenced. Therefore, pictures in a virtual view can be synthesized more efficiently if view information and a reference picture list for inter-view prediction are used. In the following description, a method of synthesizing pictures in a virtual view according to an embodiment of the present invention is explained.

25 is a block diagram of a process for predicting a current picture using pictures in a virtual view, according to one embodiment of the invention.

Referring to FIG. 25 , during inter-view prediction in multi-view video coding, it is able to predict a current picture using a picture in a view different from the current view as a reference picture. However, a picture in a virtual view is obtained using a picture in a view adjacent to that of the current picture, and then the current picture is predicted using the obtained picture in the virtual view. If so, prediction can be performed more accurately. In this case, a picture in an adjacent view or a picture in a specific view may be utilized using a view identifier indicating a view of a picture. In the case of generating a virtual view, there must be a specific syntax for indicating whether to generate a virtual view. If the syntax indicates that a virtual view should be generated, the view identifier can be used to generate the virtual view. A picture in a virtual view obtained by the view synthesis prediction unit 740 may be used as a reference picture. In this case, view identifiers can be assigned to pictures in the virtual view. In a process for performing motion vector prediction to transfer a motion vector, neighboring blocks of the current block may refer to pictures obtained by the view synthesis prediction unit 740 . In this case, in order to use a picture in the virtual view as a reference picture, a view identifier indicating a view of the picture may be utilized.

FIG. 26 is a flowchart of a process for synthesizing pictures of virtual views during inter-view prediction in MVC according to an embodiment of the present invention.

Referring to FIG. 26 , a picture in a virtual view is synthesized using a picture in a view adjacent to a view of a current picture. The current picture is then predicted using the pictures in the synthesized virtual view. In this way, more accurate prediction can be realized. In case of synthesizing pictures in a virtual view, there is a specific syntax indicating whether to perform prediction of a current picture by synthesizing pictures in a virtual view. If it is decided whether to perform prediction for the current picture, it can be encoded more efficiently. This specific syntax is defined as an inter-view synthesis prediction identifier explained below. For example, pictures in a virtual view are synthesized by a slice layer to define 'view_synthesize_pred_flag' indicating whether to perform prediction of a current picture. And, pictures in the virtual view are synthesized by the macroblock layer to define 'view_syn_pred_flag' indicating whether to perform prediction of the current picture. If 'view_synthesize_pred_flag=1', the current segment synthesizes segments in the virtual view using segments in views adjacent to the current segment's view. The synthesized segment can then be used to predict the current segment. If 'viewsynthesize_pred_flag=0', segments in the virtual view are not synthesized. Likewise, if "view_syn_pred_flag=1", the current macroblock synthesizes a macroblock in a virtual view using macroblocks in views adjacent to the current macroblock's view. The synthesized macroblock can then be used to predict the current macroblock. If "view_syn_pred_flag=0", macroblocks in the virtual view are not synthesized. Therefore, in the present invention, an inter-view synthesis prediction identifier indicating whether a picture in a virtual view is obtained is extracted from a video signal. The pictures in the virtual view can then be obtained using the inter-view synthesis prediction identifier.

As described in the foregoing description, the inter prediction unit 700 is able to use view information for identifying a view of a picture and a reference picture list for inter-view prediction. And, they can also be used to perform weighted predictions. Weighted prediction can be applied to the process for performing motion compensation. In this way, if the current picture uses a reference picture in a different view, weighted prediction can be performed more efficiently using view information and a reference picture list for inter-view prediction. A weighted prediction method according to an embodiment of the present invention is explained as follows.

FIG. 27 is a flowchart of a method for performing weighted prediction according to slice types in video signal encoding according to the present invention.

Referring to FIG. 27, weighted prediction is a method of hierarchical motion compensation prediction data samples in a P-slice or B-slice macroblock. The weighted prediction method includes an explicit mode for performing weighted prediction on a current picture using weighting coefficient information obtained from information about a reference picture, and an explicit mode for using information about a distance between the current picture and one of the reference pictures The obtained weighting coefficient information performs an implicit mode of weighted prediction on the current picture. The weighted prediction method may be applied differently according to the slice type of the current macroblock. For example, in explicit mode, weighting coefficient information may be changed depending on whether the current macroblock on which weighted prediction is performed is a macroblock of a P-slice or a macroblock of a B-slice. And, a weighting coefficient of an explicit mode may be decided by an encoder and may be delivered by being included in a slice header. On the other hand, in implicit mode, weighting coefficients can be obtained based on the relative temporal positions of list 0 and list 1 . For example, a large weighting factor may be applied if the reference picture is temporally close to the current picture. If the reference picture is temporally far away from the current picture, a small weighting factor may be applied.

First, a slice type of a macroblock to which weighted prediction is to be applied is extracted from a video signal (S2710).

Subsequently, weighted prediction may be performed on the macroblock according to the extracted slice type (S2720).

In this case, the slice type may include a macroblock to which inter-view prediction is applied. Inter-view prediction refers to predicting a current picture using information about a picture in a view different from that of the current picture. For example, a slice type may include a macroblock to which temporal prediction is applied for performing prediction using information about a picture in the same view as that of the current picture, a macroblock to which inter-view prediction is applied, and a macroblock to which temporal prediction is applied Macroblocks for both and inter-view prediction. And, the slice type may include a macroblock to which only temporal prediction is applied, a macroblock to which only inter-view prediction is applied, or a macroblock to which both temporal prediction and inter-view prediction are applied. Furthermore, slice types may include both types of macroblocks or all three types of macroblocks. This will be explained in detail later with reference to FIG. 28 . Accordingly, in the case of extracting a slice type including a macroblock to which inter-view prediction is applied from a video signal, weighted prediction is performed using information on a picture in a view different from that of the current picture. In this way, information about pictures in different views can be used using a view identifier for identifying a view of a picture.

FIG. 28 is a diagram of allowable macroblock types among slice types in video signal encoding according to one embodiment of the present invention.

Referring to Figure 28, if the P-slice type of inter-view prediction is defined as VP(ViewP), then for the P-slice type of inter-view prediction, intra-frame macroblock I, a macroblock predicted from a picture in the current view can be allowed Block P or macroblock VP predicted from a picture in a different view (2810).

In the case where the B-slice type of inter-view prediction is defined as VB(View_B), macroblocks P or B predicted from at least one picture in the current view or macroblocks predicted from at least one picture in a different view are admissible VP or VB (2820).

In the case where the slice type for which prediction is performed using temporal prediction, inter-view prediction, or both temporal prediction and inter-view prediction is defined as "Mixed", for the mixed slice type, intra macroblock I, A macroblock P or B predicted from at least one picture in the current view, a macroblock VP or VB predicted from at least one picture in a different view, or a macro predicted using both a picture in the current view and a picture in a different view Block "Mixed" (2830). In this case, in order to use a picture in a different view, a view identifier for identifying the view of the picture may be used.

29 and 30 are diagrams of syntax for performing weighted prediction according to newly defined segment types according to one embodiment of the present invention.

As described above in the description of FIG. 28 , if the slice type is decided as VP, VB or hybrid, the syntax for performing conventional weighted prediction (eg, H.264) can be modified to FIG. 29 or FIG. 30 .

For example, if the slice type is temporally predicted P-slice, add the part "if (slice_type!=VP∥slice_type!=VB)" (2910).

If the slice type is temporally predicted B-slice, the if-statement may be modified to "if (slice_type==B∥ slice_type==Mixed)" (2920).

By newly defining VP segment type and VB segment type, formats (2930, 2940) similar to FIG. 29 can be newly added. In this case, since the information on the view is added, the syntax elements respectively include a "view (view)" part. For example, there is "luma_log2_view_weight_denom, chroma_log2_view_weight_denom".

31 is a flowchart of a method of performing weighted prediction using flag information indicating whether to perform inter-view weighted prediction in video signal encoding according to the present invention.

Referring to FIG. 31 , in video signal encoding to which the present invention is applied, in the case of using flag information indicating whether weighted prediction is to be performed, more efficient encoding can be performed.

Flag information may be defined based on a segment type. For example, there may be flag information indicating whether weighted prediction will be applied to a P-slice or an SP-slice or flag information indicating whether weighted prediction will be applied to a B-slice.

Specifically, the flag information may be defined as "weighted_pred_flag" or "weighted_bipred_idc". If "weighted_pred_flag=0", it indicates that weighted prediction is not applied to P-slice and SP-slice. If "weighted_pred_flag=1", it indicates that weighted prediction is applied to P-slice and SP-slice. If "weighted_bipred_idc=0", it indicates that default weighted prediction is applied to B-slices. If "weighted_bipred_idc=1", it indicates that explicit weighted prediction is applied to B-slices. If "weighted_bipred_idc=2", it indicates that implicit weighted prediction is applied to the B-slice.

In multi-view video coding, flag information indicating whether weighted prediction is to be performed using information on an inter-view picture may be defined based on a slice type.

First, a slice type and flag information indicating whether inter-view weighted prediction will be performed are extracted from a video signal (S3110, S3120). In this case, the slice type may include a macroblock to which temporal prediction is applied for performing prediction using information about a picture in the same view as that of the current picture and inter-view prediction to which is applied for using information about A macroblock for which prediction is performed by information of a picture in a view different from that of the current picture.

A weighted prediction mode may then be decided based on the extracted segment type and the extracted flag information (S3130).

Subsequently, weighted prediction may be performed according to the decided weighted prediction mode (S3140). In this case, the flag information may include flag information indicating whether weighted prediction is to be performed using information on a picture in a view different from that of the current picture, and the aforementioned 'weighted_pred_flag' and 'weighted_bipred_flag'. This will be explained in detail later with reference to FIG. 32 .

Therefore, in the case where the slice type of the current macroblock is a slice type including a macroblock to which inter-view prediction is applied, as in the case of using flag information indicating whether weighted prediction is to be performed using information on pictures in different views than, more efficient encoding can be performed.

32 is a diagram for explaining a weighted prediction method according to flag information indicating whether to perform weighted prediction using information about a picture in a view different from that of the current picture according to one embodiment of the present invention.

Referring to FIG. 32 , for example, flag information indicating whether weighted prediction is to be performed using information on a picture in a view different from that of the current picture may be defined as 'view_weighted_pred_flag' or 'view_weighted_bipred_flag'.

If "view_weighted_pred_flag=0", it indicates that weighted prediction is not applied to the VP-slice. If "view_weighted_pred_flag = 1", then explicit weighted prediction is applied to the VP-slice. If "view_weighted_bipred_flag=0", it indicates that the default weighted prediction is applied to the VB-slice. If "view_weighted_bipred_flag=1", it indicates that explicit weighted prediction is applied to the VB-slice. If "view_weighted_bipred_flag=2", it indicates that an implicit default weighted prediction is applied to the VB-slice.

In case of applying implicit weighted prediction to a VB-slice, the weighting coefficients can be obtained from the relative distance between the current view and the different views. In case of applying implicit weighted prediction to a VB-slice, weighted prediction can be performed using a view identifier identifying a view of a picture or a picture order number (POC) provided by considering the difference of each view.

The above flag information may be included in a picture parameter set (PPS). In this case, the picture parameter set (PPS) refers to header information indicating encoding modes (for example, entropy encoding modes, quantization parameter initial values by picture unit, etc.) of all pictures. However, picture parameter sets are not attached to all pictures. If there is no picture parameter set, the picture parameter set that existed just before is used as header information.

FIG. 33 is a diagram of syntax for performing weighted prediction according to newly defined flag information according to one embodiment of the present invention.

Referring to FIG. 33 , in the multi-view video coding to which the present invention is applied, in defining a slice type including a macroblock applied to inter-view prediction and indicating whether information about a picture in a view different from that of the current picture is to be used In the case of flag information to perform weighted prediction, it is necessary to decide which weighted prediction will be performed according to the slice type.

For example, if the slice type (as shown in FIG. 33 ) extracted from the video signal is P-slice or SP-slice, weighted prediction can be performed if "weighted_pred_flag=1". In case the slice type is a B-slice, if "weighted_bipred_flag=1", weighted prediction can be performed. In case the slice type is VP-slice, if "view_weighted_pred_flag=1", weighted prediction can be performed. In case the slice type is VB-slice, if "view_weighted_bipred_flag=1", weighted prediction can be performed.

FIG. 34 is a flowchart of a method of performing weighted prediction according to NAL (Network Abstraction Layer) units according to an embodiment of the present invention.

Referring to FIG. 34, first, a NAL unit type (nal_unit_type) is extracted from a video signal (S910). In this case, the NAL unit type refers to an identifier indicating the type of the NAL unit. For example, if "nal_unit_type=5", the NAL unit is a slice of an IDR picture. And, an IDR (Instant Decoding Refresh) picture refers to a header picture of a video sequence.

Subsequently, it is checked whether the extracted NAL unit type is a NAL unit type for multi-view video encoding (S3420).

If the NAL unit type is a NAL unit type for multi-view video encoding, weighted prediction is performed using information on a picture in a view different from that of the current picture ( S3430 ). The NAL unit type may be a NAL unit type applicable to both scalable video coding and multi-view video coding or a NAL unit type only for multi-view video coding. Therefore, if the NAL unit type is for multi-view video coding, weighted prediction should be performed using information about pictures in views different from that of the current picture. Therefore, it is necessary to define a new syntax. This will be explained in detail with reference to FIGS. 35 and 36 as follows.

35 and 36 are diagrams of syntax for performing weighted prediction in a case where a NAL unit type is for multi-view video coding according to one embodiment of the present invention.

First, if the NAL unit type is the NAL unit type for multi-view video coding, the syntax for performing conventional weighted prediction (eg, H.264) can be modified to the syntax shown in FIG. 35 or FIG. 36 . For example, reference numeral 3510 indicates a syntax portion for performing conventional weighted prediction, while reference numeral 3520 indicates a syntax portion for performing weighted prediction in multi-view video coding. So, weighted prediction is only performed through the syntax part 3520 if the NAL unit type is the NAL unit type used for multi-view video coding. In this case, since information on a view is added, each syntax element includes a "view (view)" part. For example, there are "Iuma_view_log2_weight_denom, chroma_view_log2_weight_denom" and so on. And, reference numeral 3530 in FIG. 36 indicates a syntax part for performing conventional weighted prediction, and reference numeral 3540 in FIG. 36 indicates a syntax part for performing weighted prediction in multiview video coding. So, weighted prediction is only performed through the syntax part 3540 if the NAL unit type is the NAL unit type used for multi-view video coding. Also, each syntax element includes a "view" part because information about a view is added. For example, there are "luma_view_weight_ll_flag, chroma_view_weight_ll_flag" etc. Therefore, if a NAL unit type for multi-view video encoding is defined, more efficient encoding can be performed in a manner of performing weighted prediction using information about pictures in a view different from that of the current picture.

FIG. 37 is a block diagram of an apparatus for decoding a video signal according to an embodiment of the present invention.

Referring to FIG. 37 , an apparatus for decoding a video signal according to the present invention includes a slice type extraction unit 3710 , a prediction mode extraction unit 3720 and a decoding unit 3730 .

FIG. 38 is a flowchart of a method of decoding a video signal in the decoding device shown in FIG. 37 according to one embodiment of the present invention.

Referring to FIG. 38 , a method for decoding a video signal according to an embodiment of the present invention includes a step S3810 of extracting a slice type and a macroblock prediction mode, and a step S3820 of decoding a current macroblock according to the slice type and/or macroblock prediction mode.

First, the prediction scheme used by the embodiment of the present invention is explained to help understand the present invention. Prediction schemes can be classified into intra-view prediction (eg, prediction between pictures in the same view) and inter-view prediction (eg, prediction between pictures in different views). And, intra-view prediction can be the same prediction scheme as ordinary temporal prediction.

According to the present invention, the slice type extracting unit 3710 extracts a slice type of a slice including a current macroblock (S3810).

In this case, a slice type field (slice_type) indicating a slice type for intra-view prediction and/or a slice type field (view_slice_type) indicating a slice type for inter-view prediction may be provided as one of video signal syntax section to provide the fragment type. This will be described in more detail below with respect to Figures 6(a) and 6(b). And, each of the slice type for intra-view prediction (slice_type) and the slice type for inter-view prediction (view_slice_type) may indicate, for example, I-slice type (I_SLICE), P-slice type (P_SLICE), or B-slice type. Slice type (B_SLICE).

For example, if the "slice_type" of a particular slice is B-slice and the "view_slice_type" is P-slice, pass the B-slice (B_SLICE) encoding scheme in the intra-view direction (i.e., time direction) and/or pass the P-slice in the view direction. - Slice (P_SLICE) coding scheme to decode the macroblocks in that particular slice.

Meanwhile, the slice type may include P-slice type (VP) for inter-view prediction, B-slice type (VB) for inter-view prediction, and mixed slice type (Mixed) for prediction obtained by mixing two prediction types . That is, the hybrid segment type provides prediction using a combination of intra-view and inter-view prediction.

In this case, the P-slice type for inter-view prediction refers to a case where each macroblock or macroblock division included in a slice is predicted from one picture in a current view or one picture in a different view. The B-slice type used for inter-view prediction refers to predicting from "one or two pictures in the current view" or "one picture in a different view or two pictures in a different view, respectively" to be included in the slice The situation of each macroblock or macroblock division. And, the mixed slice type for the prediction resulting from mixing two kinds of predictions means from "one or two pictures in the current view", "one picture in different views or two pictures in different views respectively". ” or “respectively in one or two pictures in the current view and in one picture in a different view or in two pictures in a different view” predicts the case of each macroblock or macroblock division included in a slice.

In other words, referenced pictures and allowed macroblock types differ in each slice type, which will be explained in detail later with reference to FIG. 43 and FIG. 44 .

And, the syntax in the foregoing embodiment of the segment type will be explained in detail later with reference to FIGS. 40 and 41 .

The prediction mode extracting unit 3720 may extract a macroblock prediction mode identifier indicating whether the current macroblock is an intra-view predicted macroblock, an inter-view predicted macroblock, or a predicted macroblock obtained by mixing both types of prediction (S3820 ). To this end, the present invention defines a macroblock prediction mode (mb_pred_mode). One embodiment of the macroblock prediction mode will be explained in detail later with reference to FIG. 39 , FIG. 40 and FIG. 41 .

The decoding unit 3730 decodes the current macroblock according to the received/generated slice type and/or macroblock prediction mode of the current macroblock (S3820). In this case, the current macroblock can be decoded according to the macroblock type of the current macroblock decided from the macroblock type information. Also, the macroblock type can be determined according to the macroblock prediction mode and the slice type.

In case the macroblock prediction mode is a mode for intra-view prediction, a macroblock type is decided according to a slice type for intra-view prediction, and then a current macroblock is decoded by intra-view prediction according to the decided macroblock type.

In case the macroblock prediction mode is a mode for inter-view prediction, a macroblock type is decided according to a slice type for inter-view prediction, and then a current macroblock is decoded by inter-view prediction according to the decided macroblock type.

In the case where the macroblock prediction mode is a mode for prediction resulting from mixing two predictions, the macroblock type is decided according to the slice type used for intra-view prediction and the slice type used for inter-view prediction, and then according to the determined Each of the macroblock types decodes the current macroblock by a prediction derived from mixing the two predictions.

In this case, the macroblock type depends on the macroblock prediction mode and the slice type. Specifically, it is possible to determine a prediction scheme to be used for a macroblock type from a macroblock prediction mode, and then decide a macroblock type from macroblock type information by a slice type according to the prediction scheme. That is, one or both of the extracted slice_type and view_slice_type are selected based on the macroblock prediction mode.

For example, if the macroblock prediction mode is the mode for inter-view prediction, the macroblock can be decided from the macroblock table of the slice type (I, P, B) corresponding to the slice type (view_slice_type) for inter-view prediction type. The relationship between macroblock prediction modes and macroblock types will be explained in detail later with reference to FIG. 39 , FIG. 40 , and FIG. 41 .

FIG. 39 is a diagram of macroblock prediction modes according to an exemplary embodiment of the present invention.

In FIG. 39( a ), a table corresponding to one embodiment of the macroblock prediction mode (mb_pred_mode) according to the present invention is shown.

In the case of using only intra-view prediction (ie, temporal prediction) for a macroblock, "0" is assigned to the value of "mb_pred_mode". In the case of using only inter-view prediction for a macroblock, "1" is assigned to the value of "mb_pred_mode". In the case of using both temporal and inter-view prediction for a macroblock, "2" is assigned to the value of "mb_pred_mode".

In this case, if the value of "mb_pred_mode" is "1", that is, if "mb_pred_mode" indicates inter-view prediction, view direction List0 (ViewList0) or view direction List1 (ViewList1) is defined for inter-view prediction A list of reference images for .

In FIG. 39(b), the relationship between macroblock prediction modes and macroblock types according to another embodiment is shown.

If the value of "mb_pred_mode" is "0", only temporal prediction is used. And, the macroblock type is determined according to the slice type (slice_type) used for intra-view prediction.

If the value of "mb_pred_mode" is "1", only inter-view prediction is used. And, the macroblock type is determined according to the slice type (view_slice_type) used for inter-view prediction.

If the value of "mb_pred_mode" is "2", a hybrid prediction of temporal and intra-view prediction is used. And, two macroblock types are determined according to the slice type (slice_type) for intra-view prediction and the slice type (view_slice_type) for inter-view prediction.

Based on the macroblock prediction mode, the macroblock type is given based on the slice type as shown in Tables 1-3 below. [Please insert Form 7-12-7-14 from N6540 here as Form 1-3]

In other words, in this embodiment, the prediction scheme for a macroblock and the slice type to be referred to are decided by the macroblock prediction mode. Also, the macroblock type is determined according to the slice type.

40 and 41 are diagrams of example embodiments of syntax of a portion of a video signal received by an apparatus for decoding a video signal. As shown, the syntax has slice type and macroblock prediction mode information according to an embodiment of the present invention.

In Fig. 40, an example syntax is shown. In this syntax, a field 'slice_type' and a field 'view_slice_type' provide a slice type, and a field 'mb_pred_mode' provides a macroblock prediction mode.

According to the present invention, a 'slice_type' field provides a slice type for intra-view prediction and a 'view_slice_type' field provides a slice type for inter-view prediction. Each slice type can be an I-slice type, a P-slice type, or a B-slice type. If the value of "mb_pred_mode" is "0" or "1", a macroblock type is decided. However, in the case where the value of "mb_pred_mode" is "2", it can be seen that another macroblock type (or two types) is further determined. In other words, the syntax shown in (a) of FIG. 40 indicates that "view_slice_type" is added to further apply conventional slice types (I, P, B) to multi-view video encoding.

In Fig. 41, another example syntax is shown. In this syntax, the "slice_type" field is used to provide the slice type and the "mb_pred_mode" field is used to provide the macroblock prediction mode.

According to the present invention, the "slice_type" field may include slice type (VP) for inter-view prediction, slice type-B (VB) for inter-view prediction, and The predicted mixed fragment type (Mixed).

If the value of the 'mb_pred_mode' field is '0' or '1', a macroblock type is decided. However, in the case where the value of the "mb_pred_mode" field is "2", it can be seen that additional (ie, a total of two) macroblock types are determined. In this embodiment, segment type information exists in the segment header, which will be explained in detail with respect to FIG. 42 . In other words, the syntax shown in FIG. 41 indicates that VP, VB, and hybrid slice types are added to the conventional slice type (slice_type).

FIG. 42 is a diagram for an example of applying the segment types shown in FIG. 41 .

The diagram in Figure 42(a) shows that in the slice header, among other slice types, there can be a P-slice type (VP) for inter-view prediction, a B-slice type (VB) for inter-view prediction and the mixed segment type (Mixed) for the prediction resulting from mixing the two predictions as the segment type. Specifically, segment types VP, VB, and Mixed according to an example embodiment are added to segment types that may exist in a general segment header.

The diagram in Figure 42(b) shows that in a slice header for Multiview Video Coding (MVC), there can be a P-slice type (VP) for inter-view prediction, a B-slice type for inter-view prediction, A slice type (VB) and a mixed slice type (Mixed) for a prediction obtained from mixing two predictions are taken as slice types. Specifically, the segment type according to the example embodiment is defined in a segment header for multi-view video encoding.

The diagram in FIG. 42(c) shows that in the slice header for scalable video coding (SVC), besides the existing slice types for scalable video coding, there may be slice types for inter-view prediction ( VP), B-slice type (VB) for inter-view prediction and mixed slice type (Mixed) for prediction resulting from mixing both predictions as slice types. Specifically, slice types VP, VB, and Mixed according to example embodiments are added to slice types that may exist in a slice header of a scalable video coding (SVC) standard.

FIG. 43 is a diagram of examples of various segment types included in the segment types shown in FIG. 41 .

In FIG. 43( a ), a case where a slice type is predicted from one picture in a different view is shown. Therefore, the slice type becomes the slice type (VP) for inter-view prediction.

In FIG. 43( b ), a case where a slice type is respectively predicted from two pictures in different views is shown. Therefore, the slice type becomes a B-slice type (VB) for inter-view prediction.

In FIGS. 43( c ) and 43 ( f ), cases where a slice type is predicted from one or two pictures in the current view and one picture in a different view are shown. Therefore, the slice type becomes a mixed slice type (Mixed) for prediction obtained from mixing two kinds of predictions. Also, in FIGS. 43( d ) and 43 ( e ), a case where a slice type is predicted from one or two pictures in a current view and two pictures in a different view is shown. Therefore, the fragment type also becomes a mixed fragment type (Mixed).

FIG. 44 is a diagram of macroblocks allowed by the slice types shown in FIG. 41 .

Referring to Figure 44, the P-slice type (VP) for inter-view prediction allows intra macroblocks (I), macroblocks predicted from a picture in the current view (P), or macroblocks predicted from a picture in a different view (VP).

The B-slice type (VB) for inter-view prediction allows intra macroblocks (I), macroblocks predicted from one or two pictures in the current view (P or B) or from a picture in a different view or different Macroblock VP or VB predicted by two pictures in the view.

Also, the mixed slice type (Mixed) allows intra-frame macroblocks (I); macroblocks (P or B) predicted from one or two pictures in the current view; A macroblock (VP or VB) predicted from two pictures, or a macroblock predicted from one or two pictures in the current view, one picture in a different view, or two pictures in a different view (Mixed).

45-47 are diagrams of macroblock types of macroblocks existing in a mixed slice type (Mixed) according to an embodiment of the present invention.

In FIGS. 45( a ) and 45 ( b ), configuration schemes of a macroblock type (mb_type) and a sub-macroblock type (sub_mb_type) for macroblocks existing in a hybrid slice are shown, respectively.

In FIGS. 46 and 47, the binary representations of the prediction directions of macroblocks existing in a hybrid slice and the actual prediction directions of a hybrid slice are shown, respectively.

According to an embodiment of the present invention, the macroblock type (mb_type) is prepared by considering both the size (Partition_Size) of the macroblock partition and the prediction direction (Direction) of the macroblock partition.

And, the sub macroblock type (sub_mb_type) is prepared by considering both the size of the sub macroblock partition (Sub_Partition_Size) and the prediction direction (Sub_Direction) of each sub macroblock partition.

Referring to FIG. 45( a ), "Direction0" and "Direction1" indicate the prediction direction of the first macroblock division and the prediction direction of the second macroblock division, respectively. Specifically, in the case of an 8x16 macroblock, "Direction0" indicates a prediction direction for left 8x16 macroblock division, and "Direction1" indicates a prediction direction for right 8x16 macroblock division. The configuration principle of the macroblock type (mb_type) is explained in detail as follows. First, the first two bits indicate the partition size (Partition_Size) of the corresponding macroblock and values 0˜3 are available for the first two bits. And, in a case where a macroblock is divided into partitions, four bits following the first two bits indicate a prediction direction (Direction).

For example, in the case of a 16x16 macroblock, four bits indicating the prediction direction of the macroblock are appended to the first two bits. In the case of a 16x8 macroblock, the four bits following the first two bits indicate the prediction direction of the first partition (Direction0) and another four bits are appended to the first four bits to indicate the prediction direction of the second partition (Direction1 ). Likewise, in the case of an 8x16 macroblock, eight bits are appended after the first two bits. In this case, the first four bits of the eight bits appended to the first two bits indicate the prediction direction of the first partition and the latter four bits indicate the prediction direction of the second partition.

Referring to FIG. 45( b ), the prediction direction (Sub_Direction) of the sub macroblock is used in the same manner as the prediction direction (Direction) of the macroblock division shown in FIG. 45( a ). The configuration principle of the sub-macroblock type (sub_mb_type) is explained in detail as follows.

First, the first two bits indicate the partition size (Partition_Size) of the corresponding macroblock and the last two bits after the first two bits indicate the partition size (Sub_Partition_Size) of the sub-macroblock of the corresponding macroblock. Values 0-3 can be used for each of the first and last two bits. Then, in the case where the macroblock is divided into sub-macroblock divisions, the four bits appended to the last two bits indicate the prediction direction (Sub_Direction). For example, if the partitioned size (Partition_Size) of the macroblock is 8x8 and if the partitioned size (Sub_Partition_Size) of the sub-macroblock is 4x8, the first two bits have a value of 3, and the last two bits have a value of 2, in which The first four bits after the ones bit indicate the prediction direction for the left 4x8 block of the two 4x8 blocks, and the last four bits after the first four bits indicate the prediction direction for the right 4x8 block.

Referring to FIG. 46, the prediction direction of a macroblock is constructed using four bits. And, it can be seen that each binary representation becomes "1" depending on the case of referring to a picture at a left (L), upper (T), right (R) or lower (B) position of the current picture.

Referring to FIG. 47 , for example, in a case where the prediction direction is up (T), a picture located at an upper part in the view direction of the current picture is referred to. In the case where the prediction direction corresponds to all directions (LTRB), it can be seen that pictures in all directions (LTRB) of the current picture are referred to.

FIG. 48 is a block diagram of an apparatus for encoding a video signal according to an embodiment of the present invention.

Referring to FIG. 48 , an apparatus for encoding a video signal according to an embodiment of the present invention is described. The device includes a macroblock type determining unit 4810 , a macroblock generating unit 4820 and an encoding unit 4830 .

FIG. 49 is a flowchart of a method of encoding a video signal in the encoding device shown in FIG. 48 according to an embodiment of the present invention.

Referring to FIG. 49 , the method for encoding a video signal according to an embodiment of the present invention includes a step S4910 of determining a first macroblock type for intra-view prediction and a second macroblock type for inter-view prediction, generating a macroblock with the first Step S4920 of a first macroblock of block type and a second macroblock of a second macroblock type, step S4930 of generating a third macroblock using the first and second macroblocks, and encoding the macroblock type and macroblock of the current macroblock Step S4940 of prediction mode.

According to the present invention, the macroblock type decision unit 4810 decides the first macroblock type for intra-view prediction and the second macroblock type for inter-view prediction as described in detail above (S4910).

Subsequently, the macroblock generation unit 4820 generates a first macroblock with the first macroblock type and a second macroblock with the second macroblock type using a known prediction technique (S4920), and then uses the first and second macroblocks The block generates a third macroblock (S4930). In this case, the third macroblock is generated based on the average value between the first and second macroblocks.

Finally, the encoding unit 4830 encodes the macroblock type (mb_type) of the current macroblock and the macroblock prediction mode (mb_pred_mode) of the current macroblock by comparing the encoding efficiencies of the first to third macroblocks (S4940).

In this case, there are various methods for measuring coding efficiency. Specifically, a method with RD (Rate Distortion) cost is used in this embodiment of the invention. As known, in the RD cost method the corresponding cost is calculated using two components: the number of coding bits resulting from coding the corresponding block and a distortion value indicating the error from the actual sequence.

The first and second macroblock types may be decided in such a manner that the macroblock type having the smallest value of the RD cost explained above is selected. For example, the macroblock type having the smallest value of the RD cost among intra-view predicted macroblock types is determined as the first macroblock type. Then, the macroblock type having the smallest value of the RD cost is determined as the second macroblock type among the inter-view predicted macroblock types.

In the step of encoding the macroblock type and macroblock prediction mode, the macroblock type and prediction mode associated with one of the first and second macroblocks having a smaller RD cost may be selected. Subsequently, the RD cost of the third macroblock is determined. Finally, the macroblock type and macroblock prediction mode of the current macroblock are coded by comparing the RD cost of the selected first or second macroblock and the RD cost of the third macroblock with each other.

If the RD cost of the selected first or second macroblock is equal to or greater than the RD cost of the third macroblock, the macroblock type becomes the macroblock type corresponding to the selected first or second macroblock.

For example, if the RD cost of the first macroblock is less than the RD costs of the second and third macroblocks, then the current macroblock is set as the first macroblock type. And, the macroblock prediction mode (ie, intra-view) becomes the prediction scheme of the macroblock corresponding to the RD cost.

For example, if the RD cost of the second macroblock is smaller than the RD costs of the first and third macroblocks, the inter-view prediction scheme that is the prediction scheme of the second macroblock becomes the macroblock prediction mode of the current macroblock.

Meanwhile, if the RD cost of the third macroblock is smaller than the RD costs of the first and second macroblocks, the macroblock type corresponds to the first and second macroblock types. Specifically, the intra-view prediction and inter-view prediction macroblock types become the macroblock type of the current macroblock. And, the macroblock prediction mode becomes a hybrid prediction scheme derived from mixing intra-view and inter-view prediction.

Therefore, the present invention provides at least the following effects or advantages.

Due to various prediction schemes between views and information such as slice type, macroblock type, and macroblock prediction mode, the present invention can exclude redundant information between views; thereby improving encoding/decoding efficiency performance.

Industrial applicability

While the invention has been described and illustrated herein with reference to preferred embodiments thereof, it will be apparent to those skilled in the art that various modifications and changes can be made therein without departing from the spirit and scope of the invention. Thus, it is intended that the present invention covers the modifications and variations of this invention that come within the scope of the appended claims and their equivalents.

Claims (8)

1. A method for decoding a video signal, comprising the steps of: checking the encoding scheme of said video signal; obtaining configuration information for the video signal according to the coding scheme; identifying a total number of views using the configuration information; identifying inter-view reference information based on the total number of the views; and decoding the video signal based on the inter-view reference information, Wherein the configuration information includes at least view information for identifying a view of the video signal. 2. The method according to claim 1, further comprising the step of checking a level for view scalability of said video signal, wherein said video signal is decoded based on said level. 3. The method according to claim 1, wherein the configuration information further includes a temporal level, an inter-view group of picture flag, and an IDR flag. 4. The method according to claim 2, wherein in said decoding step, if said video signal corresponds to a base view according to said level, said video signal of said base view is decoded. 5. The method according to claim 1 , further comprising the step of: using said view information to obtain information for a picture in a view different from that of the current picture, wherein in decoding said video signal is used for said different view The information for the image in the . 6. The method of claim 5, said video signal decoding step further comprising the step of performing motion compensation using said information for said picture in said different view, wherein for said picture in said different view The information of the picture includes motion vector information and reference picture information. 7. The method according to claim 5, said video signal decoding step further comprising the steps of: extracting from the video signal an inter-view synthesis prediction identifier indicating whether to use a picture in a view adjacent to that of the current picture to obtain a picture in the virtual view; and obtaining the picture in the virtual view according to the inter-view synthesis prediction identifier, wherein said information for said pictures in said different view is used in obtaining said pictures in said virtual view. 8. The method of claim 5, said video signal decoding step further comprising the step of obtaining depth information between different views using said information for said pictures in said different views.

Images