CN107295350A

CN107295350A - Reference picture list management syntax for multiple view video coding

Info

Publication number: CN107295350A
Application number: CN201710523675.8A
Authority: CN
Inventors: 普尔温.比布哈斯.潘迪特; 苏野平; 尹鹏
Original assignee: Thomson Licensing SAS
Current assignee: InterDigital VC Holdings Inc
Priority date: 2006-10-13
Filing date: 2007-10-12
Publication date: 2017-10-24
Also published as: CN101653001A

Abstract

The present invention describes a kind of method for carrying out image reference list ordering to the multi-vision-point encoding device for encoding moving pictures, wherein described image list has the coded sequence of reference picture, needs whether the image encoded is associated with viewpoint to encode the image specified (715) for being relevant to.The sequence of image list will therefore depend on the time relationship (720) between reference picture and the image for needing to encode and the viewpoint (715) associated with reference picture changes the coded sequence of the reference picture in image reference list.

Description

Reference picture list management syntax for multiple view video coding

The application is the applying date on October 12nd, 2007, Application No. 200780038178.3, entitled " is used for The divisional application of the patent application of the reference picture list management syntax of multiple view video coding ".

The cross reference of related application

The U.S. Provisional Application 60/851,863 submitted this application claims on October 13rd, 2006 and on October 13rd, 2006 The priority of the U.S. Provisional Application 60/851,521 of submission, above-mentioned two provisional application is incorporated by as reference herein.

Technical field

The present invention relates to moving image (moving pictures) field, more particularly, to reference picture for compiling Sequence (ordering) problem during code video image.

Background technology

Many interframe coding systems use reference picture (reference picture), are helped using this reference picture In the size of reduction coded bit stream.Such result is exactly that code efficiency is higher than intra-frame coding techniques are only used only.Therefore, Many coding standards are both comprising frame in or comprising inter-frame coding, to encode a series of bit stream of mobile images.Existing In technology, different types of reference picture is used for (frame in) in coding standard, such as " I " image, its use image itself Element is encoded, in another example " B " image, it is previous by using the element from image therein and/or from two The element of reference picture (interframe) is encoded, then such as " P " image, and it is by using the element from image therein And/or the element from one previously with reference to image (interframe) is encoded.

When " B " image or " P " image are encoded and/or decoded, this image thus depends on other reference frames, so that This image can be properly encoded or build in decoding process.Coder/decoder system should provide certain type of storage Unit (memory location), so that this seed ginseng can be stored when other images are being encoded by reference picture or decoded Examine image.There is time relationship between these image types.

When the multiple views (multiple views) for considering wherein different video image need the video standard considered, this The situation of kind becomes increasingly complex.For example, Fig. 1 is the exemplary embodiment party of the reference picture structure used in multi-vision-point encoding system Formula.Specifically, shown structure is related to is regarded according to multi-vision-point encoding (MVC) scheme in the time (T0-T100) using 8 differences Point (S0-S7), the MVC schemes in July, 2006 Austria Klagenfurt JVT-T208.doc A.Vetro, Y.Su, H.Kimata, A.Smolic's " proposes in Joint Multiview Video Model (JMVM) 1.0 ".This multiple views Coding standard be based on advanced video coding (AVC) standard (G.Sullivan, T.Wiegand, A.Luthra, " H.264/AVC (ITU proposed standards and ISO/IEC 14496-10 (parts of MPEG-4 the 10th) advanced video coding) ", Palma de Mallorca, ES18-22,2004 years October).The maximum difference of the two standards is that AVC is not solved to multi-view image Coding and MVC can solve the problem that.

Please referring back to Fig. 1, such as can be seen that when coding is in the image that time T1 is associated with viewpoint S1, it is necessary to compile The image of code is associated with the image (reference picture) from same viewpoint (time T0 S1 and time T2 S1), and needs to compile The image of code is associated with the image from different points of view (time T1 S0 and time T1 S2).Therefore, when coding is associated with S1 Image when, it is important to storage device (for example can by hardware, software or its combination implement buffer, register, RAM etc.) in preserve reference picture (time T0 S1, time T2 S1, time T1 S0 and time T1 S2).

For storing the encoder of the reference picture used in AVC video standards and the general embodiment party of storage device computing Formula is shown in Figure 2.Specifically, encoder 205, encoding buffer 210 and decoding image buffer are disclosed in block diagram 200 (DPB)215.During encoding operation (coding is decoded), the image encoded by encoder 205 is present in volume In code buffer 210, and the reference pictures store of previous coding is in decoded picture buffer device 215.

But when need coding one need to use the image of the reference picture more than two when, when the image be stored in it is slow When in storage DPB 215, problem will be produced.That is DPB 215 is necessary for this reference picture and provides certain type of tissue, with Them are made to be used during encoding operation by encoder 205.It is further appreciated that when encoding the second image, it is different Reference picture must may be obtained by encoder 205 by DPB215.In the case of the time (such as AVC), this problem is non- It is often complicated, but when the image in standards for encoded video (such as MVC) when must take into consideration the reference picture in different points of view (as shown in Figure 1), this has reformed into the problem of needing to obtain reference picture.

The content of the invention

The present invention can solve the problem that the disadvantages mentioned above and deficiency of prior art.It is available for recycling the present invention relates to one kind Method and apparatus of the movable information as motion estimation prediction device to carry out Video coding.

A kind of one side in accordance with the principles of the present invention, it is proposed that encoder, it is according to from the figure decoded by decoder The information of picture carries out storage management operations to the reference picture of storage in memory, wherein this information is with being associated with the reference The view information of image is related.

There is provided a kind of modification for the video decoding operation based on multiple views for another aspect in accordance with the principles of the present invention The method of reference picture list, it is associated with multiple viewpoints with reference to figure in the video decoding operation based on multiple views As being used to encoded video image, it the described method comprises the following steps：Whether the first reference picture is determined using video encoder It is the temporal reference picture for the video image for needing coding, and by being specified to first reference picture for the time The picture number of reference picture list changes coded sequence of the reference picture in the temporal reference picture list, wherein, It is I images, B images or P images, it is necessary to which the described image of coding is that described image number, which depends on first reference picture, I images, B images or P images, it is necessary to decoding order of the video image in the multiple coded image of coding, with And need display order of the video image encoded in the multiple coded image；It is determined that needing the video encoded Whether image is associated with the viewpoint in the different points of view in the multiple viewpoint；And when expression intersects viewpoint reference picture row The corresponding viewpoint of the viewpoint of the second reference picture in table is different from the viewpoint associated by the video image for needing to encode When viewpoint is associated, the coding of second reference picture is changed, so as to specify the intersection viewpoint to join to the second reference picture The second picture number of image list is examined, wherein, it is I images, B that second picture number, which depends on second reference picture, Image or P images are, it is necessary to which the described image of coding is I images, B images or P images, it is necessary to the video image of coding The decoding order in the multiple coded image, and the video image encoded is needed in the multiple code pattern The display order as in

Pass through the detailed description carried out below in conjunction with the accompanying drawings to embodiments of the present invention, above and other side of the invention Face and feature and advantage will become obvious.

Brief description of the drawings

According to the detailed description with reference to accompanying drawings below, principle of the invention will be better understood.

Fig. 1 is the multi-vision-point encoding of the video image viewpoint according to the different time of embodiment of the present invention, wherein this Video image is encoded using reference picture in the way of shown in figure；

Fig. 2 is the illustrative embodiments of video encoder in accordance with the principles of the present invention；

Fig. 3 is the pseudo- generation in the syntactic element ref_pic_list_recording () according to used in embodiment of the present invention The embodiment of code；

Fig. 4 is in the syntactic element slice_shead_mvc_extension () according to used in embodiment of the present invention The embodiment of false code；

Fig. 5 is in the syntactic element ref_pic_list_reordering_mvc_ according to used in embodiment of the present invention The embodiment of extension () false code；

Fig. 6 is in the syntactic element ref_pic_list_reordering_mvc_ according to used in embodiment of the present invention The embodiment of extension () false code；

Fig. 7 is the block diagram in the image reference sequence reorganization sequence method according to embodiment of the present invention.

Embodiment

The principle of the present invention goes for any coding standard based on frame in and interframe.The art used in this specification Language " image " is a generic term for being used to describe various forms of video image informations, and the video image information is in ability " frame (frame) ", " field (field) ", " bar (slice) " and " image (picture) " is can be understood as in domain in itself.Can be with Understand, although term image is used to represent various elements video information, and AVC refers to using bar, wherein this reference picture can Using use the bar from identical image as " reference picture " and no matter image how quilt is divided, principle of the invention is all fitted With.

Also, in the description of the invention, be described in detail in the accompanying drawings using C language type format write it is various Instruct (syntactic element/syntax element), following nomenclature is used to the descriptor (descriptor) in this instruction.

u(n)：Use the signless integer of n-bit.When n is " v " in syntax table, bit number is according to depending on other The mode of the value of syntactic element changes.Return value (the return that parsing for this descriptor passes through function read_bits (n) Value) determine, what it was interpreted signless integer with highest significant position formerly writes (most significant bit Written first) binary representation.

ue(v)：The syntactic element of the signless integer Exp-Golomb codings of left bit first (left bit first).

se(v)：The syntactic element of the signed integer Exp-Golomb codings of left bit first.

C：The class that syntactic element is applicable is represented, i.e., one specific field should apply to that one-level.

Present description illustrates the principle of the present invention.Although not describing in this manual, ordinary skill Personnel are within the principle and scope of the present invention it is contemplated that various embodiments.

The purpose of the language of exemplary and conditionity employed in this specification is to help reader to understand by inventing The principle and concept of the invention that people is improved to prior art, the invention is not limited in this specific example and bar Part.

Also, for the principle of the invention, aspect and embodiment and particular instance all statements be intended to including etc. Same 26S Proteasome Structure and Function.In addition, this equivalent structure and function include 26S Proteasome Structure and Function that is currently known and developing later, Though i.e. structure how and for performing any element of identical function.

Referring again to Fig. 2, when (AVC and MVC) encoder 205 performs the encoding operation encoded to mobile image When, the image of coding is associated with several different values, wherein this value is placed on picture headers (such as bar head by encoder 205 Portion) in.Referred to as pic_num (image number) the first value indicates to be encoded order of the image in the sequence of video images of coding. For example, the first image in the sequence has value " 1 ", and the 4th image being encoded has value " 4 ".

Image also has the value for being referred to as image sequence number (POC), and the value determines that coded image is shown in final decoding The order shown.This represents that the frame number that coded image has likely differs from No. POC associated with the image.

Multiple view video coding introduces the 3rd value, and the 3rd value can be combined with other values is used for image.view_id (viewpoint id) value represents the viewpoint corresponding to image.For example, the view_id of the image associated with viewpoint S3 is equal to " 3 ", and it is right " 5 " should be equal in the view_id of viewpoint S5 image.The another aspect of the use of No. view_id is because can using this value To allow encoder to decouple pic_num and POC between different points of view, this enables DPB 215 to store with identical pic_num And/or POC reference picture.Therefore the image of this (have identical POC or pic_num) by using view_id values each other Distinguish.

It is appreciated that above-mentioned value can be performed (using other numeral, variable name, Hash tables etc.) by different modes.

When (as being located at http://www.rgu.ac.uk/files/avc_picmanagement_draft1.pdf work Person by Iain G Richardson " in Frame and Picture Management " as explanation) in DPB 215 During middle storage reference picture, this image has been assigned storage state, and reference picture marking is short term reference by the storage state Image (being quoted by PicNum or PicOrderCount) and long term reference image (quoted by LongTermPicNum, LongTermPicNum represents the number in LongTermPic indexes).It is appreciated that the state of reference picture (such as can lead to Cross using known storage management instruction operator in AVC) long-term, short-term and be even referred to as between the state of " non-reference " Change, " non-reference " represents that reference picture is most removed from DPB 215 at last.

The tissue of reference picture can be ordered as one or two list, and they use before coded image (bar). Reference for P images will use single image list (list0), and the reference for being used for B images will use two lists (list0 and list1).The default tissue of image in image list depends on solution when the present image being encoded is P images Code order (PicNum), display order is depended on when the present image being encoded is B images.It is designated as long term reference image Reference picture be generally assigned with LongTermPicNum and be placed on a certain position of reference listing so that the image will compare Short-term figure is retained the longer time.

AVC allows encoder 205 to change in code pattern slice in list0 (and for list1 of B images (bar)) The default order of reference picture.That is, encoder can recognize that (in above-mentioned list) reference picture can be than in reference listing The more preferable reference picture of reference picture.By that will be than with being encoded with the more relevant reference picture marking of current encoded image The reference picture of image less correlation occupies the position of lower (closer) in index on reference listing, can improve coding effect Rate.Therefore, being referred to as the reorder instruction of instruction (RPLR) of reference picture list is used to change the order of this reference picture, this Instruct shown in Figure 3.

Fig. 3 show AVC syntactic elements ref_pic_list_reordering (although being illustrated with reference to bar, can be with Understand the principle can for image perform) provide for the image list that reorders AVC instruction, wherein will be with reference to AVC to this The meaning instructed a bit is illustrated.

When considering to use reference picture list in MVC category, actually there are two groups of images to need consideration, i.e. time With reference to (temporal references) and intersect view reference (cross-view references), its it is different from AVC it It is in and only considers time reference in AVC.

Therefore, in " " Multiview Coding written by K.Muller, P.Merkle, A.Smolic, T.Wiegand In using AVC ", m12945, Bangkok, THA, in January, 2006 ", it is proposed that (it is by all video sequences for the single interleaved sequence of construction Column interleaving is in one sequence), the encoder that the interleaved sequence may then pass through AVC functions is encoded and generates compatibility Bit stream.But, the AVC embodiments the disadvantage is that, which viewpoint some image of None- identified belongs to because for pic_ The RPLR that num and POC only value must be assigned to each image of coding to perform existing AVC is instructed.

Current MVC embodiments propose the other grammer of introducing, and the grammer is injected towards sequence parameter set (SPS) to mark Know and intersect view reference.The syntactic element as shown below is used to indicate to be used for anchor image (anchor picture) and non-anchor figure The intersection view reference of picture.

It is pre- so that the reference picture from the viewpoint different from current view point is placed on into image that following step should be carried out Survey in list.

- for each i values from 0 to num_multiview_ref_for_listX-1：

- obtain viewpoint reference_view_for_list_X [i] establishments that present image is assigned from the time Image and be inserted into value decoding prediction buffer (DPB).

- index of the image is inserted into RefPicListX next empty slot.

But, this embodiment has a problem, and the initialization (and reconstruction) of reference picture list is by fixation Mode, it is impossible to change.That is, the order of the image of time and viewpoint of reporting to the leadship after accomplishing a task is inflexible.In addition, being each anchor image (i.e. I Image) and the quantity of the reference picture specified of non-anchor picture (B and P images) be identical.

Therefore, the present invention proposes a kind of for the reference picture list that reordered in the case where considering multi-view image Framework.It is appreciated that, although embodiments of the present invention are illustrated within the category of AVC and MVC environment, but the present invention Principle can also apply in other Video coding tables.

The present invention proposes a kind of new syntactic element ref_pic_list_reordering_mvc_extension (), It is shown in Fig. 4 in picture headers (bar head).

First, the present invention is proposed to the syntactic element suitable for AVC

It is fixed that num_ref_idx_l0_active_minus1 and num_ref_idx_l1_active_minus1 is carried out again The fact that justice is to reflect effective number of reference pictures in their instruction time meanings.

Num_ref_idx_l0_active_minus1 provides the maximum ginseng of the temporal reference picture list 0 for solving code-bar Examine index.Num_ref_idx_l0_active_minus1 scope is provided as follows：

If the scope that-field_pic_flag is equal to 0, num_ref_idx_l0_active_minus1 is 0 to 15, Including end points.It is for the decoding of frame macro block when MbaffFrameFlag is equal to 1, num_ref_idx_l0_active_minus1 Maximum index value and 2*num_ref_idx_l0_active_minus1+1 are the maximum index values decoded for field macro block.

- otherwise (field_pic_flag be equal to 1), num_ref_idx_l0_active_minus1 scope be 0 to 31, including end points.

Num_ref_idx_l1_active_minus1 and num_ref_idx_l0_active_minus1 has identical Semanteme, wherein l0 and list 0 is replaced by l1 and list 1 respectively.

Second, it is proposed that two new syntactic elements are indicated the quantity of effective reference picture for intersecting viewpoint：

Num_ref_idx_l0_active_view_minus1 provides the intersection viewpoint reference picture list for solving code-bar 0 maximum reference key.Num_ref_idx_l0_active_view_minus1 scope is provided as follows：

If the scope that-field_pic_flag is equal to 0, num_ref_idx_l0_active_view_minus1 is 0 To 15, including end points.It is to be used for frame when MbaffFrameFlag is equal to 1, num_ref_idx_l0_active_view_minus1 The maximum index value and 2*num_ref_idx_l0_active_view_minus1+1 of macro block decoding are to be used for the decoding of field macro block Maximum index value.

- otherwise (field_pic_flag is equal to 1), num_ref_idx_l0_active_view_minus1 scope is 0 to 31, including end points.

Num_ref_idx_l1_active_view_minus1 (another syntactic element of proposition) and num_ref_idx_ L0_active_view_minus1 has identical semantic, and wherein l0 and list 0 are replaced by l1 and list 1 respectively.

The value of the two grammers meets following equations：

num_ref_idx_l0_active_minus1+num_ref_idx_l0_active_view_minus1<=num_ ref_frames

With

num_ref_idx_l1_active_minus1+num_ref_idx_l1_active_view_minus1<=num_ ref_frames。

As previously described, it is necessary to limit the default initialization process for MVC systems.Therefore, proposed for the process Several different embodiments：

The first embodiment of the principle of the invention makes the All Time image of current view point according to MPEG-4AVC specifications In definition initialized.The list of this initialization is then respectively by the num_ref_idx_ for LIST_0 and LIST_1 Value specified in l0_active_minus1 and num_ref_idx_l1_active_minus1 is blocked (truncated).

After these images, the intersection viewpoint reference picture defined in the SPS of MVC specifications be inserted into and by for LIST_0 and LIST_1 num_ref_idx_l0_active_view_minus1 and num_ref_idx_l1_active_ Value is blocked specified in view_minus1.

Correct operating, this hair are instructed in order to allow reorder in reference picture list specified in current MVC specifications It is bright to be applied with limitation so that these instructions can only work for temporal reference picture.RPLR processing can follow time diagram closely After the initialization of picture and intersect view reference before carry out, can also the time and intersect view reference be initialised to its lack Carried out after place-saving, only temporal reference picture is reordered under latter event.

One of first embodiment has the disadvantage that intersecting viewpoint reference picture can not be reordered.

In this second embodiment, the intersection viewpoint reference picture defined in the SPS of MVC specifications is inserted into and by dividing Not Yong Yu LIST_0 and LIST_1 num_ref_idx_l0_active_view_minus1 and num_ref_idx_l1_ Value is blocked specified in active_view_minus1.

After these images, the All Time image of current view point is entered according to the definition in MPEG-4AVC specifications Row initialization.Then the list is by being respectively used to LIST_0 and LIST_1 num_ref_idx_l0_active_view_ Value specified in minus1 and num_ref_idx_l1_active_view_minus1 is blocked.

3rd embodiment is used for the above mentioned problem for solving first embodiment.Specifically, it refer in October, 2007 The PCT application (application number is undetermined) submitted for 12nd, this application requires the U.S. Provisional Application 60/ submitted on October 13rd, 2006 The priority for the U.S. Provisional Application 60/851,589 that on October 13rd, 851,522 and 2006 submits, syntactic element long_ Term_frame_idx is assigned to view_id different from the viewpoint associated with current encoded image for that will index (value) for a long time Image.By the way that long-term index is assigned into intersection visual point image, then this image can enter as being used as AVC images Row processing, the use that RPLR is instructed can act on long-term image by this.Therefore, it is possible to use AVC effectively reorders Intersect visual point image.

New RPLR instructions are proposed in the 4th embodiment.These instructions and the storage tube for controlling DPB 215 Manage ordering calculation similar, but these new commands are used in other viewpoints outside the viewpoint associated with current encoded image Image on run.These new syntax elements are shown in Figure 5, and Fig. 5 shows that MVC reference picture lists proposed by the present invention are reset Sequence grammer.

The default initialization of this embodiment is to set to be limited to reference according to H.264/MPEG-4AVC specification first All Time specified in list is referred to, in followed by the SPS of MVC extensions H.264/MPEG-4AVC for these images Defined intersection visual point image.

The partial arithmetic of this embodiment is syntactic element in ref_pic_list_reordering_mvc_extension Between separate, ref_pic_list_reordering can be called.This tissue called can cause the image based on multiple views It is processed in MVC environment and the result of the processing can in AVC environment to be located based on the image that multiple views are referred to Reason.

The various states related to index mvc_reordering_of_pic_nums_idc are shown in following table 1.

Table 1

In addition, syntactic element mvc_reordering_of_pic_nums_idc, abs_diff_pic_num_minus1 and Long_term_pic_num is used to provide from initial reference image list to changing the reference picture list decoded for bar Become.

Mvc_ref_pic_list_reordering_flag_l0 is equal to 1 prescriptive grammar element mvc_reordering_ Of_pic_nums_idc occurs providing reference picture list 0.Mvc_ref_pic_list_reordering_flag_l0 etc. Represent that this syntactic element does not occur in 0.

When mvc_ref_pic_list_reordering_flag_l0 is equal to 1, in mvc_ref_pic_list_ Mvc_reordering_of_pic_nums_idc is not equal to 3 number of times and should be no more than after reordering_flag_l0 num_ref_idx_l0_active_minus1+1。

RefPicList0 in the initial reference image list for providing to generate according to H.264/MPEG-4AVC specification [num_ref_idx_l0_active_minus1] is equal to " non-reference picture ", mvc_ref_pic_list_reordering_ Flag_l0, which should be equal to 1 and mvc_reordering_of_pic_nums_idc, should be not equal to 3, until according to H.264/ RefPicList0 [num_ref_idx_l0_ in the reference picture list that reorders of MPEG-4AVC specifications regulation generation Active_minus1] it is not equal to " non-reference picture ".

Mvc_ref_pic_list_reordering_flag_l1 is equal to 1 and represents syntactic element mvc_reordering_ Of_pic_nums_idc occurs providing reference picture list 1.Mvc_ref_pic_list_reordering_flag_l1 etc. Represent that this syntactic element does not occur in 0.

When mvc_ref_pic_list_reordering_flag_l1 is equal to 1, in mvc_ref_pic_list_ Mvc_reordering_of_pic_nums_idc is not equal to 3 number of times and should be no more than after reordering_flag_l1 num_ref_idx_l1_active_minus1+1。

Provided when decoding B bars and according to H.264/MPEG-4AVC specification in the initial reference image list of generation RefPicList1 [num_ref_idx_l1_active_minus1] is equal to " non-reference picture ", mvc_ref_pic_list_ Reordering_flag_l1, which should be equal to 1 and mvc_reordering_of_pic_nums_idc, should be not equal to 3, until RefPicList1 [num_ref_ in the reference picture list that reorders generated according to H.264/MPEG-4AVC specification regulation Idx_l1_active_minus1] it is not equal to " non-reference picture ".

Mvc_reordering_of_pic_nums_idc and difference_of_view_id and abs_diff_ Pic_num_minus1 or long_term_pic_num provide which reference picture is remapped together.mvc_reordering_ Of_pic_nums_idc value is provided in H.264/MPEG-4AVC specifications.Follow mvc_ref_pic_list_ closely The first mvc_ after reordering_flag_l0 or mvc_ref_pic_list_reordering_flag_l1 Reordering_of_pic_nums_idc value should not be equal to 3.

Abs_diff_pic_num_minus1, which adds 1 and defined, is moved into the image number currently indexed in list and image Absolute difference between number predicted value.Abs_diff_pic_num_minus1 scope should be 0 to MaxPicNum -1.Moved Dynamic image must have the view_id different from the view_id of present image.

Long_term_pic_num defines the long-term image number for being moved into the image currently indexed in list.Work as solution During code coded frame, long_term_pic_num should be equal to reference frame or the complementation distributed to labeled as " being used for long term reference " LongTermPicNum of the reference field to one of them.When decoding coded fields, long_term_pic_num should be equal to distribution LongTermPicNum to one of the reference field labeled as " being used for long term reference ".By mobile image must have with The view_id different view_id of present image.

Difference_of_view_id defines that to be moved into the view_id and view_id currently indexed in list pre- Absolute difference between measured value.

The process that progress reference picture reorders is based on default list, as follows as will be described below：

When mvc_ref_pic_list_reordering_flag_l0 is equal to 1, following steps are applicable.

- refIdxL0 is set to the index in reference picture list RefPicList0.It is initially set as 0.

- according to corresponding syntactic element mvc_reordering_of_pic_nums_idc occur in the bitstream it is suitable Ordered pair its handled.For these each syntactic elements, following steps are used.

1) if mvc_reordering_of_pic_nums_idc is equal to 0 or 1, process is adjusted specified in I below With using refIdxL0 as input, and output being assigned into refIdxL0.

2) otherwise, if mvc_reordering_of_pic_nums_idc is equal to 2, process quilt specified in II below Call, using refIdxL0 as input, and output is assigned to refIdxL0.

3) otherwise (mvc_reordering_of_pic_nums_idc is equal to 3), for reference picture list RefPicList0 operation of reordering terminates.

When mvc_ref_pic_list_reordering_flag_l1 is equal to 1, following steps are applicable.

- refIdxL1 is set to the index in reference picture list RefPicList1.It is initially set as 0.

4) if mvc_reordering_of_pic_nums_idc is equal to 0 or 1, process is adjusted specified in I below With using refIdxL1 as input, and output being assigned into refIdxL1.

5) otherwise, if mvc_reordering_of_pic_nums_idc is equal to 2, process quilt specified in II below Call, using refIdxL1 as input, and output is assigned to refIdxL1.

6) otherwise (mvc_reordering_of_pic_nums_idc is equal to 3), for reference picture list RefPicList1 operation of reordering terminates.

The scheme being set forth below is short for the view_id's for the view_id with the image different from present encoding Phase image list is reordered.

The input of this process is index refIdxLX (X is 0 or 1).

The output of this process is increment index refIdxLX.

ViewIdLX=viewIdLXPred+difference_of_view_id

Variable picNumLXNoWrap is obtained as below：

If-mvc_reordering_of_pic_nums_idc is equal to 0

if(picNumLXPred–(abs_diff_pic_num_minus1+1)<0)

PicNumLXNoWrap=picNumLXPred-(abs_diff_pic_num_minus1+1)+MaxPicNum

else

PicNumLXNoWrap=picNumLXPred-(abs_diff_pic_num_minus1+1)

- otherwise (mvc_reordering_of_pic_nums_idc is equal to 1)

if(picNumLXPred+(abs_diff_pic_num_minus1+1)>=MaxPicNum)

PicNumLXNoWrap=picNumLXPred+ (abs_diff_pic_num_minus1+1) MaxPicNum

else

PicNumLXNoWrap=picNumLXPred+ (abs_diff_pic_num_minus1+1)

ViewIdLXPred is variable viewIdLX predicted value.When process is adjusted for the first time for bar as defined in this subitem With (the mvc_reordering_of_pic_nums_ i.e. in ref_pic_list_reordering () grammer equal to 0 or 1 Idc occurs for the first time), viewIdL0Pred and viewIdL1Pred are initially set as CurrViewId.ViewIdLx's After each assignment, viewIdLX value is assigned to viewIdLXPred.

PicNumLXPred is variable picNumLXNoWrap predicted value.When process as defined in this subitem is for article by Once call the (mvc_reordering_of_pic_ i.e. in ref_pic_list_reordering () grammer equal to 0 or 1 Nums_idc occurs for the first time), picNumL0Pred and picNumL1Pred are initially set as CurrPicNum. After each assignment of picNumLXNoWrap, picNumLXNoWrap value is assigned to picNumLXPred.

Variable picNumLX is obtained as below：

if(picNumLXNoWrap>CurrPicNum)

PicNumLX=picNumLXNoWrap-MaxPicNum

else

PicNumLX=picNumLXNoWrap

PicNumLX and viewIdLX should be equal to the reference picture for being marked as " be used for short term reference " PicNum and ViewId and the PicNum for being not equal to the short-term reference picture for being marked as " being not present ".

The image with short-term image number picNumLX that following program is performed so that view_id to be equal to viewIdLX is put Put in index position refIdxLX, other remaining images are moved to the relatively rear position of list, and increase refIdxLX's Value.

For (cIdx=num_ref_idx_lX_active_minus1+1；cIdx>refIdxLX；cIdx--)

RefPicListX [cIdx]=RefPicListX [cIdx -1]

RefPicListX [refIdxLX++]=short-term reference picture with PicNum equal to picNumLX and view_id equal to viewIdLX

NIdx=refIdxLX

For (cIdx=refIdxLX；cIdx<=num_ref_idx_lX_active_minus1+1；cIdx++)

if((PicNumF(RefPicListX[cIdx])！=picNumLX) ＆＆ViewIdF (RefPicListX [cIdx])！=viewIdLX)

RefPicListX [nIdx++]=RefPicListX [cIdx]

Wherein following derivation function PicNumF (RefPicListX [cIdx])：

If-image RefPicListX [cIdx] is marked as " being used for short term reference ", PicNumF (RefPicListX [cIdx]) be image RefPicListX [cIdx] PicNum.

- otherwise (image RefPicListX [cIdx] is not labeled as " be used for short term reference ") PicNumF (RefPicListX [cIdx]) is equal to MaxPicNum.

It note that MaxPicNum value is never equal to picNumLX.

Wherein following derivation function ViewIdF (RefPicListX [cIdx])：

- ViewIdF (RefPicListX [cIdx]) is image RefPicListX [cIdx] viewId.

It is also noted that in this false code program, list RefPicListX is temporarily set to longer than needed for final list The long element of degree.After this program is performed, the element 0 of only list to num_ref_idx_lX_active_minus1 is needed It is retained.

The scheme being set forth below is used for reordering to the reference picture list comprising long term reference image.

The input of this process is index refIdxLX (X is 0 or 1).

The output of this process is increment index refIdxLX.

There is long-term image number in the viewpoint for performing the viewIdX instructions that following program is as above derived will appear in Long_term_pic_num image is placed on index position refIdxLX, by other remaining images be moved to list compared with Position, and increase refIdxLX value afterwards.

For (cIdx=num_ref_idx_lX_active_minus1+1；cIdx>refIdxLX；cIdx--)

RefPicListX [cIdx]=RefPicListX [cIdx -1]

RefPicListX [refIdxLX++]=long-term reference picture with LongTermPicNum equal to long_term_pic_num and viewId equal to viewIdLX

NIdx=refIdxLX

For (cIdx=refIdxLX；cIdx<=num_ref_idx_lX_active_minus1+1；cIdx++)

if((LongTermPicNumF(RefPicListX[cIdx])！=long_term_pic_num)s ＆＆ViewIdF (RefPicListX[cIdx]])！=viewIdLX)

RefPicListX [nIdx++]=RefPicListX [cIdx]

Wherein following derivation function LongTermPicNumF (RefPicListX [cIdx])：

If-image RefPicListX [cIdx] is marked as " being used for long term reference " LongTermPicNumF (RefPicListX [cIdx]) is image RefPicListX [cIdx] LongTermPicNum.

- otherwise (image RefPicListX [cIdx] is not labeled as " be used for long term reference "), LongTermPicNumF (RefPicListX [cIdx]) is equal to 2* (MaxLongTermFrameIdx+1).

It note that 2* (MaxLongTermFrameIdx+1) value is never equal to long_term_pic_num.

Wherein following derivation function ViewIdF (RefPicListX [cIdx])

If-image RefPicListX [cIdx] is marked as " being used for short term reference ", ViewIdF (RefPicListX [cIdx]) be image RefPicListX [cIdx] viewId.

It note that MaxPicNum value is never equal to picNumLX.In this false code program, list RefPicListX is temporarily set to an element longer than length needed for final list.After this program is performed, only list Element 0 to num_ref_idx_lX_active_minus1 be retained.

With reference to the syntactic element ref_pic_reordering_mvc_extension () shown in Fig. 6 to of the invention former Another structure of reason is illustrated and described.The computing of this syntactic element and the difference of the syntactic element described in reference picture 5 It is, list of reordering can be completed for any viewpoint, and this list of reordering only calls to complete by one, and Without calling single rec_pic_list_reordering instructions (as shown in Figure 3).

For the first embodiment under this other structure, to num_ref_idx_l0_active_minus1 and num_ Ref_idx_l1_active_minus1 is redefined again (please with effective number of reference pictures in instruction time meaning Referring to the previous section for illustrating these variables).

Present embodiment defines variable num_ref_idx_l0_active_view and num_ref_idx_l1_ Active_view, the two variables are used for the quantity for indicating to intersect the effective image in viewpoint.

Num_ref_idx_l0_active_view provides the maximum for intersecting viewpoint reference picture list 0 for solving code-bar Reference key.Num_ref_idx_l0_active_view scopes are defined as follows：

If the scope that-field_pic_flag is equal to 0, num_ref_idx_l0_active_view is 0 to 16, bag Include end points.It is the maximum for the decoding of frame macro block when MbaffFrameFlag is equal to 1, num_ref_idx_l0_active_view Index value and 2*num_ref_idx_l0_active_view are the maximum index values decoded for field macro block.

- otherwise (field_pic_flag is equal to 1), num_ref_idx_l0_active_view scope is 0 to 32, Including end points.

Num_ref_idx_l1_active_view and num_ref_idx_l0_active_view has identical semantic, Wherein l0 and list 0 are replaced by l1 and list 1 respectively.

These semantic values meet lower array function：

num_ref_idx_l0_active_minus1+num_ref_idx_l0_active_view<=num_ref_ frames

With

num_ref_idx_l1_active_minus1+num_ref_idx_l1_active_view<=num_ref_ frames

As described above, the default initialization process for MVC systems is it will be clear that ground is defined.

First, the All Time image of current view point is initialized according to the definition in MPEG-4AVC specifications.This The list of initialization and then num_ref_idx_l0_active_minus1 and num_ by being respectively used to LIST_0 and LIST_1 Value specified in ref_idx_l1_active_minus1 is blocked (truncated).

After these images, the intersection viewpoint reference picture defined in the SPS of MVC specifications is inserted into and by respectively In num_ref_idx_l0_active_view and num_ref_idx_l1_active_view for LIST_0 and LIST_1 Defined value is blocked.

Introduce and instructed with new RPLR as existing H.264/MPEG-4AVC RPLR instruction class.These instruction substitutions H.264/MPEG-4AVC the existing RPLR instructions (refer to Fig. 6) in.

New RPLR instructions have following semanteme.The instruction that whether there is is instructed to be shown in table 1 above for indicating to reorder In.

Syntactic element mvc_reordering_of_pic_nums_idc, abs_diff_pic_num_minus1 and long_ Term_pic_num is used to provide from initial reference image list to the reference picture that will be used to decode image information (such as bar) The change of list.

When mvc_ref_pic_list_reordering_flag_l0 is equal to 1, in mvc_ref_pic_list_ Mvc_reordering_of_pic_nums_idc is not equal to 3 number of times and should be no more than after reordering_flag_l0 num_ref_idx_l0_active_minus1+1+num_ref_idx_l0_active_view。

RefPicList0 in the initial reference image list for providing to generate according to H.264/MPEG-4AVC specification [num_ref_idx_l0_active_minus1+num_ref_idx_l0_active_view] is equal to " non-reference picture ", Mvc_ref_pic_list_reordering_flag_l0 should be equal to 1 and mvc_reordering_of_pic_nums_ Idc should be not equal to 3, until according in the reference picture list that reorders of H.264/MPEG-4AVC specification regulation generation RefPicList0 [num_ref_idx_l0_active_minus1+num_ref_idx_l0_active_view] is not equal to " nothing Reference picture ".

Mvc_ref_pic_list_reordering_flag_l1 is equal to 1 and represents syntactic element mvc_reordering_ Of_pic_nums_idc occurs providing reference picture list 1.

Mvc_ref_pic_list_reordering_flag_l1 is equal to 0 and represents that this syntactic element does not occur.

When mvc_ref_pic_list_reordering_flag_l1 is equal to 1, in mvc_ref_pic_list_ Mvc_reordering_of_pic_nums_idc is not equal to 3 number of times and should be no more than after reordering_flag_l1 num_ref_idx_l1_active_minus1+1+num_ref_idx_l1_active_view。

Provided when decoding B bars and according to H.264/MPEG-4AVC specification in the initial reference image list of generation RefPicList1 [num_ref_idx_l1_active_minus1+num_ref_idx_l1_active_view] is equal to " no ginseng Examine image ", mvc_ref_pic_list_reordering_flag_l1 should be equal to 1 and mvc_reordering_of_ Pic_nums_idc should be not equal to 3, until the reference picture that reorders generated according to H.264/MPEG-4AVC specification regulation RefPicList1 [num_ref_idx_l1_active_minus1+num_ref_idx_l1_active_view] in list It is not equal to " non-reference picture ".

Abs_diff_pic_num_minus1, which adds 1 and defined, is moved into the image number currently indexed in list and image Absolute difference between number predicted value.Abs_diff_pic_num_minus1 scope should be 0 to MaxPicNum -1.

Long_term_pic_num defines the long-term image number for being moved into the image currently indexed in list.Work as solution During code coded frame, long_term_pic_num should be equal to reference frame or the complementation distributed to labeled as " being used for long term reference " LongTermPicNum of the reference field to one of them.When decoding coded fields, long_term_pic_num should be equal to distribution LongTermPicNum to one of the reference field labeled as " being used for long term reference ".

3) otherwise (mvc_reordering_of_pic_nums_idc is equal to 3), for reference picture list RefPicList0 rearrangement program process terminates.

6) otherwise (mvc_reordering_of_pic_nums_idc is equal to 3), for reference picture list RefPicList1 rearrangement program process terminates.

The scheme being set forth below is used to reorder for the reference picture of short-term reference picture.

The input of this process is index refIdxLX (X is 0 or 1).

The output of this process is increment index refIdxLX.

ViewIdLX=viewIdLXPred+difference_of_view_id

It is following to derive variable picNumLXNoWrap：

If mvc_reordering_of_pic_nums_idc is equal to 0

if(picNumLXPred–(abs_diff_pic_num_minus1+1)<0)

PicNumLXNoWrap=picNumLXPred-(abs_diff_pic_num_minus1+1)+MaxPicNum

else

PicNumLXNoWrap=picNumLXPred-(abs_diff_pic_num_minus1+1)

Otherwise (mvc_reordering_of_pic_nums_idc is equal to 1)

if(picNumLXPred+(abs_diff_pic_num_minus1+1)>=MaxPicNum)

PicNumLXNoWrap=picNumLXPred+ (abs_diff_pic_num_minus1+1)-MaxPicNum

else

PicNumLXNoWrap=picNumLXPred+ (abs_diff_pic_num_minus1+1)

Variable picNumLX is obtained as below：

if(picNumLXNoWrap>CurrPicNum)

PicNumLX=picNumLXNoWrap-MaxPicNum

else

PicNumLX=picNumLXNoWrap

For (cIdx=num_ref_idx_lX_active_minus1+1；cIdx>refIdxLX；cIdx--)

RefPicListX [cIdx]=RefPicListX [cIdx -1]

NIdx=refIdxLX

For (cIdx=refIdxLX；cIdx<=num_ref_idx_lX_active_minus1+1；cIdx++)

RefPicListX [nIdx++]=RefPicListX [cIdx]

Wherein following derivation function PicNumF (RefPicListX [cIdx])：

It note that MaxPicNum value is never equal to picNumLX.

Wherein following derivation function ViewIdF (RefPicListX [cIdx])：

- ViewIdF (RefPicListX [cIdx]) is image RefPicListX [cIdx] viewId.

The input of this process is index refIdxLX (X is 0 or 1).

The output of this process is increment index refIdxLX.

For (cIdx=num_ref_idx_lX_active_minus1+1；cIdx>refIdxLX；cIdx--)

RefPicListX [cIdx]=RefPicListX [cIdx -1]

NIdx=refIdxLX

For (cIdx=refIdxLX；cIdx<=num_ref_idx_lX_active_minus1+1；cIdx++)

RefPicListX [nIdx++]=RefPicListX [cIdx]

Wherein following derivation function LongTermPicNumF (RefPicListX [cIdx])：

Wherein following derivation function ViewIdF (RefPicListX [cIdx])

MaxPicNum value is never equal to picNumLX.

It note that in this false code program, list RefPicListX is temporarily set to than length needed for final list A long element.After this program is performed, the element 0 of only list to num_ref_idx_lX_active_minus1 needs It is retained.

Second embodiment of the present invention under this other structure considers that intersecting reordering for visual point image occurs most frequently The image temporarily distributed in the time, it means that image number difference will be 0.Therefore, the starting point of present embodiment is to intersect The one side reordered of visual point image, it is more efficient than above-mentioned other embodiment.

Variable abs_diff_pic_num_minus1 is changed into abs_diff_pic_num and it is defined by this embodiment It is as follows：

Abs_diff_pic_num define be moved into the image number that is currently indexed in list and image number predicted value it Between absolute difference.

Abs_diff_pic_num scope is provided as follows：

If-reordering_of_pic_nums_idc be equal to 0, abs_diff_pic_num scope be 0 to MaxPicNum/2。

- otherwise (reordering_of_pic_nums_idc be equal to 1), abs_diff_pic_num scope be 0 to MaxPicNum/2-1.

By the above method, abs_diff_pic_num can be made to take 0 value, this is to intersect viewpoint to reorder most common feelings Condition.

Third embodiment of the present invention under this other structure also allows for the most normal hair that reorders for intersecting visual point image It is born in the image that the time temporarily distributes, it means that image number difference will be 0.

In the above-mentioned methods, abs_diff_pic_num is encoded as signless Exp-Golomb codes, so that as finger Show whether the interpolation represents the other grammer of positive or negative interpolation, it is shown as mvc_reordering_of_pic_ in figure 6 nums_idc。

Such an embodiment reduces the value that syntactic element can be used, as shown in Fig. 2 and transmitting Exp- for image number difference Golomb codes are as diff_pic_num, as shown in fig. 7, it discloses a kind of RPPL ref_pic_list_reordering_ Mvc_extension amendment definition.

Table 2

Diff_pic_num semantical definition is as follows：

Diff_pic_num defines the image number and image number predicted value for being moved into the image currently indexed in list Between difference.

Diff_pic_num scope is provided as follows：

- MaxPicNum/2-1 to MaxPicNum/2

Then picNumLXNoWrap derives as follows

PicNumLXNoWrap=picNumLXPred+diff_pic_num

The principle that Fig. 7 is the present invention is applied to the block diagram 700 that the reference picture of multi-vision-point encoding environment reorders.For this The operation of flow chart, it is assumed that there is reference picture in DPB 215.But reference picture can be different points of view.

In step 705, method considers to need whether the present image encoded is in using multi-view image (MVC and AVC) Environment in and coding environment whether be simple time environment (i.e. AVC).

If unrelated with multi-vision-point encoding in the used reference picture related to the coding of this image, operating procedure 710 with Using the known principle related to such as AVC coding standard come the reference picture list that reorders.But be encoded if desired Image be related to multi-view image, intersection viewpoint wherein between image needs to be considered, then selects step 715.

As described above, actually performing reordering for described image with several distinct programs.Above-mentioned some embodiments In the environment of (being applied to step 715) description, it is compared to consider to need the image encoded with whether reference picture has to appoint What intersects viewpoint relation.In some embodiments, image list reorders the reference related to any viewpoint of reordering Image.Then pair in other embodiment, reference pictures is carried out to all intersection viewpoints first and reordered, with being encoded The reference picture that image has same viewpoint is reordered.

Step 720 considers to need whether the image encoded has time relationship with reference picture.As described above, " P " image Using a reference listing, " B " image needs to utilize two reference listings.Therefore, the mode reordered to list is (such as Each embodiment described above) depend on this time relationship.

Step 725 is actually reordering for reference picture list.Similarly, the various modes for carrying out this operation are depended on Step 715 and 720 result, and acted according to the principle of above-mentioned embodiment.

Thus, for example, it will appreciated by the skilled person that the block diagram in specification is represented according to the principle of the invention Embodiment exemplary circuit conceptual illustration.Similarly, it will be understood that any flow chart, flow diagram, state Transition diagram, false code etc. all represent to have computer-readable medium represent and thus by the various of computer or computing device Whether process steps, this computer or processor are explicitly illustrated and irrelevant.

The function of various elements shown in accompanying drawing can combine with appropriate software by using specialized hardware and can The hardware of software is performed to provide.When provided by a-processor, the function can be by single application specific processor, single shared Processor or multiple processors (some to share) are provided.Also, term " processor " or " controller " is clear and definite Using being understood not to refer exclusively to the hardware for being able to carry out software, digital signal processor can also be included without limitation (DSP), for storing the read-only storage (ROM), random access storage device (RAM) and nonvolatile memory of software.

Other common and/or custom hardwares can also include.Similarly, any switch shown in figure is all concept Property.Its function can be even manually operated by the interaction of programmed logic, special logic, programme-control and special logic To perform, implementer can select specific technology according to application environment.

In the claims, any element being expressed as performing specific function all includes performing any of the function Mode, for example：A) combination of the circuit element of the function is performed；B) any type of software, including firmware, microcode etc., with Appropriate circuit combines to perform the software to perform the function.These claims limit principle of the invention based on The fact is that the functional form that each device quoted is provided is combined together in the way of claim is required.Therefore it can recognize Described device is functionally identical to for any these functional devices that can provide.

" embodiment " of the principle of the invention noted in the disclosure or " embodiment " represents that particular combination should Feature, structure, the key element of embodiment description are included at least one embodiment of the principle of the invention.Therefore, specification The statement " in one embodiment " or " in embodiments " that each place occurs not necessarily refer to same embodiment.

Most preferably, principle of the invention can be realized as the combination of hardware and software.Also, software can conduct Program storage unit (PSU) Shangdi application program is embedded in realize.Application program can be uploaded to comprising any appropriate architecture Machine and be executed by it.Preferably, the machine is performed on the calculating Polar Patform with hardware, and the hardware is, for example, one Or multiple central processing units (CPU), random access storage device (RAM), input and output (I/O) interface.Computer platform can be with Including operating system and micro-instruction code.Various handle with function described herein can be the microcommand generation that can be handled by CPU A part for code or a part for application program, or combinations thereof.In addition, various ancillary equipment can be connected to computer Platform, such as additional-data storage unit and print unit.

It will be further appreciated that because some system building blocks and method described in accompanying drawing are preferably real by software Existing, the actual connection between system element or processing function frame there may be not according to the programmed mode of principle of the present invention Same part.Based on the principle of the present invention, those of ordinary skill in the art are contemplated that for realizing the above-mentioned of the principle of the invention With other modes or structure.

Although with reference to presently preferred embodiment, the present invention is described, but it is understood that the invention is not limited in Disclosed embodiment.Moreover, within the principle and scope of the present invention, can modify to introduce this specification to the present invention Unaccounted any change, change, replacement or equivalent setting.Therefore, the present invention is not seen as limited by the foregoing description, But limited by scope of the following claims.

Claims

1. the method that a kind of reference picture list of encoding operation to for picture is ranked up, it is characterised in that：It is described to compile Code operation is the video encoding operations based on multiple views, wherein, the reference picture related to multiple viewpoints is used for video pictures Encoded, the reference picture list includes the time reference picture and intersection view reference for video pictures to be encoded Picture, each reference picture has the first field of the coded sequence for representing the reference picture, represented corresponding to the reference picture Viewpoint the second field and represent the reference picture long-term picture number the 3rd field, methods described include following step Suddenly：

It is determined that whether the video pictures to be encoded have with the reference picture in reference picture list intersects viewpoint relation；If deposited Then pair intersecting viewpoint relation, then reference picture is being reordered first against all intersection viewpoints, with regarding for being encoded The reference picture that frequency picture has same viewpoint is reordered, wherein considering to represent corresponding to reference picture when reordering Second field of viewpoint；

It is determined that whether the video pictures to be encoded have time relationship with the reference picture in reference picture list, if there is when Between relation, then reference picture is reordered according to the time relationship, if wherein the video pictures to be encoded are P pictures When, using a reference picture list, and if when the video pictures to be encoded are B pictures, using two reference picture lists.