JPH10174126A - Synchronization of stereoscopic video sequence - Google Patents

Abstract

PROBLEM TO BE SOLVED: To apply optimum image transmission order for minimizing a decoder input buffer size required for a system by arranging video images so as to be transmitted after respective low-order layer images corresponding to an imbalance predicted reinforce layer image. SOLUTION: The low-order layer and reinforce layer video sequences are received by a temporary multiplexer 105, the reinforce layer video is applied to a reinforce encoder 110, and the base layer video is applied to a low-order encoder 115. Afterwards, the encoded reinforce and base layers are demultiplexed by a system demultiplexer 125 through a system multiplexer 120 for transmitting them to a decoder as move streams, the encoded reinforce layer data are passed through a reinforce decoder 130, the low-order layer data are passed through a low-order decoder 135 and by combining the base layer data and reinforce layer data decoded by a temporary remultiplexer 140, reinforce and low-order layer output signals are outputted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

FIELD OF THE INVENTION The present invention relates to an apparatus and method for decoding a stereoscopic video sequence and synchronizing a display (eg, a presentation). In particular, an apparatus is provided for determining a presentation time stamp and a decoding time stamp of the enhancement layer, in addition to a corresponding optimal bit stream transmission order which minimizes the required decoder input buffer size.

[0002]

BACKGROUND OF THE INVENTION Digital technology has revolutionized the provision of video and audio services to consumers to provide much higher quality signals than analog technology and to provide additional features that were once unhandled. Digital devices are particularly advantageous for signals broadcast via cable television networks or by satellite to cable television affiliates and / or directly to home satellite television receivers. In such a system, the subscriber receives the digital data stream through a receiver / descrambler that decompresses and decrypts the data to reconstruct the original video and audio signals. The digital receiver includes a microcomputer and a memory storage element for use in its processing.

To provide a low cost receiver while providing high quality video and audio, it is necessary to limit the amount of data to be processed. In addition, the effective bandwidth for transmitting digital signals is also limited by physical constraints, existing communication protocols and government standards. Accordingly, various intra-frame data compression techniques have been developed that utilize the spatial correlation between adjacent pixels in a particular video image (eg, frame).

Further, the inter-frame compression technique uses motion compensation data and a block-matching motion estimation algorithm.
The temporal correlation between the corresponding regions of successive frames is used by using the motion estimation algorithm. In this case, a motion vector is determined for each block in the current image of the video by identifying the block in the preceding image that most closely approximates the particular current block. Thereafter, the entire current image is reconstructed at the decoder by sending the motion vectors necessary to identify the corresponding pair along with data representing the difference between the corresponding block pair. Block-matched motion estimation algorithms are particularly effective when combined with block-based spatial compression techniques such as the discrete cosine transform (DCT).

[0005] Additionally, of interest as a proposed stereoscopic video transmission format, the document ISO / IEC JTC1 / SC29 / WG11 N108 is hereby incorporated by reference.
8 Title "Proposed Draft Amendment No.3 to 13818-2 (Mu
lti-view Profile) "Motion Picture Expert Group (MPEG) MPEG-2, described in November 1995.
Multi-view profile (MVP) system.
Stereoscopic video provides a slight offset image of the same image to produce a picture combined with a deeper field, thereby creating a three-dimensional (3-D) effect. In such a system, two cameras are positioned about two inches apart to record the event with two independent video signals. The distance between the cameras is almost the same as the distance between the left and right eyes of a human. Further, for some stereoscopic video camcorders, two lenses are housed within one camcorder head and move synchronously, for example, when panning across video. The two video signals are received and recombined at a receiver to produce an image having a field of depth corresponding to normal human vision. Other special effects are also provided.

[0006] The MPEG MVP system includes two video layers transmitted in a multiplexed signal. First, the base layer (eg, the lower layer) represents the left image of the three-dimensional object. Second, the enhancement layer (e.g., auxiliary or upper) represents the right image of the object. Since the left and right images are of the same object and are only slightly offset from each other, there is often a large correlation between the base and enhancement layer video images. This correlation is used to compress the enhancement layer with respect to the base layer, thereby reducing the amount of data that needs to be transmitted within the enhancement layer to maintain a given video quality. Video quality generally corresponds to the quantization level of the video data.

[0007] The MPEG MVP system has three types of video pictures, in particular, intra-coded pictures (I-pictures), predictive-coded pictures (P-pictures), and bi-directional prediction. Encoding (bi-directionally predic
tive-coded) images (B-images). Further, the base layer includes a frame or field structured video sequence, while the enhancement layer includes only a frame structure. An I-picture completely describes a single video picture without reference to other pictures. For improved error concealment, a motion vector is provided to the I-picture. Since both the P and B pictures in the base layer are predicted from the I picture, errors in the I picture have the potential to have a greater impact on the displayed video. In addition, the images in the enhancement layer are
In a cross-layer prediction process known as (disparity prediction), prediction is made from an image in the base layer.
The prediction from one frame to another within a layer is known as temporal prediction.

In the base layer, the P picture is predicted based on the preceding I or P picture. A reference is made from the preceding I or P picture to a future P picture, which is known as forward prediction. The B image is predicted from a neighboring preceding I or P image and a neighboring succeeding I or P image.

In the enhancement layer, the P image is composed of (a) the most recently decoded image in the enhancement layer, (b) the latest base layer image in display order, or (c) the next lower layer image in display order. Predicted from Case (b) is often used when the most recent base layer image in the display order is an I-image.
In addition, the B image in the enhancement layer is calculated using (d) the most recent decoded enhancement layer image for forward prediction and the latest lower layer image in display order for backward prediction, and (e) forward Using the most recently decoded enhancement layer image for prediction and the next lower layer image in display order for backward prediction, or (f)
The prediction is performed using the most recent lower layer image in display order for forward prediction and the next lower layer image in display order for backward prediction. If the most recent lower layer image in the display order is an I image, only that I image is used for predictive coding (for example, there is no forward prediction).

Only the prediction modes (a), (b) and (d) are MPEG
Note that it is included in the MVP system. MVP
The system uses MPEG temporal sca
lability coding) is a subset of (a) to (f)
Mode.

In one additional configuration, the enhancement layer has only P and B images and no I images. Future images (ie, images that have not yet been displayed)
The reference to is called backward prediction. It should be noted that backward prediction does not occur in the enhancement layer. Therefore, the enhancement layer images are transmitted in the display order. There are situations where backward prediction is very useful in increasing the compression ratio. For example, in a scene where a door opens, the current image can be predicted based on a future image in which the door has already been opened, the one behind the door.

B-pictures provide the greatest compression, but also involve the greatest error. To eliminate the propagation of errors, B-pictures must never be predicted from other B-pictures in the base layer. P-pictures give less errors and less compression. I-pictures provide the least compression, but can also provide random access.

[0013]

Therefore, in order to decode a P-picture in the base layer, the preceding I-picture or P-picture must be valid. Similarly, B
In order to decode an image, the preceding P or I image and future P and I images must be valid. as a result,
Video images are encoded and transmitted in a dependency order such that all images used for prediction are encoded before images predicted therefrom. When the encoded signal is received at the decoder, the video image is decoded and reordered for display. Therefore, a temporary storage element is required to buffer the data before display. However, the requirement for a relatively large decoder input buffer results in increased decoder manufacturing costs. This is undesirable because the decoder is a mass product that must be manufactured at the lowest possible cost.

Therefore, there is a need to synchronize the decoding and presentation of the enhancement and base layer video sequences. Decoding and presentation synchronization for stereoscopic video are particularly important aspects of MVP. Since it is inherent that two images are tightly coupled to each other in stereoscopic video, loss of presentation or display synchronization causes many problems for the viewer, such as eye fatigue and headaches.

Further, when dealing with this problem for digitally compressed bit streams, the problem is that NTSC or PAL
Unlike for uncompressed bitstreams or analog signals that match the standard. For example, NTSC or
For the PAL signal, the picture is transmitted synchronously, so that the clock signal is derived directly from picture synch. In this case, synchronization of the two images is easily achieved using image synchronization.

However, in a digitally compressed stereoscopic bit stream, the amount of data for each image in each layer varies and depends on bit rate, image coding type and scene complexity. Therefore, the decoding and presentation timing is derived directly from the start of the image data. That is, unlike analog video transmission, digital compression bitstreams do not have the natural concept of synchronization pulses.

Therefore, it would be advantageous to provide a system for synchronizing the decoding and presentation of stereoscopic video sequences. The system must also be compatible with decoders that decode images sequentially (eg, one image at a time) or in parallel (eg, two images at a time). In addition, the system must provide an optimal image transmission order that minimizes the required decoder input buffer size. The present invention provides an apparatus having the above and other advantages.

[0018]

SUMMARY OF THE INVENTION In accordance with the present invention, there is provided a method and apparatus for aligning a transmission sequence of video images of lower and enhancement layers of a stereoscopic video sequence. In particular, the images are transmitted in such an order that the number of images that must be temporarily stored before the presentation is minimized. Furthermore, the decoding time for each image
The stamp (DTS) and the presentation time stamp (PTS) are determined to provide synchronization between the lower layer and enhancement layer images in the decoder where the decoding occurs in a continuous or parallel manner.

In particular, a method for ordering the transmission of a video image sequence within a lower layer and an enhancement layer of a stereoscopic video signal, wherein the enhancement layer includes an unbalanced predicted image predicted using a corresponding lower layer image. Is given. The method comprises:
Ordering the video images such that the unbalanced predicted enhancement layer images are transmitted after the corresponding respective lower layer images.

In the first embodiment, the lower layer is only the inner coded image (I image) including the continuous images I _Li , I _{Li + 1} , I _{Li + 2} , I _{Li + 3} , I _{Li + 4,} etc. Included and corresponding enhancement layer images are H _Ei , H
_{It is represented by Ei + 1} , _{HEi + 2} , _{HEi + 3} , _{HEi + 4,} and so on. In this case, the video images are I _Li , I _{Li + 1} , H _Ei , I _{Li + 2} , H _{Ei + 1} , I _{Li + 3} ,
H _{Ei + 2} , I _{Li + 4} , and H _{Ei + 3} are transmitted in this order (for example, sequence 1).

Alternatively, in the second embodiment, the video images are I _Li , H _Ei , I _{Li + 1} , H _{Ei + 1} , I _{Li + 2} , H _{Ei + 2} , I _{Li + 3} , H _{Ei + Three}
(For example, sequence 2).

In the third embodiment, the lower layer is an inner coded image (I image) including the continuous images I _Li , P _{Li + 1} , P _{Li + 2} , P _{Li + 3} , and P _{Li + 4.} And only the prediction-encoded image (P image), and the corresponding enhancement layer images are H _Ei , H _{Ei + 1} , H _{Ei + 2} , H _Ei
They are represented by _{Ei + 3} and _{HE + 4} , respectively. Here, the video images are I _Li , P _{Li + 1} , H _Ei , P _{Li + 2} , H _{Ei + 1} , P _{Li + 3} , H
It is transmitted in the order of _{Ei + 2} , P _{Li + 4} , H _{Ei + 3,} etc. (for example, sequence 3).

Alternatively, in the fourth embodiment, the video images are I _Li , H _Ei , P _{Li + 1} , H _{Ei + 1} , P _{Li + 2} , H _{Ei + 2} , P _{Li + 3} , H
_They are transmitted in the order of _{Ei + 3} (for example, sequence 4).

In the fifth embodiment, the lower layers are I _Li , B _{Li + 1} , P _{Li + 2} , B _{Li + 3} , P _{Li + 4} , B _{Li + 5} , P _{Li + 6} , etc. of the continuous image. , And the corresponding enhancement layer images are H _Ei , H _{Ei + 1} , H _{Ei + 2} , H _{Ei + 3} , H
_{Ei + 4} , H _{Ei + 5} , H _{Ei + 6} and so on. Video images are I _Li , P _{Li + 2} , B _{Li + 1} , H _Ei , H _{Ei + 1} , P _{Li + 4} , B _{Li + 3} , H
It is transmitted in the order of _{Ei + 2} , _{HEi + 3,} etc. (for example, sequence 5).

Alternatively, in the sixth embodiment, the video images are I _Li , H _Ei , P _{Li + 2} , B _{Li + 1} , H _{Ei + 1} , H _{Ei + 2} , P _{Li + 4} , B _{Li + 3} ,
It is transmitted in the order of H _{Ei + 3} , H _{Ei + 4,} etc. (for example, sequence 6).

Alternatively, in the seventh embodiment, the video images are I _Li , P _{Li + 2} , H _Ei , B _{Li + 1} , H _{Ei + 1} , P _{Li + 4} , H _{Ei + 2} , B _{Li + 3} ,
The transmission is performed in the order of H _{Ei + 3} (for example, sequence 7).

In the eighth embodiment, the lower layers include the continuous images I _Li , B _{Li + 1} , B _{Li + 2} , P _{Li + 3} , B _{Li + 4} , B _{Li + 5} , P _{Li + 6,} etc. Including an intra-coded image (I-picture), a predictive-coded picture (P-picture) and a continuous bidirectional predictive-coded picture (B-picture)
The corresponding enhancement layer images are H _Ei , H _{Ei + 1} , H _{Ei + 2} , H _{Ei + 3} , H _{Ei + 4} , H
_{Ei + 5} , H _{Ei + 6,} etc., respectively. Video images are I _Li , P _{Li + 3} , B _{Li + 1} , H _Ei , B _{Li + 2} , H _{Ei + 1} , H _{Ei + 2} , P _{Li + 6} , B
It is transmitted in the order of _{Li + 4} , _{HEi + 3} , _{BLi + 5} , _{HEi + 4} , _{HEi + 5} (for example, sequence 8).

Alternatively, in the ninth embodiment, the video images are I _Li , H _Ei , P _{Li + 3} , B _{Li + 1} , H _{Ei + 1} , B _{Li + 2} , H _{Ei + 2} , H _{Ei + 3} ,
The transmission is performed in the order of P _{Li + 6} , B _{Li + 4} , H _{Ei + 4} , B _{Li + 5} , H _{Ei + 5} , H _{Ei + 6} (for example, sequence 9).

Alternatively, in the tenth embodiment, the video images are I _Li , P _{Li + 3} , H _Ei , B _{Li + 1} , H _{Ei + 1} , B _{Li + 2} , H _{Ei + 2} , P
It is transmitted in the order of _{Li + 6} , _{HE + 3} , B _{+ 4} , _{HE + 4} , B _{+ 5} , _{HE + 5} (for example, sequence 10).

A corresponding device is also provided.

[0031] Additionally, a receiver is provided for processing a sequence of video images of the stereo signal including the lower layer and the enhancement layer. The receiver includes a memory, a decompression / prediction processor, and a memory manager that cooperates with the memory and the processor. The memory manager schedules the storage of the selected lower layer image in memory such that the lower layer image is processed by the decompression / prediction processor before the corresponding one of the unbalanced predicted enhancement layer images. In addition, the decryption can occur sequentially or in parallel.

[0032]

DETAILED DESCRIPTION A method and apparatus for synchronizing the decoding and presentation of a stereoscopic video image sequence is provided.

FIG. 1 is a block diagram of a coder / decoder for stereoscopic video. The MPEG MVP standard and similar devices involve the encoding of two video layers, including a lower layer and an enhancement layer.
For such applications, the lower layers are assigned to the left image,
The enhancement layer is assigned to the right image. In the coder / decoder (eg, codec) structure of FIG. 1, the lower layer and enhancement layer video sequences are temporarily remultiplexed (tempo).
ral remux) 105. Time division multiplexing (TD
Using MX), the enhancement layer video is provided to enhancement encoder 110, while the base layer video is provided to lower encoder 115. It should be noted that the lower layer video data is provided to the enhancement encoder 110 for imbalance prediction.

Thereafter, the encoded enhancement and base layers are provided to a system multiplexer 120 for transmission as a transport stream to a decoder. Typically, the transmission path is through a satellite link to the cable system headend or directly to the consumer's home. At the decoder 122, the transport stream is demultiplexed at the system demultiplexer 125. The encoded enhancement layer data is provided to enhancement decoder 130,
On the other hand, the encoded lower layer data is supplied to lower decoder 135. It should be noted that the decryption is preferably performed concurrently with the lower and enhancement layers simultaneously. other,
The enhancement decoder 130 and the lower decoder 135 may share common processing hardware, in which case the decoding is performed continuously one image at a time.

The decoded lower layer data is output from lower decoder 135 as an independent data stream, which is also provided to temporary remultiplexer 140. In temporary remultiplexer 140, the decoded base layer data and the decoded enhancement layer data are combined to provide an enhancement layer output signal as shown. Thereafter, the enhanced and lower layer output signals are provided to a display device for display.

Furthermore, the coded bitstreams for both the lower layer and the enhancement layer are encoded in a system multiplexer so that the decoder can decode every frame or field only depending on the already decoded frames or fields. Must be multiplexed at 120. However, this problem is complicated by the fact that the prediction modes for P and B pictures are different for the lower and enhancement layers. Furthermore, enhancement layer images are always transmitted in presentation order (eg, display order), but often not in lower layers. Therefore, it is often necessary to store and reorder the video images at the decoder so that the decoding and display occur in the proper order.

Additionally, difficulties arise in synchronizing the decoding and presentation of the data in the lower and enhancement layers. As described above, the video bitstreams for the lower and enhancement layers are transmitted as two elementary video streams. For a transport stream, two packet identifiers (PIDs) of the transport stream packets are specified in the transport stream program map section for the two layers. Furthermore, the timing information is carried in an additional field (for example, in a PCR_PID field) of the lower layer selection packet so as to function as a reference for timing comparison in the decoder. In particular, the sample of the 27 MHz clock is program_clock_reference
Sent in the (PCR) field. More precisely, the MPEG-2 system document ITU-T Rec. H.262, ISO / IEC 13818-1, April 27, 19, which is incorporated herein by reference.
As described in 95, the samples are carried in the program_clock_reference_base and program_clock_reference_extension fields. More detailed M
Regarding the PEG standard, the document incorporated here as a reference is ISO / IEC JTC1 / SC29 / WG11 N0702, titled "Infor
mation Technology -Generic Coding of Moving Pictur
es and Assosiated Audio, Recommendation H.262 ", 199
It is listed on March 25, 4 years.

The PCR indicates the expected time of completion of reading of the field from the bit stream in the decoder. The phase of the local clock execution at the decoder is compared to the PCR value in the bit stream at the moment when the PCR value is obtained to determine if the decoding of the video, audio and other data is synchronized. Further, the sample clock in the decoder is locked to the system clock derived from the PCR value. PCR value is ITU-T Rec. H.262
It is calculated as follows using the equations described in ISO / IEC 13818-1.

PCR (i) = PCR_base (i) × 300 + PCR_ext
(i), where PCR_base (i) = ((system_clock_frequency ×
t (i)) DIV300)% 2 33, and PCR_ext (i) = ((System _ clock _ frequency × t (i)) DIV 1 )% 300, wherein the symbol% indicates a modulo operation.

In a similar manner, for a program stream of a stereoscopic video signal, timing information is carried in the packet header as a 27 MHz clock sample in the system_clock_reference (SCR) field. The SCR value is calculated as follows using the equation described in ITU-T Rec. H.262 ISO / IEC 13818-1.

SCR (i) = SCR_base (i) × 300 + SCR_ext
(i), where SCR_base (i) = ((system_clock_frequency ×
t (i)) DIV300)% 2 33, and SCR_ext (i) = ((System _ clock _ frequency × t (i)) DIV 1 )% 300.

The identification of video packets at both the lower and enhancement layers is specified in the program stream map as two stream identifiers. For both transport and program streams, synchronization of the decoding and presentation processing for stereoscopic video is provided in packetized elementary stream (PES) packets. In particular, the presentation time stamp (PTS) and / or the decoding time stamp (DTS) are provided in an additional field of the PES header.

A PES packet is provided for each elementary video stream prior to transport or packetization of the program. If PTS and / or DTS needs to be sent to the decoder, a new PES packet is provided in the PES stream. Therefore, one important factor for synchronization is to calculate PTS and DTS accurately. PTS and DTS
Is the transport stream system target decoder (T-STD) described in ITU-T Rec. H.262 ISO / IEC 13818-1, or the program stream system target decoder.
It is determined by an encoder based on a virtual decoder model such as (P-STD).

The value of both PTS and DTS is 90 kHz per unit.
Specified within a unit period of the system clock frequency divided by 300 giving z. In particular, ITU-T Rec. H.262 ISO / I
PTS (k) = ((system_clock_frequency × tpn (k)) DIV30 as described in EC 13818-1
0)% 2 ³³ , where tpn (k) is the presentation time of the presentation unit Pn (k). Similarly, DTS (j) = ((system_clock_frequency × tdn (k)) DIV30
0)% 2 ^33, where, tdn (k) is the recovery Goka time access unit An (j). Thus, the video DTS indicates the time at which an image is required to be decoded by the STD. The video PTS indicates the time at which the decoded image is to be provided to the viewer (eg, displayed on a television screen). In addition, the time indicated by PTS and DTS is the current PCR or SCR
Evaluated for value.

The video bitstream is immediately decoded in the theoretical STD model. However, if the B-picture is in a lower layer of the stereoscopic bitstream, the bitstream arrives at the decoder in presentation (eg, display) order. In such a case, some I and / or P images must be temporarily stored in the decoder buffer in the STD until the appropriate presentation time after decoding. However, for the enhancement layer, all images arrive at the decoder in presentation order, so that PTS and DTS
The values must be offset by the same or fixed intervals.

To synchronize the lower layer and enhancement layer sequences, the corresponding images in the lower layer and the enhancement layer have the same PTS
Must have. The current method of calculating the DTS for the MPEG-2 main profile is used for calculating the DTS in the lower layer, eg, DTS _L (where “L” indicates the lower layer). Subsequent PTS and DTS values refer to the corresponding DTS _L. In particular, DTS _Li and PTS _Li indicate the DTS and PTS for the ith image in the lower layer, respectively. DTS
_Ei and PTS _Ei indicate the DTS and PTS, respectively, for the ith image in the enhancement layer. The time interval F between the presentation of successive images is then defined as F = 90 × 10 ³ / frame rate. For example, under the NTSC standard, 29.97
For a frame rate of frames / sec, F = 3,003.
F is the nominal frame time interval in a 90 kHz clock cycle, corresponding to an actual elapsed time of 3,003 cycles / 90 kHz = 0.03336 seconds. Under the PAL standard, F = 3,600 for a frame rate of 25 frames / sec.

Furthermore, the synchronization of the lower layer and enhancement layer sequences depends closely on the transmission and display order of the video sequence. Generally, for non-stereo video signals
The MPEG-2 standard does not specify a particular distribution that the I, P, and B images must take within the sequence in the base layer, but provides different degrees of compression and random access capabilities for different distributions. In one possible distribution, each image in the base layer is an I image. In other possible distributions, both I and P images are given,
Either the I, P and B images provided discontinuously are provided, or the I, P and B images provided two consecutively are provided. Generally, three or more consecutive B images are not used due to poor video quality. In the enhancement layer, a B image and a P image are provided, and an I image may additionally be provided.

FIG. 2 illustrates an enhancement layer image sequence and a first base layer image sequence for use in the apparatus of the present invention. Here, the lower layer includes only the I image. The enhancement layer image sequence is indicated by 200 and the lower layer sequence is indicated by 250. Sequences 200 and 250 are shown in display order. Each image has an image type (eg, I, B or P), a layer indication (eg, “E” for enhancement layers and “L” for lower layers), and a subscript “0” in the sequence. The zeroth image and a "1" is labeled to indicate the contiguous arrangement of the image, which is the first image in the sequence.

The enhancement layer 200 is composed of the images I _E0 (202), B _E1 (204), B _E2 (2
06), P _E3 (208), B _E4 (210), B _E5 (212), P _E6 (214), B _E7 (216), B
_E8 (218), P _E9 (220), B _E10 (222), B _E11 (224) and I _E12 (226)
including. However, the particular enhancement layer sequence shown is only an example. In any of the enhancement layer sequences discussed herein, including FIGS. 2-5, the enhancement layers are transmitted in display order and are not limited to a particular enhancement layer image type. Thus, any enhancement layer image is considered to be of a general image type (eg, _HEi (where "H" indicates an image type)).

The lower layer 250 is, in this example, I _L0 (252),
I _L1 (254), I _L2 (256), I _L3 (258), I _L4 (260), I _L5 (262), I _L6 (2
64), _IL7 (266), _IL8 (268), _IL9 (270), _IL10 (272), _IL11 (274)
And I _L12 (276). Additionally, the start of an image group (GOP) for each sequence is shown. The GOP indicates one or more continuous images to be decoded without referring to images in other GOPs. In general, the GOPs of the lower and enhancement layers are not side by side and have different lengths. For example, the start of the first GOP in enhancement layer 200 is indicated by image I _E0 (202), while the start of the second GOP is image I _E12 (226). Similarly, the start of the first GOP in lower layer 250 is indicated by image I _L2 (256), and the start of the second GOP is image I _L8 (268).

Further, the arrow shown in FIG. 2 indicates an allowable prediction mode in which the image indicated by the tip of the arrow is predicted based on the image connected to the rear end of the arrow. For example, image _BE1 (204) is predicted from image _IL1 (254). I-pictures are not predictive coded and are independent.

With respect to the image display order of FIG. 2, an advantageous transmission sequence starting with I _L2 according to the invention is I _L2 ,
_{B E1, I L3, B E2} , I L4, P E3, I L5, B E4, I L6, B E5, I L7, P E6, I L8, B
_E7 , _IL9 , _BE8 , _IL10 , _PE9 , _IL11 , _BE10 , _IL12 , _BE11, etc. (sequence 1). With respect to this picture order, each prediction coded picture arriving at the decoder does not have to be rearranged before decoding. Thus, the need for storage and processing at the decoder is reduced, thereby reducing the cost of the decoder. Other suitable image transmission sequences are _IL2 , _BE2 , _IL3 , P
_{E3, I L4, B E4,} I L5, B E5, I L6, P E6, I L7, B E7, I L8, B E8, I L9,
P _E9 , I _L10 , B _E10 , I _L11 , B _E11 , I _L12 , I _E12, etc. (sequence 2).

With respect to these image transmission sequences, all images arrive at the decoder in presentation order. Further, it is possible to determine an appropriate PTS and DTS for each image. First, the DTS of the i-th lower layer image
It is known to assume DTS _Li .

As a specific example, for the first image transmission sequence 1 of FIG. 2, decoding and presenting occur as shown in Table 1 below. Here, continuous decryption is assumed. In Table 1, the first column indicates the time of 0.5F increment using DTS _L2 as the start time, the second column indicates the decoding time of the lower layer image, and the third column indicates the decoding time of the enhancement layer image. The fourth column shows the presentation time of the lower layer and enhancement layer images.

[0055]

[Table 1] Here, it is necessary to save only the two decoded images. For example, _IL2 and _IL3 are decrypted and stored before _BE2 is received. When received, B _E2 is substantially outputted at the same time as I _L2 for immediately Fukugo of presentation.

Furthermore, for the ith image in either the lower order or the enhancement sequence, the DTS and PTS are determined from DTS _Li as follows.

[0057] _{PTS Li = DTS Li + 1.5F DTS} Ei = DTS Li + 1.5F PTS Ei = PTS Li e.g., PTS for P _E3 (208) in FIG. 2 is equal to the sum of DTS for 1.5F and I _L3. Therefore, Fukugo size and presentation 1.5 image time interval P _E3 (i.e., 1.5
F) only delayed the recovery Goka of I _L3.

Regarding the second image transfer sequence of FIG.
Decryption and presentation occurs as described in Table 2 below.

[0059]

[Table 2] Here, it is necessary to save only one decrypted image. For example, I _L2 is and stored is Fukugo of before the B _E2 is received. When received, B _E2 is outputted immediately be Fukugo of and for I _L2 presentation concurrently.

In either the lower or enhanced sequence
For the ith image, the DTS and PTS are determined from DTS _Li as follows for the transport sequence in Table 2.

PTS _Li = DTS _Li + 0.5F DTS _Ei = DTS _Li + 0.5F PTS _Ei = PTS _Li FIG. 3 illustrates the enhancement layer image sequence and the second base layer image sequence for use in the apparatus of the present invention. Things. Here, the lower layer includes both I and P images. The same elements as those in FIG. 2 are denoted by the same reference numerals. Strengthening layer 20
0 is the same as described above. The lower layer 300 includes an image sequence P _L0 (302), P _L1 (304), I _L2 (306), P _L3 (308), P _L4 (31
0), _PL5 (312), _PL6 (314), _IL8 (316), _PL9 (318), _PL10 (320), P
Includes _L11 (322) and P _L12 (326). GOP is I _L2 (306) and I
Start with _L8 (318).

Here, the prediction method is somewhat complicated. Recall that in the base layer the P picture is predictively coded using the closest preceding I or P picture. In the enhancement layer, the B picture can be predictively coded using up to three different possible modes. However, the corresponding lower layer image is I
If it is an image, only that I-image is used. Also,
In the enhancement layer, the P picture is predictively coded using the most recent enhancement layer image, the most recent lower layer image in display order, or the next lower layer image in display order. Again, when the corresponding lower layer image is an I image, only that I image is used. It should be noted that in some cases, the indicated prediction modes include additional paths.

Therefore, in the lower layer sequence 300, for example, P _L4 is encoded using P _L3 and P _L5 .
In reinforcing layer 200, P _E3 may be coded using B _E2 or P _L3. In accordance with the present invention, a suitable image transfer sequence beginning with I _L2 is I _L2 , B _E1 , P _L3 , B _E2 , P _L4 , P _E3 , P _L5 , B
_E4 , P _L6 , B _E5 , P _L7 , P _E6 , I _L8 , B _E7 , P _L9 , B _E8 , P _L10 , P _E9 , P _L11 , B
_E10 , P _L12 , B _E11, etc. (sequence 3). For this sequence, decryption and presentation occur as described in Table 3 below.

[0064]

[Table 3] Here, it is necessary to save only the two decoded images. For example, I _L2 and P _L3 are Fukugo of and stored before the B _E2 is received. When received, B _E2 is outputted immediately be Fukugo of and for I _L2 presentation concurrently.

In either the lower or enhanced sequence
For the ith image, DTS and PTS are determined from DTS _Li as follows for the transport sequence in Table 3.

PTS _Li = DTS _Li + 1.5F DTS _Ei = DTS _Li + 1.5F PTS _Ei = PTS _Li Alternatively, another suitable transfer sequence for the example of FIG. 3 is I _L2 , B _E2 , P _L3 , P _E3 , P _L4 , B _E4 , P _L5 , B _E5 , P _L6 , P _E6 , P _L7 , B
_E7 , _IL8 , _BE8 , _PL9 , _PE9 , _PL10 , _BE10 , _PL11 , _BE11 , _PL12 , _IE12, etc. (sequence 4). The decryption and presentation occurs as shown in Table 4 below.

[0067]

[Table 4] Here, only one decrypted image needs to be stored. For example, I _L2 is and stored is Fukugo of before the B _E2 is received, it is received, B _E2 is outputted immediately be Fukugo of and directly for I _L2 presentation concurrently.

In either the lower or enhanced sequence
For the ith image, the DTS and PTS are determined from DTS _Li as follows for the transport sequence in Table 4.

PTS _Li = DTS _Li + 0.5F DTS _Ei = DTS _Li + 0.5F PTS _Ei = PTS _Li FIG. 4 shows the enhancement layer image sequence used in the present invention and the third sequence.
5 shows a base layer image sequence. here,
The lower layers include I, P and B images, which are non-continuous. 2 and 3 are designated by the same reference numerals. The reinforcement layer 200 is the same as described above.
The lower layer 400 includes an image sequence P _L0 (402), B _L1 (404), I _L2 (4
06), B _L3 (408), P _L4 (410), B _L5 (412), P _L6 (414), B _L7 (416), I
_L8 (418), B _L9 (420), P _L10 (422), B _L11 (424) and P _L12 (426)
including. The GOP starts at _IL2 (406) and _IL8 (418).

Here, the prediction method is as follows. Recall that in the base layer, the B picture is predictively coded using the most recent preceding I or P picture and the most recent succeeding I or P picture. Therefore, the lower layer sequence 400
For example, B _L3 is encoded using I _L2 and P _L4 . A suitable image transmission sequence starting with I _L2 according to the invention is I _L2 , P _L4 , B _L3 , B _E2 , P _E3 , P _L6 , B _L5 , B _E4 , B _E5 , I _L8 , B
_L7, and the like _{P E6, B E7, P L10} , B L9, B E8, P E9, P L12, B L11, B E10, B E11 ( sequence 5). Alternatively, other suitable transmission sequences are I _L2 , B _E2 , P _L4 , B _L3 , P _E3 , B _E4 , P _L6 , B _L5 , B _E5 ,
P _E6 , I _L8 , B _L7 , B _E7 , B _E8 , P _L10 , B _L9 , P _E9 , B _E10 , P _L12 , B _L11 , B
_E11 , _IE12, etc. (sequence 6). Further, suitable transmission sequences are I _L2 , P _L4 , B _E2 , B _L3 , P _E3 , P _L6 , B _E4 , B _L5 ,
B _E5 , I _L8 , P _E6 , B _L7 , B _E7 , P _L10 , B _E8 , B _L9 , P _E9 , P _L12 , B _E10 , B
_L11 , _BE11, etc. (sequence 7).

In either the lower or enhanced sequence
For the i-th image, DTS and PTS are determined from DTS _Li as follows. For each image, the presentation of the image is delayed by an integer multiple of F following the decoding of the image.

For example, with respect to the first transmission sequence (sequence 5), decoding and presentation occur as shown in Table 5 below.

[0073]

[Table 5] Here, it is necessary to save only the three decoded images. For example, I _L2 , P _L4 and B _L3 are decrypted and stored before B _E2 is received, and upon receipt B _E2 is immediately decrypted and output directly for presentation at the same time as I _L2 Is done.

In either the lower or enhanced sequence
For the ith image, the DTS and PTS are determined from DTS _Li as follows for the transport sequence in Table 5.

[0075] _{PTS Li = DTS Li + (mod2} (i + 1) +1) 1.5F, DTS for all i _Ei = DTS _Li + 1.5F, with respect to i = 2 DTS _Ei = DTS
_Li + (1 + 2mod2 (i + 1)) F, PTS _Ei = PTS _Li for i> 2, for all i, where mod2 (i) is mod2 (i) when i is even
= 0 and integer i such that when i is odd, mod2 (i) = 1
This is a base 2 modulo.

For Sequence 6, decryption and presentation occur as described in Table 6 below.

[0077]

[Table 6] Here, it is necessary to save only the two decoded images. For example, P _L4 and B _L3 are decrypted and stored before P _E3 is received, and upon receipt, P _E3 is immediately decrypted and output directly for presentation at the same time as I _L2 .

In either the lower or enhanced sequence
For the ith image, the DTS and PTS are determined from DTS _Li as follows for the transport sequence in Table 6.

For PTS _Li = DTS _Li + F, i = 2, PTS _Li = DTS _Li +
DTS _Ei = DTS _Li + 0.5F, i for (3mod2 (i + 1) +1) 0.5F, i> 2
= 2 for DTS _Ei = DTS _Li + (1 + 2mod2 (i + 1)) 0.5F, for i> 2 PTS _Ei = PTS _Li , for all i, decoding and presentation for sequence 7 Occurs as described in Table 7 below.

[0080]

[Table 7] Here, it is necessary to save only the two decoded images. For example, I _L2 and P _L4 is a Fukugo of and stored before the B _E2 is received, it is received, B _E2 is outputted immediately be Fukugo of and directly for I _L2 presentation concurrently.

In either the lower or the enhanced sequence
For the ith image, the DTS and PTS are determined from DTS _Li as follows for the transport sequence in Table 7.

For PTS _Li = DTS _Li + F, i = 2, PTS _Li = DTS _Li +
DTS _Ei = DTS _Li + F, i = 2 for (4mod2 (i + 1) +1) 0.5F, i> 2
DTS _Ei = DTS _Li + (4mod2 (i + 1) +1) 0.5F for i> 2, PTS _Ei = PTS _Li for i> 2, FIG. 5 for use with the device of the invention for all i Fig. 4 illustrates an enhancement layer image sequence and a fourth base layer image sequence. Here, the lower layer includes I, P, and two consecutive B images. Figures 2-4
Are designated by the same reference numerals. The reinforcement layer 200 is the same as described above. The lower layer 500 is an image sequence B _L0 (502), B _L1 (504), I _L2 (506), B _L3 (508), B _L4 (51
0), _PL5 (512), _BL6 (514), _BL7 (516), _IL8 (518), _BL9 (520), B
Includes _L10 (522), P _L11 (524) and B _L12 (526). GOP is I _L2 (50
Start with 6) and I _L8 (518).

[0083] starting with I _L2, appropriate image sequence according to the _{invention, I L2, P L5, B} L3, B E2, B L4, P E3, B E4, I L8, B L6, B
_E5, B _L7, and the like _{P E6, B E7, P L11} , B L9, B E8, B L10, P E9, B E10 ( sequence 8). For this transmission sequence, decryption and presentation occur as described in Table 8 below.

[0084]

[Table 8] Here, it is necessary to save only the three decoded images. For example, I _L2 , P _L5, and B _L3 are decrypted and stored before B _E2 is received, and upon receipt, B _E2 is immediately decrypted and output directly for presentation at the same time as I _L2. Is done.

[0085] Within either the lower or enhanced sequence
For the ith image, the DTS and PTS are determined from DTS _Li as follows for the transport sequence in Table 8.

For PTS _Li = DTS _Li + 1.5F, i = 2, PTS _Li = DTS
_Li + (5mod2 (mod3 (i-1)) + 3) 0.5F, DTS _Ei = DTS for i> 2
DTS _Ei = DTS _Li + (3-mod2 (mod3 (i)) + 5 for _Li + 1.5F, i = 2
mod2 (mod3 (i-1)) 0.5F, PTS _Ei = PTS _Li for i> 2, for all i, where mod3 (i) is mod3 when i = 0 + 3n
(i) = 0, mod3 (i) = 1 when i = 1 + 3n, and mod3 (i) = 2 when i = 2 + 3n (n = 0,1,2, 3, ...), the base 3 modulo of the integer i.

Alternatively, another suitable transmission sequence is:
I _L2 , B _E2 , P _L5 , B _L3 , P _E3 , B _L4 , B _E4 , B _E5 , I _L8 , B _L6 , P _E6 , B _L7 , B
_E7 , _BE8 , _PL11 , _BL9 , _PE9 , _BL10 , _BE10 , _BE11, etc. (sequence 9). For this transmission sequence, the decryption and presentation occurs as described in Table 9 below.

[0088]

[Table 9] Here, it is necessary to save only the two decoded images. For example, I _L 2 and B _E2 is Fukugo of and stored before the P _L5 is received, it is received, I _L2 and B _E2
Is output for simultaneous presentation.

In either the lower or enhanced sequence
For the ith image, DTS and PTS are determined from DTS _Li as follows for the transport sequence in Table 9.

For PTS _Li = DTS _Li + F, i = 2, PTS _Li = DTS _Li +
DTS _Ei = DTS _Li + for (5mod2 (mod3 (i-1)) + 1) 0.5F, i> 2
DTS _Ei = DTS _Li + (5mod2 (mod3 (i-1)) + 1) for 0.5F, i = 2
0.5F, i> 2 with respect to PTS _Ei = PTS _Li, other suitable transmission sequence for all _{i, I L2, P L5,} B E2, B L3, P E3, B L4,
B _E4 , I _L8 , B _E5 , B _L6 , P _E6 , B _L7 , B _E7 , P _L11 , B _E8 , B _L9 , P _E9 , B _L10 ,
_BE10, etc. (sequence 10). For this transmission sequence, decryption and presentation occur as described in Table 10 below.

[0091]

[Table 10] Here, it is necessary to save only the two decoded images. For example, I _L2 and P _L5 is Fukugo of before the B _E2 is received and stored, it is received, B _E2 are output directly for Fukugoka it is and I _L2 and simultaneous presentation.

[0092] Within either the lower or enhancement sequence
For the ith image, the DTS and PTS are determined from DTS _Li as follows for the transport sequence in Table 10.

For PTS _Li = DTS _Li + F, i = 2, PTS _Li = DTS _Li +
DTS _Ei = DTS _Li + for (6mod2 (mod3 (i-1)) + 1) 0.5F, i> 2
DTS _Ei = DTS _Li + (6mod2 (mod3 (i-1)) + 1) 0.5 for F, i = 2
Note that in the above case for _FTS , PTS _Ei = PTS _Li for i> 2 and sequences 1 to 10 for all i, continuous coding was assumed. When parallel coding is used, the relationship between PTS and DTS is characterized in a more general way. In particular, when the lower layer has no B pictures but only I and / or P pictures, all pictures in both layers arrive at the decoder in presentation order. Therefore, for the ith image in either the lower or enhanced sequence, the DTS and PTS are:
Determined from _Li .

PTS _Li = DTS _Li + F, DTS _Ei = DTS _Li + F, PTS _Ei = PTS _Li This relationship is illustrated in Table 11 below. DTS _Li and DTS
The difference between _{L (i-1)} is F.

[0095]

[Table 11] For example, referring to Sequence 1 described in relation to FIG. 2, decryption and presentation occur as shown in Table 12 below.

[0096]

[Table 12] Here, only one decrypted image needs to be stored. For example, I _L2 is and stored is Fukugo of before the B _E2 is received. When received, B _E2 is outputted immediately be Fukugo of and for I _L2 substantially simultaneously presentations.

When the lower layer has a discontinuous B image, DTS
And PTS are determined from DTS _Li as follows. If the i-th picture in the lower layer is an I-picture with a "closed GOP" indicator or a P-picture followed by such an I-picture, then PTS _Li = DTS _Li + 2F holds. If the i-th picture in the lower layer is a P picture or an I picture of an "open GOP" and the (i + 1) th picture is not an I picture with a "closed GOP" indicator, PTS _Li = DTS _Li + 3F holds. If the i-th image in the lower layer is a B image, PTS _Li = DTS _Li
+ F. For the reinforcement layer, DTS _Ei = DTS _Li + 2F and PTS _Ei =
DTS _Li + 2F. In the MPEG-2 video protocol, a group of picture headers is included at the beginning of a GOP and is set to closed_gop = 0 by a 1-bit indicator (where closed_gop = 1 indicates a closed GOP). G open
The I image of the OP is treated in the same order as the P image according to the decoding order.

The decoding and presentation for non-continuous B images in the lower layers is illustrated in Table 13 below.

[0099]

[Table 13] In a specific example, the lower layer sequences are, in display order, I _L0 , B _L1 , P _L2 , B _L3 , P _L4 , B _L5 , I _L6 , I _L7, etc. The enhancement layer sequence consists of P _E0 , B _E1 ,
_BE2 , _BE3 , _BE4 , _BE5 , _PE6 , _PE7 and the like. One possible transmission order according to the present invention is I _L0 , P _L2 , B _L1 , P _E0 , P _L4 , B _E1 , B _L3 ,
_BE2 , _IL6 , _BE3 , _BL5 , _BE4 , _IL7 , _BE5 , and the like. DTS and PTS
Is determined as shown in Table 14.

[0100]

[Table 14] When the lower layer has two consecutive B images, DTS and PTS are calculated according to the following rules. If the i-th picture in the lower layer is an I-picture with a closed GOP indicator or a P-picture followed by such an I-picture, then PTS _Li = DTS _Li + 2F. If the i-th picture in the lower layer is a P picture or an I picture of an "open GOP" and the (i + 1) th picture is not an I picture with a "closed GOP" indicator, PTS _Li
= DTS _Li + 4F holds. If the i-th image in the lower layer is B
For images, PTS _Li = DTS _Li + F. For the reinforcement layer, D
TS _Ei = DTS _Li + 2F and PTS _Ei = DTS _Li + 2F.

The decoding and presentation for two consecutive B images in the lower layer is illustrated in Table 15 below.

[0102]

[Table 15] In certain instances, the lower layer sequence, etc. _{I L0, B L1, B L2} , P L3, B L4, B L5, I L6, I L7 on the display order. The enhancement layer sequence is P _E0 , B _E1 , B
_E2 , _BE3 , _BE4 , _BE5 , _PE6 , _PE7 and the like. One possible transmission order according to the present invention is I _L0 , P _L3 , B _L1 , P _E0 , B _L2 , B _E1 , I _L6 , B
_E2 , _BL4 , _BE3 , _BL5 , _BE4 , _IL7 , _BE5 , and the like. DTS and PTS are determined as shown in Table 16.

[0103]

[Table 16] The above rules applied to the frame mode can be generalized to the corresponding case of the film mode.

FIG. 6 is a block diagram of an enhancement layer decoder for stereoscopic video. The decoder 130 includes an input terminal 605 for receiving the compressed enhancement layer data and a transport level syntax parser 610 for parsing the data. The parsed data is provided to a memory manager 630 comprising a central processing unit. The memory manager 6
30 is connected to a memory 620 comprising, for example, a dynamic random access memory (DRAM). The memory manager also interfaces with the decompression / prediction processor 640,
Decoded lower level data temporarily stored in memory 620 is received via terminal 650 for subsequent use by processor 640 in decoding the unbalanced predicted enhancement layer image.

The decompression / prediction processor 640 provides various processing functions, such as error detection and correction, motion vector decoding, inverse quantization, inverse discrete cosine transform, Huffman decoding, and prediction calculations. After being processed by the decompression / prediction function 640, the decrypted enhancement layer data is output by the memory manager. Alternatively, the decrypted data is directly sent to the decompression / prediction function 640 through means not shown.
Can also be output from.

A similar structure can be used for the lower layers. In addition, enhancements and lower layers may share common hardware. For example, the memory 620 and the processor 640 can be shared. However, this is not possible if parallel coding is employed. A common clock signal (not shown) is provided to match the decoding according to the transmission sequence disclosed herein. In particular, it is necessary to temporarily store a lower layer image or another lower layer image used for prediction of an unbalanced predicted enhancement layer image before receiving predicted image data. According to the present invention, the number of images to be stored before decryption is minimized, thereby resulting in a reduction in memory size.

As can be seen, the present invention provides an advantageous image transmission technique for stereoscopic video image sequences. In particular, the images are transmitted in an order that minimizes the number of images that must be temporarily stored prior to the presentation.
Further, an example of the transmission sequence disclosed herein is MPEG
-2 Compatible with both the MVP protocol and the proposed MPEG-4 protocol. Furthermore, the decoding time stamp (DTS) and presentation
The time stamp (PTS) is determined at the decoder to provide synchronization between the lower and enhancement layers. DTS and PTS
Is set according to whether the decoding is continuous or parallel, whether the lower layer has no B-picture, has a non-continuous B-picture or has two consecutive B-pictures.

Although the invention has been described with reference to various specific embodiments, those skilled in the art will recognize that various additions and modifications may be made without departing from the spirit and aspects of the invention as set forth in the appended claims. By the way. For example, those skilled in the art will recognize that the techniques disclosed herein may be applied to other lower and enhancement layer sequences other than those specifically shown herein.

[Brief description of the drawings]

FIG. 1 is a block diagram of a coder / decoder structure for stereoscopic video.

FIG. 2 shows an enhancement layer image sequence and a first base layer image sequence for use with the apparatus of the present invention.

FIG. 3 shows an enhancement layer image sequence and a second base layer image sequence for use with the apparatus of the present invention.

FIG. 4 illustrates an enhancement layer image sequence and a third base layer image sequence for use with the apparatus of the present invention.

FIG. 5 shows an enhancement layer image sequence and a fourth base layer image sequence for use with the apparatus of the present invention.

FIG. 6 is a block diagram of an enhancement layer decoder structure for stereo video.

[Explanation of symbols]

 105 Temporary remultiplexer 110 Enhanced encoder 115 Lower encoder 120 System multiplex 122 Decoder 125 System demultiplex 130 Enhanced decoder 135 Lower decoder 140 Temporary remultiplexer

Claims (24)

[Claims] A method for arranging a sequence of video images in a lower layer and an enhancement layer of a stereoscopic video signal for transmission to a decoder, wherein the enhancement layer is an image predicted using a corresponding lower layer image. Arranging the video images such that the unbalanced predicted enhancement layer images are transmitted after the corresponding respective lower layer images. 2. The method according to claim 1, wherein the lower layers are I _Li , I _{Li + 1} , and I
_2. The method according to claim ₁ , wherein only the inner coded picture (I picture) containing _{Li + 2} is included, and the corresponding enhancement layer pictures are represented by H _Ei , H _{Ei + 1} , and H _{Ei + 2} respectively. And arranging the video images such that the video images are transmitted in the order I _Li , I _{Li + 1} , H _Ei , I _{Li + 2} . 3. The method according to claim 1, wherein the lower layer includes only an inner coded image (I image) including the continuous images I _Li and I _{Li + 1} , and the corresponding enhancement layer images are represented by H _Ei and H _{Ei + 1} , respectively. 2. The method according to claim 1, wherein the video image further comprises
Arranging the video images so that they are transmitted in the order of _Li , H _Ei , I _{Li + 1} , H _{Ei + 1} . 4. The method according to claim 1, wherein the lower layers are I _Li , P _{Li + 1} , and P
Includes only the inner coded image (I image) and the predicted coded image (P image) including _{Li + 2} , and the corresponding enhancement layer images are represented by H _Ei , H _{Ei + 1} , and H _{Ei + 2} , respectively. 2. The method according to claim 1, wherein the video image is I _Li , P
Arranging the video images so that they are transmitted in the order of _{Li + 1} , H _Ei , P _{Li + 2} . 5. The lower layer includes only an intra-coded image (I-picture) including a continuous picture I _Li and P _{Li + 1} and a predicted coded picture (P-picture), and the corresponding enhancement layer pictures are H _2. The method according to claim 1, wherein the video image is transmitted in the order of I _Li , H _Ei , P _{Li + 1} , and H _{Ei + 1} , wherein the video image is represented by _Ei and H _{Ei + 1.} Arranging the video images. 6. The method according to claim 1, wherein said lower layers are I _Li , B _{Li + 1} , and P
_Intra coded picture (I picture), predictive coded picture (P picture) and discontinuous bidirectional predictive coded picture (B picture) containing _{Li + 2}
And the corresponding enhancement layer images are H _Ei , H _{Ei + 1} , and
_2. The method according to claim 1, wherein the video image is represented by I _Li , P _{Li + 2} , B _{Li + 1} , H _Ei , H
Arranging the video images to be transmitted in the order of _{Ei + 1} . 7. The method according to claim 1, wherein the lower layers are I _Li , B _{Li + 1} , and P
_Intra coded picture (I picture), predictive coded picture (P picture) and discontinuous bidirectional predictive coded picture (B picture) containing _{Li + 2}
And the corresponding enhancement layer images are H _Ei , H _{Ei + 1} , and
_2. The method of claim 1, wherein the video image is represented by I _Li , H _Ei , P _{Li + 2} , B _{Li + 1} , H _{Ei + 2} .
Arranging the video images such that they are transmitted in the order of _{Ei + 1} , H _{Ei + 2} . 8. The method according to claim 1, wherein the lower layers are I _Li , B _{Li + 1} , and P
_Intra coded picture (I picture), predictive coded picture (P picture) and discontinuous bidirectional predictive coded picture (B picture) containing _{Li + 2}
And the corresponding enhancement layer images are H _Ei , H _{Ei + 1} , and
_2. The method of claim 1 wherein the video image is represented by I _Li , P _{Li + 2} , H _Ei , B _{Li + 1} , H _{Ei + 2} .
Arranging the video images to be transmitted in the order of _{Ei + 1} . 9. The method according to claim 1, wherein the lower layers are I _Li , B _{Li + 1} , B _{Li + 2} ,
And P _{Li + 3} , including an intra-coded image (I image), a predicted coded image (P image), and a continuous bidirectional predictive coded image (B image), and the corresponding enhancement layer images are H _Ei , H _{Ei + 1} ,
_2. The method of claim 1, wherein the video images are represented by I _Li , P _{Li + 3} , B _{Li + 1} , H _{Ei + 2} and H _{Ei +3} .
Arranging the video images so that they are transmitted in the order of H _Ei , B _{Li + 2} , H _{Ei + 1} , H _{Ei + 2} . 10. The method according to claim 1, wherein the lower layers are I _Li , B _{Li + 1} , B
Includes an intra-coded picture (I picture), a predictive coded picture (P picture) and a continuous bidirectional predictive coded picture (B picture) containing _{Li + 2} and P _{Li + 3} , and the corresponding enhancement layer pictures are respectively H _Ei ,
_{H Ei + 1, H Ei +} 2 , and a H _{Ei + 3} by the method of claim 1 where represented, further wherein the video image I _Li, H _Ei, P
_{Li + 3} , B _{Li + 1} , H _{Ei + 1} , B _{Li + 2} , _{HEi + 2} , _{HEi + 3} . 11. The method according to claim 1, wherein the lower layer is a sequence of images I _Li , B _{Li + 1} , B
Includes an intra-coded picture (I picture), a predictive coded picture (P picture) and a continuous bidirectional predictive coded picture (B picture) containing _{Li + 2} and P _{Li + 3} , and the corresponding enhancement layer pictures are respectively H _Ei ,
_2. The method according to claim 1, wherein the video image is represented by I _Li , P _{Li + 3} , H _{Ei + 1} , H _{Ei + 2} and H _{Ei + 3} .
Arranging the video images to be transmitted in the order of H _Ei , B _{Li + 1} , H _{Ei + 1} , B _{Li + 2} , H _{Ei + 2} . 12. A method for decoding a sequence of video images in a lower layer and an enhancement layer of a stereoscopic video signal in parallel, said lower layer comprising at least one intra-coded image (I-picture) and a prediction code. The method includes a coded image (P image) but does not include a bidirectional predictive coded image (B image), and is used to indicate a time for decoding each image and a time for presenting each image, respectively. Decode time
Providing a stamp (DTS) and a presentation time stamp (PTS) to the image, wherein the DTS of the i-th lower layer image is DTS _Li , and the PTS of the i-th lower layer image
Is PTS _Li , the DTS of the ith enhancement layer image is DTS _Hi , the PTS of the ith enhancement layer image is PTS _Hi , F is the time interval between the presentation of successive images, and PTS _Li =
DTS _Hi = PTS _Hi = DTS _Li + F. 13. A method for decoding a sequence of video images in a lower layer and an enhancement layer of a stereoscopic video signal in parallel, said lower layer comprising a discontinuous bidirectional predictive coded image (B).
Image), wherein a decoding time stamp (DTS) and a presentation time stamp (PTS) are provided to indicate the time for decoding and presenting each of the images, respectively. In the step of providing the image, the DTS of the i-th lower layer image is DTS _Li , the PTS of the i-th lower layer image is PTS _Li , and the DTS of the i-th enhancement layer image is DTS _Hi . , P of the i-th enhancement layer image
When TS is PTS _Hi , F is the time interval between the presentation of consecutive pictures, and when the i-th lower layer picture is an inner coded picture (I picture) with a closed GOP indicator, PTS _Li = DTS A process that is _Li + 2F. 14. The method according to claim 13, wherein the i-th lower layer image is a prediction coded image (P image) and
The method wherein PTS _Li = DTS _Li + 2F when the (i + 1) th lower layer image is an I image with a closed GOP indicator. 15. The I-picture according to claim 13, wherein the i-th lower-layer picture is a P-picture indicator and the (i + 1) -th lower-layer picture has a closed GOP indicator. If not, the method is PTS _Li = DTS _Li + 3F. 16. The method according to claim 13, wherein the i-th lower layer image is an I image having an open GOP indicator, and the (i + 1) -th lower layer image is If not an I-picture with a closed GOP indicator, P
The method where TS _Li = DTS _Li + 3F. 17. The method according to claim 13, wherein when the i-th lower layer image is a B image, PTS _Li = DTS _Li + F. 18. The method according to claim 13, wherein the DTS
_Hi = PTS _Hi = PTS _Li = DTS _Li + 2F. 19. A method for decoding a sequence of video images in a lower layer and an enhancement layer of a stereoscopic video signal in parallel, said lower layer comprising two consecutive bidirectional predictive coded images (B images). A method comprising at least one group, comprising a decoding time stamp (DTS) and a presentation time stamp (PTS) for respectively indicating the time for decoding and presenting each said image. ) To the image, wherein the DTS of the i-th lower layer image is DTS _Li , the PTS of the i-th lower layer image is PTS _Li , and the DTS of the i-th enhancement layer image is DTS _Li.
Hi , the PTS of the ith enhancement layer image is PTS _Hi , F is the time interval between presentations of successive images,
PTS _Li = DTS _Li + 2F when the i-th lower layer picture is an inner coded picture (I picture) having a closed GOP indicator. 20. The method according to claim 19, wherein the i-th lower layer image is a prediction coded image (P image) and
The method wherein PTS _Li = DTS _Li + 2F when the (i + 1) th lower layer image is an I image with a closed GOP indicator. 21. The method according to claim 19, wherein the i-th lower layer image is a P image indicator and the (i + 1) th lower layer image has a closed GOP indicator. If not, the method of PTS _Li = DTS _Li + 4F. 22. The method according to claim 19, wherein the i-th lower-layer image is an I image having an open GOP indicator, and the (i + 1) -th lower-layer image is If not an I-picture with a closed GOP indicator, P
The method where TS _Li = DTS _Li + 4F. 23. The method according to claim 19, wherein when the i-th lower layer image is a B image, PTS _Li = DTS _Li + F. 24. The method according to claim 19, wherein the DTS
_Hi = PTS _Hi = PTS _Li = DTS _Li + 2F.