WO2006040320A1

WO2006040320A1 - Method to add precision in transmitting mpeg stream over ip and device implementing the method

Info

Publication number: WO2006040320A1
Application number: PCT/EP2005/055159
Authority: WO
Inventors: Helmut Burklin; Mary-Luc Champel
Original assignee: Thomson Licensing
Priority date: 2004-10-11
Filing date: 2005-10-11
Publication date: 2006-04-20

Abstract

The invention is related to a method to send a first type of data packets comprising the steps of : - creation for each data packet of the first type of a time reference corresponding to this packet ; - creation of second type of data packet constituted by a header and a payload, the payload comprising at least one data packet of the first type ; characterized in that, the time references of each data packet of the first type contained in the payload of a packet of the second type are inserted in the header of the packet of the second type.

Description

Method to add precision in transmitting MPEG stream over IP and device implementing the method

The present invention concerns the domain of transporting audio/video programs over IP network. More precisely it concerns the transport of MPEG (Moving Picture Expert Group) transport stream using the Real Time Protocol (RTP) and the time management in this transport.

The Request For Comment RFC 3550 [3] describes general provisions how streams with timing requirements can be carried over an IP network, and also specifies that a single timestamp is attached to each RTP packet.

The document RFC 2250 [4] describes how several MPEG2 transport stream and program stream packets are to be carried inside the payload of an RTP packet.

The MPEG-2 specification [1] describes how timestamps are included inside an MPEG2 stream, how at the destination the different timestamps are used to recover a clock in synchronism with the clock at the source side, and how the different timestamps are used to feed the MPEG decoder and to control when to display the elements of the transmitted stream.

The DVB-IP specification [5] describes a generic architecture for distribution of multimedia services over IP networks. This specification adopts RFC2250 as the mechanism for transporting MPEG-2 streams over RTP.

The Pro-MPEG Wide Area Networking group has released Code of Practice #3 that also adopts RFC2250 as the transport mechanism for multimedia streams in a contribution environment. In conjunction with DVB- IP, there is now a end-to-end IP service architecture on which interoperability of equipments is assured thanks to RFC2250. A specification of ARIB (Association of Radio Industries and

Businesses) [6] describes a timestamp format that can be applied to each individual MPEG2 Transport stream packet allowing to reposition each packet in the correct temporal position of a stream, if ever this temporal information has been lost inside the transmission chain.

Since several MPEG2 packets (or other entities) are carried inside a single RTP packet, and since the RTP packet carries only a single timestamp in its header, at the destination of the RTP packet may contain insufficient information to replace all components at the correct time for succeeding processing. For full MPEG2 transport streams it is assumed that the entire MPEG2 stream is a constant bit rate stream, or that it is at least piecewise constant, i.e. that the bitrate changes only at the time where PCR or SCR timestamps are inserted inside the stream.

For the decoding of not piecewise constant streams, it is useful to distinguish the following two cases.

Usually for transmission over Ethernet type networks the size of the RTP packets is limited (to about 1500 bytes), limiting the number of TS packets that are transmitted with a single RTP timestamp (to maximal 7). Many state of the art decoders are able to decode streams in which only one out of 7 packets arrives in time, while the other packets are inserted in between without a precise timing. In fact, to guarantee non disturbed audio- video decoding it is sufficient that the decoder delays the output by the (maximum) time between two succeeding RTP packets and has supplementary buffering capacity of 6 TS packets in the decoder buffers (this time and these buffer sizes are small compared to the delay and buffer size of the RTP buffer).

In order to allow decoding also for those decoders that do not have the above mentioned facilities, each entity inside the stream may be complemented with a supplementary timestamp, e.g. the one specified by ARIB, as it is currently used in a different context (the transmission over http) in the DLNA specification [7]. The addition of supplementary timestamps to the payload is nevertheless not backwards compatible. An RTP packet that carries supplementary information in the payload is not longer compliant with RFC

2250, and so may not be labelled with the MPEG2 payload type of MP2T (33). For this reason a server which sends the same stream to two different destinations may need to use two different encapsulations and payload types. Also it becomes impossible to send the same stream as multicast to two destinations from which one accepts only streams compliant to

RFC 2250 while the other one relies on the supplementary timestamps.

The invention resolves this problem by allowing the addition of supplementary information while in the same time preserving the payload as specified in RFC 2250.

The invention is related to a method to send a first type of data packets comprising the steps of :

- creation for each data packet of the first type of a time reference corresponding to this packet ;

- creation of second type of data packet constituted by a header and a payload, the payload comprising at least one data packet of the first type ; characterized in that, the time references of each data packet of the first type contained in the payload of a packet of the second type are inserted in the header of the packet of the second type.

The invention is also related to an apparatus comprising

- means to receive a stream of data packet of a first type,

- means to create a time reference related to each received packet, - means to send a stream of data packet of a second type composed of a header and a payload where each data packet of the second type contains in its payload at least a data packet of the first type, characterized in that the apparatus further comprises means to insert in the header of each packet of the second type the time references related to the at least one data packet of the first type contained in the payload of the packet of the second type. The invention is also related to an apparatus comprising

- means to receive a stream of data packet of a second type composed of a header and a payload where each data packet of the second type contains in its payload at least a data packet of a first type,

- means to extract the data packets of the first type contained in the payload of data packets of the second type, characterized in that it further comprises means to extract from the header of data packet of the second type a time reference related to each extracted data packet of the first type.

More advantages of the invention will appear through the description of a particular, non-restricting embodiment of the invention. The embodiment will be described with reference to the following figures:

Figure 1 prior art, describes the header of a RTP packet

Figure 2 is an example of the extension header according to the present embodiment.

Figure 3 is a block diagram of a transmitter according to the present embodiment.

Figure 4 is a block diagram of a receiver according to the present embodiment.

A typical subsystem in a home network contains a server and one or more players. The server may typically contain a digital (satellite) tuner for TV reception, the player may be a TV that has a home network connection.

The digital satellite tuner will be programmed to receive a single programme out of a multiplex, this means that the analogical part of the tuner will be tuned to receive the appropriate channel, the multi-programme transport stream of this channel will then be filtered so that only those transport packets of relevance to the selected programme will remain. These transport packets arrive at the output of the filter as a sequence of packets and 'holes' (corresponding to the suppressed packets). The tuner may also contain a local system clock (synchronised to the PCR packets of the selected programme) that allows to associate a timestamp to each of the TS packets, as they leave the filter.

The server will also contain a means for collecting a number

(usually seven) TS packets, and wrap them into the payload of an RTP packet. According to the invention, we also want to add the more precise timestamp proposed by ARIB to each TS packet. This timestamp is created using a 27 Mhz clock and will date very precisely each TS packet.

As previously stated, we can't add these timestamp to the packet in the payload without breaking the compatibility with RFC 2250 that defines the payload of the RTP packet to be constituted only by the regular 188 bytes TS packets.

The idea here is to collect all these timestamps, called TTS timestamps, and to gather them in the header of the TTP packet using an extension header.

The RTP header contains the following fields as illustrated in figure 1. version (V): 2 bits.

This field identifies the version of RTP. The version defined by RFC 3550 is two (2).

padding (P): 1 bit

If the padding bit is set, the packet contains one or more additional padding octets at the end which are not part of the payload.

extension (X): 1 bit

If the extension bit is set, the fixed header MUST be followed by exactly one header extension, with a format defined below.

CSRC count (CC): 4 bits

The CSRC count contains the number of CSRC identifiers that follow the fixed header. marker (M): 1 bit

The interpretation of the marker is defined by a profile. It is intended to allow significant events such as frame boundaries to be marked in the packet stream.

payload type (PT): 7 bits

This field identifies the format of the RTP payload and determines its interpretation by the application. The payload type used for packets according to RFC 2250 is 33.

sequence number: 16 bits

The sequence number increments by one for each RTP data packet sent, and may be used by the receiver to detect packet loss and to restore packet sequence.

timestamp: 32 bits

The timestamp reflects the sampling instant of the first octet in the RTP data packet. RFC 2250 specifies a 90 kHz clock.

SSRC: 32 bits

The SSRC field identifies the synchronization source.

CSRC list: 0 to 15 items, 32 bits each The CSRC list identifies the contributing sources for the payload contained in this packet. The number of identifiers is given by the CC field.

The extension header is illustrated in figure 2 and is defined by RFC 3550 as follows :

An extension mechanism is provided to allow individual implementations to experiment with new payload-format-independent functions that require additional information to be carried in the RTP data packet header. This mechanism is designed so that the header extension may be ignored by other interoperating implementations that have not been extended. If the X bit in the RTP header is one, a variable-length header extension MUST be appended to the RTP header, following the CSRC list if present. The header extension contains a 16-bit length field that counts the number of 32-bit words in the extension, excluding the four-octet extension header (therefore zero is a valid length). Only a single extension can be appended to the RTP data header. To allow multiple interoperating implementations to each experiment independently with different header extensions, or to allow a particular implementation to experiment with more than one type of header extension, the first 16 bits of the header extension are left open for distinguishing identifiers or parameters. The format of these 16 bits is to be defined by the profile specification under which the implementations are operating.

For the purpose of the invention, it is proposed to store the TTS of the MPEG TS packets in an extension header. An example of such an extension header is given figure 2. The first field should be defined to identify our proposed extension. The length is the number of TTS timestamps stored in the extension header, usually it will be 7 as a RTP packet typically contains 7 MPEG TS packets, but of course this length will depend on the actual number of MPEG TS packets in the payload of the RTP packet. Following this length field, the timestamp are stored, one for each TS packet in the payload.

The server will send the so built RTP packet according to the corresponding RFCs, i.e. putting a 9OkHz RTP timestamp corresponding to the target transmission time of the first byte of the payload.

The networked TV receiver will receive these RTP packets, and use the RTP timestamps for de-jittering and possibly error recovery as specified in the corresponding RFCs. It will pass on the payload of these packets (containing only the MPEG2 transport stream packets) to its MPEG decoder. It may (but is not obliged to) also use the TTS timestamps in the extension header to precisely reconstruct the timing of these transport stream packet, so that the MPEG decoder receives the transport stream with exactly the same timing (with the exception of a constant delay) at which they appeared at the output of the filter in the tuner. It is proposed to add to the RTP packet of a partial transport stream an 'extension header'. The timestamps inside this header can be used by certain receivers to recalculate the precise timing of each packet of the partial transport stream. Legacy receivers will drop the extension header and process the MPEG2 stream in the usual manner.

According to an embodiment of the invention, the first packets are MPEG 2 Transport Stream packets. According to an embodiment of the invention, the timing information of these packets is formatted as TTS timestamps. According to an embodiment of the invention, the second packets are RTP packets and the timing reference is a timestamp determined by the transmitter for incorporation into individual RTP packets. According to an embodiment of the invention, the timing information is transmitted inside an extension header in the same second packet as the first packets, when the timing information is transmitted in a known order (e.g. the same as the order of the first packets) there is no difficulty to match timing information and first packets.

Another enhancement of the invention is to take advantage of the fact that a MPEG-2 timing information is given with a 27Mhz precision and such a timestamp information is actually constituted of two timing information:

• a main one with a precision of 90 kHz • a second one that when combined with first one allows to reach the precision of 27 MHz

Therefore instead of storing the whole timestamps in the extension header, it is possible to store the main part of the timestamps only once and to store each additional precision. This allows to use less space.

According to another embodiment of the invention, the timing information will be transmitted inside the payload of initial RTP packets. For that several solutions are possible:

• for each MPEG-2 TS packets, timing information can be added right before the packet inside the RTP payload. • Timing information of all MPEG-2 TS packets can be added in the payload either before or after the MPEG-2 TS packets.

Another enhancement of the invention is to take advantage of the two parts of the timing information and do the following:

• Since the main part of the timing information has a precision of 90 kHz and since RTP timestamps have a precision of 90 kHz, a first solution is to use this main timing information as the RTP timestamp.

• A second solution is to first store this main timing information either at the beginning of the RTP payload or at the end of the RTP payload right after the MPEG-2 TS packets payload. • Then, the additional timing information can be stored either, at the beginning of the RTP payload before MPEG-2 TS packets, either before each MPEG-2 TS packets, or either at the end of the RTP payload after MPEG-2 TS packets.

It is possible when storing several consecutive timing information that they do not possess the same main timing information. Nevertheless since timing information always increase between consecutive packets, if the second timing information of Packet N+1 is smaller than second timing information of Packet N, the device shall assume that the main timing information for packet N+1 shall be increased by 1 and finally actual global timing information of Packet N+1 is : (Main Timing info + 1 ) + Additional timing info. Note that this mechanism can only work when two consecutive packets are at most separated by twice the precision of additional timing info that is 2/9OkHz = 2*11 ms = 22ms. Fortunately, this is always the case for video packets (max 22ms => min 45 packets/s => min bitrate of 8.46kB/s => min bit rate of 68kb/s) .

According to another embodiment of the invention, the timing information will be transmitted in a different stream than the MPEG2 stream (i.e. with a different payload type). In this case the relationship between first packets and timing information needs some further consideration.

Three cases can be distinguished:

Both, the data packets and the timing information are sent in a single RTP session, i.e. to the same UDP port address, they share a single sequence number space. The matching of timing packets and data packets can be done either over the sequence numbers (e.g. allocating the even sequence numbers to data and the next higher odd sequence number to timing) or over the RTP timestamp (i.e. allocating to the timing information the same timestamp as to the corresponding data packet). This latter solution is possible since for a (dynamic) payload type different to 33 the choice of the timestamps is not restricted to be the transmission time of the first payload byte. If more than a single data and timing packet is transmitted with the same RTP timestamp (for high data rates), the matching becomes unambiguous while regarding the sequence numbers which must be strictly increasing in either stream.

A second solution is the use of two different port numbers, so two

RTP sessions for both streams. Matching can be achieved through the timestamps as above, or through the sequence numbers. In the latter case either both streams start with the same initial value of the sequence number, or the difference in the sequence numbers is known by other means (out of band signalling, or observation by the receiver of the first packets of each stream, or matching through the size of the packets, if not all packets carry the same number of first packets/timing information). Both solutions have advantages and disadvantages, one of the advantages of the second solution being the possibility to increment the sequence number of the data packets by one on each packet, allowing better backwards compatibility.

A third solution is, when transporting the TTS information in a RTP stream different from the one that transports the MPEG TS packets, to add in the payload of the RTP packet that is used to transport the TTS information, for each TTS information, the sequence number of the related MPEG TS packet. It is also possible to build, for each RTP packet transporting some MPEG TS packets, a corresponding RTP packet for the TTS information and then to mark this latter packet with the sequence number of the MPEG TS RTP packet. An enhancement of this latest solution could be to group TTS information and the corresponding sequence numbers of several MPEG TS RTP packets into one TTS RTP packet. These solutions do not imply any constraints regarding the choice of the port numbers used to multicast these RTP streams or any constraints regarding any synchronization between the two RTP sequence numbers.

For the three previous solutions, a possible enhancement is to take advantage of the two parts of the timing information and as described previously store the main part only once and store the additional part wherever the TTS was stored in previous solutions. The main part could be stored either in the RTP timestamp or at the beginning of the payload part corresponding to timestamp info (that is, right before the first TTS).

At the destination side, we may distinguish three cases: The legacy receiver will receive the stream, strip away the extension header (its presence is indicated by the 'X' bit in the RTP header and its length is coded in the lower part of the first extension header quadlet). It will give the TS packets to the MPEG decoder according to the RTP timestamp. It may or may not try to interpolate the timing of the 'inner' TS packets using the assumption that the stream is piecewise constant. A decoder with sufficient buffer capacity should be able to decode the stream, even if it was a partial transport stream for which the assumption of piecewise constant bit rate is not necessarily true.

A receiver with special means to process the supplementary TTS timestamps will be able to extract them from the extension header, there shall be the same number of TTS timestamps in the header as TS packets in the payload (one possible exception will be discussed below). It will be able to present each TS packet to the decoder at a time that corresponds exactly to the time that it was presented to the transmitter (+ some constant delay). Even MPEG decoders with very stringent timing requirements and minimal buffer may be used.

A third type of receiver may recognise that the extension header is present, but may not be willing or able to process the TTS time stamps. It may use this information to predict that the transmitted stream is possibly not piecewise constant, and depending on its decoders capabilities it may decide to either try to display the stream, or not to display the stream but to, e.g., display a message to the user telling that this stream can not be viewed.

The invention allows also the use of transmitters that don't support TTS timestamps (these timestamps have a resolution of 27MHz and need in general special hardware support). Such a transmitter may be allowed to add an empty extension header, at least when it is transmitting a partial transport stream. This empty extension header will signal to all receivers that the encapsulated transport stream is not necessarily piecewise constant bit rate. Receivers may now, dependent on their capabilities, decide to display the encapsulated stream, or not.

Figure 3 illustrates the creation of the TTS timestamps in the embodiment of the invention. The Transport Stream is received, for example, from a digital broadcast tuner. The stream is then transmitted to a demux which will extract the wanted service. The extracted packet, corresponding to the wanted services will be treated by the STC recovery which will generate the timestamp of the packet based on a 27MHz clock. Timestamps are transmitted to the RTP packetizer together with each packet. This packetizer put the timestamp the header extension instead of having these timestamp added to the TS packet.

Figure 4 illustrates the decoding of the timestamps at the receiver side. A de-jittered MPEG TS is generated using the timestamps as described in the figure.

References: [1 ] ISO/IEC 13818-1 :2000 Information technology - Generic coding of moving pictures and associated audio information: Systems, International Standards Organization. http^/www.iso.orq/iso/en/CataloqueDetailPaqe.CataloqueDetaiPC SNUMBER=31537 ISO/IEC 13818-2:2000 Information technology - Generic coding of moving pictures and associated audio information: Video, International Standards Organization. http://www.iso.orq/iso/en/CataloqueDetailPaqe.CataloqueDetail?C SNUMBER=31539

[2] MPEG-4 Video: ITU-T Rec H.264 | ISO/IEC 14496-10 Information Technology - coding of audio-visual objects - Part 10: Visual [3] IETF RFC 3550, RTP: A Transport Protocol for Real-Time

Applications, H. Schulzrinne et al., July 2003, available at http://www.ietf.org/rfc/rfc3550.txt

[4] IETF RFC 2250, RTP Payload Format for MPEG1/MPEG2 Video, D. Hoffman et al., January 1998, available at http://www.ietf.org/rfc/rfc2250.txt

[5] DVB TM 3022, Digital Video Broadcasting (DVB) - Transport of DVB Services over IP, ETSI, RTS/JTC-DVB-93, 2004-03-26

[6] ARIB* STD-B24 Version 3.2. Data Coding and Transmission Specification for Digital Broadcasting, Association of Radio Industries and Businesses, November 15, 2001.

[7] Digital Living Network Alliance - Home Networked Device Interoperability Guidelines Version: 1.0

Claims

1. Method to send a first type of data packets comprising the steps of :

2. Apparatus comprising

- means to receive a stream of data packet of a first type, - means to create a time reference related to each received packet,

- means to send a stream of data packet of a second type composed of a header and a payload where each data packet of the second type contains in its payload at least a data packet of the first type, characterized in that the apparatus further comprises means to insert in the header of each packet of the second type the time references related to the at least one data packet of the first type contained in the payload of the packet of the second type.

3. Apparatus comprising