JPH10303910A - Multimedia information distribution method and distribution system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To transmit an MPEG TS even when it is transmitted on a network by which a band of a LAN, etc. for which Ethernet (registered trademark) is used is difficult to be assured. SOLUTION: Data is transmitted at a smoothed rate after it is discarded (10) on the network and a decoder clock 18 is synchronized with a transmitting side by recovering a time interval of a PCR(program clock rate) by receiving terminals (14 to 18) by this method. In addition, a rate of the MPEG TS to be transferred is enabled to follow fluctuation of an effective speed of an interface at a LAN side by automatically changing an encoding mode to be discarded according to the fluctuation of the effective speed of the interface at the LAN side by relay devices (10 to 13).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multi-layered video signal capable of being transmitted in a data format of compressed video and audio information called MPEG TS even on a network such as a LAN using Ethernet where it is difficult to guarantee the bandwidth. The present invention relates to a media information distribution method and a distribution system.

[0002]

2. Description of the Related Art MPEG2 is an ISO (Interna)
Tional Standardization Or
and IS 13818, which is a technical standard for compressing and encoding video and audio. A video is treated as a sequence of images called pictures, and each picture is predicted by intra-frame prediction (I: Int
frame), forward inter-frame prediction (P: Pre
dict.), bidirectional inter-frame prediction (B: Bid)
encoding in any one of the following encoding modes. Pictures encoded in the I, P, and B encoding modes are called an I picture, a P picture, and a B picture, respectively. In order to decode a P picture, a previously decoded I picture or P picture is used as a reference picture for prediction, and in order to decode a B picture, a previously decoded I picture or P picture is predicted. Used as a reference image for For this reason, even if a B picture is lost, decoding of a picture in another mode is not affected. However, if a P picture is lost, decoding of a B picture becomes impossible, and if an I picture is lost, decoding of P and B pictures is performed. Becomes impossible.
A section from the beginning of the I picture information to immediately before the next I picture information is called a sequence, and the video stream is expressed as a continuation of the sequence. A sequence header is inserted at the beginning of each sequence, and a sequence header code (hereinafter referred to as SHC:
(Sequence Header Code) is inserted. At the beginning of each picture, a picture header is inserted, and at the start of the picture header, a picture start code (hereinafter, PSC: Picture Start) is inserted.
Code) is inserted. In the picture header, a picture coding type (hereinafter, PCT: Picture Coding T) indicating a coding mode of the picture is included.
ype).

[0003] MPEG2 is an MPEG TS.
It defines a data format for transmitting compressed video and audio information called a "compressed image". MPEG TS
A plurality of pieces of compressed information such as video, audio, and the like are fixed-length packets called TS packets as units, and PID (Packet)
Audio and video are multiplexed and demultiplexed using an identifier in a header called "IDentifier". MPEG T
The summary of the technology adopted in S is as follows. (A) The Program TS is included in the MPEG TS.
A field called k Reference (hereinafter, PCR) for defining a clock value on the transmission side is defined. By using this, the clock value of the transmitting side is periodically transmitted to the receiving side, and the increasing rate of the clock value (that is, the frequency of the clock) is synchronized with the increasing rate of the clock value of the transmitting side by the receiving side decoder. Synchronizes the decoder clock with the transmitter clock. (B) A NULL TS packet is defined as one type of TS packet. NULL TS packet is T
Inserted to keep the rate of S constant,
No special processing is performed on the UL TS packet. In this way, a method of transmitting video information using MPEG TS on a network such as an ATM, a digital CATV, and a satellite capable of guaranteeing a band has been defined and already implemented. For this reason, MPEG TS has become a substantial standard for the format of video information for distribution over a network.

[0004]

In the prior art, the AT
It is premised that information is transmitted on a network such as M, digital CATV, satellite, etc., which can guarantee a band.
However, actually, the receiving terminal may exist on a network such as a LAN using Ethernet, for which it is difficult to guarantee the bandwidth. In order to perform video distribution to such a terminal, a relay node (GW: relay device) that transfers the MPEG TS while appropriately reducing the rate is required.
That is, as shown in FIG.
Receiving terminal 5 connected to LAN 54 via network 52
5 transmits an MPEG stream to the ATM network 52.
There is no problem because the bandwidth can be guaranteed, but it is difficult to guarantee the bandwidth in the LAN 54.
The rate of the EG stream is reduced and transmitted to the receiving terminal 55 via the LAN 54. Here, as a method of reducing the rate, a method of discarding information of a picture in a certain encoding mode can be considered. However, when implementing a relay device, it is necessary to solve the following problems. That is, (Problem 1) Transmission method of TS after rate reduction By simply selecting and discarding information, looking at the rate after the reduction processing, the picture in the encoding mode that is reduced in average rate but not discarded is seen. Is being transferred, the rate remains at the same value before the reduction processing and is high. For this reason, in the time section where the rate is high, the probability of occurrence of a transmission error is the same as before the reduction processing, so that there is a problem that the rate reduction effect is small. That is, FIG.
As shown in (a), when the input information 60 is simply thinned out (62), the average rate is reduced, but the rate during the time when the information is transferred is not reduced. As a method for solving this, it is conceivable to temporarily store the TS after the rate reduction in a buffer, output the TS after smoothing the rate. However, if the rate is smoothed, it becomes impossible to synchronize the decoder clock to the transmitting side because the PCR time interval is not preserved. That is, FIG.
As shown in FIG. 0 (b), if the rate is smoothed after the rate of the input information 60 is reduced (63), the time interval of PCR is not preserved by the smoothing.

(Problem 2) Method of Determining Encoding Mode to be Discarded The effective speed (throughput) of the LAN-side (reception terminal-side) interface of the relay device fluctuates due to the fluctuation of the available transmission band on the LAN. In order to effectively use the transmission band, the relay device dynamically changes the encoding mode to be discarded, and changes the rate of the MPEG TS to be transmitted to the LAN.
It is necessary to follow the fluctuation of the effective speed of the side interface. Therefore, an object of the present invention is to solve these conventional problems and to save the time interval of the program clock rate (PCR) even when the rate is smoothed after rate reduction when transmitting over a network where bandwidth guarantee is difficult. Thus, the decoder clock can be synchronized with the transmission side, and even if the transmission band fluctuates, the transmission rate of the MPEG TS is made to follow the fluctuation of the effective speed of the LAN side interface, thereby enabling the transmission of the MPEG TS. It is an object of the present invention to provide such a multimedia information distribution method and distribution system.

[0006]

According to the multimedia information distribution method of the present invention, data is transmitted at a smoothed rate on a network, but a decoder clock is transmitted by restoring a time interval of PCR at a receiving terminal. The above problem 1 is solved by making it possible to synchronize with the side. Also, LA
The relay device automatically changes the encoding mode to be discarded in accordance with the change in the effective speed of the M-side interface, thereby changing the rate of the MPEG TS to be transferred to the LAN.
The above problem 2 is solved by following the fluctuation of the effective speed of the side interface. That is, in the first problem, the receiving side first restores the number of packets between TS packets including PCR, and then transfers the restored TS to the decoder at a rate that matches the rate of the original TS. The time interval of the PCR in the TS is restored. At this time, P
The restoration of the number of packets between TS packets including the CR is performed by the transmitting terminal using a control packet to notify the receiving terminal of the number of TS packets discarded before the TS packet including the PCR,
This is realized by the receiving terminal inserting the same number of dummy packets as the number of discarded TS packets. Also, since the number of TS packets input to the buffer 2 on the receiving side is the same as the original TS, if the buffer occupancy of the buffer 2 is adjusted to fluctuate near the initial value, the transfer rate to the decoder will be Matches the rate of the original TS. In the second problem, when the number of pictures to be transferred is larger than the effective speed of the LAN interface, the data occupancy of the buffer 1 of the relay device increases by the amount of data that cannot be transferred. At the stage where the value is exceeded, the selection level is increased and the number of pictures to be transferred is reduced. Conversely, if the network rate is too high, the amount of data occupied by the buffer 1 decreases, and when the value falls below the threshold, the selection level decreases, and the number of pictures to be transferred increases. With this mechanism, the number of pictures to be transferred is automatically increased or decreased according to a change in the effective speed of the LAN interface.

[0007]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 shows an embodiment of the present invention in which an MP is transmitted from a transmitting terminal on an ATM network to a receiving terminal on a LAN.
It is a figure of distribution of an EG stream. FIG. 10 (c)
1 is a schematic diagram of a transmission method according to the present invention. In the present invention, as shown in FIG. 10 (c), data is transmitted at a smoothed rate on a network.
Restore the CR time interval. That is, the gateway 6
In step 1, when the MPEG stream 60 is received, the TS after the rate reduction is temporarily stored in a buffer and the rate is smoothed and output (63). However, the rate restoring unit of the receiving terminal 66 again performs the PCR time interval (65). Is restored and input to the decoder. In FIG. 1, reference numerals 10 to 13 denote configurations on the relay device (gateway) side, and reference numerals 14 to 18 denote configurations on the receiving terminal side. In the relay device, when a TS packet is input from the transmitting terminal side network interface,
By adjusting the selection level by the selection level adjustment unit 13 so that the buffer (1) 12 does not overflow, the selection discarding unit 10 reduces the number of packets. Here, since the number of discarded packets is 4, the control packet insertion unit 11
Is inserted into the buffer (1) 1
Store in 2. As a result, the buffer occupancy is sent from the buffer (1) 12 to the selection level adjustment unit 13. In this way, the TS packet received from the transmitting terminal is selectively discarded, stored in the buffer (1) 12,
The TS packet stored in the buffer 12 is transmitted to the receiving terminal at the effective speed of the side network interface.
At this time, as described above, if a TS packet containing PCR or a TS packet containing SHC is received, the number of discarded TS packets up to that time and whether or not SHC has been detected are written in a payload. New TS packet is inserted. This special TS packet is called a control packet. A special PID value is used for the control packet.

Next, when the receiving terminal receives the control packet, the control packet processing unit 14 detects that 4 is written in the control packet, and inserts four dummy packets (NULL TS packets). Then, the data is accumulated in the buffer (2) 15. Thus, at the receiving terminal,
When the control packet is received, the same number of NULL TS packets as the number of discarded packets written in the payload is stored in the buffer (2) 15 instead of the discarded packets. A NULL TS packet stored in place of the discarded packet is called a dummy packet. The frequency of the clock 18 is adjusted by periodically sending the buffer occupancy from the buffer (2) 15 to the clock adjusting unit 17 so that the occupancy of the buffer 15 becomes close to the initial value. At the receiving terminal,
The TS packets stored in the buffer (2) 15 are transferred to the decoder 16 at a constant rate according to the clock. Table 1 shows the picture selection levels. As shown in Table 1, a selection level for defining a transfer target picture is defined.

[Table 1] The relay device checks the occupation amount of the buffer 1 at a certain period, and if the occupation amount of the buffer 1 is lower than a certain threshold value, the relay device decreases the selection level by one. If the occupation amount of the buffer 1 exceeds a certain threshold value in a certain cycle, the selection level is increased by one. The selection level adjustment unit 13 in FIG.
Notifies the selection discarding unit 10 of the increase / decrease value of the selection level when the level is selected.

FIG. 8 is a diagram showing the relationship between the picture selection level and the video stream state. In FIG. 8, the hatched lines indicate the transfer state, and the white frames indicate the discarded state. For example, when the picture selection level is “0” in Table 1, all of the SHC, I picture, P picture, and B picture are transferred. When the selection level is "1", only the SHC, I picture, and P picture are transferred, and the B picture is discarded. When the selection level is "2", only the SHC and I picture are transferred, and the rest are discarded.

FIG. 2 is a configuration diagram of a relay device showing one embodiment of the present invention. In the relay apparatus, as described above, the selection level is adjusted, and the clock adjustment cycle in the receiving terminal is set to a time corresponding to one sequence, and is adjusted each time a sequence header appears. In the relay device, AT
First, an MPEGTS packet received through an M network or the like is input to a stream type, SHC, and PSC detection unit 20. The stream type, SHC, and PSC detection units 20
Detects and detects the type of stream (audio, video, etc.) that transmits the input TS packet, the presence / absence and position of PCR (the byte number of the payload), the presence / absence and position of PSC, and the PCT when PSC exists. The result and the TS packet are transmitted to the transfer / discard unit 10. The transfer / discard unit 10 internally holds the state and selection level of the video stream in a memory. Table 2 shows the state of the video stream.

[Table 2]

In FIG. 2, the transfer / discard unit 10 should transfer a packet unless the stream transmitted by the packet is a video. When the stream is a video, the transfer / discard unit 10 first sets the stream type, SHC, PS
Based on the information from the C detection unit 20, the state of the video stream in the payload is updated according to Table 2. When the state of the video stream is the transfer target indicated by the selection level, the packet is processed as transfer, and when not, the packet is discarded. Also, at this time, if the state of the video stream changes to the payload and the information is to be discarded from the transfer target in the middle, the payload is rewritten to leave only the transfer target information in the payload and the data to be discarded. Is replaced with a stuffing byte (data having no meaning inserted for length adjustment). After performing the above processing, the transfer / discard unit 10
Whether the packet is transferred or discarded,
Control packet insertion unit 11 whether CR or SHC is included
, And when transferring the TS packet,
Transmit S packet. The control packet insertion unit 11 internally stores a memory for counting the number of discarded TS packets, and when notified that the packet is discarded, discards the discarded T packet.
Increase the number of S packets by one. Control packet insertion unit 11
Transfers the transferred TS packet to the buffer (1) 12. However, if a TS packet containing PCR or SHC is transmitted, a control packet describing the number of discarded TS packets and the presence or absence of SHC is inserted before the input TS packet, and the number of discarded TS packets is returned to zero.

In FIG. 2, the TS packets stored in the buffer (1) 12 are TCP / IP (Transmi
session Control Protocol / In
The transfer is performed by a transfer protocol suitable for LAN such as Internet Protocol (LAN). Further, when the TS packet including the SHC is input, the transfer / discard unit 10 transmits a selection level adjustment instruction request to the selection level adjustment unit 13. In response to this, the selection level adjustment unit 13 decreases the selection level if the buffer occupancy of the buffer 12 is below a certain threshold, and decreases the selection level if the buffer occupancy is above a certain threshold. Tell them to increase. As an example of the threshold value, the value for the decrease instruction is ８ of the buffer size, and the value for the increase instruction is ２ of the buffer size.

FIG. 3 is a diagram showing a control packet format used in the present invention. A TS packet header 31 has a PID value of 0 × 000f. Below is the payload. 32 is the first byte, SHC
Indicator is stored. 33 is the first to fifth bytes from the first, and Discarded T
P Num is stored. SHC Indicator
Is 0x01 when the next TS packet contains SHC.
And 0x00 when not included. Also, Disc
“arded TP Num” is the TS discarded until immediately before
The number of packets is represented by a 4-byte integer.

FIG. 5 is an operation flowchart of the relay apparatus of FIG. First, in the TS packet reception waiting state (step 101), it is determined whether or not the TS packet includes a PCR (step 102), and if it does, a control packet insertion process is performed (step 103). If not included, it is next determined whether or not the packet is a TS packet for transmitting video (step 104). If NO, the TS packet is transferred to the buffer 12 (step 112). If YES, it is determined whether or not the TS packet contains SHC (step 105). If the TS packet contains SHC, control packet insertion processing is performed (step 106), and the picture selection level is updated (step 107). After the update of the picture selection level, and when the TS packet does not include the SHC, the state of the video stream is updated (step 108). Next, it is determined whether or not the packet is a TS packet to be transferred (step 109).
Transfer the packet to the buffer 12 (step 11
2). If it is a TS packet that should not be transferred,
The number of discarded TS packets is increased by 1 (step 110),
The TS packet is discarded (step 111). Then, the process returns to the initial state (step 101).

FIG. 6 is a flowchart of a control packet insertion process of the relay apparatus of FIG. Control packet insertion unit 1
In 1, the control packet is first transferred to the buffer 12 (step 201). Next, the number of discarded TS packets is returned to 0 (step 202).

FIG. 4 is a block diagram of the apparatus on the receiving side showing an embodiment of the present invention. TS packets transferred by a transfer protocol suitable for a LAN such as TCP / IP are stored in the control packet processing unit 14. Control packet processing unit 1
4 stores the TS packet in the buffer (2) 15.
At this time, if the TS packet is a control packet, the same number of dummy packets as the number of discarded packets written in the payload are stored in the buffer 15. Clock section 18
Transmits an output instruction to the buffer output unit 19 every time the clock value advances by one, and responds accordingly to the buffer output unit 19.
Transfers the TS packet in the buffer 15 to the decoder 16. When receiving the control packet indicating the detection of the SHC, the control packet processing unit 14 transmits an adjustment instruction request to the clock adjustment unit 17. In response to this, the clock adjusting unit 17 calculates the increment of the clock frequency by the following equation (1) and transmits the increment to the clock. df = (k1 (b−b0) + k2 (bb ′)) / t _seq (1) where df is a clock increment, and b is currently stored in the buffer (2). The number of TS packets b 'is the value of b at the time of the previous clock adjustment b0 is the initial value of the number of TS packets in the buffer (2) t _seq is the length (unit (second)) of one sequence k1, k2 Is an example of specific values of appropriate constants k1 and k2, where k1 is 0.1,
k2 is 0.05. The initial value (b0) of the occupation amount of the buffer (2) is set to の of the buffer size, and NULL TS packets of this number are accumulated in the buffer (2).

FIG. 7 is an operation flowchart of the receiving apparatus of FIG. First, when in a waiting state for receiving a TS packet (step 301), it is determined whether or not the TS packet is a control packet (step 302). If it is not a control packet, it is stored in the output buffer 15 (step 30).
6). If it is a control packet, then the TS packet is SH
It is determined whether or not C detection is indicated (step 303), and SH
When the detection of C is indicated, the clock is adjusted (step 304), and when the detection of SHC is not indicated, a dummy TS packet is inserted (step 305). Then, the packet inserted in the output buffer 15 is accumulated (step 306).

[0018]

As described above, according to the present invention,
Even if the rate is smoothed after the rate reduction, the time interval of the program clock rate (PCR) can be preserved and the decoder clock can be synchronized with the transmission side. Rate L
Since it is possible to follow the fluctuation of the effective speed of the AN-side interface, even when transmitting on a network such as a LAN using Ethernet where it is difficult to guarantee the bandwidth, M
Transmission of PEG TS becomes possible.

[Brief description of the drawings]

FIG. 1 is a schematic diagram of a multimedia information distribution method showing one embodiment of the present invention.

FIG. 2 is a detailed configuration diagram of a relay device in FIG. 1;

FIG. 3 is a format diagram of a control packet used in the present invention.

FIG. 4 is a diagram showing a configuration of a receiving side device in FIG. 1;

FIG. 5 is an operation flowchart of the relay device shown in FIG. 2;

6 is a flowchart of a control packet insertion process of the relay device shown in FIG.

FIG. 7 is an operation flowchart of the reception-side device shown in FIG. 4;

FIG. 8 is a diagram illustrating a relationship between a picture selection level and a video stream state according to the present invention.

FIG. 9 shows a conventional MPE from a transmitting terminal to a receiving terminal.
It is a figure showing distribution of a G stream.

FIG. 10 is a comparison diagram between a conventional data transfer method on a LAN and a transfer method according to the present invention.

[Description of Signs] 10: transfer / discard unit, 11: control packet insertion unit, 12
... Buffer (1), 13 ... Selection level adjuster, 20 ... Stream type, SHC, PSC detector, 14 ... Control packet processor, 15 ... Buffer (2), 16 ... Decoder, 17 ... Clock adjuster, 18 ... Clock, 19: buffer output unit, 31: TS packet header, 32: first byte, 33: first to fifth byte from the second byte, 34: payload, 60: MPEG stream, 6
1 gateway, 63 smoothed rate, 65 clock rate, 66 receiving terminal.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁶ Identification code FI H04N 7/24 H04N 7/13 Z

Claims (6)

[Claims] 1. A fixed-length packetized multimedia information transmitted from a transmission terminal accommodated in a bandwidth guaranteed network in which a transmission band is guaranteed is accommodated in a band non-guaranteed network in which the transmission band is not guaranteed via a relay device. The relay device selectively discards only discardable packets among packets received from the transmitting terminal, counts the number of discarded packets, and periodically A control packet describing the counted number of discarded packets is generated, the number of discarded packets counted up to that point is reset, and the control packet is output together with the undiscarded packets at the transmission rate of the bandwidth non-guaranteed network. When the receiving terminal detects a control packet from a packet received from the relay device, the receiving terminal discards a packet described in the control packet. Multimedia information distribution method is characterized in that outputs a frequency of dummy packet after replacing the control packet. 2. The relay device selectively discards only packets that can be discarded among packets received from the transmitting terminal, counts the number of discarded packets, and calculates time information of the transmitting terminal from the received packets. When detecting a packet including clock reference information indicating that the control packet is generated, a control packet describing the counted number of discarded packets is generated, the number of discarded packets counted up to that point is reset, and the control packet is not discarded. The reception terminal outputs the control packet together with the packet at the transmission rate of the bandwidth non-guaranteed network. When the reception terminal detects a control packet from the packet received from the relay device, the reception terminal outputs the control packet as dummy packets as the number of discarded packets described in the control packet. After replacing with the packet, it is synchronized with the clock reference indicating the time information of the transmitting terminal. Multimedia information distribution method according to claim 1, characterized in that the output rate. 3. The method of selectively discarding only packets that can be discarded by the relay device, the method comprising: checking a transmission buffer occupancy of a packet at a fixed period; and comparing the occupancy with a preset threshold. 3. The method according to claim 1, further comprising the step of determining a type of a packet to be discarded.
The multimedia information distribution method described in 1. 4. A bandwidth guaranteed network in which a transmission bandwidth is guaranteed, a bandwidth non-guaranteed network in which a transmission bandwidth is not guaranteed, a relay device provided at a connection point between the bandwidth guaranteed network and the bandwidth non-guaranteed network, A transmission terminal accommodated in the bandwidth non-guaranteed network, and a fixed-length packetized multimedia information from the transmission terminal accommodated in the bandwidth guarantee network. A system for distributing to a receiving terminal accommodated in the bandwidth non-guaranteed network via a device, wherein the relay device selectively discards only discardable packets among packets received from the transmitting terminal. Buffer means for accumulating packets that have not been discarded, means for counting the number of discarded packets, and generating a control packet describing the number of discarded packets counted at a fixed period, Buffer means, resetting the counted number of discarded packets, and means for outputting the packets stored in the buffer means at the transmission rate of the bandwidth non-guaranteed network, the receiving terminal, Buffer means for accumulating packets received from the relay device, and when detecting a control packet from the received packet, replaces dummy packets for the number of discarded packets described in the control packet with the control packets, and stores the control packets in the buffer means. A multimedia information distribution system comprising: means for inserting; and means for outputting packets stored in the buffer means. 5. The relay device, wherein the relay device selectively discards only packets that can be discarded among the packets received from the transmitting terminal, a buffer that stores packets that have not been discarded, and the number of discarded packets. Means for counting the number of discarded packets, and means for detecting a packet containing clock reference information indicating time information of the transmitting terminal from the received packet. Produces
Means for resetting the counted number of discarded packets, means for inserting the control packet immediately before the packet including the detected clock reference information of the buffer means, and means for securing the packet stored in the buffer means to the bandwidth non-guaranteed. Means for outputting at a network transmission rate, the receiving terminal stores buffer packets received from the relay device, and, when detecting a control packet from the received packet, converts the control packet into the control packet. Means for inserting into the buffer means in place of the dummy packets for the number of discarded packets described in (1), and outputting the packets stored in the buffer means at a rate synchronized with a clock reference indicating time information of the transmitting terminal. 5. The multimedia information according to claim 4, further comprising: Delivery system. 6. A means for selectively discarding only packets that can be discarded by the relay device, checks the occupancy of the buffer means at a fixed cycle, and compares the occupancy with a preset threshold. 6. The multimedia information distribution system according to claim 4, further comprising means for determining a type of a packet to be discarded.