[go: up one dir, main page]

WO2018109500A1 - Protocole de transport de vidéo résistant aux erreurs et à faible retard sur un transit ip public - Google Patents

Protocole de transport de vidéo résistant aux erreurs et à faible retard sur un transit ip public Download PDF

Info

Publication number
WO2018109500A1
WO2018109500A1 PCT/GB2017/053772 GB2017053772W WO2018109500A1 WO 2018109500 A1 WO2018109500 A1 WO 2018109500A1 GB 2017053772 W GB2017053772 W GB 2017053772W WO 2018109500 A1 WO2018109500 A1 WO 2018109500A1
Authority
WO
WIPO (PCT)
Prior art keywords
receiver
transmitter
network
data
packets
Prior art date
Application number
PCT/GB2017/053772
Other languages
English (en)
Inventor
Alberto Boratto CARPENÈ
Pawel Kolodziej
Original Assignee
In Aria! Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by In Aria! Ltd filed Critical In Aria! Ltd
Publication of WO2018109500A1 publication Critical patent/WO2018109500A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1825—Adaptation of specific ARQ protocol parameters according to transmission conditions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0072—Error control for data other than payload data, e.g. control data
    • H04L1/0073—Special arrangements for feedback channel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/752—Media network packet handling adapting media to network capabilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/0001—Systems modifying transmission characteristics according to link quality, e.g. power backoff
    • H04L1/0009—Systems modifying transmission characteristics according to link quality, e.g. power backoff by adapting the channel coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1803—Stop-and-wait protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1806—Go-back-N protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1809—Selective-repeat protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L1/16—Arrangements for detecting or preventing errors in the information received by using return channel in which the return channel carries supervisory signals, e.g. repetition request signals
    • H04L1/18—Automatic repetition systems, e.g. Van Duuren systems
    • H04L1/1829—Arrangements specially adapted for the receiver end
    • H04L1/1835—Buffer management
    • H04L1/1838—Buffer management for semi-reliable protocols, e.g. for less sensitive applications such as streaming video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/12—Arrangements for detecting or preventing errors in the information received by using return channel
    • H04L2001/125—Arrangements for preventing errors in the return channel

Definitions

  • FEC Forward Error Correction
  • data packets are formatted in a redundant coding scheme, such that if a packet is received at the receiving node in a damaged or incomplete state, its data can be successfully reconstructed from the redundant data coded therein.
  • FEC advantageously allows for fulfillment of damaged or incomplete packets without any back communication to the sending node, which decreases latency in error correction.
  • the redundant data included in each packet means that larger packets, or a greater number of packets, are sent.
  • the increased bandwidth requirements of FEC transmission especially for high overhead FEC coding schemes, may be problematic, as bandwidth is limited.
  • a second error control method is known as Automatic Repeat reQuest (ARQ), which relies on feedback sent from the receiving node to the transmitting node to identify missing or damaged packets and to request their re-transmission.
  • ARQ Automatic Repeat reQuest
  • the receiving node sends acknowledgement messages to the sending node indicating which packets in the data window have been correctly received and decoded, for example an acknowledgement message (ACK).
  • ACK acknowledgement message
  • the feedback comprises a negative acknowledgement (NACK) for those packets that are missing or damaged.
  • This process may be continued or repeated as necessary, until each packet in the series is fulfilled, or until a timeout limit is reached.
  • Selective Repeat is part of the family of automatic repeat-request (ARQ) error control methodologies.
  • ARQ automatic repeat-request
  • the sending node and receiving node are configured such that corrupted or missing packets are identified by the receiving node.
  • Feedback messages sent from the receiver to the sending node result in the re-transmission of corrupted or missing packets, and the process is performed until each packet in the series is fulfilled, or until a timeout limit is reached.
  • Selective Repeat ARQ is widely considered to be the best match for correcting video streaming packets among all the other ARQ schemes. While ARQ provides great improvement and error control to network transmissions, it is not without shortcomings. Mainly, the requirement for a feedback loop to fulfill missing data increases latency.
  • hybrid ARQ packets sent from the transmitter to the receiving node may comprise FEC encoding to correct errors, within an ARQ error detection scheme for requesting unfulfilled packets.
  • FEC encoded blocks for the packets are generated at the sending node, but are not transmitted unless the overlaying ARQ mechanism determines that there are errors in a packet, in which case the FEC parity or other correction data is requested by the receiving node and sent by the transmitting node.
  • Hybrid ARQ can be used to increase transmission quality, yet may not be fast or efficient enough to enable high quality video transmission over lossy public networks.
  • the scope of the invention encompasses a method of transmitting data between a transmitting node and a receiving node, such as video data, over a network, such as the internet, wherein an ARQ is used to identify missing data and request retransmission of such missing data, and FEC is used in the coding of the feedback messages to increase the probability of successful transmission and speed the fulfillment of missing data.
  • the method may be carried out in a system comprising a transmission node, a receiving node, and intermediate routers.
  • the scope of the invention encompasses transmission and receiving routers configured to perform the methods of the invention as well.
  • the scope of the invention further includes hardware elements capable of carrying out the operations of the combined SR-ARQ-FEC methods.
  • Fig. 1 is a diagram depicting an overview of the general method of the invention.
  • Fig. 2 is a diagram depicting the operations of the general method of the invention.
  • Fig. 14 Detailed Description of the Invention.
  • the various elements of the invention are described next.
  • the various embodiments of the invention are directed to the transmission of data, especially video (and associated audio data) over a public network such as the internet.
  • the data transmitted in the methods of the invention may be derived from any source.
  • the data may be supplied from a storage device, a network such as the internet or a proprietary network, or a camera.
  • the source may comprise high quality video in a format such as MPEG-TS supplied over UDP/IP from a unicast or multicast LAN input.
  • the data being transmitted may comprise any data type, including data coding for voice, audio, video, and other data.
  • the data is video data from a video file.
  • the video data may comprise MPEG2-TS video data, HD video data, SD video data, and 4K-UHD resolution video data.
  • the data will be transmitted in the form of a packet.
  • Each packet within the data window is formatted according to the network protocol being used for transfer of the data.
  • the packet will comprise and addressing data specifying the destination of the packet.
  • the packet may comprise packet sequence identifiers, comprising packet sequence information indicating the packet's order in the data window.
  • the packet may further comprise transmission timing information, indicating the time of transmission, so that the receiving node can determine the age of the packet.
  • the packet may further comprise information indicating the type of encryption and/or encoding that was used to format the packet, so that it can be properly decoded at the receiver.
  • the packet will also comprise a payload comprising the data from the data source.
  • the packet may also comprise a constant length header.
  • Packets may be of any length, for example, a data packet in the range of 56 to 1372 bytes.
  • the packets comprise the data to be transmitted, e.g. video data and will be referred to herein as data packets.
  • Packets can be created from the source data using any packetization methodology known in the art.
  • the data may be encrypted for transmission and de-crypted at the destination node.
  • Any encryption methodology known in the art may be used, including, for example encryption standards such as AES 128, AES 192, or AES 256.
  • the methods of the invention may be employed on any data network, and may be used for the transmission of any type of data.
  • the methods of the invention may be employed in wired networks, wireless networks, and hybrid networks comprising both wired and wireless links.
  • the network in which the invention is applied may, for example, be the internet, a public switched telephone network, a wireless data network, or a proprietary network such as a cable network.
  • the error control methods of the invention are applied in an overlay network setting.
  • the overlay network may comprise any overlay network, for example a peer-to-peer (P2P) network, virtual private networks (VPN) or a content delivery network (CDN).
  • P2P peer-to-peer
  • VPN virtual private networks
  • CDN content delivery network
  • the network is an overlay network comprising a set of reflector (proxy) network elements (e.g. routers).
  • proxy reflector
  • the overlay network may overlay any known network, for example, the public internet.
  • the protocols described herein are applicable as a tunneling protocol, allowing tunneling of proprietary data over the public internet or other network.
  • the transmissions described herein may be accomplished by any appropriate protocol known in the art. Any unidirectional video protocol can be used for the transmission of the packets, for example, protocols based on User Datagram Protocol (UDP) can be used, for example, real-time protocol (RTP), for the transmission of real-time data.
  • UDP User Datagram Protocol
  • RTP real-time protocol
  • the data transmissions may be divided among multiple UDP sessions and transmitted over multiple UDP ports, as known in the art, to further improve the multipath reliability of the transmission.
  • Transmitters and Receivers are carried out by a transmitter and one or more receivers.
  • the network protocols of the invention are carried out by a transmitter and one or more receivers.
  • the various implementations disclosed herein will be described with respect to a unicast transmission having one transmitter and one receiver, for example, "a transmitter transmitting to the receiver.”
  • the various inventions described herein may be applied in the context of a multicast or broadcast system, wherein one (or more) transmitter sends data to a plurality of receivers, for example tens, hundreds, thousands, or even millions of receivers.
  • transmitter will refer to a source device or system from which data is transferred to the receiver, i.e. the device or system that is providing data, such as video data, to the receiver.
  • receiver will refer to the recipient device or system which receives the data.
  • the terms are fixed with respect to the identity of the devices and are not context sensitive, i.e. the device referred to as the transmitter will be referred to as such even as it acts as a receiver, receiving packets (such as NACKs) transmitted from the device referred to as the receiver.
  • the transmitter also referred to as the transmitting node, as used herein, is a device (or combination of devices) which transmits the data stream to its destination
  • the receiver also referred to as the receiving node, is a device or combination of devices which receives the transmitted data.
  • the transmitter and receiver may comprise servers or network devices of any kind, including streaming media servers, video broadcast routers, video encoders or high end IP surveillance cameras.
  • the receiver comprises a consumer receiving device such as a personal computer, mobile device, or an IP consumer Set Top Boxes, for example, with HDMI or other interface for TVs.
  • the network may comprise the public internet, having a plurality of nodes intermediate between the transmitter and the receiver.
  • the intermediate nodes do not perform any storage, processing, or logical operations on the data packets beyond ascertaining the packet' s destination and re-transmitting the packet to the next node in the network, as dictated by the selected networking protocol.
  • the network comprises an overlay network, for example a reflector (proxy) network. If the packets are being routed between nodes of an overlay network, for example a reflector network, control signals can be included in the packet. In this way, the data can be tunneled via the underlying public network.
  • an overlay network for example a reflector (proxy) network.
  • ARQ Error Correction Protocol In the various implementations of the invention, an error correction protocol is implemented, wherein the error correction protocol comprises an Automatic Repeat reQuest ("ARQ") error control protocol.
  • ARQ comprises any protocol implemented on the transmitter and the receiver by which a feedback loop between the receiver and transmitter is used to identify and direct the re-transmission of missing packets.
  • the scope of the invention encompasses any ARQ protocol, wherein the receiving node transmits acknowledgement messages ("ACKs") identifying properly received packets, and/or negative acknowledgement messages ("NACKs") identifying missing or damaged packets.
  • ACKs acknowledgement messages
  • NACKs negative acknowledgement messages
  • a single NACK specifies multiple missing packets to be retransmitted.
  • Various ARQ protocols include Stop-and-wait ARQ, Go-Back-N ARQ, Selective Repeat ARQ, and Delay Constrained ARQ.
  • the ARQ protocol implemented by the transmitter and receiver will comprise a Selective Repeat ARQ protocol.
  • a Selective Repeat ARQ protocol For example, delay constrained Selective Repeat ARQ, and variations thereof may be used.
  • selective repeat ARQ only NACKs are generated.
  • forward transmissions In the transmission of data under an ARQ error correction protocol, any number of communications will be made by the transmitter to the receiver. Such transmissions from the transmitter to the receiver will be referred to herein as “forward transmissions” or transmissions in the "forward direction.” Examples of forward transmissions include the transmission of packets from the transmitter to the receiver, including primary transmissions (packets being sent for the first time) and secondary transmissions (subsequent retransmission of packets), for example second, third, fourth, etc. retransmissions.
  • any number of communications will be made by the receiver to the transmitter.
  • Such transmissions between the receiver and the transmitter will be referred to herein as “backward transmissions,” or transmissions in the “backward direction.”
  • backward transmissions include ACKs, which are messages that identify packets or series of packets which have been successfully received at the receiver.
  • backward transmissions include NACKs, which comprise messages that identify packets or series of packets which have not been successfully received.
  • the ACKs and NACKs comprise packets which comprise address data for the destination (the transmitter) and a payload, encoded in FEC as described below, which comprises data that identifies packets (received in the case of ACKs and not received in the case of NACKs.
  • FEC FEC
  • the ARQ protocol is capable of identifying out-of-order packets
  • one or more NACKs identifying the unfulfilled packets in the series will be sent to the transmitter.
  • the receiver will reject the packet and a NACK identifying the missing packet is sent to the transmitter.
  • NACKs act as packet re-transmission requests. For a given packet or series of packets identified as missing by the system at the receiving end, NACKs will be sent at regular intervals until the packets are received, or a timeout or limit on the number of NACKs is reached.
  • Forward Error Correction Various forward and backward transmissions are made in the transmission of data under an ARQ scheme. As will be discussed below, certain of these forward and backwards transmissions may be encoded using a Forward Error Correction scheme.
  • FEC FEC
  • data is formatted in a redundant manner that allows the receiver to detect errors in the data and to correct these errors without requiring a retransmission of the packet.
  • Any number or algorithms and data formatting methods known in the art may be utilized in the selected FEC scheme.
  • FEC schemes may include systemic and non-systemic methods, interleaving methods, block codes, convolutional codes.
  • decoding of the data at the receiving node may employ any number of methods, including hard decision and soft decision algorithms.
  • the FEC schemes utilized in the practice of the invention may encompass any FEC coding and decoding schemes, including Selective Rate FEC schemes, RF5109 (ULP), RFC6015 (1-d interleaved), parity, 1- or 2-d interleaved XOR, Reed- Solomon, techniques, Bose-Chaudhuri-Hocquenghem codes, and others known in the art.
  • Selective Rate FEC schemes including Selective Rate FEC schemes, RF5109 (ULP), RFC6015 (1-d interleaved), parity, 1- or 2-d interleaved XOR, Reed- Solomon, techniques, Bose-Chaudhuri-Hocquenghem codes, and others known in the art.
  • “Stronger” FEC coding rates comprise higher proportions of redundant data. Strong FEC coding improves the odds of salvaging partially corrupted packets because the larger amounts of redundant data provides a means of reconstructing the original data. However, the transmission of larger amounts of redundant data will use up bandwidth and reduce throughput. Accordingly, “lighter,” lower coding rate FEC can be employed, which reduces bandwidth and increases throughput, although having less error control capability.
  • Coding rate can be expressed as a proportion, for example 7/8, which indicates that 7 of 8 bits in the packet are the raw data and 1 of 8 bits in the packet comprise redundant data.
  • exemplary overhead values include 1/2, 2/3, 3/4, 5/6, 7/8, and 15/16.
  • the receiving device Upon receipt of an FEC encoded packet, the receiving device will decode the packet and perform processing steps.
  • One such processing step in the decoding process is the detection of errors. Data is checked, by methods known in the art appropriate to the selected FEC coding scheme, to determine if the data is damaged, and if errors are detected, processing steps are performed to correct the errors using the redundant data within the FEC encoded packet.
  • the various embodiments of the invention encompass the combined use of FEC and ARQ for error control and data re-fulfillment.
  • the invention advantageously takes advantage of ARQ's error monitoring capabilities to efficiently identify missing data.
  • FEC is used to efficiently request and/or re-transmit missing packets, enabling the rapid completion of missing data.
  • the novelty of the invention lies in the use of FEC in combination with ARQ. FEC may be applied to certain forward and/or backward transmissions in an ARQ error control protocol.
  • the scope of the invention encompasses the use of FEC in backwards transmissions from the receiver to the transmitter.
  • the invention encompasses the use of an ARQ error correction protocol wherein all transmissions from the receiver to the transmitter comprise FEC-encoded packets.
  • NACK messages are encoded using FEC encoding.
  • NACKS In an ARQ protocol such as selective repeat ARQ, wherein NACKS are sent, the loss of NACKs will cause delays. If a NACK is not properly received by the transmitter, delay is introduced because the system must wait until the NACK is re-transmitted by the receiver and successfully received by the transmitter. Accordingly, the use of FEC in encoding the NACK packets will decrease latency by ensuring that a higher percentage of NACKs are properly received, speeding the re-transmission and fulfillment of missing packets.
  • the invention encompasses an improvement to the prior art ARQ protocols, wherein the "backwards" communications made between the receiver and transmitter that underlie the operations of such ARQ protocols are FEC encoded.
  • the "backwards" communications made between the receiver and transmitter that underlie the operations of such ARQ protocols are FEC encoded.
  • FEC encoding in these backwards transmissions greatly increases the probability of their being effectively received and acted upon.
  • the effect of this improvement is to reduce the number of necessary re-transmission requests, speeding the fulfillment of missing packets and improving the efficiency of the ARQ protocol to which the methods of the invention are applied.
  • FEC may further be applied to the transmission of packets in the forward direction.
  • FEC is applied to the primary transmission of packets to increase the percentage of successfully received packets.
  • FEC is applied to the secondary and higher order (e.g. tertiary, quaternary) retransmissions.
  • FEC is applied to both the primary and any secondary or higher order re-transmissions. The use of FEC in encoding data packets sent by the transmitter increases the likelihood of successful packet delivery and reduces the number of necessary re-transmission requests and re-transmissions.
  • the protocol can be described as a novel "two-way FEC" protocol, in contrast to the typical use of FEC schemes in only the forward direction.
  • the advantages of ARQ error detection are enhanced by the power of FEC to improve the success rate for the communications back and forth between the transmitter and receiver. The result is faster packet fulfillment and decreased latency.
  • FEC may be applied at any desired coding rate, for example at a rate from 1/2 to 15/16, for example, at 1/2, 2/3, 3/4, 5/6, 7/8, or 15/16, or intermediate values thereto.
  • the FEC coding rate may comprise a relatively lighter FEC, for example, at a coding rate in the range of 1/2 to 15/16.
  • FEC with a high coding rate e.g. 7/8-15/16 is applied in the primary transmission.
  • This lighter FEC implementation provides an optimized balance between error rate and bandwidth, with sufficient
  • the FEC coding rate may comprise a stronger FEC, for example, with coding rates in the range of 1/2. This stronger application of FEC provides optimal insurance that the communications made from the receiver to the transmitter will arrive in a usable form.
  • Fig. 1 An overview of one implementation of the general method of the invention is presented in Fig. 1.
  • data from a source is input (101) to a transmitter (102), such as a streaming media server.
  • the streaming media server packetizes the data and transmits the packets (103) over the internet or other network (104) to a receiver.
  • the packets are also sent to a buffer (109, connection with server not shown), and these freshly transmitted packets (110) are stored.
  • the receiver (hardware not shown) comprises a buffer (105) wherein received packets are stored (106). Missing packets (107) are identified by the receiver and a request for retransmission (108) of the missing packets is generated and sent over the network (104) to the transmitter.
  • This request is encoded in a forward error correction format, which greatly improves the probability of successful transmission to the transmitter.
  • the transmitter retrieves the requested, previously transmitted packets (111) from a buffer (109) and re-transmits (112) them over the network (104) to the receiver.
  • the receiver's buffer (105) reaches timeout for the window in question, the packets are output (113).
  • Dynamic Tuning An additional novel feature of the invention is the use of automatic tuning of protocol parameters in response to the changing conditions of the underlying IP transit network.
  • the various methods described above can be supplemented with automatic tuning steps, wherein the status of the network is assessed and parameters of the protocol are adjusted in response to maintain efficient data transfer to the receiving node.
  • the assessment of current network conditions is performed using tools known in the art for determining network status. Test packets, or pings, as known in the art, may be sent between the transmitter and receiver (or by other relevant devices) in both directions, including round-trip transmissions. For clarity, these network diagnostic pings are not subject to the FEC encoding utilized in the data transfer methods of the invention.
  • packet travel time may be assessed at regular intervals, for example 10 to 100 times a second.
  • Processors at the sending and/or receiving nodes will then analyze sampled results, for example, over a timespan ranging from one minute to ten minutes or more.
  • Current network conditions may encompass any measure of network bandwidth, latency, or other factors, including packet travel time between the nodes, packet drop error rates, and bit error rates, or any other parameter of network flow.
  • the parameters of the protocol of the invention may be adjusted.
  • a first parameter that may be dynamically adjusted is the receiver's binary packet buffering time (latency). For example, the lowest buffering time available at the receiver within a defined timeframe may be determined, to maintain the buffer within a selected range of fulfillment. For example, in one implementation, the buffer size may be adjusted to maintain the buffer in at least 50% fulfillment at all times.
  • a second parameter that may be dynamically adjusted in response to network conditions is the weight of FEC coding, as applied to either/or forward and backward transmissions. For example, based on available latency, the maximum number of packet retransmission attempts before timeout may be calculated, for example as the receiver endpoint binary packet buffering time divided by the average packet trip time.
  • the FEC coding should be adjusted to give a constant protection level, keeping retransmissions as low as possible, thus the lower the latency / buffer, the higher the degree of FEC coding.
  • a third parameter that may be dynamically adjusted in response to network conditions includes the ARQ timing parameters. These may be adjusted to respond to values such as the average packet transit time, for example, one-way trip time from the transmitter to the receiver (as in packet transmission) or round-trip time from the receiver to the transmitter and back (as in re-transmission request and re-transmission).
  • the minimum packet request timeout value may be set, i.e. the minimum time that the receiver will wait for a packet before sending a NACK retransmission request. In this case, the receiver "knows" not to request packets before they have time to arrive.
  • the maximum packet request timeout value may be set, i.e. the maximum time that the receiver will wait for a packet before sending a NACK retransmission request. In this case, the receiver "knows" not to wait for packets that exceed this time as they are unlikely to arrive. Accordingly, packet timeout value can be set at a value between the minimum and maximum times, by selected preferences.
  • the minimum packet request repetition timeout value may be set, i.e. the minimum time that the receiver will wait for a packet following a NACK retransmission request before sending another NACK retransmission request. In this case, the receiver "knows" not to re-request packets before the previous re-transmission request has had time to be fulfilled.
  • the maximum packet request repetition timeout value may be set, i.e. the maximum amount of time that the receiver will wait for a packet following a NACK retransmission request before sending another NACK retransmission request. In this case, the receiver "knows" not to further wait for re-requested packets that exceed this time value as they are unlikely to arrive. Accordingly, packet request repetition timeout value can be set at a value between the minimum and maximum times, by selected preferences.
  • Another ARQ timing factor that may be adjusted is the packet request give up timeout. For example, based on round-trip time from the receiver to the transmitter and back and the buffer size (in time), the maximum number of re-transmission requests that can be made within the receiver buffer time window can be calculated. Packet requests give up timeout can be set as the time at which the last such re-transmission request is timed out.
  • Hybrid Transmission Systems may be applied to improve hybrid FEC-ARQ protocols known in the art.
  • the methods of the invention are applied only when the current network conditions are sub-optimal.
  • the systems of the invention may be configured to operate without ARQ when there are no network traffic issues.
  • the system may automatically switch to operating in an ARQ mode that employs the improved methods of the invention.
  • Implementations of the Invention comprise systems, methods, computer programs for the implementation of such methods, hardware, and other elements of a data transit protocol. It will be understood by one of skill in the art that the inventions described herein encompass overlapping subject matter comprising systems, methods, computer programs, computer program products, and hardware configured to perform the operations described herein. It will be understood by one of skill in the art that the invention is practiced in any suitable network environment, broadly encompassing any type of computer, processor, router, broadcast server, reflector router, computer network, or combinations thereof and that practice of the invention is not limited to any single configuration of hardware, processors, operating systems, or programming languages.
  • the scope of the invention encompasses a method for the transmission of data, the data being transmitted by the transmitter to the receiver over a network: wherein, at the transmitter, the data is formatted as a series of packets, wherein the packets comprise data from the data source, packet sequence identifiers, and addressing data; wherein the transmitter and receiver comprise devices capable of sending and receiving data packets over and from the network; wherein the transmitter and receiver are configured to perform an ARQ protocol wherein ACK and/or NACK messages are transmitted from the receiver to the transmitter to facilitate the fulfillment of missing packets and wherein missing packets are re-transmitted by the transmitter to the receiver; and wherein the ACK and/or NACK messages transmitted from the receiver to the transmitter are encoded in a forward error correction coding scheme.
  • the general method of the invention may be implemented in any number of ways, as described herein, including, for example: wherein the data may be voice, audio, or video data; wherein if the data is video data it may comprise MPEG2-TS video data, HD video data, SD video data, or 4K-UHD resolution video data; wherein the data transmitted from the transmitter to the receiver may be encrypted, including by encryption standards AES 128, AES 192, or AES 256; wherein the transmitter and/or receiver may comprise a streaming media server, video broadcast router, video encoder, or IP surveillance camera; wherein the receiver may comprise a video broadcast router, a personal computer, a mobile device, or an IP consumer Set Top Box; wherein the network may comprise the public internet, a wired network, a wireless network, a hybrid network comprising both wired and wireless links, a public switched telephone network, a wireless data network, or a proprietary network, including a such a cable network; wherein the network may comprise an overlay network, including a reflector network, peer-to-peer
  • the method is carried out as follows:
  • a method for the transmission of data the data being transmitted by a transmitter to a receiver over a network
  • the ARQ protocol comprises a protocol wherein only NACKs are utilized and wherein the method comprises the steps of:
  • any received packets are decoded, damaged packets are identified and discarded, non-damaged packets are buffered in a packet buffer, and the identification of missing packets is carried out continuously or at periodic intervals;
  • a packet retransmission request identifying the missing packet is generated, addressed to the transmitter, and transmitted to the network, wherein such packet retransmission request is encoded by a forward error correction coding method;
  • steps (b), (c), and (d) are repeated until all packets within the series have been successfully received at the receiver, a timeout limit is reached, and
  • Output of buffered packets from the receiver is made to the next (e.g. higher order) layer of the system, where further processing may take place, for example, error correction, jitter control, and other corrective processes to account for any missing packets in the window.
  • the output data may be rendered, concatenated, displayed, played, buffered, or used to reconstruct the original source data , depending on the desired application.
  • FIG. 1 The general method described above is depicted in the diagrams of Fig. 1 and Fig. 2. With reference to Fig. 1, this diagram provides an overview of the general method of the invention.
  • FIG. 2 the operations of an exemplary implementation of the invention are depicted. In the depiction of this exemplary method, operations at the transmitter (201) and receiver (202) sides are depicted. At the transmitter side, the input of data from a source is made (203) to the transmitter (201), for example, video data from a video data source.
  • the source data is packetized (204) as a series of sequential packets, preferably FEC encoded packets.
  • packets are serially transmitted (206) to the network and at the same time, are buffered (205) and stored if needed for retransmission.
  • the network transmission (208) may be unsuccessful (packet loss) or may be successfully received and decoded (209) at the receiver . If packets are determined to be corrupted, as may be achieved by any error detection mechanisms known in the art, the damaged packets are rejected (210) and not buffered. Undamaged packets are buffered (211), for example in a dynamic circular buffer.
  • the receiver assesses which packets are missing (212), using ARQ methodologies known in the art. Missing packets assessment is repeated continuously or at regular intervals. If no missing packets of the window are detected (213), the buffered series of packets is output to the next layer (217).
  • the receiver will generate a NACK specifying packets to be retransmitted (218), and will transmit such NACK to the network to be delivered to the transmitter.
  • NACK will be encoded using FEC to improve the probability of successful transmission.
  • the retransmission request is received by the transmitter, it is decoded and the specified packets are retrieved from the buffer (205) and retransmitted (220) to the receiver, where the process is repeated until successful fulfillment of all packets in the window or timeout/retransmission limits are reached.
  • the process depicted in Fig. 2 may be auto-tuned according to current network conditions. For example, network status may be determined and appropriate adjustments may be made to the packetization scheme at the transmitter (222), the buffer size of the receiver (223), the determination of timeouts or the formatting of NACKs (224).
  • the inventions described herein comprise systems, wherein specific operations are carried out using a combination of hardware elements and non-transitory computer-readable storage medium having computer-readable program instructions stored therein, i.e. software, which directs he operations of the hardware elements.
  • the system of the invention comprises: a transmitter and a receiver, wherein the transmitter and receiver are devices capable of sending and receiving data packets over and from a network and are configured to perform an ARQ protocol wherein ACK and/or NACK messages are transmitted from the receiver to the transmitter to identify fulfilled and/or missing packets and wherein the transmitter will re-transmit packets determined to be unfulfilled or missing; and wherein the transmitter and receiver are configured such that ACK and/or NACK messages transmitted from the receiver to the transmitter are formatted in an FEC coding scheme that can be decoded by the transmitter.
  • the invention may comprise a system element.
  • the invention comprises a routing device configured to act as a transmitter in the system of the invention.
  • the invention comprises a routing device configured to act as the receiver in the system of the invention.
  • the inventions described herein comprise non-transitory memory or data storage devices which store the specific computer-readable program instructions, i.e. software, which enables practice of the operations described herein by suitable hardware devices.
  • the invention encompasses a protocol, the protocol being embedded into and carried out by various network elements and hardware devices.
  • the protocol may be embedded into physical IP routing devices, for example devices with Ethernet interfaces.
  • the protocol may be embedded into "virtualized” or “software defined” (SDN) cloud based routing environments (private or public Cloud), such as VMware or Amazon AWS.
  • the protocol may be embedded into "baseband” video routing devices like SDI Interface Routers or ASI Interface Routers.
  • the protocol may be embedded into digital video encoders or professional integrated Digital Receivers Decoders (IRDs).
  • the protocol may further be embedded into hardware elements of a video camera, for example, surveillance IP cameras of any kind.
  • the protocol may further be embedded into consumer receiving devices like personal computers, mobile devices, IP consumer Set Top Boxes, for example, with HDMI or other interface for TVs.
  • Example 1 Description of an Exemplary Embodiment. Reference is made to Fig. 1, which depicts the general process of the exemplary embodiment, described in greater detail here.
  • MPEG-TS video data transmitted from a source over UDP/IP from a unicast or multicast LAN input is received at the transmitter.
  • the data is packetized and formatted with FEC error correction coding.
  • the level of FEC coding can be dynamically adjusted, depending on network conditions.
  • the packets are stored in a dynamically adjusted size circular buffer.
  • the packets are also transmitted over a global IP transit network, such as the internet. The transmission may be single or may be divided into multiple sessions.
  • packets may be lost, arrive at the destination after delay, or may arrive out of order, which would disrupt the MPEG-TS container integrity and video quality, if not corrected.
  • error detection is applied to determine packet integrity. Damaged packets are rejected. Good packets are stored in a circular buffer.
  • a Selective Repeat Delay Constrained ARQ protocol running at the receiver recognizes missing packets.
  • NACKs which identify the missing packets are generated by the receiver, and these messages are encoded using FEC.
  • the FEC coding may be dynamically determined, with higher coding rates (more redundancy) applied if network conditions are poor.
  • the NACKs are transmitted over the network to the transmitter. The use of FEC increases the probability of successful transmission of functional NACKs to the transmitter.
  • the transmitter retrieves the requested packets from the buffer and re-transmits them over the network to the receiver. If received and if intact, the packets are stored in the buffer. Re-transmitted packets that were not successfully fulfilled will be identified as missing packets by ARQ protocols monitoring for missing packets, and additional re-transmission requests will be generated for such packets.
  • the buffer window When the buffer window has reached its limit, the received packets within the window are output. With successful fulfillment of missing packets, the output will comprise a high quality fully reconstructed MPEG-TS stream which may be output to a LAN for multicast or unicast.
  • the data may be protected with strong encryption such as AES 256.
  • the data transmission may be relayed over a proprietary overlay network of reflector proxy routers, imparting multipath capabilities.
  • This method ensures the most efficient video feed reconstruction mechanism with both with very low overhead (SR-ARQ + Selectable Rate FEC over UDP/IP on the main data feed) and very low latency.
  • the novel use of FEC in requesting re-transmission of packets minimizes the occurrence of lost NACKs, avoiding additional Re-Requests and Re- Transmissions that would increase the buffer size and thus the latency.
  • EXAMPLE 2 Performance Metrics. Using standard professional video transmission protocols and buffering technologies (i.e. one way standard FEC with non intelligent "two way-checked" buffer on public networks), average latency can be about 30X of ping travel time (nearly 5 sees minimum buffer at 150 msec of ping travel time) and bandwidth overhead of about 35% is typical, with no video container integrity guaranteed stability at the receiving node ("one way" buffer does not guarantee packet integrity).
  • Example 4 Dynamic FEC and ARQ Parameters.
  • the following illustrates implementation of dynamic adjustment of FEC and ARQ parameters in response to current network conditions.
  • the buffer size at the receiving node is set at 200 ms and average packet round trip travel time is 40 ms. In this case, five retransmissions may be achieved during the buffer window.
  • the system may be configured to implement no FEC for forward transmissions, or a very light level of FEC, for example, 15/16 (increasing overhead by about 7%).
  • system may be configured to increase the FEC level to 3/4 or even 2/3, resulting in a 30-50% increase in overhead, but improving the probability of successful transmission.
  • ARQ parameters may be adjusted in response to the increased travel time.
  • the packet request timeout value, the packet request repetition timeout value, and the packet give up timeout value may be increased to a value somewhat higher than the average travel time, in order to reduce or eliminate packet duplication, a situation that occurs when a previously received packet is received a second time, the result of requesting a retransmission of the packet before the previous retransmission attempt has had sufficient time to succeed.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Detection And Prevention Of Errors In Transmission (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

L'invention concerne le transport de données entre un émetteur et un récepteur, en particulier pour le transport de vidéo de haute qualité. Les procédés et systèmes de l'invention comprennent la nouvelle application de procédés de correction d'erreurs sans voie de retour (FEC) pour améliorer l'efficacité des protocoles de demande de répétition automatique (ARQ). Le protocole ARQ repose sur la communication du récepteur vers l'émetteur afin de répondre aux besoins des paquets manquants ou endommagés. Selon les nouveaux procédés de l'invention, ces communications du récepteur vers l'émetteur sont codées avec des procédés FEC, ce qui améliore considérablement l'efficacité du protocole ARQ et réduit le nombre de demandes de retransmission et de retransmissions, réduisant significativement la latence. Lorsqu'il est combiné avec un codage FEC des données vidéo dans le sens direct, le nouveau procédé FEC bidirectionnel est très efficace, permettant le transit de vidéo de haute qualité sur des réseaux publics avec perte.
PCT/GB2017/053772 2016-12-16 2017-12-15 Protocole de transport de vidéo résistant aux erreurs et à faible retard sur un transit ip public WO2018109500A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662435306P 2016-12-16 2016-12-16
US62/435,306 2016-12-16

Publications (1)

Publication Number Publication Date
WO2018109500A1 true WO2018109500A1 (fr) 2018-06-21

Family

ID=60812094

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2017/053772 WO2018109500A1 (fr) 2016-12-16 2017-12-15 Protocole de transport de vidéo résistant aux erreurs et à faible retard sur un transit ip public

Country Status (1)

Country Link
WO (1) WO2018109500A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020072603A1 (fr) * 2018-10-02 2020-04-09 Google Llc Connecteur de flux en direct
CN115189810A (zh) * 2022-07-07 2022-10-14 福州大学 一种面向低时延实时视频fec编码传输控制方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6104712A (en) * 1999-02-22 2000-08-15 Robert; Bruno G. Wireless communication network including plural migratory access nodes
US20030206524A1 (en) * 2002-05-06 2003-11-06 Bibhu Mohanty Method and apparatus for augmenting physical layer ARQ in a wireless data communication system
US20110243066A1 (en) * 2009-10-01 2011-10-06 Interdigital Patent Holdings, Inc. Uplink Control Data Transmission
US20150223237A1 (en) * 2012-07-02 2015-08-06 Qualcomm Incorporated Methods and apparatus for enabling fast early termination of voice frames on the uplink

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6104712A (en) * 1999-02-22 2000-08-15 Robert; Bruno G. Wireless communication network including plural migratory access nodes
US20030206524A1 (en) * 2002-05-06 2003-11-06 Bibhu Mohanty Method and apparatus for augmenting physical layer ARQ in a wireless data communication system
US20110243066A1 (en) * 2009-10-01 2011-10-06 Interdigital Patent Holdings, Inc. Uplink Control Data Transmission
US20150223237A1 (en) * 2012-07-02 2015-08-06 Qualcomm Incorporated Methods and apparatus for enabling fast early termination of voice frames on the uplink

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020072603A1 (fr) * 2018-10-02 2020-04-09 Google Llc Connecteur de flux en direct
US10686861B2 (en) 2018-10-02 2020-06-16 Google Llc Live stream connector
KR20210006983A (ko) * 2018-10-02 2021-01-19 구글 엘엘씨 라이브 스트림 커넥터
CN112313918A (zh) * 2018-10-02 2021-02-02 谷歌有限责任公司 直播流连接器
KR102562258B1 (ko) * 2018-10-02 2023-07-31 구글 엘엘씨 라이브 스트림 커넥터
CN112313918B (zh) * 2018-10-02 2023-09-05 谷歌有限责任公司 直播流连接器
CN115189810A (zh) * 2022-07-07 2022-10-14 福州大学 一种面向低时延实时视频fec编码传输控制方法
CN115189810B (zh) * 2022-07-07 2024-04-16 福州大学 一种面向低时延实时视频fec编码传输控制方法

Similar Documents

Publication Publication Date Title
US11489621B2 (en) Forward error correction for streaming data
JP4002183B2 (ja) パケット・チャネルを介するマルチメディア通信のための方法
KR101644215B1 (ko) 신뢰성 있는 데이터 통신을 위한 네트워크 추상화 계층을 파싱하는 방법 및 장치
CN101861709B (zh) 用于具有合并的自动重复请求的自适应前向纠错以在无线局域网中进行可靠多播的方法和装置
US7254765B2 (en) Method and devices for error tolerant data transmission, wherein retransmission of erroneous data is performed up to the point where the remaining number of errors is acceptable
CN102687448B (zh) 网络中可靠实时数据流传输的高效应用层自动重复请求重发的方法
US8091011B2 (en) Method and system for dynamically adjusting forward error correction (FEC) rate to adapt for time varying network impairments in video streaming applications over IP networks
US9537611B2 (en) Method and apparatus for improving the performance of TCP and other network protocols in a communications network using proxy servers
US8819532B2 (en) Methods and devices for transmitting a data stream and corresponding computer readable media
US7584404B2 (en) Method and apparatus for multimedia communication over packet channels
US20100214970A1 (en) Method and system for transmitting data packets from a source to multiple receivers via a network
CN101878612A (zh) 用于数据网络中的数据传输的方法和系统
US11381339B2 (en) System and technique for generating, transmitting and receiving network coded (NC) quick UDP internet connections (QUIC) packets
EP1931082A1 (fr) Procédé et appareil de multidiffusion de données
KR102290779B1 (ko) 멀티미디어 데이터를 송수신하는 방법 및 장치
WO2018109500A1 (fr) Protocole de transport de vidéo résistant aux erreurs et à faible retard sur un transit ip public
US9426260B2 (en) Real time communication over TCP
CN106100803A (zh) 用于作出重发决定的方法和装置
CH719252B1 (it) Metodo di trasmissione di dati video in una rete.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17821715

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17821715

Country of ref document: EP

Kind code of ref document: A1