JP7073353B2 - Systems and methods to enable communication associated with digital media distribution - Google Patents

Description

This disclosure relates to the field of digital media distribution.

Digital media playback capabilities include digital TVs, including so-called "smart" TVs, set-top boxes, laptops or desktop computers, tablet computers, digital recording devices, digital media players, video gaming devices, so-called "smart" phones. It can be incorporated into a wide range of devices, including telephones, dedicated video streaming devices, and more. Digital media content (eg, video and audio programs) can originate from multiple sources, such as, for example, wireless television providers, satellite television providers, cable television providers, and online media service providers including so-called streaming service providers. Digital media content may be delivered over a packet exchange network that includes a bidirectional network such as an Internet Protocol (IP) network and a unidirectional network such as a digital broadcast network.

Digital media content may be transmitted from the source to the receiver device (eg, a digital television or smartphone) according to a content delivery protocol model. The content delivery protocol mode can be based on one or more transmission standards. Examples of transmission standards include Digital Video Broadcasting (DVB) standards, Integrated Services Digital Broadcasting (ISDB) standards, and, for example, a set of ATSC 3.0 standards currently under development. Examples include standards created by the Advanced Television Systems Committee (ATSC), including. Current techniques for transmitting digital media content from the source to the receiver device may not be ideal.

According to one embodiment of the present disclosure, the method of signaling the information associated with the media presentation is to signal a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type and a JavaScript Object NOTation coding parameter. As including signaling one or more given arguments corresponding to the message type.

FIG. 1 is a conceptual diagram showing an example of a content delivery protocol model according to one or more techniques of the present disclosure. FIG. 2 is a block diagram showing an example of a system capable of implementing one or more techniques of the present disclosure. FIG. 3 is a block diagram showing an example of an arithmetic device capable of performing one or more techniques of the present disclosure. FIG. 4 is a block diagram showing an example of a media distribution engine capable of implementing one or more of the techniques of the present disclosure. FIG. 5 is a conceptual diagram showing an example of a communication flow according to one or more techniques of the present disclosure. FIG. 6A is a computer programming list showing an exemplary schema for each media presentation document request message. FIG. 6B is a computer programming list showing an exemplary schema for each media presentation document request message. FIG. 7A is a computer programming list showing an exemplary schema of each media presentation document request response message. FIG. 7B is a computer programming list showing an exemplary schema of each media presentation document request response message. FIG. 8A is a computer programming list showing an exemplary schema of each segment request message. FIG. 8B is a computer programming list showing an exemplary schema of each segment request message. FIG. 9A is a computer programming list showing an exemplary schema of each segment request response message. FIG. 9B is a computer programming list showing an exemplary schema of each segment request response message. FIG. 9C is a computer programming list showing an exemplary schema of each segment request response message. FIG. 9D is a computer programming list showing an exemplary schema of each segment request response message. FIG. 10A is a computer programming list showing an exemplary schema of each cancellation request message. FIG. 10B is a computer programming list showing an exemplary schema of each cancellation request message. FIG. 11A is a computer programming list showing an exemplary schema of each resource change notification message. FIG. 11B is a computer programming list showing an exemplary schema of each resource change notification message.

In general, the disclosure describes techniques that enable communication associated with the delivery of digital media content. It should be noted that the digital media content may be included as part of the audio visual service or, in some embodiments, as a dedicated audio service. Although some embodiments describe the techniques of the present disclosure for a particular transmission standard and a particular digital media format, the techniques described herein are generally in various transmission standards and digital media formats. It should be noted that it may be applicable. For example, the techniques described herein are generally DVB standards, ISDB standards, ATSC standards, Digital Terrestrial Multimedia Media Broadcast (DTMB) standards, Digital Multimedia Broadcast (DMB) standards, HybridBandBandTV standards. Of the World Wide Web Consortium (W3C) standards, Universal Plug and Play (UPnP) standards, and Moving Pictures Expert Group (MPEG) standards. It may be applicable to any of the above. Furthermore, it should be noted that the inclusion of documents in this specification by reference is for illustration purposes only and should not be constructed to limit and / or create ambiguity with respect to the terms used herein. Should. For example, if one built-in reference results in a different definition of a term than another built-in reference, and / or if that term is used herein, that term broadly includes each and every definition. It should be interpreted in a way and / or in a way that includes each of the specific definitions in the alternative.

According to another embodiment of the present disclosure, a device signaling information associated with a media presentation signals a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type, and has a JavaScript Objectation Notation coding parameter. It comprises one or more processors configured to signal one or more given arguments corresponding to the message type.

According to another embodiment of the present disclosure, the device signaling the information associated with the media presentation is a means of signaling a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type and a JavaScript Object Notification encoding. As a parameter, it includes means for signaling one or more given arguments corresponding to the message type.

According to another embodiment of the present disclosure, a non-temporary computer-readable storage medium, when executed, causes one or more processors of the device to signal a frame header indicating a dynamic adaptive streaming over hypertext transfer protocol message type. , And stores in it an instruction to signal one or more given arguments corresponding to the message type as a Javascript Object Notation encoding parameter.

Details of one or more embodiments are described in the accompanying drawings and the following specification. Other features, objectives, and advantages will be apparent from the specification and drawings, as well as from the claims.

Computational devices and / or transmission systems may be based on a model that includes one or more abstraction layers, and the data in each abstraction layer is represented according to a particular structure, such as a packet structure, a modulation scheme, and the like. An example of a model that includes a defined abstraction layer is the so-called Open Systems Interconnection (OSI) model. The OSI model defines a 7-layer stack model that includes an application layer, a presentation layer, a session layer, a transport layer, a network layer, a data link layer, and a physical layer. It should be noted that the use of the terms upper and lower with respect to the description of layers in the stack model may be based on the top layer, the application layer, and the bottom layer, the physical layer. be. Further, in some cases, the term "layer 1" or "L1" can be used to refer to the physical layer, and the terms "layer 2" or "L2" can be used to refer to the link layer. The term "layer 3" or "L3" or "IP layer" can be used to refer to the network layer.

The physical layer can generally refer to the layer in which electrical signals form digital data. For example, the physical layer can refer to a layer that defines how a modulated radio frequency (RF) symbol forms a frame of digital data. A data link layer, sometimes also referred to as a link layer, can refer to an abstraction used before physical layer processing on the transmitting side and after receiving the physical layer on the receiving side. As used in the present invention, the link layer is used to transport data from the network layer to the physical layer on the transmitting side and to transport data from the physical layer to the network layer on the receiving side. It should be noted that the sender and receiver are logical roles, and a single device can act as both the sender of one instance and the receiver of another. Is. The link layer processes various types of data (eg, video files, audio files, or application files) encapsulated in a particular packet type (eg, Internet Protocol version 4 (IPv4) packets) by the physical layer. Can be abstracted into a single generic format for. The network layer can generally refer to the layer where logical addressing occurs. That is, the network layer can generally provide addressing information (eg, IP address), which allows data packets to be delivered to specific nodes (eg, computing devices) in the network. As used in the present invention, the term network layer can refer to a layer above the link layer and / or a layer having data of a structure that can be received for link layer processing. The transport layer can refer to a layer that enables so-called interprocess communication services. Each of the session layer, presentation layer, and application layer can define how data is delivered for use by the user application. Transmission standards, including transmission standards currently under development, can include a content delivery protocol model that defines the protocols supported for each layer, and can further define one or more specific layer implementations.

FIG. 1 shows an example of a content delivery protocol model according to the techniques described herein. The techniques described herein can be implemented in systems configured to operate under Content Delivery Protocol Model 100.
The content delivery protocol model 100 can enable the delivery of digital media content. Therefore, the content delivery protocol model 100 can be implemented in a system that allows a user to access digital media content from a media service provider (eg, a so-called streaming service). The term media service or media presentation can be used to refer to a collection of media components (eg, video components, audio components, and subtitle components) that are collectively presented to the user, and the components are multiple media. It can be of type, the service can be either continuous or intermittent, and the service can be a real-time service (eg, multimedia presentation for live events) or a non-real-time service (eg, video-on). It can be a demand service).

As shown in FIG. 1, the content delivery protocol model 100 includes a physical layer and a data link layer. As mentioned above, the physical layer can generally refer to a layer in which electrical signals form digital data, and the data link layer can include a general-purpose format for processing by the physical layer. In the embodiment shown in FIG. 1, in some examples, the physical layer is a physical layer frame structure containing a defined preamble and a data payload structure containing an RF channel or one logical structure within a portion of the RF channel. And can include. In the embodiment shown in FIG. 1, the data link layer may be a data structure used to abstract various specific packet types into a single general purpose format for processing by the physical layer. As mentioned above, the network layer can generally refer to the layer where logical abstraction occurs. Referring to FIG. 1, the content delivery protocol model 100 can utilize an IP addressing scheme, eg, IPv4 and / or IPv6 as a network layer. As mentioned above, the transport layer can generally refer to a layer that enables so-called interprocess communication services. As shown in FIG. 1, the content delivery protocol model 100 can utilize a transport control protocol (TCP) as a transport layer.

As mentioned above, the content delivery protocol model 100 can enable the delivery of digital media content. In the embodiment of FIG. 1, the application may include an application that allows the user to render digital media content. For example, the content delivery protocol model can include a video player application running on a mobile device or the like. In the embodiment shown in FIG. 1, the media codec includes a basic codec (ie, a multimedia encoder / decoder implementation) that allows an application to render a digital media content format (eg, MPEG format, etc.). be able to. In the embodiment shown in FIG. 1, the content delivery protocol model 100 utilizes Dynamic adaptive streaming over HTTP (DASH) to allow a logical client to receive digital media content from a logical server. To. ＤＡＳＨは、ＭＰＥＧ ＤＡＳＨ ＩＳＯ／ＩＥＣ：ＩＳＯ／ＩＥＣ ２３００９～１：２０１４，「Ｉｎｆｏｒｍａｔｉｏｎ ｔｅｃｈｎｏｌｏｇｙ＿Ｄｙｎａｍｉｃ ａｄａｐｔｉｖｅ ｓｔｒｅａｍｉｎｇ ｏｖｅｒ ＨＴＴＰ（ＤＡＳＨ）－Ｐａｒｔ１：Ｍｅｄｉａ ｐｒｅｓｅｎｔａｔｉｏｎ ｄｅｓｃｒｉｐｔｉｏｎ ａｎｄ ｓｅｇｍｅｎｔ ｆｏｒｍａｔｓ」，Ｉｎｔｅｒｎａｔｉｏｎａｌ Ｏｒｇａｎｉｚａｔｉｏｎ ｆｏｒ Ｓｔａｎｄａｒｄｉｚａｔｉｏｎ，２ｎｄ Ｅｄｉｔｉｏｎ，５／１５ It is described in the DASH-IF profile of / 2014 (hereinafter, “ISO / IEC 23009 to 1: 2014”), which is incorporated herein by reference. As shown in FIG. 1, the DASH media presentation can include a data segment, a moving image segment, and an audio segment. In some embodiments, the DASH media presentation is a linear service or part of a linear service (eg, a single TV program, or a continuous linear TV program over a period of time) defined by a service provider. Can correspond to the set). According to DASH, a Media Presentation Document (MPD) is a metadata fragment containing a formalized description of a profile of a DASH media presentation. Fragments can include a set of extensible markup language (XML) encoded metadata fragments. The MPD content provides a resource identifier for the segment and a context for the identified resource in the media presentation. The data structures and semantics of MPD fragments are described for ISO / IEC 23009-1: 2014. Furthermore, it should be noted that a draft version of ISO / IEC 23009-1 is currently being proposed. Thus, in the examples shown in FIG. 1, the MPD can include MPDs as described in ISO / IEC 23009 to 1: 2014, currently proposed MPDs, and / or combinations thereof.

In ISO / IEC 23009 to 1: 2014, the media presentation described in the MPD can include a sequence of one or more periods, where each period is one or more adaptation sets. Can be included. It should be noted that if the adaptation set contains multiple media content components, each media content component may be described individually. Each adaptation set can include one or more representations. In ISO / IEC 23009 to 1: 2014, each representation is provided as follows. (1) As a single segment, the subsegments are aligned across the representation of the adaptation set, and (2) as a sequence of segments, each segment is by a Universal Resource Locator (URL) generated by the template. The address can be specified. The properties of each media content component may be described by an adaptation set element and / or, for example, an element in an adaptation set that includes a content component element.

As shown in FIG. 1, DASH may operate on HTTP and WebSocket protocols. That is, the DASH media presentation is described in Internet Engineering Task Force (IETF) Request for Commission (RFC) 7540 Hyperext Transfer Protocol Version 2 (HTTP / 2) (HTTP / 2) (May 2015). Described in Task Force, RFC 6455 IETF: "The WebSocket Protocol" (December 2011), and the WebSocket API (Worldwide WebSocket (W3C) Recommendation Candidate, September 20, 2012). They may be delivered via full-duplex HTTP-enabled protocols, including WebSocket protocols, each of which is incorporated herein by reference.

The WebSocket protocol allows mutual communication between a logical client and a logical server (sometimes referred to as a remote host) through the establishment of bidirectional sockets. It should be noted that in some cases, the sides or endpoints of bidirectional sockets are sometimes referred to as terminals. A bidirectional socket based on the WebSocket protocol exchanges character data encoded in Universal Transformation Format-8 (UTF-8) between a logical client and a logical server via TCP. Includes bidirectional sockets that can. 2309-6: DASH with Server Push and WebSockets (ISO / IEC JTC1 / SC29 / WG11 / W16232, June 2016) draft international standard (hereinafter, "W16232") (which is incorporated herein by reference). Defines signaling and message formats for driving the delivery of MPEG-DASH media presentations using bidirectional sockets based on the WebSocket protocol.

In W16232, the transmission of a segment from the server to the client (referred to as push in W16232) is based on a push strategy, which defines how the segment can be transmitted from the server to the client. In W16232, the push directive is a request qualifier sent from the client to the server, which allows the client to express its expectation of the server's push strategy for processing the request. Further, in W16232, an acknowledgment (also called Push Ac) is a response qualifier sent from the server to the client, which allows the server to describe the push strategy used in processing the request. can. In W16232, push directives and push acknowledgments can be communicated by sending a message containing a DASH subprotocol frame header and one or more given arguments. The semantics of the DASH subprotocol frame header as provided in W16232 are provided in Table 1.

W16232 provides the following definitions for each of STREAM_ID, MSG_CODE, E, F, EXT_LENGTH, and Extension.
STREAM_ID: The identifier of the current stream, which allows multiplexing of multiple requests / responses over the same WebSocket connection. Responses to a particular request shall use the same STREAM_ID as that request. The appearance of a new STREAM_ID indicates that a new stream has started. Upon cancellation request, termination of stream, or receipt of error, the stream identified by the carried STREAM_ID shall be closed.

MSG_CODE: Indicates the MPEG-DASH message represented by this frame. The available message codes are defined in Table 2.

E: This field is an error flag. If this field is set, the receiver can interpret the message as an error. Additional information about the error may be available in the extension header.
F: Reserved EXT_LENGTH: Provides extended data having a length of 4 bytes preceding the application data, including padding.
Extension: The extension header is an additional information that applies to the request / response in accordance with the IETF RFC7158 [The Javascript Object Notification (JSON) Data linterchange Format, March 2013, which is incorporated herein by reference]. The field must be Javascript Object Notification (JSON) encoding. 0-byte padding can be added after the extension header to align to the 4-byte boundary. It is assumed that the content is encoded in UTF-8 format. The JSON encoding of the extension header shall consist of a single root level JSON object containing zero or more name / value pairs.

As shown in Table 2, each message type contains the given arguments shown in Tables 3-8. In each of Tables 3 to 8, Type indicates a data type, URI is a character string including a Uniform Resource Identifier (URI) defined by RFC 3896, and String is UTF-. An 8-character string, an integer, where Boolean is a true or false value. In each of Tables 3-8, the radix indicates the required number of instances of the parameter in the instance of the message, where 0 indicates that the signaling of the parameter value is optional.

Referring to Tables 3 and 4, W16232 provides the following formats for PushDirective parameters.

The format of PushDirective in ABNF [IETF RFC 5234, Augmented BNF [Backus-Naur Form] for Syntax Specifications: ABNF, January 2008] format is as follows:
PUSH_DIRECTIVE = PUSH_TYPE [OWS OWS QVALUE]
PUSH_TYPE = <PushType defined by [below]>
QVALUE = <qvalue described in RFC7231>
"OWS" is defined in RFC 7230 [IETF RFC 7230, Hypertext Transfer Protocol (HTTP / 1.1): Message Syntax and Routing, June 2014], section 32.3, and represents an optional blank. ..

When multiple push directives are applied to a request, the client can apply a quality value (“qvalue”) as described for use in content negotiation of RFC7231. The client can apply a higher quality value to a directive that it wants to take precedence over an alternative directive that has a lower quality value. Note that these values are hints to the server and do not mean that the server will always choose the strategy with the highest quality value.

Referring to Tables 5 and 6, W16232 provides PushAck with the following formats:
The format of PushAck in ABNF format is as follows:
PUSH_ACK = PUSH_TYPE
In the formula, PUSH_TYPE is defined by [below].
W16232 provides PushType with the following formats:
The format of PushType in ABNF format is as follows:
PUSH_TYPE = PUSH_TYPE_NAM [OWS]; "OWS PUSH_PARAMS] PUSH_TYPE_NAME = DQUATE <URN> DQUATE
PUSH_PARAMS = PUSH_PARAM * (OWS ";" OWS PUSH_PARAM)
PUSH PARAM = 1 * VCHAR
During the ceremony
-The "<URN>" syntax is defined in RFC 2141 [IETF RFC 2141, Uniform Resource Names (URN) Syntax, May 1997].

The messages defined for signaling push directives and push acknowledgments in W16232 may not be ideal.

FIG. 2 is a block diagram showing an example of a system capable of implementing one or more of the techniques described in the present disclosure. The system 200 may be configured to communicate data according to the techniques described herein. In the embodiment shown in FIG. 2, the system 200 has one or more computing devices 202A-202N, one or more content provider sites 204A-204N, a database 206, one or more media distribution engines 208A-208N, and Includes wide area network 210. The system 200 can include software modules. Software modules are stored in memory and can be executed by the processor. The system 200 can include one or more processors and a plurality of internal and / or external memory devices. Examples of memory devices are file servers, file transfer protocol (FTP) servers, network attached storage (NAS) devices, local disk drives, or any other type that can store data. Examples include devices or storage media. The storage medium can include a Blu-ray® disc, DVD, CD-ROM, magnetic disc, flash memory, or any other suitable digital storage medium. When the techniques described herein are partially implemented in software, the device stores software instructions in a suitable non-temporary computer-readable medium and uses one or more processors. And can execute instructions in hardware.

The system 200 is configured such that digital media content such as movies, live sporting events, and data and applications and related media presentations can be delivered to and accessed by a plurality of computing devices. It represents an example of a possible system. In the embodiment shown in FIG. 2, the computing devices 202A-202N can include any device equipped for wired and / or wireless communication, such as a television, set-top box, digital video recorder, desktop. , Laptops, or tablet computers, gaming consoles, such as mobile devices, including "smart" phones, mobile phones, and personal gaming devices. It should be noted that the system 200 is illustrated as having a separate site, but such illustration is for illustration purposes only and does not limit the system 200 to a particular physical architecture. The functionality of the system 200 and the sites contained therein can be implemented using any combination of hardware implementations, firmware implementations, and / or software implementations.

FIG. 3 is a block diagram showing an example of an arithmetic device capable of implementing one or more of the techniques of the present disclosure, wherein the arithmetic device 300 receives data from a communication network and allows a user to access multimedia content. It is an example of an arithmetic device that can be configured to be. The arithmetic device 300 includes a central processing unit (s) 302, a system memory 304, a system interface 312, a voice decoder 314, a voice output system 316, a moving image decoder 318, a display system 320, and an I / O device (s). 322, and network interface 324. Central processing unit (s) 302, system memory 304, system interface 312, audio decoder 314, audio output system 316, video decoder 318, display system 320, I / O device (s) 322, and network interface Each of the 324s may be interconnected (physically, communicatively, and / or operational) for inter-component communication, one or more microprocessors, digital signal processors (DSPs),. Any of a variety of suitable circuits such as application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), discrete logic, software, hardware, firmware, or a combination thereof. It can be implemented as a hardware. Although the arithmetic device 300 is illustrated as having a separate functional block, it should be noted that such an illustration is for illustration purposes only and does not limit the arithmetic device 300 to a particular hardware architecture. .. The function of the arithmetic device 300 can be realized by using any combination of hardware implementation, firmware implementation, and / or software implementation.

The CPU (s) 302 may be configured to perform functions and / or process instructions for execution in the arithmetic device 300. The CPU (s) 302 can include single-core and / or multi-core central processing units. The CPU (s) 302 may be capable of retrieving and processing instructions, codes, and / or data structures for performing one or more of the techniques described herein. Instructions may be stored on a computer-readable medium such as system memory 304. The system memory 304 can be described as a non-temporary or tangible computer-readable storage medium. In some embodiments, the system memory 304 can provide temporary and / or long-term storage. In some embodiments, the system memory 304 or a portion thereof may be described as a non-volatile memory, and in another embodiment, a portion of the system memory 304 may be described as a volatile memory. The system memory 304 may be configured to store information that may be used by the arithmetic device 300 during operation. The system memory 304 can be used to store program instructions to be executed by the CPU (s) 302, and the program running on the arithmetic device 300 temporarily stores information during program execution. It may be used to store in. Further, the system memory 304 may be configured to store a large number of digital media content files.

As shown in FIG. 3, the system memory 304 includes an operating system 306, an application 308, and a browser application 310. The application 308 may include an application implemented or executed within the arithmetic device 300, which may be implemented or included within a component of the arithmetic device 300, thereby being operational. It can be performed and / or coupled operational / communicatively. The application 308 may include an instruction that allows the CPU (s) 302 of the arithmetic device 300 to perform a particular function. Application 308 can include algorithms expressed in computer programming statements such as for loops, while loops, if statements, and do loops. Application 308 may be developed using a particular programming language. Examples of programming languages are Java®, Jini®, C, C ++, Objective C, swift, Perl®, Python®, PhP, UNIX® Shell, Visual. Basic and Visual Basic Script can be mentioned. The browser application 310 includes a logical structure that allows the arithmetic device 300 to retrieve data using resource identifiers and / or render the retrieved data. The browser application 310 can include one or more web browsers. The browser application 310 is configured to process documents that generally correspond to website content, such as markup language documents (eg, HTML5, XML, etc.) and scripted language documents (eg, Javascript®, etc.). You may. Application 308 and browser application 310 can be run in association with operating system 306. That is, the operating system 306 may be configured to facilitate application interaction with the CPU (s) 302 of the arithmetic device 300 and other hardware components. In some embodiments, the operating system 306 may be an operating system designed to be installed in a set-top box, digital video recorder, television, and the like. Further, in some embodiments, the operating system 306 may be an operating system designed to be installed on a dedicated computing device and / or a mobile computing device. It should be noted that the techniques described herein can be utilized by devices configured to operate using any and all combinations of software architectures.

The system interface 312 may be configured to allow communication between the components of the arithmetic device 300. In one example, the system interface 312 includes a structure that allows data to be transferred from one peer device to another peer device or storage medium. For example, the system interface 312 is a peripheral component interconnect such as an Accelerated Graphics Port (AGP) -based protocol, such as a PCI Express® bus specification managed by the Peripheral Component Interconnect Expert Group. (Peripheral Component Interconnect, PCI) can include chipsets that support bus-based protocols, or any other form of structure that can be used to interconnect with peer devices (eg, proprietary bus protocols). ..

As described above, the arithmetic device 300 is configured such that the arithmetic device 300 receives data from a communication network and extracts digital media contents from the data. The extracted digital media content may be encapsulated in various packet types. The network interface 324 may be configured to allow the computing device 300 to transmit and receive data over a local area network and / or a wide area network. The network interface 324 can include a network interface card such as an Ethernet card, an optical transceiver, a radio frequency transceiver, or any other type of device configured to transmit and receive information. The network interface 324 may be configured to perform physical signaling, addressing, and channel access control according to the physical and media access control (MAC) layers used in the network. The arithmetic device 300 may be configured to analyze a signal generated according to any of the techniques described herein.

The voice decoder 314 may be configured to receive and process voice packets. For example, the voice decoder 314 can include a combination of hardware and software configured to implement aspects of a voice codec. That is, the audio decoder 314 may be configured to receive audio packets and provide audio data to the audio output system 316 for rendering. Audio data may be encoded using a multi-channel format such as that developed by Dolby and Digital Theater Systems. The audio data may be encoded using an audio compression format. Examples of audio compression formats include Motion Picture Experts Group (MPEG) format, Advanced Audio Coding (AAC) format, DTS-HD format, and Dolby Digital (AC-3) format. The audio output system 316 may be configured to render audio data. For example, the audio output system 316 can include an audio processor, a digital / analog converter, an amplifier, and a speaker system. The speaker system can include any of various speaker systems such as headphones, integrated stereo speaker system, multi-speaker system, or surround sound system.

The moving image decoder 318 may be configured to receive and process moving image packets. For example, the moving image decoder 318 can include a combination of hardware and software used to implement aspects of the moving image codec. In one example, the moving image decoder 318 is the ITU-T H.D. 262 or ISO / IEC MPEG-2 Visual, ISO / IEC MPEG-4 Visual, ITU-TH. Video data encoded according to any number of video compression standards, such as 264 (also known as ISO / IEC MPEG-4 Advanced video Coding (AVC)), and High-Efficienty Video Coding (HEVC). It may be configured to decrypt. The display system 320 may be configured to search and process moving image data for display. For example, the display system 320 can receive pixel data from the moving image decoder 318 and output the data for a visual presentation. Further, the display system 320 may be configured to output graphics (eg, a graphical user interface) associated with the moving image data. The display system 320 may be a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device capable of presenting moving image data to the user. Can include one of the various display devices of. The display device may be configured to display standard definition content, high definition content, or ultra-high precision content.

The I / O device (s) 322 may be configured to receive an input and provide an output during the operation of the arithmetic device 300. That is, the I / O device (s) 322 allows the user to select the multimedia content to be rendered. Inputs include, for example, pushbutton remote controls, devices including touch-sensitive screens, motion-based input devices, voice-based input devices, or any other type of device configured to receive user input. Can be generated from an input device. The I / O device (s) 322 may be a standardized communication protocol such as, for example, Universal Serial Bus (USB), Bluetooth®, ZigBee®, or, for example, proprietary. It may be operationally coupled to the computing device 300 using a proprietary communication protocol such as the infrared communication protocol of.

Referring again to FIG. 2, the wide area network 210 is an example of a network configured to deliver digital media content. The wide area network 210 includes a packet-based network and can operate according to a combination of one or more telecommunications protocols. Telecommunications protocols can include specialized embodiments and / or can include standardized telecommunications protocols.
Examples of standardized telecommunications protocols are the Global System Mobile Communications (GSM) standard, the code division multiple access (CDMA) standard, and the 3rd ^Generation Partnership Project (3GPP). One or more of standards, European Telecommunications Standards Institute (ETSI) standards, European standards (EN), IP standards, Wireless Application Protocol (WAP) standards, and, for example, the IEEE 802 standard. Examples include Institute of Electrical and Electronics Engineers (IEEE) standards such as (Wi-Fi). The wide area network 210 can include any combination of wireless communication media and / or wired communication media. Wide area network 210 communicates between coaxial cables, fiber optic cables, twisted pair cables, Ethernet® cables, wireless transmitters and receivers, routers, switches, repeaters, base stations, or various devices and sites. It can include any other equipment that may be useful for facilitation. In one example, the wide area network 210 may generally correspond to the Internet and its subnetworks.

Further, on the wide area network 210 can include radio and television service networks, including public radio transmission networks, public or subscription-based satellite transmission networks, and public or subscription-based cable networks, and / or over-the-top (over). The top) or Internet service providers can be included. The wide area network 210 can operate according to a combination of one or more telecommunications protocols associated with a radio and television service network. Telecommunications protocols can include specialized embodiments and / or can include standardized telecommunications protocols. Examples of standardized telecommunications protocols include DVB standards, ATSC standards, ISDB standards, DTMB standards, DMB standards, Data Over Cable Service Interface Specification (DOCSIS) standards, HbbTV standards, and W3C standards. , And UPnP standards.

Referring again to FIG. 2, the content provider sites 204A-204N represent examples of sites capable of transmitting multimedia content. For example, a content provider site may include a studio having one or more studio content servers configured to serve multimedia files and / or streams to a media distribution engine and / or database. Database 306 may include storage devices configured to store, for example, multimedia content and related data, such as descriptive data and data including executable interactive applications. The data associated with the multimedia content can be formatted according to defined data formats such as Hypertext Markup Language (HTML), Dynamic HTML, XML, and JSON, and URLs and URIs. Can include.

The media distribution engines 208A-208N may be configured to distribute digital media content via the wide area network 210. For example, the media distribution engines 208A-208N may be included in one or more broadcast stations, cable television provider sites, satellite television provider sites, internet-based television provider sites, and / or so-called streaming media service sites. The media distribution engines 208A-208N may be configured to receive data including, for example, multimedia content, interactive applications and messages and distribute the data to computing devices 202A-202N through the wide area network 210.

FIG. 4 is a block diagram showing an example of a media distribution engine capable of implementing one or more of the techniques of the present disclosure. The media distribution engine 400 may be configured to receive the data and output a signal representing the data for distribution via a communication network, for example, a wide area network 210. For example, the media distribution engine 400 may be configured to receive digital media content and output one or more data streams. A data stream can generally refer to data encapsulated in one or more sets of data packets. As shown in FIG. 4, the media distribution engine 400 includes a central processing unit (s) 402, a system memory 404, a system interface 412, a media presentation description generator 414, a segment generator 416, and a transport / network packet generator. A unit 418 and a network interface 420 are included. Each of the central processing unit (s) 402, system memory 404, system interface 412, media presentation generator 414, segment generator 416, transport / network packet generator 418, and network interface 420 is for inter-component communication. May be interconnected (physically, communically, and / or operational) for one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays. It can be implemented as any of a variety of suitable circuits such as (FPGA), discrete logic, software, hardware, firmware, or a combination thereof. Although the media distribution engine 400 is illustrated as having a separate functional block, it should be noted that such illustration is for illustration purposes only and does not limit the media distribution engine 400 to a particular hardware architecture. Is. The functionality of the media distribution engine 400 can be realized using any combination of hardware implementation, firmware implementation, and / or software implementation.

Each of the central processing unit (s) 402, system memory 404, system interface 412, and network interface 420 is the central processing unit (s) 302, system memory 304, system interface 312, and network interface 324 described above. May be similar to. The CPU (s) 402 may be configured to perform functions and / or process instructions for execution in the media distribution engine 400. The system interface 412 may be configured to allow communication between the components of the media distribution engine 400. The network interface 420 may be configured to allow the media distribution engine 400 to transmit and receive data over a local area network and / or a wide area network. The system memory 404 may be configured to store information that may be used by the media distribution engine 400 during operation. The system memory 404 may include individual memory elements contained within each of the component encapsulation unit 402, the transport / network packet generation unit 404, the link layer packet generation unit 406, and the frame builder, and the waveform generation unit 408. It should be noted that. For example, system memory 404 includes one or more buffers configured to store data for processing by components of the media distribution engine 400 (eg, First-in First-out (FIFO) buffers). be able to.

As shown in FIG. 4, the system memory includes an operating system 406, media content 408, and server application 410. The operating system 406 may be configured to facilitate application interaction with the CPU (s) 402 of the media distribution engine 400 and other hardware components. The media content 408 can include data forming a media presentation (eg, moving image component, audio component, subtitle component, etc.). The media presentation generator 414 may be configured to receive one or more components and generate a media presentation based on DASH. Further, the media presentation generation unit 414 may be configured to generate a media presentation description fragment. The segment generator 416 may be configured to receive the media component and generate one or more segments for inclusion in the media presentation. The transport / network packet generator 418 receives the transport package, and the transport layer packet (eg, UDP, transport control protocol (TCP), etc.) and the network layer packet (eg, Ipv4, etc.) corresponding to the transport package are received. It may be configured to be encapsulated in (Ipv6, compressed IP packet, etc.).

Referring again to FIG. 4, system memory 404 includes server application 410. As mentioned above, the WebSocket protocol allows mutual communication between a logical client and a logical server through the establishment of bidirectional sockets. The server application 410 can include instructions that can cause the CPU (s) 402 of the media distribution engine 400 to perform a particular function associated with the WebSocket protocol. FIG. 5 is a conceptual diagram showing an example of a communication flow according to one or more techniques of the present disclosure. FIG. 5 shows an example of a message flow for transporting an MPEG-DASH media presentation via a full-duplex WebSocket session. As mentioned above, the messages defined in W16232 for signaling push directives and push acknowledgments may not be ideal. The arithmetic device 300 and the media distribution engine 400 may be configured to exchange messages according to the techniques described herein.

In the embodiment shown in FIG. 5, in 502, the browser application 310 sends an MPD request to the server application 410. In some embodiments, the browser application 310 may also be referred to as a client, or HTTP client, or DASH client. Referring to Table 3, W16232 cannot define a normative JSON schema for the given arguments in Table 3. 6A-6B are computer programming lists showing an exemplary JSON schema for each media presentation document request message. In one example, the browser application 310 may be configured to send MPD requests to the server application 410 based on the exemplary schema shown in FIGS. 6A-6B.

Referring again to FIG. 5, at 504, the server application 410 sends an MPD request response to the browser application 310. Referring to Table 5, W16232 cannot define a normative JSON schema for the given arguments in Table 5. Further, in Table 5, the parameter status includes the "Number" type in W16232. The JSON data type "Number" corresponds to integers and floating point numbers. Floating point numbers are not required for HTTP status codes. Therefore, in some embodiments, the server application 410 may be configured to send a response message containing the parameter status using the JSON data type "Integer". In this case, it should be noted that the JSON data type may be referred to as "integer" or "integer". Referring again to Table 5, W16232 provides "mpd" as a parameter of type "MPD". This does not completely specify how the MPD is encoded as a JSON-encoded parameter, as it does not define which JSON data type is used. In one example, for the parameter "mpd", the server application 410 can use the JSON data type of "string" with encoding based on one of the following exemplary requirements.
In the media presentation description (MPD) data returned by the server, after all generated line feeds (0x0A) have been deleted and all generated double quotes (0x22) have been replaced with single quotes (0x27). , Is included in the JSON character string.
Alternatively, the media presentation description (MPD) data returned by the server is included in the JSON string after all generated line feeds (0x0A) have been deleted or escape-encoded.
Alternatively, it is assumed that the media presentation description (MPD) data returned by the server is included in the JSON character string after escape encoding.

Thus, in one example, the server application 410 can send the MPD request response to the browser application 310, and the format of the MPD request response is defined as provided in Table 9.

Further, FIGS. 7A-7B are computer programming lists showing exemplary schemas for each media presentation document request response message. In one example, the server application 410 may be configured to send an MPD request response to the browser application 310 based on the exemplary schema shown in FIGS. 7A-7B.

Referring again to FIG. 5, at 506, the browser application 310 sends a segment request to the server application 410. Referring to Table 4, W16232 cannot define a normative JSON schema for the given arguments in Table 4. 8A-8B are computer programming lists showing an exemplary JSON schema for each segment request message. In one example, the browser application 310 may be configured to send a segment request to the server application 410 based on the exemplary schema shown in FIGS. 8A-8B.

Referring again to FIG. 5, at 508, the server application 410 sends a segment request response to the browser application 310. Referring to Table 6, W16232 includes "segment" as a parameter of the type "Segment". It doesn't define which JSON data type is used, so it doesn't completely specify how the segment is encoded as a JSON-encoded parameter. In one example, the server application 410 may be configured to send a segment request response and the parameter "Segment" is not a given argument. In one example, the server application 410 may be configured to send the Segment data directly as binary data without JSON encoding after the JSON-encoded parameters including "push_ack", "headers", "status". good. 9A-9B are computer programming lists showing exemplary schemas for each of the segment request response messages. As shown in FIGS. 9A-9B, the exemplary schema does not include the parameter "Segment". It should be noted that in each of the embodiments shown in FIGS. 9A-9B, the status can have an integer data type. In one example, the server application 410 may be configured to send a segment request response to the browser application 310 based on the exemplary schema shown in FIGS. 9A-9B.

In another example, the server application 410 may be configured to send "segment" parameters as JSON, and the segment parameters may be based on the description provided in Table 10, so the correspondence in Table 6 You may replace the entry you want.

9C-9D are computer programming lists showing exemplary schemas for each segment request response message. As shown in FIGS. 9C-9D, the exemplary schema includes the parameter "Segment". It should be noted that in each of the embodiments shown in FIGS. 9C-9D, the status can have an integer data type. In one example, the server application 410 may be configured to send a segment request response to the browser application 310 based on the exemplary schema shown in FIGS. 9C-9D. In one example, the schemas in FIGS. 9C-9D can be used with Table 10. Referring to FIG. 5 again, the session is terminated by the browser application 310 sending a cancellation request to the server application 410 (510) or by the server application 410 sending a resource change notification to the browser application (512). be able to. Referring to Table 8, W16232 cannot define a normative JSON schema for the given arguments in Table 8. 10A-10B are computer programming lists showing an exemplary JSON schema for each cancellation request message. In one example, the browser application 310 may be configured to send a cancellation request message to the server application 410 based on the exemplary schema shown in FIGS. 10A-10B.

Referring to Table 7, W16232 cannot define a normative JSON schema for the given arguments in Table 7. 11A-11B are computer programming lists showing exemplary schemas for each resource change notification message. In one example, the server application 410 may be configured to send a cancellation request message to the browser application 310 based on the exemplary schema shown in FIGS. 11A-11B. As shown in Table 7, the inclusion of header parameters may be optional. In one example, if the headers parameter is not included in the instance of the end_of_stream message, EXT_LENGTH is assumed to be set to 0 in the DASH subprotocol frame (or should be set in some embodiments) and in the EXT_LENGTH field. It is assumed that there is no empty JSON parameter encoding later. It should be noted that in the absence of the headers parameter, including empty JSON string data with additional necessary padding to the 4-byte boundary is a waste of bandwidth and requires unnecessary processing. Is. In this way, the arithmetic device 300 and the media distribution engine 400 may be configured to exchange messages according to the techniques described herein.

As mentioned above, in W16232, the format of PushType in ABNF format is as follows:
PUSH_TYPE = PUSH_TYPE_NAME [OWS]; "OWS PUSH_PARAMS] PUSH_TYPE_NAME = DQUATE <URN> DQUATE
PUSH_PARAMS = PUSH_PARAM * (OWS ";" OWS PUSH_PARAM)
PUSH_PARAM = 1 * VCHAR

In some cases, PUSH_PARAM may conflict with JSON reserved characters. For example, VCHAR includes the character DQUATE ("). JSON uses DQUATE as the reserved character. In some cases, PushDirective is signaled as a JSON property, which may require complex parsing. In one example, the syntax of PUSH_PARAM can be defined with some simple modifications so as not to interfere with JSON encoding. In one example, the browser application 410 and server application 510 are preceded by DQUATE if PUSH_PARAM contains DQUATE. It may be configured to be escaped with "\" (ie,% x5C).

In another example dealing with double quotes above, the following changes could be made to W16232.
Add the following to the section "3.2 Connection": PPCHAR =% x21 /% x23-7E
The generated VCHAR is replaced with PPCHAR (Section 6.1.2 PushType, 6.1.5 URLList, 6.1.6 URLTemplate, and 6.1.7 FastStartParams).
In 6.1.5 (URL Temple), the two DQUATEs are changed to'(single quotes, ie% x27). Further, in Table 2 (Valid attributes for FastStartParams), the generated DQUATE is changed to'(single quote, that is,% x27), and'' is changed to'(single quote, that is,% x27).
The above changes use single quotes instead of double quote characters. For this reason, PPCHAR is defined to exclude the DQUATE character (% x22).

It should be noted that in W16232, the following subprotocol identifiers are used for the WebSocket subprotocol.
Sub-protocol identifier: "mpeg-dash"

In some embodiments, the browser 410 and server application 510 may be configured to provide future extensibility by including the version and / or year in the subprotocol identifier. In one example, the tag-based subprotocol identifier may be defined as:
Sub-protocol identifier: "tag: mpeg.chiariglione.org-dash, 2016: 2309-6"
In one example, the URN-based subprotocol identifier may be defined as:
Sub-protocol identifier: "urn: mpeg: dash: fdh: 2016"
Or sub-protocol identifier: "urn: mpeg: dash: fdh: 2016: 2309-6"
In one example, the subprotocol identifier and subprotocol generic name may be defined as follows:
Sub-protocol identifier: "2016.fdh.dash.mpeg.chiriglione.org"
Subprotocol generic name: "MPEG-DASH-FDH-23309-6"

In this way, the arithmetic device 300 and the media distribution engine 400 may be configured to exchange messages according to the techniques described herein.

In one or more examples, the described functionality can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, this function is stored or transmitted as one or more instructions or codes on a computer-readable medium and can be performed by a hardware-based processor. Computer-readable media correspond to tangible media such as data storage media or communication media, including any medium that facilitates the transfer of computer programs from one location to another, for example according to a communication protocol. It can include a storage medium. In this way, the computer-readable medium can generally correspond to (1) a non-temporary tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium is any that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for the introduction of the techniques described in this disclosure. It can be an available medium. Computer program products can include computer readable media.

By way of example, such computer-readable storage media are RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, other magnetic storage, flash memory, or, but not limited to. Any other medium, i.e., any other medium that can be used to store the desired program code in the form of instructions or data structures and is accessible by a computer can be included. Also, any connection is appropriately referred to as a computer-readable medium. For example, instructions use coaxial cables, fiber optic cables, twisted pairs, digital subscriber lines (DSL), or wireless technologies such as infrared, wireless and microwave from websites, servers, or other remote sources. Radio technologies such as coaxial cable, fiber optic cable, twisted pair, DSL, or infrared, radio and microwave are included in the definition of medium. However, it should be understood that computer readable and data storage media do not include connections, carrier waves, signals, or other transient media, but instead are intended for non-temporary tangible storage media. When used in the present invention, the disc and the disc are a compact disc (CD), a laser disc, an optical disc, and a digital versatile disc (Digital Versatile Disc). Includes DVD), floppy disk and Blu-ray (registered trademark) disc (Blu-ray (registered trademark) disc), where the disc normally reproduces data magnetically and the disc uses a laser. Use to optically reproduce the data. The above combinations must also be included within the scope of computer readable media.

One or more of the instructions, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. It can be executed by the processor of. Accordingly, as used in the present invention, the term "processor" can refer to any of the aforementioned structures or any other structure suitable for carrying out the techniques described herein. In addition, in some embodiments, the functionality described herein is provided within a dedicated hardware module and / or software module configured to encode and decode, or incorporated into a composite codec. Can be. The technique can also be fully implemented in one or more circuits or logic elements.

The techniques of the present disclosure can be implemented in a wide variety of devices or devices, including wireless handsets, integrated circuits (ICs), or sets of ICs (eg, chipsets). The various components, modules, or units are described herein to highlight the functional aspects of the device configured to perform the disclosed techniques, but may be realized by different hardware units. Not always necessary. Rather, as mentioned above, the various units may be combined with codec hardware units, or by a set of interactive hardware units, including the aforementioned one or more processors, along with suitable software and / or firmware. Can be provided.

Further, each functional block and various functions of the base station device and the terminal device (moving image decoder and moving image encoder) used in each of the above-described implementation forms is generally an integrated circuit or a circuit which is a plurality of integrated circuits. Can be implemented or implemented by. Circuits designed to perform the functions described herein are general purpose processors, digital signal processors (DSPs), application specific integrated or general purpose application integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other. It may include programmable logic devices, discrete gates or transistor logic, or individual hardware components, or a combination thereof. The general purpose processor may be a microprocessor, or the processor may be a conventional processor, controller, microcontroller, or state machine. The general-purpose processor or each circuit described above may be composed of a digital circuit or an analog circuit. Furthermore, if an integrated circuit technology that replaces the current integrated circuit appears due to advances in semiconductor technology, integrated circuits based on this technology will also be usable.

Various examples have been described. These and other examples are within the scope of the following claims.

This application is a related application to US Provisional Patent Application No. 62 / 408,008 filed on October 13, 2016, claiming priority under this provisional application and the content herein by reference. The whole is incorporated.

Claims (7)

A method for signaling information associated with a media presentation, said method.
Dynamically Adaptive Streaming Over Hypertext Transfer Protocol Includes signaling a frame header indicating the message type and a set of Hyperext Transfer Protocol (HTTP) 1.1 compliant header fields corresponding to the message type as Javascript Object Notification encoding parameters. Methods, including signaling header parameters . The method of claim 1, wherein the frame header indicates a media presentation description response message type , further comprising signaling media presentation description parameters. The method of claim 2, wherein the media presentation description parameter comprises an encoded character string. The method of claim 1, wherein the frame header indicates a segment request response message type and further comprises signaling a status parameter applied to the segment request response message. The method of claim 4, further comprising signaling a segment parameter, wherein the segment parameter comprises a coded string. The frame header indicates the media presentation description response message type.
The method of claim 1, wherein the header parameters include a set of header fields that apply to the media presentation description response message. The frame header indicates the segment request response message type.
The method of claim 1, wherein the header parameters include a set of header fields that apply to the segment request response message.

Images