TWI720282B - Method for receiving a watermark message and device that includes a processor configured to receive a watermark - Google Patents

Abstract

A device may be configured to signal information using watermarks. A device may be configured to determine a watermark message identifier. A device may be configured to receive a multimedia signal, parse a watermark message identifier, and receive fragment characteristic information in response to the value of the watermark identifier.

Description

Method for receiving a watermark message and device containing a processor configured to receive a watermark message

Cross-reference of related applications

The present invention generally relates to a system with audiovisual content watermarking.

In many digital broadcasting systems, a broadcasting station transmits both a stream of audiovisual (AV) content and one or more enhanced service data. The enhanced service data may be provided together with the AV content to provide information and services, or may be provided independently of the AV content to provide information and services.

In many broadcasting environments, audiovisual content and one or more enhanced service materials are not directly received by an AV presentation device from a broadcasting station. To be precise, an AV presentation device such as a TV is usually connected to a broadcast receiving device that receives the audiovisual content and the one or more enhanced service data in a compressed form and provides uncompressed audiovisual content to The AV presentation device.

In some broadcast environments, the broadcast receiving device receives audiovisual content from a server (sometimes called a multi-channel video programming distributor (MVPD)). The MVPD receives an audio-visual broadcast signal from a broadcasting station, extracts content from the received audio-visual broadcast signal, converts the extracted content into an audio-visual signal having an appropriate format for transmission, and the converted audio-visual signal The signal is provided to the broadcast receiving device. During the conversion process, the MVPD usually removes the enhanced service data provided from the broadcasting station, or may incorporate a different enhanced service data provided to the broadcast receiving device. In this way, the broadcasting station can provide audio-visual content with enhanced service data, but it will eventually be provided to The enhanced service data (if any) of the AV presentation device and/or the broadcast receiving device may be different from the enhanced service data provided by the broadcasting station.

Because the broadcast receiving device captures the audiovisual content from the signal received by the MVPD and only provides uncompressed audiovisual data to the AV presentation device, only the enhanced service data provided to the broadcast receiving device is available. In addition, the same enhanced service data provided by the broadcast station may not be provided to the broadcast receiving device and/or the AV presentation device.

After considering the following detailed description of the present invention in conjunction with the accompanying drawings, it will be easier to understand the foregoing and other objectives, features and advantages of the present invention.

An embodiment of the present invention discloses a method for receiving a watermark message, the watermark message containing at least one watermark message block, and the method includes the following steps: (a) receiving the at least one watermark message block; (b) ) Determine a watermark message id from at least one of the at least one watermark information block; (c) Determine a piece of fragment characteristic information of each of the at least one watermark information block, wherein a length of the fragment characteristic information is based on A value of the watermark message id; (d) recovering the watermark message from the at least one watermark message block based on the fragment characteristic information.

Another embodiment of the present invention discloses a method for receiving a watermark message, the watermark message containing at least one watermark message block, and the method includes the following steps: (a) receiving the at least one watermark message block; b) Determine a watermark message id from the at least one of the watermark information block; (c) Determine whether there is fragment characteristic information for each of the at least one of the watermark information block, wherein the determination exists A fragment characteristic information depends on a value of the watermark message id; (d) recovering the watermark message from the at least one watermark message block based on the fragment characteristic information.

Another embodiment of the present invention discloses one including a message configured to receive a watermark A device of a processor, the watermark message contains at least one watermark message block, and receiving the watermark message includes: (a) the device receives the at least one watermark message block; (b) the device receives the at least one watermark message from the device At least one of the blocks determines a watermark message id; (c) the device determines fragment characteristic information of each of the at least one watermark message block, wherein a length of the fragment characteristic information is based on the watermark message id A value; (d) the device recovers the watermark information from the at least one watermark information block based on the fragment characteristic information.

Another embodiment of the present invention discloses a device including a processor configured to receive a watermark message, the watermark message containing at least one watermark message block, and receiving the watermark message includes: (a) the device Receiving the at least one watermark message block; (b) the device determines a watermark message id from at least one of the watermark message blocks; (c) the device determines whether it is for the at least one of the watermark message blocks Each has fragment characteristic information, wherein the determination whether there is a fragment characteristic information depends on a value of the watermark message id; (d) the device restores the float from the at least one watermark message block based on the fragment characteristic information Watermark message.

100: content source

120: Content recognition service provision server

130: Multi-channel video program distributor

140: Enhanced service information provision server

160: Broadcast receiving device

170: Network

180: AV presentation device

190: Watermark Inserter

201: Enhanced service materials

203: Enhanced service materials

205: Provide

206: Provide

207: Broadcast

208: Transmission

209: request

211: Reply

300: Yard

310: Receiver with watermark capability

320: Watermark client

350: Metadata Server

351: Yard

353: Metadata

355: request

357: Content and Sending

360: Metadata

370: Code Database

380: Content and Sending Server

400: Content Server

500: watermark embedder

510: payload

520A, 520B, 520C: Fragments

530A, 530B, 530C: Message

600: Watermark payload extractor

610: Capture

620A, 620B, 620C: message reassembly

630A, 630B, 630C: Message

700: Expiry time value

710: Flag of Urgency

720: severity indicator

730: Deterministic indicator

800: Watermark message block

802: Watermark message identification (wm_message_id)

804:wm_message_bytes()

805: table

808: Emergency warning signal and message (EA_message())

852:EA_Expiry

854:EA_Urgency

856:EA_message_body_present

858: 4 bits reserved

860: conditional statement

862:EA_message_ID

864:EA_message_version

866:EA_message_text_length

868:EA_message_text(8*N)

900:EA_Certainty_severity_code

1000: certainty

1010: severity

1100:EA_Expiry

1102: reserved 6 bits

1104: reserved 2 bits

1200: Emergency warning flag (EA_flag)

1210:server_code

1220:interval_code

1230: trigger

1240:EA_Expiry

1250:EA_Urgency

2500: Operation

2505: Operation

2515: Operation

2550: Operation

2555: Operation

2565: Operation

Figure 1 shows a system with enhanced service information.

Figure 2 shows another system with enhanced information.

Figure 3 shows a data flow of a system with enhanced information.

Figure 4 shows another system with enhanced information.

Figure 5 shows a watermark payload.

Figure 6 shows another watermark payload.

Figure 7 shows the relationship between the watermark payloads.

Figure 8 shows the relationship between the watermark payloads.

Figure 9 shows the relationship between the watermark payloads.

Figure 10 shows another system with enhanced information.

Figure 11 illustrates obtaining synchronization and maintaining synchronization.

Figure 12 shows another watermark payload.

Figure 13 shows SDO private data.

Figure 14 shows the metadata encapsulated in the SDO private data using cmdID as the SDO payload.

Figure 15 shows a watermark embedding system.

Figure 16 shows a watermark capture system.

FIG. 17 shows an expiration time value of an emergency message, an urgency flag, a severity indicator, and a certainty indicator.

Figure 18 shows an exemplary emergency warning message.

Figure 19 shows another exemplary emergency warning message.

Figure 20 shows a set of exemplary deterministic codes and severity codes.

Figure 21 shows another exemplary emergency warning message.

Figure 22 shows another exemplary emergency warning message.

Figure 23 shows another exemplary emergency warning message.

Figure 24A shows an exemplary bitstream syntax of a watermark message block.

Figure 24B is an exemplary mapping of the field wm_message_id to the watermark message wm_message().

Fig. 24C shows an exemplary syntax of wm_message().

Figure 24D shows an exemplary syntax of the URI message.

FIG. 24E shows an exemplary mapping from the value of the uri_type field to the type of URI.

Figure 25A shows a flow of an exemplary transmission of a field based on a transmitted key field value Figure.

FIG. 25B shows a flowchart of exemplary reception of a field based on a received key field value.

Figure 26A shows another exemplary bitstream syntax of the watermark message block.

Fig. 26B is an exemplary mapping of the field wm_message_id to the watermark information wm_message().

Fig. 26C shows an exemplary syntax of wm_message().

(definition)

A uimsbf can represent the most significant bit of an unsigned integer in the first format.

When the value in the number of bit rows is equal to var, it represents a variable length field.

The once reserved field can indicate that the bit corresponding to the field is reserved for future use. Hexadecimal (also hexadecimal or hex) refers to the 16-base or base-carry system, which uses 16 different symbols, most often symbols 0-9 are used to represent values 0-9, and A, B, C, D, E, F (or alternatively, a, b, c, d, e, f) represent values of 10-15. Hexadecimal numbers usually use the prefix "0x".

x ^y can be used to indicate that an arithmetic operation corresponds to an exponentiation operation, that is, x to the power of y. In other background content, this notation can be used for unintended interpretation as exponentiation superscripts.

1, the system may include a content source 100, a content recognition service providing server 120, a multi-channel video program distributor 130, an enhanced service information providing server 140, a broadcast receiving device 160, and a network 170 And an AV presentation device 180.

The content source 100 may correspond to a broadcast station that broadcasts a broadcast signal including one or more audiovisual content streams (for example, audio and/or video). The broadcast signal may further include enhanced service data Material and/or send information. The enhanced service data preferably relates to one or more of the audiovisual broadcast streams. The enhanced data service can have any suitable format, such as, for example, service information, metadata, additional data, compiled executable files, web applications, hyperdocument markup language (HTML) documents, extensible markup language (XML) documents, Cascading style sheet (CSS) files, audio files, video files, Advanced Television Systems Committee (ATSC) 2.0 or future version content, and addresses such as Uniform Resource Locator (URL). The content recognition service providing server 120 provides a content recognition service that allows the AV presentation device 180 to recognize content based on the audiovisual content from the content source 100. The content recognition service providing server 120 may modify the audiovisual broadcast content according to the situation, such as by including a watermark. In some cases, the AV presentation device 180 is a digital video recording device.

The content recognition service providing server 120 may include a watermark inserter. The watermark inserter can insert several watermarks, which are designed to carry enhanced service data and/or signaling information without the viewer's perception or at least minimally disturbing the viewer. In other cases, an easily observable watermark can be inserted (for example, it can be easily observable or it can be easily seen in the image, and/or it can be easily observable or it can be easily heard in the audio). For example, a watermark that can be easily observed may be a logo, such as a logo of a content provider at the upper left or upper right of each frame.

The content recognition service providing server 120 may include modifying the audiovisual content to include a non-observable watermark (for example, the non-observable can be in the image and/or the non-observable can be in the audio Central Africa can be easily heard) one of the watermark inserters. For example, a watermark that is not easily observable may include security information, tracking information, data, or others. Another example includes channel, content, timing, trigger, and/or URL information.

The multi-channel video program distributor 130 receives broadcast signals from one or more broadcast stations, and usually provides the multiplexed broadcast signals to the broadcast receiving device 160. Multi-channel video program distributor 130 can perform demodulation and channel decoding on the received broadcast signal to retrieve audiovisual content and enhanced service data. The multi-channel video program distributor 130 can also perform channel coding on the captured audio-visual content and enhanced service data to generate a multiplexed signal for further distribution. The multi-channel video program distributor 130 may exclude the retrieved enhanced service data, and/or may include a different enhanced service data. The broadcast receiving device 160 can be adjusted to a channel selected by a user and receive an audiovisual signal of the tuned channel. The broadcast receiving device 160 usually performs demodulation and channel decoding on the received signal to capture the desired audiovisual content. The broadcast receiving device 160 uses any suitable technology (such as, for example, H.264/Animation Experts Group 4th Edition Advanced Video Coding (MPEG-4 AVC), H.265/High Efficiency Video Coding (HEVC), Dolby AC-3 And the Animation Expert Group 2nd Edition Advanced Audio Coding (MPEG-2 AAC)) decodes the captured audiovisual content. The broadcast receiving device 160 generally provides uncompressed audiovisual content to the AC rendering device 180.

The enhanced service information providing server 140 responds to a request from the AV presentation device 180 to provide enhanced service information to the audiovisual content.

The AV presentation device 180 may include a display, such as, for example, a TV, a notebook computer, a digital video recorder, a mobile phone, and a smart phone. The AV presentation device 180 can receive uncompressed (or compressed) audiovisual content or video or audio content from the broadcast receiving device 160, and receive a broadcast signal containing one of the encoded audiovisual content or video or audio content from the content source 100, and/or The encoded or decoded audio-visual content or video or audio content is received from the multi-channel video program distributor 130. In some cases, the uncompressed video and audio can be received via an HDMI cable. The AV presentation device 180 can receive an address of an enhanced service regarding the audiovisual content from the enhanced service information providing server 140 from the content recognition service providing server 120 via the network 170.

It should be understood that the content source 100 and the content recognition service providing server can be combined or omitted as desired. The server 120, the multi-channel video program distributor 130, and the enhanced service information providing server 140. It should be understood that these are logical roles. In some cases, parts of these entities may be separate physical devices. In other cases, parts of these logical entities may be embodied in the same physical device. For example, if necessary, the broadcast receiving device 160 and the AV presentation device 180 can be combined.

Referring to FIG. 2, a modified system may include a watermark inserter 190. The watermark inserter 190 can modify the audiovisual (for example, audio and/or video) content to include additional information in the audiovisual content. The multi-channel video program distributor 130 can receive and distribute a broadcast signal including a modified audio-visual content with a watermark.

The watermark inserter 190 preferably modifies the signal in a way that includes additional information that is not easily observable in the form of digital information (for example, visually and/or audibly). In the non-observable watermark, the inserted information can be easily identifiable in the audio and/or video. In the non-observable watermark, although the information is contained in the audiovisual content (for example, audio and/or video), it is not easy for a user to notice the information.

One purpose of watermarking is to prevent illegal copying of digital media for copyright protection. Another use of watermark is the source tracking of media. One of the watermarks further uses the descriptive information of the media. Another use of watermarks is to provide location information about where additional content associated with digital media can be received. Yet another is used to identify the content being viewed and the content source and the current point in time in the content, and then allow the device to access the required additional functionality via an Internet connection. The watermark information is included in the audio-visual content itself, as distinguished from the metadata delivered with the audio-visual content. By way of example, the watermark information can be included by using a spread spectrum technique, a quantization technique, and/or an amplitude modulation technique.

Referring to Figure 3, an exemplary data flow is shown. The content source 100 transmits a broadcast signal including at least one audiovisual content and an enhanced service data 201 to the watermark inserter 190.

The watermark inserter 190 receives the broadcast signal provided by the content source 100, and includes an easily observable and/or non-observable watermark in the audiovisual content. The modified audiovisual content with the watermark is provided to the MVPD 130 together with the enhanced service data 203.

The content information associated with the watermark may include, for example, the identification information of the content provider that provides the audiovisual content, the identification information of the audiovisual content, the time information of a content section used in the content information extraction, and the channel through which the audiovisual content is broadcast. The name, the logo of the channel where the audiovisual content is broadcast through it, the description of the channel where the audiovisual content is broadcast through it, a report period of usage information, the minimum usage time for collecting usage information, statistical data of sports events, display of useful information, interface Tool set, application, executable and/or available enhanced service information about audiovisual content.

The retrieval path of the available enhanced service data can be expressed in any way, such as the path based on the Internet Protocol (IP) or the Advanced Television Systems Committee-Mobile/Handheld System (ATSC M/H).

The MVPD 130 receives broadcast signals including watermarked audiovisual content and enhanced data services, and can generate a multiplexed signal to provide 205 to the broadcast receiving device 160. At this time, the multiplexed signal may exclude the received enhanced service data and/or may include a different enhanced service data.

The broadcast receiving device 160 can be adjusted to a channel selected by a user and receive the signal of the channel tuned to, demodulate the received signals, perform channel decoding and audiovisual decoding on the demodulated signals to generate a The audiovisual content is compressed, and then the uncompressed audiovisual content is provided 206 to the AV rendering device 180. The content source 100 can also broadcast 207 the audiovisual content to the AV presentation device 180 through a channel. The MVPD 130 can directly transmit 208 a broadcast signal including audiovisual content to the AV presentation device 180 without passing through the broadcast receiving device 160. In yet another case, part of the AV information can be sent to the AV presentation device 180 via a broadband connection. In some cases, this can be Department once managed the broadband connection. In another case, it can be an unmanaged broadband connection.

The AV presentation device 180 can receive uncompressed (or compressed) audiovisual content from the broadcast receiving device 160. In addition, the AV presentation device 180 can receive a broadcast signal from the content source 100 through a channel, and then, can adjust and decode the received broadcast signal to obtain audiovisual content. In addition, the AV presentation device 180 can receive a broadcast signal from the MVPD 130, and then, can adjust and decode the received broadcast signal to obtain audiovisual content. The AV presentation device 180 (or the broadcast receiving device 160) selects and retrieves the watermark information from one or more video frames or one of the audio samples of the received audio-visual content. The AV presentation device 180 can use the information obtained from the watermark(s) to request 209 additional information from the enhanced service information providing server 140 (or any other device). The enhanced service information providing server 140 may provide a reply 211 as a response to the AV presentation device 180.

Referring to FIG. 4, a further example includes the content source 100 that provides audiovisual content together with enhanced service data (if required) to the watermark inserter 190. In addition, the content source 100 can provide a code 300 together with the audiovisual content to the watermark inserter 190. The code 300 can be any suitable code used to identify which of a plurality of audiovisual streams should be modified with a watermark. For example, code=1 can identify the first audio-visual stream from ABC, code=2 can identify the second audio-visual stream from ABC, code=3 can identify the fourth audio-visual stream from ABC, code=4 can identify the fourth audio-visual stream from NBC Audiovisual streaming, etc. The code can contain time and location information within the audiovisual content. The code can contain other metadata (if needed). The watermark inserter 190 provides the watermarked audiovisual content and associated data to the MVPD, and the MVPD can then provide the watermarked compressed audiovisual content to the broadcast receiving device 160 (for example, a set-top box). The broadcast receiving device 160 can provide (for example, usually uncompressed) watermarked audiovisual content to the AV rendering device 180. The AV presentation device 180 may include a receiver 310 with a watermark capability and a watermark client 320. The receiver 310 with watermark capability is suitable for detecting the existence of the watermark in the audio-visual content, and extracting the watermark from the audio-visual content. Watermark data. The watermark client 320 is adapted to use the data retrieved from the watermark to request additional data based on it, and then use the additional data in an appropriate manner.

The AV presentation device 180 can use the code 300 from the extracted watermark to make a request to the unary data server 350. A code database 370 receives data including code 300 and associated metadata 360 from the content source 100. The code 300 and the associated metadata 360 are stored in the code database 370 for subsequent use. In this way, the code 300 encoded in the audiovisual content provided to the watermark inserter 190 is also stored in the code database 370 with its associated metadata 360. In the case of MVPD 130 or other cases where the associated metadata is removed or the associated metadata is changed in other ways, the AV presentation device 180 can recover the data from the metadata server 350, and the metadata server 350 uses the provided code 351 The code database 370 is queried and an associated response with one of the metadata 353 is provided to the AV presentation device 180. The reply metadata provided by the metadata server 350 is used by the AV presentation device 180 to form a request 355 provided to the content and sending server 380. The content and sending server 380 provides the selected content and sending 357 to the AV presentation device 180 in response to the request. Generally speaking, the content and sending server 380 can be different from the metadata server 350.

However, making a first request to the metadata server to obtain a response to the provided code, and then using the metadata to provide a request to the content and sending server 380 is cumbersome and prone to failure. Attributable to two different servers and/or requests used. In addition, it can increase the delay.

By way of example, metadata can be composed of one or more of the following syntax elements: (1) Content and the location of the sending server (for example, where the server is, such as its network address. Examples of network addresses are (Domain name, IPv4 address, etc.); (2) Protocols used to communicate with content and sending servers (for example, Hyper File Transfer Protocol-http, Hyper File Transfer Protocol Security-https, etc.); (3) Identify the time code of a time location in the audiovisual content (for example, where in the audiovisual content should be associated with the metadata); (4) Time-sensitive event triggering (for example, a specific location in the audiovisual content An advertisement or an event); (5) Channel identification (for example, channel specific information; local channel content); (6) Random implementation of the content by the client and the duration of the server request (for example, for load balancing). For brevity, this grammatical element can also be called the duration of the content server request; (7) and so on.

The watermark(s) embedded in the audiovisual content usually has a capacity of only a few bits of the payload information when the watermarked audiovisual broadcast has information that is not easily observable. For relatively small payload sizes, the time code (element 3 above) and/or the location of the content and sending server (element 1 above) often account for a significant percentage of the available payload, leaving a limited amount of remaining data Extra payload, which is often problematic.

In order to include enough metadata in the watermark so that both the time code and location information can be provided with additional information, it may be desirable to split the metadata across multiple watermark payloads. Likewise, each of the watermark payload is preferably contained in a different part of the audiovisual content. The data retrieved from multiple watermark payloads are combined to form a set of information that meets the needs for making a request. In the following description, the term payload may be used to indicate the watermark payload. Each of the syntax elements can be contained in a single payload, span multiple payloads, and/or be fragmented across multiple payloads. For identification purposes, each payload can be assigned a payload type. In addition, an association can be established between multiple payloads belonging to the same or approximately the same timeline position. In addition, the association can be one-way or two-way as desired.

The desired timecode data can be obtained from the payload(s) spanning several time positions of the audiovisual content. Therefore, some systems may establish rules to associate the determined time code of the audiovisual content with a specific time location. In an example, the selected time position may correspond to the time position at the end of a pre-determined watermark payload.

For example, the payload size can be 50 bits, and the required metadata can be 70 bits, so it exceeds the payload size of a single watermark. An example of meta-data that meets the needs can be as follows:

Another example of meta-data that meets the needs can be as follows:

One way to split metadata is to include content and sending server communication information (CSSCI) in one payload, and timeline information in another payload. The CSSCI payload may include, for example, where the information is (for example, the location of the content and the sending server), associated information (for example, an identifier that associates the CSSCI payload with one or more other payloads), and How the information is (for example, application layer protocol, duration of content server request). The timeline information may include, for example, associated information (e.g., an identifier that associates the timeline with one or more other payloads), information time (e.g., time code information), and which information (e.g., channel identification) ).

Referring to Figure 5, an exemplary CSSCI payload is shown.

Referring to FIG. 6, an exemplary time position payload is shown. The term time location may be used instead of the term temporal location.

The payload type can be identified by the first bit "Y". When Y is set to 0, the payload corresponds to the CSSCI payload, and the 14-bit payload identifier (P) is used to mark the CSSCI. When Y is set to 1, the payload corresponds to the time position payload, and the 14-bit payload identifier (P) is signaled to correspond to CSSCI. Therefore, different payload types with the same payload identifier (P) value are associated with each other. The identifier R indicates the duration of one of the transmission content and the sending server request. In yet another example, "Y" may correspond to a 2-bit field, where the value 00 indicates a CSSCI payload, the value 01 indicates a time location payload, and the values 10 and 11 are reserved for future use.

Referring to FIG. 7, an exemplary timeline is drawn. A first CSSCI type payload (for example, CSSCI-0) has a first set of related information P, and a second CSSCI type payload (for example, CSSCI-1) has a second set of different related information P. For CSSCI-0 and CSSCI-1, there are two different kinds of related information P to distinguish the two CSSCI payloads and identify the two CSSCI payloads. A first time position payload (for example, Timeline-0) has a first set of correlation information P that matches CSSCI-0, and a second time position payload (for example, Timeline-1) has a matching CSSCI-0 The related information P has the same first set of related information P, and a third time position payload (for example, Timeline-2) has the same second set of related information P that matches the related information P of CSSCI-1. In this way, CSSCI-0, Timeline-0; CSSCI-0, Timeline-1; and CSSCI-1, Timeline-2 are associated together as a pair with cross-watermarked information. This allows the same CSSCI type payload to be used for multiple different time position payloads.

As shown, each time location payload is associated with a previously received CSSCI type payload, and therefore, is unidirectional in its association. In the case that one of the previous CSSCI type payloads that matches a time location is not available, the system can determine that a packet has been lost or that the watermark is otherwise invalid. The loss of watermark data occurs at a certain frequency because audio-visual content is often modified by audio-visual transcoding, for example, to reduce the bit rate of the audio-visual content.

Referring to FIG. 8, an exemplary timeline is shown. A first CSSCI type payload (e.g., CSSCI-0) has a first set of associated information P, and a second CSSCI type payload (e.g., CSSCI-1) has a second set of different associated information P. There are two different associated information P for CSSCI-0 and CSSCI-1 to distinguish the two CSSCI payloads and identify the two CSSCI payloads. A first time position payload (for example, Timeline-0) has the first set of associated information P that matches CSSCI-0, and a second time position payload (for example, Timeline-1) has a match The associated information P of CSSCI-0 is the same as the first set of associated information P, and a third time position payload (for example, Timeline-2) has a match The second set of related information P is the same as the related information P of CSSCI-1. In this way, CSSCI-0, Timeline-0; CSSCI-0, Timeline-1; and CSSCI-1, Timeline-2 are associated together as a pair with cross-watermarked information. This allows the same CSSCI type payload to be used for multiple different time position payloads. As shown, two of the time position payloads are associated with a previously received CSSCI type payload, and one of the CSSCI type payloads is associated with a subsequently received time position payload, and therefore, in its association Sexuality is two-way. In the case that one of the payloads corresponding to the CSSCI type is not available for matching a time position, the system can determine that a packet has been lost or the watermark is invalid. Similarly, in the case that one of the corresponding CSSCI payloads is not available, the system can determine that a packet has been lost or the watermark is invalid. The loss of watermark data occurs at a certain frequency because audio-visual content is often modified by audio-visual transcoding, for example, to reduce the bit rate of the audio-visual content.

In one example, a CSSCI type payload (for example, CSSCI-0) has two sets of related information P0 and P1. A time position payload, for example, Timeline-0, has two sets of related information P0 and P1 that match the related information P0 and P1 of CSSCI-0. In this example, there is a two-way association for CSSCI-0 and Timeline-0, where P0 points to CSSCI-0, and P1 points to Timeline-0.

The number of bits assigned to the payload identifier (P) can be modified as desired (for example, for a desired robustness). Similarly, the number of bits assigned to I, A, T, D, L, and R can be modified as desired.

In one example, the AV presentation device 180 may maintain a list marked by a variable listC of the "c" most recently received CSSCI payloads. If necessary, "c" can be set in the watermark, or "c" can be set by the system in other ways. In this way, the AV presentation device 180 can only A limited number of CSSCI payloads must be maintained in memory. In the case of c=1, after receiving a CSSCI payload, it remains valid until receiving another CSSCI payload, as shown in FIG. 9. The payload identifier (P) can be used to detect the loss of one of the CSSCI payloads, for example, the time position payload contains one P that does not correspond to any of the CSSCI payloads in listC. In this way, the same user experience can be achieved across different AV presentation devices 180.

In an example, the AV presentation device 180 may maintain more than one list of received CSSCI payload(s). The size of each list can be different, and a set of different rules can be used to maintain each list (that is, the addition/removal of items in the list). It should be understood that this does not exclude the possibility that a subgroup of the list may have the same size and/or the same maintenance rule. As an example, there may be two lists maintained by 180, one of which contains (several) "c1" recently received CSSCI payloads, where each payload is received at one interval of (several) "0" CSSCI payloads; And the other list contains (several) "c2" recently received CSSCI payloads, where each payload is received at one interval of (several) "d" CSSCI payloads.

10, a modified system may include a content source 100, a watermark inserter 190, an MVPD 130, a broadcast receiving device 160, and an AV rendering device 180, as well as a receiver 310 and a watermark client 320 with watermark capabilities. The content server 400 can be modified to include a code database 370, a metadata server 350, and (several) content and sending servers 380. The code 300 and metadata 360 are provided by the content source 100 to the content server 400. The content and sending information are provided to the content and sending server 380.

The AV presentation device 180 can set a code in a request based on decoding one or more watermarks from the audiovisual broadcast. The content server 400 receives a request with a code from the AV presentation device 180. Then, the metadata server 350 parses the received code request, and based on the information from the code database 370, makes a request to the content and sending server 380 to determine and then provide it to the AV presentation The content and sending information of the device 180. In this way, the AV presentation device 180 only needs to make a single request to a single content server 400, and the content server 400 then provides the response to the AV presentation device 180. It should be understood that different functions of the content server 400 can be achieved by combining existing functions, separating these existing functions into more components, omitting components, and/or using any other technology.

When the time-sensitive trigger D is equal to 1, the http and/or https request URL (which will be sent to the content server 400) corresponding to one of the payload(s) in FIGS. 5 and 6 can be defined as follows: if A is equal to 0 , The http request URL is: http://IIIIIIII.IIIIIIII.IIIIIIII.IIIIIIII/LLLLLLLLL? time=TTTTTTTTTTTTTTTTTTTTTTTTT

Otherwise, the https request URL is: https://IIIIIIII.IIIIIIII.IIIIIIII.IIIIIIII/LLLLLLLLL? time=TTTTTTTTTTTTTTTTTTTTTTTTT

The above IIIIIIII.IIIIIIII.IIIIIIII.IIIIIIII corresponds to the 32-bit IP address sent in the CSSCI payload.

In one example, a subgroup of URLs carrying specific information (such as the location of the content server, communication protocol, communication port, registration information, and document library on the content server) is carried according to a specific payload type.

In some implementations, a decoding process that can access information across multiple payloads can be used to derive a value of a syntax element. For example, the time code can be fragmented into multiple watermark payloads, and then reassembled to construct a complete time code. In an example, the time code may correspond to a time position within the audiovisual content. In one example, the time code can correspond to the timeline data of the audiovisual content.

For example, the payload size can be 50 bits, and the required metadata can be 66 bits, so it exceeds the payload size of a single watermark. An example of meta-data that meets the needs can be as follows:

Another example of meta-data that meets the needs can be as follows:

Referring to FIG. 11, a state transition diagram illustrates a technique for calculating the time code. In order to obtain a time code synchronization, a number of consecutive payloads starting with a payload type "start sync" are followed by payloads of the type "not start sync", one of which totals equal to "r". By using a total of "r" consecutive payloads (each with some time information contained in it), The time synchronization is determined by calculating an anchor time. After the anchor time code is calculated, the time code can be updated by receiving an additional payload including part of the time code information in one of the ways to determine the next time code without receiving another total of "r" consecutive payloads. One technique to achieve this time synchronization is to divide the time code in the continuous payload and the incremental time code in each of the continuous payload. When synchronization is lost, such as due to changing the channel, perform the Get Synchronization procedure. A video display device enters the initial "synchronized" state when it is turned on for the first time.

Referring to FIG. 12, an exemplary structure of a watermark payload is shown. Z indicates the payload type, where Z equal to 1 indicates the start of time synchronization, and Z equal to 0 indicates that time synchronization has not started. S indicates the time synchronization payload bit used to determine the absolute time code. M indicates the time synchronization payload bit used to maintain the time code.

By way of example, the AV rendering device 180 may receive n=7 consecutive watermark payloads, where the first payload has Z=1, and the subsequent watermark payloads have Z=0. The bits corresponding to "SSSS" are captured from the (t-n+1)th watermark payload to the tth watermark payload, and cascaded together to obtain a time code "T _t "at a time position One is represented by 28 bits. The anchor time code "C _t "is also set to "T _t ". “T _t ”can be expressed as SSSS _z=1,t·n+1 …SSSS _z=0,t·l SSSS _z=0,t ; “C _t ”=“T _t ”. In another example, the derived value can be added with a constant (to select a future time) and/or multiplied by a constant (to change the granularity). In yet another alternative example, the derived value is mapped to another value by using a mapping function.

After the initialization synchronization is obtained, the anchor time and the effective load time are updated with each payload. For example, this can be executed as follows: T _t =f(C _t-1 ,MMMM _t )

C _t =g(T _t )

Among them, f represents a mapping function that takes 2 values as input and outputs 1 value; g represents a mapping function that takes 1 value as input and outputs 1 value; / represents integer division toward zero, for example, 7/4 and -7/-4 are truncated to 1, and -7/4 and 7/-4 are truncated to -1. In an example: T _t =C _t-1 +MMMM _t

C _t =T _t

As described above, for every "n" payloads, the bit corresponding to "SSSS" can also be used to determine the anchor time. The anchor time determined using "SSSS" must match the anchor time derived above, and can be used to verify the correctness of the maintained time code.

Because the watermark can span a non-zero time, _{the time position of the time code T t} can be determined by a set of rules, such as, for example, T _t can correspond to a time at the end of the t-th watermark payload.

It should be understood that multiple syntax elements can be combined to form a code. Then, the codes can be mapped by the AV rendering device 180 or using another server to map to different syntax element values. For example, server information (for example, the content(s) and the location of the sending server and/or application layer protocol, etc.) and the time code are combined into a single code. Then, the single code can be mapped to a time position in the uncompressed audiovisual stream and the position of (several) content and sending servers. In this way, a single request for additional information can be made to the server.

A limited number of bits can be used in the time code in this way to permit conflicts in the time code. For example, using 20 bits for the time code allows up to 12 days of independence at a granularity of 1 second. After 12 days, the code space corresponding to the time code will be reused, causing conflicts.

In one example, the watermark payload can be encapsulated in a standard development organization (SDO) private data command using cmdID as the SDO payload. As an example, the watermark payload of FIG. 5 or FIG. 6 can be encapsulated as an SDO payload. A cmdID value of 0x05 may refer to a watermark-based interactive service trigger (Trigger Declared Object (TDO) model). A cmdID value 0x06 Can refer to a watermark-based interactive service trigger (direct execution model). This promotes the reuse of existing sections and re-assembly modules for triggering transportation builds. If necessary, the segmented command can be embedded in the watermark. It is expected that SDO private data, such as the one shown in Figure 13, contains packets as part of SDO_payload(). In some examples, the watermark payload received in this way can be passed to the receiver to handle one of these defined cmdID types and/or modules. Then, if the watermark payload packet needs to be separated into multiple packets, the segmentation and reassembly functionality of that module can be reused, depending on the capacity of the selected watermark scheme in terms of the number of bits.

The parameter type T is a 2-bit field, which indicates whether the instance of the SDOPrivatedata command is part of a segmented variable length command, such as CEA-708 (CEA: "Digital Television (DTV) Closed Captioning, CEA-708- E, Consumer Electronics Association, June 2013") as defined in section 7.1.11.2, and if so, regardless of whether the instance is the first, middle or last segment. The Type field in the SDOPrivateData command is coded as specified in section 7.1.11.2 of CEA-708. pr is a flag, which when set to "1" indicates that the content of the command is confirmed to be a related program. When the flag is set to "0", the content of the command is not so confirmed. Length (L) is an unsigned integer indicating the number of bytes (within the range of 2 to 27) after the header, and it is represented as the group of bits L ₄ to L ₀ in the SDOPrivateData command, where L ₄ is the highest Effective, and L ₀ is the least effective. cid (cmdID) is an 8-bit field that identifies the SDO that has defined the syntax and semantics of the SDO_payload() data structure to be followed. The metadata can be encapsulated in the SDO private data using cmdID as the SDO payload, as shown in Figure 14.

The payload defined in FIGS. 5 and 6 can be encapsulated in a standard development organization (SDO) private data (SDOPrivateData) command using cmdID as the SDO payload. A cmdID value 0x05 and 0x06 can refer to the encapsulation of the payload defined in FIG. 5 and FIG. 6 respectively. This promotion has Segmentation and reuse of reassembled modules built for triggering transportation. If necessary, the segmented command can be embedded in the watermark. It is expected that SDO private data, such as the one shown in Figure 13, contains the payload packet as part of SDO_payload().

The payload defined in FIG. 12 can be encapsulated in a standard development organization (SDO) private data command using cmdID as the SDO payload. A cmdID value 0x05 can refer to the encapsulation of the payload as defined in FIG. 12. This promotes the reuse of existing sections and reassembly modules built for triggering transportation. If necessary, the segmented command can be embedded in the watermark. It is expected that SDO private data, such as the one shown in Figure 13, contains packets as part of SDO_payload().

Referring to FIG. 15, a transmitter of the system can receive one or more messages 530A, 530B, 530C, which are embedded as a watermark in a body (for example, audio and/or video content). One or more messages 530A, 530B, and 530C can be encapsulated in the form of one or more fragments 520A, 520B, and 520C. By way of example, each message can be packaged into a corresponding fragment. By way of example, each message can be packaged in the form of one or more corresponding fragments. By way of example, a message can be divided, each of which corresponds to a message fragment. In some cases, a message of more than one permitted length of one fragment can be propagated to a plurality of corresponding fragments. In some cases, a long message can propagate in a plurality of corresponding fragments. In one example, each of the fragments is encoded to only transmit when the other fragments are not needed for transmission. The transmitter can receive the message fragment(s) and create a series of one or more payloads 510 to be embedded in the body. In some cases, the series may include embedding and/or sending the same message fragment(s) multiple times. In a typical example, a payload is embedded in a unit of the body (for example, an image of video and/or a segment of audio). Each of the payload 510 may include (several) fragmented additional headers and signaling information. For example, the body of a video image and/or an audio segment can be received by a watermark embedder 500, and the watermark is embedded The device 500 embeds the payload 510 therein to generate a marked body.

In an exemplary system, it may be required that if one image in a video segment carries a watermark, all images in the video segment will carry a watermark. Then, a receiver can detect the loss of the image by detecting that the watermark segment is not detected in the current video segment, and an image in the earlier video segment contains a watermark. . A video segment will correspond to a group of consecutive images. In a receiver, a video segment can be identified by the watermark extractor through some external components.

Referring to FIG. 16, a decoder or receiver of the system may receive one or more labeled bodies, such as those provided by the transmitter of FIG. 15. A watermark payload extractor 600 extracts payload(s) from the marked body(s). One or more message fragments can be retrieved 610 from one or more payloads. The result of capturing 610 is a series of one or more message fragments. Each of the one or more message fragments (for example, using the header information of the message fragments) can be appropriately grouped and input into a message reassembly 620A, 620B, 620C. The result of the message reassembly 620A, 620B, 620C is a series of messages 630A, 630B, 630C. Each of the messages 630A, 630B, and 630C can be the result of one or more pieces of reassembly, it can be the result of one or more payloads, and it can be the result of one or more labeled bodies. In an example, the extracted and reassembled messages 1 (630A), ..., message (N-1) (630B), and message N (630C) in Fig. 16 will be the same as messages 1 ( 530A),..., message (N-1) (530B), message N (530C). By way of example, message reassembly may be related to cascading message data contained in a group of message fragments in a specific order.

In one example, a "1X" video watermark (transmission format) delivers 30 valid data bytes per video frame, and a "2X" video watermark (transmission format) system delivers 60 bits per frame group. They are sometimes called 1X system and 2X system respectively.

The message fragments may include type information, which indicates the specific type of information carried in the fragments. For example, the message type may indicate that the information includes a subset of a set of predefined syntax elements (eg, content identifier, media time). In some cases, the values used by some syntax elements can be used to determine the exact subset of the syntax elements contained in the message fragment. For example, the message type may indicate that the information may include a channel identifier (ID). For example, the message type may indicate that the information may include a uniform resource identifier (URI) and a URI type. In another example, the message type may indicate that the information includes a content identifier.

In one example, a message fragment may include a content identifier that may correspond to an entertainment identifier registration (EIDR).

In one example, a message fragment can include a content identifier that can correspond to an Ad-ID used to track advertising assets.

In one example, the message fragment may include length information about variable length information contained therein.

In an example, the watermark payload may include a message.

In one example, the message can be contained in a message fragment.

In one example, a watermark payload can carry one or more message fragments.

In one example, a message fragment may include length information about variable length information contained therein, such as URI, Ad-ID.

In one example, the message fragment may include length information about one of the first variable length information included in the message fragment. The first variable length information may include a fixed length part and a second variable length information. The length of the second variable length information can be derived as the length of the first variable length information minus the length of the fixed length part. The length of the fixed-length part can be derived in any suitable way. For example, the fixed-length part can be based on the message type, the length of the first variable-length information, and the attributes Derived from the length of the syntax element of a fixed-length part contained in the message fragment. In one example, the length of the portion of the second variable length information included in a message fragment can be derived as the length of the first variable length information minus the length of the fixed length portion included in the message fragment. In one example, the fixed-length portion included in a message fragment may be included discontinuously. In one example, the fixed-length portion included in a message fragment can be located on either side of the second variable-length information. In one example, the fixed length part is only partly included in the message fragments. In one example, the fixed length part may not be included in the message fragments.

In some audiovisual environments, the system is expected to have the ability to time-shift audiovisual content. Generally, this refers to recording audiovisual content on a storage medium (such as a hard disk drive), and then watching the recorded performance at a later time, even if the recording has not been completed. In some audiovisual environments, the system can also be expected to have trick mode functions, such as replay of previously recorded content, pause, pause live broadcast, skip to the next segment, skip to the last segment, resume broadcast of live content, and so on. In some audiovisual environments, it is expected that the system can enable user preferences and interactive applications to be covered as needed in an emergency alert (EA) situation. Generally, emergency warnings are important messages from the federal government, state government, or local government that provide emergency information (such as earthquakes, floods, and other national and/or regional events). For these emergency warnings that are usually provided with audiovisual content, it can be expected to cover the graphics displayed on the AV presentation device 180 (such as video overlays or other graphic content), so that the emergency warning message can be easily viewed on the AV presentation device. To one way to present. For example, in the case of a viewer watching video content on an AV presentation device (such as a TV) and another window opened on an AV presentation device that interacts with an interactive TV application, it is desirable to cover both the video content and the interactive TV application. This makes it easy to see emergency warning messages on the AV presentation device. In some cases where the video content is blocked by another application (such as an interactive TV application), it may not be sufficient to only display the emergency warning message in the video content. In some audiovisual environments, in All transmitted broadcast services are received from one of the MVPDs, such as cable, satellite, or Internet Protocol Television (IPTV) operators. To the extent that the broadcast television services are not available to the viewer, the system should be able to enable the receiver To retrieve the missing components of the service via an alternative network (for example, a broadband network connection). Generally, this may include emergency warning messages and their contents, which may not be available to an AV presentation device 180, because a broadcast receiver device 160 (for example, a set-top box) that receives audiovisual content is being used to one of the AV presentation devices High-definition multimedia interface (HDMI), which only provides uncompressed audio and video information to the AV presentation device while omitting other types of components that might otherwise be expected to be provided to the AV presentation device. It should be understood that the AV presentation device may be any device capable of presenting audio and/or video content, and in a multi-screen interactive TV session, the devices may be networked together.

When presenting the broadcast audiovisual content provided by a broadcaster at the same time, any emergency warning message contained in the audiovisual content (such as embedded in one of the watermarks contained in the audio and/or video content), has the watermark capability The receiver 310 and the AV presentation device 180 of the watermark client 320 will detect and respond to the emergency warning signal. However, in the case where the viewer has time-shifted audiovisual content, when the AV presentation device 180 receives the time-shifted audiovisual content together with the watermark containing the emergency alert signal, the AV presentation device 180 will also detect and respond to the emergency alert. signal. Although this delayed detection and response may be appropriate when the time shift has a minimum duration, it can cause a disruption to the viewer’s experience when the time shift does not have a minimum duration. This is because of the urgency. Warnings are usually no longer relevant. By way of example, when the time shift does not have a minimum duration, the AV presentation device 180 with the watermark capable receiver 310 and the watermark client 320 will detect and respond to the emergency warning signal, which can be related to modifying the video The content may also be related to removing any other applications currently presented on the AV presentation device 180, thereby causing an unnecessary destruction of the viewing experience.

Referring to FIG. 17, it is expected that an emergency warning watermark included in the audio and/or video content includes an expiration time value 700. The expiration time value 700 indicates a time value representing a time range corresponding to the emergency alert. For example, the time range can be expressed in minutes in the case of audio and video watermarks, or in seconds in the case of video watermarks. Preferably, the time range is consistent with the text content of the warning message of the broadcaster. For example, for the warning message of a broadcaster of "flood warning from effective to 5pm", the time range until 5PM should be appropriate. It is also expected that the emergency warning watermark included in the audio and/or video content includes an urgency flag 710. The urgency flag 710 sends a signal to inform the device of the degree of timely attention to the expectation of the emergency alert. For example, if the urgency flag 710 is set, all objects displayed on the screen (for example, an interactive TV application running on the AV presentation device 180 (such as a TV)) can be cleared, even if the emergency alert message is still captured. The same applies to the remaining part, so that the emergency warning message can be presented in a more urgent way. For example, if the urgency flag 710 is not set, the object displayed on the screen does not need to be cleared in this timely manner when the remaining part of the emergency alert message is still being captured. In the case where the urgency flag 710 is not set, the emergency warning message can be further analyzed and matched to further confirm its applicability to the current viewer. For example, further processing may include geographic location processing to determine whether the message is applicable to a specific viewer.

It is also expected that the emergency warning watermark included in the audio and/or video content includes a severity indicator 720. For example, the severity indicator 720 may include a series of values such as, for example, extremely severe, severe, moderate, mild, and/or unknown. In this way, the emergency warning signal can provide information about the severity of the emergency.

It is also expected that the emergency warning watermark included in the audio and/or video content includes a certainty indicator 730. For example, the certainty indicator 730 may include a series of values such as, for example, observed, likely, possible, impossible, and/or unknown. Sequentially, emergency warning signals can be provided Information about the certainty of an emergency.

By providing an emergency warning watermark including an expiration time value 700, an urgency flag 710, a severity indicator 720, and/or a certainty indicator 730, the broadcaster can flexibly inform the receiver that it is suitable for the environment Time-sensitive emergency warnings include redistribution via an MVPD broadcast receiving device 160 and/or time-shifted use of audiovisual content. Preferably, the emergency warning signal including the expiration time value 700, the urgency flag 710, the severity indicator 720 and/or the certainty indicator 730 is set in the audio watermark and/or the video watermark of the audiovisual content. In addition, by providing an emergency alert signal including an expiration time value 700, an urgency flag 710, a severity indicator 720, and/or a certainty indicator 730, the receiver can appropriately recognize the time-sensitive alert and provide an appropriate Response. In addition, by providing an emergency warning signal including an expiration time value 700, an urgency flag 710, a severity indicator 720 and/or a certainty indicator 730, it is promoted to reduce unnecessary damage to the viewer’s experience, especially when In the case of time-shifted audiovisual content. In addition, by providing an emergency alert signal including an expiration time value 700, an urgency flag 710, a severity indicator 720, and/or a certainty indicator 730, the viewer is provided with information so that the viewer can appropriately respond to the emergency alert signal.

Referring to FIG. 18, the structure of the watermark message block 800 carried in the payload of a medium-capable watermark technology (such as a video watermark) may include a watermark message identification (wm_message_id) 802, which indicates that the watermark message The type of message sent in block 800, such as an emergency warning signal and a message. A watermark message block 800 may include a complete wm_message() or a fragment of wm_message(). A table 805 can be used to select a set of appropriate watermark decoding and/or processing based on the type of wm_message_id 802. In the case that wm_message_id is 0x05, 806 indicates that the watermark message block 800 includes an emergency warning signal and a message (EA_message()) 808. When indicating that fragmentation is not used wm_message_bytes() 804 contains a complete instance of wm_message() identified by the value of wm_message_id, otherwise wm_message_bytes() contains a fragment of wm_message(). Other structures of the watermark message can be used as desired.

The structure of EA_message()808 can include one or more different data fields. EA_message() 808 may include an EA_Expiry 852, which may be a 26-bit integer value representing the coordinated universal time (UTC) of one minute granularity when the current emergency message ends. An EA_Expiry value of 0 indicates that the end time of the alert is unknown. In the receiving device, the UTC of the current time can be compared with the UTC of EA_Expiry 852. If the UTC of the current time is less than or equal to the UTC of EA_Expiry 852, the emergency alert event is still suitable for processing accordingly. In the case where the value of EA_Expiry 852 is 0 and the warning expiration time is unknown, the AV presentation device 180 can automatically present the warning message. EA_Expiry 852 corresponds to an expiry time value of 700.

EA_message() 808 may include an EA_Urgency 854, which may be a 1-bit value indicating the urgency of an emergency alert event. A value of 1 signal to inform the AV presentation device 180 (such as a TV) to pay attention in time is better. A value of 0 sends a signal to inform the AV presentation device 180 (such as a TV) that the warning is of normal urgency in nature. The AV presentation device 180 can further propagate the signal to one or more accompanying devices that are currently in a networked multi-screen interactive TV session with the AV presentation device 180 (such as a television). EA_Urgency 854 corresponds to the urgency flag 710.

EA_message() 808 may include an EA_message_body_present 856, which may be a 1-bit value indicating the existence of additional information about EA_message 808. EA_message() 808 may include padding bytes to align one of 4 bits 858 reserved. EA_message() 808 may include a conditional statement 860 that sends additional information about EA_message 808.

The additional data may include EA_message_ID 862, which can provide an ID for the emergency alert message.

The additional data may include EA_message_version 864 which can provide a version number for the emergency alert message.

The additional data may include an EA_message_text_length 866, which may be an 8-bit unsigned integer of the length of a given EA_message_text 866.

The additional information may include EA_message_text(8*N)868, which may be a text string of the emergency alert text.

It should be understood that the watermark message and/or any other fields therein can be structured in any suitable manner. It should be understood that fewer and/or greater numbers of bits can be used for signaling. It should be understood that the data is preferably received in the audio and/or video watermark, but can also be obtained in any other way.

Referring to FIG. 19, another example for sending a watermark message in a video may include using EA_Certainty_severity_code 900 to indicate the certainty and/or severity of the corresponding emergency message instead of reserving 4 bits 858.

Referring to Figure 20, a table shows the different combinations of certainty and severity. The certainty 1000 may include a series of values such as, for example, observed, likely, possible, impossible, and/or unknown. In order to represent 5 values with two bits, unknown and impossible can be combined. The severity 1010 may include a series of values such as, for example, extremely severe, severe, moderate, mild, and/or unknown. In order to express 5 values with two bits, unknown and lighter can be combined.

Referring to FIG. 21, another example for sending a watermark message in a video may include using reserved 6 bits 1102 instead of reserved 4 bits 858. In addition, sending watermark messages in the video can include using EA_Expiry 1100 (32 bits) instead of EA_Expiry 852 (26 bits). The 32 bits provide additional granularity to use the second granularity to send UTC timecode more appropriately.

Referring to FIG. 22, another example for sending a watermark message in a video may include using reserved 2 bits 1104 instead of reserved 6 bits 1102. In addition, the watermark message sent in the video can include EA_Certainty_severity_code 900.

Referring to FIG. 23, the structure of the watermark message block 800 included in the watermark suitable for audio content may include an emergency warning flag (EA_flag) 1200 with one bit, which indicates the message sent by the watermark message The type, such as an emergency warning signal. When EA_flag has a value of 0, the watermark message is not an emergency alert type. In this case, the watermark message preferably includes a server_code 1210, which can be a 22-bit code used to query an audio watermark server to obtain further information about the non-emergency warning message. The query may have the following form "http://{server_code}.vp1.tv/atsc30/interval_code", where interval_code 1220 indicates a timeline position corresponding to server_code 1210 in the video content. A trigger 1230 may be provided to indicate that the previous one or more server_code and/or interval_code watermark data should be executed.

When EA_flag 1200 has a value of 1, the watermark message is an emergency alert type. In this case, the watermark message preferably includes server_code 1210, which can be a 22-bit code used to query the audio watermark server to obtain further information about the emergency alert message. The query may have the following form "http://{server_code}.vp1.tv/atsc30/AEA/?zip=zipcode", where the query includes an AV with a receiver 310 with watermark capability and a watermark client 320 The five-digit postal ZIP code of the presentation device 180 enables the server to provide relevant emergency warning information to the AV presentation device. The watermark information can also include EA_Expiry 1240, which can be a 22-bit code used to determine the expiration time. The watermark message may also include EA_Urgency 1250, which indicates the urgency of the watermark message in a manner similar to EA_Urgency 854.

A system that uses audiovisual watermarking may include a requirement: broadcasters using this watermarking technology should ensure that whenever a broadcaster sends an EA event elsewhere in the transmitted signal, the EA flag should be set accordingly To 1, and wm_message_id should be set to 0x05 accordingly.

A system that uses audiovisual watermarking may include a requirement: the broadcaster using this watermarking technology should ensure that whenever a broadcaster sends a signal elsewhere in the transmitted signal, no EA event is valid, then the EA flag should be set accordingly To 0, and wm_message_id should not be set to 0x05 accordingly.

24A shows an exemplary bit stream structure of a video watermark message block (wm_message_block()), where: wm_messgae_id is a value that uniquely identifies the syntax and semantics of the data byte carried in the message block.

wm_message_version is a 4-bit value, which can be incremented if and only when anything in wm_message() changes, and the word wraps back to 0 after the value reaches 15.

fragment_number specifies the number of fragments of the current message minus 1 or 2 bits.

last_fragment is a 2-bit value that specifies the number of fragments used to deliver the last fragment of a complete wm_message(). One of the values "00" in last_fragment indicates that fragmentation is not used (wm_message() contained in it is complete). A value of "01" in last_fragment indicates that wm_message() will be delivered in two parts, a value of "10" indicates that wm_message() will be delivered in three parts, and a value of "11" indicates that wm_message() will be delivered in four parts Deliver it. The pair of the value fragment_number and the value last_fragment can be regarded as the signal "part M of N".

wm_message_bytes()-When the value of last_fragment is 0, wm_message_bytes() can be one of the watermark messages identified by the value of wm_message_id Complete examples. When the value of last_fragment is non-zero, wm_message_bytes() can be a fragment of the watermark message wm_message(). The cascade of all instances of wm_message_bytes() with a given number of wm_message_id and wm_message_version results in a complete wm_message() associated with that wm_message_id. The assembly of wm_message() from one of the one or more wm_message_block() can be as shown in Figure 24C. wm_message_block(i) may indicate the i-th instance, for example, the corresponding wm_message_block() instance whose fragment_number value is equal to i.

Fig. 24B is an exemplary mapping from wm_message_id to wm_message(). It is used to determine the bytes contained in wm_message_byte().

In an example system, fragment_number is constrained to be less than or equal to last_fragment.

Figure 24D shows an exemplary URI message for delivering one of various types of URIs. URI messages can be sent in fragments (for example, the value of last_fragment in the message header can be non-zero). The value of the field uri_type identifies the type of URI. The value of the field uri_strlen will follow the number of characters in the URI_string() field when sending. The field URI_string() is a URI consisting of characters, and its value can be limited to the Internet Engineering Task Force (IETF) Request for Opinion (RFC) 3986 (https://www. ietf.org/rfc/rfc3986.txt) allowable values of the Uniform Resource Identifier (URI). The length of the URI string (URI_string()) can be as given by the value of uri_strlen. After the reassembled character string, if the URI is sent in the fragment, it will be constrained to become a valid URI per RFC 3986.

In one example, a pre-determined uri_type value can indicate that the URI message includes the URL of a Common Alerting Protocol (CAP) message. For example, according to the code given in Figure 24E, the value 0x04 of uri_type indicates that the URI message contains the URL of a CAP message. The value 0x04 is only an example. instead Alternatively, another value can be used to indicate that the URI message contains the URL of a CAP message. For example, a value of 0x09 or 0xf0 can be used.

In one example, some URI types (or all URI types) are associated (for example, as indicated by the uri_type field), for example, an expiration field is sent in the URI message. The expiration field may contain date and time information after which the URI is no longer valid. The expiration date and time information can be from the low-order 32 bits of the 42-bit second field of the Precision Time Protocol (PTP) time stamp defined in the Institute of Electrical and Electronics Engineers (IEEE) 1588 (2008).

In one example, it is associated with some URI types (or all URI types), for example, sending a valid_from field in the URI message. The valid_from field can contain date and time information for which the URI has not been valid before. The valid_from date and time information can be the low-order 32 bits from the 42-bit second field of the PTP time stamp.

FIG. 25A shows a flowchart of an exemplary transmission of a field based on a key field value. A key field value of 2500 is transmitted in the watermark. The length of a field 2505 can be determined based on the transmitted key field value. A number of bits corresponding to the determined length can be used to transmit the field value 2515. In one example, the watermark is a video watermark. In one example, the key field is the watermark message id. In one example, the fields whose length is determined are the current number of fragments and the last number of fragments. In one example, the length of the current number of fragments is the same as the length of the last number of fragments.

FIG. 25B shows a flowchart showing exemplary reception of a field based on a key field value. Receive a key field 2550 in the watermark. Determine the length of a field based on the key field value 2555. Analyze the number of bits corresponding to the determined length from the watermark receiving field 2565. In one example, the watermark is a video watermark. In one example, the key field is the watermark message id. In one example, the fields whose length is determined are the current number of fragments and the last number of fragments. In one example, the length of the current number of fragments is the same as the length of the last number of fragments.

In one example, a derived CAP message can be sent in the video watermark (for example, derived_emergency_alert_message() in FIG. 26B). Once the exported CAP message can contain all the information or a subgroup of the information from the original CAP message.

In one example, the original CAP message can be received from an external warning issuance system.

In one example, the original CAP message can be formatted as specified in OASIS: "Common Alert Agreement." OASIS: The current version of the "Common Alerting Protocol" is 1.2 and is described at http://docs.oasis-open.org/emergency/cap/v1.2/CAP-v1.2-os.html.

In one example, a derived CAP message may include information other than the original CAP message. In one example, once the CAP message is derived, specific information can be excluded from the original CAP message.

In one example, once the CAP message is derived, the format specified in OASIS: "Common Alerting Protocol" as described above can be used.

In one example, even when the original CAP message does not exist, the derived CAP message can be sent in the video watermark. For example, a station detects a local emergency before the international warning system.

In one example, even when there is an original CAP message, the derived CAP message may not be sent in the video watermark.

In one example, the derived CAP message can be sent over the air (download).

In one example, the exported CAP message can correspond to a warning/alert message.

In an exemplary system, the length of a specific field in the video watermark message block can be derived based on the value of some other fields in the video watermark message.

In an exemplary system, the maximum value of some fields in the video watermark message block can be derived based on the values of some other fields in the video watermark message block.

In one example, the derived CAP message may be an emergency alert message that has not been formatted as a CAP message.

FIG. 26A shows an exemplary bitstream structure of a video watermark message block (wm_message_block()), where the length and maximum value of the fields fragment_number and last_fragment depend on the value of the wm_message_id field. This allows a message to be fragmented into more than one specific single fixed number of fragments. For example, when using the bit stream structure shown in FIG. 24A, a message can be fragmented into at most four fragments. Each fragment may have limited data capacity. Therefore, when the structure shown in FIG. 24A is used, the total data capacity is limited. In a comparison that allows a larger number of bits for the fragment_number and last_fragment fields, a larger number of fragments are allowed for the message. Therefore, larger messages can be sent. While sending smaller messages, avoid the extra cost of sending a higher number of bits for the fragment_number and last_fragment fields. Therefore, message-dependent fragment field signaling provides a good compromise between flexibility and bit saving.

The following is an exemplary semantics of one of the various fields in Figure 26A: wm_message_id is a value that uniquely identifies the syntax and semantics of the data bytes carried in the message block.

wm_message_version is a 4-bit value, which can be incremented if and only when anything in wm_message() changes, and the word wraps back to 0 after the value reaches 15.

fragment_number specifies the number of fragments of the current message minus one value. The number of fragments field has a length equal to num_fragment_bits bits, as specified in Table A below.

last_fragment is a value specifying the number of fragments used to deliver the last fragment of a complete wm_message(). The last_fragment field has a length equal to the num_fragment_bits bits specified in Table A below. For the last_fragment field, the allowed unsigned integer value range is 0 to (2 ^{num_fragment_bits} -1). A string of "0" of length num_fragment_bits in last_fragment indicates that fragmentation is not used (wm_message() contained in it is complete).

wm_message_bytes()-When the value of last_fragment is 0, wm_message_bytes() can be a complete example of the watermark message identified by the value of wm_message_id. When the value of last_fragment is non-zero, wm_message_bytes() can be a fragment of the other watermark message wm_message(). The cascade of all instances of wm_message_bytes() with a given number of wm_message_id and wm_message_version results in a complete wm_message() associated with that wm_message_id. The assembly of wm_message() from one of the one or more wm_message_block() instances can be as shown in Figure 26C. wm_message_block(i) may indicate the i-th instance, for example, the corresponding wm_message_block() instance whose fragment_number value is equal to i.

In one example, wm_message_id not only defines the num_fragment_bits (in bits) of some fields, but also defines the watermark message block The size of (wm_message_block()) is the length of the reserved field that is a multiple of 8 bits.

In one example, a decimal value of 0 in the last_fragment field indicates that fragmentation is not used (the information contained in it is complete). One of the decimal value "xyz" in last_fragment indicates that the message will be delivered in the xyz part.

In one example, a range of wm_message_id values will correspond to one of the maximum number of allowed message fragments.

In an example, a range of wm_message_id values will correspond to a value (in bits) of num-fragment_bits. In one example, a range of wm_message_id values will define the length of the reserved field so that the size of the watermark message block (wm_message_block()) is a multiple of 8 bits.

A system using audiovisual watermarks can be handled by the broadcaster, which includes setting the expiration time to 0 to reduce the need to determine the appropriate duration and/or end time.

A system that uses audiovisual watermarks can determine the expiration time based on other elements included in the audiovisual content or otherwise available to the display device.

In addition, each functional block or various features of the base station device and the terminal device (video decoder and video encoder) in each of the foregoing examples can be implemented or executed by a circuit. The circuit is usually an integrated circuit or a plurality of An integrated circuit. Circuits designed to perform the functions described in this manual include a general-purpose processor, a digital signal processor (DSP), a dedicated or general-purpose integrated circuit (ASIC), a programmable gate array (FPGA), or Other programmable logic devices, discrete gate or transistor logic, or a discrete hardware component, or a combination thereof. The general-purpose processor may be a microprocessor, or alternatively, the processor may be a conventional processor, a controller, a microcontroller, or a state machine. The general-purpose processor or the circuits described above can be configured by a digital circuit, or can be configured by an analog circuit. In addition, when due to advances in semiconductor technology When a technology of manufacturing an integrated circuit that replaces one of the current integrated circuits appears, the integrated circuit manufactured by this technology can also be used.

The program running on the device according to the present invention can be a program that causes a computer to function by controlling a central processing unit (CPU) to implement the embodiments of the present invention. Programs or information processed by such programs can be temporarily stored in a volatile memory, such as a random access memory (RAM) or a hard disk drive (HDD) or a non-volatile memory, such as a flash Memory or other memory storage system.

Programs that can implement the functions of the present invention can be recorded on a computer-readable recording medium. These functions can be implemented by causing a computer system to read the programs recorded on the recording medium and execute the programs. The "computer system" may be a computer system embedded in the device, and may include an operating system or hardware, such as peripheral devices. The "computer-readable recording medium" can be a semiconductor recording medium, an optical recording medium, a magnetic recording medium, a recording medium that dynamically stores a program in a short period of time, and any other recording medium readable by a computer.

The features or functional blocks of the devices used in the foregoing embodiments can be implemented or executed in a circuit, for example, an integrated circuit or a plurality of integrated circuits. Circuits designed to perform the functions described in this specification can include a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable gate array (FPGA), or other Programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor may be a microprocessor, or may be any conventional processor, controller, microcontroller, or state machine. The above-mentioned circuit may be a digital circuit or may be an analog circuit. In a case where an advanced semiconductor technology is introduced to replace one of the current integrated circuit technologies, one of the new integrated circuit technologies can be used to implement the present invention.

It should be understood that the scope of the patent application is not limited to the precise configuration and components shown above. Available at Various modifications, changes and changes are made in the configuration, operation and details of the systems, methods and equipment described in this article without departing from the scope of the patent application.

100‧‧‧Content source

120‧‧‧Content Recognition Service Providing Server

130‧‧‧Multi-channel video program distributor

140‧‧‧Enhanced service information provision server

160‧‧‧Broadcast receiver

170‧‧‧Internet

180‧‧‧AV presentation device

Claims (2)

A method for receiving a watermark message, the watermark message containing at least one watermark message block, the method comprising the following steps: (a) receiving the at least one watermark message block; (b) from the at least one watermark At least one of the message blocks determines a watermark message id; (c) recovering the watermark message from the at least one watermark message block; and the watermark message id indicates a type of the watermark message; the watermark message The type includes content identifier, media time, and URI; if the value of the watermark message id is 0x03, the type of the watermark message is URI. A device including a processor configured to receive a watermark message, the watermark message containing at least one watermark message block, comprising: (a) the device receives the at least one watermark message block; (b) The device determines a watermark message id from at least one of the at least one watermark message block; (c) the device restores the watermark message from the at least one watermark message block; and the watermark message id indicates the watermark A type of message; the type of the watermark message includes content identifier, media time, and URI; if the value of the watermark message id is 0x03, the type of the watermark message is URI.

Images