CN100459682C - Closed caption tagging system - Google Patents

Abstract

A closed caption tagging system provides a mechanism for inserting tags into an audio or video television broadcast stream prior to or at the time of transmission. The tags contain command and control information that the receiver translates and acts upon. The receiver receives the broadcast stream and detects and processes the tags within the broadcast stream which is stored on a storage device that resides on the receiver. Program material from the broadcast stream is played back to the viewer from the storage device. The receiver performs the appropriate actions in response to the tags. Tags indicate the start and end points of a program segment. The receiver skips over a program segment during playback in response to the viewer pressing a button on a remote input device or it automatically skips over program segments depending on the viewer's preferences. Program segments such as commercials are automatically replaced by the receiver with new program segments that are selected based on various criteria. Menus, icons, and Web pages are displayed to the viewer based on information included in a tag. The viewer interacts with the menu, icon, or Web page through an input device with the receiver performing the associated actions. If a menu or actions requires that the viewer exits from the playback of the program material, then the receiver saves the exit point and returns the viewer back to the same exit point when the viewer has completed the interaction session. Menus and icons are used to generate leads, generate sales, and schedule the recording of programs. A one-touch recording option is provided. An icon is displayed to the viewer telling the viewer that an advertised program is available for recording at a future time. The viewer presses a single button on an input device causing the receiver to schedule the program for recording.

Description

Apparatus and method for scheduling television program recording via advertisements in a television broadcast data stream

This application is a divisional application of a patent application with an application date of September 20, 2000, an application number of 00815914.9 (the international application number is PCT/US 00/25847), and a titled "system for adding labels to closed captions".

technical field

The invention relates to multimedia audio-visual data flow. More specifically, the present invention relates to tagging of multimedia audio-visual television broadcast data streams.

Background technique

Video cassette recorders (VCRs) have changed the lives of television viewers around the world. VCRs can provide time-flexible TV programs to match their lifestyles.

Viewers can store television programs on tape using a VCR. VCRs allow viewers to play, rewind, fast-forward, and pause stored program content. These features allow viewers to pause an ongoing program at any time they prefer; fast-forward to skip unwanted content or commercials; and replay favorite segments. However, VCR cannot record video and play information content at the same time.

Recently, digital video recorders (DVRs) have entered the commercial market. DVRs allow viewers to store television programs on hard disks. In this way, viewers are freed from the confines of tape. Viewers can pause, rewind and fast-forward live broadcasts. However, DVRs do more than just record shows.

Enabling programs to be stored locally in digital form provides program planners with more options than ever before. Today, advertisements (commercials) can be changed dynamically and targeted specifically to specific viewers according to viewing habits. Commercials can be stored locally in the viewer's DVR and presented at any time.

DVR enables viewers to broadcast interactive programs. In general, previews of upcoming programs are aired during normal program broadcasts. The viewer must then remember the show date, time and channel of the show in order to record or watch the show. DVRs allow viewers to immediately schedule recordings of the show.

The only downside: Current DVRs cannot interact with viewers at this level. There is currently no way to tell the DVR that a commercial is immediately followed by a particular program or other advertisement. Additionally, there is currently no way to inform the DVR that commercials are replaceable.

It would be beneficial to provide a closed captioning system that would allow program providers to transmit frame specific data over the broadcast medium. It would also be beneficial to provide a closed captioning system that allows receivers to dynamically interact with viewers and configure themselves based on program content.

Contents of the invention

The present invention provides a closed caption marking system. The present invention enables content providers to transmit frame-specific data and commands integrated into an AV broadcast data stream via a broadcast medium. In addition, the present invention enables the receiver to interact with the viewer in a dynamic manner and to configure itself according to the content of the audio-visual data stream.

The preferred embodiment of the present invention provides a mechanism for inserting tags into audio or video television broadcast data streams. Tags are inserted into the broadcast data stream before or at the time of broadcast. These tags contain command and control information that the receiver interprets and acts upon.

The foregoing receiver receives the broadcast data stream, and detects and processes tags in the broadcast data stream. The broadcast data stream is stored on a storage device on the receiver. Program material from the broadcast data stream is presented to the viewer from the storage device.

During the tag processing phase, the receiver will take appropriate actions in response to the tags. These tags can provide content providers or system administrators with a large amount of flexible use space, so as to construct infinite operation functions.

Labels indicate the beginning and end of program segments. When the viewer presses a button on the remote control input device, the receiver responds by skipping a segment of the program during the broadcast. The receiver can also automatically skip program segments according to the viewer's preference.

The receiver automatically replaces program segments, such as commercials, with new program segments. New program segments are selected according to different criteria, such as location, time of day, program material, viewer's viewing habits, viewer's preference for programs, or viewer's personal data. The new program segment is stored at the local end or the remote end of the receiver.

Menus, icons and web pages are presented to the viewer based on the information contained in the tags. Interaction between viewers and menus, icons or web pages is through input devices. The receiver performs actions associated with menus, icons or web pages and viewer input. If a menu or action requires the viewer to exit the broadcast of the program material, the receiver will retain the exit point and return the viewer to the same exit point again after the viewer completes the interaction process.

Menus and icons are used to generate product introductions, merchandising, and to schedule recordings. One-touch recording options are available here. Viewers can see an icon, which informs that a commercial program can be recorded at a certain time in the future. Viewers press a single button on the input device to schedule the receiver to record the program. The receiver can also record the program currently in the broadcast data stream in the storage device according to the information in a certain tag.

Tags can be used to index program material. This allows the viewer to jump to a specific index within the program.

Other aspects and advantages of the present invention can be understood with reference to the following descriptions of the principles of the present invention and accompanying drawings.

Description of drawings

Figure 1 is a high-level schematic block diagram of a preferred embodiment of the present invention;

Figure 2 is a schematic block diagram of a preferred embodiment of the present invention utilizing multiple input and output modules;

3 is a schematic diagram of the moving picture compression standard (MPEG) data stream and its video and audio components of the present invention;

4 is a schematic block diagram of a parser and four direct memory access (DMA) input engines included in the media switch of the present invention;

Figure 5 is a schematic block diagram of the components of the packetized elementary stream (PES) buffer of the present invention;

6 is a schematic diagram of the PES buffer structure established according to the analysis piece of the media switch output ring buffer;

Fig. 7 is a schematic block diagram of the media switch of the present invention and various components connected thereto;

Figure 8 is a high level overview block diagram of the program logic of the present invention;

Fig. 9 is a class hierarchical block diagram of the program logic of the present invention;

Fig. 10 is a schematic block diagram of a preferred embodiment of the clipping cache of the present invention;

11 is a schematic block diagram of a preferred embodiment of the analog broadcast studio video mixer of the present invention;

12 is a schematic block diagram of a closed caption parser of the present invention;

Figure 13 is a high-level schematic block diagram of a preferred embodiment of the present invention utilizing a VCR as an integrator of the present invention;

Fig. 14 is a schematic block diagram of a preferred embodiment of inserting a label into a video data stream according to the present invention;

Fig. 15 is a schematic block diagram of a preferred embodiment in which tags are inserted into a video data stream based on a server in the present invention;

Figure 16 is a diagram illustrating the user interface used by the present invention to insert tags into a video data stream;

Fig. 17 is a screen illustrating a prompt icon according to the present invention, wherein the image is displayed in the lower left corner of the screen;

Fig. 18 is a schematic block diagram of the transmission path of the video data stream of the present invention;

Fig. 19 is a schematic block diagram of marking the starting point and end point of a program segment of a certain video data stream and playing a new program segment according to the present invention;

20 is a schematic block diagram of a preferred embodiment of interpreting tags inserted into a video data stream according to the present invention;

Figure 21 is a diagram illustrating a screen displaying the program recording options of the present invention;

Figure 22 is a viewer remote control device illustrating the present invention;

Figure 23 is a schematic block diagram of a series of screens that generate product presentations and merchandising.

Detailed ways

The present invention is implemented in a closed captioning system. The system constructed in accordance with the present invention allows content providers to transmit frame-specific data and commands integrated in an audiovisual television broadcast data stream via the broadcast medium. The present invention also allows the receiver to interact with the audience in a dynamic manner, and to perform self-setting according to the content of the audio-visual data stream.

A preferred embodiment of the present invention provides a marking and interpreting system, which enables the content provider to mark the video and audio data stream transmitted through the television broadcasting medium with a specific picture. The receiver interprets and acts upon the tags embedded within the received data stream. Tag data enables receivers to dynamically interact with viewers through menus and action icons. Tags also provide dynamic configuration of receivers.

Please refer to Figure 1. A preferred embodiment of the present invention includes an input unit 101 , a media switch 102 , and an output unit 103 . The input unit 101 can receive various forms of television input data streams—such as the National Television Standards Committee (NTSC) standard or Phase Alternation Line (PAL) broadcasts, as well as digital data streams—such as Digital Satellite System (DSS), digital Broadcasting Service (DBS) or Advanced Television Standards Committee (ATSC) standards. DBS, DSS, and ATSC are based on standards such as Motion Picture Video Compression Standard 2 (MPEG2) and MPEG2 Transport. The MPEG2 transmission standard is a standard for formatting digital data streams from television source transmitters so that television receivers can unpack the incoming data streams to find programs within the multiplexed signal. The input section 101 generates an MPEG data stream. MPEG2 transport multiplexing supports multiple programs within the same broadcast channel where video and audio feeds and private data reside. The input section 101 can tune the channel to a particular program, extract a particular MPEG program from it, and provide that program to the rest of the system. The analog TV signal is encoded into a similar MPEG format using separate video and audio encoders, so that the rest of the system does not know how the signal was obtained. Various standard methods can be used to modulate information into the vertical blanking interval (VBI) of an analog television signal; for example, the North American Broadcast Teletext Standard (NABTS) can be used to modulate information into scans 10 to 20 of an NTSC signal line of sight, while the (US) Federal Communications Commission (FCC) regulates Line 21 for closed captioning (CC) and Extended Data Service (EDS). Such signals are decoded by the input and transmitted to other components as if they were transmitted via an MPEG2 dedicated data channel.

The media switch 102 is responsible for coordinating the operations among the microprocessor CPU (Central Processing Unit) 106 , the hard disk or storage device 105 and the memory 104 . The input data stream is converted to an MPEG data stream and transmitted to the media switch 102 . The media switch 102 temporarily stores the MPEG data stream in memory. If the user watches live TV, the media switch 102 performs two tasks: the data stream is sent to the output unit 103 , and the data stream is written to the hard disk or storage device 105 at the same time.

Output section 103 receives an incoming MPEG data stream and generates an analog television signal according to NTSC, PAL or other desired television standard. The output unit 103 includes an MPEG decoder, an on-screen display (OSD) generator, an analog TV encoder, and an audio logic circuit. An OSD generator enables program logic to provide an image superimposed on the final analog TV signal. In addition, the input can modulate information provided by the program logic onto the VBI of the output signal using a variety of standard formats, including NABTS, CC, or EDS.

Please refer to Figure 2. The present invention can be easily expanded to accommodate multiple input sections (tuners) 201, 202, 203, 204, and each input section can be adjusted to different types of inputs. At the same time, multiple output modules (decoders) 206, 207, 208, 209 can also be added. Special effects—such as picture-in-picture—can also be implemented using multiple decoders. When the user is watching a certain TV program, the media switch 205 can record another program. This means that while a data stream is being stored on disk, another data stream can be fetched from disk.

Please refer to Figure 3. The incoming MPEG data stream 301 has interleaved video 302 , 305 , 306 , and audio 303 , 304 , 307 data segments. These data segments must be segmented and recombined to form separate video 308 and audio 309 data streams or cached information. The above is necessary because a separate decoder is used to convert the MPEG data segments into their original audio or video components. This individual transmission process must generate time-series information so that the decoder can be properly synchronized for accurate signal playback.

Media switches enable program logic to associate the correct time-series information with each data segment, possibly embedding that information directly within the data stream. The time-series information for each data segment is called a timestamp. These time stamps start at zero and increase monotonically every time the system starts. This allows the present invention to find any particular point in any particular segment of video data. For example, if the system needs to read five seconds into an already cached incoming continuous video stream, the system need only initiate a forward read and look for the appropriate time stamp.

Binary searches can be performed on stored files to navigate into the data stream. Storing each data stream as a series of fixed-size data segments can increase the speed of the binary search because of uniform time stamps. If the user wants to start in the middle of the program, the system will continue to perform binary searches on the stored data segments until the system finds an appropriate point, at which point the desired result is obtained with the least amount of information. If the signal is stored as an MPEG data stream, the data stream must be parsed linearly from the starting point to find the desired position.

Please refer to Figure 4. The media switch contains four incoming direct memory access (DMA) engines 402, 403, 404, 405, and each DMA engine has an associated buffer 410, 411, 412, 413. Conceptually, each DMA engine has a pointer 406 , pointer's boundary 407 , next pointer 408 , and next pointer's boundary 409 . Each DMA pertains to a particular type of information; for example, video information 402 , audio information 403 , and parsing events 405 . The buffers 410, 411, 412, 413 are circular and collect specific information. The DMA engine increments pointers 406 to the attached buffers until boundary 407 is reached, then loads the next pointer 408 and boundary 409 . Setting the values of the pointer 406 and the next pointer 408 and their corresponding boundaries to be the same, a ring buffer can be established. The next pointer 408 can be set to a different address to provide vectored DMA.

The input data stream will pass through the parser 401 . Parser 401 finds MPEG events by parsing the data stream, which indicate the beginning of video, audio or private data segments. For example, when the parser 401 finds a certain video event, it will direct the data flow to the video DMA engine 402 . The parser 401 caches the data and imports it into the video buffer 410 through the video DMA engine 402 in a DMA manner. At the same time, the parser 401 directs an event to the event DMA engine 405 , and the DMA engine 405 generates an event in the event buffer 413 . When the parser 401 encounters an audio event, it redirects the byte stream to the audio DMA engine 403 and generates an event in the event buffer 413 . Likewise, when the parser encounters a private data event, it redirects the byte stream to the private data DMA engine 404 and directs an event to the event buffer 413 . When an event is stored in the event buffer, the media switch notifies the program logic through the interrupt mechanism.

Please refer to Figures 4 and 5. Parser 401 fills event buffer 413 with events. Each event 501 in the event buffer has fields for offset 502 , event type 503 and time stamp 504 . After the events are stored in the buffer, the parser 401 will provide the type and offset of each event. For example, when an audio event occurs, the event type column will be set to audio event, and the offset will indicate the position in the audio buffer 411 . The program logic will know where the start of the audio buffer 411 is, and add offsets to find events in the data stream. The address offset 502 tells the program logic where the next event will occur, but not where the end point will occur. The previous event is stored in a cache so that the end of the current event and the length of the data segment can be found.

Please refer to Figures 5 and 6. When the program logic is interrupted by the media switch 601 , the program logic reads the events accumulated in the event buffer 602 . Program logic may generate a sequence of logical data segments 603 corresponding to parsed MPEG data segments 615 from these events. The program logic converts the offset 502 into the actual address 610 of each data segment, and uses the previous cache event to record the event length 609 . If the data stream results from the encoding of an analog signal, it will not contain a Program Time Stamp (PTS) value. The PTS value is used by the decoder to properly provide the resulting result. Therefore, the program logic uses the generated time stamp 504 to calculate the analog PTS value for each data segment, and stores this value in the logical data segment time stamp 607 . In terms of digital TV broadcasting data streams, the PTS value has been programmed into the data stream. The program logic extracts this information and stores it in the logical data segment timestamp 607 .

The program logic continues to collect logical data segments 603 until it reaches the fixed capacity of the buffer. When the capacity limit is reached, the program logic creates a new buffer—called a Packet Elementary Stream (PES) 605 buffer, which contains sequential logical data segments 603 and accompanying control information. Each logical data segment is directed to the ring buffer 604 - eg the video buffer 613 filled by the media switch 601 . The new buffer is then passed to other logic elements, which can further process the data stream in the buffer in some way, such as decoding it or writing it to a storage medium. Therefore, the processor does not copy the MPEG data from one location in memory to another. The result is a more cost-effective design because the required memory bandwidth and processor bandwidth are reduced.

A unique property of the conversion of an MPEG data stream into a PES buffer is that the data accompanying the logical data segments need not be present in the buffer itself as described above. When the PES buffer is written into the storage medium, these logical data segments are written into the storage medium where they appear in logical order. The above process results in the components of the data stream being collected in a single stream data linear buffer on the storage medium, whether the components of the data stream are located in video, audio or dedicated data ring buffers. The buffer is read back from the storage medium in a single transfer, and the logical segment information is updated to correspond to the actual location in the buffer 606 . Higher-level program logic is unaware of the transformation process, as it only deals with logical data segments; thus, streaming data can be easily processed without going through the CPU's location in DRAM at all copy data between.

One of the unique technical features of media switches is their ability to handle high-speed data in an efficient and cost-effective manner. A media switch can perform the following functions on a low-cost platform: record video and audio data, send video and audio data, send video and audio data to disk, and retrieve video and audio data from disk. Typically, media switches operate asynchronously and independently of the microprocessor's CPU and use their DMA capabilities to move large amounts of data with minimal CPU intervention.

Please refer to FIG. 7 , the input end of the media switch 701 is connected to the MPEG encoder 703 . Also included are circuits dedicated to providing MPEG audio data 704 and VBI data 702 to the media switch 701 . If digital TV signals are processed, the MPEG encoder 703 is replaced by an MPEG2 transport demultiplexer, while the MPEG audio encoder 704 and VBI decoder 702 are omitted. The splitter multiplexes the extracted audio, video and dedicated data channel streams through the video input media switch port.

Parser 705 is used to parse the input stream from MPEG encoder 703, audio encoder 704 and VBI decoder 702, or in the case of digital television broadcast streams, the input stream from the transport demultiplexer . The parser 705 detects the beginning of all important events, frames, and sequence headers in the video or audio data stream, all of which are necessary for the program logic to play the data stream correctly and execute special Functions such as: fast forward, rewind, play, pause, fast/slow play, index, and fast/slow reverse play.

When the parser 705 recognizes a segment of video, audio data, or a given private data, it places a tag 707 in the FIFO buffer 706 . The DMA 709 controls when these tags are removed. The tag 707 and the segment's DMA address are put into the event queue 708 . Frame type information whether it is video I-frame, video B-frame, video P-frame, video packet elementary data stream, audio packet elementary data stream, sequence header, or audio frame or private data packet is stored together with the relevant ring buffer The offset in the zone is stored in the event queue 708, and various information is stored in the ring buffer. After the program logic is transferred to the DRAM 714, the program logic running in the CPU 713 checks the ring buffer for events.

The media switch 701 has a data bus 711 connected to a CPU 713 and a DRAM 714. Address bus 712 is also commonly used between media switch 701, CPU 713 and DRAM 714. A hard disk or storage device 710 is connected to one port of the media switch 701 . The media switch 701 outputs the data stream to an MPEG video decoder 715 and a separate audio decoder 717 . The signal from the audio decoder 717 includes audio prompts, which are generated by the system in response to user commands on the remote control or other internal events. The decoded audio output by the MPEG decoder is digitally mixed 718 with the separate audio signal. The finally obtained signal includes video signal, audio signal and on-screen display, and will be sent to the TV 716 .

The media switch 701 takes the 8-bit data and sends it to disk, while another data stream is extracted from the disk and sent to the MPEG decoder 715 . All DMA engines described above can operate at the same time. The media switch 701 can be implemented in hardware by using a Field Programmable Gate Array (Field Programmable Gate Array, FPGA), an Application Specific Integrated Circuit (ASIC), or a discrete logic circuit.

The program logic only needs to search the circular event buffer in the DRAM 714 to identify the start position and type of each frame, instead of parsing a large amount of data streams to find the possible start position of each frame. This kind of processing method can save a large amount of CPU processing power, and can keep the real-time requirement of CPU 713 in the minimum state. CPU 713 does not need to be very fast at any time. The media switch 701 provides time to the CPU 713 to complete work as much as possible. The parser mechanism 705 and the event queue 708 free the CPU 713 from analyzing audio data, video data and buffers, and the real-time nature of the data stream in a low-cost manner. This approach also allows the use of very low clock rate buses in a CPU environment and works with less expensive memory than other approaches.

The CPU 713 has the function of queuing up a DMA transfer, and can set up the next DMA transfer while it is idle. This allows the CPU 713 to have a large amount of time to serve the DMA controller 709. Due to the larger waiting time allowed, the CPU 713 can respond to DMA interrupts within a larger time window. An MPEG stream, whether extracted from an MPEG2 transmission or encoded from an analog television signal, is usually encoded using a technique called Variable Bit Rate (VBR) encoding. This technique changes the amount of data required to represent a sequence of images by the amount of movement between those images. This technique can greatly reduce the bandwidth of the signal, and fast-moving sequences (such as basketball games) can be encoded with a larger bandwidth. For example, the Hughes DTV satellite system encodes at a rate of 1 to 10 megabits per second across the required bandwidth, varying from frame to frame. Without this structure, it would be difficult for any computer system to keep up with such rapidly changing data rates.

Referring to FIG. 8 , the program logic in the CPU has three conceptual components: source 801 , conversion 802 and sink 803 . Source 801 generates data buffers. Transformer 802 processes data buffers, while sink 803 consumes data buffers. Transforms are responsible for allocating and scheduling queues of data buffers on which they operate. Buffers are allocated to the data source as if "empty" and returned as "full". The buffer is then queued and provided to the sink in a "full" state, and the sink will return it with a "empty" state buffer.

Source 801 receives data from an encoder (eg, a digital satellite receiver). The source obtains the buffer of the data from the downconversion, seals the data into the buffer, and then pushes the buffer into the transmission channel in the manner described above. The source object 801 knows nothing about other conditions of the system. The sink 803 consumes the buffer, fetches the buffer from the upconversion, transmits the data to the decoder, and releases the buffer for reuse.

Two transformations 802 are used here: spatial and temporal transformations. Examples of spatial transformations are image convolution or compression/decompression as buffered data passes through. Time transformations are used when there is no representable relationship between buffered data entering the system and buffered data leaving the system. This transformation is to write the buffered data to a file 804 located on the storage medium. The buffered data is fetched later and sent along the transmission channel, properly sequenced in the data stream.

Referring to Figure 9, it shows the C++ class hierarchy derived from the program logic. TiVo Media Core (Tmk) 904, 908, and 913 switch operating system core. This core provides operational functions such as memory allocation, synchronization and threading. TmkCore 904, 908 and 913 take the memory from the media core as an object, which can be provided to the operator for constructing a new object or deleting an object. Each object (source 901, transformation 902 and sink 903) has been defined as multi-threaded and can operate in parallel processing.

The TmkPipeline classes 905, 909 and 914 are responsible for controlling the flow through the system. The transmission channel is directed to the next transmission channel from the source 901 to the sink 903 in the flow. For example, if the transfer channel is to be paused, an event called "pause" is sent to the first object in the transfer channel. The event is forwarded all the way down the transmission channel to the next object. This process is done in an asynchronous manner for the data passing through the transport channel. Therefore, in applications such as telephony, the flow control of the MPEG data stream is asynchronous and separate from the data stream itself. This approach allows the use of simple logic designs while being capable enough to support the previously described technical features, including pause, rewind, fast forward, and other functions. Furthermore, this architecture allows for fast and efficient switching between stream sources, since buffered data is simply discarded and the decoder is reset with a single event, after which data from the new stream will pass through the transmission channel. This is required, for example, when the input captures a channel change, or when switching between a live broadcast signal from the input and a stored data stream.

The source object 901 is TmkSource 906 and the conversion object 902 is TmkXfrm 910. The above are intermediate classes that define the standard behavior in the transport channel. Conceptually, it swaps buffers along the transport channel. The source object 901 takes data out from the entity data source (such as a media switch) and stores it in the PES buffer. In order to obtain the buffer, the source object 901 requests a buffer (allocEmptyBuf) from the downstream object in the transport channel. The source object 901 will be closed until sufficient memory is available. This means that the transmission channel adjusts itself; it has automatic flow control. When the source object 901 is full of the buffer, it returns the buffer to the transformation 902 through the pushFullBuf function.

The sink 903 is also subject to flow control. The sink will call nextFullBuf to inform the switch 902 that it is ready for the next full buffer. This action can block the sink 903 until a buffer is ready. When the sink 903 completes a buffer (ie, it has consumed all the data in the buffer), it calls releaseEmptyBuf. ReleaseEmptyBuf returns the buffer to transition 902 . For example, transformation 902 may then hand the buffer back to source object 901 for refilling. Besides having the advantage of automatic flow control, this approach also limits the amount of buffer-specific memory by allowing a fixed configuration of buffers to be forced. This is an important feature to achieve cost-effectiveness in a limited DRAM environment.

The MediaSwitch class 909 calls the allocEmptyBuf method of the Tmk clip cache 912 object and receives the PES buffer from it. The ring buffer is then left in the media switch hardware and a PES buffer is generated. The MediaSwitch class 909 fills the buffer and pushes it back into the Tmk Clip Cache 912 object.

The Tmk clip cache 912 maintains a cache file 918 in the storage medium. The Tmk Clip Cache 912 will also keep two pointers in this cache: the advance pointer 919, which shows where the next buffer from the source 901 is to be inserted; and the present pointer 920, which points to the one currently in use buffer.

The buffer currently pointed to by the pointer is processed by the sail decoder class 916 . Sail decoder class 916 communicates with decoder 921 in hardware. Decoder 921 produces a decoded television signal which is in turn encoded into an analog NTSC, PAL or other analog television signal. When the sail decoder class 916 is done with the buffer, it will call releaseEmpthBuf.

Various structures can make the system easy to test and debug. Each layer can be tested independently to ensure that it performs in an appropriate manner, and the various types can be gradually combined to achieve the required function, while maintaining the ability to effectively test each object.

The control object 917 is used to receive instructions from the user and send events to the transmission channel to control the execution status of the transmission channel. For example, if the user has a remote control and is watching a TV program, after the user presses pause, the control object 917 will send an event to the sink 903 to inform it of the pause. The sink 903 will stop asking for new buffers. Currently the pointer 920 will stay where it is. When the sink 903 receives another event telling it to play, the sink 903 will start fetching the buffer again. The system is in perfect sync and the fetch starts where the frame left off.

The remote may also contain a fast forward button. When the fast forward button is pressed, the control object 917 sends an event to the transition 902 telling it to move forward two seconds. Conversion 902 finds that a time length of two seconds requires three buffers to be moved forward. It then sends a reset event to the downstream channel so that any data in the queue or state that may be present in the hardware decoder is forced out. This is a critical step because the structure of the MPEG data stream requires that a state must be maintained for multiple frames which would become invalid due to relocation of the pointers. It then moves the current pointer 920 forward three buffers. When the sink 903 calls nextFullBuf next time, it will get a new current buffer. The same method can be applied to the fast rewind function, which is achieved by moving the current pointer 920 backward with the transition 902 .

The system clock reference is located in the decoder. The system clock base is sped up for fast playback and slowed down for slow playback. The sink asks the entire buffer to speed up or slow down purely based on the clock rate.

Referring to Figure 10, the other two objects derived from the TmkXfrm class are stored in the transmission channel for easy disk access. One of them is called Tmk Clip Reader 1003 and the other is called Tmk Clip Writer 1001 . The buffer enters the Tmk clip writer 1001 and is pushed into a file located on the storage medium 1004 . The Tmk clip reader 1003 requests a buffer of a file from the storage medium 1005 . Tmk clip reader 1003 only provides allocEmptyBuf and pushFullBuf methods, while Tmk clip writer 1001 only provides nextFullBuff and releaseEmptyBuf methods. Thus, the Tmk Clip Reader 1003 performs the same function as the input or "push" side of the Tmk Clip Cache 1002, and the Tmk Clip Writer 1001 performs the same function as the output or "pull" side of the Tmk Clip Cache 1002 function.

Referring to Figure 11, it shows a preferred embodiment for achieving multiple functions. Source 1101 has a television signal input. This source transmits data to push switch 1102, which is a transform derived from TmkXfrm. The push switch 1102 has multiple output terminals, which can be switched by the control object 1114 . This means that a part of the transmission channel can be stopped, while another transmission channel can be started arbitrarily according to the user. Users can switch between different storage devices. Push switch 1102 may output to Tmk clip writer 1106 , which goes to storage 1107 or writes to cache switch 1103 .

An important feature of this device is that it can easily select the recorded portion of the incoming signal under the control of the program logic. Based on information such as the current moment, a specific length of time, or via the viewer pressing the remote control, the Tmk clip writer 1106 can be switched to record a portion of the signal and turned off at some later time. This toggle typically results in a "toggle" event being sent to the push switch 1102 object.

Another method for enabling selective recording is by modulating the information into the VBI, or depositing into an MPEG dedicated data channel. Data decoded from the VBI or a dedicated data channel is sent to the program logic. Program logic checks this data to determine whether the data indicates that recording of the modulated television signal should begin. Likewise, this information may also indicate when the recording should be terminated, or another data item may be tuned to signal when the recording should be terminated. Start and stop indicators may be explicitly modulated into the signal, or other information stored in the signal in a standard manner may be used to encode this information.

Referring to FIG. 12, an example is shown of how program logic scans words contained in a closed caption (CC) field to determine start and stop times and initiate recording with a specific word or phrase. These include NTSC or PAL field 1201 data streams. CC bytes are extracted from each odd field 1202 and added to ring buffer 1203 for processing by word parser 1204 . Word parser 1204 collects words until a word boundary is encountered, which is typically a blank word, period, or other descriptive character. Recall from the above that the MPEG audio and video segments are collected into a series of fixed-size PES buffers. A special data segment is added to each PES buffer to preserve words extracted from the CC field 1205 . Therefore, the CC information will be kept in sync with the audio and video, and will be correctly presented to the viewer while the stream is playing. This also allows stored data streams to provide processing for CC information when the program logic is idle, which can spread the load, reduce costs and improve efficiency. In this case, the words stored in the special data field can be sent directly to the state table logic 1206.

During data stream recording, each word is locked in a table 1206, which indicates the recognition action for that word. This action may simply change the state of the recognizer state machine 1207, or may cause the state machine 1207 to issue an action request, such as "start recording", "stop recording", "view phrase", or other similar requests. Indeed, a recognized word or phrase may cause the transmission channel to be switched; for example, a different voiceprint channel may be overlaid if an unwanted language is used in the program.

It should be noted that the parsing state table 1206 and the recognizer state machine 1207 can be modified or changed at any point in time. For example, different tables and state machines may be provided for each input channel. In another case, the units may be switched based on time of day or other events.

Referring to FIG. 11 , a pull switch 1104 is added, which is an output to a sink 1105 . The sink 1105 will call nextFullBuf and releaseEmptyBuf to get or return the buffer from the pull switch 1104 . Pull switch 1104 may have any number of inputs, one of which may be clip action 1113 . The remote control switches between input sources. The control object 1114 sends an event to the pull switch 1104 to inform it to switch from the current input source to any input source selected by the control object.

Clipping actions are provided for arranging a number of different stored signals in a predictable and controllable manner, possibly adding viewer-selected control via the remote control. Thus, it is exposed as a derivative of the TmkXfrm object and receives a "toggle" event for switching to the next stored signal.

This allows program logic or the user to create custom video output sequences. Any number of segments of video data can be arranged and combined, eg, as program logic or as a user using a broadcast studio video mixer. Tmk clip readers 1108 , 1109 and 1110 are assigned and hooked into pull switch 1104 . Pull switch 1104 toggles between Tmk clip readers 1108, 110g and 1110 to combine video and audio clips. Due to the way the transmission channel is constructed, flow control is automated. Push and Pull switches are like video switches in broadcast studios.

The derived classes and resulting objects described herein can be combined in any manner to create a number of different and useful configurations for storing, retrieving, switching, and viewing television broadcast data streams. For example, if multiple outputs and inputs are available, one input can be viewed while another is stored, while a second output can be used to generate a window of images within images to preview previously stored data streams. These configurations represent a unique and novel application of software translation that can utilize a single cost-effective device to achieve the effects of expensive and complex hardware solutions.

Referring to Figure 13, it shows a high level system diagram and shows its implementation with VCR backup equipment. The output module 1303 transmits the TV signal to the VCR1307. This allows users to record TV programs directly on video tape. The present invention allows the user to queue programs to be recorded from disk to video tape, and to schedule the time when the programs are sent to the VCR1307. Banner pages (EPG data) can be sent to the VCR 1307 before a program is delivered. Longer titles can be adjusted to fit smaller tapes by speeding up the playback rate or omitting frames.

The output of the VCR 1307 can also be directed back to the input module 1301 . In this configuration, the VCR is used as a backup system for the media switch 1302 . All excess storage or lower priority programming will be sent to the VCR1307 for later retrieval.

The input module 1301 can decode and pass on to the rest of the system information encoded in the VBI. The output module 1303 can be encoded into the output VBI data provided by the rest of the system. The program logic can be arranged to encode different types of identification information into the output signal which will be recorded on tape using the VCR 1307. Playing back the tape allows the program logic to read back the identification information so that the television signal recorded on the tape can be processed correctly. For example, a particular program can be recorded on tape along with information about when it was recorded and the source network. When the program is played back to the input module, this information can be used to control the storage and presentation of the signal to the viewer, etc.

Those skilled in the art will readily appreciate that this mechanism can be used to introduce various data items into program logic that would not be considered television signals. For example, software updates or other data may be added to the system. Program logic receiving this data from a television broadcast data stream may enforce control over how the data is processed, such as requiring an authentication procedure and/or decrypting embedded information based on some pre-knowledgeable key. This approach also works on conventional broadcast signals, thereby introducing an efficient method of providing non-television control information and data to program logic.

In addition, those skilled in the art will readily understand that although the above description is specific to a VCR, any multimedia recording device such as a digital video disc-random access memory (DVD-RAM) recorder can easily replace a VCR.

Although the present invention has been described herein with reference to preferred embodiments, those skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. For example, the invention can be applied to detect crimes in gambling establishments. The input of the present invention is connected to the camera surveillance system of the casino. Recorded video data will be quickly buffered and output to an external VCR at the same time. While the external VCR is loading real-time input video, the user can switch to either image feed, viewing (eg, reversing, playing, slow-playing, fast-playing, etc.) a specific segment of the recorded image.

Architecture of Video Data Stream Labeling

Please refer to Figure 12 again. Tags are abstract events that appear in the television broadcast data stream 1201 . These tags may be embedded in the VBI of the analog signal, or embedded in a dedicated data channel of the MPEG2 multiplex. As described above, tags may be embedded in closed caption (CC) fields and fetched into ring buffer 1203 or a memory allocation structure. The word parser 1204 will identify unique tags during the process of scanning the CC data. These tags are decorated with standard CC control codes. Tags can also be generated implicitly; for example, based on the current time and the program being watched.

The present invention provides a system called TiVo Video Tag Authoring (TVTAG) for inserting tags (TiVo Tags) into video data streams prior to broadcast. Referring to Figures 14, 16 and 17, the TVTAG system includes a video output source 1401, a compatible device 1402 for inserting closed caption information in the VBI and outputting captioned video, a monitor 1405, and a software program for controlling the VBI insertion device , causing it to include into the video data stream 1406 a captioned tag data object in the closed captioning information format. The tagged video data is immediately transmitted or stored on an appropriate medium for subsequent transmission.

In most basic implementation cases, the TVTAG software 1406 is responsible for controlling the VBI inserter 1402 . The TVTAG software 1406 interfaces with the VBI plug-in device 1402 through a standardized computer interface and device control code protocol. When the operator watching the video monitor 1405 judges that the desired tag insertion point has been reached, he presses a button to generate a TiVo tag data object, which is sent to the VBI insertion device 1402 and incorporated into the video Data streams for transmission 1404 or storage 1403 .

The TVTAG software has the additional function of controlling the video input source 1401 and the video output storage device 1403 . The operator selects a particular video 1602 and can pause the video input stream to facilitate the image element 1702 to be overlaid on the monitor and locate the image element using a pointing device such as a mouse. Positioning of the image element 1702 can also be done through the operator interface 1601 . The operator uses the X position 1605 and the Y position 1604 to input the image position.

Image elements and positioning information are then incorporated into TiVo tag objects (described below) and recorded with timecode or video frames. When the operator is satisfied, playback and recording can be resumed. Subsequently, the most accurate label is issued by the insertion device.

Please refer to Figure 15. In another preferred embodiment of the TVTAG system, the software program takes a standard Internet Protocol web page and displays the web page 1505 for the operator. The web page will cause the script executed on the remote server 1504 to generate the TiVo tag object. The server 1504 controls the VBI insertion device 1502 , the video source 1501 and the recording device 1503 . The remote operator 1505 can obtain the low-bandwidth or high-bandwidth version of the video data stream from the server 1504 as a reference for inserting tags. Once the necessary tag data objects have been generated and transmitted, they can later be batched by the server 1504 .

In another preferred embodiment of the present invention, software and widely used non-linear video editing systems are integrated into "insert" objects, thereby allowing TiVo tag data objects to be inserted during video generation. In this embodiment, the nonlinear editing system acts as the source and storage system controller, and also provides image placement functionality, which allows for the placement of frame-accurate TiVo tag data objects.

Please refer to Figure 18. Tags are integrated into the video data stream before or on video source 1801 . The video data stream is then transmitted via satellite 1802 , cable or other terrestrial transmission method 1803 . Receiver 1804 receives the video data stream, recognizes tags, and performs appropriate actions in response to the tags. Viewers watch the final video data stream through a monitor or TV set 1805 .

The present invention provides an architecture that supports actions based on tags in a video stream. Examples of flexible uses provided by TiVo tags include:

• Want to know when to watch a broadcast network's promos, so viewers can choose to record upcoming programs that will air in the future. A TiVo tag attached to the program trailer indicates the date, time and channel on which the program will air. Details about proactive promos are described in detail below.

• A common problem occurs in baseball overtime. VCRs and digital video recorders (DVRs) cut off the recording process whenever a baseball game is longer than the time to enter commercials. A TiV0 tag is transmitted in the video data to indicate that the recording process must continue. When the game is over, a TiVo tag is also transmitted to tell the system to stop recording.

• Boxing matches often end abruptly, causing VCRs and DVRs to record substitute programs during the remaining retention time. Send a TiV0 tag to indicate that the show is over and tell the system to stop recording.

·Please refer to Figure 19. Advertisement tagging is used to broadcast advertisements stored locally or remotely, so as to replace advertisements outside the country or region. In video data stream 1901, tags superimposed on program segments 1902 (commercials or other data segments) are tagged with techniques such as the TVTAG system described above. The TiVo tag will inform the present invention 1905 about where the old program segments 1902 start and end. A separate tag 1903 can be appended to inform the invention 1905 about the duration of the old program segment 1902, or tags can be appended to the start 1903 and end 1904 of the old program segment to indicate the start and end of the data segment 1902. When a tag is detected, the present invention 1905 will search for a new program segment 1906, and directly broadcast a new data segment to replace the old program segment 1902, and when the broadcast is over, it will return to the original program 1901. The viewer will not feel the conversion process described above.

There are three options at this point:

1) The system 1905 can continuously cache the original program; thus, if the viewer 1907 reverses the program 1901 and plays the program again, he will see the overlapping data segments;

2) The old program segment 1902 will also be replaced in the cache; therefore, the viewer will not see overlapping data segments; or

3) The system can quickly cache the original data segment 1902, and reinterpret the tag during playback. However, without smart tag prefetching, this feature would only work if the viewer had enough backup that the system would know the first tag in the overlapping segment.

The above problem can be solved by adding the length of the old program segment to the start point 1903 and end point 1904 tags. Another processing method is label matching, so that the system identifies the end label 1904 from the start label 1903 . When the system 1905 performs fast forward or reverse to pass one of the tags, the system 1905 can know that it has to look for the other tag. Tags 1903 and 1904 are paired to contain a unique identifier. The system 1905 can then search forward or backward for matching tags and replace the old program. When the system performs tag prefetching, it has time or frame length limitations. This limitation is included in labeling or normalization. Including the bounds in the label is the most flexible way to do this.

For example, the program segments to be played are selected by location, time of day, program content, or by a preference engine (described in US Patent Application Serial No. 09/422,121 owned by the applicant of the present application). Using the preference engine, appropriate programs 1906 from local or server storage can be selected based on the viewer's personal data. Personal data includes viewers' viewing habits, program preferences and other personal information. Stored program segments 1906 also have program objects describing their characteristics, which are used to search for the most suitable preference guide.

Obviously, there must be a looping mechanism in commercials to avoid running out of commercials. By creating an offset bias for the currently viewed program relative to the program data, the preference guide can further offset this bias and use this offset bias to match the commercial list with the program 1906 on disk. For example, if a viewer is watching a soap opera, and the viewer's preference tends to be sports, the present invention will select a beer advertisement instead of a diaper advertisement.

Labels can also be used to form conditional selections. Tags have their own way of measuring likes. In this case, the preference measurement is compared with the preference tendency, and if the comparison result shows a high correlation, the present invention will not change the commercial. If the comparison result is low correlation, it will trigger the application of the above method

It should be noted that in all cases, the system 1905 has sufficient time to be able to make a selection. The transport channel structure typically fast-buffers 1/2 second of video data to provide a lot of time to change the data stream between input and output. If longer time is required, more buffers can be added to the transmission channel. When the broadcast of the program on the disk is finished, the system will form a time delay of the same length by reading the previous data stream.

It should also be noted that commercials can also be detected using the method described above, described in U.S. Patent Application Serial No. 09/187,967, entitled "Analog Video Tagging and Encoding System", The above-disclosed application is also owned by the applicant of this case. The same type of substitution as described above can also be used when using the tags described in the preceding patent applications.

Please refer to Figure 19 and Figure 22. The tag may perform a commercial "zapping" function. Tags can be used to mark the start 1903 and end 1904 of a commercial so it can be skipped or fetched first. The viewer only needs to press the jump key 2205 on the remote controller 2201. The system seeks to the end tag and replays the frame immediately following that tag. The number of skipped commercials depends on the number of buffered video streams.

The system 1905 can automatically skip commercials for live programs, or skip commercials for programs that are pre-recorded and stored in memory 1906, according to viewer preferences. To skip the commercial advertisements in live programs, according to the above method, there must be a large amount of buffering in the channel. Allowing the system to skip commercials recorded in the program allows the viewer to watch the program continuously without any commercial interruptions.

• Tags attached to programs can be used as indexes. For example, the viewer can jump to each index in the program by pressing the jump key 2205 on the remote control 2201 .

· Labels can also be used for system functions. As mentioned above, the system can store the program content locally for its own use. The system 1905 has to get the program material in some way. This can be achieved by tuning to a specific channel during offline hours. The system 1905 searches the data stream 1901 for a tag telling it to start recording. The recording includes a number of program segments delimited by tags 1903, 1904, which can identify content or preferences. A tag at the end in the data stream tells the system 1905 to stop the recording process. Program segment 1906 is stored locally and used for subsequent indexing according to the above method.

The present invention has the following design points:

· This design provides clear compartmentalization mechanisms and strategies.

• From an internal point of view, tags can be viewed as abstract events that trigger policy modules. The source channel object is responsible for mapping received label information to these internal abstractions.

• Abstract tags are stored in the PesBuf data stream as if it were another data segment. This allows processing of tags of any size with precise time information. It also allows tags to persist as part of the recorded program, so appropriate actions can be taken whenever the program is viewed.

• Tabs can be updated with information about current shows and future shows. Such information is reserved for recorded programs.

• Can record tags as they pass through the system. In addition, this information can also be uploaded. It is not necessary to retain all the information that accompanies a label.

• Labels can be generated on an individual schedule. For example, use network workstation records to generate tags based on time and network viewed. Time-based tags remain in the recorded stream.

Time-Based Labels

Please refer to FIG. 20 , the time-based tags are processed by the time-based tag identifier 2012 . This object 2012 listens for channel change events, and when switching to a known broadcast network, the object receives a "time record" of that broadcast network. If there is a certain time record, the object 2012 will create a tag schedule according to the current time. When the time of each tag occurs, the object 2012 sends an event to the source 2001 to indicate that the tag has been inserted. The source object 2001 inserts the label into the next available position in the PesBuf currently being constructed. The next "available" location may be determined based on frame boundaries or other conditions.

The role of the source object

The source object 2001 is responsible for inserting tags into the PesBuf stream it processes. It is assumed here that there are separate source objects for analog input and digital TV sources.

Tags can appear in the simulated data stream in a number of different ways:

— in the EDS field.

- Implicit use of CC fields.

- Modulation to VBI, perhaps using the ATVEF specification.

— A time-based approach.

In a digital TV broadcast data stream, or after conversion from analog to MPEG:

- In-band, using TiVo tagging technology.

— MPEG2 dedicated data channel.

- MPEG2 data flow characteristics (such as frame boundaries, etc.).

—Time-based labels.

The source object 2001 is not responsible for parsing tags and taking any action. Instead, the source object 2001 should only be responsible for identifying potential tags in the data stream and adding them to the PesBuf data stream.

Tag identification and action

Conceptually, all tags fall into two broad categories: tags that must take an action when received (such as recording a program), and tags that must take some action when displayed—that is, when the program is viewed.

Processing of received labels

Tags that must take action on receive are handled as follows: A new Receive Tag Mechanism subclass 2003 of the Tmk Push Switch class 2002 is created. When the incoming data stream passes through the sub-class 2003 between the source object 2001 and the program cache transformation 2013, the sub-class 2003 can recognize the receiving tag and take appropriate action.

Receiving tags are usually processed once and then expired.

Display label processing

Labels that must take some action when displayed are handled as follows: Create a new display label mechanism subclass 2007 of the TmkPullSwitch class 2008. When the output data stream passes through the secondary class 2007 between the program cache switch 2013 and the sink object 2011, the secondary class 2007 can recognize the display tag and take appropriate action.

Handling of labeling procedures

Only when the TagReceptionPolicy object 2009 appears in the current channel, can the processing of receiving tags be performed. Only when the TagPresentationPolicy object of the source channel exists, can the tag be displayed.

The TagPolicy object describes which tags will be recognized and which actions are allowed.

When the input channel changes, the receiving tag object will be notified, and at the same time, the object will extract the TagReceptionPolicy object 2009 of the channel (if this object exists), and follow the defined procedures.

When the output channel changes, the display tag object will be notified, and at the same time, the object will get the TagPresentationPolicy object 2010 of the channel (if this object exists), and follow the defined procedure.

tag record

Receipt of tags can be recorded in the database. This action only occurs if a TagReceptionPolicy object 2009 exists and the tag is set as a record attribute. For example, the record attribute may be set, but still not allowed to perform receive actions. This enables passive recording of actions within the incoming data stream.

Changes in channel handling

It is important to support data updates about the current show content. The following strategies are proposed:

—When the input source is changed or a new show content starts, a copy of the show object will be made, and other operations in the channel will make up this copy.

- Update tags are of the receive tag; where policy permits, update the replicated projection object.

- If the current showing is to be recorded, a copy of the showing is stored with it so that the stored program has the appropriate information to be stored with it.

- The video recorder must be notified that the object of the projection has changed so that it does not prematurely cut off the recording of, for example, a baseball game.

Tag Interpretation and Tag State Machine

The use of tags is very flexible because the Tag Interpreter 2005 interprets standardized abstract tags and the TiVo Tag State Machine 2006 executes operational tags once the TagPolicy object has been used to identify a valid tag. An interpreted tag initiates a predefined set of actions. Each group of actions has been pre-programmed and set in the system.

State machine tags are operational tags that do not carry executable code, but execute program steps. This allows tag originators to combine such tags to perform specially tailored actions on the TiVo system. State machine tags achieve the same results as interpreted tags can, but with the flexibility to dynamically change the set of actions performed.

Abstract Interpretation Tags

The set of abstract tags that can be used is defined in a list called tag/action-list. Such a list is usually stored within a database object; there are several abstract tags defined therein. These actions can be divided into three main categories:

- Actions (which may include interactions) that can be seen by the viewer.

- Meta information about the data stream (eg channel, time, show length, etc.).

-TiVo control tab.

Tags that cause changes to the on-board database or covert recordings must be authenticated. This work is done through control tags.

Tabs visible to viewers

— menu

This label indicates that the viewer is to be offered a choice. The data attached to this tag tells what the option is, as well as other relevant data such as display styles. The menu has a time-out function.

The purpose of a menu label is to provide choices to the viewer. If the viewer is not present or interested, the menu should disappear immediately. The operating principle of the menu can be used to pause the currently broadcast program according to the situation. The display method of the menu is not necessarily a list.

— conditional launch of another show

This tag indicates that another program should be played when a condition is true. This condition is analyzed by the strategy module. A condition can always be true.

—Conditionally entrusted to another program

This tab will revert to the previous show. When the program ends, the stack of another program will be lifted up automatically. If the channel has been changed or the viewer enters the TiVo central menu area, all other programs are popped up.

Highlighting other programs is a method of inserting arbitrary sequences into programs being watched. Conditional data is not evaluated at the top level. Instead, policy modules must check this data for selection. For example, this method can be used to form "nested" advertisements.

— conditional display indicator

Such labels cause indicators to appear on the screen. Indicators are named, and the set of active indicators can be queried at any time. Such a tag or tag policy may indicate a timeout value for the boot indicator.

— Conditional clear indicator bar

This label clears the active indicator. If the channel is changed or the viewer enters the TiVo center menu area, all indicators will be cleared.

Indicators are another way to provide viewers with choices without interrupting the airing of the show. Indicators can also be used to indicate conditions for which data streams may be of interest. For example, an "ActivePromo" can be created by providing a PID as part of the tag data to make the program available for selection. When the indicator is displayed, if the viewer presses a certain button, the program is scheduled to be recorded.

meta information tag

—Current broadcast information

This type of tag is typically used to hold information about the currently running program. Each tag is usually linked to a certain piece of information, such as start time, end time, duration, etc. Such tags can be used to "extend" the recording of an event.

- Future broadcast information

This tag is similar to the tag above, but it contains information about future showings. There are two relevant situations here:

• The above information is broadcast information about a certain item already existing in the database. This database object is updated as appropriate.

• The above information is broadcast information that does not exist. At this time, a new playback object will be created and started by this label.

TiVo Control Tags

— authorize modification

This tag is usually encrypted with the current month's security key. The lifetime of this certification is set by the operating principle, which may be one or two hours. Therefore, if changes to the state of the TiVo system at the local end are allowed, the tags must be continuously rebroadcasted.

Such tags are intended to prevent deliberate (or accidental) tampering using inherently non-secure tagging mechanisms such as Extended Data Service (EDS). If the broadcast network provides EDS information, it should first be determined that the labels provided by the broadcast network are correct, and damage to the label transmission system is unlikely to occur. Second, it should cooperate with the broadcast network to provide an authentication system that transmits authentication tags only on the broadcast network. Unauthenticated tags MUST NOT be inserted into the PES stream via source objects.

—Conditionally record the current program

This tag causes the currently watched program to be stored to disk from that point on. Recording stops when the program ends.

— Conditionally stop recording the current program

This tab will stop recording the program you are currently watching.

—Conditional recording of future programs

The showing object ID is provided here (perhaps directly in the future show tag). Shows are scheduled to be recorded with background prioritizing them lower than the viewer's explicit choice.

— Conditional unrecording of future shows

Provide the playback object ID here. If a recording was scheduled for this object by a previous tag, the recording will be cancelled.

These tags and future show tags can be inserted in an encrypted and secure format. The source object will only insert such tags into the PES stream if they have been properly validated.

One use of these tabs is to automatically launch TiVo catalogs - such as loop collections, commercials, slots, etc. Later downloads will cause this directory to be "mounted" and available.

—Store files conditionally

This type of tag is used to transfer data within a data stream to be stored on disk. For example, broadcast web pages go through this mechanism.

—Conditionally store objects

This tag is used to transfer objects in the data stream to be stored on disk. Object stored procedures follow standardized object update rules.

The following is an example implementation using a show tag inserted in the Closed Caption (CC) portion of the data stream. The CC portion of the data stream is selected because the signal is preserved when it is transmitted to the user's receiving device before it is decoded. The rest of the VBI signal is not guaranteed to be the same. When encoding to MPEG-2, many satellite systems strip everything but closed captions.

There are very strict bandwidth restrictions on CC data streams. The data rate of the CC data stream is two 7-bit bytes per video frame. Furthermore, to avoid collisions with control codes, the data must start at 0x20, thus effectively limiting its transfer rate to approximately 6.5-bit bytes (truncated to 6-bit bytes for simplicity). Furthermore, the aforementioned bandwidth is approximately equal to 360 bits/second. If the channel is shared with the actual CC data, the above transmission rate will be further reduced. Additionally, additional control codes need to be transmitted to prevent CC-capable TVs from attempting to display the TiVo tag as CC text.

Basic Label Design

This paragraph explains how to arrange tags within a closed caption data stream. A general knowledge of closed captioning specifications is assumed here; however, this is not a determining factor.

make the label invisible

A TiVo tag placed in the data stream should not interfere with the display on a closed caption capable TV. This purpose can be achieved by the following method: first send a "reload subtitle" command (send twice to avoid mistakes), followed by a string of characters describing the label, these characters are followed by "erase non-display memory" Command (twice to avoid mistakes). The effect of the above procedure is to load the text into memory outside the screen, and then clear the memory. Normally closed captioning capable TVs will not display this text (according to the EIA-701 standard).

As long as the closed caption decoder is not in "scroll" or "scroll" mode, the above method must work. In this mode, the "reload subtitles" command will cause the text to be deleted. To overcome this problem, even if the TiVo tag is transmitted to the second closed captioning channel, the TiVo tag will still be received and recognized. According to the above method, even if the closed caption channel 1 is set as scrolling text, the tag can still be transmitted through the closed caption channel 2.

Label encoding

The text transmitted in a TiVo tag consists of the letters "Tt", followed by a single character representing the length of the tag, followed by the tag content, followed by a cyclic redundancy check (CRC) of the tag content. The "Tt" letter is sufficiently unique that it is unlikely to encounter the same letter in general CC data. In addition, general CC data often starts with a position control code to indicate where the text should be displayed on the screen. Such location data is not required since the on-screen display will not be used. Thus, the very low probability of encountering the letters "Tt" immediately after the "resubtitle loading" control code almost guarantees that this combination is a TiVo tag (however this implementation does not need to rely on the fact that this is the case).

The single character representing the tag length is calculated by adding 0x20 to the tag length. For example, if its length is 3 characters, the length character used is 0x23 ('#'). Thus, the above implementation can be extended to a length of 95 (because there are only 96 characters in the character set), and the maximum length is defined as 63. If longer tags are required, interpretations of the other 32 possible values can be added for the length character.

The possible values for the tag itself are defined in the Tag Types section below.

The above-mentioned CRC is a 16-bit CRC-CCITT (Consultative Committee for International Telegraph and Telephone) standard code (ie, polynomial=x^16+x^12+x^5+1). The CRC code is placed in the data stream as three separate characters. The first character is calculated by adding 0x20 to the six most significant bits of the CRC. The next character is calculated by adding 0x20 to the second most significant bit of the CRC. The last character is calculated by adding 0x20 to the last four bits of CRC.

label type

This paragraph will detail examples of TiVo tags. It should be noted that each tag sequence begins with at least one byte, and this byte represents the type of tag.

iPreview Tags

Please refer to Figure 17. An iPreview tab contains four items of information. The first item of information is the 32-bit program identification code of the preview program. The second piece of information contains the running time of the teaser program. The third item of data is the position on the screen 1701 where the iPreview prompt 1702 is placed, and the last item of information is the size used by the iPreview prompt.

The screen position prompted by iPreview is the ratio of the screen width and high resolution. The X coordinate uses 9 bits to distinguish the width, so the final coordinate is as follows: X=(x_resolution/511*xval). If xval is 10, then on a 720x486 screen (using CCIR656 resolution), the X coordinate is 14. The Y coordinate uses 8 bits to distinguish the height, so the final coordinate is as follows: Y=(y_resolution/255*yval). The X, Y coordinates indicate the position of the upper left corner of the debug graph.

If the X and Y values are set to the maximum possible values (ie x=511, y=255), it means that the maker leaves the determination of its position to the system. The system will place the debug map at the preset position. The reason for using maximum values for preset positions is that the "real" positions are never set to these values, as doing so would push the entire modulation map out of frame.

The Size column is a four-digit number that represents the size of all hint images. The 16 possible values in this column correspond to preset image sizes, which should be provided in advance by the set-top box (video adapter).

The timeout is limited to a 10-digit number representing the number of frames left in the trailer. This will limit the lifetime of the tag to 34 seconds. This tag must be repeated if a teaser has a longer duration. It should be noted that the above timeout limit is "artificially limited" to 10 digits to avoid errors. This prevents it from affecting subsequent commercials when a creator uses a malformed timeout in a tag.

Version is the number used to compile the version to identify the preview program itself. The full closed caption character set is used here instead of bit-packing the number (thus limiting it to 6 bits), which results in 96 possible values instead of 64 (2^6) possible values. Therefore, version numbers should be in the range 0-95. Reserved characters are not currently used. Such characters must be present in order for the control codes to be properly terminated on 2-byte boundaries.

The first character of the Ipreview tag is always "i".

All data fields are packed on the same bit boundary, and then divided into six bit values, which are converted into characters (by adding 0x20) and sent. The order of the above columns is as follows:

32 bits: program identification code (ID)

9 digits: X coordinate

8 bits: Y coordinate

4 bits: image size

10 digits: timeout limit

1 character: version

1 character: Reserved column

The data field has a total of 66 bits, which requires 11 characters to transmit, plus 1 character for the version field, and 1 character for the reserved field. The exact content of each character is as follows:

1) 0x20+ identification code (ID) [31:26]

2) 0x20+ID[25:20]

3) 0x20+ID[19:14]

4) 0x20+ID[13:8]

5) 0x20+ID[7:2]

6) 0x20+ID[1:0]X[8:5]

7) 0x20+X[4:0]Y[7]

8) 0x20+Y[6:1]

9) 0x20+Y[0] size (size)[3:0]

10) 0x20+Y[0] size[3:0] timeout (timeout)[9]

11) 0x20+timeout[8:3]

12) 0x20+timeout[2:0]

13) 0x20+ version (version)

14) Reserved

Including the first character "i", the length of the iPreview tag is 14 characters + 3 CRC characters. Adding the tag header (3 characters) makes the total length 20 characters and can be transmitted over 10 frames. Plus the 4 frames used to send the two "reload subtitles" and "erase undisplayed memory", this means that the iPreview tag will take up 14 frames (0.47 seconds) when broadcast.

A complete iPreview tab contains:

"Reload subtitle", "Reload subtitle", Tt1(0x20+17=0x31=0110001="1") i<13 characters iPreview tag>, 3 characters CRC, erase undisplayed memory, erase undisplayed memory

parity debug characters

Currently, parity bits are used as parity. However, since the CRC is already included, the parity bit does not need to have the function of error checking. Taking this design a step further, there are more clever ways to use the parity bits. Since closed caption receivers will necessarily ignore any characters with wrong parity, this preferred way of utilizing the bandwidth-limited CC channels can be achieved by intentionally using wrong parity. This eliminates the need to reload subtitles and delete undisplayed memory, and makes it easier to "scatter" TiVo tags in existing CC data.

iPreview audience interaction

Please refer to Figure 17, Figure 20, Figure 21 and Figure 22. The iPreview tag causes the tag interpreter 2005 to display the iPreview prompt 1702 on the screen 1701 . The iPreview prompt 1702 is to inform the viewer that the program preview in operation is available, and the viewer can inform the TiVo system to record the program that will be broadcast in the future. The viewer responds to the iPreview prompt 1702 by pressing the select button 2204 on the remote control 2201 .

The tag interpreter 2005 waits for user input. Pressing the select button 2204 will automatically schedule the program recording schedule by the tag interpreter 2005 according to the viewer's preset preferences, thus becoming a one-touch recording function, that is, presenting the recording option screen 2101 to the viewer. At this point, the viewer can highlight the option 2102 on the recording menu and press the selection key 2204 to schedule the program to be recorded.

The tag itself has been interpreted by the tag interpreter 2005. The tag interpreter 2005 will wait for any input from the viewer through the remote control 2201 . Once the viewer presses the select key 2204, the tag interpreter 2005 will inform the TiVo system of the scheduled recording of the program described in the 32-digit program ID code in the iPreyiew tag.

Please refer to Figure 20, Figure 22 and Figure 23. iPreview also has other uses. Each use is governed by program content and display screen images. Obviously, the system cannot interpret the program content, but the combination of the image and the program ID tells the tag interpreter 2005 what action to take. Below are two examples that generate leases and sales.

The process of generating a rental may take place, for example, while a car advertisement is being played. When the iPreview image appears on the screen, the viewer knows that they can enter the interactive menu by pressing the select key 2204 .

A menu screen 2302 is displayed by the tab interpreter 2005, which provides the viewer with options to obtain more information 2303 or watch a car video 2304. Viewers can press live TV button 2202 and leave at any time. If the viewer chooses to obtain more information 2303 through the up and down arrow keys 2203 and the selection key 2204, the information desired by the viewer will be sent to the manufacturer 2305 by the label interpreter 2005, and a product introduction will be generated accordingly. The viewer presses the select key 2204 to return to the program.

Sales may be generated when advertisements for products such as music albums are shown. The iPreview image 2301 will appear on the screen. The viewer can press the select key 2204 to cause the tag interpreter 2005 to display a menu screen 2307 .

The menu screen 2307 provides the viewer with the option to buy the product 2308 or leave 2309. If the viewer chooses "Yes" 2308 to purchase the product, the label interpreter 2005 will send the order form together with the purchase information 2310 of the viewer to the manufacturer. If the ad is for a music album, viewers may also choose to watch a music video produced by the music producer.

Whenever the system returns the viewer to the program, the viewer will pick up where he left off. This will give the viewer a sense of continuous play.

The concept of redirection can be easily extended to the Internet. The iPreview icon will appear as above. When the viewer presses the selection key 2204 on the remote control 2201, the webpage will be presented to the viewer accordingly. The viewer can then interact with the web page, and when the interaction is complete, the system returns the viewer to where they left off viewing the program.

Using the preference engine described above, the information presented to a viewer in forming a product introduction or sales process can be conveniently tailored to a specific audience. The viewer's viewing habits, program preferences and personal information are used to select the menus, options and screen displays presented to the viewer. Each menu, option, and screen display has an associated program object, and these objects are compared with the viewer's preferences.

For example, if a viewer is male and the ad is provided by Chevrolet, when the viewer presses the select button, a picture of a truck will appear. If the viewer is a woman, a picture of a convertible car will appear.

It should be noted that the tag state machine 2006 described below is fully capable of performing the same steps as the tag interpreter 2005 in the above example.

TiVo Tag State Machine

Please refer to Figure 20 again. A preferred embodiment of the present invention provides a Tag State Machine (TSM) 2006 . The tag state machine is a mechanism for processing abstract tags, which can generate actions that can be seen by the viewer through the TiVo receiver.

Creating proactive promos is a simple example of this. As mentioned above, active promos are previews of upcoming shows, and viewers can immediately choose to have the TiVo system record the show when it actually airs.

There are several potential complications in the simple example above: some indicator must be generated to alert the viewer to the opportunity; the indicator must appear and disappear accurately; the correct identifier for the program in question must be provided; active programs appear A previewed program may be viewed at a different time than its original airing time.

There are also challenges in setting up and managing TiVo tags. It is important to minimize changes to existing broadcast methods and technologies. This presentation mechanism should be as simple as possible to allow easy integration into broadcast data streams and result in robust and reliable operating characteristics.

How Tags Work

As mentioned above, it has been considered that the available bandwidth for transmitting tags is limited. For example, the VBI has limited free space that it scrambles to use. Even in digital television signals, the amount of out-of-band data conveyed is modest, since most users of this signal are primarily concerned with television programming choices.

Thus, tags become simple objects of only a few bytes in size. More complex actions are established by sequentially transmitting multiple tags.

The nature of broadcast transmissions implies that labels will be lost due to signal problems, sunspots, etc. TSM utilizes some kind of mechanism to deal with missing tags and make sure that no unintended actions are taken due to missing tags.

Generally speaking, the tag actions that viewers can watch are only related to the channel they watch; it is assumed here that the tag status is discarded when the channel is changed.

Entity tags are translated into abstract tags by the source object 1901 receiving the entity tags. Since no executable code is attached to the tag, the tag is not an "active agent"; manipulating the TSM results in modifications and changes that are visible to viewers, but the basic operation of the TiVo receiving system is not affected by the sequence of tags. If the tags could contain executable code—such as a Java byte stream for ATVEF—the quality of the TiVo viewing experience could be adversely affected by poorly written or malicious software.

All tag actions are governed by a matching policy object that matches the current channel. Any action is enabled or disabled by this object; if no policy object exists, all tag actions are blocked.

Basic abstraction tags

All abstract tags share a common underlying structure. The following components appear in all abstract tags:

— Tag type (1 byte)

Type 0 is not allowed. Type 255 indicates an "extended" tag in case more than 254 tag values may be required in the future.

— tag sequence (1 byte)

This unsigned column increments with each label in the sequence. For tags that are not part of a sequence, this column must be set to 0. A tag sequence of 1 represents the start of a new sequence; conceptually, a sequence can be of any length, but it must consist of an ordered sequence of fewer than 255 tags. composed of fragments.

Each tag type has a built-in sequence length (may be 0); the sequence number is introduced to handle tag omissions or other forms of tag loss in the data stream. In general, whenever an error occurs in one sequence, the entire tag sequence is discarded and the state machine is reset.

Tags should be checked and summed within the scope of the entity. If the checksum does not match, the source object discards the tag. Doing so will cause a sequence error and reset the state machine.

— tag time stamp (8 bytes)

This is the synchronous time within the television broadcast data stream at which the tag is recognized. This time is synchronized with all other displays produced by the TiVo receiver. This part will not be transmitted, but generated by the receiver itself.

—Length of label data (2 bytes)

This is the length of any data in parallel with the tag. This data is interpreted according to the tag type. Entity scope interpreters should perform at least some level of error checking on the data.

Tag State Machine (TSM)

The TSM is part of the tag presentation mechanism, which is connected to the video playback.

Conceptually, a TSM manages an abstract stack of integers that are at least 32 bits in precision, or large enough to hold an object identifier. This object identification code is an abstraction that may or may not indicate a real object on the TiVo receiver - otherwise the object may have to be mapped to the correct object. The stack size is limited to 255 data items to prevent denial of service damage.

TSM also manages a set of variables. Variables are named 2-byte integers. Variable name 0 is reserved. Label sequences operate on "user" variables; such variables are between 1 and 2^15-1. The "system" variable is handled by the TSM and contains values about the current TiVo receiver, such as: object ID of the current program, TSM revision and other useful information. The names of these variables are between 2^5 and 2^16-1. The number of user variables is limited within the TSM; and TSM variables indicate this boundary.

Tag data is a sequence of TSM instructions. Execution of these commands begins when the tag is recognized and allowed. TSM instructions are byte-oriented, and certain instructions may contain additional bytes to support their functionality.

Available TSM commands can be divided into several types:

Data Movement Instructions

push_byte—follows the instruction and pushes bytes onto the stack.

push_short—follows the instruction to push a short number onto the stack.

push_word—Follows the instruction and pushes the word onto the stack.

variable access instruction

push_var—Push down a 16-bit named variable following the instruction.

pop_var—follows the instruction to pop up a 16-bit named variable.

copy_var—follows the instruction to copy from the stack to a 16-bit named variable.

stack operation instructions

swap—Swaps the stack values of the top two layers.

pop—Pops up the topmost stacked value.

arithmetic instruction

add_byte—follows the instruction to add the signed byte to the top of the stack.

add_short—follows the instruction to add a signed short to the top of the stack.

add_word—follows the instruction and adds the signed word to the top of the stack.

and—Adds together the top and next stacked data items, popping up the stack and pushing down the new value.

or—ORs the top and bottom stacked data items, pops up the stack and pushes down the new value.

conditional instruction

(only for comparison of unsigned variables)

brif_zero—follow the instruction to branch to a signed 16-bit offset if the top level of the stack is zero.

brif_nz—Follow instruction to branch to signed 16-bit offset if top of stack is non-zero.

brif_gt—Follow the instruction to branch to the signed 16-bit offset if the top of the stack is greater than the next stack data item.

brif_ge—Follow instruction to branch to signed 16-bit offset if top of stack is greater than or equal to next stack data item.

brif_le—Follow the instruction to branch to the signed 16-bit offset if the top level of the stack is less than or equal to the next stack data item.

brif_lt - follow instruction to branch to signed 16-bit offset if top of stack is less than next stack data item.

brif_set—follows the instruction to branch to a signed 16-bit offset if there is a set of bits when the top of the stack and the next stack data item are joined together with the AND operator.

action command

exec—Executes the tag action on the object identifier named at the top of the stack

fin—terminates the label without taking any action

system variable

32768(TAG)—The value of the current tag

.

.

.

GMT:

32769 (YEAR)—Current year (starts with 0).

32770(MONTH)—Current month (1-12).

32771(DAY)—the day of the month (1-31).

32772 (WDAY)—Day of the week (1-7, starting on Sunday).

32773(HOUR)—the hour of the day (0-23).

32774 (MIN)—Minute of hour (0-59).

32775 (SEC)—Second of minute (0-59).

Status of TiVo receiver:

32800 (SWREL)—software version (marked as x.x.x in bytes)

32801 (NTWRK) - Object ID of the currently tuned network

32802 (PRGRM) - Object ID of the currently tuned program

32803(PSTATE)—The current state of the output channel:

0—normal playback

1—Pause

2—Slow motion

10—reverse speed 1

11—reverse speed 2

...

20—fast forward speed 1

21—fast forward speed 2

Tag execution status:

32900(IND)—Indicator code to show or clear.

32901(PDURING)—Channel status during label execution.

32902 (ALTP) - Alternate Program Object ID to advance play stack.

32903 (SELOBJ) - Recorded Program Object ID when the indicator is selected.

33000 (MENU1) - String object number for menu item 1.

33001 (MENU2) - String object number for menu item 2.

...

33009 (MENU10) - String object number for menu item 10.

33100 (PICT1) - Picture object number for menu item 1.

33101 (PICT2) - Picture object number for menu item 2.

...

3310g (PICT10) - Picture object number for menu item 10.

33200 (MSELOBJ1) - Recorded Program Object ID when selecting a menu item.

33201 (MSELOBJ2) - Recorded Program Object ID when selecting a menu item.

...

33209 (MSELOBJ10) - Recorded Program Object ID when selecting a menu item.

Label

—Push down another show

—Upload another program (upload automatically after the program ends)

— Raise indicator

— lower indicator

— menu

Label Enforcement Policy

Execution strategy is determined by TSM. Some suggestions are as follows:

— menu

Menus are arranged according to standardized TiVo menu benchmarks. Generally speaking, the menu will appear on the live video. Selecting an option usually initiates a recording dialog. It would be nice to be able to pause the data channel while manipulating the menu.

-indicator

Please refer to Figure 17 and Figure 22. Indicators are displayed in an array of small images at the bottom. In the normal viewing state, the up and down arrow keys 2203 on the remote controller 2201 do not work. For an indicator, the up arrow key 2203 turns the indicator to the left, and the down arrow key turns it to the right. A small box appears around the selected indicator. Pressing the select key 2204 initiates the action. According to preset conditions, new indicators will be selected; when an indicator is deleted, if there is a previously selected indicator, the indicator will be highlighted.

- Programs to choose from

Alternative programs shall appear as part of the video stream with full fast forward/rewind controls. Jumping to the live program button 2202 first lifts up the available program stacks to be empty stacks.

People who are familiar with this technology can easily know that although the closed caption data stream is used as an example for illustration, other transmission methods can also be used, such as: EDS field, VBI, MPEG2 dedicated data channel, etc.

Although the present invention is described here with a preferred embodiment, those skilled in the art can easily understand that other applications can be used to replace the embodiment presented here without departing from the spirit and scope of the present invention. Accordingly, the invention is only to be defined by the appended claims.

Claims (8)

1. the method via the advertisement television program record scheduling time-histories of television broadcast stream is characterized in that, comprises:

Receive described television broadcast stream;

For the program of broadcasting future is broadcast promotional advertisement in the described television broadcast stream;

Show icon, can be to inform the described program of spectators for recording, described icon is to show according to the label that inserts in the described television broadcast stream;

Receiving the singly-bound of described spectators on a remote input units pushes;

The time-histories of recording of described program is ranked.

2. the method for claim 1 is characterized in that, when arrival was ranked the time, described program was stored in a memory device.

3. method as claimed in claim 2 is characterized in that described memory device is a hard disk.

4. method as claimed in claim 2 further comprises:

Whether detection described program during the described program of storage finishes too early; And

If program finishes too early, stop the described program of storage automatically.

5. device via the advertisement television program record scheduling time-histories of television broadcast stream, described device comprises:

Be used to receive the module of described television broadcast stream;

The program that will be used to broadcast future is broadcast the module of the promotional advertisement in the described television broadcast stream;

Be used to show the module of icon to inform that the described program of spectators can Gong be recorded, described icon is to show according to the label that inserts in the described television broadcast stream;

Be used to receive the module that the singly-bound of described spectators on a remote input units pushed;

The module of recording time-histories of described program is used to be ranked.

6. device as claimed in claim 5 further comprises:

Memory device; And

Wherein, when arrival was ranked the time, described program was stored in described memory device.

7. device as claimed in claim 6 is characterized in that described memory device is a hard disk.

8. device as claimed in claim 6 further comprises:

Detect module, wherein said detection module detects whether described program finishes too early during the described program of storage; And

Stop module, wherein said termination module stops the described program of storage automatically when program finishes too early.

Images