AU1359299A - A digital multimedia editing and data management system - Google Patents

Description

WO 99/05821 PCT/US98/15795 -1 A DIGITAL MULTIMEDIA EDITING AND DATA MANAGEMENT SYSTEM Field of the Invention The present invention relates generally to an audio/video data management and editing system, and more particularly, to a method and apparatus for editing and managing multimedia data in a multimedia production environment. Background of the Invention The process for producing broadcast news programs has undergone several changes over the last few decades. Changes in available technology, as well as increased competition brought about by the expansion of cable outlets and other news sources, have forced news broadcasters to use their resources more effectively while striving to differentiate their programming. In a typical news production operation, there are four major processes that occur during the production of a news program. These processes include video production, graphics production, text production and on-air operations. The results of these processes are rarely effectively combined until the actual broadcast of the news program. The video production process included the generation and editing of tape-based video for broadcast using videotape retrieved from a videotape archive or produced from one or more sources (i.e., cameras, either studio or field recorded). The text production process includes the scripting and editing of text gathered from several sources including a text archive. Similar to the tape production process and the text production process, the graphics production process includes the generation and editing of graphics data gathered from several sources. In order to produce the final news product for broadcast, the results of the video production process, the graphics production process and the text production process must be properly integrated during the on-air operations to produce the finished product. Existing news broadcast systems provide the capability for the integration of the video production process, the graphics production process and the on-air operations to provide complete management of audio and video elements of the news program from acquisition, through editing, distribution and on-air play. An integrated process in accordance with one such system includes a disk-based video production process integrated with a media production process and on air operations. The use of disk-based digital audio/video storage systems, digital networks, and digital nonlinear editing systems has allowed for successful integration of the WO 99/05821 PCTIUS98/15795 -2 video production, graphics production and on-air operations. Several products are available from Avid Technology, Inc., Tewksbury, MA, for providing such an the integrated system. One digital multimedia newsroom production system from Avid Technology allows users of the system to create, browse, and catalog multimedia assets. A journalist workstation is provided that allows a user of the workstation to access multimedia assets stored in a multimedia archive. A user of the journalist workstation can create a story board of a multimedia product for broadcast. The story board may include frames of low resolution video from several different sources, text from several sources such as news wire copy, archived text, and text created by the user while creating the story board, graphics, audio, still pictures, and any other form of multimedia data. The created story board includes a list of events, such as film clips, text, graphics, audio and still pictures which are to be displayed during specified portions of a broadcast. Traditionally, each of the events are forwarded to an associated device that is capable of displaying that event. Thus, the video clips are forwarded as a list of events to a video playback unit, the text is forwarded to a text display unit, the still pictures are forwarded to a still picture unit, and so on. Each of these units is individually controlled by a broadcast person. During broadcast, the selected events are integrated with live action. The integration of events is typically controlled by a director, with the director controlling the operations of each of the broadcast persons at each of the display devices. One drawback of such a production system is that the increased manpower required to control the sequence of events requires close synchronization by the director and introduces opportunities for error. Summary of the Invention A machine control subsystem is used to provide centralized control during broadcast in a 5 news production system. A director workstation is coupled to a server, which is in turn coupled to any number of external devices that may be used during broadcast to air and display video or audio images. The director workstation uses a graphical user interface to display a list of events that are to occur during the production. The events include all events that are to be aired by any of the coupled devices. A director at the workstation may then control the production of all of 0 the types of events from a centralized control station. To select an event or a sequence of events for production, the director may simply select the event from an event list, and the event will be broadcast without the director having to know particular characteristics of the device to display WO 99/05821 PCT/US98/15795 -3 the event. The director workstation is coupled to a server. The server includes device manager software for interfacing with and controlling the operation of all of the externally coupled devices. By providing a centralized communication pathway between the director workstation and the devices, the server may forward status information about the devices to the director 5 workstation to facilitate redirection of resources by the director. One aspect of the invention is a news production system including a workstation coupled to a network for receiving a list of events to be broadcast during a newscast. At least one server is coupled to the workstation for controlling a plurality of different types of devices. The server also receives commands from the workstation identifying at least one event for broadcasting on 0 one of the different types of devices. Brief Description of the Drawings Figure 1 is a block diagram of a news production system incorporating the present invention; and 5 Figure 2 illustrates an exemplary graphical user interface for use in providing centralized control during broadcast in the news production system of Figure 1. Description Figure 1 illustrates one embodiment of a digital multimedia newsroom production system o 10 that is shown to include three major subsystems, a video production system 12, a client server/core newsroom system 14 and an operations system 18. The components of the core newsroom system and the are interconnected using a digital network 20. The digital network 20 is implemented using an Ethernet network having a data rate greater than or equal to 100 Mb/s. The network 20 couples the three operative components 12, 5 14 and 16 together via bridges 22 and 24. The operation of each of the subsystems 12, 14 and 16 of the system are described in greater detail below. Video Production Subsystem The video production system 12 provides audio/video capture, media data editing, and management and control of high quality multimedia data suitable for broadcast. Multimedia o data is defined as any form of information that can be represented in a digital form. The video production system includes a capture manager 24 coupled to control a news cutter/media WO 99/05821 PCT/US98/15795 -4 recorder 23. One example of a media recorder is the Avid Media RecorderTM available from Avid Technology Inc., Tewksbury, Massachusetts. The capture manager controls the conversion of the NTSC or PAL video into digital form, and forwards the converted data to an encoding station 26. At the encoding station 26, the video data is compressed in either JPEG or MPEG format the compressed audio/video is forwarded over network 20 via bridge 22 to a number of news servers 42a-42c in server subsystem 14. An asset manager 27 controls the indexing and storage of the compressed video data. A storage array 30 is also included for retaining archived audio/video data. Access to images stored in RAID array 30 is controlled via a browse server 32. A digital playback system 31 is coupled via a switch 38 to components in the video subsystem. The digital playback system 31 is a digital, disk-based playback system that manages the broadcast to air of multimedia data produced and stored within the video production system. The digital playback system plays materials stored either locally, at storage devices 30, or on the media server 42a-42c. In a preferred embodiment of the present invention, the digital playback system is implemented using an Avid AIRPLAY broadcast system available from Avid Technology, Inc. The number of graphics workstations, such as graphics workstation 28, are used for generating and editing graphics material for broadcast and storage in the video production system. In a preferred embodiment, the graphics workstation 28 is implemented using a MATADOR workstation available from Avid Technology. More details about the operations and features of the video subsystem 12 are described in co-pending application entitled A Multimedia System with Improved Data Management Systems, filed on even date herewith, by Jason Loveman, et al. Server Subsystem The server subsystem 14 is shown to include a number of journalist workstations, such as video editing workstation 40, coupled to the network 20 via a 10 base T hub 37 to news servers 42a - 42c. The number of journalist workstations used in the server subsystem is based on several factors including the amount of network activity generated by each user of the workstations and by the amount of delay each user will tolerate in accessing the system. In a preferred embodiment of the present invention, each journalist workstation 40 is implemented using an MPC III compliant workstation.

WO 99/05821 PCTIUS98/15795 -5 The journalist workstation provides access to multimedia data from a variety of sources and includes the tools (i.e., software) necessary to create a multimedia story board of a news story for broadcast. The multimedia data available to the journalist includes the low resolution MPEG video data captured by the media recorder 23. Each of the journalist workstations 5 advantageously includes a video port (not shown) for receiving video from, for example, a video tape recorder. Each of the journalist workstations also includes a serial port (not shown) for controlling the video tape recorder. Each of the news servers 42a-42c provides the management and storage of multimedia data in the newsroom environment. The news servers are configured as distributed processors 0 with mirrored data bases to provide maximum reliability and performance. Other centralized functions, such as communications functions, are managed by the news servers 42a-42c. The news servers may be implemented, for example, using an Avid NewsServer workstation available from Avid Technology, Inc. The news servers have external connections 122 for providing access to news wire services and to allow remote access to the news server from users 5 external to the core news room system. The server subsystem 14 may also include an interface such as interface 45 for providing connections to the digital network 20 for user terminals such as user terminal 46. The user terminals may be one of several different terminals used in prior art systems primarily for text processing and communications functions. In addition, a number of device controllers (not 0 shown) may also be coupled to the digital network 20 to provide control of several multimedia devices, such as teleprompters, from the journalist workstations. The journalist workstation 40 executes numerous functions in the news room. For example, the workstation 40 captures, stores and catalogs news wire text. It is used to create, store and catalog news story text. It is also used for capture, editing, playback, storage and 5 cataloging of high resolution video. It is used for editing and browsing of low resolution video. All these functions are used by a journalist at the workstation 40 to create a rundown. The rundown is a list of events that are to occur during the broadcast. The rundown is forwarded from the server subsystem 14 over to the machine control subsystem 16. Machine Control Subsystem 0 The machine control subsystem 16 includes three main functional components; one or more machine control servers 50, a technical directors workstation 52, and a serial port connector 54. The serial port device 54 provides connection pathways between the machine WO 99/05821 PCT/US98/15795 -6 control server and a number of other external devices. One serial port device for use in such a system is the Avanstar connector provided by Digi International, Inc., Eden Prairie, Minnesota. Device manager software 51, executing at the machine control servers 50 and, optionally on devices coupled to the machine control servers 50, controls the transmission of data to the external devices, such as still image store 56, character generator 58, video switcher 60 and video playback unit 62. The still image store is for controlling the display of still images, stored in the new servers 42a-42c. The character generator 58 stores and displays character information, received over network 20, such as text for teleprompters, and text that is to be overlaid on an image (such as identification characters over a still image or video clip). The video switcher 60 controls the display of video sequences stored in news server 42a-42c. The video playback unit 62 cntrols the issuance of audio during production. The functions of the technical director workstation 52 and the machine control servers 50 are described in more detail further below. Technical Directors Workstation The Technical director workstation 52 provides intelligent, user-friendly, interactive 5 control for newsroom production. The workstation may be, for example, an Avid Workstation running Windows NT. The control of components coupled to the Technical Directors Workstation is managed via a graphical user interface, such as that shown in Figure 2. The graphical user interface 70 allows the user to select events for playing on one of the devices such as still store 56 or character generator 58. The user may select one event, or may D alternatively select a range of machine events which will then play in sequential order without stopping. An event list is provided at the user interface 70 for displaying a portion of the run down received from the server 14. The data sent from the server 14 to the machine control subsystem 16 is communicated using a suitable protocol. The protocol is a packet type protocol. The information for each event may include, for example, a short name, or 'slug' 70b identifying 5 the subject matter of the event, a status 70c of the event, a tape ID 70d for referencing a video segment, a total time 70e of the event, a back time 70f, indicating the expected completion time of the event, a device field 70g, indicating the device on which the event is to be broadcast, a device status field 70h, indicating whether the device is currently in use, and a channel field 70i for indicating a channel on which the audio is to be broadcast. 0 A user may select an event, or sequence of events, by depressing a mouse button over an event, and dragging the mouse to select a section of events for processing. Some available functions that can be performed on the event are outlined in the machine pull down menu 80.

WO 99/05821 PCTIUS98/15795 -7 These functions include a play, cue, stop, reload, and block play function. The play function causes the event to be forwarded to the device listed in field 70g of the event. The cue function also forwards an event to the device listed in its associated device field 70g, placing it behind a currently executing event. The stop function 80c stops the display of the event at the device, and the reload function 80d allows for the event to be reloaded at the device. The block play function 80e allows for multiple events to be sent as a 'block' over to the server. In one embodiment, a user selects an event, and all of the events suceeding the selected event, up until a break point, are forwarded as a group to the server for processing. Thus the block play function allows for all of the events until the next break period to be played in sequence, without intervention from the director. Thus the block play is a useful tool for displaying sequences of video clips for news programs that principally use pre-taped programming. The play function 80a is also used to support compound events. Compound events are a series of machine control events that are linked together at the journalist workstations 40 and appear as one single event that incorporates all of the linked events. Compound events differ from block play, since in block play there are still multiple events that are joined together to play sequentially. The introduction of compound events allows for events, such as a video, and a character generation font, that are broadcast together, to be linked together as one event. Each of the events in the compound event series include an optional offset from the beginning, so that they do not all start at once. For example, a video clip may include sound bits from Alan Green and Bob White. A compound event for the clip may look like: play the clip; at 13 seconds after the start, display "Alan Green" for 10 seconds; 5 at 32 seconds after the start, display "Bob White" for 10 seconds; when the video clip is finished, display a still store image. The 'block play' menu item in the graphical user interface is provided for attaching events together. However most compound events are received explicitly from the new server 14 as part of the run down. The compound event may be explicitly described, as in the above 0 example, or described using a template of events, time offsets, and source materials. Thus, the compound events allow for support of common operations such as bringing up a title over a video clip at the same time the clip starts playing. The compound event allows WO 99/05821 PCTIUS98/15795 -8 multiple events to be represented as a single, macro event in a rundown list. In addition to displaying the rundown list, the graphical user interface 70 additionally includes a device status window 82 for displaying the status of the attached devices. By displaying the status of the devices, a user may readily determine whether or not that device is 5 available, thus giving the user the flexibility to alter event/device assignments according to the present system status. A portion of one rundown list is shown in Figure 2. Although only one rundown is displayed at a given time, multiple rundowns may be stored at the technical director workstation. When a first rundown has completed, the director may select a next rundown from a list of o rundowns, or alternatively rundowns may be automatically provided to the graphic user interface 70 in sequence. By providing control over all of the devices using a graphical user interface, control over the entire production is centralized at one workstation, thus requiring intervention by only one operator and removing the ambiguity presented by having multiple operators controlling multiple 5 external devices. In addition, by providing control using a graphical user interface, device control may be automated, and the manual operator need not know the particulars of controlling each of the specific external devices. An additional benefit of providing the status of all of the events and devices coupled, to the system in one graphical user interface is that it allows the director to over ride device o assignments and channel assignments based on the status of the devices as viewed at the interface. For example, the director may override the channel assignment to maintain continuity if there are two back to back audio clips directed to different channels. In addition, if one of the devices becomes unavailable, the director may easily modify the run down to change device assignments of subsequent events. 5 Although an example graphical user interface and associated functions have been described and illustrated in Figure 2, it should be understood that the user interface may be augmented to provide other types of functions that facilitate broadcast production. Thus the graphics user interface is meant only as an example, and not intended to be limiting, but rather intended to introduce a concept of centralized production control. 30 Machine Control Server The machine control server 50 is the central contact point for all of the components in the machine control subsystem 16. The machine control server 50 also maintains a composite list of WO 99/05821 PCTIUS98/15795 -9 events as defined by the rundown. When an event or group of events is selected by the technical director workstation, the event data, including device, event ID, or slug, and channel information, is forwarded to the machine control server 50. The machine control server forwards the received events in the form of commands, based 5 on the received type of event and device data, to the associated device using the device manager software 51. Device manager software is typically provided with an associated device, and includes protocol and data management routines for controlling communication with that external device. Device manager software that is to be used with the MC servers should also be able to communicate with the server 14 according to the protocol. As a result, device status 0 information may be made readily available to the user to facilitate device and channel reassignment during broadcast. The machine control server thus provides a centralized communication pathway for exchanging data between the server 14, the technical director workstation 52 and each of the coupled devices 56-62. Although only 8 devices are shown coupled to the machine control 5 server 50, in one embodiment the server 50 is capable of supporting 32 devices simultaneously. As technology advances, it is envisioned that future embodiments will include the capability of supporting more than 32 devices. As shown in Figure 1, in one embodiment the machine control subsystem may include dual servers 50a and 50b, configured to provide redundant support. Thus, in the event of a o failure of one of the servers 50a or 50b, control is switched to the other server. This process is referred to as a 'warm start'. The warm start, which allows for a continuance of processing by the redundant server within two minutes, occurs in response to a combination of commands entered at the technical directors workstation 32. Serial lines to individual devices are switched to the redundant server using the AB switch 65. The switching operation may occur manually, Z5 or may be performed automatically via a command from the technical director workstation. The machine control server additionally supports the concept of virtual devices, as seen by the journalists. When writing a story, a journalist can specify a machine control event for a named device that actually represents some combination of devices. For example, two AirPlays and a switch can be made to look like one dual channel Airplay to the journalist. 30 Although the machine control subsystem has been described above with reference to an Avid News broadcast system, it should be understood that it may be readily adapted to support otheer types of news broadcast systems. For example, the machine control subsystem 16 can WO 99/05821 PCTIUS98/15795 -10 also be used with NewsView, a news product for Novell® computer systems. A computer process would be provided for monitoring the News View database, and communicating changes to the machine control server 50 using the Avid Machine control protocol via network 20. Thus a method and apparatus for providing centralized control and communication during 5 the production phase of a news broadcast has been described. A graphical user interface allows for a director, at one workstation, to control a variety of types of events that occur on multiple types of devices. By providing centralized control at the workstation, the director may track the availability of resources, and update the device selection as required during live production. In addition, the graphical user interface allows for sequences of events to be selected to simplify the 0 production of multiple, pre-generated video clips. The foregoing embodiments are merely illustrative and not meant to be limiting, having been presented by way of example only. Numerous modifications and other embodiments are within the scope of one of ordinary skill in the art and are contemplated as falling within the scope of the invention as defined by the appended claims and equivalents thereto. 5

Claims (10)

1. A news production system comprising: a workstation coupled to a network for receiving a list of events to be broadcast during a newscast; and at least one server, coupled to said workstation, for controlling a plurality of different types of devices and for receiving commands from said workstation identifying at least one event for broadcasting on one of said plurality of different types of devices.

2. A news production system, comprising: a journalist workstation for providing a play list indicating a series of events to be displayed during a broadcast; a director workstation, coupled to said journalist workstation, for receiving said events, said workstation including an interface for allowing a user to select one of said events of said play list to be broadcast; and a server, coupled to said director workstation and to a plurality of different types of broadcast devices, for controlling the broadcast of an event received from said director workstation on one of said broadcast devices.

3. The news production system according to Claim 2, wherein said interface further comprises: means for displaying said list of events to said user; means for said user to select one of said list of events from said display; and means for forwarding said selected event to said server. 5

4. The news production system according to claim 2, wherein said interface is a graphical user interface, comprising event information and device status information, and wherein said selection of one of said event causes said event and at least one command associated with said event to be forwarded to said server. o

5. The news production system according to Claim 4, wherein said graphical user interface additionally includes a command menu, for selecting said command to be forwarded to said server with said event. WO 99/05821 PCT/US98/15795 -12

6. The news production system according to claim 4, wherein said command menu includes a block play command for causing multiple selected events to be forwarded together to said server. 5

7. The news production system according to claim 4, wherein said event is a compound event, comprising multiple events, and wherein said selection of said compound event results in said multiple events being forwarded together to said server.

8. The news production system according to claim 4, wherein a selected device is also D forwarded with said event from said workstation to said server.

9. The news production system according to Claim 8, wherein said selected device is a virtual device representing more than one of said plurality of different devices, and wherein said selection of said virtual device results said server forwarding the event to at least one of said 5 devices associated with said virtual device name.

10. A news production system comprising: a graphical user interface listing a plurality of events to be broadcast on one or more of a plurality of existing, said events subdivided by break points; and 0 a block play command, at said graphical user interface, for forwarding a plurality of successive events between a selected event in said listing and an succeeding breakpoint to one or more broadcast devices.