BR9917091B1 - "METHOD IMPLEMENTED ON A MACHINE, APPARATUS, VIDEO EDITING SYSTEM, ELECTRONIC EQUIPMENT, AND MACHINE READING MEDIA THAT HAS A PLURALITY OF INSTRUCTIONS IN IT TO IMPLEMENT THE VIDEO EDITING SERVICES." - Google Patents

Description

Report of the Invention Patent for "METHOD IMPLEMENTED ON A MACHINE, APPARATUS, VIDEO EDITING SYSTEM, ELECTRONIC EQUIPMENT, AND MACHINE READING MEDIA THAT HAS STORED IN IT A PLURALITY OF INSTRUCTIONS FOR IMPLEMENTING THE SERVICES." BACKGROUND OF THE INVENTION FIELD OF THE INVENTION The present invention relates to the field of entertainment systems and in particular to a method and apparatus for editing a video recording with audio selections.

KEY INFORMATION

Numerous advances have been made in recent years in the field of consumer electronics in general, and entertainment systems in particular. Indeed, many households in the United States now have a television and video recording / playback device, for example, a VCR, a versatile digital disc (known as a digital video disc, or DVD), laser disc, and the equivalent. In addition, more and more homes now have camcorders, also commonly referred to as a cam-corder, with which they make their own films, documenting audio and video, a child's birthday, soccer games, vacations and equivalent. Similarly, while not entirely replacing "still" photographs, many marriages are now captured on videotape as well as in a photo album.

The most advanced models of these prior art camcorders have features which allow the user to edit and manipulate the recording. For example, some camcorders allow a user to manipulate the recording to add a title, or credits, at the beginning of the recording. Some camcorders allow a user to "lighten / darken", for example, to slowly focus from a "blurry" image to a clear image, or from total darkness to a clear image, and vice versa. More advanced video recording / playback features include features that allow a user to edit a recording to add a title or credits to a video recording.

None of the above AN devices, however, allow a user to edit a video recording to add audio content, such as an enhanced soundtrack, poem, sonnet, or other audio without overwriting the original audio content of the recording. of video. For example, if someone wanted to add background music to their ocean video recording using a typical prior art home A / V equipment, they would have to rewrite the audio track of the video recording, thereby replacing the sound. of the ocean with "background" music, which becomes the primary audio content of video recording. Those skilled in the art will recognize that an audio "mixer", for example, a device which receives two signals and combines them into a composite signal common to professional editing equipment could well solve this dilemma, allowing an editor to add the recording. video with an audio. However, audio mixers typical of professional editing systems are expensive, difficult to install and use. That is, typical prior art audio mixers require the user to select audio content to add to the video recording, the user needs to synchronize the audio selection with the primary audio content, and adjust the various levels (for example). , the volume) of the audio.

Thus, even if the cost of such a professional mixer was not prohibitively expensive, the user of a typical home entertainment system would likewise find it excessively difficult to use a professional audio mixer to edit his home movies. Consequently, while it may be desirable to be able to add background music or other audio effects to home movies, the consumer electronics industry has so far failed to satisfy this need.

Thus, there is a need for a method and apparatus for editing a video recording with audio selections that is not overloaded with the above deficiencies and limitations associated with the prior art.

SUMMARY OF THE INVENTION

In accordance with the teachings of the present invention, a method and apparatus for editing a video recording with audio selections is provided. In a first embodiment of the present invention, a method for editing a video recording includes receiving a signal that includes video content and analyzing the video content of the received signal to identify the visual attributes which characterize the video content. Based, at least in part, on the identified visual attributes of video content, an audio selection with which to increase the received signal is identified from a plurality of available audio selections.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention will be described by way of exemplary embodiments, but not limitations, illustrated in the accompanying drawings in which like references denote similar elements, and in which: Figure 1 is a block diagram illustrating an entertainment system. embodying the teachings of the present invention;

Figure 2 is a block diagram of an A / V editing system according to one embodiment of the present invention: Figure 3 illustrates a flow chart of an example of a method for automatically enlarging a video recording with a selection of audio in accordance with the teachings of the present invention;

Figure 4 is an illustration of an example of a video channel of a received A / V signal showing the quantization fields used to characterize the visual attributes of the A / V signal according to one embodiment of the present invention;

Figure 5 illustrates a flow chart of an example of a method for automatically characterizing the primary audio content of a video recording, according to one embodiment of the present invention;

Figure 6 is an illustration of an example of an audio selection database according to one embodiment of the present invention;

Figure 7 is a block diagram of an exemplary computer system for use as an A / V editing system in accordance with one embodiment of the present invention; and Figure 8 is a block diagram illustrating an exemplary software architecture for implementing an A / V editing system in accordance with one embodiment of the present invention.

DETAILED DESCRIPTION

In the following description, for purposes of explanation, specific numbers, materials and configurations are presented to provide a complete understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without specific details. In other cases, well-known features have been omitted or simplified for ease of explanation. Further, for ease of understanding, certain steps of the method are outlined as separate steps, however, these separately outlined steps should not be considered as necessarily dependent on order in their performance.

Referring to Figure 1, a block diagram illustrating an example of an entertainment system incorporating the teachings of the present invention is presented. According to the embodiment illustrated in Figure 1, the entertainment system 100 is shown including an audio / video (A / V) editing system 128 incorporating the teachings of the present invention. As will be described in more detail below, in accordance with one embodiment of the present invention, the A / V editing system 128 receives a signal from a video recording / playback device containing video content with a video stream and, optionally, audio content within an audio stream of the received signal (hereinafter referred to generically as an A / V signal), after which the A / V editing system 128 parses the content. A / V signal that identifies the visual attributes which characterize the video content and, based at least in part on the identified visual attributes, the A / V 128 editing system identifies an appropriate audio selection from a plurality of available audio selections with which to increase the received A / V signal. Accordingly, one skilled in the art will appreciate that the entertainment system 100 includes the innovative A / V editing system 128 provides a user of such an entertainment system with a means for automatically editing and enhancing home movies and other video recordings with audio selections automatically chosen.

As shown in the illustrated exemplary embodiment of Figure 1, the A / V editing system 128 may well be used with a wide variety of A / V components. According to Figure 1 the entertainment system 100 is shown comprising the signal routing multiplexers 108 and 112, a plurality of video recording / playback devices, for example a video cassette recorder / player ( VCR 116, a versatile digital disc (known as a digital video disc, or DVD) 118, a laser disc 120, a camcorder 122 and the equivalent, a television / monitor 126, and a plurality of cumulatively referred to audio components. like audio system 132, each communicatively coupled with system 100 as shown in Figure 1. Except for the A / V editing system 128 incorporated with the teachings of the present invention, each element of system 100 is intended to represent a wide variety of commonly available A / V components and, as their respective functions and features are well known in the art, they need not be further described herein.

As illustrated in the entertainment system 100 of Figure 1, the A / V signal may emanate from any of a number of sources. In the illustrated exemplary embodiment of Figure 1, entertainment system 100 is capable of receiving an A / V signal from wireless sources and / or from wired sources. That is, the A / V editing system 128 may receive the A / V signal from any of a number of broadcast sources including, for example, television broadcasts 103 received by antenna 102, or broadcasts. 105 received by satellite dish 104. Similarly, entertainment system 100 also receives A / V signals from wired sources such as, for example, Internet resources, Intranet resources, and cable television broadcasts. Thus, according to the illustrated embodiment of Fig. 1, line 106 is intended to represent any of a variety of line transport means including, but not limited to, a single old telephone service line (POTS). , an Integrated Services Digital Network (ISDN) line, a cable line, an Ethernet line, a T1 / E1 line, etc. which provides an A / V signal from a corresponding variety of wired services. Similarly, the A / V editing system 128 may receive the A / V signal from any of the plurality of video recording / playback devices (116-122) described above. In an alternative embodiment, the television / monitor 126 and A / V editing system 128 may well receive an A / V signal transmitted directly from individual antenna / wire sources, or from MUX 108 via line 110. Accordingly, those skilled in the art will appreciate that system 100 is if not a definite example to illustrate the diverse nature of the signal sources available for the A / V 128 editing system, and that larger or smaller capacity systems may well be suitable. replaced without departing from the spirit and scope of the present invention.

In one embodiment, the A / V editing system 128 may be a computer system incorporated with the teachings of the present invention, as will be further discussed with respect to Figure 7 below. In another embodiment, the A / V editing system 128 may be a "pickup" box equipped with the required processing power incorporated with the teachings of the present invention. Alternatively, the A / V editing system 128 may well be incorporated into individual elements within the system 100 (e.g., the television system, or the VCR). Thus, according to the illustrated exemplary embodiment of Figure 1, system 100 is intended to represent any of a number of entertainment systems found in many homes that are capable of receiving an AN signal from any of a number of alternative sources.

Having introduced the concept of the above innovative AN 128 editing system within the context of entertainment system 100, Figure 2 provides a block diagram illustrating an exemplary architecture of an A / V 200 editing system suitable for use in entertainment system 100, incorporating the teachings of the present invention. In the illustrated exemplary embodiment of Figure 2, the A / V editing system 200 is shown comprising a video analysis module 202, an audio analysis module 208, a controller 206, a display device 216, an input device 218 and audio files 212, each of which is communicatively coupled as shown. Although audio files 212 are shown in the exemplary embodiment illustrated in Figure 2 as being external to controller 206, those skilled in the art will appreciate that such audio files may well be stored in a mass storage device (not shown) within the controller. 206. Further, those skilled in the art will appreciate that in alternative embodiments, audio files 212 may well be located at a remote location accessible via the Internet and line 106, or audio files 212 may well be located in a computer system. audio (e.g., audio system 132), wherein line 214 represents the interconnection between the A / V 200 editing system and the audio system. Similarly, in an alternative embodiment, display device 216 may be removed by using television (TV) / monitor 126 as the video display for A / V editing system 200.

Given the architectural description of the exemplary A / V editing system provided in Figure 2, an exemplary method for automatically enhancing video recording with an audio selection, according to the teachings of the present invention, will be developed with reference to the In particular, for ease of explanation and not limitation, the operation of the A / V 200 editing system will be developed with reference to Figure 3 and continued reference to Figure 2. As illustrated in Figure 3, The exemplary method for automatically enhancing a video recording with an audio selection begins with the initial step of determining whether the augmentation function is enabled, step 302. This means, in one embodiment, that a user of the A editing system / V 200 may disable the audio boost feature via user input device 218. If, in step 302, it is determined that The audio boost feature of the A / V 200 editing system has been disabled, the editing section continues without automatic audio boost, step 304.

If, however, the audio boost feature of the A / V 200 editing system is enabled, the A / V 200 editing system loads a predetermined amount of video recording via an A / V signal for analysis, step 306. In one embodiment, the A / V editing system 200 loads the total video recording into accumulators (not shown) within video analysis module 206 and audio analysis module 208 for audio analysis and augmentation. . In an alternative embodiment, the A / V editing system 200 loads a subset of the entire video recording for analysis and augmentation. More specifically, according to the latter embodiment, the A / V 200 editing system loads two- or three-minute segments, i.e. samples, of video recording into the analysis and augmentation accumulators, where the sample length corresponds to at an average length of an audio file in audio files 214. In another embodiment, the A / V 200 editing system loads individual scenes of video recording into accumulators. In one embodiment, the A / V editing system 200 performs an initial video recording analysis to identify each of a plurality of scenes comprising video recording, and incrementally loads the individual scenes in the analysis accumulator to the audio boost, to be described more fully below.

Having loaded the predetermined amount of video recording via an A / V signal for analysis at step 306, the received AN signal is simultaneously analyzed by video analysis module 202 and audio analysis module 208 at steps 308 and 310, respectively. That is, according to the illustrated embodiment of Figure 3, the A / V signal is simultaneously provided to the video analysis module 202 which analyzes the video content of the A / V signal as well as to the video module. audio analysis 208 which analyzes the audio content of the A / V signal. In particular, the video analysis module 202 analyzes the video content embedded within the video stream of the received A / V signal and characterizes the video content in terms of any of a number of visual attributes, step 308. In one embodiment, the video analysis module 202 "splits the received video stream into a number of quantization regions, or quadrants, and analyzes the video content within each of the video stream quantization regions. An example A video stream divided into such quantization regions is shown in the illustration of Figure 4.

Moving on to Figure 4, an illustration of a video stream with its corresponding quantization regions is shown. In particular, video stream 400 is shown comprising a number of "frames" (e.g., a predetermined amount of video stream) of the video stream referenced as 402a, 402b through 402n, respectively. As illustrated in Figure 4, the quantization regions of table 402b are shown as quantization region 1 (Q-) 404a to quantization region 9 (Qg) 404n. Thus, according to an exemplary embodiment of the present invention, the video analysis module 202 analyzes the video content within each of the quantization regions of each video stream frame to characterize the video content in terms of its attributes. visual.

In one embodiment, for example, the video analysis module 202 analyzes each of the quantization regions 404a through 404n of the video stream 400 for color attributes, for example, if the video content of the received AN signal is of a "cold" color (blues, whites), a "warm" color (reds, yellows), or a "warm" or "earthy" color (browns, oranges), etc. and issues a scale from 0 ("cold" (white)) to 10 ("hot" (red)), with the appropriate gradations between them. In another embodiment the video analysis module 202 analyzes each of the quantization regions 404a through 404n of the video stream 400 for visual lighting attributes, for example, whether the video content is "bright" or "dark". In another embodiment, the video analysis module 202 analyzes each of the quantization regions 404a through 404n of the video stream 400 for visual content and motion attributes, for example, whether the video contains a city or field landscape. , whether the video contains people, and whether they are active or sedentary. An example of a system for quantifying motion / action within video content is described in Co-pending US Patent Application No. 08 / 918,681 entitled "Video Data Compression Bit Rate Control" by commonly designated Adnan Allatar to the signatory of the present invention. In yet another embodiment, the video analysis module 202 analyzes the quantization regions 404a through 404n of the video stream 400 for each of the aforementioned visual attributes.

Returning to the exemplary method shown in Figure 2, in addition to video analysis step 308, audio analysis module 208 analyzes the audio content (hereinafter the primary audio content), if any, received in the audio stream. audio from the received A / V signal and identifies the audio attribute information which characterizes the primary audio content embedded in the audio stream, step 310. For the illustrated embodiment, the purpose of identifying the audio attribute information in step 310 is for determining the level, for example, the volume, of the augmented audio selection thereby ensuring that primary audio content, if any, is not "muffled" or "drowned" by the relative level of the augmented audio selection. An exemplary method for analyzing the primary audio content of the received A / V signal (e.g., step 310) is shown in Figure 5.

Referring to Figure 5, an exemplary method for analyzing the primary audio content of the received AN signal is shown in accordance with one embodiment of the present invention. In the illustrated exemplary embodiment of Figure 5, audio analysis begins with audio analysis module 208 determining whether the received AN signal includes an audio stream containing audio content (i.e., primary audio content), If the audio analysis module 208 determines that the audio stream does not carry any audio content, an indication that the audio selection last selected by the A / V 200 editing system will provide the only audio content for When recording, the audio analysis module 208 provides a level indication to the controller 206 via line 214, step 504. In such a circumstance, the level indication will adjust the volume of the audio selection chosen by the edição editing system. / at a "high" level, as it will provide the only audio content for this segment of video recording. If, however, audio parsing module 208 determines that the ΑΛ / received signal contains audio content, step 502, then the following audio parsing module 208 determines whether primary audio content contains voices, step 506 In one embodiment, the audio analysis module 208 employs any of a number of available speech recognition devices with which it performs this task.

If, in step 506, audio analysis module 208 determines that primary audio content consists of voices, audio analysis module 208 issues a level ID to controller 206 via line 214 which ensures that voices do not alternatively, if at step 506 the audio analysis module 208 determines that the primary audio content is not voices, a determination is made at step 510 if the primary audio content consists of music. In one embodiment, the audio analysis module 208 analyzes the relative levels and width of the frequency spectrum that characterizes the primary audio content to make this determination. For example, a spectral analysis in which primary audio content spans a broad frequency spectrum with large gradients within the spectrum that fluctuate over time, provides an indication that primary audio content consists of music. In an alternative embodiment, the audio analysis module 208 may be preprogrammed with a number of music selections against which to compare primary audio content.

Regardless of the analysis method, if audio analysis module 208 determines that primary audio content is not music, step 510, and has already determined that primary audio content is not voices, audio analysis module 208 determines that the audio selection last selected by the A / V 200 editing system should be background audio, and accordingly outputs a level indication to controller 206 via line 214 to ensure that the primary audio content is not " muffled "by selecting background audio, step 508. If, however, audio analysis module 208 determines in step 510 that the primary audio content consists of music, audio analysis module 208 provides this attribute information. 206 to controller 206 via line 214. Subsequently, controller 206 alerts the user of the A / V editing system 200 via display device 216 with the option to overwrite review primary audio content (for example, music). For continuity and ease of explanation only, the function in which the controller 206 provides the user of an A / V 200 editing system with this option is presented in the illustrated embodiment of Figure 5 as step 512, although those versed in will appreciate that this step may well be completed later in method 300 of Figure 3.

If the user chooses to overwrite the primary audio content of the received A / V signal, the audio selection chosen by the A / V 200 editing system becomes the primary audio content of the composite signal generated by the A / V editing system. V 200 and is therefore adjusted to an appropriate level, step 504. However, if in step 512, the user of the A / V 200 editing system chooses not to overwrite the primary audio content consisting of music, the section Editing continues without the audio increasing of this predetermined amount of the received A / V signal, step 514.

Therefore, according to one embodiment of the present invention, the content of the audio stream is merely analyzed for the purpose of adjusting a level by which to increase the received A / V signal. That is, in the illustrated exemplary embodiment, the function of the audio analysis module 208 is to identify the audio attributes of the primary audio content to ensure that the recording level (e.g., volume) of the augmented audio adjusted by the controller 206 do not "muffle" the primary audio content. Those skilled in the art will appreciate, however, that in other embodiments, analysis of audio content may well serve additional functions without departing from the spirit or scope of the present invention.

Continuing with the exemplary method of Figure 3, having received the attribute information from the video analysis module 202 in step 308, the controller 206 identifies an appropriate audio selection from a plurality of audio selections contained in the 212 based audio files, at least in part on the received visual attribute information, step 312. In one embodiment, the controller 206 relies on a database which references the appropriate audio selection based on any of a number of corresponding visual attributes. An example of a database suitable for use by controller 206 is shown in Figure 6.

According to the illustrated exemplary embodiment of Figure 6, database 600 is shown comprising a number of cross-referenced audio selections with a number of visual attributes. As shown in the illustrated exemplary embodiment of Figure 6, database 600 references audio selections against corresponding visual attributes in a two-dimensional database. In the illustrated embodiment, the y-axis is characterized by gender identification 602 of the audio selections. Those skilled in the art will appreciate that gender information 602 is only one of a number of alternative means by which information within database 600 can be organized. Thus, a number of suitable alternative proposals exist for organizing database information 600, without departing from the spirit or scope of the present invention. The x-axis of database 600 is characterized by cross-reference audio selections 604 against corresponding visual attributes such as, for example, color attributes 606, lighting attributes 608, and content / emotion attributes 610. According to one embodiment of the present invention, the A / V editing system 200 may be preloaded with a number of audio selections, wherein the audio selections stored in database 600 are cross-referenced against various attribute information. In another embodiment, the ΑΛ / 200 editing system provides a user interface through which audio selections can be added or deleted from audio files 212, wherein controller 206 automatically updates database 600 with additions. / deletions as appropriate. Still, those skilled in the art will appreciate that although database 600 is shown as a two-dimensional database, that is for ease of explanation only. That is, databases of greater or less complexity may be beneficially replaced by database 600, with a corresponding effect on the amount and complexity of the information contained therein.

Returning to the illustrated embodiment of Figure 3, having automatically chosen an audio selection with which to enhance video recording based at least in part on the visual attributes identified from the video recording in step 312, controller 206 alerts the system user 200 AN edit with the option to accept the audio selection from the controller, or rejecting it in favor of another selection, step 314. If, in step 314, the user accepts the audio selection from controller 206, controller 206 synchronizes audio selection with video recording, blending audio selection with primary audio content, if any, at levels automatically determined by audio analysis module 208, as appropriate, and outputs a composite signal comprising the signal AN signal received increased with the automatically identified audio selection. In one embodiment, while mixing the automatically identified audio selection with the primary audio content, controller 206 coordinates the "timing" (e.g., speed) of the audio selection with the movement rate identified in the video content, or over time of the primary audio content.

If, however, the user rejects the audio selection from controller 206 in step 314, the user is provided with an interface through which the user can access database 600 of the available audio selections, where the user chooses one. audio selection with which to increase video recording, step 316. In step 318, controller 206 determines whether the end of video recording has been reached. If gone, the method ends. Alternatively, if controller 206 determines that the end of video recording has not yet been reached, the method continues with step 306, and the next predetermined amount of video for editing is loaded into the ΑΛ / 200 editing system as appropriate. .

Therefore, according to the teachings of the present invention, the AN editing system 200 analyzes the video content of a received AN signal and characterizes the video content in terms of a number of visual attributes. Additionally, the AN 200 editing system analyzes the audio content of the received AN signal, for example, the primary audio content, to automatically identify a level at which to "mix" the audio selection with the primary audio content. Having developed the visual attribute information, the AN 200 editing system chooses an audio selection from a plurality of available audio selections based at least in part on the identified visual attribute information. Having identified an audio selection with which to increase the primary audio content, when receiving user approval the AN 200 editing system "mixes" the audio selection with the primary audio content to a level automatically determined according to the audio attribute information determined from the audio stream. According to one embodiment of the present invention, audio selection is a musical composition, for example, a song. In an alternate embodiment, audio selection is a poem, sonnet, or other lyrical composition that is automatically selected by the A / V 200 editing system to improve the arrangement shown by the visual attributes of the received A / V signal.

According to one embodiment of the present invention, the A / V editing system 200 is a computer system suitably configured to analyze a video stream and characterize the video content of a received A / V signal in terms of its attributes. based, at least in part, on those visual attributes in which the computer system identifies an audio selection with which to increase the received A / V signal, and generates a composite of the received A / V signal and audio automatically identified for recording. As will be described in more detail below, the A / V 200 editing system is intended to represent a broad category of computer systems known in the art. An example of such a computer system is a desktop computer system equipped with high performance microprocessor (s), such as the manufactured and commonly available Pentium® Pro processor, Pentium® Pro processor, or Intel Pentium® II processor. Corporation of Santa Clara, California; including any of a number of audio and video input and output peripherals / interfaces for receiving, digitizing, compressing and decompressing audio and video signals. It should be appreciated that the size and design of the housing for an A / V 200 editing system can be changed, allowing it to visually fit better with an entertainment system, for example entertainment system 100. Consequently, the A / V 200 editing system may well be incorporated within a "collection" box incorporated with the teachings of the present invention. Figure 7 provides a block diagram of a computer system (e.g., system 700) incorporated with the teachings of the present invention. In one embodiment, system 700 is an AN 128 editing system of Figure 1. In the illustrated embodiment, system 700 includes at least one processor (e.g., processor 702) and a cache memory 704 coupled together as shown. . Additionally, system 700 includes a high performance input / output (I / O) bar 706 and a standard I / O bar 708 as shown. Host bridge 710 couples processor 702 with high-performance I / O bar 706, while 712 I / O bar bridge couples high-performance I / O bar 706 to standard I / O bar 708. Attached to the high-performance I / O bar 706 are network / communication interface 724, system memory 714, audio / video interface card 730, A / V editor 732, and video memory 716. In turn, the display device 718 is coupled to video memory 716. Coupled to the standard I / O bar 708 are the mass storage device 720, the keyboard and a pointing device 722, and the I / O ports. 726. In one embodiment, the keyboard and pointing device are coupled to the standard I / O bar 708 with a serial communication interface cable, while in alternate embodiments they may be communicatively coupled with an infrared (IR) interface. or a radio frequency (RF) interface.

With continued reference to Figure 7, elements 702-730 perform their conventional functions as known in the art. In particular, network / communication interface 724 is used to provide communication between system 700 and any of a wide range of conventional networks, such as Ethernet, plug-in, Internet, etc. Similarly, the audio / video interface card 730 is used to receive transmission communications from any of a wide range of conventional wired and wireless transmission media such as RF transmissions, satellite transmissions, transmissions. by cable, etc. Mass storage device 720 is used to provide permanent storage for data and programming instructions to implement the functions described above, while system memory 714 is used to provide temporary storage of data and programming instructions. when run by processor 702. I / O ports 726 are one or more serial and / or parallel communication ports used to provide communication between additional peripheral devices which may be coupled to system 700 (e.g. stereo, high speakers, etc.). Collectively, the coupled elements in system 700 are intended to represent a broad category of hardware systems, including but not limited to commonly used Pentium® processor, Pentium® Pro processor, or commonly available Pentium® II processor-based computer systems. Intel Corporation of Santa Clara, California.

In one embodiment, the A / V editor 732 includes a video analysis module 202 and an audio analysis module 208 of the A / V editing system 200, while controller 206, display device 216 and user interface device 218 of the A / V editing system 200 corresponds to processor 702, display device 718, and keyboard and pointing device 722, respectively, of system 700 of Figure 7. In one embodiment, the input files Audio 216 are stored in mass storage 720, or are remotely located and communicatively coupled with system 700 via the network / communication interface 724. In one embodiment, system 700 receives an A / V signal from the network / communication interface 724 and / or 730 audio / video tuning interface, parses video content for visual attribute information, and automatically identifies an audio selection with which to increase the received A / V signal , according to the above teachings. In an alternative embodiment, system 700 receives an A / V signal via an antenna (not shown) coupled to one of the I / O ports 726 and automatically identifies an appropriate audio selection with which to increase the A / V signal. Received.

It should be appreciated that various components of the system 700 may be rearranged. For example, cache 704 may be in the chip with processor 702. Alternatively, cache 704 and processor 702 may be packaged together as a "processor module", with processor 702 being referred to as the "processor core". Furthermore, mass storage device 720, keyboard and sharpening device 722, and / or display device 718 and video memory 716 may not be included in system 700. In addition, peripheral devices shown coupled to the standard I / O bar 708 may, in alternative embodiments, be coupled to the high performance I / O bar 706; or, in some implementations only a single bar may exist with system 700 components being coupled to the single bar. Further, additional components may be included in system 700, such as additional processors, storage devices, or memories.

In one embodiment, instead of including a separate A / V editor 732, the innovative features of the present invention discussed above can be implemented as a series of software routines run by system 700 of Figure 7. These software routines run a plurality or series of instructions to be executed by a processor, such as processor 702 in system 700. Initially, the instruction sets are stored in a storage device, such as mass storage device 720. It should be appreciated that instruction sets can be stored on any conventional storage device, such as a floppy disk, a CD ROM, a magnetic tape, a versatile digital disc (DVD) (also referred to as a digital video disc), a laser disc, a ROM, a flash memory, etc. It should also be appreciated that instruction sets do not need to be stored locally, and can be received from a remote storage device, such as a server on a network, via the network / communication interface 724. Instructions are copied from the storage device. such as mass storage device 720 for system memory 714 and then accessed and executed by processor 702. In one embodiment, these software routines are written in the C ++ programming language. It should be appreciated, however, that these routines can be implemented in any of a wide variety of programming languages. In alternative embodiments, the present invention may be implemented on discrete hardware or firmware. For example, an application specific integrated circuit (ASIC) may be programmed with the above described functions of the present invention. Figure 8 is a block diagram illustrating software elements comprising an exemplary software architecture in accordance with one embodiment of the present invention. In particular, exemplary software architecture 800 is shown comprising an A / V editor application 802, an A / V editor agent 804 with an associated video analysis module 806 and an audio analysis module 808, and a operating system 810 with associated drivers, and a BIOS 822. As shown in the illustrated exemplary embodiment of Figure 8, the A / V editor application 802 interfaces with the A / V editor agent 804 and provides the user interface for A / V 128 editing system of Figure 1.

In one embodiment, the A / V editor 804 is coupled to and capable of obtaining information from database 812 and audio files 814. In an alternative embodiment, audio files 814 and / or database 812 are integrated modules of the A / V editor agent 804. As shown in Figure 8, the A / V editor agent 804 receives the video signal through a communication port via an appropriate trigger within the 810 operating system. In this embodiment, the video analysis module 806 performs the steps of analyzing the video content of the received A / V signal for any of a number of visual attributes, while the audio analysis module 808 analyzes the audio content of the received A / V signal (e.g. primary audio content), if any, to determine the relative level (e.g. volume) at which audio selection should be recorded in accordance with the teachings of the present invention described ac magnet. Based at least in part on input from video analysis module 806, A / V editor 804, accessing database 812, identifies an audio selection which substantially corresponds to the identified visual attributes of the video content of the signal. received, and upon user approval of the identified audio selection, the 802 A / V editor combines the received A / V signal with the identified audio selection at a level automatically selected by the audio analysis module 808 to provide an AΛ / composite signal including augmented audio selection which is output via an appropriate operating system trigger 810.

As alluded to above, BIOS 822 provides an interface between the operating system 810 and the various I / O devices attached to the hardware system. Operation system 810 is a software service which provides an interface between BIOS 822 and the agente / 804 editor agent as well as other software applications, if any, being executed by a computer system within which the present invention. is practiced (eg system 700). The operating system 810 provides an interface, such as a graphical user interface (GUI), between the user and the system controller. According to one embodiment of the present invention, operating system 810 is the Windows ™ 95 operating system available from Microsoft Corporation of Redmond, Washington. However, it should be appreciated that the present invention may be used with any other conventional operating system, such as, for example, other versions of Microsoft Windows ™ (for example, Windows ™ 3.0, Windows ™ 3.1, Windows ™ NT , or Windows ™ CE), Microsoft DOS, OS / 2, available from International Business Machines Corporation of Ar-monk, New York, the Apple Macintosh Operating System, available from Apple Computer Incorporated of Cupertino, California, the NeXTSTEP® operation available from Apple Computer Incorporated, or the UNIX operating system available from Santa Cruz Operations of Santa Cruz, California.

Thus, in accordance with the teachings of the present invention, a video editing system analyzes the video content of a video recording and, based at least in part on the visual attributes of the video content, automatically identifies an audio selection with which increases video recording to a recording level automatically selected by the A / V editing system that does not override the primary audio content, if any, of the video recording.

While the method and apparatus of the present invention has been described in terms of the embodiments illustrated above, those skilled in the art will recognize that the invention is not limited to the embodiments thus described. The present invention may be practiced with modifications and changes within the spirit and scope of the appended claims. For example, although shown as separate components, the A / V 128 editing system may well be incorporated into any of the system components (for example, a television / monitor / recorder / video player device). In addition, the A / V editing system 128 may not include all the elements shown in Figures 2 or 7, or may alternatively include additional elements without departing from the scope and spirit of the present invention. . Accordingly, the description should be viewed as illustrative rather than restrictive of the present invention.

Thus, a method and apparatus for editing a video recording with audio selections has been described.

Claims (22)

A method implemented on a machine comprising: receiving a signal that includes video content; analyze the video content of the received signal to identify the visual attributes of the video content; identifying an appropriate audio selection from a plurality of available audio selections with which to increase the received signal based at least in part on the identified visual attributes of the video content; characterized in that it further comprises: identifying a recording level at which to increase the received signal with the identified audio selection, including, (i) determining whether the received signal includes audio content, (ii) identifying an audio attribute indicating whether the audio content includes voices, whether the received signal is determined to include audio content, and (iii) selecting a recording level based at least in part on the identified audio attribute; and utilizing the identified audio selection and its identified recording level to augment the received signal with the identified audio selection to form a composite audio / video (A / V) signal comprising at least video content and audio selection. identified audio. Method according to claim 1, characterized in that analyzing the video content of the received signal includes characterizing the video content of the received signal in terms of visual color attributes. Method according to claim 1, characterized in that analyzing the video content of the received signal includes characterizing the video content of the received signal in terms of visual lighting attributes. Method according to claim 1, characterized in that analyzing the video content of the received signal includes characterizing the video content of the received signal in terms of visual motion attributes. Method according to claim 1, characterized in that analyzing the video content of the received signal comprises at least one of: characterizing the video content of the received signal in terms of visual color attributes; characterize the video content of the received signal in terms of visual lighting attributes; and characterizing the video content of the received signal in terms of visual motion attributes. Apparatus comprising: an input port through which the apparatus receives a signal having video content; a video analysis circuit coupled to the input port operative for analyzing the video content of the received signal to identify the visual attributes of the video content; characterized in that it further comprises: an audio analysis circuit coupled to the input port to determine whether the received signal includes audio content and to identify an audio attribute indicating whether the audio content includes voices, if it is determined that the received signal includes audio content; and a controller coupled to the video analysis circuit and audio analysis circuit operant for selecting an audio from a plurality of audio selections with which to increase the received signal based at least in part on the identified visual attributes of the audio. video content of the received video signal, the controller must also identify, based on the identified audio attribute, a recording level at which the received signal increases with the identified audio selection. Apparatus according to claim 6, further comprising a controller-coupled mass storage device operable for storing and retrieving each of the plurality of audio selections. Apparatus according to claim 6, characterized in that the video analysis circuit is operative to quantify the color visual attributes of the video content. Apparatus according to claim 6, characterized in that the video analysis circuit is operative to identify the visual illumination attributes of the video content. Apparatus according to claim 6, characterized in that the video analysis circuit is operative to identify the visual attributes of motion of the video content. Apparatus according to claim 6, characterized in that the video analysis circuit is operative to identify the color visual attributes, the lighting visual attributes and the motion visual attributes of the video content. Apparatus according to claim 6, characterized in that the controller is operative to determine the tagged audio selection at an automatically tagged recording level and mix the tagged audio selection with the primary audio content of the received signal. Apparatus according to claim 6, characterized in that the plurality of audio selections are stored in a remote audio system, communicatively coupled to the apparatus via an audio interface, wherein the controller is operative. to identify and retrieve the appropriate audio selections from the remote audio system through the audio interface. Apparatus according to claim 6, characterized in that the plurality of audio selections are stored on a network server, communicatively coupled to the apparatus through a network connection, wherein the controller is operative to identify and retrieve the appropriate audio selections from the network server through the network connection. A video editing system comprising: an input port through which the editing system receives a signal having video content; a video analysis circuit coupled to the input port operative for analyzing the video content of the received signal to identify the visual attributes of the video content; and further comprising: an audio analysis circuit coupled to the input port to determine if the received signal includes audio content and to identify an audio attribute indicating whether the audio content includes voices, if it is determined that the received signal includes audio content; a controller coupled to the video analysis circuit and audio analysis circuit operant for selecting an audio selection from a plurality of audio selections with which to increase the received signal based at least in part on the identified visual attributes video content, and the controller must also identify, based on the identified audio attribute, a recording level at which the signal received with the identified audio selection increases. System according to claim 15, characterized in that it further comprises a mass storage device coupled to the controller operable to store and retrieve each of the plurality of audio selections. System according to claim 15, characterized in that the video analysis circuitry is operative to quantify the visual color attributes of the video content, the visual lighting attributes of the video content and / or the attributes. motion visuals of video content. System according to claim 15, characterized in that the controller is operative to determine the identified audio selection at an automatically identified recording level and mix the identified audio selection with the primary audio content of the received signal. . System according to claim 15, characterized in that the plurality of audio selections are stored in a remote audio system, communicatively coupled to the editing system via an audio interface, wherein the controller is operating. -to identify and retrieve the appropriate audio selections from the remote audio system through the audio interface. System according to claim 15, characterized in that the plurality of audio selections are stored on a network server, communicatively coupled to the editing system via a network connection, wherein the controller is operative for identify and retrieve the appropriate audio selections from the network server through the network connection. 21. Electronic equipment comprising: a primary audio / video (A / V) functional unit operative to provide a signal that includes video content; and an attached video editing system responsive to the primary A / V functional unit, the attached video editing system including (i) an input port coupled to the primary A / V functional unit to receive the signal that has a video content, (ii) a video analysis circuit coupled to the input port operative to analyze the video content of the received signal to identify the visual attributes of the video content, characterized by the fact that it further comprises: (iii) an audio analysis circuit coupled to the input port to determine if the received signal includes audio content and to identify an audio attribute indicating whether the audio content includes voices, if it is determined that the received signal includes audio content. audio; and a controller coupled to the video analysis circuit and audio analysis circuit operant for selecting an audio selection from a plurality of audio selections with which to increase the received signal based at least in part on the identified visual attributes. video content of the received signal, the controller must also identify, based on the identified audio attribute, a recording level at which the received signal increases with the identified audio selection. Machine readable medium having stored therein a plurality of instructions for implementing video editing services, wherein video editing services include: a service for parsing the video content of a received signal to identify attributes video content visuals, a service for identifying an appropriate audio selection from a plurality of available audio selections with which to augment the video content of the received signal based at least in part on the identified visual attributes of the video content of the video. received signal; characterized by the fact that it further comprises: a service for identifying a recording level at which to increase the received signal with the identified audio selection, including: determining whether the received signal includes audio content, identifying an audio attribute indicating whether the audio content includes voices, if it is determined that the received signal includes audio content, and selects the recording level based at least in part on the identified audio attribute; and a service for utilizing the identified audio selection and its identified recording level to augment the received signal with the identified audio selection to form a composite audio / video (A / V) signal comprising at least the video content and the selection of i-dentified audio.