JP2005065191A - Movie metadata automatic creation device and movie metadata automatic creation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image metadata automatic creation device for automatically assigning metadata to a moving image.
A moving image input unit that inputs moving image data including audio data, an audio separating unit that extracts audio data from the moving image data, and time information on the moving image data in which the audio data is recorded, and a moving image A speech recognition dictionary in which a phrase for specifying a spatial position on an image and a phrase to become metadata are registered in advance, and a speech image is recognized from the speech data by referring to the speech recognition dictionary. The speech recognition means for separating and extracting the phrase that identifies the spatial position above and the phrase that serves as metadata, and converting them into character data, the spatial position on the moving image, and the character data of the phrase that serves as metadata And metadata storage means for associating and storing time information as metadata.
[Selection] Figure 1

Description

  The present invention relates to a moving picture metadata automatic creation apparatus and a moving picture metadata automatic creation program for automatically assigning metadata to a moving image.

  The price of digital video cameras and the use of IT in daily life have progressed, and moving images have become familiar, following text and photographs so far. In order to search for a captured moving image or to use it effectively as data later, it is conceivable that some kind of index is provided on the moving image to organize and manage it. In general, indexing is generally performed by relying on automatically entered data such as date and time, or by manually inserting the data while reproducing a moving image. Therefore, conventionally, attempts have been made to automatically index by the following methods (1) to (5).

(1) A method of using average color information of each frame constituting a moving image as feature information of the moving image (Non-Patent Document 1). In addition, there is a method of using a histogram of each frame as feature information for a long time.
(2) A method of searching for an arbitrary moving image scene using human words (Non-Patent Document 2). In this method, the positional relationship, movement, and change between objects are associated with human language in advance. Then, an arbitrary moving image scene is searched with human words by semi-automatically extracting an object in the moving image having a positional relationship, movement or change corresponding to the human language.
(3) “Video search method and apparatus using acoustic keywords” in which acoustic information corresponding to the video information of the video information is subjected to feature analysis and converted into an acoustic feature parameter time series (Patent Document 1). The keyword sound to be the video search key is subjected to feature analysis, converted into a keyword sound feature parameter time series, and the two are compared to search for the video information using the sound included in the sound information as a keyword.
(4) By comparing the audio in the video confirmed using voice recognition technology with text content such as the minutes, manuscripts, and presentation materials related to the video, the video for each slide of the minutes and presentation materials “Parliamentary video search system” (Non-patent Document 3) that can automatically create metadata to which time information from the head of the video is added.
(5) “Video knowledge management system” (Non-Patent Document 4) that enables text search by automatically recognizing and generating metadata.
Japanese Patent Laid-Open No. 06-68168 IEICE IEICE technical report IE98-83, "A study on moving image retrieval using image print information" IEICE Transactions D-II Vol.J80-D-II No.6 pp1590-1599 (1997) http://www.nec.co.jp/press/en/0306/0502.html http://www.photron.co.jp/esolution/products/powerindex/index.html

By the way, considering the use of “search”, the index only needs to have “character information” to be searched and “time information” indicating the position on the time axis, and it can be said that the conventional technology is sufficient. However, moving images are used not only for taking pictures and watching them as they are, but also for editing and using them for teaching materials. Recently, there are indexes that have not only simple information such as moving image character information + time information but also character information + time information + space information. Some of them can be pasted and played back. When such an apparatus is used, information on characters can be referred to in addition to information on moving images, so that it is expected to be used effectively in various teaching materials such as sports, academics, manuals, and business. In addition, a method has been realized in which control information at the time of playback such as pause or slow playback is incorporated as an object in a moving image, and a video provider conveys more information to a viewer.
However, in the conventional technology, since only “text information” and “time information” are acquired in mind, the index or object metadata with spatial information in the new usage of moving images is used as a moving image. However, there is a problem that it cannot be automatically inserted. Therefore, a complicated editing work with manual intervention is required, and there is a problem that work efficiency deteriorates.

  The present invention has been made in view of such circumstances, and provides a moving picture metadata automatic creation apparatus and a moving picture metadata automatic creation program capable of automatically assigning metadata to a moving image. Objective.

  According to the first aspect of the present invention, moving image input means for inputting moving image data including audio data, audio data and time information on the moving image data recorded with the audio data are extracted from the moving image data. Recognizing the speech data by referring to the speech separation means, a speech recognition dictionary in which a phrase specifying a spatial position on a moving image and a phrase to be metadata are registered in advance, and the speech recognition dictionary Thus, a speech recognition means for separating and extracting a phrase for specifying a spatial position on a moving image and a phrase to be metadata from the speech data, and converting each of them into character data, and a spatial position on the moving image And a metadata storage means for associating and storing the character data of the word to be the metadata and the time information as metadata.

  According to the second aspect of the present invention, the spatial position on the moving image is identified by referring to a position conversion table in which words and phrases specifying the spatial position on the moving image and position data on the screen are defined in advance. It is characterized by performing.

  The invention according to claim 3 further includes means for creating the position conversion table by designating the number of divisions on the screen and registering it in the speech recognition dictionary.

  The invention according to claim 4 is characterized in that the position conversion table is created based on a recognition result obtained by image recognition of the moving image data.

  The invention according to claim 5 is characterized in that the metadata storage means stores the metadata as a character data file.

  According to the sixth aspect of the present invention, moving image input means for inputting moving image data including audio data, audio data and time information on the moving image data recorded with the audio data are extracted from the moving image data. Referencing the speech data by referring to the speech separation means, the speech recognition dictionary in which words and phrases for controlling the playback of the motion image are registered in advance, and the speech recognition dictionary, the motion data can be extracted from the speech data. Voice recognition means for extracting a phrase for controlling reproduction and converting it into character data, character data for controlling reproduction of the moving image, and metadata storage means for associating and storing the time information as metadata. It is characterized by that.

  The invention according to claim 7 is a moving image input process for inputting moving image data including audio data, and extracts audio data and time information on the moving image data recorded with the audio data from the moving image data. Recognizing the speech data by referring to the speech separation dictionary, a speech recognition dictionary in which a phrase specifying a spatial position on a moving image and a phrase to be metadata are registered in advance, and the speech recognition dictionary A speech recognition process that separates and extracts a phrase that specifies a spatial position on the moving image and a phrase that serves as metadata from the voice data, and converts each of the phrase into character data; and a spatial position on the moving image Further, the present invention is characterized in that a computer performs metadata storage processing for associating and storing character data of a word as metadata and the time information as metadata.

  According to an eighth aspect of the present invention, moving image input processing for inputting moving image data including audio data, audio data and time information on the moving image data recorded with the audio data are extracted from the moving image data. By referring to the voice recognition dictionary in which words and phrases for controlling playback of the moving image are registered in advance, and the voice recognition dictionary, the voice data is recognized, thereby moving the moving image from the voice data. A computer recognizes a speech recognition process that extracts a phrase that controls reproduction and converts it into character data, a character data that controls reproduction of the moving image, and a metadata storage process that associates and stores the time information as metadata. It is made to carry out.

  According to the present invention, characters such as memos, markings, and the like can be given at a designated time and a designated position on the screen. For example, when a word such as “upper left” is spoken during shooting, an effect is obtained that an index can be assigned to a spatially relevant position of a temporally relevant position of a moving image. In addition, an object for executing playback control such as pause or slow playback can be given at a specified time. Therefore, by automatically generating metadata, a work that has been manually performed after photographing as a conventional editing work can be performed during photographing, and work efficiency can be improved. In addition, since a moving image in which characters and marks are displayed and moving images are automatically controlled during image reproduction can be created, the photographing side can give more information to the browsing side. In addition, since the space (position) information on the moving image and the time axis on the moving image are associated with the metadata, an index can be added to the designated spatial position, and the search of the moving image can be performed. The necessary data (video position) can be quickly accessed. Also, by setting the number of divisions on the screen, a table for converting the position can be automatically created and reflected in the dictionary, so that it is possible to save the trouble of registering the word string at the spatial position in the dictionary. Furthermore, since a character data file (for example, a report) can be automatically linked to a moving image and created, an effect of facilitating the organization and management of moving images can be obtained.

  Hereinafter, an automatic moving image metadata creation apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a moving image metadata automatic creation apparatus according to the embodiment. In this figure, reference numeral 1 denotes an automatic metadata creation apparatus. Reference numeral 2 denotes a moving image recording apparatus, which includes a video camera, a digital camera, a small personal computer, a camera-equipped mobile phone, and the like used as a moving image recording apparatus. Reference numeral 12 denotes a moving image editing unit that captures a moving image from the moving image recording apparatus 2 and creates metadata from a voice recognition result. Reference numeral 17 denotes a position conversion table for converting the word string in the dictionary and the position on the screen. Reference numeral 20 denotes an audio separation unit that extracts an audio part from a moving image. Reference numeral 21 denotes a speech recognition unit that recognizes speech and converts it into characters, and creates and manages a recognition dictionary 22 used for speech recognition.

  Next, an outline of the operation of the apparatus will be described with reference to FIG. First, a user arranges word strings to be used as moving image metadata, and creates a recognition dictionary 22 that is managed by the speech recognition unit 21 of the automatic metadata creation apparatus 1.

  Next, the user creates a position conversion table 17 for converting a word string (upper right, lower left, etc.) indicating a position on the screen in the dictionary 22 into a position (pixel, etc.) on the screen. This table may be automatically created by specifying how many screens are to be divided, added to the dictionary, or automatically created in combination with an image recognition device. Associated with.

  Next, a moving image is photographed using the moving image recording apparatus 2. At this time, the photographer consciously utters the words registered in the dictionary 22 and the words registered in the dictionary of the system side. For example, in a conference recording scene, an agenda and a position to be pasted as an index are uttered. In the personal profile creation scene, the user speaks the name and the position to paste it as an index. In a scene where it is desired to automatically pause at the time of reproduction, “pause” is spoken.

  Next, the captured moving image is captured by the moving image editing unit 12 of the metadata automatic creation device 1. In response to this, the automatic metadata creation apparatus 1 extracts only audio data from the moving image by the audio separation unit 20. Then, the automatic metadata creation apparatus 1 recognizes the voice data by the voice recognition unit 21. At this point, if a character data file (such as a document for organizing moving images) storing the speech recognition result is required, it is created using the speech recognition result.

  Next, the automatic metadata creation apparatus 1 creates the metadata (index or moving picture control object) of the corresponding part of the moving image by the moving picture editing unit 12, and creates the character data file created in the storage database. And moving images with metadata attached.

  By performing this operation, the user can browse a moving image in which metadata is pasted as an index or an object, or an automatically generated character data file.

  Next, the detailed configuration of the metadata automatic creation device 1 shown in FIG. 1 will be described with reference to FIG. In this figure, the same parts as those in the apparatus shown in FIG. Reference numeral 11 denotes an input / output interface that enables moving image transfer with the moving image recording apparatus 2. Reference numeral 13 denotes a moving image capturing unit that captures moving images from the moving image recording apparatus 2. Reference numeral 14 denotes a metadata creation unit that creates metadata serving as an index and an object from the speech recognition result. Reference numeral 15 denotes an index creation unit that searches the corresponding position conversion table 17 from the dictionary file name and creates metadata serving as an index based on the recognition result. Reference numeral 16 denotes a moving image control unit that creates metadata serving as an object for moving image control based on the recognition result. Reference numeral 18 denotes a screen division unit that automatically selects a word string representing a spatial position according to the number of screen divisions, creates the position conversion table 17 and registers it as a word string representing the spatial position in the speech recognition dictionary 22. . Reference numeral 19 denotes a dictionary management unit that creates and manages a recognition dictionary 22. Reference numeral 23 denotes a character data file creation unit that creates a character data file of a specified format based on the result of speech recognition. Reference numeral 24 denotes a file management unit that manages a database that stores moving images and the like and cooperates with other systems. Reference numeral 25 denotes a storage database (DB) for storing the created moving image and the like. Reference numeral 31 denotes a video input unit composed of video cameras and camera parts of various devices. Reference numeral 32 denotes an audio input unit including an internal microphone or an external microphone. Reference numeral 33 denotes a moving image creation unit that inputs signals from the video input unit 31 and the audio input unit 32 and generates a moving image. Reference numeral 3 denotes an image recognition apparatus that recognizes an image at a designated time and creates a position conversion table 17.

  Next, with reference to FIG. 2, the operation of preparation before moving image shooting (when the word string representing the spatial position is automatically acquired by the screen dividing unit 18) will be described. First, the dictionary 22 for speech recognition used for speech recognition is taken into the system by the dictionary management unit 19 ((A) in FIG. 2). The dictionary contains words that indicate position (such as “upper right” and “lower left”), index type (such as “memo” and “arrow”), content (“words you want to display”), video control information (“pause” ”,“ Slow ”). This can be either a dictionary created by the user or a dictionary provided by the system.

  Subsequently, the number of screen divisions is acquired from the user, and a word string representing a spatial position is automatically selected by the screen division unit 18 to create the position conversion table 17 ((B) of FIG. 2). The user designates the number of screen divisions (for example, 6 divisions). The screen dividing unit 18 has a word string corresponding to the number of divisions in advance. Then, the dictionary management unit 19 registers the word sequence indicating the spatial position in the corresponding speech recognition dictionary 22 ((C) in FIG. 2). Thus, the position conversion table 17 shown in FIG. 6 is created, and a word string indicating position information is registered in the speech recognition dictionary 22 (may be manually registered in the dictionary).

  Next, with reference to FIGS. 2 and 3, an operation for automatically creating metadata (when the image recognition apparatus is not used) will be described. Here, a description will be given by taking as an example an operation of pasting a memo index of “point” on the image at the upper left of the screen and adding a pause object at an arbitrary timing.

  First, the user composes the input from the video input unit (camera) 31 and the audio input unit (microphone) 32 of the moving image recording apparatus 2 by the moving image creation unit 33 to create a moving image (FIG. 2 (1)). )). At this time, the user speaks “upper left, memo, point” at the timing at which an index is desired, and “pause” at the timing at which an object is desired.

  Subsequently, after shooting the moving image, the user connects the input / output interfaces 11 and 34 of the moving image recording apparatus 2 and the automatic metadata creation apparatus 1 to enable transfer of moving images (FIG. 2 ( 2)). In response to this, the metadata automatic creation device 1 captures the captured moving image by the moving image capturing unit 13 of the moving image editing unit 12 (FIG. 2 (3), step S1).

  Next, the metadata automatic creation device 1 extracts only the sound data from the moving image by the sound separation unit 20 (FIGS. 2 (4) and (5), step S2). Then, the voice recognition unit 21 recognizes the voice data extracted based on the voice recognition dictionary 22 created in advance (step S3), and as a result, obtains character information of the utterance content of the photographer, Time information of the file at the time of voice recognition (for example, information that can specify a temporal position such as how many seconds after the beginning of the file) is acquired. The character data file creation unit creates the character data file shown in FIG. 8 using the voice recognition result (FIG. 2 (6)).

  Next, the metadata automatic creation device 1 calls the index creation unit 15 or the movie control unit 16 by the metadata creation unit 14 of the movie editing unit 12 (FIG. 2 (7), step S5). The determination of which one to call is made by comparing the acquired character information with the dictionary, or by distinguishing the dictionary for the index and the object and acquiring the dictionary name.

  Next, the called index creating unit 15 searches the position conversion table 17 based on the dictionary name used for speech recognition (FIG. 2 (8), step S8). Then, the index creating unit 15 confirms the spatial position by comparing the character information of the speech recognition result with the position conversion table 17 searched in step S8, and creates metadata as an index shown in FIG. 8 (FIG. 2). (9), steps S9, S10). On the other hand, the called moving image control unit 16 creates metadata as an object for moving image control shown in FIG. 9 based on the character data of the voice recognition result (FIG. 2 (10), step S6).

  Next, in the metadata automatic creation device 1, the metadata creation unit 14 of the moving image editing unit 12 attaches the metadata that becomes the index or object created in step S6 or S10 to the moving image (FIG. 2 (11), Step S7). Then, the file management unit 24 stores the moving image or character data file to which the generated metadata is attached in the storage database 25 (FIG. 2 (12)).

  Next, with reference to FIG. 4, an automatic metadata creation operation when the image recognition apparatus 3 is used will be described. Here, only parts different from the operation shown in FIG. 3 will be described. The operation shown in FIG. 4 performs the same operation as steps S1 to S5 shown in FIG. When it is determined that the index is created (YES in step S5), the image recognition device 3 recognizes the image of the time of the acquired time information (for example, 20 seconds after the head) and acquires the character acquired at the same timing. Based on the information (FIG. 10), the position conversion table 17 shown in FIG. 11 is created (step S8a). Then, the index creation unit 15 compares the character information of the speech recognition result with the position conversion table 17 created in step S8a (step S8b), determines the spatial position (step S9a), and sets the metadata serving as the index. Create (step S10).

  In this way, it is possible to view a moving image in which indexes and objects are pasted at spatial and temporal positions spoken during shooting.

  Next, with reference to FIG. 5, the operation when the number of screen divisions is designated will be described. As described above, in order to obtain metadata with spatial information from a captured moving image using voice recognition, it is necessary to convert the voice recognition result (character) into information indicating the spatial position on the screen, such as pixels. There is. In the above description, this conversion is realized by referring to the position conversion table 17. Two methods of screen division and image recognition have been shown as methods for creating the position conversion table 17. Here, the screen division method will be described.

  First, since steps S1 to S4 are the same as the operations shown in FIG. 3, the description thereof will be omitted, and only the portions different from the operations shown in FIG. 3 will be described here. Here, as an example, an operation will be described in which the number of divisions is changed when zooming in on each operation point (power button, etc.) from a state in which an entire image of the device is captured in a scene where device-based manual video is created.

  First, when shooting a moving image, the user zooms in on the operation point and speaks “operation point” (this changes the number of divisions). Then, utter the location and contents of the index to be pasted, such as “Upper right, memo, this button is the power button”. After shooting, the user connects the moving image recording device 2 and the metadata automatic creation device 1 to recognize voice. The operation so far is the same as the operation described above.

  Next, the metadata automatic creation device 1 specifies the number of screen divisions from the voice recognition result, and selects the position conversion table 17. The number of screen divisions is specified as 9 divisions at the operation point and 4 divisions at the whole display. The metadata automatic creation device 1 automatically creates metadata such as indexes using the position conversion table until the position conversion table 17 is changed (steps S11 to S14).

  In this way, utterance of the number of divisions (numbers), utterances such as the usage of the object to be photographed and moving images, zooming in / out of the camera, use of image recognition (for example, changing the number of divisions depending on the object to be photographed) For example, the number of divisions can be determined on the system side at any timing during moving image shooting.

Next, a usage example of the metadata automatic creation device 1 described above will be described.
(A) Manual video By applying it to manuals such as equipment, it becomes easier to understand by using moving images even if it is difficult to understand with manual books. For example, when the user speaks the “upper right button”, it is possible to place a mark on the corresponding position by using a circle on the button, instructions such as caution points and usage that should not be used, and screen recognition.
(B) Labor-saving at the help desk By providing images with marks and memos taken of how to deal with frequently asked questions, it is possible to secure satisfaction on the question side and save labor on the answer side. .
(C) Review video of the class By taking a video of the class scene, it is possible to review after the class by automatically putting notes on the utterances of “This is important” and “Exiting the exam”. .
(D) Product introduction video This can be applied to the introduction of things such as model rooms that cannot be seen without going to the site, and famous places. For example, if you say “Points in the lower right”, you can mark sales points such as doors and kitchens, and if you say “Central Cape XX” and the name of a famous place, you can add a title and mark to that position.
(E) How to use for home Use a video while traveling, such as “Eiffel Tower is at the center of the screen”, mark or character on a famous place, or play at a sports day as “A's goal scene, pause” Sometimes it is not necessary to perform complicated editing work at home, such as not overlooking a decisive scene.
(F) Video materials for sports training, etc. Mark human parts. For example, if you say `` Do not move your head '' or `` Be careful about the angle of your knees '' during shooting, you can use screen recognition to mark your head and knees with a mark such as a circle, Can be done.

  By applying to such an example, it is possible to automatically create a video as it is after editing and greatly reduce the editing work, so that it is possible to widen the use scene of moving images.

  As described above, the voice is separated from the moving image, and the word (spatial information) indicating the position on the screen and the index type and the word (character information) indicating the content associated therewith or the moving image control object are shown by the voice recognition. While acquiring a word (character information) and simultaneously acquiring time information indicating a position on the time axis in the moving image file, the acquired spatial information is converted from a word to a spatial position ( Therefore, it is possible to automatically create moving image metadata having character information + time information + spatial information.

  The program for realizing the processing unit in FIG. 2 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to automatically execute moving image metadata. A creation process may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a section for transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the above-described unit. Furthermore, what can implement | achieve the part mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

It is a block diagram which shows the structure of one Embodiment of this invention. It is a block diagram which shows the detailed structure of the metadata automatic production apparatus 1 shown in FIG. It is a flowchart which shows the operation | movement which produces metadata automatically. It is a flowchart which shows the operation | movement which produces metadata automatically. It is a flowchart which shows the operation | movement which determines the division | segmentation number of a screen. It is explanatory drawing which shows the table structure of the position conversion table. It is explanatory drawing which shows an example of a speech recognition result. It is explanatory drawing which shows an example of metadata (index). It is explanatory drawing which shows an example of metadata (object). It is explanatory drawing which shows an example of character information. It is explanatory drawing which shows an example of the position conversion table.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Metadata automatic creation apparatus 11 ... Input-output interface 12 ... Movie editing part 13 ... Moving image capture part 14 ... Metadata creation part 15 ... Index creation part 16 ... Moving image control unit 17 ... position conversion table 18 ... screen division unit 19 ... dictionary management unit 20 ... voice separation unit 21 ... voice recognition unit 22 ... voice recognition dictionary 23 ... Character data file creation unit 24... File management unit 25... Storage database 2 .. moving image recording device 31... Video input unit 32. ... I / O interface 3 ... Image recognition device

Claims (8)

Moving image input means for inputting moving image data including audio data;
Audio separation means for extracting audio data and time information on the video data recorded with the audio data from the video data;
A speech recognition dictionary in which a phrase specifying a spatial position on a moving image and a phrase serving as metadata are registered in advance;
By referring to the voice recognition dictionary and recognizing the voice data, a phrase specifying a spatial position on a moving image and a phrase serving as metadata are separated and extracted from the voice data. Voice recognition means for converting to data,
Automatic creation of moving image metadata, comprising: metadata storage means for associating a spatial position on the moving image, character data of a word and phrase serving as metadata, and the time information, and storing them as metadata apparatus.   2. The spatial position on the moving image is specified by referring to a position conversion table in which a phrase specifying the spatial position on the moving image and position data on the screen are defined in advance. The video metadata automatic creation device described in 1.   The apparatus for automatically creating moving picture metadata according to claim 2, further comprising means for creating the position conversion table by designating a division number on a screen and registering the position conversion table in the voice recognition dictionary.   The moving image metadata automatic creation device according to claim 2, wherein the position conversion table is created based on a recognition result obtained by recognizing the moving image data.   The moving image metadata automatic creation apparatus according to claim 1, wherein the metadata storage means stores the metadata as a character data file. Moving image input means for inputting moving image data including audio data;
Audio separation means for extracting audio data and time information on the video data recorded with the audio data from the video data;
A speech recognition dictionary in which words and phrases for controlling reproduction of moving images are registered in advance;
A speech recognition means for extracting the words controlling the reproduction of moving images from the speech data by referring to the speech recognition dictionary and converting the speech data into character data;
An apparatus for automatically creating moving image metadata, comprising: metadata storage means for storing character data for controlling reproduction of the moving image and the time information in association with each other. Moving image input processing for inputting moving image data including audio data;
Audio separation processing for extracting audio data and time information on the video data recorded with the audio data from the video data;
A speech recognition dictionary in which a phrase specifying a spatial position on a moving image and a phrase serving as metadata are registered in advance;
By referring to the voice recognition dictionary and recognizing the voice data, a phrase specifying a spatial position on a moving image and a phrase serving as metadata are separated and extracted from the voice data. Voice recognition processing to convert to data,
Moving image metadata, characterized by causing a computer to perform metadata storage processing for associating the spatial position on the moving image, the character data of the word and phrase serving as metadata, and the time information and storing them as metadata. Automatic creation program. Moving image input processing for inputting moving image data including audio data;
Audio separation processing for extracting audio data and time information on the video data recorded with the audio data from the video data;
A speech recognition dictionary in which words and phrases for controlling reproduction of moving images are registered in advance;
A speech recognition process of referring to the speech recognition dictionary and extracting the words controlling the reproduction of moving images from the speech data by recognizing the speech data, and converting them into character data;
An automatic moving image metadata creation program that causes a computer to perform a character storage process for storing character data for controlling reproduction of the moving image and the time information as metadata in association with each other.

Images