KR20060100646A - Method for searching specific location of image and image search system - Google Patents

Abstract

The present invention relates to a method and system for receiving a search word from a user and searching for a specific position where a dialogue related to the input search word is reproduced in an image. According to an aspect of the present invention, there is provided a method of searching for an image object, the method comprising: dividing the image object into a plurality of image stream data, associating a segment identifier with each of the divided image stream data, and audio from the image stream data. Extracting data, correlating the extracted audio data with a related partition identifier, and converting the extracted audio data into symbol data that can be recognized by a predetermined script alignment unit by using a predetermined speech recognition technique. Identifying text data in the script data corresponding to the video object, including the converted symbol data, and correlating the identified text data and the associated segmentation identifier in the script aligning unit. It is characterized by including. According to the present invention, a user can search an image using arbitrary text, and search and provide a specific position of a video to be reproduced by using a dialogue including a search word. There is an advantage that can provide a method for searching for a specific position of the image to perform the image search and the image retrieval system conveniently.

Video, video search, playback time, dialogue, Viterbi search

Description

METHOD AND SYSTEM FOR SEARCHING THE POSITION OF AN IMAGE THING}

1 is a view for explaining the network configuration of the image search system of the present invention.

2 is a block diagram illustrating an internal configuration of an image retrieval system according to a preferred embodiment of the present invention.

3 is a diagram illustrating an example of searching for a specific position of an image by using a keyword in a text form according to the present invention.

4 is a flowchart illustrating an image search method according to a preferred embodiment of the present invention.

5 is a flowchart illustrating an example of a method of performing an image search according to another exemplary embodiment of the present invention.

6 is an internal block diagram of a general purpose computer device that may be employed to perform the image retrieval method according to the present invention.

<Explanation of symbols for the main parts of the drawings>

200: Image retrieval system 210: Identifier association unit

220: audio extraction unit 230: speech recognition unit

240: script alignment unit 250: interface unit

260: location search unit 270: search result provider

The present invention relates to an image retrieval method and an image retrieval system, and more particularly, to a method and system for receiving a search word from a user and searching for a specific position where a dialogue related to the input search word is reproduced in an image. It is about.

Image providing services such as movies, music, and broadcasting, which are mainly performed offline, are being implemented on-line with the development of communication networks such as the Internet. In particular, the image providing service transmitted directly to the user's terminal through a communication network can be provided to the user in real time to overcome the limitations of space and time, the use area is gradually expanding to all fields of the multimedia field.

The user may feel a need to search for a specific point that he / she wants in the image while playing the stored image. However, since the image data is stored in a complex form different from the general text, it is not easy to construct the image data corresponding to the corresponding point into the database, and thus it is not easy to search.

Accordingly, in order to search for a specific position of an image, a conventional method such as '2 times faster' or 'fast forward' should be followed while watching a playback screen of an image. That is, the search method in the existing multimedia providing service is completely dependent on the playback time of the video material, and the accurate location search is possible only if the user correctly identifies the running (playing) time related to the specific location of the video to be searched. Done. If the playing time is not known properly, the existing multimedia providing service has to repeatedly perform the 'fast forward view' or 'fast forward view' function, and provides only the approximate location information of the relevant point. The search takes a lot of time and is inaccurate.

In this case, if there is an image search method for determining a specific position of an image only by directly inputting a dialogue to be searched for, the user may be able to ensure a more accurate search within a shorter search time.

Therefore, by generating a video search request according to the input of the user's desired dialogue, and in response to the generated video search request to search the specific position of the video image to be reproduced, the user can optimally meet the user's video search request New image retrieval models are urgently needed.

The present invention has been made to solve the above problems, by allowing the user to search the image using the input dialogue, and search for a specific position of the image to reproduce the input dialogue, more convenient and convenient image search An object of the present invention is to provide a method for searching for a specific position of an image to be performed and an image search system.

In addition, an object of the present invention is to divide the image by sorting the script data corresponding to the image using the segmentation identifier associated with the segmentation of the image, and to identify the segmentation identifier associated with the sentence of the script data including the search word The present invention provides a method and an image retrieval system for retrieving a specific position of an image for retrieving stream data.

Another object of the present invention is to provide a method and an image retrieval system for retrieving a specific position of an image that optimally supports an image retrieval request using dialogue using text data extracted from the image.

According to an aspect of the present invention, there is provided a method of searching for an image object, the method comprising: dividing the image object into a plurality of image stream data, and mapping a segment identifier to each of the divided image stream data; Extracting audio data from the video stream data, correlating the extracted audio data to an associated segmentation identifier, and recognizing the extracted audio data in a predetermined script alignment unit using a predetermined speech recognition technique. Converting into symbol data in the form, identifying the text data in the script data corresponding to the video object, including the converted symbol data in the script aligning unit, and the identified text data and associated segmentation. And correlating the identifiers. It shall be.

In addition, as a technical configuration for achieving the above object, a system for retrieving a video object comprises: an identifier association unit for dividing the video object into a plurality of video stream data, and corresponding to the divided identifier for each of the divided video stream data; An audio extraction unit for extracting audio data from the image stream data, a voice recognition unit for converting the extracted audio data into symbol data that can be recognized by a predetermined script alignment unit using a predetermined speech recognition technique; A script alignment unit for identifying text data in the script data corresponding to the video object including the converted symbol data and correlating the identified text data with an associated segmentation identifier, and an image including a search word from a user. An interface unit for receiving a search request; In the script data, a position search unit for identifying text data including the input search word, searching for a segmentation identifier correlated with the identified text data, and extracting image stream data corresponding to the searched segmentation identifier; And a search result providing unit provided to the user.

Hereinafter, a method and an image retrieval system for retrieving a specific position of an image of the present invention will be described with reference to the accompanying drawings.

The term " image material " as used continuously herein refers to image information that is ordered to be transmitted from a user via a communication network and correspondingly also supplied to the user via the communication network, for example at a desired time at the user's request. It may refer to video information provided by a video on demand (VOD) that enables a user to watch a desired video. In addition, the image stream data is a unit-specific image constituting a video object, and is generated by dividing the entire video object based on a predetermined unit, for example, a reproduction time, a frame number, a scene, and the like.

1 is a view for explaining the network configuration of the image search system of the present invention.

The image retrieval system 100 deviates from the positional identification of the image by the playback time as in the prior art, so that the user 130 receives the desired dialogue in text form and provides the user 130 with the position of the image corresponding thereto. It will support the search method.

First, the image providing server 110 serves to provide image information related thereto through the communication network 140 in response to an order request for an image received from the user 130, for example, a VOD server supporting a VOD service. It can be illustrated. That is, the image providing server 110 provides desired image information to the user 130 having a predetermined contract relationship, and enables the user 130 to watch the image through the communication network 140 and the terminal means 135. It plays a role. In addition, the image providing server 110 may be provided with a lady disk (ready disk, not shown), a portion of the video information to be processed in advance in the image information database 120 to copy to the lady disk As a result, service requests of the multiple users 130 may be promptly processed in real time.

In addition, the image providing server 110 may include the image retrieval system 100 of the present invention inside or outside, and searches for a specific position of the image in response to the image retrieval request of the user 130 and retrieves information about the image. The user 130 is controlled to be notified.

The image information database 120 refers to a large capacity storage medium that stores a plurality of image information collected from a predetermined image provider. The image information database 120 takes the form of an array that connects a plurality of storage media. The image information database 120 considers the category of the image, the image provider supplying the image, the grade of the image, the quality of the image, and the like. Can be categorized and stored systematically.

The user 130 has a terminal means 135 for accessing the image retrieval system 100 of the present invention, and provides a search service for a specific position of an image by searching and inputting a desired dialogue in a text form among predetermined images. It may mean a receiving Internet user or a VOD service user.

The terminal unit 135 maintains a connection state with the image retrieval system 100 through a communication network 140 such as the Internet and is a device for displaying image information provided to the user 130. In addition, the terminal means 135 drives a predetermined user interface (UI) for inputting a search word on the screen, and interlocks with the user 130 inputting a character (search word) related to an arbitrary dialogue in the user interface. To generate an image search request. The terminal means 135 is provided with predetermined memory means such as, for example, a personal computer, a handheld computer, a personal digital assistant (PDA), an MP3 player, an electronic dictionary, a mobile phone, a smartphone, and the like. It may be a concept of collectively referred to as a terminal having a predetermined computing power by mounting.

The image retrieval system 100 may provide a search service for a specific position of an image using a character by allowing the user 130 to input a dialogue that was reproduced in the image as a search word. Hereinafter, a detailed configuration of the image retrieval system 200 of the present invention will be described with reference to FIG. 2.

2 is a block diagram illustrating an internal configuration of an image retrieval system according to a preferred embodiment of the present invention.

Image retrieval system 200 of the present invention, identifier association unit 210, audio extraction unit 220, speech recognition unit 230, script aligning unit 240, interface unit 250, location search unit 260 ) And a search result providing unit 270.

First, the identifier association unit 210 is an apparatus for dividing a video object into a plurality of video stream data, and for assigning a segmentation identifier to each of the divided video stream data. Here, the segmentation identifier is used to identify a point where segmented video stream data is located among all video objects, and the segmentation identifier is a playback time (start time and end time), frame number, and scene related to video stream data corresponding to the segmentation identifier. Information may be included. That is, the identifier association unit 210 divides and stores each of the plurality of video stream data constituting the video material into a partition identifier, and controls to search for specific video stream data through identification of the partition identifier. In addition, the identifier association unit 210 may correspond to each video stream data in the order of time flow, that is, the time-series division identifiers. For example, the identifier association unit 210 may correspond to the video stream data that is first divided in time. T1 'can then be sequentially associated with a series of segmentation identifiers, and the segmentation identifier' Tn 'can be correlated to the video stream data that is finally divided into the nth segment.

The audio extractor 220 extracts audio data from video stream data. The audio extractor 220 classifies video data such as an image and a picture and audio data such as sound and sound for each video stream data. Specific audio data may be selectively extracted according to the selection of the user 130.

The speech recognition unit 230 is a device that converts the extracted audio data into symbol data that can be recognized by the predetermined script alignment unit 240 using a predetermined recognition technique. That is, the voice recognition unit 230 is symbol data as a sign meaningful to the script aligning unit 240 involved in alignment processing (eg, position identification processing of data data included in the script data) related to the script data related to the video material. In the process of generating a function, for example, the automatic sorting rule generation is performed to convert the script sorter 240 into symbol data that can be recognized. Here, the automatic phonetic notation rule generation may be composed of text normalization and GTP (Grapheme-to-Phoneme) conversion process. For example, when there is a phrase "3-point-7-5 percent" in English audio data, the text normalization results in "three point seven to five percent". sp P OY N TD sp S EH V AX N sp T AX sp F AY V sp P AXR S EH N TD sp ". Each of these transformed symbols correlates to speech recognition models. In addition, the speech recognition technique is a technique for changing the audio data (sound) output during image playback into text-type characters (sentences) having time information, and may be representative of the Viterbi-search technique. have. Speech recognition by the Viterbi search technique selects the phoneme model with the highest accuracy while passing the script reconstructed into the preset phoneme model through the extracted audio data, and determines the characters corresponding to the selected phoneme model. Is performed by. In addition, in the process of extracting feature information to be used for speech recognition from speech data, for example, Sequential Fast Pole Filtering (SFPF) is used to perform a predetermined compensation process on audio data, thereby extracting the extracted audio data. It is possible to improve the alignment accuracy.

The script aligning unit 240 is an apparatus for identifying text data in the script data corresponding to the video material and including the changed symbol data in a stochastic manner, and correlating the identified text data with an associated segmentation identifier. That is, the script aligning unit 240 is a device for identifying the position of the text data converted in the predetermined script data, and the text data of the script data having the highest degree of agreement (similarity) with the symbol data converted by speech recognition. The location of is identified based on real time. Here, the script data is a set of scripts (or subtitles) corresponding to the video stream data, for example, created in advance by an image provider for producing or supplying the corresponding video material, and provided to an operator who operates the system at a predetermined point in time. . In addition, the script data is composed of one or more combinations of letters, numbers, special characters, or sentences consisting of these combinations. For example, the combinations or sentences may be distinguished in consideration of a scene reproduced in an image. .

In addition, the script aligning unit 240 correlates the segmentation identifier associated with the symbol data converted to the identified text data in the script data, and sorts the combination or sentence of the script data based on the correlated segmentation identifier. can do. As described above, the partition identifier is correlated in time series with each video stream data, and in conjunction with this, the script aligning unit 240 may rearrange the script data in time series based on the partition identifier.

Accordingly, the script data of the present invention can correspond to the partition identifier replacing the existing time information for each combination / statement, so that it can be seen that the specific combination / statement of the script data is reproduced as a sound at any position of the image. do.

The interface unit 250 is a device that receives an image search request including a search word from the user 130. That is, the interface unit 250 receives an image search request generated as the user 130 inputs a search word (metabolism) to a predetermined user interface.

The location retrieval unit 260 is an apparatus for identifying text data including the input search word in the script data and searching for a segmentation identifier correlated with the identified text data. That is, the location search unit 260 identifies a combination / sentence of the script data including at least a part of the input search word (metabolism) and searches for a partition identifier matched thereto. In particular, when there are a plurality of positions (combination / sentence) of the script data including the search word, the position search unit 260 calculates the similarity between each position and the search word, and searches for a predetermined number whose calculated similarity value is equal to or larger than a set value. do. In the present embodiment, the technique for calculating the similarity is specifically limited. For example, the similarity may be calculated as a numerical value in consideration of phonemes, syllables, the degree of coincidence of words, the degree of similarity in meaning, and the like. Such a similarity calculation technique can be flexibly selected by an operator who operates the system. However, in the process of searching for the position (text data) by the position search unit 260 of the present invention, calculation is performed for each phoneme unit, and the result of the previous step affects the rear end, so that the current determined position is final. There is no guarantee that it will be determined. Therefore, the operator can flexibly select and apply the similarity calculation technique with higher accuracy according to the operating system environment.

The search result providing unit 270 is a device that extracts and provides the video stream data corresponding to the found partition identifier to the user 130. That is, the search result providing unit 270 checks the video stream data corresponding to the segmentation identifier searched by the location search unit 260, and uses the checked video stream data as a result of the video search request. ) To control exposure to

Therefore, according to the present invention, the user 130 allows the image search using the metabolism of the image as a search word, and is more accurate than the conventional search method of searching in dependence on time by searching for a specific position of the image playing the input metabolism. It has the advantage of creating a convenient image search environment.

3 is a diagram illustrating an example of searching for a specific position of an image by using a keyword in a text form according to the present invention.

First, FIG. 3) illustrates that a segmentation identifier is mapped to a portion (or all) of an image, and the identifier association unit 210 of FIG. 2 is configured to a predetermined reference (play time, frame, scene, etc.). Based on this, the video object is divided into a plurality of video stream data. At this time, the partition identifiers T1 to T4 correspond to the video stream data. In FIG. 3), as an example of the present invention, a segmentation identifier corresponds to each of a plurality of video stream data on the basis of a reproduction time.

That is, as shown in iv) of FIG. 3, the predetermined memory means may record a reproduction time when the divided video stream data is reproduced and a partition identifier corresponding thereto. Accordingly, when a specific segmentation identifier is searched for, the video stream data corresponding to the specific segmentation identifier, more specifically, the information on the reproduction time at which the video stream data is reproduced can be known. For example, when the video stream data and the partition identifier recorded in the memory means are the same as in iv) of FIG. 3, and the partition identifier 'T1' is searched by a predetermined process, the video search system 200 has a playback time of '00. Video stream data # 1 of '03: 24-00: 06: 17 'can be identified as a search result.

In FIG. 3, ii) illustrates the recognition of speech using audio data extracted from each video stream data after the script is converted to GTP.

In (iii) of FIG. 3, the speech recognized result identifies the position of the start and end of each sentence of the script on the audio. At this time, the script aligning unit 240 correlates the segmentation identifier associated with the corresponding text data to each identified position. Accordingly, the image retrieval system 200 of the present invention can identify a time point at which a specific position of the script data is reproduced in the video object as the segmentation identifier.

Then, for example, when the search term 'selection is instant' is input by the user 130, the image search system 200 may sense the moment of selection of the location of the script data including the search term 'selection is instant'. Important 'and search for the associated partition identifier' T1 '. In addition, the image retrieval system 200 may provide the user 130 with the image stream data # 1 (for example, playback time 00:03:24 to 00:06:17) corresponding to the retrieved 'T1' as a search result. Can be.

Therefore, according to the present invention, the user 130 inputs a dialogue (selection moment) among predetermined video images, thereby obtaining an effect of searching for a predetermined position of the video image where the corresponding dialogue is reproduced.

The workflow of the image retrieval system 200 according to the present invention having such a configuration will be described in detail.

4 is a flowchart illustrating an image search method according to a preferred embodiment of the present invention.

The image retrieval method according to the present embodiment is performed by the image retrieval system 200 described above.

First, the image retrieval system 200 divides an image into a plurality of image stream data, and associates a segmentation identifier with respect to each of the divided image stream data (S410). The step S410 is a process of dividing the video material into predetermined units and allocating a segmentation identifier for each of the video material. For example, the video material may be divided based on a playback time, a frame, a scene, and the like. In (iii) of FIG. 3, the video material is divided into playback time units and the partition identifiers are correlated to each other. Image stream data # 1 corresponding to the playback time 00:03:24 to 00:06:17 It corresponds to the partition identifier T1.

In addition, the image retrieval system 200 extracts audio data from the image stream data, and correlates the extracted audio data with an associated segmentation identifier (S420). In operation S420, the video data included in the video stream data and the audio data synchronized with the video stream data are separated. In this embodiment, only the audio data is extracted.

Next, the image retrieval system 200 extracts the extracted audio data by using a predetermined speech recognition technique, and extracts the extracted audio data into symbol data that can be recognized by the predetermined script arranging unit 240. The conversion is made (S430). This step (S430) is a process of textualizing the voice reproduced in the corresponding video stream data, for example, by passing the audio data to a predetermined map (model map) using a video search, and compared with the phoneme model The symbol data determined to be similar can be determined. At this time, the symbol data is generated in a form that can be identified by the script sorter 240 to be described later to be used for the sorting process for the script data.

In addition, the image retrieval system 200 identifies the text data in the script data corresponding to the video material, including a plurality of symbol data transformed by the script aligning unit (S440). This step (S440) is a process of comparing the converted symbol data with the text data of the script data which is the Devon of the image, and identifying the same or most similar text data with the text data.

Subsequently, the image retrieval system 200 correlates the identified text data and the associated segmentation identifier (S450). This step (S450) is a process of aligning the script data using the partition identifier, the correlated partition identifier corresponds to, for example, symbol data involved in the identification of text data or audio data involved in the conversion of the symbol data. Can be determined by the partition identifier. As a result, the image retrieval system 200 of the present invention can identify the position of the image of which the predetermined sentence of the script data is reproduced by using the partition identifier (see FIG. 3).

Therefore, according to the present invention, the script data corresponding to the video object can be sorted by using a segmentation identifier associated with the segmentation of the video object, thereby accurately identifying a specific position of the video object in which a predetermined sentence of the script data is reproduced. Can be obtained.

In addition, the image search system 200 receives an image search request including a search word from the user 130 and identifies text data including the input search word in script data (S460). This step (S460) is a process of comparing the search word input for the image search request and the script data that is the script of the image, and identifying the location of the text data, that is, the script data including the search word (S460). That is, the image retrieval system 200 generates a video retrieval request according to a user inputting a predetermined dialogue from the user 130 as a search word, and identifies a sentence position of script data identical or similar to the input dialogue.

Next, the image search system 200 searches for a segmentation identifier correlating to the identified text data (S470). This step (S470) is a process of positioning the dialogue input by the user 130 in the script data. In particular, when a plurality of specific locations of the script data including the input search word are identified in the step S470, the image search system 200 selects a predetermined location based on the similarity calculated for each location. . Here, the similarity may be calculated as, for example, a similar probability value in consideration of a matching ratio between a predetermined sentence and text data of the script data, calculation of the same number of phonemes, use of a synonym, and the like. do.

For example, suppose that the script data is the same as i) of FIG. 3 and input as a search word 'this restaurant makes the ribs delicious'. At this time, the image search system 200 is the location of the script data, including 'restaurant, ribs' among the search terms entered by the user' this restaurant is good ribs (split identifier T3) 'and' this restaurant The ribs are full text (split identifier T5). Thereafter, the image retrieval system 200 may allow a relatively high similarity to be calculated in consideration of the use of synonyms, such as 'this restaurant is good at ribs', which is the location of the script data associated with the segmentation identifier T3. If the number of the segment identifiers to be searched is limited to one, the image search system 200 may allow only the segment identifier T3 to be searched in response to the search term 'this restaurant makes the ribs delicious' inputted for the image search request. .

In addition, the image retrieval system 200 extracts the image stream data correlated with the retrieved segment identifier and provides the image to the user 130 (S480). This step (S480) is a process of transmitting the video stream data corresponding to the found segment identifier through the communication network 140 as a search result for the video search request. For example, with respect to the segmentation identifier T3 retrieved in step S470, the image retrieval system 200 may provide the user 130 with image stream data # 3 related to the playback time 00:08:26 to 00:10:00.

Therefore, according to the present invention, by allowing the user 130 to input the dialogue as a search word and searching for a position where the input search word is reproduced in the image, a convenient and accurate image search method can be obtained.

Hereinafter, as another embodiment of the present invention, it will be described to perform an image search through the text input of the user 130 with respect to the video material without the script data.

5 is a flowchart illustrating an example of a method of performing an image search according to another exemplary embodiment of the present invention.

First, the image retrieval system 200 divides an image into a plurality of image stream data, and extracts audio data from each of the divided image stream data (S510). The step S510 is a process of dividing an image into unit data based on a predetermined criterion, for example, a playback time, a frame, a scene of the image, and separating audio for each of the divided unit data.

In addition, the image retrieval system 200 converts the extracted audio data into predetermined symbol data by using a predetermined speech recognition technique, and refers to predetermined script data by referring to text data that is stochasticly similar to the converted symbol data. It identifies (S520). In operation S520, the image retrieval system 200 may record the identified text data and the image stream data associated with the identified text data in a predetermined memory means. That is, the image retrieval system 200 finally infers the text data by using the audio data extracted from the image material, and stores the image stream data and the text data corresponding to the extracted audio data. For example, when the image stream data is divided based on the reproduction time, and the text data converted from the specific image stream data is 'our love forever', the image retrieval system 200 reproduces the image in response to 'our love forever'. You can record the time '1: 01: 25-1: 02: 00'.

Next, the image search system 200 receives an image search request including a search word from the user 130 and searches for text data including the search word with reference to a memory means (S530). This step (S530) is a process of receiving a specific dialogue of the image from the user 130 as a search word, and searching for text data identical or similar to the input dialogue. In particular, when a plurality of text data including an input search word is searched for in a plurality of text data, the image search system 200 may select predetermined text data according to a similarity based on the degree of correspondence with the search word. Will be chosen. That is, the image retrieval system 200 may calculate, for example, a similar probability value in consideration of the match ratio between the search word and the text data, the calculation of the same number of phonemes, the use of synonyms, and the like, and searches for text data having a predetermined similar probability value or more. .

In addition, the image retrieval system 200 extracts and provides the image stream data corresponding to the retrieved text data to the user 130 (S540). In operation S480, as a result of the search for the video search request, the video stream data recorded in the memory means is transmitted to the user 130 through the communication network 140 in response to the found partition identifier.

Therefore, according to the present invention, even if the script of the video is not provided, the service of optimally supporting the video search request using the metabolism can be obtained by using the text data extracted from the video.

Embodiments of the invention include a computer readable medium containing program instructions for performing various computer-implemented operations. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical recording media such as CD-ROMs, DVDs, magnetic-optical media such as floppy disks, and ROM, RAM, flash memory, and the like. Hardware devices specifically configured to store and execute the same program instructions are included. The medium may be a transmission medium such as an optical or metal wire, a waveguide, or the like including a carrier wave for transmitting a signal specifying a program command, a data structure, or the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

6 is an internal block diagram of a general purpose computer device that may be employed to perform the image retrieval method according to the present invention.

Computer device 600 includes one or more processors 610 coupled with a main memory device including random access memory (RAM) 620 and read only memory (ROM) 630. The processor 610 may also be called a central processing unit (CPU). As is well known in the art, the ROM 630 serves to transfer data and instructions to the CPU unidirectionally, and the RAM 620 typically transfers data and instructions bidirectionally. Used to. RAM 620 and ROM 630 may include any suitable form of computer readable media. Mass storage 640 is bidirectionally coupled to processor 610 to provide additional data storage capability, and may be any of the computer readable recording media described above. The mass storage device 640 is used to store programs, data, and the like, and is a secondary memory device such as a hard disk which is generally slower than the main memory device. Certain mass storage devices such as CD ROM 660 may be used. The processor 610 may include one or more input / output interfaces such as a video monitor, trackball, mouse, keyboard, microphone, touchscreen display, card reader, magnetic or paper tape reader, voice or handwriting reader, joystick, or other known computer input / output device. 650 is connected. Finally, the processor 610 may be connected to a wired or wireless communication network through the network interface 670. Through this network connection, the procedure of the method described above can be performed. The apparatus and tools described above are well known to those skilled in the computer hardware and software arts.

The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention.

Although specific embodiments of the present invention have been described so far, various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims below, but also by the equivalents of the claims.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above-described embodiments, which can be variously modified and modified by those skilled in the art to which the present invention pertains. Modifications are possible. Accordingly, the spirit of the present invention should be understood only by the claims set forth below, and all equivalent or equivalent modifications thereof will belong to the scope of the present invention.

As can be seen from the above description, according to the present invention, the existing search that allows the user to search the image depending on time by allowing the user to search the image using the dialogue, and search and provide a specific position of the image to reproduce the input dialogue A method and image search system for searching for a specific position of an image to perform an image search more simply and conveniently can be provided.

In addition, according to the present invention, by using the partition identifier associated with the segmentation of the video image, the script data corresponding to the video material is aligned, and the segmented video image by identifying the segmentation identifier associated with the sentence of the script data including the search word It is possible to provide a method and a video search system for searching for a specific position of an image to search stream data.

According to the present invention, it is possible to provide a method and an image retrieval system for retrieving a specific position of an image that optimally supports an image retrieval request using dialogue using text data extracted from the image.

In addition, according to the present invention, when searching for a specific section of the image, a method and image for searching a specific position of the image using an optimized text search function, rather than a monotonous search by the user's vision, depending on the key frame A search system can be provided.

Claims (10)

In the method for searching for video material, Dividing the video object into a plurality of video stream data and assigning a partition identifier to each of the divided video stream data; Extracting audio data from the video stream data and correlating the extracted audio data with an associated partition identifier; Converting symbol data of a form recognizable by a predetermined script alignment unit from the extracted audio data using a predetermined speech recognition technique; Identifying text data in the script data corresponding to the video object including the converted symbol data in the script aligning unit; And Correlating the identified text data with an associated segmentation identifier Image retrieval method comprising a. The method of claim 1, Receiving an image search request including a search word from a user; Identifying, within the script data, text data including the entered search term Retrieving a segmentation identifier that correlates to the identified text data; And Extracting the video stream data corresponding to the retrieved partition identifier and providing the same to the user Image retrieval method comprising a. The method of claim 2, The step of identifying the text data containing the search word, And when a plurality of text data is identified in the script, the predetermined text data is selected based on the similarity calculated for each text data. The method of claim 1, The script data is an image retrieval method, characterized in that consisting of one or more combinations of letters, numbers, special symbols. The method of claim 1, And the video stream data is generated by dividing the video object based on one of a reproduction time, a frame number, and a scene. In the method for searching for video material, Dividing the video object into a plurality of video stream data; Extracting audio data from the video stream data; Converting the extracted audio data into predetermined symbol data using a predetermined speech recognition technique, and identifying text data probabilisticly similar to the converted symbol data with reference to predetermined script data; Recording the identified text data and video stream data associated with the identified text data in a predetermined memory means correspondingly; Receiving an image search request including a search word from a user; Searching for text data associated with the search word with reference to the memory means; And Extracting image stream data corresponding to the retrieved text data and providing the same to the user Image retrieval method comprising a. The method of claim 6, The step of searching for text data associated with the search word, And when a plurality of the text data are searched, the predetermined text data is selected according to the similarity based on the degree of correspondence between the text data and the search word. The method according to claim 1 or 6, The speech recognition technique is a Viterbi-Search technique for characterizing the extracted audio data by comparing with a phoneme model. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 1 to 7. In the system for searching video material, An identifier association unit for dividing the video object into a plurality of video stream data and corresponding to a segmentation identifier for each of the divided video stream data; An audio extracting unit extracting audio data from the video stream data; A speech recognition unit for converting the extracted audio data into symbol data that can be recognized by a predetermined script alignment unit by using a predetermined speech recognition technique; A script alignment unit for identifying text data in the script data corresponding to the video material including the converted symbol data and correlating the identified text data with an associated segmentation identifier; An interface unit for receiving an image search request including a search word from a user; A position search unit for identifying text data including the input search word in the script data and searching for a segmentation identifier correlated with the identified text data; And Search result providing unit for extracting the video stream data corresponding to the searched partition identifier and provides the user to the extracted stream; Image retrieval system comprising a.

Images