JPH11161661A - Information retrieval device - Google Patents

Abstract

(57) [Summary] (with correction) [PROBLEMS] To provide an information search system that generates a search key using a video signal or an audio signal and searches for related information based on the search key. SOLUTION: A search key extraction device 2 for extracting a search key from an input audio signal or video signal, an instruction device 26 for selecting a keyword from the extracted search key and making a search request, and an information search based on the keyword And a control device 23 that outputs a keyword requested to be searched by the pointing device to the information search engine and outputs information searched from the information search engine to the monitor device 5. Further, an information extraction device 28 for storing the search history information in the information storage device 17 and analyzing the search history information is provided. By adopting such an information search device, the trouble of input is unnecessary, a simple information search can be performed, and an appropriate information search can be performed according to a user.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information retrieval system for generating a retrieval key using a video signal or an audio signal and retrieving related information based on the retrieval key.

[0002]

2. Description of the Related Art In recent years, a large amount of information has been overflowing with the development of a highly information-oriented society. Therefore, the importance of an information retrieval system for efficiently obtaining only necessary information from a large amount of information is increasing. In particular, WWW (Wo
Due to the development of ld wide web, there is a demand for an information search system that enables even a user with little experience of using the information search system to easily obtain desired information.

[0003] As such an information retrieval system, for example, RCAAU (http://www.k.jp) developed at Kyoto University for retrieving information on the Internet.
uamp. kyoto-u. ac. jp / labs / i
nfocom / mondou /). RCAA
In U, input multiple character strings such as Japanese and English,
Information related to those character strings, for example, a URL (Uniform Re
source Locator), which is widely used as an information collection tool.

[0004]

SUMMARY OF THE INVENTION However, RCA
In an information retrieval system represented by an AU, since a user needs to enter a character string as a search key using a keyboard or a mouse, the user cannot immediately enter a character string as a key even if he or she thinks of it. In addition, there is a problem that a load of input is large unless a user is skilled in operating the device. In addition, since there is no information other than a character string as a search key, for example, information on a use environment such as information that can be extracted from a TV program broadcast in a specific time zone, and a use history of a user such as a search execution result performed previously There is a problem that a search in consideration of information cannot be performed.

[0005]

According to the present invention, there is provided means for extracting a search key from an input audio signal or video signal, and instruction means for selecting a keyword from the extracted search key and issuing a search request. Information search engine means for performing an information search based on the keyword, monitor means for displaying the searched information, and outputting a keyword requested to be searched by the instruction means to the information search engine means,
Control means for outputting information retrieved from the information retrieval engine means to the monitor means.

[0006] Further, the information search apparatus further includes an information storage means for inputting and storing the searched information and search history information from the control apparatus and outputting the information based on a request from the instruction means. I will provide a.

Further, the present invention provides an information retrieval apparatus to which an information extraction means for analyzing the retrieval history information stored in the information storage means and outputting the result based on a request from the instruction means is added.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS (Embodiment 1) An embodiment of an information retrieval system according to claim 1 of the present invention will be described with reference to FIG. 1 which is a block diagram.

General TV tuners, VCRs and D
The VD device or the like outputs the signal as an NTSC or PAL signal, and the digital data generator 1 receives the video signal and the audio signal from these, converts the signal into digital data, and outputs the digital data to the search key extracting device 2. It is a device to do.

As the digital data generator 1 as described above, for example, there is a device called Super CVI manufactured by Canopus. When an NTSC signal is input to the same device, in the case of a video, the video is converted into a sequence of digital data (bitmap image), and 30 frames of digital data can be output per second. . In the case of voice, the device outputs an analog voice signal. By applying the analog signal to an analog-to-digital converter, digital voice data can be output in real time, and the digital data generator 1 is configured as a whole. The digital data generator 1 can generally handle a plurality of input signals, and can switch the source of digital data generated by a command signal from the controller 3.

The search key extracting device 2 generates a search key to the information search engine 4 based on the input from the digital data generating device 1 and outputs the key to the control device 3 together with the video / audio signal input from the digital data generating device 1. I do.

[0012] In the case of a video, among the bitmap images extracted by the digital data generator 1, an image with subtitles, such as a news program, is first searched for a portion having characters. At this time, for example, an area surrounded by a line in which a color component included in a certain vertical and horizontal line is greatly changed is searched. Next, two colors are identified: a color in which the color points are continuous with respect to the area and which is considered to constitute a character, and a color which appears next to the color and constitutes the background. Then, based on the two colors, it is calculated which of the two colors is closer to each point in the area to determine whether the point is a character portion or a background portion. as a result,
Bitmap images can be converted to monochrome character image data. The distance between colors is, for example, assuming that a color component is represented by an RGB vector of c (r, g, b).
Assuming that the color components of the digital data are c1 and c2 in chronological order, | c1−c2 | = sqrt ((r1−r2) ＾ 2
+ (G1-g2) ＾ 2 + (b1-b2) ＾ 2). It is an object of the present invention to provide an apparatus that performs accurate and high-speed character recognition after binarizing a color considered to constitute a character and a color considered to constitute a background into monochrome. By using the character recognition device proposed in Japanese Patent Application Laid-Open No. Hei 6-20096 or the like, it is possible to extract a character string such as subtitles displayed on a bitmap image.

Further, by analyzing a character string and extracting only meaningful search keys, or by referring to a dictionary of similar characters, even if there is some misrecognition, the character string is converted into a correct search key to perform recognition accuracy. Is output to the control device 3. In addition to the extracted search key, date and time, channel, video (image) data, audio data, and the like can be output.

In the case of voice, the digital data generator 1
The digital audio data obtained by
A character string is extracted by inputting to a speech recognition device proposed in JP-A-6-138895 or the like for the purpose of providing a speech recognition device excellent in recognition performance of sampling data by conversion under noise. be able to. At this time, inside the speech recognition device, the character string obtained as the Japanese syllabary (a string of kana characters) is compared with a word dictionary similar to that used in the kanji conversion, and a word string containing the most suitable kanji is used. Is converted to a search key. Further, by performing morphological analysis, it is possible to adopt only the noun as a search key, and the like, and this is output to the control device 3. At this time, in addition to the extracted search key, date and time, channel information, video (image) data, audio data, and the like can be output.

The instruction device 6 is a device that can generate a search request signal, and notifies the control device 3 that a search request is made according to the user's intention. For example, a remote control and its signal receiving unit and its processing circuit,
Or it can simply consist of one button directly connected to the system.

The control device 3 is a device for controlling the operation of each device and the flow of data, and a block diagram of its structure is shown in FIG. CPU 101 and main storage device (memory in control device)
102, an auxiliary storage device 103, an input device 104, an output device 105, and the like, and can be configured in the same manner as a general-purpose PC (personal computer).

The information search engine 4 is, for example, an RCAAU (btp: //www.uamp.ky) for performing information search on the Internet developed at Kyoto University.
oto-u. ac. jp / labs / infocom /
For example, it can receive a keyword as a search key, and output a URL of a site related to the keyword or information related to the keyword as a search result. In this case, a telephone line or LAN (L
(Local Area Network) via a modem or a LAN board. However, it is not limited to information retrieval on the Internet, but may simply search for a Japanese dictionary or encyclopedia. In this case, a large-capacity storage device such as a CD-ROM or a DVD is connected.

The monitor device 5 is a device that displays a video or a search result by a digital signal from the digital data generator 1 and outputs audio. Monitor device 5
Comprises a CRT, a liquid crystal display, a plasma display, a speaker, and the like. Also, both a digital signal and an analog signal such as NTSC, which is a normal TV broadcast system, may be input, and switching may be performed.

First, a TV antenna line is connected to the digital data generator 1, the digital data generator 1 generates a video or audio digital signal, and outputs the digital signal to the search key extraction device 2. ing. This digital signal is also output directly to the monitor device 5 or via the control device 3 so that the user can view the digital signal in the same manner as a normal TV broadcast program. However, if the monitor device 5 is configured to receive an analog signal, the analog signal before conversion into digital data by the digital data generator 1 may be directly output to the monitor device 5.

Next, the processing procedure will be described with reference to the flowchart shown in FIG. First, in step S1, the digital data generator 1 receives the signal. In step S2, the data is converted into digital data. In step S3, a search key is extracted from the digital data. At this time, a character string may be actually extracted, such as when a caption is displayed in a news program, or no character string may be extracted, such as when no character is displayed. Only when a column is extracted, a search key that is an extracted character string is transmitted to the control device 3, and the control device 3 stores this information in the main storage device 102. In step S4, it is monitored whether there is an instruction signal from the instruction device 6.

If there is no instruction signal from the instruction device 6, the processing from step S1 to step S4 is performed in the background (TV broadcasts and the like are always output to the monitor device 5 and the user is watching ordinary TV broadcasts. (Without changing the external appearance) as quickly and repeatedly as possible.
It is desirable to repeat as frequently as possible to extract all characters appearing on the screen and output voice.

For example, it is assumed that the user wants to search for information related to the information of interest while watching a TV program or the like of interest. The contents of a normal program are often displayed as captions with related keywords or output through voice. In such a case,
The user uses the pointing device 6 to search for information.

Specifically, for example, consider a case where a TV broadcast of news relating to baseball is being received. FIG.
1 shows a TV screen of news about baseball. At this time, for example, when the pitcher who is a baseball game has achieved no hit no run, "no hit no run achieved"
Assume that a caption such as appears or is output as audio. FIG. 12 shows a TV screen when a no-hit no-run caption appears. Suppose that the user wants to gain knowledge on achieving no-hit no-run. At this time, the caption "No hit no run achieved" appeared or was output as audio,
In steps S1 to S3 of the flowchart, characters such as "no hit", "no run", and "achieved" are extracted and stored in the main storage device 102. Table 1 shows an example of data stored in the main storage device. In the case of Table 1, the configuration is made up of a bitmap as digital data, extracted characters as search keys, date and time, and channels.

[0024]

[Table 1]

Next, a description will be given of a case where the user uses the pointing device 6 to search for information. FIG. 8 shows an external view of the pointing device 6. As shown in FIG. 8, the pointing device 8 has a search start request button 6a, a selection button 6b, a video source switching button 6c, a search history button 6d, and a search sequence button 6e.

The user presses the search start request button 6a of the pointing device 6. At this time, steps S4 and S4 shown in FIG.
At 5, the search start request signal is transmitted to the control device 3. Next, in step S6, the control device 3 displays a search keyword selection menu on the monitor device 5 based on the search key information stored in the main storage device 102. For example, "No hit" and "No run" as shown in FIG.
A search keyword screen presented by a search key such as "achievement" is displayed. In steps S7 and S8, a keyword is selected from the keyword selection button 6b of the pointing device 6.
For example, three of "no hit", "no run", and "achieved" are selected. FIG. 10 shows a search keyword screen in which a black circle is added before the selected keyword. The dot before the unselected search key remains as a dot. Next, in step S9, the control device 3 outputs those keywords to the information search engine 4 as search keys. In step S10, the control device 3 receives the search result from the information search engine 4, and in step S11, the search result is obtained. Is output to the monitor device 5 to display and output the search result.
FIG. 13 shows a diagram displaying the search results. As a result, it is possible to search for related information relating to “achieving no hit no run”.

When displaying the search results, the display of the general TV broadcast or the like is interrupted as shown in FIG. 13 and the display is switched to display the search results or the TV broadcast image is displayed. The search results may be displayed in an overlapping manner. FIG. 14 shows a display screen of the search result in this case. Alternatively, the screen may be divided into two, and one may display a search result on the other, such as a general TV broadcast. FIG. 15 shows a display screen of the search result in this case.

The content of the information depends on what information the information search engine 4 can search, such as simply drawing a Japanese language dictionary or encyclopedia, requesting a search from an information search server via a network, or the like. It can also be said. When the output of the result is completed, the processing is repeated from Start (step S1). In addition, the user presses the video source switching button 6c of the pointing device 6 to find the information of interest in the same manner as changing channels while watching TV, and the data becomes the original data for keyword extraction. It is also possible to generate a source switching request signal such as changing the channel of the TV which is a signal.

Referring to FIG. 5, a flow chart of the generation of a search key instruction signal of the instruction device 6 will be described. In steps S101 and S103, the instruction device 6 continues to check whether there is a search start request or a source switching request of the user. Then, the instruction device 6 performs step S101,
When the generation of the search start request signal or the source switching request signal is known in S103, an instruction signal notifying the fact is transmitted to the control device 3 in steps S102 and S104.

When receiving the source switching request signal from the instruction device 6, the control device 3 outputs a command such as a channel setting to the digital data generating device 1 in step S12, and starts the process again from Start (step S1). To resume.

In the present embodiment, it is assumed that the TV broadcast, which is the signal serving as the original data for keyword extraction, is always output to the monitor device 6. However, the present invention is not limited to this. A TV signal, which is a signal serving as data, may not be displayed on the monitor device 5.
In this case, the character extraction is continued from the signal in the same manner and the search key is stored in the main storage device 102, but the TV broadcast and the like are not displayed on the monitor device 5 at the same time. When a user issues a search request to this information search system, a search keyword selection menu is displayed on the monitor device 5.

In addition, character extraction processing (steps S1 to S1)
3) and the search process (steps S5 to S11) are independent processes and do not have to be performed simultaneously. Character extraction process (step S1) is limited to a certain time zone.
To S3), or the search processing (steps S4 to S11) may be performed only during a certain time period. That is, the processing may be omitted in a certain time zone. For example,
Character extraction processing from about 9 am to 10 am (step S1
To S3) may be repeated, and the search process (steps S4 to S11) may be started at 11:00 at night, and the keyword extracted on that day may be checked to search for information again.

(Embodiment 2) An embodiment of the information retrieval system according to claim 2 of the present invention will be described with reference to the block diagram shown in FIG. 2 and the flowchart shown in FIG. This embodiment is characterized in that an information storage device 17 is added in comparison with the first embodiment. Accordingly, the control device 13,
The pointing device 16 has more functions than the control device 3 and the pointing device 6 of the first embodiment, but the other parts are the same as those of the first embodiment.

The information storage device 17 is a device for storing various digital information, and is configured using a storage device such as a flash memory or a magnetic disk device.

The control device 13 generates search history information in which keywords, search results, video data, audio data, date and time, channel information, and the like are associated in addition to the functions described in the first embodiment. This is output to the information storage device 17, and the information storage device 17 stores the search history information. Of course, such a process may be performed before executing the information search. Table 2 shows a case where the search history information obtained from a source such as a television includes date and time, channel, keyword, search result, and video / audio signal.

[0036]

[Table 2]

When the user presses the search history button 6d of the pointing device 16 shown in FIG. 8 after the user performs steps S1 to S4 according to the flowchart of FIG. Transmits a search history request signal to the control device 13. A brief description will be given with reference to FIG. 5 which is a flowchart for generating a search history request signal at this time. Step S101 described in the first embodiment
After checking whether there is a search request and a source switching request in steps S104 to S104, the search history button 6 is checked in step S105.
It is checked whether d has been pressed, and if it has been pressed in step S106, a search history signal is generated.

In step S4 of FIG.
3 receives the search history request signal, in step S22, requests the search history information already stored in the information storage device 17, and displays a search keyword selection menu based on the search history information obtained from the information storage device 17. Make up,
It is displayed on the monitor device 5. FIG. 16 shows a display example of search history information.

The subsequent processing is the same as when a search keyword selection menu is constructed based on the information in the main storage device 102. That is, the user can easily perform the information search using the same search key as the previously performed information search by pressing the search history button 6d.

For example, when the information shown in Table 2 is stored in the information storage device 17, the user can press the search history button 6d to display the history searched by the user so far. , You can easily search based on it.

(Embodiment 3) An embodiment of the information retrieval system according to claim 3 of the present invention will be described with reference to the block diagram shown in FIG. 3 and the flowchart shown in FIG. This embodiment is characterized in that an information extraction device 28 is added as compared with the first embodiment. The information extraction device 28 performs processing such as frequency analysis such as calculating a time zone with a high access to the series of search history information stored in the information storage device 17 and feature extraction such as checking a keyword to check a user's preference. And can be configured in the same manner as a general-purpose PC (personal computer). Information extraction device 2
8 can always refer to the contents of the information storage device 17 asynchronously with the control device 3.

As in the second embodiment, when the user activates the information search engine 4 and saves the search results in the information storage device 17 at step S21 shown in FIG. 17 can detect that the information has been stored, retrieve the stored search result from the information storage device 17, and check, for example, what keyword was used for the search. The search history information in the past is checked. If the search is performed by the same keyword as before, the number of times of the search by the keyword is counted by referring to the contents of the information storage device 17, and the absolute number and the relative number of times to the whole Is calculated and configured as search sequence information. Table 3 shows the search sequence information at this time. Table 3 shows new information names obtained by analyzing the search history information, and the resulting values.

[0043]

[Table 3]

This search sequence information is stored in step S shown in FIG.
At 31 the data is output to the information storage device 17 again and stored.

In addition, for example, by enumerating from which information source the keyword used in the search is extracted, for example, the keyword used in the search may be used as the TV 8c.
Assuming that 50% is extracted from h, this user can make an guess that half of the time watching TV is watching 8ch, and transmit information via the network. If a means for referring to the contents of the information storage device 17 such as a means is provided, a highly reliable audience rating survey focusing on individuals can be performed.

When the user presses the search sequence button 6e of the pointing device 26 of FIG. 8, the pointing device 26 in steps S109 to S110 of FIG. The device 23 requests the information storage device 17 to output the search sequence information calculated in advance by the information extracting device 28.
In step S32 of FIG. 4, a keyword selection menu is generated based on the input from the information storage device 17 and output to the monitor device 5. At this time, for example, keywords having the largest number of search executions in the past search history may be displayed in order. Subsequent processing (steps S7 to S11)
Is equivalent to a case where a search keyword selection menu is configured based on information in the main storage device 102.

For example, assuming that the contents as shown in Table 3 are stored in the information storage device 17, when the user presses the search sequence button 6e of the pointing device 26, the information is rearranged in the order of the keyword search frequency as shown in FIG. The selected keyword selection menu is displayed on the monitor device 5. It can be said that a keyword such as “Internet” is selected and the information search is executed again by the search key.

Here, the keyword may be called for general information related to the keyword such as data (image-sound) from which the keyword is extracted, and the information retrieval is also performed in this case.

The information extracting device 28 and the control device 3 can be implemented as one device or as independent devices that operate asynchronously.

[0050]

According to the information retrieval apparatus according to the first aspect,
Since the user does not need to input or edit a character string serving as a search key, a user unfamiliar with the operation of the device can easily perform a search.

According to the information search apparatus of the present invention, search history information such as date and time, channel, video, audio, search result, and search condition can be stored as one piece of information without the user inputting it. By retrieving and displaying the search history information later, it is possible to confirm their relevance and use it as a new search key.

According to the information search apparatus of the third aspect, the search history information such as the date and time channel, video, audio, search result, search condition, and the like stored in association with each other is analyzed so that the information is included in the original information. For example, new information can be obtained or a new search key can be generated, so that an appropriate search can be performed according to the user.

[Brief description of the drawings]

FIG. 1 is a block diagram showing Embodiment 1 of an information search system according to the present invention.

FIG. 2 is a block diagram showing a second embodiment of the information search system according to the present invention.

FIG. 3 is a block diagram showing a third embodiment of the information search system according to the present invention.

FIG. 4 is a flowchart of the information search system according to the present invention.

FIG. 5 is a flowchart of a search key instruction of the information search system according to the present invention.

FIG. 6 is a diagram showing a structure of a control device in the information search system according to the present invention.

FIG. 7 is a schematic diagram of a bitmap image handled in the information search system according to the present invention.

FIG. 8 is an external view of a pointing device in the information search system according to the present invention.

FIG. 9 is a diagram illustrating a display example of a search keyword selection menu in the information search system according to the present invention.

FIG. 10 is a diagram illustrating a display example regarding selection of a search keyword in the information search system according to the present invention.

FIG. 11 is a diagram illustrating a display example of a display device in the information search system according to the present invention.

FIG. 12 is a diagram illustrating another display example of the display device in the information search system according to the present invention.

FIG. 13 is a diagram illustrating a display example of a search result in the information search system according to the present invention.

FIG. 14 is a diagram illustrating another display example of a search result in the information search system according to the present invention.

FIG. 15 is a diagram illustrating another display example of a search result in the information search system according to the present invention.

FIG. 16 is a diagram illustrating a display example of search history information in the information search system according to the present invention.

FIG. 17 is a diagram illustrating a display example of search sequence information in the information search system according to the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 video / audio signal generator 2 search key extraction device 3, 13, 23 control device 4 information search engine 5 monitor device 6, 16, 26 pointing device 17 information storage device 28 information extraction device 101 CPU 102 main storage device 103 auxiliary storage device 104 input device 105 output device

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁶ Identification symbol FI H04N 5/78 510 G06F 15/40 370E H04L 11/20 101B G11B 27/00 E G06F 13/00 355

Claims (3)

[Claims] 1. A means for extracting a search key from an input audio signal or video signal, an instruction means for selecting a keyword from the extracted search key and making a search request, and performing an information search based on the keyword An information search engine means, a monitor means for displaying the searched information, a keyword requested by the search means from the instructing means being output to the information search engine means, and the information searched from the information search engine means being monitored. Control means for outputting to the means. 2. The information processing apparatus according to claim 1, further comprising an information storage unit that inputs and stores the searched information and search history information from the control device, and outputs the information based on a request from the instruction unit. Information retrieval device. 3. The apparatus according to claim 2, further comprising an information extracting unit that analyzes the search history information stored in the information storing unit and outputs a result based on a request from the instruction unit. Information retrieval device.