JP3762191B2 - Information input method, information input device, and storage medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an information input method, an information input device, and a storage medium, and more particularly to a technology for inputting character information using a speech recognition technology.
[0002]
[Prior art]
When inputting character information such as characters and symbols into an input item (input form) of a graphical user interface (GUI), generally, a pointing device such as a keyboard and a mouse is used. In the case of a keyboard, characters and symbols are typed, and in the case of a pointing device, a desired character or symbol is selected and input from selection candidates.
[0003]
[Problems to be solved by the invention]
However, when inputting character information to a desired input item using a pointing device such as a keyboard or a mouse, all operations from selection of the input item to input of character information must be performed manually. There was a problem that it was very difficult to handle in a difficult environment.
[0004]
In view of the above-described problems, an object of the present invention is to easily input character information to a plurality of input items displayed on a display unit without performing manual input.
[0005]
[Means for Solving the Problems]
An information input method of the present invention is an information input method for inputting character information to a plurality of input items displayed on a display, and includes a reception step of receiving speech and one or more of a plurality of grammatical rules. A speech recognition step for recognizing character information from the speech, and a selection step for selecting an input item for inputting the character information based on the additional information of the input item and the grammatical rule used in the speech recognition step. And the additional information is information indicating a correspondence relationship between the input item and the grammar rule, and the speech recognition step uses the grammar rule associated with the input item for which character information is not input. Character information is recognized from voice.
[0006]
An information input device according to the present invention is an information input device for inputting character information to a plurality of input items displayed on a display, and includes a receiving means for receiving speech and one or more of a plurality of grammatical rules. Voice recognition means for recognizing character information from the voice, and selection means for selecting an input item for inputting the character information based on the additional information of the input item and the grammatical rules used in the voice recognition means. And the additional information is information indicating a correspondence relationship between the input item and the grammar rule, and the speech recognition means uses the grammar rule associated with the input item for which character information is not input. Character information is recognized from voice.
[0007]
DETAILED DESCRIPTION OF THE INVENTION
(First embodiment)
FIG. 1 is a block diagram showing the configuration of the information input device in this embodiment.
In FIG. 1, reference numeral 1 denotes a display device for displaying a graphical user interface (GUI) of this embodiment. The display device 1 includes a display such as a CRT, a liquid crystal panel, or a plasma display panel. A central processing unit 2 performs processing such as numerical calculation and control.
[0008]
A storage device 3 includes a RAM, a ROM, a magnetic disk, an optical disk, a semiconductor memory, a hard disk device, and a combination thereof. The storage device 3 holds the GUI of the present embodiment, a control program necessary for the processing procedure of the present embodiment, and an operating system (OS) that manages this control program.
[0009]
Reference numeral 4 denotes a voice recognition unit. The voice recognition unit 4 performs acoustic processing on the voice input from the microphone 5 and performs language processing on the result of the acoustic processing. The storage device 3 holds an acoustic model 10 used in acoustic processing, N types of grammatical rules 111 to 11n and word dictionaries 121 to 12n used in language processing.
[0010]
Here, each grammar rule 111 to 11n is a grammar rule that is most suitable for recognition of a predetermined type of character information, and each word dictionary 121 to 12n is a word dictionary corresponding to each grammar rule 111 to 11n. The voice recognition processing in the voice recognition unit 4 can also be realized by software.
[0011]
The display device 1, the central processing unit 2, the storage device 3, and the voice recognition unit 4 are connected by a bus 16. The central processing unit 2 reads out a control program necessary for the processing procedure of the present embodiment from the storage device 3, and integrates the speech recognition processing of the speech recognition unit 4, the display processing of the display device 1, and the read / write processing of the storage device 3. To control.
[0012]
FIG. 2 is a flowchart showing a processing procedure of the information input apparatus of this embodiment. The processing performed at each step of the flowchart shown in FIG. 2 is realized by the central processing unit 2 controlling each processing unit based on a control program stored in the storage device 3.
[0013]
First, in step S1, the GUI of this embodiment is read from the storage device 3 and displayed on the display device 1. This GUI includes one or a plurality of input forms (also referred to as input items), and additional information is set in each input form. This additional information is information indicating a grammar rule optimum for recognition of a predetermined type of character information and a correspondence relationship, and is, for example, an index (grammar ID) for identifying the type of grammar rule. The GUI according to the present embodiment is described in a description language such as HTML (Hyper Text Markup Language) or XML (Extensible Markup Language).
[0014]
An example of the GUI of this embodiment will be described with reference to FIG. FIG. 3 is a diagram showing a GUI including three input forms. This GUI is an application program GUI for searching for a route between stations. In FIG. 3, 6 is a departure station name, 7 is an arrival station name, and 8 is an input form for inputting a departure time.
[0015]
Both the input form 6 and the input form 7 are input forms for inputting station name information, and these are associated with grammar rules that are optimal for recognizing station name information. In the present embodiment, the grammar rule 111 (“grammar 1” in FIG. 1) is described as a grammar rule corresponding to the input forms 6 and 7.
[0016]
On the other hand, since the input form 8 is an input form for inputting time information, it is associated with a grammar rule that is optimal for recognition of time information. In the present embodiment, the grammar rule 112 (“grammar 2” in FIG. 1) will be described as a grammar rule corresponding to the input form 8.
[0017]
Next, in step S2, a grammar rule associated with an input form for which character information has not yet been input is recognized, and the recognized grammar rule is set. When there is one unfilled input form, one type of grammar rule corresponding to the input form is set, and when there are two or more input forms, one or more types of grammar rules are set. For example, in the case of the GUI shown in FIG. 3, grammar rules 111 and grammar rules 112 are set.
[0018]
In step S3, a voice input is accepted. The voice uttered by the user is converted into an electrical signal by the microphone 5 and then supplied to the voice recognition unit 4.
[0019]
In step S4, the speech input in step S3 is recognized by using one or more kinds of grammar rules set in step S2. For example, in the case of the GUI shown in FIG. 3, station name information and time information are recognized from the voice input in step S2, using the grammatical rules 111 and 112 and the word dictionaries 121 and 122.
[0020]
In step S4, when there is one kind of grammar rule set in step S2, the result of the acoustic processing is subjected to language processing using the grammar rule and a word dictionary corresponding to the grammar rule. The character information obtained from the grammatical rules is used as the recognition result in step S4.
[0021]
On the other hand, when there are two or more types of grammar rules set in step S2, the sound processing result is subjected to language processing using each grammar rule and a word dictionary corresponding to each grammar rule. Then, character information whose likelihood with the input speech is equal to or greater than a predetermined value among character information obtained from each grammar rule is set as a recognition result in step S4.
[0022]
In step S5, it is determined whether or not the input form for inputting the character information obtained in step S4 can be uniquely determined based on the correspondence between each input form and the grammatical rule. In the present embodiment, the input form for inputting character information is automatically determined from the grammatical rule that recognizes the character information obtained in step S4 by the correspondence between each input form and the grammar rule.
[0023]
Therefore, for example, when character information is recognized from one kind of grammar rule and the grammar rule corresponds to only one input form, one input form for inputting the character information obtained in step S4 is provided. Can only be determined automatically. With this configuration, it is possible to automatically select and determine an input form desired by the user without causing the user to select an input form in advance.
[0024]
However, in the following case, since the input form for inputting the character information obtained in step S4 cannot be determined as one, the process of step S6 is executed.
[0025]
1) When multiple grammar rules recognize vocabulary of the same sound. For example, when “Sendai” (/ seNdai /) is registered in the word dictionary A used by the grammar rule A, and “predecessor” (/ seNdai /) is registered in the word dictionary B used by the grammar rule B. In this case, the user pronounces / seNdai /. In this case, “Sendai” is recognized in the grammatical rule A, and “predecessor” is recognized in the grammatical rule B, and both become recognition results in step S4, and one input form cannot be determined.
[0026]
2) A single grammar rule corresponds to multiple input forms. This is the case, for example, when the grammar rule 111 corresponds to the two input forms 6 and 7, as in the GUI of FIG. In this case, one input form for inputting character information recognized by the grammatical rule 111 cannot be determined as one.
[0027]
In step S6, the user is notified of input forms that are selection candidates, and the user is allowed to select one input form. The user selects one input form by inputting the selection candidate form name into the microphone 5. Thereby, it is possible to easily select an input form without using a keyboard or a pointing device.
[0028]
Here, there are various methods for notifying selection candidates. For example, when all input forms are displayed on one screen as in the GUI of FIG. 3, the color of the selection candidate's surroundings, background graphics color, and design are changed. The selection candidate is notified to the user by a method such as changing the color and font.
[0029]
On the other hand, if not all input forms are displayed on one screen as in the GUI of FIG. 4, the form names of the selection candidates are displayed together in a separate window. The user is notified of selection candidates by a method such as presenting by voice. With this configuration, it is possible to notify the user of input forms that are selection candidates in an easy-to-understand manner.
[0030]
In step S7, the character information obtained from the grammar rules corresponding to the input form is displayed on the input form determined in step S5 or the input form determined in step S6.
[0031]
Here, when all the input forms are not displayed on one screen as in the GUI of FIG. 4, when character information is displayed on an input form that is only partially displayed or not all displayed. Automatically scrolls the GUI so that the input form is centered on the screen. A specific example will be described with reference to FIGS. When displaying character information on the input form 11 (form 4 in FIG. 4) that is not completely displayed, the GUI in FIG. 4 scrolls as shown in FIG. 5 and arranges the input form 11 in the center of the screen. With this configuration, it is possible to easily notify where the input form selected by the user is on the GUI.
[0032]
In step S8, it is determined whether or not the character information displayed on the input form is correct. If the displayed character information is not correct, the user inputs the voice uttered “No” to the microphone 5. In this case, the character information displayed on the input form is cleared and the process of step S2 is executed.
[0033]
On the other hand, when the displayed character information is correct, the user inputs the voice uttered “Yes” to the microphone 5. In this case, the character information displayed on the input form is determined as an input for the input form (step S9).
[0034]
In step S10, it is determined whether or not there is an input form that has not been input. If there is an input form that has not been input, the process of step S2 is executed. If there is no input form that has not been input, the process is performed. finish.
[0035]
As described above, according to the present embodiment, the keyboard or pointing device is used by determining the input form for inputting the character information recognized from the user's voice according to the correspondence between the input form and the grammatical rule. It is possible to easily input character information into an input form desired by the user without any problem.
[0036]
(Second Embodiment)
FIG. 6 is a flowchart illustrating a processing procedure according to the second embodiment. Since the configuration of the information input device of the second embodiment is the same as that of the first embodiment, description thereof is omitted. The processing performed in each step of the flowchart shown in FIG. 6 is realized by the central processing unit 2 controlling each processing unit based on a control program stored in the storage device 3.
[0037]
First, in step S11, the GUI of the second embodiment is read from the storage device 3 and displayed on the display device 1. This GUI includes one or a plurality of input forms (also referred to as input items), and additional information is set in each input form. The additional information includes information indicating a correspondence relationship with a grammar rule optimum for recognition of a predetermined type of character information, and information indicating a correspondence relationship with a predetermined character string (also referred to as a keyword). The character string is a sentence, a phrase, a word, or the like that supports the input of the form name of the input form or character information to the input form, and is displayed on the GUI. In the second embodiment, the grammar rule corresponding to each input form is configured to recognize character information including a keyword.
[0038]
An example of the GUI of the second embodiment will be described with reference to FIG. FIG. 7 is a diagram showing a GUI including three input forms. This GUI is an application program GUI for searching for a route between stations. In FIG. 7, 71 is a departure station name, 72 is an arrival station name, and 73 is an input form for inputting a departure time.
[0039]
Both the input form 71 and the input form 72 are input forms for inputting station name information. The keyword of the input form 71 is “from the station”, and the keyword of the input form 72 is “to the station”. As in the first embodiment, the input forms 71 and 72 are associated with the grammar rule 111 (“grammar 1” in FIG. 1), which is the grammar rule optimal for the recognition of station name information.
[0040]
On the other hand, the input form 73 is an input form for inputting time information, and the keyword is “time is”. As in the first embodiment, the input form 73 is associated with the grammar rule 112 (“grammar 2” in FIG. 1), which is the grammar rule optimal for the recognition of time information.
[0041]
Next, in step S12, as in the first embodiment, a grammar rule associated with an input form for which character information has not yet been input is recognized, and the recognized grammar rule is set. For example, in the case of the GUI shown in FIG. 7, the grammar rule 111 and the grammar rule 112 are set.
[0042]
In step S13, a voice input is accepted. At this time, the user utters the keyword of the input form together with the character information to be input to the desired input form. The voice uttered by the user is converted into an electrical signal by the microphone 5 and then supplied to the voice recognition unit 4.
[0043]
In step S14, as in the first embodiment, the speech input in step S13 is recognized using one or more grammar rules set in step S12. For example, in the case of the GUI shown in FIG. 7, the station name information and time information are recognized from the voice input in step S12 using the grammatical rules 111 and 112 and the word dictionaries 121 and 122.
[0044]
In step S15, the input form for inputting the character information obtained in step S14 is selected based on the correspondence between each input form and the grammatical rule, and the keyword of the selected input form is compared with the keyword included in the character information. To determine one input form.
[0045]
In this embodiment, not only the correspondence between each input form and the grammar rule, but also the character information is input from the grammar rule that recognized the character information obtained in step S14 using the correspondence between each input form and the keyword. The input form to be automatically determined. With this configuration, it is possible to automatically select and determine an input form desired by the user without causing the user to select an input form in advance.
[0046]
In step S16, the character information obtained in step S14 is displayed on the input form determined in step S15. However, keywords are excluded. For example, when the character information obtained in step S14 is “From XX station”, the character information “XX” is displayed on the input form 71 corresponding to the keyword “From station”.
[0047]
Here, when all the input forms are not displayed on one screen as in the GUI of FIG. 4, when character information is displayed on an input form that is only partially displayed or not all displayed. As in the first embodiment, the GUI is automatically scrolled so that the input form is arranged at the center of the screen.
[0048]
In step S17, it is determined whether or not the character information displayed on the input form is correct. If the displayed character information is not correct, the user inputs the voice uttered “No” to the microphone 5. In this case, the character information displayed on the input form is cleared and the process of step S12 is executed.
[0049]
On the other hand, when the displayed character information is correct, the user inputs the voice uttered “Yes” to the microphone 5. In this case, the character information displayed on the input form is determined as an input for the input form (step S18).
[0050]
In step S19, it is determined whether or not there is an input form that has not been input. If there is an input form that has not been input, the process of step S2 is executed. If there is no input form that has not been input, the process is performed. finish.
[0051]
As described above, according to the second embodiment, the keyboard or pointing device is used by determining the input form for inputting the character information recognized from the user's voice according to the correspondence between the input form and the keyword. It is possible to easily input character information into an input form desired by the user without doing so.
[0052]
(Third embodiment)
In the second embodiment, the example in which the input form for inputting the character information recognized from the user's voice is determined according to the correspondence between the input form and the keyword has been described.
[0053]
In contrast, in the third embodiment, an example will be described in which an input form for inputting character information recognized from the user's voice is determined according to the input form candidate list. Here, the candidate list indicates character information that can be entered in the input form.
[0054]
In this case, the additional information of each input form is information indicating a correspondence relationship with a grammar rule optimum for recognition of a predetermined type of character information, and a candidate list indicating character information that can be input to the input form.
[0055]
The input form for inputting character information recognized from the user's voice is automatically determined based on the correspondence between each input form and the grammar rule and the candidate list of each input form. With this configuration, it is possible to automatically select and determine an input form desired by the user without causing the user to select an input form in advance.
[0056]
As described above, according to the third embodiment, the input form for inputting the character information recognized from the user's voice is determined according to the input form candidate list without using a keyboard or a pointing device. Character information can be easily input into an input form desired by the user.
[0057]
(Another embodiment of the present invention)
Each of the above-described embodiments may be applied to a system composed of a plurality of devices or an apparatus composed of a single device.
[0058]
In addition, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, and a nonvolatile memory may be used as a recording medium for storing a program code of a control program for realizing the functions of the above-described embodiments A card, ROM, etc. can also be used.
[0059]
In addition, when the program code of the above-described control program realizes the functions described in the above-described embodiment in cooperation with an OS (operating system) or other application software running on the central processing unit 2 It goes without saying that such program code is included in the embodiment of the present invention.
[0060]
Further, after the program code of the control program is stored in the memory provided in the function expansion board or function expansion unit, the CPU or the like provided in the function expansion board or function expansion unit performs actual processing based on the instruction of the program code. The present invention also includes a case where the function of the above-described embodiment is realized by performing part or all of the above-described processing.
[0061]
【The invention's effect】
According to the present invention, character information is recognized from speech using grammar rules associated with input items for which character information has not been input. Therefore, grammar rules can be appropriately limited, and processing accuracy and The processing speed can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an information input device according to an embodiment.
FIG. 2 is a flowchart illustrating a processing procedure according to the first embodiment.
FIG. 3 is a diagram illustrating an example of a GUI according to the first embodiment.
FIG. 4 is a diagram showing another example of the GUI (before scrolling) in the present embodiment.
FIG. 5 is a diagram showing another example (after scrolling) of the GUI according to the present embodiment.
FIG. 6 is a flowchart illustrating a processing procedure according to the second embodiment.
FIG. 7 is a diagram illustrating an example of a GUI according to the second embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Display apparatus 2 Central processing unit 3 Memory | storage device 4 A / D converter 5 Microphone 6 Departure station name input form 7 Arrival station name input form 8 Departure time input form 11 Input object form

Claims (11)

An information input method for inputting character information into a plurality of input items displayed on a display unit,
A receiving step for receiving audio;
A speech recognition step for recognizing character information from the speech using one or more of a plurality of grammar rules;
A selection step of selecting an input item for inputting the character information based on the additional information of the input item and the grammatical rule used in the speech recognition step;
The additional information, Ri information der showing the correspondence between the grammar rules and the input item,
The speech recognition step recognizes character information from the speech using a grammatical rule associated with an input item for which character information is not input .   The information input method according to claim 1, wherein when there are a plurality of input item candidates for inputting the character information, the user is notified of the candidates.   The information input method according to claim 2, wherein an input item for inputting the character information is selected from the candidates based on a user's voice.   4. The control according to claim 1, wherein when the input item selected in the selection step is outside the display screen, the input item is controlled to be displayed in the display screen. The information input method described.   5. The information input method according to claim 1, wherein each of the plurality of grammar rules is a grammar rule for recognizing a predetermined type of character information. An information input device for inputting character information to a plurality of input items displayed on a display,
Receiving means for receiving audio;
Speech recognition means for recognizing character information from the speech using one or more of a plurality of grammar rules;
Selection means for selecting an input item for inputting the character information based on the additional information of the input item and the grammatical rules used in the speech recognition means;
The additional information, Ri information der showing the correspondence between the grammar rules and the input item,
The speech recognition means recognizes character information from the speech using a grammatical rule associated with an input item for which character information is not input . The information input device according to claim 6 , wherein when there are a plurality of input item candidates for inputting the character information, the candidates are notified to the user. The information input device according to claim 7 , wherein an input item for inputting the character information is selected from the candidates based on a user's voice. When the input item selected in the selection means is outside the display screen, to any one of claims 6-8, wherein the input fields and controls to be displayed on the display screen The information input device described. Wherein the plurality of grammar rules Each of the information input device according to any one of claims 6-9, characterized in that a grammar rule for recognizing the predetermined type of character information. A computer-readable storage medium storing a program for causing a computer to execute the information input method according to any one of claims 1 to 5 .

Images