JP5318030B2 - Input support apparatus, extraction method, program, and information processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an input support device capable of displaying conversion candidates requested by a user, that is, improving the accuracy of predicted conversion even if a character string entered from a keyboard or the like is short. <P>SOLUTION: An input support device 100 for predicting input of text by a user and extracting conversion candidates comprises a voice recognition part 107 for performing voice recognition on speech waveforms that are based on input voice, a key input information acquisition part 111 for acquiring key input information corresponding to key operations done by the user, and a conversion candidate generation part 113 for, based on the result of the voice recognition by the voice recognition part 107 and the key input information, generating associated text as a conversion candidate. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to an input support apparatus, an extraction method, a program, and an information processing apparatus that predict text input by a user and extract conversion candidates.

2. Description of the Related Art Conventionally, an input support apparatus that displays conversion candidates input using a keyboard or the like is widely known as a method for inputting information to a personal computer, a mobile communication terminal, or the like.
For example, when a character is input using a keyboard and converted into a kanji-kana mixed character string, if there are a plurality of conversion candidates, an input support device that refers to input by voice and narrows down conversion candidates is known. (For example, patent document 1). According to such an input support apparatus, the efficiency at the time of specifying a conversion candidate can be improved.

JP-A-8-123481

  However, the above-described conventional input support apparatus has the following problems. That is, since the input support device described above displays the character string stored in the input support device as a conversion candidate based on the character string input to the keyboard or the like, if the input character string is short, the conversion candidate is There is a problem that many conversion candidates to be displayed cannot be narrowed down and conversion candidates requested by the user cannot be displayed sufficiently.

  Therefore, the present invention has been made in view of such a situation, and even when the character string input to the keyboard or the like is short, the conversion candidate requested by the user can be displayed. An object of the present invention is to provide an input support device, an extraction method, a program, and an information processing device that can improve accuracy.

  In order to solve the problems described above, the present invention has the following features. First, the first feature of the present invention is an input support device (input support device 100) that predicts text input by a user and extracts conversion candidates, and includes a speech waveform (speech waveform 131) based on the input speech. ) For voice recognition (voice recognition unit 107), acquisition means (key input information acquisition unit 111) for acquiring key input information (key input information 180) corresponding to the user's key operation, and the voice The gist of the invention is to provide conversion candidate creation means (conversion candidate creation unit 113) for creating a related text as a conversion candidate based on the result of speech recognition by the recognition means and the key input information.

  Such an input support apparatus includes a voice recognition unit that performs voice recognition on a voice waveform based on an input voice, an acquisition unit that acquires key input information corresponding to a user's key operation, and a conversion candidate creation unit. Prepare.

  The conversion candidate creation means creates related text as a conversion candidate based on the result of speech recognition by the speech recognition means and the key input information. That is, the conversion candidate creation means can display the conversion candidate requested by the user even when the character string input to the keyboard or the like is short. Therefore, it is possible to provide an input support apparatus that can improve the accuracy of predictive conversion.

  A second feature of the present invention relates to the first feature of the present invention, wherein the conversion candidate creation means is a speech conversion that is text corresponding to the speech waveform based on a result of speech recognition by the speech recognition means. A speech conversion candidate acquisition unit (speech conversion candidate acquisition unit 115) for acquiring a candidate (speech conversion candidate 140); and a limiting unit (limitation unit 117) for narrowing down the speech conversion candidates based on the key input information. This is the gist.

  A third feature of the present invention relates to the first or second feature of the present invention, and further includes a confirmation unit (a determination unit 127) for determining a word from the conversion candidate based on a user's designation. The gist of the recognition means is to perform voice recognition again for the voice waveform that has not been processed by the determination means.

  A fourth feature of the present invention relates to the third feature of the present invention, and is related word storage means (related word) for storing a word (basic word 191) and a related word (related word 193) related to the word. A storage unit 125) and a presentation unit (presentation unit 123) that preferentially presents the conversion candidate corresponding to the related word related to the word determined by the determination unit from the related word storage unit. Is the gist.

  A fifth feature of the present invention is an extraction method in which a computer predicts a text input by a user and extracts conversion candidates, and performs speech recognition on a speech waveform based on the input speech (step S120). A step of acquiring key input information corresponding to the user's key operation (step S130), and a step of creating a related text as a conversion candidate based on the result of the voice recognition and the key input information (step) S150).

  The sixth feature of the present invention is summarized as a program for causing a computer to execute the extraction method according to the fifth feature of the present invention.

  The gist of a seventh feature of the present invention is an information processing apparatus for distributing the program described in the sixth feature of the present invention.

  According to the present invention, even when the character string input to the keyboard or the like is short, the conversion candidate requested by the user can be displayed, that is, the input support apparatus, the extraction method, the program, and the like that can improve the accuracy of predictive conversion. An information processing apparatus can be provided.

It is a functional block diagram of the input assistance apparatus 100 which concerns on embodiment of this invention. It is a conceptual diagram for demonstrating operation | movement of the input assistance apparatus 100 which concerns on embodiment of this invention. It is a figure which shows an example of the information memorize | stored in the input assistance apparatus 100 which concerns on embodiment of this invention. It is a conceptual diagram for demonstrating operation | movement of the input assistance apparatus 100 which concerns on embodiment of this invention. It is a conceptual diagram for demonstrating operation | movement of the input assistance apparatus 100 which concerns on embodiment of this invention. It is a figure which shows an example of the information memorize | stored in the input assistance apparatus 100 which concerns on embodiment of this invention. It is a flowchart which shows the conversion candidate extraction operation | movement of the input assistance apparatus 100 which concerns on embodiment of this invention. It is a flowchart which shows the detail of the speech recognition process operation | movement of the input assistance apparatus 100 which concerns on embodiment of this invention. It is a flowchart which shows the determination processing operation | movement regarding the input assistance apparatus 100 which concerns on embodiment of this invention.

  Next, embodiments of the present invention will be described with reference to the drawings.

  In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. Further, the drawings are schematic, and the ratio of each dimension is different from the actual one.

  Therefore, specific dimensions and the like may be determined in consideration of the following description, and include portions where the relationship and ratio of the dimensions are different between the drawings.

  In the following embodiments, (1) the overall configuration of the input support apparatus 100, (2) the operation of the input support apparatus 100, (3) the operation and effect, and (4) other embodiments will be described.

(1) Overall Configuration of Input Support Device 100 FIG. 1 is a functional block diagram of the input support device 100 according to the present embodiment.

  The input support device 100 is a mobile communication terminal device (a mobile phone or a smartphone), and predicts a text input by a user and extracts a conversion candidate 150. In the following embodiment, the input support apparatus 100 describes, as a specific example, predicting an input of a sentence “company visit” based on an input of “K” or “ka” by a user. The predictive conversion function described in the following embodiments may be incorporated into a well-known predictive conversion function. Further, the input support apparatus 100 may have not only a predictive conversion function but also a normal kana-kanji conversion function.

  The input support device 100 includes a voice input device such as a microphone, a key input information input device such as a key button or a touch panel, and an output device such as a liquid crystal display, in addition to the following hardware resources. The input support device 100 is configured by a computer (single or plural) having hardware resources such as a control device such as a CPU, a storage device such as a memory and an HDD, and a communication device such as a communication I / F that communicates with a communication network. . The following blocks included in the input support apparatus 100 are realized by executing a computer program using hardware resources such as the computer control apparatus described above. These blocks and means do not have to be arranged on a single computer, and a configuration in which they are distributed as necessary may be adopted.

  As shown in FIG. 1, the input support apparatus 100 includes a control unit 101, a voice acquisition unit 103, a voice storage unit 105, a voice recognition unit 107, an acoustic model DB 108a, a language model DB 108b, a voice conversion candidate storage unit 109, key input information. An acquisition unit 111, a conversion candidate creation unit 113, a key input conversion candidate creation unit 119, a conversion candidate dictionary storage unit 121, a presentation unit 123, a related word storage unit 125, a determination unit 127, and an output unit 129 are provided. Further, the conversion candidate creation unit 113 includes a speech conversion candidate acquisition unit 115 and a limitation unit 117.

  These functional units are constituted by so-called computers, and are related to a CPU (Central Processing Unit) as an arithmetic / control device, a RAM (Random Access Memory) and a ROM (Read Only Memory) as a storage medium, a communication interface, and the like. It is realized with. Hereinafter, each functional unit will be described.

(1.1) Control unit 101
The control unit 101 is a control device such as a CPU that controls the input support device 100. The control unit 101 controls the operation of the functional unit shown below.

(1.2) Voice acquisition unit 103
The voice acquisition unit 103 acquires the input voice via a microphone or the like, and converts the voice into a voice waveform 131.

(1.3) Voice storage unit 105
The voice storage unit 105 stores the voice waveform 131 of the voice input by the voice acquisition unit 103.

(1.4) Voice recognition unit 107
FIG. 2 is a conceptual diagram for explaining the operation of the voice recognition unit 107 of the input support apparatus 100 according to the present embodiment. Specifically, FIG. 2A is a conceptual diagram of the speech waveform 131 converted by the speech recognition unit 107. FIG. 2B is a diagram illustrating an example of a phonetic symbol 133 that is replaced by the speech recognition unit 107 based on the speech waveform 131. FIG. 2C is a conceptual diagram of the speech conversion candidate 140 converted by the speech recognition unit 107 based on the phonetic symbol 133.

  As shown in FIG. 2, the voice recognition unit 107 performs voice recognition on a voice waveform 131 based on the input voice. The speech recognition unit 107 converts the speech waveform 131 into a phonetic symbol 133 using the acoustic model DB 108a, and converts the phonetic symbol 133 into a speech conversion candidate 140 made up of a word string using the language model DB 108b. Speech recognition by various recognition methods. Specifically, as illustrated in FIG. 2A, the speech recognition unit 107 analyzes the speech waveform 131 stored in the speech storage unit 105 and converts it into information obtained by extracting feature amounts. Furthermore, the speech recognition unit 107 analyzes the speech waveform 131 and converts the extracted feature quantity into information.

  Next, as shown in FIG. 2B, the speech recognition unit 107 replaces the speech waveform 131 with a phonetic symbol 133 based on the speech waveform 131 and the acoustic model DB 108a. Specifically, the speech recognition unit 107 divides the speech waveform 131 into speech sections that are words or phrases based on the feature amount of the speech waveform 131 and the like. Next, the speech recognition unit 107 extracts a phonetic symbol 133 corresponding to the speech waveform 131 from the acoustic model DB 108 a, thereby replacing a predetermined speech section of the speech waveform 131 with a plurality of phonetic symbols 133. Further, the speech recognition unit 107 assigns and stores probabilities based on the acoustic model DB 108a to the plurality of phonetic symbols 133.

  For example, based on the feature amount of the speech waveform 131, the speech recognition unit 107 uses the speech waveform 131 based on the acoustic model DB 108 a based on the phonetic symbol 133 such as “Kaishahoumon”, “Haishakoumon”, and “Aishohoumon”. Replace with probability.

  Finally, as shown in FIG. 2C, the speech recognition unit 107 performs speech recognition on the phonetic symbol 133 using a word stored in the language model DB 108b based on a probability statistical method. The language model DB 108b stores the speech conversion candidate 140 and the probability of the speech conversion candidate 140 for the phonetic symbol 133. For example, the speech recognition unit 107 sets “dentist”, “company”, “love car”, “start” for phonetic symbols 133 such as “Kaisha”, “Haishakou-mon”, “Aisho-ho-mon” based on the language model DB 108b. , Etc., and voice conversion candidates 143 such as “school gate”, “visit”, and “hormone”. That is, the speech conversion candidate 140 is text data corresponding to the speech waveform 131.

  In addition, after the speech conversion candidate 140 is confirmed by the confirming unit 127, the speech recognizing unit 107 performs speech recognition again on the speech waveform 131 that has not been processed by the confirming unit 127. In this case, the speech recognition unit 107 has a probability of selecting a word related to the speech conversion candidate 140 determined by the determination unit 127 (that is, a conversion candidate 150 described later) among the words stored in the language model DB 108b by the user. The speech conversion candidate 140 can be used as a high word.

(1.5) Acoustic model DB 108a
The acoustic model DB 108a stores phonetic symbols and a plurality of voice waveforms corresponding to the phonetic symbols or a voice waveform model that is a statistical voice waveform model of a voice waveform. For example, the acoustic model DB 108 a stores hiragana phonetic symbols from “a” to “n” and a plurality or statistical speech waveform models corresponding to each phonetic symbol.

(1.6) Language model DB 108b
The language model DB 108b includes a dictionary part and an n-gram. The dictionary part includes phonetic symbols and words corresponding to the phonetic symbols. The dictionary unit converts the appearance probability of each word into data according to the usage frequency. That is, the cumulative probability of words for a predetermined phonetic symbol in the dictionary part is 100%. For example, for the phonetic symbol “Kaisha”, the dictionary unit has a 70% probability of “company”, a 20% probability of “dentist”, and a 5% probability of “love car”. And the cumulative probability of the word for the phonetic symbol “Kaisha” is 100%.

  n-gram defines how words are arranged as a function of the grammar description part. The n-gram is created based on a so-called n-gram grammar consisting of a table showing the appearance frequency of N character strings or word combinations in a predetermined character string. That is, n-gram has a table indicating the appearance frequency of the nth word when the immediately preceding n-1 word is designated. Thus, for example, in 1-gram, “Cabinet” and “Inner corner” can be “Cabinet” rather than “Inner corner” when the previous word is a person name based on the text data before and after. If the previous word is “Catcher”, there is data that the probability of “inner angle” is high.

(1.7) Speech conversion candidate storage unit 109
FIG. 3 is a diagram illustrating an example of information stored in the speech conversion candidate storage unit 109 of the input support apparatus 100 according to the present embodiment.

  As illustrated in FIG. 3, the speech conversion candidate storage unit 109 stores an identification number 171, a speech waveform element 173, and a speech conversion candidate 140.

  The identification number 171 is a number assigned to each voice waveform element 173. The voice waveform element 173 is a part of the voice waveform 131 and is a waveform corresponding to the phonetic symbol 133 converted into the voice conversion candidate 140 by the voice recognition unit 107.

(1.8) Key input information acquisition unit 111
The key input information acquisition unit 111 acquires key input information 180 corresponding to the user's key operation. Specifically, the key input information acquisition unit 111 acquires key input information 180 input by the user in accordance with an operation of a keyboard, a touch panel, or the like. For example, the key input information acquisition unit 111 acquires key input information 180 such as “K” or “ka”.

(1.9) Conversion candidate creation unit 113
FIG. 4 is a conceptual diagram for explaining the operation of the voice recognition unit 107 of the input support apparatus 100 according to the present embodiment.

  As shown in FIG. 4, the conversion candidate creation unit 113 creates related text as conversion candidates 150 based on the result of speech recognition by the speech recognition unit 107 and key input information 180. Specifically, the conversion candidate creation unit 113 determines the presence / absence of a related speech conversion candidate 140 based on the result of speech recognition by the speech recognition unit 107 and the key input information 180, and the speech conversion candidate 140, Based on the key input information 180, related text is created as a conversion candidate 150. Also, the conversion candidate creation unit 113 creates the speech waveform 131 by connecting it to each phrase or word like the conversion candidate 151 and the conversion candidate 153.

  The conversion candidate creation unit 113 includes a speech conversion candidate acquisition unit 115 and a limitation unit 117. The speech conversion candidate acquisition unit 115 acquires the result of speech recognition by the speech recognition unit 107 by acquiring the speech conversion candidate 140 corresponding to the speech waveform 131 from the speech conversion candidate storage unit 109. For example, the speech conversion candidate acquisition unit 115 acquires the speech conversion candidate 141 and the speech conversion candidate 143 from the speech conversion candidate storage unit 109 as the speech conversion candidate 140 corresponding to the speech waveform 131.

  The limiting unit 117 creates related text as conversion candidates 150 by narrowing down the voice conversion candidates 140 based on the key input information 180. Specifically, the limiting unit 117 narrows down the speech conversion candidates 140 based on the key input information 180 and the first character of the phonetic symbol 133 of the speech conversion candidate 140. For example, the limiting unit 117 includes “K” and “ka” as the key input information 180 and “Haisha”, “Kaisha”, “Aisha”, and “Kaishi” as the first characters of the phonetic symbol 133 of the speech conversion candidate 141. The conversion candidate 151 is created by narrowing down the speech conversion candidates 141 to “Kaisha” and “Kaishi” whose first character of the phonetic symbol 133 is “Ka line”. Note that the limiting unit 117 narrows down the speech conversion candidates 141 to “Kaisha” and “Kaishi” whose first character of the phonetic symbol 133 is “ka” based on “ka” that is the key input information 180. Good.

  The restriction unit 117 narrows down the speech conversion candidates 143 based on the new key input information 180 after the conversion candidates 151 are determined for the speech conversion candidates 143 following the speech conversion candidates 141. Note that, when there is no speech conversion candidate 143 that matches the key input information 180, the limiting unit 117 determines that there is no corresponding speech conversion candidate 143. In this case, only the key input conversion candidate 160 created by the key input conversion candidate creation unit 119 described later is presented to the user via the presentation unit 123.

  Further, the limiting unit 117 can narrow down the voice conversion candidates 140 based not only on the key input information 180 consisting of a single character but also on the key input information 180 consisting of a plurality of characters. Specifically, the voice conversion candidates 141 can be further narrowed down based on new key input information 180 following the top key input information 180. The narrowing down of the limiting unit 117 when the speech conversion candidates 143 are “Koumon”, “Houmon”, and “Horumon” will be described below.

  For example, when the key input information 180 is “H” and the speech conversion candidates 143 are “Koumon”, “Houmon”, and “Hormon”, the limiting unit 117 includes the key input information 180 and the speech conversion candidates 143. Based on the above, the speech conversion candidates 143 are narrowed down to “Homomon” and “Holmon”. Here, when “O” and “U” are continuously input as the new key input information 180, the limiting unit 117 sets the speech conversion candidate 143 to “Honmon” based on the new key input information 180. Refine to.

(1.10) Key input conversion candidate creation unit 119
The key input conversion candidate creation unit 119 creates a key input conversion candidate 160 based on the key input information 180 corresponding to the user's key operation. Specifically, the key input conversion candidate creation unit 119 acquires the key input information 180 input by the user from the key input information acquisition unit 111 in accordance with the operation of the keyboard, the touch panel, the mouse, and the like. The key input conversion candidate 160 corresponding to is extracted from the conversion candidate dictionary storage unit 121.

  For example, the key input conversion candidate creation unit 119 acquires key input information 180 such as “K” or “ka”, and “return” or “mosquito” whose first character corresponding to the key input information 180 is “ka line”. ”,“ Yesterday ”,“ Health ”and the like are extracted from the conversion candidate dictionary storage unit 121. The key input information acquisition unit 111 sets the key input conversion candidate 160 to “return” or “mosquito” whose first character of the phonetic symbol 133 is “ka” based on “ka” as the key input information 180. You may narrow down.

(1.11) Conversion candidate dictionary storage unit 121
The conversion candidate dictionary storage unit 121 stores phonetic symbols and key input conversion candidates 160 corresponding to the phonetic symbols. The phonetic symbols and key input conversion candidates 160 are registered in advance as a dictionary. The phonetic symbols and key input conversion candidates 160 are registered based on the history of words used by the user. For example, the conversion candidate dictionary storage unit 121 stores the word determined by the determination unit 127 as the key input conversion candidate 160 in association with the pronunciation symbol of the word determined by the determination unit 127.

(1.12) Presentation unit 123
FIG. 5 is a conceptual diagram for explaining the operation of the presentation unit 123 of the input support apparatus 100 according to the present embodiment. Specifically, FIG. 5A is a diagram illustrating an example of the conversion candidate 150 displayed on the presentation unit 123. FIG. 5B is a conceptual diagram illustrating the conversion candidate 151 determined by the determination unit 127 and the operation of the presentation unit 123 accompanying the determination of the conversion candidate 151.

  As shown in FIG. 5, the presentation unit 123 presents at least one of the corresponding conversion candidate 150 and key input conversion candidate 160 for the speech waveform 131 or the key input information 180. Specifically, the presentation unit 123 presents the conversion candidate 150 corresponding to the voice waveform 131 or the key input information 180 via an output device such as a liquid crystal display. For example, as illustrated in FIG. 5A, the presentation unit 123 presents conversion candidates 151 and conversion candidates 153 corresponding to the speech waveform 131 or the key input information 180. The presentation unit 123 illustrated in FIG. 5A presents a so-called lattice expression that presents a plurality of conversion candidates 151 and a plurality of conversion candidates 153. For the conversion candidate 151 and the conversion candidate 153, the presenting unit 123 is based on only the words that have a high probability of being selected by the user based on the language model DB 108b or the related word storage unit 125 described later, that is, based on the n-best sentence. Each word may be presented.

  Further, as shown in FIGS. 5B and 6, the presentation unit 123 corresponds to the related word 193 related to the basic word 191 with the conversion candidate 151 a that is the word determined by the determination unit 127 as the basic word 191. The conversion candidates are preferentially presented as conversion candidates 153a. For example, when the user selects “company” which is the conversion candidate 151a, the presentation unit 123 presents “visit” which is a related word 193 related to “company” of the basic word 191 to a higher rank than “school gate”. By doing so, it is preferentially presented as the conversion candidate 153a.

(1.13) Related word storage unit 125
FIG. 6 is a diagram illustrating an example of information stored in the related word storage unit 125 of the input support apparatus 100 according to the present embodiment. As shown in FIG. 6, the related word storage unit 125 stores basic words 191 and related words 193 related to the basic words 191.

  The basic word 191 is registered based on a history of words used by the user. For example, the basic word 191 is a word stored in the language model DB 108b. The basic word 191 is sequentially registered based on the input from the user.

  The related word 193 is a word related to the basic word 191. For example, the related word 193 is a word stored in the language model of the voice storage unit 105 in association with the basic word 191. The related word 193 is sequentially stored in association with the basic word 191 based on the input from the user.

  The related word storage unit 125 may be the n-gram itself of the language model DB 108b.

(1.14) Determination unit 127
The determination unit 127 determines a word from the conversion candidate 150 based on the user's designation. Specifically, the determination unit 127 determines a word from the conversion candidate 150 based on user instruction information from a key input information input device such as a key button or a touch panel, or an input device such as a mouse. For example, as illustrated in FIGS. 5A and 5B, the determination unit 127 determines “company” from the conversion candidate 151 as the conversion candidate 151 a based on the designation of the user.

(1.15) Output unit 129
The output unit 129 outputs the conversion candidate 151a determined by the determination unit 127 to the user. Specifically, the output unit 129 outputs the conversion candidate 151a via an output device such as a liquid crystal display. The output unit 129 can also output the conversion candidate 151a as data via a network or the like.

(2) Operation of Input Support Device 100 Regarding the operation of the input support device 100, (2.1) Overall schematic operation, (2.2) Speech recognition processing operation of the speech recognition unit 107, (2.3) Conversion candidate creation unit A description will be given of the order of determination processing operations relating to the voice conversion candidates 113.

(2.1) Overall Schematic Operation FIG. 7 is a flowchart showing the conversion candidate extraction operation of the input support apparatus 100 according to the embodiment.

  As shown in FIG. 7, in step S <b> 100, the control unit 101 determines whether the input from the user is based on voice or key input information 180. Specifically, if the input from the user is based on voice, control unit 101 advances the process to step S110. If the input from the user is based on the key input information 180, the control unit 101 advances the process to step S130.

  In step S <b> 110, the voice acquisition unit 103 acquires the input voice via a microphone or the like, and converts the voice into a voice waveform 131. The voice storage unit 105 stores the voice waveform 131 of the voice input by the voice acquisition unit 103.

  In step S120, the speech recognition unit 107 performs speech recognition on the speech waveform 131 stored in the speech storage unit 105 using the acoustic model DB 108a and the language model DB 108b, and creates a speech conversion candidate 140. Further, the speech conversion candidate storage unit 109 stores a speech conversion candidate 140, an identification number 171 associated with the speech conversion candidate 140, and a speech waveform element 173.

  In step S130, the key input information acquisition unit 111 acquires key input information 180 corresponding to the user's key operation.

  In step S <b> 140, the conversion candidate creation unit 113 determines whether or not there is a related speech conversion candidate 140 based on the result of speech recognition by the speech recognition unit 107 and the key input information 180.

  Specifically, the speech conversion candidate acquisition unit 115 of the conversion candidate creation unit 113 acquires the speech conversion candidate 140 corresponding to the speech waveform 131 from the speech conversion candidate storage unit 109, thereby performing speech recognition by the speech recognition unit 107. Get the result.

  Further, the limiting unit 117 of the conversion candidate creation unit 113 narrows down the voice conversion candidates 140 based on the key input information 180, thereby determining the presence or absence of the voice conversion candidates 140 based on the key input information 180.

  Specifically, when it is determined that there is a speech conversion candidate 140 corresponding to the key input information 180, the limiting unit 117 advances the processing to step S150. If the limiting unit 117 determines that there is no voice conversion candidate 140 corresponding to the key input information 180, the limiting unit 117 performs voice recognition by the voice recognition unit 107 again by returning the process to step S120.

  In step S <b> 150, the conversion candidate creation unit 113 creates related text as the conversion candidate 150 based on the result of speech recognition by the speech recognition unit 107 and the key input information 180. Specifically, the limiting unit 117 of the conversion candidate creation unit 113 creates related text as the conversion candidates 150 by narrowing down the speech conversion candidates 140 based on the key input information 180.

  In step S <b> 160, the presentation unit 123 presents the corresponding conversion candidate 150 for the speech waveform 131. The presentation unit 123 may present the corresponding key input conversion candidate 160 for the speech waveform 131 in response to the processing of the key input conversion candidate creation unit 119 in step S210.

  In step S170, the confirmation unit 127 confirms whether or not the word is confirmed from the conversion candidates 150 based on the user's designation. Specifically, when the determination unit 127 determines a word from the conversion candidate 150 based on the user's specification, the process proceeds to step S180. On the other hand, if the word is not confirmed from the conversion candidate 150 based on the user's designation, the determination unit 127 advances the process to step S200.

  In step S180, the output unit 129 outputs the conversion candidate 151a determined by the determination unit 127 to the user.

  In step S190, the control unit 101 determines whether or not there is a phrase next to the confirmed conversion candidate 151a or a conversion candidate 150 for the next word for the speech waveform 131. For example, for the speech waveform 131, the control unit 101 confirms the conversion candidate 153 as the next phrase or the next word of the confirmed conversion candidate 151a, and proceeds to the next process with the conversion candidate 153 present.

  Specifically, if the control unit 101 determines that the next phrase of the confirmed conversion candidate 151a or the next word conversion candidate 150 is present for the speech waveform 131, the process returns to step S170 to confirm. The next conversion candidate 150 of the conversion candidate 151a is processed. When the control unit 101 determines that the next phrase of the confirmed conversion candidate 151a or the conversion candidate 150 of the next word is absent for the speech waveform 131, the conversion candidate extraction process ends.

  In step S200, the key input information acquisition unit 111 confirms input of new key input information 180 corresponding to the user's key operation, and advances the process to step S210.

  In step S210, the key input conversion candidate creation unit 119 creates a key input conversion candidate 160 based on the key input information 180 corresponding to the user's key operation.

(2.2) Voice Recognition Processing Operation of the Voice Recognition Unit 107 FIG. 8 is a flowchart showing details of the voice recognition processing operation of the voice recognition unit 107 according to the embodiment. Specifically, FIG. 8 shows a detailed operation of processing in which the speech recognition unit 107 in step S120 of FIG. 7 performs speech recognition.

  In step S <b> 121, the speech recognition unit 107 analyzes the speech waveform 131 stored in the speech storage unit 105 and converts it into information obtained by extracting feature amounts. Further, the speech recognition unit 107 replaces the speech waveform 131 with a phonetic symbol 133 based on the speech waveform 131 and the acoustic model DB 108a.

  In step 123, the speech recognition unit 107 creates a speech conversion candidate 140 for the phonetic symbol 133 based on the language model DB 108b.

  In step 125, the speech conversion candidate storage unit 109 stores the identification number 171, the speech waveform element 173, and the speech conversion candidate 140 in association with each other.

  Thereby, the voice recognition unit 107 ends the voice recognition processing operation.

(2.3) Determination Processing Operation Related to Speech Conversion Candidate 140 by Conversion Candidate Creation Unit 113 FIG. 9 is a flowchart showing a determination processing operation related to speech conversion candidate 140 by the conversion candidate creation unit 113 according to the embodiment. Specifically, FIG. 9 shows a detailed operation of a process in which the conversion candidate creation unit 113 in step S140 of FIG.

  In step S 141, the speech conversion candidate acquisition unit 115 of the conversion candidate creation unit 113 acquires the speech conversion candidate 140 corresponding to the speech waveform 131 from the speech conversion candidate storage unit 109, so that the speech recognition result by the speech recognition unit 107 is obtained. To get.

  In step S <b> 143, the limiting unit 117 of the conversion candidate creation unit 113 narrows down the speech conversion candidates 140 based on the key input information 180, thereby determining the presence / absence of the speech conversion candidates 140 based on the key input information 180.

  Thereby, the conversion candidate creation part 113 complete | finishes the determination processing operation regarding a speech conversion candidate.

(3) Action / Effect As described above, according to the present embodiment, the input support apparatus 100 uses the voice recognition unit 107 that performs voice recognition on the voice waveform 131 based on the input voice, and the user's key operation. The key input information acquisition unit 111 that acquires the key input information 180 corresponding to, and the conversion candidate creation unit 113 are provided.

  The conversion candidate creation unit 113 creates related text as the conversion candidate 150 based on the result of speech recognition by the speech recognition unit 107 and the key input information 180. That is, the conversion candidate creation unit 113 can display the conversion candidate 150 requested by the user even when the character string input to the keyboard or the like is short. Therefore, the input support apparatus 100 that can improve the accuracy of predictive conversion can be provided.

  As described above, the input support device 100 is preferably used in such a manner that a sentence to be input is first input over a single sentence or the entire sentence, and a word is determined from the head of the sentence based on a user's key operation. In particular, the input support device 100 can be suitably used by being applied to a mobile communication terminal because it can reduce user key operations.

  In the embodiment, the conversion candidate creation unit 113 includes a speech conversion candidate acquisition unit 115 that acquires a speech conversion candidate 140 that is text corresponding to the speech waveform 131 based on the result of speech recognition by the speech recognition unit 107, and a key input. And a limiting unit 117 that narrows down the voice conversion candidates 140 based on the information 180.

  For this reason, the conversion candidate creation unit 113 can narrow down the speech conversion candidates 140 recognized by the speech recognition unit 107 based on the key input information 180. That is, the conversion candidate creation unit 113 can further improve the accuracy of predictive conversion by using the speech waveform 131 and the key input information 180.

  In the embodiment, the input support apparatus 100 further includes a determination unit 127 that determines a word from the conversion candidate 150 based on a user designation. The voice recognition unit 107 performs voice recognition again on the voice waveform 131 that has not been processed by the determination unit 127.

  For this reason, the speech recognition unit 107 can perform speech recognition again based on the conversion candidate 150 in which the word is confirmed by the confirmation unit 127. That is, since the speech recognition unit 107 performs speech recognition again using the confirmed word information, it is possible to further improve the accuracy of predictive conversion by displaying words that are likely to be linked to the confirmed word as candidates. .

  In the embodiment, the input support apparatus 100 includes a related word storage unit 125 that stores a word and a related word 193 related to the word, and a related item that is related to the word determined by the determining unit 127 from the related word storage unit 125. And a presentation unit 123 that preferentially presents conversion candidates 150 corresponding to the word storage unit 125. For this reason, the presentation unit 123 can preferentially present the related word 193 based on the related word storage unit 125. Therefore, it is possible to provide the input support device 100 that can more effectively present the word requested by the user.

(4) Other Embodiments In the embodiment described above, the speech recognition unit 107 converts the speech waveform 131 into the phonetic symbol 133 using the acoustic model DB 108a, and converts the phonetic symbol 133 from the word string using the language model DB 108b. The speech recognition is performed by a so-called bottom-up recognition method in which the speech conversion candidate 140 is converted.

  The present invention is not limited to this, and the speech recognition unit 107 assumes a speech conversion candidate 140 using the language model DB 108b, converts it into a phonetic symbol 133 using the dictionary unit of the language model DB 108b, and uses the acoustic model DB 108a. Thus, speech recognition may be performed by a so-called top-down recognition method in which the phonetic symbol 133 and the corresponding speech waveform 131 are collated. In this case, in the process of performing speech recognition shown in FIG. 8, the speech recognition unit 107 performs steps S <b> 121 and S <b> 123 simultaneously to create a speech conversion candidate 140 from the speech waveform 131.

  In the embodiment described above, the input support device 100 includes a voice input device such as a microphone and a key input information input device such as a key button or a touch panel, and predicts text input by the user and extracts the conversion candidates 150.

  The present invention is not limited to this, and voice and key input information may be input from the client terminal to the input support apparatus 100 via a communication network or the like. In this case, the input support apparatus 100 can present the conversion candidate 150 to the user at the client terminal by transmitting the conversion candidate 150 to the original client terminal via a network or the like.

  In the embodiment described above, the input support device 100 is exemplified as a mobile communication terminal device (a mobile phone or a smartphone). The present invention is not limited to this, and the input support apparatus 100 may be a PC (Personal Computer) or a car navigation system used by the user.

  In the embodiment described above, each block included in the input support apparatus 100 is described. However, the present invention is not limited to this, and for example, a computer program that executes the function of each block described above may be used. May be an information processing device that distributes.

  In the embodiment described above, the speech recognition unit 107 replaces the speech waveform 131 with the phonetic symbol 133 based on the speech waveform 131 and the acoustic model DB 108a, and uses the words stored in the language model DB 108b to use the speech conversion candidates. 140 is created.

  The present invention is not limited to this. For example, the speech recognition unit 107 performs speech recognition on the speech waveform 131 using the words stored in the conversion candidate dictionary storage unit 121 and corresponding to the key input information 180. Also good. In this case, the conversion candidate dictionary storage unit 121 stores the word determined by the determination unit 127 as the key input conversion candidate 160 in association with the pronunciation symbol of the word determined by the determination unit 127. According to this, the speech recognition unit 107 can create the word corresponding to the key input information 180 as the speech conversion candidate 140 while confirming by the confirmation unit 127. That is, the voice recognition unit 107 can create a word corresponding to the key input information 180 as the voice conversion candidate 140 while being frequently used by the user. Accordingly, the voice recognition unit 107 can create a word that is intended to be created by the user as the voice conversion candidate 140.

  As described above, the present invention naturally includes various embodiments that are not described herein. Therefore, the technical scope of the present invention is defined only by the invention specifying matters according to the scope of claims reasonable from the above description.

  For example, each configuration diagram, flowchart, conceptual diagram, and the like are merely examples, and the presence / absence of each element, its order, specific contents, and the like can be changed as appropriate. As an example, manual block designation is not indispensable, and in the present invention, elements such as each means are not limited to an arithmetic control unit of a computer but may be realized by an electronic circuit based on wired logic or the like.

DESCRIPTION OF SYMBOLS 100 Input support apparatus 101 Control part 103 Voice acquisition part 105 Voice storage part 107 Voice recognition part 108a Acoustic model DB
108b Language Model DB
109 Speech conversion candidate storage unit 111 Key input information acquisition unit 113 Conversion candidate creation unit 115 Speech conversion candidate acquisition unit 117 Limit unit 119 Key input conversion candidate creation unit 121 Conversion candidate dictionary storage unit 123 Presentation unit 125 Related word storage unit 127 Determination unit 129 Output unit 131 Speech waveform 133 Phonetic symbol 140, 141, 143 Speech conversion candidate 150, 151, 151a, 153, 153a Conversion candidate 160 Key input conversion candidate 171 Identification number 173 Speech waveform element 180 Key input information 191 Basic word 193 Related word

Claims (6)

An input support device that predicts text input by a user and extracts conversion candidates,
Speech recognition means for performing speech recognition on a speech waveform based on the input speech;
Obtaining means for obtaining key input information corresponding to a user's key operation;
Based on the result of speech recognition by the speech recognition means , a speech conversion candidate that is text corresponding to the speech waveform is acquired, and among the acquired speech conversion candidates, a speech conversion candidate that matches the key input information is selected. If there is a voice conversion candidate that matches the key input information, the voice conversion candidate is presented, and if there is no voice conversion candidate that matches the key input information, only the key input information is used. Voice conversion candidate acquisition means for presenting key input conversion candidates created by
An input support apparatus comprising: Further comprising a confirming means for confirming a word from the conversion candidate based on a user designation;
The voice recognition means
The input support apparatus according to claim 1, wherein speech recognition is performed again on the speech waveform that has not been processed by the determination unit. Related word storage means for storing a word and a related word related to the word;
The input according to claim 2, further comprising: a presentation unit that preferentially presents the conversion candidate corresponding to the related word related to the word confirmed by the confirmation unit from the related word storage unit. Support device. An extraction method in which a computer predicts input of text by a user and extracts conversion candidates,
Performing speech recognition on a speech waveform based on the input speech;
Obtaining key input information corresponding to a user's key operation;
Based on the result of the speech recognition, obtain speech conversion candidates that are text corresponding to the speech waveform, narrow down speech conversion candidates that match the key input information from the obtained speech conversion candidates, If there is a voice conversion candidate that matches the input information, the voice conversion candidate is presented. If there is no voice conversion candidate that matches the key input information, the voice conversion candidate is created based only on the key input information. Presenting key input conversion candidates ;
The extraction method characterized by including.   A program for causing a computer to execute the extraction method according to claim 4.   An information processing apparatus for distributing the program according to claim 5.

Images