JP2005134442A - Speech recognition device and method, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To speed up the processing while preventing accuracy of speech recognition from deteriorating. <P>SOLUTION: A word preliminary selection part 56 selects one or more words following the pre-acquired words of a word string as a candidate of a speech recognition result; a word candidate clustering part 57 classifies the candidate words into the same sound words; a matching part 58 calculates a sound score about a word candidate which has a highest initial value of the score among the classified word candidates; and the sound score substitutes for the sound score of another word candidate included in the same word candidate set. Thereafter, a re-evaluation part 59 sequentially corrects word connecting relations between words in the word string as a candidate of the speech recognition result. For example, this invention is applicable to a speech dialog system. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a speech recognition apparatus and method, a recording medium, and a program, and more particularly, a speech recognition apparatus and method, a recording medium, and a recording medium capable of speeding up processing while preventing deterioration in accuracy of speech recognition. Regarding the program.

  In recent years, products and services using voice recognition have been actively put into practical use.

  Speech recognition is a technology that automatically determines a word sequence corresponding to an input speech signal. By combining this technology with an application, various products and services are possible.

  A basic configuration example of the speech recognition apparatus is shown in FIG. In FIG. 1, a voice uttered by a user is captured by a microphone (microphone) 1, sampled by an AD (Analog to Digital) conversion unit 2, and converted into digital data. The feature extraction unit 3 performs processing such as frequency analysis on the digital data output from the AD conversion unit 2 at appropriate time intervals, and converts the digital data into parameters representing the acoustic characteristics of the spectrum and other sounds. As a result, a time series of audio feature values is obtained. This parameter is transmitted from the feature extraction unit 3 to the matching unit 4. The matching unit 4 uses the data stored in the acoustic model database (DB) 5, the dictionary database (DB) 6, and the grammar database (DB) 7 to use the feature amount series supplied from the feature extraction unit 3. Is matched, and the speech recognition result is output.

  The acoustic model database 5 is a model that retains acoustic features such as individual phonemes and syllables used in a target language, and a hidden Markov model (HMM) or the like is used. The dictionary database 6 holds information related to pronunciation of individual words to be recognized, thereby associating the words with the acoustic model described above. As a result, an acoustic standard pattern corresponding to each word included in the dictionary database 6 is obtained. The grammar database 7 describes how individual words described in the dictionary database 6 can be chained. The grammar database 7 includes a description based on a regular grammar and a context-free grammar, and a statistical word chain probability. (N-gram) is used.

  The matching unit 4 uses the acoustic model database 5, the dictionary database 6, and the grammar database 7 to determine a word sequence that most closely matches the feature amount sequence input from the feature extraction unit 3. For example, when a hidden Markov model (HMM) is used as the acoustic model, a value obtained by accumulating the appearance probability of each feature amount according to the feature amount series is used as an acoustic evaluation value (hereinafter referred to as an acoustic score). The acoustic score is obtained corresponding to each word by using the standard pattern described above.

  For example, when a bigram is used as a grammar, the linguistic accuracy of each word based on the chain probability with the immediately preceding word is quantified, and the value is a linguistic evaluation value (hereinafter referred to as a language score). As given.

  Then, by comprehensively evaluating the acoustic score and the language score, the word sequence that best matches the input speech signal is determined.

  As an example, let us consider a case where a user utters “It is a nice weather today”. In this case, for example, a series of words such as “today”, “ha”, “good”, “weather”, and “sound” is obtained as a recognition result. At this time, an acoustic score and a language score are given to each word.

  Various methods for performing such speech recognition processing have been proposed. Generally, speech recognition processing requires a large amount of calculation and storage capacity, and in order to speed up speech recognition processing, There is a demand for a device having a higher-speed calculation processing capacity and a large capacity memory (storage means). Therefore, a technique has been proposed for speeding up the speech recognition processing while keeping the amount of calculation and storage capacity required for the speech recognition processing low.

  For example, the applicant narrows down the candidate of the next word following the word already obtained as a speech recognition result by performing a preliminary selection, and executes a matching process for the word candidate narrowed down by the word preliminary selection, A word string that is a candidate for the speech recognition result is identified, and then the word connection relationship between the words in the word sequence that is the candidate for the speech recognition result is corrected, and the speech recognition result is obtained based on the corrected word connection relationship. A method of sequentially determining a word string hypothesis corresponding to an input speech by determining a word string has been proposed (see, for example, Patent Document 1). With this method, the speech recognition process can be executed more efficiently.

  In addition, a bundle search method has been proposed as one method for speeding up speech recognition processing (see, for example, Non-Patent Document 1). An outline of the bundle search method proposed in Non-Patent Document 1 will be described with reference to FIG. In FIG. 2, A, B, and C represent word candidates. In FIG. 2, a circle represents a node, and a line segment connecting the circles represents an arc. In FIG. 2, the direction from left to right represents the passage of time.

  Further, the cumulative score up to word A is set as score S1, and the cumulative score up to word B is set as score S2.

  In the case of the example in FIG. 2, the same word C appears at different positions in the grammar. That is, the word C appears also at the position next to the word A, and also appears at the position next to the word B. Node 2 and node 4 are located at the same time t1.

  In the conventional speech recognition process that does not use the bundle search method, the score of the word C following the word A and the score of the word C following the word B are calculated independently. On the other hand, in the speech recognition process using the bundle search method, for example, when the score S2 of the word C following the word A is calculated by performing the matching process using the score S1 as an initial value, the difference in scores (Δ = By using S2-S1) and estimating the score T2 of the word C following the word B as T2 = T1 + Δ, the matching process for the same word candidate C is completed once. Thereby, the original score calculation of the word C following the word B can be omitted, and as a result, the speed of the speech recognition process is increased.

In this way, the bundle search method targets speech recognition processing based on frame-synchronized matching processing, and the same word is used for the problem that matching processing with words at different positions in the grammar is performed separately. Measures are taken so that the matching process is performed once.
JP 2001-242848 A "Acceleration of continuous speech recognition using bundle search method" IEICE Transactions, November 1992, D-22, Vol.J75-D-II, No.11, pp.1761-1769

  However, it is difficult to increase the processing speed without lowering the accuracy of the speech recognition with the conventional technology that tries to speed up the speech recognition processing while keeping the calculation amount and the storage capacity required for the speech recognition processing low. There was a problem of being.

  For example, in the method described in Patent Document 1, since the subsequent matching process is performed on all word candidates determined by the word preliminary selection, there is a problem that the amount of matching processing becomes relatively large. there were. In addition, when the acoustic score or language score determined by the matching process depends on the immediately preceding word, etc., in order to perform the matching process strictly, the matching process starting at the same time It is necessary to perform matching processing as many as the number, and this is also a factor for increasing the amount of matching processing.

  For example, FIG. 3 and FIG. 4 show examples of word connection relation information when recognizing a voice “open a window”. In FIGS. 3 and 4, a circle represents a node, and a line segment connecting the circles represents an arc. In FIGS. 3 and 4, the direction from left to right represents the passage of time. In FIG. 3, the matching process for the input speech proceeds until time t 1, and the word string partial hypothesis “window” through node 1, node 2, and node 3, and the word string partial hypothesis “door” through node 1 and node 4. Is required. Here, it is assumed that four word candidates “open”, “open”, “open”, and “close” starting from time t1 are obtained by the word preliminary selection process. FIG. 4 shows the word connection relation information at this time. As shown in FIG. 4, matching processing is performed for each of the word candidates “open”, “open”, “dawn”, and “close”, and four matching processes are required.

  That is, in FIG. 4, arcs are extended from the node 3 at the end of the word string partial hypothesis “window” to each of the nodes 5 to 8. The arc extending from node 3 to node 5 corresponds to the word candidate “open”, the arc extending from node 3 to node 6 corresponds to the word candidate “open”, and the arc extending from node 3 to node 7 is the word candidate The arc extending from the node 3 to the node 8 corresponding to “open” corresponds to the word candidate “close”. A matching process is performed on each of the four word candidates, and respective scores are obtained.

  Also, when applying an acoustic model (eg crossword triphone) or language model (bigram) that depends on the word connected before, the two word string partial hypotheses “window” and “door” connected before It is necessary to perform matching processing for each. Therefore, as shown in FIG. 4, in addition to the matching process for the word string partial hypothesis “window”, it is necessary to execute the matching process for the same word candidate also in the word string partial hypothesis “door”. .

  That is, in FIG. 4, arcs are extended from the node 4 at the end of the word string partial hypothesis “door” to each of the nodes 9 to 12. An arc extending from node 4 to node 9 corresponds to the word candidate “open”, an arc extending from node 4 to node 10 corresponds to the word candidate “open”, and an arc extending from node 4 to node 11 corresponds to the word candidate The arc extending from the node 4 to the node 12 corresponding to “open” corresponds to the word candidate “close”. A matching process is performed on each of the four word candidates, and respective scores are obtained.

  As a result, the number of word candidates (in the case of FIG. 4, “open”, “empty”, “dawn”, and “close”) is added to the number of immediately preceding word string partial hypotheses (in FIG. 4, “window” The matching process needs to be executed as many times as the number obtained by multiplying (“2” and “door”) (in the case of FIG. 4, 4 × 2 = 8 times), and the amount of calculation increases.

  On the other hand, the bundle search method described in Non-Patent Document 1 has a problem that the matching process is not a process based on strict dynamic programming. That is, for example, in FIG. 2, when the matching process related to the same word C of interest is performed using a crossword triphone based on dynamic programming, even if the same word is used, the word connected before in the grammar The acoustic model of the focused word C changes according to (Word A and Word B). That is, the acoustic model of the word C when following the word A in FIG. 2 and the acoustic model of the word C when following the word B are different.

  However, when the above-described score estimation method based on the bundle search method is applied, the acoustic model of the word C following the word B is substantially substituted with the acoustic model of the word C following the word A. Therefore, an accurate acoustic score cannot be obtained. Furthermore, when the matching process related to the same word C of interest is performed based on dynamic programming, the transition time to the word of interest C changes, but if the score estimation method by the bundle search method is applied, the word of interest There is a problem that the transition time t1 to C is fixed between the same words.

  In the bundle search method, as a result, the estimated score value T2 = T1 + Δ may differ from the exact score value. This causes a deterioration in recognition accuracy.

  The speech recognition apparatus of the present invention shares a part of the processing for each word group in which one or more word candidates following the already obtained word in the word string are classified for each word having the same attribute. The word connection relationship information indicating the connection relationship between the words in the word string that is a candidate for the speech recognition result, generated based on the result of the matching process performed by the matching processing execution unit And a correcting means.

  A classifying unit that classifies one or more word candidates following the already obtained word in the word string as the word group for each word having the same attribute; The processing execution means can share a part of the processing for each word group classified by the classification means.

  A selection means for selecting one or more word candidates following the already obtained word from the word string is further provided, and the classification means includes the one or more words selected by the selection means. Can be classified as the word group for each word having the same attribute.

  One or more candidate words following the already obtained word can be classified into the word group for each word having the same pronunciation.

  One or more word candidates following the already-sought word can be classified into the word group for each of the acoustically similar words.

  Storage means for storing the word connection relation information generated based on the result of the matching process by the matching process execution means is further provided, and the correction means includes the word connection relation information stored by the storage means. Can be fixed.

  When a state transition model for calculating an acoustic score of a partial word string is constructed by the modifying means to modify the word connection relation information, a memory for storing the constructed state transition model Means may be further provided, and the correction means may use the state transition model stored by the storage means when recalculating the acoustic score of the same partial word string.

  When a state transition model for calculating an acoustic score of a word is constructed by the matching process execution means, a storage means for storing the constructed state transition model is further provided, and the matching process execution means In the case where the acoustic score of the same word is calculated again, the state transition model stored in the storage unit can be used.

  A storage means for storing the value as the progress of the matching process calculated by the matching process execution means is further provided, and the value as the progress of the process stored by the storage means is stored in the matching process execution means. The matching processing of the plurality of word strings that are candidates for the speech recognition result can be alternately executed.

  In the speech recognition method of the present invention, a part of the processing is shared for each word group in which one or more word candidates following the already obtained word in the word string are classified for each word having the same attribute. Connection processing information indicating a connection relationship between words in a word sequence that is a candidate for a speech recognition result, generated based on a matching processing execution step in which the matching processing is performed and a matching processing result by the processing of the matching processing execution step And a correction step for correcting.

  The program of the recording medium of the present invention performs a part of the processing for each word group in which one or more word candidates following the already obtained word in the word string are classified for each word having the same attribute. A matching process execution step for sharing and matching processing, and a word connection relation that indicates a connection relation between words in a word sequence that is a candidate for a speech recognition result generated based on the result of the matching process by the process of the matching process execution step And a correction step for correcting the information.

  The program of the present invention allows a computer that controls processing for determining a word string corresponding to an input speech to select one or more word candidates that follow the already obtained word from the word string and assign the same attribute. For each word group classified for each word it has, a matching process execution step that shares a part of the process and performs a matching process, and a voice recognition result generated based on the result of the matching process by the process of the matching process execution step A correction step of correcting word connection relation information indicating connection relations between words in a candidate word string is executed.

  In the speech recognition apparatus and method, the recording medium, and the program according to the present invention, a word group in which one or more word candidates subsequent to the already obtained word in the word string are classified for each word having the same attribute. Each time, a part of the process is shared and the matching process is performed, and the word connection relation information indicating the connection relation between the words in the word string that is a candidate of the speech recognition result generated based on the result of the matching process is corrected. The

  The present invention can be applied to, for example, a voice interaction system when searching a database by voice, operating various devices, or inputting data to each device. More specifically, for example, a database search device that displays map information corresponding to a place name inquiry by voice, an industrial robot that sorts luggage for voice instructions, a voice instead of a keyboard The present invention can be applied to a dictation system that creates text by input, a dialog system in a robot that performs conversation with a user, and the like.

  According to the present invention, voice recognition processing can be executed. In particular, it is possible to increase the speed of speech recognition processing while suppressing an increase in calculation amount and storage capacity required for processing and maintaining accuracy of speech recognition.

  BEST MODE FOR CARRYING OUT THE INVENTION The best mode of the present invention will be described below. The correspondence relationship between the disclosed invention and the embodiments is exemplified as follows. Although there is an embodiment which is described in the specification but is not described here as corresponding to the invention, it means that the embodiment corresponds to the invention. It doesn't mean not. Conversely, even if an embodiment is described herein as corresponding to an invention, that means that the embodiment does not correspond to an invention other than the invention. Absent.

  Further, this description does not mean all the inventions described in the specification. In other words, this description is an invention described in the specification and is not claimed in this application, that is, an invention that will be filed in division in the future, appearing by amendment, and added. The existence of is not denied.

  According to the present invention, a voice recognition device is provided. This speech recognition apparatus uses one or more word candidates (for example, “open”, “dawn”, “empty”, and “close”) following a word that has already been obtained in a word string with the same attribute. A matching process in which a part of the process (for example, calculation of an acoustic score) is shared and matched for each word group (for example, “open” and “shimeru”) classified for each word having (for example, a homophone) A word connection relationship indicating a connection relationship between words in a word string that is a candidate for a speech recognition result generated based on the result of the matching process performed by the execution unit (for example, the matching unit 58 in FIG. 5). And correction means for correcting the information (for example, the re-evaluation unit 59 in FIG. 5).

  According to the present invention, a voice recognition device is provided. In this speech recognition apparatus, classification means (for example, classifying one or more word candidates subsequent to the already obtained word in the word string as the word group for each word having the same attribute) 5 is further provided, and the matching processing execution means may share a part of the processing for each word group classified by the classification means. it can.

  According to the present invention, a voice recognition device is provided. The speech recognition apparatus further includes selection means (for example, the word preliminary selection unit 56 in FIG. 5) for selecting one or more word candidates subsequent to the already obtained word from the word string. The classifying means may classify the one or more word candidates selected by the selecting means as the word group for each word having the same attribute.

  According to the present invention, a voice recognition device is provided. In this speech recognition apparatus, one or more candidate words following the already obtained word can be classified into the word group for each word having the same pronunciation.

  According to the present invention, a voice recognition device is provided. In this speech recognition apparatus, one or more candidate words following the already obtained word can be classified into the word group for each of the acoustically similar words.

  According to the present invention, a voice recognition device is provided. The speech recognition apparatus further includes storage means (for example, the word connection relation storage unit 60 in FIG. 5) for storing the word connection relation information generated based on the result of the matching process performed by the matching process execution means. The correcting means can correct the word connection relation information stored in the storing means.

  According to the present invention, a voice recognition device is provided. In this speech recognition apparatus, when a state transition model for calculating an acoustic score of a partial word string is constructed by the modifying means to modify the word connection relation information, the constructed state A storage means (for example, the reevaluation process storage unit 201 in FIG. 22) for storing the transition model is further provided, and the correction means recalculates the acoustic score of the same partial word string. The state transition model stored by the storage means can be used.

  According to the present invention, a voice recognition device is provided. In this speech recognition apparatus, when a state transition model for calculating an acoustic score of a word is constructed by the matching processing execution unit, a storage unit (for example, FIG. 25) that stores the constructed state transition model. The word model storage unit 301) is further provided, and when the acoustic score of the same word is calculated again in the matching processing execution unit, the state transition model stored in the storage unit is used. Can be made.

  According to the present invention, a voice recognition device is provided. In this speech recognition apparatus, a storage unit (for example, a matching process storage unit 401 in FIG. 28) that stores a value calculated by the matching process execution unit as a midway of the matching process is further provided. The matching processing execution means may be configured to alternately execute the matching processing of the plurality of word strings that are candidates for the speech recognition result by using the value as the intermediate progress stored by the storage means. it can.

  According to the present invention, a speech recognition method is provided. In this speech recognition method, one or more word candidates (for example, “open”, “dawn”, “open”, and “close”) following a word that has already been obtained in a word string are assigned the same attribute. A matching process in which a part of the processing (for example, calculation of an acoustic score) is shared for each word group (for example, “open” and “shime”) classified for each word having (for example, a homophone). A word indicating a connection relationship between words in a word sequence that is a candidate for a speech recognition result, generated based on the result of the matching process executed in the process execution step (for example, step S7 in FIG. 7) and the process in the matching process execution step. And a correction step (for example, step S4 in FIG. 7) for correcting the connection relation information.

  According to the present invention, a program similar to the speech recognition method is provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 5 shows a configuration example of a speech recognition apparatus to which the present invention is applied.

  The voice uttered by the user is input to a microphone (microphone) 51 in FIG. 5, and the microphone 51 converts the input voice into an audio signal as an electric signal. This audio signal is supplied to the AD converter 52. In the AD conversion unit 52, the audio signal that is an analog signal from the microphone 51 is sampled, quantized, and converted into audio data that is a digital signal. This audio data is supplied to the feature extraction unit 53.

  The feature extraction unit 53 performs acoustic processing on the audio data from the AD conversion unit 52 for each appropriate frame, thereby extracting, for example, a feature quantity such as MFCC (Mel Frequency Cepstrum Coefficient) and the like to the control unit 54. Supply. In addition, the feature extraction unit 53 can extract other feature quantities such as, for example, a spectrum, a linear prediction coefficient, a cepstrum coefficient, and a line spectrum pair. Note that the sequence of feature amounts of the voice uttered by the user output from the feature extraction unit 53 is supplied to the control unit 54 in units of frames, and the control unit 54 receives from the feature extraction unit 53. The feature amount is supplied to the feature amount storage unit 55.

  The control unit 54 refers to the word connection relationship information stored in the word connection relationship storage unit 60 and controls the matching unit 58 and the reevaluation unit 59. Further, the control unit 54 generates word connection relation information based on the acoustic score, the language score, and the like as the matching process result obtained by the matching process performed by the matching unit 58, and the word connection relation information is used to generate the word connection relation information. The stored contents of the connection relation storage unit 60 are updated. Further, the control unit 54 corrects the stored contents of the word connection relation storage unit 60 based on the output of the reevaluation unit 59. Further, the control unit 54 determines and outputs a final speech recognition result based on the word connection relationship information stored in the word connection relationship storage unit 60.

  Here, the word connection relationship information represents a connection (chain or concatenation) relationship between words constituting a word sequence that is a candidate for a final speech recognition result, and includes an acoustic score and a language score of each word, and each The start time and end time of the utterance corresponding to the word are included.

  In the present specification, description will be made on the assumption that “the higher the value of the acoustic score and the language score, the better”. However, conversely, even if it is assumed that “the smaller the value of the acoustic score and the language score, the better”, if the sign of the acoustic score and the language score is reversed, the present invention Can be applied.

  FIG. 6 shows an example of word connection relationship information stored in the word connection relationship storage unit 60 using a graph structure. In FIG. 6, the graph structure as the word connection relation information includes an arc representing a word (portion indicated by a line segment connecting the circles in FIG. 6) and a node representing a boundary between the words (circles in FIG. 6). Part). The same applies to FIGS. 8 to 11, 14, 16, 17, 19 to 21, 24, and 26 described later.

  The node has time information, and this time information represents the extraction time of the feature amount corresponding to the node. As will be described later, the extraction time is the time at which the feature amount output by the feature extraction unit 53 is obtained, with the start time of the speech section being 0. Accordingly, in FIG. 6, the time information of the node 1 corresponding to the start of the speech section, that is, the head of the first word is 0. The node is the start and end of the arc, but the time information of the start node (start node) or the end node (end node) is the start time or end time of the utterance of the word corresponding to the node, respectively. It is time.

  In FIG. 6, the time from left to right represents the passage of time, and therefore, among the nodes on the left and right of a certain arc, the left node is the start node and the right node is the end node. The same applies to FIGS. 8 to 11, 14, 16, 17, 19 to 21, 24, and 26 described later.

  The arc has a word corresponding to the arc, and an acoustic score and a language score of the word, and the arc is sequentially connected with a node that is a terminal node as a start node, thereby performing speech recognition. A sequence of words as candidate results is constructed. In FIG. 6, the word, the acoustic score, and the language score corresponding to each arc are shown in parentheses in the immediate vicinity of each arc. That is, in the example of FIG. 6, the word corresponding to the arc 1 is “room”, the acoustic score is A1, and the language score is L1. The word corresponding to arc 2 is “O”, the acoustic score is A2, and the language score is L2. The word corresponding to the arc 3 is “open”, the acoustic score is A3, and the language score is L3. The word corresponding to the arc 4 is “window”, the acoustic score is A4, and the language score is L4. The word corresponding to the arc 5 is “O”, the acoustic score is A5, and the language score is L5.

  In the control unit 54, first, an arc corresponding to a probable word as a speech recognition result is connected to the node 1 representing the start of the speech section. In the example of FIG. 6, an arc 1 corresponding to “room” and an arc 4 corresponding to “window” are connected. Whether or not the word is likely to be a speech recognition result is determined based on the acoustic score and language score obtained by the matching unit 58.

  In the same manner, the end node 2 corresponding to the end of the arc 1 corresponding to “room”, the end node 3 corresponding to the end of arc 2 corresponding to “O”, and the end of the arc 4 corresponding to “window”. Similarly, arcs corresponding to probable words are connected to each of the terminal nodes 5.

  By connecting arcs as described above, one or more paths composed of arcs and nodes are formed from left to right starting from the start of the speech section. Are reached at the end of the voice section (time T in the embodiment of FIG. 6), the control unit 54 determines, for each path formed from the start to the end of the voice section, an arc that constitutes the path. The sound score and the language score possessed by are accumulated, and a final score is obtained. Then, for example, a word string corresponding to an arc constituting a path having the highest final score is determined and output as a speech recognition result.

  In the above-described case, the arcs are always connected to the nodes in the speech section, and the path extending from the start to the end of the speech section is configured. In the process of configuring such a path, For a path that is clearly unsuitable as a speech recognition result from the scores for the paths that have been constructed so far, the path configuration should be terminated at that point (the arc is not connected thereafter). Is possible.

  Further, according to the path configuration rule as described above, the end of one arc becomes the start node of one or more arcs to be connected next, and basically the path is configured so that the branches and leaves expand. Exceptionally, if the end of one arc matches the end of another arc, that is, the end node of one arc and the end node of another arc are shared by the same node. There is a case.

  That is, when a bigram is used as a grammar rule, two arcs extending from another node correspond to the same word, and when the end time of the utterance of the word is also the same, The ends of the two arcs coincide.

  Although it is possible not to share the nodes, it is preferable to do so from the viewpoint of increasing the memory capacity.

  Further, the word connection relation information stored in the word connection relation storage unit 60 can be referred to as needed by the word preliminary selection unit 56, the matching unit 58, and the reevaluation unit 59. .

  In FIG. 5, the feature quantity storage unit 55 stores a series of feature quantities supplied from the control unit 54. The feature quantity storage unit 55 stores all of the supplied feature quantity time series so that the time series can be used retroactively. The control unit 54 uses the start time of the voice section as a reference (for example, 0), and the time when the feature amount output by the feature extraction unit 53 is obtained (hereinafter, referred to as extraction time as appropriate) together with the feature amount. The feature amount storage unit 55 stores the feature amount together with its extraction time. The feature amount stored in the feature amount storage unit 55 and the extraction time thereof can be referred to by the word preliminary selection unit 56, the matching unit 58, and the reevaluation unit 59 as necessary.

  When the word preliminary selection unit 56 is instructed by the matching unit 58 to perform word preliminary selection using the time of the designated node as the start time, the word connection relation storage unit 60, the acoustic model database (DB) 61, A feature quantity storage unit performs word preliminary selection processing in which the matching unit 58 selects one or more words to be subjected to matching processing while referring to the dictionary database (DB) 62 and the grammar database (DB) 63 as necessary. The feature amount stored in 55 is used. The word preliminary selection unit 56 transmits one or more words selected as a result of the word preliminary selection processing to the word candidate clustering unit 57. When the word preliminary selection unit 56 selects, for example, four word candidates “open”, “open”, “open”, and “close” starting from time t1 in FIG. 6 by the word preliminary selection process. The four word candidates are transmitted to the word candidate clustering unit 57.

  The word candidate clustering unit 57 examines the pronunciations of the word candidates supplied from the word preliminary selection unit 56 and clusters word candidates (sound words) having the same pronunciation. Thereby, one or more word candidate sets are generated. For example, when word candidates “open”, “open”, “open”, and “close” are supplied from the word preliminary selection unit 56, the word candidate clustering unit 57 uses the same “open”, “open”, and “open”. Clustering as a word candidate (sound word) having pronunciation, collecting it as one word candidate set, and letting “close” be another word candidate set having only that as an element. The word candidate clustering unit 57 assigns classification information for identifying the word candidate set to which the word candidate belongs to each word candidate after the word candidate clustering process supplied from the word preliminary selection unit 56 is completed. To the matching unit 58.

  Based on the control from the control unit 54, the matching unit 58 instructs the word preliminary selection unit 56 to perform word preliminary selection using the time of the designated node as the start time. Further, the matching unit 58 refers to the word connection relationship storage unit 60, the acoustic model database 61, the dictionary database 62, and the grammar database 63 as necessary, and the word obtained as a result of the clustering process from the word candidate clustering unit 57 The matching process for the candidate is performed using the feature amount stored in the feature amount storage unit 55, and the acoustic score, the language score, and the end time for each word candidate are determined.

  That is, the acoustic model database 61 stores an acoustic model representing acoustic features such as individual phonemes and syllables in the speech language for speech recognition. Here, since speech recognition is performed based on the continuous distribution HMM method, for example, an HMM (Hidden Markov Model) is used as the acoustic model. The dictionary database 62 stores a word dictionary in which information about pronunciation (phoneme information) is described for each word (vocabulary) to be recognized. The grammar database 63 stores grammar rules (language model) describing how the words registered in the word dictionary of the dictionary database 62 are linked (connected). Here, as the grammar rule, for example, a rule based on context-free grammar (CFG), statistical word chain probability (N-gram), or the like can be used.

  The matching unit 58 calculates the initial value of the score for each word candidate included in the word candidate set obtained as a result of the clustering process from the word candidate clustering unit 57, and for the word having the largest initial value, the matching unit 58 By connecting the acoustic model stored in the acoustic model database 61 by referring to the word dictionary, a word acoustic model (word model) is constructed, and the word candidate is used based on the feature amount using the word model. Determine the acoustic score and end time for each set. The matching unit 58 also refers to the grammar database 63 to determine the language score of the word candidate included in the word candidate set.

  The matching unit 58 supplies the control unit 54 with the acoustic score for each word candidate set, the end time, and the language score for each word candidate obtained as a result of the matching process.

  Based on the control from the control unit 54, the re-evaluation unit 59 refers to the acoustic model database 61, the dictionary database 62, and the grammar database 63 as necessary, and stores the word connection relationship stored in the word connection relationship storage unit 60. Information reevaluation is performed using the feature quantity stored in the feature quantity storage unit 55, and the reevaluation result is supplied to the control unit 54.

  That is, the re-evaluation unit 59 traces back in time from the node before performing the matching process on the node that should continue the matching process in the word connection relation information stored in the word connection relation storage unit 60. The reevaluation for the partial word sequence is performed using the acoustic model database 61, the dictionary database 62, and the grammar database 63. At this time, as will be described later, since a partial word string can be evaluated without fixing the word boundary, the word boundary is determined based on dynamic programming. Then, based on the result of the re-evaluation, the word connection relation information stored in the word connection relation storage unit 60 is corrected. This process is performed immediately before issuing the next matching process execution command to the matching unit 58. For example, paying attention to the two word string hypotheses “window” and “door” in FIG. 6, each reevaluation is performed, and thereafter, the word preliminary selection unit 56, the word candidate clustering unit 57, and the matching unit 58 Processing is performed. Similarly, when the newly extended word string hypotheses “open window” and “open door” are stored in the word connection relationship storage unit 60, the word string hypothesis is also re-evaluated later. As a result, correct matching processing using the acoustic model is performed for the same word “open”, depending on the previous word string hypotheses “window” and “door”, and the word boundary correction and sound The score will be corrected.

  As described above, the word connection relationship storage unit 60 stores the word connection relationship information supplied from the control unit 54 until a recognition result of the user's voice is obtained.

  As described above, the word candidate clustering unit 57 reduces the number of matching processes by clustering word candidates having the same pronunciation, and once the result is stored in the word connection relation storage unit 60, the stored word string By re-evaluating the hypothesis in the re-evaluation unit 59, the acoustic model depending on the immediately preceding word string hypothesis that is not used in the matching process is applied, and as a result, the correct word boundary and acoustic score are obtained. It becomes possible to correct.

  Next, the speech recognition process by the speech recognition apparatus in FIG. 5 will be described with reference to the flowchart in FIG.

  When the user speaks, the voice as the speech is converted into digital voice data via the microphone 51 and the AD conversion unit 52 and supplied to the feature extraction unit 53. The feature extraction unit 53 sequentially extracts voice feature amounts for each frame from the audio data supplied thereto, and supplies the extracted feature amounts to the control unit 54.

  The control unit 54 recognizes a speech section by some method. In the speech section, the feature amount sequence supplied from the feature extraction unit 53 is associated with the extraction time of each feature amount, and the feature is detected. The quantity is supplied to and stored in the quantity storage unit 55.

  Further, after the start of the speech section, the control unit 54 generates a node representing the start of the speech section (hereinafter, referred to as an initial node as appropriate) in step S1, and supplies the node to the word connection relation storage unit 60 for storage. . That is, the control unit 54 stores, for example, the node 1 in FIG. 6 in the word connection relation storage unit 60 in step S1.

  Then, the process proceeds to step S <b> 2, and the control unit 54 refers to the word connection relationship information in the word connection relationship storage unit 60 to determine whether there is a midway node.

  That is, as described above, in the word connection relation information shown in FIG. 6, for example, a path extending from the start to the end of the speech interval is formed by connecting an arc to the terminal node. In step S2, an end node that has not yet been connected to the arc and has not reached the final time T of the voice section is searched as an intermediate node (for example, node 6 in FIG. 6). It is determined whether or not such a halfway node exists.

  As described above, the voice section is recognized by some method, and the time corresponding to the terminal node can be recognized by referring to the time information of the terminal node, so that the arc is connected. Whether or not a terminal node that is not present is a midway node that has not reached the end of the speech segment can be determined by comparing the last time of the speech segment with the time information of the termination node.

  If it is determined in step S2 that there is an intermediate node, the process proceeds to step S3, and the control unit 54 sets one of the intermediate nodes existing in the word connection relation information as an arc connected to the intermediate node. The node is selected as a node for determining a word (hereinafter, referred to as an attention node as appropriate).

  That is, when there is only one halfway node in the word connection relation information, the control unit 54 selects that halfway node as the node of interest. In addition, when there are a plurality of halfway nodes in the word connection relation information, the control unit 54 selects one of the plurality of halfway nodes as the attention node. Specifically, for example, the control unit 54 refers to time information possessed by each of a plurality of intermediate nodes, and selects, as the node of interest, the one with the earliest time represented by the time information (the one on the voice segment start side). To do. When there are a plurality of intermediate nodes with the oldest time at the same time (for example, in the case of the example shown in FIG. 17 described later), the control unit 54 selects all of the intermediate nodes with the oldest time as nodes of interest. Select as.

  Thereafter, the control unit 54 outputs, to the matching unit 58 and the reevaluation unit 59, a command for performing the matching process using the time information of the node of interest as the start time (hereinafter referred to as a matching process command as appropriate).

  When the re-evaluation unit 59 receives the matching processing command from the control unit 54, the process proceeds to step S 4, and by referring to the word connection relation storage unit 60, the path from the initial node to the node of interest (hereinafter referred to as partial A word string (hereinafter referred to as a partial word string as appropriate) represented by an arc constituting a path) is recognized, and the partial word string is re-evaluated. That is, the partial word string is a word string that is a candidate for a speech recognition result obtained by the matching unit 58 performing a matching process on the word preliminarily selected by the word preliminary selection unit 56 as described later. Although it is an intermediate result, the re-evaluation unit 59 evaluates the intermediate result again.

  Specifically, the re-evaluation unit 59 reads a feature amount series corresponding to the partial word sequence from the feature amount storage unit 55 in order to recalculate the language score and the acoustic score for the partial word sequence. That is, the re-evaluation unit 59, for example, a series of feature amounts (from the time indicated by the time information of the initial node that is the first node of the partial path to the time indicated by the time information of the node of interest ( The feature amount series) is read from the feature amount storage unit 55. Further, the re-evaluation unit 59 refers to the acoustic model database 61, the dictionary database 62, and the grammar database 63, and uses the feature amount sequence read from the feature amount storage unit 55 to determine the language score and the acoustic score for the partial word string. Is recalculated. This recalculation is performed without fixing the word boundaries of the words constituting the partial word string. Therefore, the re-evaluation unit 59 recalculates the language score and the acoustic score of the partial word string, so that the word boundary of each word constituting the partial word string is determined based on the dynamic programming. It will be.

  When the re-evaluation unit 59 newly obtains the language score and the acoustic score of each word of the partial word string and the word boundary as described above, the word connection relation storage unit 60 uses the new language score and acoustic score. A node constituting a partial path corresponding to a partial word string in the word connection relation storage unit 60 by correcting a language score and an acoustic score of an arc constituting a partial path corresponding to a partial word string of The time information possessed by is corrected. In the present embodiment, correction of the word connection relation information by the reevaluation unit 59 is performed via the control unit 54.

  An example of the process of the re-evaluation unit 59 described above will be described with reference to FIGS. FIG. 8 shows one of the word string hypotheses stored in the word connection relation storage unit 60 extracted as a graph. In FIG. 8, arc 1 connecting node 0 at time 0 to node 2 at time a corresponds to the word candidate “window”, the acoustic score of the word candidate “window” is A1, and the language score is L1. . An arc 2 connecting node 2 at time a to node 3 at time t corresponds to the word candidate “O”, the acoustic score of the word candidate “O” is A2, and the language score is L2.

  When the control unit 54 is instructed to perform the matching process using the time information of the node 3 as the start time, the re-evaluation unit 59 sets the word string hypothesis “window to correspond to the nodes 1 to 3”. Is re-evaluated for the word string. As a result, as shown in FIG. 9, the time a of the node 2 that is the word boundary between “window” and “wo” is corrected to the time b, and the acoustic score A1 given to the arc 1 is corrected to A1b. The acoustic score A2 given to the arc 2 is corrected to A2b. Thereafter, it is assumed that matching processing is performed by the matching unit 58 on the word “open”, starting at time t. As a result, as shown in FIG. 10, the arc 3 corresponding to the word candidate “open” is added with the node 4 at the time u as the terminal node. In FIG. 10, the acoustic score of the word candidate “open” is A3, and the language score is L3.

  Thereafter, when the control unit 54 is further instructed to perform the matching process using the time information of the node 4 as the start time, the reevaluation unit 59 “opens” corresponding to the nodes 2 to 4. Re-evaluate the word string. As a result, as shown in FIG. 11, the time t of the node 3 serving as the word boundary between “open” and “open” is corrected to the time s, and the acoustic score A2b given to the arc 2 is corrected to A2c, The acoustic score A3 given to the arc 3 is corrected to A3b.

  When the re-evaluation unit 59 finishes correcting the word connection relationship information in the word connection relationship storage unit 60, the re-evaluation unit 59 supplies the fact to the matching unit 58 via the control unit.

  As described above, when the matching unit 58 receives the matching processing command from the control unit 54 and then receives a notification from the reevaluation unit 59 via the control unit 54 that the correction of the word connection relation information has been completed, The node and the time information possessed by the node are supplied to the word preliminary selection unit 56, each of which requests a word preliminary selection process, and proceeds to step S5.

  When the word preliminary selection unit 56 receives a request for word preliminary selection processing from the matching unit 58, in step S5, the word preliminary selection unit 56 performs word preliminary selection processing for selecting a word candidate to be an arc connected to the target node. This is performed on words registered in the word dictionary.

  That is, the word preliminary selection unit 56 recognizes the start time of the feature amount series used to calculate the language score and the acoustic score from the time information of the node of interest, and sets the necessary feature amount after the start time. The series is read from the feature amount storage unit 55. Further, the word preliminary selection unit 56 configures the word model of each word registered in the word dictionary of the dictionary database 62 by connecting the acoustic model of the acoustic model database 61, and stores the feature amount based on the word model. The acoustic score is calculated using the feature amount sequence read from the unit 55.

  The word preliminary selection unit 56 calculates the language score of the word corresponding to each word model based on the grammar rules stored in the grammar database 63.

  In addition, the word preliminary selection unit 56 refers to the word connection relation information to calculate the acoustic score of each word (word corresponding to the arc whose terminal node is terminated) immediately before the word. Can be performed using a crossword model that depends on.

  In addition, the word preliminary selection unit 56 calculates the language score of each word by referring to the word connection relation information based on the bigram that defines the probability that the word is linked to the immediately preceding word. Is possible.

  When the word preliminary selection unit 56 obtains the acoustic score and the language score for each word as described above, a score obtained by comprehensively evaluating the acoustic score and the language score is hereinafter referred to as a word score as appropriate). The upper L words are supplied to the word candidate clustering unit 57 as word candidates to be subjected to matching processing.

  Here, in the word preliminary selection unit 56, the word is selected based on the word score obtained by comprehensively evaluating the acoustic score and the language score of each word. However, in the word preliminary selection unit 56, for example, It is possible to select a word based on only the acoustic score or the language score.

  In addition, the word preliminary selection unit 56 uses only the first part of the feature quantity series read from the feature quantity storage unit 55 and uses some of the first parts of the corresponding words based on the acoustic model in the acoustic model database 61. It is also possible to obtain a phoneme and select a word whose first part matches the phoneme.

  Further, the word preliminary selection unit 56 refers to the word connection relation information, recognizes the part of speech of the immediately preceding word (the word corresponding to the arc whose target node is the terminal node), and the part of speech of the word following the part of speech. It is also possible to select a word with the most likely part of speech.

  That is, any method may be used as a word selection method in the word preliminary selection unit 56, and ultimately a word may be selected at random.

  When the word candidate clustering unit 57 receives L word candidates used for the matching process from the word preliminary selection unit 56, in step S6, the word candidate clustering unit 57 performs the clustering process on the word candidates.

  That is, the word candidate clustering unit 57 classifies the L word candidates into words (sound words) having the same pronunciation, and sets these as word candidate sets. Therefore, for example, when the word candidates “open”, “dawn”, “empty”, and “close” are supplied from the word preliminary selection unit 56, the word candidate clustering unit 57 opens the word candidate “sound word”. ”,“ Dawn ”, and“ open ”are classified into one word candidate set“ open ”, and the word candidate“ close ”is classified into one word candidate set“ close ”. After classifying the word candidates into word candidate sets, the word candidate clustering unit 57 assigns each word candidate with classification information indicating to which word candidate set the word candidates are classified, and the word candidates are sent to the matching unit 58. Supply.

  When the matching unit 58 receives the L word candidates to which the classification information is assigned from the word candidate clustering unit 57, the matching unit 58 performs matching processing on the word candidates in step S7. Here, the matching process in step S7 in FIG. 7 will be described in detail with reference to the flowchart in FIG. In the following description, as an example, the node 6 in FIG. 6 is selected as the node of interest (step S3 in FIG. 7), and the word candidates “open”, “open”, “open” by the word preliminary selection unit 56, And “close” are selected (step S5 in FIG. 7), and the word candidate clustering unit 57 closes the word candidate set “open” including the word candidates “open”, “open”, and “open”, and the word candidate “close”. A case will be described in which the word candidate set “Shime” including “is clustered (step S6 in FIG. 7).

  In step S101 of FIG. 12, the matching unit 58 refers to the word connection relationship information stored in the word connection relationship storage unit 60 to identify the word string hypothesis immediately before connecting the word candidates, and for the word string hypothesis. Calculate the score.

  The score corresponding to the word string hypothesis is given by the cumulative value of the acoustic score and the language score. For example, if the score corresponding to the word string hypothesis “window” is S (window), the score corresponding to the word string hypothesis “window” composed of arc 4 and arc 5 shown in FIG. Is given by

S (window) = (A4 + L4) + (A5 + L5)

  In other words, the score corresponding to the word string hypothesis is obtained by adding the total score obtained by adding the acoustic score and language score of each word included in the word string hypothesis to all the words included in the word string hypothesis. Is calculated.

  In step S102, the matching unit 58 selects one word candidate set to be processed in steps S103 to S107 from the word candidate sets supplied from the word candidate clustering unit 57. For example, the word candidate clustering unit 57 performs clustering into the word candidate set “open” including the word candidates “open”, “open”, and “open”, and the word candidate set “shime” including the word candidate “close”. In this case, the matching unit 58 selects one of the word candidate sets “open” and “shimeru”.

  Next, in step S103, the matching unit 58 obtains the language scores of all the word candidates included in the word candidate set selected in step S102 from the probability based on the bigram, for example. For example, when the word candidate set “open” is selected in step S102, in step S103, the matching unit 58 “opens” in the language score “open window” when “open” follows “open”. "Is followed by a language score, and a" window "is followed by" open "followed by a language score.

  In step S104, the matching unit 58 calculates the initial score based on the cumulative score up to the word immediately before connecting the word candidate calculated in step S101 and the language score of the word candidate calculated in step S103. Is calculated.

  The initial value of the score differs depending on the word string hypothesis connected before. For example, when the word string hypothesis “open window” is followed by “open”, the initial value U1 is given by the following equation.

U1 = S (open window) + L (open | open window)
... (1)

  In the above equation (1), L (open | window) represents a language score when “open” is followed by “open”. In the case of using a bigram, this language score is given by a two word chain probability of (open).

  Similarly, when “break” follows the word string hypothesis “window”, the initial value U2 is given by the following equation. Note that L (dawn | window) represents a language score when “dawn” follows “open window”.

U2 = S (window) + L (dawn | window)
... (2)

  Similarly, when the word string hypothesis “window” is followed by “open”, the initial value U3 is given by the following equation. L (open | window) represents a language score when “open” is followed by “open”.

U3 = S (window) + L (open | window)
... (3)

  In this way, the score of the word string hypothesis (for example, “window” in FIG. 6) reaching the node (for example, node 6 in FIG. 6) connecting the new word candidate set is included in the word candidate set. By adding the language score when each word (for example, “open”, “dawn”, or “open”) is connected, the initial value of the score corresponding to each word candidate can be calculated. In the above example, three score initial values U1 to U3 are obtained for one word string hypothesis and three word candidates.

  FIG. 13 shows an example in which combinations of word string hypotheses and word candidates and initial values U1 to U3 of the respective scores are given. That is, in FIG. 13, U1 represents an initial value when “open” follows the word string hypothesis “window”, and U2 when “open” follows the word string hypothesis “window”. U3 represents the initial value when “open” follows the word string hypothesis “window”.

  Next, in step S105, the matching unit 58 determines the word candidate having the largest initial value of the calculated score as a word representing the word candidate set including the word candidate (hereinafter also referred to as a representative word). In step S106, the acoustic score of the representative word determined in step S105 is calculated. For example, when the initial value U1 of the combination (open the window) in FIG. 13 is the highest, the matching unit 58 executes a matching process for “open”. Here, the acoustic model corresponding to “open” is dependent on the immediately preceding word string hypothesis “window”. Specifically, the matching unit 58 recognizes the start time of the feature amount series used to calculate the acoustic score from the time information of the node of interest (in the example of FIG. 6, the node 6), and starts the start A series of necessary feature quantities after the time is read from the feature quantity storage unit 55. Further, the matching unit 58 recognizes the phoneme information of the representative word “open” by referring to the dictionary database 62, reads out the acoustic model corresponding to the phoneme information from the acoustic model database 61, and connects it. Construct a word model.

  Then, the matching unit 58 calculates the acoustic score of the representative word using the feature amount series read from the feature amount storage unit 55 based on the word model configured as described above. The matching unit 58 can calculate the acoustic score of the word based on the crossword model by referring to the word connection relation information.

  In step S107, the matching unit 58 determines the acoustic score of the representative word obtained in step S106 as the acoustic score of other word candidates included in the same word candidate set. As a result, for example, in step S106, if an acoustic score starting at time t1 in FIG. 6 and ending at a certain time is obtained for the representative word “open”, this acoustic score is used for the remaining word string. A combination of a hypothesis and a word candidate is also used as an approximate value as it is, and matching processing for the remaining two word candidates is not performed. That is, for example, for the combination of (open the window) and the combination of (open the window), the calculation of the acoustic score is omitted, and the acoustic score calculated by the combination of (open the window) is used instead.

  In step S108, the matching unit 58 determines whether or not there is an unselected word candidate set. If there is an unselected word candidate set, the process returns to step S102, and the processes after step S102 described above are performed. repeat. For example, as described above, after the processing of step S102 to step S107 is performed for the word candidate set “open” among the word candidate sets “open” and “shimeru”, in step S108, the matching unit 58 determines whether or not It is determined that the selected word candidate set “Shimeru” exists, and the process returns to step S102. Thereafter, the processing from step S102 to step S107 is also executed for the word candidate set “Shimeru”. If only one word candidate is included in the word candidate set as in the word candidate set “Shime”, the process of step S107 is skipped.

  If the matching unit 58 determines in step S108 that there is no unselected word candidate set, the matching process 58 in FIG. 12 ends, and the process proceeds to step S8 in FIG.

  As described above, the matching unit 58 obtains the acoustic score and the language score for all the word candidate sets from the word candidate clustering unit 57, and proceeds to step S8.

  In step S8, a word score obtained by comprehensively evaluating the acoustic score and the language score is obtained for each word candidate, and the word connection relation information stored in the word connection relation storage unit 60 is updated based on the word score. .

  That is, in step S8, the matching unit 58 obtains a word score for the word candidate, and compares the word score as an arc connected to the node of interest, for example, by comparing the word score with a predetermined threshold. We narrow down from. Then, the matching unit 58 supplies the word remaining as a result of the narrowing down to the control unit 54 together with the acoustic score, the language score, and the end time of the word.

  In the matching unit 58, the end time of the word is recognized from the extraction time of the feature value used for calculating the acoustic score. In addition, when a plurality of extraction times with high probability as the end time are obtained for a certain word, a set of each end time and a corresponding acoustic score and language score is obtained for the word by the control unit 54. To be supplied.

  When receiving the acoustic score, language score, and end time of the word supplied from the matching unit 58 as described above, the control unit 54 stores each word from the matching unit 58 in the word connection relation storage unit 60. The target node in the word connection relation information is set as the start node, the arc is extended, and the arc is connected to the end node corresponding to the position of the end time. Further, the control unit 54 gives a corresponding word, an acoustic score and a language score to each arc, and gives a corresponding end time as time information to the end node of each arc. And it returns to step S2 and the same process is repeated hereafter.

  As a result, for example, word connection relationship information as shown in FIG. 14 is stored in the word connection relationship storage unit 60. FIG. 14 is included in the word candidates “open”, “open”, and “open” included in the word candidate set “open”, and the word candidate set “shimeru” with respect to the node 6 in FIG. 6. An example in which word candidates “close” are connected is shown. The arc 6 corresponds to the word candidate “open”, and the acoustic score A6 and the language score L6 are calculated as usual. The arc 7 corresponds to the word candidate “dawn”, and the language score L7 is calculated as usual, but the acoustic score is substituted by the acoustic score A6 of the word candidate “open”. That is, the acoustic score of the word candidate “bright” is not calculated. The arc 8 corresponds to the word candidate “open”, and the language score L8 is calculated as usual, but the acoustic score is substituted with the acoustic score A6 of the word candidate “open”. That is, the acoustic score of the word candidate “open” is not calculated.

  The arc 9 corresponds to the word candidate “close”, and the acoustic score A9 and the language score L9 are calculated as usual.

  With the above processing, it is possible to reduce the calculation amount of the speech recognition processing. In the case of the above example, conventionally, the acoustic score was obtained for each of the three combinations of (open the window), (open the window), and (open the window). The matching unit 58 needs to obtain the acoustic score only in the combination with the highest initial value (open the window) in step S106 of FIG. 12, and thus the number of calculations of the acoustic score is reduced to one third. Can do.

  In the conventional case, the acoustic score has to be calculated for each of the word candidates “open”, “dawn”, “empty”, and “close”, so the acoustic score is calculated four times in total. On the other hand, the matching unit 58 performs the word candidate set “Shime” including the word candidate “Close” once for the word candidate set “Open” including the word candidates “Open”, “Dawn”, and “Open”. It is only necessary to calculate once for each, a total of two times. Therefore, the number of calculations can be reduced by two times obtained by subtracting the number of calculations of 2 in the case of FIG.

  In the example of FIG. 14, each word candidate has only one end point (terminal node time) of T, but there may be a plurality of candidates for each word candidate end point (terminal node time). is there.

  Here, the principle of obtaining the end point in step S106 in FIG. 12 will be described. Consider a case where the matching process is performed on the word candidate “open” with the node 6 in FIG. 6 as the start time. In this case, as described above, the matching process for the word candidate “open” is performed with the start time t1 and the initial score value U1. The matching unit 58 acquires an acoustic standard pattern corresponding to the word candidate “open” from the pronunciation information stored in the acoustic model database 61 and the dictionary database 62.

  When a hidden Markov model (HMM) is used as an acoustic model, the standard pattern is a state transition model as shown in FIG. 15, and transition probabilities are given to transitions between the states, and the output probability density function is expressed in each state. Given. At this time, the initial value U1 is given to the initial state 101, and the acoustic score is calculated using the feature amount time series starting at time t1 in FIG. A score accumulation method called Viterbi Search is applied to the score calculation. As a result, the accumulated score in the final state 102 is obtained every hour. The accumulated score in the final state 102 obtained at each time is the acoustic score obtained for “open”.

  Here, when paying attention to a certain time, the cumulative score in one state between the initial state 101 and the final state 102 can be compared with the cumulative score in another state, and only a state with a relatively good cumulative score is enabled. While applying the Viterbi Search method. This is a technique called beam search, which is a widely used technique for improving the efficiency of matching processing. Also in the final state 102 in FIG. 15, it is possible to determine that only the time when the cumulative score becomes relatively good is valid. Therefore, the time when the final state 102 becomes valid is determined as the end point of the processing of the matching unit 58.

  Usually, the time at which the final state 102 becomes valid is not necessarily determined uniquely, and therefore, it may become effective at a plurality of times. FIG. 16 shows an example of word connection relationship information stored in the word connection relationship storage unit 60 when the final state 102 becomes valid at two times t2 and T.

  FIG. 16 includes the word candidates “open”, “open”, and “open” included in the word candidate set “open” and the word candidate set “shime” with respect to the node 6 in FIG. 6. In this example, the word candidates “closed” are connected.

  In FIG. 16, arc 6 corresponds to the word candidate “open”, and the time of the end node is the end time T of the speech section. The acoustic score A6 and language score L6 of this word candidate “open” are calculated as usual. The arc 7 corresponds to the word candidate “open”, and the time of the terminal node is the time t2 before the end time T of the speech section. The acoustic score A7 of the word candidate “open” is calculated at time t2 in the score calculation process for the word candidate “open”, and then the acoustic score A6 is calculated at time T. Since the word candidate “open” is the same word candidate as the arc 6, the language score is the same value L 6 as the arc 6.

  The arc 8 corresponds to the word candidate “dawn”, and the time of the terminal node is the end time T of the speech section, similar to the word candidate “open” of the arc 6. The language score L8 of this word candidate “dawn” is calculated as usual, but the acoustic score is substituted by the acoustic score A6 of the word candidate “open” (word candidate corresponding to arc 6) having the same end point. ing. That is, the acoustic score of the word candidate “bright” corresponding to the arc 8 is not calculated. The arc 9 corresponds to the word candidate “dawn”, and the time of the terminal node is the time t2 as in the word candidate “open” of the arc 7. The acoustic score of the word candidate “dawn” is substituted by the acoustic score A7 of the word candidate “open” (word candidate corresponding to the arc 7) having the same end point. That is, the acoustic score of the word candidate “bright” corresponding to the arc 9 is not calculated. Further, since this word candidate “bright” (word candidate corresponding to arc 9) is the same word candidate as arc 8, the language score is the same value L8 as arc 8.

  Further, the arc 10 corresponds to the word candidate “open”, and the time of the terminal node is the end time T of the speech section, similar to the word candidate “open” of the arc 6. The language score L10 of this word candidate “open” is calculated as usual, but the acoustic score is substituted with the acoustic score A6 of the word candidate “open” (word candidate corresponding to arc 6) having the same end point. ing. That is, the acoustic score of the word candidate “open” corresponding to the arc 10 is not calculated. The arc 11 corresponds to the word candidate “open”, and the time of the terminal node is the time t2 as in the word candidate “open” of the arc 7. The acoustic score A7 of the word candidate “open” (word candidate corresponding to the arc 7) having the same end point is substituted for the acoustic score of the word candidate “open”. That is, the acoustic score of the word candidate “bright” corresponding to the arc 11 is not calculated. Further, since this word candidate “open” (word candidate corresponding to the arc 11) is the same word candidate as the arc 10, the language score is the same value L10 as the arc 10.

  The arc 12 corresponds to the word candidate “close”, and the time of the end node is the end time T of the speech section. The acoustic score A12 and the language score L12 of this word candidate “close” are calculated as usual. The arc 13 corresponds to the word candidate “close”, and the time of the terminal node is the time t2 before the end time T of the speech section. The acoustic score A13 of this word candidate “close” is calculated as usual. Since the word candidate “close” is the same word candidate as the arc 12, the language score is the same value L 12 as the arc 12.

  In the example of FIG. 16, in the process of step S106 of FIG. 12, candidates for two end points (end node times) are obtained for each word candidate. In this case, for the word candidates included in the same word candidate set, the acoustic score can be calculated once for each time candidate (T and t2). Conventionally, although it was possible to complete the score calculation once for the same word candidate, since the acoustic score of each of the different word candidates was obtained, a total of four times for the arc 6 to arc 14 You will ask for a score. On the other hand, for the word candidates included in the same word candidate set, the matching unit 58 only needs to calculate the acoustic score once for each end point candidate. That is, in FIG. 16, the number of calculations of the acoustic score is only two.

  Therefore, the number of calculations can be reduced by two times obtained by subtracting the number of calculations of 2 in the case of FIG. This is the same as the case of one end point shown in FIG. Even if the number of end point candidates is 3 or more, the same effect of reducing the number of calculations can be obtained.

  In the examples of FIGS. 14 and 16, the word string hypothesis that connects the word candidate sets is only one “window” (that is, only one node 6 is connected to the word candidate set). However, there may be a plurality of word string hypotheses connecting word candidate sets at the same time. That is, if there are a plurality of intermediate nodes at the same time in step S3 of FIG. 7, a plurality of intermediate nodes are selected. FIG. 17 shows an example of word connection relation information when there are a plurality of word string hypotheses connecting word candidate sets at the same time.

  The graph shown in FIG. 17 is obtained by adding an arc 6 to the graph shown in FIG. That is, in FIG. 17, an arc 6 corresponding to the word candidate “door” is further connected to the node 1. The node 7 is a terminal node of the arc 6, and the time information held by the node 7 is the same as the time information held by the node 6.

  As shown in FIG. 17, there may be a plurality of word string hypotheses (two “window” and “door”) that connect word candidate sets at the same time t1.

  In this case, in step S101 of FIG. 12, the score corresponding to each word string hypothesis is given as the cumulative value of the acoustic score and the language score. For example, if the scores corresponding to the word string hypotheses “window” and “door” are S (window) and S (door), respectively, the word string hypotheses “window” and “door” shown in FIG. The scores corresponding to are given by the following equations.

S (window) = (A4 + L4) + (A5 + L5)
S (door) = A6 + L6

  Then, in step S103 of FIG. 12, a language score is obtained for each word candidate, and in step S104, an initial score value is obtained. The initial value of the score required for matching the three word candidates “open”, “open”, and “dawn”, which are homophones, differs according to the word string hypothesis connected before. Become. The initial value when the word string hypothesis “window” is followed by “opening”, “opening”, and “opening” is expressed by the above-described equations (1), (2), and (3). They are given as U1, U2, and U3, respectively.

  The initial value U4 when “open” follows the word string hypothesis “door”, the initial value U5 when “open” follows the word string hypothesis “door”, and the word string hypothesis “door”. The initial value U6 in the case where “empty” follows for “” is given by the following equations, respectively. Note that L (open | door) represents the language score when “open” follows “door”, and L (open | door) represents the language score when “open” follows “door”. L (open | door) represents a language score when “open” is followed by “open”.

U4 = S (door) + L (open | door)
... (4)

U5 = S (door) + L (dawn | door)
... (5)

U6 = S (door) + L (open | door)
... (6)

  In the above formula, L (open | door) represents a language score when “open” follows “door”, and L (open | door) follows “door” followed by “open”. L (open | door) represents a language score when “open” is followed by “open”. These language scores are obtained in step S103 of FIG.

  In this manner, initial values of six scores are obtained for two word string hypotheses “window” and “door” and three word candidates “open”, “dawn”, and “open”. Become. FIG. 18 shows an example in which combinations of word string hypotheses and word candidates and respective initial value scores U1 to U6 are given. That is, in FIG. 18, U1 represents an initial value when “open” follows the word string hypothesis “window”, and U2 when “open” follows the word string hypothesis “window”. U3 represents the initial value when “open” follows the word string hypothesis “window”, and U4 represents the case where “open” follows the word string hypothesis “door”. U5 represents an initial value when “open” follows the word string hypothesis “door”, and U6 represents an initial value when “open” follows the word string hypothesis “door”. Represents.

  In step S105 in FIG. 12, the matching unit 58 selects one of the combinations shown in FIG. 18 that has the best initial value, and performs matching processing on the corresponding word candidate based on the combination. . For example, when the initial value U1 is the best and the combination of (open window) is selected, the matching unit 58 performs matching processing for the word candidate “open” in step S106 of FIG. Here, the acoustic model corresponding to the word candidate “open” is dependent on the immediately preceding word string hypothesis “window”. As a result, an acoustic score starting from time t1 in FIG. 17 and ending at a certain time is obtained for “open”.

  In step S107, the matching unit 58 directly uses the acoustic score obtained in step S106 as an approximate value for the remaining combinations of word string hypotheses and word candidates, and matching processing for the remaining five word candidates is performed. Don't do it. That is, for example, the combinations of (open window, open), (open window, open), (door, open), (door, open), and (door, open), As shown in FIG. 19, the calculation of the acoustic score is omitted, and the acoustic score calculated by the combination of (open the window) is used instead.

  That is, FIG. 19 shows the word candidates “open”, “open”, and “open” included in the word candidate set “open”, and the word candidate set “shime” for the nodes 6 and 7 in FIG. ] Shows an example of word connection relation information when word candidates “close” included in “” are connected.

  In FIG. 19, the arc 7 connected to the node 6 corresponds to the word candidate “open”, and the acoustic score A7 and the language score L7 are calculated as usual. The arc 8 connected to the node 6 corresponds to the word candidate “open”, and this language score L8 is calculated as usual, but the acoustic score is substituted by the acoustic score A7 of the word candidate “open”. ing. That is, the acoustic score of the word candidate “bright” corresponding to the arc 8 is not calculated. The arc 9 connected to the node 6 corresponds to the word candidate “open”, and the language score L9 is calculated as usual, but the acoustic score is an acoustic score A7 of the word candidate “open”. Substituting. That is, the acoustic score of the word candidate “open” corresponding to the arc 9 is not calculated. The arc 10 connected to the node 6 corresponds to the word candidate “close”, and the acoustic score A10 and the language score L10 are calculated as usual.

  The arc 11 connected to the node 7 corresponds to the word candidate “open”, and the language score L11 is calculated as usual, but the acoustic score is the sound of the word candidate “open” corresponding to the arc 7. A score A7 is substituted. That is, the acoustic score of the word candidate “open” corresponding to the arc 11 is not calculated. The arc 12 connected to the node 7 corresponds to the word candidate “open”, and the language score L12 is calculated as usual, but the acoustic score is the sound of the word candidate “open” corresponding to the arc 7. A score A7 is substituted. That is, the acoustic score of the word candidate “bright” corresponding to the arc 12 is not calculated. The arc 13 connected to the node 7 corresponds to the word candidate “open”, and the language score L13 is calculated as usual, but the acoustic score is the word candidate “open” corresponding to the arc 7. The acoustic score A7 is substituted. That is, the acoustic score of the word candidate “open” corresponding to the arc 13 is not calculated. The arc 14 corresponds to the word candidate “close”, and this language score L14 is calculated as usual, but the acoustic score is substituted by the acoustic score A10 of the word candidate “close” corresponding to the arc 10. doing. That is, the acoustic score of the word candidate “close” corresponding to the arc 14 is not calculated.

  With the above processing, it is possible to reduce the calculation amount of the speech recognition processing. In the case of the above example, conventionally, (open the window), (open the window), (open the window), (open the door), (open the door), and (open the door) 6 While the acoustic score is obtained for each of the two combinations, the matching unit 58 needs to obtain the acoustic score only for the combination having the highest initial value (open the window). The number of calculations can be reduced to 1/6.

  In the conventional case, acoustic scores must be calculated for all combinations of word string hypotheses “window” and “door” and word candidates “open”, “open”, “open”, and “close”, respectively. Since it had to be done, the acoustic score is calculated 8 times in total. On the other hand, the matching unit 58 performs the word candidate set “Shime” including the word candidate “Close” once for the word candidate set “Open” including the word candidates “Open”, “Dawn”, and “Open”. It is only necessary to calculate once for each, a total of two times. Therefore, the number of calculations can be reduced by 6 times obtained by subtracting the number of calculations 2 times in the case of FIG. 19 from the conventional number of calculations 8 times.

  When there is one word string hypothesis at the same time to connect the word candidate sets shown in FIG. 14, the number of calculations can be reduced by two times as described above, but the word candidate sets shown in FIG. 19 are connected. If there are two word string hypotheses at the same time, the number of calculations can be reduced by six times. For example, if there are three or more word string hypotheses at the same time connecting the word candidate sets, the number of reductions in the number of calculations compared to the prior art is even greater. That is, when the number of word string hypotheses at the same time connecting word candidate sets increases, the number of calculations can be further reduced. As a result, it is possible to greatly reduce the calculation amount of the speech recognition process. Further, for example, in step S3 in FIG. 7, a plurality of word string hypotheses at the same time are selected, and in the process in step S106 in FIG. 12, a plurality of word end point candidates that are representative of the word candidate set are specified. In this case, the calculation amount of the speech recognition process can be further greatly reduced.

  Returning to FIG. 7, the word connection relation information is sequentially updated based on the processing result of the matching unit 58 and further corrected by the re-evaluation unit 59 by the above-described processing. Therefore, the word preliminary selection unit 56 and the matching unit 58 can always perform processing using the word connection relation information.

  In addition, when updating the word connection relation information, the control unit 54 shares the terminal nodes as described above if possible.

  If it is determined in step S2 in FIG. 7 that there is no midway node, the process proceeds to step S9, and the control unit 54 refers to the word connection relation information, so that each path configured as the word connection relation information is obtained. By accumulating word scores, a final score is obtained, for example, a word string corresponding to an arc constituting a path having the largest final score is output as a voice recognition result for the user's utterance, and the process is terminated. .

  As described above, the word preliminary selection unit 56 selects one or more words following the already obtained word in the word string that is a candidate for the speech recognition result, and the word candidate clustering unit 57 selects homophones. Are classified into one set, and an acoustic score is calculated for the representative word candidates of the classified word candidate set in the matching unit 58, and other words included in the same word candidate set with the acoustic score. Candidate acoustic scores are substituted to form word strings that are candidates for speech recognition results. Then, the re-evaluation unit 59 corrects the word connection relationship between the words in the word string that is a candidate for the speech recognition result, and the control unit 54 uses the corrected word connection relationship as a word to be the speech recognition result. The column is confirmed. Therefore, highly accurate speech recognition can be performed while suppressing an increase in resources required for processing.

  That is, since the acoustic score of all word candidates selected by the word preliminary selection unit 56 is not calculated by the matching unit 58, the homophones are set as one set, and the acoustic score of the representative word is calculated. The amount of calculation can be reduced. In addition, since the word boundary of the word connection information is corrected in the re-evaluation unit 59, the time information of the node of interest is highly accurate in representing the word boundary. In the word preliminary selection unit 56 and the matching unit 58, Processing is performed using a feature amount sequence after the time represented by such highly accurate time information. Therefore, even if the criteria for selecting a word in the word preliminary selection unit 56 and the criteria for narrowing down the word in the matching unit 58 are strengthened, the possibility that a correct word is excluded as a speech recognition result is extremely reduced. be able to.

  When the criteria for selecting a word to be selected in the word preliminary selection unit 56 are strengthened, the number of words to be subjected to the matching process in the matching unit 58 is reduced. As a result, the amount of calculation required for the processing of the matching unit 58 and Memory capacity can also be reduced.

  In addition, the matching unit 58 diverts the acoustic score of the representative word of the word candidate set to other word candidates included in the same word candidate set, so the relationship with the immediately preceding word is not considered, and the acoustic score Is not necessarily accurate, but since the word boundary of the word connection information is corrected in the re-evaluation unit 59, the final acoustic score can be corrected to a more accurate value.

  Furthermore, in the word preliminary selection unit 56, even if a word starting from a certain time among the words constituting the word string as a correct speech recognition result is not selected at that time, it is deviated from that time. If the wrong time is selected, the re-evaluation unit 59 corrects the wrong time, and a word string as a correct speech recognition result can be obtained. That is, even if there is a selection omission of the word constituting the word string as the correct speech recognition result in the word preliminary selection unit 56, the re-evaluation unit 59 corrects the omission of selection and obtains the correct speech recognition result. A word string can be obtained.

  Accordingly, the re-evaluation unit 59 can correct an error in the word selection in the word preliminary selection unit 56 in addition to an error in the end time detection in the matching unit 58.

  By the way, when the speech recognition process is performed by the above-described method, the same process may be repeated many times during the process in the reevaluation unit 59. For example, it is assumed that word connection relation information corresponding to the graph structure shown in FIG. 20 is stored in the word connection relation storage unit 60. In FIG. 20, the word string hypothesis “open the door” is formed by arc 1, arc 2, and arc 3 that connect node 1, node 2, node 3, and node 4. The word string hypothesis “window” is formed by the arc 4 and the arc 5 that connect the node 1, the node 5, and the node 6. In FIG. 20, node 6 is an intermediate node. FIG. 21 shows an example in which the subsequent word candidate “open” is connected to the node 6 of FIG. 20 by the matching process. In the example of FIG. 21, arc 6 and arc 7 corresponding to the same word candidate “open” extend from node 6 to node 7 and node 8 having two different end times t1 and t2.

  As shown in the example of FIG. 21, when a plurality of nodes having different end times corresponding to the same word candidate are obtained, a re-evaluation process for the word strings from node 5 to node 7, Consider the reevaluation for word strings up to node 8. Focusing on these two re-evaluation processes, the re-evaluation start time is the same, and the word strings “open” and “open” to be re-evaluated are also the same. The only difference is the end time. That is, in this example, the same word string is re-evaluated from the same start time. In particular, as described above, since the end time is not fixed in the matching process in the matching unit 58, and a word hypothesis having a plurality of times as end times is generated, the same word string is re-started from the same start time. The situation where evaluation is performed frequently occurs.

  Therefore, in the speech recognition apparatus shown in FIG. 22, the re-evaluation process for the same word string from the same start time is performed more efficiently. FIG. 22 is configured by adding a reevaluation process storage unit 201 to the reevaluation unit 59 of the speech recognition apparatus of FIG. 5, and the same parts as those of the speech recognition apparatus of FIG. doing.

  With reference to FIG. 21, the reevaluation unit 59 and the reevaluation process storage unit 201 of the speech recognition apparatus in FIG. 22 will be described in detail. Now, when performing the matching process starting from the node 7 in FIG. 22, a reevaluation is performed on a partial word sequence that goes back in time from the node 7. Here, a case will be described in which reevaluation is performed by going back two words. In this example, the word string “open” and “open” is re-evaluated. Here, the start time is a time t0 corresponding to the node 5, and the end time is a time t1 corresponding to the node 7. In order to perform re-evaluation, a state transition model corresponding to the word string “open” and “open” is first constructed based on the acoustic model database 61 and the dictionary database 62. FIG. 23 shows this state transition model. As the acoustic model database 61, a three-state left-to-right phoneme HMM is used. At this time, phonemes HMMs “o”, “a”, “k”, “e”, “r”, and “u” corresponding to the word strings “open” and “open” are connected based on the pronunciation of the dictionary database 62, and the initial node 251 is connected. A state transition model having an end node 253 and a word boundary node 252 and transitioning between the respective nodes is configured. Nodes in the state transition model are indicated by circles, and arcs that transition from left to right in each node and arcs that self-transition in each node are indicated by solid lines connecting the nodes. The output probability density function is given to the three nodes corresponding to each phoneme, and the transition probability is given to the arc that transits between the nodes. Furthermore, a language score is given at a node that transitions to a word, that is, at an initial node 251 and a word boundary node 252. Using such a state transition model, a re-evaluation process based on a feature amount sequence from a start time t0 to an end time t1 is performed.

  In the re-evaluation process, first, an initial value score is given to the initial node 251, and then the score value in each node is updated while using the feature amount sequence in time order. The update of the score value is a process of accumulating an acoustic score and a language score based on dynamic programming. When the update of the score value up to the end time t1 is completed, an optimum route is searched as a state transition sequence from the initial node 251 to the end node 253, and based on the search result, the time t ′ ( The acoustic score given to “O” between the initial node 251 and the word boundary node 252 and the acoustic score given to “open” between the word boundary node 252 and the end node 253 are determined. As described above, the process of constructing the state transition model, updating the score value on the state transition model, searching for the optimum route, and determining the acoustic score of the word boundary time t ′ and “open” and “open”, This is a re-evaluation process. Then, the word boundary time t ′, the acoustic score A 5 ′ (not shown) given to “O”, and the acoustic score A 6 ′ (not shown) given to “open” are transmitted to the control unit 54, and the word connection relation memory is stored. The node and the arc corresponding to the word string that is the object of reevaluation stored in the unit 60 are corrected.

  Next, in the case of performing the matching process starting from the node 8 in FIG. 21, as in the case of the node 7, a re-evaluation process is performed on a partial word sequence that goes back in time from the node 8. However, the state transition model constructed at that time is the same as that described above corresponding to the node 7, and the update of the score value on the state transition model has a start time of t0. The same. Therefore, by updating the score value using the state transition model of FIG. 23 described above and continuing until time t1 until time t2, and searching for the optimum route, the word between node 5 and node 8 in FIG. The re-evaluation process for the column is completed, and the corresponding word boundary time and the acoustic score of each word can be determined.

  Therefore, the re-evaluation unit 59 in FIG. 22 performs the state evaluation model constructed at the time when the re-evaluation processing on the word string between the node 5 and the node 7 is completed, and the score value of each node at the time t1, and The update history of the score value necessary for searching for the optimum route is stored in the reevaluation process storage unit 201 as reevaluation process data. Then, when performing the reevaluation process on the word string between the node 5 and the node 8 in FIG. 21, the reevaluation process data up to time t1 stored in the reevaluation process process storage unit 201 is used, and the time t1 The re-evaluation process is restarted from and the score value is updated until time t2. As a result, the process for constructing the state transition model and the process for updating the score value from the start time t0 to the time t1 are shared.

  For example, as shown in FIG. 24, when a large number of hypotheses with different end times, such as node 3 to node 6, are generated for the same word W2 expanded from the same node 2, The re-evaluation process as the starting point has to be performed for each of the word strings of arc 2 to arc 5. However, in the speech recognition apparatus of FIG. 22, these re-evaluation processes are shared, and the result In other words, the amount of calculation is the same as when the reevaluation process is performed once for the word string from node 1 to the node 6 with the latest end time. That is, it can be considered that a re-evaluation processing result for another end time is obtained in the middle of the re-evaluation processing for the word string between the node 1 and the node 6. That is, the reevaluation process for the word string between the node 1 and the node 6 is stopped halfway and stored in the reevaluation process storage unit 201 so that the process can be continued as necessary. .

  Since the process can be continued while being stored in the reevaluation process storage unit 201, the process can be omitted if not necessary. For example, if a process called so-called hypothesis branching that rejects hypotheses for word string hypotheses with poor score values is performed, not all word string hypotheses are re-evaluated, so only when necessary It is important to be able to handle re-evaluation.

  In addition, in order to refer to the re-evaluation process data stored in the re-evaluation process process storage unit 201, the corresponding word string and time information, or the node and arc information stored in the word connection relation storage unit 60 are used. It can be easily realized. Further, in the case where the matching process is to be continued from the node stored in the word connection relation storage unit 60 in the order of the time held by the node, the re-evaluation unit 59 performs the reevaluation. The processing is also performed in order of such time.

  Therefore, for example, in FIG. 21, when the re-evaluation process is first performed on the word string between the node 5 and the node 7, the re-evaluation unit 59 stores the word connection relation storage unit 60 when the process ends. The node stored in is searched, and the node that uses the reevaluation process data next is searched. In the example of FIG. 21, this is the node 8 that differs only in the end time. If such a node is found, the re-evaluation unit 59 stores the re-evaluation process data in the re-evaluation process process storage unit 201 and at the same time establishes a link from the retrieved node. If the node is not found, the re-evaluation process is not shared, and the re-evaluation unit 59 does not store the re-evaluation process data in the re-evaluation process process storage unit 201. In this way, not only can the storage capacity of the re-evaluation process storage unit 201 be reduced, but it is also easy to refer to the re-evaluation process data stored in the re-evaluation process storage unit 201 by using a link. Will be able to do it. In other words, if there is a link, the process is continued using the re-evaluation process data to which the link is attached, and if there is no link, the re-evaluation process is performed from the time of the first node.

  By doing so, it is possible to further reduce the amount of calculation required for the speech recognition processing.

  By the way, in the conventional speech recognition apparatus, the matching process for the same word may be repeated many times in the matching unit 58 with different times as start times. For example, in FIG. 20, it is assumed that a word candidate “open” whose start time is the time corresponding to the node 3 is determined by the word preliminary selection unit 56. On the other hand, it is assumed that the word candidate “open” having the time corresponding to the node 6 as the start time is determined by the word preliminary selection unit 56. In such a case, the matching process is performed on the word candidate set “open” including the same word candidate “open”, with two different times corresponding to the nodes 3 and 6 as the start times. For this reason, when the state transition model necessary for the calculation for score calculation is dynamically constructed when performing the matching process, the same model is constructed many times, resulting in an increase in the processing amount.

  FIG. 25 shows an example of the configuration of a speech recognition apparatus that can prevent duplication of processing as described above. 25 has a configuration in which a word model storage unit 301 is added to the matching unit 58 of the voice recognition device of FIG. 5. The same parts as those of the voice recognition device of FIG. The code | symbol is attached | subjected.

  The operation of the speech recognition apparatus in FIG. 25 will be described with reference to FIGS.

  FIG. 26 shows an example of the graph structure of the word connection relation information stored in the word connection relation storage unit 60. In FIG. 26, consider a case where matching processing is performed with node 3 as a starting point. First, suppose that the word candidate set “open” corresponding to the word candidate “open” is given from the word candidate clustering unit 57 with the time ta corresponding to the node 3 as the start time. At this time, when the initial value of the score of the word candidate “open” is the highest, a state transition model (hereinafter also referred to as a word model) corresponding to the word candidate is configured based on the acoustic model database 61 and the dictionary database 62. Is done. FIG. 27 shows this word model. Assuming that a three-state left-to-right phoneme HMM is used as the acoustic model database 61, the phoneme HMMs “a”, “k”, “e”, “e” corresponding to the word candidate “open” based on the pronunciation of the dictionary database 62. “r” and “u” are connected, and an initial node 351 and an end node 352 are held, and a state transition model for transitioning between the respective nodes is configured. The description of the nodes and arcs in the state transition model is the same as the description for FIG.

  Using such a word model, score calculation is performed with time ta as the start time. In the score calculation, the initial value of the score is given to the node 351, and the score value at each node is updated while using the feature amount series after the time ta in time order. The update of the score value is a process of accumulating acoustic scores based on dynamic programming. Since the end time is not fixed, the score value of the end node 352 is checked at each time, and when the score value is good, that is, when the score is higher than a certain threshold, for example, from the initial node 351 to the end node 352 A word hypothesis is generated with the accumulated score as the acoustic score. A word hypothesis corresponding to a plurality of end times may be generated for the same word candidate. The generated word hypothesis is transmitted to the control unit 54 and stored in the word connection relation storage unit 60. Further, the matching unit 58 in FIG. 25 stores a word model (state transition model) corresponding to the word candidate “open” constructed for score calculation in the word model storage unit 301.

  Subsequently, when performing the matching process starting at the node 5 in FIG. 26, the word candidate clustering unit 57 gives the word candidate set “open” including the word candidate “open”, and the word candidate “open” score. The initial value is assumed to be the highest. In this case, the conventional speech recognition apparatus needs to construct a word model corresponding to the word candidate “open” again for score calculation. On the other hand, in the speech recognition apparatus in FIG. 25, the matching unit 58 reads and uses the word model corresponding to the word candidate “open” stored in the word model storage unit 301. By doing so, the process of constructing the word model is not necessary, and only the score calculation needs to be performed.

  As described above, regarding the construction of the word model performed in the matching process, the constructed model is stored in the word model storage unit 301, and the stored word model is reused in the subsequent matching process. Thus, it is possible to prevent an increase in the processing amount due to the word model corresponding to the same word being built many times. In particular, when matching processing for the same word with different start times frequently occurs, the efficiency increases.

  In addition, regarding the word model as described above, it is possible to remove the word model construction process from the matching process by creating a corresponding word model for all words included in the dictionary database 62 in advance. It is. However, in this case, it is necessary to store the word models for all the words, and it is necessary to provide storage means having a large storage capacity. On the other hand, in the speech recognition apparatus of FIG. 25, when it becomes necessary, the word model is constructed, so that the storage capacity is small. In particular, when an acoustic model or the like that depends on the immediately preceding word is used, a word model that depends on the immediately preceding word is constructed, so that the difference in storage capacity is large.

  By the way, regarding the word model stored in the word model storage unit 301, the storage capacity of the word model storage unit 301 continues to increase only by storing the constructed word model. Therefore, for example, when the speech recognition process is completed in units of utterances, the word model stored in the word model storage unit 301 is deleted from the word model storage unit 301, or the number of stored word models is constant. If the number exceeds the number, the word model stored in the word model storage unit 301 can be deleted from the word model storage unit 301 to prevent the storage capacity from increasing excessively.

  By the way, in the matching unit 58, when matching processing for a word candidate having a certain time as a start time is completed and matching processing for a word candidate having a different time as a start time is performed, an audio signal input in real time is obtained. When processing is to be performed, there arises a problem that the other matching process cannot be started unless an audio signal up to the time required to complete one matching process is input.

  For example, in FIG. 20, a case is considered where matching processing for “open” is performed with node 3 as a starting point, and processing is performed until time T. In consideration of processing the input voice in real time, the matching process starting from the node 3 is not completed unless the voice input up to time T is completed. On the other hand, when focusing on the matching process starting from the node 6, the matching process can be started when the voice after time t is input even if the voice input up to time T is not completed. . Therefore, it is desirable that the matching process starting from the node 6 can be started even if the matching process starting from the node 3 is not completed.

  FIG. 28 shows an example of the configuration of a speech recognition apparatus that can process input speech in real time. FIG. 28 shows a configuration in which a matching process storage unit 401 is added to the matching unit 58 of the speech recognition apparatus of FIG. 5, and the same parts as those of the speech recognition apparatus of FIG. It is.

  The operation of the speech recognition apparatus in FIG. 28 will be described with reference to FIG. In FIG. 26, consider a case where matching processing is performed with node 3 as a starting point. Here, it is assumed that a word candidate set is given from the word candidate clustering unit 57 to the matching unit 58 using the time ta corresponding to the node 3 as a start time. At this time, as matching processing, processing such as construction of a word model, update of a score value on the word model, and generation of a word hypothesis is performed in the same manner as in the speech recognition apparatus of FIG.

  By the way, when processing the input voice in real time, even if the matching process is performed with the time ta as the start time, the voice after the time ta may not be completely used. For example, when the matching process is necessary until time T, the matching process cannot be completed in a situation where only the sound up to time tc is input. Therefore, the matching unit 58 in FIG. 28 updates the score value on the word model using only the feature quantity series up to time tc, and the constructed word model and each state on the model are updated. The score value and the update history of the score value are stored in the matching process storage unit 401 as matching process process data.

  Subsequently, the matching unit 58 performs a matching process starting from the node 5 in FIG. Here, a word candidate set is given from the word candidate clustering unit 57 with the time tb corresponding to the node 5 as the start time. Regarding the word candidate with the highest initial value of the first score in the word candidate set, a process of constructing a word model, updating a score value on the word model, and generating a word hypothesis is shown in FIG. This is the same as the case of 26 nodes 3.

  Since voice is input even during this processing, the end time of the input voice shifts to the right (newer time) from tc in the figure. Assume that the end time of the input voice is shifted to time td. Also in the matching process of the word connected to the node 5, if the matching process up to the time T is necessary, the matching process cannot be completed at the time td. The score value of each of the above states and the update history of the score value are stored in the matching process storage unit 401 as matching process process data. Here, with respect to the matching process starting from the node 3, since the matching process is performed only until the time tc, the process can be advanced to the time td. Therefore, the matching unit 58 uses the matching process data stored in the matching process storage unit 401 to restart the matching process starting at the node 3 from time tc. As described above, the matching process for the portion after node 3 and the portion after node 5 are alternately advanced according to the input voice.

  As described above, according to the input of the voice, after proceeding within the range in which the matching process can be advanced, the process is temporarily stopped and stored, and if necessary, the matching process can be continued. The processing speed can be increased when processing the input voice in real time.

  In the above description, the voice recognition devices of FIGS. 22, 25, and 28 have been described independently. However, the voice recognition devices are not shown in FIG. 22, FIG. 25, and FIG. It is also possible to have any two or all three of the evaluation process storage unit 201, the word model storage unit 301, and the matching process storage unit 401.

  In addition, regarding a state transition model used in the re-evaluation process and a word model (state transition model) used in the matching process, when focusing on a certain word, the state transition model may be the same. In such a case, since the state transition model itself does not depend on time, it may be used in common. That is, the word model stored in the word model storage unit 301 of FIG. Thereby, it is possible to improve the efficiency of the process of building the state transition model in the re-evaluation unit 59.

  As described above, the word candidate clustering unit 57 clusters the word candidates determined by the word preliminary selecting unit 56, and the matching process is performed only once for each obtained word candidate set. The amount can be greatly reduced. The matching process does not depend on the number of immediately preceding word string hypotheses. Further, in the reevaluation unit 59, the acoustic model depending on the immediately preceding word string hypothesis is applied, and the word boundary and acoustic score determined in the matching process are correctly corrected. Almost never occurs. That is, the processing speed can be increased without reducing the accuracy of voice recognition.

  Actually, as a result of applying the method of the present embodiment to large vocabulary continuous speech recognition with a vocabulary number of 60,000, it was confirmed that the processing amount required for the matching processing was reduced to 1/10 or less. In this experiment, the partial word string hypotheses to be noted at each time are limited to about 30, and the number of candidates extracted by word preliminary selection is about 1000 words as word candidates to be developed from the word string hypotheses. In this case, a comparison was made with and without applying the method of the present embodiment.

  Conventionally, when focusing on a certain time, assuming that the number of word candidates is 1000 words and the partial word string hypothesis is about 30, the matching process is performed 30000 times. Is no longer dependent on the number of word string hypotheses, so that it can be suppressed to at least 1000 times. Further, matching processing for the same pronunciation is also reduced, and in the case of Japanese with many homonyms, processing reduction of about several percent to 10 percent is expected. That is, it is expected to be 1/30 or less. However, in the process of matching processing, the processing of hypothesis pruning called so-called beam search was performed, so that the amount of processing of each matching was originally suppressed to some extent. It seems that it was reduced.

  Further, as in the speech recognition apparatus of FIG. 22, a re-evaluation process is provided for a re-evaluation process for the same word string having the same start time but different end times. By providing means for temporarily stopping and storing the process and means for continuing the process using the stored process for reevaluation, it is possible to increase the efficiency of the process required for reevaluation.

  In addition, as in the speech recognition apparatus of FIG. 25, a means for storing a word model for score calculation constructed for performing score calculation in matching processing is provided, and when score calculation for the same word is necessary, By using the stored word model, sharing (efficiency) of processing required for word model construction is realized.

  Furthermore, like the speech recognition apparatus of FIG. 28, it is provided with means for temporarily stopping and storing the matching process, and means for continuing the process using the stored matching process, so that input is performed in real time. Since it becomes possible to proceed with the matching process for the audio signal to be processed, it is possible to effectively use the calculation resources.

  As a result, high speed speech recognition processing is realized.

  Here, the acoustic model database 61, the dictionary database 62, and the grammar database 63 have been described as being shared by the word preliminary selection unit 56, the matching unit 58, and the reevaluation unit 59. It is also possible to use different acoustic model databases, dictionary databases, and grammar databases. For example, as the grammar, the word preliminary selection unit 56 can use a unigram, the matching unit 58 can use a bigram, and the reevaluation unit 59 can use a trigram.

  In the above description, the word candidate clustering unit 57 clusters the word candidates (sound words) having the same pronunciation (classified based on the attribute of “words having the same pronunciation”) as an example. As described above, even if the pronunciation is not exactly the same, a distance scale is defined that shows the acoustic similarity between words, and the distance scale is smaller than the set threshold, that is, acoustic similarity It is also possible to further reduce the amount of matching processing by clustering high-frequency word candidates (classifying on the basis of an attribute of “acoustically similar words”).

  Note that the present invention can be applied to, for example, home or business game machines, mobile phones, mobile terminal devices, and other electrical appliances.

  The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, various functions can be executed by installing a computer in which the programs that make up the software are installed in dedicated hardware, or by installing various programs. For example, it is installed from a recording medium or the like into a general-purpose personal computer or the like.

  FIG. 29 is a diagram showing an example of the internal configuration of a personal computer 500 that executes such processing. A CPU (Central Processing Unit) 501 of the personal computer executes various processes according to a program stored in a ROM (Read Only Memory) 502. A RAM (Random Access Memory) 503 appropriately stores data and programs necessary for the CPU 501 to execute various processes. An input unit 506 including a mouse, a keyboard, a microphone, an AD converter, and the like is connected to the input / output interface 505, and a signal input to the input unit 506 is output to the CPU 501. The input / output interface 505 is also connected to an output unit 507 including a display, a speaker, a DA converter, and the like.

  Further, a storage unit 508 configured from a hard disk or the like and a communication unit 509 that performs data communication with other devices via a network such as the Internet are connected to the input / output interface 505. The drive 510 is used when data is read from or written to a recording medium such as the magnetic disk 521, the optical disk 522, the magneto-optical disk 523, and the semiconductor memory 534.

  As shown in FIG. 29, a program storage medium for storing a program that is installed in a computer and can be executed by the computer includes a magnetic disk 521 (including a flexible disk), an optical disk 522 (CD-ROM (Compact Disk- A package medium consisting of a read only memory), a DVD (Digital Versatile Disk), a magneto-optical disk 523 (including an MD (Mini-Disk)), or a semiconductor memory 524, or a program temporarily or permanently. A ROM 502 to be stored, a hard disk constituting the storage unit 508, and the like are configured. The program is stored in the program storage medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via an interface such as a router or a modem as necessary.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

It is a block diagram which shows the structural example of the conventional speech recognition apparatus. It is a figure explaining a bundle search method. It is a figure which shows the example of the word connection relation information by a graph structure. It is a figure which shows the example of the word connection relation information by a graph structure. It is a block diagram which shows the structural example of the speech recognition apparatus to which this invention is applied. It is a figure which shows the example of the word connection relationship information memorize | stored in the word connection relationship memory | storage part. It is a flowchart explaining the speech recognition process of the speech recognition apparatus of FIG. It is a figure explaining a reevaluation process. It is a figure following FIG. 8 explaining a reevaluation process. It is a figure following FIG. 9 explaining a reevaluation process. It is a figure following FIG. 10 explaining a reevaluation process. It is a flowchart explaining the process of FIG.7 S7 in detail. It is a figure explaining the example of the combination of the initial value of a score. It is a figure which shows the example of the word connection relationship information memorize | stored in the word connection relationship memory | storage part. It is a figure which shows the example of a word model (state transition model). It is a figure which shows the example of the word connection relationship information memorize | stored in the word connection relationship memory | storage part. It is a figure which shows the example of the word connection relationship information memorize | stored in the word connection relationship memory | storage part. It is a figure explaining the example of the combination of the initial value of a score. It is a figure which shows the example of the word connection relationship information memorize | stored in the word connection relationship memory | storage part. It is a figure which shows the example of the word connection relationship information memorize | stored in the word connection relationship memory | storage part. It is a figure which shows the example of the word connection relationship information memorize | stored in the word connection relationship memory | storage part. It is a block diagram which shows the other structural example of the speech recognition apparatus to which this invention is applied. It is a figure which shows the example of a word model (state transition model). It is a figure which shows the example of the word connection relationship information memorize | stored in the word connection relationship memory | storage part. It is a block diagram which shows the other structural example of the speech recognition apparatus to which this invention is applied. It is a figure which shows the example of the word connection relationship information memorize | stored in the word connection relationship memory | storage part. It is a figure which shows the example of a word model (state transition model). It is a block diagram which shows the other structural example of the speech recognition apparatus to which this invention is applied. It is a block diagram which shows the structural example of the personal computer to which this invention is applied.

Explanation of symbols

  51 microphone, 52 AD conversion unit, 53 feature extraction unit, 54 control unit, 55 feature quantity storage unit, 56 word preliminary selection unit, 57 word candidate clustering unit, 58 matching unit, 59 re-evaluation unit, 60 word connection relation storage unit , 61 Acoustic model database (DB), 62 Dictionary database (DB), 63 Grammar database (DB), 201 Re-evaluation process storage unit, 301 Word model storage unit, 401 Matching process storage unit

Claims (12)

In a speech recognition device that determines a word string corresponding to an input speech,
A matching process in which a part of the processing is shared for each word group in which one or more word candidates following the already obtained word in the word string are classified for each word having the same attribute. Processing execution means;
Correction means for correcting word connection relation information indicating a connection relation between words in the word string, which is a candidate for a speech recognition result, generated based on a result of matching processing by the matching processing execution means. Voice recognition device. Classifying means for classifying one or more word candidates following the already obtained word in the word string as the word group for each word having the same attribute,
The speech recognition apparatus according to claim 1, wherein the matching process execution unit shares a part of the process for each word group classified by the classification unit. A selection means for selecting one or more candidates for the word following the already requested word from the word string;
The speech recognition according to claim 2, wherein the classifying unit classifies the one or more word candidates selected by the selection unit as the word group for each word having the same attribute. apparatus. The speech recognition apparatus according to claim 1, wherein one or more word candidates subsequent to the already obtained word are classified into the word group for each word having the same pronunciation. The speech recognition apparatus according to claim 1, wherein one or more word candidates following the already obtained word are classified into the word group for each of the acoustically similar words. A storage unit that stores the word connection relation information generated based on the result of the matching process performed by the matching process execution unit;
The speech recognition apparatus according to claim 1, wherein the correction unit corrects the word connection relation information stored by the storage unit. When a state transition model for calculating an acoustic score of a partial word string is constructed by the modifying means to modify the word connection relation information, the constructed state transition model is stored. A storage means,
The said correction means utilizes the said state transition model memorize | stored by the said memory | storage means, when calculating the said acoustic score of the said same partial word sequence again. Voice recognition device. In the case where a state transition model for calculating an acoustic score of a word is constructed by the matching processing execution unit, the matching processing execution unit further includes a storage unit that stores the constructed state transition model,
The speech recognition according to claim 1, wherein the matching process execution unit uses the state transition model stored in the storage unit when recalculating the acoustic score of the same word. apparatus. A storage unit that stores a value calculated by the matching process execution unit as a mid-course of the matching process;
The matching processing execution means alternately executes the matching processing of a plurality of the word strings that are candidates for the speech recognition result using the value as the intermediate progress stored by the storage means. The speech recognition apparatus according to claim 1. In the speech recognition method of the speech recognition apparatus for determining a word string corresponding to the input speech,
A matching process in which a part of the processing is shared for each word group in which one or more word candidates following the already obtained word in the word string are classified for each word having the same attribute. Process execution steps;
A correction step of correcting word connection relationship information indicating a connection relationship between words in a word string that is a candidate for a speech recognition result, which is generated based on the result of the matching processing by the processing of the matching processing execution step. A feature of speech recognition. A program for a speech recognition device that determines a word string corresponding to an input speech,
A matching process in which a part of the processing is shared for each word group in which one or more word candidates following the already obtained word in the word string are classified for each word having the same attribute. Process execution steps;
A correction step of correcting word connection relationship information indicating a connection relationship between words in the word string, which is a candidate of a speech recognition result, generated based on the result of the matching processing by the processing of the matching processing execution step. A recording medium on which a computer-readable program is recorded. A computer that controls the process of determining a word string corresponding to the input speech,
A matching process in which a part of the processing is shared for each word group in which one or more word candidates following the already obtained word in the word string are classified for each word having the same attribute. Process execution steps;
A correction step for correcting word connection relationship information indicating a connection relationship between words in the word string, which is a candidate of a speech recognition result, generated based on the result of the matching processing by the processing of the matching processing execution step; A program characterized by that.

Images