JP6000094B2 - Speaker adaptation device, speaker adaptation method, and program - Google Patents

Description

  The present invention relates to a speaker adaptation device, a speaker adaptation method, and a program for a speech recognition device that receives a speech signal distorted by the influence of noise, reverberation, a communication path, or the like.

  Hereinafter, a conventional feature correction technique will be briefly described with reference to the speech recognition apparatus shown in FIG. FIG. 1 is a block diagram showing a configuration of a conventional speech recognition apparatus including a feature amount correction process.

  As shown in FIG. 1, the conventional speech recognition apparatus 9 includes a feature amount extraction unit 91, a feature amount correction unit 92, a correction acoustic model storage unit 93, a feature amount conversion unit 94, and a speech recognition decoder unit 95. A recognition acoustic model storage unit 96, a language model storage unit 97, and a pronunciation dictionary storage unit 98. The acoustic model storage unit 93 for correction stores an acoustic model dedicated to correcting acoustic features. The acoustic model storage unit 96 for recognition stores an acoustic model dedicated to speech recognition. The language model storage unit 97 stores language models. The pronunciation dictionary storage unit 98 stores a pronunciation dictionary. The speech recognition device 9 receives a speech signal distorted by the influence of noise, reverberation, a communication path, etc. as an input, and outputs a word or a word series representing the utterance content, that is, a speech recognition result. Hereinafter, the distorted speech is referred to as degraded speech, the degraded speech signal is referred to as a degraded signal, and the acoustic feature amount of the degraded speech is referred to as a degraded feature amount. The deterioration signal is input to the feature amount extraction unit 91. The feature quantity extraction unit 91 divides the degraded signal into short-time frames, converts the degraded signal of each short-time frame into an acoustic feature quantity, and outputs a time series of the acoustic feature quantity. Hereinafter, the time series of acoustic feature quantities is referred to as an acoustic feature quantity series or a feature quantity series. As the acoustic feature amount, a log mel frequency spectrum coefficient, a mel frequency cepstrum coefficient (MFCC), or the like is used. Hereinafter, the feature amount series of deteriorated speech is referred to as a deteriorated feature amount sequence. The deterioration feature amount series is input to the feature amount correction unit 92. The feature amount correcting unit 92 uses the acoustic model stored in the correction acoustic model storage unit 93 to correct the influence of distortion superimposed on the degradation feature amount of each short-time frame, and the corrected acoustic feature amount Output time series. The feature quantity correction unit 92 is constructed by feature quantity enhancement means such as VTS (Vector Taylor Series) enhancement (Non-Patent Document 1) and Algonquin (Non-Patent Document 2). Hereinafter, the acoustic feature quantity in which the influence of the distortion is corrected is referred to as a corrected feature quantity, and the time series of the corrected feature quantity is referred to as a corrected feature quantity series. The corrected feature quantity series output from the feature quantity correction unit 92 is input to the feature quantity conversion unit 94. The feature amount conversion unit 94 converts the corrected feature amount series into a time series of feature amount expression used by the speech recognition decoder unit 95, and outputs this. Hereinafter, the feature quantity representation used by the speech recognition decoder unit 95 is referred to as a recognition feature quantity, and the time series of recognition feature quantities is referred to as a recognition feature quantity series. As the feature quantity for recognition, a MFCC connected to a delta cepstrum is used. The recognition feature value series output from the feature value conversion unit 94 is input to the speech recognition decoder unit 95. The speech recognition decoder unit 95 calculates a word or a word sequence that best fits the input recognition feature quantity series while referring to a speech recognition-dedicated acoustic model, language model, pronunciation dictionary, etc. Output as a result. In the figure, components (93, 96, 97, 98) indicated by cylindrical symbols are storage units that store parameters that define the model represented by the component, and the processing in which these parameters refer to the model. Read by the unit.

  In the above description, the feature amount correction technology has been described assuming application to a speech recognition device. However, the feature amount correction technology is not limited to a speech recognition device, but a speech recognition program, a noise suppression device, and a noise suppression program. Etc.

PJ Moreno, B. Raj, and RM Stern, “A vector Taylor series approach for environmental-independent speech recognition,” in Proc. Int. Conf. Acoust., Speech, Signal Process., Vol. 2, 1996, pp. 733 -736. B. J. Frey, L. Deng, A. Acero, and T. Kristjansson, “ALGONQUIN: iterating Laplace ’s method to remove multiple types of acoustic distortion for robust speech recognition,” in Proc. Eurospeech, 2001, pp. 901-904.

  As described above, the feature amount correction unit 92 uses an acoustic model that is used to correct the influence of distortion, which is different from that used by the speech recognition decoder unit 95. Hereinafter, this acoustic model is referred to as a correction acoustic model. The acoustic model dedicated to speech recognition referred to by the speech recognition decoder unit 95 is called a recognition acoustic model and is distinguished from the correction acoustic model. The acoustic model for correction is expressed using a mixed normal distribution and has a simpler structure than the acoustic model for recognition based on the hidden Markov model. The acoustic model for correction expresses the distribution of acoustic features in a short frame of clean speech that does not include distortion, and is learned in advance using a speech corpus.

  In order to increase the reliability of the speech recognition apparatus, it is desirable to learn the correction acoustic model using a large amount of user's voices recorded in advance. Such a correction acoustic model is called a specific speaker model. However, since it is actually difficult to collect a large amount of user's voice in advance, in many cases, the correction acoustic model is learned using the voice of one or more unspecified speakers. Such a correction acoustic model is called an unspecified speaker model. The unspecified speaker model also has a certain feature amount correction effect, but the speech recognition accuracy obtained by correcting the feature amount using the unspecified speaker model is inferior to the recognition accuracy obtained by the specific speaker model.

  On the other hand, speaker adaptation that modifies the “recognition” acoustic model of an unspecified speaker to match the characteristics of the user's speech using a small amount of speech data obtained from the user of the speech recognition device Technology is known. Specifically, the speaker adaptation technology receives as input a set of acoustic feature values extracted from the small amount of utterance data and a set of parameter values of a recognition acoustic model that defines an unspecified speaker model, The parameter values are modified to match the acoustic characteristics of the user, and a set of modified parameter values is output. A parameter value that defines a correction acoustic model before speaker adaptation, that is, an unspecified speaker model, is referred to as a pre-adaptation parameter value, and a parameter value after speaker adaptation is referred to as a post-adaptation parameter value.

  However, the conventional speaker adaptation technology for the “recognition” acoustic model cannot be used as it is for speaker adaptation of the “correction” acoustic model. For example, as a measure for collecting user's voice in advance, it is conceivable to use voice recorded when the user has used a voice recognition device in the past. However, since these voices actually contain distortion, the acoustic feature quantity used for speaker adaptation is a deteriorated feature quantity. Even if these deteriorated feature quantities are used for speaker adaptation of a correction acoustic model that expresses clean acoustic feature quantities, the speech recognition accuracy is not improved. Accordingly, an object of the present invention is to provide a speaker adaptation device for obtaining a correction acoustic model that is appropriately adapted to a speaker so that speech recognition accuracy can be effectively improved.

  The speaker adaptation apparatus of the present invention includes a frame selection unit and a parameter correction unit. The acoustic feature quantity of distorted speech is called a deterioration feature quantity. An acoustic model used for correcting the influence of distortion is referred to as a correcting acoustic model. A correction acoustic model learned using the voice of an unspecified speaker is called an unspecified speaker model. The frame selection unit receives a set of pre-recorded deterioration feature values, extracts a deterioration feature value with a low degree of deterioration from the set of deterioration feature values, and uses the extracted set of deterioration feature values as adaptation feature values. Output as a set. The parameter correction unit receives a set of feature values for adaptation and a set of pre-adaptation parameter values that are parameter values that define an unspecified speaker model, and adjusts the set of pre-adaptation parameter values to the feature values for adaptation. And a set of post-adaptation parameter values that are speaker-adapted parameter values are output.

  According to the speaker adaptation apparatus of the present invention, it is possible to obtain a correction acoustic model that is appropriately adapted to a speaker.

The block diagram which shows the structure of the conventional speech recognition apparatus containing a feature-value correction process process. 1 is a block diagram illustrating a configuration of a speaker adaptation device according to Embodiment 1 of the present invention. The flowchart which shows operation | movement of the speaker adaptation apparatus which concerns on Example 1 of this invention. FIG. 3 is a block diagram illustrating a configuration example of a frame selection unit according to the first embodiment. 5 is a flowchart illustrating an operation example of a frame selection unit according to the first embodiment. FIG. 3 is a block diagram illustrating a configuration example of a parameter correction unit according to the first embodiment. 5 is a flowchart illustrating an operation example of a parameter correction unit according to the first embodiment. The block diagram which shows the structure of the speaker adaptation apparatus which concerns on Example 2 of this invention. The flowchart which shows operation | movement of the speaker adaptation apparatus which concerns on Example 2 of this invention. FIG. 6 is a block diagram illustrating a configuration example of a frame selection unit according to the second embodiment. 9 is a flowchart illustrating an operation example of a frame selection unit according to the second embodiment. FIG. 6 is a block diagram illustrating a configuration example of a reverberation time estimation unit according to the second embodiment. 10 is a flowchart illustrating an operation example of a reverberation time estimation unit according to the second embodiment. The block diagram which shows the structure which combined the speaker adaptation apparatus with the speech recognition apparatus. The block diagram which shows the structure of the computer which implement | achieves the speaker adaptation apparatus of this invention.

Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function, and duplication description is abbreviate | omitted. Note that I, N _i , K, and N appearing below are all integers of 1 or more, and i, k, and n are all integers.

  A speaker adaptation apparatus according to Embodiment 1 of the present invention will be described below with reference to FIGS. FIG. 2 is a block diagram showing the configuration of the speaker adaptation apparatus according to Embodiment 1 of the present invention. FIG. 3 is a flowchart showing the operation of the speaker adaptation apparatus according to Embodiment 1 of the present invention.

The input to the speaker adaptation apparatus 1 is an acoustic feature value (acoustic feature value is a D-dimensional vector) obtained from a short-time frame of a user's recorded speech signal that has been recorded in advance, that is, a set of degraded feature values. Y = {y _{i (n)} ; 1 < i < I, 1 < n < N _i }, and a set Λ of pre-adaptation parameter values that define an unspecified speaker model. Here, I is the number of utterances included in Y, and each _Ni is the number of short-time frames included in the i-th utterance. In the present invention, it is assumed that the acoustic model for correction is expressed in the form of a mixed normal distribution. That is, Λ consists of a weighting coefficient ω _k (1 < k < K) for each normal distribution, an average vector m _k (1 < k < K), and an accuracy matrix R _k (1 < k < K), and Λ = { ω _k , m _k , R _k ; 1 < k < K}. Here, K is the number of normal distributions included in the mixed normal distribution. The output from the speaker adaptation device is a set Θ of post-adaptation parameter values that defines the corrected acoustic model for correction after the speaker adaptation process. Θ is the modified weighting factor ~ ω _k (1 < k < K), the modified mean vector ~ m _k (1 < k < K), the modified accuracy matrix ~ R _k (1 < k < K) And can be written as Θ = {~ ω _k , ~ m _k , ~ R _k ; 1 < k < K}.

As shown in FIG. 2, the speaker adaptation apparatus 1 of the present embodiment includes a frame selection unit 11 and a parameter correction unit 12. The degradation feature amount set Y input to the speaker adaptation apparatus 1 is input to the frame selection unit 11. The frame selection unit 11 selects a set of acoustic features X = {x _n ; 1 < n < N} that has a high SN ratio and can be used for speaker adaptation from Y, and outputs this (S11). Frame selection step). Since X is a subset of Y, X⊆Y, N < N ₁ + ... + N _I holds. Specifically, the frame selection unit 11 calculates the SN ratio value of each y _{i (n)} (1 < i < I, 1 < n < N _i ) or the index value correlated with the SN ratio. Then, only when the calculated value is larger than a predetermined threshold, it is determined that y _{i (n)} can be used for speaker adaptation of the correction acoustic model because the degree of deterioration is small, and y _{i (} Include _n) in X. The acoustic feature quantity included in X is called the adaptation feature quantity.

The adaptation feature amount set X output from the frame selection unit 11 is input to the parameter correction unit 12 together with the pre-adaptation parameter value set Λ. The parameter correction unit 12 uses one of the speaker adaptation means for the recognition acoustic model and adapts the set of parameter values included in Λ to the adaptation feature amount based on the set X. Thus, speaker adaptation is performed to calculate a set Θ of parameter values after adaptation (S12, parameter correction step). Θ thus obtained is output from the speaker adaptation apparatus 1. As speaker adaptation means that can be used in the frame selection unit 12, for example, the maximum posterior probability adaptation method (reference non-patent document 1) and the maximum likelihood linear regression method (reference non-patent document 2) are known.
(Non-patent document 1) J. Gauvain and C.-H. Lee, “Maximum a posteriori estimation for multivariate Gaussian mixture observation of Markov chains,” IEEE Trans. Speech Audio Process., Vol. 2, no. 2, pp 291-298, 1994.
(Reference Non-Patent Document 2) C. Legetter and P. Woodland, “Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models,” Comput. Speech Language, vol. 9, no. 2, pp. 171-185, 1995.

<Frame selection unit 11: configuration example assuming additive noise environment>
Hereinafter, a configuration example of the frame selection unit 11 will be described with reference to FIGS. 4 and 5. FIG. 4 is a block diagram illustrating a configuration example of the frame selection unit 11 of the present embodiment. FIG. 5 is a flowchart showing an operation example of the frame selection unit 11 of the present embodiment. As illustrated in FIG. 4, the frame selection unit 11 according to the present exemplary embodiment includes, for example, a noise estimation unit 111, an SN ratio estimation unit 112, and a first threshold processing unit 113. The configuration example of the frame selection unit 11 shown in FIG. 4 is assumed to be used in an additive noise environment, and does not use the pre-adaptation parameter value set Λ, and receives only the degradation feature amount set Y as an input.

The noise estimation unit 111 receives a set Y of deteriorated feature quantities as an input, and estimates and calculates a noise feature quantity d _{i (n)} that is an acoustic feature quantity of noise included in Y using a known noise estimation method. A set {d _{i (n)} ; 1 < i < I, 1 < n < N _i } of the obtained noise feature values is output. (SS111, noise estimation substep). As a noise estimation method, a method using speech segment detection (reference non-patent document 3), a method assuming that the first few frames of each utterance of degraded speech are composed of noise only (reference non-patent document 4), an IMCRA method (reference) Non-patent document 5) and the like are known.
(Reference Non-Patent Document 3) SF Boll, “Suppression of acoustic noise in speech using spectral subtraction,” IEEE Trans. Acoust., Speech, Signal Process., Vol. 27, no. 2, pp. 113-120, 1978.
(Non-patent document 4) Y. Ephraim, “Speech enhancement using a minimum mean square error short-time spectral amplitude estimator,” IEEE Trans. Acoust., Speech, Signal Process., Vol. 32, no. 6, pp. 1109-1121, 1984.
(Non-patent document 5) I. Cohen, “Noise spectrum estimation in adverse environments: improved minima controlled recursive averaging,” IEEE Trans. SAP, vol. 11, no. 5, pp. 466-475, 2003.

For example, in the second method, di _(n) is calculated according to the following equation.

However, each D _i is the number of frames at the beginning of the utterance that is assumed to consist only of noise. The SN ratio estimation unit 112 receives the deterioration feature quantity set Y and the noise feature quantity set {d _{i (n)} ; 1 < i < I, 1 < n < N _i } as inputs, and receives each acoustic feature quantity y _{i. For (n)} , the scalar value r _{i (n)} defined by the following equation is calculated, and the set of calculated SNRs {r _{i (n)} ; 1 < i < I, 1 < n < N _i } Output (SS112, SN ratio estimation substep).

Here, exp (·) and | · | represent the norm of the exponential function and the vector, respectively. This value is an estimated value of the SN ratio, and is simply referred to as an SN ratio hereinafter. The first threshold value processing unit 113 receives the degradation feature quantity set Y and the SN ratio set {ri _(n) ; 1 < i < I, 1 < n < _Ni } as inputs, and the corresponding SN ratio r _{i A} set of acoustic features y _{i (n)} where _(n) is greater than a predetermined threshold H == y _{i (n)} ; r _{i (n)} > H, 1 < i < I, 1 < n < N _i } And output the feature feature set X (SS113, first threshold value processing sub-step).

<Parameter Correction Unit 12: Configuration Example Using Maximum A posteriori Probability Adaptation Method>
Hereinafter, a configuration example of the parameter correction unit 12 will be described with reference to FIGS. 6 and 7. FIG. 6 is a block diagram illustrating a configuration example of the parameter correction unit 12 of the present embodiment. FIG. 7 is a flowchart showing an operation example of the parameter correction unit 12 of this embodiment. As shown in FIG. 6, the parameter correction unit 12 of the present embodiment includes, for example, a parameter initialization unit 121, a super parameter setting unit 122, a count initialization unit 123, a distribution rate calculation unit 124, and a parameter update unit. 125, a count increase means 126, and a convergence determination means 127.

  The configuration example of the parameter correction unit 12 shown in FIGS. 6 and 7 is a configuration example using the maximum posterior probability adaptation method. The details of the processing of the parameter correction unit 12 shown in FIGS. 6 and 7 are described in Reference Non-Patent Document 1 including the background concept, so only the processing flow is described in this specification. Note that the maximum a posteriori probability adaptation method is merely described as one specific example of speaker adaptation technology, and the parameter correction unit is used by using another speaker adaptation technology such as maximum likelihood linear regression. 12 can also be realized.

The parameter initialization means 121 initializes the corrected parameters ~ ω _k (1 < k < K), ~ m _k (1 < k < K), ~ R _k (1 < k < K) (SS121, Parameter initialization sub-step). As an initialization method, for example, parameters of an unspecified speaker model can be used as in the following equation.

However, the initialization method is not limited to the above equation, and the corrected parameters may be initialized using any other method. Next, the super parameter setting means 122 sets the value of the super parameter (hyper parameter) of the prior distribution used for the maximum posterior probability adaptation (SS 122, super parameter setting sub-step). The Dirichlet distribution is used as the prior distribution of the weighting coefficient, and the normal-Wishart distribution is used as the combined prior distribution of the mean vector and the accuracy matrix. That is, the prior distribution g (ω ₁ ,..., Ω _K ) of the weighting coefficient and the combined prior distribution g (m _k , R _k ) of the average vector and the accuracy matrix are given by the following equations, respectively.

tr (·) represents a matrix trace. Further, (v _k , τ _k , μ _k , α _k , U _k ; 1 < k < K) is a super parameter that defines the prior distribution. These values are set as follows, for example.

However, the value of the super parameter is not limited to these and may be set freely. The maximum posterior probability adaptation method is based on the EM algorithm and includes iterative processing. Next, the count initialization unit 123 sets the repetition count C to 1 before starting the repetition process (SS123, count initialization sub-step). Next, the distribution rate calculation means 124 calculates a distribution rate c _kn defined by the following equation for 1 < k < K and 1 < n < N (SS124, distribution rate calculation substep).

Next, the parameter update unit 125 updates the corrected parameter according to the following equation (SS125, parameter update substep).

However, in the parameter update substep, it is not always necessary to update all modified parameters. For example, only the average vector corrected by equation (15) may be updated. The count increment means 126 calculates C = C + 1 and increments the repeat count by 1 (SS126, count increment substep). The convergence determination unit 127 determines whether or not the EM algorithm has converged. If the EM algorithm has not converged (SS127 NO), the process returns to the distribution ratio calculation substep (SS124). If it has converged (SS127 YES), the flow ends. The convergence determination can be performed based on, for example, whether or not the repetition count exceeds the threshold value C _max . However, the convergence condition is not limited to this, and it may be determined whether or not the convergence has occurred based on the amount of change of the parameter that has been changed repeatedly.

  Hereinafter, the speaker adaptation apparatus according to the second embodiment will be described with reference to FIGS. 8 to 11. FIG. 8 is a block diagram showing the configuration of the speaker adaptation apparatus 2 according to the second embodiment of the present invention. FIG. 9 is a flowchart showing the operation of the speaker adaptation apparatus 2 according to the second embodiment of the present invention. FIG. 10 is a block diagram illustrating a configuration example of the frame selection unit 21 according to the present embodiment. FIG. 11 is a flowchart showing an operation example of the frame selection unit 21 of the present embodiment.

  As shown in FIG. 8, the speaker adaptation device 2 of this embodiment includes a frame selection unit 21 and a parameter correction unit 12. Since the parameter correction unit 12 is the same as the parameter correction unit 12 of the first embodiment, description thereof is omitted. Hereinafter, the frame selection unit 21 that is different from the first embodiment will be described.

<Frame selection unit 21: configuration example assuming reverberation environment>
The frame selection unit 21 according to the present embodiment is assumed to be used in a reverberant environment, and receives a set Λ of pre-adaptation parameter values and a set Y of deteriorated feature values as inputs. The frame selection unit 21 selects a set of acoustic features X = {x _n ; 1 < n < N} that can be used for speaker adaptation from Y with a short reverberation time, and outputs this (S21). Frame selection step). Specifically, the frame selection unit 21 estimates the reverberation time of each utterance i (1 < i < I) using the input set Y of deteriorated feature quantities and the set of pre-adaptation parameter values Λ. Then, only when the estimated reverberation time is smaller than a predetermined threshold, y _{i (n)} is a deterioration feature amount with a small degree of deterioration, and is determined to be usable for speaker adaptation of the correction acoustic model, Include y _{i (n)} in X.

More specifically, as shown in FIG. 10, the frame selection unit 21 includes a reverberation time estimation unit 211 and a second threshold processing unit 212. The reverberation time estimation means 211 receives as input the set Λ of pre-adaptation parameter values and the set Y of deterioration features, and uses a known reverberation time estimation method to determine the deterioration feature amount {y _{i (1)} ,. · estimates y _{i (Ni)}} from the reverberation time T ₆₀ (i), the set of estimated reverberation time {T ₆₀ (i); outputting the 1 <i <I} (SS211 , reverberation time estimation Sub-step). The reverberation time estimation can be performed, for example, by using the reverberation time estimation method described in Reference Non-Patent Document 6 or using the reverberation correction method described in Reference Non-Patent Document 7.
(Reference Non-Patent Document 6) R. Ratnam, DL Jones, BC Wheeler, WD O'Brien Jr., CR Lansing, and AS Feng, “Blind estimation of reverberation time,” J. Acoustical Society of America, vol. 114, no. 5, pp. 2877-2892, 2003.
(Reference Non-Patent Document 7) Takuya Yoshioka, Tomohiro Nakatani, “Dynamic Static Approach for Estimating Highly Responsive and Accurate Strain Feature Models,” vol. 2011-SLP-89, no. 22, 2011.

Alternatively, instead of automatically estimating the reverberation time, a person may be allowed to hear the sound and determine the degree of reverberation. As the reverberation time becomes longer, the distortion superimposed on the voice increases, so the reverberation time can be regarded as an index having an inverse correlation with the SN ratio. Therefore, the second threshold processing means 212 receives the degradation feature quantity set Y and the reverberation time set {T ₆₀ (i); 1 < i < I} as inputs, and the corresponding reverberation time T ₆₀ (i) set of predetermined threshold R is smaller than acoustic features y _{i (n)} X =; seek _{{y i (n) T 60} (i) <R, 1 <i <I, 1 <n <n i}, computed The set X of feature values for adaptation is output (SS212, second threshold value processing sub-step).

<Reverberation time estimation means 211: configuration example using the reverberation correction method of Reference Non-Patent Document 7>
Hereinafter, a configuration example of the reverberation time estimation unit 211 will be described with reference to FIGS. 12 and 13. FIG. 12 is a block diagram illustrating a configuration example of the reverberation time estimation unit 211 of the present embodiment. FIG. 13 is a flowchart showing an operation example of the reverberation time estimation means 211 of the present embodiment.

The reverberation time estimation unit 211 is a configuration example that estimates the reverberation time T ₆₀ (i) by using the reverberation correction method of Reference Non-Patent Document 7. The reverberation correction method (reference non-patent document 7) estimates three types of parameters called attenuation rate b _i , distortion variance σ _i , and shift h _i, and the attenuation rate is closely related to the reverberation time. However, these parameters have the same dimensions as the acoustic feature y _{i (n)} . Therefore, the reverberation time T ₆₀ (i) is calculated through the attenuation rate b _i obtained using the method of Reference Non-Patent Document 7. Here, in order to simplify the notation, the index i representing the utterance is omitted. The index i is an integer of 1 or more. Further, it is assumed that a log mel filter bank is used as the acoustic feature quantity, and the accuracy matrix of the correction acoustic model is a diagonal matrix, that is, R _k = diag (r _k ) (1 < k < K). Each r _k represents a vector composed of diagonal components of the accuracy matrix, and diag (·) represents an operation for converting the vector into a diagonal matrix.

  As shown in FIG. 12, the reverberation time estimation unit 211 includes, for example, a variable initialization unit 2110, a count initialization unit 2111, a synthesis unit 2112, a coefficient calculation unit 2113, a processing branch unit 2114, and a first update unit. 2115, a second update unit 2116, a count increase unit 2117, a convergence determination unit 2118, and an attenuation rate conversion unit 2119.

  The variable initialization unit 2110 initializes the unknown variables of the attenuation rate b, distortion variance σ, and shift h (SS2110, variable initialization substep). Arbitrary values can be used as initial values of these variables. For example, the following initial values can be used.

The method is based on the EM algorithm and includes iterative processing. The count initialization unit 2111 sets the repetition count C to 1 before starting the repetition process (SS2111, count initialization substep). The synthesizer 2112 calculates the first coefficient ψ _{k, n} and the second coefficient υ _{k, n} defined by the following equations for all 1 < k < K and 1 < n < N (SS2112, synthesize Sub-step).

  Where Δ is a given positive integer, and f (•) and g (•) are functions defined by the following equations, respectively.

Note that multiplication, division, power calculation, and function calculation are applied to each vector element. The coefficient calculation unit 2113 performs the third coefficient ω _{k, n} , the fourth coefficient l _{k, n} and the fifth coefficient e _{k, n} defined by the following equations for all 1 < k < K and 1 < n < N. Is calculated (SS2113, coefficient calculation substep).

The process branching unit 2114 branches the process to the first update substep (SS2115) if the repetition count C is an odd number (SS2114 YES), and to the second update substep (SS2116) if it is an even number (SS2114NO).

  The first update unit 2115 updates the attenuation rate b and the distortion variance σ according to the following equation (SS2115, first update substep).

The second update unit 2116 updates the shift h according to the following equation (SS2116, second update substep).

The count increment unit 2117 calculates C = C + 1 and increments the repeat count by 1 (SS2117, count increment substep). The convergence determination unit 2118 determines whether or not the EM algorithm has converged, and if not converged (SS2118NO), returns to the synthesis substep (SS2112), and if converged (SS2118YES), the attenuation rate conversion substep ( The process proceeds to SS2119) (SS2118, convergence determination substep). The convergence determination can be made based on, for example, whether or not the repetition count exceeds a predetermined threshold C _max . However, the convergence condition is not limited to this, and it may be determined whether or not the convergence has occurred based on the amount of change in the attenuation rate that has been changed repeatedly. The attenuation rate conversion unit 2119 obtains the reverberation time T ₆₀ (i) based on the calculated attenuation rate b _i (hereinafter, the utterance index i is specified for clarity). Specifically, T ₆₀ (i) is calculated by the following equation using Q as a predetermined constant and avg (·) as an operation for obtaining the average of vector elements (SS2119, attenuation rate conversion substep).

<Frame Selection Unit Included in Speaker Adaptation Apparatus of Present Invention>
As described above, as a specific configuration example of the frame selection unit included in the speaker adaptation apparatus of the present invention, the frame selection unit 11 is disclosed in the first embodiment and the frame selection unit 21 is disclosed in the second embodiment. The frame selection unit 11 according to the first embodiment is characterized in that a degradation feature quantity set Y is input and a degradation feature quantity having a high SN ratio is output from the degradation feature quantity set Y as an adaptation feature quantity set X. . The frame selection unit 21 according to the second embodiment is characterized in that the degradation feature quantity set Y is input, and the degradation feature quantity having a short reverberation time is output from the degradation feature quantity set Y as the adaptation feature quantity set X. did. However, the frame selection unit included in the speaker adaptation apparatus of the present invention is not limited to the configuration disclosed in the first and second embodiments. The speaker adaptation apparatus of the present invention has a high SN ratio because the SN ratio varies greatly depending on the short-time frame due to the fact that the voice signal is non-stationary and the environment in which the voice is distorted varies. Focusing on the fact that there are almost clean short-time frames, out of the deterioration feature quantities including distortion, take out short-time frames with little distortion, that is, deterioration, and make corrections prepared in advance as an unspecified speaker model The idea is to adapt the acoustic model to the speaker. Therefore, the frame selection unit of the present invention is configured to extract a deterioration feature amount with a low degree of deterioration from the deterioration feature amount set Y, and output the extracted deterioration feature amount set as a set of adaptation feature amounts. It is sufficient to use parameters that indicate any degree of deterioration other than the SN ratio and reverberation time.

<Voice recognition experiment>
Hereinafter, the results of performing a speech recognition experiment on speech recorded in a reverberant environment by combining the speaker adaptation device 2 of the second embodiment of the present invention with the conventional speech recognition device 9 shown in FIG. 1 will be described. To do. FIG. 14 is a block diagram showing a configuration in which the speaker adaptation device 2 is combined with the speech recognition device 9. As shown in FIG. 14, parameters stored in the correction acoustic model storage unit 93 of the speech recognition device 9 can be read and written from the outside. As a result, the speaker adaptation device 2 reads a set of pre-adaptation parameter values initially stored in the correction acoustic model storage unit 93. Then, the speaker adaptation device 2 overwrites the generated acoustic parameter storage unit 93 with the set of post-adaptation parameter values. The frame selection unit 21 of the speaker adaptation device has the configuration as described in the second embodiment. The speech recognition device 9 and the speaker adaptation device 2 are realized by causing a computer to execute a speech recognition program and a speaker adaptation program that describe the procedure of processing executed by these devices.

  Speech recognition experiments were performed using the apparatus of FIG. The experiment used the learning, evaluation, and adaptation data sets from the 20,000-word Wall Street Journal database. The learning data set was used to learn the acoustic model for recognition and the acoustic model for correction before speaker adaptation. In order to simulate reverberant speech, the evaluation data set was used after convolution of each utterance included in the data set and an impulse response measured in advance. The evaluation data set consists of 8 speakers, and for each speaker, the adaptation data set includes personal adaptation data for 50 utterances. For the speech included in the adaptation data set, speaker adaptation data including reverberation was simulated and used by convolving impulse responses recorded in several different reverberation environments.

  As a result of the experiment, the word error rate without any feature correction was 92.14%. When speaker adaptation was not performed, that is, when feature value correction was performed using pre-adaptation parameter values, the word error rate improved to 54.93%. When speaker adaptation was performed and feature values were corrected using post-adaptation parameter values, the word error rate was further improved to 51.83%. This result shows the effectiveness of speaker adaptation of the correction acoustic model proposed in the present invention.

  Although the present invention has been described with reference to specific embodiments, the present invention is not necessarily limited to the above-described embodiments and examples. The present invention can be implemented in various forms within the scope of the technical idea already described. In addition, the various processes described above are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. Needless to say, other modifications are possible without departing from the spirit of the present invention.

  Further, the speaker adaptation apparatus of the present invention described above, for example, causes the recording unit of the computer 8 shown in FIG. 15 to read a program describing the processing contents of the functions that each apparatus should have, the arithmetic processing unit 81, and the output The processing functions are realized on a computer by operating the device 82, the input device 83, the storage device 84, and the like.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good.

  Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer). In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (3)

The acoustic feature quantity of the distorted speech including at least one utterance is defined as the degradation feature quantity.
The acoustic model used for correcting the influence of distortion is the acoustic model for correction,
When the acoustic model for correction learned using the voice of an unspecified speaker is an unspecified speaker model,
A set of degradation features recorded in advance is input, a degradation feature having a low degree of degradation is extracted from the degradation feature, and the extracted degradation feature is output as a set of adaptation features A frame selection unit to perform,
The set of feature values for adaptation and the set of pre-adaptation parameter values that are parameter values that define an unspecified speaker model are input, and the set of pre-adaptation parameter values is spoken to match the feature values for adaptation. And a parameter modification unit that outputs a set of post-adaptation parameter values that are speaker-adapted parameter values,
avg (·) represents an operation for obtaining an average of vector elements, i represents a natural number representing an index of an utterance included in the distorted speech, b _i represents an attenuation rate of the i-th utterance , and a predetermined constant is defined. It shall be expressed as Q,
The frame selection unit
Using a set of the inputted deterioration characteristic quantity and the set of the adaptive pre-parameter values, i-th speech degradation characteristic of the reverberation time T ₆₀ the _(i),

And estimated constant, the estimated reverberation time is extracted as the deterioration characteristic quantity degree is smaller the deterioration less than a predetermined threshold value deterioration characteristic quantity speaker adaptation apparatus. A speaker adaptation method performed by a speaker adaptation device, comprising:
The acoustic feature quantity of the distorted speech including at least one utterance is defined as the degradation feature quantity.
The acoustic model used for correcting the influence of distortion is the acoustic model for correction,
When the acoustic model for correction learned using the voice of an unspecified speaker is an unspecified speaker model,
A set of deterioration feature values recorded in advance is input, the deterioration feature amount having a small degree of deterioration is extracted from the set of deterioration feature values, and the set of extracted deterioration feature values is set as a set of feature values for adaptation. A frame selection step to output;
The set of feature values for adaptation and the set of pre-adaptation parameter values that are parameter values that define an unspecified speaker model are input, and the set of pre-adaptation parameter values is spoken to match the feature values for adaptation. A parameter modification step for outputting a set of post-adaptation parameter values that are speaker-adapted parameter values,
avg (·) represents an operation for obtaining an average of vector elements, i represents a natural number representing an index of an utterance included in the distorted speech, b _i represents an attenuation rate of the i-th utterance , and a predetermined constant is defined. It shall be expressed as Q,
The frame selection step includes:
Using a set of the inputted deterioration characteristic quantity and the set of the adaptive pre-parameter values, i-th speech degradation characteristic of the reverberation time T ₆₀ the _(i),

And estimated constant, speaker adaptation method of reverberation time the estimated extracts as the deterioration characteristic quantity degree is smaller the deterioration less than a predetermined threshold value deterioration characteristic quantity. A program for causing a computer to function as the speaker adaptation device according to claim 1 .

Images