JP3427500B2 - Membership calculation device and HMM device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice recognition device.

[0002]

2. Description of the Related Art HMM (Hidden Markov Model) has been widely used in the field of speech recognition. One of them is the HMM (F
VQ / HMM). IEICE Technical Report SP
93-27 (June 1993) has a synergistic and additive F
VQ / HMM is described, especially synergistic FVQ / H
It is worth noting that MM shows excellent performance.

FIG. 1A is a block diagram for explaining the general principle of FVQ / HMM.

Reference numeral 100 denotes a feature extraction unit, which converts an input voice to be recognized into a feature vector every 10 msec, for example. As the feature amount, for example, cepstrum and its regression coefficient are often used recently.

Reference numeral 101 is a vector quantizer, which converts the feature vector into a membership vector. Reference numeral 102 denotes a codebook, and the vector quantization is performed based on the information of this codebook.

The codebook 102 is obtained by clustering a training vector set for creating a codebook into M clusters, labeling each cluster, and storing a representative vector of each cluster in a searchable form by the label. . Here, the training vector set is obtained by using the feature extraction unit 100 or a feature extraction unit that operates in a similar manner as the feature vectors of various voices uttered in advance for codebook creation. It consists of the converted into. The representative vector is usually an average vector of each cluster.

The belonging degree vector is a vector having the degree of belonging of the feature vector at each time point to each cluster as an element, and the feature vector at the time point t is y.
_t, C ₁ the cluster, ..., and C _M, y if _t attribution degree for C _m and u _tm of the membership vector y _t is converted u _t = (u _t1, .. ., u _tM ) ^T. Hereinafter, in the present application, the vector is a vertical vector, and T represents transposition. There are various possible definitions of u _tm here, but C
_When the representative vector of _m is μ _{m and} the Euclidean distance between y _t and μ _m is d _tm = [(y _t −μ _m ) ^T (y _t −μ _m )] ^1/2 , for example,

[0008]

[Equation 1]

[JG Bezdek: “Pattern R
ecognition with Fuzzy Objective Function Algorith
m ", Plenum Press, New York (1981).).

In order to reduce the amount of calculation, actually, the degree of membership is not calculated for all the clusters, but is calculated for the kth smallest cluster from the cluster with the smallest d _tm . That is, the elements forming the degree-of-association vector u _t are the upper k (k-k
"nearest neighbor" is a value calculated by (Equation 1), and other values are set to 0.

Reference numeral 103 denotes an HMM storage unit which stores HMMs corresponding to respective recognition units w = 1, ..., W such as words and syllables to be recognized. Reference numeral 104 denotes a likelihood calculation unit, which calculates the likelihood of the input voice of each HMM, that is, the feature vector sequence y ₁ , ..., From the belonging degree vector sequence obtained at the output of the vector quantization unit. The degree of occurrence of y _T from each HMM is calculated.

Reference numeral 105 denotes a determination unit, which is used when the degree of occurrence of y ₁ , ..., Y _T from HMMw is L ^w.

[0013]

[Equation 2]

Is calculated, and w ^* is used as the recognition result. (FIG. 1) (b) is an explanatory view for explaining the principle of the HMM. q ₁ , ..., q _{J + 1} is a state, a _ij is a state i to a state j
The transition probability to, ω _i (y _t ) is the degree of occurrence of y _t in state i. If these symbols are used, the degree L that the series of feature vectors y ₁ , ..., Y _T is generated from this HMM is as shown in (Equation 3). However, X = (x ₁ , x ₂ , ..., x _{T + 1} =
q _{J + 1} ) is a sequence of states, assuming the final state J + 1,
Assume that state J + 1 is reached at time T + 1, and state J +
In the case of 1, no vector is generated. Also, π _i
Is the probability that the state is i at t = 1, and λ is the parameter set of the HMM.

[0015]

[Equation 3]

The likelihood calculating unit 104 calculates the likelihood L ^w corresponding to the recognition unit w in accordance with (Equation 3) for w = 1, ..., W. The above-mentioned ω _i (y _t ) Various HMMs are defined according to the definition method of. The problem here is F
In VQ / HMM, ω _i (y _t ) is defined in principle as follows. (1) In the case of synergistic FVQ / HMM

[0017]

[Equation 4]

The name synergistic comes from the latter expression (Equation 4). (2) Additive FVQ / HMM

[0019]

[Equation 5]

The name "additive type" comes from the expression (Equation 5). Here, b _im is the probability of occurrence of cluster m in state i, and u _tm is the degree of membership of y _t in cluster m. As described above, the addition or multiplication in (Equation 4) or (Equation 5) is actually performed only in the upper K cluster of the degree of membership, and in this case, (Equation 4) (Equation 5) becomes 6) It becomes like (Equation 7). However, for h (k), y _t is k
This is the second highest cluster.

[0021]

[Equation 6]

[0022]

[Equation 7]

For the actual likelihood calculation, the Viterbi method is often used instead of calculating (Equation 3) as it is, and it is common to use it in the form of addition after logarithmizing it. That is,

[0024]

[Equation 8]

## EQU1 ## Let L'be the likelihood. (Equation 8) can be efficiently calculated by dynamic programming. That is,

[0026]

[Equation 9]

Let φ _i (1) = log π _i and t = 2, ...,
Recursively for T + 1, i = 1, ..., J + 1,

[0028]

[Equation 10]

Is calculated as The recognition result is L
Since there is no big difference between using and L ',
The Viterbi method is generally used for recognition.
In the calculation of the recurrence formula of (Equation 9), log ω _j (y _t )
It is necessary for calculation in the case of a synergistic type, by storing the log b _im instead of b _im, by using the equation (6) the first half of the equation, log omega _j calculations (y _t) Since only the sum of products is required, it is most advantageous to use the synergistic type in terms of calculation amount.

[0030]

In the above conventional example, the degree of membership u _tm is calculated by ( _Equation 1). This is the objective function in fuzzy clustering

[0031]

[Equation 11]

To

[0033]

[Equation 12]

It is derived by the criterion of minimizing under the condition of. However, (Equation 11) is given in a descending direction, is not derived theoretically or experimentally, and cannot be said to be optimal as a model in the actual world.

[0035]

When the number of clusters is M and the observation vector is y, it consists of a hierarchical neural network, and each unit of its input layer is associated with each element of y, and the m-th layer of the output layer. The output of the number unit is the degree of membership of y in cluster m (= 1, ..., M) or the posterior probability of cluster m in y (hereinafter, this is also referred to as membership degree). A means of calculating the degree of belonging.

[0036]

When the number of clusters is M and the observation vector is y, each unit of the input layer of the hierarchical neural network corresponds to each element of y, and the output of the m-th unit of the output layer is the cluster of y. The degree of membership to m (= 1, ..., M) or the posterior probability of cluster m to y (hereinafter also referred to as the degree of membership).

[0037]

EXAMPLES When C _m is a cluster m, u _tm can be interpreted as a posterior probability P (C _m | y _t ), and FVQ / HMM is a posterior probability of a cluster defined by a degree of membership. Can be interpreted. Obviously, if this idea is applied and the posterior probability of C _m with respect to y _t can be obtained by some other method, this should be used as the degree of membership of FVQ / HMM.

On the other hand, the hierarchical neural network is
Each unit of the input layer is made to correspond to each element of the feature vector to be identified, and the m-th unit of the output layer is made to correspond to the cluster m, and a vector sequentially given to the input layer by using the learning vector set. On the other hand, if learning is performed so that the output of the unit in the output layer corresponding to the cluster to which the vector should belong becomes 1 and the outputs of other units become 0, when the unknown input y _t is input. It is known that the output obtained as the output of the m-th unit corresponds to P (C _m | y _t ). (Takio Kurita: “Method of determining the number of hidden layer units in a three-layer neural network based on the information criterion”), IEICE Transactions D-II, Vol.J73-D- II, No.
11, pp.1872-1878 (November 1990)). The present invention utilizes this fact.

Therefore, the model creating procedure is as follows. (1) Creating a codebook The training vector set is clustered and divided into M clusters. Labels 1, ..., M are assigned to each cluster,
By its label, the centroid of each cluster, ie
The representative vector of each cluster is stored in a searchable form. Specifically, the LBG algorithm or the like is used. (2) Learning of neural network The neural network to be used here is shown in (Fig. 3). This example is a case of three layers. They are called the input layer, the intermediate layer, and the output layer, respectively. ○ is called a unit, and the input / output characteristics are 1 in the input layer and 1 / [1 in the intermediate layer.
+ Exp (-θ)], 1 or 1 / [1 + exp (-
θ)] in many cases. Here, θ is the input level. 1 / [1 + exp (-θ)] is called a sigmoid function. The k-th unit of the u-th layer and the j-th unit of the u + 1-th layer are connected with a weight w ^u _k ^{u + 1} _j , and if the output of the k-th unit of the u-th layer is o ^u _k , then the u + 1-th unit The input of the j-th unit of the layer is i ^{u + 1} _j = Σ _k o ^u _k w ^u _k ^{u + 1} _j .

The learning of the neural network is to determine these weighting factors by giving a desired output as a teacher signal to the learning input vector.
This is a value that minimizes the squared error between the actual output and the desired output, and is obtained by iterative calculation. In the present invention, specifically, it is performed as follows.

When the output of the k-th unit of the output layer is o ^U _k , in the above clustering, o ^U _k = 1 (k = m), o ^U _k = for the learning vector of y _n εC _m. 0 (k ≠
m) to learn. However, n = 1, ..., N is a serial number attached to the training vector. At this time, the objective function to be minimized is as shown in (Equation 13). However,
δ _km is Kronecker δ, and when k = m, δ _km
= 1 and k ≠ m, δ _km = 0.

[0042]

[Equation 13]

The minimization of (Equation 13) is calculated by a known method called back propagation (omitted). (3) Learning of HMM In learning of HMM, initial probability π _i , state transition probability a _ij , cluster occurrence probability b _im are learning data (a set of feature vector sequences obtained corresponding to recognition units such as words). It can be performed by a well-known method called Baum-Welch (omitted). In this case, the difference from the conventional model of the present invention is that the calculation of u _tm in ( _Equation 4) to ( _Equation 7) is used as the output of the neural network for the input y _t .

When recognizing words 1, ..., W, a learning feature vector sequence group is obtained from the voiced speech corresponding to words 1 ,.

The recognition procedure is as follows. (1) Calculation of Likelihood A method of calculating the likelihood of HMM for the input speech will be described. FIG. 2 shows an embodiment of the likelihood calculation device according to the present invention. The HMM for which the likelihood should be calculated is already calculated as described above and is stored in the HMM storage unit 204. 20
Reference numeral 0 is a voice input terminal for inputting a voice to be recognized. Reference numeral 201 denotes a feature extraction unit, which converts the feature vector y _t every 10 msec (called a frame), for example. t is a frame number. As a feature quantity, LPC
Well-known ones such as a cepstrum and a cepstrum are used, and y _t is usually a vector of ten-odd dimension having these feature quantities as elements. 202 is a degree of membership calculation unit is constituted by a neural network of a hierarchical as described above, (posterior probability of each cluster for y _t) membership to each cluster of _{y t u tm = P (C} m | y _t ) is calculated. A likelihood calculating unit 203 calculates (Equation 9) based on the degree of membership. 205 is an output terminal of the calculated likelihood. (2) Recognition (FIG. 4) is an embodiment of the voice recognition device according to the present invention. Reference numeral 206 denotes an HMM storage unit, which has recognition units 1, ...,
A model corresponding to W, that is, HMM 1, HMM 2, ...,
HMM W is stored. Reference numerals 200 to 203 are blocks having the same functions as the blocks having the same numbers in (FIG. 2). However, in 203, the degree of membership calculator 2
HMM 1, HMM 2, ..., HM for output 03
MW The likelihoods of all models are calculated. A determination unit 207 calculates (Equation 2) when the likelihood for the recognition unit w is L ^w . A terminal 208 outputs the recognition result w ^* .

[0046]

As described above, since the neural network for calculating the posterior probability of clusters is directly learned from the actual data, y _t can be obtained in a more realistic manner.
The posterior probability of each cluster can be obtained, and the recognition rate is expected to improve.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of a conventional FVQ / HMM.

FIG. 2 is a block diagram showing an embodiment of likelihood calculation according to the present invention.

FIG. 3 is a diagram showing an embodiment of a voice recognition device according to the present invention.

FIG. 4 is a block diagram showing an embodiment of a voice recognition device according to the present invention.

[Explanation of symbols]

100 feature extraction unit 101 Vector quantizer 102 Codebook 103 HMM storage unit 104 Likelihood calculator 105 determination unit

Continuation of the front page (56) References Yasuo Tan, Toshiro Ejima, Proposal of a network Fuzzy Partition Model using multiple input / output elements and its basic properties, IEICE Technical Report [Pattern Recognition / Understanding], September 21, 1989, PRU89-45, p. 39-46 Eiichi Tsubo, Junichi Nakahashi, Synergistic FVQ / HMM, Technical Report of IEICE [Voice], Japan, June 18, 1993, SP93-27, p. 25-32 Eiichi Tsubo, Junichi Nakahashi, A Study on FVQ / HMM, Proceedings of the 1994 Autumn Meeting of the Acoustical Society of Japan, Japan, October 5, 1992, 2-1-2, p. 81-82 Junichi Nakahashi, Eiichi Tsubo, Effect of HMM Structure Reflection on Code Vector Creation, Proceedings of the 5th Spring Research Conference of ASJ, Japan, March 17, 1993, 2-4-3 , P. 23-24 Kato Yoshinaga, Sugiyama Masahide, Continuous Speech Recognition Using Fuzzy Partition Model, IEICE Technical Report [Speech], Japan, June 30, 1992, SP92-28, p. 31-37 Eiichi Tsubo, Junichi Nakahashi, Speech recognition with dissimilarity between feature vectors as dissimilarity between attribution vectors, IEICE Transactions D-II, Japan, December 1996, Vol. J79-D-II No. 12, p. 2039-2046 (58) Fields investigated (Int.Cl. ⁷ , DB name) G10L 15/10 G10L 15/14 G10L 15/16 G06F 15/18 560 JISST file (JOIS)

Claims (14)

(57) [Claims] 1. When the number of clusters is M and the observation vector is y, it is composed of a hierarchical neural network, each unit of its input layer is associated with each element of y, and the m-th (= 1, 1, The output of unit No ...., M) is defined as the degree of membership of y to cluster m or the posterior probability of cluster m to y (hereinafter, this is also referred to as the degree of membership) .
The degree of membership calculation means and the probability of occurrence of cluster m in b _m and y
When the degree of membership in cluster m is u _m , the degree of occurrence of y
Calculate (observed degree of y) as a function of b _m and u _m
Where u _m is calculated by the degree-of-attribute calculating means.
A device for calculating the degree of occurrence of observation vectors characterized by being transmitted
Place 2. The logarithm of the degree of occurrence is u _m of the logarithm of b _m .
Is calculated as a weighted arithmetic mean with
The observation vector generation degree calculation device according to claim 1, which is a characteristic . 3. The degree of occurrence is the weight of b _m , with u _m as the weight.
The calculation is performed as an attached geometric mean.
The observation vector generation degree calculation device described . 4. The degree of occurrence is a weight of b _m with a weight of u _m
2. The calculation is performed as an arithmetic mean with addition.
The observation vector generation degree calculation device described . 5. A hidden Markov model (HMM: Hidden Mar
kov Model), and the cluster m (=
1, ..., M) occurrence probability b _im , and the observation vector at time t.
Let _{k utl} be the degree of membership in the cluster m of y _t and u _tm .
Then, the occurrence degree of y _t in state i (y _t is observed
Degree) as the function of b _im and u _tm
Occurrence degree calculation means and observation vector series y ₁ , y ₂ , ..., y
_The degree of occurrence of _T from the HMM is calculated from the observation vector
Observation vector calculated using the calculation result of the degree calculation method
And u _tm is the number of clusters
Is M and the observation vector is y, the hierarchical neural
It consists of a network and each of its input layer units
It corresponds to each element of y, and the mth unit of the output layer
Of the output of y to the cluster m (= 1, ..., M) of y
Calculated by a device for calculating degree of belonging characterized by
Likelihood calculation device characterized by the following. 6. W HMs corresponding to recognition units 1, ..., W
M and a cluster in the state i of the hidden Markov model w
The probability of occurrence of m (= 1, ..., M) is b ^w _im , at time t
The observation vector is y _t , and the degree of membership of y _t in cluster m is u _tm
And the occurrence degree of y _t in the state i of HMM w
Calculate ( observed degree of y _t ) as a function of b ^w _im and u _tm
Observation vector generation degree calculation means to be issued and observation vector
Degree L ^{w that the} sequence y ₁ , y ₂ , ..., y _T is generated from HMM ^w
Using the calculation result of the observation vector generation degree calculation means
, W, 1, ..., W calculation of observation vector series
The maximum w among the degree-of-life calculation means and L ¹ , ..., L ^W
Result determination means for calculating and determining w as a recognition result
And u _tm has the number of clusters M and the observation vector y
And consists of a hierarchical neural network,
Each of its input layer units corresponds to each element of y
The output of the m-th unit in the output layer to the cluster m of y
Characterized by the degree of membership in (= 1, ..., M)
An identification feature characterized by being calculated by a degree of membership calculator
Intelligence device. 7. The logarithmic value of the degree of occurrence is u _m of the logarithmic value of b _m .
Is calculated as a weighted arithmetic mean with
The recognition device according to claim 6, which is used as a signature . 8. The degree of occurrence is a weight of b _m with a weight of u _m
7. The calculation is performed as a geometric mean with a weight.
The recognition device described . 9. The degree of occurrence is the weight of b _m , with u _m as the weight.
7. The calculation is performed as an arithmetic mean with addition.
The recognition device described . 10. The HMM of the recognition unit w in the state i
Probability of occurrence of raster m (= 1, ..., M) is b ^w _im at time t
The observation vectors in y _t , the degree of membership of y _t in cluster m
_Is u _tm , the degree of occurrence of y _t in state i (y _t
(Observed degree of) as a function of b ^w _im and u _tm
Observation vector generation degree calculation means and observation vector series
The degree of occurrence of y ₁ , y ₂ , ..., y _T from the HMM is described above.
Calculation using the calculation result of the observation vector generation degree calculation means
Observation vector sequence occurrence degree calculating means and recognition unit w
HMMs corresponding to
Of the observation vector sequence for learning of
Maximize the occurrence degree calculated by the liveness degree calculation means.
Parameters to learn the parameters of HMM w
And learning means, u _tm is M number of clusters, observation vector
A torque when you and y, or hierarchical neural network
Each of the units in its input layer to each element of y
The output of the m-th unit in the output layer
It is specified that the degree of belonging to the raster m (= 1, ..., M)
Characteristic that it is calculated by a device for calculating the degree of belonging
HMM creation device. 11. The logarithmic value of the degree of occurrence is the logarithmic value of b _m ,
Calculate as a weighted arithmetic mean with u _m as the weight
The HMM creating apparatus according to claim 10, wherein 12. The degree of occurrence is the weight of b _m , with u _m as the weight.
Calculated as a weighted geometric mean
10. The HMM creation device according to item 10 . 13. The degree of occurrence is the weight of b _m , with u _m as the weight.
The calculation is performed as a weighted arithmetic average.
10. The HMM creation device according to item 10 . 14. Clustering means for clustering a set of learning vectors, and when a learning vector belonging to the mth cluster is input, the output of the mth unit of the output layer is 1, and the output of units of other output layers is The membership degree calculation device according to claim 1, wherein the weighting coefficient is learned as 0.