CN107103914B - High-quality voice conversion method - Google Patents

Abstract

The invention discloses a high-quality voice conversion method, which comprises the steps of firstly replacing a K-Means algorithm in a traditional GMM (Gaussian mixture model) by a self-organizing clustering algorithm, realizing training and classification of speaker personality characteristic parameters (MFCC) with an EM (effective noise model) algorithm iterative loop, then carrying out training of bilinear frequency warping and amplitude companding to obtain a conversion function required by voice conversion, and then carrying out high-quality voice conversion by using the conversion function. Aiming at the correlation between the space distribution condition of the voice characteristic parameters and the Gaussian mixture model, the invention uses the iterative self-organizing clustering algorithm to realize the determination of the mixing degree, solves the problem that the Gaussian mixture model is inaccurate when the voice characteristic parameters are classified, combines the improved Gaussian mixture model with the bilinear frequency bending and amplitude companding, constructs a high-quality voice conversion system, and has practical value in the voice conversion field.

Description

A high-quality voice conversion method

technical field

The invention relates to the field of voice conversion, in particular to a high-quality voice conversion system and an implementation method.

Background technique

Voice conversion refers to changing the voice personality characteristics of the source speaker so that it has the voice personality characteristics of the target speaker, even if the voice spoken by one person sounds like the voice spoken by another person after conversion, while retaining the semantics. There are usually two indicators to measure the effect of speech conversion: similarity (the similarity between the converted speech and the personality characteristics of the target speaker's speech) and clarity (the sound quality of the converted speech).

Typical speech conversion methods include: statistical mapping method represented by Gaussian Mixture Model (GMM, Gaussian Mixture Model). The error between the target voice and the target voice is the smallest, so as to achieve better voice similarity conversion, but the sound quality after conversion is not ideal; the voice spectrum conversion based on formant mapping represented by Frequency Warping (FW, Frequency Warping) method, which utilizes the correlation between the physiological characteristics of the human vocal tract and the formant parameters, and achieves a good sound quality conversion effect, but the conversion effect on the speech similarity is unsatisfactory.

In order to improve the above problems, speech researchers have done a lot of work: Jian Zhihua uses the transition probability matrix of the target speech frame to describe the timing information of the speech frame, and uses the Viterbi search algorithm to find the best GMM component of each frame of speech, avoiding the traditional The speech conversion algorithm based on GMM causes the discontinuity between the spectrum frames caused by the loss of the timing information of the speech frames, and also reduces the over-smoothing of the speech spectrum caused by the weighted averaging, enhances the formants, and improves the traditional The performance of the speech conversion algorithm based on GMM; Zhang Bing of Soochow University considered the global variance when setting the objective function (error function), which improved the over-smoothing phenomenon; Jian Zhihua et al. introduced the concept of compressed sensing in the speech conversion system, An improved speech conversion algorithm is obtained. In this conversion model, the line spectrum pair parameter vectors of continuous multi-frame speech are compressed into short vectors, and these short vectors are used for speech training and conversion; Bi Xing adopts the frequency of formant binary mapping. The bending method improves the traditional frequency bending method, and the similarity of the converted speech is improved. The scholar Daniel Erro combines GMM and FW technology, and the converted speech obtained achieves a good balance in terms of speech similarity and sound quality. However, Daniel Erro uses GMM in speech conversion to perform soft classification training with fixed mixing degree of speech feature parameters. , which limits the improvement space of the speech conversion effect. The reason is that the statistical distribution of the speech feature parameters of different people is not considered, and the GMM mixing degree is closely related to the statistical distribution of the feature parameters.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a high-quality speech conversion method, which takes into account the difference in the statistical distribution of speech characteristic parameters of different people, and provides a method that can use the iterative self-organization algorithm ISODATA to convert the characteristic parameters according to the different target speakers. Initial clustering, and then combined with the EM algorithm to self-organize the classification of the changed feature parameters, and then combined with the subsequent BLFW+AS training to obtain the conversion function, realizing a high-quality speech conversion. The invention has good practical value and can be used in the fields of movie dubbing, voice translation, confidential communication and the like.

The technical scheme adopted by the present invention is divided into a training part and a conversion part, and the details are as follows:

1), the training part steps:

1-1) Obtain the parallel corpus of the source speaker and the target speaker;

1-2) Use AHOcoder speech analysis model to extract speech feature parameters and logarithmic fundamental frequency;

1-3) carry out DTW to the speech characteristic parameter in step 1-2);

1-4) Use the iterative self-organization algorithm (ISODATA) to replace the K-Means algorithm in the traditional GMM model, and perform GMM training with the feature parameters in the EM algorithm step 1-3) to obtain the GMM parameters λ, P(X|λ );

1-5) Use the posterior conditional probability matrix P(X|λ) in step 1-4) to perform Bilinear Frequency Warping plus Amplitude Scaling (BLFW+AS, BLFW+AS) training, Obtain the frequency bending factor α(x,λ) and the amplitude adjustment factor s(x,λ), where x refers to the spectral characteristic parameter of the speech, λ refers to the GMM model parameter, so as to construct the BLFW+AS conversion function; use the logarithmic base The mean and variance of the frequency establish the conversion function between the source speech pitch frequency and the target speech pitch frequency;

2), the conversion part of the steps:

2-1) Input the source speaker voice to be converted;

2-2) Use AHOcoder speech analysis model to extract characteristic parameters and logarithmic fundamental frequency;

2-3) Use the improved GMM and the parameter λ obtained during training to obtain the posterior conditional probability matrix

2-4) Substitute the frequency bending factor α(x, λ) and the amplitude adjustment factor s(x, λ) into the BLFW+AS conversion function to obtain the converted characteristic parameters;

2-5) Substitute the logarithmic fundamental frequency into the fundamental frequency conversion function obtained when training to obtain the converted logarithmic fundamental frequency;

2-6) Use the AHOdecoder speech synthesis model to synthesize the converted speech with the converted feature parameters and the logarithmic fundamental frequency.

Wherein: the number of Gaussian classifications described in steps 1-4) of the training part is determined according to the specific distribution of ISODATA of the speaker's speech feature parameters.

The frequency bending factor and amplitude adjustment factor in steps 1-5) of the training part are obtained through BLFW+AS training according to the posterior conditional probability matrix obtained by ISODATA+GMM training.

beneficial effect

1. The present invention uses the combination of improved GMM and BLFW+AS to realize a speech conversion system, which can self-organize and adjust the number of components of GMM according to the distribution of speech feature parameters of different speakers, thereby realizing high-quality speech conversion.

2. The present invention realizes a complete high-quality voice conversion system, so it has practical effects in the application scenario of voice conversion.

Description of drawings

FIG. 1 is a schematic diagram of the training and conversion of the present invention.

FIG. 2 is a flow chart of the realization of the speech feature parameter clustering algorithm involved in the present invention.

FIG. 3 is a comparison of the MCD value after the speech conversion between the improved GMM and BLFW+AS combined speech conversion system involved in the present invention and the traditional GMM and GMM+BLFWAS.

FIG. 4 is a comparison of the MOS value after the voice conversion between the improved GMM and BLFW+AS combined voice conversion system involved in the present invention and the traditional GMM and GMM+BLFWAS.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings.

The high-quality voice conversion method of the present invention is divided into two parts: the training part is used to obtain parameters and conversion functions required for voice conversion, and the conversion part is used to convert the source speaker's voice into the target speaker's voice.

1), as shown in Figure 1, the implementation steps of the training part:

1-1) Obtain the parallel corpus of the source speaker and the target speaker, and the open source CMUARCTIC corpus of Carnegie Mellon University can be used to obtain the parallel corpus;

1-2) the present invention uses AHOcoder speech analysis model to extract the speech Mel cepstral coefficient (MFCC, Mel-Frequency Cepstral Coefficient) and logarithmic fundamental frequency parameter logf ₀ of source speaker and target speaker respectively;

1-3) Perform channel length normalization (VTLN, VocalTract Length Normalization) and dynamic time warping (DTW, Dynamic Time warping) on the MFCC parameters of the source and target speech in step (2);

1-4) Establish the ISODATA+GMM model, use the expectation maximization (EM, Expectation-Maximization) algorithm for training, and use the ISODATA iteration method to obtain the initial value of the EM training. The traditional Gaussian mixture model is represented as follows:

X表示一个D维随机向量；N(X；μ_q,Σ_q)，q＝1,…,Q表示子分布。每个子分布是一个D维的联合高斯概率分布，可用下式表示：where ω _q , q=1,...,Q represents the mixing weight, and

X represents a D-dimensional random vector; N(X; μ _q , Σ _q ), q=1, . . . , Q represents a sub-distribution. Each subdistribution is a D-dimensional joint Gaussian probability distribution, which can be expressed as:

where μ _i is the mean vector, Σ _i is the covariance matrix, λ={ω _q , μ _q , Σ _q } is the model parameter of the GMM model, and the estimation of λ can be realized by the maximum likelihood estimation method (ML, Maximum Likelihood) , the purpose of maximum likelihood estimation is to maximize the conditional probability P(X/λ). For the speech feature parameter vector set X={x _n , n=1,2,...,N}, there are:

at this time:

The EM algorithm can be used to solve formula (4), as the iterative conditions in the EM calculation process satisfy P(X/λ ^k )≥P(X/λ ^k-1 ), K is the number of iterations until the model parameter λ. In the iterative process, the iterative formulas of the Gaussian component weight coefficient w _q , the mean vector μ _q , and the covariance matrix Σ _q are as follows:

In the improved GMM training, the expected number of classes of the feature parameter data, the initial number of cluster centers, the minimum number of samples allowed in each class, the upper limit of the relative standard deviation of the distribution of each feature component within the class, the two classes The lower limit of the minimum distance between the centers, the maximum number of "merging" operations that can be performed in each iteration, and the maximum number of iterations allowed are comprehensively analyzed, and the Gaussian mixture degree is dynamically self-organized. The training process is as follows:

1. Set the expected number of classes c of the initial feature parameter data of ISODATA, the initial number of cluster centers N _c , the minimum number of samples allowed in each class θ _n , and the upper limit θ of the relative standard deviation of the distribution of each feature component within the class _s , the minimum distance lower bound θ _D between the two types of centers, the maximum number of "merging" operations L that can be performed in each iteration, and the maximum number of iterations I allowed.

2. As shown in Figure 2, use the ISODATA algorithm to initialize the GMM.

(1) Divide each sample in the sample set { _xi } into a certain class according to the nearest neighbor rule.

(2) Calculate the parameters after classification: various centers of gravity, average distance within the class and overall average distance

(a) Calculate the center of gravity of each class:

(b) Calculate the average distance from the sample to the centroid in each category

(c) Calculate the overall average distance of each sample to its intra-class center

3. Iterative training using the EM algorithm.

4. If the trained model meets the following logical requirements:

(the number of samples within a class < θ _n )∪ (the number of classes ≥ 2c)∩ (the distance between the centers of the two classes < θ _D ),

It is considered that these two components contain less information and are similar in composition, and can be merged:

Calculate the distance D _ij between various centers:

D _ij =||z _i -z _j ||,i=1,2,...,N _c -1; j=i+1,...,N _c (12)

Merge is judged according to θ _D. Compare _{Di ij} with θ _D , and arrange those _{Di ijs} that are less than θ _D in increasing order, taking the first L. Starting from the smallest D _ij , the corresponding two classes are merged, and the merged cluster centers are calculated. In each iteration, a class can be merged at most once.

After the merge operation, the number of categories is: N _c =N _c - the number of categories that have been merged

If the trained model meets the following logical requirements:

, it is considered that the Gaussian component contains excessive information and should be split:

(1) Calculate the standard deviation vector from the samples within each class to the class center

σ _j =(σ _1j ,σ _2j ,...,σ _nj ),j=1,2,...,N _c (13)

(2) Calculate its components

Among them, j is the class number, k is the component number, n is the vector dimension, z _kj is the kth component of z _j , and x _ki is the kth component of x _i ;

(3) Find the maximum component σ _jmax in the standard deviation vector σ _j within each class

和

分别是在原z_j中相应的分量加上和减去kσ_jmax，而其它分量不变，其中0≤k≤1。分裂后，I_p＝I_p+1，转至步骤2-(2)。Split the class ω _j into two classes, the original z _j cancels and let N _c =N _c +1. The center of these two new classes

and

are the addition and subtraction of kσ _jmax to the corresponding components in the original z _j , respectively, while other components remain unchanged, where 0≤k≤1. After splitting, _Ip = _Ip +1, go to step 2-(2).

The algorithm ends if the number of iterations I _p =I or the process converges. Otherwise, I _p =I _p +1, if the parameters need to be adjusted, go to step 2-(1); if the parameters are not changed, go to step 2-(2), and go to the next iteration.

5. After ISODATA+GMM training, the posterior conditional probability matrix P(X|λ) is obtained, and λ is saved.

1-5) Use the source speech feature parameter X and target speech feature parameter Y obtained in step (3) and the posterior conditional probability matrix P(X|λ) obtained in step (4) for training to obtain the frequency bending factor and amplitude adjustment factor to construct bilinear frequency warping (BLFW, Bilinear FrequencyWarping) and amplitude adjustment (AS, AmplitudeScaling) speech conversion functions, which are expressed as follows:

F(x)=W _α(x,λ) +s(x,λ) (16)

1-6) Establish the conversion relationship between the source voice pitch frequency and the target voice pitch frequency:

where μ,σ ² are used to represent the mean and variance of the logarithmic fundamental frequency logf ₀ .

1-7) Through the above steps, we established the conversion relationship between the source and target speech feature parameters --- formula (16), the conversion relationship between the source voice and target voice logarithmic fundamental frequency --- formula (20 ).

2), the specific implementation steps of the conversion part as shown in Figure 1:

2-1) Input the source speaker voice to be converted;

2-2) use the AHOdecoder speech analysis model to extract the voice 20-order MFCC feature parameter X of the source speaker and the source voice logarithmic fundamental frequency parameter _logf 0×;

2-3) Using λ={w _i , μ _i , Σ _i } and the feature parameter X extracted in step (2) obtained during ISODATA+GMM training, and substitute it into formula (1) to obtain the posterior conditional probability matrix P( X/λ);

2-4) Use the frequency bending factor α(x/λ) and amplitude adjustment factor s(x/λ) obtained during BLFW+AS training and the posterior conditional probability matrix P(X/λ) obtained in step (3) ), after substituting into formulas (16), (17), (18) and (19) respectively, the MFCC feature parameter Y of the converted speech is obtained;

2-5) utilize the source voice logarithmic fundamental frequency parameter logf _0X obtained in step (2), substitute formula (20), obtain the logarithmic fundamental frequency parameter logf _0Y of voice after conversion;

2-6) Use the AHOdecoder speech synthesis model to take Y in step (4) and logf _0Y in step (5) as input to obtain the converted speech logf _0Y .

3), performance evaluation.

3-1) Mel-cepstral distortion (MCD), an objective performance indicator, more reflects the transformed speaker similarity.

where the MFCC parameter is the 20-D Mel cepstral parameter, which we denote by V _d (t), 0≤d≤19. We remove the first dimension parameter when computing the MCD. T is the total number of frames after the MFCC is aligned with DTW. The smaller the MCD value, the higher the transformed speaker similarity. Figure 3 shows the comparison of the MCD values after the voice conversion between the improved GMM and BLFW+AS speech conversion system and the traditional GMM and GMM+BLFWAS.

3-2) Mean Opinion Score (MOS), a subjective performance indicator, more reflects the quality of speech synthesis.

The MOS evaluation standard divides the quality of the test speech into 5 grades, and the detailed scoring system is shown in Table 1.

Table 1 MOS evaluation criteria

Grading voice level evaluation standard 5 excellent High sound quality, no noise 4 good High sound quality, almost inaudible noise 3 middle Moderate sound quality with slight noise 2 Difference Medium to low sound quality, but with a lot of noise 1 inferior Low sound quality, unbearable

The higher the MOS test score, the higher the intelligibility and naturalness of the converted speech. The mathematical expression for the MOS score is as follows:

Among them, M is the total number of people participating in the test, N is the number of test sentences, and score _n,m is the score of the mth person on the nth test sentence. The larger the MOS value, the higher the speech synthesis quality. Figure 4 shows the comparison of the MOS values after the voice conversion between the improved GMM and BLFW+AS speech conversion system and the traditional GMM and GMM+BLFWAS.

To sum up, a high-quality speech conversion system should satisfy the above three criteria at the same time: the spectrogram of the converted speech is closer to the spectrogram of the target speech, the distortion of the spectrum is lower, and the converted speech is less distorted than other models. The MCD value of the model is the smallest, indicating that the distortion of the spectrum of the model is lower, the spectral similarity between the converted speech and the target speech is better, and the MOS value is the largest, indicating that the sound quality of the converted speech of the model is better.

The above are only preferred examples of the present invention for detailed and exemplary description. Those skilled in the art can obtain technical solutions through various equivalent substitutions based on the above-mentioned specific examples, all of which shall be included in the rights of the present invention. requirements and their equivalents.

Claims (3)

1. A high-quality speech conversion method, including a training part and a conversion part: 1), the training part steps: 1-1) Obtain the parallel corpus of the source speaker and the target speaker; 1-2) Use AHOcoder speech analysis model to extract speech feature parameters and logarithmic fundamental frequency; 1-3) carry out DTW to the speech characteristic parameter in step 1-2); 2), the conversion part of the steps: 2-1) Input the source speaker voice to be converted; 2-2) Use AHOcoder speech analysis model to extract characteristic parameters and logarithmic fundamental frequency; 2-3) Use ISODATA+GMM and the parameter λ obtained during training to obtain a posteriori conditional probability matrix; 2-4) Substitute the frequency bending factor α(x, λ) and the amplitude adjustment factor s(x, λ) into the bilinear frequency bending and amplitude adjustment conversion function to obtain the converted characteristic parameters, where x refers to The spectral characteristic parameter of speech, λ refers to the GMM model parameter; 2-5) Substitute the logarithmic fundamental frequency into the fundamental frequency conversion function obtained when training to obtain the converted logarithmic fundamental frequency; 2-6) Use the AHOdecoder speech synthesis model to synthesize the converted speech with the converted characteristic parameters and the logarithmic fundamental frequency; It is characterized in that, in the step of the training part, it also includes: 1-4) Use the iterative self-organization algorithm ISODATA to set the initial value of the GMM training for the characteristic parameters in step 1-3), and use the EM algorithm to perform the GMM training to obtain the GMM parameter λ and the posterior conditional probability matrix P(X| λ), where X refers to the spectral feature parameter set of speech; 1-5) Use the posterior conditional probability matrix P(X|λ) in step 1-4) to perform BLFW+AS training to obtain the frequency bending factor α(x,λ) and the amplitude adjustment factor s(x,λ) , so as to construct the BLFW+AS conversion function; use the mean and variance of the logarithmic fundamental frequency to establish the conversion function between the source speech pitch frequency and the target speech pitch frequency. 2. A kind of high-quality speech conversion method according to claim 1, is characterized in that, parameter λ in training part step 1-4) is decided according to the concrete distribution of speaker's speech characteristic parameter. 3. a kind of high-quality speech conversion method according to claim 1, is characterized in that, in training part step 1-5), frequency bending factor and amplitude adjustment factor are the posterior conditional probability matrix that obtains according to ISODATA+GMM training to train.

Images