JPH1195796A - Voice synthesizing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice synthesizing method by which synthesized voices of superior tone quality can be abtained, the size of a voice element dictionary is compact, and the change in voice quality is easily performed. SOLUTION: In a analysis section 100, the voice elements segmented by a pitch waveform segmenting section 101 are inputted into an LPC analysis section 102 and expressed in the forms of residual signals and LPC coefficients. A set of these spectrum parameters and the residual signals is stored in a residual signal storage section 103 and an LPC coefficient storage section 104 as a voice element dictionary. In an analysis section 200, a selecting section 210 selects a set of the residual signals and the spectrum parameters in accordance with the phoneme symbol string given by a sentence analysis.rhythm control section. Then, voice elements are generated by passing them through a synthesis filter 202, which is constructed of the selected residual signals and the selected spectrum parameters. Then, a pitch period control by a pitch synchronization waveform superimposing method and a duration length control are conducted in a rhythm control section 203 against the voice elements and synthesized voices are generated by connecting these voice elements an element connecting section 204.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a speech synthesis method suitable for text speech synthesis, and more particularly to a speech synthesis method for generating a synthesized speech from information such as a phoneme symbol string, a pitch, and a phoneme duration.

[0002]

2. Description of the Related Art Creating a speech signal artificially from an arbitrary sentence is called text-to-speech synthesis. Text-to-speech synthesis is generally performed in three stages: a language processing unit, a phonemic processing unit, and a speech synthesis unit. The input text is first subjected to morphological analysis and syntactic analysis in the language processing unit, and then to accent and intonation processing in the phonological processing unit, resulting in phonological symbol strings, pitch, phonological duration, etc. Is output. Finally, the speech signal synthesizer generates a synthesized speech from information such as a phoneme symbol string, pitch, and phoneme duration.

[0003] A speech synthesis method used for such text speech synthesis must be a method capable of synthesizing an arbitrary phoneme symbol string with an arbitrary prosody. A speech synthesis method capable of synthesizing an arbitrary phoneme symbol string as speech is roughly classified into an LPC analysis / synthesis method and a waveform editing method.

The LPC analysis / synthesis method is described in, for example, Reference (1).
Applying LPC analysis to speech signals as introduced in "Ito and Sato:" Pitch control method in speech synthesis using segmented residuals ", Acoustic Theory 2-7-18 (1989-3)" LPC
In this method, a spectrum parameter and a residual signal are obtained, and prosody is controlled and connected at the level of the residual signal. This method has the advantage that the voice quality can be easily changed by manipulating the LPC coefficients and that the size of the speech unit dictionary for synthesis is relatively small, but the sound quality of the synthesized voice is so-called nose-like It was lacking in nature and was insufficient.

[0005] On the other hand, a waveform editing method is described in, for example, Reference (2).
"Hirokawa, Hakoda, Sato:" Waveform Selection Method Considering Spectrum Continuity in Waveform Editing Synthetic Method ", Acoustic Theory 2-6-10
(1990-9) ", Reference (3)" Iwata et al .: "Japanese Text-to-Speech Synthesis for Personal Computer Software", Acoustic Theory 2-8-13
(1993-10) "and Reference (4)" Koyama, Koizumi: "VC
As described in “Study of waveform rule synthesis method using V as a basic unit”, IEICE Technical Report SP96-8 (1996-5), etc., the pitch period and continuation of speech units extracted from actual speech waveforms This is a method of synthesizing voice by changing the time length and connecting, and it is considered that it is relatively easy to achieve high sound quality, and has been actively studied.

Further, from the standpoint that it is better not to perform signal processing such as analysis and synthesis in order to improve sound quality, a speech waveform having the same phonological environment and prosodic environment is connected in the longest unit from a natural speech database. (N. Campbell and A.W. Black: “CHAT”
R: Arbitrary speech synthesis system with natural speech waveform connection, IEICE Technical Report SP96-7 (1996-5).

[0007] These systems have the advantage that synthesized speech with higher sound quality can be generated than the analysis and synthesis system, but have the problem that the size of the speech unit dictionary is increased. Also,
Since the spectral parameters are not explicitly expressed, there is a problem that it is difficult to change the voice quality.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-mentioned conventional problems, and has a superior speech quality of a synthesized speech, a compact speech unit dictionary size, and an easy-to-change speech quality. The purpose is to provide.

[0009]

In order to solve the above problems, a speech synthesis method according to the present invention represents a speech unit in the form of a residual signal and spectral parameters such as LPC coefficients, and converts the residual signal. A speech unit is created by passing through a synthesis filter configured according to spectral parameters, prosody control is performed on the speech unit, and a speech unit after the prosody control is connected to generate a synthesized speech. And

More specifically, a speech unit is expressed in the form of a residual signal and a spectrum parameter, and a set of the residual signal and the spectrum parameter is stored as a speech unit dictionary. A speech unit is created by selecting a set of a residual signal and a spectrum parameter according to a column and passing the selected residual signal through a synthesis filter configured according to the selected spectrum parameter. Prosody control is performed, and speech units after prosody control are connected to generate a synthesized speech signal.

In prosody control, it is preferable to control the pitch period by applying a pitch-synchronized waveform superposition method to a speech unit obtained by a synthesis filter.
In prosody control, the duration of the speech unit may be further controlled.

According to such a speech synthesis method according to the present invention, in the conventional residual drive speech synthesis method, the prosody is controlled at the level of the residual signal, whereas the level of the speech unit is controlled. To control the prosody and connect the speech units after the prosody control, so that a synthesized speech having a sound quality equivalent to that of the waveform editing method can be obtained.

In this case, if the pitch-synchronized waveform superposition method is used for controlling the pitch period in the prosody control, a clearer and higher-quality sound can be synthesized. Further, in the present invention, since a speech unit prepared as a speech unit dictionary is represented by a set of a spectral parameter such as a residual signal and an LPC coefficient,
The size of the speech unit dictionary is also compact.

Further, by expressing a speech unit by a set of a spectrum parameter and a residual signal, it is possible to easily change the voice quality of a synthesized speech by operating the spectrum parameter.

[0015]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing an embodiment in which the speech synthesis method according to the present invention is applied to a text speech synthesis system. This speech synthesis system is roughly divided into an analysis unit 100 and a synthesis unit 200.

The analyzing section 100 includes a pitch waveform extracting section 101 for extracting a pitch waveform from an input speech waveform,
LPC analysis of extracted patch waveforms (linear prediction analysis)
, And the residual signal and the spectral parameter LPC
LPC analysis unit 102 for extracting coefficients, and LPC analysis unit 1
02 and a residual signal storage unit 103 for storing a set of the residual signal and the LPC coefficient extracted as a speech unit dictionary.
It comprises a PC coefficient storage unit 104.

On the other hand, the synthesizing unit 200 transmits a sentence
According to a phoneme symbol string obtained by analyzing a text to be subjected to text synthesis by the prosody control unit, the residual signal storage unit 103 and the LPC coefficient storage unit 104 of the analysis unit 100 store a set of residuals corresponding to each phoneme symbol. A speech unit selection unit 201 for selecting and extracting a difference signal and an LPC coefficient, and a synthesis filter 202 configured according to the selected LPC coefficient and generating a speech unit using the selected residual signal as an input.
And a prosody control unit 203 that controls the prosody of the generated speech unit according to the pitch period and duration information given from the sentence analysis and prosody control unit, and a speech unit after the prosody control. Unit connection unit 2 that generates synthesized speech
04.

Next, the detailed processing procedure of the analyzer 100 will be described with reference to the flowchart shown in FIG. First,
A voice waveform is input to the analyzer 100 (step S1).
1). As the speech waveform, for example, a representative speech unit created as described later is used.

Next, a pitch waveform extracting section 101 multiplies an input speech waveform by a window function of a pitch cycle length to cut out a waveform corresponding to the pitch cycle, and then performs a pitch-synchronized LPC analysis in an LPC analyzing section 102 (step S12). To S13). In this case, since the discrete spectrum of the speech waveform is smoothed by the window function, a spectrum envelope in which the influence of the fundamental frequency is reduced can be obtained.

As a result of the LPC analysis in step S12, the speech unit is represented by a set of the residual signal and the LPC coefficient in units of pitch period. Among these, the residual signal is stored in the residual signal storage unit 1.
03, the LPC coefficients are stored in the LPC coefficient storage unit 104 as a speech unit dictionary in association with each other (step S14).

Next, a detailed processing procedure of the synthesizing unit 200 will be described with reference to a flowchart shown in FIG. At the time of speech synthesis, a sentence analysis / prosody control unit (not shown) supplies a phoneme symbol string, a pitch period, and a duration (phoneme duration).
Is given. First, according to the phoneme symbol string, the residual signal storage unit 103 forming the speech unit dictionary and the LPC
From the coefficient storage unit 104, the selection unit 201 selects and reads a set of the residual signal and the LPC coefficient corresponding to each phoneme symbol (step S21).

Next, the LPC selected in step S21
A speech unit is created by configuring the synthesis filter 202 with the coefficients and inputting the residual signal selected in step S21 to the synthesis filter 202 (step S21).
22-S23).

Next, the prosody control unit 203 controls the prosody, that is, the control of the pitch period, according to the pitch period and duration information provided by the sentence analysis and prosody control unit for the speech unit created in step S23. And the duration time is controlled.

More specifically, the pitch period is controlled by applying the pitch-synchronized waveform superposition method (PSOLA) to the speech unit created in step S23 in the same manner as in the waveform editing method (step S24). . The pitch-synchronized waveform superposition method is described in, for example, reference (6) “F. Charpentier and M. Stella:
“Diphone Synthesis Using an Overlap-add Techniqef
or Speech Waveforms concateration ”, Proc.ICASSP 8
6, pp. 2015-2018 (1986) '', but in the present embodiment, in order to enable speech synthesis with higher sound quality, the pitch synchronous waveform superposition method is performed as follows. The pitch period is controlled based on the pitch.

In general, the sound quality of synthesized speech largely depends on the smoothness of voiced sound. Therefore, in the present embodiment, in order to make the change in the pitch cycle smoother, the given pitch cycle is interpolated in units of samples. If the center times of the j-th frame and the j + 1-th frame are respectively t ₁ and t ₂ and the pitch periods are p ₁ and p ₂ , the pitch period p (t) at time t is It is expressed by an equation.

[0026]

(Equation 1) Also, the position of the pitch mark from t ₁ to t ₂ m _{k (k}
= 1, 2,... N), the following equation holds.

[0027]

(Equation 2) The following equations are obtained from the equations (1) and (2).

[0028]

(Equation 3)

In the control of the pitch period in the prosody control unit 203, the speech unit created by the synthesis filter 202 is superimposed on the basis of the position of the pitch mark thus obtained. That is, for example, the head of a speech unit is arranged at the position of each pitch mark on the time axis, and these are superimposed on the zero signal. In this case, the overlapping portions of adjacent speech units corresponding to the positions of the respective pitch marks are added, and the non-overlapping portions remain as original speech units.

The prosody control unit 202 further controls the duration time (step S25). In the control of the duration, it is important how the pitch marks of the original speech waveform and the synthesized speech waveform are associated with each other. In this embodiment, temporal mapping is performed by a function in this association. According to this method, by properly defining the mapping function, it is possible to freely control the thinning / interpolation of the pitch waveform according to the nature of the speech unit to be connected.

Next, connection is performed between the speech units subjected to the prosody control (control of the pitch period and the duration) in this way (step S26). In the present embodiment, in order to reduce distortion due to discontinuity of the waveform in the connection unit, CV and VC units are used as speech units, and each vowel unit is connected by a vowel stationary unit. At this time, the pitch waveforms of the vowels to be connected are connected by weighted addition over the entire vowel section. In this way, a synthesized speech in which an arbitrary sentence (text) is converted into an audio signal is generated.

Next, a description will be given of a speech unit learning method associated with the present invention. Traditionally, speech units have been created by manual trial-and-error techniques. It was necessary to repeatedly perform a series of operations for evaluating the synthesized speech.

On the other hand, as a method for automatically generating a speech unit from a speech database, for example, reference (7) “Nakashima,
Hamada, "Rule Synthesis Method by Clustering Based on Acoustic Updates", IEICE D-II, vol.J-72-D-II, No.8, pp.1177-1179 (1
989-8) ”, the phoneme environment clustering (CO
C: Context Oriented Clustering) method is known.
This method is a method of clustering speech segments cut out from a speech database under the constraints of a phoneme environment based on the dispersion of spectral parameters, and using the centroid of each cluster as a representative speech segment.

This phoneme environment clustering method has a feature that a representative speech unit can be determined based on a statistical evaluation criterion without relying on a priori knowledge. However, it is difficult to control a pitch cycle which is a problem in speech synthesis. Since the accompanying distortion is not taken into account, the sound quality of the synthesized speech is not always sufficient.

Therefore, the distortion of the synthesized speech including the distortion caused by performing the prosody control (control of the pitch period and the duration) is defined, and a learning method of the representative speech unit for minimizing the distortion will be described. .

FIG. 4 is a block diagram of a representative speech unit closed loop learning system used in this embodiment. This learning method can be actually applied to various synthesizers and synthesis units, but here, a case where the learning method is applied to learning of CV and VC speech units used in the speech synthesis system described above will be described. . In this method, after speech units are generated by learning, LPC coefficients of a synthesis filter and a residual signal are obtained.

In the learning, first, a large number of speech units in a speech synthesis unit are cut out from the speech database 401 as a preliminary preparation, and these are used as representative speech unit candidates 402. At the same time, training data 403 for learning is created by a similar method. Next, the pitch period and the duration of the representative speech unit candidate are analyzed (404), and the pitch period and the duration of the representative speech unit candidate are analyzed and changed using the training data 403 as a target (405). Synthesize speech units. In this way, speech units are generated for all combinations of the representative speech unit candidates 402 and all the training data.

Next, the distortion of the generated speech unit with respect to the training data is calculated and evaluated (405).
A representative speech unit that minimizes the sum of distortions for all the training data is searched and selected from the above-described candidate representative speech units (406), and this is set as a representative unit.

This learning method is called closed-loop learning in the sense that the evaluation result of the synthesized speech unit is fed back to the learning of the speech unit. Hereinafter, a specific example of a distortion measure and a method of selecting a representative speech unit, which are important in this learning method, will be described.

(Distortion scale) The distortion scale of learning needs to reflect the result of the subjective evaluation well. Further, since the power of the synthesized speech is controlled by the speech synthesis system, it is necessary to evaluate the representative speech unit at a level where the power is normalized. In consideration of the above, the distortion of the synthesized speech unit is defined by the following equation.

[0041]

(Equation 4)

[0042] Here, r _j represents the training data, s _ij denotes a synthesized speech segment according to a representative speech unit candidate u _i that the goal of r _j. (Selection of representative speech unit) Assuming that the number of representative speech units per synthesis unit is n and the number of candidate representative speech units is N, the selection of the representative speech unit is a combination of selecting n out of N candidates. Then, there is a problem of searching for a set of representative speech units that minimizes the next cost function.

[0043]

(Equation 5)

Here, M is the number of training data. When a set of representative speech units that minimizes the cost function of Expression (9) is obtained, all training data can be clustered into clusters corresponding to the representative speech units.

FIG. 5 shows an example in which two representative speech units are selected from four representative speech unit candidates. In this example, among any two combinations of u _{1 to} u ₄ , u ₂
And the cost function of the combination of u ₃ is minimized. As a result, u ₂ and u ₃ are selected as representative speech units.

(Evaluation experiment) Diphone of CV and VC
Is a synthesis unit, and 1 is calculated for each synthesis unit by the above-described method.
An experiment to create representative speech units was performed. Speech unit data and representative speech unit candidates used for training are cut out from a speech database to which a phoneme label has been attached by inspection, and a total of 302 CVs and Vs are obtained by the closed loop learning method described above.
C representative speech unit was created. The time required for learning was Sun
-About 1.5 hours with Ultra2.

FIG. 6 shows the value of the cost function when the number of speech units per synthesis unit (CV, VC) is increased. From this figure, as the number of speech units increases, the distortion of the synthesized speech decreases. It turns out that it decreases monotonically.

It has been known that the sound quality of a synthesized sound is improved by selectively using speech units according to power and pitch. However, according to the conventional method based on trial and error, it takes a lot of effort and time to create a representative speech unit, and it is not easy to increase the number of representative speech units.

On the other hand, according to the above-described closed-loop learning method, given labeled speech data, speech segments can be automatically created in a short time, and an arbitrary number of representative speech segments are generated. It is easy. Moreover, it is possible to create a rule for selection based on a measure of distortion of synthesized speech, instead of selecting speech units based on a priori knowledge such as power and pitch. In other words, it is possible to generate a speech unit selection rule by clustering the training data into clusters of the selected representative speech units and extracting factors common to the clusters.

Next, the sound quality of the synthesized speech obtained by the above-described speech synthesis system was evaluated. The generated representative speech unit is supplied to the analysis unit as the speech input of FIG. 1 and is divided into a residual signal and an LPC coefficient via the pitch waveform cutout unit 101 and the LPC analysis unit 102, and the LPC coefficient is stored in the residual signal storage unit 103. It was stored in the coefficient storage unit 104 as a speech unit dictionary. Upon accumulation, the residual signal and the LPC coefficient were encoded by applying a vector-scalar quantization technique. As a result, the data amount is about 150 kbytes per speaker, which is very compact, 1/10 to 1/20 as compared with the waveform editing method. Therefore, the speech synthesis system of the present embodiment can be easily incorporated into a portable information terminal such as a PDA, a car navigation system, or the like.

Seven stages (−3: very bad to +3 :) by a total of 10 (same number of men and women) general subjects including 7 university students
As a result of the subjective evaluation of (very good), the sound quality of the synthesized speech obtained by the speech synthesis system of the present embodiment is, on average, the average for male and female speakers and various sentences compared to the conventional cepstrum synthesis type speech synthesis system. The subject obtained an evaluation that the sound quality was improved by 2.5 points, the clarity was significantly improved, and the sound quality was softer and closer to real voice.

[0052]

As described above, according to the speech synthesis method of the present invention, a speech unit is represented by a set of a residual signal and a spectral parameter such as an LPC coefficient, and is generated by the residual signal and the spectral parameter. Control of the tone of the speech unit to be generated, it is possible to generate clear and high-quality synthesized speech, and it is easy to change the voice quality by manipulating the spectrum parameters. It can be compact.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a speech synthesis system according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a processing procedure on the analysis side in the embodiment.

FIG. 3 is a flowchart showing a processing procedure on the synthesis side according to the embodiment;

FIG. 4 is a block diagram for explaining a closed-loop learning system for representative speech units.

FIG. 5 is a diagram showing an example of representative speech unit selection based on distortion of a synthesized speech unit.

FIG. 6 is a diagram showing the relationship between the number of representative speech segments and a cost function.

[Explanation of symbols]

 Reference Signs List 100 voice analysis unit 101 pitch waveform cutout unit 102 LPC analysis unit 103 residual signal storage unit 104 LPC coefficient storage unit 200 voice synthesis unit 201 selection unit 202 LPC synthesis filter 203 prosody control unit 204 Voice unit connection

Claims (4)

[Claims] A speech unit is expressed in the form of a residual signal and a spectrum parameter, and the residual signal is passed through a synthesis filter configured according to the spectrum parameter to produce a speech unit. A speech synthesis method comprising: performing prosody control using the speech units after the prosody control to generate synthesized speech. 2. A speech unit is represented in the form of a residual signal and a spectrum parameter, a set of the residual signal and the spectrum parameter is stored as a speech unit dictionary, and the residual is stored in accordance with a given phoneme symbol string. A speech unit is created by selecting a set of a difference signal and a spectrum parameter, and passing the selected residual signal through a synthesis filter configured according to the selected spectrum parameter, and performing prosody control on the speech unit. A speech synthesis method comprising: connecting speech units after prosody control to generate synthesized speech. 3. The speech according to claim 1, wherein the pitch period is controlled by applying a pitch-synchronized waveform superposition method to speech segments obtained by the synthesis filter during the prosody control. Synthesis method. 4. A speech synthesis method according to claim 3, wherein said prosody control further controls a duration of a speech unit.