JPH0247700A - Speech synthesizing method - Google Patents

Abstract

PURPOSE:To hold clearness and naturalness by varying the spectral envelope of a unit speech waveform according to 1st and 2nd spectral variation components. CONSTITUTION:A speech frequency correction part 4 obtains the pitch frequency of a voiced section and its time track from a unit voice obtained by a unit voice selection part 2. A formant correction part 6 determines formant as to the voiced section according to the shape (resonance frequency) of the spectral envelope and finds its track. Then a frequency characteristic variation part 8 varies the spectral envelope of the waveform which is expanded or compressed according to the variation of a pitch frequency according to the spectral variation found by the formant correction part and the spectral variation found by the vocal chord frequency correction part. Consequently, the phoneme and naturalness of each unit speech are maintained and a speech of high quality to which phoneme information is added is synthesized.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a speech synthesis method for outputting by synthesizing pre-stored speech based on output speech information.

There are also known speech synthesis methods such as the rule synthesis method, which synthesizes from parameters stored as unit speech such as phonemes and syllables and prosodic information such as accent and intonation generated according to predetermined rules during synthesis. was.

[Summary of the Invention] The present invention, when connecting pre-stored voices and outputting meaningful voices, takes into account the influence on the synthesized voice due to the connection and the addition of prosodic information,
This is a speech synthesis method that does not impair the clarity and naturalness of synthesized speech.

E. Prior Art Conventionally, in this type of technology, ■Recording 1JA3J, which connects and reproduces the sounds of words and phrases recorded in advance;
Parameter editing that records the parameters obtained by once analyzing the waveform and controls the synthesizer using these parameters during playback [Problems to be solved by the invention] However, in the methods of ■ and ■, Since the output voice is selected from among the voices uttered and registered by a person in advance, the sound quality can be maintained among the registered voices, but in order to connect and output the voices as they are, vocal cord vibration (
Pitch) frequency and formant discontinuity occurred, resulting in unnatural sound.

In addition, method (■) outputs a sound source that has been passed through a vocal tract characteristic filter, using an impulse corresponding to the pitch frequency, white noise, or an estimated vocal cord waveform as a sound source during synthesis, and even in these cases, it is possible to change the accent and intonation. In other words, when the pitch period of speech changes, the formant changes at the same time, resulting in phonologically unclear speech, and unnaturalness at the connection point between the unit speech to be synthesized. Ta.

Therefore, the purpose of the present invention is to solve the above-mentioned conventional problems,
Even when fine-grained pitch and formant control is used to improve the quality of synthesized speech, such as by making the units of speech synthesis smaller or adding prosodic information such as intonation, the phonology of the synthesized speech can be maintained. An object of the present invention is to provide a voice synthesis method that has the naturalness of human voice.

Another object of the present invention is to provide a method for measuring the perceptual characteristics of human language speech by making it possible to control phonology based on prosody such as intonation and formant frequency.

[Means for Solving the Problems J] To this end, in the present invention, unit speech information and prosody information are determined based on output speech information, and unit speech information is determined based on unit speech information determined from unit speech data stored in advance. To select unit voice data corresponding to voice, calculate or extract each of the pitch period, spectral envelope, formant locus, and unit voice waveform from the selected unit voice data, and connect the calculated or extracted unit voice waveforms. ,
and to add prosodic information, the calculated or extracted pitch period is changed, the spectral envelope due to the pitch change is calculated in the changed pitch period, and the spectral envelope due to the pitch change and the calculated or extracted spectral envelope are calculate a first spectral change based on the calculated or extracted formant locus to connect the calculated or extracted unit speech waveforms, and calculate a spectral envelope due to the formant change based on the changed formant locus. Then, a second spectral change is calculated based on the spectral envelope due to the formant change and the calculated or extracted spectral envelope, and the spectrum of the unit speech waveform related to the pitch period change is calculated based on the first and second spectral changes. The present invention is characterized in that after changing the envelope and connecting the vehicle position audio waveform with the changed spectrum envelope, the connected audio is output.

[Operation] According to the above configuration, it is possible to synthesize high-quality speech in which intonation necessary for meaningful speech is added while maintaining the phonology and naturalness of each unit speech.

[Example] Hereinafter, an example of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram of a speech synthesis system showing one embodiment of the present invention. In the figure, 2 is a unit voice selection section, 4
6 is a vocal cord frequency correction unit, 6 is a formant correction unit, and 8 is a frequency characteristic change unit, each of which is configured in an electronic computer,
With this configuration, speech synthesis processing is performed using memory such as ROM, RAM, or disk memory.

When output speech information such as a sentence is manually input to the unit speech selection section 2, the output speech information is first converted into a phoneme symbol string as discrete linguistic information, as well as the duration of the speech, accent, intonation, and pause. etc., is determined. Furthermore, a string of unit sounds constituting the determined phoneme symbol string is selected from a pre-stored set of unit sounds.

Note that instead of determining the phoneme symbol string based on the output speech information, words, characters, etc. may be determined, and the unit speech string may be selected based on this. In addition, although the unit speech to be stored is a waveform in this example, by storing linear prediction coefficients or pitch and formant trajectories related to the unit speech as data of the unit speech, the pitch period and spectral envelope shown below can be stored. , without calculating the formant locus,
It may also be extracted directly.

Next, in the vocal cord frequency correction section 4, the unit voice selection section 2
From the unit speech obtained in step 1, the voiced sound section is determined, and the pitch frequency of the voiced sound section and its time trajectory are obtained. Furthermore, by referring to the pitch frequency locus of the preceding and succeeding unit voices, the pitch frequency is changed so that the loci are smoothly connected, and components such as accent and intonation are further added as changed components, and each voice is adjusted according to this new pitch frequency. The duration of the waveform is expanded or contracted for each pitch period. As a result, the pitch frequencies are smoothly connected, and the accent, intonation, etc. as prosodic information are controlled. Also, the amount of change in the spectral envelope that has changed due to this processing is determined.

The formant correction unit 6 determines the formant for the voiced sound section from the shape of the spectrum envelope (resonance frequency), and obtains its locus. Next, by referring to the formant frequency trajectories of the preceding and succeeding unit voices, changes are made to the formant frequencies so that they are smoothly connected. Also, the amount of change caused by the formant change from the spectral envelope of the original speech is determined.

The frequency characteristic changing section 8 changes the spectral envelope of the waveform expanded and contracted according to the pitch frequency change according to the spectral change obtained by the formant correction section and the spectral change obtained by the vocal fold frequency correction section. .

The details of the processing in each part above are shown in Figures 2 (A) and (B).
) will be explained with reference to the block diagram and flowchart shown in FIG. FIG. 2 (8) shows details of the configuration shown in FIG. The formant correction unit 6 is controlled by the expansion/contraction unit 44, the spectrum envelope extraction unit 46, and the correction extraction unit 48, the formant correction unit 6 is controlled by the spectrum envelope extraction unit 62, the spectrum envelope control unit 64, and the correction extraction unit 66, and the frequency characteristic change unit 8 is The FF7 section 82, the spectral envelope changing section 84, and the IFF7 section 86 are each configured. In addition, the numbers attached to the left side of each step in the flowchart of Figure 2 (B) indicate the number of each part in Figure 2 (A), and the processing is performed in the part indicated by the number with the corresponding step. Indicates that it will be executed.

In the above configuration, the unit voice storage section 22 has vowels -
Conversion of pre-pronounced speech using a set of phoneme sequences such as consonants and vowels. Number of bits: 1zbtt, m frequency: 15k
What is converted into ^/D by llz is stored as a unit voice. Here, the speech selection unit 24 determines a phonetic symbol string according to predetermined phonetic rules based on output audio information such as an input sentence, and further determines duration, accent, intonation, pause, etc. according to prosody rules.

Furthermore, the corresponding unit speech is selected from the unit speech storage section 22 based on the determined phoneme symbol string.

The pitch extraction control unit 42 determines whether each extracted unit voice is a voiced section or a silent section based on the presence or absence of voice power, and then compares the voice in the voiced section with a first-order correlation coefficient and a zero crossing. The number is calculated and the voiced consonant interval and voiced consonant interval are distinguished. This is to ensure reliable discrimination of high-frequency components in speech by examining both the first-order correlation coefficient and the number of zero crossings.

Here, the determined time length of the silent section and the waveform of the silent consonant section are recorded as they are in the memory.

Furthermore, the pitch extraction control unit 42 performs linear predictive analysis on the speech waveform in the voiced section using a so-called voice guidance inverse filter to obtain a residual waveform, and performs correlation analysis on this residual waveform to peak the correlation. Find the pitch period from the interval. This is performed for the entire voiced sound section of the unit speech.

Next, for each of the pitch periods found, changes are made so that the connection with the unit voice immediately before being connected (from a past time) is audibly smooth, and the accent and intonation are correct for the entire phrase or sentence. is added to calculate a new pitch period sequence.

That is, first, the average pitch period of the entire pitch period in the determined unit voice is determined as a geometric mean pitch, taking into consideration the human @ feeling.

Here, if Pn is the n-th pitch period and the total number of pitches in a unit voice is L, the average pitch period Pave is expressed as Pave-(P, xP2x-xPL)"',
When the average pitch period of the immediately preceding unit voice is Pavel, the average pitch period adjustment R is +1-Pavel/P
ave. In addition, let Qn be the period change coefficient calculated from the accent rule and intonation rule, and then use the following formula for L pitches in the pitch period sequence when the average pitch period is adjusted as shown in Figure 3. Adjustments are made using the indicated coefficient Sn so that the pitch period series are smoothly connected.

= 1 (L/'2
≦B≦L) Here, P' is the last pitch period of the corrected pitch period sequence of the immediately preceding (past) unit voice.

If the above adjustments are combined and Iln is the total adjustment amount, it is expressed as Rn-R-Qn-5n, and new pitch period information can be obtained by multiplying Pn by Qn for each of L pitch periods. It will be done.

At this time, in the spectrum envelope extraction section 62.

Find the spectral envelope of the original speech. That is, as shown in FIG. 4, the pitch start point is set immediately before the point where the waveform level suddenly increases from the basic unit audio waveform, and the next pitch is determined based on the pitch period initially determined by the pitch extraction control section 42. Define one pitch section with the end point one sample before the start point of ■20 with the center of the pitch section as the center of the analysis window.
Install windows of about m5ec. This windowing enables short-time spectrum analysis using a finite number of sample values, and this windowing performs linear prediction analysis again based on the data to calculate linear prediction coefficients α1 to α2. Here, p is the order of linear predictive analysis, and generally about P-10 is used for female voices, and about P-14 is used for male voices.

Furthermore, the spectral envelope 11(k) of the original speech is determined using the above-mentioned linear prediction coefficients α1 to α2 according to the following equation.

(k; 1 to N) Here, N is a power of 2 that is larger than the number of samples, and is 512.

This process is repeated while shifting one pitch section until the voiced sound section ends.

Further, the waveform expansion/contraction section 44 expands/contracts the waveform for each pitch according to the new pitch period information obtained by the pitch extraction control section 42 . In other words, when the number of samples for one pitch of the basic unit speech waveform is k, and the number of samples corresponding to the changed pitch is °, if you want to shorten the pitch period, start at the 'th sample point from the start of the pitch section. If you want to truncate the waveform and extend the pitch period, use the spectrum envelope extraction section 62.
Using the linear prediction coefficients α1 to α2 obtained in the above, sample values from the +1st to the m-k'th m- are obtained as shown in the following equation to obtain the subsequent waveform.

x (m) = a, x (m-1) + a, x (ll-2)
+...+ a, x (m-P) However, this process is
It is repeated until the voiced sound section ends while shifting the pitch section, but at this time, the speech rate changes by the expansion and contraction of the pitch period, so the utterance time of the basic unit voice is thinned out in waveform units of 1 pitch period, and the same waveform is repeated. keep it long.

Further, the duration length is also corrected here based on the prosody information obtained from the voice selection section 24 using the same means.

Furthermore, as a result of changing the pitch, there is a large discontinuity between the final sample point of the waveform of a pitch section and the starting sample point of the next pitch section, so Approximation using cubic curves is performed using the least squares method using data of several samples, and the data are connected continuously.

When the above-described processes by the pitch extraction control section 42, waveform expansion/contraction section 44, and spectrum envelope extraction section 62 are completed, first, in the spectrum envelope extraction section 46, the waveform expansion/contraction section 44
Centering on one pitch section of the waveform obtained by changing the pitch period obtained from , the spectral envelope i (k) after the pitch change is determined by the following equation.

(k=1~N) (1) Here, N is assumed to be 512 as described above.

Next, the correction extracting unit 48 extracts the spectral envelope 1 of the original speech.
1(k), the amount of change in the spectral envelope (k) that is distorted due to the change in pitch period is calculated.

That is, Ll(k)·H(k)/Kai(k) is calculated for each pitch period and stored in the memory.

In addition, the spectral envelope control unit 64 uses α1 to α2 obtained by the spectral envelope extraction unit 62 as coefficients to calculate a complex number Zl”-Z, which is a root of P that satisfies the formula shown below.
Find p.

1+a,z-'+ a 2z-2+ ・・-+
CE, Z-'-0 Among these P roots, there are pairs of conjugate complex roots, and one pair of conjugate complex roots can correspond to one formant. That is, from these roots Z, the resonant frequency Fl and its bandwidth a are determined by the following formula, and are recorded in memory, and the above-mentioned process is shifted for each pitch interval until the voiced sound interval in the unit speech ends. Repeat until.

F+-FS/(2π)・arg(zl)81-Fs/π
llog(lz+l) where Fs is the sampling frequency.

Furthermore, from a series of resonant frequencies, a resonant frequency with a narrow bandwidth is selected in order from the lowest frequency, taking into account its bandwidth and continuity, as the first formant, second formant, third formant, etc. Find the frequency locus.

Furthermore, as shown in FIG. 5, the spectral envelope control unit 64 converts the locus of the formant and its bandwidth to the final sample point of the immediately preceding unit speech and the process required to improve the connectivity with the immediately preceding unit speech. Using several samples before and after the starting sample point in the unit speech, interpolation is performed by cubic curve approximation using the least squares method as described above, and the samples are continuously connected.

Next, once a new formant frequency locus and f-bandwidth are determined as described above, new linear prediction coefficients are determined as follows.

That is, F
lo Let the bandwidth be BIo, and find a new root Zl' using the following equation. Note that Zlo includes a pair of conjugate complex roots corresponding to formants.

Zl'-exp(-7CB+'/Fs"j2 πp+'
/ps) The p-order equation with these P Zl' as roots is (1-L'Z-') (1-Z2°X-')・" (1
-Zp'Z-')-0, and when this formula is expanded, Z
If the coefficient of
0, and the coefficients β, ˜β2 give the new linear prediction coefficients. Using this new linear prediction coefficient, the spectral envelope 1n(k) is determined by the following equation.

(k=1~N) (2) Here, N is assumed to be 512.

The correction valve extractor 66 calculates the amount of change in the spectrum r (k) that has been changed due to the formant change, with respect to the spectrum envelope 11(k) of the original speech obtained by the spectrum envelope extractor 62. That is, V (k) −II (k
) /F2 (k) wo Calculate for each hit cycle and record in memory.

In the frequency characteristic changing section 8, first, in the FF7 section 82,
The pitch extraction control unit 42 multiplies N samples x(1) to x(N) in the pitch period whose pitch has been changed by a time window w (i) as shown in the following equation to obtain y(1) to y( N). That is, y(+)su(i)・x(i) 1≦
i≦N where w(i)J, 5-(1-cos(yr i/L))
1≦i<Lw(i)=I
L≦i<N−Lw(
i) −0,5・[l+cos(π (i−N+L)/L
)] N-L≦i≦N The above y (i) is subjected to N-point fast Fourier transform and transformed into the frequency domain to become Y (k).

Next, in the spectrum conversion section 84, the correction valve extraction section 6
The change component V (k
) and the change amount U(k) due to the cycle change calculated by the correction valve extraction unit 48 to change Y(k). That is, Toshi (k) −tl (k) ・V (k) −Y (k
) Obtain a frequency domain representation r(k) corrected as 1≦ and ≦N.

Next, the IFFT unit 86 converts this r(k) into a time-domain audio waveform by inverse fast Fourier transform. (k), and in order to reduce the effect of distortion at the edges during waveform connection among the N pieces of data obtained, a Hamming window of 20 m5ec is applied to the center of the sample data and cut out.

Next, as shown in FIG. 6, y (k) is shifted by 10 nSec, and the above-described series of operations of correcting the spectrum envelope and cutting out is repeated, and the waveform cut out just before is superimposed to form continuous audio.

Furthermore, when the processing of one voiced sound section is completed, it is connected to the unvoiced consonant section or silent section stored in the memory,
The process then moves on to the next voiced section. After processing all the unit sounds in this way, the final synthesized sound is converted to D/
A conversion is performed to produce output audio.

By the way, there are still many unknowns about the perceptual characteristics of human speech sounds. How does a physical change in the speech signal waveform affect the phonology of language, that is, the perceptual characteristics of humans? For example, how does it sound when the pitch or prosody of the voice is slightly changed? When trying to test a person using an actual person as a speaker, it may be difficult to produce exactly the same voice due to changes in physical condition, etc., and it is generally difficult to produce subtle articulations.

For these reasons, a method for adjusting artificially generated sounds has been needed. However, when conventional methods are used, unnaturalness occurs due to discontinuities in vocal fold vibration frequencies and formant frequencies, making delicate psychological experiments difficult.

In contrast, according to the above-described embodiments of the present invention, this difficulty can be overcome because fine control can be performed while maintaining the naturalness of the voice.

[Effects of the Invention] As is clear from the above explanation, according to the present invention, it is possible to prevent deterioration of sound quality due to discontinuity when connecting pre-stored unit sounds, and to prevent the deterioration of sound quality due to discontinuity, which affects the naturalness and individuality of the original sound. It becomes possible to synthesize speech by adding intonation, accent, etc. without giving any.

Further, according to the present invention, speech can be synthesized by arbitrarily changing prosodic properties such as intonation and accent, and phonological properties based on formant frequency, etc., so that it becomes possible to provide a method for measuring the perceptual characteristics of human language speech.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a speech synthesis system showing an embodiment of the present invention, FIG. 2 (A) is a block diagram showing details of the system shown in FIG. 1, and FIG. 2 (6) is a block diagram of a speech synthesis system shown in FIG. A flowchart showing the processing of the system shown in (A), Fig. 3 is a diagram to explain the conversion method that maintains the continuity of the locus of vocal fold vibration frequency, and Fig. 4 shows the method for cutting out the waveform of the pitch section. FIG. 5 is a diagram showing a conversion method that maintains the continuity of formant frequency bandwidth, and FIG. 6 is a diagram showing a method for connecting processed waveforms. 22... Unit audio storage section, 24... Audio selection section, 42... Pitch extraction control section, 44... Waveform expansion/contraction section, 46... Spectral envelope extraction section, 48... Correction component extraction Dobe, 62... Spectrum envelope extraction unit, 64... Spectrum envelope control unit, 66... Correction extraction unit, 82... FFT unit, 84... Spectral envelope modification unit, 86...・IFFT Department. b?・To-splash ~ Shin? -r;! 5 μm platen 2 of 4'J Figure 5 Figure 5

Claims (1)

[Scope of Claims] 1) Determining unit voice information and prosody information based on output voice information, and determining unit voice corresponding to the unit voice from among pre-stored unit voice data based on the determined unit voice information. selecting data, calculating or extracting each of the pitch period, spectral envelope, formant locus, and unit speech waveform from the selected unit speech data, and connecting the calculated or extracted unit speech waveforms, and the prosody. In order to add information, the calculated or extracted pitch period is changed, a spectral envelope due to the pitch change is calculated in the changed pitch period, and the spectral envelope due to the pitch change and the calculated or extracted spectral envelope are calculate a first spectral change based on the above, change the calculated or extracted formant locus in order to connect the calculated or extracted unit speech waveforms, and change the formant based on the changed formant locus. calculate a second spectrum change based on the spectrum envelope due to the formant change and the calculated or extracted spectrum envelope; and calculate the pitch period based on the first and second spectrum changes. A speech synthesis method comprising: changing the spectral envelope of a unit speech waveform related to the change; connecting the unit speech waveforms whose spectral envelopes have been changed; and then outputting the connected speech. 2) A speech synthesis device characterized by synthesizing speech using the method according to claim 1.