JPH1195798A - Method and device for voice synthesis - Google Patents

Abstract

PROBLEM TO BE SOLVED: To efficiently synthesize human voices with the degree of freedom in the length of sound and intervals. SOLUTION: Phonemes such as fundamental fifty sounds, voiced sounds and contracted sounds are uttered by an announcer with fundamental sound lengths and intervals and the sound waveforms are taken in as PCM data. Then, plural unit segments d1, d2,... are defined on the time axis of the sound waveforms of one phoneme, the sound waveforms are Fourier-transformed for every segment, four representative frequencies are extracted and corresponding four MIDI data are defined. Then, the defined four MIDI data are arranged on four tracks T1 to T4. Then, the MIDI data of the segments d1 to d9 are arranged at positions k1 to k9 and the data are classified into the data that contribute to vowels, and the data that contribute to consonants. During the reproducing of specified phonemes while specifying sound lengths and intervals, compensation on sound lengths (delta time) and interval (note numbers) is conducted against the MIDI data, which contribute to vowels, and a reproducing is performed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech synthesizing method and a speech synthesizing apparatus.
It relates to technology for synthesizing human speech and singing voice.

[0002]

2. Description of the Related Art A method of artificially synthesizing a human voice has been used in various fields, and various electronic devices having a built-in computer have a function of simulating a conversation. . The most widespread method for encoding human speech is PCM (Pulse Code Modul).
ation) technology, which is widely used in the field of digitizing and handling audio signals. The basic principle of this PCM method is that an analog audio signal is sampled at a predetermined sampling frequency, and the signal strength at each sampling is quantized and represented as digital data. The more it is, the more faithful it is possible to reproduce the original sound. However, the higher the sampling frequency and the number of quantization bits, the larger the required information amount. Therefore, as a technique for reducing the amount of information as much as possible, an ADPCM (Adaptive Differentia) that encodes only a signal change difference is used.
l PulseCode Modulation) is also used.

[0003] When an electronic device has a function of simulating a conversation, the PCM method is usually used. For example, if a human voice is encoded and recorded for each word by the PCM method and a necessary word is decoded when necessary, a specific word can be reproduced. If a plurality of words are reproduced in a predetermined order, it is possible to have a pseudo conversation.

On the other hand, MIDI (Musical Instrume) was born from the idea of encoding musical instrument sounds by electronic musical instruments.
The Digital Interface (nt Digital Interface) standard has also been actively used with the spread of personal computers. Code data according to the MIDI standard (hereinafter, MID)
I data) is basically data that describes the operation of playing a musical instrument, such as which keyboard key of the musical instrument was played and with what strength. The MIDI data itself contains the actual sound. No waveform is included. Therefore, when reproducing the actual sound, the MI which stores the waveform of the musical instrument sound is used.
A DI sound source is required separately. However, the P
Compared to the case where sound is recorded by the CM method, the amount of information is extremely small. This encoding and decoding technology based on the MIDI standard is now widely used in software for playing musical instruments, practicing musical instruments, composing music, etc. using a personal computer, and is also widely used in fields such as karaoke and game sound effects. Have been.

[0005]

If the above-described PCM technique is used, it is possible to synthesize human speech to some extent faithfully. However, since the coded data by PCM has the waveform information of the spoken voice as it is, the amount of information is considerably large, and the data processing load has to be heavy. Further, at present, a data processing technique of changing a sound length or a pitch of voice data encoded by PCM has not been established at a practical level, and a sound length and a pitch can be freely changed. It is difficult to play. For this reason, the spoken voice reproduced by the conventional voice synthesis method must be a so-called "stick reading" monotonous voice without any accent or clause. Therefore, the singing voice and the like cannot be synthesized by the conventional PCM method.

On the other hand, if an encoding method based on the MIDI standard is adopted, it is possible to reproduce a sound having a sufficient sound quality with a very small amount of information and to freely change the sound length and the pitch. . For this reason, in the field of handling musical instrument sounds, MIDI encoding is actively used. However, as described above, since the MIDI standard itself is originally for encoding the operation of musical instrument performance, it is difficult to synthesize human voice and singing voice by the conventional MIDI encoding method. is there.

Therefore, the present invention efficiently converts human voice,
It is another object of the present invention to provide a speech synthesis method and a speech synthesis device capable of synthesizing with a freedom of a sound length and a pitch.

[0008]

[Means for Solving the Problems]

(1) According to a first aspect of the present invention, in a method of synthesizing a human voice corresponding to a specific phoneme, a human pronunciation waveform is captured as data for one phoneme, and a plurality of waveforms are provided on the time axis of the pronunciation waveform. A code section defining a unit section, creating a code code indicating a representative frequency and its intensity included in the sound waveform in the unit section for each unit section, and defining a code code group corresponding to one phoneme When a process is performed for each required phoneme, a phoneme code table that records code groups defined for each phoneme is prepared, and a synthesis instruction to synthesize a human voice corresponding to a specific phoneme is given. Then, by referring to the prepared phoneme code table, a code code group defined for a specific phoneme is extracted, and the extracted code code group is reproduced using a predetermined sound source. It synthesizes human voices by producing them.

(2) The second aspect of the present invention is the above-mentioned first aspect.
In the voice synthesizing method according to the aspect, when setting a plurality of unit sections on the time axis of the sounding waveform, setting is performed such that adjacent unit sections partially overlap on the time axis. is there.

(3) A third aspect of the present invention is the above-mentioned first aspect.
Alternatively, in the speech synthesis method according to the second aspect, a plurality of P code codes are created by defining a plurality of P representative frequencies for one unit section, and these code codes are separated into P tracks. When the code codes are reproduced, the code codes stored in the P tracks are simultaneously reproduced to obtain a P channel reproduction sound.

(4) The fourth aspect of the present invention is the above-mentioned first aspect.
In the speech synthesis method according to the third to third aspects, a code code group defined for each phoneme is classified into a code code contributing to a vowel and a code code contributing to a consonant, and recorded in a phoneme code table. In the synthesis instructions for phonemes,
Including a parameter indicating the length of sound in generating the specific phoneme, and when performing reproduction based on the given synthesis instruction, the parameter contributes to the vowel of the code codes extracted from the phoneme code table. The code length is corrected based on the parameter indicating the sound length.

(5) The fifth aspect of the present invention is the above-mentioned first aspect.
In the speech synthesis method according to the third to third aspects, a code code group defined for each phoneme is classified into a code code contributing to a vowel and a code code contributing to a consonant, and recorded in a phoneme code table. In the synthesis instructions for phonemes,
A parameter indicating a pitch at which the specific phoneme is generated is included, and when performing reproduction based on a given synthesis instruction, the parameter contributes to a vowel of the code codes extracted from the phoneme code table. With regard to the code code, the pitch is corrected based on a parameter indicating the pitch.

(6) The sixth aspect of the present invention is the above-described fourth aspect.
Alternatively, in the speech synthesis method according to the fifth aspect, a plurality of P code codes are created by defining a plurality of P representative frequencies for one unit section, and these code codes are sorted based on the frequencies. For code groups obtained for phonemes consisting of vowels only, which are stored separately in P tracks, all code codes stored in the P tracks are considered to be code codes contributing to vowels. For the code group obtained for the phoneme, the code codes contributing to the consonants are all the code codes contained in the P tracks, and for the code group obtained for the phoneme including vowels and consonants, The code codes stored in the n tracks on the side are code codes that contribute to vowels, and the code codes stored in the remaining (Pn) tracks are used. The issue code is obtained so as to contribute code code consonants.

(7) The seventh aspect of the present invention is directed to the above-mentioned fourth aspect.
Alternatively, in the speech synthesis method according to the fifth aspect, for code groups obtained for phonemes consisting only of vowels, all code codes are code codes contributing to vowels, and code codes obtained for phonemes consisting only of consonants are obtained. For a group, all code codes are used as code codes contributing to consonants, and for a code code group obtained for phonemes including vowels and consonants, a plurality of types of pronunciation waveforms are produced by changing the length of the phonemes. A code code for each of these plural types of sound waveforms is created, and a code code that indicates only a change within a predetermined allowable range among the plurality of generated code codes that contributes to a consonant. And the remaining code codes are code codes that contribute to vowels.

(8) The eighth aspect of the present invention is the above-described fourth aspect.
Alternatively, in the speech synthesis method according to the fifth aspect, for code groups obtained for phonemes consisting only of vowels, all code codes are code codes contributing to vowels, and code codes obtained for phonemes consisting only of consonants are obtained. For a group, all code codes are used as code codes contributing to consonants, and for code groups obtained for phonemes including vowels and consonants, a plurality of pronunciation waveforms are captured by changing the pitch of the phoneme. A code code group is created for each of the plurality of tone waveforms, and among the created code code groups, a code code showing only a change within a predetermined allowable range is regarded as a code code contributing to a consonant. , And the remaining code codes are code codes that contribute to vowels.

(9) The ninth aspect of the present invention is the above-mentioned first aspect.
In the voice synthesizing method according to the eighth to eighth aspects, a representative frequency included in a sound waveform in a unit section is indicated by a note number, an intensity of the representative frequency is indicated by velocity, and a time corresponding to the section length of the unit section is indicated. By expressing the data in delta time, MIDI code data is created, and when reproducing the created MIDI code data, a MIDI sound source having an acoustic waveform based on human voice is used.

(10) In a tenth aspect of the present invention, in the speech synthesis method according to the first to ninth aspects, code codes similar to each other are determined under a predetermined condition for a plurality of adjacent unit sections. In some cases, a process of replacing these similar code codes with an integrated code code extending over a plurality of unit sections is performed.

(11) According to an eleventh aspect of the present invention, in a speech synthesizer for synthesizing a human voice corresponding to a specific phoneme, synthesis instruction data for designating a specific phoneme and a sound length to be synthesized is input. For each of the necessary phonemes and the required phonemes, code groups for reproducing the pronunciation waveform of a human pronounced at the reference pitch are classified into code codes contributing to vowels and code codes contributing to consonants. A code group for a specific phoneme to be synthesized is read from the phoneme code table based on the phoneme code table recorded and the synthesis instruction data, and a code code that contributes to a vowel in the read code code group is read out. Using a code code correction unit that corrects the sound length based on the synthesis instruction data, and a predetermined sound source, reproduce the corrected code code group, and utter a synthesized voice of a human voice. Those provided with sound reproduction unit.

(12) According to a twelfth aspect of the present invention, in a speech synthesizer for synthesizing a human voice corresponding to a specific phoneme, synthesis instruction data indicating a specific phoneme to be synthesized and a pitch is input. For the data input unit and the required individual phonemes, code codes for reproducing the pronunciation waveform of a human pronounced at the reference interval are classified and recorded as code codes contributing to vowels and code codes contributing to consonants. A code group for a specific phoneme to be synthesized is read from the phoneme code table based on the obtained phoneme code table and the synthesis instruction data, and code codes that contribute to vowels in the read code code group are synthesized. Using a code code correction unit that corrects the pitch based on the instruction data, and a predetermined sound source, reproduce the corrected code code group and produce a synthetic voice of a human voice. Those provided with sound reproduction unit.

[0020]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below based on an embodiment shown in the drawings. In this specification, the procedure of the former part of this embodiment will be described with reference to FIGS. 1 to 4, and subsequently, the procedure of the latter part of this embodiment will be described with reference to FIGS. 5 and 6. I do. And finally, referring to FIG.
The configuration of the speech synthesizer according to the present invention will be described.

§1. Procedure Figure 1 of the front part of the present invention is a diagram showing the basic concept of the encoding processing for phonemes which is a front portion of the speech synthesis method according to the present invention. The first stage of the present invention is a process for preparing a phoneme code table, which is equivalent to a preparation stage for performing speech synthesis.
The phoneme code table is a table in which a code code group is recorded for each of a large number of phonemes. In a later part described later, a synthesized sound is reproduced based on the code codes recorded in the phoneme code table. In the present application, the “phoneme” means a voice that can be recognized as a group of sounds when it is pronounced, and in the case of Japanese, it substantially corresponds to the pronunciation of one phonogram.

In the preceding stage described here, a human voice is synthesized for each phoneme unit. First, a process of capturing a human pronunciation waveform as data for one phoneme is performed. For example, if you ask a human to pronounce the letter “sa” and import it as data,
A sound waveform W as shown in FIG. In the embodiment described here, this sound generation waveform W is sampled at a predetermined period by using a general PCM technique, and is taken into a computer as digital data.

Subsequently, a plurality of unit sections are set on the time axis of the sound generation waveform W. FIG. 1B shows a plurality of unit sections d corresponding to the time axis t of the sound generation waveform W shown in FIG.
An example in which 1, d2, d3,... Are set is shown. When setting the unit section, as shown in the example in the figure,
It is preferable that adjacent unit sections partially overlap on the time axis. In this example, the sounding waveform is 22 kHz
The sampling frequency is fetched as digital data, the section length of each unit section is set to 1024 samples (approximately 47 msec), and the head of an adjacent unit section is set to 20 samples (approximately 0.9 msec).
Each unit section is set while shifting in order by msec).

When the setting of the unit section is completed, a code code indicating the representative frequency and its intensity included in the sound waveform W in the unit section is created for each unit section, and a code code corresponding to one phoneme is generated. A code code definition process for defining a group is executed. Hereinafter, this code code definition processing will be described based on a specific example using Fourier transform.

First, a unit section d as shown in FIG.
, D2, d3,... Are set, Fourier transform is performed on each of the sound waveforms W belonging to each unit section,
Create a spectrum. FIG. 2A shows an example of a spectrum created for the unit section d1. In this spectrum, the frequency components (0 to Fs: where Fs is a sampling frequency) included in the sound waveform W in the unit section d1 are indicated by the frequency f defined on the horizontal axis, and on the vertical axis. The complex intensity A for each frequency component is indicated by the defined complex intensity A. As a method for obtaining such a spectrum, various methods are known in addition to the Fourier transform, and any method may be used. Further, if a method of directly creating a spectrum from an analog waveform is used, the sounding waveform W
There is no need to digitize using the CM method.

Next, a plurality of code codes are discretely defined corresponding to the frequency axis f of the spectrum. In this example, a note number N used in MIDI data is used as a code code, and 1 from N = 0 to 127 is used.
28 code codes are defined. Note number N
Is a parameter indicating a musical scale of a note. For example, a note number N = 69 indicates a “ra tone (A3 tone)” at the center of the keyboard of a piano, and corresponds to a sound of 440 Hz. As described above, since a predetermined frequency is associated with each of the 128 note numbers, 128 note numbers N are discretely defined at predetermined positions on the frequency axis f of the spectrum. Become.

Here, the note number N indicates a logarithmic scale in which the frequency doubles when the octave is increased by one octave, and thus does not correspond linearly to the frequency axis f. Therefore, the frequency axis f is represented by a logarithmic scale, and an intensity graph in which the note number N is defined on the logarithmic scale axis is created. FIG. 2B shows an intensity graph for the unit section d1 created in this way. The horizontal axis of this intensity graph is obtained by converting the horizontal axis of the spectrum shown in FIG. 2A to a logarithmic scale, and note numbers N = 0 to 127 are plotted at equal intervals. On the other hand, the vertical axis of this intensity graph is obtained by converting the complex intensity A of the spectrum shown in FIG. 2A into the effective intensity E, and indicates the intensity at the position of each note number N. Generally, the complex intensity A obtained by Fourier transform is represented by a real part R and an imaginary part I, but the effective intensity E can be obtained by an operation of E = (R ² + I ² ) ^1/2 .

The intensity graph of the unit section d1 obtained in this manner shows the ratio of each frequency component corresponding to the note number N = 0 to 127 with respect to the frequency components included in the sound waveform W belonging to the unit section d1 as the effective intensity. It can be called a graph shown. Therefore, P note numbers are selected from a total of 128 note numbers based on each effective strength shown in the strength graph. Here, for convenience of explanation, it is assumed that P = 4 and four note numbers are selected from a total of 128 candidates. The frequencies corresponding to the four note numbers need to be representative frequencies indicating representative values of the frequency components included in the sound waveform W belonging to the unit section d1. In other words, by playing these four note numbers simultaneously, it is necessary to reproduce a sound as close as possible to the sounding waveform in the unit section d1. One method of selecting a note number that satisfies such requirements is shown in FIG.
This is a method of selecting four note numbers in descending order of the execution intensity E in the intensity graph shown in FIG. If a note number having a large execution strength E is selected in the unit section d1, the unit section d1 can be represented by the selected note number.

In the example shown in FIG. 2B, four note numbers Np (d1,1), Np (d1,2), Np (d
1,3) and Np (d1,4) are selected. When such a note number is selected, a value of the execution strength for each of these note numbers is also obtained. Now, in the example shown in FIG. 2B, the execution intensities of the selected note numbers are Ep (d1, 1) and Ep, respectively.
If (d1,2), Ep (d1,3), Ep (d1,4) are obtained, after all, the sound waveform W of the unit section d1 is approximately expressed by the following four sets of code codes. can do.

Code 1: Np (d1, 1), Ep (d1, 1) Code 2: Np (d1, 2), Ep (d1, 2) Code 3: Np (d1, 3), Ep ( d1,3) Code 4: Np (d1,4), Ep (d1,4) The processing for the unit section d1 has been described above. The same applies to the unit sections d2, d3, d4,. Is performed, and the sound waveform W of each unit section is approximately represented by the four sets of code codes. For example, for unit section d2, code code 5: Np (d2,1), Ep (d2,1) code code 6: Np (d2,2), Ep (d2,2) code code 7: Np (d2, 3), Ep (d2,3) Code code 8: Four data pairs of Np (d2,4), Ep (d2,4) are obtained.

FIG. 1C is a conceptual diagram of a code group obtained by the above-described coding. here,
One set of code codes is indicated by one quarter note, and each code code is arranged at a position corresponding to the time axis t of the sound generation waveform W. That is, the four notes 1-4 arranged on the position K1 shown by the broken line correspond to the above-mentioned code codes 1 to 4 for the unit section d1, and the position K2 shown by the broken line
The four notes 5 to 8 arranged above correspond to the above-described code codes 5 to 8 for the unit section d2. here,
Four tracks T1 to T to accommodate each code code
4 are defined, and four code codes for one unit section are separately accommodated in these four tracks T1 to T4. In the above example, since the unit lengths of the unit sections are common, the notes are all quarter notes of the same note length, but the pitch differs according to each note number. In the expression using musical notes shown in FIG. 1 (c), information on the strength is not shown, but the code code indicated by each musical note has information on its own strength. For example, the code code 2 represented by the note 2 arranged on the track T1 in FIG. 1C has a note number Np (d1, 2) and an intensity Ep as described above.
This is a code representing (d1, 2).

In the example shown here, when four tracks are accommodated in four tracks, the four code codes are sorted based on the frequency, and the code code having the highest frequency (in other words, the largest code code) is used. The track number T1 is stored in the track T1, and the code code having the lowest frequency (in other words, the lowest note number) is stored in the track T4. More specifically, FIG.
Among the four note numbers shown in (b), Np (d1,1) having the lowest frequency and its intensity Ep (d
1 (1) is accommodated in the track T4 of FIG. 1 (c), and the code code 2 composed of Np (d1,2) having the highest frequency and its intensity Ep (d1,2).
Are stored in the truck T1 in FIG. 1 (c). After all,
The track T1 contains the code code on the high note side, and the track T4 contains the code code on the low note side.

In this way, a group of code codes as shown in FIG. 1 (c) is defined for a specific phoneme "sa" as shown in FIG. 1 (a). The code code definition processing in the present invention is, as described above, for a specific phoneme,
This can be said to be a process of defining a predetermined code group as shown in FIG. Here, the code code definition processing for one phoneme “sa” is shown, but all phonemes requiring similar processing (for Japanese, 5
0 sound and muddy sound, relentless sound, prompting sound, etc.), and a unique code code group is defined for each phoneme. The table in which the code codes defined for each phoneme are recorded in this manner is referred to as a “phoneme code table” in this specification. In short, a phoneme code table is prepared by recording a code group as shown in FIG. 1C for a plurality of phonemes.

In order to define a code group for each phoneme, it is necessary to prepare a sounding waveform generated by an actual person for each phoneme. It is preferable to use one that is pronounced at intervals. Specifically, to those who have trained pronunciation such as announcer, with the same pitch and pitch,
We asked them to produce all the phonemes that consisted of the 50 sounds of Aiueo and all of the necessary phonemes, such as the muddy, relentless, and consonant sounds, and fetched them as reference phonetic waveforms for each phoneme. What is necessary is just to define a code group. As will be described later, since the tone strength, tone length, and pitch can be corrected, the accuracy of the tone length and pitch at the time of having the announcer generate the tone need not be so strict. If it is difficult to pronounce a single phoneme naturally, such as a consonant phoneme found in a foreign language, ask the appropriate words to be pronounced and cut out the part corresponding to the phoneme on the signal waveform. Can also be taken.

By decoding the code group prepared on the phoneme code table in this way, it is possible to approximately reproduce the sounding waveform of the original phoneme. For example, by reproducing four tracks of MIDI data simultaneously as shown in FIG. 1 (c) and obtaining a reproduced sound of four channels, a sound close to the sounding waveform of the original phoneme "sa" can be reproduced. Becomes possible. In order to make the reproduced sound as close as possible to human voice, it is preferable to perform reproduction using a MIDI sound source (a sound source generally called VOICE) having an acoustic waveform based on human voice.

It is not always necessary to adopt the MIDI format as the encoding format in the present invention. However, since the MIDI format is the most widespread as this type of encoding format, the MIDI format is practically used. Most preferably, code data is used. In MIDI format, "Note On"
Data or "note-off" data exists with "delta time" data interposed. "Note on"
The data is data that designates a specific note number N and a velocity V to instruct the start of performance of a specific sound, and the “note off” data is a specific note number N
And data indicating the end of the performance of a specific sound by designating the velocity and the velocity V. The “delta time” data is data indicating a predetermined time interval. Velocity V is a parameter indicating, for example, the speed at which a piano keyboard or the like is depressed (velocity at the time of note-on) and the speed at which a finger is released from the keyboard (velocity at the time of note-off). Or it indicates the strength of the performance end operation.

In this embodiment, as described above, for the i-th unit section di, four note numbers Np
(Di, 1), Np (di, 2), Np (di, 3),
Np (di, 4) are obtained, and for each of them, the effective intensities Ep (di, 1), Ep (di, 2), Ep (d)
i, 3) and Ep (di, 4) are obtained. Therefore, in the present embodiment, MIDI code data is created by the following method. First, note number N described in "note-on" data or "note-off" data
Is the obtained note number Np (di, 1),
Np (di, 2),... Are used as they are. on the other hand,
As the velocity V described in the “note-on” data or the “note-off” data, the obtained effective intensities Ep (di, 1), Ep (di, 2),.
The value is obtained by multiplying 127 by the square root of the effective intensity E after the standardization. That is, assuming that the maximum value of the effective intensity E is Emax, the value V obtained by the calculation of V = (E / Emax) ^1/2 · 127 is used as the velocity. Alternatively, a value V obtained by a calculation of V = log (E / Emax) .127 + 127 (when V <0, V = 0) may be used as the velocity. Also,
The “delta time” data may be set according to the length of each unit section.

Here, for convenience of explanation, four code codes are defined for one unit section, and these are separately arranged in four tracks so that 4-channel stereo reproduction is performed at the time of reproduction. A general device having a function of reproducing MIDI encoded data can perform stereo reproduction of 8 channels or 16 channels. Therefore, in practice, it is preferable to define eight or sixteen code codes for one unit section and to separately arrange them on eight tracks or sixteen tracks.

The code codes recorded in the phoneme code table can be integrated as needed. That is, in the code group shown in FIG.
When notes of the same pitch are continuously arranged on the same track, by integrating them, the number of notes can be reduced. For example, note 2 and note 7 arranged on track T1 have the same pitch, so that both can be integrated into a half note. Note that the actual code data includes not only pitch information (note number) but also strength information. However, even if the strengths are different, if the pitches are the same, there is a great problem in integrating them. Does not occur. If the strength was different,
For example, a process of adopting the larger intensity as the integrated code data intensity may be performed. In short, when there are code codes similar to each other under a predetermined condition for a plurality of adjacent unit sections, a process of replacing these similar code codes with an integrated code code extending over the plurality of unit sections is performed. Is possible.

§2. Classification Based on Vowels and Consonants If a phoneme code table can be prepared by the procedure of §1 described above, a desired phoneme can be reproduced for the time being.
However, in order to be able to reproduce an arbitrary phoneme at an arbitrary pitch and an arbitrary pitch in the procedure of §3 described later, a code code group defined for each phoneme is replaced with a code code contributing to a vowel. It is preferable to classify into a code code contributing to a consonant and record it in a phoneme code table. For example, FIG. 1 (c) shows a group of code codes corresponding to the phoneme "sa". Each code code shown here mainly contributes to vowels and mainly contributes to consonants. Can be classified as In other words, since the pronunciation waveform W of the phoneme "sa" shown in FIG. 1A includes a vowel component and a consonant component, each code code shown in FIG. That is, the one defined based on the vowel component and the one defined based on the consonant component are mixed. Each code is
The merit of classifying into those contributing to vowels and those contributing to consonants will be described later in §3.
When recording each defined code code in the phoneme code table, it is preferable to add information indicating which classification belongs to each code code.

In a general Japanese phoneme, "consonant" +
Since it takes the form of "vowel", the sounding time of the consonant precedes the sounding time of the vowel, and the "consonant part" on the time axis
It seems that the "vowel part" can be separated from the "vowel part", but according to the experiment performed by the present inventor, it is very difficult to separate the "consonant part" and the "vowel part" on the time axis. I understood. For example, the phoneme “sa” is composed of a combination of a consonant “S” and a vowel “A”.
In the code code group shown in (c), for example, code codes arranged at positions K1 to K3 corresponding to the preceding unit section contribute to the consonant "S", and positions K4 to K4 corresponding to the succeeding unit section. It has been found that even if a classification based on a time axis such that the code code arranged in K9 contributes to the vowel "A" is performed, the classification is not actually correct.

According to an experiment conducted by the inventor of the present application, more accurate classification can be performed by performing classification based on the level of frequency rather than classification based on the time axis. The simplest classification method for implementing the present invention is a classification method based on the level of the frequency. In general, vowels have low frequency components and consonants have high frequency components. Therefore, it is possible to roughly classify low-frequency code codes (small note numbers) as code codes contributing to vowels, and high-frequency code codes (high note numbers) as code codes contributing to consonants. is there.
The code group shown in FIG. 1 (c) is sorted based on the frequency as described above, so that the track T1
Contains a high-frequency code code, and the track T4
Contains a low frequency code code. Therefore, if a classification method based on the level of the frequency is applied, for example, as shown in FIG. 4, the code codes stored in the track T4 are all codes contributing to vowels, and are stored in the tracks T1 to T3. A simple classification can be made that all the code codes are codes that contribute to consonants.

According to an experiment conducted by the inventor of the present invention, when a code code group composed of eight tracks is defined, if a phoneme is composed of vowels and consonants, only one track on the lowest frequency side contributes to vowels. If the remaining seven tracks are code codes that contribute to consonants, relatively accurate classification can be performed.

Incidentally, like "A", "I", "U",
For code groups obtained for phonemes consisting only of vowels, all code codes contained in all tracks may be used as code codes contributing to vowels, and conversely, phonemes consisting only of consonants (for Japanese, With regard to the code code group obtained for (nonexistent), all code codes contained in all tracks may be used as code codes that contribute to consonants.

Although the above-described frequency-based classification method is useful in that the classification operation is simple, correct classification results are not always obtained when this classification method is applied uniformly. Not always. For example, as shown in FIG. 3, a code code is stored in each of four tracks T1 to T4 (the pitch of each note is omitted for convenience of illustration). Let us consider a case in which the encoded code contributes to a vowel and the other encoded code contributes to a consonant. According to the above-described classification method, even in such a case, the code codes contained in the track T4 are all assumed to contribute to the vowels, and the code codes contained in the tracks T1 to T3 are uniform. All are considered to contribute to consonants.

As the present inventor repeated experiments for defining code groups for various sound waveforms,
A rule that can be used for more accurate classification has been found. In other words, for the same phoneme, the pitch is changed or the pitch is changed, and the pitch is changed to produce a plurality of pronunciation waveforms.
When a code code group is defined for each of these sounding waveforms by the method described in §1, the code code contributing to the consonant has little change, while the code code contributing to the vowel has a large change. In other words, changes in pitch and pitch occur mainly in vowels. Therefore, even when the pitch or pitch is changed, a code code with little change can be considered to be a code code that contributes to a consonant, while a code code that greatly changes by changing the pitch or pitch contributes to a vowel. Can be considered as a code code.

If this property is used, more accurate classification is possible. For example, for the phoneme "sa", the announcer sounds at various pitches or at various pitches, and a plurality of different sounding waveforms are captured. At this time, it is desirable that the degree of change of the pitch and the pitch is rich, but a highly accurate trial of changing the pitch at equal intervals is not required at all. Then, a code code group is created for each of the plurality of tone waveforms, and among the created code code groups, a code code showing only a change within a predetermined allowable range is contributing to a consonant. The remaining code codes may be used as code codes that contribute to vowels. If this classification method is adopted, fairly accurate classification becomes possible even when vowels / consonants are separated as shown in FIG. That is, in the case of the example of FIG. 3, if the pitch or pitch at the time of sounding is changed, the pitch or intensity of a note without a circle is not significantly changed, whereas a note with a circle is added. For notes that are present, there will be significant changes in pitch and intensity.

By adopting the above classification method,
The code group for each phoneme prepared on the phoneme code table obtained by the procedure of the preceding part of the present invention can be classified into code codes that contribute to vowels and code codes that contribute to consonants.

§3. Procedure of the latter part of the present invention As described above, the phoneme code table is prepared by the procedure of the former part of the present invention. The procedure in the latter part of the present invention is a procedure for synthesizing a human voice corresponding to a specific phoneme using the phoneme code table.

When it is desired to synthesize a human voice corresponding to a specific phoneme, basically, synthesis instruction data that specifies a phoneme to be synthesized is prepared, and a synthesis instruction data is synthesized from a phoneme code table based on the synthesis instruction data. A code group defined for the target phoneme may be extracted, and the extracted code group may be reproduced using a predetermined sound source. For example,
In the case of synthesizing the words "Sakura has bloomed"
Synthetic instruction data specifying seven phonemes “Sakura Gasita” is prepared. Based on the synthetic instruction data, first, a code code group defined for the phoneme “sa” is extracted from the phoneme code table and reproduced. And the phoneme "ku"
May be extracted from the phoneme code table and reproduced, and... May be sequentially reproduced for each phoneme.

However, when the synthesis instruction data for specifying only phonemes is used, a so-called “stick reading” is used.
In this state, only the sound of "Sakura Gasita" is reproduced, and the intonation peculiar to human speech cannot be reproduced. As described above, when preparing the phoneme code table, a person who has trained pronunciation such as an announcer can give the same pitch and the same pitch the 50 sounds of Aiou ...
We asked all the phonemes that consist of repetitive sounds, prompting sounds, etc. to be pronounced, and defined the code codes for each phoneme based on this reference pronunciation waveform. It can only perform unnatural speech synthesis without any intonation.

In the present embodiment, a phoneme, a tone length, a pitch, and an intensity are designated in the synthesis instruction data, so that an arbitrary phoneme can be reproduced at an arbitrary pitch, an arbitrary pitch, and an arbitrary intensity. . This allows speech synthesis with a high degree of freedom. That is, in the present embodiment, as synthesis instruction data given to synthesize one phoneme,
A format including four codes as shown in FIG. 5 is used. The phoneme code is a code for specifying a phoneme to be synthesized, and for example, a character code such as an ASCII code or a JIS code can be used. The tone length code is a code for specifying the length when reproducing the phoneme to be synthesized. For example, a numerical value using a unit of real time sec can be used as it is. The pitch code is a code for specifying a pitch when a phoneme to be synthesized is reproduced, and for example, a general scale code used in music can be used as it is. Further, the intensity code is a code for specifying the intensity of the sound when the phoneme to be synthesized is reproduced, and for example, a numerical value indicating the relative amplitude of the reproduced sound can be used.

FIG. 6 is a diagram showing an example of specific combining instruction data described using the format shown in FIG. In this example, a singing voice composed of six phonemes “Sakura Sakura” is synthesized. For the six phonemes, as shown in the figure, a pitch code, a pitch code, and an intensity code are set variously, and a singing voice having a melody or intonation is synthesized.

When specific synthesis instruction data as shown in FIG. 6 is given, a singing voice synthesized by the following method may be actually reproduced. First,
Based on each code described in the first line of FIG. 6, the phoneme "sa" is reproduced at a pitch of 1 sec, a pitch of C3, and an intensity of 5 in the following procedure. First, based on the character code of “sa”, a code group defined for the phoneme “sa” is extracted by referring to the phoneme code table. This code code group is accommodated in a plurality of tracks as shown in FIG. 3, for example.
Classification is made as to whether the chord contributes to a vowel or a consonant. Subsequently, the length, pitch, and intensity of the extracted code code group are corrected. That is, in the case of the above example, the sound length at the time of reproduction is 1 sec.
, The pitch should be C3 and the intensity should be 8.

The correction of the tone length is a process of adjusting the delta time in the case of a MIDI code. For example, if the original code group recorded in the phoneme code table is defined based on the reference sounding waveform generated by the announcer with a tone length of 0.5 sec, each code code is defined on the time axis. In this case, it is sufficient to perform a correction process of expanding the image by two times. However, this tone length correction processing is performed only for code codes that contribute to vowels. This is because, as described above, since the change in tone length has the property of mainly appearing in vowels, if tone length correction processing is applied to code codes that contribute to consonants, an unnatural sounding waveform will be synthesized. Because it will be done. Therefore, for example, in the case of the example shown in FIG. 3, the note length of only the note with a circle is corrected.

On the other hand, in the case of a MIDI code, the pitch correction is a process of changing a note number.
For example, if the original code group recorded in the phoneme code table is defined based on the reference tone waveform generated by the announcer at the pitch of C2, the instruction by the pitch code is one octave higher. Since the pitch is C3, a correction process for raising each code code by one octave as a whole may be performed. However, this interval correction process is also performed only for code codes that contribute to vowels. This is because, as described above, a change in pitch has the property of mainly appearing in a vowel, so that when pitch correction processing is performed on a code code contributing to a consonant, an unnatural sounding waveform is synthesized. Because it will be. Accordingly, for example, in the case of the example shown in FIG. 3, the pitches of only the notes with circles are shifted upward by one octave.

In the case of MIDI code, the correction of the intensity is a process of changing the velocity. If a numerical value to be given to the intensity code is predetermined to be, for example, 5 as a standard value, the correction may be performed so that the velocity of each code code is 8/5 times. This intensity correction process may be performed uniformly for all code codes without distinction between code codes contributing to vowels and code codes contributing to consonants.

If reproduction is performed using a predetermined sound source based on the code codes subjected to the above-described corrections, based on the synthesis instruction data described in the first line of FIG. Will be reproduced. The reproduction of the synthesized sound based on the synthesis instruction data described in the second to sixth lines is performed in exactly the same manner.

As described above, according to the voice synthesizing method of the present invention, by changing the content of the synthesis instruction data, the phoneme, the pitch, the pitch, and the intensity can be freely selected, and an arbitrary singing voice can be synthesized. It becomes possible to do. For English,
The sentence is required to have the same pitch change as the singing voice, such as changing into a question sentence due to the difference in intonation of the sentence, but individual words need only be controlled in intensity by strong and weak accents. On the other hand, in the case of Japanese, sentences do not usually show inflection, but different words such as "chopsticks" and "bridges" and "rain" and "candy" are changed by changing the pitch instead of accents. Often expressed. Even in such a case, if the speech synthesis method according to the present invention is used, it is possible to perform correct word synthesis in consideration of the pitch.

§4. Speech Synthesis Apparatus According to the Present Invention Finally, an apparatus configuration for implementing the above-described speech synthesis method will be briefly described with reference to the block diagram of FIG. The main components of the speech synthesizer shown in FIG.
A data input unit 10; a phoneme code table 20; a code code correction unit 30;

The data input unit 10 has a function of inputting synthesis instruction data D for specifying a specific phoneme to be synthesized and its pitch, pitch and intensity. The synthesis instruction data D is provided, for example, in a format as shown in FIG. If data of such a format is prepared for the required number of phonemes, words and lyrics can be expressed as continuous phonemes.

On the other hand, the phoneme code table 20 includes, for each required phoneme, a code code group for reproducing a sounding waveform of a person who has sounded at a reference pitch and a reference pitch, a code code and a consonant that contribute to a vowel. Are recorded in the form of code codes contributing to. More specifically,
For each of the 50 phonemes, and the phonemes that are dull sounds, resounding sounds, and prompting sounds, the announcer etc. provide the reference pitch (for example,
0.5 sec) and a reference pitch (for example, C2 tone), and a code code group is created for each phoneme according to the procedure described in §1 based on the sound waveform. The group may be recorded as the phoneme code table 20. At this time, using the method described in §2,
For each code code, whether it is a code code that contributes to vowels,
A code code contributing to a consonant is also recorded.

The code code correcting section 30 is provided with the data input section 1
0 is read from the phoneme code table 20 based on the synthesis instruction data D input as a code code group for a specific phoneme to be synthesized, and code codes that contribute to vowels in the read code code group are: It has a function of correcting the pitch and pitch based on the synthesis instruction data D. For example, when specific synthesis instruction data as shown in FIG. 6 is given, as described in §3, first, based on the phoneme code “sa” described in the first row, the phoneme code table 20, a code code group defined for the phoneme “sa” is extracted. Subsequently, based on the pitch code and the pitch code, correction is performed on the code codes contributing to the vowels in the extracted code code group so that the pitch at the time of reproduction is 1 sec and the pitch is C3. . Further, based on the intensity codes, correction is performed on all the extracted code code groups so that the intensity at the time of reproduction becomes 8. In the case of the MIDI code, the correction for the tone length is performed by correcting the delta time, the correction for the pitch is performed by correcting the note number, and the correction for the intensity is performed by correcting the velocity.

The sound reproducing unit 40 uses a predetermined sound source to
It has a function of reproducing the corrected code code group and producing a synthesized voice of a human voice, and in the example shown in the figure, is composed of a control unit 41, a sound source unit 42, an amplifier unit 43, and a speaker 44. The control unit 41 is a part that performs basic reproduction processing based on the corrected code code group provided from the code code correction unit 30. The sound source section 42 generally has a “VOIC
This is a type of MIDI sound source called "E", which stores an acoustic waveform created based on human voice and has a function of providing an acoustic waveform corresponding to predetermined MIDI data. The control unit 41 sequentially reads out necessary acoustic waveforms from the sound source unit 42 based on a given code code group (in this case, MIDI data), synthesizes a reproduced sound, and converts the synthesized sound into an analog signal. Output as This analog signal is amplified by the amplifier unit 43 and the speaker 4
4 will be reproduced as audio.

In each component shown in FIG. 7, the code code correction unit 30 can be realized by incorporating dedicated software into a general-purpose computer, and the phoneme code table 20 is stored in a storage device (accessible by this computer). (A memory or various storage media). The data input unit 10 can be realized by various input devices for inputting the synthesis instruction data D to the computer, and the audio reproduction unit 40 can be realized by a MIDI device for a computer.

By using this speech synthesizer, it is possible to perform a speech synthesis in which the pitch and pitch are freely set, instead of a so-called "stick reading" speech synthesis. It is also possible to synthesize singing voices according to the melody. In addition, since the synthesis instruction data D does not need to include an actual tone generation waveform, the information amount of the synthesis instruction data D itself can be extremely low. Therefore, the synthesis instruction data D is
Like general MIDI data, it is suitable for distribution via a communication line, and the speech synthesizer according to the present invention can be expected to be used in the field of communication karaoke. Alternatively, since the synthesis instruction data D can be expressed by a barcode or the like, if the data input unit 10 is provided with a barcode reader, the synthesis instruction data D can be displayed on a printed material containing music scores and lyrics. It is also possible to print and distribute in the form of a code, and it is also possible to transmit this printed matter by a method such as facsimile.

[0067]

As described above, according to the speech synthesizing method and the speech synthesizing apparatus according to the present invention, human speech can be efficiently processed.
In addition, synthesis can be performed with a degree of freedom in pitch and pitch.

[Brief description of the drawings]

FIG. 1 is a diagram showing a basic concept of a phoneme encoding process which is a former part of a speech synthesis method according to the present invention.

FIG. 2 is a diagram illustrating a method of selecting a note number indicating a representative frequency for each unit section in the encoding process illustrated in FIG.

FIG. 3 is a diagram showing a state in which code data contributing to vowels and code data contributing to consonants are classified for each code data contained in four tracks in the encoding process shown in FIG. 1;

FIG. 4 is a diagram showing a state in which code data contributing to vowels and code data contributing to consonants are classified for each track.

FIG. 5 is a diagram showing an example of a format of synthesis instruction data for indicating a phoneme to be synthesized in the voice synthesis method according to the present invention.

FIG. 6 is a diagram showing specific combining instruction data described in the format shown in FIG. 5;

FIG. 7 is a block diagram illustrating a basic configuration of a speech synthesis device according to an embodiment of the present invention.

[Explanation of symbols]

1 to 8: Code data (note) 10: Data input unit 20: Phoneme code table 30: Code code correction unit 40: Voice reproduction unit 41: Control unit 42: Sound source unit 43: Amplifier unit 44: Speaker A: By Fourier transform Obtained complex intensity D: synthesis instruction data d1 to d9: unit interval E: effective intensity f: frequency Fs: sampling frequency K1 to K9: position on the time axis corresponding to unit interval d1 to d9 N: note number Np (d1 , 1) to Np (d1, 4) ... note numbers selected for unit section d1 T1 to T4 ... track W ... sounding waveform

Claims (12)

[Claims] 1. A method for synthesizing a human voice corresponding to a specific phoneme, wherein a human pronunciation waveform is taken as data for one phoneme, and a plurality of unit sections are set on a time axis of the pronunciation waveform. It is necessary to perform a code code definition process for creating a code code indicating a representative frequency and its intensity included in a sound waveform in the unit section for each individual unit section and defining a code code group corresponding to the one phoneme. A phoneme code table in which code groups defined for each phoneme are recorded, and when a synthesis instruction for synthesizing a human voice corresponding to a specific phoneme is given, the phoneme By referring to a code table, a code group defined for the specific phoneme is extracted, and the extracted code group is reproduced using a predetermined sound source. Speech synthesis method characterized by synthesizing a human voice through the. 2. The speech synthesis method according to claim 1, wherein when setting a plurality of unit sections on the time axis of the sounding waveform,
A speech synthesis method characterized in that settings are made such that adjacent unit sections partially overlap on a time axis. 3. The speech synthesis method according to claim 1, wherein a plurality of P code codes are created by defining a plurality of P representative frequencies for one unit section, and these code codes are represented by P code. A speech synthesizing method characterized in that the code codes stored in the P tracks are simultaneously reproduced to obtain a P channel reproduced sound when the code codes are reproduced in separate tracks. 4. The speech synthesis method according to claim 1, wherein code groups defined for each phoneme are classified into code codes contributing to vowels and code codes contributing to consonants. Recorded in the phoneme code table, the synthesis instruction for the specific phoneme includes a parameter indicating the length of sound for generating the specific phoneme, and when performing reproduction based on the synthesis instruction, A voice synthesizing method, wherein a code length that contributes to a vowel among code codes extracted from a code table is corrected based on a parameter indicating the voice length. 5. The speech synthesis method according to claim 1, wherein a group of code codes defined for each phoneme is classified into a code code contributing to a vowel and a code code contributing to a consonant. It is recorded in a phoneme code table, and a synthesis instruction for a specific phoneme includes a parameter indicating a pitch at which the specific phoneme is pronounced. When performing reproduction based on the synthesis instruction, a phoneme code A voice synthesizing method characterized in that, among code codes extracted from a table, a code code contributing to a vowel is corrected in pitch based on a parameter indicating the pitch. 6. The speech synthesis method according to claim 4, wherein a plurality of P code codes are created by defining a plurality of P representative frequencies for one unit section, and these code codes are converted into frequency codes. For the code codes obtained for phonemes consisting of vowels only by separating them into P tracks by sorting on the basis of For code groups obtained for phonemes consisting only of consonants, all code codes contained in P tracks are used as code codes contributing to consonants, and code codes obtained for phonemes containing vowels and consonants are used. For the group, the code codes stored in the n tracks on the low frequency side are used as code codes contributing to vowels, and the remaining (P-n) Speech synthesis method characterized by the code code contained in the click and contribute code code consonants. 7. A speech synthesis method according to claim 4 or 5, wherein, for a code group obtained from a phoneme composed only of vowels, all code codes are code codes contributing to vowels, and a phoneme composed only of consonants is obtained. For the code group obtained for, all code codes are considered to be code codes contributing to consonants, and for the code code group obtained for phonemes including vowels and consonants, the phonemes are generated by changing the pitch. A plurality of types of sound waveforms are captured, a code code group is created for each of the plurality of sound waveforms, and a code code that indicates only a change within a predetermined allowable range among the plurality of code code groups created. A speech synthesizing method, wherein code codes contributing to consonants are used, and the remaining code codes are code codes contributing to vowels. 8. A speech synthesis method according to claim 4 or 5, wherein, for a code group obtained for a phoneme consisting only of vowels, all code codes are code codes contributing to vowels, and a phoneme consisting only of consonants is obtained. For the code group obtained for, all code codes are considered to be code codes contributing to consonants, and for the code group obtained for phonemes including vowels and consonants, a plurality of codes are generated by changing the pitch of the phoneme. A plurality of pronunciation waveforms are fetched, a code group is created for each of the plurality of pronunciation waveforms, and a code code showing only a change within a predetermined allowable range in the created plurality of code code groups is consonant. A speech synthesis method characterized by using code codes contributing to vowels and code codes contributing to vowels. 9. The speech synthesis method according to claim 1, wherein a representative frequency included in a sound waveform in a unit section is indicated by a note number, and an intensity of the representative frequency is indicated by a velocity. By representing the time corresponding to the section length of the unit section by the delta time, the MIDI format code data is created, and when reproducing the created MIDI format code data, the MIDI format has an acoustic waveform based on human voice. A speech synthesis method using a sound source. 10. The voice synthesizing method according to claim 1, wherein, for a plurality of adjacent unit sections, if there are code codes similar to each other under a predetermined condition, these similar code codes are replaced by: A speech synthesizing method characterized by performing a process of replacing with an integrated code code spanning a plurality of unit sections. 11. An apparatus for synthesizing a human voice corresponding to a specific phoneme, comprising: a data input unit for inputting synthesis instruction data for designating a specific phoneme and a sound length to be synthesized; For phonemes, a phoneme code table in which code groups for reproducing a pronunciation waveform of a human pronounced at a reference pitch are classified into code codes contributing to vowels and code codes contributing to consonants, and the phoneme code table is recorded. Based on the instruction data, a code group for a specific phoneme to be synthesized is read from the phoneme code table, and code codes that contribute to vowels in the read code group are read based on the synthesis instruction data. Using a code code correction unit for correcting the tone length, and using a predetermined sound source, reproduce the corrected code code group,
A voice synthesizer, comprising: a voice reproducing unit configured to generate a synthesized voice of a human voice. 12. A device for synthesizing a human voice corresponding to a specific phoneme, comprising: a data input unit for inputting synthesis instruction data for designating a specific phoneme to be synthesized and a pitch; A phoneme code table in which code groups for reproducing a human pronunciation waveform pronounced at the reference interval are classified into code codes contributing to vowels and code codes contributing to consonants, and the synthesis instruction data is recorded. , A code group for a specific phoneme to be synthesized is read from the phoneme code table, and a code code that contributes to a vowel in the read code group is read out of a pitch based on the synthesis instruction data. Using a code code correction unit for performing correction, and a predetermined sound source, reproducing the corrected code code group,
A voice synthesizer, comprising: a voice reproducing unit configured to generate a synthesized voice of a human voice.