JP2002525663A - Digital voice processing apparatus and method - Google Patents

Abstract

(57) [Summary] A digital speech processing apparatus having a prosody generation means for generating a text prosody and an editing means for displaying and correcting the generated prosody.

Description

DETAILED DESCRIPTION OF THE INVENTION

The present invention relates to an apparatus and a method for digital audio processing or audio generation. Systems that digitally output speech have traditionally been applied where synthesized speech is acceptable or desired. The present invention relates to a system that enables the synthetically generating speech to give a natural impression.

In a system for digital speech generation, information about prosody and intonation is automatically generated, for example as described in EP 0 689 706. In some systems, it is possible to insert additional commands into the text screen and then pass them on to the speech generating means, for example as shown in EP0598599. These commands are entered, for example, as special characters that are not pronounced, as described, for example, in EP0598598.

A command inserted into a text screen may include a notation related to a characteristic of a speaker (for example, a parameter of a speaker model). EP0762384 describes these speaker characteristics in a graphical user interface.
) Describes a system that can be inserted on a screen.

[0004] Speech synthesis is based on, for example, a waveform sequence in the case of EP0831460.
(As) using the auxiliary information stored in the data bank. However, regarding the pronunciation of a word that is not stored in the data bank, it is necessary to set rules for pronunciation in the program. Distortion and artefacts will occur if only individual series are combined and no special measures are taken. This problem (typically "
The term "segmental quality" has been largely resolved today (eg, "Volker Kraft: Concatenation of natural language components for speech synthesis:
Conditions, skills and evaluation. German Institute of Engineers Status Report (VDI) Series 10, No. 46
8, VDI Publishing, 1997 ("Volker Kraft; Verkettung naturlichsprachlicher Baust
ein zur Sprachsynthese: Anforderungen, Techniken und Evaluierung. Fortsc
hr-Ber. VDI Reihe 10, Nr. 468, VDI-Verlag 1997 ")"). However, there are some problems in recent speech synthesis.

One problem with digital audio output is, for example, multilingual ability.

Another problem concerns the improvement of prosodic quality, the quality of intonation. For example, (e.g., "Volker Kraft: Concatenation of Natural Language Components for Speech Synthesis: Conditions, Techniques and Evaluations. Status Report of the German Institute of Engineers (
VDI) Series 10, No. 468, VDI Publishing, 1997 ("Volker Kraft; Verkettung natur
lichsprachlicher Baustein zur Sprachsynthese: Anforderungen, Techniken u
nd Evaluierung. Fortschr-Ber. VDI Reihe 10, Nr. 468, VDI-Verlag 1997 ")
See. This issue is due to orthographic input information
) Alone is not enough to build intonation. It also depends on higher levels such as semantics and pragmatics, the situation of the speaker, and the type of speaker. In general, the quality of today's audio output systems is acceptable if the listener expects or tolerates synthesized speech. However, often the quality of the synthesized speech is considered to be insufficient or unsatisfactory.

Accordingly, it is an object of the present invention to provide an apparatus and a method for digital voice processing capable of generating a synthesized voice with improved quality. Another object of the present invention is to
It is to generate a speech that gives a natural impression synthetically. The scope of application covers from simple text sound generation for multimedia applications to sound generation for movie dubbing, radio dramas and audio books.

[0008] Even when synthetically generated speech gives a natural impression,
cal) It is necessary to be able to intervene in order to produce an effect. Another object of the invention is to provide the possibility of such an intervention.

[0009] The invention is defined in the independent claims. The dependent claims define a particular embodiment of the invention.

[0010] The object of the present invention is substantially solved by providing the ability to modify the prosody generated for text by an editor.

Certain embodiments of the present invention allow for the addition of additional properties to synthetically generated speech in addition to prosody editing.

[0012] Thus, the starting point is written text. However, in certain embodiments, the user has a wide range of possibilities to intervene in order to achieve sufficient quality (especially with respect to prosody) and to produce a staging effect. Users have a supervisory role, define speakers on the system and give them voice rhythms and prosody,
Assign pronunciation and intonation.

The present invention provides for generating phonetic transcriptions for textual text, and based on rules that modify or modify the generated phonetic transcriptions,
Preferably, it has the ability to modify the phonetic transcription. This can, for example, generate a particular accent for the speaker.

In yet another preferred embodiment of the invention, a dictionary means is provided in which words in one or more languages are stored together with their pronunciation. In the latter case,
Has multilingual processing ability. That is, texts in different languages can be processed.

[0015] Preferably, the editing of the generated phonetic notation or prosody is performed by an editor that is readily available, such as, for example, a graphical user interface.

In yet another embodiment of the present invention, a predetermined or user-defined or modified speaker model is considered in the speech processing. Thus, different speaker characteristics are realized, whether male or female, and different speakers, such as, for example, Bayerischer, Schwabischer, or Northern Germany. it can.

In a particularly preferred embodiment, the device comprises: dictionary. In this dictionary, pronunciations for all words are stored in phonetic notation (hereinafter "phonetic notation" means an arbitrary phonetic notation. One example is SAMPA notation (SAM
PA-notation). See, for example, "Multilingual Speech Input / Output Evaluation, Methodology and Standardization, Standard Computer-Compatible Notation, Pages 29-31, Split Project 258
9 (SAM) Fin Report SAM-UCC-037 ("Multilingual speech input / output asse
ssment, methodology and standardization, standard computer-compatible tr
anscription, pp 29-31, in Esprit Project 2589 (SAM) Fin Report SAM-UCC-0
37 "). ) Or educational materials, such as "International Phonetics Association Standards: International Phonetics and Their Use. International Phonetics Association, Department of Phonetics, University College London (" The Principles of the International Phonetik Association: Ad
escription of the International Phonetic Alphabet and the Manner of Usin
g it.International Phonetic Association, Dept. Phonetics, Univ. Colleg
e of London ". ) International Phonetic
script). Translator (conversion means). This translator converts the input text into phonetic transcriptions and generates prosody. Editor (editing means). The editor allows text to be entered, assigned to speakers, and the resulting phonetic transcription and prosody to be displayed and modified. Input module. With this input module, a speaker model can be defined. A system for digital speech generation. The system has the ability to generate from phonetic transcriptions and prosodic signals representing spoken speech or data representing such signals and to process different speaker models. System composed of digital filters, etc. (for reverberation, echo, etc.) With this system, certain effects can be produced. Sound archive. Mixing device. In this mixing device, the generated audio signal can be mixed with the sound from the archive, and can be corrected by the sound effect. The invention can be implemented in hybrid form by software and hardware, or entirely in software. The generated digital audio signal is
It can be output by a specific device for digital audio or by a sound board of a personal computer (PC).

Hereinafter, the present invention will be described with reference to the drawings according to some embodiments.

FIG. 1 is a block diagram of an apparatus for generating digital audio according to an embodiment of the present invention.

In the embodiments of the present invention described below, the present invention has one or more digital processing devices and some components that can be realized by a combination thereof. These operations are described below.

The dictionary 100 includes a simple table (one for each language) in which words of the language are stored along with pronunciations. The table may optionally be expanded to include additional words and their pronunciation. For a specific purpose, for example for generating a rounding, an additional table may be provided which records different audio data in one language. One table in the dictionary is assigned to each different speaker.

The translator 110 generates phonetic notations by replacing words of the input text based on phonetic correspondence in the dictionary. In the speaker model, when modifiers (correction means) described later are stored, those modifiers are used to correct the pronunciation.

Further, a prosody is generated using a learning function (heuristics) known in speech processing. Such a learning function is described in, for example, the model of Fujisaki (199).
2) or other acoustic methods and then perceptual models such as d'Alessandro and Mertens
(1995). Both older language models are described, for example, in "Thierry Dutoit: An Introduction to Text-to-Speech Synthesis, Clubert 1997 (" Thierry Dutoit: An Intr
oduction to Text-to-Speech Synthesis, Kluwer 1997 ")".
It also describes a method of segmentation (setting of breaks), also performed by the translator.

The choice of method is a minor issue. The translator only generates a version of the prosody that can be modified by the user.

The editor 120 provides the user with pronunciation, intonation, and stress (accent
), speed, volume, pause, etc.
(strument).

First, a user assigns a speaker model to text segments to be processed. The composition and operation will be described later in detail. In response to this assignment, the translator adapts the phonetics and possibly the prosody to the speaker model and generates a new one. The phoneme is displayed to the user in the form of phonetic notation, and the prosody is displayed, for example, in symbolic notation (score) adopted from music. The user can modify this, listen to individual text segments, re-enter, and so on.

The text itself can be saved in an editor if it cannot be captured directly from another text processing system.

The speaker model 130 is, for example, parameterized for speech generation. These models model the characteristics of human vocal organs. The function of the vocal cords is represented by a sequence of pulses, of which only the frequency (pitch) can be corrected. Other characteristics of the vocal organs (oral, nasal) can be realized by digital filters. Those parameters are stored in the speaker model. Standard models (children, young women, elderly men, etc.) are stored. Based on these, the user can generate and store additional models by selecting and correcting parameters. The stored parameters are used together with the prosody information for intonation at the time of speech generation described later.

[0029] Special characteristics of the speaker, such as dullness or speech impairment, can also be entered. These are used by the translator to modify the pronunciation. A simple example of such a modification is to replace "∫t" with "st" in phonetic notation (to generate a rounding out of Hamburg).

The speaker model relates, for example, to the rules that the translator follows in generating phonetic transcriptions. Different speaker models follow different rules. However, the speaker model may correspond to a set of filter parameters for processing an audio signal according to a specified audio characteristic. Of course, various combinations of aspects of these two speaker models are also conceivable.

The role of the speech generation unit 140 is to provide, based on the given text and phonological and prosodic additional information (generated by a translator and edited by an editor), numerical data representing a digital speech signal Stream (numerical da
ta stream). This data stream is output to the output device 150.
(A digital audio device or a sound board in a PC) is converted into an analog sound signal (text to be output).

To generate speech, conventional text-to-speech conversion methods (pronunciation and prosody are pre-generated) can be used. In general, rule-based synthesizers and concatenation-based synthesizers
nation-based synthesizers).

A rule-based synthesizer operates using rules for the production of sounds and sound-to-sound transitions. These synthesizers operate with less than 60 parameters, but determining these parameters is labor intensive. However, very good results are obtained with these types of synthesizers. An overview and details of these types of systems can be found in Thierry Dutoit: An Introduction to Text-to-Speech Synthes.
is, Kluwer 1997 ")".

A synthesizer based on concatenation is easy to operate. These operate using a database that stores every sound pair. These are easily connected. However, forming a system that achieves good quality requires high computing power.
These types of systems are described in "Thierry Dutoit: An Introduction to Text-to-Speech Synthes.
is, Kluwer 1997 ")" and (for example, "Volker Kraft: Concatenation of Natural Language Components for Speech Synthesis: Conditions, Techniques and Evaluations. Status Report of the German Institute of Engineers (
VDI) Series 10, No. 468, VDI Publishing, 1997 ("Volker Kraft; Verkettung natur
lichsprachlicher Baustein zur Sprachsynthese: Anforderungen, Techniken u
nd Evaluierung. Fortschr-Ber. VDI Reihe 10, Nr. 468, VDI-Verlag 1997 ")
"It is described in.

In principle, any type of system can be used. In rule-based synthesizers, prosodic information has a direct effect on rules. In systems based on concatenation, they are stacked in an appropriate manner.

For generating the specific sound effect 160, a known technique for digital signal processing, for example, a digital filter (for example, a band-pass filter for a telephone effect), a reverberation generator, or the like can be used. These can also be applied to sounds stored in the archive 170.

In the archive 170, road noise, railway noise, children's noise, sea noise,
Sounds such as background music are stored. The archive can be arbitrarily expanded. The archive may be simply a collection of digitized sound files, or a database in which sound is stored as BLOBs (binary large objects).

In the mixing device 180, the generated audio signal is combined with a background sound. The volume of all signals can be adjusted before the combination. Further, sound effects can be applied to each signal individually or in common.

The signal thus generated is passed to a suitable device 150 for digital audio, for example a sound board of a PC, and is acoustically checked or acoustically output. Further, a storage means (not shown) is provided so that the signal can be stored and transmitted to an intended medium later in an appropriate manner.

As the mixing device, a device realized by hardware has been known for a long time, and it may be used, or may be realized by software. Also, it may be formed as a part of a program.

Those skilled in the art will readily be able to modify the above embodiments. For example, as a further embodiment of the present invention, instead of the output device 150, a further computer connected to the mixing device 180 via a network can be used. In this case, the generated audio signal can be transmitted to another computer via a computer network, for example, the Internet.

In still another embodiment, the audio signal generated by the audio generation unit 140 can be directly transmitted to the output device 150 without passing through the mixing device 180. Further similar modifications will be apparent to those skilled in the art.

[Brief description of the drawings]

FIG. 1 is a block diagram of an apparatus for generating digital audio according to an embodiment of the present invention.

──────────────────────────────────────────────────続 き Continued on the front page (71) Applicant 2, Kalkarra Cr, Mt. Duneed Vic 3216, Australia

Claims (21)

[Claims] 1. A digital speech processing apparatus comprising: a prosody generation means for generating a prosody for a text; and an editing means for displaying and modifying the generated prosody. 2. The apparatus according to claim 1, further comprising a conversion unit for converting the text to a phonetic transcription, wherein the conversion unit includes a unit for displaying and correcting the generated phonetic transcription. Equipment. 3. The method according to claim 1, wherein the prosody generation means and / or the conversion means generate the prosody and / or the phonetic notation based on or dependent on a specific speaker model. The described device. 4. Apparatus according to claim 1, further comprising means for selecting and / or modifying one or more speaker models. 5. The apparatus according to claim 4, wherein the means for modifying the speaker model comprises means for modifying a phonetic transcription element for rounding. 6. A device for digital speech processing according to claim 1, wherein the speech is based on a phonetic notation that may have been edited using the editing means and / or based on the prosody. Means for generating a signal. 7. The apparatus according to claim 6, wherein said voice signal generating means further comprises speaker model processing means for generating said voice signal based on or dependent on a specific speaker model. 8. The method of claim 7, wherein said speaker model processing means comprises at least one of: a digital filter system; and means for employing a set of filter parameters representing a particular speaker model. apparatus. 9. Apparatus according to claim 7, wherein said speaker model processing means comprises means for selecting and / or modifying a speaker model. 10. The apparatus according to claim 6, further comprising sound effect generating means for generating a sound effect. 11. The apparatus according to claim 10, wherein said sound effect generating means has at least one of digital filter means for modifying the generated audio signal, and reverberation generating means for generating a reverberation effect. The described device. 12. An apparatus according to claim 6, further comprising: archiving means for storing a sound; and mixing means for mixing the generated audio signal with the sound stored in the archiving means. An apparatus according to claim 1. 13. Apparatus according to claim 1, further comprising a graphical user interface for editing the generated phonetic transcription and / or prosody. 14. Apparatus according to claim 1, further comprising means for modifying speech rhythm and / or pronunciation and / or intonation. 15. The apparatus according to claim 1, further comprising display means for displaying prosody by symbolic notation. 16. Apparatus according to claim 1, further comprising dictionary means for storing words in one or more languages together with their pronunciation. 17. The apparatus according to claim 16, wherein different voice data is stored in the dictionary means for at least one dictionary entry. 18. The apparatus according to claim 6, further comprising means for converting said digital audio signal into an audio signal. 19. A digital speech processing method comprising: generating a prosody of a text; displaying the generated prosody; and editing the generated and displayed prosody. 20. The method according to claim 1, further comprising generating digital voice.
20. The method according to claim 19, wherein the method according to claim 19 is used. 21. A method according to claim 19 or 20, comprising a medium readable by a computer for storing and transmitting digital data, in particular a data carrier, wherein the data to be stored and / or transmitted to a computer. A computer program product comprising a sequence of computer-executable instructions for causing a computer to execute.