CN1310209C - Speech and music regeneration device - Google Patents

Abstract

The voice/music reproducing device related to the present invention is made up of intermediate equipment such as regenerating synthesized voice signals, a sound source (20) and a loudspeaker for regenerating desired voice or melody according to the synthesized voice signals; wherein the intermediate equipment converts the script data (11) , user timbre parameters (12) and user phrase synthesis dictionary data (13) are combined, and regenerate synthetic speech signals etc. according to default timbre parameters (18) and default synthesis dictionary data (19). When an HV-script describing various events is used as scenario data, desired waveform data, music phrase data including note information, and formant frame data are reproduced in combination as appropriate according to the type of event.

Description

Speech and music reproduction device

technical field

The present invention relates to a voice and music reproduction device, in particular to a voice and music reproduction device which converts character information into voice or music reproduction while reproducing specific words through speech synthesis.

Background technique

Conventionally, there have been designed character string-to-voice conversion devices that convert character string information such as e-mails into voice output. Japanese Laid-Open Patent Publication No. 2001-7937 shows an example of a conventional string-to-speech conversion device, in which string information is divided into units of text, and the content is displayed on a display while outputting voice.

Known methods include a method of regenerating waveform data (or sampling data) made by sampling musical phrases or speech phrases, or by SMF (standard MIDI file) or SMAF (synthetic music mobile application file; synthetic music mobile application file) A method for reproducing the short sentence of the music after forming a short sentence of music by waiting for note information. For example, Japanese Laid-Open Patent Publication No. 2001-51688 discloses an e-mail reading device, which can separate the character string information and tone information in the e-mail, and reproduce the respective pronunciations.

However, in the existing string-to-speech conversion device, since the string information is divided into units of text sections (sentences (clause) or phrases) for voice output, the voice output is a pronunciation unit (or character unit) When the connection point of the pronunciation unit is reproduced, compared with the normal speaking voice, the listener feels disharmony. That is, in the conventional character string speech conversion device, it is impossible to change and output the sound with good sound quality for the entire text, in other words, it is not possible to output the natural speech close to the human speech language.

A method considered to solve the above-mentioned problem is, for example, to sample the speech of each stanza (hereinafter referred to as "phrase") in advance, store it as speech data, and output it as a corresponding speech waveform during reproduction. But, in order to improve the quality of speech output, this method must increase sampling frequency, therefore, just must save the speech data of large capacity, in the relatively limited device of storage capacity such as portable telephone (handy phone or mobile phone), will Accompanied by technical difficulties.

In addition, in the conventional method of reproducing waveform data generated from music or voice samples, or the conventional method of reproducing the music data by constructing one piece of music data from note information such as SMF or SMAF, the reproduction timing of the music or voice is not Since it is described in the form of a text file, it is difficult to combine voice reproduction based on character string information with waveform data reproduction or music data reproduction according to the user's intention.

Contents of the invention

In order to solve the above-mentioned problems, the object of the present invention is to provide a kind of voice reproduction device, which can make desired text sections (or phrases) made of character string information etc. .

Another object of the present invention is to provide a voice/music reproducing device in which a user can easily perform voice reproduction or a combination of waveform data reproduction and music data reproduction, thereby enabling voice and music reproduction faithful to the user's intention.

According to the speech reproducing apparatus of the present invention, a database composed of formant frame data corresponding to predetermined utterance units is stored as synthesis dictionary data, and the synthesis dictionary is used when giving information about character strings forming the utterance units in parallel. data for speech synthesis. Here, when formant frame data is replaced with arbitrary user phrase data and character string information is given, speech synthesis is performed using synthesis dictionary data replaced with the user phrase data. The timbre parameters of the processed formant frame data are added to the user phrase data. In addition, at the time of speech synthesis, a predetermined data exchange format including user phrase data is used. This data exchange format is, for example, the SMAF file format, and may include not only user phrase data but also various pieces and music reproduction information.

Specifically, the above-mentioned speech reproducing apparatus comprises default synthesis dictionary data storing formant frame data corresponding to predetermined pronunciation units, and an intermediate device, application program, and interface (API) for replacing the formant frame data with user phrase data. ), converter, driver and sound source. In this way, a desired phrase composed of character string information can be reproduced as a sound with good sound quality, and the sound can be reproduced with an appropriate change in timbre.

The audio/music reproducing device according to the present invention stores script data (that is, HV-script) in which the pronunciation of characters or reproduction instructions of data for pronunciation stored in advance are described. Based on the script data, a speech signal corresponding to the characters is generated to generate a desired speech, and a speech signal corresponding to speech data is generated to generate a desired speech or musical sound. Here, the data for utterance is composed of, for example, waveform data generated from sampled speech or music, and a synthesized utterance signal is generated based on the waveform data. Also, when the data for sound generation is music data including musical note information, a musical sound signal corresponding to the musical note information is generated based on the music data. Also, when formant control parameters (or formant frame data) that characterize the pronunciation of the above-mentioned characters are stored, an audio signal is generated based on the formant control parameters. The script data may be created arbitrarily by the user. In this case, the script data takes the form of a predetermined file created by text input.

Specifically, when explaining various events described in the HV-script, and when the type of the event indicates waveform data, the waveform data is read out and reproduced. On the other hand, when the type of the event indicates a phrase In the case of data, the playback processing of the music phrase data is performed. At this time, the note data is read out and reproduced based on the time information in the music phrase data. In addition, corresponding to other events, a character string input by synthesis dictionary data is converted into a formant frame string, and speech synthesis is performed. In this way, the user can easily perform reproduction of voice, reproduction of waveform data, and reproduction of music data in combination.

Description of drawings

FIG. 1 is a block diagram showing the configuration of a speech reproducing apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram showing the allocation relationship between pronunciation units and phrase IDs.

FIG. 3 is an example of the content of phrase synthesis dictionary data.

FIG. 4 is an example of the format of a SMAF file.

FIG. 5 is a functional block diagram showing an example of an HV authoring tool.

FIG. 6 is a block diagram showing the configuration of a portable communication terminal to which the voice reproduction device is applied.

FIG. 7 is a flowchart showing a process of creating user phrase synthesis dictionary data.

FIG. 8 is a flowchart showing a process of reproducing user phrase synthesis dictionary data.

FIG. 9 is a flowchart showing a SMAF file creation process.

Fig. 10 is a flowchart showing playback processing of SMAF files.

Fig. 11 is a block diagram showing the configuration of a voice/music reproducing apparatus according to a second embodiment of the present invention.

FIG. 12 is a diagram showing an example of the distribution relationship between each event, waveform data, and music phrase data.

Fig. 13 is a flowchart showing speech/music reproduction processing in the second embodiment.

Fig. 14 is a block diagram showing the configuration of a mobile phone provided with the audio/music reproducing device of the second embodiment.

Fig. 15 is a block diagram showing the configuration of a voice/music reproducing apparatus according to a third embodiment of the present invention.

Fig. 16 is a flowchart showing the operation of the speech/music reproducing device shown in Fig. 15 .

Detailed ways

Embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a speech reproducing apparatus according to a first embodiment of the present invention.

That is, the voice reproduction device 1 shown in FIG. 1 has application software (application software) 14, middleware API (middleware application program interface; middleware application program interface) 15, converter 16, driver 17, default tone parameters (default tone colorparameter) 18, default composition dictionary data (default composition dictionary date) 19 and sound source 20, input script data (script data) 11, user timbre parameter 12 and user phrase composition dictionary data (user phrase composition dictionary data) 13 (variable length) to reproduce the speech.

The speech reproduction device 1 basically adopts a method of reproducing speech by formant composition according to a CSM (composition sinusoidal modeling: composite sine wave model) speech synthesis method using FM (frequency modulation) sound source resources. In the present embodiment, user phrase synthesis dictionary data 13 is defined, and the voice reproduction device 1 assigns user phrases to tone color parameters in units of phonemes by referring to the user phrase synthesis dictionary data 13 . When the user phrase synthesis dictionary data 13 is assigned to the timbre parameters in this way, at the time of reproduction, the speech reproduction device 1 replaces the phonemes registered in the default synthesis dictionary data with user phrases, and then performs speech synthesis based on the replacement data. In addition, the above-mentioned "phoneme" is the smallest unit of pronunciation, and in the case of Japanese, it is composed of a vowel (vowel) and a consonant (consonant).

Next, the detailed configuration of the speech reproducing device 1 will be described.

In FIG. 1, scenario data 11 is data defining a data format for reproducing "HV (human voice): voice synthesized by the above method". That is, the script data 11 represents a data format for speech synthesis including a pronunciation character string including a prosody symbol (intonation symbol), event data for setting the sound of pronunciation, event data for controlling the above-mentioned application software 14, etc. , a form of text input to facilitate manual input by the user.

The definition of the data format in the script data 11 is language-dependent and may be defined in various languages, but in this embodiment, the definition in Japanese is adopted.

The user phrase synthesis dictionary data 13 and the default synthesis dictionary data 19 extract 8 groups of formant frequencies and formant frequencies by sampling and analyzing actual human voices with pronunciation character units (for example, "あ", "い" in Japanese, etc.). Intensity (formant level) and pitch are used as parameters, and the parameters are made into formant frame data in advance to correspond to the pronunciation character unit, which is equivalent to being stored in the database of the pronunciation character unit. The user phrase synthesis dictionary data 13 is a database built outside the intermediate device, and the user can arbitrarily log in the formant frame data to the database. Therefore, the registration content of the user phrase synthesis dictionary data 13 can be completely replaced by the default through the intermediate device API15 Stored contents of the synthesized dictionary data 19 . That is, the content of the default synthetic dictionary data 19 can be completely replaced with the content of the user phrase synthetic dictionary data 13 . On the other hand, the default synthetic dictionary data 19 is a database built in the intermediate device.

The user phrase synthesis dictionary data 13 and the default synthesis dictionary data 19 are preferably provided with two types for male voice quality and female voice quality respectively, and the voice output of the voice reproduction device 1 is changed according to the cycle of each frame, but are registered in the user phrase synthesis The frame period of the formant frame data in the dictionary data 13 and the default composite dictionary data 19 is set to, for example, 20 ms.

The user timbre parameters 12 and the default timbre parameters 18 are parameter groups for controlling the sound quality in the voice output of the voice reproduction device 1, that is, the user timbre parameters 12 and the default timbre parameters 18 can be configured, for example, with 8 groups of resonance frequencies and formant strengths. Change (that is, designation of the change amount of the resonance frequency and formant intensity registered in the user phrase synthesis dictionary data 13 and the default synthesis dictionary data 19) and the designation of the basic waveform for formant synthesis, thus, it is possible to create produce a variety of tones.

The default timbre parameter 18 is a timbre parameter pre-set in the intermediate device as a default value, and the user timbre parameter 12 is a parameter that can be arbitrarily made by the user, and is set and saved outside the intermediate device, and is passed through the intermediate device. API15 to extend the parameters of the content of the default timbre parameter 18.

The application software 14 is software for reproducing the scenario data 11 .

The intermediate device API 15 forms an interface between the application software 14 composed of software and the converter 16, driver 17, default tone color parameters 18, and default synthesis dictionary data 19 composed of intermediate devices.

The converter 16 interprets the script data 11 and uses the driver 17 to finally convert it into a formant frame data string formed by concatenating the frame data.

The driver 17 generates a formant frame data string based on the pronunciation character string included in the script data 11 and the default synthesis dictionary data 19, and processes the formant frame data string by interpreting the timbre parameters.

The sound source 20 outputs a synthesized utterance signal corresponding to the output data of the converter 16, and outputs the synthesized utterance signal to a speaker to emit sound.

The technical features of the speech reproducing apparatus 1 of this embodiment will be described below.

The user timbre parameter 12 includes a parameter for assigning a phrase ID stored in the user phrase synthesis dictionary data 13 to any pronunciation unit. FIG. 2 shows an example of assigning pronunciation units and phrase IDs. Here, the assignment relationship between syllables (mora) and phrase IDs is shown. In the case of Japanese, the so-called syllable means "beat", and corresponds to, for example, a kana character unit.

By assigning a phrase ID to each utterance unit, the utterance unit specified in the user timbre parameter 12 is specified not with reference to the default synthesis dictionary data 19 but with reference to the user phrase synthesis dictionary data 13 . In the user timbre parameter 12, it is preferable that an arbitrary number of utterance units can be specified from one timbre parameter.

The above-mentioned allocation of phrase IDs to each pronunciation unit in the user timbre parameters 12 is an example of this embodiment, as long as it corresponds to the pronunciation unit, other methods can also be used.

Next, details of the user phrase synthesis dictionary data 13 will be described, and FIG. 3 shows an example of the content of the user phrase synthesis dictionary data 13 . The user phrase synthesis dictionary data 13 stores formant frame data composed of 8 groups of resonance frequency, formant intensity and pitch. The so-called "phrase" in Fig. For sentences with unified syllables, the definition scale of the "phrase" does not need to be specially specified, and it can also be sentences of any scale such as words, syllables, and articles.

As a tool for creating user phrase synthesis dictionary data 13, it is necessary to load and analyze normal sound files (files with extensions such as *.wav, *.aif) to generate resonances consisting of 8 sets of resonance frequencies, formant strengths, and pitches. Analysis tools for peak frame data.

The script data 11 includes event data indicating a change in sound quality, but the user tone parameter 12 can be specified by using the event data.

For example, as a description example of the scenario data 11, when using Japanese hiragana characters and alphanumeric characters, "TJK12 みなさんX10あか" can be set. In this example, "K" represents event data designating the default timbre parameter 18, and "X" represents event data designating the user timbre parameter 12. "K12" is a code for specifying a specific default tone parameter from various default tone parameters, and "X10" is a code for specifying the user tone parameter shown in FIG. 2 from various user tone parameters.

In the above-mentioned example, the synthesized speech to be reproduced is "minasankonnichiwaSuzuki desu.". Here, "みなさん" is a synthesized voice reproduced with reference to the default timbre parameters 18 and default synthesized dictionary data 19; That is, the sentence "みなさん" is a synthesized speech in which the formant frame data of the four phonemes "み", "な", "さ", and "ん" are read out and reproduced from the default synthetic dictionary data 19; "んにちは" and "Suzuki desu" are synthesized voices in which the formant frame data of the respective phrase units are read from the user phrase synthesis dictionary data 13 and reproduced.

In the example in FIG. 2 , although three pronunciation units such as "あ", "い", and "か" are shown, as long as they are characters and symbols that can be expressed in text, there are no special regulations. In the above example, the sentence "こんにちは" is pronounced with the diacritic "あ" following "X10", and the sentence "Suzuki desu" is pronounced with the diacritic "か". Therefore, after the above-mentioned pronunciation example, in the case of performing the original pronunciation of "あ", it is only necessary to insert a symbol that returns the reference target to the default synthetic dictionary data 19 (for example, "X○○", "○" inserts a specified numbers, etc.) will do.

Next, the data exchange format of the music reproduction sequence data (SMAF: synthetic music mobile application format) used by the speech reproduction device 1 of the present embodiment will be described with reference to FIG. 4 . Fig. 4 shows the SMAF file format, and this SMAF is a kind of data distribution and the data exchange format that is used for mutual utilization with sound source expression music use, in portable terminal machine (Personal Digital Assistant (PDA), personal computer, cellular phone etc. ) is a specification of a data format for reproducing multimedia content.

The SMAF file 30 of the data interchange format shown in FIG. 4 has a data unit called a chunk as a basic structure. A block is composed of a fixed-length (8-byte) header and an arbitrary-length body, and the header is divided into two parts: a 4-byte block ID and a 4-byte block size. The block ID is used as an identifier of the block, and the block size indicates the length of the main body. The SMAF file 30 itself and various data contained therein also constitute the entire block structure.

As shown in Figure 4, SMAF file 30 is made up of content information block (contents info chunk) 31, selection data block (optional data chunk) 32, score track block (score track chunk) 33 and HV block (HV chunk) 36.

The content information block 31 stores various management information about the SMAF file 30, for example, information such as the level, type, copyright information, genre name, song name, artist name, lyricist/composer name, etc. of the stored content. The selection data block 32 stores information such as copyright information, genre name, song name, artist name, lyricist/composer name, and the like. In the SMAF file 30, it is not necessary to set the operation data block 32 either.

The musical score track block 33 is a block for storing the sequence track of the musical piece sent to the sound source, and includes a setup data chunk (setup data chunk) 34 (option) and a sequence data chunk (sequence data chunk) 35 .

The created data block 34 is a block for storing timbre data of a sound source, etc., and also stores exclusive message (exclusive message) sentences, such as timbre parameter registration information as exclusive information.

The sequence data block 35 is a block for storing actual performance data, and stores HV note-on (HV note-on, 'HV' representing human voice) which determines the playback timing of the scenario data 11 mixed with other sequence events. . Here, HV and other music events are distinguished by the channel designation of the HV.

The HV chunk 36 includes an HV setup data chunk (HV setup data chunk) 37 (option), an HV user phrase dictionary chunk (HV user phrase dictionary chunk) 38 (option), and an HV-S chunk 39 .

The HV creation data block 37 stores HV user tone parameters and information for specifying channels for HV, and the HV-S block 39 stores HV scenario data.

The HV user phrase dictionary block 38 stores the content of the user phrase synthesis dictionary data 13, and among the HV user timbre parameters stored in the HV creation data block 37, parameters for assigning syllables and phrase IDs shown in FIG. 2 must be included.

By applying the SMAF file 30 shown in FIG. 4 to the timbre parameters of this embodiment, the synthesized speech (HV) can be reproduced synchronously with the music, and the contents of the user phrase synthesis dictionary data 13 can also be reproduced.

Then, the HV authoring tool (HV authoring tool), which is a tool for making the user phrase synthesis dictionary data 13 shown in FIG. 1 and the SMAF file 30 shown in FIG. 4 , is described with reference to FIG. 5 . FIG. 5 is a block diagram showing an example of functions and specifications of the HV authoring tool.

In the case of making the SMAF file 30, the HV authoring tool 42 reads in the SMF file (standard MIDI file) 41 (including the note that determines the pronunciation timing of the HV) made by a MIDI (instrument digital interface; musical instrument digital interface) sequencer in advance. Open), according to the information obtained from the HV script UI (HV script user interface) 44 and the HV voice editor (HV voice editor) 45, the conversion process to the SMAF file 43 (equivalent to the aforementioned SMAF file 30) is performed.

The HV voice editor 45 is an editor having a function of editing HV user tone parameters (corresponding to the aforementioned user tone parameters 12 ) included in the HV user tone file 48 . This HV voice editor 45 can assign user phrases to arbitrary syllables in addition to editing various HV tone color parameters.

The interface of the HV voice editor 45 has a menu for selecting a syllable, and has a function of assigning an arbitrary sound file 50 to the syllable. The voice file 50 distributed by the interface of the HV voice editor 45 is analyzed by the waveform analyzer 46 to generate formant frame data consisting of eight sets of resonance frequencies, formant strengths, and pitches. The formant frame data can be output as individual files (ie, HV user tone color file 48, HV user synthesis dictionary file 49).

The HV script UI 44 can directly edit the HV script data, and the HV script data can be input and output as individual files (ie, the HV script file 47 ). In addition, the HV authoring tool 40 according to this embodiment may be composed only of the above-mentioned HV authoring tool 42 , HV script UI 44 , HV voice editor 45 , and waveform analyzer 46 .

Next, an example in which the voice reproduction device 1 of this embodiment is applied to a portable communication terminal will be described with reference to FIG. 6 .

The portable communication terminal 60 is, for example, a device equivalent to a mobile phone, and includes a CPU 61, a ROM 62, a RAM 63, a display unit 64, a vibrator 65, an input unit 66, a communication unit 67, an antenna 68, a speech processing unit 69, a sound source 70, and a speaker. 71 and bus 72. The CPU 61 performs overall control of the portable communication terminal 60, and the ROM 62 stores control programs such as various communication control programs and programs for playing back music, and various constant data and the like.

RAM63 is used as a work area, storing music files and various applications at the same time. The display unit 64 is composed of, for example, a liquid crystal display (LCD), and the vibrator 65 vibrates when the mobile phone receives an incoming call. The input unit 66 is composed of operation elements such as a plurality of keys, and these operation elements instruct registration processing of user tone parameters, user phrase synthesis dictionary data, and HV script data in accordance with user operations. The communication unit 67 is composed of a modem or the like, and is connected to an antenna.

The voice processing unit 69 is connected to a microphone and a receiver speaker (for example, a loudspeaker and an earphone; e.g., microphone and earphone), and has a function of encoding and decoding voice signals for communication. The sound source 70 reproduces the music based on the music file stored in the RAM 63 or the like, and simultaneously reproduces the audio signal and outputs it to the speaker 71 . The bus 72 is used to carry out data transfer between the constituent elements of the cellular phone composed of the CPU 61, ROM 62, RAM 63, display unit 64, vibrator 65, input unit 66, communication unit 67, voice processing unit 69 and sound source 70. transmission path.

The communication unit 67 can download the HV script file or the SMAF file 30 shown in FIG. 4 from a predetermined content server (Contents Server) or the like, and store it in the RAM 63. The ROM 62 stores the application program 14 of the speech reproduction device 1 shown in FIG. 1 and the program of the intermediate device. The CPU 61 reads and starts the application program 14 and the program of the intermediate device. The CPU 61 interprets the HV scenario data stored in the RAM 63 to generate formant frame data, and sends the formant frame data to the sound source 70 .

Next, the operation of the speech reproducing apparatus 1 of this embodiment will be described. First, a method of creating the user phrase synthesis dictionary 13 will be described. FIG. 7 is a flowchart showing a method of creating the user phrase synthesis dictionary 13 .

First, in step S1, the HV tone color referring to the user phrase synthesis dictionary 13 is selected by the HV creation tool 42 shown in FIG. 5, and the HV voice editor 45 is started. Then select the syllable to use with the HV voice editor 45, and load the sound file. Thus, in step S2, the HV voice editor 45 generates and outputs user phrase dictionary data (corresponding to the HV user synthesized dictionary file 49).

Then, the HV tone parameters are edited with the HV voice editor 45, and then, at step S3, the HV voice editor 45 generates and outputs user tone parameters (corresponding to the user tone file 48).

Then, a syllable to be reproduced is described by describing a sound quality change event specifying a corresponding HV tone in the HV scenario data using the HV scenario UI 44 . Next, in step S4, the HV scenario UI 44 generates and outputs HV data (corresponding to the HV scenario file 47).

Next, the playback operation of the user phrase synthesis dictionary data 13 in the speech playback device 1 will be described with reference to FIG. 8 . FIG. 8 is a flowchart showing the playback operation of the user phrase synthesis dictionary data 13 in the speech playback device 1 .

First, in step S11, the user tone parameter 12 and the user phrase synthesis dictionary data 13 are registered in the intermediate device of the speech reproduction apparatus 1 . Then, the scenario data 11 is registered in the intermediate device of the voice reproduction apparatus 1, and reproduction of the HV scenario data is started in step S12.

At the time of reproduction, in step S13, it is monitored whether a sound quality change event (X event) specifying the user tone parameter 12 is included in the scenario data 11 or not.

In step S13, if the voice quality change event is found, search for the phrase ID assigned to the syllable from the user's timbre parameter 12, and read out the data corresponding to the phrase ID from the user's phrase synthesis dictionary data 13, then, in step S14, The dictionary data of corresponding syllables in the default synthesized dictionary data 19 managed by the HV driver is replaced with the user phrase synthesized dictionary data 13 . The replacement process in step S14 may be performed before the HV scenario data is reproduced.

After step S14 ends, or under the situation that step S13 does not find sound quality change event, flow process advances to step S15, converter 16 interprets script data 11 (when carrying out the processing of step S14, the script after the exchanging process of this step S14 Data) syllables, and finally converted into formant frame string data with the HV driver.

In step S16, the data converted in step S15 is reproduced by the sound source 20 .

After this, proceed to step S17, judge whether the reproduction of script data 11 has ended, under the situation of not ending, return to step S13, on the other hand, if end, just end user phrase synthesizing dictionary data 13 shown in Fig. 8 regeneration treatment.

Next, a method of creating the SMAF file 30 shown in FIG. 4 will be described with reference to FIG. 9 . FIG. 9 is a flowchart showing a method of creating the SMAF file 30 .

First, according to the procedure shown in FIG. 7, user phrase synthesis dictionary data 13, user tone parameters 12, and scenario data 11 are created (see step S21).

Then, in step S22, the SMF file 41 including events for controlling the sounding of music data and HV scenario data is created.

Next, the SMF file 41 is input to the HV authoring tool 42 shown in FIG. 5, and the HV authoring tool 42 converts the SMF file 41 into a SMAF file 43 (equivalent to the aforementioned SMAF file 30) (refer to step S23).

Then, the user's timbre parameter 12 made in the aforementioned step S21 is imported into the HV creation data block 37 in the HV block 36 of the SMAF file 30 shown in FIG. To the HV user phrase dictionary block 38 in the HV block 36 of the SMAF file 30, the input SMAF file 30 is thus created and output (refer to step S24).

Next, the playback processing of the SMAF file 30 will be described with reference to FIG. 10 , which is a flowchart of the playback processing of the SMAF file 30 .

First, in step S31, the SMAF file 30 is registered in the intermediate device of the speech reproducing apparatus 1 shown in FIG. 1 . Here, normally, the voice reproduction device 1 registers the music data portion in the SMAF file 30 in the music reproduction unit of the intermediate device, and prepares for reproduction.

In step S32, the speech reproducing apparatus 1 determines whether the HV block 36 is included in the SMAF file 30 or not.

If the result of determination in step S32 is "Yes", the flow proceeds to step S33, and the speech reproducing apparatus 1 interprets the content of the HV block 36 .

In step S34, the speech reproducing apparatus 1 registers the user's timbre parameters, registers the user's phrase dictionary data, and registers the HV scenario data.

When the determination result in step S32 is "No", or when the registration process in step S34 is completed, the process proceeds to step S35, and the speech reproduction device 1 interprets the blocks in the music reproduction unit.

Then, the speech reproducing device 1 starts interpreting the sequence data (that is, the actual performance data) in the sequence data block 35 in response to the "start" signal, thereby reproducing the music (refer to step S36).

In the music playback described above, the speech playback device 1 sequentially interprets the events included in the sequence data, and in this process, judges whether or not each event corresponds to HV note-on (see step S37).

If the result of determination in step S37 is "Yes", the flow proceeds to step S38, and the speech reproducing apparatus 1 starts reproducing the HV script data of the HV block designated by the HV note-on.

After step S38 ends, speech reproduction device 1 carries out the regeneration processing of user phrase synthesis dictionary data shown in Figure 8, that is, in the reproduction of HV script data in step S38, speech reproduction device 1 judges whether there is the specified user's timbre parameter 12 The sound quality change event (X event) of (refer to step S39).

In the case where there is the above-mentioned voice quality change event, that is, when the determination result of step S39 is "Yes", the process proceeds to step S40, where the phrase ID assigned to the syllable is searched from the user timbre parameter 12, and the phrase ID is synthesized from the user phrase synthesis dictionary. The data 13 reads the data corresponding to the phrase ID, and replaces the dictionary data of the corresponding syllable in the default synthesized dictionary data 19 managed by the HV driver with the user phrase synthesized dictionary data. The replacement processing in step S40 may be performed before the HV scenario data is reproduced.

After step S40 ends, or under the situation that no sound quality change event is found in step S39, the process proceeds to step S41, where the converter 16 interprets the syllables of the script data 11, and finally converts them into formant frame string data with the HV driver.

Then, the flow proceeds to step S42, and the HV reproduction unit of the sound source 20 reproduces the data converted in step S41.

Thereafter, the flow proceeds to step S43, and the speech reproducing apparatus 1 judges whether or not the reproduction of the music has been completed. When the music reproduction is completed, the reproduction process of the SMAF file 30 is terminated. On the other hand, when the music reproduction is not completed, the flow returns to step S37.

In step S37, if the event in the sequence data is not HV note-on, the voice reproduction device 1 recognizes the event as part of the music data and converts it into sound source reproduction event data (see step S44).

Then the flow proceeds to step S45, and the voice reproducing apparatus 1 reproduces the data converted in step S44 in the music reproduction part of the sound source 20.

As described above, the present embodiment adopts the speech reproduction method using formant synthesis of FM sound source, which has the following three advantages.

(1) Phrases preferred by the user can be assigned, that is, voice reproduction can be performed with a timbre closer to a preferred timbre without depending on a fixed dictionary.

(2) Since a part of the default synthesized dictionary data 19 is replaced with the user phrase synthesized dictionary data 13, it is possible to avoid an excessive increase in data capacity in the voice reproduction apparatus 1. Since a part of the default synthetic dictionary data 19 can be replaced with any phrase, pronunciation by phrase unit can be realized, and the connection points between each pronunciation unit generated in the existing synthesized speech by pronunciation unit can be eliminated. The sensation of auditory dissonance caused.

(3) Since arbitrary phrases can be specified in the HV script data, speech synthesis of syllable units and speech pronunciation of phrase units can be used together.

In addition, according to the present embodiment, compared with the method of reproducing waveform data formed by sampling phrases in advance, it is possible to realize a change in timbre according to the formant intensity. Although the data size and quality in this embodiment depend on the frame rate, compared with the conventional method using sampled waveform data, it is possible to realize high-quality speech reproduction with a much smaller data capacity. Therefore, for example, the speech reproducing device 1 of this embodiment can be easily incorporated in a portable communication terminal such as a mobile phone, and therefore, contents such as e-mails can be reproduced with high-quality speech.

Fig. 11 is a block diagram showing the configuration of an audio/music reproducing apparatus according to a second embodiment of the present invention. Here, the HV-script (i.e., HV-script data) is equivalent to a file defining a format for reproducing speech, and is a file defining a pronunciation character string, pronunciation In order to make creation easier for the user, the file of speech synthesis data for the setting of the sound and the information for the reproduction application is created by text input.

The HV-script is read by application software such as a text editor, and may be described as long as it can be described in a file format that can be edited by text. As an example, a text file created by a text editor can be cited. HV-script is language-dependent and can be defined in various languages. In this embodiment, HV-script is defined in Japanese.

Symbol 101 represents an HV-script player (HV-Script player), which is used to control reproduction or stop of the HV-script. Here, when an HV-scenario is registered in the HV-scenario player 101 and an instruction to reproduce the HV-scenario is received, the HV-scenario player 101 starts interpreting the HV-scenario. Then, according to the type of the event described in the HV-script, any one of the HV driver 102, the waveform reproduction player 104, and the phrase reproduction player 107 is processed according to the event.

The HV driver 102 reads out from a ROM (read-only memory) not shown in the figure and refers to the synthesized dictionary data. The human voice has a predetermined resonance depending on the structure of the human body (such as the shape of the vocal cords or oral cavity). For peaks (that is, natural spectrum), the synthetic dictionary data stores the parameters of formants related to speech in correspondence with the pronunciation characters. Synthetic dictionary data is equivalent to using the parameters obtained by sampling and analyzing the actual sound for each phonetic character unit (for example, phoneme units such as "あ" and "い" in Japanese) as formant frame data in advance. stored database.

For example, in the case of the aforementioned CSM (Composite Sinusoidal Modeling: Composite Sine Wave Model) speech synthesis method, the synthetic dictionary data stores 8 sets of resonance frequencies, formant strengths, and pitches as parameters. Such a speech synthesis method has an advantage that the amount of data is very small compared with a reproduction method of waveform data generated by sampling speech. In addition, parameters for controlling the sound quality of the reproduced speech (for example, parameters for specifying changes to eight sets of resonance frequencies and formant strengths, etc.) may also be stored in the synthesized dictionary data.

The HV driver 102 interprets pronunciation character strings and the like including prosodic symbols in the HV-script, converts them into formant frame strings using synthesized dictionary data, and outputs them to the HV sound source 103 . The HV sound source 103 generates an utterance signal from the formant frame sequence output from the HV driver 102 and outputs it to the adder 110 .

The waveform reproduction player 104 reproduces or stops waveform data such as pre-sampled speech, music, and analog sound. Symbol 105 represents a waveform data RAM (waveform data random-access memory), which stores default waveform data in advance. The user can store user waveform data in the user data RAM 112 in the waveform data RAM 105 via a registration API (registration application program interface) 113 . When the waveform reproduction player 104 receives a reproduction instruction from the HV-scenario player 101 , it reads the waveform data from the waveform data RAM 105 and outputs it to the waveform reproduction unit 106 . Waveform regenerator 106 generates an audio signal from the waveform data output from waveform regenerator 104 and outputs it to adder 110 . The sampled waveform data is not limited to PCM (pulse code modulation; pulse-code modulation), for example, the voice compression format of MP3 (moving picture experts group layer 3; moving picture experts group layer 3) mode can also be adopted.

The phrase reproduction player 107 reproduces or stops music phrase data (or music data). The music phrase data is in the SMF format and is composed of note information representing the pitch and volume of the uttered sound, and time information representing the utterance time of the uttered sound. Reference numeral 108 represents a RAM for music phrase data, in which default music phrase data is stored in advance. The user can store user music phrase data in the user data RAM 112 in the music phrase data RAM 108 via the login API.

Phrase regeneration player 107 once accepts the reproduction instruction from HV-script player 101, just reads out the music phrase data with RAM108 from music phrase data, and carries out the time management of the note information in the music phrase data, according to the music phrase The time information described in the sentence data outputs the note information to the short speech source 109 . The short voice source 109 generates a tone signal based on the note information output from the phrase reproduction player 107 and outputs it to the adder 110 . The FM method, the PCM method, etc. can be used as the short sound source 109, but the sound source method need not be limited as long as the music phrase data reproduction function is provided.

Adder 110 synthesizes the utterance signal output from HV sound source 103 , the speech signal output from waveform regenerator 106 , and the musical tone signal output from short speech source 109 , and outputs the synthesized signal to speaker 111 . The speaker 111 emits speech and/or musical tones according to the synthesized signal of the adder 110 .

It is also possible to perform processing simultaneously in the HV driver 102, the waveform reproduction player 104, and the phrase reproduction player 107, thereby making the sound signal based on the pronunciation signal, the voice signal and the tone signal (in addition, the voice signal and the tone signal can also be collectively referred to as "voice signal and tone signal"). Sound signal") voice and melody sound at the same time. Alternatively, the HV-script player 101 may manage the processing timing of the HV driver 102, the waveform reproduction player 104, and the phrase reproduction player 107, and simultaneously reproduce speech and music based on the respective processes. In this embodiment, simultaneous processing by the HV driver 102, the waveform reproduction player 104, and the phrase reproduction player 107 is prohibited. In addition, in Fig. 11, for the convenience of explanation, separate RAMs are respectively provided as the RAM 105 for waveform data, the RAM 108 for music phrase data, and the RAM 112 for user data, but these functions can also be allocated to different storages in a single RAM. area.

FIG. 12 shows a definition example of an event for reproducing waveform data or musical phrase data (hereinafter collectively referred to as "audio data") described in the HV-scenario. The initial word "D" of an event means default definition, and the meaning of "○" means user definition. As a category of each event, it is assigned as a waveform or a phrase. The default waveform data pre-stored in RAM105 for waveform data or the default music phrase data stored in RAM108 are allocated to the default definition (D0～D63). In the default definition, 64 default Waveform data and default song phrase data. Allocate sampled waveform data or music phrase data arbitrarily created by the user to the user definition (○0～○63), and 64 sampled waveform data or music phrase data can be allocated in the user definition.

An event whose category is waveform data shown in FIG. 12 and data indicating a relationship with the waveform data represented by the event are stored in the waveform data RAM 105 in advance. Incidentally, an event whose category is a phrase and data indicating a relationship with the music phrase data represented by the event are stored in the music phrase data RAM 108 in advance. When the user has registered the waveform data or music phrase data in the RAM 112 for user data, these data are updated.

As an HV-script, for example, "TJK12 みなさん○0です. D20" is described. "T" in "TJK12" described at the beginning is a symbol indicating the start of the HV-script, and "J" designates the country character code. Here, it means HV- The script is described in Japanese. "K12" is a symbol for setting the sound quality, indicating that the twelfth sound quality is designated. Also, “minasan” and “desu” are interpreted by the HV driver 102 , and Japanese voices such as “minasan” and “desu” are emitted from the speaker 111 . When a pronunciation character string such as "みなさん" and "です" includes a prosodic symbol indicating a pronunciation state such as intonation (or intensity), voice with added intonation (or intensity) is emitted.

In the user event "○0", for example, waveform data of a sampled voice uttering "Suzuki" is registered. The user event "○0" is interpreted by the waveform reproduction player 104, whereby the voice of "Suzuki" is emitted from the speaker 111. In addition, in the user event "○20", for example, cheerful short music phrase data is registered. This user event "○20" is interpreted by the phrase playback player 107, thereby emitting cheerful music sounds from the speaker 111. At this time, the reproduced voice becomes "みなさんSuzukiです" (when a phrase of the music is reproduced), and only the part of "Suzuki" is reproduced as waveform data. The pronunciation of the speech generated by the reproduction of the waveform data is more natural in the reproduction of the connection points of the pronunciation units than the tone reproduced by the speech synthesis of the pronunciation units such as "minasan" or "desu". In addition, the reproduction of the waveform featuring the pronunciation of the phrase called "Suzuki" enables the user to effectively hear the reproduced voice. As described above, by describing the event specifying the reproduction of the waveform data or the music phrase data by the HV-script, it is possible to arbitrarily designate the reproduction timing of the waveform data or the music phrase data. The setting of the description related to the HV-script is a so-called design matter, and is not limited to the method described above.

Next, the operation of the speech/music reproducing apparatus according to this embodiment will be described using the flowchart of FIG. 13 . First, the user creates an HV-script using a text editor, and registers it in the HV-script player 101 (see step S101). At this time, if there is waveform data or music phrase data defined by the user, the login API 113 reads the waveform data or music phrase data from the user data RAM 112 . The login API 113 stores the waveform data in the waveform data RAM 105 and stores the music phrase data in the music phrase data RAM 108 .

Once the user gives a start instruction (step S103), the HV-script player 101 starts interpreting the HV-script (refer to step S102). HV-script player 101 judges whether the HV-script includes the event starting from "D" or "○" (step S104); is waveform data (step S105). If the type of the event is waveform data, the HV-script player 101 instructs the waveform reproduction player 104 to perform the processing, and the waveform reproduction player 104 reads out the waveforms of numbers following "D" or "○" from the waveform data RAM 105. data, and output to the waveform regenerator 106 (step S106). The waveform regenerator 106 generates a speech signal based on the waveform data, and outputs it to the speaker 111 through the adder 110 (step S107). In this way, the speaker 111 emits a corresponding voice.

Also, if the event type is not waveform data in step S105, the flow proceeds to step S108, and the HV-scenario player 101 determines whether the event type is music phrase data. When the type of the event is musical phrase data, the HV-script player 101 instructs the phrase reproduction player 107 to perform the processing. Phrase reproduction player 107 reads out the waveform data of the numbers following "D" or "○" from the music phrase data with PAM108, and outputs the note information in the music phrase data to the phrase according to the time information in the music phrase data. The sound source 109 (refer to step S109). The short voice source 109 generates a musical tone signal according to the note information, and outputs it to the speaker 111 through the adder 110 (step S110). In this way, the speaker 111 emits music sounds. Also, in step S108, if the event type is judged not to be music phrase data, the speech/music playback device of this embodiment considers it an event that cannot be processed, and the flow proceeds to step S113.

In step S104 , when no event starting from “D” or starting from “○” is described in the HV-script, the HV-script player 101 instructs the HV driver 102 to process it. The HV driver 102 converts the character string into a formant frame string using the synthesized dictionary data, and outputs it to the HV sound source 103 (see step S111). The HV sound source 103 generates a sounding signal according to the formant frame string, and outputs it to the speaker 111 through the adder 110 (refer to step S112). In this way, the speaker 111 emits a corresponding voice.

The HV-scenario player 101 judges whether or not the interpretation has been completed until the last description of the HV-scenario every time the event-related processing ends (step S113 ). When there are still descriptions to be explained, the process returns to step S104. On the other hand, when all the descriptions of the HV-script have been explained, the audio/music reproduction process shown in FIG. 13 is terminated.

In the case of "TJK12 みなさん○0です.D20" shown as the description example of the HV-script of this embodiment, after the sound of the waveform data defined by the event "○0" is finished, the next sentence "です" voice. For example, under the situation that HV-script player 101 carries out the interpretation of the event of wave data (or music short sentence data), temporarily suspend the reproduction of its next event, finish by wave reproduction player 104 (or phrase reproduction player) 107) When uttering is being performed, a signal indicating the end of utterance is output from the waveform reproduction player 104 to the HV-script player 101 .

In addition, when the HV driver 102, the waveform playback player 104, and the phrase playback player 107 are allowed to simultaneously perform playback processing, these playback processing can also be controlled by describing the HV-script. For example, when the HV-script describes "TJKl2 みなさん○0 3です. D20", "" (space) and "3" following "○0" are used to represent the event of setting a specified silent period, so that in Control is performed so that the voice reproduced by the HV driver 102 is silent while the voice of the sentence "Suzuki" indicated by "○0" is uttered. In the case where "TJK12 こんにちは. D20 みなさん○03です." is described in the HV-script, the music specified by "D20" and the voice called "みなさんSuzukiです" are pronounced simultaneously.

Fig. 14 is a block diagram showing the configuration of a mobile phone equipped with the audio/music reproducing device of this embodiment. Here, reference numeral 141 denotes a CPU for controlling each part of the mobile phone. Reference numeral 142 denotes an antenna for transmitting and receiving data. Reference numeral 143 denotes a communication section, which modulates the data for transmission and outputs it to the antenna 142, and demodulates the data for reception received by the antenna. Reference numeral 144 represents a voice processing section, which converts the voice data of the other party output from the communication section 143 into a voice signal and outputs it to a near-ear speaker (or earphone, not shown) when the portable phone is talking. The input voice signal (not shown) is converted into voice data and output to the communication unit 143 .

Reference numeral 145 denotes a sound source, which has the same functions as the HV sound source 103, waveform regenerator 106, and short speech source 109 shown in FIG. Reference numeral 146 represents a speaker for emitting desired voice or musical sound. Reference numeral 147 denotes an operation section operated by a user. Reference numeral 148 denotes a RAM for storing waveform data and music phrase data and the like defined by HV-script or the user. Reference numeral 149 denotes a ROM which stores programs executed by the CPU 141, and synthesized dictionary data, default waveform data, default music phrase data, and the like. Reference numeral 150 denotes a display unit, which displays the result of the user's operation or the status of the mobile phone on the screen. Reference numeral 151 denotes a vibrator, which generates vibration upon receiving an instruction from the CPU 141 when a call is received in the cellular phone. The above functional blocks are connected to each other via the bus B.

This mobile phone has a function of generating waveform data from speech, sends the speech input from the microphone to speech processing unit 144, converts it into waveform data, and stores the waveform data in RAM 148 . When the music phrase data is downloaded from the WEB server via the communication unit 143 , the music phrase data is stored in the RAM 148 .

CPU 141 performs the same processing as HV-script player 101, HV driver 102, waveform playback player 104, and phrase playback player 107 shown in FIG. In addition, the CPU 141 also interprets the events described in the HV-script read from the RAM 148 . When the event indicates pronunciation by speech synthesis, CPU 141 reads out from ROM 149 and refers to synthesized dictionary data, converts character strings described in the HV-script into formant frame strings, and outputs them to sound source 145 .

When the event indicates reproduction of the waveform data, the CPU 141 reads out the waveform data of numbers following “D” or “◯” in the HV-script from the RAM 148 or the ROM 149 and outputs the waveform data to the sound source 145 . Under the situation that event represents the reproduction of music phrase, CPU141 reads the music phrase data of the number that continues "D" or "○" in the HV-script from RAM148 or ROM149, and according to the music phrase data in this music phrase data, The time information outputs the note information in the music phrase data to the sound source 145 .

Sound source 145 generates a synthesized speech signal from the formant frame string output from CPU 141 and outputs it to speaker 146 . Then, an audio signal is generated based on the waveform data output from the CPU 141 and output to the speaker 146 . Furthermore, a musical sound signal is generated based on the musical phrase data output from the CPU 141 and output to the speaker 146 . The speaker 146 emits speech or musical tones as appropriate based on the synthesized utterance signal, speech signal or musical tone signal.

When the user operates the operation unit 147 to start software for text editing, the user can create an HV-script while recognizing the content displayed on the screen of the display unit 150 , and store the thus-created HV-script in the RAM 148 .

In addition, HV-scripts made by users can be applied to incoming call ringtones. At this time, the matter of using the HV-script when the cellular phone receives an incoming call is stored in RAM 148 as setting information. That is, when communication unit 143 receives call information transmitted from another mobile phone via antenna 142 , communication unit 143 notifies CPU 141 of an incoming call. The CPU 141 that has received the notification of the incoming call reads out the setting information from the RAM 148, reads out the HV-script indicated by the setting information from the RAM 148, and starts its interpretation. Subsequent processing is as described above. That is, the speaker 146 emits a voice or a musical sound according to the type of the event described in the HV-script.

In addition, users can add HV-scripts to e-mails and send them to other terminals. In addition, CPU 141 may interpret the text itself of the e-mail in HV-script format, and after receiving an instruction from the user, output the HV-script playback instruction to speech processing unit 144 according to the description in the e-mail. All functions of the HV-script player 101, the HV driver 102, the waveform reproduction player 104, and the phrase reproduction player 107 need not be borne by the CPU 141. For example, any one of the above-mentioned functions may be performed by the sound source 145 .

The applicable objects of the present embodiment are not limited to cellular phones, for example, it is suitable for portable terminals such as PHS (personal handphone system: (personal handy phone system) registered trademark in Japan) or PDA (personal digital assistant), The aforementioned voice and music are reproduced.

In addition, as a flexible application example of this embodiment, an HV-script created by a user can be input into a portable mobile terminal such as a mobile phone, so that a general user can not only easily create text for speech synthesis, but also easily An HV-script for reproducing stereotyped sampled waveform data or musical phrase data is created. In addition, when the audio/music reproducing device of this embodiment is provided in the portable terminal for sending and receiving, the user can operate the portable mobile terminal to attach HV-script to e-mail and send and receive it. In this way, not only text for speech synthesis but also stereotyped sample data or music phrase data can be appropriately reproduced from an e-mail received by the recipient's mobile terminal. In addition, it is also possible to use HV-script voice and reproduction of music as ringtones for incoming calls.

Next, a voice/music reproducing apparatus according to a third embodiment of the present invention will be described with reference to FIGS. 15 and 16. FIG. The third embodiment is formed by combining the aforementioned first embodiment and the second embodiment. The intermediate equipment implements HV regeneration, waveform regeneration and music phrase regeneration. The sound source produces a synthetic pronunciation signal according to these 3 kinds of data. The signals of the two systems are synthesized and output to the speaker.

Here, FIG. 15 is based on the structure of FIG. 1 and combines the partial structures of FIG. 11. The symbols 211-219 correspond to the symbols 11-19 in FIG. 113. That is, the RAM for user data in FIG. 11 is connected to the intermediate device in FIG. 1 through the user data API, and the waveform reproduction player and the phrase reproduction player in FIG. 11 are added in the intermediate device. The waveform reproduction player and the phrase reproduction player The player is respectively connected to the RAM for waveform data and the RAM for music phrase data. In addition, the application software also functions as the HV scenario player in FIG. 11, and the HV scenario is converted to HV through the intermediate device AP1 according to the type of event described in the HV-script, any of a waveform reproduction player and a music phrase player. Indicates processing. In addition, the sound source has the three functions of the HV sound source, the waveform generator, and the music phrase sound source in Fig. 11, and their respective output signals are synthesized by the adder and uttered by the speaker. Note that the operation of each component shown in FIG. 15 is the same as that of the corresponding component shown in FIGS. 1 and 11 , and thus detailed description thereof will be omitted.

Fig. 16 is a flowchart showing the operation of the voice/musical instrument reproduction device shown in Fig. 15 . It is based on the flow chart shown in Figure 13, and a part of the flow chart shown in Figure 8 is added, the labels S211-S216 correspond to the labels S11-S16 in Figure 8, and the labels S304-S310, S312, S313 correspond to Figure 13 The labels S104～S110, S112, S113. That is, when the determination result in step S104 of FIG. 13 is "No", the same processing as steps S13, S14, and S15 of FIG. 8 is executed, and then HV sound source reproduction processing is executed in step S312. In this way, by inputting an HV-script, it is possible to reproduce the voice of the HV sound source, the reproduction of the wave data of the player based on the waveform reproduction, and the reproduction of the music phrase based on the note information of the music phrase reproduction player. . In addition, the processing of each step shown in FIG. 16 is the same as that in FIG. 8 and FIG. 13 , so detailed description thereof will be omitted.

^ま$5し＞10た。”就是在“はじま

ました”(即发音的字符串)内附加规定的音调(イソトネ一シヨソ)而进行语音合成起来的，这里，“^”、“$”、“＞”等相当于韵律符号。对该韵律符号后面的文字(在韵律符号后紧接数值的情况下，接续在该数值后面的文字)附加规定的仰扬(语调)。Finally, the prosodic symbols used in the foregoing embodiments will be described. For example, "は^3じま$ described in the HV-script

^ま$5し＞10た. "is in" はじま

ました" (that is, a character string of pronunciation) is added with a prescribed tone (イソトネ一シヨソ) to perform speech synthesis. Here, "^", "$", ">" and the like are equivalent to prosodic symbols. The prosody symbols The subsequent character (in the case of the rhythmic symbol immediately following the numerical value, the character following the numerical value) is given a predetermined uplift (intonation).

Specifically, "^" indicates that the pitch is raised during the pronunciation, "$" indicates that the pitch is lowered during the pronunciation, and ">" indicates that the volume of the pronunciation is decreased, and speech synthesis is performed according to these symbols. In the case where the prosodic symbol is followed by a numerical value, the numerical value is used to specify the amount of change in the additional intonation. For example, in the case of the sentence "は^3じま", "は" is pronounced according to the standard pitch and volume, and "じ" is raised by the pitch "3" in the pronunciation, and the following "ま" is pronounced according to the raised pitch .

In this way, when a predetermined intonation (or pitch) is added to a character included in the uttered language, the above-mentioned prosodic symbol (and also a numerical value indicating the amount of pitch change) is described before the character. The above-mentioned prosodic symbols are used to control pitch or volume in pronunciation, but are not limited thereto. For example, symbols for controlling tone quality or speed may also be used. By adding such a symbol to the HV-script, the state of pronunciation such as pitch can be properly reflected.

In addition, this invention is not limited to the said Example, The change within the range of invention is included in this invention.

Claims (10)

1. A voice reproducing device comprises a storage device, a login device, an input device and a voice synthesis device; wherein,

the storage device stores synthetic dictionary data in which formant frame data corresponding to a phonetic character indicating a predetermined phonetic unit is associated with the phonetic character and stored in advance;

the registration device registers user phrase data indicating formant frame data used in place of formant frame data corresponding to a spoken word stored in the synthetic dictionary data into the user dictionary data in accordance with an operation by a user;

the input device inputs script data including a character string composed of a plurality of pronounced characters and event data indicating a replacement of formant frame data corresponding to at least a portion of the pronounced characters of the character string;

the speech synthesis device interprets the inputted scenario data, reads formant frame data from the synthesis dictionary data based on uttered characters other than at least a part of the character string, reads the user phrase data from the user dictionary data based on the event data and the part of the character string, and generates a synthesized speech based on the read formant frame data and the read user phrase data.

2. The voice reproducing device according to claim 1, further comprising music reproducing means for reproducing music based on the music reproduction information;

inputting a data exchange format in the input device, the data exchange format being an information structure including music reproduction information for reproducing music and music reproduction information containing the script data and the user phrase data, and synchronously reproducing music reproduction based on the music reproduction information and voice reproduction based on the voice reproduction information;

the music reproduction device reproduces the music reproduction information included in the data exchange format;

the voice synthesizing device reproduces the voice reproduction information included in the data exchange format.

3. A portable terminal device comprising the speech reproducing device according to claim 1 or 2.

4. A voice reproducing device is composed of a first storage device for storing voice data, a second storage device for storing script data, a reproducing instruction device, a synthesized pronunciation signal generating device, a voice signal generating device and a synthesized voice generating device; wherein,

the script data describes a character string composed of a pronunciation character representing a predetermined pronunciation unit and event data instructing reproduction of the voice data;

the reproduction instructing means reads the scenario data from the second storage means, instructs the sound generation based on a character string in the scenario data, and instructs reproduction of the sound data based on event data in the scenario data;

a synthesized speech signal generation device that performs speech synthesis based on a speech instruction of the character string from the reproduction instruction device and generates a synthesized speech signal;

the audio signal generating means reads the audio data from the first storage means in accordance with the reproduction instruction of the audio data from the reproduction instructing means, and generates an audio signal based on the audio data;

the synthesized speech device generates a synthesized speech from the synthesized utterance signal, and generates a sound from the sound signal.

5. The speech reproducing apparatus according to claim 4, wherein the sound data is waveform data generated by sampling a predetermined sound.

6. The speech reproducing apparatus according to claim 4, wherein the sound data is music data including note information indicating a pitch and a volume of a sound to be sounded.

7. The speech reproducing apparatus according to claim 4, wherein the synthesized speech signal generating means stores formant control parameters characterizing the speech of characters, and performs speech synthesis using the formant control parameters corresponding to character strings in the script data.

8. The speech reproducing apparatus according to any one of claims 4 to 7, wherein the script data is described in a file made of text data.

9. The speech reproducing apparatus according to claim 4, wherein synthesis dictionary data is provided which associates formant frame data corresponding to a phonetic character indicating a predetermined phonetic unit with the phonetic character and which is stored in advance;

registering user phrase data indicating other formant frame data used in place of formant frame data corresponding to a spoken word stored in the synthetic dictionary data in user dictionary data in accordance with an operation by a user;

when the script data includes event data indicating replacement of formant frame data corresponding to at least a part of phonetic characters of the character string, the synthesized phonetic signal generating device reads formant frame data from the synthesized dictionary data based on phonetic characters other than the part of phonetic characters, reads the user phrase data from the user dictionary data based on the event data and the part of character string, and generates the synthesized phonetic signal based on the read formant frame data and the read user phrase data.

10. A portable terminal device comprising the speech reproducing device according to any one of claims 4 to 9.

Images