JP2015118222A - Voice synthesis system and voice synthesis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice synthesis system allowing a user of a voice synthesis service to easily know available corrections on a synthetic voice.SOLUTION: The voice synthesis system receives a read-out text data and outputs the synthetic voice of the read-out text data. The voice synthesis system includes a correction candidate information generation processing section that generates a piece of correction candidate information indicating a points to be corrected in the synthetic voice and description of the correction candidate based on the information used for generating the synthetic voice.

Description

  The present invention relates to a speech synthesis system and a speech synthesis method.

  2. Description of the Related Art Conventionally, there are text-to-speech synthesis techniques that convert text into speech and read it out, and text-to-speech synthesis systems that use it. Application destinations of such technologies include, for example, guide voice in car navigation, mail reading and voice dialogue interfaces on mobile phones / smartphones, screen readers for visually impaired people, and reading functions of electronic books.

  In such a system, speech synthesis processing is often performed by a PC (Personal Computer) that reproduces synthesized speech, but in recent years, by combining Web technologies, commercialization of speech synthesis services using a server has become possible. Is progressing. In this speech synthesis service, when the read-out text is transmitted to the server, the server returns synthesized speech data as a conversion result. By realizing such a web-based speech synthesis conversion process (service), it becomes easy to respond to many speech synthesis conversion requests or to provide services to many users simultaneously. Become. Usually, in the speech synthesis service, the service is completed by one exchange of sending a read-out text and receiving the converted synthesized speech.

JP 2012-108378 A

  Since speech synthesis processing automatically determines the reading of words and intonations, it is inevitable that there are places where reading errors and intonation / accents are unnatural. For this reason, some conventional speech synthesis programs have a mechanism for re-synthesizing speech after correcting an error or an unnatural part noticed by the user. For example, Patent Document 1 discloses a technique for recognizing a voice improvement instruction from a user and correcting synthesized voice data based on the voice recognition.

  The speech synthesis service is usually divided into a server side and a client side, the speech synthesis processing is performed on the server side, and the user on the client side can usually only listen to the returned synthesized speech. However, even if the reading or inflection becomes unnatural as a result of an analysis error or the like as described above, the client side does not know what corrections can be made to the synthesized speech, so the client side has errors or inaccuracies. It cannot have a function to correct natural parts.

  An object of the present invention is to provide a technology that allows a user who uses a speech synthesis service to easily know what corrections can be made to synthesized speech.

  The present inventors have found that the above-mentioned problem can be solved by generating correction candidate information indicating a correctable portion and the contents of correction candidates in the synthesized voice based on information when the synthesized voice is generated.

  In order to solve the above problems, for example, the configuration described in the claims is adopted. The present application includes a plurality of means for solving the above-described problems. For example, a speech synthesis system that receives a read-out text and outputs a synthesized speech obtained by reading the read-out text, and generates the synthesized speech. A speech synthesis system is provided that includes a correction candidate information generation processing unit that generates correction candidate information indicating a correctable portion and the contents of correction candidates in the synthesized speech based on the information at the time.

  According to another example, the first step of generating a synthesized speech obtained by reading out the input read-out text, and a correctable portion and correction in the synthesized speech based on information when the synthesized speech is generated And a second step of generating correction candidate information indicating the content of the candidate.

  According to the present invention, it is possible to receive, as a result of the speech synthesis process, the synthesized speech as the conversion result, the correctable portion and the correctable content for the synthesized speech. Therefore, it is possible to easily know how the user who uses the speech synthesis service can correct the synthesized speech.

  Further features related to the present invention will become apparent from the description of the present specification and the accompanying drawings. Further, problems, configurations and effects other than those described above will be clarified by the description of the following examples.

It is a figure explaining the basic composition of the speech synthesis system in the present invention. It is a figure explaining the internal process of the reading production | generation process part in the basic composition of a speech synthesizer system, and a reading correction candidate information generation process part. It is a figure explaining the internal process of the prosody correction candidate information generation process part in the basic composition of a speech synthesizer system. It is a figure explaining the internal process of the reading selection process part in a basic composition of a speech synthesizer system, and a word reading correction candidate information generation process part. It is an example of data of word network information. It is an output data example of the optimal word string selection process. It is an example of output data of a word integration process. It is an example of data of an optimal morpheme sequence. It is a flowchart of the word reading correction candidate information generation process in the basic configuration of the speech synthesis system. It is an example of data of word reading correction candidate information. It is an example of output data of a division | segmentation determination process. It is an example of output data of a division | segmentation determination process. It is an example of data of delimiter correction candidate information. It is an example of output data of a joint determination process. It is an example of data of joint correction candidate information. It is an example of data of reading correction candidate information. It is a figure which shows the processing structure of the speech synthesis system at the time of the normal speech synthesis request | requirement. It is a figure which shows the processing structure of the speech synthesis system at the time of a correction | amendment and improvement instruction | indication request | requirement. It is an example of data of meta information with ID of the speech synthesis service of the first embodiment. It is a figure which shows the structure of the server side system in 2nd Example. It is a figure which shows the structure of the client side system in 2nd Example. It is an example of data of a user information database. It is a data example of the reading correction candidate information output with respect to the English text. It is an example of data after the data of FIG. 20 is corrected by the service content correction part. It is an example of output data of correction candidate information. It is an example of a screen displayed in a client side system.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. The accompanying drawings show specific embodiments in accordance with the principle of the present invention, but these are for the understanding of the present invention, and are never used to interpret the present invention in a limited manner. is not.

  First, the basic configuration of the speech synthesis system according to the present invention will be described. FIG. 1 is a diagram for explaining the basic configuration of a speech synthesis system according to the present invention. This speech synthesis system receives a read-out text and outputs a synthesized speech obtained by reading out the read-out text. The specific configuration described below of the speech synthesis system may be implemented by software program code that implements the functions of the embodiments. That is, the basic configuration of the speech synthesis system can be realized by storing a predetermined program as a program code in a memory and executing each program code by the central processing unit.

  In the following description, the information handled in this embodiment will be described using a “table” structure. However, such information does not necessarily have to be represented by a table data structure, such as a list, DB, queue, etc. It may be expressed by a data structure or other. Therefore, in order to show that it does not depend on the data structure, various data may be simply referred to as “information” below.

  In this basic configuration, it is possible to generate correction candidate information together with synthesized speech by extending the conventional speech synthesis processing. The speech synthesis system includes a reading generation processing unit 102, a prosody generation processing unit 103, and a waveform generation processing unit 104. When the reading text 101 is input, the reading generation processing unit 102 performs text analysis processing such as morpheme analysis processing, reading separation processing, and delimiter position determination processing, and reading / accent of the reading text is determined. Subsequently, the prosody generation processing unit 103 determines the duration of each syllable, the fundamental frequency, the power, and the like. Finally, the synthesized voice waveform data is generated by the waveform generation processing unit 104 and output as the read-out voice 105. The implementation methods of the reading generation processing unit 102, prosody generation processing unit 103, and waveform generation processing unit 104 are disclosed in various known documents, and thus will not be described here.

  In this embodiment, compared to the above-described conventional speech synthesis system, the reading correction candidate information generation processing unit 106, the prosody correction candidate information generation processing unit 107, the waveform correction candidate information generation processing unit 108, and the correction candidate information integration A processing unit 109 is added. The reading correction candidate information generation processing unit 106 generates reading correction candidate information that is correction information in a reading generation process for generating a synthesized speech reading. The prosody modification candidate information generation processing unit 107 generates prosody modification candidate information that is modification information in the prosody generation process for generating a synthesized speech prosody. Further, the waveform correction candidate information generation processing unit 108 generates waveform correction candidate information that is correction information in the waveform generation processing for generating the waveform of the synthesized speech. It is a feature of the present invention that these processing units can output correction candidate information 110 indicating the portion of the synthesized speech that can be corrected and the contents of the correction candidates in addition to the read-out speech 105.

  In the following, an example of processing of the reading correction candidate information generation processing unit 106, prosody correction candidate information generation processing unit 107, waveform correction candidate information generation processing unit 108, and correction candidate information integration processing unit 109 will be described. FIG. 2A explains internal processing of the reading generation processing unit 102 and the reading correction candidate information generation processing unit 106.

  The reading generation processing unit 102 is roughly composed of a word division processing unit 202, a reading selection processing unit 203, a delimitation determination processing unit 204, and a combination determination processing unit 205. These are the minimum constituent elements necessary for the reading generation process in the speech synthesis process, but other constituent elements such as a dependency analysis process may be added depending on the embodiment. Even with these additional components, the generality of the description in this embodiment is not lost. In addition, since methods for realizing the word division processing unit 202, the reading selection processing unit 203, the delimitation determination processing unit 204, and the combination determination processing unit 205 are disclosed in various known documents, they will not be described here.

  The reading correction candidate information generation processing unit 106 includes a word reading correction candidate information generation processing unit 207, a delimiter correction candidate information generation processing unit 208, a combined correction candidate information generation processing unit 211, a reading correction candidate information integration processing unit 209, Consists of The word reading correction candidate information generation processing unit 207 generates word reading correction information that is correction information regarding the reading of the words in the synthesized speech. The delimiter correction candidate information generation processing unit 208 generates delimiter correction candidate information that is correction information related to the delimiter position of the synthesized speech. Further, the combination correction candidate information generation processing unit 211 generates combination correction candidate information that is correction information related to the position of the accent combination of the synthesized speech.

  In this embodiment, each of the reading selection processing unit 203, the delimitation determination processing unit 204, and the combination determination processing unit 205 includes a word reading correction candidate information generation processing unit 207, a delimitation correction candidate information generation processing unit 208, and a combination correction candidate. An information generation processing unit 211 is added. The reading correction candidate information integration processing unit 209 integrates the output results from the word reading correction candidate information generation processing unit 207, the delimiter correction candidate information generation processing unit 208, and the combined correction candidate information generation processing unit 211, thereby The correction candidate information 210 is output.

  FIG. 3 illustrates internal processing of the reading selection processing unit 203 and the word reading correction candidate information generation processing unit 207. The reading selection processing unit 203 is roughly composed of an optimum word string selection processing unit 302, a word integration processing unit 303, and a reading classification processing unit 304. The read-out text 201 input to the speech synthesis system is converted into word network information 301 by the word division processing unit 202. This process is called a general morphological analysis, and since an algorithm is disclosed in various known documents, it will not be described here. The word network information 301 has a network structure with a limited link structure generally called a word lattice. FIG. 4 shows an example of the word network information 301. FIG. 4 shows a lattice structure in which all words partially matching the notation of the sentence “Today is sunny” are extracted from the dictionary and word candidates are listed.

  When the word network information 301 is input to the reading selection processing unit 203, first, the optimal word string selection processing unit 302 determines the optimal word string from the word network information. The optimum word string is a word string that can be a combination of words that is most suitable for Japanese among all combinations of words in the word network information 301. FIG. 5 shows an example of the optimum word string for the word network information of FIG. In FIG. 5, the word string linked by the arrows from the beginning of the sentence to the end of the sentence is the optimum word string. This optimal word string determination process is realized by calculating the cost of words and links, the appearance probability of words and links, and searching for a combination of words with the smallest cost sum or the largest product of appearance probabilities. The This process is also a known technique disclosed in various documents.

  The subsequent word integration processing unit 303 performs a process of collecting those that can be integrated as morpheme candidates. “Things that can be integrated” refer to words of the same length that start from the same position and can be exchanged as parts of speech. For example, in the example of FIG. 5, regarding the character string “today”, two word candidates for reading are output: a word candidate 501 for reading “kyou” and a word candidate 502 for reading “konnichi”. ing. Of these, the word candidate 501 of “kyou” is output as the optimum word candidate string as a more appropriate reading, but the word candidate 502 of “konnichi” may be read more correctly. is there. Since these two words are words of the same length starting from the same position and are both nouns, they are equivalent as morpheme candidates. Therefore, these two words are candidates that can be integrated. The word integration processing unit 303 integrates the morpheme candidates with the optimum word string shown in FIG. 5 and converts it into the word string shown in FIG. In FIG. 6, the reading “kyou” and the reading “konnichi” are associated with the character string “today”.

  The subsequent reading processing unit 304 determines the most appropriate reading from words having a plurality of readings in the optimum word string. In this process, for example, an appropriate one is determined from a plurality of readings using information such as a co-occurrence probability database that collects information that a certain word and a certain word are likely to co-occur. In the case of the example in FIG. 6, the fact that the word “today (kyou)” and the word “hare” are likely to co-occur exists in the co-occurrence probability database. (Kyou) ”is selected. Through the above processing, an optimum morpheme string (FIG. 7) is obtained as the output of the reading selection processing unit 203, which is a word division result determined up to how to read.

  The processing described above is the flow of processing of the conventional reading selection processing unit 203, but this embodiment is characterized in that the word reading correction candidate information generation processing unit 306 of FIG. 3 is added. The word reading correction candidate information generation processing unit 306 in FIG. 3 corresponds to the word reading correction candidate information generation processing unit 207 in FIG. 2A. The word reading correction candidate information generation processing unit 306 uses the optimal word string information having a plurality of readings output from the word integration processing unit 303 and the most appropriate reading information determined by the reading processing unit 304 to generate a word. A process of generating word reading correction candidate information 307 related to reading is performed.

  Hereinafter, the processing flow of the word reading correction candidate information generation processing unit 306 will be described with reference to FIG. The word reading correction candidate information generation processing unit 306 includes the integrated optimum word string information (FIG. 6) output from the word integration processing unit 303 and the optimum word string information (FIG. 7) output from the reading processing unit 304. Entered.

  When the process starts (S801), it is determined whether there is i-th word information after the initialization process (S802) (S803). If there is i-th word information, the i-th word information is acquired (S804). For example, in the example of FIGS. 6 and 7, the first word is “today”. If there is no i-th word information, the character string S is output as it is (S809).

  After obtaining the i-th word information (S804), it is determined based on the integrated optimum word string information whether or not a plurality of readings are set for this word (S805). If there are not a plurality of readings, the notation character string of the word is added to the variable s as it is in step S807, i is incremented, and the process proceeds to the next loop process (S808). On the other hand, if there are a plurality of readings, a word reading correction candidate designation tag addition process is performed (S806). In the example of FIG. 6, the first word “today” has two readings “Kyo” and “Konichi”. Therefore, for the word “today”, a word reading correction candidate designation tag is added in step S806. After the word reading correction candidate designation tag is added, i is incremented and the process proceeds to the next loop process (S808).

  There are various forms of the word reading correction candidate designation tag. For example, a form such as <WORD YOMI = “kyou” OTHER = “konnichi”> today </ WORD> can be an example. . This tag indicates that the reading “kyou” is determined for the character string “today”, and “konnichi” is another reading candidate. FIG. 9 is an example of word reading correction candidate information 307 output as a result of processing by the word reading correction candidate information generation processing unit 306. This character string form is only one example, and can be expressed in various forms such as a table form and a list form.

  Subsequent to the reading selection processing unit 203, processing is executed in a delimitation determination processing unit 204. The delimiter determination processing unit 204 determines where to place a pause (pause) in the text to be read out. Various methods can be used for the delimiter determination algorithm here, but since there are many methods disclosed in publicly known documents, description thereof will not be given here. For example, there is a method of inserting a delimiter only at a place where a character string “,” (reading mark) is present, or a method of inserting a delimiter after a specific part of speech (for example, a particle “ha”). This is an example in the case of Japanese, but the delimiter can naturally be determined in other languages as well, and may be determined according to the grammar and expression.

  Here, it is assumed that the delimiter decision processing unit 204 outputs delimiter position candidates and delimiter insertion probabilities there. FIG. 10 shows an example of the expression form of the break position candidates and the break insertion probability. The delimitation determination processing unit 204 performs determination processing of the delimiter position in the reading designation intermediate expression based on the delimiter position candidate information as shown in FIG. 10, and outputs it. In the delimiter position determination process, for example, a position where the delimiter insertion probability is a predetermined threshold value or more can be determined as the delimiter position. FIG. 11 is an output example of the break position in the reading designation intermediate expression.

  The processing described above is the processing flow of the conventional delimiter determination processing unit 204, but this embodiment is characterized in that the delimiter correction candidate information generation processing unit 208 of FIG. 2A is added. This delimiter correction candidate information generation processing unit 208 uses the delimiter position candidate information as shown in FIG. 10 and delimiter position information (FIG. 11) output from the delimiter determination processing unit 204 to correct delimiters into delimiter position candidates. Add candidate tags to the input text. The processing flow is the same as the processing flow for inserting the word reading correction candidate tag shown in FIG. As a result of this processing, for example, character string information as shown in FIG. 12 is output. Here, the PROB attribute indicates a break insertion probability calculated internally, and the MODE attribute indicates whether a break is actually output in the synthesized speech. Therefore, in the example of FIG. 12, it is determined that “there is” a break after “today”, and the break insertion probability is “0.8”. Similarly, in FIG. 12, after “sunny”, it is determined that the break is determined to be “none”, and the break insertion probability is “0.2”.

  Subsequent to the delimitation determination processing unit 204, processing is executed in the combination determination processing unit 205. In this processing, it is determined whether or not accent coupling occurs in Japanese, and the accent position is moved based on the determination. The “accent combination” is, for example, a character string “speech synthesis” in which two words “speech (once)” and “synthesis (gosei)” (accent type is 1 type and 0 type, respectively) are consecutive. When it appears at, it means that the accent position is moved to “Onsego Gosei” instead of being pronounced “Onse Go Gosei” in the original accent position. The processing related to the accent connection is also performed by the conventional speech synthesis processing, and its processing form is disclosed in various known documents. Accent coupling occurs between two accent phrases, and when it occurs, the result is predictable from the accent coupling rules. Therefore, the combination determination processing unit 205 determines flag information as to whether or not each accent phrase is to be combined with the subsequent accent phrase.

  The processing described above is the flow of the conventional combination determination processing unit 205, but this embodiment is characterized in that the combination correction candidate information generation processing unit 211 of FIG. 2A is added. The combined correction candidate information generation processing unit 211 uses the accent phrase combination flag information as shown in FIG. 13 to add an accent combination candidate tag to the accent combination candidate portion to the input text. FIG. 13 shows an example of the determination result determined by the combination determination processing unit 205. In FIG. 13, an accent combination flag 1301 and an accent combination candidate location flag 1302 are associated with each accent phrase. The accent combination flag 1301 is a flag indicating a place where the accent combination determined by the combination determination processing unit 205 occurs, and the accent combination candidate position flag 1302 is a flag indicating a candidate where the accent combination can occur. The location of the accent phrase “speech” is determined to cause an accent combination, and the locations of the accent phrases “high quality” and “speech” are determined as candidates that may cause the accent combination.

  The processing flow of the combined correction candidate information generation processing unit 211 is the same as the processing flow for inserting the word reading correction candidate tag shown in FIG. As a result of this processing, for example, character string information as shown in FIG. 14 is output. Here, the part where the “ACC” tag is added is a candidate part that can be accent-coupled, and the part where the MODE attribute is “ON” indicates the part that is actually accent-coupled in the synthesized speech.

  When output information is output from the word reading correction candidate information generation processing unit 207, the delimiter correction candidate information generation processing unit 208, and the combined correction candidate information generation processing unit 211, they are input to the reading correction candidate information integration processing unit 209. The The reading correction candidate information integration processing unit 209 executes integration processing of the three tagged character strings that are output. The tagged character string output from the above three processes is obtained by adding tag information to the same input character string. For this reason, the reading correction candidate information integration processing unit 209 performs integration processing for embedding all the three tag information in the same input character string. The purpose of this integration process is to facilitate the analysis process of the tagged character string on the side of the tagged character string recipient (speech synthesis service user), and the integration process is not essential if there is no problem with redundancy. .

  In this integration process, the start tag and the end tag must be nested. As a result of the reading generation process, the position where the tag is embedded is almost in units of words (morphemes). Therefore, there are almost no cases where the start / end tags set in units of words cannot be nested. Tag integration processing in such a case can be realized only by considering simple tag association using a stack. FIG. 15 illustrates an example of reading correction candidate information output from the reading correction candidate information integration processing unit 209.

  As described above, in the speech synthesis system according to the present embodiment, when read-out text is input, meta information describing a correctable portion and a correctable candidate is output in addition to the synthesized speech. As a result, the user's system that has received this meta information can instruct the user to make corrections by analyzing this information. In addition, the user-side system automatically notifies the speech synthesis system of correction candidates, so that another type of synthesized speech can be acquired and presented to the user.

  In the above description, the method for generating the reading correction candidate information using the information from the reading generation processing unit 102 in FIG. 1 has been described in detail, but the prosody generation processing unit 103 and the waveform generation processing unit 104 perform similar processing. Is possible. The prosody generation processing unit 103 and the waveform generation processing unit 104 determine internal information in speech synthesis. For example, in the prosody generation processing unit 103, a pitch pattern, speech speed, phoneme fundamental frequency, phoneme duration, etc. for each part of the text to be read (sentence, phrase (exhalation paragraph), accent phrase, word, etc.), etc. Is determined. The prosody modification candidate information generation processing unit 107 can output prosody modification candidates for these pieces of information.

  FIG. 2B explains the internal processing of the prosody generation processing unit 103 and the prosody modification candidate information generation processing unit 107. The prosody correction candidate information generation processing unit 107 includes a duration correction candidate information generation processing unit 221, a fundamental frequency correction candidate information generation processing unit 222, a height correction candidate information generation processing unit 223, and a speech speed correction candidate information generation process. Unit 224 and prosodic correction candidate information integration processing unit 225.

  The continuation length correction candidate information generation processing unit 221 generates continuation length correction candidate information that is correction information related to the phoneme continuation length of the synthesized speech. The fundamental frequency correction candidate information generation processing unit 222 generates basic frequency correction candidate information that is correction information related to the phoneme fundamental frequency of the synthesized speech. Also, the height correction candidate information generation processing unit 223 generates height correction candidate information that is correction information related to the pitch pattern of the synthesized speech portion. Furthermore, the speech speed correction candidate information generation processing unit 224 generates speech speed correction candidate information that is correction information regarding the speech speed of the portion of the synthesized speech. The prosodic correction candidate information integration processing unit 225 integrates the correction candidate information output from these processing units 221 to 224, and outputs prosodic correction candidate information 226.

  The prosody modification candidate information generation processing unit 107 can indicate the prosody modification candidate information 226 by adding, for example, a PITCH tag or a SPEED tag to the read-out text. In the example of FIG. 22, correction candidates for the pitch (pitch) pattern for each part of the text to be read are indicated as PITCH tags, and correction candidates for the speech speed for each part of the text to be read are indicated as SPEED tags. .

  Further, the waveform generation processing unit 104 determines a segment (speech component) used in the synthesized speech. The waveform correction candidate information generation processing unit 108 generates segment correction candidate information that is correction information related to the segments used in the synthesized speech. For example, the waveform correction candidate information generation processing unit 108 lists the IDs of voice component data actually used in the voice synthesis process, and lists other voice component data IDs that could be used instead. Thus, the waveform correction candidate information can be output.

  FIG. 22 shows an example of the correction candidate information 110 that is finally output from the correction candidate information integration processing unit 109. The correction candidate information integration processing unit 109 integrates information output from the reading correction candidate information generation processing unit 106, the prosody correction candidate information generation processing unit 107, and the waveform correction candidate information generation processing unit 108.

[First embodiment]
In the first embodiment, a speech synthesis system that accepts correction and improvement instruction requests for speech synthesis results will be described using the basic configuration of FIG. In the present embodiment, a speech synthesis service in a server / client configuration using the basic configuration of the present invention is assumed. The server side constitutes a speech synthesizer using the speech synthesis method of the present invention, and this speech synthesizer generates synthesized speech for the read-out text requested from the client (user) side and transmits it. Do service. At this time, by using the meta information of the reading text generated by the basic configuration of the present invention, it is possible to indicate to the client side what kind of correction / improvement is possible for the generated synthesized speech. it can.

  On the client side, based on the received meta information, a request for correction and improvement instruction by the client side system or the user is created and returned to the server side system. In the server side system, by processing the correction and improvement instruction request, it is possible to generate again the synthesized speech modified as intended by the client side and return it to the client side. Further, in the server side system, the contents of the speech synthesis data and the speech synthesis parameters can be updated so that the contents of the correction and improvement instructions are reflected in the subsequent speech synthesis processing.

  With the above processing, in the first embodiment, in the speech synthesis service in the server / client configuration, the modified synthesized speech can be generated by reflecting the correction and improvement instructions instructed by the client side, and the correction and improvement instructions are given. It is possible to realize a speech synthesis service that can reflect the contents of the above in future speech synthesis processing.

  Hereinafter, the configuration of this embodiment will be described with reference to FIGS. 16A and 16B. 16A and 16B are diagrams illustrating the configuration of the server-side system (speech synthesis server system) of this embodiment. The speech synthesis server system has different processing contents when a normal speech synthesis is requested and when a correction and improvement instruction is requested.

  FIG. 16A shows a processing configuration when a normal speech synthesis request is made. The speech synthesis server system includes speech synthesis context generation processing unit 1602, speech synthesis data storage device 1610, context storage device 1611, speech synthesis processing unit with meta information 1603, and analysis unit as components for speech synthesis. An ID setting processing unit 1605 and a service response generation processing unit 1608 are provided.

  Hereinafter, the flow of processing from when the read-out text is input until the service response including the synthesized speech and its meta information is returned will be described. First, when the read-out text 1601 in the service request information from the user is input, the speech synthesis context generation processing unit 1602 executes the process. The speech synthesis context generation processing unit 1602 extracts speech synthesis parameters set by the user (client) who has made the speech synthesis request from a database (not shown), and generates speech synthesis context information. The generated speech synthesis context information is stored in the context storage device 1611. The user can be determined based on the user ID. This user ID is often embedded in the service request information in which the read-out text 1601 is stored.

  Here, a normal relational database can be used as it is as the database storing the speech synthesis parameters for each user. The information stored in this database includes a user ID as key information, a speaker ID of synthesized speech, a speech speed, a speech height, a volume, and the like. Furthermore, depending on the speech synthesis system, there is a specification that the symbol can be read / not read and the distinction between digit reading / bar reading of the numeric string can be specified as a parameter. When such a speech synthesis system is used, These parameters are also elements of speech synthesis parameters acquired for each user. There are other speech synthesis systems that can determine the context of the text to be read and change the reading or inflection. For example, the reading / inflection when a certain sentence A is read out separately may differ from the reading / inflection when the same sentence A is read out after another sentence B. In the case of such a speech synthesis system, information for specifying the current context can be output. The speech synthesis context generation processing unit 1602 may also output such context specifying information as a speech synthesis parameter.

  The speech synthesis context generation processing unit 1602 further generates speech synthesis request ID information for specifying the current speech synthesis request. Since the speech synthesis server system provides services to a plurality of users at the same time, it is necessary to identify when the synthesized speech is generated for which user based on this information. No special configuration is necessary for this processing. What is necessary is just to give a unique sequence number to each synthesized speech when the speech synthesis server system receives a speech synthesis request. Thus, the speech synthesis context generation processing unit 1602 generates a speech synthesis request ID and a speech synthesis parameter set for specifying the current synthesized speech based on the user ID, and stores them in the context storage device 1611. Store.

  Subsequently, the read-out text 1601 is input to the speech synthesis processing unit 1603 with meta information. The speech synthesis processing unit with meta information 1603 is implemented with the speech synthesis processing with meta information described in the basic configuration of FIG. As a result of this processing, the synthesized speech 1606 read out from the read-out text and the sentence analysis meta information 1604 that is the analysis result of the read-out sentence are output. The synthesized voice 1606 corresponds to the reading voice 105 described with reference to FIG. The sentence analysis meta information 1604 corresponds to the correction candidate information 110 described with reference to FIG. Since the processing here has already been described in the basic configuration, it will be omitted.

  Note that the speech synthesis processing unit with meta information 1603 acquires speech synthesis data from the speech synthesis data storage device 1610 and executes speech synthesis processing with meta information. This speech synthesis data includes at least dictionary data used by the reading generation processing unit 102, prosody data used by the prosody generation processing unit 103, waveform data used by the waveform generation processing unit 104, and the like.

  The sentence analysis meta information 1604 is then input to the ID setting processing unit 1605 for each analysis unit. The ID setting processing unit 1605 for each analysis unit performs processing for setting an ID (identifier) for each analysis unit for the sentence analysis meta information described in the basic configuration. The analysis unit referred to here is a sentence that constitutes the reading text 1601, a phrase that constitutes the sentence (a unit that is read in a pause between pauses), an accent phrase that constitutes the phrase (an intonation unit having one accent nucleus), This refers to the words (morphemes) that make up the accent phrase, and the poses that are included in the sentence. In this way, IDs are added to various units constituting the text to be read out 1601, so that it is easy to identify to which part the correction and improvement instruction request from the client side is. As an ID assigning method for each analysis unit, for example, a method of generating by adding IDs for each analysis unit in order to the speech synthesis request ID can be considered.

  FIG. 17 shows the result of assigning IDs by analysis unit to the reading correction candidate information of FIG. Not only the reading correction candidate information but also the prosodic correction candidate information, the waveform correction candidate information, and the entire correction candidate information 110 obtained by integrating them can be assigned IDs by analysis unit by the same method. The ID setting processing unit 1605 for each analysis unit outputs the correction candidate information to which the ID for each analysis unit is given as meta information 1607 with ID. The meta information 1607 may be stored in the context storage device 1611.

  Finally, the service response generation processing unit 1608 generates service response information 1609 for the speech synthesis service request using the synthesized speech 1606 and the ID-added meta information 1607. This service response information corresponds to HTTP protocol response information and the like. If synthetic voice data is set as the main data of the response information and the meta information with ID is set as a header attribute value of the response information, the service response information here can be realized.

  According to the above flow, when the read-out text 1601 is input, the speech synthesis server system according to the present embodiment stores the parameter set for specifying and reproducing the speech synthesis request in the context storage device 1611, and then the text. Can be output, and the ID-added meta information 1607 indicating the correctable candidate location and the correctable candidate content for the synthesized speech.

  Next, a flow that the speech synthesis server system according to the present embodiment processes in response to a correction and improvement instruction request from the user will be described. FIG. 16B shows a processing configuration when a correction and improvement instruction is requested. The speech synthesis server system includes a modification instruction interpretation processing unit 1652, a speech synthesis context selection processing unit 1654, a context storage device 1611, a meta-information-synthesized speech synthesis processing unit 1603 as components for correction and improvement instruction requests. A speech synthesis data storage device 1610, an improved part determination processing unit 1660, an improved data reflection processing unit 1659, an improved data creation processing unit 1661, and a service response generation processing unit 1608 are provided. In FIG. 16B, the same components as those in FIG. 16A have the same functions.

  When a correction and improvement instruction request is transmitted from the client side to the speech synthesis server system of the present embodiment, the improvement instruction information 1651 and the target ID 1653 are first separated therefrom. As a configuration method of the correction and improvement instruction request information, a request information format of the HTTP protocol can be used. Here, as the target ID 1653 for designating the correction and improvement target, the ID for each analysis unit passed at the time of the speech synthesis request can be included. Further, the improvement instruction information 1651 can include information designating a correction result desired by the user (client) from among the correction candidates described in the correction candidate information passed at the time of speech synthesis. The separated improvement instruction information 1651 is input to the improvement instruction interpretation processing unit 1652, while the target ID 1653 is input to the speech synthesis context selection processing unit 1654.

  The speech synthesis context selection processing unit 1654 extracts speech synthesis context information from the context storage device 1611 based on the target ID 1653. The speech synthesis request ID when the synthesized speech is generated can be extracted from the target ID 1653. In brief, when the target ID 1653 is generated in such a manner that the analysis unit ID is added to the speech synthesis request ID as described above, only the part of the speech synthesis request ID is extracted from the target ID 1653. When the speech synthesis request ID and the target ID are associated with each other by another configuration method (for example, a conversion table), the speech synthesis request ID can be extracted by referring to the conversion table. By searching the context storage device 1611 using the speech synthesis request ID extracted from the target ID 1653 as key information, various speech synthesis parameters when the synthesized speech is generated, or parameter information including context information is acquired. be able to. The extracted speech synthesis parameter set (context information) is output to the speech synthesis processing unit 1603 with meta information.

  The speech synthesis processing unit with meta information 1603 is implemented with the speech synthesis processing with meta information described in the basic configuration of FIG. The speech synthesis process performed here is performed when the text to be read and corrected this time is synthesized previously (that is, when the synthesized speech that is the target of the correction and improvement instruction request is created). The speech synthesis process is repeated again. As a result, the same synthesized speech 1657 and sentence analysis meta information 1658 as those output last time (that is, the sentence analysis meta information 1604 in FIG. 16A) are output. In this embodiment, the sentence analysis meta-information 1658 is generated again by repeating the previous speech synthesis process at the time of the correction and improvement request. Of course, the sentence analysis meta-information 1658 is generated when the previous generation is performed. It is also possible to take a method of storing them in a database in association with In this case, after the speech synthesis request ID is separated from the target ID 1653, the sentence analysis meta information 1658 can be acquired by referring to this database.

  The speech synthesis processing unit with meta information 1603 refers to speech synthesis data stored in the speech synthesis data storage device 1610. Similar to the configuration in FIG. 16A, the speech synthesis data includes dictionary data used by the reading generation processing unit 102, prosody data used by the prosody generation processing unit 103, waveform data used by the waveform generation processing unit 104, and the like. At least included. The speech synthesis processing unit with meta information 1603 can output synthesized speech 1657 using these pieces of information.

  The sentence analysis meta information 1658 output in this way is input to the improved part determination processing unit 1660. Based on the sentence analysis meta information 1658 and the target ID 1653, the improvement part determination processing unit 1660 determines which part the user (client) side is instructing to correct and improve. If the analysis unit ID described in the sentence analysis meta-information at the time of the previous speech synthesis is used as the target ID 1653, the improvement portion is determined only by searching for the analysis unit ID in the current sentence analysis meta-information 1658. can do.

  The improvement instruction interpretation processing unit 1652 interprets the improvement instruction information 1651 and extracts at least one correction and improvement information of reading correction information, prosody correction information, and waveform correction information from the user (client) side. The correction and improvement information output from the improvement instruction interpretation processing unit 1652 and the improvement point information (analysis unit ID) output from the improvement point determination processing unit 1660 are input to the improvement data creation processing unit 1661.

  The improved data creation processing unit 1661 updates the synthesized speech so that the speech synthesis result at the designated location becomes the designated improvement instruction, and outputs the synthesized speech to the service response generation processing unit 1608. The service response generation processing unit 1608 generates service response information 1609 for the correction / improvement instruction request using the updated synthesized speech and ID-added meta information.

  Also, the improved data creation processing unit 1661 generates information for updating the content of the original speech synthesis data so that the speech synthesis result at the designated location becomes the designated improvement instruction. Based on the information generated here, the improved data reflection processing unit 1659 extracts the data to be updated from the data for speech synthesis stored in the speech synthesis data storage device 1610 and designates the data for the update. Perform the updated processing. As described above, the location and content of the correction and improvement instruction request designated by the user are determined, and the update of the speech synthesis data for implementing the correction and improvement is realized.

[Second Embodiment]
In the second embodiment, a speech synthesis service that accepts correction and improvement instruction requests for speech synthesis results will be described using the basic configuration of the present invention and the configuration described in the first embodiment.

  In the present embodiment, a voice service with the same server / client configuration as in the first embodiment is assumed. Furthermore, in this embodiment, based on the system of the first embodiment, the service content provided by the user level (such as restrictions on parts that can be modified) is changed or applied to speech synthesis in a foreign language other than Japanese. The case will be described. Furthermore, in the client side system, a function for performing GUI display of a portion where the synthesized speech can be corrected, and for transmitting a speech correction and improvement instruction from the user to the server using the GUI display will be described. .

  FIG. 18A is a diagram illustrating a configuration of a server-side system in the second embodiment. FIG. 18B is a diagram illustrating a configuration of a client-side system in the second embodiment. The entire system of this embodiment is composed of a server side system and a client side system. The server-side system is a system that is operated and operated by the provider providing the speech synthesis service, and the client-side system is a system (program) used by the user who receives the speech synthesis service. These systems are connected to each other via a network not described in FIGS. 18A and 18B.

  Further, the server side system and the client side system are configured by an information processing apparatus such as a personal computer or a workstation. Each system includes a central processing unit, an auxiliary storage device, a main storage device, an output device, and an input device. For example, the central processing unit is composed of a processor (or a processing unit) such as a CPU (Central Processing Unit). For example, the auxiliary storage device is a hard disk, the main storage device is a memory, the output device is a display, a speaker, and the like, and the input device is a keyboard and a pointing device (such as a mouse). Various data used in the present invention is stored in the storage device of the information processing apparatus.

  The server-side system includes a user information acquisition unit 1802, a speech synthesis selection unit 1803, a user information database 1804, a speech synthesis data database 1806, a service content modification unit 1807, and the modifications and improvements described in the first embodiment. A speech synthesis system 1805 that receives an instruction request.

  The server side system first receives the read request information 1801 from the client side system. The reading request information 1801 is speech synthesis service request information using a network protocol such as HTTP, for example. In this information, at least read-out text information to be synthesized and user information for identifying a user who receives the service are stored. In the case of the improvement instruction request phase, information such as the improvement instruction information described in the first embodiment may be stored in the reading request information.

  The user information acquisition unit 1802 separates user information from the reading request information 1801. The user information acquisition unit 1802 can know what service content is permitted for the user by searching the user information database 1804 for the user information.

  The speech synthesis selection unit 1803 determines available speech synthesis means based on the user information. For example, when a set of speech synthesizers according to languages such as Japanese speech synthesizer and English speech synthesizer is available as speech synthesizer, English speech synthesis service is not permitted by examining user information The user can make an error response to the English text reading request. The speech synthesis selection unit 1803 has a function of selecting speech synthesis means that can be used by the user or returning an error response by referring to available speech synthesis service information set for each user.

  The speech synthesis selection unit 1803 selects, from the speech synthesis data database 1806, speech synthesis data necessary for the reading request and data usable by the user. By using the data by the speech synthesis system 1805, data that is appropriate for the reading request and that can be used by the user is appropriately selected. In this embodiment, by switching various speech synthesis data, various speech synthesis services (for example, speech synthesis in different languages such as Japanese, English, Chinese, etc., announcer tone, conversation tone) are performed by the same speech synthesis system 1805. However, in some cases, not only the speech synthesis data is switched but also the speech synthesis program itself executed by the speech synthesis system 1805 must be switched. It can be the case. In that case, in addition to the speech synthesis data database 1806, a method may be employed in which a speech synthesis program database is newly provided and the speech synthesis program itself is switched by the speech synthesis selection unit 1803.

  The speech synthesis system 1805 is a speech synthesis system having the functions described in the first embodiment (FIGS. 16A and 16B). The speech synthesis system 1805 converts the read text specified by the read request information 1801 into synthesized speech using available speech synthesis data. At the same time, the speech synthesis system 1805 generates meta information with ID, which is correction candidate information indicating the portion that can be corrected in the synthesized speech and the contents of the correction candidates.

  Finally, the service content correction unit 1807 performs processing for deleting correction candidate information that is not permitted to be provided to the user from the meta information with ID generated by the speech synthesis system 1805. For example, in the case of a setting in which a user is permitted only to change and read words and change accents and not to change prosody changes such as intonation and rhythm, the user is involved in prosody from the meta-information with ID output by the speech synthesis system 1805. Delete correction candidate information. As a result, it is possible to provide a service that provides only the correction information permitted for the user. Information regarding corrections permitted for the user is stored in the user information database 1804.

  As illustrated in FIG. 18B, the client side system includes a correction candidate list presentation unit 1852, an improvement request target instruction unit 1855, and an improvement instruction request creation unit 1857. The correction candidate list presenting unit 1852 performs output to the display device 1853 and the speaker device 1854 that are presentation devices. Further, the improvement request target instruction unit 1855 receives an input from a mouse or a keyboard 1856 that is an instruction device.

  In the client-side system, first, a new text-to-speech synthesis request is issued from the client-side system to the server-side system. However, since this first speech synthesis request process is not a unique part in the present invention, it is omitted in FIGS. 18A and 18B. FIG. 18A shows a processing flow when the first speech synthesis processing request is issued and the service response information 1851 is returned from the server side system in response to the reading request.

  When the service response information 1851 for a certain reading request is returned, the correction candidate list presentation unit 1852 takes out the synthesized speech and the ID-added meta information included in the response information, and presents the correction location to the user. A display device 1853 is used for visually presenting the corrected portion using the GUI, and a speaker device 1854 is used for checking the synthesized speech of the read-out text, checking the voice of the corrected portion, and checking the voice of the correction result of the corrected portion. used. Details of the correction candidate list presentation unit 1852 will be described later.

  The correction candidate list presentation unit 1852, the display device 1853, and the speaker device 1854 present the speech synthesis result for the read-out text or the correction result after the correction / improvement instruction request is made by GUI or voice. At this time, the user using the client side system can issue a correction and improvement instruction request for the read-out synthesized speech through the GUI. The user designates what correction and improvement instruction request is to be made for which part of the text to be read on the GUI by using an input device such as a mouse or a keyboard 1856. The improvement request target instruction unit 1855 analyzes information from the input device and determines what correction and improvement requests are instructed for which part of the text to be read. Finally, the improvement instruction request creating unit 1857 creates a correction and improvement instruction request to be transmitted to the server-side system based on the correction and improvement request for the reading text position designated by the user.

  Next, for each of the server-side system and the client-side system, an embodiment of the technology of each component will be described in more detail.

  The user information acquisition unit 1802 separates user information from the reading request information 1801. This user information is information for identifying the user, and means information such as a user ID. When the read request information 1801 is transmitted by a protocol such as an HTTP request, the user information is embedded as HTTP protocol header information or in the main part information of the HTTP request in a format unique to this system. The user information acquisition unit 1802 searches the user information database 1804 using the user information separated by the user information acquisition unit 1802, and acquires the service content permitted for the user.

  FIG. 19 shows an example of the contents of the user information database 1804. The user information database 1804 stores at least a user ID for identifying the user and service information permitted for the user. Permitted service information includes at least information that can be acquired (individual tag information output from the processing unit such as reading, prosody, waveform, etc.) and whether an improvement request can be issued. In the case of a system for multilingual speech synthesis, information such as a usable reading language is further included as permitted service information. Accordingly, as illustrated in FIG. 19, the user information database 1804 may include at least a user ID 1901, an available language 1902, information 1903 that can be acquired, and whether or not an improvement request is possible 1904 as configuration items.

  For example, in the example of FIG. 19, a reading service of “Japanese” and “English” can be provided to the user with the user ID “1234567”, and reading (WORD, ACC, PAUSE) is performed for Japanese. (Tag) and prosody (PITCH, SPEED tag) information can be obtained, and improvement instruction requests for them can be obtained. On the other hand, only English (PAUSE) information is provided for an English service, and an improvement instruction request for it cannot be accepted. Information retrieved from the user information database 1804 is passed to the speech synthesis selection unit 1803, the speech synthesis data database 1806, and the service content modification unit 1807, where appropriate information is selected or deleted.

  The speech synthesis selection unit 1803 selects speech synthesis means based on the reading language information permitted by the user searched in the user information database 1804. The speech synthesis means may be selected when the speech synthesis program and the speech synthesis data are different for each language, or when the speech synthesis program is common and only the speech synthesis data is different. Even if it is the latter, it is necessary to specify the language of the text to be read to the speech synthesis program. This is because even if the text uses the same alphabet, it is often impossible to specify the language.

  In the embodiment of FIG. 18A, the speech synthesis selection unit 1803 is configured to pass only the language information of the text to be read to the speech synthesis system 1805. That is, it is assumed that the speech synthesis system 1805 that is a speech synthesis program can use the same system regardless of the reading language.

  On the other hand, if the speech synthesis program differs for each language, an appropriate speech synthesis program is selected by the speech synthesis selection unit 1803 and the program is executed in the speech synthesis system 1805. In this case, in addition to the speech synthesis selection unit 1803, there is a speech synthesis program database storing a speech synthesis program for each language, an appropriate program is selected from the database, and the speech synthesis system is a program execution unit. In 1805, the program is executed.

  In the present embodiment, it is assumed that only the language of the read-out text is output from the speech synthesis selection unit 1803. As a method for specifying the language of the text to be read, it is common to specify language information as an attribute value in the text request information 1801. The speech synthesis selection unit 1803 separates the attribute value from the reading request and outputs it to the subsequent stage.

  As another embodiment, the speech synthesis selection unit 1803 acquires not only language information but also information such as an utterance style and outputs the information to the speech synthesis system 1805 or the speech synthesis data database 1806. Is also possible. For example, when the speech synthesis program and speech synthesis data differ depending on reading tone, spoken tone, etc., it is necessary to acquire speech style information by the speech synthesis selection unit 1803 and switch between the program and data. Even in such a case, it is possible to cope by extracting the speech style information described in the reading request information 1801 and passing it to the speech synthesis system 1805 or the speech synthesis data database 1806.

  The voice synthesis data database 1806 stores a plurality of voice synthesis data used by the voice synthesis system 1805. As described above, data for each language, such as data for Japanese speech synthesis and data for English speech synthesis, and data for each utterance style, such as Japanese reading tone data and Japanese spoken tone tone data, are stored. Therefore, based on the information acquired by the speech synthesis selection unit 1803, speech synthesis data used in actual processing is retrieved and output. The speech synthesis data database 1806 also stores dictionary data, prosody data, waveform data, and the like used in the speech synthesis system 1805.

  The speech synthesis system 1805 is the speech synthesis means described in the basic configuration or the first embodiment. Although detailed description thereof is omitted, it is possible to process a plurality of languages as described above. In the reading generation process, the information of reading and accent is information unique to Japanese, so the information analyzed according to the language can change. For example, in the case of English, instead of reading and accent, pronunciation and stress information are information analyzed by the reading generation processing unit 102. For example, in the case of Chinese, pronunciation (pinyin) and tone are information analyzed by the reading generation processing unit 102. As described above, the information that is analyzed and output by the reading generation processing unit 102 may vary depending on the language, but the text analysis processing of each language is disclosed in various documents, and the method may be used. In addition, regarding the method of reading the analysis information for each language analyzed by those methods and outputting it as correction candidate information, the format and processing algorithm are described in the basic configuration described above, even if there is a difference in the type of information. Those described for the synthesis can be used as they are.

  FIG. 20 shows an example of the reading correction candidate information output for the English text. In this reading correction candidate information, the WORD tag has a PRON attribute indicating pronunciation instead of the Japanese YOMI attribute. This reading correction candidate information has a STRESS tag indicating stress instead of an ACC tag indicating accent. Of course, this format is one embodiment, and other tags can be set.

  Finally, the service content correction unit 1807 selects only the information output from the user information database 1804 and permitted for the target user. In this process, tag information that is not permitted to be provided to the target user is deleted from the output information with correction candidates as shown in FIGS. For example, when the current target user is the user with the user ID “1234567” and the read-out text is in English, tag information other than the PAUSE tag is deleted from the information with correction candidates (FIG. 20) output from the speech synthesis system 1805. Service response information 1808 including information as indicated by 21 and synthesized speech is output. Thereby, it is possible to provide information permitted for each user.

  Next, a method for realizing the client-side system in FIG. 18B will be described. First, the client side system receives service response information 1851 from the server side system. This information is a set of synthesized voice data and information with correction candidates (FIGS. 17, 20, and 21) for the reading request information 1801 previously transmitted from the client-side system. These pieces of information are output to the correction candidate list presentation unit 1852.

  The correction candidate list presentation unit 1852 performs processing for displaying the information with correction candidates on the display device 1853 graphically. In the speech synthesizer, the text to be read is graphically displayed on the screen to confirm the reading and inflection of the generated synthesized speech, and each part is designated by an input device such as a mouse, so that the synthesized speech of that part is displayed. The product to be confirmed is realized. For example, a screen as shown in FIG. An interface has already been realized in which an audio part corresponding to “Kyo” is reproduced from the speaker device 1854 by clicking the “today” part with the mouse.

  In addition to such graphical presentation, the correction candidate list presentation unit 1852 uses information such as readings, accents, and prosody as shown in FIG. Graphically display the realization value and its changeable value. For example, FIG. 23 is an example, and here, regarding the reading, the current value (Kyo) and the changeable value (Konichi) of the reading / accent of the part corresponding to “today” are graphically presented in a menu format. Yes. The user instructs the values to be changed to the client side system by instructing these values with a mouse or the like.

  In the example of FIG. 23, only reading is illustrated, but, of course, the same method is used for accents, positions where pauses are possible, prosodic information such as pitch and speech speed, ID of speech component data used for synthesized speech, and the like. Graphical presentation is possible. A client-side system that can graphically display the generated synthesized speech and change its reading, accent, prosody, etc. has already been realized. It is of course possible to use such an existing client side system in the client side system of the present invention.

  When the user designates one of the values that can be changed with the mouse or the like, the improvement request target instruction unit 1855 determines which ID element corresponds to the position. For example, when “Connichi” is clicked with the mouse in FIG. 23, it is determined that the “today” part, which is the element of the correction candidate information ID “2013306301230123456W001” in FIG. 22, is the element designated by the user. Furthermore, it is easy to determine that “Connichi” has been selected as the reading correction candidate.

  As described above, the improvement request target instruction unit 1855 instructs the user to change the synthesized speech by giving an instruction with an input device such as a mouse on the reading text indicating the synthesized speech graphically displayed on the client side system. And two kinds of information, improvement information, can be acquired. These two pieces of information are converted into a format such as XML or JSON in the improvement instruction request creation unit 1857, and are sent to the speech synthesis server system (system shown in FIG. 16A and FIG. 16B or FIG. 18A) as improvement instruction information 1858. Sent. How the improvement instruction request is processed on the server system side is as described in the first embodiment.

  As described above, in the server / client speech synthesis system according to the second embodiment, the server-side system can set a fine service level for each user and only provide information permitted to the user. . In the client side system, the synthesized speech is displayed graphically within the range of information permitted for the user himself, and the desired value is selected from among the correction candidates passed from the server side system. It is possible to correct and improve the synthesized speech.

  According to the above embodiment, the server-side system can use any information that has been used to generate the synthesized speech in each of the reading generation, prosody generation, and waveform generation processing that constitutes the speech synthesis processing as an alternative candidate. Describes whether there is information as meta information. Then, the server side system returns information obtained by adding the correction candidate information as meta information to the read-out text together with the synthesized speech to the client side system. According to this configuration, it is possible to receive, as meta information, a correctable portion and correctable content for the synthesized speech, as well as the synthesized speech of the conversion result as a result of the speech synthesis process. By using this information in the client side system, a user can easily create a user interface for designating a correction location and correction content. Thereby, the synthesized speech which the user of a speech synthesis service requires can be easily created.

  In addition, this invention is not limited to the Example mentioned above, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. In addition, a part of the configuration of one embodiment may be replaced with the configuration of another embodiment, and the configuration of another embodiment may be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  The functions, processing units, processing means, and the like of the server side system and the user side system may be realized by hardware by designing a part or all of them, for example, with an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. In this case, a non-transitory computer readable medium in which the program code is recorded is provided to the information processing apparatus (computer), and the information processing apparatus (or CPU) is a non-transitory computer readable medium. The program code stored in is read. As the non-transitory computer-readable medium, for example, a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or a storage medium such as an IC card, an SD card, or a DVD is used.

  Further, the program code may be supplied to the information processing apparatus by various types of temporary computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the information processing apparatus via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  In the above-described embodiments, control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

101: Reading text 102: Reading generation processing unit 103: Prosody generation processing unit 104: Waveform generation processing unit 105: Reading speech 106: Reading correction candidate information generation processing unit 107: Prosody correction candidate information generation processing unit 108: Waveform correction candidate information Generation processing unit 109: correction candidate information integration processing unit 110: correction candidate information 201: reading text 202: word division processing unit 203: reading selection processing unit 204: delimitation determination processing unit 205: combination determination processing unit 207: word reading correction candidate Information generation processing unit 208: Separation correction candidate information generation processing unit 209: Reading correction candidate information integration processing unit 210: Reading correction candidate information 211: Combined correction candidate information generation processing unit 221: Continuous length correction candidate information generation processing unit 222: Basic Frequency correction candidate information generation processing unit 223: Height correction candidate information generation processing 224: Spoken speed correction candidate information generation processing unit 225: Prosody modification candidate information integration processing unit 226: Prosody modification candidate information 301: Word network information 302: Optimal word string selection processing unit 303: Word integration processing unit 304: Reading classification processing unit 306 : Word reading correction candidate information generation processing unit 307: Word reading correction candidate information 501 and 502: Word candidate 1301: Accent combination flag 1302: Accent combination candidate location flag 1601: Text to read out 1602: Speech synthesis context generation processing unit 1603: Meta information Speech synthesis processing unit 1604: sentence analysis meta information 1605: ID setting processing unit 1606 for each analysis unit: synthesized speech 1607: meta information with ID 1608: service response generation processing unit 1609: service response information 1610: speech synthesis data storage device 161 : Context storage device 1651: Improvement instruction information 1652: Improvement instruction interpretation processing unit 1653: target ID
1654: speech synthesis context selection processing unit 1657: synthesized speech 1658: sentence analysis meta-information 1659: improved data reflection processing unit 1660: improved portion determination processing unit 1661: improved data creation processing unit 1801: reading request information 1802: user information acquisition unit 1803: Speech synthesis selection unit 1804: User information database 1805: Speech synthesis system 1806: Speech synthesis data database 1807: Service content modification unit 1808: Service response information 1851: Service response information 1852: Correction candidate list presentation unit 1853: Display device 1854: Speaker device 1855: Improvement request target instruction unit 1856: Mouse / keyboard 1857: Improvement instruction request creation unit 1858: Improvement instruction information 1901: User ID
1902: Available language 1903: Acquired information 1904: Whether improvement request is possible

Claims (15)

A speech synthesis system that receives a read-out text and outputs a synthesized speech obtained by reading out the read-out text,
A speech synthesis system comprising: a correction candidate information generation processing unit that generates correction candidate information indicating a correctable portion and correction candidate contents in the synthesized speech based on information when the synthesized speech is generated. . The speech synthesis system according to claim 1,
The correction candidate information generation processing unit
A reading correction candidate information generation processing unit that generates reading correction candidate information that is correction information in the reading generation processing for generating the reading of the synthesized speech;
A prosody modification candidate information generation processing unit for generating prosody modification candidate information which is modification information in the prosody generation process for generating the prosody of the synthesized speech;
A waveform correction candidate information generation processing unit that generates waveform correction candidate information that is correction information in the waveform generation processing for generating the waveform of the synthesized speech;
A speech synthesis system comprising at least one of the following. The speech synthesis system according to claim 2,
The reading correction candidate information generation processing unit
A word reading correction candidate information generation processing unit that generates word reading correction information that is correction information related to the reading of a word in the synthesized speech;
A delimiter correction candidate information generation processing unit that generates delimiter correction candidate information that is correction information regarding the delimiter position of the synthesized speech;
A speech synthesis system comprising at least one of a combined correction candidate information generation processing unit that generates combined correction candidate information, which is correction information related to the position of accent combining of the synthesized speech. The speech synthesis system according to claim 2,
The prosody modification candidate information generation processing unit
A continuation length correction candidate information generation processing unit that generates continuation length correction candidate information that is correction information related to the phoneme continuation length of the synthesized speech;
A fundamental frequency correction candidate information generation processing unit that generates basic frequency correction candidate information that is correction information related to the phoneme fundamental frequency of the synthesized speech;
A height correction candidate information generation processing unit that generates height correction candidate information that is correction information related to a pitch pattern of the synthesized speech portion;
A speech speed correction candidate information generation processing unit that generates speech speed correction candidate information that is correction information related to the speech speed of the portion of the synthesized speech;
A speech synthesis system comprising at least one of the following. The speech synthesis system according to claim 2,
The waveform correction candidate information generation processing unit includes a unit correction candidate information generation processing unit that generates unit correction candidate information that is correction information related to a unit used in the synthesized speech. . The speech synthesis system according to claim 1,
A speech synthesis system, further comprising an improved data creation processing unit that updates the synthesized speech based on improvement instruction information that is improvement information of the synthesized speech. The speech synthesis system according to claim 6,
A speech synthesis data storage device for storing speech synthesis data for generating the synthesized speech;
An improved data reflection processing unit for updating the voice synthesis data in the voice synthesis data storage device based on the improvement instruction information;
A speech synthesis system, further comprising: The speech synthesis system according to claim 1,
A user information database that stores information about services provided to users;
A service content correction unit for correcting the content of the correction candidate information based on the information about the service;
A speech synthesis system, further comprising: The speech synthesis system according to claim 1,
The speech synthesizer system is characterized in that the correction candidate information is information obtained by adding a tag indicating the correctable portion and the content of the correction candidate to the read-out text data. The speech synthesis system according to claim 9,
The speech synthesis system, wherein the correction candidate information includes an identifier for each correctable portion. The speech synthesis system according to claim 1,
A server device that includes the correction candidate information generation processing unit and transmits the synthesized speech and the correction candidate information;
A client device that receives the synthesized speech and the correction candidate information from the server device;
A speech synthesis system comprising: The speech synthesis system according to claim 11.
The client device includes an improvement instruction request creating unit that creates improvement instruction information regarding improvement information selected from the correction candidate information,
The said server apparatus is provided with the improvement data creation process part which updates the said synthesized speech based on the said improvement instruction information, The speech synthesis system characterized by the above-mentioned. The speech synthesis system according to claim 11.
The client device is
A correction candidate list presentation unit for displaying the correction candidate information on a display device;
An improvement request target instruction unit that receives improvement information selected from the correction candidate information displayed on the display device;
A speech synthesis system, further comprising: A first step of generating a synthesized speech in which the input text is read out;
A second step of generating correction candidate information indicating a correctable portion and correction candidate content in the synthetic voice based on information when the synthetic voice is generated;
A speech synthesis method comprising: The speech synthesis method according to claim 14.
The second step includes
Generating reading correction candidate information that is correction information in a reading generation process for generating a reading of the synthesized speech;
Generating prosody correction candidate information that is correction information in the prosody generation process for generating the prosody of the synthesized speech;
Generating waveform correction candidate information that is correction information in a waveform generation process for generating a waveform of the synthesized speech;
A speech synthesis method comprising at least one of the following.

Images