CN107910004A - Speech translation processing method and device - Google Patents

Abstract

The embodiment of the invention provides a speech translation processing method and device, and belongs to the technical field of language processing. The method comprises the following steps: in the process of broadcasting the first synthesized voice signal, if a mixed voice signal containing part of the first synthesized voice signal is received, the broadcasting of the first synthesized voice signal is stopped. And filtering part of the first synthesized voice signals from the mixed voice signals to obtain the voice signals to be translated in the current round, and using the voice signals as target voice signals. And acquiring a second synthesized voice signal based on the target voice signal, and broadcasting the second synthesized voice signal. When a mixed voice signal containing part of the first synthesized voice signal is received, the broadcasting of the first synthesized voice signal is stopped, and the broadcasting of the second synthesized voice signal is also stopped. Because any party in the communication process can interrupt the broadcasting process at any time according to the full-duplex mode, the broadcasting process does not need to be waited to finish each time, and the communication efficiency can be improved.

Description

Speech translation processing method and device

technical field

Embodiments of the present invention relate to the technical field of language processing, and more specifically, to a speech translation processing method and device.

Background technique

At present, language communication has become an important issue faced by different ethnic groups when they communicate with each other. For example, in a two-person or multi-person meeting, voice translation can be realized through an automatic voice translation system. Among them, the automatic speech translation system usually consists of three parts: speech recognition, machine translation and speech synthesis. The source language voice signal is obtained through speech recognition to obtain the source language text data, and then the source language text data is translated into the target language text data through machine translation, and finally the target language text data is synthesized by speech to obtain the target language voice signal and broadcast. When performing voice translation, it is necessary to wait until the last round of target language voice signals are broadcast before translating the next round of source language voice signals.

Because in real-time communication, the listener may be able to understand the meaning of what the speaker wants to say when the last round of broadcast process is halfway through, and the listener needs to wait for the end of the last round of broadcast process before proceeding to the next round. Rounds of communication, resulting in low communication efficiency.

Contents of the invention

In order to solve the above problems, an embodiment of the present invention provides a voice translation processing method and device for overcoming the above problems or at least partially solving the above problems.

According to a first aspect of an embodiment of the present invention, a speech translation processing method is provided, the method comprising:

In the process of broadcasting the first synthesized speech signal, if a mixed speech signal containing part of the first synthesized speech signal is received, the broadcast of the first synthesized speech signal is stopped, and the first synthesized speech signal is translated through the last round and what is obtained after speech synthesis;

Filter out part of the first synthesized speech signal from the mixed speech signal, obtain the speech signal to be translated in the current round, and use it as the target speech signal;

Based on the target voice signal, a second synthesized voice signal is obtained, and the second synthesized voice signal is broadcast. The second synthesized voice signal is obtained by translating and voice-synthesizing the target voice signal.

In the method provided by the embodiment of the present invention, in the process of broadcasting the first synthesized voice signal, if a mixed voice signal including part of the first synthesized voice signal is received, the broadcast of the first synthesized voice signal is stopped. Part of the first synthesized speech signal is filtered out from the mixed speech signal to obtain the speech signal to be translated in the current round as the target speech signal. Obtain a second synthesized voice signal based on the target voice signal, and broadcast the second synthesized voice signal. Since any party in the communication process can interrupt the broadcast process at any time according to the full-duplex mode, instead of waiting for a round of broadcast process to end every time, it can improve communication efficiency and enable users of different languages to communicate More natural flow.

With reference to the first possible implementation of the first aspect, in the second possible implementation, the first synthesized speech signal and the target speech signal are of the same language type or the first synthesized speech signal and the target speech signal are different Language type.

In combination with the first possible implementation of the first aspect, in a third possible implementation, before broadcasting the second synthesized voice signal, the method further includes:

Acquiring recognized text data obtained after performing speech recognition on the target speech signal, obtaining target text data obtained after translating the recognized text data, and performing speech synthesis on the target text data to obtain a second synthesized speech signal.

In combination with the third possible implementation of the first aspect, in a fourth possible implementation, the target text data obtained after translating the recognized text data includes:

Determine the source language type corresponding to the recognized text data, and determine the target language type corresponding to the source language type according to the preset correspondence relationship;

Input the target language type and recognized text data into the translation codec model, and output the target text data.

In combination with the fourth possible implementation of the first aspect, in the fifth possible implementation, based on the voiceprint features in the target voice signal, determine the preset language type corresponding to the voiceprint feature as the corresponding source language type target language type.

In combination with the third possible implementation of the first aspect, in a sixth possible implementation, the target text data obtained after translating the recognized text data includes:

If it is determined that the information conveyed by the target speech signal and the first synthesized speech signal are related to each other, the recognized text data is translated based on the speech signal and/or the translation result of the round before the current round to obtain the target text data.

In combination with the third possible implementation of the first aspect, in a seventh possible implementation, speech synthesis is performed on the target text data to obtain a second synthesized speech signal, including:

Acquire speech broadcast parameters, input the target text data and speech broadcast parameters into the speech synthesis model, and output the second synthesized speech signal, the speech broadcast parameters include at least the timbre parameters used when broadcasting the second synthesized speech signal.

According to a second aspect of the embodiments of the present invention, there is provided a speech translation processing device, the device comprising:

Stop the broadcast module, for broadcasting the first synthesized voice signal, if a mixed voice signal containing part of the first synthesized voice signal is received, stop broadcasting the first synthesized voice signal, the first synthesized voice signal is Obtained after the last round of translation and speech synthesis;

A filter module, configured to filter out part of the first synthesized speech signal from the mixed speech signal, to obtain the speech signal to be translated in the current round, and use it as the target speech signal;

The broadcasting module is used to obtain a second synthesized voice signal based on the target voice signal, and broadcast the second synthesized voice signal. The second synthesized voice signal is obtained by translating and speech-synthesizing the target voice signal.

According to a third aspect of the embodiments of the present invention, a speech translation processing device is provided, including:

at least one processor; and

at least one memory communicatively coupled to the processor, wherein:

The memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the speech translation processing method provided in any one of the various possible implementations of the first aspect.

According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute any of the various possible implementations of the first aspect. A speech translation processing method provided by a possible implementation manner.

It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory, and are not intended to limit the embodiments of the present invention.

Description of drawings

Fig. 1 is a schematic flow chart of a speech translation processing method according to an embodiment of the present invention;

Fig. 2 is a schematic flow chart of a speech translation processing method according to an embodiment of the present invention;

Fig. 3 is a schematic flow chart of a speech translation processing method according to an embodiment of the present invention;

4 is a block diagram of a speech translation processing device according to an embodiment of the present invention;

Fig. 5 is a block diagram of a speech translation processing device according to an embodiment of the present invention.

Detailed ways

The specific implementation manners of the embodiments of the present invention will be further described in detail below in conjunction with the drawings and embodiments. The following examples are used to illustrate the embodiments of the present invention, but are not intended to limit the scope of the embodiments of the present invention.

At present, when people of different languages communicate, it is usually realized through an automatic voice translation system. Among them, the automatic speech translation system usually consists of three parts: speech recognition, machine translation and speech synthesis. The source language voice signal is obtained through speech recognition to obtain the source language text data, and then the source language text data is translated into the target language text data through machine translation, and finally the target language text data is synthesized by speech to obtain the target language voice signal and broadcast. When performing speech translation, it is necessary to wait until the last round of speech signals in the target language is broadcasted before the next round of translation, speech synthesis and broadcasting can be performed.

For example, user A communicates with user B, user A speaks English, and user B speaks Chinese. User A speaks a sentence of English, gets a sentence of Chinese through the process of translation and speech synthesis, and broadcasts it. After finishing the broadcast of the whole sentence, user A can continue to speak a sentence of English or user B can speak a sentence of Chinese, and repeat the above-mentioned process of translation, speech synthesis and broadcast. That is to say, user A and user B need to wait until the end of the system broadcast to receive new voice data, and perform translation, speech synthesis and broadcast.

Considering that after the user finishes speaking a sentence of voice data, it may be necessary to supplement or modify the voice data just spoken. In addition, when the synthesized voice signal is broadcast, the user who listens to the broadcast may be able to understand the intention of the speaking user without listening to the end. For the above situation, if the previous round of full-sentence broadcast is completed according to the above process, and then the next round of translation, speech synthesis and broadcast is performed, it will be more time-consuming. In view of the above problems, an embodiment of the present invention provides a speech translation processing method. The method can be applied to a terminal or system with functions of voice collection, translation, synthesis, and broadcast, and can be applied to two or more communication scenarios, which is not specifically limited in the embodiments of the present invention. Referring to Fig. 1, the method includes: 101. In the process of broadcasting the first synthesized speech signal, if a mixed speech signal containing part of the first synthesized speech signal is received, stop broadcasting the first synthesized speech signal, and first The synthesized speech signal is obtained after the last round of translation and speech synthesis; 102. Filter out part of the first synthesized speech signal from the mixed speech signal to obtain the speech signal to be translated in the current round and use it as the target speech signal; 103 . Obtain a second synthesized voice signal based on the target voice signal, and broadcast the second synthesized voice signal, where the second synthesized voice signal is obtained by translating and speech-synthesizing the target voice signal.

In the above step 101, the first synthesized speech signal is obtained after collecting, translating and speech-synthesizing the source language signal of the previous round. In the process of broadcasting the first synthesized voice signal, it is possible to simultaneously monitor whether there is a new voice signal in the source language, that is, monitor whether any user speaks something that needs to be translated and broadcasted. Specifically, monitoring may be performed by starting a monitoring thread, which is not specifically limited in this embodiment of the present invention. During the monitoring process, since the first synthesized voice signal of the previous round is still broadcasting, the monitored voice signal contains not only the new source language voice signal (that is, the user's new speech), but also some first synthesized voice signals. A synthesized speech signal, whereby a mixed speech signal comprising a portion of the first synthesized speech signal is received. After receiving the mixed voice signal containing part of the first synthesized voice signal, it means that a user has made a speech at this time, and the speech may be supplemented or modified by the user who spoke in the last round, thereby interrupting the first synthesized voice signal speak during the broadcast process. Or, it is also possible that the user who listened to the broadcast understands the intention conveyed by the user who spoke in the previous round before finishing listening, and thus interrupts the broadcast process of the first synthesized voice signal to speak.

Since the mixed speech signal includes not only the mixed speech signal containing part of the first synthesized speech signal, but also the speech signal to be translated in the current round, and the subsequent speech signal to be translated in the current round needs to be translated, speech synthesized and Broadcasting, so in the above step 102, part of the first synthesized speech signal needs to be filtered out from the mixed speech signal, so as to obtain the speech signal to be translated in the current round. The embodiment of the present invention does not specifically limit the manner of filtering out part of the first synthesized speech signal from the mixed speech signal, including but not limited to filtering out part of the first synthesized speech signal from the mixed speech signal by means of echo cancellation. Among them, the calculation process of echo cancellation can be shown as follows:

Taking the monitoring device as a microphone as an example, assuming that the broadcast part of the first synthesized speech signal is s(t), the channel transfer function received by the mth microphone is h _m (t), and the speech signal to be translated newly input by the user is x _m (t), then the observed signal y _m (t) received by the microphone is shown in the following formula:

y _m (t)=s(t)*h _m (t)+x _m (t)

When there is no new input speech signal x _m (t) to be translated, the channel transfer function h _m (t) can be estimated in advance. When there is a new input speech signal x _m (t) to be translated, echo cancellation can be performed on the mixed speech signal. Since y _m (t), s (t), and h _m (t) are known, the speech signal to be translated in the current round can be calculated by the following formula and used as the target speech signal. The specific formula is as follows:

x' _m (t)=y(t)-s(t)*h _m (t)

After the target voice signal is obtained, the target voice signal can be translated and speech synthesized to obtain the second synthesized voice signal of the current round, and the second synthesized voice signal is broadcast.

In the method provided by the embodiment of the present invention, in the process of broadcasting the first synthesized voice signal, if a mixed voice signal including part of the first synthesized voice signal is received, the broadcast of the first synthesized voice signal is stopped. Part of the first synthesized speech signal is filtered out from the mixed speech signal to obtain the speech signal to be translated in the current round as the target speech signal. Obtain a second synthesized voice signal based on the target voice signal, and broadcast the second synthesized voice signal. Since any party in the communication process can interrupt the broadcast process at any time according to the full-duplex mode, instead of waiting for a round of broadcast process to end every time, it can improve communication efficiency and enable users of different languages to communicate More natural flow.

Based on the content of the above-mentioned embodiments, any party in the communication process can interrupt the broadcast process according to the needs, so that the speaking user can interrupt the broadcast process of his own speech, and the user listening to the broadcast can interrupt the broadcast of the last round of user speech process. Correspondingly, as an optional embodiment, the first synthesized speech signal and the target speech signal are of the same language type, or the first synthesized speech signal and the target speech signal are of different language types.

Wherein, when the speech signal before the translation of the first synthesized speech signal is of the same language type as the target speech signal, it may be that the user who spoke in the last round interrupted the broadcast process of the last round of his own speech, or It may be that the user speaking in the current round uses the same language type as the user speaking in the previous round, and the user speaking in the current round interrupts the broadcast process of the user speaking in the previous round. When the first synthesized speech signal and the target speech signal are of different language types, it may be that the user listening to the broadcast interrupted the broadcast process of the last round of user speech, or it may also be the user who spoke last round Interrupted the broadcast process of the last round of his own speech, and spoke in a language type different from the previous round. Based on the above content, the embodiments of the present invention are applicable to communication scenarios in different languages between two or more people, such as two or more people conferences.

Based on the content of the foregoing embodiments, before broadcasting the second synthesized voice signal, it is necessary to obtain the second synthesized voice signal. Correspondingly, as an optional embodiment, an embodiment of the present invention provides a method for obtaining a synthesized speech signal. Referring to Fig. 2, the method includes: 201, obtaining the recognition text data obtained after performing speech recognition on the target voice signal; 202, obtaining the target text data obtained after translating the recognition text data; 203, performing speech synthesis on the target text data , to obtain the second synthesized speech signal.

Based on the content of the foregoing embodiments, as an optional embodiment, an embodiment of the present invention further provides a method for acquiring target text data. Referring to Fig. 3, the method includes: 2011, determine the source language type corresponding to the recognition text data, and determine the target language type corresponding to the source language type according to the preset correspondence; 2012, input the target language type and the recognition text data into the translation editor Decode the model and output the target text data.

In the above step 2011, it is first necessary to determine the source language type corresponding to the recognized text data. The embodiment of the present invention does not specifically limit the manner of determining the source language type corresponding to the text data, including but not limited to the following two manners.

The first way: based on the acoustic feature determination of the target speech signal.

Specifically, the acoustic features of the target speech signal can be extracted, such as spectral features: Mel-Frequency Cepstral Coefficients (MFCC), perceptual linear predictive coefficients (Linear Predictive Coding, PLP), etc., and the acoustic features can be input to language recognition A model to predict the language of the target speech signal. The output result of the language recognition model is the probability that the target speech signal is each language type, and the language with the highest probability is selected as the language corresponding to the target speech signal, that is, the source language type corresponding to the recognized text data is determined. Among them, the language recognition model is generally a classification model commonly used in pattern recognition. Specifically, it can be constructed by collecting a large number of speech signals in advance, extracting the acoustic features of each speech signal, and marking the language type of each speech signal.

The second way is to determine based on the recognition result of the target speech signal.

Specifically, use the speech recognition model corresponding to each language currently involved to perform speech recognition on the target speech signal, obtain the recognition text data corresponding to each language of the target speech signal and the corresponding recognition confidence, and select the recognition model with the highest recognition confidence. The language corresponding to the text data is used as the language of the target speech signal. Wherein, the voice recognition process generally includes: first, endpoint detection is performed on the target voice signal to obtain the starting point and the ending point of the effective voice segment. Then, feature extraction is performed on the effective speech segment obtained by endpoint detection, and then the extracted feature data and the pre-trained acoustic model and language model are used for decoding to obtain the corresponding recognition text of the current speech data and the confidence of the corresponding recognition text.

For example, assume that the language corresponding to the target voice signal is Chinese; the currently involved languages are Chinese and English. When performing language recognition on the target speech signal, use the Chinese speech recognition model and the English speech recognition model to perform speech recognition on the target speech signal, and obtain the Chinese recognition text data corresponding to the target speech signal and the corresponding recognition confidence of 0.9, and the English recognition text data and The corresponding recognition confidence is 0.2. Select the corresponding language of the recognized text data with higher recognition confidence, that is, Chinese as the corresponding language of the target voice signal. Further, the recognition confidence and language model score of the recognized text data corresponding to each language can be fused, and the language corresponding to the recognized text data with the highest fusion score can be selected as the language corresponding to the target speech signal. Wherein, the fusion method may be a linear weighting method, which is not specifically limited in this embodiment of the present invention.

After determining the source language type corresponding to the recognized text data, the target language type corresponding to the source language type can be determined according to the preset correspondence relationship. Wherein, the source language type is the language used by the speaking user, and the target language type generally refers to other language types except the source language type among the languages involved in the current user communication. Wherein, the type of the target language may be one or more types, which respectively correspond to communication between users who use two languages and users who use multiple languages.

For example, if the languages involved in the current user communication are Chinese, English, French, Thai, Hindi, German, etc., it can be pre-determined that the Chinese voice signal is translated into English and French each time, that is, English and French as the target language type.

In addition to setting a fixed target language type in advance, the target language type corresponding to the source language type can also be determined based on the preference or demand of the speaking user. Correspondingly, as an optional embodiment, the embodiment of the present invention does not specifically limit the method of determining the target language type corresponding to the source language type according to the preset correspondence relationship, including but not limited to: based on the voiceprint features in the target voice signal , determining the preset language type corresponding to the voiceprint feature as the target language type corresponding to the source language type.

For example, if the leader's speech in a meeting needs to be automatically translated into English, the target language type of the leader's speech can be determined in advance as English, and a corresponding relationship with the voiceprint features of the leader's speech can be established, so that the leader can be extracted in the meeting. After the voiceprint features of the speech, it can be directly determined that the target language type is English.

In the above step 2012, when inputting the target language type and the recognized text data into the translation codec model and outputting the target text data, a neural network-based translation codec model can be used to translate the recognized text data into the corresponding target language text data, which is not specifically limited in this embodiment of the present invention. Among them, the translation codec model corresponding to each language type can be constructed in advance using a large amount of training data.

In addition to translating the recognition text data by translating the codec model, the embodiment of the present invention also provides a method for obtaining the target text data in consideration of the correlation between the context and content of the speech, including but not limited to: if If it is determined that the information conveyed by the target speech signal and the first synthesized speech signal are related to each other, the recognized text data is translated based on the speech signals and/or translation results of the round before the current round to obtain the target text data.

Specifically, it can be determined whether the information conveyed by the target voice signal of the current round and the first synthesized voice signal of the previous round are related to each other, if the two are related (such as during the broadcast process, the speaking user interrupts the broadcast and needs to supplement ), then during translation, the target speech signal of the current round can be fused with the speech signal that has been translated in the previous round before translation, and finally the translation result will be broadcast; or after translation, the two rounds of speech After the translation results of the signals are fused, they are broadcasted; it is also possible to combine the translated speech signals of the previous round, the translation results of the previous round, the target speech signal, and the translation results of the target speech signal to combine the translation results of the two rounds In addition, in the above-mentioned process of translating the recognized text data of the current round, in addition to using the voice signal and/or translation results of the previous round as a reference, the previous n rounds can also be used as a reference. The speech signal and/or the translation result are used as a reference, which is not specifically limited in this embodiment of the present invention. Wherein, n is greater than or equal to 1.

After the target text data is acquired, speech synthesis may be performed on the target text data to obtain a second synthesized speech signal. Correspondingly, as an optional embodiment, the embodiment of the present invention does not specifically limit the method of performing speech synthesis on the target text data to obtain the second synthesized speech signal, including but not limited to: obtaining speech broadcast parameters, The target text data and speech broadcast parameters are input to the speech synthesis model, and the second synthesized speech signal is output, and the speech broadcast parameters include at least the timbre parameters used when broadcasting the second synthesized speech signal.

Wherein, when performing speech synthesis, a fixed speaker model can be selected for synthesis, for example, a synthesis model with a neutral and thick voice can be used to synthesize the corresponding second synthesized speech signal. Of course, you can also choose a personalized speaker model for synthesis. Specifically, the voice translation system may include a variety of voices with different timbres, which can be selected by the user or selected by the system based on the user information of the current user, which is not specifically limited in this embodiment of the present invention. Wherein, the user information includes but not limited to the user's gender, age, timbre and so on. For example, if the user listening to the broadcast is a male, the system can automatically select a female speaker model to synthesize a second synthesized voice signal uttered by a female.

Of course, voice conversion can also be used to convert the synthesized voice into a voice similar to the user's timbre for broadcasting. For example, after translating the speech signal input by user A to be translated, voice conversion is used to make the timbre of the converted speech signal similar to that of user A, and the converted speech signal is broadcast.

In the method provided by the embodiment of the present invention, in the process of broadcasting the first synthesized voice signal, if a mixed voice signal including part of the first synthesized voice signal is received, the broadcast of the first synthesized voice signal is stopped. Part of the first synthesized speech signal is filtered out from the mixed speech signal to obtain the speech signal to be translated in the current round as the target speech signal. Obtain a second synthesized voice signal based on the target voice signal, and broadcast the second synthesized voice signal. Since any party in the communication process can interrupt the broadcast process at any time according to the full-duplex mode, instead of waiting for a round of broadcast process to end every time, it can improve communication efficiency and enable users of different languages to communicate More natural flow.

In addition, since the recognized text data is translated based on the speech signals and/or translation results of the rounds before the current round, the relevance of the context of speech can be fully utilized to improve translation accuracy.

Finally, since the target language type corresponding to the source language type can be determined according to the preset corresponding relationship, for example, the target language type can be determined based on the voiceprint characteristics in the target voice signal, so as to meet the user's preferences and needs when translating, and realize personalized translation. customization.

It should be noted that all the above optional embodiments may be combined in any way to form optional embodiments of the present invention, which will not be repeated here.

Based on the content of the foregoing embodiments, an embodiment of the present invention provides a speech translation processing device, the speech translation processing device is configured to execute the speech translation processing method in the foregoing method embodiments. Referring to Figure 4, the device includes:

Stop broadcasting module 401, for broadcasting the first synthesized voice signal, if a mixed voice signal containing part of the first synthesized voice signal is received, stop broadcasting the first synthesized voice signal, the first synthesized voice signal It is obtained after the last round of translation and speech synthesis;

The filtering module 402 is used to filter out part of the first synthesized speech signal from the mixed speech signal, obtain the speech signal to be translated in the current round, and use it as the target speech signal;

The broadcasting module 403 is configured to acquire a second synthesized voice signal based on the target voice signal, and broadcast the second synthesized voice signal. The second synthesized voice signal is obtained by translating and speech-synthesizing the target voice signal.

As an optional embodiment, the first synthesized speech signal and the target speech signal are of the same language type, or the first synthesized speech signal and the target speech signal are of different language types.

As an optional embodiment, the device also includes:

The first obtaining module is used to obtain the recognized text data obtained after performing speech recognition on the target speech signal;

The second acquiring module is used to acquire target text data obtained after translating the recognized text data;

The speech synthesis module is configured to perform speech synthesis on the target text data to obtain a second synthesized speech signal.

As an optional embodiment, the second acquisition module includes:

A determining unit, configured to determine the source language type corresponding to the recognized text data, and determine the target language type corresponding to the source language type according to the preset correspondence relationship;

The translation unit is used to input the target language type and recognized text data into the translation codec model, and output the target text data.

As an optional embodiment, the determining unit is configured to determine, based on the voiceprint features in the target voice signal, a preset language type corresponding to the voiceprint features as a target language type corresponding to the source language type.

As an optional embodiment, the second acquisition module is configured to, when it is determined that the information conveyed by the target speech signal and the first synthesized speech signal are related to each other, based on the speech signal and/or translation of the previous round of the current round As a result, the recognized text data is translated to obtain target text data.

As an optional embodiment, the speech synthesis module is used to obtain speech broadcast parameters, input target text data and speech broadcast parameters to the speech synthesis model, and output the second synthesized speech signal, and the speech broadcast parameters include at least the second synthesized speech signal. Timbre parameters used for voice signals.

The device provided by the embodiment of the present invention stops broadcasting the first synthesized voice signal if a mixed voice signal including part of the first synthesized voice signal is received during broadcasting of the first synthesized voice signal. Part of the first synthesized speech signal is filtered out from the mixed speech signal to obtain the speech signal to be translated in the current round as the target speech signal. Obtain a second synthesized voice signal based on the target voice signal, and broadcast the second synthesized voice signal. Since any party in the communication process can interrupt the broadcast process at any time according to the full-duplex mode, instead of waiting for a round of broadcast process to end every time, it can improve communication efficiency and enable users of different languages to communicate More natural flow.

In addition, since the recognized text data is translated based on the speech signals and/or translation results of the rounds before the current round, the relevance of the context of speech can be fully utilized to improve translation accuracy.

Finally, since the target language type corresponding to the source language type can be determined according to the preset corresponding relationship, for example, the target language type can be determined based on the voiceprint characteristics in the target voice signal, so as to meet the user's preferences and needs when translating, and realize personalized translation. customization.

An embodiment of the present invention provides a speech translation processing device. Referring to FIG. 5, the device includes: a processor (processor) 501, a memory (memory) 502 and a bus 503;

Wherein, the processor 501 and the memory 502 complete the mutual communication through the bus 503 respectively;

The processor 501 is used to call the program instructions in the memory 502 to execute the speech translation processing method provided by the above-mentioned embodiments, for example, including: in the process of broadcasting the first synthesized speech signal, if a part of the first The mixed voice signal of the synthesized voice signal, then stop broadcasting the first synthesized voice signal, the first synthesized voice signal is obtained after the last round of translation and speech synthesis; filter out part of the first synthesized voice signal from the mixed voice signal, Obtain the speech signal to be translated in the current round, and use it as the target speech signal; based on the target speech signal, obtain the second synthesized speech signal, and broadcast the second synthesized speech signal, the second synthesized speech signal is to translate the target speech signal and voice obtained after synthesis.

An embodiment of the present invention provides a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the speech translation processing method provided in the above-mentioned embodiments, for example, including: In the process of broadcasting the first synthesized voice signal, if a mixed voice signal containing part of the first synthesized voice signal is received, then stop broadcasting the first synthesized voice signal, the first synthesized voice signal is translated and Obtained after speech synthesis; filter out part of the first synthesized speech signal from the mixed speech signal, obtain the speech signal to be translated in the current round, and use it as the target speech signal; obtain the second synthesized speech signal based on the target speech signal, and broadcast The second synthesized speech signal, which is obtained by translating and speech-synthesizing the target speech signal.

Those of ordinary skill in the art can understand that all or part of the steps for realizing the above-mentioned method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the It includes the steps of the above method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

The above-described embodiments such as information interaction equipment are only illustrative, and the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may Located in one place, or can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative effort.

Through the above description of the implementations, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic Discs, optical discs, etc., include several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute various embodiments or some partial methods of the embodiments.

Finally, the method of the present application is only a preferred implementation, and is not intended to limit the scope of protection of the examples of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the embodiments of the present invention shall be included in the protection scope of the embodiments of the present invention.

Claims (10)

1. A voice translation processing method, characterized in that, comprising: In the process of broadcasting the first synthesized voice signal, if a mixed voice signal containing part of the first synthesized voice signal is received, the broadcast of the first synthesized voice signal is stopped, and the first synthesized voice signal is Obtained after the last round of translation and speech synthesis; Filtering out part of the first synthesized speech signal from the mixed speech signal to obtain the speech signal to be translated in the current round as the target speech signal; Obtaining a second synthesized voice signal based on the target voice signal, and broadcasting the second synthesized voice signal, where the second synthesized voice signal is obtained by translating and speech-synthesizing the target voice signal. 2. The method according to claim 1, wherein the first synthesized speech signal and the target speech signal are of the same language type or the first synthesized speech signal is different from the target speech signal Language type. 3. The method according to claim 1, wherein, before the broadcasting of the second synthesized speech signal, further comprising: Acquiring recognized text data obtained after performing speech recognition on the target speech signal, obtaining target text data obtained after translating the recognized text data, and performing speech synthesis on the target text data to obtain the second synthesized voice signal. 4. The method according to claim 3, wherein said acquiring the target text data obtained after translating said recognition text data comprises: Determine the source language type corresponding to the identified text data, and determine the target language type corresponding to the source language type according to the preset correspondence relationship; Input the target language type and the recognized text data into the translation codec model, and output the target text data. 5. The method according to claim 4, wherein said determining the target language type corresponding to the source language type according to the preset corresponding relationship comprises: Based on the voiceprint feature in the target voice signal, determine a preset language type corresponding to the voiceprint feature as the target language type corresponding to the source language type. 6. The method according to claim 3, wherein said acquiring the target text data obtained after translating said recognition text data comprises: If it is determined that the target speech signal and the information conveyed by the first synthesized speech signal are related to each other, then based on the speech signal and/or the translation result of the round before the current round, the recognized text data is translated to obtain The target text data is obtained. 7. The method according to claim 3, wherein said performing speech synthesis on said target text data to obtain said second synthesized speech signal comprises: Acquiring speech broadcast parameters, inputting the target text data and the speech broadcast parameters into the speech synthesis model, and outputting the second synthesized speech signal, the speech broadcast parameters at least include the The tone parameters to use. 8. A speech translation processing device, characterized in that, comprising: stop broadcasting module, for broadcasting the first synthesized voice signal, if a mixed voice signal containing part of the first synthesized voice signal is received, then stop broadcasting the first synthesized voice signal, the The first synthesized speech signal is obtained after a previous round of translation and speech synthesis; A filtering module, configured to filter out part of the first synthesized speech signal from the mixed speech signal, to obtain the speech signal to be translated in the current round, and use it as the target speech signal; The broadcast module is used to obtain a second synthesized voice signal based on the target voice signal, and broadcast the second synthesized voice signal, the second synthesized voice signal is obtained after the target voice signal is translated and speech synthesized owned. 9. A speech translation processing device, characterized in that, comprising: at least one processor; and at least one memory communicatively coupled to the processor, wherein: The memory stores program instructions executable by the processor, and the processor can execute the method according to any one of claims 1 to 7 by calling the program instructions. 10. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the computer according to any one of claims 1 to 7. Methods.