CN107315742A - The Interpreter's method and system that personalize with good in interactive function - Google Patents

Abstract

The invention provides an anthropomorphic spoken language translation method with a man-machine dialogue function, which comprises the following steps: performing intelligent speech recognition on the source language voice to obtain the source language text; processing the source language text and the dialogue scene to perform anthropomorphic Man-machine dialogue and communication; perform machine translation and get translation results. The invention also provides an anthropomorphic spoken language translation system with man-machine dialogue function. According to the needs of the translation task, the present invention conducts man-machine dialogue with the user when necessary, can significantly improve the user's translation experience in complex application scenarios, and improve the accuracy of translation semantics.

Description

Anthropomorphic spoken language translation method and system with man-machine dialogue function

technical field

The invention relates to the field of computer and artificial intelligence, in particular to a spoken language translation method and a corresponding system which adds an anthropomorphic man-machine dialogue mechanism into the translation process.

Background technique

With the popularization and application of the Internet and the rapid advancement of globalization, oral translation, as an effective solution to the problems of high cost, high threshold, and imbalance between supply and demand of human translation, has a strong role in many scenarios such as daily life, business negotiation, and international communication. market demand.

The oral translation technology of two languages is composed as shown in Figure 1, including speech recognition, speech synthesis and two-way translation technology of source language and target language. Among them, two-way speech recognition and two-way translation are technologies that must be included, while speech synthesis is optional depending on the translation application scenario and equipment.

In the traditional automatic spoken language translation method, the user generally inputs the source language speech to be translated, automatically recognizes and translates it, and directly presents the natural speech of the target language to the other user. Spoken language recognition or translation is only an end-to-end software from the user's perspective. (as shown in picture 2).

Constrained by the complexity and variability of human communication language, even human translators will communicate with interlocutors in various ways in order to obtain the accurate connotation of the speech to be translated. However, the current machine spoken language translation method is an end-to-end rendering translation method that does not deal with the complexity of the actual scene and the semantic complexity, which is obviously difficult to meet the accuracy requirements. At the same time, because translation as a software service lacks human-computer communication with users, it is also difficult to meet the requirements of scene friendliness in actual application scenarios. How to improve the accuracy of oral translation and user experience in actual complex scenarios is a problem that needs to be solved at present.

Contents of the invention

(1) Technical problems to be solved

In view of the above technical problems, the present invention provides an anthropomorphic spoken language translation method and system with man-machine dialogue function. The core point of the present invention is to add a man-machine dialogue module on the basis of the original speech recognition and translation, which captures, processes and recognizes the acoustic scene, speaker scene, prosodic scene, language scene, etc. Conduct man-machine dialogues with users when necessary to obtain accurate translation experience that can significantly improve users' translation experience in complex application scenarios and improve the accuracy of translation semantics.

(2) Technical solution

According to one aspect of the present invention, there is provided an anthropomorphic spoken language translation method with a man-machine dialogue function, which includes the following steps: performing intelligent speech recognition on the source language speech to obtain the source language text; Processing, anthropomorphic man-machine dialogue and communication; machine translation, translation results.

According to another aspect of the present invention, there is also provided an anthropomorphic spoken language translation system with man-machine dialogue function, which includes: a speech recognition module, a man-machine dialogue management module, a machine translation module, and the speech recognition module is used to interpret the source language Intelligent voice recognition is performed on the voice to obtain the source language text; the man-machine dialogue management module is used to process the source language text and dialogue scenes to perform anthropomorphic man-machine dialogue communication; the machine translation module is used to perform machine translation to obtain the translation result.

(3) Beneficial effects

It can be seen from the above technical solutions that the anthropomorphic spoken language translation method and system with man-machine dialogue function of the present invention have at least one of the following beneficial effects:

(1) The present invention can significantly improve the accuracy of translation performance in complex application scenarios;

(2) The present invention makes it more convenient for users to use, and does not need to do any other redundant operations during the conversation;

(3) The present invention makes user translation and interactive experience more intelligent and humanized.

Description of drawings

FIG. 1 is a schematic diagram of oral translation technology of two languages in the prior art.

Fig. 2 is a schematic diagram of an automatic spoken language translation system in the prior art.

Fig. 3 is a schematic diagram of an anthropomorphic spoken language translation system with man-machine dialogue function according to the present invention.

FIG. 4 is a structural schematic diagram of a speech recognition module of an anthropomorphic spoken language translation system with man-machine dialogue function according to the present invention.

Fig. 5 is a detailed schematic diagram of an anthropomorphic spoken language translation system with man-machine dialogue function according to the present invention.

Fig. 6 is a schematic diagram of a method for acquiring source language speech input by a speaker in the first embodiment of the present invention.

Fig. 7 is a schematic diagram of a method for man-machine dialogue with a speaker in the first embodiment of the present invention.

Fig. 8 is a schematic diagram of a method for visually presenting the current system state to the speaker in the first embodiment of the present invention.

FIG. 9 is a schematic diagram of a method for intelligently outputting translation results to the other party in the dialogue in the first embodiment of the present invention.

Fig. 10 is a schematic diagram of a method for acquiring meeting information and creating a meeting in the second embodiment of the present invention.

Fig. 11 is a schematic diagram of a method for intelligently hosting a conference process in the second embodiment of the present invention.

Fig. 12 is a schematic diagram of a method for visually displaying the current conference status to participants in the second embodiment of the present invention.

Fig. 13 is a schematic diagram of a translation method based on an anthropomorphic spoken language translation system without screen display according to the third embodiment of the present invention.

detailed description

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in further detail below in conjunction with specific embodiments and with reference to the accompanying drawings. Apparently, the described embodiments are some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The present invention provides a kind of anthropomorphic spoken language translation method with man-machine dialogue function, as shown in Figure 3, it comprises the following steps: obtain source language speech (being the language of user A); Carry out intelligent speech recognition to source language speech, Get the source language text; process the source language text and the dialogue scene, and carry out anthropomorphic human-machine dialogue communication; perform machine translation to obtain the target language text; perform speech synthesis to obtain the target language voice (that is, the language of user B); output the target language voice.

In particular, for the source language speech and the target language speech in Figure 3, the source language speech is the language before translation, and the target language speech is the language after translation. The two are relative. For the same user, the The source language voice and the obtained target language voice are the same language. For example, the language of user A is Chinese, and the language of user B is English. User A sends Chinese (source language voice), and after translation, user B gets English (target language voice). voice); user B sends out English (voice in the source language), and after translation, user A gets Chinese (voice in the target language).

When intelligent voice recognition is performed on the source language voice, intelligent voice detection means and language recognition means are used on the translation interface to automatically discriminate the interlocutor's voice and language information, so that people using the translation system do not need to input keys based on two languages To conduct a dialogue; specifically, the probability score of each frame of speech/non-speech and the probability score of each frame of source language/target language are given, and the speech is simultaneously decoded bilingually. On this basis, it is meaningful to use the comprehensive output of information recognition results. As shown in Figure 4, the source language speech and the target language speech are two different language speeches, both of which refer to the language speech before translation.

As shown in Figure 5, when processing the source language text and the dialogue scene, it includes the processing of the acoustic scene, the speaker scene, the prosodic scene, and the language scene. Information about the user's translation experience in the scenario. Specifically, (1) perform acoustic background perception on the scene where the translation system is located. The information to be perceived includes but is not limited to detecting dynamic background noise information (such as signal-to-noise ratio, noise category, etc.) and intelligently processing the acoustic background perception information The results are comprehensively processed; (2) The scene of the interlocutor is intelligently sensed, and the information to be sensed includes but is not limited to speaker information, language information, whether there are multiple people speaking, voice separation information of multiple people speaking, and other speaker scene information, etc., and Comprehensively process these results (speaker scene perception information and intelligent processing results); (3) carry out intelligent perception of prosodic scenes, perception information includes but not limited to the middle pause of input speech, sentence boundary, speech rate, fundamental frequency, resonance peak and other supersegmental prosodic features, prosody analysis confidence and intelligent processing of translation round boundaries, etc., and comprehensively process these results (prosodic scene perception information and intelligent processing results); (4) perform intelligent perception of language scenes, namely Contextual intelligent processing of the source language text, including but not limited to: the names of people, places and organizations extracted from the text to be translated; possibly misrecognized words contained in the text to be translated; fragments of colloquial language contained in the text to be translated and Repetition; the absence of obvious components contained in the text to be translated; the reversal of the time sequence contained in the text to be translated; the time and number phrase language information contained in the text to be translated; industry terms, daily abbreviations, new words on the Internet, Ancient poems, idioms, common sayings, allegorical sayings, etc.; and these results (context intelligent processing results) are comprehensively processed.

The content of man-machine dialogue communication includes but is not limited to: if the above feature extraction needs to give friendly prompts to users, this includes prompting users to improve the use environment, speaker environment, prompting users to use correctly, etc.; If the judgment result requires further semantic explanation, start a man-machine dialogue to obtain the user's semantic explanation information for correct translation. There are clear prompts in the anthropomorphic translation interface during the above-mentioned dialogue, and such prompts include but are not limited to sound, graphics, icons, etc.

In addition, different states of the translation system during use can be displayed to users in visual or non-visual forms, including but not limited to: displaying different states of the translation system to users through visual anthropomorphic images; Mediums show users different states of the translation system.

In the present invention, the user's feedback on the content of the man-machine dialogue is acquired through contact or non-contact forms, including but not limited to: realizing the man-machine dialogue by clicking and touching the hardware equipment involved in the embodiment of the present invention Feedback or confirmation of content; Feedback or confirmation of man-machine dialogue content is realized through voice interaction.

The present invention also provides a kind of anthropomorphic spoken language translation system with man-machine dialogue function, it comprises: input module, is used for obtaining source language speech (namely the language of user A); Speech recognition module, is used for source language speech Intelligent speech recognition to obtain source language texts; man-machine dialogue management module, used to process source language texts and dialogue scenes, and perform anthropomorphic man-machine dialogue communication; machine translation module, used to perform machine translation to obtain target language texts; The speech synthesis module is used for speech synthesis to obtain the target language speech (that is, the language of user B); the output module is used for outputting the target language speech.

The first embodiment: the dialogue system of oral translation between the two parties on the mobile phone

In this embodiment, a two-party oral translation dialogue system on a mobile phone is provided. The system provides an end-to-speak translation dialogue function for both parties in the dialogue, and initiates a man-machine dialogue to the user when necessary to improve the user's translation experience.

(1) Obtain the source language speech input by the speaker, and the input method is optional according to the speaker's use environment and usage habits (as shown in Figure 6).

If the speaker's current environment is not conducive to direct use of voice input, the system provides alternatives for direct input of source language text;

If the speaker is used to manually specifying the language of the two parties in the conversation, the system provides a button for manually specifying the language, and at the same time allows manual switching of the language when the speaker changes;

If the speaker is accustomed to using the system to automatically identify the current language, the system provides the function of automatic language switching when the two parties are talking, so that the speaker does not need to manually specify the language of the current input voice;

If the speaker is used to manually clicking to determine the boundary of voice input, the system provides a voice input button, and the state of the speaker's operation of the button is used to obtain the boundary of the speaker's input voice. At the same time, the system automatically recognizes the language according to whether the speaker chooses To determine the number of voice input buttons, if the speaker chooses to manually specify the language, there are two voice input buttons, and the two parties in the dialogue operate their own voice input buttons respectively. If the speaker chooses the system to automatically identify the language, the two dialogue parties share the same voice input button;

If the speaker is accustomed to using the system to automatically identify the boundary of voice input, the system provides the function of automatic detection of voice input breakpoints, so that when the speaker pauses or stops voice input, the boundary of voice input is automatically recognized, and the currently acquired The voice input of the phone is handed over to the subsequent processing flow;

(2) Conduct a man-machine dialogue with the speaker when necessary, and the content of the man-machine dialogue can be selected according to the speaker's different acoustic scenes, speaker scenes, language scenes, etc. (as shown in Figure 7).

If the anthropomorphic Mediator dynamically calculates that the background noise intensity exceeds the set threshold when acquiring the speaker's voice input, the system will suggest the speaker to re-input the voice or change the input method;

If the anthropomorphic Mediator judges that there is a voice input that exceeds the language category set by the dialogue parties through processing the input voice, the system will suggest the speaker to reset the language option;

If the anthropomorphic Mediator judges that there are multiple speakers inputting voice at the same time by processing the input voice, the system will suggest that the speakers input voice in turn to obtain a better translation experience;

If the anthropomorphic Mediator analyzes the language scene and semantic confusion of the text to be translated by processing the text to be translated after automatic speech recognition of the input voice, when the complexity of the language scene or the semantic confusion exceeds the preset threshold, the The system will initiate a man-machine dialogue to obtain further explanations from the speaker on complex language scenarios and semantic confusion. The complex scenarios and semantic confusion here include but are not limited to:

If the text to be translated contains names of people, places and institutions, and there is ambiguity between them or their context, the system will suggest the speaker to confirm the word boundaries and structures of names of people, places and institutions;

If the text to be translated may contain misrecognized words, the system will advise the speaker to confirm whether the words are consistent with their real input;

If the text to be translated contains many colloquial language fragments and repetitions, the system will automatically analyze and reconstruct the text to be translated, and submit the reconstructed language expression to the speaker for confirmation. The structured language expression is handed over to the subsequent process. If the speaker vetoes it, the system will suggest that the speaker restructure the language and re-express its semantics in a more fluent manner;

If the text to be translated contains obvious missing components, the system will automatically complete the components of the text to be translated, and submit the completed language expression to the speaker for confirmation. If the confirmation is passed, the completed language expression will be submitted to the follow-up Process processing, if the speaker vetoes, the system will suggest that the speaker re-enter with a more complete language structure expression;

If the time sequence of the text to be translated is reversed, the system will automatically adjust the time sequence of the text to be translated, and submit the adjusted language expression to the speaker for confirmation. If the confirmation is passed, the adjusted language expression will be submitted to the subsequent process for processing. If the speaker rejects it, the system will suggest that the speaker re-enter the input with the normal timing;

If the text to be translated contains temporal numeral phrases, and there is ambiguity between the phrase itself or its context, the system will advise the speaker to confirm the phrase boundary and structure of temporal numeral phrases, including but not limited to: cardinal numbers, ordinal numbers , decimals, fractions, probability words, multiple words, approximate numbers, individual quantifiers, measure words, compound quantifiers, infinitive quantifiers, verb quantifiers, time quantifiers, noun quantifiers, time, duration, quarters, months, weeks, solar terms, festivals, Chronicle;

If the text to be translated contains proprietary phrases, and there is ambiguity between the phrase itself or its context, the system will suggest the speaker to confirm the phrase boundary and structure of the proprietary phrases. The proper nouns include but are not limited to: industry terms, Daily abbreviations, new words on the Internet, ancient poems, idioms, sayings, allegory.

At the same time, in the process of the above-mentioned human-computer dialogue, the communication and interaction methods of the human-computer dialogue include but are not limited to: realizing human-computer communication and interaction through voice question and answer; displaying prompts and requirements for the speaker through text; acknowledgment and reply.

If the anthropomorphic Mediator obtains the prosodic scene information of the input speech by processing and recognizing the input speech, then the system uses the prosodic scene information to help improve the speaker's translation service experience. The prosodic scenes here include but are not limited to:

The middle pause information of the input speech, if the anthropomorphic Mediator collects and obtains the middle pause information of the input speech by processing and recognizing the input speech, then the system will intelligently judge the semantic correlation and semantic center of gravity between the speaker's spoken speech fragments according to the information, And use it as the optimization basis for subsequent process processing;

Sentence boundary of the input voice, if the anthropomorphic Mediator collects and obtains the sentence boundary information of the input voice by processing and recognizing the input voice, then the system will intelligently judge the chapter segmentation of the text to be translated according to the information, and use this as a follow-up process Processing and providing segmentation information;

The intonation and emotion of the input voice. If the anthropomorphic Mediator collects and acquires the intonation and emotion information of the input voice by processing and recognizing the input voice, then the system will intelligently judge the semantic center of gravity and mid-sentence and sentence-end punctuation of the text to be translated according to the information. And provide semantic and emotional information for the subsequent process;

The translation round boundary of the input speech. If the anthropomorphic Mediator acquires the translation round boundary information of the input speech by processing and recognizing the input speech, the system will intelligently reset the memory module according to the information and use it as a new round of translation Basis for dialogue.

(3) Visually display the current system state to the speaker (as shown in Figure 8). A special anthropomorphic image is set in the visual interface to show the current system status to the speaker. The special anthropomorphic image includes but is not limited to: cartoons, stars, animals, robots, etc.; at different stages of the translation service provided by the system, the special anthropomorphic image Status includes but is not limited to:

When one of the dialogue parties acts as a speaker to input voice, the special anthropomorphic image is based on the language or speaker of the input voice, and faces the speaker in the state of listening to the input voice;

When initiating a man-machine dialogue to the speaker, the dedicated anthropomorphic image faces the speaker in the state of requesting answers, friendly prompts, intelligent judgments, etc. according to the actual scene of the man-machine dialogue;

When obtaining the speaker's reply to the content of the man-machine dialogue, the special anthropomorphic form faces the speaker in the state of listening to the reply, understanding, thanking, etc. according to the speaker's reply content;

When outputting the translation results to the other party, the dedicated anthropomorphic image faces the other direction in a state of speaking and communicating.

(4) Intelligently output translation results to the other party in the dialogue (as shown in FIG. 9 ). Anthropomorphic Mediator obtains the above-mentioned acoustics, semantics, prosody and other information of the speaker through man-machine dialogue and intelligent processing, and when necessary, adds it to the translation result and outputs it to the other party in the dialogue simultaneously. The output methods include but are not limited to: use the output text to mark red Mark the key parts of the output translation results by means of , bold, etc.; display the speaker's emotion and semantic focus by means of the accent and repetition of the output voice; use the method of adding automatic explanation to explain the rare words and professional concepts in the output text.

The second embodiment: multi-party oral translation conference system on the mobile phone

In this embodiment, a multi-party spoken language translation conference system on a mobile phone is provided. The system provides participants with an end-to-end multi-party spoken language conference translation function, provides an intelligent conference moderator function, and provides a conference call to the participants when necessary. Machine dialogue to improve the translation experience of the meeting.

(1) Obtain meeting information and create a meeting (as shown in Figure 10). The meeting creator specifies the meeting identification code, which is the basis for the unique identification of the meeting, and other meeting participants participate in the specified meeting by entering the meeting identification code; the meeting creator specifies the meeting name, and the meeting name is the content summary or participation of the meeting The information of the participants is reflected; the creator of the meeting specifies all the languages of the meeting, and the participants can only choose their own language among the languages selected by the creator of the meeting; the participants enter their names respectively, and their names are used as the names of the participants in the meeting The identification basis is reflected in the meeting interface and the anthropomorphic Mediator conversation.

(2) Anthropomorphic Mediator initiates a multi-party oral translation conference and intelligently hosts the conference process (as shown in Figure 11). If the meeting creator chooses the participants to host the meeting, the anthropomorphic Mediator will hand over the function of hosting the meeting to the meeting creator and the participants after starting the multi-party oral translation conference. These functions include but are not limited to:

If the conference creator selects the speaking mode of the participants as the microphone competition mode, the speaking order is determined by the participants themselves. When one participant speaks, the system will reject other participants' requests to speak until the When the participant finishes speaking, other participants can apply to speak. If there are multiple people applying to speak at the same time, the system will determine the next speaker according to the order in which the application requests arrive;

If the conference creator selects the speaking mode of the participants as the microphone designation mode, the speaking order is specified by the conference creator. When a participant applies to the conference creator to speak, the conference creator can grant the speaking right to the participant , and other participants cannot apply for the right to speak while the participant is speaking;

If the meeting creator manually specifies the speaking time limit, the system will remind the participants when the speaking time limit is reached;

If the meeting creator does not specify a limit on the speaking time, the system will not limit the speaking time of the participants, which is controlled by the participants themselves;

If the conference creator does not set the automatic monitoring of the acoustic scene, speaker scene, language scene and prosodic scene of the participant's voice, the anthropomorphic Mediator will not perform any intelligent processing on the participant's speech and its translation results, and directly Distribute translation results to other participants;

If the conference creator sets the automatic monitoring of the acoustic scene, speaker scene, language scene and prosodic scene of the participant's voice, the anthropomorphic Mediator will intelligently process the participant's speech and its translation results, and will process The result is returned to the conference creator, who decides whether to communicate or confirm further with the speaker, wherein the automatic monitoring of the acoustic scene, speaker scene, language scene and prosodic scene includes but is not limited to those involved in the first embodiment Various recognition and processing of input speech, text to be translated or translation results.

If the meeting creator chooses the anthropomorphic Mediator to host the meeting intelligently, the anthropomorphic Mediator will enable the function of hosting the meeting after starting the multi-party oral translation conference. These functions include but are not limited to:

The anthropomorphic Mediator can intelligently identify and process the meeting process and meeting records, and at the same time, intelligently determine the speaking order according to the order information of the participants' application for speaking;

Anthropomorphic Mediator intelligently judges the current speaker's speech content, speech speed and conference progress, and intelligently reminds the current speaker's speech length, content length, speech speed, etc. These smart reminders include:

If the speaker's speaking time is too long, the anthropomorphic Mediator will automatically remind the speaker to pay attention to the speaking time;

If the speaker's speech is too long, the anthropomorphic Mediator will automatically remind the speaker to segment the speech to achieve better translation performance;

If the speaker speaks too fast, the anthropomorphic Mediator automatically reminds the speaker to slow down and express with a more gentle rhythm.

Anthropomorphic Mediator intelligently monitors the acoustic scene, speaker scene, language scene and prosodic scene of the current speaker's input voice, and intelligently recognizes and processes the meeting process and meeting records, and dynamically decides the human-machine dialogue with the speaker. Necessity, among them, the reference factors for the dynamic decision-making of the anthropomorphic Mediator include but are not limited to:

If the anthropomorphic Mediator intelligent perception participants are more familiar with the conference theme, the system will increase the threshold or threshold for man-machine dialogue; if the anthropomorphic Mediator intelligent perception participants are less familiar with the conference theme, then The system will lower the threshold or threshold for engaging in human-computer dialogue;

If the anthropomorphic Mediator intelligently perceives the urgency of the meeting process is high, the system will increase the threshold or threshold for man-machine dialogue; if the anthropomorphic Mediator intelligently senses the urgency of the meeting process is low, the system will reduce the man-machine dialogue threshold or threshold.

Anthropomorphic Mediator's intelligent monitoring of the acoustic scene, speaker scene, language scene and prosodic scene of the current speaker's input voice includes but is not limited to the input speech, text to be translated or translation results involved in the first embodiment identification and processing.

(3) Visually display the current conference status to the participants (as shown in FIG. 12 ). Set up a special anthropomorphic image in the visual interface to show the current meeting status to the participants. The special anthropomorphic image includes but not limited to: cartoon shape, star, animal, robot, etc.; in different states of the meeting, the state of the special anthropomorphic image is simultaneously changes, including but not limited to:

When a participant applies for a speech, the dedicated anthropomorphic image faces the applicant in a state of waiting to listen, and at the same time informs other participants of the participant's information who applies for a speech;

When a participant speaks, the dedicated anthropomorphic image faces the speaker in a state of listening, and at the same time faces other participants in a state of speaking;

When the anthropomorphic Mediator conducts man-machine dialogue with the speaker, the dedicated anthropomorphic image faces the speaker in the state of requesting answers, friendly prompts, intelligent judgments, etc. according to the actual scene of the man-machine dialogue, and at the same time faces other participants in a waiting state By;

When the meeting creator modifies the relevant settings of the meeting, the special anthropomorphic image will notify other participants of the modification, including but not limited to: meeting theme, meeting identification code, meeting language, meeting host, speech time limit, speech mode, meeting participants.

(4) Anthropomorphic Mediator intelligently determines the end of the meeting

If the meeting creator chooses to end the meeting manually, the meeting can be terminated by clicking the function button of ending the meeting, and other participants will be forced to end the meeting;

If the meeting creator chooses to have the anthropomorphic Mediator intelligently decide to end the meeting, the anthropomorphic Mediator can intelligently determine the boundary of the meeting process by identifying and processing the meeting process and meeting records, and terminate the meeting after the meeting process ends.

In addition, after the meeting is over, the system will provide participants with meeting-related information, including but not limited to: meeting minutes, meeting statistical information, meeting personnel list, and meeting minutes.

The third embodiment: an anthropomorphic spoken language translation system based on no-screen display

In this embodiment, an anthropomorphic spoken language translation system based on no-screen display is provided. The system provides end-to-end anthropomorphic translation services to users without a screen. The system adopts the following technical solution (as shown in FIG. 13 ):

(1) Obtain information about the speaker

In the case of no screen display, the system acquires relevant information about the speaker through intelligent processing of the speaker's input voice, and the relevant information includes but is not limited to:

Optionally, the system requests all speakers to speak a common phrase in turn when starting up, so as to obtain the language information of the dialogue participants;

The system automatically identifies the speaker, and uses the identification result as an important basis for language identification, man-machine dialogue, and dialogue records for different speakers.

(2) Obtain the source language voice input by the speaker

After the dialogue starts, the system obtains the complete source language voice through the intelligent processing of the speaker's input voice. The intelligent processing of the input voice includes but is not limited to:

The system automatically detects the breakpoints of the speech input by the speaker, so as to intelligently identify the boundaries of the input speech and obtain complete speech fragments.

(3) Conduct man-machine dialogue with the speaker when necessary. The content of the man-machine dialogue can be selected according to the speaker's different acoustic scenes, speaker scenes, language scenes, etc.

The anthropomorphic Mediator starts the man-machine dialogue when necessary through the intelligent monitoring of the acoustic scene, speaker scene, language scene and prosody scene of the current input voice. The intelligent detection and man-machine dialogue include but are not limited to the first embodiment the content involved.

(4) In the case of no screen, the anthropomorphic Mediator shows the user the status of the current dialogue through different states of the voice

The anthropomorphic Mediator uses sound as a medium to display the current state of the conversation to the user. The ways of using sound as a medium include but are not limited to:

Anthropomorphic Mediator supports different states such as man-machine dialogue and translation result output through the gender of the voice;

Anthropomorphic Mediator supports different states such as man-machine dialogue and translation result output through the tone of the voice. In man-machine dialogue, anthropomorphic Mediator uses soft negotiating and requesting tone; when outputting translation results, anthropomorphic Mediator uses Use an objective, serious tone of expression;

Anthropomorphic Mediator supports different states such as man-machine dialogue and translation result output through the prefix background music of the sound. When the result is generated, the anthropomorphic Mediator will insert a short informative thick music before outputting.

(5) Intelligently output the voice of the translation result to the other party in the dialogue

Anthropomorphic Mediator obtains the above-mentioned acoustics, semantics, prosody and other information of the speaker through human-computer dialogue and intelligent processing, and when necessary, adds it to the translation result and outputs it to the other party in the dialogue simultaneously. The output methods include but are not limited to: using the accent of the output voice , repetition and other ways to show the speaker's emotional and semantic focus.

It should be noted that, in the accompanying drawings or in the text of the specification, implementations that are not shown or described are forms known to those of ordinary skill in the art, and are not described in detail. In addition, the above definitions of each element and method are not limited to the various specific structures, shapes or methods mentioned in the embodiments, and those skilled in the art can easily modify or replace them.

To sum up, the present invention provides an anthropomorphic spoken language translation method and system with man-machine dialogue function. The core point of the present invention is to add a man-machine dialogue module on the basis of the original speech recognition and translation, which captures, processes and recognizes the acoustic scene, speaker scene, prosodic scene, language scene, etc. Conduct man-machine dialogues with users when necessary to obtain accurate translation experience that can significantly improve users' translation experience in complex application scenarios and improve the accuracy of translation semantics.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. a kind of anthropomorphic spoken language translation method with man-machine dialogue function, it comprises the following steps: Perform intelligent speech recognition on the source language speech to obtain the source language text; Process the source language text and dialogue scenes, and carry out anthropomorphic man-machine dialogue communication; Perform machine translation and get translation results. 2. The anthropomorphic spoken language translation method according to claim 1, wherein, when carrying out intelligent speech recognition to source language speech, specifically comprise the following steps: Give the probability score of each frame of speech/non-speech, and the probability score of each frame of source language/target language; Simultaneously perform bilingual decoding on speech; Use information to synthesize and output meaningful recognition results. 3. The anthropomorphic spoken language translation method according to claim 1, wherein, when processing the source language text and the dialogue scene, it includes perceiving the acoustic scene, the speaker scene, the prosodic scene, and the language scene. 4. The anthropomorphic spoken language translation method according to claim 3, wherein when perceiving an acoustic scene, the information to be perceived includes: dynamic background noise information. 5. The anthropomorphic spoken language translation method according to claim 3, wherein, when the scene of the interlocutor is perceived, the information to be sensed includes: speaker information, language information, whether there are many people speaking, voice separation information of many people speaking, and others Speaker scene information. 6. The anthropomorphic spoken language translation method according to claim 3, wherein, when the prosody scene is perceived, the information to be sensed includes: a pause in the input speech, sentence boundary, speech rate, fundamental frequency, formant and prosody analysis confidence degrees and translation turn boundaries. 7. The anthropomorphic spoken language translation method according to claim 3, wherein, when the language scene is perceived, the information to be perceived includes: Names of people, places and institutions extracted from the text to be translated; Misidentified words that may be included in the text to be translated; Fragments and repetitions of colloquial language contained in the text to be translated; missing obvious elements contained in the text to be translated; Chronological inversions contained in the text to be translated; Language information of time, number, phrases contained in the text to be translated; Industry terminology, daily abbreviations, new words on the Internet, ancient poems, idioms, common sayings, allegories contained in the text to be translated. 8. The anthropomorphic spoken language translation method according to claim 1, wherein, when carrying out anthropomorphic man-machine dialogue communication, its communication content includes: Prompts on correct usage methods, tips on improving the usage environment or the speaker's environment; Obtain semantic description information when further semantic description is required. 9. An anthropomorphic spoken language translation system with man-machine dialogue function, comprising: The speech recognition module is used for performing intelligent speech recognition on the source language speech to obtain the source language text; The man-machine dialogue management module is used to process source language texts and dialogue scenes, and carry out anthropomorphic man-machine dialogue communication; The machine translation module is used for performing machine translation and obtaining translation results. 10. The anthropomorphic spoken language translation system according to claim 9, wherein the different states of the translation system are shown through visualized anthropomorphic images, or different states of the translation system are shown through non-visualized sound media.