CN112417896B - Domain data acquisition method, machine translation method and related equipment - Google Patents

Abstract

The application provides a domain data acquisition method, a machine translation method and related equipment, wherein the domain data acquisition method can determine a first target value corresponding to a training corpus in a training corpus set of a general domain, and because the first target value can represent the matching degree of the corresponding training corpus and a designated domain, the training corpus in the designated domain can be screened out from the training corpus set of the general domain based on the first target value corresponding to the training corpus in the training corpus set of the general domain. On the basis, the application also provides a machine translation method, which can utilize a pre-established domain translation model to realize translation of the text in the appointed domain, and because the domain translation model is obtained by fine tuning a general translation model by adopting a large number of training corpuses in the appointed domain, the method is a translation model which can be suitable for the appointed domain, and the text in the appointed domain is translated by utilizing the translation model, so that a relatively accurate translation result can be obtained.

Description

A domain data acquisition method, machine translation method and related equipment

Technical field

This application relates to the technical field of natural language processing, and in particular to a field data acquisition method, machine translation method and related equipment.

Background technique

Language communication has become an important issue for mutual communication between different language and racial groups. It is a dream pursued by mankind to realize barrier-free and free communication at any time, in any place and in any language. The traditional language service industry uses manual accompanying interpretation, consecutive interpretation, and simultaneous interpretation to solve the problem of language communication barriers. However, due to insufficient manpower and cost constraints, it cannot meet the needs of ordinary people for communication in different languages.

Machine translation is the process of using computers to convert one natural language (source language) into another natural language (target language). Machine translation can significantly save translation time, improve translation efficiency, meet the translation needs of information with high timeliness requirements or massive texts, greatly reduce labor costs, and more importantly, it makes cross-language communication accessible to everyone. Everyone can have the ability, and language barriers are no longer obstacles for people to obtain information and services.

At some point, there will be some translation tasks in specific fields. However, the current machine translation methods are mostly machine translation methods based on general domain translation models. When using this machine translation method to translate texts in specific fields, the translation is accurate. The degree is not high. For this reason, it is necessary to build a translation model for specific fields. It is understandable that to build a translation model in a specific field, training corpus in a specific field is often required. However, in some specific fields, it is difficult to collect training corpus, which results in a small amount of training corpus in a specific field, and in certain fields, it is difficult to collect training corpus. Insufficient amount of training corpus in the field will make it difficult to build a domain-specific translation model with better performance. For this reason, a method of obtaining training corpus in a specific field is urgently needed.

Contents of the invention

In view of this, this application provides a field data acquisition method, a machine translation method and related equipment to filter out training corpus in a specified field from a collection of training corpus in a general field, so as to use the training corpus in the specified field to construct an accurate translation A highly accurate domain translation model can accurately translate texts in specified fields. The technical solution is as follows:

A method for obtaining domain data, including:

Obtain the training corpus in the general field and the initial training corpus in the specified field;

Using the training corpus in the general field to establish a general translation model;

Based on the general translation model and the initial training corpus of the designated field, a first target value corresponding to the training corpus in the training corpus of the general field is determined, wherein a piece of training corpus corresponds to a first target value, and the first target value is determined. A target value can represent the degree of matching between the corresponding training corpus and the specified field;

Based on the first target value corresponding to the training corpus in the training corpus in the general field, the training corpus in the specified field is screened from the training corpus in the general field.

Optionally, the domain data acquisition method also includes:

Use the training corpus in the general field to establish a general language model, and use the initial training corpus in the specified field to establish a domain language model;

Based on the general language model and the domain language model, a second target value corresponding to the training corpus in the training corpus set in the general domain is determined, wherein a piece of training corpus corresponds to a second target value, and the second target value can Characterize the degree of relevance of the corresponding training corpus to the designated field;

Based on the first target value corresponding to the training corpus in the training corpus in the general field, screening the training corpus in the specified field from the training corpus in the general field includes:

Based on the first target value and the second target value corresponding to the training corpus in the training corpus in the general field, the training corpus in the specified field is screened out from the training corpus in the general field.

Optionally, determining the first target value corresponding to the training corpus in the training corpus of the general field includes:

Use the training corpus in the general field to establish a general language model, and use the initial training corpus in the specified field to establish a domain language model;

Based on the general language model and the domain language model, a second target value corresponding to the training corpus in the training corpus set in the general domain is determined, wherein a piece of training corpus corresponds to a second target value, and the second target value can Characterize the degree of relevance of the corresponding training corpus to the designated field;

Based on the second target value corresponding to the training corpus in the training corpus in the general field, select candidate training corpus from the training corpus in the general field;

Determine the first target value corresponding to each candidate training corpus screened out;

Based on the first target value corresponding to the training corpus in the training corpus in the general field, screening the training corpus in the specified field from the training corpus in the general field includes:

Based on the first target value corresponding to each selected candidate training corpus, the training corpus in the designated field is selected from the selected candidate training corpus.

Optionally, based on the general translation model and the initial training corpus in the specified field, determine the first target value corresponding to a target training corpus in the training corpus in the general field, including:

Determine the gradient of each training corpus in the initial training corpus in the specified field on the general translation model;

Calculate the average value of the gradient of each training corpus in the initial training corpus set in the specified field on the general translation model as the average field gradient;

Determine the gradient of the target training corpus on the general translation model;

The distance between the gradient of the target training corpus on the general translation model and the average value of the field gradient is calculated as the first target value corresponding to the target training corpus.

Optionally, each piece of training corpus in the training corpus in the general field and each piece of training corpus in the initial training corpus in the specified field include: source language text and corresponding target language text;

The use of the training corpus in the general field to establish a general language model, and the use of the initial training corpus in the specified field to establish a domain language model include:

Utilize the source language text in the training corpus set in the general field to train the language model, and the trained language model is used as the source language end universal language model;

Utilize the target language text in the training corpus in the general field to train the language model, and the language model obtained by training is used as the target language side general language model;

Use the source language text in the initial training corpus of the specified field to train the language model, and the trained language model is used as the source language side domain language model;

The language model is trained using the target language text in the initial training corpus of the specified field, and the trained language model is used as the target language end domain language model.

Optionally, based on the general language model and the domain language model, determine the second target value corresponding to a target training corpus in the training corpus of the general domain, including:

Calculate the posterior probabilities of the target training corpus on the general language model and the domain language model respectively;

According to the determined posterior probability, the second target value corresponding to the target training corpus is determined.

Optionally, calculating the posterior probabilities of the target training corpus on the general language model and the domain language model respectively includes:

Calculate the posterior probabilities of the target training corpus on the source language side general language model, the source language side domain language model, the target language side general language model, and the target language side domain language model;

Determining the second target value corresponding to the target training corpus according to the determined posterior probability includes:

According to the posterior probabilities of the target training corpus on the source language side universal language model and the posterior probabilities on the source language side domain language model, the probability of the target training corpus on the source language side language model is determined. Score;

According to the posterior probabilities of the target training corpus on the target language side general language model and the posterior probabilities of the target language side domain language model, the probability of the target training corpus on the target language side language model is determined. Score;

The score of the target training corpus on the source language side language model and the score of the target training corpus on the target language side language model are used as the second target value corresponding to the target training corpus.

A machine translation method for a specified domain, including:

Obtain the source language text to be translated in the specified field;

Input the source language text to be translated into a pre-established domain translation model to obtain the target language text corresponding to the source language text to be translated;

Wherein, the domain translation model adopts the initial training corpus of the specified domain, and the training corpus obtained from the training corpus of the general domain using the domain data acquisition method described in any of the above, and is obtained by adjusting the general translation model. The general translation model is trained using the training corpus in the general field.

Optionally, using the initial training corpus in the designated field and the training corpus selected from the training corpus in the general field, the process of adjusting the general translation model includes:

Mix the training corpus from the initial training corpus set in the specified field with the training corpus screened out from the training corpus set in the general field;

The general translation model is adjusted using the mixed training corpus.

Optionally, using the initial training corpus in the designated field and the training corpus selected from the training corpus in the general field, the process of adjusting the general translation model also includes:

The translation model adjusted using the mixed training corpus is further adjusted using the training corpus in the initial training corpus set in the specified field.

A domain data acquisition device, including: a data acquisition module, a universal translation model establishment module, a first target value determination module and a data screening module;

The data acquisition module is used to obtain training corpus in general fields and initial training corpus in specified fields;

The universal translation model establishment module is used to establish a universal translation model using the training corpus in the general field;

The first target value determination module is used to determine the first target value corresponding to the training corpus in the training corpus in the general field based on the general translation model and the initial training corpus in the specified field, wherein a training The corpus corresponds to a first target value, and the first target value can represent the degree of matching between the corresponding training corpus and the designated field;

The data screening module is configured to filter out the training corpus in the specified field from the training corpus in the general field based on the first target value corresponding to the training corpus in the general field training corpus.

Optionally, the first target value determination module determines the first target value corresponding to a target training corpus in the training corpus in the general field based on the general translation model and the initial training corpus in the specified field. , specifically used to determine the gradient of each training corpus in the initial training corpus in the designated field on the universal translation model, and calculate the gradient of each training corpus in the initial training corpus in the designated field on the universal translation model. The average value of the gradient, as the domain gradient average, determines the gradient of the target training corpus on the universal translation model, and calculates the difference between the gradient of the target training corpus on the universal translation model and the domain gradient average The distance is used as the second target value corresponding to the target training corpus.

A machine translation device for a designated field, including: a source language text acquisition module and a translation module;

The source language text acquisition module is used to obtain the source language text to be translated in a specified field;

The translation module is used to input the source language text to be translated into a pre-established domain translation model to obtain the target language text corresponding to the source language text to be translated;

Wherein, the domain translation model adopts the initial training corpus of the designated domain and the training corpus selected from the training corpus of the general domain by the domain data acquisition device described in any of the above, and is obtained by adjusting the general translation model. The general translation model is trained using the training corpus in the general field.

A domain data screening device includes: a memory and a processor;

The memory is used to store programs;

The processor is used to execute the program and implement each step of the domain data acquisition method described in any one of the above.

A readable storage medium with a computer program stored thereon, characterized in that when the computer program is executed by a processor, each step of the domain data acquisition method described in any one of the above is implemented.

It can be seen from the above solution that the domain data acquisition method provided by this application first obtains a training corpus in a general field and an initial training corpus in a specified field, then uses the training corpus in the general field to establish a universal translation model, and then based on the universal translation model and The initial training corpus in the specified field determines the first target value corresponding to the training corpus in the general field training corpus. Finally, based on the first target value corresponding to the training corpus in the general field training corpus, the designated training corpus is filtered out from the general field training corpus. training corpus in the field. Through the domain data acquisition method provided by this application, training corpus in a specified field can be obtained from a collection of training corpus in general fields. On this basis, this application also provides a machine translation method that can utilize a pre-established domain translation model. To realize the translation of text in a specified field, the domain translation model is obtained by fine-tuning a general translation model using a large amount of training corpus in the specified field. Therefore, it is a translation model that can be adapted to the specified field. This translation model is used to translate texts in the specified field. , can obtain more accurate translation results.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only These are embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without exerting creative efforts.

Figure 1 is a schematic flow chart of a domain data acquisition method provided by an embodiment of the present application;

Figure 2 is a schematic flow chart of another domain data acquisition method provided by an embodiment of the present application;

Figure 3 is a schematic flow chart of yet another domain data acquisition method provided by an embodiment of the present application;

Figure 4 is a schematic flow chart of adjusting a general translation model using an initial training corpus in a specified field and training corpus selected from a training corpus in a general field provided by the embodiment of the present application;

Figure 5 is a schematic diagram of preventing model deviation through KL regular constraints provided by the embodiment of the present application;

Figure 6 is a schematic structural diagram of a domain data acquisition device provided by an embodiment of the present application;

Figure 7 is a schematic structural diagram of a domain data acquisition device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present invention.

In view of the difficulty in collecting training corpus in a specific field, in order to obtain enough training corpus, the inventor of this case thought that the training corpus in a specific field can be screened out from the training corpus in a general field, so that the selected training corpus can be compared with the training corpus from a specific field. The training corpus collected in the domain is used as the training corpus for building the domain translation model.

In order to filter out the training corpus in a specific field from the training corpus in the general field, the inventor of this case conducted in-depth research and finally proposed a solution. The general idea of the solution is: determine the training corpus in the general field and concentrate the training corpus with the specific field. The degree of matching in the field is based on the matching degree between the training corpus in the general field and the training corpus in the specific field. The training corpus in the specific field is screened from the training corpus in the general field.

On the basis of the above-mentioned domain data acquisition scheme, the inventor of this case also provides a machine translation method for specific fields. The general idea of this method is to use the training corpus of general fields to build a general-field translation model, and then use it to The collected training corpus and the training corpus selected from the training corpus in the general field are fine-tuned to the general field translation model to obtain a translation model in a specific field. By inputting the source language text in the specific field into the translation model in the specific field, you can get Corresponding target language text.

The domain data acquisition method and machine translation method provided by this application can be applied to terminals with data processing capabilities, and can also be applied to a single server or a server cluster composed of multiple servers. Next, the domain data acquisition method and machine translation method provided by this application will be introduced through the following embodiments.

First embodiment

Please refer to Figure 1, which shows a schematic flowchart of a domain data acquisition method provided by an embodiment of the present application. The method may include:

Step S101: Obtain a training corpus in a general field and an initial training corpus in a specified field.

Among them, each training corpus in the general domain training corpus and each training corpus in the initial training corpus in a specified field includes source language text and its corresponding target language text, that is, each training corpus is a text pair.

It should be noted that the training corpus in the general field includes training corpus from multiple fields mixed together, and the training corpus in the initial training corpus in the specified field is the training corpus collected directly in the specified field. In this embodiment, the training corpus in the general field includes training corpus in the specified field. What this embodiment aims to achieve is to filter out the training corpus in the specified field from the training corpus in the general field, and then combine the filtered training corpus with The training corpus in the initial training corpus of the specified field constitutes the training corpus of the specified field.

Step S102: Establish a general translation model using training corpus in general fields.

Specifically, the translation model is trained using the training corpus in the general domain training corpus, and the trained translation model is the universal translation model. The process of training a translation model using training corpus set in a general domain training corpus is an existing technology, and will not be described in detail in this embodiment.

Step S103: Based on the general translation model and the initial training corpus in the specified field, determine the first target value corresponding to each training corpus in the training corpus in the general field.

Among them, the first target value can represent the degree of matching between the corresponding training corpus and the specified field.

It should be noted that the higher the matching degree between a training corpus and the specified field, the greater the possibility that the training corpus is the training corpus in the specified field. On the contrary, the lower the matching degree between a training corpus and the specified field, it means that the training corpus is training corpus in the specified field. The less likely it is that the training corpus is training corpus in the specified field.

Considering that in the field of deep learning/machine learning, the reverse update gradients of training corpus with similar characteristics or characteristics are often consistent, and training corpus in the same field must have similar characteristics, that is, those belonging to the same field The reverse gradient of the training corpus on the model is consistent. Based on this, the present invention thinks that the gradient distance can be used to measure the matching degree of the training corpus and the specified field.

In view of this, based on the general translation model and the initial training corpus in the specified field, a target training corpus (this application refers to the training corpus with a target value to be determined as the "target training corpus") corresponding to the training corpus in the general field is determined. The process for the first target value may include:

Step a1: Determine the gradient of each training corpus in the initial training corpus in the specified field on the general translation model.

Specifically, for each piece of training corpus in the initial training corpus in the specified field, input it into the universal translation model, and calculate its gradient in the universal translation model. Gradient/> The calculation formula is as follows:

in, Represents the gradient operator, θ refers to the model parameters in the universal translation model. It should be noted that θ can be a parameter of the entire universal translation model, or some of the parameters. Optionally, θ can be a universal translation model. The last layer of model parameters.

Step a2: Calculate the average of the gradients of each training corpus in the initial training corpus in the specified field on the general translation model as the average field gradient

Step a3: Determine the gradient of the target training corpus on the general translation model.

The gradient of the target training corpus on the general translation model is determined in the same way as the gradient of the initial training corpus in the specified field on the general translation model.

Step a4: Calculate the gradient of the target training corpus on the general translation model and the average domain gradient The distance is used as the first target value corresponding to the target training corpus.

That is, the first target value S _d corresponding to the target training corpus is:

Among them, dist(.,.) refers to calculating the distance between two tensors. The distance between two tensors can be, but is not limited to, cosine distance.

Step S104: Based on the first target value corresponding to each training corpus in the training corpus in the general field, select the training corpus in the specified field from the training corpus in the general field.

Based on the first target value corresponding to each training corpus in the training corpus in the general field, there are many ways to filter out the training corpus in the specified field from the training corpus in the general field: In one possible implementation, Sort the training corpus in the general domain training corpus from high to low in order of the first target value, and take the first N training corpus as the training corpus in the specified field. Of course, the first target value can also be sorted from low to high. Sort each training corpus in the training corpus set in the general field in the order of In a possible implementation, a threshold T can be set, and the training corpus with a first target value greater than the threshold T can be selected from the training corpus in the general field as the training corpus in the specified field.

The domain data acquisition method provided by the embodiment of the present application can determine the first target value corresponding to each training corpus in the general domain training corpus based on the general translation model and the initial training corpus in the specified field, because the first target value can characterize Corresponds to the matching degree of the training corpus with the specified field. Therefore, based on the first target value corresponding to each training corpus in the general field training corpus, it is possible to filter out the training corpus with a higher degree of matching with the specified field from the general field training corpus set. . The domain data acquisition method provided by the embodiments of the present application can more accurately filter out the training corpus in the specified field from the training corpus in the general field. In addition, the embodiments of the present application screen the training corpus from the model gradient level, which makes subsequent use When the selected training corpus is used to build a translation model in a specified field, the training corpus can be directly coupled with the model, which can effectively improve the domain translation capabilities of the translation model in the specified field.

Second embodiment

In order to improve the screening accuracy of training corpus, this embodiment provides another domain data acquisition method. Please refer to Figure 2, which shows a schematic flow chart of the domain data acquisition method. The method may include:

Step S201: Obtain a training corpus in a general field and an initial training corpus in a specified field.

Among them, the training corpus in a general field includes training corpus in multiple fields mixed together, and the initial training corpus in a specified field is training corpus collected directly in the specified field.

Step S202a: Establish a general translation model using training corpus in general fields.

Specifically, the translation model is trained using the training corpus in the general domain training corpus, and the trained translation model is the universal translation model.

Step S202b: Use the training corpus in the general field to establish a general language model, and use the initial training corpus in the specified field to establish a domain language model.

Among them, the process of establishing a general language model using a training corpus in a general field may include: using the source language text in the training corpus in the general field to train the language model, and the trained language model serves as a source language side general language model; using the source language text in the general field The target language text in the training corpus is used to train the language model, and the trained language model is used as a general language model on the target language side. That is, the universal language model in this embodiment includes a source language side universal language model and a target language side universal language model.

Among them, the process of establishing a domain language model using the initial training corpus of the specified field may include: using the source language text in the initial training corpus of the specified field to train the language model, and the trained language model is used as the source language side domain language model; using the specified The target language text in the domain's initial training corpus is used to train the language model, and the trained language model is used as the target language side domain language model. That is, the domain language model in this embodiment includes a source language side domain language model and a target language side domain language model.

Step S203a: Based on the general translation model and the initial training corpus in the specified field, determine the first target value corresponding to each training corpus in the training corpus in the general field.

Among them, the first target value can represent the degree of matching between the corresponding training corpus and the specified field.

The process of determining the first target value corresponding to each piece of training corpus in the general field training corpus set can be referred to steps a1 to a4 in the first embodiment, which will not be described in detail in this embodiment.

Step S203b: Based on the general language model and the domain language model, determine the second target value corresponding to each training corpus in the general domain training corpus.

Among them, the second target value can represent the degree of correlation between the corresponding training corpus and the specified field.

It should be noted that the higher the correlation between a training corpus and the specified field, the greater the possibility that the training corpus is the training corpus in the specified field. On the contrary, the lower the correlation between a training corpus and the specified field, it means that the training corpus is training corpus in the specified field. The less likely it is that the training corpus is training corpus in the specified field.

Specifically, based on the general language model and the domain language model, the process of determining the training corpus in the general domain and concentrating the second target value corresponding to one target training corpus may include:

Step b1: Calculate the posterior probabilities of the target training corpus on the general language model and domain language model respectively.

Specifically, the posterior probabilities of the target training corpus on the source language side general language model, the source language side domain language model, the target language side general language model, and the target language side domain language model are calculated respectively.

Step b2: Determine the second target value corresponding to the target training corpus according to the determined posterior probability.

Specifically, based on the determined posterior probability, the process of determining the second target value corresponding to the target training corpus includes:

According to the posterior probability of the target training corpus on the source language side general language model and the posterior probability on the source language side domain language model, the score of the target training corpus on the source language side language model is determined; according to the target training corpus, respectively The posterior probability on the general language model on the target language side and the posterior probability on the domain language model on the target language side are used to determine the score of the target training corpus on the target language side language model; the score of the target training corpus on the source language side language model is determined. The score and the score of the target training corpus on the target language side language model are used as the second target value corresponding to the target training corpus.

More specifically, it is assumed that the posterior probability of the target training corpus on the source language side universal language model is The posterior probability of the target training corpus on the source language domain language model is/> Then the score of the target training corpus on the source language side language model/> for:

Assume that the posterior probability of the target training corpus on the target language side universal language model is The posterior probability of the target training corpus on the target language domain language model is/> Then the score of the target training corpus on the target language side language model/> for:

Among them, x is the source language text in the target training corpus, and y is the target language text in the target training corpus.

Step S204: Based on the first target value and the second target value corresponding to each training corpus in the general field training corpus set, select the training corpus in the specified field from the general field training corpus set.

In a possible implementation, for each training corpus in the general field training corpus: the first target value corresponding to the training corpus can be used as the score of the first dimension, and the training corpus can be used in the source language The score on the model is used as the score of the second dimension, the score of the training corpus on the target language side language model is used as the score of the third dimension, the scores of these three dimensions are fused, and the fused score is used as the training corpus The corresponding target score, after obtaining the target score corresponding to each training corpus in the general field training corpus set, is based on the target score corresponding to each training corpus in the general field training corpus set, from the general field training corpus set Filter out the training corpus in the specified field. Specifically, the N training corpus with the highest target score can be selected from the training corpus in the general field as the training corpus in the specified field.

Assume that the first target value corresponding to a piece of training corpus x is S _d , and the score of this piece of training corpus x on the source language side language model is The score of this training corpus x on the target language side language model is/> In a possible implementation, S _d ,/> can be directly and/> Sum, and the value obtained by summing is used as the target score corresponding to the training corpus x. In another possible implementation, S _d ,/> can be predetermined and/> The corresponding weights α, β and γ respectively, according to the weights α, β and γ, S _d , /> and/> Weighted summation, the value S obtained by the weighted summation is used as the target score corresponding to the training corpus x, that is:

The domain data acquisition method provided by the embodiments of the present application can determine the first target value corresponding to each training corpus in the general domain training corpus based on the general translation model and the initial training corpus in the specified field. It can also be based on the general language model and The domain language model determines the second target value corresponding to each training corpus in the general domain training corpus. Since the first target value can characterize the matching degree of the corresponding training corpus with the specified domain, the second target value can characterize the matching degree of the corresponding training corpus with the specified domain. The degree of relevance of the field. Therefore, based on the first target value and the second target value corresponding to each training corpus in the general field training corpus, the training corpus in the specified field can be more accurately screened out from the general field training corpus. .

Third embodiment

In order to improve the screening efficiency and screening accuracy of training corpus, this embodiment provides yet another domain data acquisition method. Please refer to Figure 3, which shows a schematic flow chart of the domain data acquisition method. The method may include:

Step S301: Obtain a training corpus in a general field and an initial training corpus in a specified field.

Among them, the training corpus in a general field includes training corpus in multiple fields mixed together, and the initial training corpus in a specified field is training corpus collected directly in the specified field.

Step S302: Use the training corpus in the general field to establish a general language model, and use the initial training corpus in the specified field to establish a domain language model.

Step S303: Based on the general language model and the domain language model, determine the second target value corresponding to each training corpus in the general domain training corpus.

Among them, the second target value can represent the degree of correlation between the corresponding training corpus and the specified field.

For the specific implementation process of this step, please refer to the specific implementation process of "Step S203b: Based on the general language model and the domain language model, determine the second target value corresponding to each training corpus in the general domain training corpus set" in the second embodiment. The embodiments will not be described in detail here.

Step S304: Based on the second target value corresponding to the training corpus in the training corpus in the general field, select candidate training corpus from the training corpus in the general field to form a candidate training corpus.

In a possible implementation, each training corpus in the training corpus in the general field can be sorted according to the order of the second target value from large to small, and the top M training corpus is taken as the candidate training corpus to form the candidate training corpus. set (of course, you can also sort the training corpus in the general field training corpus in ascending order of the second target value, and take the last M training corpus as candidate training corpus to form a candidate training corpus); in another In one possible implementation, a threshold T1 can be set, and the training corpus with a second target value greater than the threshold T1 in the general field training corpus is used as candidate training corpus to form a candidate training corpus set.

Step S305: Establish a general translation model using training corpus in general fields.

Specifically, the translation model is trained using the training corpus in the general domain training corpus, and the trained translation model is the universal translation model.

It should be noted that this embodiment does not limit step S305 to be executed after step S304, as long as step S305 is executed after step S301 and before step S306.

Step S306: Based on the universal translation model and the initial training corpus in the specified field, determine the first target value corresponding to each training corpus in the candidate training corpus.

Specifically, first determine the gradient of each training corpus on the universal translation model in the initial training corpus of the specified field, and then calculate the average of the gradient of each training corpus on the universal translation model in the initial training corpus of the specified field, as the domain Gradient average, then, determine the gradient of each training corpus in the candidate training corpus on the general translation model, and finally, for each training corpus in the candidate training corpus, calculate its gradient on the general translation model and the domain gradient average The distance between the values is used as its corresponding first target value, thereby obtaining the first target value corresponding to each training corpus in the candidate training corpus set. The process of determining the gradient of a piece of training corpus on the universal translation model can be found in the first embodiment, and will not be described again in this embodiment.

Step S307: Based on the first target value corresponding to each training corpus in the candidate training corpus set, select the training corpus in the specified field from the candidate training corpus set.

Optionally, you can sort each training corpus in the candidate training corpus set in descending order of the first target value, and take the top N (the size of N can be set according to the specific application) training corpus as the designated field. training corpus. Of course, you can also sort the training corpus in the candidate training corpus in ascending order of the first target value, and take the last N training corpus as the training corpus in the specified field. You can also set a threshold T2 , select the training corpus whose first target value is greater than the threshold T2 from the candidate training corpus set as the training corpus in the specified field.

Considering that the calculation of the gradient is relatively time-consuming, in order to improve the screening efficiency, the domain data acquisition method provided in this embodiment is first based on the second target value (language model score) corresponding to each training corpus in the general domain training corpus, from the general domain Candidate training corpus is screened from the training corpus set, and then based on the first target value (gradient distance) corresponding to each candidate training corpus in the candidate training corpus set, training corpus in the specified field is screened out from the candidate training corpus set. The domain data acquisition method provided by this embodiment can not only accurately filter out the training corpus in the specified field from the training corpus in the general domain, but also, because it only needs to calculate the gradient for the candidate training corpus, there is no need to calculate the gradient for all the training corpus. , Therefore, the amount of calculation is reduced, thereby improving the efficiency of screening training corpus.

Fourth embodiment

Based on the above embodiments, this embodiment provides a machine translation method for a specified field. The method may include:

Obtain the source language text to be translated in the specified field; input the source language text to be translated into the pre-established domain translation model to obtain the target language text corresponding to the source language text to be translated.

Among them, the domain translation model adopts the initial training corpus of the specified domain and the training corpus obtained from the training corpus of the general domain using the domain data acquisition method provided by any of the above embodiments. The general translation model is obtained by adjusting the general translation model. It is trained using training corpus in general fields.

Please refer to Figure 4, which shows a schematic flow chart of adjusting the general translation model using the initial training corpus in the specified field and the training corpus filtered out from the training corpus in the general field, which can include:

Step S401: Mix the training corpus from the initial training corpus in the specified field with the training corpus filtered out from the training corpus in the general field.

Step S402: Use the mixed training corpus to fine-tune the general translation model, and the fine-tuned model is used as the domain translation model T _in1 .

Use the mixed training corpus to fine-tune the general translation model, that is, use the mixed training corpus to further train the general translation model. The translation model obtained after training is the domain translation model T _in1 . This domain translation model T _in1 can compare Accurately translate the text to be translated in the specified field.

Preferably, in order to obtain a domain translation model with better performance, this embodiment may also include the following steps:

Step S403: Fine-tune the domain translation model T _in1 using the training corpus in the initial training corpus set of the specified domain, and the fine-tuned model is used as the final domain translation model T _in .

After the domain translation model T _in1 is obtained through step S402, the domain translation model T _in1 is fine-tuned more precisely using high-quality training corpus in the specified domain. Use high-quality training corpus in the specified field to fine-tune the domain translation model T _in1 , that is, use the training corpus in the initial training corpus of the specified field to further train the domain translation model T _in1 .

Considering that there is a small amount of high-quality training corpus, this embodiment adds a KL regular constraint when fine-tuning the domain translation model T _in1 to prevent the model from going astray.

Specifically, the strategy to prevent model deviation through KL regular constraints is: after obtaining T _in1 , add a T _in1 , as shown in Figure 5, one of the parameters of T _in1 is fixed (T _in1 - in Figure 5 fixed), another T _in1 is adjusted, that is, the parameters are adjusted (T _in in Figure 5). The high-quality training corpus is input into T _in and T _in1 -fixed in Figure 5 respectively, and T _in outputs the probability distribution P(y |x), _{T in1 -fixed} output probability distribution Q (y |

The proximity of two probability distributions can be measured by relative entropy. Relative entropy is also called KL divergence or information divergence. It is an asymmetry measure of the difference between two probability distributions. Relative entropy can measure the difference between two probability distributions. When the two probability distributions are the same, their relative entropy is zero. When the difference between the two probability distributions increases, their relative entropy will also increase. Specifically, the relative entropy of the probability distribution P and the probability distribution Q Entropy or KL divergence can be calculated by:

After the training converges, the final model T _in is the translation model in the specified field.

In the machine translation method for the designated field provided in this embodiment, since the domain translation model of the designated field is obtained by fine-tuning the general translation model using a large amount of training corpus in the designated field, it is a translation model that can be adapted to the designated field. This translation model translates texts in specified fields and can obtain accurate translation results.

It should be noted that the domain data screening method provided in the above embodiments can be used to obtain training corpus in multiple different fields, and further training corpus in multiple different fields can be used (each domain can include filtered training corpus and collected training corpus. corpus) to fine-tune the general translation model to obtain domain translation models in multiple different fields, thereby achieving accurate translation of texts to be translated in multiple different fields.

Fifth embodiment

The embodiment of the present application also provides a domain data acquisition device. The domain data acquisition device provided by the embodiment of the present application is described below. The domain data acquisition device described below and the domain data acquisition method described above can be mutually referenced.

Please refer to Figure 6, which shows a schematic structural diagram of a domain data acquisition device provided by an embodiment of the present application, which may include: a data acquisition module 601, a general translation model establishment module 602, a first target value determination module 603, and a data filtering module 604.

Data acquisition module 601, used to obtain training corpus in general fields and initial training corpus in specified fields;

Universal translation model building module 602, used to establish a universal translation model using the training corpus in the general field;

The first target value determination module 603 is used to determine the first target value corresponding to the training corpus in the training corpus set in the general field based on the general translation model and the initial training corpus set in the designated field, wherein a training corpus Corresponding to a first target value, the first target value can represent the degree of matching between the corresponding training corpus and the designated field;

The data screening module 604 is configured to filter out the training corpus in the specified field from the training corpus in the general field based on the first target value corresponding to the training corpus in the training corpus in the general field.

Optionally, the domain data acquisition device provided by the embodiment of the present application may also include: a general language model establishment module, a domain language model establishment module, and a second target value determination module.

A general language model building module is used to build a general language model using the training corpus in the general field.

A domain language model establishment module is used to establish a domain language model using the initial training corpus of the designated domain.

The second target value determination module is configured to determine the second target value corresponding to the training corpus in the training corpus of the general domain based on the general language model and the domain language model. Among them, a piece of training corpus corresponds to a second target value, and the second target value can represent the degree of correlation between the corresponding training corpus and the designated field;

The data screening module 604 is specifically used to filter out the training in the specified field from the training corpus in the general field based on the first target value and the second target value corresponding to the training corpus in the training corpus in the general field. corpus.

Optionally, the first target value determination module 603 includes: a general language model establishment module, a domain language model establishment module, a second target value determination module, a candidate training corpus screening submodule, and a candidate corpus first target value determination module.

A general language model building module is used to build a general language model using the training corpus in the general field.

A domain language model establishment module is used to establish a domain language model using the initial training corpus of the designated domain.

The second target value determination module is configured to determine the second target value corresponding to the training corpus in the training corpus of the general domain based on the general language model and the domain language model. Among them, a piece of training corpus corresponds to a second target value, and the second target value can represent the degree of correlation between the corresponding training corpus and the designated field;

The candidate training corpus screening sub-module is used to screen candidate training corpus from the training corpus in the general field based on the second target value corresponding to the training corpus in the training corpus in the general field.

The first target value determination module of the candidate corpus is used to determine the first target value corresponding to each filtered candidate training corpus.

The data screening module 604 is specifically used to screen out the training corpus in the specified field from the screened candidate training corpus based on the first target value corresponding to each of the screened candidate training corpus.

Optionally, when the first target value determination module 603 determines the first target value corresponding to a target training corpus in the training corpus in the general field based on the general translation model and the initial training corpus in the specified field, Specifically used for:

Determine the gradient of each training corpus in the initial training corpus in the specified field on the universal translation model; calculate the average value of the gradient of each training corpus in the initial training corpus in the designated field on the universal translation model , as the domain gradient average; determine the gradient of the target training corpus on the universal translation model; calculate the distance between the gradient of the target training corpus on the universal translation model and the domain gradient average, as the The first target value corresponding to the target training corpus.

Each piece of training corpus in the training corpus in the general field and each piece of training corpus in the initial training corpus in the specified field include: a source language text and a corresponding target language text.

When the general language model establishment module uses the training corpus in the general field to establish a general language model, and uses the initial training corpus in the specified field to establish a domain language model, it is specifically used to:

Use the source language text in the training corpus in the general field to train the language model, and the trained language model is used as the source language end universal language model; use the target language text in the training corpus in the general field to train the language model, and the trained language model The language model serves as a general language model on the target language side.

When the domain language model establishment module uses the initial training corpus of the designated domain to establish a domain language model, it is specifically used to:

Use the source language text in the initial training corpus of the designated field to train the language model, and the trained language model is used as the source language side domain language model; use the target language text in the initial training corpus of the designated field to train the language model, and train The obtained language model is used as the target language side domain language model.

When the second target value determination module determines the second target value corresponding to a target training corpus in the training corpus of the general domain based on the general language model and the domain language model, it is specifically used to:

Calculate the posterior probabilities of the target training corpus on the general language model and the domain language model respectively; determine the second target value corresponding to the target training corpus based on the determined posterior probabilities.

When calculating the posterior probabilities of the target training corpus on the general language model and the domain language model respectively, the second target value determination module is specifically used to:

Calculate the posterior probabilities of the target training corpus on the source language side general language model, the source language side domain language model, the target language side general language model, and the target language side domain language model.

When the second target value determination module determines the second target value corresponding to the target training corpus according to the determined posterior probability, it is specifically used to:

According to the posterior probabilities of the target training corpus on the source language side universal language model and the posterior probabilities on the source language side domain language model, the probability of the target training corpus on the source language side language model is determined. Score; according to the posterior probability of the target training corpus on the target language side universal language model and the posterior probability on the target language side domain language model, determine the target training corpus on the target language side language model The score of the target training corpus on the source language side language model and the score of the target training corpus on the target language side language model are used as the second target value corresponding to the target training corpus.

The domain data acquisition device provided in this embodiment can more accurately filter out the training corpus in the specified field from the training corpus in the general domain.

Sixth embodiment

This embodiment provides a machine translation device for a designated field. The machine translation device provided by this embodiment will be described below. The machine translation device described below and the machine translation method described above may be mutually referenced.

The machine translation device for a specified field provided in this embodiment may include: a source language text acquisition module and a translation module.

The source language text acquisition module is used to obtain the source language text to be translated in the specified field.

A translation module is used to input the source language text to be translated into a pre-established domain translation model to obtain the target language text corresponding to the source language text to be translated.

Wherein, the domain translation model adopts the initial training corpus of the specified domain and the training corpus obtained from the training corpus of the general domain using the domain data acquisition device provided in the above embodiment, and is obtained by adjusting the general translation model. The general translation model The model is trained using the training corpus in the general field.

The machine translation device for a specified domain provided in this embodiment may also include: a domain translation model building module.

The domain translation model building module may include a first adjustment module.

The first adjustment module is used to mix the training corpus from the initial training corpus set in the specified field with the training corpus selected from the training corpus set in the general field, and use the mixed training corpus to adjust the general translation model.

Optionally, the domain translation model building module may also include a second adjustment module.

The second adjustment module is configured to further adjust the translation model adjusted using the mixed training corpus using the training corpus in the initial training corpus set in the specified field.

Since the domain translation model used by the machine translation device provided in this embodiment is obtained by fine-tuning a general translation model using a large amount of training corpus in the specified domain, it is a translation model that can be adapted to the specified domain, and the translation model is used to perform translation in the specified domain. Translate the text to get accurate translation results.

Seventh embodiment

The embodiment of the present application also provides a field data acquisition device. Please refer to Figure 7, which shows a schematic structural diagram of the field data acquisition device. The field data acquisition device may include: at least one processor 701, at least one communication interface 702 , at least one memory 703 and at least one communication bus 704;

In the embodiment of the present application, the number of the processor 701, the communication interface 702, the memory 703, and the communication bus 704 is at least one, and the processor 701, the communication interface 702, and the memory 703 complete communication with each other through the communication bus 704;

The processor 701 may be a central processing unit CPU, or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, etc.;

Memory 703 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), etc., such as at least one disk memory;

Wherein, the memory stores a program, and the processor can call the program stored in the memory, and the program is used for:

Obtain the training corpus in the general field and the initial training corpus in the specified field;

Using the training corpus in the general field to establish a general translation model;

Based on the general translation model and the initial training corpus of the designated field, a first target value corresponding to the training corpus in the training corpus of the general field is determined, wherein a piece of training corpus corresponds to a first target value, and the first target value is determined. A target value can represent the degree of matching between the corresponding training corpus and the specified field;

Based on the first target value corresponding to the training corpus in the training corpus in the general field, the training corpus in the specified field is screened from the training corpus in the general field.

Optionally, the detailed functions and extended functions of the program may refer to the above description.

Eighth embodiment

The embodiment of the present application also provides a readable storage medium, which can store a program suitable for execution by the processor, and the program is used for:

Obtain the training corpus in the general field and the initial training corpus in the specified field;

Using the training corpus in the general field to establish a general translation model;

Based on the general translation model and the initial training corpus of the designated field, a first target value corresponding to the training corpus in the training corpus of the general field is determined, wherein a piece of training corpus corresponds to a first target value, and the first target value is determined. A target value can represent the degree of matching between the corresponding training corpus and the specified field;

Based on the first target value corresponding to the training corpus in the training corpus in the general field, the training corpus in the specified field is screened from the training corpus in the general field.

Optionally, the detailed functions and extended functions of the program may refer to the above description.

Ninth embodiment

The embodiment of the present application also provides a machine translation device for a specified field. The machine translation device may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus;

In the embodiment of the present application, the number of processors, communication interfaces, memories, and communication buses is at least one, and the processor, communication interface, and memory complete communication with each other through the communication bus;

The processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, etc.;

The memory may include high-speed RAM memory, or may also include non-volatile memory (non-volatile memory), such as at least one disk memory;

Wherein, the memory stores a program, and the processor can call the program stored in the memory, and the program is used for:

Obtain the source language text to be translated in the specified field;

Input the source language text to be translated into a pre-established domain translation model to obtain the target language text corresponding to the source language text to be translated;

Wherein, the domain translation model adopts the initial training corpus of the specified domain, and the training corpus obtained from the training corpus of the general domain using the domain data acquisition method provided by any of the above embodiments, and is obtained by adjusting the general translation model. The general translation model is trained using the training corpus in the general field.

Optionally, the detailed functions and extended functions of the program may refer to the above description.

Tenth embodiment

The embodiment of the present application also provides a readable storage medium, which can store a program suitable for execution by the processor, and the program is used for:

Obtain the source language text to be translated in the specified field;

Input the source language text to be translated into a pre-established domain translation model to obtain the target language text corresponding to the source language text to be translated;

Wherein, the domain translation model adopts the initial training corpus of the specified domain, and the training corpus obtained from the training corpus of the general domain using the domain data acquisition method provided by any of the above embodiments, and is obtained by adjusting the general translation model. The general translation model is trained using the training corpus in the general field.

Optionally, the detailed functions and extended functions of the program may refer to the above description.

Finally, it should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or any such actual relationship or sequence between operations. Furthermore, the terms "comprises," "comprises," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element is defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article or apparatus including the stated element.

With regard to the above description of the disclosed embodiments, the features described in each embodiment in this specification can be replaced or combined with each other, so that those skilled in the art can realize or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be practiced in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (13)

1. A domain data acquisition method, characterized by comprising:

acquiring a training corpus of a general field and an initial training corpus of a designated field;

establishing a general translation model by using the training corpus of the general field;

determining a first target value corresponding to a training corpus in the general field based on the general translation model and the initial training corpus in the specific field, wherein one training corpus corresponds to the first target value, and the first target value can represent the matching degree of the corresponding training corpus and the specific field;

screening the training corpus of the appointed field from the training corpus set of the general field based on a first target value corresponding to the training corpus in the training corpus set of the general field;

wherein determining a first target value corresponding to a target training corpus in the general domain based on the general translation model and the initial training corpus in the specific domain comprises:

determining the gradient of each training corpus in the initial training corpus set in the appointed field on the general translation model;

calculating the average value of gradients of each training corpus on the general translation model in the initial training corpus set in the appointed field, and taking the average value as a field gradient average value;

Determining a gradient of the target training corpus on the general translation model;

and calculating the distance between the gradient of the target training corpus on the general translation model and the domain gradient average value as a first target value corresponding to the target training corpus.

2. The domain data acquisition method according to claim 1, characterized by further comprising:

establishing a general language model by using the training corpus of the general field, and establishing a field language model by using the initial training corpus of the appointed field;

determining a second target value corresponding to a training corpus in a training corpus set of the general field based on the general language model and the field language model, wherein one training corpus corresponds to a second target value, and the second target value can represent the correlation degree between the corresponding training corpus and the specified field;

the screening the training corpus of the specified domain from the training corpus set of the general domain based on a first target value corresponding to the training corpus in the training corpus set of the general domain includes:

and screening the training corpus in the appointed field from the training corpus set in the universal field based on a first target value and a second target value corresponding to the training corpus in the training corpus set in the universal field.

3. The method according to claim 1, wherein determining the first target value corresponding to the training corpus in the training corpus set of the general domain includes:

establishing a general language model by using the training corpus of the general field, and establishing a field language model by using the initial training corpus of the appointed field;

determining a second target value corresponding to a training corpus in a training corpus set of the general field based on the general language model and the field language model, wherein one training corpus corresponds to a second target value, and the second target value can represent the correlation degree between the corresponding training corpus and the specified field;

screening candidate training corpuses from the training corpuses in the general field based on a second target value corresponding to the training corpuses in the general field;

determining a first target value corresponding to each screened candidate training corpus;

the screening the training corpus of the specified domain from the training corpus set of the general domain based on a first target value corresponding to the training corpus in the training corpus set of the general domain includes:

And screening the training corpus in the appointed field from the screened candidate training corpuses according to the first target value corresponding to each screened candidate training corpus.

4. A domain data acquisition method according to claim 2 or 3, wherein each of the training corpora in the training corpus set of the general domain and each of the training corpora in the initial training corpus set of the specified domain include: a source language text and a corresponding target language text;

the building a general language model by using the training corpus of the general domain and building a domain language model by using the initial training corpus of the appointed domain comprises the following steps:

training a language model by using a source language text in a training corpus in the general field, wherein the trained language model is used as a source language end general language model;

training a language model by using a target language text in a training corpus in the general field, wherein the trained language model is used as a general language model of a target language end;

training a language model by using the source language text in the initial training corpus of the appointed field, wherein the trained language model is used as a source language end field language model;

Training a language model by using the target language text in the initial training corpus of the appointed domain, wherein the trained language model is used as a target language end domain language model.

5. A method of acquiring domain data according to claim 2 or 3, wherein determining a second target value corresponding to a target training corpus in the training corpus of the general domain based on the general language model and the domain language model comprises:

calculating posterior probabilities of the target training corpus on the general language model and the domain language model respectively;

and determining a second target value corresponding to the target training corpus according to the determined posterior probability.

6. The method of claim 5, wherein the calculating posterior probabilities of the target training corpus on the generic language model and the domain language model, respectively, comprises:

calculating posterior probabilities of the target training corpus on a source language end general language model, a source language end field language model, a target language end general language model and a target language end field language model respectively;

the determining, according to the determined posterior probability, a second target value corresponding to the target training corpus includes:

Determining the score of the target training corpus on the source language end language model according to the posterior probability of the target training corpus on the source language end general language model and the posterior probability of the target training corpus on the source language end field language model;

determining the score of the target training corpus on the target language end language model according to the posterior probability of the target training corpus on the target language end general language model and the posterior probability of the target language end field language model;

and the score of the target training corpus on the source language end language model and the score of the target training corpus on the target language end language model are used as a second target value corresponding to the target training corpus.

7. A machine translation method for a specified domain, comprising:

acquiring a source language text to be translated in a designated field;

inputting the source language text to be translated into a pre-established domain translation model to obtain a target language text corresponding to the source language text to be translated;

the domain translation model is obtained by adjusting a general translation model by adopting an initial training corpus of a designated domain and a training corpus obtained from a training corpus of a general domain by adopting the domain data obtaining method according to any one of claims 1-6, and the general translation model is obtained by training the training corpus of the general domain.

8. The machine translation method according to claim 7, wherein the process of adjusting the universal translation model using the initial corpus of training for the specified domain and the corpus of training selected from the corpus of training for the universal domain comprises:

mixing the training corpus in the initial training corpus set of the appointed field with the training corpus screened from the training corpus set of the general field;

and adjusting the general translation model by using the mixed training corpus.

9. The machine translation method according to claim 8, wherein the process of adjusting the universal translation model using the initial corpus of training for the specified domain and the corpus of training selected from the corpus of training for the universal domain further comprises:

and adopting the training corpus in the initial training corpus set in the appointed field to further adjust the translation model adjusted by the mixed training corpus.

10. A domain data acquisition apparatus, comprising: the system comprises a data acquisition module, a general translation model establishment module, a first target value determination module and a data screening module;

the data acquisition module is used for acquiring a training corpus in the general field and an initial training corpus in the appointed field;

The universal translation model building module is used for building a universal translation model by utilizing the training corpus in the universal field;

the first target value determining module is configured to determine a first target value corresponding to a training corpus in the general domain based on the general translation model and the initial training corpus in the specific domain, where one training corpus corresponds to a first target value, and the first target value can represent a matching degree of the corresponding training corpus and the specific domain;

the data screening module is used for screening the training corpus in the appointed field from the training corpus set in the general field based on a first target value corresponding to the training corpus in the training corpus set in the general field;

the first target value determining module is specifically configured to determine a gradient of each training corpus in the initial training corpus in the designated domain on the general translation model when determining a first target value corresponding to a target training corpus in the designated domain based on the general translation model and the initial training corpus in the designated domain, calculate an average value of gradients of each training corpus in the initial training corpus in the designated domain on the general translation model, as a domain gradient average value, determine a gradient of the target training corpus on the general translation model, and calculate a distance between the gradient of the target training corpus on the general translation model and the domain gradient average value, as a first target value corresponding to the target training corpus.

11. A machine translation apparatus for a specified domain, comprising: the system comprises a source language text acquisition module and a translation module;

the source language text acquisition module is used for acquiring a source language text to be translated in a designated field;

the translation module is used for inputting the source language text to be translated into a pre-established domain translation model to obtain a target language text corresponding to the source language text to be translated;

the domain translation model is obtained by adjusting a general translation model by adopting an initial training corpus of a designated domain and a training corpus obtained from a training corpus of a general domain by adopting the domain data obtaining device according to claim 10, and the general translation model is obtained by training the training corpus of the general domain.

12. A field data acquisition apparatus, characterized by comprising: a memory and a processor;

the memory is used for storing programs;

the processor is configured to execute the program to implement the steps of the domain data acquisition method according to any one of claims 1 to 6.

13. A readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the respective steps of the domain data acquisition method according to any one of claims 1 to 6.