CN118468899A - Translation method and device for example-aware machine translation large language model - Google Patents

Abstract

The present invention relates to the field of machine translation, and in particular to a translation method and device of a machine translation large language model based on example perception, the method comprising: constructing sentence-level perception examples and document-level perception examples; constructing example-perception training data according to sentence-level perception examples and document-level perception examples, and obtaining a trained machine translation large language model according to the training data and low-rank adaptive LoRA fine-tuning technology; constructing domain translation examples and document-level translation examples; optimizing the trained machine translation large language model according to the domain translation examples and document-level translation examples, obtaining a constructed machine translation large language model with improved example perception ability, and then obtaining a translation result. The present invention can not only provide customized solutions for specific needs such as specific domain translation and document-level translation, but also significantly improve translation efficiency and performance without sacrificing translation quality.

Description

Translation method and device for example-aware machine translation large language model

Technical Field

The present invention relates to the technical field of machine translation, and in particular to a translation method and device for a large language model of machine translation based on example perception.

Background Art

As large language model technology has made significant progress in academia and industry, and has shown impressive performance in machine translation, large language models for machine translation have gradually become a hot research topic in the field of natural language processing. By providing relevant examples for contextual learning or fine-tuning the large model through supervised learning with translation data, the large model can be adapted to translation tasks in multiple languages, demonstrating excellent translation capabilities. Therefore, it has received widespread attention from researchers in recent years.

At present, the large language model technology for machine translation includes prompt-based methods and fine-tuning-based methods to use large language models. The prompt-based large language model for machine translation activates the translation ability of the model by providing examples to the basic large language model, providing a training-free method that can be quickly deployed according to specific translation needs (such as domain-specific translation). However, this method is constrained by fixed parameters, which limits its adaptability and translation depth. In contrast, the fine-tuning-based large language model for machine translation enhances translation capabilities through supervised fine-tuning on machine translation-specific datasets, thereby improving performance through parameter updates. However, this method often ignores the contextual learning ability of the large language model, especially in some translation scenarios that require example perception. Only sentence-level translation using simple instructions will limit the translation ability of the large language model for machine translation.

Summary of the invention

In order to solve the shortcomings of example perception and learning in existing machine translation, how to overcome the limitations of the large language model of machine translation based on prompts and fine-tuning, so as to achieve more efficient and more accurate machine translation, the embodiment of the present invention provides a translation method and device of a large language model of machine translation based on example perception. The technical solution is as follows:

On the one hand, a translation method of a large language model for machine translation based on example perception is provided, the method is implemented by a translation device for a large language model for machine translation, and the method includes:

S1. Obtain data to be translated.

S2. Input the data into the constructed large language model for machine translation with improved example perception capabilities.

S3. Obtain translation results based on a large language model of machine translation with improved data and example perception capabilities.

The process of building a large language model for machine translation with improved sample perception capabilities includes:

S21. Construct sentence-level perception examples and document-level perception examples.

S22. Construct example-aware training data based on the sentence-level perception examples and the document-level perception examples; and obtain a trained large language model for machine translation through low-rank adaptive LoRA fine-tuning technology based on the example-aware training data.

S23. Construction domain translation examples and document level translation examples.

S24. According to the domain translation examples and the document-level translation examples, the trained machine translation large language model is optimized for translation, so as to obtain a constructed machine translation large language model with improved example perception capability.

Optionally, constructing sentence-level perception examples and document-level perception examples in S21 includes:

S211. Obtain the original training set.

S212. In the original training set, randomly select multiple translation pairs and use the multiple translation pairs as sentence-level perception examples.

S213: Select any translation pair in the original training set, obtain the first multiple translation pairs of the selected translation pair in the original training set, and use the obtained multiple translation pairs as document-level perception examples.

Optionally, a sentence-level perception example is shown in the following formula (1):

(1)

In the formula, Indicates Example of sentence-level perception of translation pairs, represents the number of translation pairs, The format of the representation example is, Indicates The source sentence of the translation pair, Indicates The target sentence of a translation pair.

Optionally, a document-level perception example is shown in the following formula (2):

(2)

In the formula, represents a document-level aware example, represents the order of sentences in the original training set, represents the number of translation pairs, The format of the representation example is, Indicated in The first sentence before Source sentence, Indicated in The first sentence before target sentence.

Optionally, in S22, example-aware training data is constructed according to the sentence-level awareness examples and the document-level awareness examples; and according to the example-aware training data, a trained large language model for machine translation is obtained by low-rank adaptive LoRA fine-tuning technology, including:

S221. Concatenate the sentence-level perception examples and the document-level perception examples with the original training data, and mix the concatenated training data through Bernoulli probability to obtain example-perception training data.

S222. Obtain a basic large language model for machine translation.

S223. Use low-rank adaptive LoRA fine-tuning technology to add adjustable parameters to the basic machine translation large language model to obtain a machine translation large language model with adjustable parameters.

S224. According to the example perception training data, a large machine translation language model with adjustable parameters is trained to obtain a trained large machine translation language model.

Optionally, the training loss of the large language model of machine translation with adjustable parameters in S223 is as shown in the following formula (3):

(3)

In the formula, represents the training loss of a large language model for machine translation with adjustable parameters, represents the cross entropy loss, Represents the target sentence The probability distribution of represents the target sentence, represents the source sentence of the translation, Indicates the instructions being translated, express A collection of translation pairs , Represents the parameters of the basic machine translation language model is frozen during training, Indicates an adjustable parameter.

Optionally, the construction domain translation example and the document level translation example in S23 include:

S231. Obtain an original training set.

S232. According to the R-BM25 retrieval method, the translation pairs in the original training set are scored and sorted, and the translation pairs with scores exceeding a preset threshold are selected as domain translation examples.

S233. For a target sentence in a test set, obtain the translation source sentences and target sentences of the first multiple sentences of the target sentence, input the translation source sentences and target sentences of the first multiple sentences into a trained machine translation large language model, obtain translation examples generated by the machine translation large language model, and obtain document-level translation examples based on the domain translation examples and the translation examples generated by the machine translation large language model.

On the other hand, a translation device based on an example-aware machine translation large language model is provided, and the device is applied to a translation method based on an example-aware machine translation large language model, and the device includes:

The acquisition module is used to obtain the data to be translated.

The input module is used to input data into the constructed large language model for machine translation with improved example-awareness.

The output module is used to obtain the translation results based on the large language model of machine translation with improved data and example perception capabilities.

The process of building a large language model for machine translation with improved sample perception capabilities includes:

S21. Construct sentence-level perception examples and document-level perception examples.

S22. Construct example-aware training data based on sentence-level perception examples and document-level perception examples; based on the example-aware training data, obtain a trained large language model for machine translation through low-rank adaptive LoRA fine-tuning technology.

S23. Construction domain translation examples and document level translation examples.

S24. According to the domain translation examples and the document-level translation examples, the trained machine translation large language model is optimized for translation, so as to obtain a constructed machine translation large language model with improved example perception capability.

Optionally, the input module is further configured to:

S211. Obtain the original training set.

S212. Randomly select multiple translation pairs from the original training set, and use the multiple translation pairs as sentence-level perception example training data.

S213: Select any translation pair in the original training set, obtain the first multiple translation pairs of the selected translation pair in the original training set, and use the obtained multiple translation pairs as document-level perception example training data.

Optionally, a sentence-level perception example is shown in the following formula (1):

(1)

In the formula, Indicates Example of sentence-level perception of translation pairs, represents the number of translation pairs, The format of the representation example is, Indicates The source sentence of the translation pair, Indicates The target sentence of a translation pair.

Optionally, a document-level perception example is shown in the following formula (2):

(2)

In the formula, represents a document-level aware example, represents the order of sentences in the original training set, represents the number of translation pairs, The format of the representation example is, Indicated in The first sentence before Source sentence, Indicated in The first sentence before target sentence.

Optionally, the input module is further configured to:

S221. Concatenate the sentence-level perception examples and the document-level perception examples with the original training data, and mix the concatenated training data through Bernoulli probability to obtain example-perception training data.

S222. Obtain a basic large language model for machine translation.

S223. Use low-rank adaptive LoRA fine-tuning technology to add adjustable parameters to the basic machine translation large language model to obtain a machine translation large language model with adjustable parameters.

S224. According to the example perception training data, a large machine translation language model with adjustable parameters is trained to obtain a trained large machine translation language model.

Optionally, the training loss of the large language model of machine translation with adjustable parameters is as shown in the following formula (3):

(3)

In the formula, represents the training loss of a large language model for machine translation with adjustable parameters, represents the cross entropy loss, Represents the target sentence The probability distribution of represents the target sentence, represents the source sentence of the translation, Indicates the instructions being translated, express A collection of translation pairs , Represents the parameters of the basic machine translation large language model is frozen during training, Indicates an adjustable parameter.

Optionally, the input module is further configured to:

S231. Obtain an original training set.

S232. According to the R-BM25 retrieval method, the translation pairs in the original training set are scored and sorted, and the translation pairs with scores exceeding a preset threshold are selected as domain translation examples.

S233. For a target sentence in a test set, obtain the translation source sentences and target sentences of the first multiple sentences of the target sentence, input the translation source sentences and target sentences of the first multiple sentences into a trained machine translation large language model, obtain translation examples generated by the machine translation large language model, and obtain document-level translation examples based on the domain translation examples and the translation examples generated by the machine translation large language model.

On the other hand, a translation device for a large language model for machine translation is provided, the translation device for a large language model for machine translation comprising: a processor; a memory, the memory storing computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the translation methods for a large language model for machine translation based on example-perceptual enhancement is implemented.

On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any of the above-mentioned translation methods of the large language model of machine translation based on example-aware boosting.

The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

In the embodiment of the present invention, a lightweight example-aware fine-tuning method and an example screening hybrid strategy are used to effectively integrate the prompt-based and fine-tuning methods, thereby improving the adaptability and performance of the translation model in various translation scenarios. In addition, the present invention pays special attention to using examples to promote the translation of rare words, while proposing strategies to mitigate the negative impact that noise demonstrations may have on translation quality. In this way, the present invention can not only provide customized solutions for specific needs such as translation in specific fields and document-level translation, but also significantly improve translation efficiency and performance without sacrificing translation quality.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

FIG1 is a flow chart of a translation method of a large language model for machine translation based on example perception provided by an embodiment of the present invention;

FIG2 is an overall flow chart of a translation method of a large language model of machine translation based on example perception provided by an embodiment of the present invention;

FIG3 is a flow chart of an example perception training method provided by an embodiment of the present invention;

FIG4 is a flow chart of an example perceptual reasoning method provided by an embodiment of the present invention;

FIG5 is a block diagram of a translation device of a large language model of machine translation based on example perception provided by an embodiment of the present invention;

FIG6 is a schematic diagram of the structure of a translation device for a large language model for machine translation provided by an embodiment of the present invention.

DETAILED DESCRIPTION

The technical solution of the present invention is described below in conjunction with the accompanying drawings.

In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and/or" can be both, or it can be either of the two.

In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

In the embodiments of the present invention, sometimes a subscript such as _W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

The embodiment of the present invention provides a translation method of a large language model for machine translation based on example perception, which can be implemented by a translation device of a large language model for machine translation, which can be a terminal or a server. As shown in FIG1 , the flow chart of the translation method of a large language model for machine translation based on example perception is shown in FIG1 . The processing flow of the method may include the following steps:

S1. Obtain data to be translated.

S2. Input the data into the constructed large language model for machine translation with improved example perception capabilities.

S3. Obtain translation results based on a large language model of machine translation with improved data and example perception capabilities.

As shown in FIG. 2 , the process of constructing a large language model for machine translation with improved perception capability may include the following steps S21-S24:

S21. Construct sentence-level perception examples and document-level perception examples.

Optionally, the above step S21 may include the following steps S211-S213:

S211. Obtain the original training set.

S212. In the original training set, randomly select multiple translation pairs and use the multiple translation pairs as sentence-level perception examples.

In a feasible implementation, the original training data of the large language model for machine translation are randomly selected by taking advantage of the fact that they are all from the same field.

Specifically, in the usual contextual learning framework, the model selects Examples as perception examples To translate the target sentence, the target sentence The probability distribution of is as follows:

(1)

In the formula, Represents the target sentence The probability distribution of Indicates the length of the target sentence.

By leveraging learned knowledge and contextual information, the model is able to generate more accurate and natural translation results.

FIG3 shows the specific process of example perception training. For sentence-level translation, considering that the original training data are all in the same field, the present invention randomly selects A translation pair is used as a sentence-level perception example of the present invention, and the specific structure is as follows:

(2)

In the formula, Indicates Example of sentence-level perception of translation pairs, represents the number of translation pairs, The format of the representation example is, Indicates The source sentence of the translation pair, Indicates The target sentence of the translation pair, " indicates the separator between examples.

Therefore, the probability of translation is During the training process, sentence-level examples are selected from the entire training data.

S213: Select any translation pair in the original training set, obtain the first multiple translation pairs of the selected translation pair in the original training set, and use the obtained multiple translation pairs as document-level perception examples.

In a feasible implementation, document-level selection is performed by utilizing the fact that the original training data of the large language model of machine translation are all logical paragraphs.

Specifically, for document-level translation, considering that in the context learning process of the model, it is crucial to understand the logical relationship in the context, and the training set usually consists of continuous paragraphs rather than unrelated sentences. To this end, the present invention randomly selects the pair of parallel training data in front of it and next to it in the original training set. An example is given as a document-level perception example of the present invention, and the specific structure is as follows:

(3)

In the formula, represents a document-level aware example, represents the order of sentences in the original training set, represents the number of translation pairs, The format of the representation example is, Indicated in The first sentence before Source sentence, Indicated in The first sentence before target sentence.

Various exemplary forms of the present invention are shown in Table 1:

Table 1

S22. Construct example-aware training data based on sentence-level perception examples and document-level perception examples; based on the example-aware training data, obtain a trained large language model for machine translation through low-rank adaptive LoRA fine-tuning technology.

In a feasible implementation, the constructed sentence-level and document-level training data are mixed, and the LoRA (Low-Rank Adaptation) method is used for fast and efficient training.

Optionally, the above step S22 may include the following steps S221-S224:

S221. Concatenate the sentence-level perception examples and the document-level perception examples with the original training data, and mix the concatenated training data through Bernoulli probability to obtain example-perception training data.

In a feasible implementation, after constructing sentence-level perception examples and document-level perception examples, considering the different application scenarios faced by the large language model of machine translation, the present invention randomly selects the example types to be used through Bernoulli distribution for mixing. This mixed use method improves the flexibility and efficiency of the model in different translation scenarios. The specific distribution is as follows:

(4)

in, To select sentence-level demonstrations during training The probability of selecting document-level demonstration The probability of .

S222. Obtain a basic large language model for machine translation.

S223. Use low-rank adaptive LoRA fine-tuning technology to add adjustable parameters to the basic machine translation large language model to obtain a machine translation large language model with adjustable parameters.

In a feasible implementation, in order to further optimize the model performance, the present invention introduces lightweight LoRA fine-tuning technology. By adding a small number of adjustable parameters to the model, the learning ability and adaptability of the model are improved while ensuring the efficiency of the model. This method can fully utilize the capabilities of the pre-trained model and effectively fine-tune it for specific tasks. Given the parameters of the basic model , the present invention adds additional adjustable parameters to the model Therefore, the training samples The training loss is:

(5)

In the formula, represents the training loss of a large translation model with adjustable parameters, represents the cross entropy loss, Represents the target sentence The probability distribution of represents the target sentence, represents the source sentence of the translation, Indicates the instructions being translated, and equivalence, Representation parameters Frozen during training.

Since the present invention uses lightweight LoRA fine-tuning, and The comparison of the parameter sizes should satisfy:

(6)

S224. According to the example perception training data, a large machine translation language model with adjustable parameters is trained to obtain a trained large machine translation language model.

S23. Construction domain translation examples and document level translation examples.

Optionally, the above step S23 may include the following steps S231-S233:

S231. Obtain an original training set.

S232. According to the R-BM25 retrieval method, the translation pairs in the original training set are scored and sorted, and the translation pairs with scores exceeding a preset threshold are selected as domain translation examples.

In a feasible implementation, as shown in FIG4 , the examples are scored and ranked using the R-BM25 method, and the present invention selects several examples with the highest scores as high-quality examples used in the reasoning of the present invention as examples for reasoning.

Among them, the R-BM25 retrieval method: the present invention first uses conventional BM25 to screen examples, then extracts all word ngrams and their counts from the translated source sentences of the retrieved BM25 examples and the target sentences of the test set, let S and Q represent the sets of translated source sentence ngrams and target sentence ngrams from the BM25 retrieval examples and the test set, respectively, re-rank according to the matching degree of ngrams, and use the following formula to calculate the R-BM25 score:

(7)

(8)

In the formula, Indicates the matching rate of ngram, Indicates the number of ngrams to be calculated. Indicates the R-BM25 score.

Specifically, in terms of domain translation optimization, the present invention significantly improves the translation performance of the machine translation model in a specific domain by combining the R-BM25 (Best Match25) retrieval method and the incremental learning framework. By accurately selecting translation pairs with high domain similarity to the sentences to be translated as model input demonstrations, the present invention ensures that the model can access contexts and expressions that are highly relevant to the target domain. This demonstration selection method based on N-Gram (language model) matching and language similarity calculation not only deepens the model's understanding of specific domains, but also enhances the accuracy and naturalness of its translation. In this way, the model can dynamically filter out the most relevant examples from a large data set, thereby providing rich and accurate contextual information during the translation process, greatly improving the adaptability and performance of the model in translation tasks in diverse domains. The present invention uses retrieval-based examples for domain translation, satisfying the equation . By retrieving the scoring function and search corpus , the retrieved R-BM25 example is:

(9)

In the formula, Indicates the filtered examples, Indicates selection The examples with the highest scores.

S233. For a target sentence in a test set, obtain the translation source sentences and target sentences of the first multiple sentences of the target sentence, input the translation source sentences and target sentences of the first multiple sentences into a trained machine translation large language model, obtain translation examples generated by the machine translation large language model, and obtain document-level translation examples based on the domain translation examples and the translation examples generated by the machine translation large language model.

In a feasible implementation, examples selected by the R-BM25 method and logical pre-translation content generated by the large language model of machine translation are used as examples for reasoning.

The target sentence is each piece of data in the test set. For each piece of test data, the multiple adjacent translated training data in front of it are used as examples. Since the test set is a logical text, each sentence is related to the previous sentences, so the adjacent translated data are used as translation examples.

Specifically, in terms of document-level translation optimization, the present invention adopts an innovative online adaptation method and hybrid demonstration strategy, which effectively utilizes the contextual relationship within the document to improve translation quality. By selecting sentences adjacent to the target sentence and the translations generated by the model itself in real time as examples, and combining them with relevant field examples retrieved through R-BM25, the document-level translation optimization strategy of the present invention, by comprehensively utilizing the direct contextual information generated by the model itself and the field-related information retrieved externally, enhances the model's capture of the entire document context and learning of sentence-related knowledge, and also improves its adaptability to document style and topic. In order to further utilize the intrinsic contextual information of the same document, the present invention uses the sentences previously translated by the model itself when translating a sentence. Using K-shot document-level demonstration , sentence Translation of The generation method is as follows:

(10)

Furthermore, in order to ensure the translation quality with specific information (such as domain, style or theme) in document-level translation, the present invention proposes to use retrieval-based demonstration and document-level presentation Therefore, the final mixed example decoding criterion is:

(11)

in, and Represents the number of sentence-level and document-level examples respectively. For the total number of examples ,have , where Indicates splicing.

S24. According to the domain translation examples and the document-level translation examples, the trained machine translation large language model is optimized to obtain a constructed machine translation large language model with improved example perception capability.

Table 2

Table 2 shows the COMET and d-BLEU scores of the prompt-based machine translation large language model, the fine-tuning-based machine translation large language model, the original machine translation large language model, and the example-aware improvement-based machine translation large language model of the present invention. The present invention selects the best-performing model among the fine-tuning-based machine translation large language models as the original machine translation large language model, represents the base model used by the present invention, the example-based machine translation large language model represents the effect obtained by the original machine translation large language model using the translation examples screened by the present invention, the example-aware machine translation large language model represents the effect obtained by training using the method of the present invention and using the translation examples screened by the present invention, and 7B and 13B represent the parameter amounts of the model. According to the main results of Table 2, the present invention finds that the performance of the original fine-tuning-based machine translation large language model may be impaired by examples, while the example-aware improvement-based machine translation large language model proposed by the present invention can learn from examples significantly and stably, and the results on all data sets are better than the baseline, showing the significant improvement of the context learning ability brought by the example-aware improvement-based machine translation large language model. Finally, the large language model for machine translation based on example-aware improvement can enhance example-awareness under different model sizes and surpasses the original large language model for machine translation on all datasets, demonstrating the stable superiority of the large language model for machine translation based on example-aware improvement on different model sizes.

In the embodiment of the present invention, a lightweight example-aware fine-tuning method and an example screening hybrid strategy are used to effectively integrate the prompt-based and fine-tuning methods, thereby improving the adaptability and performance of the translation model in various translation scenarios. In addition, the present invention pays special attention to using examples to promote the translation of rare words, while proposing strategies to mitigate the negative impact that noise demonstrations may have on translation quality. In this way, the present invention can not only provide customized solutions for specific needs such as translation in specific fields and document-level translation, but also significantly improve translation efficiency and performance without sacrificing translation quality.

FIG5 is a block diagram of a translation device based on an example-aware machine translation large language model according to an exemplary embodiment, and the device is used for a translation method based on an example-aware machine translation large language model. Referring to FIG5 , the device includes an acquisition module 310, an input module 320, and an output module 330. Among them:

The acquisition module 310 is used to acquire data to be translated.

The input module 320 is used to input data into the constructed large language model of machine translation with improved example perception capability.

The output module 330 is used to obtain a translation result based on the machine translation large language model with improved data and example perception capabilities.

The process of constructing a large language model for machine translation with example-aware capabilities includes:

S21. Construct sentence-level perception examples and document-level perception examples.

S22. Construct example-aware training data based on sentence-level perception examples and document-level perception examples, and obtain a trained large language model for machine translation through low-rank adaptive LoRA fine-tuning technology.

S23. Construction domain translation examples and document level translation examples.

S24. Construct example-aware training data based on sentence-level perception examples and document-level perception examples; based on the example-aware training data; based on the example-aware training data, obtain a trained large language model for machine translation through low-rank adaptive LoRA fine-tuning technology.

Optionally, the input module 320 is further configured to:

S211. Obtain the original training set.

S212. In the original training set, randomly select multiple translation pairs and use the multiple translation pairs as sentence-level perception examples.

S213: Select any translation pair in the original training set, obtain the first multiple translation pairs of the selected translation pair in the original training set, and use the obtained multiple translation pairs as document-level perception examples.

Optionally, a sentence-level perception example is shown in the following formula (1):

(1)

In the formula, Indicates Example of sentence-level perception of translation pairs, represents the number of translation pairs, The format of the representation example is, Indicates The source sentence of the translation pair, Indicates The target sentence of a translation pair.

Optionally, a document-level perception example is shown in the following formula (2):

(2)

In the formula, represents a document-level aware example, represents the order of sentences in the original training set, represents the number of translation pairs, The format of the representation example is, Indicated in The first sentence before Source sentence, Indicated in The first sentence before target sentence.

Optionally, the input module 320 is further configured to:

S221. Concatenate the sentence-level perception examples and the document-level perception examples with the original training data, and mix the concatenated training data through Bernoulli probability to obtain example-perception training data.

S222. Obtain a basic large language model for machine translation.

S223. Use low-rank adaptive LoRA fine-tuning technology to add adjustable parameters to the basic machine translation large language model to obtain a machine translation large language model with adjustable parameters.

S224. According to the example perception training data, a large machine translation language model with adjustable parameters is trained to obtain a trained large machine translation language model.

Optionally, the training loss of the large language model of machine translation with adjustable parameters is as shown in the following formula (3):

(3)

In the formula, represents the training loss of a large language model for machine translation with adjustable parameters, represents the cross entropy loss, Represents the target sentence The probability distribution of represents the target sentence, represents the source sentence of the translation, Indicates the instructions being translated, express A collection of translation pairs , Represents the parameters of the basic machine translation language model is frozen during training, Indicates an adjustable parameter.

Optionally, the input module 320 is further configured to:

S231. Obtain an original training set.

S232. According to the R-BM25 retrieval method, the translation pairs in the original training set are scored and sorted, and the translation pairs with scores exceeding a preset threshold are selected as domain translation examples.

S233. For a target sentence in a test set, obtain the translation source sentences and target sentences of the first multiple sentences of the target sentence, input the translation source sentences and target sentences of the first multiple sentences into a trained machine translation large language model, obtain translation examples generated by the machine translation large language model, and obtain document-level translation examples based on the domain translation examples and the translation examples generated by the machine translation large language model.

In the embodiment of the present invention, a lightweight example-aware fine-tuning method and an example screening hybrid strategy are used to effectively integrate the prompt-based and fine-tuning methods, thereby improving the adaptability and performance of the translation model in various translation scenarios. In addition, the present invention pays special attention to using examples to promote the translation of rare words, while proposing strategies to mitigate the negative impact that noise demonstrations may have on translation quality. In this way, the present invention can not only provide customized solutions for specific needs such as translation in specific fields and document-level translation, but also significantly improve translation efficiency and performance without sacrificing translation quality.

FIG6 is a schematic diagram of the structure of a translation device for a large language model of machine translation provided by an embodiment of the present invention. As shown in FIG6 , the translation device for a large language model of machine translation may include the translation device for the large language model of machine translation based on example perception shown in FIG5 . Optionally, the translation device 410 for a large language model of machine translation may include a first processor 2001.

Optionally, the translation device 410 for machine translation of a large language model may further include a memory 2002 and a transceiver 2003 .

The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

The following is a detailed introduction to the various components of the translation device 410 of the machine translation large language model in conjunction with FIG. 6 :

The first processor 2001 is the control center of the translation device 410 of the machine translation large language model, and can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

Optionally, the first processor 2001 may perform various functions of the translation device 410 for machine translating a large language model by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as CPU0 and CPU1 shown in FIG. 6 .

In a specific implementation, as an embodiment, the translation device 410 of the machine translation large language model may also include multiple processors, such as the first processor 2001 and the second processor 2004 shown in FIG6 . Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and/or processing cores for processing data (such as computer program instructions).

The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, a random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing the desired program code in the form of an instruction or data structure and capable of being accessed by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and be coupled to the first processor 2001 through an interface circuit (not shown in FIG. 6 ) of the translation device 410 for the large language model of machine translation, which is not specifically limited in the embodiment of the present invention.

The transceiver 2003 is used to communicate with a network device or a terminal device.

Optionally, the transceiver 2003 may include a receiver and a transmitter (not shown separately in FIG6 ), wherein the receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

Optionally, the transceiver 2003 can be integrated with the first processor 2001, or can exist independently and be coupled to the first processor 2001 through the interface circuit (not shown in FIG. 6 ) of the translation device 410 of the machine translation large language model, which is not specifically limited in the embodiment of the present invention.

It should be noted that the structure of the translation device 410 of the machine translation large language model shown in FIG6 does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

In addition, the technical effects of the translation device 410 for the large language model of machine translation can refer to the technical effects of the translation method for the large language model of machine translation based on example perception described in the above method embodiment, which will not be repeated here.

It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

It should be understood that the term "and/or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character "/" in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and/or" relationship. Please refer to the context for specific understanding.

In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (10)

1. A method for translating a machine translation large language model based on example awareness, the method comprising:

S1, acquiring data to be translated;

s2, inputting the data into a constructed machine translation large language model with improved example perceptibility;

s3, according to the data and the machine translation large language model with the improved example perceptibility, a translation result is obtained;

the construction process of the machine translation large language model with the improved example perceptibility comprises the following steps:

S21, constructing sentence-level perception examples and document-level perception examples;

S22, constructing example perception training data according to the sentence-level perception examples and the document-level perception examples; according to the example perception training data, a trained machine translation large language model is obtained through a low-rank adaptation LoRA fine tuning technology;

S23, constructing a domain translation example and a document level translation example;

And S24, according to the domain translation example and the document level translation example, performing translation optimization on the trained machine translation large language model to obtain a constructed machine translation large language model with improved example perceptibility.

2. The method for translating a large language model based on example perception machine according to claim 1, wherein constructing sentence-level perception examples and document-level perception examples in S21 includes:

s211, acquiring an original training set;

s212, randomly selecting a plurality of translation pairs in an original training set, and taking the plurality of translation pairs as sentence-level perception examples;

S213, selecting any translation pair in the original training set, acquiring a plurality of translation pairs before the selected translation pair in the original training set, and taking the acquired translation pairs as document-level perception examples.

3. The method for translating a large language model for example-aware machine translation of claim 2, wherein said sentence-level aware examples are represented by the following formula (1):

（1）

In the method, in the process of the invention, Represent the firstSentence-level perception examples of the individual translation pairs,Representing the number of translation pairs,The format of the examples is shown and,Represent the firstThe source sentences of the individual translation pairs are,Represent the firstTarget sentences of the translation pairs.

4. The method for translating a large language model for example-aware machine translation of claim 2, wherein said document-level aware examples are represented by the following formula (2):

（2）

In the method, in the process of the invention, A document-level perception example is represented,Representing the order of sentences in the original training set,Representing the number of translation pairs,The format of the examples is shown and,Is shown in the firstFirst of every sentenceThe number of source sentences that are to be processed,Is shown in the firstFirst of every sentenceAnd (3) target sentences.

5. The method for translating a large language model for example-aware-based machine translation of claim 1, wherein said step S22 constructs example-aware training data from said sentence-level aware examples and document-level aware examples; according to the example perception training data, a trained machine translation large language model is obtained through a low-rank adaptation LoRA fine tuning technology, which comprises the following steps:

S221, splicing the sentence-level perception examples and the document-level perception examples with original training data, and mixing the spliced training data through Bernoulli probability to obtain example perception training data;

s222, acquiring a basic machine translation large language model;

s223, adding adjustable parameters to the basic machine translation large language model by adopting a low-rank adaptation LoRA fine tuning technology to obtain a machine translation large language model with adjustable parameters;

S224, training the machine translation large language model with the adjustable parameters according to the example perception training data to obtain a trained machine translation large language model.

6. The method for translating a large machine translation language model based on example perception according to claim 5, wherein the training loss of the large machine translation language model with adjustable parameters in S223 is represented by the following formula (3):

（3）

In the method, in the process of the invention, Represents a training penalty for parameter-tunable machine translation large language models,Representing the cross-entropy loss,Representing a target sentenceIs a function of the probability distribution of (1),The target sentence is represented and the target sentence is displayed,The source sentence that is to be translated is represented,Representing an instruction in the translation,Representation ofAggregation of individual translation pairs，Parameters representing underlying machine translated large language modelsIs frozen during the training process and is then frozen,Representing the adjustable parameter.

7. The method for translating a large language model for example-aware machine translation according to claim 1, wherein the constructing domain translation example and the document level translation example in S23 comprises:

s231, acquiring an original training set;

S232, according to an R-BM25 retrieval method, scoring and sorting are carried out on translation pairs in an original training set, and translation pairs with scores exceeding a preset threshold are selected as field translation examples;

S233, aiming at a target sentence in a test set, acquiring translation source sentences and target sentences of a plurality of sentences in front of the target sentence, inputting the translation source sentences and target sentences of the plurality of sentences in front to a trained machine translation large language model to obtain a translation example generated by the machine translation large language model, and obtaining a document-level translation example according to the field translation example and the translation example generated by the machine translation large language model.

8. A translation apparatus of an example-aware machine translation large language model for implementing the translation method of an example-aware machine translation large language model according to any one of claims 1 to 7, characterized in that the apparatus comprises:

The acquisition module is used for acquiring data to be translated;

The input module is used for inputting the data into the constructed machine translation large language model with improved example perceptibility;

the output module is used for obtaining a translation result according to the data and the machine translation large language model with the improved example perceptibility;

the construction process of the machine translation large language model with the improved example perceptibility comprises the following steps:

S21, constructing sentence-level perception examples and document-level perception examples;

S22, constructing example perception training data according to the sentence-level perception examples and the document-level perception examples; according to the example perception training data, a trained machine translation large language model is obtained through a low-rank adaptation LoRA fine tuning technology;

S23, constructing a domain translation example and a document level translation example;

And S24, optimizing the trained machine translation large language model according to the domain translation example and the document level translation example to obtain a constructed machine translation large language model with improved example perceptibility.

9. A translation apparatus for machine translating a large language model, the translation apparatus for machine translating a large language model comprising:

A processor;

a memory having stored thereon computer readable instructions which, when executed by the processor, implement the method of any of claims 1 to 7.

10. A computer readable storage medium having stored therein program code which is callable by a processor to perform the method of any one of claims 1 to 7.