[go: up one dir, main page]

CN118395996B - An automatic evaluation method for machine translation based on deep cross-network - Google Patents

An automatic evaluation method for machine translation based on deep cross-network Download PDF

Info

Publication number
CN118395996B
CN118395996B CN202410872045.1A CN202410872045A CN118395996B CN 118395996 B CN118395996 B CN 118395996B CN 202410872045 A CN202410872045 A CN 202410872045A CN 118395996 B CN118395996 B CN 118395996B
Authority
CN
China
Prior art keywords
machine translation
translation
feature vector
sentence
quality
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410872045.1A
Other languages
Chinese (zh)
Other versions
CN118395996A (en
Inventor
李茂西
魏嘉琴
万旻晖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangxi Normal University
Original Assignee
Jiangxi Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangxi Normal University filed Critical Jiangxi Normal University
Priority to CN202410872045.1A priority Critical patent/CN118395996B/en
Publication of CN118395996A publication Critical patent/CN118395996A/en
Application granted granted Critical
Publication of CN118395996B publication Critical patent/CN118395996B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/51—Translation evaluation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3344—Query execution using natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/205—Parsing
    • G06F40/211—Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/58—Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Data Mining & Analysis (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Machine Translation (AREA)

Abstract

本发明公开一种基于深度交叉网络的机器译文自动评价方法，步骤为：获取训练集，对训练集进行规范化处理；提取独立表征模式下句子级别机器译文质量特征向量；提取统一表征模式下句子级别机器译文质量特征向量；提取机器译文质量交叉特征向量；预测机器译文质量；训练基于深度交叉网络的机器译文自动评价模型。预测方法步骤为；将机器译文与人工参考译文输入上述基于深度交叉网络的机器译文自动评价模型预测机器译文质量；同时采用大语言模型向量数据库直接对机器译文和人工参考译文进行句向量表征，计算机器译文与人工参考译文的余弦相似度，将预测的机器译文质量与余弦相似度线性加权获取机器译文质量的分值。

The present invention discloses a method for automatic evaluation of machine translation based on a deep cross network, the steps of which are: obtaining a training set, normalizing the training set; extracting a sentence-level machine translation quality feature vector under an independent representation mode; extracting a sentence-level machine translation quality feature vector under a unified representation mode; extracting a machine translation quality cross feature vector; predicting the quality of machine translation; and training a machine translation automatic evaluation model based on a deep cross network. The prediction method steps are: inputting machine translation and artificial reference translation into the above-mentioned machine translation automatic evaluation model based on a deep cross network to predict the quality of machine translation; at the same time, using a large language model vector database to directly perform sentence vector representation on machine translation and artificial reference translation, calculating the cosine similarity between machine translation and artificial reference translation, and linearly weighting the predicted machine translation quality with the cosine similarity to obtain the machine translation quality score.

Description

Machine translation automatic evaluation method based on deep crossover network
Technical Field
The invention relates to the technical field of natural language processing, in particular to a machine translation automatic evaluation method based on a deep cross network.
Background
In research and application of machine translation, automatic evaluation of machine translation plays an important role. With the vigorous development of a pre-training language model in recent years, the automatic machine translation evaluation method based on the neural network extracts the deep characterization of the machine translation and the manual reference translation through the pre-training language model, and builds the neural network to compare the difference of the deep characterization of the machine translation and the manual reference translation so as to predict the quality of the machine translation.
The automatic machine translation evaluation method based on the neural network is generally based on a double-tower architecture. The method comprises the steps of independently extracting the depth representation of a machine translation to form a machine translation representation tower, independently extracting the depth representation of an artificial reference translation to form an artificial reference translation representation tower, and then performing simple interaction on double towers, such as double-tower feature vector splicing, element multiplication and absolute value subtraction operation on the same position of the double-tower feature vector, and the like, wherein the feature interaction modes do not directly and explicitly model the relationship between elements at different positions of the double-tower feature, namely feature cross operation. The low-order combination features and the high-order combination features generated by the cross operation play an important role in automatic evaluation of the machine translation. Therefore, the invention provides a machine translation automatic evaluation method based on a depth intersection network, which utilizes the depth intersection network to carry out depth interaction on the machine translation characteristics and the manual reference translation characteristics so as to capture semantic relations between the machine translation characteristics and the manual reference translation characteristics, thereby improving the effect of the machine translation automatic evaluation.
Disclosure of Invention
The invention provides a machine translation automatic evaluation method based on a deep cross network, which is used for improving the correlation between the machine translation automatic evaluation and human evaluation.
The technical scheme adopted by the invention is as follows: a machine translation automatic evaluation method based on a depth cross network comprises the following steps:
step S1, a training set is obtained, normalization processing is carried out on the training set, and the training set after normalization processing is obtained; the training set is composed of a plurality of different samples, and each sample comprises a machine translation, a manual reference translation and a human evaluation score of the machine translation;
S2, extracting sentence-level machine translation quality feature vectors in an independent characterization mode; inputting the machine translation and the manual reference translation in each sample in the normalized training set into a cross-language pre-training model respectively, outputting a machine translation sub-word level feature vector in an independent characterization mode and a manual reference translation sub-word level feature vector in the independent characterization mode, carrying out interaction on the machine translation sub-word level feature vector in the independent characterization mode and the manual reference translation sub-word level feature vector in the independent characterization mode by using an external attention mechanism to obtain a machine translation interaction feature vector in the independent characterization mode and a manual reference translation interaction feature vector in the independent characterization mode, connecting the machine translation interaction feature vector in the independent characterization mode and the manual reference translation interaction feature vector in the independent characterization mode, carrying out average pooling operation, and outputting a sentence level machine translation quality feature vector in the independent characterization mode;
S3, extracting sentence-level machine translation quality feature vectors in a unified characterization mode; performing character string connection on the machine translation and the manual reference translation in each sample in the training set after normalization processing to obtain a translation joint character string, inputting the translation joint character string into a cross-language pre-training model, outputting sub-word level feature vectors in a unified characterization mode, performing interaction on the sub-word level feature vectors in the unified characterization mode by using a self-attention mechanism to obtain interaction feature vectors in the unified characterization mode, performing average pooling operation on the interaction feature vectors in the unified characterization mode, and outputting sentence level machine translation quality feature vectors in the unified characterization mode;
S4, extracting a machine translation quality cross feature vector; splicing the sentence-level machine translation quality feature vector in the independent characterization mode in the step S2 and the sentence-level machine translation quality feature vector in the unified characterization mode in the step S3, inputting the spliced sentence-level machine translation quality feature vector into a deep cross network containing 4 stacked cross layers, and outputting the machine translation quality cross feature vector;
s5, predicting the quality of the machine translation; inputting the machine translation quality cross feature vector in the step S4 into a three-layer feedforward neural network, and outputting a machine translation quality score;
S6, training a machine translation automatic evaluation model based on a depth cross network; and training parameters of the automatic machine translation evaluation model based on the depth cross network by minimizing the mean square error loss on the training set after the normalization processing according to the machine translation quality score output in the step S5 and the human evaluation score of the machine translations in the training set after the normalization processing in the step S1, so as to obtain the automatic machine translation evaluation model based on the depth cross network after training.
Further, in step S1, the training set is composed of a plurality of different samples, and each sample is specifically:
given a training set of samples d= { (h, r), y }, where d represents a training sample, h represents a machine translation, r represents an artificial reference translation, y represents a human evaluation score for machine translation h, the human evaluation score being a real value between 0-1.
Further, in step S2, a sentence-level machine translation quality feature vector in an independent characterization mode is extracted, which specifically includes:
Step S21, inputting the machine translation h and the artificial reference translation r into a cross-language pre-training model XLM-RoBERTa respectively, and performing sub-word segmentation on the machine translation h and the artificial reference translation r by using a sub-word segmentation method SENTENCEPIECE algorithm through the cross-language pre-training model XLM-RoBERTa to respectively obtain sub-word sequences comprising m sub-words and n sub-words:
；
Wherein m and n respectively represent the number of the subwords contained in the machine translation and the artificial reference translation after being segmented by using a subword segmentation method SENTENCEPIECE algorithm; h 1,h2,hm represents the 1 st sub word, the 2 nd sub word and the m th sub word after the machine translation is segmented by using the sub word segmentation method SENTENCEPIECE algorithm; r 1,r2,rn represents the 1 st sub word, the 2 nd sub word and the nth sub word after the manual reference translation is segmented;
S22, outputting a machine translation sub-word level feature vector in an independent characterization mode and a manual reference translation sub-word level feature vector in the independent characterization mode according to the positions of the sub-words and the sub-words in sentences by a cross-language pre-training model XLM-RoBERTa, wherein the cross-language pre-training model XLM-RoBERTa is shown as a formula (1) and a formula (2);
(1)；
(2)；
Wherein v h and v r respectively represent a machine translation and an artificial reference translation using a cross-language pre-training model XLM-RoBERTa to output a machine translation sub-word level feature vector in an independent characterization mode and an artificial reference translation sub-word level feature vector in an independent characterization mode, and XLM-RoBERTa ()' represents a cross-language pre-training model XLM-RoBERTa;
Step S23, interacting the machine translation sub-word level feature vector in the independent characterization mode and the manual reference translation sub-word level feature vector in the independent characterization mode by using an external attention mechanism to obtain a machine translation interaction feature vector in the independent characterization mode and a manual reference translation interaction feature vector in the independent characterization mode, wherein the machine translation interaction feature vector in the independent characterization mode and the manual reference translation interaction feature vector in the independent characterization mode are shown as a formula (3) and a formula (4):
(3)；
(4)；
Wherein, For machine translation interaction feature vectors in the independent characterization mode,Manually referencing the translation interaction feature vector in the independent characterization mode; multiHead () represents a multi-headed attention mechanism function, which contains three parameters, query, key and value, which is converted into an external attention mechanism when the query is not identical to the key and value;
Step S24, connecting the machine translation interaction feature vector in the independent characterization mode and the manual reference translation interaction feature vector in the independent characterization mode, carrying out an average pooling operation, and outputting the sentence-level machine translation quality feature vector in the independent characterization mode as shown in a formula (5):
(5)；
wherein vs hr represents the sentence-level machine translation quality feature vector in the independent characterization mode; avgPooling () represents the average pooling operation.
Further, in step S3, a sentence-level machine translation quality feature vector in a unified characterization mode is extracted, which specifically includes:
step S31, the machine translation h and the manual reference translation r are connected in character strings to obtain a translation joint character string, as shown in a formula (6):
(6)；
Wherein hr is a translation joint string, "</s >" represents the start and stop characters of the string;
step S32, inputting the translation joint character string into a cross-language pre-training model XLM-RoBERTa, and performing sub-word segmentation on the translation joint character string by using a sub-word segmentation method SENTENCEPIECE algorithm through the cross-language pre-training model XLM-RoBERTa to obtain a sub-word sequence containing p sub-words, wherein the sub-word sequence is shown in a formula (7):
(7)；
Wherein, p represents the number of the subwords contained in the translation joint character string after the subwords are segmented by using a subword segmentation method SENTENCEPIECE algorithm; hr 1,hr2,hrp represents the 1 st subword, the 2 nd subword and the p th subword of the translation joint string after being segmented by using the SENTENCEPIECE algorithm;
Step S33, a cross-language pre-training model XLM-RoBERTa outputs a subword level feature vector in a unified characterization mode according to the subwords and the positions of the subwords in sentences, as shown in a formula (8):
(8)；
Wherein v hr represents a subword level feature vector in the unified characterization mode;
Step S34, the sub-word level feature vectors in the unified characterization mode are interacted by using a self-attention mechanism to obtain interaction feature vectors in the unified characterization mode, as shown in a formula (9):
(9)；
Wherein, Representing interaction feature vectors in a unified characterization mode, multiHead ()' representing a multi-head attention mechanism function, wherein the multi-head attention mechanism function comprises three parameters of query, key and value, and the multi-head attention mechanism is converted into a self-attention mechanism when the query is identical to the key and the value;
step S35, carrying out average pooling operation on the interaction feature vectors in the unified characterization mode, and outputting sentence-level machine translation quality feature vectors in the unified characterization mode, wherein the sentence-level machine translation quality feature vectors are shown in a formula (10):
(10)；
where vu hr represents the sentence-level machine translation quality feature vector in the unified characterization mode.
Further, in step S4, a machine translation quality cross feature vector is extracted, which specifically includes:
step S41, splicing the sentence-level machine translation quality feature vector in the independent characterization mode in step S2 and the sentence-level machine translation quality feature vector in the unified characterization mode in step S3, as shown in a formula (11):
(11)；
Wherein x 0 represents a machine translation splice feature vector, and the symbol 'three' represents a vector splice operation;
Step S42, inputting the machine translation splicing feature vector into a depth cross network containing 4 stacked cross layers, and outputting a machine translation quality cross feature vector as shown in formula (12), formula (13), formula (14) and formula (15):
(12)；
(13)；
(14)；
(15)；
wherein x 1,x2,x3,x4 is the cross feature vector output by the 1 st, 2 nd, 3 rd and 4 th stacked cross layers respectively, and x 4 is taken as the machine translation quality cross feature vector; sign' "Is Hadamard product operation, W 1,W2,W3,W4 and b 1,b2,b3,b4 are the linear weight parameters and bias of the 1 st, 2 nd, 3 rd and 4 th stacked cross layers respectively; it should be noted that the deep crossover network may comprise several stacked crossover layers, and the present invention uses 4 stacked crossover layers, which are obtained from a large amount of experimental experience.
Further, in step S5, the quality of the machine translation is predicted, specifically:
Inputting the machine translation quality cross feature vector in the step S4 into a three-layer feedforward neural network, and outputting a machine translation quality score; as shown in equation (16):
(16)；
Wherein Score is the quality Score of the machine translation, and Feed-Forward is a three-layer feedforward neural network;
further, in step S6, the mean square error is lost, as shown in formula (17);
(17)；
where MSE represents the mean square error loss, N represents the number of samples in the training set, i represents the ith sample in the training set, y (i) represents the human evaluation Score of the machine translation of the ith sample in the training set, score (i) represents the machine translation quality Score of the prediction of the ith sample.
Further, another technical scheme adopted by the invention is as follows: a machine translation automatic evaluation method based on a depth cross network further comprises the following steps:
s7, carrying out standardization processing on the manual reference translation of the machine translation to be evaluated;
s8, inputting the normalized machine translation to be evaluated and the manual reference translation of the machine translation to be evaluated in the S7 into the machine translation automatic evaluation model based on the depth cross network trained in the S6, and predicting the quality score of the machine translation;
S9, calculating the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation based on the large language model vector library; respectively inputting the normalized machine translation to be evaluated and the artificial reference translation of the machine translation to be evaluated in the step S7 into a large language model vector library, directly outputting the machine translation sentence vector representation and the artificial reference translation sentence vector representation, and calculating the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation;
Step S10, calculating a final prediction value of the quality of the machine translation; and linearly weighting the machine translation quality score predicted in the step S8 and the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation based on the large language model vector library in the step S9 to obtain a final machine translation quality prediction score.
Further, in step S9, the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation based on the large language model vector library is calculated, specifically:
Inputting the normalized machine translation to be evaluated and the artificial reference translation of the machine translation to be evaluated in the step S7 into a large language model vector database Chromadb respectively, and directly outputting a machine translation sentence vector representation and an artificial reference translation sentence vector representation, wherein the machine translation sentence vector representation and the artificial reference translation sentence vector representation are shown as a formula (18) and a formula (19);
(18)；
(19)；
Wherein E h,Er represents the machine translation sentence vector representation and the manual reference translation sentence vector representation respectively, chromadb zephyr-7b () represents the Chromadb vector database output function with the large language model zephyr-7b as the base large model;
Calculating cosine similarity of the translation sentence vector representation of the machine and the artificial reference translation sentence vector representation, as shown in a formula (20);
(20)；
Wherein CosSim (h, r) represents cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation, and T represents transposition operation.
Further, in step S10, a final prediction score of the machine translation quality is calculated, specifically:
the machine translation quality score predicted in the step S8 and the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation based on the large language model vector library in the step S9 are linearly weighted, as shown in a formula (21);
（21）；
Wherein, Representing a final predictive value of machine translation quality; score represents the machine translation quality Score predicted in step S8; and 0.8 is linear interpolation weight, and is obtained according to experimental experience.
The beneficial effects of the invention are as follows: the invention decomposes a machine translation automatic evaluation method based on a depth cross network into extracting sentence-level machine translation quality feature vectors in an independent characterization mode by using a pre-training model, an external attention mechanism and average pooling; extracting sentence-level machine translation quality feature vectors in a unified characterization mode by using a pre-training model, a self-attention mechanism and average pooling according to the whole information of the machine translation and the manual reference translation; the method comprises the steps of utilizing an experiment to verify that a depth intersection network containing 4 stacked intersection layers is effective to extract machine translation quality intersection feature vectors, and inputting the machine translation quality intersection feature vectors into a feedforward neural network to automatically predict the machine translation quality; meanwhile, a large language model vector database is adopted to directly represent sentence vectors of the machine translation and the artificial reference translation, cosine similarity of the machine translation and the artificial reference translation is calculated, and the automatic predicted machine translation quality and the cosine similarity of the machine translation and the artificial reference translation are linearly weighted to obtain the score of the machine translation quality so as to improve the automatic evaluation effect of the machine translation.
Drawings
FIG. 1 is a schematic flow chart of a machine translation automatic evaluation model training method based on a depth cross network;
FIG. 2 is a schematic flow chart of the machine translation automatic evaluation method based on the deep cross network;
FIG. 3 is a schematic diagram of a machine translation automatic evaluation model structure based on a depth cross network according to the invention;
Fig. 4 is a schematic diagram of a deep crossover network structure of the present invention containing 4 stacked crossover layers.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and embodiments.
As shown in fig. 1, this embodiment works as follows, and is a machine translation automatic evaluation method based on a deep crossover network, which includes the following steps:
step S1, a training set is obtained, normalization processing is carried out on the training set, and the training set after normalization processing is obtained; the training set is composed of a plurality of different samples, and each sample comprises a machine translation, a manual reference translation and a human evaluation score of the machine translation;
S2, extracting sentence-level machine translation quality feature vectors in an independent characterization mode; inputting the machine translation and the manual reference translation in each sample in the training set after normalization processing into a cross-language pre-training model respectively, outputting a machine translation sub-word level feature vector in an independent characterization mode and a manual reference translation sub-word level feature vector in the independent characterization mode, carrying out interaction on the machine translation sub-word level feature vector in the independent characterization mode and the manual reference translation sub-word level feature vector in the independent characterization mode by using an external attention mechanism to obtain a machine translation interaction feature vector in the independent characterization mode and a manual reference translation interaction feature vector in the independent characterization mode, connecting the machine translation interaction feature vector in the independent characterization mode and the manual reference translation interaction feature vector in the independent characterization mode, carrying out average pooling operation, and outputting a sentence level machine translation quality feature vector in the independent characterization mode;
S3, extracting sentence-level machine translation quality feature vectors in a unified characterization mode; performing character string connection on the machine translation and the manual reference translation in each sample in the training set after normalization processing to obtain a translation joint character string, inputting the translation joint character string into a cross-language pre-training model, outputting sub-word level feature vectors in a unified characterization mode, performing interaction on the sub-word level feature vectors in the unified characterization mode by using a self-attention mechanism to obtain interaction feature vectors in the unified characterization mode, performing average pooling operation on the interaction feature vectors in the unified characterization mode, and outputting sentence level machine translation quality feature vectors in the unified characterization mode;
S4, extracting a machine translation quality cross feature vector; the sentence-level machine translation quality feature vector in the independent characterization mode in the step S2 and the sentence-level machine translation quality feature vector in the unified characterization mode in the step S3 are spliced, then a deep intersection network containing 4 stacked intersection layers is input, and a machine translation quality intersection feature vector is output;
s5, predicting the quality of the machine translation; inputting the machine translation quality cross feature vector in the step S4 into a three-layer feedforward neural network, and outputting a machine translation quality score;
S6, training a machine translation automatic evaluation model based on a depth cross network; and training parameters of the automatic machine translation evaluation model based on the depth cross network by minimizing the mean square error loss on the training set after the normalization processing according to the machine translation quality score output in the step S5 and the human evaluation score of the machine translations in the training set after the normalization processing in the step S1, so as to obtain the automatic machine translation evaluation model based on the depth cross network after training.
As shown in fig. 2, a machine translation automatic evaluation method based on a deep cross network further includes:
s7, carrying out standardization processing on the manual reference translation of the machine translation to be evaluated;
s8, inputting the normalized machine translation to be evaluated and the manual reference translation of the machine translation to be evaluated in the S7 into the machine translation automatic evaluation model based on the depth cross network trained in the S6, and predicting the quality score of the machine translation;
S9, calculating the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation based on the large language model vector library; respectively inputting the normalized machine translation to be evaluated and the artificial reference translation of the machine translation to be evaluated in the step S7 into a large language model vector library, directly outputting the machine translation sentence vector representation and the artificial reference translation sentence vector representation, and calculating the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation;
Step S10, calculating a final prediction value of the quality of the machine translation; and linearly weighting the machine translation quality score predicted in the step S8 and the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation based on the large language model vector library in the step S9 to obtain a final machine translation quality prediction score.
FIG. 3 is a schematic diagram of a machine translation automatic evaluation model structure based on a depth cross network according to the invention;
Fig. 4 is a schematic diagram of a deep crossover network structure of the present invention containing 4 stacked crossover layers.
In step S1, a sample in the training set is specifically:
given a training set of samples d= { (h, r), y }, where d represents a training sample, h represents a machine translation, r represents an artificial reference translation corresponding to the machine translation, y represents a human evaluation score for the machine translation h, the human evaluation score being an intervention of a real value between 0-1.
Table 1: one training sample example in a training set
Further, in step S2, a sentence-level machine translation quality feature vector in an independent characterization mode is extracted, which specifically includes:
Inputting the machine translation h and the artificial reference translation r into a cross-language pre-training model XLM-RoBERTa respectively, and performing sub-word segmentation on the machine translation h and the artificial reference translation r by using a sub-word segmentation method SENTENCEPIECE algorithm through the cross-language pre-training model XLM-RoBERTa to respectively obtain sub-word sequences comprising m sub-words and n sub-words:
；
Wherein m and n respectively represent the number of the subwords contained in the machine translation and the artificial reference translation after being segmented by using a subword segmentation method SENTENCEPIECE algorithm; h 1,h2,hm represents the 1 st sub word, the 2 nd sub word and the m th sub word after the machine translation is segmented by using the sub word segmentation method SENTENCEPIECE algorithm; r 1,r2,rn represents the 1 st sub word, the 2 nd sub word and the nth sub word after the manual reference translation is segmented;
Outputting a machine translation sub-word level feature vector in an independent characterization mode and a manual reference translation sub-word level feature vector in the independent characterization mode according to the positions of the sub-words and the sub-words in sentences by a cross-language pre-training model XLM-RoBERTa;
(1)；
(2)；
Wherein v h and v r respectively represent a machine translation and an artificial reference translation using a cross-language pre-training model XLM-RoBERTa to output a machine translation sub-word level feature vector in an independent characterization mode and an artificial reference translation sub-word level feature vector in an independent characterization mode, and XLM-RoBERTa ()' represents a cross-language pre-training model XLM-RoBERTa;
Alternatively, the cross-language pre-training model XLM-RoBERTa uses the basic model "XLM-roberta-large" model therein, has 24 transducer encoder hidden layers and 16 self-attention heads, and outputs 1024-dimensional vectors for each subword.
And the machine translation sub-word level feature vector in the independent characterization mode and the manual reference translation sub-word level feature vector in the independent characterization mode are interacted by using an external attention mechanism to obtain a machine translation interaction feature vector in the independent characterization mode and a manual reference translation interaction feature vector in the independent characterization mode:
(3)；
(4)；
Wherein, For machine translation interaction feature vectors in the independent characterization mode,Manually referencing the translation interaction feature vector in the independent characterization mode; multiHead () represents a multi-headed attention mechanism function, which contains three parameters, query, key and value, which is converted into an external attention mechanism when the query is not identical to the key and value;
Connecting the machine translation interaction feature vector in the independent characterization mode with the manual reference translation interaction feature vector in the independent characterization mode, carrying out average pooling operation, and outputting the sentence-level machine translation quality feature vector in the independent characterization mode:
(5)；
Wherein vs hr represents the sentence-level machine translation quality feature vector in the independent characterization mode; avgPooling () represents the average pooling operation.
Further, in step S3, a sentence-level machine translation quality feature vector in a unified characterization mode is extracted, which specifically includes:
and (3) carrying out character string connection on the machine translation h and the artificial reference translation r to obtain a translation joint character string:
(6)；
Wherein hr is a translation joint string, "</s >" represents the start and stop characters of the string;
the translation joint strings of the training sample example in the attached table 1 are:
</s>This document is proposed by the Ministry of Industry and Information Technology.</s></s>TheMinistry of Industry and Information Technology is responsible for the proposal and administration of this document.</s>;
Inputting the translation joint character string into a cross-language pre-training model XLM-RoBERTa, and performing sub-word segmentation on the translation joint character string by using a sub-word segmentation method SENTENCEPIECE algorithm through the cross-language pre-training model XLM-RoBERTa to obtain a sub-word sequence containing p sub-words:
(7)；
Wherein, p represents the number of the subwords contained in the translation joint character string after the subwords are segmented by using a subword segmentation method SENTENCEPIECE algorithm; hr 1,hr2,hrp represents the 1 st subword, the 2 nd subword and the p th subword of the translation joint string after being segmented by using the SENTENCEPIECE algorithm;
outputting a subword level feature vector in a unified characterization mode according to the subwords and the positions of the subwords in sentences by the cross-language pre-training model XLM-RoBERTa;
(8)；
Wherein v hr represents a subword level feature vector in unified characterization mode, XLM-RoBERTa ()' represents a cross-language pre-training model XLM-RoBERTa;
Interaction is carried out on the subword level feature vectors in the unified characterization mode by using a self-attention mechanism to obtain interaction feature vectors in the unified characterization mode:
(9)；
Wherein, Representing interaction feature vectors in a unified characterization mode, multiHead ()' representing a multi-head attention mechanism function, wherein the multi-head attention mechanism function comprises three parameters of query, key and value, and the multi-head attention mechanism is converted into a self-attention mechanism when the query is identical to the key and the value;
and carrying out average pooling operation on the interaction feature vectors in the unified characterization mode, and outputting sentence-level machine translation quality feature vectors in the unified characterization mode:
(10)；
Where vu hr represents the sentence-level machine translation quality feature vector in the unified characterization mode.
Further, in step S4, a machine translation quality cross feature vector is extracted, which specifically includes:
Splicing the sentence-level machine translation quality feature vector in the independent characterization mode in the step S2 and the sentence-level machine translation quality feature vector in the unified characterization mode in the step S3:
(11)；
Wherein x 0 represents a machine translation splice feature vector, and the symbol 'three' represents a vector splice operation;
Inputting the machine translation splicing feature vector into a depth intersection network containing 4 stacked intersection layers, and outputting a machine translation quality intersection feature vector:
(12)；
(13)；
(14)；
(15)；
wherein the symbol is' "Is Hadamard product operation, W 1,W2,W3,W4 and b 1,b2,b3,b4 are the linear weight parameters and bias of the 1 st, 2 nd, 3 rd and 4 th stacked cross layers respectively; x 1,x2,x3,x4 is the cross feature vector output by the 1 st, 2 nd, 3 rd and 4 th stacked cross layers respectively, and x 4 is taken as the machine translation quality cross feature vector; it should be noted that the deep crossover network may include a plurality of stacked crossover layers, and the present patent uses 4 stacked crossover layers obtained according to a great deal of experimental experience.
Further, in step S5, the quality of the machine translation is predicted, specifically:
inputting the machine translation quality cross feature vector in the step S4 into a three-layer feedforward neural network, and outputting a machine translation quality score;
(16)；
Wherein Score is the machine translation quality Score, and Feed-Forward is a three-layer feedforward neural network.
Further, in step S6, the mean square error loss is shown in formula (17);
(17)；
where MSE represents the mean square error loss, N represents the number of samples in the training set, i represents the ith sample in the training set, y (i) represents the human evaluation Score of the machine translation of the ith sample in the training set, score (i) represents the machine translation quality Score of the prediction of the ith sample.
Further, in step S9, the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation based on the large language model vector library is calculated, specifically:
Inputting the normalized machine translation to be evaluated and the artificial reference translation of the machine translation to be evaluated in the step S7 into the large language model vector database Chromadb respectively, and directly outputting the machine translation sentence vector representation and the artificial reference translation sentence vector representation:
(18)；
(19)；
Wherein Chromadb zephyr-7b () represents a Chromadb vector database output function with the large language model zephyr-7b as the base large model; h, r respectively represent the machine translation to be evaluated and the manual reference translation of the machine translation to be evaluated; e h,Er represents the machine translation sentence vector representation and the manual reference translation sentence vector representation respectively;
calculating cosine similarity of the translation sentence vector representation of the machine and the artificial reference translation sentence vector representation:
(20)；
Wherein CosSim (h, r) represents the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation.
Further, in step S10, a final prediction score of the machine translation quality is calculated, specifically:
The machine translation quality score predicted in the step S8 and the cosine similarity of the machine translation sentence vector representation and the artificial reference translation sentence vector representation based on the large language model vector library in the step S9 are linearly weighted:
（21）；
Wherein, Representing a final predictive value of machine translation quality; score represents the machine translation quality Score predicted in step S8; and 0.8 is a linear interpolation weight, and is obtained according to a large amount of experimental experience.
The machine translation automatic evaluation method DCN-MTE based on the depth cross network is tested on the news field dataset Newstest with the tasks of Deying, zhongying and England directions for automatic evaluation and evaluation of the machine translation of the international machine translation conference at the 7 th time. The automatic machine translation evaluation method BLEU, the automatic machine translation evaluation method chrF, the automatic machine translation evaluation method YiSi-1, the automatic machine translation evaluation method BLEURT-20, the automatic machine translation evaluation method UNITE, the automatic machine translation evaluation method BERTScore, the automatic machine translation evaluation method MS-COMET-22, the automatic machine translation evaluation method COMET-22 and the like are used as comparison methods, wherein the automatic machine translation evaluation method COMET-22 is the performance optimal method participating in evaluation.
And in performance measurement, following the official practice in the automatic evaluation task of the machine translation of the 7 th international machine translation conference, respectively using different methods of Kendel correlation coefficient and Pelson correlation coefficient evaluation to evaluate the correlation with human evaluation on sentence level and system level, wherein the larger the Kendel correlation coefficient or the Pelson correlation coefficient is, the better the automatic evaluation effect of the machine translation is.
Table 2: and automatically evaluating and evaluating sentence level correlation and system level correlation of different machine translations in the directions of task Deying, zhongying and Ying in the 7 th international machine translation conference machine translation with human evaluation.
The sentence level correlation and the system level correlation of the automatic evaluation method of the machine translations of the international machine translation conference with human evaluation in the directions of the automatic evaluation task of the machine translations of 7 th, the Chinese and the English are shown in the attached table 2. The data in Table 2 shows that the machine translation automatic evaluation method DCN-MTE based on the deep cross network is superior to the machine translation automatic evaluation methods BLEU, chrF, yiSi-1, BLEURT-20, UNITE, BERTScore, MS-COMET-22, the machine translation automatic evaluation method COMET-22 and the like in terms of the comprehensive values of sentence level correlation and system level correlation. The machine translation automatic evaluation method based on the depth cross network has the advantages that DCN-MTE is higher than the optimal system COMET-22 participating in evaluation by 0.002 on sentence level correlation and higher than the optimal system COMET-22 on system level correlation by 0.015.
The method for automatically evaluating the machine translation based on the depth intersection network can carry out depth interaction on the machine translation features and the manual reference translation features and capture semantic relations between the machine translation features and the manual reference translation features, and can consistently improve the effect of automatically evaluating the machine translation.
The methods of the present disclosure have general applicability because the methods of the present disclosure are not presented for two particular languages. Although the present disclosure has been experimentally verified in only three translation directions among german, chinese and english, the present disclosure is equally applicable to other language pairs such as chinese-japanese and chinese-vietnamese.
The protection of the present invention is not limited to the above embodiments. Variations and advantages that would occur to one skilled in the art are included in the invention without departing from the spirit and scope of the inventive concept, and the scope of the invention is defined by the appended claims.

Claims (8)

1.一种基于深度交叉网络的机器译文自动评价方法，其特征在于：步骤如下：1. A method for automatic evaluation of machine translation based on deep cross-network, characterized by the following steps: 步骤S1，获取训练集，对训练集进行规范化处理，获得规范化处理后的训练集；训练集由多个不同样本组成，每个样本包括机器译文、人工参考译文和机器译文的人类评价分值；Step S1, obtaining a training set, normalizing the training set, and obtaining a normalized training set; the training set is composed of a plurality of different samples, each sample including a machine translation, a manual reference translation, and a human evaluation score of the machine translation; 步骤S2，提取独立表征模式下句子级别机器译文质量特征向量；将规范化处理后的训练集内每个样本中机器译文和人工参考译文，分别输入到跨语言预训练模型，输出独立表征模式下机器译文子词级别特征向量和独立表征模式下人工参考译文子词级别特征向量，将独立表征模式下机器译文子词级别特征向量和独立表征模式下人工参考译文子词级别特征向量使用外部注意力机制进行交互得到独立表征模式下机器译文交互特征向量和独立表征模式下人工参考译文交互特征向量，连接独立表征模式下机器译文交互特征向量和独立表征模式下人工参考译文交互特征向量，并进行平均池化操作，输出独立表征模式下句子级别机器译文质量特征向量；Step S2, extracting sentence-level machine translation quality feature vectors under the independent representation mode; inputting the machine translation and manual reference translation in each sample in the training set after normalization into the cross-language pre-training model respectively, outputting the machine translation subword-level feature vector under the independent representation mode and the manual reference translation subword-level feature vector under the independent representation mode, using the external attention mechanism to interact the machine translation subword-level feature vector under the independent representation mode and the manual reference translation subword-level feature vector under the independent representation mode to obtain the machine translation interaction feature vector under the independent representation mode and the manual reference translation interaction feature vector under the independent representation mode, connecting the machine translation interaction feature vector under the independent representation mode and the manual reference translation interaction feature vector under the independent representation mode, and performing an average pooling operation, and outputting the sentence-level machine translation quality feature vector under the independent representation mode; 步骤S3，提取统一表征模式下句子级别机器译文质量特征向量；将规范化处理后的训练集内每个样本中机器译文和人工参考译文进行字符串相连得到译文联合字符串，将译文联合字符串输入到跨语言预训练模型，输出统一表征模式下子词级别特征向量，将统一表征模式下子词级别特征向量使用自注意力机制进行交互，得到统一表征模式下交互特征向量，将统一表征模式下交互特征向量进行平均池化操作，输出统一表征模式下句子级别机器译文质量特征向量；Step S3, extracting the sentence-level machine translation quality feature vector under the unified representation mode; connecting the machine translation and the manual reference translation in each sample in the training set after normalization to obtain a translation joint string, inputting the translation joint string into the cross-language pre-training model, outputting the subword-level feature vector under the unified representation mode, using the self-attention mechanism to interact the subword-level feature vectors under the unified representation mode to obtain the interaction feature vector under the unified representation mode, performing an average pooling operation on the interaction feature vector under the unified representation mode, and outputting the sentence-level machine translation quality feature vector under the unified representation mode; 步骤S4，提取机器译文质量交叉特征向量；将步骤S2中独立表征模式下句子级别机器译文质量特征向量和步骤S3中统一表征模式下句子级别机器译文质量特征向量拼接，输入到含有4个堆叠交叉层的深度交叉网络，输出机器译文质量交叉特征向量；Step S4, extracting a machine translation quality cross feature vector; concatenating the sentence-level machine translation quality feature vector under the independent representation mode in step S2 and the sentence-level machine translation quality feature vector under the unified representation mode in step S3, inputting them into a deep cross network containing 4 stacked cross layers, and outputting a machine translation quality cross feature vector; 步骤S5，预测机器译文质量；将步骤S4中机器译文质量交叉特征向量输入三层前馈神经网络，输出机器译文质量分值；Step S5, predicting the quality of machine translation; inputting the cross-feature vector of the machine translation quality in step S4 into a three-layer feedforward neural network, and outputting the machine translation quality score; 步骤S6，训练基于深度交叉网络的机器译文自动评价模型；根据步骤S5中输出的机器译文质量分值和步骤S1中规范化处理后的训练集内机器译文的人类评价分值，通过最小化在规范化处理后的训练集上的均方差损失来训练基于深度交叉网络的机器译文自动评价模型的参数，得到训练后的基于深度交叉网络的机器译文自动评价模型；Step S6, training a machine translation automatic evaluation model based on a deep cross network; according to the machine translation quality score output in step S5 and the human evaluation score of the machine translation in the training set after normalization in step S1, the parameters of the machine translation automatic evaluation model based on a deep cross network are trained by minimizing the mean square error loss on the training set after normalization, so as to obtain a trained machine translation automatic evaluation model based on a deep cross network; 步骤S2中提取独立表征模式下句子级别机器译文质量特征向量，具体为：In step S2, the sentence-level machine translation quality feature vector in the independent representation mode is extracted, which is specifically: 步骤S21，将机器译文h和人工参考译文r分别输入到跨语言预训练模型XLM-RoBERTa，由跨语言预训练模型XLM-RoBERTa使用子词切分方法SentencePiece算法对机器译文h和人工参考译文r进行子词切分，分别得到包含m个子词和n个子词的子词序列：Step S21, the machine translation h and the manual reference translation r are respectively input into the cross-language pre-training model XLM-RoBERTa, and the cross-language pre-training model XLM-RoBERTa uses the subword segmentation method SentencePiece algorithm to segment the machine translation h and the manual reference translation r into subwords, and obtain subword sequences containing m subwords and n subwords respectively: ； ; 其中，m和n分别表示机器译文和人工参考译文使用子词切分方法SentencePiece算法切分后包含的子词个数；h1，h2，hm表示机器译文使用子词切分方法SentencePiece算法切分后的第1个子词，第2个子词，第m个子词；r1，r2，rn表示人工参考译文切分后的第1个子词，第2个子词，第n个子词；Wherein, m and n represent the number of subwords contained in the machine translation and the manual reference translation respectively after segmentation using the subword segmentation method SentencePiece algorithm; h 1 , h 2 , h m represent the first subword, the second subword, and the mth subword of the machine translation after segmentation using the subword segmentation method SentencePiece algorithm; r 1 , r 2 , r n represent the first subword, the second subword, and the nth subword of the manual reference translation after segmentation; 步骤S22，跨语言预训练模型XLM-RoBERTa根据子词和子词在句子中的位置输出独立表征模式下机器译文子词级别特征向量和独立表征模式下人工参考译文子词级别特征向量，如公式（1）和公式（2）所示；Step S22, the cross-language pre-training model XLM-RoBERTa outputs a subword-level feature vector of a machine translation in an independent representation mode and a subword-level feature vector of a manual reference translation in an independent representation mode according to the subword and the position of the subword in the sentence, as shown in formula (1) and formula (2); (1)； (1); (2)； (2); 其中，vh和vr分别表示机器译文和人工参考译文使用跨语言预训练模型XLM-RoBERTa输出的独立表征模式下机器译文子词级别特征向量和独立表征模式下人工参考译文子词级别特征向量，XLM-RoBERTa(•)表示跨语言预训练模型XLM-RoBERTa；Wherein, v h and v r represent the machine translation subword-level feature vector and the manual reference translation subword-level feature vector under the independent representation mode output by the cross-language pre-training model XLM-RoBERTa, respectively. XLM-RoBERTa(•) represents the cross-language pre-training model XLM-RoBERTa; 步骤S23，将独立表征模式下机器译文子词级别特征向量和独立表征模式下人工参考译文子词级别特征向量,使用外部注意力机制进行交互，得到独立表征模式下机器译文交互特征向量和独立表征模式下人工参考译文交互特征向量，如公式（3）和公式（4）所示：Step S23, the sub-word level feature vector of the machine translation under the independent representation mode and the sub-word level feature vector of the manual reference translation under the independent representation mode are interacted using an external attention mechanism to obtain the interactive feature vector of the machine translation under the independent representation mode and the interactive feature vector of the manual reference translation under the independent representation mode, as shown in formula (3) and formula (4): (3)； (3); (4)； (4); 其中，为独立表征模式下机器译文交互特征向量，为独立表征模式下人工参考译文交互特征向量；MultiHead(•)表示多头注意力机制函数，多头注意力机制函数中包含查询、键和值三个参数，当查询与键和值不相同时多头注意力机制转化为外部注意力机制；in, is the interactive feature vector of machine translation in the independent representation mode, is the interactive feature vector of the artificial reference translation in the independent representation mode; MultiHead(•) represents the multi-head attention mechanism function, which contains three parameters: query, key and value. When the query is different from the key and value, the multi-head attention mechanism is transformed into an external attention mechanism; 步骤S24，连接独立表征模式下机器译文交互特征向量和独立表征模式下人工参考译文交互特征向量，并进行平均池化操作，输出独立表征模式下句子级别机器译文质量特征向量如公式（5）所示：Step S24, connect the interactive feature vector of the machine translation in the independent representation mode and the interactive feature vector of the manual reference translation in the independent representation mode, and perform an average pooling operation to output the sentence-level machine translation quality feature vector in the independent representation mode as shown in formula (5): (5)； (5); 其中，vshr表示独立表征模式下句子级别机器译文质量特征向量；AvgPooling()表示平均池化操作；Where vs hr represents the sentence-level machine translation quality feature vector in the independent representation mode; AvgPooling() represents the average pooling operation; 步骤S3中提取统一表征模式下句子级别机器译文质量特征向量，具体为：In step S3, the sentence-level machine translation quality feature vector under the unified representation mode is extracted, which is specifically: 步骤S31，将机器译文h和人工参考译文r进行字符串相连得到译文联合字符串，如公式（6）所示：Step S31, the machine translation h and the manual reference translation r are connected by string to obtain a translation joint string, as shown in formula (6): (6)； (6); 其中，hr为译文联合字符串，“</s>”表示字符串的起止符；Among them, hr is the translation combined string, and “</s>” indicates the start and end characters of the string; 步骤S32，将译文联合字符串输入跨语言预训练模型XLM-RoBERTa，由跨语言预训练模型XLM-RoBERTa使用子词切分方法SentencePiece算法对译文联合字符串进行子词切分，得到包含p个子词的子词序列，如公式（7）所示：Step S32, input the translation joint string into the cross-language pre-training model XLM-RoBERTa, and the cross-language pre-training model XLM-RoBERTa uses the subword segmentation method SentencePiece algorithm to segment the translation joint string into subwords, and obtain a subword sequence containing p subwords, as shown in formula (7): (7)； (7); 其中，p表示译文联合字符串使用子词切分方法SentencePiece算法子词切分后包含的子词个数；hr1，hr2，hrp表示译文联合字符串使用SentencePiece算法切分后的第1个子词，第2个子词，第p个子词；Wherein, p represents the number of subwords contained in the translation joint string after the subword segmentation method SentencePiece algorithm is used; hr 1 , hr 2 , hr p represent the first subword, the second subword, and the pth subword of the translation joint string after the SentencePiece algorithm is used; 步骤S33，跨语言预训练模型XLM-RoBERTa根据子词和子词在句子中的位置输出统一表征模式下子词级别特征向量，如公式（8）所示：Step S33, the cross-language pre-trained model XLM-RoBERTa outputs a subword-level feature vector under a unified representation mode according to the subword and the position of the subword in the sentence, as shown in formula (8): (8)； (8); 其中，vhr表示统一表征模式下子词级别特征向量；Among them, v hr represents the subword level feature vector under the unified representation mode; 步骤S34，将统一表征模式下子词级别特征向量使用自注意力机制进行交互得到统一表征模式下交互特征向量，如公式（9）所示：Step S34, the subword level feature vectors in the unified representation mode are interacted using the self-attention mechanism to obtain the interactive feature vector in the unified representation mode, as shown in formula (9): (9)； (9); 其中，表示统一表征模式下交互特征向量，MultiHead(•)表示多头注意力机制函数，多头注意力机制函数中包含查询、键和值三个参数，当查询与键和值相同时多头注意力机制转化为自注意力机制；in, represents the interactive feature vector in the unified representation mode, MultiHead(•) represents the multi-head attention mechanism function, which contains three parameters: query, key and value. When the query is the same as the key and value, the multi-head attention mechanism is transformed into a self-attention mechanism; 步骤S35，将统一表征模式下交互特征向量进行平均池化操作，输出统一表征模式下句子级别机器译文质量特征向量，如公式（10）所示：Step S35, average pooling operation is performed on the interactive feature vector under the unified representation mode, and the sentence-level machine translation quality feature vector under the unified representation mode is output, as shown in formula (10): (10)； (10); 其中，vuhr表示统一表征模式下句子级别机器译文质量特征向量。Among them, vu hr represents the sentence-level machine translation quality feature vector under the unified representation mode. 2.根据权利要求1所述的一种基于深度交叉网络的机器译文自动评价方法，其特征在于：步骤S1中训练集由多个不同样本组成，每个样本具体为：2. According to the method for automatic evaluation of machine translation based on deep cross network in claim 1, it is characterized in that: the training set in step S1 is composed of a plurality of different samples, each sample is specifically: 给定训练集中一个样本d={(h, r), y}，其中d表示一个训练样本， h表示机器译文，r表示人工参考译文， y表示对机器译文h的人类评价分值，人类评价分值介入0-1之间的实数值。Given a sample d={(h, r), y} in the training set, d represents a training sample, h represents the machine translation, r represents the manual reference translation, and y represents the human evaluation score of the machine translation h. The human evaluation score is a real value between 0 and 1. 3.根据权利要求2所述的一种基于深度交叉网络的机器译文自动评价方法，其特征在于：步骤S4中提取机器译文质量交叉特征向量，具体为：3. According to the method for automatic evaluation of machine translation based on deep cross network in claim 2, it is characterized in that: in step S4, the cross feature vector of machine translation quality is extracted, specifically: 步骤S41，将步骤S2中独立表征模式下句子级别机器译文质量特征向量和步骤S3中统一表征模式下句子级别机器译文质量特征向量拼接，如公式（11）所示：Step S41, concatenate the sentence-level machine translation quality feature vector under the independent representation mode in step S2 and the sentence-level machine translation quality feature vector under the unified representation mode in step S3, as shown in formula (11): (11)； (11); 其中，x0表示机器译文拼接特征向量，符号“⊕”表示向量拼接运算；Among them, x 0 represents the machine translation concatenation feature vector, and the symbol “⊕” represents the vector concatenation operation; 步骤S42，将机器译文拼接特征向量输入含有4个堆叠交叉层的深度交叉网络，输出机器译文质量交叉特征向量，如公式（12）、公式（13）、公式（14）和公式（15）所示：Step S42, input the machine translation concatenation feature vector into a deep cross network containing 4 stacked cross layers, and output the machine translation quality cross feature vector, as shown in formula (12), formula (13), formula (14) and formula (15): (12)； (12); (13)； (13); (14)； (14); (15)； (15); 其中，x1，x2，x3，x4分别为第1个、第2个、第3个、第4个堆叠交叉层输出的交叉特征向量，将x4作为机器译文质量交叉特征向量；符号“”为哈达玛乘积运算，W1，W2，W3，W4和b1，b2，b3，b4分别为第1个、第2个、第3个、第4个堆叠交叉层的线性权值参数和偏置；需要说明的是深度交叉网络包含若干个堆叠交叉层。Among them, x 1 , x 2 , x 3 , x 4 are the cross feature vectors output by the first, second, third, and fourth stacked cross layers respectively, and x 4 is taken as the cross feature vector of machine translation quality; the symbol “ ” is the Hadamard product operation, W 1 , W 2 , W 3 , W 4 and b 1 , b 2 , b 3 , b 4 are the linear weight parameters and biases of the first, second, third and fourth stacked cross layers respectively; it should be noted that the deep cross network contains several stacked cross layers. 4.根据权利要求3所述的一种基于深度交叉网络的机器译文自动评价方法，其特征在于：步骤S5中预测机器译文质量，具体为：4. According to the method of automatic evaluation of machine translation based on deep cross network in claim 3, it is characterized in that: the quality of machine translation is predicted in step S5, specifically: 将步骤S4中机器译文质量交叉特征向量输入三层前馈神经网络，输出机器译文质量分值；如公式（16）所示：The machine translation quality cross feature vector in step S4 is input into the three-layer feedforward neural network, and the machine translation quality score is output; as shown in formula (16): (16)； (16); 其中，Score为机器译文质量分值，Feed-Forward为三层前馈神经网络。Among them, Score is the quality score of the machine translation, and Feed-Forward is a three-layer feedforward neural network. 5.根据权利要求4所述的一种基于深度交叉网络的机器译文自动评价方法，其特征在于：步骤S6中均方差损失，如公式(17)所示；5. The method for automatic evaluation of machine translation based on deep cross network according to claim 4, characterized in that: the mean square error loss in step S6 is as shown in formula (17); (17)； (17); 其中，MSE表示均方差损失，N表示训练集中样本的数量，i表示训练集中第i条样本，y(i)表示训练集中第i条样本机器译文的人类评价分值，Score(i)表示第i条样本预测的机器译文质量分值。Where MSE represents mean square error loss, N represents the number of samples in the training set, i represents the i-th sample in the training set, y (i) represents the human evaluation score of the machine translation of the i-th sample in the training set, and Score (i) represents the quality score of the machine translation predicted by the i-th sample. 6.根据权利要求5所述的一种基于深度交叉网络的机器译文自动评价方法，其特征在于：还包括以下步骤：6. According to claim 5, a method for automatic evaluation of machine translation based on deep cross network is characterized in that it also includes the following steps: 步骤S7，对待评价机器译文、待评价机器译文的人工参考译文进行规范化处理；Step S7, normalizing the machine translation to be evaluated and the manual reference translation of the machine translation to be evaluated; 步骤S8，将步骤S7中规范化处理后的待评价机器译文、待评价机器译文的人工参考译文输入至步骤S6中训练后的基于深度交叉网络的机器译文自动评价模型，预测机器译文质量分值；Step S8, inputting the machine translation to be evaluated after the normalization processing in step S7 and the manual reference translation of the machine translation to be evaluated into the machine translation automatic evaluation model based on the deep cross network trained in step S6 to predict the quality score of the machine translation; 步骤S9，计算基于大语言模型向量库的机器译文句向量表征和人工参考译文句向量表征余弦相似度；将步骤S7中规范化处理后的待评价机器译文、待评价机器译文的人工参考译文分别输入大语言模型向量库，直接输出机器译文句向量表征和人工参考译文句向量表征，计算机器译文句向量表征和人工参考译文句向量表征的余弦相似度；Step S9, calculating the cosine similarity between the machine translation sentence vector representation and the manual reference translation sentence vector representation based on the large language model vector library; inputting the machine translation to be evaluated and the manual reference translation of the machine translation to be evaluated after the normalization processing in step S7 into the large language model vector library respectively, directly outputting the machine translation sentence vector representation and the manual reference translation sentence vector representation, and calculating the cosine similarity between the machine translation sentence vector representation and the manual reference translation sentence vector representation; 步骤S10，计算机器译文质量最终预测分值；线性加权步骤S8中预测的机器译文质量分值和步骤S9中基于大语言模型向量库的机器译文句向量表征和人工参考译文句向量表征余弦相似度，获取机器译文质量最终预测分值。Step S10, calculating the final predicted score of the machine translation quality; linearly weighting the machine translation quality score predicted in step S8 and the cosine similarity of the machine translation sentence vector representation based on the large language model vector library and the artificial reference translation sentence vector representation in step S9 to obtain the final predicted score of the machine translation quality. 7.根据权利要求6所述的一种基于深度交叉网络的机器译文自动评价方法，其特征在于：步骤S9中计算基于大语言模型向量库的机器译文句向量表征和人工参考译文句向量表征余弦相似度，具体为：7. According to the method of automatic evaluation of machine translation based on deep cross network in claim 6, it is characterized in that: in step S9, the cosine similarity between the machine translation sentence vector representation based on the large language model vector library and the artificial reference translation sentence vector representation is calculated, specifically: 将步骤S7中规范化处理后的待评价机器译文、待评价机器译文的人工参考译文分别输入大语言模型向量数据库Chromadb，直接输出机器译文句向量表征和人工参考译文句向量表征，如公式(18)和公式(19)所示；The machine translation to be evaluated and the manual reference translation of the machine translation to be evaluated after the normalization processing in step S7 are respectively input into the large language model vector database Chromadb, and the machine translation sentence vector representation and the manual reference translation sentence vector representation are directly output, as shown in formula (18) and formula (19); (18)； (18); (19)； (19); 其中，Eh，Er分别表示机器译文句向量表征和人工参考译文句向量表征，Chromadbzephyr-7b()表示以大语言模型zephyr-7b为基座大模型的Chromadb向量数据库输出函数；Wherein, E h , E r represent the vector representation of machine translation sentence and the vector representation of manual reference translation sentence respectively, Chromadb zephyr-7b () represents the Chromadb vector database output function of the large model based on the large language model zephyr-7b; 计算机器译文句向量表征和人工参考译文句向量表征的余弦相似度，如公式(20)所示；Calculate the cosine similarity between the machine translation sentence vector representation and the manual reference translation sentence vector representation, as shown in formula (20); (20)； (20); 其中，CosSim(h, r)表示机器译文句向量表征和人工参考译文句向量表征的余弦相似度，T表示转置操作。Among them, CosSim(h, r) represents the cosine similarity between the machine translation sentence vector representation and the manual reference translation sentence vector representation, and T represents the transposition operation. 8.根据权利要求7所述的一种基于深度交叉网络的机器译文自动评价方法，其特征在于：步骤S10中计算机器译文质量最终预测分值，具体为：8. According to the method of automatic evaluation of machine translation based on deep cross network in claim 7, it is characterized in that: the final predicted score of the machine translation quality is calculated in step S10, specifically: 线性加权步骤S8中预测的机器译文质量分值和步骤S9中基于大语言模型向量库的机器译文句向量表征和人工参考译文句向量表征余弦相似度，如公式(21)所示；The machine translation quality score predicted in step S8 and the cosine similarity between the machine translation sentence vector representation based on the large language model vector library and the manual reference translation sentence vector representation in step S9 are linearly weighted, as shown in formula (21); （21）； (twenty one); 其中，表示机器译文质量最终预测分值；Score表示步骤S8中预测的机器译文质量分值；0.8为线性插值权重，根据实验经验取得。in, represents the final predicted score of the machine translation quality; Score represents the machine translation quality score predicted in step S8; 0.8 is the linear interpolation weight, which is obtained based on experimental experience.
CN202410872045.1A 2024-07-01 2024-07-01 An automatic evaluation method for machine translation based on deep cross-network Active CN118395996B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410872045.1A CN118395996B (en) 2024-07-01 2024-07-01 An automatic evaluation method for machine translation based on deep cross-network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410872045.1A CN118395996B (en) 2024-07-01 2024-07-01 An automatic evaluation method for machine translation based on deep cross-network

Publications (2)

Publication Number Publication Date
CN118395996A CN118395996A (en) 2024-07-26
CN118395996B true CN118395996B (en) 2024-08-30

Family

ID=91985421

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410872045.1A Active CN118395996B (en) 2024-07-01 2024-07-01 An automatic evaluation method for machine translation based on deep cross-network

Country Status (1)

Country Link
CN (1) CN118395996B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115437626A (en) * 2022-08-17 2022-12-06 北京航空航天大学 A method and device for automatically generating OCL sentences based on natural language
CN117910482A (en) * 2024-03-19 2024-04-19 江西师范大学 Automatic machine translation evaluation method based on depth difference characteristics

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111261146B (en) * 2020-01-16 2022-09-09 腾讯科技（深圳）有限公司 Speech recognition and model training method, device and computer readable storage medium
WO2022020467A1 (en) * 2020-07-22 2022-01-27 Unbabel, Inc. System and method for training multilingual machine translation evaluation models
CN115952808A (en) * 2022-12-20 2023-04-11 中国科学技术大学 Multilingual machine translation quality assessment method, model, equipment and storage medium
CN117034961B (en) * 2023-10-09 2023-12-19 武汉大学 A BERT-based quality assessment method for Chinese-French translation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115437626A (en) * 2022-08-17 2022-12-06 北京航空航天大学 A method and device for automatically generating OCL sentences based on natural language
CN117910482A (en) * 2024-03-19 2024-04-19 江西师范大学 Automatic machine translation evaluation method based on depth difference characteristics

Also Published As

Publication number Publication date
CN118395996A (en) 2024-07-26

Similar Documents

Publication Publication Date Title
US12254865B2 (en) Multi-dialect and multilingual speech recognition
Liu et al. Diffsinger: Singing voice synthesis via shallow diffusion mechanism
Li et al. Multi-Language Multi-Speaker Acoustic Modeling for LSTM-RNN Based Statistical Parametric Speech Synthesis.
US7103544B2 (en) Method and apparatus for predicting word error rates from text
CN111783477B (en) Voice translation method and system
CN117910482B (en) An automatic evaluation method for machine translation based on deep difference features
CN111710326A (en) English voice synthesis method and system, electronic equipment and storage medium
CN117043857A (en) Method, apparatus and computer program product for English pronunciation assessment
WO2019167296A1 (en) Device, method, and program for natural language processing
CN114169346B (en) A machine translation method using part-of-speech information
CN113539268A (en) End-to-end voice-to-text rare word optimization method
CN117292680A (en) Voice recognition method for power transmission operation detection based on small sample synthesis
Zhang et al. Dropdim: A regularization method for transformer networks
Bai et al. A universal bert-based front-end model for mandarin text-to-speech synthesis
Bai et al. Integrating knowledge into end-to-end speech recognition from external text-only data
CN118260711B (en) A multimodal emotion recognition method and device
CN118395996B (en) An automatic evaluation method for machine translation based on deep cross-network
CN117711386A (en) Training of voice recognition model, voice recognition method, device, equipment and medium
US20240153508A1 (en) End-to-End Speech Recognition Adapted for Multi-Speaker Applications
CN116958343A (en) Facial animation generation method, device, equipment, medium and program product
Tang et al. SQuAD-SRC: A Dataset for Multi-Accent Spoken Reading Comprehension.
CN115240712A (en) Multi-mode-based emotion classification method, device, equipment and storage medium
CN114708848A (en) Method and device for acquiring size of audio and video file
CN118410812B (en) Machine translation quality estimation method based on large language model
CN112989845A (en) Chapter-level neural machine translation method and system based on routing algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant