CN104699685B - Model modification device and method, data processing equipment and method, program - Google Patents

Abstract

The present application discloses a model update device and method, data processing device and method, which are used to update the target model in the multi-model system, wherein each model in the multi-model system is pre-trained in different ways for the training data set The obtained model, the model updating device includes: a pseudo-label acquisition unit, which uses a calibration model to process the data set to be tested, and uses the processed result as a pseudo-label; the first feature distribution acquisition unit obtains the data set to be tested based on the pseudo-label. Feature distribution; the second feature distribution acquisition unit obtains the feature distribution of the training data set based on the target model; the adjustment unit adjusts the feature space division of the target model based on the feature distribution of the training data set and the feature distribution of the test data set, so that The training data set and the test data set have similar distributions for the feature space division; and an update unit uses the training data set to update the target model based on the adjusted feature space division.

Description

Model update device and method, data processing device and method, program

technical field

The present application relates to the field of data processing, in particular to a model updating device and a model updating method for updating a model in data processing, and a data processing device and a data processing method using the model updating device and model updating method.

Background technique

With the progress of society and the rapid development of information technology, how to efficiently process massive amounts of information has become particularly important. With the help of computer technology, various data processing methods, such as entity relationship extraction, object recognition, data mining, etc., have become common processing methods. These treatments are usually based on one or more models. For example, in entity relationship extraction, machine learning-based methods extract features from relationship samples, then train models on relationship samples, and use the trained model to identify entity relationships during actual extraction. In other words, the used model is generally trained based on the training data set.

As mentioned above, using the trained model to process the test data set is actually based on the assumption that the training data set and the test data set are homogeneous. However, in many cases, this assumption is not fully established, resulting in the model obtained based on the training data set is not suitable for the test data set, especially when new data is generated, this problem is particularly prominent. Moreover, it takes a lot of time and manpower to label samples on new data, which makes updating the model more expensive.

Contents of the invention

A brief overview of the invention is given below in order to provide a basic understanding of some aspects of the invention. It should be understood that this summary is not an exhaustive overview of the invention. It is not intended to identify key or critical parts of the invention nor to delineate the scope of the invention. Its purpose is merely to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.

In view of the requirements described in the background section, the present invention focuses on how to update the model at a relatively low cost to make it close to real data, thereby improving the performance of the system. Aiming at this problem, the present invention proposes a model update device and method for updating the model by dividing the feature space based on pseudo-feedback, wherein the difference between the test data set and the training data set is used to adjust the The feature space partition of the model, and use the adjusted feature space partition to update the model.

According to one aspect of the present invention, a model update device for updating the target model in the multi-model system is provided, wherein each model in the multi-model system is a model obtained by pre-training in different ways for the training data set, the The model update device includes: a pseudo-label acquisition unit configured to use a model different from the target model in the multi-model system as a calibration model to process the data set to be tested, and use the processed result as a pseudo-label; the first feature distribution acquisition unit, The second feature distribution acquisition unit is configured to obtain the feature distribution of the training data set based on the target model; the adjustment unit is configured based on the feature distribution of the training data set and The feature distribution of the test data set is used to adjust the feature space division of the target model, so that the training data set and the test data set have similar distributions for the feature space division; and the update unit is configured to be based on the adjusted feature space division Use the training dataset to update the target model.

According to another aspect of the present invention, there is provided a data processing device using a multi-model system to process a data set to be tested, including the above-mentioned model updating device.

According to another aspect of the present invention, there is provided a model update method for updating the target model in the multi-model system, wherein each model in the multi-model system is a model obtained by pre-training in different ways for the training data set, The method for updating the model includes: using a model different from the target model in the multi-model system as a calibration model to process the data set to be tested, and using the processed result as a pseudo-label; obtaining the feature distribution of the data set to be tested based on the pseudo-label; The target model obtains the feature distribution of the training data set; based on the feature distribution of the training data set and the feature distribution of the test data set, the feature space division of the target model is adjusted, so that the training data set and the test data set have the same characteristics for the feature space division. similar distribution; and using the training dataset to update the target model based on the adjusted feature space partition.

According to another aspect of the present invention, there is provided a data processing method using a multi-model system to process a data set to be tested, including the above-mentioned model updating method.

According to other aspects of the present invention, corresponding computer program codes, computer-readable storage media and computer program products are also provided.

These and other advantages of the present invention will be more apparent through the following detailed description of preferred embodiments of the present invention with reference to the accompanying drawings.

Description of drawings

In order to further illustrate the above and other advantages and features of the present application, the specific implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings. The drawings are incorporated in and form a part of, together with the following detailed description. Elements having the same function and structure are denoted by the same reference numerals. It should be understood that these drawings depict only typical examples of the application and should not be considered as limiting the scope of the application. In the attached picture:

Fig. 1 is a structural block diagram showing a model updating device according to an embodiment of the present application;

FIG. 2 is a schematic diagram showing an example of a comparison between a feature distribution of a training data set and a feature distribution of a test data set according to an embodiment of the present application;

Fig. 3 is a structural block diagram showing an adjustment unit according to an embodiment of the present application;

Fig. 4 is a structural block diagram showing a data processing device according to an embodiment of the present application;

Fig. 5 is a structural block diagram showing a data processing device according to another embodiment of the present application;

Fig. 6 is a schematic diagram showing several tables collected in a specific example of entity relationship mining;

Fig. 7 is a flowchart illustrating a model updating method according to one embodiment of the present application;

Fig. 8 is a flow chart showing the adjustment steps in the model update method according to one embodiment of the present application; and

FIG. 9 is a block diagram of an exemplary structure of a general-purpose personal computer in which methods and/or apparatuses according to embodiments of the present invention can be implemented.

detailed description

Exemplary embodiments of the present invention will be described below with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in this specification. It should be understood, however, that in developing any such practical embodiment, many implementation-specific decisions must be made in order to achieve the developer's specific goals, such as meeting those constraints related to the system and business, and those Restrictions may vary from implementation to implementation. Moreover, it should also be understood that development work, while potentially complex and time-consuming, would at least be a routine undertaking for those skilled in the art having the benefit of this disclosure.

Here, it should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only the device structure and/or processing steps closely related to the solution according to the present invention are shown in the drawings, and the Other details not relevant to the present invention are described.

The following description proceeds in the following order:

1. Model update device

2. Data processing device

3. Model update method and data processing method

4. Computing equipment for implementing the apparatus and method of the present application

[1. Model update device]

First, the configuration and structure of a model updating apparatus 100 according to an embodiment of the present application will be described with reference to FIG. 1 . The model updating device 100 is used to update the target model in the multi-model system, wherein each model in the multi-model system is a model pre-trained in different ways for the training data set. These models alone and in combination are used to perform detection on the test data. The model updating device 100 according to this embodiment can update each of multiple models or at least a part of the multiple models based on pseudo-feedback, thereby improving the performance of each model, and then making the model obtained by merging each model The performance is better and the detection effect is better. Hereinafter, the model to be updated is sometimes referred to as a target model.

As shown in Figure 1, the model update device 100 includes: a pseudo-label acquisition unit 101 configured to use a model different from the target model in the multi-model system as a calibration model to process the data set to be tested, and use the processed result as a pseudo-label The first feature distribution acquisition unit 102 is configured to obtain the feature distribution of the data set to be tested based on the pseudo-label; the second feature distribution acquisition unit 103 is configured to obtain the feature distribution of the training data set based on the target model; the adjustment unit 104, configured to adjust the feature space division of the target model based on the feature distribution of the training data set and the feature distribution of the test data set, so that the training data set and the test data set have similar distributions for the feature space division; and the update unit 105, configured to use the training data set to update the target model based on the adjusted feature space partition.

In the model training phase, use the existing training data set to train multiple models in different ways. For example, the plurality of models includes models obtained based on one or more of the following methods: Support Vector Machine (SVM), Random Forest, Decision Tree, K-Nearest Neighbor (KNN) algorithm, Boosting algorithm, Naive Bayes Algorithm and Maximum Entropy Algorithm. It should be noted that the above-mentioned algorithms given here are only examples, and besides that, models can also be obtained based on other various algorithms.

At the same time, during the model training process, the performance of each model on the training set can also be obtained. The above-mentioned multi-model system can target various applications, such as entity relationship extraction, topic division, object recognition, and so on. A well-trained multi-model system is suitable for a test data set that is homogeneous to the training data set, but the test data set is often different from the training data set, so that when the trained multi-model system processes the test data set Difficult to obtain accurate results.

Therefore, the model updating apparatus 100 shown in FIG. 1 can be used to update each model in the multi-model system based on the data set to be tested. Specifically, one model in the multi-model system is first selected as the target model, which represents the model to be updated or adjusted. Then, a calibration model for adjusting the target model based on its processing results is selected from the remaining models.

As an example, a multi-model system includes maximum entropy model, SVM, random forest, and KNN. Suppose you want to adjust the maximum entropy model to improve system performance, then the maximum entropy model is the target model, and the remaining SVM, random forest, and KNN models are all Can be used as a candidate for the calibration model.

In one embodiment, the calibration model may be the best performing model other than the target model in the multi-model system. For example, in the above example, SVM has the best performance among SVM, random forest and KNN, so you can choose SVM as the calibration model.

Alternatively, a model obtained using an algorithm that is complementary in principle to the target model can be selected as the calibration model. The complementary in principle mentioned here means that the principles on which the algorithms are based or the information of the data used by the algorithms are substantially complementary to each other. For example, traditional production models such as Bayesian networks and maximum entropy algorithms tend to represent the distribution of data from a statistical point of view, which can reflect the similarity of the same type of data itself, while traditional discriminative models such as SVM and KNN look for differences between different categories. The optimal classification surface among them reflects the differences among heterogeneous data. Therefore, from a certain point of view, it can be considered that the models obtained by these two algorithms are complementary. On the contrary, since random forest is an upgrading algorithm of decision tree, the two are based on the same principle, so they are not complementary.

In addition, the calibration model can also be a model obtained by fusing other models than the target model in the multi-model system. The way of fusion will be described in detail later.

After the calibration model is selected, the pseudo-label acquisition unit 101 uses the calibration model to process the data set to be tested, and uses the processed result as a pseudo-label. Next, this pseudo-label is used as feedback to adjust the feature space division of the target model, where the feature space represents the range of feature values.

Specifically, the first feature distribution obtaining unit 102 takes the obtained pseudo-label as a correct processing result, and obtains the feature distribution of the data set to be tested based on the pseudo-label. Similarly, the second feature distribution obtaining unit 103 obtains the feature distribution of the training data set based on the target model.

FIG. 2 is a graph showing an example of a comparison between the feature distribution of the test data set and the feature distribution of the training data set according to an embodiment of the present application. As shown in Figure 2, the horizontal axis represents the feature value, the vertical axis represents the category of the sample, the solid circles and triangles above represent the pseudo-label-based feature distribution of the data set to be tested, and the hollow circles and triangles below represent the training data set In addition, the dots and triangles represent two kinds of sample categories, which are determined according to the pseudo-label for the test data set, and determined based on the target model for the training data set.

It can be seen that the sample distribution of the test data set and the training data set are different. Next, the adjustment unit 104 adjusts the feature space division of the target model based on this difference, so that the training data set and the test data set have similar distributions for the feature space division. In other words, the adjustment unit 104 adjusts the target model by using the characteristics of the data of the test data set, so that the adjusted target model takes into account the difference between the test data set and the training data set, so that the processing result is more accurate. Improved performance.

Then, the updating unit 105 uses the training data set to retrain on the adjusted feature space division to adjust parameters, so as to obtain an updated model. In one embodiment, the model updating apparatus 100 may be configured to update periodically, but it is not limited thereto, and the time interval for updating or the way of starting the update may be specifically selected according to the application scenario and the processing capability of the device. In addition, the model updating device 100 can update a part or all of the models in the multi-model system.

The structure and configuration of the adjustment unit 104 according to an embodiment of the present application will be described below with reference to FIG. 3 . The adjustment unit 104 shown in FIG. 3 includes: a partition module 4001, configured to divide the feature space of the training data set and the data set to be tested into multiple regions; a distribution calculation module 4002, configured to distribution and the feature distribution of the data set to be tested, calculate the distribution of one or more adjacent areas on the training data set and the data set to be tested; the distance calculation module 4003 is configured to calculate the adjacent one or more a distance between the two distributions of regions; and a merging module 4004 configured to merge the one or more adjacent regions as a division of the feature space when the distance is less than a predetermined threshold.

Still taking FIG. 2 as an example, the partition module 4001 divides the feature space represented by the horizontal axis into multiple regions. For example, if the feature space is [0,1], the feature space can be divided into 100 regions, where the number of divided regions determines the granularity of adjustment, the more the number, the finer the granularity.

Then, the distribution calculation module 4002 is based on the feature distribution of the training data set and the feature distribution of the test data set, starting from the first area to calculate the adjacent one or more areas on the training data set and the test data set distributed. This is equivalent to a sliding window, and the size of each area is equivalent to the sliding step. Hereinafter, one or more areas in the window are called an interval. The distance calculation module 4003 calculates the distance between the two distributions calculated by the distribution calculation module 4002, and when the distance is smaller than a predetermined threshold, the determination module 4004 combines one or more corresponding regions as a division of the feature space. Repeatedly, the entire feature space can be re-divided.

Among them, the distance calculation module 4003 can calculate the distance between distributions through various methods, such as chi-square distance, KL distance, Resistor distance, Topsoe distance and so on. Taking the KL distance as an example, it can be calculated according to the following formula (1):

Among them, P _tr represents the probability distribution of the training dataset, P _ts represents the probability distribution of the test data set, _fi represents the feature value in the interval in question, and y represents the set of sample categories. As shown in Figure 2, the curve of KL distance can be obtained based on formula (1), the value of the corresponding eigenvalue at the peak of the curve is the position to be divided, and the area between the two peaks is merged into a division of the feature space .

In the case that the data on a certain interval in the data set to be tested is missing, that is, there is no data sample in the eigenvalue interval in the data set to be tested, the distance calculation module 4003 uses the Bayesian metric to calculate the above-mentioned adjacent one or more The distribution of regions (i.e. intervals) on the training dataset is used as the distance. This is because it is expected to still use the original feature space division method due to the lack of corresponding data on the data set to be tested. The formula of the Bayesian measure is shown in the following formula (2).

Among them, y _j represents the jth sample category, y _k represents the kth sample category, and there are m sample categories in total, and f represents the feature value.

The pseudo code of the above algorithm is given below as an example, and it should be understood that this is only for the purpose of illustration and is not intended to be limiting.

initialization:

Set the step size T for interval merging

Set the value range of the feature [MINB,MAXB], set lowb=MINB,

upb=lowb+T

Set the Bayesian threshold θb, the Bayesian distribution Bp of the previous interval and the distribution Bc of the current interval

Set the threshold θd of the KL distance, the KL value Dp of the previous interval and the KL value Dc of the current interval

While upb<MAXB

Calculate Bp,Dp

upb=lowb+T

Calculate Bc,Dc

if(|Bc-Bp|>θb or |Dc-Dp|>θd)

divide [lowb,upb-T)

lowb=upb-T

else

upb+=T

end if

end while

In the above pseudocode, the step size T of interval merging is equivalent to the size of each region of the feature space, and its size determines the granularity of adjustment, which can be determined according to actual applications or empirical values. In addition, although the above shows the calculation method of using the Bayesian metric to calculate the distribution on the interval when the data of a certain interval in the data set to be tested is missing, it is not limited to this, and other strategies can also be used, such as Assign a fixed value, etc.

As mentioned above, the model updating device 100 processes the data set to be tested by using a calibration model different from the target model in the multi-model system, and obtains the difference between the data set to be tested and the training data set based on the processing result, and according to the difference Adjusting the division of the feature space of the target model makes the target model closer to the data set to be tested, that is, the target model has a higher degree of correct processing of the data set to be tested, thereby improving its performance.

[2. Data processing device]

FIG. 4 shows the configuration of a data processing device 200 according to an embodiment of the present application. The data processing device 200 uses a multi-model system to process the data set to be tested, including the model updating device 100 as described above. Wherein, in order not to obscure the spirit and scope of the present application, FIG. 4 omits other common components that may be included in the data processing device 200, such as input and output interfaces, display units, and the like. In addition, since the structure and configuration of the model updating apparatus 100 have been described in detail above, details will not be repeated here.

By including the model update device 100, the data processing device 200 can obtain better performance when processing the test data set including new data.

5 is a structural block diagram of a data processing device 300 according to another embodiment of the present application. In addition to the model updating device 100, the data processing device 300 also includes: a control unit 301 configured to perform control so that the model updating device 100 updating each model in the multi-model system as a target model; and a fusion unit 302 configured to fuse the updated models to obtain a final model, wherein the data processing device 300 uses the final model to be tested for the data set to process.

In one embodiment, the following formula (3) can be used to fuse the models:

Among them, there are K models in the multi-model system, P _i is the output value of the fused model on sample i, P _ik is the output value of the kth model on sample i, and α is the weight value.

It can be understood that the data processing device 300 can further improve its performance by using the final model obtained by merging each updated model to process the data to be tested.

The operation of the data processing apparatus 300 is described below through a specific example. In this example, it is necessary to judge the affiliation of two entities from the network, that is, the author and the article. Due to the influence of the author's duplicate name, change of unit, change of research direction, reliability of information source, and inconsistency of multiple information sources, this judgment It's not easy. In other words, this example involves entity relationship mining. However, it should be understood that the scope of application of the data processing device 300 of the present application is not limited thereto, but can be widely applied to various data processing occasions using a multi-model system.

Make the collected information into several tables, as shown in Figure 6, including author table, article table, conference table, journal table, article-author table, each table includes different fields. On the basis of these tables, a batch of labeled items in the format of "author ID, article ID, yes/no" is obtained by manual labeling to indicate the affiliation relationship between author ID and article ID.

Assume that the multi-model system used includes five classifiers (models) including random forest, SVM, Bayesian network, boosting, and maximum entropy. These classifiers are first trained on the training data set to get the performance of each classifier.

For example, first select the maximum entropy as the target model, then random forest, SVM, Bayesian network, and boosting are candidate calibration models. Next, the SVM is selected as the calibration model, for example, according to a performance-optimized strategy. The pseudo-label acquisition unit 101 uses the selected SVM to classify the data set to be tested to obtain the pseudo-label information of the data set to be tested, that is, "author ID, article ID, yes/no".

For a feature that is a continuous value between [0,1], such as the name similarity feature indicating the degree of similarity between two names, the size of the selected area, such as 0.01, the partition module 4001 divides the feature space into 100 area. Starting from 0, for one or more adjacent areas, the distribution calculation module 4002 calculates its distribution on the training data set and the data set to be tested, and the distance calculation module 4003 calculates the training data set and the data set to be tested according to the distribution The distribution distance between them, such as the KL distance. If the KL distance difference is smaller than a predetermined threshold in two or more regions, the merging module 4004 merges these regions, and uses the final merging result as a new feature space division.

As an example, for example, the feature space division of the name similarity feature of the maximum entropy model is [0,0.2), [0.2,0.4), [0.4,0.6), [0.6,0.8) and [0.8,1 ], and [0,0.3), [0.3,0.5), [0.5,0.8) and [0.8,1) after the above repartition. The maximum entropy model is then retrained on the training dataset using this feature space partition.

If there are multiple features, all the feature spaces can be re-divided according to the same steps, and the update unit 105 uses the training data set to retrain the maximum entropy classifier in the new feature space.

Next, the control unit 301 may select other classifiers as target models, and repeat the above operations until all target models are updated. It should be understood that although the case of updating all target models is shown here as an example, this is not necessarily the case, and only one or several models may be updated.

Finally, the fusion unit 302 fuses all the updated models to obtain the final model. In the case that only one or several models are updated, the fusion unit 302 uses them as they are during fusion for other models that have not been updated.

Experiments have proved that by using the updated single model or the final fusion model to process the test data set, the classification results with higher accuracy can be obtained than the corresponding model before the update, that is, the system performance is improved.

[3. Model update method and data processing method]

The implementations of the model updating device and the data processing device according to the present invention are described above with reference to the accompanying drawings, and a model updating method and data processing method are actually described in the process. The method will be briefly described below with reference to Figs. 7 and 8, details of which can be referred to the previous descriptions of the model updating device and the data processing device.

Fig. 7 shows a model update method for updating a target model in a multi-model system according to an embodiment of the present application, wherein each model in the multi-model system is a model obtained by pre-training in different ways for the training data set, The method for updating the model includes: using a model different from the target model in the multi-model system as a calibration model to process the data set to be tested, and using the processed result as a pseudo-label (S11); obtaining the feature distribution of the data set to be tested based on the pseudo-label (S12); obtain the feature distribution of the training data set based on the target model (S13); adjust the feature space division of the target model based on the feature distribution of the training data set and the feature distribution of the test data set, so that the training data set and the test data set The dataset has a similar distribution for the feature space partition (S14); and the target model is updated using the training dataset based on the adjusted feature space partition (S15).

Wherein, the calibration model may be the model with the best performance other than the target model in the multi-model system.

In addition, the calibration model and the target model can be respectively obtained by using complementary algorithms in principle.

In an embodiment, the calibration model may also be a model obtained by fusing other models than the target model in the multi-model system.

Wherein, a plurality of models may include models obtained based on one or more of the following methods: support vector machine, random forest, decision tree, K nearest neighbor node algorithm, Boosting algorithm, naive Bayesian algorithm and maximum entropy algorithm. It should be understood that the category of models is not limited thereto. The multi-model system described above can be used in various applications, including but not limited to entity-relationship extraction.

In an embodiment, the above-mentioned model updating method may be performed periodically. The above-mentioned model updating method can also be executed according to requirements.

Fig. 8 shows the flowchart of the sub-steps of step S14. Step S14 includes: dividing the feature space of the training data set and the test data set into multiple regions (S401); The feature distribution of the data set, calculating the distribution of one or more adjacent areas on the training data set and the test data set (S402); calculating the distance between the two distributions of the adjacent one or more areas , and when the distance is less than a predetermined threshold, the one or more adjacent regions are combined as a division of the feature space (S403).

Wherein, the above distance may be KL distance.

In one embodiment, a Bayesian metric may be used to calculate the distribution of one or more adjacent regions on the training data set as the distance.

As mentioned above, the present application also provides a data processing method using a multi-model system to process a data set to be tested, including the model updating method described above with reference to FIG. 7 and FIG. 8 .

In one embodiment, the data processing method further includes: updating each model in the multi-model system as a target model; and merging the updated models to obtain a final model, and using the final model to The data set to be tested is processed.

Relevant details in the above embodiments have been given in detail in the description of the model updating device and the data processing device, and will not be repeated here.

[3. Computing equipment for implementing the device and method of the present application] Each component module and unit in the above device can be configured by means of software, firmware, hardware or a combination thereof. Specific means or manners that can be used for configuration are well known to those skilled in the art, and will not be repeated here. In the case of realizing by software or firmware, the program constituting the software is installed from a storage medium or a network to a computer having a dedicated hardware configuration (for example, a general-purpose computer 900 shown in FIG. 9 ), and when the computer is installed with various programs, Capable of performing various functions, etc.

In FIG. 9 , a central processing unit (CPU) 901 executes various processes according to programs stored in a read only memory (ROM) 902 or programs loaded from a storage section 908 to a random access memory (RAM) 903 . In the RAM 903, data required when the CPU 901 executes various processes and the like is also stored as necessary. The CPU 901 , ROM 902 , and RAM 903 are connected to each other via a bus 904 . An input/output interface 905 is also connected to the bus 904 .

The following components are connected to the input/output interface 905: an input section 906 (including a keyboard, a mouse, etc.), an output section 907 (including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), A storage part 908 (including a hard disk, etc.), a communication part 909 (including a network interface card such as a LAN card, a modem, etc.). The communication section 909 performs communication processing via a network such as the Internet. A driver 910 may also be connected to the input/output interface 905 as needed. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 910 as necessary, so that a computer program read therefrom is installed into the storage section 908 as necessary.

In the case of realizing the above-described series of processing by software, the programs constituting the software are installed from a network such as the Internet or a storage medium such as the removable medium 911 .

Those skilled in the art should understand that such a storage medium is not limited to the removable medium 911 shown in FIG. 9 in which the program is stored and distributed separately from the device to provide the program to the user. Examples of the removable media 911 include magnetic disks (including floppy disks (registered trademark)), optical disks (including compact disk read-only memory (CD-ROM) and digital versatile disk (DVD)), magneto-optical disks (including trademark)) and semiconductor memory. Alternatively, the storage medium may be a ROM 1402, a hard disk contained in the storage section 1408, or the like, in which the programs are stored and distributed to users together with devices containing them.

The invention also proposes a program product storing machine-readable instruction codes. When the instruction code is read and executed by a machine, the above-mentioned method according to the embodiment of the present invention can be executed.

Correspondingly, a storage medium for carrying the program product storing the above-mentioned machine-readable instruction codes is also included in the disclosure of the present invention. The storage medium includes, but is not limited to, a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, and the like.

Finally, it should also be noted that the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, but also Other elements not expressly listed, or inherent to the process, method, article, or apparatus are also included. Furthermore, without further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising said element .

Although the embodiments of the present invention have been described in detail above with reference to the accompanying drawings, it should be understood that the above-described embodiments are only used to illustrate the present invention, rather than to limit the present invention. Various modifications and changes can be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the present invention. Accordingly, the scope of the present invention is limited only by the appended claims and their equivalents.

Through the above description, the embodiments of the present invention provide the following technical solutions, but are not limited thereto.

Additional Note 1. A model update device for updating the target model in the multi-model system, wherein each model in the multi-model system is a model obtained by pre-training in different ways for the training data set, and the model update device includes :

A pseudo-label acquisition unit configured to use a model different from the target model in the multi-model system as a calibration model to process the data set to be tested, and use the processed result as a pseudo-label;

a first feature distribution acquisition unit configured to obtain a feature distribution of the data set to be tested based on the pseudo-label;

a second feature distribution acquisition unit configured to acquire the feature distribution of the training data set based on the target model;

An adjustment unit configured to adjust the feature space division of the target model based on the feature distribution of the training data set and the feature distribution of the test data set, so that the training data set and the test data set has a similar distribution for this feature space partition; and

An update unit configured to use the training data set to update the target model based on the adjusted feature space partition.

Supplement 2. The model updating device according to Supplement 1, wherein the adjustment unit includes:

a partition module configured to divide the feature space of the training data set and the test data set into multiple regions;

The distribution calculation module is configured to calculate the distribution of adjacent one or more regions in the training data set and the test data based on the feature distribution of the training data set and the feature distribution of the test data set, respectively. distribution on the set;

a distance calculation module configured to calculate the distance between two distributions of adjacent one or more regions; and

A combining module configured to combine the one or more adjacent regions as a division of the feature space when the distance is smaller than a predetermined threshold.

Supplement 3. The model updating device according to Supplement 2, wherein the distance is KL distance.

Supplement 4. The model updating device according to Supplement 2, wherein the distance calculation module is configured to use a Bayesian metric to calculate the distance between the adjacent one or more regions on the training data set The distribution of is used as the distance.

Supplement 5. The model updating device according to any one of Supplements 1 to 4, wherein the calibration model is a model with the best performance other than the target model in the multi-model system.

Supplement 6. The model updating device according to any one of Supplements 1 to 4, wherein the calibration model and the target model are respectively obtained by using complementary algorithms in principle.

Supplement 7. The model updating device according to any one of Supplements 1 to 4, wherein the calibration model is a model obtained by fusing other models than the target model in the multi-model system.

Supplement 8. The model updating device according to any one of Supplements 1 to 4, wherein the multiple models include models obtained based on one or more of the following methods: support vector machine, random Forest, Decision Tree, K-Nearest Neighbor Algorithm, Boosting Algorithm, Naive Bayesian Algorithm, and Maximum Entropy Algorithm.

Supplement 9. The model updating device according to any one of Supplements 1 to 4, wherein the multi-model system is used for entity relationship extraction.

Supplement 10. The model updating device according to any one of Supplements 1 to 4, wherein the model updating device is configured to update periodically.

Supplement 11. A data processing device using a multi-model system to process a data set to be tested, comprising the model updating device according to any one of Supplements 1 to 10.

Supplement 12. The data processing device according to Supplement 11, further comprising:

a control unit configured to perform control such that the model updating means updates each model in the multi-model system as the target model; and

a fusion unit configured to fuse the updated models to obtain a final model,

Wherein, the data processing device uses the final model to process the data set to be tested.

Additional note 13. A model update method for updating the target model in the multi-model system, wherein each model in the multi-model system is a model obtained by pre-training in different ways for the training data set, and the model update method includes :

Using a model different from the target model in the multi-model system as a calibration model to process the data set to be tested, and use the processed result as a pseudo-label;

Obtaining the feature distribution of the data set to be tested based on the pseudo-label;

Obtaining the feature distribution of the training data set based on the target model;

Adjust the feature space division of the target model based on the feature distribution of the training data set and the feature distribution of the test data set, so that the training data set and the test data set have a characteristic for the feature space division. similar distribution; and

The target model is updated using the training dataset based on the adjusted feature space partition.

Supplementary Note 14. The model updating method according to Supplementary Note 13, wherein the step of adjusting the feature space division of the target model includes:

Dividing the feature space of the training data set and the test data set into a plurality of regions;

Based on the feature distribution of the training data set and the feature distribution of the test data set, calculate the distribution of one or more adjacent regions on the training data set and the test data set;

calculating a distance between two distributions of one or more adjacent regions, and merging the one or more adjacent regions as a partition of the feature space when the distance is smaller than a predetermined threshold.

Supplementary Note 15. The model updating method according to Supplementary Note 14, wherein the distance is KL distance.

Supplementary Note 16. The model updating method according to Supplementary Note 14, wherein a Bayesian metric is used to calculate the distribution of the one or more adjacent regions on the training data set as the distance.

Supplementary Note 17. The model updating method according to any one of Supplementary Notes 13 to 16, wherein the calibration model and the target model are respectively obtained using complementary algorithms in principle.

Supplementary Note 18. The model updating method according to any one of Supplementary Notes 13 to 16, wherein the multiple models include models obtained based on one or more of the following methods: support vector machine, random Forest, Decision Tree, K-Nearest Neighbor Algorithm, Boosting Algorithm, Naive Bayesian Algorithm, and Maximum Entropy Algorithm.

Supplementary Note 19. A data processing method using a multi-model system to process the data set to be tested, including the model updating method according to any one of Supplementary Notes 13 to 18.

Supplement 20. The data processing method described in Supplement 19, further comprising:

updating each model in the multi-model system as the target model; and

The updated models are fused to obtain a final model, and the data set to be tested is processed using the final model.

Claims (10)

1. A model updating device for updating the target model in the multi-model system, wherein each model in the multi-model system is a model obtained by pre-training in different ways for the training data set, and the model updating device includes: A pseudo-label acquisition unit configured to use a model different from the target model in the multi-model system as a calibration model to process the data set to be tested, and use the processed result as a pseudo-label; a first feature distribution acquisition unit configured to obtain a feature distribution of the data set to be tested based on the pseudo-label; a second feature distribution acquisition unit configured to acquire the feature distribution of the training data set based on the target model; An adjustment unit configured to adjust the feature space division of the target model based on the feature distribution of the training data set and the feature distribution of the test data set, so that the training data set and the test data set has a similar distribution for this feature space partition; and An update unit configured to use the training data set to update the target model based on the adjusted feature space partition. 2. The model updating device according to claim 1, wherein the adjustment unit comprises: a partition module configured to divide the feature space of the training data set and the test data set into multiple regions; The distribution calculation module is configured to calculate the distribution of adjacent one or more regions in the training data set and the test data based on the feature distribution of the training data set and the feature distribution of the test data set, respectively. distribution on the set; a distance calculation module configured to calculate the distance between two distributions of adjacent one or more regions; and A combining module configured to combine the one or more adjacent regions as a division of the feature space when the distance is smaller than a predetermined threshold. 3. The model updating device according to claim 2, wherein the distance is a KL distance. 4. The model updating apparatus according to claim 2, wherein the distance calculation module is configured to use a Bayesian metric to calculate the distribution of the adjacent one or more regions on the training data set as the distance. 5. The model updating device according to any one of claims 1 to 4, wherein the calibration model is a model with the best performance other than the target model in the multi-model system. 6. The model updating device according to any one of claims 1 to 4, wherein the calibration model and the target model are respectively obtained using complementary algorithms in principle. 7. The model updating device according to any one of claims 1 to 4, wherein said respective models include models obtained based on one or more of the following methods: support vector machine, random forest, decision Tree, K-Nearest Neighbor Algorithm, Boosting Algorithm, Naive Bayesian Algorithm, and Maximum Entropy Algorithm. 8. The model updating device according to any one of claims 1 to 4, wherein the model updating device is configured to update periodically. 9. A data processing device using a multi-model system to process a data set to be tested, comprising the model updating device according to any one of claims 1-8. 10. A model update method for updating a target model in a multi-model system, wherein each model in the multi-model system is a model obtained by pre-training in different ways for a training data set, and the model update method includes: Using a model different from the target model in the multi-model system as a calibration model to process the data set to be tested, and use the processed result as a pseudo-label; Obtaining the feature distribution of the data set to be tested based on the pseudo-label; Obtaining the feature distribution of the training data set based on the target model; Adjust the feature space division of the target model based on the feature distribution of the training data set and the feature distribution of the test data set, so that the training data set and the test data set have a characteristic for the feature space division. similar distribution; and The target model is updated using the training dataset based on the adjusted feature space partition.