CN108776697A - A kind of multi-source data collection cleaning method based on predicate - Google Patents

Abstract

The present invention proposes that a kind of method that the multi-source data collection cleaning method based on predicate is provided effectively can identify most reliable data item from isomorphism multi-source data concentration, is related to the fields such as data cleansing, data fusion.The method includes：1) predicate is excavated with automated method, and the predicate excavated is filtered；2) confidence level of each entity attributes value is concentrated according to predicate derived data；3) attribute value confidence level is established with the relationship between data source confidence level, calculates data source confidence level；4) data source confidence level and attribute value confidence level is combined to find out the highest data item of confidence level.For multiple data sources, the present invention can filter out redundancy, mistake and out-of-date data, leave the highest data of confidence level to coming from different data sources but the identical information of content is analyzed, for subsequent data analysis lay a good foundation, it is of great significance to the efficiency and accuracy rate of follow-up data processing.

Description

A predicate-based cleaning method for multi-source datasets

technical field

The invention relates to the fields of data cleaning, data fusion and the like, in particular to a predicate-based multi-source data set cleaning method.

Background technique

In the information age, description data for the same event or object can be found from a large number of data sources. At the same time, due to time errors, format errors, accuracy, completeness, semantic ambiguity, etc., data for the same entity from different data sources There is an inconsistency in the description of . After collecting data from different data sources, resolving inconsistencies between descriptive data belonging to the same entity is crucial for subsequent data analysis. A simple voting strategy—selecting descriptions supported by more data sources—is not suitable for the current Web environment, and it is necessary to consider the credibility of data sources, the credibility of the data itself, and some prior knowledge to design more complex cleaning Strategy. Existing cleaning strategies mainly include the following:

The application document of Chinese Patent No. 201410387772 discloses "a bus traffic condition processing system and method based on traffic multi-source data fusion", which integrates traffic data describing bus traffic conditions from different data sources to obtain road condition information for display. Its input is specific traffic data, and it does not judge the credibility according to the predicate, nor calculate the credibility of the data source according to the relationship between the data and the data source.

Chinese patent application document No. 201110369877 discloses "a multi-source data integration platform and its construction method", which manages different data, and there is no consistency problem among these data.

The US Patent No. US 8190546 application document discloses "Dependency between sources intruth discovery", which establishes a probabilistic graph model through the copy relationship between data sources to evaluate the credibility of data sources and data, and does not involve the use of predicates to evaluate data credibility.

Contents of the invention

Purpose of the invention: In order to overcome the current problem of inconsistency in describing the data of the same entity in multi-source data fusion, that is, it is difficult to determine the initial value of data credibility in the problem of multi-source data consistency, and how to combine data source credibility and data To solve the problem of credibility, the present invention provides a method for cleaning multi-source datasets based on data source credibility and data credibility. The data credibility is calculated by setting predicates, and then the data source can be calculated through the data credibility. Reliability, and finally find the data with the highest reliability to achieve the purpose of data cleaning.

Technical solution: In order to achieve the above technical effects, the present invention proposes a predicate-based multi-source data set cleaning method, including steps:

(1) Build a predicate model: define priority predicates, status predicates and interaction predicates; among them,

The priority predicate is Prior(A _i , A _j ), which means that the priority of attribute A _i is higher than that of attribute A _j ;

The status predicates are: Among them, t _i represents statement i, Indicates the attribute value of attribute A _k in sentence i, means predefined and φ(t _i , t _j ) represents the condition satisfied between the predefined t _i and t _j ; Stat(A _k ) represents when t _i and t _j satisfy the conditions P and φ, t _i has a mass higher than t _j ;

The interaction predicate is: Inter _δ (A ₁ ,...,A _l ), which means that when the data satisfies the condition δ, the quality of the attribute value of the attribute A ₁ ,...,A _l of the piece of data is poor;

(2) Perform predicate mining on the data set to be cleaned through the predicate model defined in step (1), and obtain priority predicates, status predicates and interaction predicates in the data set;

(3) Deduce the attribute value credibility of each data in the data set according to the obtained predicate, including steps:

(3-1) All attribute value credibility of the data in the initialization data set is 0, and influence factor η is set for each attribute value of each piece of data, and η is a constant;

(3-2) Use state predicates and interaction predicates to update the credibility of each attribute value of each piece of data. When updating, first use the state predicates to update and then use the interaction predicates to update, or first use the interaction predicates to update and then use the state predicates to update;

The steps to update the credibility of each attribute value of the data using the state predicate are as follows: enumerate two data t _i and t _j in the data set in pairs, if t _i and t _j satisfy the state predicate on the attribute A _k : then attribute value The credibility minus η;

The steps to use the interaction predicate to update the credibility of each attribute value of the data are: traverse all the data in the data set, if a piece of data satisfies a certain interaction predicate Inter _δ (A ₁ ,...,A _l ), then the data attribute A ₁ ,... _, the credibility of the attribute value of Al minus η;

(3-3) After the step (2) is completed, use the priority predicate to update the attribute value credibility of each piece of data. When updating, execute the priority predicate in order of priority from high to low;

The steps of executing the priority predicate Prior(A _i , A _j ) are: if the attribute values of multiple pieces of data on attribute A _j have the same credibility, sort them in ascending order according to the attribute value credibility of A _i , According to the sorted order, add n-1 to the attribute value reliability of A _j of the nth-ranked data;

(3-4) After obtaining the credibility of all attribute values, for multi-valued attributes, return all attribute values with credibility greater than or equal to the preset threshold as the result; for attributes that only need to return one result, perform step (4) to (6);

(4) Normalize the credibility of all attribute values; according to the formula Calculate the credibility of all data sources in the data set to be cleaned; where, λ _i represents the credibility of data source D _i , t represents a piece of data in data source D _i , d(t) represents the credibility of data t, The credibility of data t is equal to the sum of the credibility of all attribute values of the data;

(5) According to the formula Update the credibility of each attribute value, D' means that attribute value is provided for attribute A _j The data source; after updating, return to step (4);

(6) Repeat steps (4) to (5) until the credibility of all attribute values converges; for an attribute that only needs to return one result, find out the attribute value with the highest reliability under this attribute as the final result.

Further, the definition method of the priority predicate is: for the attributes A _i and A _j , if p _score (A _i )<p _score (A _j ), then define the priority predicate Prior(A _i , A _j ) , indicating that the priority p _score (A _i ) of attribute A _i is higher than the priority p _score (A _j ) of attribute A _j ; Among them, H(A _i ) represents the Shannon entropy of attribute A _i , and p _n (A _i ) represents the proportion of null values in all attribute values of attribute A _i .

Further, both the state predicate and the interaction predicate are obtained through a first-order logic predicate mining method.

Further, before cleaning the data set, manually mark all the attributes of all the data sets, and mark whether each attribute needs to return one result or multiple results. If an attribute only needs to return one result, mark the attribute as For a single-valued attribute, the value of the attribute with the highest reliability under the attribute is the final result during cleaning; if there may be multiple results for an attribute, mark the attribute as a multi-valued attribute, and the attribute under the attribute with a reliability greater than the expected value is selected during cleaning. Set all attributes of Threshold to the final result.

Beneficial effect: compared with the prior art, the present invention has the following advantages:

There is no need to assume that there is only one correct value for an attribute, nor does it depend on crowdsourcing, and it does not require a lot of manual intervention. It uses automatically mined predicates and the relationship between data sets and attribute values to find highly credible attribute values. The present invention scores the credibility of attribute values by digging self-defined predicates, finds attribute values whose credibility is higher than the preset threshold for multi-answer attributes as a result, and combines data source credibility and attribute values for the remaining attributes The relationship between value credibility further updates the attribute value credibility, and finding the attribute value with the highest credibility as a result is of great significance for improving the efficiency and accuracy of data analysis. By adopting the technical scheme of the invention, engineering personnel can realize related software relatively easily.

Description of drawings

Fig. 1 is a flowchart of the present invention;

Fig. 2 is a schematic flow chart of calculating the credibility of attribute values in the updated data source in the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings.

Shown in Fig. 1 is flow chart of the present invention, and the present invention mainly comprises following several parts:

a) First define three predicates:

1) Priority predicate: For attributes A _i and A _j , if p _score (A _i )<p _score (A _j ), then define a priority predicate Prior(A _i , A _j ), which means the priority of attribute A _i Level p _score (A _i ) is higher than the priority p _score (A _j ) of attribute A _j ; Among them, H(A _i ) represents the Shannon entropy of attribute A _i , and p _n (A _i ) represents the proportion of null values in all attribute values of attribute A _i .

The calculation formula of H(A _i ) is: H(A _i )＝-∑ _x∈X p(x)log ₂ p(x), X is the range of the attribute value of attribute A _i , and p(x) represents the attribute value x's weighting of all property values (excluding null values).

2) State predicate: The state predicate is a first-order logical predicate, and its form is:

It means that t _i and t _j satisfy the conditions P and φ, then the quality of t _i is higher than that of t _j .

conditions in the above definition And f _i (v ₁ , v ₂ ) can be replaced by v ₁ =v ₂ or v ₁ ≠v ₂ . P in the state predicate definition can be replaced by 6 pre-defined predicates, namely P ₁ (v ₁ , v ₂ ), P ₂ (v ₁ , v ₂ ), P ₃ (v ₁ , v ₂ ), P ₄ (v ₁ , v ₂ ), P ₅ (v ₁ , v ₂ ), P ₆ (v ₁ , v ₂ ). P ₁ (v ₁ , v ₂ ), P ₂ (v ₁ , v ₂ ) are suitable for attribute values of numeric types, P ₁ (v ₁ , v ₂ ) means that v ₁ is larger than v ₂ , P ₂ (v ₁ , v ₂ ) means that v ₁ is smaller than v ₂ ; P ₃ (v ₁ , v ₂ ), P ₄ (v ₁ , v ₂ ) are suitable for attribute values of character types, and P ₃ (v ₁ , v ₂ ) means that v ₁ Longer than v ₂ , P ₄ (v ₁ , v ₂ ) indicates that v ₁ is shorter than v ₂ ; P ₅ (v ₁ , v ₂ ), P ₆ (v ₁ , v ₂ ) are suitable for character type attribute values, character A more detailed string means that it contains more information, which is measured by comparing the amount of information contained in two strings with the Shannon entropy formula. P ₅ (v ₁ , v ₂ ) means that v ₁ is more detailed than v ₂ , and P ₆ (v ₁ , v ₂ ) means that v ₁ is simpler than v ₂ .

3) Interaction predicate: The interaction predicate is a first-order logical predicate, and its form is Inter _δ (A ₁ ,…,A _l ), which means that when a piece of data satisfies the condition δ, then the attribute value of A ₁ ,…,A _l of this piece of data low quality.

in the above definition Among them, P _i ′ can be replaced by any predicate from P ₁ to P ₆ , and can also be replaced by the following four predicates: P ₇ (v ₁ , v ₂ ), P ₈ (v ₁ , v ₂ ), P ₉ (v ₁ , v ₂ ), P ₁₀ (v ₁ , v ₂ ); among them, P ₇ (v ₁ , v ₂ ), P ₈ (v ₁ , v ₂ ) are suitable for attribute values of character types, and P ₇ (v ₁ , v ₂ ) means that v ₁ contains v ₂ , P ₈ (v ₁ , v ₂ ) means that v ₁ does not contain v ₂ ; P ₉ (v ₁ , v ₂ ), P ₁₀ (v ₁ , v ₂ ) are suitable for character types and attribute values of numerical type, P ₉ (v ₁ , v ₂ ) indicates that v ₁ is equal to v ₂ , and P ₁₀ (v ₁ , v ₂ ) indicates that v ₁ is not equal to v ₂ .

Then perform predicate mining according to the data set: for the priority predicate, it can be based on the formula The priority of all attributes of the calculation data set is obtained; for the state predicate and the interaction predicate, according to the definition of its first-order logic predicate, it is automatically obtained by the first-order inductive learning method. After obtaining the state predicates and interaction predicates, in order to further improve the usability of the predicates, domain experts can be asked to remove invalid predicates to obtain the final usable predicates;

b) Initialize the reliability of all attribute values as 0, and manually set the parameter η (influence factor) to a real number. Then execute the three types of predicates in the following order to derive the credibility of the attribute values of each entity in the dataset;

1) Use state predicates: Enumerate two pieces of data in the data set in pairs, if these two pieces of data satisfy a certain state predicate then will The reliability minus η.

2) Use the interaction predicate: traverse all the data in the data set, if a piece of data satisfies a certain interaction predicate Inter _δ (A ₁ ,…,A _l ), then the attribute value credibility of the piece of data A ₁ ,…,A _l Subtract η.

3) Use priority predicates: Since the priority of attributes is determined by It is determined that in order to make the credibility of the current attribute value the latest, the priority predicate with higher priority must be executed first, that is, the priority predicate with the smaller sum of the p _score of the two attributes included. The function of the priority predicate Prior(A _i , A _j ) is, if two pieces of data t ₁ and t ₂ satisfy the condition At this time, it can be judged by the credibility of attribute A _i and Which is better. The method is, for all multiple data with the same reliability on A _j , sort them in ascending order according to the reliability of A _i , and according to the sorted order, the attribute of A _j in the nth-ranked data Add n-1 to the value credibility, so that the value of A _j with the same credibility is distinguished according to the higher priority A _i . Also note that for attributes that need to return multiple results, if the confidence is negative, there is no need to apply the priority predicate.

After obtaining the credibility of all attribute values, for attributes that need to return multiple results, return all attribute values with credibility greater than or equal to the preset threshold as results, and for attributes that need to return one result, continue with the following steps.

c) According to Calculate the credibility of the data source, and normalize the credibility of all data sources, that is, ∑ _i λ _i =1, as shown in Figure 2. reuse Update the credibility of each attribute value. The credibility of each attribute value is equal to the sum of the credibility of the data source that provides the attribute value multiplied by its original credibility. Note that the data source with a null value is credible Degree refers to the credibility of its own data source, excluding the credibility of other data sources that provide null values. Next, the credibility of all attribute values is also normalized, so that for any attribute, the sum of the credibility of all possible values is 1. Repeat the above steps until the credibility of the data source and the credibility of each attribute value converge.

d) Finally, for an attribute that only needs to return one result, find out the attribute value with the highest reliability under this attribute as the final result, and combine the result in b) as the final result.

Below in conjunction with specific examples, the embodiment of the present invention is described:

We let the credibility of a data source D _i be λ _i , the attribute set of the data source is {A ₁ ,...,A _n }, and t∈D _i is a piece of data of the data source, where Indicates that t corresponds to the attribute value of A _j . Then let d(t) represent the credibility of the piece of data, Indicates the attribute value credibility. The credibility of a piece of data is equal to the sum of the credibility of all attribute values of the piece of data, namely:

The credibility of a data source is equal to the average of the credibility of all the data it contains, that is, the sum of the credibility of all data divided by the number of data:

At the same time, let D′ be the property value that provides the property value for property A _j data source, the property value The credibility of is the sum of the credibility of all data sources that provide the value multiplied by its original credibility:

Example: The cleaning data set is shown in the following table. There are 5 pieces of data and 5 data sources in total, among which t _i comes from the data source D _i and describes the data of a scientific researcher named Mary.

Clean the dataset table

First of all, simply observe the data set first, do a manual preprocessing, and clear some obviously unreasonable data, so that the subsequent data cleaning efficiency is higher and the effect is better.

For example, in the data of t ₅ in the above data set, the salary is a negative value, which is obviously unreasonable, and the values of the three attributes of Research Area, Affiliation and Publication in this data are not very meaningful, so t ₅ can be This piece of data is directly deleted as noise without participating in subsequent cleaning operations.

Looking at the data of t ₄ again, the value of its publication attribute is "-", which is also unreasonable data, but because the values of other attributes of the data of t ₄ still have reference value, you can directly use the publication Value changed to "null".

After the above simple preprocessing, the obtained data set is as follows:

In the first step, predicate mining is performed on the dataset.

Mining Priority Predicates

For priority predicates, count the entropy of all attributes and the proportion of null values.

Entropy calculation formula: H(A _i )＝-∑ _x∈X p(x)log ₂ p(x)

Among them, p(x) represents the proportion of attribute value x to all attribute values (excluding null values);

Taking salary as an example, there are three attribute values, namely 142k, 120k and 88k.

in, Then the entropy of attribute salary can be obtained as:

In the same way:

p _n (Salary) = p _n (ResearchArea) = 0

can get:

According to the above less than relationship, three priority predicates can be defined:

Mining state predicates:

Automatically obtained by the first-order logic predicate mining algorithm First Order Inductive Learner:

Mining interaction predicates:

Also automatically obtained by the first-order logical predicate mining algorithm First Order Inductive Learner:

The second step is to derive the credibility of the attribute values of each entity.

Initialize the reliability of all attribute values to 0, set the impact factor η=1, and set the reliability threshold to 0 for attributes that need to return multiple results. Then use the corresponding predicates in a certain order (different predicate execution orders may produce different results).

Both the state predicate and the interaction predicate act on the attribute value, and the attribute value will not change, so the relationship between the state predicate and the state predicate, between the interaction predicate and the interaction predicate, and between the state predicate and the interaction predicate are independent of each other. Called in any order.

However, priority predicates act on the trustworthiness of attribute values, so priority predicates must be used after all state and interaction predicates.

In addition, a certain order must be observed between priority predicates and priority predicates. In order to make the credibility of the current attribute value up-to-date, the priority predicate with higher priority needs to be executed first, that is, the priority predicate with the smaller sum of the priorities of the two attributes included. For priority predicates on cleaning dataset tables, The sum of the p _scores is 3.5, The sum of the p _scores is also 3.5, The sum of the p _scores is 4.11, so it should be according to The order in which the priority predicates are executed.

After executing the state predicate, the credibility of all attribute values is shown in Table 1. There are 4 pieces of data here, which need to be compared in pairs, and a total of 16 comparisons have been made. Taking t ₁ and t ₂ as examples, according to the state predicate because So will minus 1.

Table 1

Salary Research Area Affiliation Publication t ₁ (D ₁ ) 0 0 0 0 t ₂ (D ₂ ) -1 0 0 0 t ₃ (D ₃ ) -2 0 0 0 t ₄ (D ₄ ) -2 0 0 0

After executing the interaction predicate, the credibility of all attribute values is shown in Table 2. Here according to the interaction predicate and Decrease the credibility of all attribute values with null Affiliation and Publication values by 1.

Table 2

Salary Research Area Affiliation Publication t ₁ (D ₁ ) 0 0 0 0 t ₂ (D ₂ ) -1 0 0 -1 t ₃ (D ₃ ) -2 0 0 0 t ₄ (D ₄ ) -2 0 -1 -1

After executing the priority predicate, the credibility of all attribute values is shown in Table 3. Taking Research Area as an example, the initial value of the Research Area column is {0, 0, 0, 0], according to the priority predicate The value of the Research Area column can be updated according to {0, -1, -2, -2) of the Salary column. Reorder the values with the same credibility in the Research Area column according to the ascending order of the Salary column, add 0, 1, and 2 respectively after sorting, and get {2, 1, 0, 0] after restoring the order. In the same way, execute the priority predicate again and

table 3

Salary Research Area Affiliation Publication t ₁ (D ₁ ) 0 2 2 1 t ₂ (D ₂ ) -1 1 1 -1 t ₃ (D ₃ ) -2 0 0 0 t ₄ (D ₄ ) -2 0 -1 -1

Then mark all the attributes. A researcher can only have one salary and one affiliation at the same time, so Salary and Affiliation only return one result, but a researcher can have multiple research fields and works, so ResearchArea and The Publication property should return multiple result values. For multi-valued attributes, all attribute values greater than or equal to the threshold 0 are returned at this time, that is, for Research Area, the returned result is {Data integration, data cleaning Data cleaning&Google Knowledge managementInformation retrieve}; for Publication, the returned result is {Data integration, Adiagnostic tool for data errors}.

The third step is to calculate the credibility of the data source.

then pass Map the credibility values of all attribute values to (0, 1), and normalize, the results are shown in Table 4. and according to Calculate the credibility of all data sources.

Table 4

Salary Research Area Affiliation Publication lambda t ₁ (D ₁ ) 0.496353 0.33723 0.369959 0.413275 0.404204 t ₂ (D ₂ ) 0.26698 0.2799 0.307065 0.152035 0.251495 t ₃ (D ₃ ) 0.118333 0.191435 0.210014 0.282655 0.200609 t ₄ (D ₄ ) 0.118333 0.191435 0.112963 0.152035 0.143692

finally passed and The two formulas update the credibility of all attribute values iteratively until the credibility of all attribute values converges. Note that after updating the credibility of attribute values by column each time, the credibility of the column needs to be normalized. Taking the first update process as an example, for the attribute value credibility of the Salary column:

{0.496353, 0.26698, 0.118333, 0.118333]

→{0.496353×0.404204, 0.26698×0.251495, 0.118333×0.344301, 0.118333×0.344301]

→{0.200628, 0.0671441, 0.0407422, 0.0407422)

Similarly, for the attribute value credibility of the Research Area column:

{0.33723, 0.2799, 0.191435, 0.191435) → {0.500009, 0.258216, 0.140871, 0.100903]

For the attribute value credibility of the Affiliation column:

{0.369959, 0.307065, 0.210014, 0.112963) → {0.404204, 0.251495, 0.200609, 0.143692)

For the attribute value credibility of the Publication column:

{0.413275, 0.152035, 0.282655, 0.152035} → {0.588542, 0.134713, 0.199777, 0.0769686]

Finally update the credibility of the data source:

λ = {0.5168, 0.209168, 0.164478, 0.109554)

Repeat the above process until convergence, and the final results are shown in Table 5.

table 5

Salary Research Area Affiliation Publication lambda t ₁ (D ₁ ) 1 1 1 1 1 t ₂ (D ₂ ) 0 0 0 0 0 t ₃ (D ₃ ) 0 0 0 0 0 t ₄ (D ₄ ) 0 0 0 0 0

The fourth step is to get the result.

According to Table 5, the best attribute value among the Salary and Affiliation attributes can be selected, that is, the attribute value with the highest reliability is the result. The result of Salary is {142k}, and the result of Affiliation is {Amazon}.

The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications are also possible. It should be regarded as the protection scope of the present invention.

Claims (4)

1. A predicate-based multi-source data set cleaning method, characterized in that, comprising steps: (1) Build a predicate model: define priority predicates, status predicates and interaction predicates; among them, The priority predicate is Prior(A _i , A _j ), which means that the priority of attribute A _i is higher than that of attribute A _j ; The status predicates are: Among them, t _i represents statement i, Indicates the attribute value of attribute A _k in sentence i, means predefined and φ(t _i , t _j ) represents the condition satisfied between the predefined t _i and t _j ; Stat(A _k ) represents when t _i and t _j satisfy the conditions P and φ, t _i has a mass higher than t _j ; The interaction predicate is: Inter _δ (A ₁ ,...,A _l ), which means that when the data satisfies the condition δ, the quality of the attribute value of the attribute A ₁ ,...,A _l of the piece of data is poor; (2) Perform predicate mining on the data set to be cleaned through the predicate model defined in step (1), and obtain priority predicates, status predicates and interaction predicates in the data set; (3) Deduce the attribute value credibility of each data in the data set according to the obtained predicate, including steps: (3-1) All attribute value credibility of the data in the initialization data set is 0, and influence factor η is set for each attribute value of each piece of data, and η is a constant; (3-2) Use state predicates and interaction predicates to update the credibility of each attribute value of each piece of data. When updating, first use the state predicates to update and then use the interaction predicates to update, or first use the interaction predicates to update and then use the state predicates to update; The steps to update the credibility of each attribute value of the data using the state predicate are as follows: enumerate the two data t _i and t _j in the data set in pairs, if t _i and t _j satisfy the state predicate on the attribute A _k : then attribute value The credibility minus η; The steps to use the interaction predicate to update the credibility of each attribute value of the data are: traverse all the data in the data set, if a piece of data satisfies a certain interaction predicate Inter _δ (A ₁ ,...,A _l ), then the data attribute A ₁ ,... _, the credibility of the attribute value of Al minus η; (3-3) After the step (2) is completed, use the priority predicate to update the attribute value credibility of each piece of data. When updating, execute the priority predicate in order of priority from high to low; The steps of executing the priority predicate Prior(A _i , A _j ) are: if the attribute values of multiple pieces of data on attribute A _j have the same credibility, sort them in ascending order according to the attribute value credibility of A _i , According to the sorted order, add n-1 to the attribute value reliability of A _j of the nth-ranked data; (3-4) After obtaining the credibility of all attribute values, for multi-valued attributes, return all attribute values whose credibility is greater than or equal to the preset threshold as the result; for attributes that only need to return one result, perform step (4) to (6); (4) Normalize the credibility of all attribute values; according to the formula Calculate the credibility of all data sources in the data set to be cleaned; where, λ _i represents the credibility of data source D _i , t represents a piece of data in data source D _i , d(t) represents the credibility of data t, The credibility of data t is equal to the sum of the credibility of all attribute values of the data; (5) According to the formula Update the credibility of each attribute value, D' means that attribute value is provided for attribute A _j The data source; after updating, return to step (4); (6) Repeat steps (4) to (5) until the credibility of all attribute values converges; for an attribute that only needs to return one result, find out the attribute value with the highest reliability under this attribute as the final result. 2. A predicate-based multi-source data set cleaning method according to claim 1, wherein the method for defining the priority predicate is: for attributes A _i and A _j , if p _score (A _i )<p _score (A _j ), then define a priority predicate Prior(A _i , A _j ), which means that the priority p _score (A _i ) of attribute A _i is higher than the priority p _score (A _j ) of attribute A _j ; Among them, H(A _i ) represents the Shannon entropy of attribute A _i , and p _n (A _i ) represents the proportion of null values in all attribute values of attribute A _i . 3. A predicate-based multi-source data set cleaning method according to claim 2, characterized in that, both the state predicate and the interaction predicate are obtained through a first-order logic predicate mining method. 4. A predicate-based multi-source data set cleaning method according to claim 3, characterized in that, before the data set is cleaned, all attributes of all data sets are manually marked, and each attribute needs to be marked Return one result or multiple results. If an attribute only needs to return one result, mark the attribute as a single-valued attribute. When cleaning, the attribute with the highest reliability under the attribute is the final result; if there may be multiple As a result, the attribute is marked as a multi-valued attribute, and all attributes under the attribute whose reliability is greater than a preset threshold are valued as final results during cleaning.