Open AccessArticle

Knowledge Graph Alignment Network with Node-Level Strong Fusion

Shuang Liu

^1,*,

Man Xu

Yufeng Qin

¹ and

Niko Lukač

School of Computer Science and Engineering, Dalian Minzu University, Dalian 116600, China

Faculty of Electrical Engineering and Computer Science, University of Maribor, SI-2000 Maribor, Slovenia

Author to whom correspondence should be addressed.

Appl. Sci. 2022, 12(19), 9434; https://doi.org/10.3390/app12199434

Submission received: 4 August 2022 / Revised: 17 September 2022 / Accepted: 18 September 2022 / Published: 20 September 2022

(This article belongs to the Special Issue Natural Language Processing (NLP) and Applications)

Download

Browse Figures

Versions Notes

Abstract

Entity alignment refers to the process of discovering entities representing the same object in different knowledge graphs (KG). Recently, some studies have learned other information about entities, but they are aspect-level simple information associations, and thus only rough entity representations can be obtained, and the advantage of multi-faceted information is lost. In this paper, a novel node-level information strong fusion framework (SFEA) is proposed, based on four aspects: structure, attribute, relation and names. The attribute information and name information are learned first, then structure information is learned based on these two aspects of information through graph convolutional network (GCN), the alignment signals from attribute and name are already carried at the beginning of the learning structure. In the process of continuous propagation of multi-hop neighborhoods, the effect of strong fusion of structure, attribute and name information is achieved and the more meticulous entity representations are obtained. Additionally, through the continuous interaction between sub-alignment tasks, the effect of entity alignment is enhanced. An iterative framework is designed to improve performance while reducing the impact on pre-aligned seed pairs. Furthermore, extensive experiments demonstrate that the model improves the accuracy of entity alignment and significantly outperforms 13 previous state-of-the-art methods.

Keywords:

knowledge graph; entity alignment; graph convolutional network; knowledge fusion

1. Introduction

Knowledge graphs (KGs) store the knowledge of the human world in the form of triples. With the worldwide sharing of knowledge, an increasing number of multi-lingual KGs are constructed, such as DBPedia [1] and YAGO [2]. However, KGs are constructed from different data sources, each KG may be incomplete, and the knowledge they contain can complement each other. Entity alignment is a key step in knowledge fusion, whose purpose is to discover entities with the same meaning.

In recent years, many entity alignment models have been the embedding-based method. First, entities in the KGs are embedded into a uniform vector space, and then their similarity is calculated by comparing the distances between entities. Compared with traditional feature-base methods, the embedding-base methods can effectively model entity information and reduce the cost of designing features manually. Embedding-based approaches can be divided into two types. The first type is translation-based approaches --TransE [3], with the key assumption “

h + r \approx t

” applied to each relation triple. That is, the sum of the header entity vector and the relation vector is regarded as the tail entity vector, such as multilingual knowledge graph embedding (MTransE) model [4], iterative and parameter sharing (IPTransE) method [5] and bootstrapping (BootEA) method [6]. The second type is GCN-based approaches. Kipf et al. proposed a graph-based neural network model called GCNs [7], a convolutional method that directly manipulates the graph structure, and entity embedding representations are obtained by aggregation of neighboring entity information around GCN, including the embedding-based KG alignment method (GCN-Align) [8], Sun et al.’s method (AliNet) [9], Relation-aware Dual-Graph Convolutional Network (RDGCN) [10] and Neighborhood Matching Network (NMN) [11]. These methods only learn and improve the entity structure information and are lacking reflecting the entity meaning in all aspects.

In the field of entity alignment, most previous research was based on the structure assumptions that entities with the same or similar neighborhoods are usually regarded as equivalent entities. These methods are judged by comparing adjacent entities. Figure 1 shows the English version of “Oban” and the Chinese version of “Stonehaven” as the central entity respectively. The central entity in Figure 2 is “Steam Engine” in English and “England” in Chinese. If only the alignment is judged by comparing neighborhoods, then these two pairs of entities will be incorrectly judged as aligned.

However, as shown in Figure 1, if the attribute information “Postal Code” is added, which is unique attribute value label, it can correctly be determined that “Oban” and “Stonehaven” are non-equivalent entities. In Figure 2, considering the influence of relation information, it can be found that “Steam Engine” and “England” have different correspondences when facing the entities of “James Watt”, “Industrial Revolution” and “University of Glasgow”. In Figure 2, by contrast, it can be found that the relation between the two central entities “Steam Engine”, “England” and their common neighbor entities “James Watt”, “Industrial Revolution”, “University of Glasgow” are different. For example, the “steam engine” was improved by “James Watt”, but the relation between “England” and “James Watt” is the birthplace. The difference in the correspondence between them can be judged by the fact that these two entities with similar structure are not equivalent entities. Thence, attribute information and relation information can improve the accuracy of entity alignment.

Therefore, this paper proposes a new embedded entity alignment model SFEA that integrates multi-faceted information with node-level strong fusion. The aim of SFEA is to make full use of multi-faceted information to obtain a more realistic and accurate entity representation, it can closely fuse information from different aspects and obtain entity representations in a node-level fine manner. The structure and attribute are encoded by GCN, and the attribute and entity name are used as the basis in the learning of entity structure. The stacking of multi-layer GCN can make the information of different aspects more closely fuse in the unit of node in the process of dissemination. In addition, in order to alleviate the structural heterogeneity, it joins the highway mechanism and the attention mechanism for neighborhood selection. The TransE-based approach is employed to approximate relation embeddings, and entity alignment is further facilitated using the way that sub-alignment tasks interact with each other. At the same time, the iterative framework is used to optimize the entity representations. The way in which multiple aspects of information are tightly fused also makes the model more stable, reducing the reliance on pre-aligned seed pairs. The approach (SFEA) has been evaluated on two datasets. Experimental results show that this approach performs better results and is more robust compared with state-of-the-art methods for entity alignment.

This paper makes the following technical contributions:

SFEA learns entity representations using various entity information of entity attribute, structure, relation and entity name. Entity alignment is performed through node-level tight fusion, so that multi-faceted information can be fully utilized in entity alignments.
SFEA introduces highway gating mechanism and attention mechanism in the process of entity structure learning, selects the importance of neighbors at the same layer, and flexibly aggregates neighbors at different layers.
SFEA uses sub-alignment tasks interaction and iterative strategy to further optimize entity representations, to obtain more accurate entity embedding representations.

2. Related Work

Early entity alignment required substantial human involvement to design different features [12] and rules [13] for each task, or by importing external data sources [14,15] to align. This kind of method is limited by cost and difficult to reuse, simultaneously. The large KGs will have greater difficulties to achieve alignment.

Most of the existing entity alignment methods are based on embedding, TransE is one of the main technologies for KGs embedding. MTransE was the first method to extend the idea of the translation model into the field of cross-lingual entity alignment, designing three cross-lingual transformations of entities and relations. IPTransE jointly encoded entities and relations of different KGs into a unified semantic space through a small set of aligned entity seeds, and proposed an interactive strategy to expand the seed sets. Later BootEA transformed entity alignment into a classification problem and designed a bootstrapping process to reduce error accumulation by alignment editing method. Such methods rely on pre-aligned seed sets and cannot model complex relationships between entities.

Graph convolutional network (GCN) is another major technique for KGs embedding and has made significant progress in recent years. GCN-Align is the first method to use graph convolutional neural network for entity alignment, with entities of each language being embedded into a unified vector space through GCNs. While traditional GCN ignores the relations in KGs, Relational Graph Convolutional Networks (RGCN) [16] improved GCNs by specific matrix coding for each relation. But it also introduced a large number of parameters, which greatly increase the requirements on computational power. Vectorized relational graph convolutional network (AVR-GCN) [17] extended RGCN with similar translation feature. RDGCN constructed dual relation graphs to learn relation information in KGs. AliNet learned the structure of entities through GCNs and attention mechanism, alleviating the structure heterogeneity of KGs. NMN designed a graph sampling strategy to select more important entity neighborhoods and discovered new aligned entities through graph matching. The joint learning (HGCN) method [18] conducted joint learning on the structure and relation of entities, and performed relation alignment in an unsupervised manner. Attributed Graph Neural Network (AttrGNN) [19] learned the structure and attribute of entities respectively by dividing sub-graphs. However, the information union in these methods is still at the aspect level and the operation is relatively rough, resulting in the original advantages being consumed in the process of information union.

However, the information union in these methods is still at the aspect level and the operation is relatively rough, resulting that the original advantages will be consumed in the process of information union. Therefore, a fine fusion at the node level is proposed in this paper to greater play to the characteristics of various aspects of entities and enhance entity alignment.

3. Problem Definition

Formally, a KG is denoted as

G = {E, R, A, V, T_{R}, T_{A}}

, where

E

R

A

V

denote the sets of entities, relations, attributes and attribute values, respectively.

T_{R}

T_{A}

represent the sets of relation triples

< h, r, t >

and attribute triples

< e, a, v >

, where

h

t

e \in E

r \in R

a \in A

v \in V

In this paper, two knowledge graphs are defined in different languages as

G_{1}

and

G_{2}

Ψ

is used to denote the a set of pre-aligned seeds and

Ψ = {(e_{1}, e_{2}) |e_{1} \in E_{1}, e_{2} \in E_{2}}

will be considered as training data. The task is to discover potential equivalent entities in knowledge graphs by

Ψ

4. Method

In this section, the proposed model SFEA is first introduced. After that, the various parts of the model are described in detail. This method implements strong fusion embedding of attribute and structure through GCNs, attention mechanism and highway networks. And sub-alignment tasks are performed in a semi-supervised manner and entity representations are updated to achieve better alignment results using an iterative strategy finally.

4.1. Overview of SFEA

The overall architecture of the SFEA model is shown in Figure 3. First, the feedforward neural network is used to model nodes through attribute information, and names of importing entity resources. Here, the two are used as the foundation to expand upward to learn nodes of structure, forming strong fusion. Multi-layer GCNs are used to explore the structure of nodes while passing out the obtained alignment signal by layers. Then, the strong fusion entity representations are used to approximate the relation representations, and the impact of the relation information is mapped to the entity representations through two sub-alignment tasks and updated synchronously. Finally, the newly discovered entities are retrieved in the alignment section and iterated back into the training process.

In the Figure 3,

H_{e_{1}}^{o u t}

and

H_{e_{2}}^{o u t}

represent the entity representations learned from the two knowledge graphs, respectively,

H_{r_{1}}^{o u t}

and

H_{r_{2}}^{o u t}

represent the relation representations learned from the two knowledge graphs, respectively, then

S^{E}

S^{R}

and

S^{E^{o u t}}

respectively represent by distance calculation obtain the similarity matrix of initial entity alignment, the similarity matrix of relationship alignment and the optimized entity alignment matrix.

4.2. Strong Fusion Embedding of Attribute and Structure

In this stage, the attribute information of entities is first learned, so that nodes have alignment cues for attributes before learning structure information. Through multi-layer GCNs stacking, not only the multi-hop neighbor information of entities is learned, but also the structure and attribute alignment signals learned in each layer are transmitted to each neighbor node, to achieve strong fusion.

In the first step, the information is learned by a fully-connected (FC) layer, that is considered the entity vectors initialization. The hidden representations of attribute at 0th layer are denoted as

h_{a}^{0}

. Then, attribute information is learned in the form of triples of <entity, attribute, attribute value>, where entities and attribute values are regarded as nodes in the graph and attribute is regarded as edges between nodes, using GCN to encode it to represent the entity by aggregating the nodes (i.e., attribute values) around the entity, denoted as

h_{a}^{1}

, which is computed by:

h_{a}^{1} = σ (\sum_{i \in N} W^{1} h_{a_{i}}^{0}),

(1)

where

N

represents the set of all nodes connected to the central entity in the attribute graph,

W^{1}

denotes a learnable weight matrix, and

σ

is the

Re L U

function.

Entity name is initialized with pre-trained English word vectors by Glove, which is denoted as

h_{e}^{n a m e}

. Attribute representations and entity names vectors are aggregated through highway gating mechanism. As the initial vector of the learned structure, the calculation formula is computed as follows:

h_{e}^{0} = T (h_{a}^{1}) \cdot h_{e}^{n a m e} + (C (h_{a}^{1})) \cdot h_{a}^{1},

(2)

T (h_{a}^{1}) = s i g n m o i d (h_{a}^{1} W_{T} + b_{T}),

(3)

C = 1 - T,

(4)

where

T

is the transform gate that represents the proportion of the information entering the next layer after passing the GCN operation, and

C

is the carry gate which denotes the proportion of the information entering the next layer directly without the GCN operation.

W_{T}

denotes the weight matrix of the transform gate, and

b_{T}

denotes the bias of the transform gate.

Structure information (i.e., relation triples) is also learned by taking the form of a graph. In <head entity, relation, tail entity>, head entities and tail entities are as nodes in the graph and relation as edges. Learning the entity structure via GCNs, it also transmits the alignment signal of the attribute to each node. The input for GCN layer is a node feature matrix,

H_{e}^{0} = {h_{e_{1}}^{0}, h_{e_{2}}^{0}, \dots, h_{e_{n}}^{0} |h_{e_{i}}^{0} \in R^{d_{0}}}

, where n is the number of nodes of a KG,

d^{0}

is the number of features in 0th layer of structure, and

H_{e}^{0}

denotes the 1th layer of strong fusion and the 0th layer of learning structure information.

H_{e}^{1}

is the first layer of embedded structure information and the second layer of strong fusion. In this layer, the one-hop neighborhood of entities is modeled and learned based on attribute information and entity names. Due to a central node, there are a large number of neighbor nodes associated with it. But some of neighbor nodes cannot have an effective effect on the entity alignment of a central node and have the potential to bring noise because of their diversity. Therefore, attention mechanism is added [20] to calculate the weight between entities to highlight the more important parts of the neighborhood. The calculation formula is as follows:

H_{e}^{1} = σ (\sum_{j \in Ψ_{1} (i)} α_{i j} W^{1} H_{e_{j}}^{0} + H_{e_{i}}^{0}),

(5)

where

Ψ_{1} (i)

denotes the set of one-hop neighbor entities of entity

e_{i}

σ

is the

Re L U

function, and

W^{1}

is a learnable weight matrix.

s_{i j} = v^{T} \tanh (W H_{e_{j}}^{0} + q),

(6)

where

s_{i j} \geq 0

v

W

q

are learnable weight matrices.

Attention weights are normalized by the SoftMax function to enable comparability between different entities.

α_{i j} = s o f t \max (s_{i j}),

(7)

Stacking of multi-layer GCN can learn multi-order neighborhood structure, and

H_{e}^{l}

is updated using forward propagation as:

H_{e}^{l} = σ (\sum_{j \in Ψ_{l} (i)} α_{i j} W^{l} H_{e_{j}}^{(l - 1)} + H_{e_{i}}^{(l - 1)}),

(8)

where

Ψ_{l} (i)

denotes the set of l-hop neighbor entities of entity

e_{i}

σ

is the

Re L U

function, and

W^{l}

is a learnable weight matrix.

Aggregation between the GCN layers by highway gates,

H_{e}^{o u t (l + 1)}

is the final entity representations after inter-layer aggregation, the attribute-aware structure representations. The aggregation works as follow:

H_{e}^{out (l + 1)} = T (H_{e}^{l}) \cdot H_{e}^{(l + 1)} + (C (H_{e}^{l})) \cdot H_{e}^{l},

(9)

4.3. Updating Entity Representations by Sub-Tasks

In this stage, the relation representations are approximately calculated though the entity representations (the strong fusion representations) obtained in the previous stage. The entity representations are further updated by initial entity alignment and relation alignment.

4.3.1. Relation Embedding

In the relation triples

< h, r, t >

h

represents the head entities,

t

represents the tail entities, and

r

represents the relations connecting the head entities and the tail entities. Therefore, the surface semantic of the relations are related to the head entities and the tail entities. The relation representation

H_{r}

is approximately computed by the vector representation of the head entities and the tail entities from the previous stage.

H_{r} = g (H_{e_{h}}^{r}, H_{e_{t}}^{r}),

(10)

where

r \in R_{1} \cup R_{2}

{(e_{h}, e_{t} \in E_{1}) \cup (e_{h}, e_{t} \in E_{2}) |(e_{h}, r, e_{t}) \in (T_{1} \cup T_{2})}

H_{e_{h}}^{r}

H_{e_{t}}^{r}

denote the vector representation of different head entities and tail entities, which are connected to relation

r

g (,)

representing the operation of the mean after summing.

4.3.2. Initial Entity Alignment

Whether two entities are aligned or not is judged by the similarity between the entities (the distance between the two entities in the embedding space). If the distance between the entities is closer, the two entities are more similar and the probability of being equivalent entities is higher. Conversely, the farther distance and the smaller similarity imply the lower the possibility of being a pair of equivalent entities.

The alignment problem is achieved though finding the closest entity

e_{2}

from

G_{2}

to entity

e_{1}

from

G_{1}

. The distance is calculated as:

d i s (e_{1}, e_{2}) = {‖H_{e_{1}}^{o u t} - H_{e_{2}}^{o u t}‖}_{L_{1}},

(11)

where

{‖\cdot‖}_{L_{1}}

denotes the 1-norm measure for vectors (the Manhattan distance).

To enable the distance between aligned entity pairs to be as close as possible, and the distance between non-aligned entities to be as large as possible, the minimized margin-based loss function is as follows.

L_{E} = \sum_{(e_{1}, e_{2}) \in Ψ} \sum_{(e_{1}^{'}, e_{2}^{'}) \in Ψ^{'}} \max {0, d i s (e_{1}, e_{2}) - d i s (e_{1}^{'}, e_{2}^{'}) + γ},

(12)

where

Ψ

denotes a set of pre-aligned seed pairs,

Ψ^{'}

denotes the corresponding set of negative samples, which are generated by nearest neighbor sampling [21],

e_{1}^{'}

and

e_{2}^{'}

are negative entities representing

e_{1}

and

e_{2}

γ

is a margin hyper-parameter.

4.3.3. Relation Alignment

This sub-task has the same principle as entity alignment. It calculates the distance between two relations in the embedding space. The closer the distance is, the more similar the relations are.

d i s (r, r^{'}) = {‖H_{r} - H_{r^{'}}‖}_{L_{1}},

(13)

Based on the translation assumption from TransE, the vector representations of the tail entities can be obtained by summing the vectors of the head entities and the relations—“

h + r = t

”. A function to calculate relation loss,

L_{r}

is designed as:

L_{R} = \sum_{(h, r, t) \in (T_{R}^{1} \cup T_{R}^{2})} ‖h + r - t‖,

(14)

where

T_{R}^{1}

and

T_{R}^{2}

separately denote the sets of relation triples in two knowledge graphs.

It is also necessary to incorporate the relation information into the process of entities modeling. The relations are put into the entity representations. It updates the entity representations and relation representations via initial entity alignment and relation alignment and combines the entity loss and relation loss, to jointly learn the embeddings of entities and relation. The objective is as:

L = L_{E} + μ \cdot L_{R},

(15)

where

μ

is a coefficient used to balance the loss of entity alignment and the loss of relation alignment.

4.4. Iterative Strategy

In large KGs, the number of aligned seeds is usually insufficient and the cost of manually labeling aligned entities is too high. In order to extend the data for training and improve the practicality of the model, an iterative strategy is designed here. The newly aligned entities are added to the training set and fed back through the process again. Iterative strategy in SFEA is as follows (Algorithm 1).

Algorithm 1. Iterative strategy

Input: Knowledge Graphs

G_{1}

G_{2}

; Pre-aligned seed entity pairs

Ψ

Output: A set of aligned entities

Q

1. Repeat the initialization model:

2. for

e_{i} \in E_{1} | (G_{1} = (E_{1}, R_{1}, A_{1}, V_{1}, T_{R}^{1}, T_{A}^{1}))

3. for

e_{j} \in E_{2} | (G_{2} = (E_{2}, R_{2}, A_{2}, V_{2}, T_{R}^{2}, T_{A}^{2}))

d i s (e_{i}, e_{j})

5. Compare dis

\to e_{i}^{'}, (e_{i}^{'} \in E_{2})

//find the nearest entity in

G_{2}

;

Q = Ψ = Ψ \cup (e_{i}, e_{i}^{'})

e_{i} \in E_{1}, {e^{'}}_{i} \in E_{2}

7. End for

8. Until no more aligned entities

9. Return

Q

;

5. Experiments

5.1. Datasets

To evaluate the performance of the model (SFEA), the DBP15K is adopted, which is a subset of the large-scale knowledge graph DBpedia. DBP15K is a standard dataset commonly used for entity alignment experiments and it has a cross-language nature. It contains a total of four different languages and each language is widely used in the human world, including DBP_ZH-EN (Chinese-English), DBP_JA-EN (Japanese–English) and DBP_FR-EN (French–English), each with 15,000 reference entity alignments and about 400,000 triples. Moreover, to evaluate the performance of the model in hard environments, a sparse dataset S-DBP15K based on DBP15K is also used. This dataset is proposed by NMN. The number of relation triples in non-English languages in DBP15K is removed in different proportions to form S-DBP15K. By increasing the neighborhood differences of entities in different KGs, the structure information of different KGs is in an unbalanced state. Table 1 presents the detailed statistics of DBP15K and S-DBP15K. The entire experiment uses the same dataset split rate as previous studies, with 30% as training data and 70% as test data.

5.2. Experimental Settings

This experiment stacks two layers of GCNs to learn the two-hop neighborhood of entities. The dimension of hidden layer of GCN is set as 300. Adma is used as an optimizer, and the learning rate is 0.001. Google Translate is first used to translate all non-English entity names into English, and

g l o v e .840 B .300 d

are used to pre-train English word vectors. Besides, the nearest 100 entities and the nearest 20 relations are selected as the candidate sets. Each positive sample generates 125 negative samples. The margin

γ

and

ρ

are set as 1. SFEA is implemented in the environment of TensorFlow. The experiment is conducted on a server with Inter(R)Xeon(R)CPU E5-1630 v4 @ 3.70GHz NVIDIA Quadro K1200 GPU and 8GB memory.

5.3. Metrics

Following convention, Hits@1, Hits@10 and MRR are used as evaluation metrics. A Hits@k score is computed by measuring the proportion of correctly aligned entities ranked in the top k list. A higher Hits@k score indicates better performance. Hits@1 is equivalent to precision. Recall and F1-measure also have the same value as Hits@1, since the nearest neighbor search always finds the corresponding object for each entity to be aligned. MRR is the summation value of the reciprocal ranks of results. MRR is commonly used to measure the effectiveness of search algorithms. It is also widely used in the evaluation of entity alignment effectiveness.

5.4. Experimental Results

In this section, experimental results are listed to show this complete model consistently outperforms all competing approaches in data sets. Ablation experiments are performed to demonstrate the effectiveness of each part of the model. Then, the effect of pre-aligned seed data size on model performance is analyzed. The results show that this method requires significantly fewer seed pairs, but achieves better performance than the best prior method. Next, experiments are conducted on the neighborhood range of entities to analyze that learning n-hop neighborhood structures can have a better impact on entity alignment. Finally, the influence of different aspects of information on entity alignment is compared.

5.4.1. DBP15K Results

Table 2 shows the performance of different methods on the entity alignment task. The results of Hits@1 and Hits@10 are in percentage (%). Numbers in bold denote the best results among all models. The underline indicates the best effect in the comparison models. The experimental results show that SFEA significantly outperforms all baselines on DBP15K and further confirms the effectiveness of the proposed model.

This experiment contains some unmentioned entity alignment methods. Meta Relation Aware Entity Alignment (MRAEA) [22] is proposed as a meta-relations-aware entity alignment that directly models cross-language entity embedding, by focusing on the meta-semantics of the node’s incoming and outgoing neighbors and their connected relation. Yang et al. (HMAN) [23] applied GCNs to combine multi-aspect information of entities, and proposes two uses of a pre-trained multilingual BERT model to bridge cross-lingual gaps. Multiview highway graph convolutional network (MHGCN) [24] is employed as a highway graph convolutional network (GCN) for entity embedding in each view, weights and fuses the multiple views from each view. The Entity Visual Alignment (EVA) model [25] presents as an unsupervised approach and leverages visual similarities of entities to build a seed dictionary from scratch and expand the dictionary with iterative learning. The Structure and Semantics Preserving network (SSP) [26] jointly leveraged the global KG structure and local semantics preserving network for entity alignment.

Specifically, among all TransE-based models, BootEA performs the best because it adopts a bootstrap strategy to iteratively expand the seed alignments. This indicates that the iterative strategy can significantly improve the performance of entity alignment. For GCN-based models, in only considering structure information model, NMN has better results than AliNet, because of a more efficient graph sampling strategy. Among the models considering relation information, MRAEA has better results than RDGCN, because it mines more relational hidden semantics. HMAN and AttrGNN both consider attribute information. AttrGNN considers different types of attribute information, achieves the best results in the baselines, which proves the importance of attribute information for entity alignment. However, the proposed SFEA model can achieve all the values of Hits@1 to be higher than 85%, those of Hits@10 higher than 93%, and those of MRR higher than 0.88. As far as the better performing baseline models HGCN, AttrGNN, MHGCN and EVA are concerned, although these baseline models also use different aspects of information, they use simple merging at the aspect level in the process of information fusion. The information is not fully utilized to obtain rougher entity representations. However, SFEA closely integrates information from different aspects, and makes use of the advantages of each type of information to achieve better entity alignment.

Table 3 shows the comparison results of different methods on DBP15K for relation alignment. -R for models that compute approximately relation embedding though entity embedding. It can be seen from the experimental results SFEA achieves best results on relation alignment.

The relation embedding representations required for relation alignment are calculated from the embedding representation of the head entity and the embedding representation of the tail entity. Therefore, the high performance of SFEA in the relation alignment stage is affected by the entity embedding representation in the SFEA model. Compared with the best comparison model HGCN, SFEA has a significant improvement, because HGCN uses a simple addition merging method, which will cause insufficient use of information. The precise entity embeddings obtained by SFEA demonstrate that the model improvement is effective.

The relation alignment between HGCN and RDGCN are better in the comparison models because HGCN combines structure and relation, while RDGCN has the best performance in the comparison models because RDGCN uses dual relation graphs to improve relation learning. However, compared to all models, SFEA employs strong fusion and sub-alignment tasks to obtain approximate relation embeddings. The accurate entity embedding results obtained by SFEA show that SFEA has a significant improvement and the model improvement is effective.

5.4.2. Ablation Studies

Ablation studies are conducted on the proposed SFEA model and the results are shown in Table 4. Specifically, (1) (w/o SF) denotes information from different aspects is aggregated through the connection operation. (2) (w/o UE) denotes the entity representations are not updated by sub-alignment tasks. (3) (w/o IS) denotes the SFEA model without iteration strategy. From the experimental results, it can be observed that (w/o SF) has the largest performance drop, which confirms that the strong fusion strategy can obtain more accurate and realistic entity embedding representations. In addition, the sub-alignment tasks also play a role in updating the entity representations. Iterative strategy also improves model performance.

5.4.3. S-DBP15K Results

On a more difficult dataset, S-DBP15K for SFEA was evaluated. The experimental results are shown in Table 5. The performance of SFEA is significantly better than that of the baseline models, which proves that the SFEA model still shows good results on the heterogeneous KGs.

The analysis is as follows.

Comparing the experimental results of S-DBP15K and DBP15K, the performance of all models is degraded. This shows that with the lack of triple information, the structure information and relation information are greatly reduced, resulting in inaccurate entity representations learned by the models.
The SFEA achieves the best results in three different language environments. In the comparison models, BootEA and GCN-Align models perform very poorly on sparse datasets. BootEA and GCN-Align models rely too much on relation triples and entity structure, so entity alignment does not work well in sparse hard cases. The bootstrap iteration method is used in BootEA, and its effect is slightly better than the GCN-Align. RDGCN used structure information, relation information, and name information. When faced with insufficient structure information and relation information, attribute information in SFEA plays a certain role.
Comparing the experimental results of the deformation of the three variants of SFEA, the without strong fusion (w/o SF) has the largest drop, indicating that in the case of unbalanced structure information of KGs, the strong fusion method in the SFEA model plays a key role. The influence of insufficient structure information is alleviated by tightly merging various information at the node level. Without sub-tasks alignment module (w/o UE) and without iterative strategy (w/o IS) also plays a role in the stability of the model. They further optimize the entity representations.
NMN performs well in the comparison model because it adopts a neighborhood sampling strategy to alleviate the impact of data sparsity. Through the comparison of experimental results, the experimental results of (w/o IS) are already better than NMN, indicating the efficiency of the strong fusion and subtask update modules in the model.

5.4.4. Analysis

An experiment is conducted on the proportional sensitivity of pre-aligned seeds. The proportion of pre-aligned seed pairs is adjusted between 10% and 40% in steps of 10% at a time. MRAEA is the model with superior stability in baseline, as shown in Figure 4. From the previous introduction in this paper, it can be seen that relation embedding is calculated by entity embedding. So, SFEA is compared with HGCN-R, the baseline model with the best performance in the relation alignment comparison experiment, as shown in Figure 5. Hits@1 as a calibration indicator is used on DBP15K.

According to the analysis in Figure 4 and Figure 5, the alignment effectiveness of SFEA and comparison models on the DBP15K also gradually improve with the gradual increase in the proportion of pairs of pre-aligned seeds. In this proportion range, SFEA consistently achieves better results and a more gradual increase in performance compared to the comparison model. In other words, SFEA not only has better alignment accuracy and is insensitive to seed scale, but is more stable and robust.

Figure 6 shows that SFEA considers different hop counts for neighborhood information. It can be seen that the performance of SFEA also degrades when it has more layers of GCNs. Although more layers can learn more neighborhood information, those multi-hop neighborhoods may introduce more noise into the model. Therefore, the neighbors of two hops will obtain better results, which indicates that aggregating the neighbor information of two hops is enough.

Four aspects of information are respectively modeled, including attribute information (attr), relationship information (rel), structure information (struc) and entity name information (name). As shown in Figure 7, the entity alignment performance of these four aspects, SFEA (w\o SF), and SFEA model are studied. It can be concluded as follows:

In the comparison of four different aspects of information, it can be observed that entity names perform better than other alone aspects of information. The reason is that the English and non-English language versions in the DBP15K dataset are all from Wikipedia, with a large number of entity pairs for which the non-English entities can be converted into names that are highly similar or even the same as the corresponding English entities after machine translation. However, in practical situations, the alignment task needs to deal with the same objects with different names, and different objects with the same names. So, entity alignment cannot just align by entity name without considering other information, or placing entity names too high.
The effect of the (w\o SF) model is not obviously compared to using only structure information, which shows that this simple aspect-level connection operation does not efficiently integrate information from different aspects, and even existing alignment signals would be lost. In addition, the SFEA model has obvious improvement compared with other deformations. It is proven that the node-level strong fusion strategy can effectively utilize different entity information.

6. Conclusions

This paper proposes a new embedded entity matching framework. SFEA solves the problem of loss caused in the process of information fusion of different aspects and information unbalance in different KGs. In this model, a new node-level strong fusion method is used. By learning from all aspects of the information, and in multi-hop neighborhoods, node-level fusion becomes closer and closer, forming a more refined fusion process. Through the learning of multiple aspects of information and the attention mechanism and the highway gate mechanism, the dependence of the entity alignment task on the structure information of the neighborhood is dropped while the influence of the unimportant part of the neighborhood is reduced. It makes the model less susceptible to the heterogeneity of the knowledge graph. The entity representations are further optimized through the interaction between the sub-alignment task modules (initial entity alignment and relation alignment) and an iterative strategy is designed. Through extensive experimental verification, the entity alignment performance of the SFEA model is significantly improved and new SOTA performance is achieved.

For future work, different types of properties will be distinguished and learned; abstract or contextual information that relates to knowledge graphs and can be obtained will also be incorporated into entity alignment. In order to be closer to the real situation, subsequent work will consider exploring entity alignment in different scenarios. For example, the work will be carried out in the direction of entity alignment in knowledge graphs constructed under different data sources.

Author Contributions

S.L. wrote the paper and supervised the work; M.X. performed the experiments; Y.Q. conceived and designed the experiments; N.L. gave some suggestions on the experiments. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Liaoning Province Economic and Social Development Research Project 2023 of Provincial Social Science Association (grant no.2023lslybkt-039), 2019 National Natural Science Foundation of China (grant no.61876031).

Acknowledgments

The authors would like to thank all anonymous reviewers and editors for their helpful suggestions for the improvement of this paper.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

Lehmann, J.; Isele, R.; Jakob, M.; Jentzsch, A.; Kontokostas, D.; Mendes, P.N.; Hellmann, S.; Morsey, M.; van Kleef, P.; Auer, S.; et al. DBpedia—A large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web 2015, 6, 167–195. [Google Scholar] [CrossRef]
Suchanek, F.M.; Kasneci, G.; Weikum, G. Yago: A core of semantic knowledge. In Proceedings of the 16th International Conference on World Wide Web, Banff, AB, Canada, 8–12 May 2007; pp. 697–706. [Google Scholar]
Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; Yakhnenko, O. Translating embeddings for modeling multi-relational data. Adv. Neural Inf. Processing Syst. 2013, 2, 2787–2795. [Google Scholar]
Chen, M.; Tian, Y.; Yang, M.; Zaniolo, C. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, Lake Tahoe, NV, USA, 19–25 August 2016. [Google Scholar]
Zhu, H.; Xie, R.; Liu, Z.; Sun, M. Iterative entity alignment via knowledge embeddings. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), Montreal, QC, Canada, 19–27 August 2017; pp. 4258–4264. [Google Scholar]
Sun, Z.; Hu, W.; Zhang, Q.; Qu, Y. Bootstrapping Entity Alignment with Knowledge Graph Embedding. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 13–19 July 2018. [Google Scholar]
Kipf, T.N.; Welling, M. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations, Toulon, France, 24–27 April 2016. [Google Scholar]
Wang, Z.; Lv, Q.; Lan, X.; Zhang, Y. Cross-lingual knowledge graph alignment via graph convolutional networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October–4 November 2018; pp. 349–357. [Google Scholar]
Sun, Z.; Wang, C.; Hu, W.; Chen, M.; Dai, J.; Zhang, W.; Qu, Y. Knowledge graph alignment network with gated multi-hop neighborhood aggregation. In Proceedings of the AAAI Conference on Artificial Intelligence, Palo Alto, CA, USA, 11–15 October 2020; pp. 222–229. [Google Scholar]
Wu, Y.; Liu, X.; Feng, Y.; Wang, Z.; Yan, R.; Zhao, D. Relation-aware entity alignment for heterogeneous knowledge graphs. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), Macao, China, 10–16 August 2019; pp. 5278–5284. [Google Scholar]
Wu, Y.; Liu, X.; Feng, Y.; Wang, Z.; Zhao, D. Neighborhood matching network for entity alignment. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, online, 5–10 July 2020; pp. 6477–6487. [Google Scholar]
Sarasua, C.; Simperl, E.; Noy, N.F. Crowdmap: Crowdsourcing ontology alignment with microtasks. In Proceedings of the International Semantic Web Conference, Boston, MA, USA, 11–15 November 2012; pp. 525–541. [Google Scholar]
Mahdisoltani, F.; Biega, J.; Suchanek, F. Yago3: A knowledge base from multilingual wikipedias. In Proceedings of the 7th Biennial Conference on Innovative Data Systems Research, CIDR Conference, Asilomar, CA, USA, 4–7 January 2014. [Google Scholar]
Suchanek, F.M.; Abiteboul, S.; Senellart, P. Paris: Probabilistic alignment of relations, instances, and schema. VLDB Endow. 2011, 5, 157–168. [Google Scholar] [CrossRef]
Wang, X.; Liu, K.; He, S.; Liu, S.; Zhang, Y.; Zhao, J. Multi-source knowledge bases entity alignment by leveraging semantic tags. Chin. J. Comput. 2017, 40, 701–711. [Google Scholar]
Schlichtkrull, M.; Kipf, T.N.; Bloem, P.; Van Den Berg, R.; Titov, I.; Welling, M. Modeling Relational Data with Graph Convolutional Networks. In European Semantic Web Conference; Springer: Berlin/Heidelberg, Germany, 2018; pp. 593–607. [Google Scholar] [CrossRef]
Ye, R.; Li, X.; Fang, Y.; Zang, H.; Wang, M. A Vectorized Relational Graph Convolutional Network for Multi-Relational Network Alignment. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), Macao, China, 10–16 August 2019; pp. 4135–4141. [Google Scholar]
Wu, Y.; Liu, X.; Feng, Y.; Wang, Z.; Zhao, D. Jointly learning entity and relation representations for entity alignment. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP) and 9th International Joint Conference on Natural Language Processing (IJCNLP), Hong Kong, China, 3–7 November 2019; pp. 240–249. [Google Scholar]
Liu, Z.; Cao, Y.; Pan, L.; Li, J.; Chua, T.S. Exploring and evaluating attributes, values, and structures for entity alignment. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic, 8–12 November 2020; pp. 6355–6364. [Google Scholar]
Chaudhari, S.; Mithal, V.; Polatkan, G.; Ramanath, R. An attentive survey of attention models. ACM Trans. Intell. Syst. Technol. 2021, 12, 1–32. [Google Scholar] [CrossRef]
Mao, X.; Wang, W.; Wu, Y.; Lan, M. Are Negative Samples Necessary in Entity Alignment? An Approach with High Performance, Scalability and Robustness. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, New York, NY, USA, 1–5 November 2021; pp. 1263–1273. [Google Scholar]
Mao, X.; Wang, W.; Xu, H.; Lan, M.; Wu, Y. MRAEA: An efficient and robust entity alignment approach for cross-lingual knowledge graph. In Proceedings of the 13th International Conference on Web Search and Data Mining, Houston, TX, USA, 3–7 February 2020; pp. 420–428. [Google Scholar]
Yang, H.W.; Zou, Y.; Shi, P.; Lu, W.; Lin, J.; Sun, X. Aligning cross-lingual entities with multi-aspect information. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP) and 9th International Joint Conference on Natural Language Processing (IJCNLP), Hong Kong, China, 3–7 November 2019. [Google Scholar]
Gao, J.; Liu, X.; Chen, Y.; Xiong, F. MHGCN: Multiview highway graph convolutional network for cross-lingual entity alignment. Tsinghua Sci. Technol. 2022, 27, 719–728. [Google Scholar] [CrossRef]
Liu, C.; Collier, R. Visual pivoting for (unsupervised) entity alignment. AAAI Conf. Artif. Intell. 2021, 35, 4257–4266. [Google Scholar] [CrossRef]
Nie, H.; Han, X.; Sun, L.; Wong, C.M.; Chen, Q.; Wu, S.; Zhang, W. Global structure and local semantics-preserved embeddings for entity alignment. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, Yokohama, Japan, 7–15 January 2021; pp. 3658–3664. [Google Scholar]

Figure 1. An example of non-equivalent entities(structure-attribute).

Figure 2. An example of non-equivalent entities (structure-relation).

Figure 3. Overall architecture of SFEA with two-hop neighborhood aggregation.

Figure 4. Different proportions of pre-aligned seed (entity alignment).

Figure 5. Different proportions of pre-aligned seed (relation alignment).

Figure 6. Different layers of GCN (entity alignment).

Figure 7. Impact of different aspects of information (entity alignment).

Table 1. Statistics of DBP15K and S-DBP15K. (Rel.triples and Attr.triples stand for relations and attributes respectively. Tri.Remain in S-DBP15K Indicates the proportion of retained relation triples compared to DBP15K).

Datasets		DBP15K					S-DBP15K
Datasets		Entities	Relations	Attributes	Rel.Triples	Attr.Triples	Tri.Remain
ZH-EN	Chinese	66,469	2830	8113	153,929	379,684	26%
ZH-EN	English	98,125	2317	7173	237,674	567,755	100%
JA-EN	Japanese	65,744	2043	5882	164,373	354,619	41%
JA-EN	English	95,680	2096	6066	233,319	497,230	100%
FR-EN	French	66,858	1379	4547	192,191	528,665	45%
FR-EN	English	105,889	2209	6422	278,599	576,543	100%

Table 2. Performance on entity alignment.

Method	DBP15K_ZH-EN			DBP15K_JA-EN			DBP15K_FR-EN
Method	Hits@1	Hits@10	MRR	Hits@1	Hits@10	MRR	Hits@1	Hits@10	MRR
JAPE	0.412	0.745	0.490	0.363	0.685	0.476	0.324	0.667	0.430
BootEA	0.629	0.848	0.703	0.623	0.854	0.701	0.653	0.874	0.731
GCN-Align	0.413	0.744	0.549	0.399	0.745	0.546	0.373	0.745	0.532
AliNet	0.539	0.826	0.628	0.549	0.831	0.645	0.552	0.852	0.657
RDGCN	0.708	0.846	0.746	0.767	0.895	0.812	0.886	0.957	0.911
HGCN	0.720	0.857	0.768	0.766	0.897	0.813	0.892	0.961	0.917
NMN	0.733	0.869	0.781	0.785	0.912	0.827	0.902	0.967	0.924
MRAEA	0.757	0.929	0.827	0.757	0.933	0.826	0.780	0.948	0.849
HMAN	0.562	0.851	0.670	0.557	0.860	0.674	0.550	0.876	0.666
AttrGNN	0.796	0.929	0.845	0.783	0.921	0.834	0.918	0.977	0.910
MHGCN	0.732	0.867	0.793	0.767	0.891	0.821	0.864	0.923	0.898
EVA	0.761	0.907	0.814	0.762	0.913	0.817	0.793	0.942	0.847
SSP	0.739	0.925	0.808	0.721	0.935	0.800	0.739	0.947	0.818
SFEA	0.855	0.931	0.883	0.876	0.953	0.904	0.942	0.983	0.959

Table 3. Performance on relation alignment.

Method	DBP15KZH-EN		DBP15KJA-EN		DBP15KFR-EN
Method	Hits@1	Hits@10	Hits@1	Hits@10	Hits@1	Hits@10
MTransE-R	0.328	0.576	0.310	0.561	0.189	0.443
BootEA-R	0.453	0.854	0.414	0.798	0.302	0.604
GCN-Align-R	0.702	0.828	0.639	0.818	0.420	0.538
HGCN-R	0.703	0.854	0.650	0.836	0.425	0.566
RDGCN-R	0.785	0.861	0.718	0.845	0.476	0.582
SFEA	0.813	0.891	0.767	0.881	0.514	0.628

Table 4. Ablation study of the proposed model (entity alignment).

Method	DBP15K_ZH-EN			DBP15K_JA-EN			DBP15K_FR-EN
Method	Hits@1	Hits@10	MRR	Hits@1	Hits@10	MRR	Hits@1	Hits@10	MRR
SFEA (w/o SF)	0.625	0.764	0.675	0.645	0.771	0.691	0.753	0.845	0.786
SFEA (w/o UE)	0.818	0.924	0.859	0.840	0.941	0.877	0.923	0.979	0.945
SFEA (w/o IS)	0.816	0.911	0.852	0.846	0.937	0.881	0.925	0.977	0.945
SFEA	0.855	0.931	0.883	0.876	0.953	0.904	0.942	0.983	0.959

Table 5. Performance on entity alignment (S-DBP15K).

Method	S-DBP15K_ZH-EN		S-DBP15K_JA-EN		S-DBP15K_FR-EN
Method	Hits@1	Hits@10	Hits@1	Hits@10	Hits@1	Hits@10
BootEA	0.122	0.275	0.278	0.526	0.327	0.532
GCN-Align	0.114	0.264	0.213	0.451	0.292	0.489
HGCN	0.529	0.644	0.586	0.756	0.772	0.886
RDGCN	0.607	0.746	0.693	0.829	0.836	0.926
NMN	0.620	0.751	0.703	0.844	0.863	0.94
SFEA	0.665	0.787	0.751	0.868	0.883	0.955
(w/o SF)	0.550	0.696	0.655	0.804	0.761	0.868
(w/o UE)	0.609	0.767	0.728	0.856	0.821	0.923
(w/o IS)	0.630	0.780	0.741	0.866	0.846	0.945

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Liu, S.; Xu, M.; Qin, Y.; Lukač, N. Knowledge Graph Alignment Network with Node-Level Strong Fusion. Appl. Sci. 2022, 12, 9434. https://doi.org/10.3390/app12199434

AMA Style

Liu S, Xu M, Qin Y, Lukač N. Knowledge Graph Alignment Network with Node-Level Strong Fusion. Applied Sciences. 2022; 12(19):9434. https://doi.org/10.3390/app12199434

Chicago/Turabian Style

Liu, Shuang, Man Xu, Yufeng Qin, and Niko Lukač. 2022. "Knowledge Graph Alignment Network with Node-Level Strong Fusion" Applied Sciences 12, no. 19: 9434. https://doi.org/10.3390/app12199434

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Knowledge Graph Alignment Network with Node-Level Strong Fusion

Abstract

1. Introduction

2. Related Work

3. Problem Definition

4. Method

4.1. Overview of SFEA

4.2. Strong Fusion Embedding of Attribute and Structure

4.3. Updating Entity Representations by Sub-Tasks

4.3.1. Relation Embedding

4.3.2. Initial Entity Alignment

4.3.3. Relation Alignment

4.4. Iterative Strategy

5. Experiments

5.1. Datasets

5.2. Experimental Settings

5.3. Metrics

5.4. Experimental Results

5.4.1. DBP15K Results

5.4.2. Ablation Studies

5.4.3. S-DBP15K Results

5.4.4. Analysis

6. Conclusions

Author Contributions

Funding

Acknowledgments

Conflicts of Interest

References

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI