Abstract
Recently, multi-hop reasoning over incomplete Knowledge Graphs (KGs) to predict missing facts has attracted widespread attention due to its desirable effectiveness and interpretability. It typically adopts the Reinforcement Learning (RL) framework and traverses over the KG to reach the target answer and find evidential paths. However, existing methods often give all reached paths equal hit rewards. Intuitively, not all paths have the same contribution to the proof of the reasoning process. Moreover, the severely sparse rewards obtained after a multi-step traversal are usually insufficient to encourage a sophisticated RL-based model to work well. In order to tackle the above two problems, we propose a novel Counterfactual-guided and Curiosity-driven Knowledge Graph multi-hop Reasoning model (CoCuKGR). CoCuKGR constructs counterfactual relation reasoning tasks to estimate the semantic contribution to the query relation of each path and give each arrival path a different soft reward that can distinguish its validity. In addition, our method leverages the curiosity mechanism to generate curiosity-driven intrinsic rewards, which can not only alleviate the reward sparsity issue but also drive the agent to explore the environment more thoroughly to find more abundant paths. Experimental results show that our proposed model outperforms existing multi-hop reasoning methods significantly.
Supported by the National Key R&D Program of China under Grant Nos. 2021ZD0112501 and 2021ZD0112502; the National Natural Science Foundation of China under Grant Nos. 62172185 and 61876069; Jilin Province Key Scientific and Technological Research and Development Project under Grant Nos. 20180201067GX and 20180201044GX; and Jilin Province Natural Science Foundation under Grant No. 20200201036JC.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, vol. 26 (2013)
Das, R., et al.: Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. arXiv preprint arXiv:1711.05851 (2017)
Li, R., Cheng, X.: Divine: a generative adversarial imitation learning framework for knowledge graph reasoning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2642–2651 (2019)
Lin, X.V., Socher, R., Xiong, C.: Multi-hop knowledge graph reasoning with reward shaping. arXiv preprint arXiv:1808.10568 (2018)
Meilicke, C., Chekol, M.W., Ruffinelli, D., Stuckenschmidt, H.: Anytime bottom-up rule learning for knowledge graph completion. In: IJCAI, pp. 3137–3143 (2019)
Pathak, D., Agrawal, P., Efros, A.A., Darrell, T.: Curiosity-driven exploration by self-supervised prediction. In: International Conference on Machine Learning. PMLR, pp. 2778–2787 (2017)
Shen, Y., Chen, J., Huang, P.-S., Guo, Y., Gao, J.: M-walk: learning to walk over graphs using monte carlo tree search. arXiv preprint arXiv:1802.04394 (2018)
Trouillon, T.P., Bouchard, G.M.: Complex embeddings for simple link prediction, November 23 2017. US Patent App. 15/156,849
Wan, G., Pan, S., Gong, C., Zhou, C., Haffari, G.: Reasoning like human: hierarchical reinforcement learning for knowledge graph reasoning. In: IJCAI, pp. 1926–1932 (2020)
Xiong, W., Hoang, T., Wang, W.Y.: Deeppath: a reinforcement learning method for knowledge graph reasoning. arXiv preprint arXiv:1707.06690 (2017)
Yang, B., Yih, W., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shi, D., Li, A., Yang, B. (2022). Counterfactual-Guided and Curiosity-Driven Multi-hop Reasoning over Knowledge Graph. In: Bhattacharya, A., et al. Database Systems for Advanced Applications. DASFAA 2022. Lecture Notes in Computer Science, vol 13245. Springer, Cham. https://doi.org/10.1007/978-3-031-00123-9_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-00123-9_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-00122-2
Online ISBN: 978-3-031-00123-9
eBook Packages: Computer ScienceComputer Science (R0)