[go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

QPGCN: graph convolutional network with a quadratic polynomial filter for overcoming over-smoothing

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Graph convolutional networks have developed rapidly these years. Over-smoothing is an important factor that makes it difficult to deepen the networks, affecting the further development of graph convolutional networks. There have been some studies to solve the over-smoothing issue. However, the shapes of the filters in the existing graph convolutional networks are fixed for various data, and the convolution operations in some networks require eigendecomposition. Starting from the relationship between filters and over-smoothing, we propose a novel way to overcome over-smoothing in this paper. In this way, a novel quadratic polynomial filter (QPF) is proposed, and then a quadratic polynomial graph convolutional network (QPGCN) is derived. Without increasing the complexity of the network, QPGCN can adaptively learn the shape of QPF and does not require eigendecomposition, which can better alleviate over-smoothing. The extensive experiments on Cora, Citeseer, Pubmed, DBLP and Ogbn-arxiv datasets show that QPGCN achieves state-of-the-art performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. If all f(λi) = 1,i ∈ [N], \(\text {diag}\left (F(\vec {\lambda }) \right ) = I_{N}\) and the convolution operation will be \(F \star _{\mathcal {G}} \boldsymbol {x} = U\left [ I_{N} \right ] U^{T}\boldsymbol {x} = \boldsymbol {x} \).

  2. Since GCN employed an augmented adjacency matrix, an approximate filter \(F_{GCN}(\vec {\lambda } ) \approx \vec {1}-\bar {d}/\left (\bar {d}+1 \right ) \vec {\lambda }\) was used [1], where \(\bar {d}\) is the average degree.

References

  1. Balcilar M, Renton G, Héroux P, Gaüzère B, Adam S, Honeine P (2020) Bridging the gap between spectral and spatial domains in graph neural networks. arXiv:abs/2003.11702

  2. Bo D, Wang X, Shi C, Shen H (2021) Beyond low-frequency information in graph convolutional networks. In: Thirty-fifth AAAI conference on artificial intelligence, AAAI 2021, thirty-third conference on innovative applications of artificial intelligence, IAAI 2021, the eleventh symposium on educational advances in artificial intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. https://ojs.aaai.org/index.php/AAAI/article/view/16514. AAAI Press, pp 3950–3957

  3. Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In: 2nd international conference on learning representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, conference track proceedings

  4. Chen J, Ma T, Xiao C (2018) Fastgcn: fast learning with graph convolutional networks via importance sampling. In: 6th International conference on learning representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, conference track proceedings

  5. Chung F (1997) Spectral graph theory. Published for the Conference Board of the mathematical sciences by the American Mathematical Society, Providence

    MATH  Google Scholar 

  6. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in neural information processing systems 29: annual conference on neural information processing systems 2016, December 5-10, 2016, Barcelona, Spain, pp 3837–3845

  7. Duvenaud D, Maclaurin D, Aguilera-Iparraguirre J et al (2015) Convolutional networks on graphs for learning molecular fingerprints. In: Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pp 2224–2232

  8. Gao H, Wang Z, Ji S (2018) Large-scale learnable graph convolutional networks. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, KDD 2018, London, UK, August 19-23, 2018. ACM, pp 1416–1424

  9. Gilmer J, Schoenholz S S, Riley P F, Vinyals O, Dahl G E (2017) Neural message passing for quantum chemistry. In: Proceedings of the 34th international conference on machine learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, proceedings of machine learning research, vol 70. PMLR, pp 1263–1272

  10. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. 2005 IEEE international joint conference on neural networks, 2005, vol 2, pp 729–734

  11. Hamilton W L, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, december 4-9, 2017, Long Beach, CA, USA, pp 1024–1034

  12. Hammond D K, Vandergheynst P, Gribonval R (2011) Wavelets on graphs via spectral graph theory. Appl Comput Harmon Anal 30(2):129–150. https://doi.org/10.1016/j.acha.2010.04.005

    Article  MathSciNet  MATH  Google Scholar 

  13. Henaff M, Bruna J, LeCun Y (2015) Deep convolutional networks on graph-structured data. arXiv:1506.05163v1

  14. Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J (2020) Open graph benchmark: datasets for machine learning on graphs. arXiv:2005.00687

  15. Huang W-, Zhang T, Rong Y, Huang J (2018) Adaptive sampling towards fast graph representation learning. In: Advances in neural information processing systems 31: annual conference on neural information processing systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp 4563–4572

  16. Kipf T N, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: 5th International conference on learning representations, ICLR 2017, Toulon, France, April 24-26, 2017, conference track proceedings

  17. Levie R, Monti F, Bresson X, Bronstein M M (2019) Cayleynets: graph convolutional neural networks with complex rational spectral filters. IEEE Trans Signal Process 67(1):97–109. https://doi.org/10.1109/TSP.2018.2879624

    Article  MathSciNet  MATH  Google Scholar 

  18. Li Q, Han Z, Wu X-M (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the thirty-second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, pp 3538–3545

  19. Li R, Wang S, Zhu F, Huang J (2018) Adaptive graph convolutional neural networks. In: Proceedings of the thirty-second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, pp 3546–3553

  20. Min Y, Wenkel F, Wolf G (2020) Scattering GCN: overcoming oversmoothness in graph convolutional networks. In: Advances in neural information processing systems 33: annual conference on neural information processing systems 2020, NeurIPS 2020, December 6-12, 2020, virtual

  21. Monti F, Boscaini D, Masci J, Rodolà E, Svoboda J, Bronstein M M (2017) Geometric deep learning on graphs and manifolds using mixture model cnns. In: 2017 IEEE Conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, pp 5425–5434

  22. Niepert M, Ahmed M, Kutzkov K (2016) Learning convolutional neural networks for graphs. In: Proceedings of the 33nd international conference on machine learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, JMLR workshop and conference proceedings, vol 48. JMLR.org, pp 2014–2023

  23. Oono K, Suzuki T (2020) Graph neural networks exponentially lose expressive power for node classification. In: 8th International conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020

  24. Pan S, Wu J, Zhu X, Zhang C, Wang Y (2016) Tri-party deep network representation. In: Proceedings of the twenty-fifth international joint conference on artificial intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016. IJCAI/AAAI Press, pp 1895–1901

  25. Rong Y, Huang W, Xu T, Huang J (2020) Dropedge: towards deep graph convolutional networks on node classification. In: 8th International conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020

  26. Sen P, Namata G, Bilgic M, Getoor L, Gallagher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93–106. https://doi.org/10.1609/aimag.v29i3.2157

    Google Scholar 

  27. Susnjara A, Perraudin N, Kressner D, Vandergheynst P (2015) Accelerated filtering on graphs using Lanczos method. arXiv:1509.04537v3

  28. Velickovic P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: 6th International conference on learning representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, conference track proceedings

  29. Wang K, Shen Z, Huang C et al (2020) Microsoft academic graph: when experts are not enough. Quant Sci Stud 1(1):396–413. https://doi.org/10.1162/qss_a_00021, https://direct.mit.edu/qss/article-pdf/1/1/396/1760880/qss_a_00021.pdf

    Article  Google Scholar 

  30. Wu F, Jr A H S, Zhang T, Fifty C, Yu T, Weinberger K Q (2019) Simplifying graph convolutional networks. In: Proceedings of the 36th international conference on machine learning, ICML 2019, 9-15 june 2019, Long Beach, California, USA, proceedings of machine learning research, vol 97. PMLR, pp 6861–6871

  31. Wu J, He J, Xu J (2019) Demo-net: degree-specific graph neural networks for node and graph classification. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019. ACM, pp 406–415

  32. Zhao L, Akoglu L (2020) Pairnorm: tackling oversmoothing in gnns. In: 8th International conference on learning representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020

  33. Zhou K, Huang X, Li Y, Zha D, Chen R, Hu X (2020) Towards deeper graph neural networks with differentiable group normalization. In: Larochelle H, Ranzato M, Hadsell R et al (eds) Advances in neural information processing systems 33: annual conference on neural information processing systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. https://proceedings.neurips.cc/paper/2020/hash/33dd6dba1d56e826aac1cbf23cdcca87-Abstract.htmlhttps://proceedings.neurips.cc/paper/2020/hash/33dd6dba1d56e826aac1cbf23cdcca87-Abstract.html

  34. Zhuang C, Ma Q (2018) Dual graph convolutional networks for graph-based semi-supervised classification. In: Proceedings of the 2018 World Wide Web conference on World Wide Web, WWW 2018, Lyon, France, April 23-27, 2018. ACM, pp 499–508

Download references

Acknowledgments

This study was supported by Science Research Project of Liaoning Department of Education of China (No. LJKZ0008) and Science and Technology Development Project of Liaoning Province of China (No. 2021JH6/10500127).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Northeastern University, Chuangxin Road, Shenyang, 110169, Liaoning, China

    Gongce Wu, Shukuan Lin, Xiaoxue Shao, Peng Zhang & Jianzhong Qiao

Authors
  1. Gongce Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shukuan Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoxue Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianzhong Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shukuan Lin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, G., Lin, S., Shao, X. et al. QPGCN: graph convolutional network with a quadratic polynomial filter for overcoming over-smoothing. Appl Intell 53, 7216–7231 (2023). https://doi.org/10.1007/s10489-022-03836-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03836-2

Keywords