Abstract
Graph embedding is one of the most efficient dimensionality reduction methods in machine learning and pattern recognition. Many local or global graph embedding methods have been proposed and impressive results have been achieved. However, little attention has been paid to the methods that integrate both local and global structural information without constructing complex graphs. In this paper, we propose a simple and effective global structure guided neighborhood preserving embedding method for dimensionality reduction called GSGNPE. Specifically, instead of constructing global graph, principal component analysis (PCA) projection matrix is first introduced to extract the global structural information of the original data, and then the induced global information is integrated with local neighborhood preserving structure to generate a discriminant projection. Moreover, the \(L_{2,1}\)-norm regularization is employed in our method to enhance the robustness to occlusion. Finally, we propose an iterative optimization algorithm to solve the proposed problem, and its convergence is also theoretically analyzed. Extensive experiments on four face and six non-face benchmark data sets demonstrate the competitive performance of our proposed method in comparison with the state-of-the-art methods.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Li Y, Chai Y, Yin H et al (2020) A novel feature learning framework for high-dimensional data classification. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-020-01188-2
Hu Q, Zhang L, Zhou Y et al (2018) Large-scale multimodality attribute reduction with multi-kernel fuzzy rough sets. IEEE Trans Fuzzy Syst 26(1):226–238
Li J, Mei C, Xu W et al (2015) Concept learning via granular computing: a cognitive viewpoint. Inf Sci 298:447–467
Qian J, Yang J, Xu Y et al (2020) Image decomposition based matrix regression with applications to robust face recognition. Pattern Recognit 102:107204
Shang R, Chang J, Jiao L et al (2019) Unsupervised feature selection based on self-representation sparse regression and local similarity preserving. Int J Mach Learn Cybern 10:757–770
Wang X, Dong L, Yan J (2012) Maximum ambiguity-based sample selection in fuzzy decision tree induction. IEEE Trans Knowl Data Eng 24:1491–1505
Wu W, Qian Y, Li T et al (2017) On rule acquisition in incomplete multi-scale decision tables. Inf Sci 378:282–302
Shahdoosti H, Tabatabaei Z (2020) Object-based feature extraction for hyperspectral data using firefly algorithm. Int J Mach Learn Cybern 11:1277–1291
Fang X, Teng S, Lai Z et al (2018) Robust latent subspace learning for image classification. IEEE Trans Neural Netw Learn Syst 29(6):2502–2515
Wang X, He Y (2016) Learning from uncertainty for big data: future analytical challenges and strategies. IEEE Syst Man Cybern Mag 2:26–31
Qian J, Yang J, Tai Y et al (2016) Exploring deep gradient information for biometric image feature representation. Neurocomputing 213:162–171
Ma M, Deng T, Wang N et al (2019) Semi-supervised rough fuzzy Laplacian eigenmaps for dimensionality reduction. Int J Mach Learn Cybern 10:397–411
Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemom Intell Lab Syst 2(1–3):37–52
Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720
Cai D, He X, Zhou K et al (2007) Locality sensitive discriminant analysis. In: Proceedings of 2007 international joint conference on artificial intelligence (IJCAI07), pp 1713–1726
Park S, Kwak N (2018) Independent component analysis by lp-norm optimization. Pattern Recognit 76:752–760
Mi J, Zhang Y, Li Y et al (2020) Generalized two-dimensional PCA based on \(\ell _{2, p}\)-norm minimization. Int J Mach Learn Cybern 11:2421–2438
Hu Q, Zhang S, Xie Z et al (2014) Noise model based \(\nu\)-support vector regression with its application to short-term wind speed forecasting. Neural Netw 57:1–11
Lai Z, Bao J, Kong H et al (2020) Discriminative low-rank projection for robust subspace learning. Int J Mach Learn Cybern 11:2247–2260
Jenssen R (2010) Kernel entropy component analysis. IEEE Trans Pattern Anal Mach Intell 32(5):847–860
Xiong F, Gou M, Camps O et al (2014) Person re-identification using kernel-based metric learning methods. In: Proceedings of the European conference on computer vision, pp 1–16
Li H, Jiang T, Zhang K (2004) Efficient and robust feature extraction by maximum margin criterion. IEEE Trans Neural Netw 17(1):157–165
Tenenbaum J, De Silva V, Langford J (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323
Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326
Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396
Fang X, Xu Y, Li X et al (2018) Regularized label relaxation linear regression. IEEE Trans Neural Netw Learn Syst 29(4):1006–1018
He X, Niyogi P (2003) Locality preserving projections. In: Proceedings of the 16th international conference on neural information processing systems, pp 153–160
He X, Cai D, Yan S et al (2005) Neighborhood preserving embedding. In: Proceedings of the tenth IEEE international conference on computer vision (ICCV05), pp 1208–1213
Pang Y, Zhang L, Liu Z et al (2005) Neighborhood preserving projections (NPP): a novel linear dimension reduction method. In: Proceedings of international conference on intelligent computing, pp 117–125
Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recognit 43(1):331–341
Cai W (2017) A dimension reduction algorithm preserving both global and local clustering structure. Knowl Based Syst 118:191–203
Fang X, Han N, Wong W et al (2019) Flexible affinity matrix learning for unsupervised and semisupervised classification. IEEE Trans Neural Netw Learn Syst 30(4):1133–1149
Yin M, Gao J, Lin Z (2016) Laplacian regularized low-rank representation and its applications. IEEE Trans Pattern Anal Mach Intell 38(3):504–517
Liu Z, Shi K, Zhang K et al (2020) Discriminative sparse embedding based on adaptive graph for dimension reduction. Eng Appl Artif Intell 94:103758
Shen X, Liu S, Bao B et al (2020) A generalized least-squares approach regularized with graph embedding for dimensionality reduction. Pattern Recognit 98:107023
Gou J, Yi Z, Zhang D et al (2018) Sparsity and geometry preserving graph embedding for dimensionality reduction. IEEE Access 6:75748–75766
Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Graph Stat 15(2):265–286
Hu Q, Li L, Zhu P (2013) Exploring neighborhood structures with neighborhood rough sets in classification learning. In: Rough Sets and Intelligent Systems-Professor Zdzisław Pawlak in Memoriam, Springer, pp 277–307
Qian J, Yang J, Zhang N et al (2014) Histogram of visual words based on locally adaptive regression kernels descriptors for image feature extraction. Neurocomputing 129:516–527
Golub G, Van Loan C (1996) Matrix computations. Johns Hopkins University Press, Baltimore
Nie F, Huang H, Cai X et al (2010) Efficient and robust feature selection via joint \(\ell _{2,1}\)-norms minimization. In: Advances in neural information processing systems, pp 1813–1821
Yang Y, Shen H, Ma Z et al (2011) \(\ell _{2,1}\)-norm regularized discriminative feature selection for unsupervised learning. In: Proceedings of the 22nd international joint conference on artificial intelligence, pp 1589–1594
Lai Z, Mo D, Wong W et al (2018) Robust discriminant regression for feature extraction. IEEE Trans Cybern 48(8):2472–2484
Wen J, Han N, Fang X et al (2019) Low-rank preserving projection via graph regularized reconstruction. IEEE Trans Cybern 49(4):1279–1291
Nie F, Zhu W, Li X (2019) Structured graph optimization for unsupervised feature selection. IEEE Trans Knowl Data Eng 33(3):1210–1222
Phillips P, Moon H, Rizvi S et al (2000) The FERET evaluation methodology for face-recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104
Martinez A (1998) The AR face database. CVC Tech. Report#24
Sim T, Baker S, Bsat M (2002) The CMU pose, illumination, and expression (PIE) database. In: Proceedings of fifth IEEE international conference on automatic face gesture recognition, pp 53–58
Georghiades A, Belhumeur P, Kriegman D (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660
Pohlert T (2014) The pairwise multiple comparison of mean ranks package (PMCMR). R Packag 27(2020):10
Benavoli A, Corani G, Mangili F (2016) Should we really use post-hoc tests based on mean-ranks. J Mach Learn Res 17(1):152–161
van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(2605):2579–2605
Acknowledgements
The author would like to thank the Editor-in-Chief, editors and anonymous reviewers for their kind help and valuable comments. The work was supported in part by the National Natural Science Foundation of China (Nos. 61806127, 62076164, 61976145), in part by Guangdong Basic and Applied Basic Research Foundation (No. 2021A1515011861), in part by Shenzhen Institute of Artificial Intelligence and Robotics for Society, in part by Shenzhen Science and Technology Program (No. JCYJ20210324094601005), and in part by the Bureau of Education of Foshan (Nos. 2019XJZZ05).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gao, C., Li, Y., Zhou, J. et al. Global structure-guided neighborhood preserving embedding for dimensionality reduction. Int. J. Mach. Learn. & Cyber. 13, 2013–2032 (2022). https://doi.org/10.1007/s13042-021-01502-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-021-01502-6