Abstract
In this paper, we introduce a new error measure, integrated reconstruction error (IRE) and show that the minimization of IRE leads to principal eigenvectors (without rotational ambiguity) of the data covariance matrix. Then, we present iterative algorithms for the IRE minimization, where we use the projection approximation. The proposed algorithm is referred to as COnstrained Projection Approximation (COPA) algorithm and its limiting case is called COPAL. Numerical experiments demonstrate that these algorithms successfully find exact principal eigenvectors of the data covariance matrix.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Abed-Meraim K., Chkeif A., Hua Y. (2000), Fast orthonormal PAST algorithm. IEEE Signal Processing Letters 7(3): 60–62
Ahn J.H., Choi S. and Oh J.H.: A new way of PCA: integrated-squared-error and EM algorithms. In: Proceedings IEEE Int’l Conference Acoustics, Speech, and Signal Processing. Montreal, Canada (2004).
Ahn J.H., Oh J.H. (2003), A constrained EM algorithm for principal component analysis. Neural Computation 15(1): 57–65
Baldi P., Hornik K. (1989), Neural networks for principal component analysis: learning from examples without local minima. Neural Networks 2: 53–58
Brockett R.W. (1991), Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems. Linear Algebra and Applications 146: 79–91
Choi S. On Variations of Power Iteration. In: Proceedings Int’l Conference Artificial Neural Networks, 2: 145–150, Warsaw, Poland: Springer (2005).
Cichocki A., Unbehauen R.(1992), Neural networks for computing eigenvalues and eigenvectors. Biological Cybernetics 68: 155–164
Diamantaras K.I., Kung S.Y. (1996). Principal Component Neural Networks: Theory and Applications. New York, John Wiely & Sons Inc
Hua Y., Xiang Y., Chen T., Abed-Meraim K., Miao Y. (1999), A new look at the power method for fast subspace tracking. Digital Signal Processing 9: 297–314
Jolliffe I.T. (2002). Principal Component Analysis, 2nd edn. Berlin, Spinger-Verlag
Oja E. (1989), Neural networks, principal component analysis, and subspaces. International Journal of Neural Systems 1: 61–68
Roweis S.T.: EM algorithms for PCA and SPCA, In: Advances in Neural Information Processing Systems, 10: 626–632, Cambridge, MA: MIT press (1998).
Sanger T.D. (1989), Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Networks 2(6): 459–473
Tipping M.E., Bishop C.M.(1999), Probabilistic principal component analysis. Journal of the Royal Statistical Society B 61(3): 611–622
Xu L. (1993), Least MSE reconstruction: a principle for self-organizing nets. Neural Networks 6: 627–648
Yang B. (1995), Projection approximation subsapce tracking. IEEE Transaction Signal Processing 43(1): 95–107
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License ( https://creativecommons.org/licenses/by-nc/2.0 ), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Choi, S., Ahn, JH. & Cichocki, A. Constrained Projection Approximation Algorithms for Principal Component Analysis. Neural Process Lett 24, 53–65 (2006). https://doi.org/10.1007/s11063-006-9011-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-006-9011-z