Abstract
We describe symmetries of feedforward networks in terms of their corresponding groups, which naturally act on and partition weight space. This leads to an algorithm that generates representative weight vectors in a specific fundamental domain. The closure of this domain turns out to be a manifold with singular points. We derive a canonical metric for the manifold that can be implemented efficiently even for large networks. One application would be the clustering of resulting weight vectors of an experiment in order to identify inadequate models or learning methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
A. Ossen and S.M. Rüger, “An analysis ofthe metric structure of the weight space of feedforward networks and its application to time series modeling and prediction”, in Proceedings of the European Symposium on Artificial Intelligence (ESANN), pp. 315–322. D Facto Publications: Brussels, 1996.
A.M. Chen, H.-m. Lu and R. Hecht-Nielsen, “On the geometry of feedforward neural network error surfaces”, Neural Computation, Vol. 5, No. 6, pp. 910–927, 1993.
R.O. Duda and P.E. Hart, Pattern Classification and SceneAnalysis, John Wiley & Sons: New York, 1973.
E.L. Lawler, J.K. Lenstra, A.H.G. Rinnooy Kan and D.B. Shmoys (eds), The Traveling Salesman Problem, John Wiley & Sons: Chichester, 1985.
I.D. Macdonald,The Theory of Groups, Oxford University Press: Oxford 1975.
H.J. Sussmann, “Uniqueness of the weights for minimal feedforward nets with a given input– output map”, Neural Networks, Vol. 5, pp. 589–593, 1992.
H. White, Artificial Neural Networks –Approximation & Learning Theory, Blackwell: Oxford/Cambridge, 1992.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Rüger, S.M., Ossen, A. The Metric Structure of Weight Space. Neural Processing Letters 5, 1–9 (1997). https://doi.org/10.1023/A:1009657318698
Issue Date:
DOI: https://doi.org/10.1023/A:1009657318698