Abstract
We consider the combination of the outputs of several classifiers trained independently for the same discrimination task. We introduce new results which provide optimal solutions in the case of linear combinations. We compare our solutions to existing ensemble methods and characterize situations where our approach should be preferred.
Preview
Unable to display preview. Download preview PDF.
References
Bates, J.M. and Granger, C.W.J. (1969): The Combination of Forecasts. Operational Research Quaterly, Vol. 20, 451–468.
Bishop, C.M. (1995): Neural Networks for Pattern Recognition, Clarendon Press, Oxford.
Guermeur, Y., d'Alché-Buc, F. and Gallinari, P. (1997): Combinaison Linéaire Optimale de Classifieurs. XXIX-ièmes Journées de Statistique, 425–428.
Guermeur, Y. and Gallinari, P. (1996): Combining Statistical Models for Protein Secondary Structure Prediction. ICANN'96, Bochum, 599–604.
LeBlanc, M. and Tibshirani, R. (1993): Combining estimates in regression and classification. Technical Report 9318, Department of Preventive Medicine and Biostatistics and Department of Statistics, University of Toronto.
Rosen, J.B. (1960): The Gradient Projection Method for Nonlinear Programming. Part 1. Linear Constraints. J. Soc. Indust. Appl. Math., Vol. 8, N o 1, 181–217.
Perrone, M.P. and Cooper, L.N. (1993): When Networks Disagree: Ensemble Methods for Hybrid Neural Networks. Technical Report, Institute for Brain and Neural Systems, Brown University, Providence, Rhode Island.
Rost, B. and Sander, C. (1993): Prediction of Protein Secondary Structure at Better than 70% Accuracy. J. Mol. Biol., 232, 584–599.
Wolpert, D.H. (1992): Stacked Generalization. Neural Networks, Vol. 5, 241–259.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Guermeur, Y., d'Alché-Buc, F., Gallinari, P. (1997). Optimal linear regression on classifier outputs. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, JD. (eds) Artificial Neural Networks — ICANN'97. ICANN 1997. Lecture Notes in Computer Science, vol 1327. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0020201
Download citation
DOI: https://doi.org/10.1007/BFb0020201
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63631-1
Online ISBN: 978-3-540-69620-9
eBook Packages: Springer Book Archive