Abstract
In this paper we consider large scale distributed committee machines where no local data exchange is possible between neural network modules. Regularization neural networks are used for both the modules as well as the combiner committee in an embedded architecture. After the committee training no module will know anything else except its own local data. This privacy preserving obligation is a challenging problem for trainable combiners but crucial in real world applications. Only classifiers in the form of binaries or agents can be sent to others to validate their local data and sent back average classification rates. From this fully distributed and privacy preserving mutual validation a coarse-grained matrix can be formed to map all members. We demonstrate that it is possible to fully exploit this mutual validation matrix to efficiently train another regularization network as a meta learner combiner for the committee.
Chapter PDF
Similar content being viewed by others
Keywords
References
Tresp, V.: Committee Machines. In: Hu, Y.H., Hwang, J.N. (eds.) Handbook of Neural Network Signal Processing. CRC Press LLC (2002)
Drucker, H.: Fast Committee Machines for Regression and Classification. In: KDD 1997 Proceedings (1997)
Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Computation 7, 219–269 (1995)
Evgeniou, T., Pontil, M., Poggio, T.: Regularization Networks and Support Vector Machines. Advances in Computational Mathematics 13, 1–50 (2000)
Poggio, T., Smale, S.: The mathematics of learning: Dealing with data. Notices of the American Mathematical Society 50(5), 537–544 (2003)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
Jain, A.K., Duin, R.P.W., Mao, J.: Statistical Pattern Recognition: A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(1), 4–37 (2000)
Wang, L., Fu, X.: Data Mining with Computational Intelligence. Springer (2005)
Bottou, L., Chapelle, O., DeCoste, D., Weston, J.: Large Scale Kernel Machines. Neural Information Processing Series. MIT Press, Cambridge (2007)
Kashima, H., Ide, T., Kato, T., Sugiyama, M.: Recent Advances and Trends in Large-scale Kernel Methods. IEICE Transactions on Information and systems E92-D (7), 1338–1353 (2009)
Prodromidis, A., Chan, P.: Meta-learning in a distributed data mining system: Issues and approaches. In: Proceedings of the Fourteenth International Conference on Machine Learning, pp. 211–218 (1998)
Clifton, C., Kantarcioglu, M., Vaidya, J., Lin, X., Zhu, M.: Tools for Privacy Preserving Distributed Data Mining. ACM SIGKDD Explorations 4(2), 1–7 (2003)
Kantarcioglu, M., Vaidya, J.: Privacy-Preserving Naive Bayes Classifier for Horizontally Partitioned Data. In: IEEE Workshop on Privacy-Preserving Data Mining (2003)
Yi, X., Zhang, Y.: Privacy-preserving naïve Bayes classification on distributed data via semi-trusted mixers. Information Systems 34(3), 371–380 (2009)
Yu, H., Jiang, X., Vaidya, J.: Privacy-Preserving SVM using nonlinear Kernels on Horizontally Partitioned Data. In: SAC Conference (2006)
Xiong, L., Chitti, S., Liu, L.: k nearest neighbour classification across multiple private databases. In: Proceedings of the ACM Fifteenth Conference on Information and Knowledge Management, November 5-11 (2006)
Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 993–1001 (1990)
Perrone, M.P., Cooper, L.N.: When networks disagree: ensemble method for neural networks. In: Mammone, R.J. (ed.) Neural Networks for Speech and Image Processing. Chapman & Hall, Boca Raton (1993)
Krogh, A., Vedelsby, J.: Neural networks ensembles, cross validation and active learning. In: Advances in Neural Information Processing Systems 7. MIT Press, Cambridge (1995)
Wilson, G.: Parallel Programming for Scientists and Engineers. MIT Press, Cambridge (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 IFIP International Federation for Information Processing
About this paper
Cite this paper
Kokkinos, Y., Margaritis, K.G. (2012). A Regularization Network Committee Machine of Isolated Regularization Networks for Distributed Privacy Preserving Data Mining. In: Iliadis, L., Maglogiannis, I., Papadopoulos, H. (eds) Artificial Intelligence Applications and Innovations. AIAI 2012. IFIP Advances in Information and Communication Technology, vol 381. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33409-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-33409-2_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33408-5
Online ISBN: 978-3-642-33409-2
eBook Packages: Computer ScienceComputer Science (R0)