Abstract
We consider the problem of writing on a Hopfield network. We cast the problem as a supervised learning problem by observing a simple link between the update equations of Hopfield network and recurrent neural networks. We compare the new writing protocol to existing ones and experimentally verify its effectiveness. Our method not only has a better ability of noise recovery, but also has a bigger capacity compared to the other existing writing protocols.
This work is supported partly by China 973 program (No. 2015CB358700), by the National Natural Science Foundation of China (No. 61772059, 61421003), and by the Beijing Advanced Innovation Center for Big Data and Brain Computing (BDBC) and State Key Laboratory of Software Development Environment (No. SKLSDE-2018ZX-17).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
More generally, function \(f_W\) may take the form \(f_W(\mathbf{x})=\mathrm {sign}(W\mathbf{x}+\mathbf{b})\), where \(\mathbf{b}\) is an off-set or threshold vector. In the context of this paper, dropping this offset term \(\mathbf{b}\) is without loss of generality.
References
Anderson, J.R., Bower, G.H.: Human Associative Memory. Psychology Press, London (2014)
Bottou, L.: Largescale machine learning with stochastic gradient descent. In: Lechevallier, Y., Saporta, G. (eds.) Proceedings of COMPSTAT 2010, pp. 177–186. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-7908-2604-3_16
Cheng, K.S., Lin, J.S., Mao, C.W.: The application of competitive hopfield neural network to medical image segmentation. IEEE Trans. Med. Imaging 15(4), 560–567 (1996)
Ding, J., Sun, Y.Z., Tan, P., Ning, Y.: Detecting communities in networks using competitive hopfield neural network. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2018)
Duan, S., Dong, Z., Hu, X., Wang, L., Li, H.: Small-world hopfield neural networks with weight salience priority and memristor synapses for digit recognition. Neural Comput. Appl. 27(4), 837–844 (2016)
Elman, J.L.: Finding structure in time. Cogn. Sci. 14, 179–211 (1990)
Folli, V., Leonetti, M., Ruocco, G.: On the maximum storage capacity of the hopfield model. Front. Comput. Neurosci. 10, 144 (2017)
Hillar, C., SohlDickstein, J., Koepsell, K.: Efficient and optimal binary hopfield associative memory storage using minimum probability flow. arXiv preprint arXiv:1204.2916 (2012)
Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982)
Kanter, I., Sompolinsky, H.: Associative recall of memory without errors. Phys. Rev. A 35(1), 380 (1987)
Kobayashi, M.: Multistate vector product hopfield neural networks. Neurocomputing 272, 425–431 (2018)
Krotov, D., Hopfield, J.J.: Dense associative memory for pattern recognition. In: Advances in Neural Information Processing Systems, pp. 1172–1180 (2016)
Lee, K.Y., Sode-Yome, A., Park, J.H.: Adaptive hopfield neural networks for economic load dispatch. IEEE Trans. Power Syst. 13(2), 519–526 (1998)
McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)
McEliece, R., Posner, E., Rodemich, E., Venkatesh, S.: The capacity of the hopfield associative memory. IEEE Trans. Inform. Theory 33(4), 461–482 (1987)
Rebentrost, P., Bromley, T.R., Weedbrook, C., Lloyd, S.: Quantum hopfield neural network. Phys. Rev. A 98(4), 042308 (2018)
Storkey, A.: Increasing the capacity of a hopfield network without sacrificing functionality. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 451–456. Springer, Heidelberg (1997). https://doi.org/10.1007/BFb0020196
Wang, S.: Classification with incomplete survey data: a hopfield neural network approach. Comput. Oper. Res. 32(10), 2583–2594 (2005)
Zhen, H., Wang, S.N., Zhou, H.J.: Unsupervised prototype learning in an associative-memory network. arXiv preprint arXiv:1704.02848 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Bao, H., Zhang, R., Mao, Y., Huai, J. (2019). Writing to the Hopfield Memory via Training a Recurrent Network. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11671. Springer, Cham. https://doi.org/10.1007/978-3-030-29911-8_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-29911-8_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29910-1
Online ISBN: 978-3-030-29911-8
eBook Packages: Computer ScienceComputer Science (R0)