Abstract
In general, Q-learning needs well-defined quantized state spaces and action spaces to obtain an optimal policy for accomplishing a given task. This makes it difficult to be applied to real robot tasks because of poor performance of learned behavior due to the failure of quantization of continuous state and action spaces. In this paper, we proposed a fuzzy-based CMAC method to calculate the contribution of each neighboring state to generate a continuous action value in order to make motion smooth and effective. A momentum term to speed up training has been designed and implemented in a multi-agent system for real robot applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Kaebling, L., Littman, M.L., Moore, A.W.: Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research 5(4), 237–285 (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press/Bradford Books (March 1998)
Balch, T.: Learning Roles: Behavioral Diversity in Robot Teams. In: Sec. 8
Tan, M.: Multi-Agent Reinforcement Learning: Independent vs. Cooperative Agents. In: Proceedings of the Tenth International Conference on Machine Learning, pp. 330–337 (June 1993)
Jong, E.D.: Non-Random Exploration Bonuses for Online Reinforcement Learning. In: Sec. 8
Claus, C., Boutilier, C.: The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems. In: Sec. 8
Hwang, K.S., Lin, C.S.: Smooth Trajectory Traking of Three-Link Robot: A Self-Organizing CMAC Approach. IEEE Transactions on Systems, Man, and Cybernetics—PART B: Cybernetics 28(5) (1998)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning Internal Representations by Error Propagation. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing: Exploration in the Microstructure of Cognition, vol. 1, ch. 8, pp. 318–362. MIT Press, Cambridge (1986)
Takahashi, Y., Takeda, M., Asada, M.: Continuous Valued Q-learning for Vision-Guided Behavior Acquistion. In: Proceedings of 1999 IEEE/SICE/RSJ International Conference on Multisensor Fusion and Intergration for Intelligent Systems, pp. 255–260 (1999)
Brown, M., Harris, C.J.: A Perspective and Critique of Adaptive Neurofuzzy Systems Used for Modeling and Control Applications. Int. J. Neural Syst. 6(2), 197–220 (1995)
Jou, C.-C.: A fuzzy Cerebellar Model Articulation Controller. In: IEEE Int. Conf. Fuzzy Systems, March 1992, pp. 1171–1178 (1992)
Littman, M.L.: Markov Games as a Framework for Multi-Agent Reinforcement Learning. In: Proceedings of the Eleventh International Conference on Machine Learning, New Brunswick, pp. 157–163 (1994)
Stone, P., Veloso, M.: Multiagent Systems: A Survey from a Machine Learning Perspective (1997)
Sen, S. (ed.): Collected Papers from the AAAI 1997 Workshop on Multiagent Learning. AAAI Press, Menlo Park (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hwang, KS., Chen, YJ., Lin, TF. (2006). Q-Learning with FCMAC in Multi-agent Cooperation. In: Wang, J., Yi, Z., Zurada, J.M., Lu, BL., Yin, H. (eds) Advances in Neural Networks - ISNN 2006. ISNN 2006. Lecture Notes in Computer Science, vol 3971. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11759966_89
Download citation
DOI: https://doi.org/10.1007/11759966_89
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34439-1
Online ISBN: 978-3-540-34440-7
eBook Packages: Computer ScienceComputer Science (R0)