Abstract
Autonomous state generalization problem is a key issue in the research field of behavior learning of reactive agents, and many approaches have been proposed in recent years. However, those existing methods have a diversity in their criteria of state generalization or “how to define the similarity or distance between different sensor inputs”, while it is not yet clear how this difference in the criteria would affect the entire learning process. In this paper, we first classify and examine those conventional heuristic criteria of state generalization, and then propose a new general framework for unifying all of them. This novel general criterion is based on minimization of weighted sum of entropies in multiple behavior outcomes of agents. An experimental study in the latter part suggests that this state generalization criterion enables a reactive agent to construct or reconstruct its state space in a more efficient and flexible way.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Albus, J., Lacaze, A., Meystel, A.: Multiresolutional intelligent controller for baby robot. Proceedings of the 10th International Symposium on Intelligent Control (1995)
Asada, M., Noda, S., Hosoda, K.: Action-based sensor space categorization for robot learning. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (1996) 1502–1509
Chapman, D., Kaelbling, L.P.: Input generalization in delayed reinforcement learning: An algorithm and performance comparisons. Proceedings of Twelfth International Joint Conference on Artificial Intelligence (1991) 726–731
Gaskett, C, Wettergreen, D., Zelinsky, A.: Q-learning in continuous state and action spaces. Proceedings of 12th Australian Joint Conference on Artificial Intelligence (1999)
Ishiguro, H., Sato, R., Ishida, T.: Robot oriented state space construction. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (1996) 1496–1501
Moore, A. W., Atkeson, C. G.: The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces. Machine Learning, Vol. 21. (1995) 199–233
Sutton, R. S., Barto, A. G.: Reinforcement Learning. MIT Press (1998)
Takahashi, Y., Asada, M., Hosoda, K.: Reasonable performance in less learning time by real robot based on incremental state space segmentation. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (1996) 1518–1524
Takahashi, Y., Takeda, M., Asada, M.: Continuous valued q-learning for vision-guided behavior acquisition. Proceedings of 1999 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI’99) (1999)
Ueno, A., Hori, K., Nakasuka, S.: Simultaneous learning of situation classification based on rewards and behavior selection based on the situation. Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (1996) 1510–1517
Yairi, T., Nakasuka, S., Hori, K.: State abstraction from heterogeneous and redundant sensor information. Proceedings of International Conference on Intelligent Autonomous Systems 5 (IAS-5) (1998) 234–241
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yairi, T., Hori, K., Nakasuka, S. (2000). Unified Criterion of State Generalization for Reactive Autonomous Agents. In: Mizoguchi, R., Slaney, J. (eds) PRICAI 2000 Topics in Artificial Intelligence. PRICAI 2000. Lecture Notes in Computer Science(), vol 1886. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44533-1_36
Download citation
DOI: https://doi.org/10.1007/3-540-44533-1_36
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67925-7
Online ISBN: 978-3-540-44533-3
eBook Packages: Springer Book Archive