Abstract
Flexibility in decision making is essential for adapting to dynamically changing scenarios. A probabilistic reversal learning task is one of the experimental paradigms used to characterize the flexibility of a subject. Recent studies hypothesized that in addition to a reward history, a subject may also utilize a “cognitive map” that represents the latent structures of the task. We conducted experiments on a probabilistic reversal learning task and performed model-based analysis using two types of reinforcement learning (RL) models, with and without state representations of the task. Based on statistical model selection, the RL model without state representations was selected for explaining the behavior of the average of all the subjects. However, the individual behaviors of approximately 20% subjects were explained using the RL model with state representation and by the probabilistic estimation of the current state. We inferred that these results possibly indicate the variations in the development of the orbitofrontal cortex of the subjects.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Collins A, Koechlin E (2012) Reasoning, learning, and creativity: frontal lobe function and human decision-making. PLoS Biol 10(3):e1001293
Collins AG, Frank MJ (2013) Cognitive control over learning: creating, clustering, and generalizing task-set structure. Psychol Rev 120(1):190
Cools R, Clark L, Owen AM, Robbins TW (2002) Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci 22(11):4563–4567
Costa VD, Tran VL, Turchi J, Averbeck BB (2015) Reversal learning and dopamine: a Bayesian perspective. J Neurosci 35(6):2407–2416
Daw N, Courville A (2008) The pigeon as particle filter. Adv Neural Inf Process Syst 20:369–376
Hampton AN, Bossaerts P, O’doherty JP (2006) The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci 26(32):8360–8367
Hartley CA, Somerville LH (2015) The neuroscience of adolescent decision-making. Curr Opin Behav Sci 5:108–115
Izquierdo A, Brigman J, Radke A, Rudebeck P, Holmes A (2017) The neural basis of reversal learning: an updated perspective. Neuroscience 345(2):12–26
Saez A, Rigotti M, Ostojic S, Fusi S, Salzman C (2015) Abstract context representations in primate amygdala and prefrontal cortex. Neuron 87(4):869–881
Schuck NW, Cai MB, Wilson RC, Niv Y (2016) Human orbitofrontal cortex represents a cognitive map of state space. Neuron 91(6):1402–1412
Scott WA (1962) Cognitive complexity and cognitive flexibility. Sociometry 25(4):405–414
Wilson RC, Takahashi YK, Schoenbaum G, Niv Y (2014) Orbitofrontal cortex as a cognitive map of task space. Neuron 81(2):267–279
Acknowledgements
This study was supported by JSPS KAKENHI Grant Number JP16H06397.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was presented in part at the 25th International Symposium on Artificial Life and Robotics (Beppu, Oita, January 22-24, 2020).
About this article
Cite this article
Masumi, A., Sato, T. Model-based analysis of learning latent structures in probabilistic reversal learning task. Artif Life Robotics 26, 275–282 (2021). https://doi.org/10.1007/s10015-020-00674-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10015-020-00674-8