Cortical substrates for exploratory decisions in humans

Nathaniel D. Daw, John P. O’Doherty, Peter Dayan, Ben Seymour, Raymond J. Dolan · Nature · 2006

Summary

It is shown, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma, and a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes is suggested.

Type: article
Published: 2006-06-15
Cited by: 2,229
References: 33
Access: Open access
DOI: https://doi.org/10.1038/nature04766
PubMed: https://pubmed.ncbi.nlm.nih.gov/16778890/
PMC: https://www.ncbi.nlm.nih.gov/pmc/articles/2635947/
OpenAlex: https://openalex.org/W2077611535
Semantic Scholar: https://api.semanticscholar.org/CorpusID:4420107

Keywords

Curiosity, Functional magnetic resonance imaging, Task (project management), Exploit, Dilemma

References

Progress in statisticsJoseph Gani, K. Sarkadi, I. Vincze — 1975
Abstract reward and punishment representations in the human orbitofrontal cortexJohn P. O’Doherty, Morten L. Kringelbach, Edmund T. Rolls — 2001
A framework for mesencephalic dopamine systems based on predictive Hebbian learningP. Read Montague, Peter Dayan, TJ Sejnowski — 1996
Learning in embedded systemsLeslie Pack Kaelbling — 1993
Tracking the hemodynamic responses to reward and punishment in the striatum.Mauricio R. Delgado, Leigh E. Nystrom, Catherine Fissell — 2000
FMRI visualization of brain activity during a monetary incentive delay task.Brian Knutson, Andrew F. Westdorp, Erica Kaiser — 2000
Activity in posterior parietal cortex is correlated with the relative subjective desirability of action.Michael C. Dorris, Paul W. Glimcher — 2004
Temporal difference models and reward-related learning in the human brain.John P. O’Doherty, Peter Dayan, Karl Friston — 2003
Cognitive planning in humans: neuropsychological, neuroanatomical and neuropharmacological perspectives.Adrian M. Owen — 1997
The cognitive and neuroanatomical correlates of multitasking.Paul W. Burgess, Emma Veitch, Angela Costello — 2000
Matching Behavior and the Representation of Value in the Parietal CortexLeo P. Sugrue, Greg S. Corrado, William T. Newsome — 2004
The Architecture of Cognitive Control in the Human Prefrontal CortexÉtienne Koechlin, Chrystèle Ody, Frédérique Kouneiher — 2003
Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal CortexJay A. Gottfried, John P. O’Doherty, Raymond J. Dolan — 2003
Anterior prefrontal cortex: insights into function from anatomy and neuroimagingNarender Ramnani, Adrian M. Owen — 2004
The role of locus coeruleus in the regulation of cognitive performance.Marius Usher, Jonathan D. Cohen, David Servan‐Schreiber — 1999
Separate Neural Systems Value Immediate and Delayed Monetary RewardsSamuel M. McClure, David Laibson, George Loewenstein — 2004
Neural correlates of decision variables in parietal cortexMichael L. Platt, Paul W. Glimcher — 1999
Dopamine: generalization and bonusesSham M. Kakade, Peter Dayan — 2002
Midbrain dopamine neurons encode a quantitative reward prediction error signal.H Bayer, Paul W. Glimcher — 2005
Reinforcement Learning: An IntroductionSutton, Richard S, Barto, Andrew G — 1998

Cited by

Neuroeconomics: Best to go with what you know?勝良馬場, 正樹松川 — 2006
Decision-Making Dysfunctions in Psychiatry—Altered Homeostatic Processing?Tsai-Yu Huang, C. F. Huang, Gil‐Ho Kim — 2007
Foundations of Neuroeconomics: From Philosophy to PracticeNicholas Gibbins, Steve W. Harris, Danius T. Michaelides — 2008
Neural mechanisms mediating association of sympathetic activity and exploration in decision-making.María del Rosario Fernández, José Emilio Palomero Pescador — 2013
Reward-based decision making in pathological gambling: the roles of risk and delay.Gregory Evangelos Koulouras — 2015
Depression and Alzheimer's Disease: Novel Postmortem Brain Studies Reveal a Possible Common MechanismNaomi Nihonmatsu-Kikuchi, Yoshitaka Hayashi, Xiu‐Jun Yu — 2013
Active inference and epistemic valueJudith Luciani — 2015
Medication adherence as a learning process: insights from cognitive psychologyGeoffrey Macnab — 2017
Probing the temporal dynamics of the exploration-exploitation dilemma of eye movements.Facundo J Gutiérrez — 2018
Human reinforcement learning: Insights from intracranial recordings and stimulationAshwin G. Ramayya — 2014
Incentive processing and inhibitory control in adolescents and young adultsCharles F. Geier — 2009
Decision Making, Movement Planning,Julia Trommershäuser, Laurence T. Maloney, Michael S. Landy — 2008
Bayesian modeling of human sequential decision-making on the multi-armed bandit problemDaniel E. Acuña, Paul Schrater — 2008
Proactive action preparation: seeing action preparation as a continuous and proactive process.Giovanni Pezzulo, Dimitri Ognibene — 2012
Proper embodiment: the role of the body in affect and cognitionStapleton, Mog — 2012
The “working” of working memoryEarl K. Miller — 2013
Dopaminergic enhancement of cognition in old ageR. Chowdhury — 2013
Criteria and awareness in perceptual decisionStephen M. Fleming — 2011
Activités spécifiques du cortex cingulaire antérieur et du cortex préfrontal dorsolatéral et interactions lors de l’adaptation des comportementsMarie-Pierre Rothé — 2010
The economics, neurobiology and pharmacology of intertemporal choice in humansAlex Pine — 2011

Cortical substrates for exploratory decisions in humans

Summary

Keywords

References

Cited by

Related papers