Spatiotemporal Neural Characterization of Prediction Error Valence and Surprise during Reward Learning in Humans

Elsa Fouragnan, Filippo Queirazza, Chris Retzler, Karen Mullinger, Marios G. Philiastides

Research output: Contribution to journalArticle

10 Citations (Scopus)

Abstract

Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.
LanguageEnglish
JournalScientific Reports
Volume7
Issue number4762
DOIs
Publication statusPublished - 6 Jul 2017

Fingerprint

Reward
Learning
Electroencephalography
Magnetic Resonance Imaging
Reversal Learning
Avoidance Learning

Cite this

Fouragnan, Elsa ; Queirazza, Filippo ; Retzler, Chris ; Mullinger, Karen ; Philiastides, Marios G. / Spatiotemporal Neural Characterization of Prediction Error Valence and Surprise during Reward Learning in Humans. In: Scientific Reports. 2017 ; Vol. 7, No. 4762.
@article{b8a46c1d637f4fedbc97ebf6c1c2d433,
title = "Spatiotemporal Neural Characterization of Prediction Error Valence and Surprise during Reward Learning in Humans",
abstract = "Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.",
author = "Elsa Fouragnan and Filippo Queirazza and Chris Retzler and Karen Mullinger and Philiastides, {Marios G.}",
year = "2017",
month = "7",
day = "6",
doi = "10.1038/s41598-017-04507-w",
language = "English",
volume = "7",
journal = "Scientific Reports",
issn = "2045-2322",
publisher = "Nature Publishing Group",
number = "4762",

}

Spatiotemporal Neural Characterization of Prediction Error Valence and Surprise during Reward Learning in Humans. / Fouragnan, Elsa; Queirazza, Filippo; Retzler, Chris; Mullinger, Karen; Philiastides, Marios G.

In: Scientific Reports, Vol. 7, No. 4762, 06.07.2017.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Spatiotemporal Neural Characterization of Prediction Error Valence and Surprise during Reward Learning in Humans

AU - Fouragnan, Elsa

AU - Queirazza, Filippo

AU - Retzler, Chris

AU - Mullinger, Karen

AU - Philiastides, Marios G.

PY - 2017/7/6

Y1 - 2017/7/6

N2 - Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.

AB - Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.

UR - https://www.nature.com/srep/

U2 - 10.1038/s41598-017-04507-w

DO - 10.1038/s41598-017-04507-w

M3 - Article

VL - 7

JO - Scientific Reports

T2 - Scientific Reports

JF - Scientific Reports

SN - 2045-2322

IS - 4762

ER -