Redes neuronales y mapas cerebrales durante el aprendizaje del

Redes neuronales y mapas cerebrales durante el
aprendizaje del lenguaje y la música
Antoni Rodríguez-Fornells
Catalan Institution for Research and Advanced Studies (ICREA)
Cognition and Brain Plasticity Unit
Department of Basic Psychology
IDIBELL – Institut de Investigació Biomedica de Bellvitge
Campus Hospital de Bellvitge
University of Barcelona
www.brainvitge.org
Cognition and Brain Plastivity Unit
www.brainvitge.org
Meltzoff, Kuhl, Movellan, Terrence & Sejnowski, Science 2009
“Suppose and adult and a child human arrive on Mars and
discover that there are Martians who seem to speak a language
to one another. If the adult and child human stay on Mars
for several years and try to learn Martian, what do you think
will be the outcome?”
L. Gleitman and E. Newport, 1995
Invitations to Cognitive Science, Language
From Gupta et al., 2004
Why do we want to study language learning?
Language Learning streams
Bilingualism
Language
Learning
mechanisms
- Dorsal audio-motor
- Ventral meaning integration
General Learning mechanisms
- Episodic lexical interface
- Reward-motivation circuits
Individual Differences:
Brain Plasticity
Res. Framework:
i. Human Simulation paradigm
ii. Comparisons with other learning
domains
iii. The interface between cognitive
control, reward-motivation and lang.
learning
iv. Indiv.Differences in Lang.
Learning
v. Infants and Patients: Intervention
& re-learning strategies
Translational research:
Re-learning and
neurorehab.
Selective or strategic use of
one-mechanism
Brain mechanisms involved in “on-line Language” Learning (first evidences)
a1. Speech segmentation
problem
Cunillera et al., NI 2009
Lopez-Barroso et al., Cer. C. 2011; PNAS 2013
De Diego-Balaguer et al., Plos One, 2007
a2. Learning the meaning
of new-words
Mestres-Misse et al., Cer. C. 2007
Mestres-Misse et al., JOCN 2008, 2009 NI 2010
Rodriguez-Fornells et al., PTRSL 2010
Speech segmentation and word recognition problem
How do infants segment words from continous speech?
Can you see silences
signaling the onset of each word?
Saffran et al., 1996: a statistical learning mechanism might be used for language acquisition in infants
2 minutes of exposition to nonsense syllabes (8 months):
golabopadotibidakutupirogolabopadotigolabobidakutupiro....
Transition P syllables within a word (go-la) = 1
Transition P syllables in between words (bo-pa)= 0.33
words:
golabo
padoti
bidaku
tupiro
mean = 7.9 seg.
non-words:
dakupa
labopa
kutupi
pirogo
mean = 8.8 seg.
But, which is the output of the statistical learning mechanism in infants segmenting speech?
(Saffran et al., 2001)
Familiarization phase (Language A: pabiku tibudo golatu daropi )
English frame
I like my
We saw the
You play with
What a nice
pabiku
tibudo
tudaro
pigola
Nonsense frame
Zy fike my
Gee baw fuh
Foo dray miff
Vut luh kife
pabiku
tibudo
tudaro
pigola
Design
Language conditions:
pirutabagolitokudapirutabagoligukibobagolitokudapirutagukibo....
Random conditions:
pigukukidatagotolibobarubakutorupigubotakidaligolitokubabogo….
1st block
2nd block
3rd block
4th block
Behavioral test
Language A
2 min.
2 min.
2 min.
2 min.
Baseline A
piruta
Language B
Baseline B
bobago
Audio-motor interface in speech segmentation
Language conditions:
pirutabagolitokudapirutabagoligukibo
Random conditions:
pigukukidatagotolibobarubakutorupi
McNealy et al., JN 2006
Cunillera et al., Neuroimage 2009
Dorsal audio-motor interface involved in
Speech segmentation
Superior Longitudinal fasciulus (SLF)
Efference copy of the “articulated
possible new word”, and fine-tuning the
template-matching algorithm
Mapping the encoded
sequence of sounds into a
sequence of articulatory
gestures (for keeping the
segmented word active
through rehearsal –
Phonological WM)
Output: ordered sequence of
auditory representations
feedforwaded to the PMC
Template-matching
algorithm = detection of
coincidences btw the stored
representations and
“phonological templates”
derived from previous exposures
(Warren et al., 2005)
Rodriguez-Fornells et al., 2009
Phil. Trans R.S. B-Biol. Sci.
Rauschecker & Scott, 2009; Hickok &
Poeppel, 2003, 2007; Scott & Wise 2004
Functional connectivity of the dorsal
audio-motor interface
Identification of three different dorsal learning
networks involved in word learning using
Independent Component Analysis (ICA)
Two different studies fMRI (3T), same
paradigm:
1st cohort: 27 Spanish participants
2nd cohort:16 German participants
Lopez-Barroso, Ripolles, et al. (2015 Neuroimage)
Auditory-motor coupling hypothesis in MUSIC LEARNING
actions to sounds
sounds to actions
Dorsal audio-motor coupling induced by MUSIC
LEARNING in chronic stroke hemiparetic patients
Rodriguez-Fornells et al., 2012
A. N.Y. A. Sciences
Under review. Nature Sc. Reports
Word learning
(speech segmentation)
Rodriguez-Fornells et al., 2009
Phil. Trans R.S. B-Biol. Sci.
NEW questions
Language
Learning
Bilingualism
Learning streams
- Dorsal audio-motor
- Ventral meaning integration
- Episodic lexical interface
Individual Differences:
Brain Plasticity
Translational research:
Re-learning and
neurorehab.
Selective or strategic use of
one-mechanism
Individual Differences in Speech Segmentation
and Arcuate Fasciculus tractography (Diffusion
Tensor Imaging DTI)
Language conditions:
pirutabagolitokudapirutabagoligukibo
Lopez-Barroso, Catani, et al.. (PNAS 2013)
Individual Differences in Speech Segmentation
and Arcuate Fasciculus tractography (Diffusion
Tensor Imaging DTI)
Lopez-Barroso, Catani, et al.. (PNAS 2013)
> The AF and in relation to auditory-motor integration, might
represent a key step for language evolution.
> Comparative studies indicate that apes and monkeys have
homologs for Broca’s and Wernicke’s areas in terms of grey
matter connected by a dorsal and a ventral pathway (Rilling et al.,
2011). However, the trajectory of the AF is different between
species. Middle and inferior temporal gyri terminations from the
inferior frontal gyrus are more prominent in humans than in
macaques and chimpanzees (Rilling et al., 2008).
> These findings suggest that the ability to integrate auditorymotor information confers some advantages in the manipulation
of the representation of acoustic stimuli, thus allowing its storage
in the long-term memory.
Rilling et al., 2012
Rilling et al., 2008
Ontogenetically, the AF develops slower than other associative pathways, including the ventral
pathway (Perani et al., 2011). The AF’s terminations connecting with Broca’s area show
progressive development during childhood, still under development at the age of 7 y (Brauer
2011). However, the connections with premotor cortex, those responsible to auditory-motor
integration, can be tracked in newborns (Brauer et al., 2013, Perani et al., 2011), suggesting a role
in early language acquisition.
Brauer et al., 2013
Brain language streams involved in “on-line Language” Learning
a1. Speech segmentation
problem
a2. Learning the meaning
of new-words
Cunillera et al., NI 2009
Lopez-Barroso et al., Cer. C. 2011
De Diego-Balaguer et al., Plos One, 2007
Mestres-Misse et al., Cer. C. 2007
Mestres-Misse et al., JOCN 2008, 2009 NI 2010
Rodriguez-Fornells et al., PTRSL 2010
Word-learning task (Mestres-Missé et al., Cer. Cor. 2007; J. Cog Neuropsco. 2008)
Non-word meaning M+
Mario always forgets where he leaves the rankey
It was expensive the repair of the rankey
I punctured again the wheel of the rankey
Non-word no-meaning MI have bought the tickets for the garty
On the construction-site you must wear a garty
Everyday I buy two loaves of fresh garty
Real word R
My brother Peter is buying a bit of salt
The rice we made needs a bit more salt.
My doctor advised me not to eat with salt.
a. Cloze probability of the sentences (N = 160)
1ª frase: 6,11% (SD= 10,38)
2ª frase: 28,97% (SD= 18,97)
3ª frase: 76,05% (SD= 17,75)
b. Meaning extraction after reading the
sentences (N = 15): 91.12 % (10.73)
c. Meaning extraction in the ERP experiment
app. 84% (N = 24)
(Mestres-Missé et al., Cer. Cor. 2007)
Mestres-Missé et al., Cerebral Cor. 2007
See also recent studies using this task: Batterink & Neville, 2011; Borovsky, Elman & Kutas, 2012
Implications of the Meaning Integration
interface in the contextual learning task
Non-word meaning - M+
Non-word no-meaning - MReal word - R
Mestres-Misse et al., JoCN 2008
Mestres-Misse et al., JoCN 2008
Question?
i. Reward – motivated Language Learning. fMRI-DTI combined study
Language Learning streams
Language
Learning
mechanisms
- Dorsal audio-motor (development)
- Ventral meaning integration
General Learning mechanisms
- Episodic lexical interface
- Reward-motivation circuits
Individual Differences
Is there any pleasure associated to language learning experiences?
Ripolles et al., Current Biology, 2014
a. Speech provides critical emotional value to children promoting infant-parent bounding
(hearing and adult voice provides comfort and pleasure experiences in early stages of
development) (Kuhl et al., 2005).
b. Infant-directed prosodically enriched speech
communication of affect –arousal modulation).
(attract
infant’s
attention
and
c. In stressful situations children experience increased oxytocin release upon hearing
their mother’s voice (Seltzer 2010; DeCasper & Fifer, 1980). This hormone release
promotes affiliative behaviors and is closely related to emotion and reward processing
(Strathearn et al., 2009).
d. Infants pleasure depends on communication. This could explain the internal “drive to
learn new words” (intrinsically motivated). Importance of social interaction in
Language Learning (Bruner, 1983; Vigotsky, 1962)
e. Social gating perspective in Language acquisition (Bloom, Tomasello, Kuhl, Goldstein).
Infant language is gated by the motivational properties (attention and arousal) inherent
in social interactions (Kuhl, 2007).
a. At EVOLUTIONARY LEVELS, recent cortical language and communication systems
might have relayed on existing subcortical (emotion-reward) mechanisms (Syal and
Finlay, 2011; Panksepp, 1998), reinforcing human motivation to learn a new language
(inner drive).
b. The initial development of a ‘‘protolanguage’’ in human ancestors was probably crucial for
sharing information-knowledge and emotions, improving success on reward-seeking
behaviors, bonding social groups, and increasing the chances of group survival in
competitive environments.
c.
This protolanguage might have been
naturally selected and reinforced by
interlinking
it
with
ancient
brain
mechanisms involved in hedonic reward
processing
d. Finally, adult individuals that are language
learning
“addicts”
(language
learning
seekers).
Rodriguez-Fornells et al., 2010
Phil. Trans R.S. B-Biol. Sci.
Dopamine and Reward-Motivated Learning
Lisman & Grace, The hippocampal-VTA
loop controlling the entry of information
in Long Term Memory. Neuron 2005
Evidences of dopaminergic modulation of word
learning (Knecht et al., An. Neurol 2004)
Neural correlates of listening to highly rewarding music
Salimpoor & Zatorre NN 2012, Science 2014
Sescouse et al., NI2013
Ventral striatum (NAc)
Ventral striatum (NAc)
Camara et al., JN 2010
Ripolles et al., Current Biology 2014
Sescouse et al., NI2013
Ripolles et al., CB 2014
Ripolles et al., CB 2014
CONCLUSIONS on motivation-reward and Language learning:
 Thus, consistent with the songbird’s instinct to learn to sing, human beings also
display an urge to acquire language (Fitch, 2010), and both adaptive behaviors might
be driven by similar, phylogenetically older, reward-related circuits.
 Songbirds possess area X, a striatal nucleus analog to the human basal ganglia,
which is crucial for song learning in both young and adult birds (Margoliash &
Nusbaum, 2009). In addition, area X receives midbrain dopaminergic projections
(Mooney, 2009) and shows increased FoxP2 (a gene associated in humans with
language and speech) expression during periods of learning (Scharff & Haesler,
2005).
 Reduced cortical and reward subcortical connectivity in autistic children (which might
difficult the capacity of them to experience speech as a rewarding episode) and
influencing the correct development of their communication (Abrams et al., PNAS
2013).
Doupe et al., TICS 2005
CONCLUSIONS on motivation-reward and Language learning:
 Thus, consistent with the songbird’s instinct to learn to sing, human beings also
display an urge to acquire language (Fitch, 2010), and both adaptive behaviors might
be driven by similar, phylogenetically older, reward-related circuits.
 Songbirds possess area X, a striatal nucleus analog to the human basal ganglia,
which is crucial for song learning in both young and adult birds (Margoliash &
Nusbaum, 2009). In addition, area X receives midbrain dopaminergic projections
(Mooney, 2009) and shows increased FoxP2 (a gene associated in humans with
language and speech) expression during periods of learning (Scharff & Haesler,
2005).
 Reduced cortical and reward subcortical connectivity in autistic children (which might
difficult the capacity of them to experience speech as a rewarding episode) and
influencing the correct development of their communication (Abrams et al., PNAS
2013).
Previous studies have shown that insensitivity to the human voice may underlie the hallmark
communication deficits of autism spectrum disorders (ASD) (Kanner, 1943).
The social motivation theory of ASD suggest that deficits in representing the reward value of social
stimuli, including voice-speech, impedes children with ASD from actively engaging with these stimuli
and consequently impairs social skill development (Dawson et al., 2002)
The findings suggest that weak connectivity between speech and reward centers impede children with
ASD from experiencing speech as pleasurable, thus influencing how social communication skills
develop, according to the authors
Abrams et al., 2013
CONCLUSIONS on motivation-reward and Language learning:
 This hypothesis favors current perspectives, which emphasize that language was an
evolutionary innovation built on different preexisting cognitive capabilities, probably
‘‘hijacking’’ old evolutionary solutions as reward-reinforcement mechanisms.
 Language learning could then rely on the interaction between general-domain
cognitive abilities (e.g., theory of mind, associative learning, analogical processing, or
joint attention) and more-specific linguistic ones (Bloom 1992).
Syal and Finlay, 2011
Future questions:
i.
Comparison of infant neurophysiological mechanisms in place for
Language Learning with adult mechanisms.
ii. If infants isolated and “learn” the word, how is it going to be represented
and structured in their Mental Lexicon  we need longitudinal studies
iii. Memory systems (the interplay between episodic and semantic memory)
iv. Individual differences in the plasticity of the different language learning
streams (dorsal and ventral). Role of metaplasticity in these language
streams (or how initial learning could constrain future learning).
v. Plan strategic rehabilitation programs based on preserved grey and white
matter regions in aphasic patients and based on the function of the
different language learning streams (stroke-aphasia project)
vi. On-going studies on infant perinatal stroke (MRI – fine grained language
learning evaluation)
vii. Follow-up the study of pleasure and reward in Language Learning in at risk
infants (ASD)
Ramon y Cajal, 1904, “Texture of the Nervous system of man and the
vertebrates”.
“Todo el mundo sabe que la habilidad de un pianista para tocar su
instrumento y su adaptación a esta profesión, requiere de muchos años de
gimnasia mental y muscular. Pero para poder comprender este fenómeno
tan importante, es necesario aceptar que, además de el refuerzo que se
produzca en las vías cerebrales pre-existentes, nuevas vías deben
crearse mediante la ramificación o crecimiento progresivo de los
terminales dendríticos y los procesos axonales.”
Supported by:
 Ramon y Cajal research program from the MCYT, Spain
 Spanish Government, I+D programs BSO2002, SEJ2005
 Catalan Government (SGR research groups, NECOM group)
www.brainvitge.org
 Volkswagenstifung (2005-2007) European coordinated program
 Fundació La Marato (programa Neurociencies, 2007)
 Icrea Junior and Senior programs
•
Clement Francois, Pablo Ripolles & Claudia Peñazola
•
Diana Lopez-Barroso, Toni Cunillera, Anna Mestres-Misse, Ruth de Diego Balaguer
•
Dept. Neuropsychology and Klinik für Neurologie II. Otto von Guericke University.
Magdeburg: T.F. Münte, M. Rotte, & C. Tempelmann & Toemme Noesselt
•
Department of Psychology, University of Turku: M. Laine
•
Temple University: Nadine Martin