Abstract concepts

From affordances to abstractness:
An embodied and grounded
perspective on language
anna m. borghi
1 University of Bologna
2 Institute of Sciences and Technologies of Cognition, CNR, Rome
[email protected]
Web-site: http://laral.istc.cnr.it/borghi
Rome, Modelact, June 7, 2016
OUTLINE
Language and affordances – theories of reuse
Beyond language grounding: Words As Tools
that modify space perception
Abstract concepts: Words As social Tools
(WAT): !"#$%&'()*+$,+-$&.+/*(/$%0"$&$%+'(1+
20(340/&0)5/$)0'2+0(,$%"'&0$(
Acquisition
Embodiment: activation of the mouth
Linguistic variability
Why importance of language/
Why mouth activation?
AFFORDANCES BETWEEN
PERCEPTION AND ACTION
Affordances (Gibson, 1979) action
opportunities that the environment offers
to the subject
Affordances concern BOTH perception
and action, they refer both to the world
and to the individuals, they are variable
Ellis & Tucker (2000): microaffordances: brain assemblies of
visuomotor associations concerning
specific action components – beyond a
purely externalist view (Gibson)
(for a review Thill et al. 2013)
STABLE AND VARIABLE AFFORDANCES
Affordances
can b
be (Borghi & Riggio, 2009; 2015):
d
!"#$%&stable' / permanent ! based on longterm visuomotor associations. E.g., size.
canonical (subset of stable affordances): e.g.,
canonical orientation - more dependent on the
context
!"#$%&temporary'/variable ! based on online
visual information. E.g., current orientation of
an object handle.
No dichotomy but continuum
Borghi & Riggio,2009; Borghi & Riggio, 2015; Sakreida et al., under
review
STABLE AND VARIABLE AFFORDANCES
META-ANALYSIS: VENTRO-DORSAL
VS. DORSO-DORSAL STREAM
Meta-analysis:
l i studies
t di
1995-2015
Stable: stronger lefthemisphere and frontoparietal activations
Variable: bilateral,
dorsally-located frontoparietal activations.
Stable: ventrodorsal V-D stream
Variable: dorsodorsal D-D route
(Rizzolatti & Matelli,
2003)
Sakreida et al., under review
!
LANGUAGE AND
STABLE VS. VARIABLE AFFORDANCES
action
ti
vs. observation
b
ti
sentences
t
(e.g. grasp / look at
the brush)
followed by images of everyday objects
graspable with a power vs. a precision grip
with affordances presented in canonical or in
reversed orientation (upright vs reversed)
Task: Is the object in the picture the same as the
object mentioned in the sentence (Zwaan et al,
2002)?
Borghi & Riggio, 2009
LANGUAGE AND
STABLE VS. VARIABLE AFFORDANCES
TRUE ITEMS: RTs were faster
with upright than with reversed objects - i.e.
in case of correspondence between the canonical
affordance location (up, down), and the field in which it
was presented (upper, lower).
with action than with observation verbs.
LANGUAGE AND
STABLE VS. VARIABLE AFFORDANCES
/0123 45362
/0123(45362(
Factors:
Sentence
(action –
observation
Orientation
(upright –
reversed)
Grip (same –
different)
with action sentences RTs were slower with
objects graspable with the same grip than with
objects graspable with a different grip. Interference.
thus: during sentence comprehension simulation
of the object grip
Language encodes "#$%&'($))*+,$-.'"((size/grip,
canonical orientation)
CONTEXT-DEPENDENCY OF
AFFORDANCES: LANGUAGE AND BODY
Verbs: observation, manipulation, function
Task: Respond if the picture-verb combination
makes sense (e.g., bottle-pour, not bottle-cut)
3D objects: near, far
CONTEXT-DEPENDENCY OF
AFFORDANCES: LANGUAGE AND BODY
1-.'2-.,($03
45&.02-.,($03
B>>
!
6-0$ 78,9:;,-%<
A?>
!
A>>
@?>
!
!
@>>
=?>
=>>
"#$%&'($
)%&'($
*+,-./0&'($
manipulation and function verbs: faster responses when
objects are in the near than in the far space; no difference
is present for observation verbs.
function slightly faster than manipulation, especially in the
peripersonal space: object represented in terms of what
you do with them.
AFFORDANCES, CONTEXT AND
AUTOMATICITY: OPEN ISSUES
Affordances are flexible and vary depending on
6$(&*7& (distance from the body)
Linguistic context (kind of verb)
Can we still say they are automatic (van Elk et al,
2013; van Dam et al., 2014)? C*78'#9#9*-8+
differently solved depending on the task?
Manipulation vs. function? Linguistic task:
FUNCTION
Stable vs. variable affordances? Linguistic
task: STABLE affordances
:*+;<9=(>?@AB(:*+;<9=(C9;;9* D>?@EF(
LANGUAGE: SIMULATION AND
BEYOND
Language and simulation:
Language recruits perception, action and emotional
systems (re-use) (Anderson, 2010, 2014; Gallese,
2008) affordances
But it also modifies them: e.g. it is more sensitive
to stable and to functional affordances
Beyond simulation: language is also a tool
performing actions in the physical and social space:
words modify our space perception (presence of
others)
Linguistic and social info have a major influence
on how we represent abstract concepts
:*+;<9=(2.*+*&&9=(G$&9;9*+'=(:$&,$""$++'=(5H77*&9-9=(>?@I
WORDS AS TOOLS THAT MODIFY
SPACE PERCEPTION
For each trial 3 objects located
in the near, border and far space
Children were asked to grasp
one object and put it into a
black box in the indicated hole.
Training: “Pick up the right object
and place it in the box (different
shaped holes)”
RAKE
border, far
WORD
border, far (children
had to use the word color to obtain
the object)
SHORT RAKE border
2.*+*&&9=(J$(K+$#9=(L9.*(M(:*+;<9=(>?@!
WORDS AS TOOLS THAT MODIFY
SPACE PERCEPTION
Estimations (pre/post training):
Verbal
Motor-based
!"#$%&'()*+!
WORDS AS TOOLS THAT MODIFY
SPACE PERCEPTION
!"#$%&'()*+!
Main effect of Space : far space M = 19.44 cars
border space M = 13.85 cars
No effect of Kind of training / Phase.
16
WORDS AS TOOLS THAT MODIFY
SPACE PERCEPTION
MAXIMAL DISTANCE: Main effect of Training. Motor
estimations relative to the border but not the far space
changed significantly after the training sessions.
Both tools and words modify space perception 17
extending the bodily space.
OUTLINE
Language and affordances - reuse
9&'-2*+'(1+:'%0'-2*+',,$%1'()*/
;2*70-020&<=+10/&'()*+,%$"+&.*+-$1<+'(1+20(340/&0)+)$(&*7&
Beyond simulation: Words As social Tools
>7&*(/0$(+$,+&.*+-$102<+-$%1*%/
?-/&%')&+)$()*#&/=+@$%1/+?/+/$)0'2+A$$2/+B@?AC
Embodiment: mouth activation
Linguistic diversity
Why importance of language/mouth activation?
WHAT ARE ABSTRACT CONCEPTS?
FRASI
FRASI
ABSTRACTION AND ABSTRACTNESS
Capability to abstract: sophisticated human ability
Difference ABSTRACTION – ABSTRACTNESS
Categorization always implies some sort of abstraction: e.g.
dog – animal. Animal more general, but still it may evoke a
collection of single, bounded and concrete referents.
Here focus on abstractness. E.g. freedom, fantastic
vedi metafore; idiomi
19
WHAT ARE ABSTRACT CONCEPTS?
DICHOTOMY OR CONTINUUM?
-$&&2*
Difficulty in defining abstract vs. concrete words.
CONTINUUM, no dychotomy
,%**1$"
Different kind of grounding. More abstract
entities = Not clearly bounded objects as
referents. E.g. “truth” (talk by K. PASTRA)
Complexity. “abstract concepts often capture
vedi metafore;
idiomi
complex configurations of physical
and mental
events” (Barsalou, 2003). E.g. “cause”
Meaning variability. Higher meaning
variability, both within and across subjects. E.g.
“freedom”
D$%3.0 E+D0(F$,/F08+GHIJ
20
PROBLEM – EG THEORIES
AND ABSTRACT WORDS
Abstract words = crucial test for embodied theories
Most evidence limited to concrete and highly imageable
words (focus on action words and sentences)
How can we claim that abstract words are grounded in
perception, action, emotion systems?
(see work on abstract emotion terms by Moseley, 2012,
2013; Dreier et al., 2015 talk by PULVERMUELLER)
ABSTRACT CONCEPTS
AND EMBODIED THEORIES
Abstract concepts (e.g. “freedom”, “philosophy”) =
the challenge to explain them has become urgent due
to the widespread of embodied and grounded theories
Important only for embodied theories? NO!
e.g. distributional theories (meaning = co-occurrence
of words in large corpora)
do not have specific problems in explaining abstract
concepts
but they are unable to convincingly explain all kinds
of concepts due to a symbol grounding problem
(talk by ANGELO CANGELOSI)
WAT (Words As social Tools) VIEW
Words are tools, not pointers: Beyond a
referential view of words grounding
Words are social tools which:
help us to interact with the social and physical
environment
modify perception and categorization
extend our cognitive abilities
Beyond a purely embodied view, at the bridge with
an extended view of cognition.
D$%3.08+9)$%$2208+6'2030$%*8+D'21'//'%%*8+A4""$20(08+GHIK
WAT VIEW
ON ABSTRACT CONCEPTS
WAT (Words As social Tools) view (Borghi
& Cimatti, 2009, 2012;
Borghi & Binkofski, 2014).
WAT proposal on abstract concepts:
focus on the social aspects
involved in language
acquisition (Vygotskij, 1934)
WAT PROPOSAL
ON ABSTRACT CONCEPTS
abstract concepts:
t b
between
t
sensorimotor
i
t and
d
social/linguistic experience
1. Importance of linguistic/social
experience for their representation
2. Linguistic/social acquisition
3. Activation of language
activation
Mouth
4. Linguistic variability
D$%3.0 E+60"'&&08+GHHMN+D$%3.0 E+D0(F$,/F08+GHIJ
GL
()*+),-0%
12()*+),-%
3,1
&()*+),-''
&.)//''
O Concrete concepts/words: categories formed on a
sensorimotor basis: e.g., ball
O
Abstract concepts/words: social and linguistic experiences
help us to put together a variety of bodily states, of internal and
external experiences etc. e.g. fantasy
ABSTRACT CONCEPTS ACQUISITION:
ROLE OF LINGUISTIC KNOWLEDGE
Li
Linguistic
i ti modality
d lit off acquisition.
i iti
Lit
Literature
t
on
Modality of Acquisition (Wauters et al., 2003):
perceptual vs. linguistic word acquisition. With age
increase of linguistic acquisition. E.g. pencil, art, tundra
Great amount of known words. In order to acquire
“hard” words, children need to master a consistent
amount of words and of linguistic knowledge (Gleitman
et al., 2005).
Influence of language. Influence of language more
marked in the conceptualization of words referring to
relations (verbs, prepositions) than to objects or
animated entities (nouns) (Gentner, 2006; Gentner &
Boroditski, 2011)
>N
ABSTRACT CONCEPTS ACQUISITION:
ROLE OF SOCIAL COMPETENCES
Infants: importance of linguistic and social input.
Mothers produce abstract words (e.g. «all gone»;
«more») also in absence of their referent (Bergelson &
Swingley, 2013): importance of the linguistic/social input.
Infants. Social abilities. When children start to fixate
the words said by their parents . Abstract words
comprehension: sophisticated social cognition abilities
are required: 10 months, 14 months: ability to follow
others’ gaze and engage in joint action (Bergelson &
Swingley, 2013)
Children.Testimony. Literature on testimony: 3-4
year olds: able and keen to monitor information sources,
particularly in relation to abstract domains (god, religion,
death, etc.)
>O
ABSTRACT CONCEPTS: BOTH
GROUNDED AND EMBODIED
E.g. fantasy,
ntasy fantastic,
fantastic freedom,
freedom free
;'(&'/<P
;%**1$"P
Grounded. Both concrete AND abstract words
are grounded in experiences.
Embodied. Due to their link with language,
abstract concepts should activate the
mouth-related motor system more, while
concrete concepts activate vedi
the hand-related
metafore; idiomi
motor system more.
Attention! This does not imply that abstract
concepts are not represented also in terms of
their semantic content! (Wilson-Mendenhall et
al., 2013)
29
ACQUISITION OF NOVEL CATEGORIES
AND ACTIVATION OF THE MOUTH
Training 1. participants either manipulated novel
objects / observed groups of objects interacting in novel
ways. Concrete defined as: manipulable, single object as
referent.
Training 2: they read the linguistic labels referring to
concepts (e.g., “fusapo”) (labels could be accompanied
by an explanation of their meaning).
;4/'#$
B)$()%*&*C
Q'(0,'
B'-/&%')&C
D$%3.08+;24"0(08+60"'&&08+R'%$))$ E+9)$%$2208+GHII+
I?
ACQUISITION OF NOVEL CATEGORIES
AND ACTIVATION OF THE MOUTH
!"#$%&#'(")*+#,&-.
Property verification
ti
ttask.
k R
Results:
lt Th
The microphone
i
h
use
has an advantage over the keyboard use for abstract words –
activation of the MOUTH
The advantage is more marked in the explanation condition.
This is not simply due to phono-articulatory aspects: the
advantage disappears when the label and explanation contrast
with the already formed perceptually based category
I@
ACQUISITION OF NOVEL CATEGORIES
AND ACTIVATION OF THE MOUTH
Training
i i
((mimicking
i i ki
off conceptual
t l acquisition):
i iti )
Participants directly manipulate entities (objects/relations)
The experimenter verbally explains the word meaning
! Objects (concrete) eg chair ! Relations (abstract) e.g. above
Label + verbal explanation
! Label + verbal explanation
e.g. “CALONA is a staggered e.g. “BANOTO is when 2 objects touch
stack with a yellow protrusion
protrusion” each
ea other
her to form a concavity
concavity”
P+$-9#*=(2.*+*&&9 M(:*+;<9=(>?@E
ACQUISITION OF NOVEL CATEGORIES
AND ACTIVATION OF THE MOUTH
Do they belong to
the same category?
Categorical recognition task:
Before and after linguistic training.
The performance with concrete
categories was better than that with
abstract categories.
The linguistic training facilitated
more abstract than concrete
concepts.
With no linguistic training hand
responses were facilitated, while
with linguistic training the
hand-mouth difference
disappeared.
II
BODY PART RATINGS:
ABSTRACT SENTENCES AND MOUTH
Mental state, Emotional, Math-related
sentences – «She memorizes the
procedure», «She feels happy», «She
determines the sum».
Body part rating: evaluate how much
the action described in each sentence
involved the mouth, the hand, and the
leg
Results: involvement of the mouth for
mental states and of both mouth and
hand for emotional sentences, of the
hand for math-related ones.
P<9*=(Q$;<9 M(5'##$7$-#9=(>?@I
IA
BODY PART RATINGS:
ABSTRACT WORDS AND MOUTH
The activation of linguistic
information and the mouth can
depend not on abstractness but on
heterogeneity of a category
RATINGS (see Ghio et al., 2013):
how much is the hand/mouth
involved in a possible action with the
target?
Database by Barca et al. (2011)
Difference between :
! Concrete Compact (e.g., penguin)
! Concrete Heterogeneous (e.g. tool)
Vs. Abstract (e.g., comfort)
P+$-9#*=(2.*+*&&9 M(:*+;<9=(>?@E
35
RESPONSE TIMES:
ABSTRACT WORDS AND MOUTH
10 abstract and 10 concrete Italian words
(database by Della Rosa et al., 2010) – significantly different
for Abstractness, Concreteness, Imageability, Acquisition
Modality, Age of Acquisition, Contextual Availability. Not
different in Familiarity and Average Word Length.
E.g. hat, hen, flag – justice, career, freedom.
Each combined with a concrete and an abstract
definition rated by an independent group on 7 point scale
Concrete definitions: perceptual features (e.g., «flag»
«the Italian one is white red and green») or specific
examples and situations (e.g. «carrier»: «when you obtain
a promotion at work»)
Abstract definitions: more «scientific», including
taxonomies: e.g. «hen» «domestic bird, it belongs to the
family of chickens“.
I!
R$+.*-' M(:*+;<9=(9-(8+'8S
RESPONSE TIMES:
ABSTRACT WORDS AND MOUTH
Definition (abstract, concrete) as prime,
word (abstract, concrete) as target.
Task: press the button when the definition is
appropriate.
Respond by pressing a button (response box) with the
hand or with the teeth. Hand-mouth responses
manipulated within participants, between blocks.
S'%)$(* E+D$%3.08+0(+#%*#T
IN
RESPONSE TIMES:
ABSTRACT WORDS AND MOUTH
Main effects of words and definitions: Abstract words and
abstract definitions are slower than concrete ones.
Interaction words / definitions: with abstract words the
advantage of concrete definitions is more pronounced than
with concrete words.
Consistent with the higher difficulty of abstract words and
the higher need to «ground» them in concrete experiences
IO
RESPONSE TIMES:
ABSTRACT WORDS AND MOUTH
Main effect: hand responses faster than mouth responses.
Interaction effector / kind of words: with abstract words the
advantage of the hand over the mouth responses is less
pronounced
WAT confirmed: first study in which a difference in the
activation of mouth and hands is found with «real» concrete
and abstract words and with response times.
Further study with lexical decision (Mazzuca & Borghi, in prep).
I
T
IT
ABSTRACT WORDS AND MOUTH:
DEVELOPMENT
Longitudinal study
Effect of pacifier on word acquisition in children
Late use of pacifier: influence on the acquisition of
abstract words? How about emotive words?
:$+.$=(6$UUH.$ M(:*+;<9=(9-(8+'8S(
A?
ABSTRACT WORDS
AND ACOUSTIC MODALITY
Extrinsic Simon task (De Houwer, 2003)
Abstract and concrete words (database by Dalla
Rosa et al, 2011)
SEE POSTER SESSION
DY!XUA
>6UV!WX
UVY9>
2.'++$#9=(1H;&9=(:*+;<9 $-,(L9.*&'##9 9-(8+'8S(
A@
WAT PROPOSAL
ON ABSTRACT CONCEPTS
abstract concepts:
t b
between
t
sensorimotor
i
t and
d
social/linguistic experience
1. Importance of linguistic/social
experience for their representation
2. Linguistic/social acquisition
3. Activation of language
activation
Mouth
4. Linguistic variability
D$%3.0 E+60"'&&08+GHHMN+D$%3.0 E+D0(F$,/F08+GHIJ
JG
WAT AND ABSTRACT WORDS:
CROSSLINGUISTIC EVIDENCE
Language diversity: abstract
concepts should be more influenced
by differences across languages
compared to concrete concepts
Malt et al. (1999). Categorization of
containers. Spanish, English,
Chinese speakers. Words extension
differs, but convergences in sorting.
Many studies on effects of different
languages on categorization of TIME
(Boroditsky, Casasanto, others…)
More evidence required
AI
A STUDY ON ITALIAN SIGN
LANGUAGE (LIS)
Sign llanguages are ttypically
i ll hi
highly
hl iiconic,
i th
thus it
might seem obvious that signs are grounded in
perception and action.
But with signs referring to abstract concepts, a
mixture of actions and linguistic information is used: –
language complements perception and action
Linguistic information could derive from different
sources:
from the same sign language (e.g., the LIS
IMPOSSIBLEAA sign derives from the LIS sign
POSSIBLEAA),
from a foreign sign language as ASL (e.g.,
LANGUAGE/LINGUA and LINGUISTICS)
from spoken/written Italian (e.g., TRUE).
:*+;<9=(G$89+.9=(P9$-)+',$=(Q*&#'++$=(>?@A
44
A STUDY ON ITALIAN SIGN
LANGUAGE (LIS)
LIS: VERO/VERITA’
(TRUE/TRUTH): Initialization:
initial letter of the corresponding
word V.
LIS: LINGUA (LANGUAGE):
Initialization: L shaped
hands, initially located in
proximity of the mouth,
move symmetrically
forward with a wrist
rotation
DIFFERENT KINDS
OF ABSTRACT CONCEPTS
O
O
O
O
O
Emotions: more “embodied” than pure abstract concepts
Numbers: a peculiar subset of abstract concepts –
Finger counting – activation of the hand Ghio et al., 2013
SNARC effects: relations numbers-space-body
Task: add or subtract 3 to a starting number for 22
seconds and to say the result of each calculation aloud,
for additions and subtractions, respectively), while they
are performing an ascending or descending movement
taking the elevator or the stairs (Figure 1).
1H;&9=(:$+*-9=(0-'&&9=(:*+;<9 M(L9.*&'##9=(>?@I
A!
NUMBERS
AND GROUNDING
D
Dependent
d t variable:
i bl N
Number
b off
correct calculations
Congruency effect with the
elevator
Lack of congruency with the
stairs: dual-task (climbing +
counting); movement direction:
not vertical
No effect offline, no effect in
imagery condition
Hence: during number
processing online simulation
with the whole body movement
Anelli et al., 2014: similar
results during left-right walking
Number: a peculiar subset of
abstract concepts
AN
OUTLINE
Language and affordances – re-use
9&'-2*+'(1+:'%0'-2*+',,$%1'()*/
;2*70-020&<=+10/&'()*+,%$"+&.*+-$1<+'(1+20(340/&0)+)$(&*7&
Beyond simulation: Words As social Tools
Extension of the bodily borders
?-/&%')&+)$()*#&/=+@$%1/+?/+/$)0'2+A$$2/
Acquisition of abstract concepts
Linguistic and social experiences
Embodiment: mouth activation
W$:*2+)'&*3$%0*/+')Z40/0&0$(+/&410*/
Y'&0(3+/&410*/
9&41<+$(+[$%1/+'(1+1*,0(0&0$(/
?+/&41<+$(+!&'20'(+903(+\'(34'3*+ B\!9C
Why importance of language? (talk by D’ausilio)
Why mouth activation?
WHY IS LANGUAGE SO IMPORTANT
FOR ABSTRACT CONCEPTS?
Labels are sort of glue
Words are social tools
Inner speech
Language augments our computational abilities
Language is a control and prediction system
JM
WHY IS LANGUAGE SO IMPORTANT
FOR ABSTRACT CONCEPTS?
Labels
s as «glue» - abstract concepts refer to
more sparse and diverse experiences compared
to concrete concepts. Labels help us «build» the
category.
Lupyan (2012; Boutonnet & Lupyan, 2015). After
hearing the name «dog» visual processing is
facilitated compared to when hearing a sound.
More crucial for abstract concepts?
LH
WHY IS LANGUAGE SO IMPORTANT
FOR ABSTRACT CONCEPTS?
W d as
Words
s social
i l tools
tools.
l To understand
d
d abstract
b
words
d
we need to rely on other people’s opinions and to
possess sophisticated social cognition abilities
Abstract concepts: Word tracking strategy (J.
Prinz, 2002; 2012): abstract words (e.g.,
“democracy”) are grasped in part through concrete
images, in part through verbal skills. Tracking of
definitions used by other members of our community
to help reference.
LI
WHY IS LANGUAGE SO IMPORTANT
FOR ABSTRACT CONCEPTS?
Inner speech.
eech
h W
Words
d can b
become internalized
i
li d and
d
support our thought processes: e.g., speaking to
ourselves helps us to better memorize and plan our
actions (Vygotsky, 1986; Alderson-Day & Fernyhough,
2015).
More crucial for abstract concepts?
LG
WHY IS LANGUAGE SO IMPORTANT
FOR ABSTRACT CONCEPTS?
Language
e as a way to improve
i
our computational
i
l
abilities (e.g. Clark, 1998).
Abstract concepts: Representational pluralism
(Dove, 2009, 2011, 2014): Importance of language
to enhance thought processes, but in his view «the
acquisition of language creates a new dis-embodied
semantic system, one that has many of the
properties usually associated with the amodal
symbol systems favored by traditional cognitive
science.»
LK
WHY IS LANGUAGE SO IMPORTANT
FOR ABSTRACT CONCEPTS?
Language as a controll system and
d a predictive
di i
system
Language is not only a communication system, but a
control system that programs human mind manipulating
sensorimotor experiences (Lupyan & Bergen, 2015).
Language can provide a means of building predictions: for
example, listening to a word can help our visual system to
process noisy inputs (Lupyan & Clark, 2015).
Stronger for abstract concepts?
EA
LJ
WHICH MECHANISM UNDERLIES THE
ACTIVATION OF THE MOUTH?
Re-enactment? (possibly through inner speech)
We could represent abstract words re-enacting the
linguistic/social experience of their acquisition, including
the sounds of the words, and the explanations of their
meanings
Supporting evidence:
acquisition of novel words: the responses with the
mouth are faster when explanations are provided
the mouth responses are faster after linguistic training
abstract words are associated to the acoustic modality?
To investigate: role only of the original acquisition?
GLUE
SIGN TRACKING
EE
LL
WHICH MECHANISM UNDERLIES THE
ACTIVATION OF THE MOUTH?
Re-explanation?
l
ti
Inner speech: We could reexplain to ourselves their meaning, formulating
predictions against which sensory experiences can be
assessed
Compatible evidence:
mental states concepts yield higher mouth ratings
abstract words are associated to the acoustic
modality?
AUGMENTED COMPUTATIONAL ABILITY
LANGUAGE AS A CONTROL SYSTEM
INNER SPEECH
E!
L]
WAT AND INTROSPECTIVE VIEW:
MOUTH ACTIVATION?
Possibility to reconcile WAT and introspective view of
abstract concepts (Barsalou & Wiemer-Hastings, 2005)?
Grounding in situations and introspection. More
social aspects of situations and introspective properties
elicited with ACWs E.g., Barsalou & Wiemer-Hastings (2005).
Role of introspection: due not only to content (e.g.
high presence of mental words) but to an internal
mechanism (e.g. talking to oneself?) – this could
explain the mouth activation.
EN
L^
WAT PROPOSAL
ON ABSTRACT CONCEPTS
abstract concepts:
1. Importance of linguistic and social
experience for their representation
2. Higher cross-linguistic variability
3. Mouth activation: which mechanisms?
re-enhactment of the linguistic/social
experience of their acquisition
talking to oneselves?
both?
EO
L_
OUTLINE
Language and affordances
9&'-2*+'(1+:'%0'-2*+',,$%1'()*/
;2*70-020&<=+10/&'()*+,%$"+&.*+-$1<+'(1+20(340/&0)+)$(&*7&
Beyond simulation: Words As social Tools
Extension of the bodily borders
?-/&%')&+)$()*#&/=+@$%1/+?/+/$)0'2+A$$2/
Acquisition of abstract concepts
Linguistic and social experiences
Embodiment: mouth activation
W$:*2+)'&*3$%0*/+')Z40/0&0$(+/&410*/
Y'&0(3+/&410*/
9&41<+$(+[$%1/+'(1+1*,0(0&0$(/
9&41<+$(+!&'20'(+903(+\'(34'3*+ B\!9C
Why importance of language/mouth activation?
Y*`*(')&"*(&+$,+&.*+')Z40/0&0$(+*7#*%0*()*P
Y*`*7#2'('&0$(+ $,+&.*+[$%1+"*'(0(3P
@?A+'(1+0(&%$/#*)&0:*+:0*[P
Thanks!
collaborators
Gianluca Baldassarre, Laura Barca, Ferdinand Binkofski, Cristiano
Castelfranchi, Daniele Caligiore, Felice Cimatti, Martin Fischer, Andrea
Flumini, Claudia Gianelli, Carmen Granito, Luisa Lugli, Davide
Marocco, Claudia Mazzuca, Roberto Nicoletti, Mariagrazia Ranzini,
Claudia Scorolli, Elisa Scerrati, Annalisa Setti, Luca Tummolini,
Edoardo Zarcone
!?
WAT AND BRAIN REPRESENTATION:
ABSTRACT CONCEPTS AND LANGUAGE
solid fMRI result: involvement of areas related
to language production and comprehension
during abstract concepts processing, as the left
inferior frontal gyrus and the left middle
temporal gyrus (Binder et al., 2009; Hoffman et al.,
2015; Sakreida et al., 2013; Wang et al., 2010)
META-ANALYSIS:
STABLE vs. VARIABLE AFFORDANCES
Meta-analysis
l i – fMRI
studies 1995-2015: 71
studies stable, 50 variable
and mixed (uncertain), 3
raters
Stable: reaching, grasping,
pointing tasks with
unchanging objects or
objects with constant size,
shape, weight - (canonical
included within stable).
Variable: variations of the
same object in size, shape,
weight, location, orientation
during the task, different
perspectives of an object
Sakreida et al., submitted
!>