Psych 156A/ Ling 150: Acquisition of Language II

Transcription

Psych 156A/ Ling 150: Acquisition of Language II
Psych156A/Ling150:
AcquisitionofLanguageII
Announcements
HW3isduebytheendofclasstoday
Reviewquestionsareavailableforstructure
Lecture16
StructureI
Onlinecourseevaluationsareavailableforthisclass-please
fillthemout!:)
ComputationalProblem:
Figureouttheorderofwords(syntax)
ComputationalProblem:
Figureouttheorderofwords(syntax)
Jarethjugglescrystals
Jarethjugglescrystals
SubjectVerbObject
SubjectVerbObject
NounVerbNoun
NPNP
DependsongrammaticalcategorieslikeNounsandVerbs(and
theirassociatedphrases(NP)),butalsoonmoreprecise
distinctionslikeSubjectsandObjects.
SomeNounPhrasedistinctions:
Subject=usuallytheagent/actoroftheaction,“doer”:Jareth
Object=usuallytherecipientoftheaction,“doneto”:crystals
Importantidea:Theobservablewordorderspeakersproduce(like
SubjectObjectVerb)istheresultofasystemofwordorderrulesthat
speakersunconsciouslyusewhentheyspeak.Thissystemofwordorder
rulesiscalledsyntax.
ComputationalProblem:
Figureouttheorderofwords(syntax)
Jarethjugglescrystals
Jarethjugglescrystals
SubjectVerbObject
SubjectVerbObject
OnewaytogenerateSubjectVerbObjectorder:
Thelinguisticsystemspecifiesthatorderasthegeneralpatternof
thelanguage.AnexampleofthiskindofsystemisEnglish.
English
ComputationalProblem:
Figureouttheorderofwords(syntax)
AnotherwaytogenerateSubjectVerbObjectorder:
ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but
theVerbinmainclausesmovestothesecondpositionandsomeotherphrase
(liketheSubject)movestothefirstposition.Anexamplelanguagelikethisis
German.
SubjectVerbObject
German
ComputationalProblem:
Figureouttheorderofwords(syntax)
SubjectObjectVerb
ComputationalProblem:
Figureouttheorderofwords(syntax)
Jarethjugglescrystals
Jarethjugglescrystals
SubjectVerbObject
SubjectVerbObject
AnotherwaytogenerateSubjectVerbObjectorder:
ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but
theVerbinmainclausesmovestothesecondpositionandsomeotherphrase
(liketheSubject)movestothefirstposition.Anexamplelanguagelikethisis
German.
AnotherwaytogenerateSubjectVerbObjectorder:
ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but
theVerbinmainclausesmovestothesecondpositionandsomeotherphrase
(liketheSubject)movestothefirstposition.Anexamplelanguagelikethisis
German.
movementrules
German
____ VerbSubjectObjectVerb
movementrules
German
SubjectVerbSubjectObjectVerb
ComputationalProblem:
Figureouttheorderofwords(syntax)
ComputationalProblem:
Figureouttheorderofwords(syntax)
Jarethjugglescrystals
Jarethjugglescrystals
SubjectVerbObject
SubjectVerbObject
AthirdwaytogenerateSubjectVerbObjectorder:
ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but
theObjectmovesaftertheVerbincertaincontexts(theObjectisunexpected
information).Kannadaisalanguagelikethis.
AthirdwaytogenerateSubjectVerbObjectorder:
ThelinguisticsystemspecifiesSubjectObjectVerbasthegeneralpattern,but
theObjectmovesaftertheVerbincertaincontexts(theObjectisunexpected
information).Kannadaisalanguagelikethis.
movementrule
Kannada
SubjectObjectVerb
ComputationalProblem:
Figureouttheorderofwords(syntax)
Kannada
SubjectObjectVerbObject
ComputationalProblem:
Figureouttheorderofwords(syntax)
Jarethjugglescrystals
Jarethjugglescrystals
SubjectVerbObject
SubjectVerbObject
German
English
SubjectVerbObject
SubjectVerbSubjectObjectVerb
Kannada
SubjectObjectVerbObject
Thelearningproblem:Howdochildrenknowwhich
systemtheirlanguageuses?
German
English
SubjectVerbObject
SubjectVerbSubjectObjectVerb
Kannada
SubjectObjectVerbObject
Thisisahardquestion!
Childrenonlyseetheoutputofthesystem(theobservableword
orderofSubjectVerbObject).
Translationisnotsoeasy:
morethanjustword-by-word
Syntax:Onereasonwhytranslationissohard
http://www.nbc.com/nbc/The_Tonight_Show_with_Jay_Leno/headlines/
Translationisnotsoeasy:
morethanjustword-by-word
Translationisnotsoeasy:
morethanjustword-by-word
translate.google.com
translate.google.com
Hebrew
HaitianCreole
Literally:
Throughdangersimmenseanddifficultiesnotnumbered,there-istomefightingthroughmyherecastletransitioncitygoblintakebackyou
childthere-wasto-youstolen.
Literally:
Throughdangercountlessanddifficultiescountless,Iwasfighthow
meheretheymansionthemorefarthancitiestheGoblintheytake
backchildrenofthatyouwasthiefit.
Translationisnotsoeasy:
morethanjustword-by-word
translate.google.com
Abouthumanknowledge:
Language&variation
Hindi
Literally:
Untoldanduncountabledifficultiesthreatsmediumthrough,Iyou
stoleisthatchildrenbacktaketheghostcitybeyondpalacethehere
yourmethodsfromfightfought.
NavajoCodeTalkers
Crucialcryptographicmethodusedin
WorldWarII
NavajoCodeTalkerParadox(Baker2001)
EnglishmustbeverydifferentfromNavajo
JapanesecoulddecodeEnglish,but
couldn’tdecodeNavajowhentheydidn’t
knowitwasNavajo.
http://en.wikipedia.org/wiki/Code_talker#Use_of_Navajo
“…JohnstonsawNavajoasansweringthemilitaryrequirementforan
undecipherablecode.NavajowasspokenonlyontheNavajolandsofthe
AmericanSouthwest,anditssyntaxandtonalqualities,nottomentiondialects,
madeitunintelligibletoanyonewithoutextensiveexposureandtraining.One
estimateindicatesthatattheoutbreakofWorldWarIIfewerthan30nonNavajoscouldunderstandthelanguage….”
https://www.youtube.com/watch?v=5rSvm3m8ZUA
(~3minvideo)
EnglishmustbesimilartoNavajo
EnglishcanbetranslatedintoNavajoandbackwithnolossof
meaning.(Languagesarenotjustaproductoftheculture-pastoral
Arizonalifestylecouldn’thavepreparedthecodetalkersforPacific
Islandhightechwarfare.Yet,translationwasstillpossible.)
Typesofvariation
Morphology(wordforms)
English:invariantwordforms
“thegirliscrying”,“Iamcrying”
Navajo:noinvariantforms(theremaybe100-200prefixesforverb
stems)
At’éédyicha.“Girlcrying”
Yishcha.“Iamcrying”
(yi+sh+cha)
Ninááhwiishdlaad.“Iamagainplowing”
(ni+náá+ho+hi+sh+l+dlaad)
Typesofvariation
Wordorder(syntax)
English:SubjectVerbObject(invariantwordorder)
“Theboysawthegirl”
Navajo:SubjectObjectVerb,ObjectSubjectVerb
(varyingwordorders,meaningdependsonlyonverb’sform)
Ashkiiat’éédyiyiiltsá
boygirlsaw
“Theboysawthegirl”
Ashkiiat’éédbiilstá
boygirlsaw
“Thegirlsawtheboy”
Typesofvariation
wals.info:TheWorldAtlasofLanguageStructures
Typesofvariation
Let’slookatsyntax…
Jacklaughs.
Thistellsusthatmostlanguages
havetheSubjectcomebeforethe
Verb…butnotalldo.
LaughsJack.
Typesofvariation
Let’slookatsyntax…
Typesofvariation
Let’slookatsyntax…
WhatvaluedoesEnglishhave?
WhataboutFijian?
WhataboutSpanish?
HowarethedifferentSubjectand
Verbordersdistributedaroundthe
world?
Similarities&differences:Parameters
Chomsky:Differentcombinationsofdifferentbasic
elements(parameters)wouldyieldtheobservable
languages(similartothewaydifferentcombinationsof
differentbasicelementsinchemistryyieldmany
different-seemingsubstances).
Thinkingaboutsyntacticvariation
Similarities&differences:Parameters
BigIdea:Arelativelysmallnumberofsyntax
parametersyieldsalargenumberofdifferent
languages’syntacticsystems.
Similarities&differences:Parameters
BigIdea:Arelativelysmallnumberofsyntax
parametersyieldsalargenumberofdifferent
languages’syntacticsystems.
5different
parametersof
variation
Similarities&differences:Parameters
BigIdea:Arelativelysmallnumberofsyntax
parametersyieldsalargenumberofdifferent
languages’syntacticsystems.
Similarities&differences:Parameters
BigIdea:Arelativelysmallnumberofsyntax
parametersyieldsalargenumberofdifferent
languages’syntacticsystems.
2different
parameter
valuesofone
parameter
Totallanguages
thatcanbe
represented
=2*2*2*2*2
=25
=32
Similarities&differences:Parameters
Learninglanguagestructure
Chomsky:Childrenarebornknowingtheparameters
ofvariation.ThisispartofUniversalGrammar.Input
fromthenativelinguisticenvironmentdetermines
whatvaluestheseparametersshouldhave.
BigIdea:Arelativelysmallnumberofsyntax
parametersyieldsalargenumberofdifferent
languages’syntacticsystems.
English
Japanese
Tagalog
Navajo
French
…
Learninglanguagestructure
Learninglanguagestructure
Chomsky:Childrenarebornknowingtheparameters
ofvariation.ThisispartofUniversalGrammar.Input
fromthenativelinguisticenvironmentdetermines
whatvaluestheseparametersshouldhave.
Chomsky:Childrenarebornknowingtheparameters
ofvariation.ThisispartofUniversalGrammar.Input
fromthenativelinguisticenvironmentdetermines
whatvaluestheseparametersshouldhave.
English
Japanese
Learninglanguagestructure
Chomsky:Childrenarebornknowingtheparameters
ofvariation.ThisispartofUniversalGrammar.Input
fromthenativelinguisticenvironmentdetermines
whatvaluestheseparametersshouldhave.
Navajo
Generalizationsaboutlanguagestructure
Greenberg’swordordergeneralizations
Greenberg’swordordergeneralizations
Navajo
Navajo
Japanese
Japanese
Basicwordorder:
SubjectObjectVerb
Basicwordorder:
SubjectObjectVerb
Ashkiiat’éédyiyiiltsá
boygirlsaw
Jareth-gaHoggle-obutta
JarethHogglehit
“Theboysawthegirl”
“JarethhitHoggle”
Greenberg’swordordergeneralizations
Greenberg’swordordergeneralizations
Navajo
Navajo
Postpositions:
NounPhrasePostposition
‘éé’biihnáásdzá
clothingintoI-got-back
“Igotbackinto(my)clothes.”
Japanese
Postpositions:
NounPhrasePostposition
Jareth-gaSarahtokurumada
JarethSarahwithcarby
Londonniitta
Londontowent
Japanese
PossessorbeforePossessed
PossessorbeforePossessed
PossessorPossession
PossessorPossession
Chidíbi-jáád
Carits-leg
Toby-noimooto-ga
Toby’ssister
“thecar’swheel”
“Toby’ssister”
“JarethwenttoLondonwithSarahby
car.”
Greenberg’swordordergeneralizations
Greenberg’swordordergeneralizations
Navajo
English
Japanese
Basicwordorder:
SubjectObjectVerb
Basicwordorder:
SubjectObjectVerb
Postpositions:
NounPhrasePostposition
Postpositions:
NounPhrasePostposition
PossessorbeforePossessed
PossessorPossession
PossessorbeforePossessed
PossessorPossession
Despitethedifferencesinthelanguages(andtheirculturalhistories),
bothJapaneseandNavajoareverysimilarwhenviewedthroughthese
threestructuraldescriptions.
Edo(Nigeria)
Greenberg’swordordergeneralizations
Greenberg’swordordergeneralizations
English
English
Edo(Nigeria)
Basicwordorder:
SubjectVerbObject
Basicwordorder:
SubjectVerbObject
SarahfoundToby
ÒzómiénAdésuwá
OzofoundAdesuwa
Edo(Nigeria)
Prepositions:
PrepositionNounPhrase
Prepositions:
PrepositionNounPhrase
JarethgavethecrystaltoSarah
ÒzórhiénénéebénéAdésuwá
OzogavethebooktoAdesuwa
Greenberg’swordordergeneralizations
Greenberg’swordordergeneralizations
English
English
Edo(Nigeria)
PossessedbeforePossessor
PossessedbeforePossessor
PossessionPossessor
PossessionPossessor
questofSarah
OmoOzó
childOzo
(alternative:Sarah’squest)
“childofOzo”
Edo(Nigeria)
Basicwordorder:
SubjectVerbObject
Basicwordorder:
SubjectVerbObject
Prepositions:
PrepositionNounPhrase
Prepositions:
PrepositionNounPhrase
PossessedbeforePossessor
PossessionPossessor
PossessedbeforePossessor
PossessionPossessor
Again,despitethedifferencesinthelanguages(andtheircultural
histories),bothEnglishandEdoareverysimilarwhenviewedthrough
thesethreestructuraldescriptions.
Greenberg’swordordergeneralizations
Greenbergfoundforty-five“universals”oflanguages-patterns
overwhelminglyfollowedbylanguageswithunsharedhistory(Navajo
&Japanese,English&Edo)
Notallcombinationsarepossible-somepatternsrarelyappear
Ex:SubjectVerbObjectlanguage(English/Edo-like)+postpositions
(Navajo/Japanese-like)
Onepotentialparameter
English
Italian
SubjectVerb
SubjectVerb
Jarethverrá
Jarethwill-come
“Jarethwillcome.”
“Jarethwillcome.”
grammatical
grammatical
Moral:Languagesmaybemoresimilarthantheyfirstappear“onthe
surface”,especiallyifweconsidertheirstructuralproperties.
Onepotentialparameter
English
Italian
*VerbSubject
VerbSubject
VerráJareth
Will-arriveJareth
*WillarriveJareth
Onepotentialparameter
English
*Verb
Willcome
grammatical
Verb
Verrá
He-will-come
“Hewillcome”
“Jarethwillarrive”
ungrammatical
Italian
ungrammatical
grammatical
Onepotentialparameter
Onepotentialparameter
Expletivesubjects:wordswithoutcontent
(maybemoredifficulttonotice)
English
Italian
SubjectVerb
SubjectVerb
English
Italian
*VerbSubject
*Verb
VerbSubject
Verb
Raining.
Piove.
It-rains.
“It’sraining.”
“It’sraining.”
Notokaytoleaveout
expletivesubject“it”.
Okaytoleaveout
expletivesubject“it”.
Thesewordorderpatternsmightbefairlyeasytonotice.They
involvethecombinationsofSubjectandVerbthatare
grammaticalinthelanguage.Achildmightbeabletonoticethe
prevalenceofsomepatternsandtheabsenceofothers.
Onepotentialparameter
That-traceeffectforsubjectquestions
English
Whodoyouthink(*that)willcome?
Requiresno“that”inembeddedclause,despite
allowing“that”indeclarativesandobject
questions
Ithink(that)HogglewillsaveSarah.
Whodidyouthink(that)Hogglewouldsave?
Italian
Onepotentialparameter
That-traceeffectforsubjectquestions
English
Italian
CredicheJarethverrá.
YouthinkthatJarethwill-come.
“YouthinkthatJarethwillcome.”
Checrediche__verrá?
Whothink-youthatwill-come?
“Whodoyouthinkwillcome?”
Allows“that”intheembedded
clauseofasubjectquestion(and
declarativeclauses).
Onepotentialparameter
English
Italian
SubjectVerb
SubjectVerb
*VerbSubject
VerbSubject
*Verb
TheValueofParameters:Learningthehardstuffby
noticingtheeasypatterns
Verb
Notokaytoleaveout
expletivesubject“it”.
Okaytoleaveout
expletivesubject“it”.
Requiresspecialactionfor
embeddedsubjectquestions.
Doesnotrequirespecialaction
forembeddedsubject
questions.
Alltheseinvolvethesubjectinsomeway-coincidence?
Idea:No!There’salanguageparameterinvolvingthesubject.
TheValueofParameters:Learningthehardstuffby
noticingtheeasypatterns
Englishvs.Italian:SubjectParameter
Bigidea:Ifallthesestructuralpatternsaregeneratedfromthesame
linguisticparameter(e.g.a“subject”parameter),thenchildrencan
learnthehard-to-noticepatterns(likethepatternsofembeddedsubject
questions)bybeingexposedtotheeasy-to-noticepatterns(likethe
optionaluseofsubjectswithverbs).Thehard-to-noticepatternsare
generatedbyonesettingoftheparameter,whichchildrencanlearn
fromtheeasy-to-noticepatterns.
Children’sknowledgeoflanguagestructurevariationisbelievedby
linguisticnativiststobepartofUniversalGrammar,whichchildrenare
bornwith.
Englishvs.Italian:SubjectParameter
English
SubjectVerb
Italian
SubjectVerb
Easierto
notice
*VerbSubject
VerbSubject
Hardtonotice
*Verb
Verb
Expletives
Itrains
Piove.
It-rains.
EmbeddedSubject-questionformation(easytomiss)
Whodoyouthink(*that)willcome?
Checrediche__verrá?
Whothink-youthatwill-come?
Anotherpossibleparameter
Syntax:theHeadDirectionalityparameter(Baker2001,Cook&Newson
1996):headsofphrases(ex:NounsofNounPhrases,VerbsofVerb
Phrases,PrepositionsofPrepositionPhrases)areconsistentlyineither
theleftmostorrightmostposition
Japanese/Navajo:Head-Last
VerbPhrase:
ObjectVerb
Postpositions:
NounPhrasePostposition
VP
NP
Object
Verb
PP
NP
Object
P
postposition
UniversalGrammar:Parameters
Anotherpossibleparameter
Syntax:theHeadDirectionalityparameter(Baker2001,Cook&Newson
1996):headsofphrases(ex:NounsofNounPhrases,VerbsofVerb
Phrases,PrepositionsofPrepositionPhrases)areconsistentlyineither
theleftmostorrightmostposition
Edo/English:Head-First
VerbPhrase:
VerbObject
Prepositions:
PrepositionNounPhrase
NP
Object
Edo/English
NP
Subject
S
VP
VP
NP
Subject
NP Verb
Object
PP
PP
P
Preposition
Japanese/Navajo
S
VP
Verb
Atthislevelofstructuralanalysis(parameters),languagesdiffervaryminimally
fromeachother.Thismakeslanguagestructuremucheasierforchildrento
learn.Alltheyneedtodoissettherightparametervaluesfortheirlanguage,
basedonthedatathatareeasytoobserve.
NP
Object
Verb
NP
Object
PP
NP P
Object postposition
P NP
preposition Object
Parameters
Butwhatarelinguisticparametersreally?Howdotheywork?What
exactlyaretheysupposedtodo?
Aparameterismeanttobesomethingthatcanaccountformultiple
observationsinsomedomain.
Parameterforastatisticalmodel:determineswhatthemodel
predictswillbeobservedintheworldinavarietyofsituations
Parameterforourmental(andlinguistic)model:determineswhat
wepredictwillbeobservedintheworldinavarietyofsituations
Statisticalparameters
Thenormaldistributionisa
statisticalmodelthatuses
twoparameters:
-µ forthemean
-σ forthestandarddeviation
Statisticalparameters
Supposethisisamodelofhow
manyminuteslateyou’llbe
toclass.
Let’susethemodelwithµ =
0,andσ2=0.2.(blueline)
Ifweknowthevaluesoftheseparameters,wecanmakepredictionsaboutthe
likelihoodofdatawerarelyorneversee.
Statisticalparameters
Supposethisisamodelofhow
manyminuteslateyou’llbe
toclass.
Let’susethemodelwithµ =
0,andσ2=0.2.(blueline)
Howlikelyareyoutobe5minuteslate,giventheseparameters?
Notverylikely!Wecantellthisjustbyknowingthevaluesofthetwo
statisticalparameters.Theseparametervaluesallowustoinferthe
probabilityofsomeobservedbehavior.
Statisticalparameters
Observingdifferentquantitiesof
datawithparticularvalues
cantelluswhichvaluesofμ
andσ2aremostlikely,ifwe
knowwearelookingto
determinethevaluesofμ
andσ2infunctionφ(X)
Observingdatapointsdistributedlikethegreencurvetellsusthatμislikelyto
bearound-2,forexample.
Linguisticprinciplesvs.linguisticparameters
Statisticalvs.Linguisticparameters
Importantsimilarity:
Wedonotseetheprocessthat
generatesthedata,butonly
thedatathemselves.This
meansthatinordertoform
ourexpectationsaboutX,
weare,ineffect,reverse
engineeringtheobservable
data.
Ourknowledgeoftheunderlyingfunction/principlethatgeneratesthesedata-
φ(X)-aswellastheassociatedparameters-μ,andσ2-allowsustorepresentan
infinitenumberofexpectationsaboutthebehaviorofvariableX.
Linguisticprinciplesvs.linguisticparameters
Bothprinciplesandparametersareoftenthoughtofasinnatedomain-specific
abstractionsthatconnecttomanystructuralpropertiesaboutlanguage.
Linguisticparameterscorrespondtothepropertiesthatvaryacrosshuman
languages.Comparison:μandσ2determinetheexactformofthecurvethat
representsthelikelihoodofobservingcertaindata.Whiledifferentvaluesfor
theseparameterscanproducemanydifferentcurves,thesecurvessharetheir
underlyingformduetothecommoninvariantfunction.
Bothprinciplesandparametersareoftenthoughtofasinnatedomain-specific
abstractionsthatconnecttomanystructuralpropertiesaboutlanguage.
Linguisticprinciplescorrespondtothepropertiesthatareinvariantacrossall
humanlanguages.Comparison:theequation’sform–itisthestatistical
“principle”thatexplainstheobserveddata.
Theutilityofconnectingtomultipleproperties
Thefactthatparametersconnecttomultiplestructuralproperties
thenbecomesaverygoodthingfromtheperspectiveofsomeone
tryingtoacquirelanguage.Thisisbecauseachildcanlearnabout
thatparameter’svaluebyobservingmanydifferentkindsof
examplesinthelanguage.
“Thericherthedeductivestructureassociatedwithaparticular
parameter,thegreatertherangeofpotential‘triggering’datawhich
willbeavailabletothechildforthe‘fixing’oftheparticular
parameter”–Hyams(1987)
Theutilityofconnectingtomultipleproperties
Whyhard-to-learnstructuresareeasier
Let’sassumeanumberofpropertiesareallconnectedtoparameter
P,whichcantakeoneoftwovalues:aorb.
Parameterscanbeespeciallyusefulwhenachildistryingtolearn
thethingsaboutlanguagestructurethatareotherwisehardto
learn,perhapsbecausetheyareverycomplexproperties
themselvesorbecausetheyappearveryinfrequentlyinthe
availabledata.
P
aorb?
P1
P2
P3
P4
P5
Whyhard-to-learnstructuresareeasier
HowdowelearnwhetherP4showsbehavioraorb?
OnewayistoobservemanyinstancesofP4.
P
P5
ButwhatifP4occursveryrarely?Wemightneverseeanyexamples
ofP4.
aorb?
P1
P2
P3
P4
Whyhard-to-learnstructuresareeasier
P
P1
P2
P3
aaaaaaaaaa…
P4
P5
???
aorb?
Whyhard-to-learnstructuresareeasier
Fortunately,ifP4isconnectedtoP,wecanlearnthevalueforP4by
learningthevalueofP.Alsofortunately,PisconnectedtoP1,P2,
P3,andP5.
P
Whyhard-to-learnstructuresareeasier
Step1:ObserveP1,P2,P3,orP5.Inthiscase,alltheobserved
examplesofthesestructuresarebehaviora.
aorb?
P
P1
P2
P3
P1
P2
P3
P4 ???
P4 ???
P5
P5
Whyhard-to-learnstructuresareeasier
Step2:UsethisknowledgetosetthevalueofparameterPtoa.
P
P1
P2
P3
P4
P5
???
aaa
aaaaaaaaaaa…
Whyhard-to-learnstructuresareeasier
Step3:SinceparameterPisconnectedtoP4,wecanpredictthatP4will
alsoshowbehaviora-eventhoughwe’veneverseenanyexamplesof
it!(WecanalsoinferP3andP5thesameway.)
a
aaa
aaaaaaaaaaa…
aorb?
P
P1
P2
P3
P4
P5
a
aaa
aaaaaaaaaaa…
a
HierarchicalBayesianlearninglinks:
Overhypotheses
Whyacquisitioniseasier
P
P1
P2
P3
P4
P5
a
aaa
aaaaaaaaaaa…
a
a
Thishighlightsanotherbenefitof
parameters-wedon’thavetolearn
thebehaviorofeachstructure
individually.Instead,wecanobserve
somestructures(ex:P1andP2)and
infertherightbehaviorforthe
remainingstructures(P3,P4,andP5).
OverhypothesesinhierarchicalBayesianlearningare
generalizationsmadeatamoreabstractlevel,whichcovermany
differentdatatypes.
Inthisway,theyaresimilarinspirittolinguisticparameters.
Thatis,insteadofhavingtomake5
decisions(oneforP1,P2,P3,P4,and
P5),weactuallyonlyneedtomake
onedecision-isPaorb?
a
HierarchicalBayesianlearninglinks:
Overhypotheses
HierarchicalBayesianlearninglinks:
Overhypotheses
Overhypothesisexample
Overhypothesisexample
Supposeyouareobservingthecontentsofmarblebags.
Thefirstbagyoulookathas20blackmarbles.
20
HierarchicalBayesianlearninglinks:
Overhypotheses
HierarchicalBayesianlearninglinks:
Overhypotheses
Overhypothesisexample
Overhypothesisexample
Thesecondbagyoulookathas20whitemarbles.
Thethirdandfourthbagsyoulookathave20blackmarbles.
20
20
20
HierarchicalBayesianlearninglinks:
Overhypotheses
20
20
20
HierarchicalBayesianlearninglinks:
Overhypotheses
Overhypothesisexample
Overhypothesisexample
Yougetafifthbagandpulloutasinglemarble.It’swhite.Whatdoyou
predictaboutthecolordistributionoftherestofthemarblesinthe
bag?
Mostadultspredictthisbagwillcontain19otherwhitemarbles,fora
totalof20whitemarbles.
1
20
20
20
20
1
20
20
20
20
20
HierarchicalBayesianlearninglinks:
Overhypotheses
HierarchicalBayesianlearninglinks:
Overhypotheses
Overhypothesisexample
Overhypothesisexample
Whatifyouthengetasixthbagandpulloutasinglepurplemarble
fromit?
Mostadultswouldpredictthattheother19marblesinthatbagare
purpletoo,for20purplemarblestotal.
1
20
20
20
20
1
20
1
20
HierarchicalBayesianlearninglinks:
Overhypotheses
Overhypothesisexample
20
20
1
20
20
HierarchicalBayesianlearninglinks:
Overhypotheses
Overhypothesisexample
Whydoesthishappen?Itseemslikeyou’relearningsomethingabout
thecolordistributioningeneral(notjustforaparticularbag):all
marblesinabaghavethesamecolor.Thisallowsyoutomake
predictionswhenyou’veonlyseenasinglemarbleofwhatevercolor
fromabag.
1
20 1
20
20
20
20
20
20
observed
20
20
1
20
20
20
1
20
HierarchicalBayesianlearninglinks:
Overhypotheses
HierarchicalBayesianlearninglinks:
Overhypotheses
Overhypothesisexample
Overhypothesisexample
overhypothesis
allthesamecolor
allblack
20
allwhite
20
allblack
20
overhypothesis
allthesamecolor
allblack
1
20
1
20
20
HierarchicalBayesianlearninglinks:
Overhypotheses
allblack
allwhite
20
20
allblack
20
makepredictions
allblack
1
20
1
20
20
HierarchicalBayesianlearninglinks:
Overhypotheses
Linguisticoverhypothesisexample
Linguisticoverhypothesisexample
Supposeyouareobservingthewordorderofsentencesyouhear.
ThefirstsentenceyouhearhastheVerbbeforetheObject(“Seethe
penguin?”)
VerbObject
HierarchicalBayesianlearninglinks:
Overhypotheses
HierarchicalBayesianlearninglinks:
Overhypotheses
Linguisticoverhypothesisexample
Linguisticoverhypothesisexample
ThesecondsentenceyouhearhasthePrepositionbeforetheObject
(“Ilikethepenguinontheiceberg”)andalsotheVerbbeforethe
Object(“Ilikethepenguinontheiceberg”).
Thesedatatellyouaboutwordorderforverbsandobjectsandalso
aboutwordorderforprepositionsandtheirobjects.
observed
VerbObject
PrepositionObject
VerbObject
VerbObject
HierarchicalBayesianlearninglinks:
Overhypotheses
PrepositionObject
VerbObject
HierarchicalBayesianlearninglinks:
Overhypotheses
Linguisticoverhypothesisexample
Linguisticoverhypothesisexample
Inaddition,theyarerelatedviatheheaddirectionalityparameter,
whichfunctionsasanoverhypothesis.
Knowingthevalueofthisparameterallowsyoutopredictotherword
orderpropertiesofthelanguage.
Headdirectionality=headfirst
headfirst
VerbObject
headfirst
PrepositionObject
VerbObject
Headdirectionality=headfirst
headfirst
VerbObject
makepredictions
headfirst
PrepositionObject
VerbObject
PossessionPossessor
HierarchicalBayesianlearninglinks:
Overhypotheses
Learningoverhypotheses
Learningoverhypotheses:Dewar&Xu(2010)
9-month-olds
Question:
Whenprovidedwithpartialevidenceaboutafew
objectsinafewcategories,caninfantsforma
moreabstractgeneralization(anoverhypothesis)
thatthenappliestoanewcategory?
Bayesianlearnercomputationalmodelsareabletolearn
overhypotheses,providedtheyknowwhattheparametersare
andtherangeofvaluesthoseparameterscantake(ex:Kemp,
Perfors,&Tenenbaum2006).
Whataboutreallearners?
Learningoverhypotheses:Dewar&Xu(2010)
9-month-olds
Trainingtrials:
Observefourdifferentobjects
pulledoutbyexperimenterwho
hadhereyesclosed-theobjects
aredifferentcolorsbutalways
havethesameshape.
Learningoverhypotheses:Dewar&Xu(2010)
9-month-olds
Experimentaltrials:
Expectedoutcome(assuming
infantshadtheoverhypothesis
thatalltheobjectsfromasingle
boxshouldbethesameshape)
=
Experimenterpullsouttwo
objectswiththesamenew
shape.
Infantsshouldnotbesurprised.
Learningoverhypotheses:Dewar&Xu(2010)
9-month-olds
9-month-olds
Experimentaltrials:
Unexpectedoutcome(assuming
infantshadtheoverhypothesis
thatalltheobjectsfromasingle
boxshouldbethesameshape)
=
Experimenterpullsouttwo
objectswithdifferentshapes,
onewhichisnewandonewhich
isold.
Infantsshouldbesurprised.
Learningoverhypotheses:Dewar&Xu(2010)
9-month-olds
Controltrials:
Expectedoutcome(assuming
infantshadtheoverhypothesis
thatalltheobjectsfromasingle
boxshouldbethesameshape)
=
Experimenterpullsouttwo
objectswiththesamenew
shape.
Infantsshouldnotbesurprised.
Learningoverhypotheses:Dewar&Xu(2010)
9-month-olds
Controltrials:
Unexpectedoutcomecontrol=
Experimenterpullsouttwo
objects,onewithanewshape
thatcamefromthenewboxand
onewithanoldshapethatcame
fromanoldboxthatcontained
thatshape.
Learningoverhypotheses:Dewar&Xu(2010)
Infantsshouldnotbesurprised
thisoutcomeiscompatiblewith
theoverhypothesis.(The
overhypothesisisactually
irrelevanthere.)
Results:
Infantsintheexperimental
conditionlookedlongeratthe
unexpectedoutcome(~14.28s)
whencomparedtotheexpected
outcome(~11.32s).
Theyweresurprisedatthe
evidencethatdidn’tsupportthe
overhypothesis!
Learningoverhypotheses:Dewar&Xu(2010)
9-month-olds
9-month-olds
Results:
Infantsinthecontrolcondition
didnotlooklongeratthe
expectedoutcomeascompared
totheunexpectedoutcome
controlthathadthesame
objectspresent(~10.3-11.0s).
Learningoverhypotheses:Dewar&Xu(2010)
Theywerenotsurprisedatthe
evidencethatwascompatible
withtheoverhypothesis,evenif
theevidenceinvolvedtwo
differentlyshapedobjects.
Overallresult:
9-month-oldsappearableto
formoverhypothesesfromvery
limiteddatasets.
Hopefully,thismeanstheycan
alsouselinguisticparametersto
learn,sinceparametersare
similartooverhypothesesabout
language!
Summary:Linguisticparameters
Parametersmakeacquisitioneasierbecausehard-to-learn
structurescanbelearnedbyobservingeasy-to-learnstructuresthat
areconnectedtothesameparameters.
Questions?
Linguisticparametersaresimilartostatisticalparametersinthat
theyareabstractionsabouttheobservabledata.Forlinguistic
parameters,theobservabledataarelanguagedata.
Parametersmaybesimilartooverhypotheses,whichBayesian
learnersand9-month-oldsarecapableoflearning.
Youshouldbeabletodoupthroughquestion11on
thestructurereviewquestions.
ExtraMaterial
Syntax:Onereasonwhynaturallanguage
comprehensionissohardforcomputers
SolvingtheLanguageProblem
(ArtificialIntelligence)
SolvingtheLanguageProblem
(ArtificialIntelligence)
HAL9000from2001:ASpaceOdyssey(1968)
2012:Apple’sSiriisgettingcloser,thoughstillhasproblems…
PerfectproductionandcomprehensionofEnglish.
http://bits.blogs.nytimes.com/2012/07/15/with-apple’s-siri-aromance-gone-sour/?_php=true&_type=blogs&_r=0
1960s:Languagenotconsideredoneofthe“hard”problemsofartificialintelligence.
2010:Gettingbetterbutstillnotperfect.
http://www.research.att.com/~ttsweb/tts/demo.php
SolvingtheLanguageProblem
(ArtificialIntelligence)
SolvingtheLanguageProblem
(ArtificialIntelligence)
Contrast:Chess-playing.
Updatefor2011onamachine’sabilitiestodowhathumansdo:
In1997,aprogramnamedDeepBluebeat
thereigningworldchampioninchess.Itdid
thisbyhavingenoughcomputational
resourcestoinvestigateeverymoveoption
beforeitactuallymadethechessmove.This
showsthatcomputers’poorperformanceon
languageisnotaboutinsufficient
computationalpower,sincethereisenough
computationalpowertosolvethechessplayingproblem(whichsomepeoplemight
consideraverydifficultproblem).
Manvs.Machine(Watson)inJeopardy
&howhardaproblemlanguagecomprehensionandproductionis
SolvingtheLanguageProblem
(ArtificialIntelligence)
2013:Trueon-the-flylanguagecomprehensionisstillprettyhard,aswellas
determiningtheanswerto“commonsense”questionsthatarephrased
naturally.
http://www.sciencedaily.com/releases/2013/07/130715151059.htm
“Oneofthehardestproblemsinbuildinganar€ficialintelligence,Sloansaid,isdevising
acomputerprogramthatcanmakesoundandprudentjudgmentbasedonasimple
percep€onofthesitua€onorfacts-thedic€onarydefini€onofcommonsense.
CommonsensehaseludedAIengineersbecauseitrequiresbothaverylargecollec€on
offactsandwhatSloancallsimplicitfacts—thingssoobviousthatwedon'tknowwe
knowthem.Acomputermayknowthetemperatureatwhichwaterfreezes,butwe
knowthaticeiscold.”-JeanneGalatzer-Levy
“We'res€llveryfarfromprogramswithcommonsense-AIthatcananswer
comprehensionques€onswiththeskillofachildof8,"saidSloan.Heandhis
colleagueshopethestudywillhelptofocusa~en€ononthe"hardspots"inAI
research.
http://www.youtube.com/watch?v=dr7IxQeXr7g
(approximately9minvideo)
Watsonvs.allhumanity
h~ps://www.youtube.com/watch?v=WFR3lOm_xhE
(approximately4minvideo)
Typesofvariation
Vocabulary
English“think”verbs:think,know,wonder,suppose,assume,…
Multipletypesoftheactionverb“think”.Eachhascertainusesthatare
appropriate.
“Iwonderwhetherthegirlsavedherlittlebrotherfromthe
goblins.”[grammatical]
*“Isupposewhetherthegirlsavedherlittlebrotherfromthe
goblins.”[ungrammatical]
Typesofvariation
Vocabulary
English“think”verbs:think,know,wonder,suppose,assume,…
Navajo“carry”verbs:dependsonobjectbeingcarried
aah(carryasolidround-ishobject)
kaah(carryanopencontainerwithcontents)
lé(carryaflexibleobject)
Typesofvariation
Sounds:EachlanguageusesaparticularsubsetofthesoundsintheInternational
PhoneticAlphabet,whichrepresentsallthesoundsusedinallhumanlanguages.
There’softenoverlap(ex:“m”,“p”areusedinmanylanguages),butlanguagesalso
maymakeuseofthelesscommonsounds.
lesscommonEnglishsounds:“th”[T], “th”[D]
lesscommonNavajosounds:“whisperedl”,“nasalizeda”,…