Partan (2002)
Transcription
Partan (2002)
Single and Multichannel Signal Composition: Facial Expressions and Vocalizations of Rhesus Macaques (Macaca mulatta) Author(s): Sarah R. Partan Reviewed work(s): Source: Behaviour, Vol. 139, No. 8 (Aug., 2002), pp. 993-1027 Published by: BRILL Stable URL: http://www.jstor.org/stable/4535968 . Accessed: 28/07/2012 15:18 Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp . JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. . BRILL is collaborating with JSTOR to digitize, preserve and extend access to Behaviour. http://www.jstor.org SINGLE AND MULTICHANNEL SIGNAL COMPOSITION: FACIAL EXPRESSIONS AND VOCALIZATIONSOF RHESUS MACAQUES (MACACA MULATTA) by SARAH R. PARTAN12) (Animal BehaviorGroup,Universityof California,Davis) (Acc. 7-IV-2002) Summary A methodfor simultaneouslyexaminingvisual and vocal componentsof expressivebehavior is described, compiled from video recordings of social behavior of a free-ranginggroup of rhesus macaques on Cayo Santiago, Puerto Rico. I developed a catalog of expressive movements, and chronicled detailed informationon visual and vocal components of 1215 individual behaviors. Two thirds of the events recorded were silent, supportingthe idea 1)Departmentof Psychology,Universityof SouthFloridaSt. Petersburg,1400 SouthSeventh Avenue, St. Petersburg,FL 33701, USA. e-mail:[email protected] 2) This work is based on my dissertationresearchin Peter Marler'slaboratory.I would like to give my heartfeltthanks to Dr. Marlerfor his encouragement,support,and advice, as well as for his comments on several draftsof the manuscript.I am gratefulto Marc Hauser for encouraging me to work on rhesus monkeys, and to William Mason for helping me to understandrhesus behavior duringpilot observationsat the CaliforniaRegional Primate Research Center.This project was developed duringdiscussions with Drs. Marler,Mason, Hauser,ChristopherEvans and Joseph Macedonia.I would like to thankCharlesSnowdon, CorreighGreene, Jill Soha, Katya Partan,and two anonymousreviewersfor comments on the currentmanuscript;Dr. Mason, JohnEndler,and Arlene Alvaradofor commentson prior versions; and J.A.R.A.M. van Hooff for a suggestion on terminology.Neal Willits and Tim Allis provided key statistical advice; Renee Allen and Marc Fourriergave indispensable assistance in the field, and Jeannine Logan, Rebecca Wylie, and Virginia Price in the laboratory;Bill Patrickand Todd Hughes helped produceFigure 4. Finally, I would like to thankJohn Berard,Matt Kessler, the University of PuertoRico, and the CaribbeanPrimate ResearchCenterfor providingaccess to the field site and the genealogical data, and Edgar Davila for introducingme to the monkeys.This studywas fundedby grantsfromthe National Science Foundation,the L.S.B. LeakeyFoundation,the AnimalBehaviorSociety, and Sigma xi. ( KoninklijkeBrill NV, Leiden, 2002 Behaviour139, 993-1027 Also availableonline - 994 PARTAN that visual behaviors are primary for short distance communication in these macaques. Clustersof expressive componentsdetectedby PrincipalComponentAnalysis and Multiple CorrespondenceAnalyses correspondedto threatening,submissive,and affiliativebehaviors describedpreviously,providingquantitativesupportboth for these previousdescriptionsand for the suggestion thatthese threepoles of behaviorare importantin daily social interaction. Silent expressions involved a greater variety of mouth positions than did vocalizations, which were produced with stereotypedmouth shapes. Other components of the face, not involvedwith articulation,were nonethelessassociatedwith particularvocalizations:specific associationswere found among barks,ears retracted,and head loweredon the one hand,and pant-threats,ears forward, and head raised on the other.Screams and squeaks were highly stereotyped,combinedwith prototypicalgrimace mouthpositions, crouchingand retreating. Girney vocalizations were accompanied by lipsmacking. Grunts were unaccompaniedby otherexpressivecomponents,evoking the suggestionthatthey may be predominantlyneutral in valence. Introduction Although communicationinvolves the use of multiple sensory channels, most researchersfocus on one sensory channel at a time, in isolation from the full repertoireof the animal. The combinationof channels, however, can have importantramificationsfor signal meaning and efficacy (Marler, 1965; Partan& Marler, 1999; Rowe, 1999). Human facial expression and visual articulatorymovements,for example,play a role in speech perception (McGurk& MacDonald, 1976; Massaro, 1998). In birds, odor cues from prey can interactwith visual stimuli of particularcolors to produce food aversionsthat do not occur withoutthe odor (Rowe & Guilford, 1996), and odor plays an importantrole in combinationwith other cues during sexual behaviorin baboons (Goldfoot, 1982). Signal componentsin multichannel displays can be redundant(e.g. Conner,1987), or each componentmay play an independentrole (e.g. Fusani et al., 1997). It is difficult, however, to determinethe functionalrole of each componentof multicomponentsignals, as Green (1975, p. 87) mentioned for Japanese macaques: "...their vocal behavior is inextricablytied to simultaneousolfactory, tactile, and visual signals, hence considerationsolely of evoked responses cannot disentangle the roles of the concurrentsignals available by different sensory modalities .. Before parsingthe role of each componentin communication,one needs to determinewhich particularcomponentsarecombinedsimultaneouslyinto multimodalsignals duringsignal production.Althoughresearchersstudying SIGNAL COMPOSITION 995 the receptiveside of communicationhave developed sophisticatedmethods for measuringthe perceptionof multichannelsignals (e.g. visual influences on human speech perception,Massaro, 1998), there are to date few established methodsfor quantifyingnaturalmultichannelsignalproduction. This study describes how naturalvisual signals of facial expressionand body postureare associatedwith vocalizationsof rhesusmacaques(Macaca mulatta),bridgingthe separateanalyses of visual and vocal signals of this species (e.g. primarilyvisual analysesby Hinde & Rowell, 1962; van Hooff, 1962; Maxim, 1982, 1985; Zeller, 1986, 1996;Maestripieri& Wallen,1997; and primarily vocal analyses by Rowell, 1962; Gouzoules et al., 1984; Hauser, 1991, 1996; Hauser& Marler,1993). Although some authorshave described visual and vocal signals together (e.g. Altmann, 1962; Rowell & Hinde, 1962; Lindberg, 1971; Mason, 1985; Kalin et al., 1992), none have reportedthe frequencieswith which particularvisual componentsare associated with particularvocal components.This is an importantfirst step towards an understandingof the multimodalnatureof communication.A similar approachhas been taken by Adams & Schoel (1982) in a study of stumptailmacaque(Macaca arctoides) communication:vocalizationswere analyzedseparatelybut in parallelwith the simultaneousfacial and postural componentsof behavior.My work also builds on the work of Hauseret al. (1993), who suggestedthateach class of rhesusvocalizationis accompanied by an unique articulatorygesture.Here I not only examine mouthposition, but also includefacial expressionsandbody posturesuninvolvedwith actual phonationthatnonethelesspredictablyaccompanyeach vocalization. One goal is to explorethe variabilityin facial expressivecomponentsused during communication.I constructeda catalog of expressive movements of the face and head, grouping behaviors into 'morphological'categories (classifiedby body part:eye, ear,head,etc.; cf. Reynolds, 1976). Withineach category,I definedlogical, mutuallyexclusive statesfor each body part(e.g. the ear categoryincludes earsforward,retracted,orflappedbackandforth). This categorizationscheme is more detailed than previous rhesus catalogs with regard to facial movements (see Zeller, 1986, 1996, for a detailed look at the face of other Macaca species) but may not be as detailed with regardto gross body movements.Reynolds (1976) thoroughlydocumented and cross-referencedterms used by early rhesus monkey observers,and I attempthere to use many of the terms of my predecessorsin the hope of achieving marginalconsistency and common understanding.My terms for 996 PARTAN the vocalizationsfollow most closely those used by Rowell & Hinde (1962) and Hauseret al. (1993). The behaviorof the animals was recordedaction by action in sequence. An 'action' was identifiedas a discreteunit of behavior,which may include multiple simultaneous components. This designation was based on my impressionof a behavioral'unit.'Behavior,althoughit occurs in continuous streams, can be described as being made up of sequences of more or less discrete actions. The identificationof these action units can, however, be difficult. I separatedthe behavioralstream into discrete actions based partiallyon physical changes in the body (e.g. the head moving from raised to lowered), and partially on timing considerations (if two components of behavior occurred simultaneously,such as head lowered and bark, I consideredthem to be componentsof the same action, whereasif there was a time lag between them, they were consideredto be two separateactions). Green (1975) also emphasizeddynamic transitionsin patternsas an aid to classification. Drummond(1981) provides an extensive and interestingdiscussion of how we categorize and describe behavior. He notes that the stream of behavior can be segmented at many differentlevels, each appropriateto a differenttype of analysis. A common definitionof a 'unit' is as a reliably recurringphenomenon(Drummond,1981, p. 13-14). Ethologists may have to use intuition to determinewhere the breaks in behavior occur, but our intuitionis based on tractabledata such as regularityin patternsor bouts of behavior(Drummond,1981; see also Marler& Hamilton,1966, Chapter20). The ultimatetest of our intuitionis to ask the animalsthemselves how they categorizetheirown signals (see Marler,1982). Describing real, complex behavior,involving multiple simultaneoussignal components,is a challenge. To simplify, I included only the visual and acoustic channels. In addition,I typically analyzed only two classes of behavior simultaneously:vocalizations,along with one class of visual behaviors (e.g. eye gaze, or mouth position). My rationalewas to documentthe bimodal visual and vocal behaviorsof these animals. Detailed quantitative analyses of the compositions of facial and vocal expressions are presented in an effort to describehow visual and auditorysignal componentsare combined in this populationof rhesus monkeys. For furthersuggestions of how to analyzecomplex multicomponentbehaviors,see van Hooff (1982), Bradbury& Vehrencamp(1998), and Deputte(2000). 997 SIGNALCOMPOSITION TABLE 1. Demographicsof main studyanimals Study Subjects GroupY GroupV Adults (6 yrs +) Females Males1) 19 12 13 8 Subadults(4-5 yrs) Females Males 8 8 4 3 Juveniles(1-3 yrs) Females Males 15 14 6 6 Infants(<1 yr) Females Males 11 9 6 3 96 49 Total 1)The numberof adult males was approximatesince males occasionally transferredamong groupsduringthe study. Methods Subjectsand studysite The subjectswere free-rangingmale and female rhesusmacaques,Macaca mulatta,on Cayo Santiago, a 15.2-hectareisland off the southeastcoast of Puerto Rico (Rawlins & Kessler, 1986). The monkeys ranged freely and formed social groups, were provisionedwith food, and had no naturalpredators.All of the animalswere trappedas yearlings and given tattoos and ear notches for identification.At the time of the study, there were six social groups on the island, rangingin size from approximately49 to 354 animalsper group.The study was conductedon two groups: group 'Y' and group 'V' (see Table 1 for age demographicsof the groups). Twenty-twoadult females and 13 adult males were the primaryfocal subjects. Occasionalsamples were takenfrom animalsin neighboringgroups. Videotaperecordingmethods All datawere collected on audio and videotapefor subsequentanalysis. Adult animalswere filmed during 30-minute focal-animalfollows. The primaryfocal animals had at least four samples each. Two simultaneousvideo recordingsof the animals were taken: one camera filmed a close-up shot of the facial expressions of the focal animal, and the other took a wide-angle view that also encompassed the surroundinganimals. I collected 270 hours of video tape footage on VHS & SVHS tape, summing from both cameras, along with supplementaryaudio cassette tape recordings,during March and September-Decemberof 1994. Close-up videos used a PanasonicAG460 movie cameraand wide-anglevideos used a PanasonicAG455. Both cameraswere mountedon Bogen 3179 tripods.A SennheiserMKH 816 shotgunmicrophonewith windcoverwas strappedto the close-up video cameraand fed into the externalaudio inputfor betteraudio recordingson the videotape.I filmed only when the focal animal was interactingwith other monkeys at relatively close range, so signals 998 PARTAN propagatedby solitary monkeys, such as branch-shakingand food-calling (coos), were not common. Signals concernedwith mating were infrequentbecause I collected data primarily duringthe nonbreedingseason. (Publishedrecords,such as Rawlins& Kessler,1985, indicate that the Cayo Santiago breeding season was from approximatelyJuly to November in the 1970's to early 80's. However,the season has shiftedforwardslowly over the years, currently beginningin May and mostly over by September[M. Gerald,pers. comm., December2001].) Data collectionfrom videotape Of the 270 hours recorded on both cameras, I watched 230 hours of tape (approximately 130 hours of real time). In many instances, the two videos had to be watched one at a time because they displayed different views of the same scene, and included differentplayers. While watching the tapes, I sampled for social interactionsusing the 'behavior-sampling' method(Martin& Bateson, 1993, p. 87). Specifically,I sampledfor any interactioninvolving vocal or visual communicationbetween two or more individuals.This broad samplingrule includedall social interactions,with the exceptionof groominginteractionsalreadyunderway at the time the camerastartedto film, since they usuallydid not includevocal or visual signals. If two animalsbeing filmedbegan to groom duringthe sampleperiod,however,this behavior was included. Once an interactionwas located and data collection began, data were collected continuously until either (a) the individualsceased interacting,or (b) one or both ran off screen and out of view of both cameras. Since I was recordingin the field, with trees, bushes, and interveningtopography,many interactionsin the databaseended prematurely(i.e. the animals went out of sight before they were done interacting).However, this is not problematicfor the purposes of this study, since I am interestedhere in the structureof visual and auditory signals, ratherthanthe outcomes of the interactions. I described 402 social interactions(both agonistic and affiliative), watching the interactions in detail, often proceeding frame by frame to identify the components used in the signals. The individualactions that each animal performed(which often included multiple simultaneouscomponents,as defined in the introduction)were logged sequentially,totaling to 1215 actions. The median numberof actions per interactionwas 2 (the range was 1-26; the mean was 3). This means that most interactionswere short, with one animal emitting a display and the other a response. Focal animals were all adults, but younger animals were included when they interactedwith focal subjects.Although the primaryfocal animals each had 4-10 focal follows, the total numberof actions performedby each focal animalranged from 5 to 65 (dataon the numberof times each individualcontributedavailableon requestto the author). The databasecontainedone entryfor each action, with the following fields:the time of the event (video frame);the identities of the actor and recipient(s)involved, including their sex and age; anda set of eight componentsof the signaler'sbehavior.These included:vocalization type, mouth position, eye gaze, ear position, head position, tail position, body posture, and overall movement.If a particularbody partwas out of view, as could happenif the animal's face was on the camerascreen but its tail was not, I left blank the correspondingcell in the database.I did not include observationswith blank cells in my analyses. For each of the eight componentsof the signal, I compiled a list of possible expressive positions or states (Table 2). This list provides a detailed repertoireof rhesus monkey communicativebehavior,focused largelyon movementsof the face and head. I avoidedusing SIGNALCOMPOSITION TABLE 999 2. Repertoire of communicativebehaviors of rhesus macaques: expressivecomponentsof theface, voice and body Vocal behavior: No vocalization Bark Pant-threat Scream Squeak Bark/Scream Grunt Girney Coo Gecker Copulationcall Othervocalization Mouth position: Neutral Open mouth Grimace Tense mouth Lipsmack(LS) LS with tongue protrusion Chin-up1) Chin-up& LS Teeth-chatter Toothchomp Puckeredlips Yawn Bite Mouthmatches vocalization Eye gaze2): Neutral silence loud, voiced, harsh(broadband)sound softer,more breathyharshsoundusually emittedin a rapid bout of threeunits loud sound, can be harshor tonal (includesnoisy, archedand tonal screams) shortpunctuatedharsh(pulsed) scream mixed call that startswith barkand ends with scream quiet, harshsoundusually not repeated frequencymodulated,quiet sound, may be narrow-or broadband tonal (narrow-band)sound staccatoharshsounds,usually from infants shortharshscream-likecalls given by some males during copulation vocalizationwhich cannotbe put in any of the above classes mouthis closed, lips closed, loose lowerjaw droppedso lips form 'o' shape,upperteeth covered lips retractedhorizontallyto expose teeth,jaws can be togetheror apart mouthis closed, lip cornersdrawnback to form straightline lips moved repeatedlytogetherand apart;may be audible same as above but with tongue rhythmicallymoving in and out of mouth mouthis closed, lips togetherand often pursed,chin is angledup same as above, but with lips makingsmall, rapidmovements mouthrapidlyopens & closes; teeth exposed;may be audible jaw opens and closes but lips remainclosed; may be audible lips drawnforwardtogether,cheeks furrowed mouthopens widely in stereotypedgaping movement mouthopened to bite othermonkey mouthopens only enough to emit a sound, andonly for the durationof the sound,then closes eyes relaxed,may be half shut, not looking in any particular direction PARTAN 1000 TABLE Stare Look at Look away Look between Look around Ear position: Neutral 2. (Continued) direct,prolonged,unwaveringlook at specific individual look in directionof specific individual look in directionobviously other than thatof specific individual look rapidlybetween two or more otherindividuals look rapidlyin severaldirections,often at distant individual(s)/group(s) Earsback Ear flap ears relaxed,in mediumposition (about45 degrees from head) ears pointingperpendicularlyout from head ears retractedtightly to scalp ears move quickly backwardsand forwardsmore thanonce Eyebrow position: Neutral Eyebrowsraised Eyebrowslowered Eyebrowflash eyebrowsrelaxed eyebrowsraised and remainup eyebrows lowered and remaindown eyebrowsmove quickly up and down Earsforward Head position3): Neutral Head raised Head lowered Headjerk Head bob Tail position: Neutral Tail vertical Tail up looped Tail up crooked Tail 45 degrees Tail parallel Tail wave head relaxed head held up in high position head held in low position, neck usually angled forward head moved quickly and abruptlyup and down head moved slowly and smoothly up and down, often repeated tail held in relaxed,low position tail held up perpendicularto body, straightfrom base to tip tail held straightup from base, but tip is curledin tight loop tail held straightup from base, but tip is bent over tail held up at 45 degrees from base, tip often slightly bent tail held straightout parallelto body tail held low, waving Body positions & movements: Neutral posturerelaxed Crouch posturewith belly low, armsand legs bent suddenand quick movementof upperor entirebody towards Lunge recipient mountothermonkey in typical posture Mount lean whole body away from anothermonkey Lean away quick hit to ground,usually one handonly Slap ground 1001 SIGNAL COMPOSITION TABLE 2. (Continued) Groom stereotypedmovementof approachinganotherindividual very closely, then suddenlybreakingapproachby bendingarms, turningquickly aroundand moving away (headusually swings aroundlast, so that actoris looking at recipientuntil last moment) bounce up and down on branchor otherobject, holding on with feet so thatthe substratemoves as well body suddenlybecomes still use handto grabhold of othermonkey use handlarmto hit at othermonkey use handto gently touch othermonkey hairis raisedon body fast-pacedagonistic interactioninvolving chasing, tumbling, biting, barking,screamingall at once presentown body for mountingusually by moving tail aside and presentingrearend to othermonkey presentown body for groomingusually by moving neck or side or back towardsothermonkey,looking away,and becoming still handssystematicallycombing throughothermonkey'shair Overall movement: No change Approach Retreat Chase Go in severaldirections Pass by Leave Follow remainstationary move towardsothermonkey move rapidlyaway from othermonkey run afterothermonkey move back and forth pass anotherwithoutstopping leave anotherwith whom it had been associating(not rapidly) walk close behindanothermonkey,same direction Dip-turn Branchshake Freeze Rough grab Swat Gentle touch Piloerection Fight Present-sexual Present-for-grooming 1)Identicalto 'muzzle-up'(Partan,1998). 2) Eye gaze directionwas determinedby head orientationas well as eye gaze. 3) Includeshead movementsin the vertical,ratherthanhorizontal,domain. names with functional connotationsto describe the elements of the expressions, with the exception of pant-threat.I kept the 'threat'as a partof this name to be consistent with the previousliterature(e.g. Rowell, 1962; Rowell & Hinde, 1962; Hauseret al., 1993). All expressive components were defined as a departurefrom a neutral state (based on Sade, 1973). A neutral expression was defined as the countenance of a relaxed monkey (Fig. la). Neutralpositions of each body partwere recordedalong with expressivepositions (see Table 2). Expressive positions are mutually exclusive within categories. For example, when an individual'shead is raised, it cannot simultaneouslybe lowered; or if an animal is barking,it cannotbe also cooing. The only categoryfor which internalmutualexclusivity does not hold is the 'body' category,becauseit is possible, for instance,for an animaltofreeze and crouchat the same time. In these cases I enteredthe most pronouncedbehavior,putting 1002 (t) PARTAN ~~~~~~(b) Fig. I. (a) Sub-adultrhesus male (X85) with neutralfacial expression. Photo by author. (b) Adult female (E76) with silent stare and open-mouth. Digitized from videotape using Adobe Premieresoftwareand Panasonicediting stations. the secondary behavior in an 'other' category. I chose to lump all body postures into this one categorybecause I wanted to focus specifically on facial expressions and vocalizations. Body posturewas of secondaryinteresthere, althoughcertainlyof great importancein social behavior. Eye gaze direction is difficult to assess on videotape. I have followed Altmann (1962) in his distinction of stare from look at, and I have included three other active states of the eyes (see Table 2): look betweeni,look arounid(perhapssimilarto Altmann'sunit #35, 'looks apprehensively'),and look away (similarto Altmann'sunit #34, 'avoids staringat'). In each of the latterthree cases the eyes move quickly between, among, or away from social targets were relaxed eyes, often half shut in the environment,respectively.In contrast,eyes nieutrail or gazing off into space, not moving quickly. In cases where I could not assess eye direction from the videotape,I left the correspondingdatacell blank. Head position was troublesometo categorizebecause the head often moves in conjunction with other behavior, such as yawning (Deputte, 1994). Movements of the head are also incorporatedinto gaze behavior. I decided to prioritize mouth and eye behavior, since I was primarily interested in facial expression. Therefore yawn was categorized as a mouth behavior,and all eye gaze directions involving both head and eye movements were categorized as eye behaviors (e.g. look arounidand look between). Head movements that occurredin the vertical domain (raised, lowered, bobbed,jerked) were included in the head position category. My use of the term head bob refers to a slow, rhythmic up and down motion, whereas head jerk is a sudden sharp movement, not repeated. I follow Hinde & Rowell (1962) and Reynolds (1976) in their use of the term 'jerk' for the latter behavior (departingfrom Altmann, 1962, and Drickamer,1975, in their use of the term 'bob' to refer to quick head movementsduringthreat). Vocalizationswere classified by ear from the audio trackof the videotape,aftercategories were established via spectrographic analysis and verified by comparison to published SIGNALCOMPOSITION 1003 spectrograms(Rowell & Hinde, 1962; Hauser & Marler, 1993) and discussions with other researchers(P. Marler,W. Mason, & M. Hauser,pers. comm.). Although rhesus screams have been subdividedinto five categories by Gouzoules et al. (e.g. 1984), I could reliably distinguish only two types by ear. One type, which I labeled scream, included 'noisy', 'tonal', and 'arched' screams (Gouzoules et al., 1984); the second type, which I called squeak,includedscream-likevocalizationsthatwere very shortin duration('pulsed' screams, Gouzouleset al., 1984). If a vocalizationhad severalsyllables (as was commonwith screams andpant-threats),I enteredit only once into the database,to avoid inflatingthe numberof vocalizations. Pilot observationsof mouth positions during vocal behaviorindicatedthat some vocalizations were producedwith a utilitarianor purely articulatoryuse of the mouth:the mouth opened only to the degree and durationnecessaryfor sound emission. Othervocalizations, however,were accompaniedby elaboratemouthmovementsthatexceeded eitherthe duration of the call or the degree of opening necessaryto producethe sound. I distinguishedbetween these possibilitiesby includinga categorycalled mouthmatchesvocalization(Table2), along with the expressive mouth positions. This allowed me to set aside articulatorymovements for particularanalyses (noted below). Note that all vocalizationsare 'simultaneouslymultimodal' in that each occurs with a particularmouthposition, providingboth visual and vocal stimuli, regardlessof whetherthe mouth is open as long as or longer than the call. A finergrained temporalanalysis might consider the latter cases to include unimodal signaling of mouthopening, followed by multimodalsignalingduringthe call, followed againby the unimodal mouthposition afterthe call has ended. However,here the behavioralunit includedthe vocalizationin the context of the visual signals thatencompassedit; any actionthatincluded a vocalizationwas considered 'multimodal'. Data analysis All dataanalysiswas done using SAS software.Any entries(observations)with missing data for the behaviorbeing analyzedwere omittedpriorto analysis. This reducedthe sample size differentiallyfor the differentanalyses,dependingon which behaviorswere being assessed. I carriedout three main analyses. FirstI examinedthe entiredata set to determinewhich components co-occurred,using Principal ComponentAnalysis (PCA) to examine overall relationships among the variables. Since PCA requires numeric data, I transformedthe nominal categories into a series of indicatorvariables which read '1' or '0' depending on whetherthe particularbehaviorwas presentor absent(this methodis also used anddescribed by Deputte,2000). I used all eight behavioralclasses (vocalizations,eye, ear, head position, etc.). When I transformedthe data, there were a total of 57 columns, because the 8 classes included 57 unique behavioralcomponentsor elements that were included in the analysis. (All behavioralcomponentsin Table 2 were includedin the PCA except articulatorymouth matchesvocalizationmovementsand some otherrarecomponents.)The raw datamatrixfor the PCA analysis consisted of the 57 columns correspondingto each behavioralcomponent, and 1215 rows correspondingto each event. The analysis thereforeproduceda correlation matrixwith 57 columns and 57 rows, correlatingeach of the behavioralelements with each other element. This matrix was then analyzed to produce 57 principalcomponents,the top six of which had eigenvalueslarge enough (> 1.9) to warrantdiscussion. The second analysis was a comparisonof the silent and vocal behaviorsof the animals, basedon the frequenciesof occurrenceof each expressivecomponentduringsilent andvocal behavioras a whole. 1004 PARTAN In the thirdset of analyses I examinedin detail the structureof the bimodal,visual/vocal expressions.I conductedMultipleCorrespondenceAnalyses (MCA's)to explorethe distribution of componentstwo or threeclasses of behaviorat a time, andpresentedthe resultsgraphically. The frequencydata were taken from columns in the main database,with behavioral componentsacrossthe columns, andone composite action per row. A correspondenceanalysis, like a PrincipalComponentAnalysis, detects trends in the data by establishing which componentsare associated (see van der Heijden et al., 1990, for detailed explanation).The analysis then defines 'dimensions' to describe the strongestassociations.Like the PCA, the correspondenceanalysis has many dimensions, but only the two most importantones are plotted in the graphs. Strong associations between variablesare shown in two ways on the MCA graphs:by distance from the origin, and by trajectory.If two behavioralcomponents plot out on the same trajectory,they are closely associated,andthe fartherthey areaway from the origin, the strongerthe association. (Diagonal lines drawnon the graphsmake it easier to see which variablesare associated.) Note that for the correspondenceanalysis of mouth positions and vocalizations,I excluded the articulatorymouthmatchesvocalizationcategory. Finally, I documentedwhich visual componentswere associated with each vocalization, using frequencymatricesand histograms. For all analyses, behavior was pooled across individuals. Although I am aware of the potential dangers of pooling data for parametrictests (Machlis et al., 1985; Leger & Didrichsons, 1994), my data are nominal frequencies that cannot be averaged. I cannot, for example, find the mean of 10 open mouth threats and 5 fear grimaces to come up with an 'average expression' for a given individual who produced these 15 expressions. FurthermoreI collected data on myriad components of expression, with the individuals involvedin manydifferentpossible dyadic combinations,makingthe concept of an 'average' even less appropriate.Instead,the focus of analysis is on the frequencieswith which multiple expressivecomponentsare combined,across all cases of expression. Results Overallrelationshipsamong the behaviors Principal Component Analysis yielded six top components (with largest eigenvalues; Table 3). The clusters revealed by the PCA correspond to behavioral suites that have been reportedpreviously in a more anecdotal fashion. Component1 The most striking association in the data was among open mouth, stare, earsforward, and head lowered (Fig. lb depicts the open mouthand stare). This suite of behaviorscorrespondsto descriptionsof silent threatfrom the literature.Grimacedid not occur with this cluster. 1005 SIGNALCOMPOSITION TABLE Component 3. Principal ComponentAnalysis FactorWeight Variable Coefficient Interpretation PC 1 3.67 stare head low open mouth ears forward [grimace] 0.38 0.31 0.28 0.28 -0.23 PC 2 2.75 bark look-around scream move around 0.31 0.27 0.23 0.23 Vocalfight PC 3 2.59 tail up approach 0.34 0.31 Approach PC 4 2.32 girney lipsmack ears back tail wave [grimace] 0.28 0.24 0.24 0.23 -0.22 Affiliation PC 5 2.08 look-at lunge approach 0.38 0.35 0.32 PC 6 1.94 look-at grimace retreat ears back chin-up [bark] 0.35 0.31 0.24 0.21 0.21 -0.22 Silent threat Aggressive approach Submission /Male solicitation Top six components.All variableswith Icoefficientl>0.2 are listed for each component.A negativecoefficientindicatesthatthe variablewas negativelyassociated. Component 2 The second principalcomponentinvolvedbark,scream,lookingaround,and movingin several directions.This is a typical suite of behaviorsinvolvedin agonistic interactions. Component 3 Steady, walking approach with tail raised. Typical behavior of dominant males (Hinde, 1966; Lindburg, 1971). 1006 PARTAN TABLE4. Comparisonof visual componentsassociated withsilent and vocal expressions SILENT N VOCAL Row % Column % N Row % Column % Total Mouth: Neutral Open Mouth Grimace Lipsmack Chin-up Yawn Bite ToothChomp TongueProtrusion MatchesSound ColumnTotal 249 108 88 69 16 9 9 8 8 n/a 564 98.03 63.53 58.28 82.14 100.00 100.00 100.00 100.00 88.89 0.00 69.46 44.15 19.15 15.60 12.23 2.84 1.60 1.60 1.42 1.42 0.00 100.00 5 62 63 15 0 0 0 0 1 102 248 1.97 36.47 41.72 17.86 0.00 0.00 0.00 0.00 11.11 100.00 30.54 2.02 25.00 25.40 6.05 0.00 0.00 0.00 0.00 0.40 41.13 100.00 254 170 151 84 16 9 9 8 9 102 812 Eye: Look-at Stare Neutral Look-between Look Away Look-around ColumnTotal 252 161 99 64 40 33 649 73.68 73.18 89.19 65.98 83.33 32.35 70.54 38.83 24.81 15.25 9.86 6.16 5.08 100.00 90 59 12 33 8 69 271 26.32 26.82 10.81 34.02 16.67 67.65 29.46 33.21 21.77 4.43 12.18 2.95 25.46 100.00 342 220 111 97 48 102 920 Ear: Neutral Back Forward Flapped ColumnTotal 458 100 55 9 622 72.01 66.67 75.34 39.13 70.52 73.63 16.08 8.84 1.45 100.00 178 50 18 14 260 27.99 33.33 24.66 60.87 29.48 68.46 19.23 6.92 5.38 100.00 636 150 73 23 882 Head: Neutral Lowered Jerked Raised Bobbed ColumnTotal 575 82 20 13 12 702 69.70 80.39 80.00 68.42 85.71 71.27 81.91 11.68 2.85 1.85 1.71 100.00 250 20 5 6 2 283 30.30 19.61 20.00 31.58 14.29 28.73 88.34 7.07 1.77 2.12 0.71 100.00 825 102 25 19 14 985 Body: Neutral Lunge Lean Away Crouch 515 45 30 22 69.69 59.21 90.91 50.00 71.63 6.26 4.17 3.06 224 31 3 22 30.31 40.79 9.09 50.00 74.92 10.37 1.00 7.36 739 76 33 44 1007 SIGNALCOMPOSITION TABLE4. (Continued) SILENT VOCAL N Row % Column % N Row % Column % Total Freeze Slap Ground Gentle Touch Rough Grab Present Swat Shift Position Dip-turn Fight Piloerection ColumnTotal 22 17 16 15 11 10 9 5 2 0 719 91.67 89.47 94.12 100.00 100.00 76.92 90.00 100.00 25.00 0.00 70.63 3.06 2.36 2.23 2.09 1.53 1.39 1.25 0.70 0.28 0.00 100.00 2 2 1 0 0 3 1 0 6 4 299 8.33 10.53 5.88 0.00 0.00 23.08 10.00 0.00 75.00 100.00 29.37 0.67 0.67 0.33 0.00 0.00 1.00 0.33 0.00 2.01 1.34 100.00 24 19 17 15 11 13 10 5 8 4 1018 Movement: None Approach Retreat Pass By Chase Back & Forth Follow Leave ColumnTotal 411 172 98 25 22 15 11 5 759 71.98 78.54 63.64 100.00 53.66 37.50 91.67 100.00 71.13 54.15 22.66 12.91 3.29 2.90 1.98 1.45 0.66 100.00 160 47 56 0 19 25 1 0 308 28.02 21.46 36.36 0.00 46.34 62.50 8.33 0.00 28.87 51.95 15.26 18.18 0.00 6.17 8.12 0.32 0.00 100.00 571 219 154 25 41 40 12 5 1067 Therewere a total of 760 silent behaviorsand 356 vocal ones, but the totals for each section above are smaller,owing to dataunavailablebecause a particularbody partcould not be seen in certaincases. Component4 Girney,lipsmack,ears back, tail wave. This is typical affiliativebehavior. Grimacedoes not occur with these behavioralelements. Component5 Lunge,chase, look at. This is typical of an aggressiveapproach. Component6 Retreat, grimace, look at, ears back, chin-up. The first two behaviorsare typical of submission. The last three are components of a solicitation behaviordescribedby Altmann(1962, p. 378), in which a male approacheda female with a stereotypedposture,head tilted back, chin up, and lips pursed 1008 PARTAN ii . i _ .. I i,,~~~~~~~~~~~~ ....... . _ .... ...... ............ .. ~~~~~~~~~.... _ ..... | Fig 2. U . . .......,,.. _.,..... .....,,,t, , .: I'l"......._ ~~~~~~~~~~~d ....F;},... ,~~~~~~~~~~~~~~~~~~ .. ...... Images from videotape of vocalizing animals. Spectrograms were collected from videotape at the same frame as the picture, using a Kay Digital Sona-Graphmodel 7800. Horizontalbars mark I-kHz intervals(1-8 kHz). (a) Adult male barking,with open mouth and ears back. (b) Adult female (845) giving a pant-threatvocalization.(c) Sub-adultfemale (X70) giving a broadband('noisy') scream. (d) Adult male (C78) grunting. (e) Sub-adult male girneying and waving his tail. Althoughthis girney contains primarilybroadbandcomponents, girneys can also include narrow-bandsounds (e.g. see spectrogramin Kalin et al., 1992). Figure 2a reprinted goes multimodal- with permission from: Partan, S. & Marler, P.: Communication Science 283(5406), p. 1272-1273; copyright 1999 American Association for the Advancementof Science. and sometimes rapidlysmacking.He typically approachedvery close to the female, almost touched his face to hers, and then immediately tured away with a stereotyped 'dip' (arms bend quickly down), and walked away. Only males were seen performingthe chin-up and dip-turn; when a recipientwas identified, it was female (Partan, unpublished data). Comparison of silent and vocal expressions Four of the top six Principal Components were silent, and two were vocal. To compare silent and vocal behavior, I explored which visual components were SIGNAL COMPOSITION 1009 associated with vocalizations. The presence or absence of a vocalization could be reliablydeterminedfor 1116 out of the 1215 total events recorded. Of these 1116 events, 68% (760) were silent and 32% (356) included vocalizations.Silent and vocal expressionswere differentiallyaccompanied by the various visual signal components(Table4). The 'Vocal' column in Table4 lumps the six majorvocalizations,each of which occurred20 times or more: barks, pant-threats, screams, squeaks, grunts, and girneys (see Fig. 2 for spectrograms).Omittedfrom the table are vocalizationsthat were recordedfewer thanten times (bark-screammixes, coos, geckers,copulation calls, harmonicarches, and unclassifiablecalls). The 'row %' column in Table 4 compares the silent to vocal behavior. Silent expressionsinvolvedmore visual componentsthanvocal expressions. This can be demonstratedby a quick scan over the 'row %' columns to see the disparityin the numberof behaviors at 100% for the silent group comparedto the vocal. The mouthwas most often in a neutralposition duringsilence, but many other mouth positions were used. All possible eye positions were recorded duringsilent expressions,the most common being looking at the recipient, followed by staring and neutral eye position. All ear and head positions were also recorded during silent expressions, with the neutral position predominantin both cases. Head bobs were predominantlysilent. Many body and arm postures occurredduring silent expressions, although most of the time the body was neutral.Presents,grabs, anddip-turnswere always silent; lean away, touch,freeze, and slap ground usually so. During most silent expressionsthe animalwas stationary.Pass by and lunge were always silent;follows usually were as well. Expressions including vocalizations incorporatedfewer visual components than did silent expressions overall. Mouth positions during the six majorvocalizationsincludedopen mouths,grimaces, lipsmacks,and mouth matchingthe vocalization.Therewere no vocalizationsduringyawns, bites, tooth chomps,or chin-upmovements. Lookingaroundoccurredmoreduringvocalizationsthanduringsilent behavior;lookingaway, however,predominantlyoccurredduringsilent behavior.Ear-flapping,lunging,crouching,fighting,andpiloerectionoccurredproportionatelymore often duringvocal thansilent behaviors,as did retreating, chasing, and movingback andforth. PARTAN 1010 Structureof multimodalsignals: Visualbehaviorsassociated with each vocalization I used Multiple CorrespondenceAnalysis to examine in greaterdetail how behaviorswere associatedwhen consideringnot the entire data set, as in the above PCA analysis, butjust two of the eight majorcategoriesat a time (e.g. vocalization and mouth position). Associations among vocalizations and mouthpositions are shown in Fig. 3, using only 'expressive'mouthpositions (all positions except mouthmatchesvocalization).The componentssplit into three groups, each of which contains behavioralelements devoted to one of three functions describedpreviously in the literature:threat,submission, and affiliation (Altmann, 1962; Hinde & Rowell, 1962; Rowell & Hinde, 1962; Lindburg,1971). Threatincludedbark,pant-threat,bark-scream,and * ,: .....' ! 4 ,J:A . 146, * : .,:.; , .. ..; . .:.fs..e.j. .. t:..'* ".'. Fig. 3. Multiple correspondence analysis of vocalizations (circles) and mouth positions (triangles). Associations among variables are shown by common trajectory from the origin. Aggressive components clustered on upper left; submissive ones on right; neutral and friendly of to facilitate facilitate grouping thevariables. variables. lines are are included included to ofthe grouping ones on on lower lower left. left. Diagonal ones Diagonal lines SIGNALCOMPOSITION * 1011 1 3-U 11..~~N" 1: Fig. 4. ..._^K 1. ... Multiple correspondenceanalysis of vocalizations (circles), mouth positions (triangles), and eye gaze (squares). open mouth; submission included scream, squeak, and grimace; affiliation includedno vocalization,girney,grunt, coo, lipsmack,and chin-up. To test whether these three clusters still appearedwhen combined with a visual component uninvolvedwith phonation,I added eye gaze direction to the vocal and mouth data and ran anotherMCA (Fig. 4). The same three clustersoccurred.Stare fell into the aggressivecluster;look aroundand look betweenfell with submission;look at and neutraleye with affiliation. Along with the above associations predictedfrom the literature,the next MCA analyses revealed several new and unexpected associations. These depict a difference in visual accompaniments of the two vocalizations considered to be aggressive, bark and pant-threat. Associations among vocalizationsand ear position are shown in Fig. 5a. Threateningbehaviors areon the right,andnonthreatening(affiliativeandsubmissive)behaviorsare on the left. The nonthreateningvocalizations were loosely associated with _,, . . : . , . .. _ . i . 1012 PARTAN ...' . i: I _-'' s' -1' ''' ",;-'*'' . '*.';:*/.' "ji '-' .:': "*' 'y v l:****:' , .. v * * .' '-: * -,w .'' **r , *.+i9 i*'<'.' } *::'*. -* ;*' -.w !; :y_ ^ .:' . ****:* ' . -: . *t i . e! ! |l , !* '*'*^ '^* *' :" 'j .EX ^ j i ^ ^*i l . '"' * .-- C» AN' * * .:.*'_'; . ,,i ': i. ^ *''i, *:,n:, ' f: ' '-' * ' .., ,- l . .','' .'.e · !".' "¢ i-I, . . .. .. t.;. A ..t ' '- ,' ...... . . : : ^ .^ .'*.l ^ -4, i ;- .--.'.', .'s1V; i: ' '.;*':; _0 - F- - .. (a) . · ;-- . '.. 1 ---;·~ · .. .. .. . . -~;~r . CL- .rcu·* 1.L-~~;-i,l;i:ii~P I~Li ·~+ ........ .' -* "t ii; ji:^'i > ' ' Fig. 5. Multiple correspondenceanalysis of (a) vocalizations (circles) and ear positions (triangles), and (b) vocalizations (circles) and head positions (triangles). In both graphs, aggressive behaviors fell to the right of the origin, and submissive and affiliative ones fell primarilyto the left. neutral ear position. The interesting finding is that the threat vocalization bark was associated with ears back, while pant-threat was with ears forward. Figure 5b shows the associationsamong vocalizationsand head position. Affiliation and submission are on the left, weakly associated with neutral head position. On the right, pant-threat vocalizations were associated with head raised or forward, whereas barks were with head lowered. , . ._ F-.s : 1013 SIGNAL COMPOSITION .. -0-00- -.& . . . - . - - .~.· .- . ^.^ *.,... : .,., ..-" ': :..., .:.'.:. ::'- e . . : ,." '. :. "' . , . -' .. . ... .... . . 7 . . .. . .. 'i '".: l[ :,~.~.'...-. ..... ·,. ...::. .*., .- ,:.[ :~. -~ .. .. -1 .711.M. 1U : , -:...:..-: ,-'...:. ::..: ''" ;...... :¥ ~.. . ,, ,:, ',',, · . ,. -- uy - t - : : - il T,. ,: '...4.. . .'"~: ...' .. ,',.:.:..:''.. ."'".... .S,i:-:t ; * ·WqL' :*. ..'-'^; . .... · . .. . .:. Ti- i· -i -I, ;.. IT; · - · ~ , i · -. ·· .-I- .. · I -· .. ·- VE ... - I i- ·.l :i-· 7 · : r~~~~~~~~~~ .' 'jit l Ak': --I...f.L',' '..:':.'.. '.. :".. (b.) Fig. 5. (Continued). I systematicallydocumentedthe visual componentsthat occurredsimultaneouslywith each of the six majortypes of vocal behaviors(Fig. 6). These are presentedbelow, organizedaccordingto the visual components. Mouthposition Mouth position was tightly linked to vocalization type (Fig. 6a). Barks and pant-threats were accompanied by rounded open mouths, screams and squeaks were accompanied by grimaces, and girneys (and, to a lesser extent, grunts) were accompaniedby lipsmacks.Each vocalizationalso had a significantproportionof mouth shapes that simply matched the sound, 1014 PARTAN synchronizedin time, opening and closing without any furtherexpressive posture. For grunts, almost 80% involved no mouth expression other than opening slightly for the grunt. The lowest matching scores were found among screams and, especially, squeaks, which often occurredin the midst of prolongedsilent grimaces. These vocalizationswere still simultaneously multimodal:the squeaksco-occurredin time with grimaces. Eye gaze All six vocalizationswere accompaniedby substantialproportionsof looking directlyat the recipientandof lookingaround(Fig. 6b). The girneyinvolved a higher percent of looking at the recipient (67% of all girneys), and a lower proportionof lookingaround(only 17%).Barksandpant-threatswere accompaniedby a high proportionof staring (37% of barks, and 50% of pant-threats).Screams and squeaks had the highest proportionsof looking between the recipient and a thirdparty (13% of screams; 30% of squeaks) and of looking away from the recipient(5% of screams; 15%of squeaks). Earposition Ear position (Fig. 6c) was predominantly(>80%) neutral for screams, squeaks, girneys, and grunts. Barks and pant-threatsdiffered dramatically. Ears were neutral in only 51% of barks and 40% of pant-threats.During barks, the most common nonneutralposition was retracted(28%), but they could also be juttingforward(10%)orflappedbackandforth(11%).Inpantthreatsthe ears were most oftenforward (35%) but also could be retracted (15%) orflapped (10%). Head position Headposition (Fig. 6d) was always neutralfor screams,squeaks,andgrunts. During girneys, the head was predominantly(92%) neutral, but in the remainderof the cases showed a distinctive bobbing movement not seen during any other vocalizations.Barks and pant-threatswere most variable in head position.Barkswere accompanied16%of the time by head lowered, 3% by headjerked, and 2% by head raised. Pant-threatswere accompanied more often thanbarksby head raised (20%)andheadjerked (10%),andless often by head lowered (10%). SIGNAL COMPOSITION 1015 Body posture A wide array of body postures was observed (Fig. 6e). Grunts, girneys, and pant-threats were given at least 95% of the time with neutral body position. The remaining girneys were accompaniedby reaching out and gently touching the recipient. The remainingpant-threats involved body crouches. Barks, screams, and squeaks were relatively active. Barks were accompanied 16% by lunges; 4% by crouching, and 3% by piloerection. Screams were given 16% with crouching, 12% with lunging, 7% with tumbling duringfighting, 4% with swatting, and 2% with leaning away. Squeakswere emitted 7% with lunging or crouching,and 4% with leaning away,freezing, or shiftingposition. Overallmovement Movement (Fig. 6f) showed similar patternsto body posture, except for girneys. 40% of girneys were stationary;52% involved approachingthe recipient. Gruntsandpant-threatswere 90% stationary;for the remainder, the actor approachedduring grunts, and either approachedor chased the recipientduringpant-threats.Barkswere heardduringchases or approaches 13%each, duringretreats5%,and6%while movingbackandforth.Screams and squeaks were commonly accompaniedby retreats (42% of screams; 36%of squeaks).They also were heardduringapproaches(12%of screams; 16%of squeaks)andwhile moving backandforth (14%of screams;10%of squeaks). Discussion This study provides quantitativesupportfor two areas of literature.First, the data supportthe idea of three 'poles' of behaviorimportantduringsocial communication:aggression,submission,and affiliation.Deputte(2000, p. 118) calls this the "3 A's concept (affiliation, aggression, and avoidance),"and Mason (1985) also singles out these three motivationalstates. In the presentstudy this was shown most clearly in the MultipleCorrespondence Analyses of the associationsamongmouthposition, vocalization,and eye gaze (see Figs 3 & 4). Previous work identifiedbehavior involved in these threedomainsin rhesus (i.e. open mouthsand barksbeing aggressive; 1016 PARTAN (a) 100% [ eta protr. U~~~~~~~~~~~~~~~~~~~~ t 13Lipsmack *~~~~~~~~~~~~~~~~~~~~0Gr 80% 10% Ope~~~~~~~~~~~~~~~Qn 12 mouth E Matches Sound 60% 40% 20% BARK(102) (b)K PANT(20) (111)OPANT (20) 0f% - Fig. 6 Visual expressive components SCREAM(62) SQUEAK(25) GIRNEY(20) GRUNT (19) SCREAM(59)SQUEAK(27)GIRNEY(23)GRUNT(20)Neutral 12LookAway associated with each 13Look-between Stare 1ULook-around * Look-ut major 60% 40% 20% 0% BARK(113) (c) PANT(25) SCREAM(67) SQUEAK(27) GIRNEY(24) BRUNT (25) Neutral 10% 9S% * Flapped U2Forward 80% E Back 70% 60% 50% 40% 30% 20% 10% BARK(It ) PANT(20) SCREAM(09) SQUEAK(27) GIRNEY(23) GRUNT(20) Fig. 6. Visual expressive components associated with each major vocalization. (a) Mouth position; (b) eye gaze direction; (c) ear position; (d) head position; (e) body posture; (f) overall movement. The N's are based on the numberof times that the vocalization was 1017 SIGNAL COMPOSITION (d) 100% Nur 90% OBobbed B~~~~~~~~~~~~~~~~~~~ 80% U Raised 70% *Lowered 60% 50% 40% 30%{ 20% -- 10% I BARK (110) (e) ( PANT(20) SCREAM(75) SQUEAK(29) GIRNEY(24) GRUNT (20) 700 O Neutral 80% 90% Shift position Touch MuGentle El~~~~~~~~~~~~~~~~~~~~UF 80% EoSlapGround 60% 12LeanAway MSwat 13Piloerection U Fight 50% U3Crouch U Lunge 70% 40% 30% 20%- 10% 0% BARK (120) 7\ (1) PANT(20) SCREAM(86) SQUEAK(29) GIRNEY(24) GRUNT (20) 100% ~~~~~~~~~~~~~~~~~~ONo * Follow 13Chaue &FoSth HEBach 90% eO% Retreat NApproach U~~~~~~~~~~~~~~~~~~~ 70% 60% 00% BARK (120) PANT(20) SCREAM(92) SQUEAK(31) GIRNEY(25) GRUNT(20) Fig. 6. (Continued)recordedin conjunctionwith any position of the mouth, eyes, etc. The N's differ slightly among panels owing to cases lost hecause the particularvisual component could not he seen. 1018 PARTAN screams, squeaks, and grimaces submissive;and girneys and lipsmacksaffiliative; Altmann, 1962; Hinde & Rowell, 1962; van Hooff, 1962, 1967; Rowell & Hinde, 1962; Marler, 1965; Lindburg,1971; Drickamer,1975; Redican, 1975; Mason, 1985; Boccia, 1986; Kalin et al., 1992; Maestripieri, 1997; Maestripieri& Wallen, 1997; Partan,1998). Van Hooff (1973) providedquantitativeevidence for five behavioralstates importantin chimpanzees, including the three discussed here as well as play and excitement. He pointed out that the determinationof these groupingsdepends partially on the degree of lumpingand splittingof the initialbehavioralelements that make up the catalog. Adams & Schoel (1982) found six motivationalstates in male stumptailmacaques,includingthe threediscussedhere, and defense (which I includedwith submission),sexual behavior,and display. Second, these data providequantitativesupportfor the previousdescriptions of behavioralclustersin rhesusmonkeys.For example,the top componentsof the PrincipalComponentAnalysis, each of which involveda suite of behaviorsthat clusteredtogether,correspondedto behaviorsreported(usually in a nonquantitativefashion)to have been observedtogether. Comparisonof silent and vocal expressions Silent visual expressions dominated the communicationsignals used by these monkeys at the close distances at which they were observed. This agrees with Rowell's (1962) suggestion that the visual channel is primary for rhesus macaques, and supportsthe argumentthat, owing to selective pressuresimposed by the environment,terrestrialprimates such as rhesus macaques rely more heavily on vision while arborealones rely more on audition(Altmann,1967; Redican, 1975). A higherproportionof behaviors in this studywere vocal (31.9%),however,thanwas trueof Altmann's(1962, 1965, 1967) study,in which only 5.1% of behaviorsincluded vocalizations. These figures are difficult to compare, particularlybecause Altmann split some categories of visual behaviorsmore finely than I did, increasing the proportionof visual to vocal signals in his studyrelativeto mine. The silent expressionsreportedhere were highly complex; in fact, silent expressionsas a whole incorporateda widerrangeof visual componentsthan did expressionsaccompaniedby vocalizations(see Table4). This may partly be due to physicalconstraintsplacedon the mouthwhen an animalvocalizes, inhibitingthe concurrenceof vocalizationswith certainmouth movements SIGNAL COMPOSITION 1019 such as yawning, biting, tooth chomping, or pursing or compressing the lips. However,these particularmouth movements do not entirely preclude vocalizing:yawns have been observedto coincide occasionallywith barksin grey-cheekedmangabeys,for example (Deputte, 1994). In additionto mouthpositions, some body posturesneveroccurredwhile an animalvocalized, but did occur duringsilent behavior(see Table4). All occasions when an individualpresented its body to another,whether for grooming, occasional sexual interactions,or to indicate submission, were silent. Also silent were chin-up and dip-turn, which Hinde and Rowell referredto as 'dancing' (see Fig. 8b & c in Hinde & Rowell, 1962, p. 17). They did not mentionany vocalizationsoccurringduringthis behavior. Altmann also described this behavior in detail, althoughhe did not name it. He consideredit to be an 'extremeform' of 'smacks lips at' (Altmann, 1962, p. 378), and it did not occur in combinationwith any vocal units. This behavior may be analogous to the 'stylized trot' describedby Green (1975, p. 65) for Japanesemacaques.The facial componentmay be similar to the 'LEN' observedin pigtailmacaques(Lips protruded-Eyebrows raisedNeck forward;Jensen & Gordon, 1970, p. 268) and the 'pucker'described by Maestripieri& Wallen (1997). The males I studied were silent during the chin-up and dip-turnbehaviorregardlessof whether or not they were lipsmacking. Hauser (1993) has reportedthat male rhesus monkeys who vocalize duringcopulation,perhapsanalogous,aremorelikely to be attacked by othermales thanthose who copulatesilently. Visualsignals associated with vocalizations Multimodalsignals are an importantpartof the rhesus macaquerepertoire: in this studythey madeupjust over 30%of all behaviorsin the database.For example,barkandpant-threatvocalizationswere accompaniedby particular visual components(open mouth,staring, ears neutralor back, head neutral or lowered,body neutralor lunging,and approaching,chasing or remaining stationary)as suggested in the literature(Altmann, 1962; Hinde & Rowell, 1962; van Hooff, 1962, 1967; Rowell & Hinde, 1962; Redican, 1975; de Waalet al., 1976; Boccia, 1986). Novel findingsinclude a contrastin the frequenciesof the visual accompanimentsof the two aggressive vocalizations.Pant-threatswere more often accompaniedby staring, ears in the forward position, and head raised 1020 PARTAN orforward than were barks. Ear position duringthreathas been described by others to be retracted(e.g. Drickamer,1975), but the two threatening vocalizations have not been distinguishedpreviously in terms of these associated visual signals. Staring or looking at anotherindividualmay be a method of indexing with the eyes, indicatingto whom a vocalizationis directed (Itani, 1963; Altmann, 1967; Mitchell, 1972; Argyle & Cook, 1976; Green & Marler, 1979). Ears in the forwardposition may serve a similar function, indicating the intended addressee (Partan& Marler,2002). The correlationsbetween pant-threatand indexicaleye and ear positions suggest that pant-threatsare typically directedat one or more targetedindividuals, whereas barks are propagatedmore generally.Barks were more often accompaniedby looking around and looking between other individualsthan were pant-threats. 'Broadcast' signals are not directed to any recipient in particularbut are presented generally for all to see or hear. Other examples of broadcastsignals in rhesusincludebranchshaking,yawning,and tail raising. In the stumptailmacaque,broadcastsignals such as branchshaking (called 'bouncing') and patrollingwere also associated with barks (Adams & Schoel, 1982). The visual accompanimentsof the two submissivevocalizations,screams andsqueaks,differed.Both were generallyaccompaniedby grimacing,looking at, around,or between two other animals,neutralears, heads, and bodies, and retreatingor remainingstationary(Fig. 6), in agreementwith previous studies (Altmann, 1962; Hinde & Rowell, 1962; van Hooff, 1962, 1967; Rowell & Hinde, 1962; de Waalet al., 1976; Maestripieri& Wallen, 1997). Not previously documented,however, are that the ears were never held forward or flapped during submissive vocalizations,and that screams and squeaks differed in the proportionsof their visual accompaniments. Squeaks (shorterthan screams) occurredwith prolonged, stereotypicalgrimacing mouth positions, and were more often accompaniedby looking between the recipientand a thirdanimalthanwere screams.Squeaksneveroccurredduringactualfighting,however;they were morelikely to occurduring breaksin fighting or afterthe fight was over. Since looking between can be used for recruitment,squeaksmay have been used more often than screams when soliciting supportfrom allies presentat the scene. Two acoustically distinct vocalizations,girney and grunt, were the only ones to be sometimes accompaniedby the affiliativebehaviorsof lipsmacking and, in the case of girneys, head bobbing (Fig. 6). Vocalizing animals SIGNAL COMPOSITION 1021 usually looked at the recipient, had neutral ear, head, and body positions, and either approached(in the case of girneys) or remainedstationary.The ears, as with submissivebehaviors,were neverheldforward orflapped during these two vocalizations.Girneysandgruntsdifferedin the proportionsof theirvisual accompaniments.Girneyswere most often given with lipsmacking, an affiliativesignal (Hinde & Rowell, 1962; Mason, 1985; Maestripieri & Wallen, 1997). Gruntswere much more likely than girneys to be given withoutany mouthexpressionat all. Gruntswere also more likely thangirneysto be accompaniedby neutralpositionsof the eyes, ears,head,andbody, andgruntswere usually not accompaniedby movementin any particulardirection. Adams & Schoel (1982) found that stumptailmacaquegruntswere likewise not significantlyassociatedwith any particularacts or postures. Rowell & Hinde (1962) suggested that rhesus girneys (or 'girns') are affiliative,and Kalin et al. (1992) providedempirical evidence for this in infant monkeys. Hauser et al., (1993) classified both girneys and grunts as affiliative.Hauser & Marler(1993) stated that grunts occur in multiple contexts, including both affiliativeand food-related.Kaldor(1996) lumped grunts with girneys in her analyses and empiricallyfound them togetherto be affiliative. My data are consistent with the suggestion that girneys are affiliative,but I have little evidence that grunts have any emotionalvalence whatsoever. Mouthposition Mouthposition differs from the othervisual expressionsdiscussed in that it is intricatelytied with vocal production.Althoughvocalizationsand mouth shapes must to some extent be associated, there is disagreementabout the order of events on an evolutionarytimescale. Darwin (1872, p. 91) mentioned this as an "obscurepoint, namely, whether the sounds which are produced under various states of the mind determine the shape of the mouth, or whether its shape is not determinedby independentcauses, and the sound thus modified".Rowell (1962) suggested that vocalizations are secondaryto visual signals, postures, and breathingpatternsof rhesus monkeys. Andrew(1963) agreedthat vocalizationsare dependenton mouth shapes, althoughhe suggestedthatthe mouth shapesoriginatedprimarilyas protectiveresponses towardnoxious environmentalstimuli. Ohala (1984), however, indicated that the vocalizations were primary:he suggested that 1022 PARTAN mouth shape originatedin orderto produce specific types of sound. Ohala rules,' which based his ideas on Morton's (1977) 'motivational-structural predictthatan aggressiveanimalwill attemptto appearlargeby using sounds of lower frequencies,and submissiveanimalswill try to appearsmall, using higher-pitchedsounds (but see Hauser et al. 1993). Ohala suggested that large size could also be communicatedby a large resonantchamber,such as that createdby the roundedmouth of a macaquedisplayingan open-mouth threat.Conversely,smallness could be conveyed by high resonance,which would occur if the monkey pulled its lips back, shorteningthe length of the resonance chamber.Ohala suggested that throughritualization,the mouth shapes, originally in service of vocalizations,became independentand are now communicativeon theirown. I distinguishedbetween articulatoryand expressive movements of the mouth. I found that the most common mouth shapes accompanyingeach vocalization either matched the vocalization exactly (i.e. the mouth was opened only to the degree necessary to produce the sound) or were of a shape perhapsconstrainedby the requisite sound production(grimace for screamsand squeaks;roundedopen mouthfor barks;neutralor lipsmacking for girneys).This providesquantitativesupportfor the illustrationsin Hauser et al. (1993) of typical articulatorygestures accompanying eight main vocalizations of rhesus macaques. These data suggest that the mouth and the voice are not emancipatedduring bimodal (vocal) production.During unimodal,silent (visual) behavior,the mouth took on a variety of postures and movementsnot observedduringvocalizations,althoughthe silent open mouths,grimaces, and lipsmackswere similarin gross structureto the vocal ones. I compared the proportionof each vocalization that was accompanied by matchingmouth shapes (see Fig. 6a). Gruntshad the highest scores for mouthpositions exactly matchingthe vocalization,indicatingthattherewas no particularsilent expressiveshapeof the mouththataccompaniedgrunting. Hauseret al. (1993) found, similarly,that grunts were producedwith little separationof the lips. I foundthatscreamsandsqueaks,in contrastto grunts, had low scores for mouthmatching,indicatingthatthese vocalizationswere often accompaniedby extended mouth expressions, in this case grimaces. Girneys also had low scores for mouthmatching,because they were often producedsimultaneouslywith extensive lipsmacking. SIGNAL COMPOSITION 1023 Matching mouth positions may carry no new information,being purely redundantwith the vocalization, whereas an extended expressive mouth position may provide additionalinformation.For example, an expressive mouth position such as a prolonged, exaggeratedgrimace might reflect a higher intensity response than a more utilitarian(smaller,briefer)grimace thatsimply matcheda scream(see Maestripieri,1997; Partan,1998). Marler (1992) discussed the idea that certain componentsof a signal may reflect intensityof responsewhile othersreflectreferentialinformation. Conclusion The detailed associations among visual and vocal components of rhesus monkey expressions demonstratea method for quantifyingthe structureof multichannelsignals. The results provide new insights into signal structure and quantitativesupportfor previousdescriptionsof rhesusbehaviorfound in the literature.That particularvocalizations are not always paired with the exact same visual expressions reflects the fact that all behavior is probabilistic (as Altmann, 1965, discussed for sequences of behavior). However,some vocal expressionswere consistentlypairedwith predictable visual components;this was more often the case for screams and squeaks thanfor barksandpant-threats.Silent expressionswere the most variablein termsof the numbersof associatedvisual components. References Adams,D.B. & Schoel, M.W. (1982). A statisticalanalysisof the social behaviorof the male stumptailmacaque(Macaca arctoides).- Am. J. Primatol.2, p. 249-273. Altmann,S.A. (1962). A field studyof the sociobiology of rhesusmonkeys,Macaca mulatta. - Ann. N.Y. Acad. Sci. 102, p. 338-435. - (1965). Sociobiology of rhesus monkeys. II: Stochasticsof social communication. J. theoret.Biol. 8, p. 490-522. - (1967). The structureof primatesocial communication.- In: Social Communication Among Primates (S.A. Altmann, ed.). The University of Chicago Press, Chicago, p. 325-362. Andrew, R.J. (1963). The origin and evolution of the calls and facial expressions of the primates.-Behaviour 20, p. 1-109. Argyle, M. & Cook, M. (1976). Gaze and mutual gaze. - CambridgeUniversity Press, Cambridge. Boccia, M.L. (1986). Groomingsite preferencesas a form of tactile communicationandtheir role in the social relationsof rhesus monkeys. - In: Currentperspectivesin primate social dynamics(D.M. Taub& F.A. King, eds). VanNostrandReinholdCo., New York. 1024 PARTAN Bradbury,J.W.& Vehrencamp,S.L. (1998). Principlesof animalcommunication. Sinauer Associates, Inc., Sunderland,Massachusetts. Conner,W.E. (1987). Ultrasound:Its role in the courtshipof the arctiidmoth, Cynia tenera. - Experientia43, p. 1029-1031. Darwin, C. 1872/1897. The expression of the emotions in man and animals. - Second edition. D. Appleton& Co., New York. Deputte, B.L. (1994). Ethological study of yawning in primates.I. Quantitativeanalysis and study of causation in two species of Old World monkeys (Cercocebus albigena and Macacafascicularis). - Ethology 98, p. 221-245. (2000). Primate socialization revisited: Theoretical and practical issues in social ontogeny.- Adv. Study Behavior29, p. 99-157. Drickamer,L.C. (1975). Quantitativeobservationof behavior in free-rangingMacaca mulatta: Methodologyand aggression. -Behaviour 55, p. 209-236. Drummond,H. (1981). The natureand descriptionof behaviorpatterns.- In: Perspectives in ethology, Vol. 4 (P.P.G.Bateson & P.H. Klopfer,eds). Plenum,New York,p. 1-33. Fusani, L., Hutchison, R.E. & Hutchison, J.B. (1997). Vocal-posturalco-ordinationof a sexually dimorphicdisplayin a monomorphicspecies: The Barbarydove. - Behaviour 134, p. 321-335. Goldfoot, D.A. (1982). Multiplechannelsof sexual communicationin rhesusmonkeys:Role of olfactorycues. - In: Primatecommunication(C.T. Snowdon, C.H. Brown & M.R. Petersen,eds). CambridgeUniversityPress, Cambridge,p. 413-428. Gouzoules, S., Gouzoules, H. & Marler, P. (1984). Rhesus monkey (Macaca mulatta) screams:Representationalsignalingin the recruitmentof agonisticaid. - Anim. Behav. 32, p. 182-193. Green, S. (1975). Variationof vocal patternwith social situation in the Japanese monkey (Macaca fuscata): A field study. - In: Primate behavior (L.A. Rosenblum, ed.). Academic Press, New York,p. 1-102. & Marler,P. (1979). The analysis of animal communication.- In: Handbook of behavioralneurobiology,Vol. 3; Social behaviorand communication(P. Marler& J.G. Vandenbergh,eds). PlenumPress, New York. Hauser, M.D. (1991). Sources of acoustic variationin rhesus macaque (Macaca mulatta) vocalizations.- Ethology 89, p. 29-46. - (1993). Rhesus monkey copulationcalls: Honest signals for female choice? - Proc. R. Soc. Lond. B 254, p. 93-96. - (1996). Vocal communicationin macaques:Causes of variation.- In: Evolutionary ecology and behaviorof macaques(J.E.Fa & D. Lindburg,eds). CambridgeUniversity Press, Cambridge. - , Evans, C.S. & Marler,P. (1993). The role of articulationin the productionof rhesus monkey,Macaca mulatta,vocalizations.- Anim. Behav. 45, 423-433. - & Marler,P. (1993). Food-associatedcalls in rhesus macaques (Macaca mulatta):I. Socioecological factors.- Behav. Ecol. 4, p. 194-205. van derHeijden,P.G.,de Vries, H. & van Hooff, J.A.R.A.M.(1990). Correspondenceanalysis of transitionmatrices,with special attentionto missing entriesandasymmetry.- Anim. Behav.40, p. 49-64. Hinde, R.A. (1966). Ritualizationand social communicationin rhesus monkeys. - Phil. Trans.Roy. Soc. London251, p. 285-294. SIGNALCOMPOSITION 1025 & Rowell, T.E. (1962). Communicationby posturesandfacial expressionsin the rhesus monkey (Macaca mulatta).- Proc. Zool. Soc. Lond. 138, p. 1-21. van Hooff, J.A.R.A.M. (1962). Facial expressions in higher primates.- Symp. Zool. Soc. Lond. 8, p. 97-125. (1967). The facial displaysof the Catarrhinemonkeysandapes.- In:Primateethology (D. Morris,ed.). Aldine Pub. Co., Chicago, p. 7-68. (1973). A structuralanalysis of the social behaviour of a semi-captive group of chimpanzees.- In: Social communicationand movement:Studies of interactionand expression in man and chimpanzee (M.V. Cranach& I. Vine, eds). Academic Press, London,p. 75-162. (1982). Categoriesand sequencesof behavior:Methodsof descriptionand analysis.In: Handbookof methods in nonverbalbehaviorresearch(K.R. Scherer& P. Ekman, eds). CambridgeUniversityPress, Cambridge,p. 362-439. Itani,J. (1963). Vocal communicationof the wild Japanesemonkey.- Primates4, p. 11-66. Jensen,G.D. & Gordon,B.N. (1970). Sequencesof mother-infantbehaviorfollowing a facial communicativegestureof pigtail monkeys.- Biol. Psychiatry2, p. 267-272. Kaldor,E.L. (1996). The meaning and social function of grunt and girney vocalizationsin Macaca mulatta.- Msc thesis, Universityof California,Los Angeles. Kalin, N.H., Shelton, S.E. & Snowdon,C.T. (1992). Affiliativevocalizationsin infantrhesus macaques(Macaca mulatta).- J. Comp. Psychol. 106, p. 254-261. Leger,D.W.& Didrichsons,I.A. (1994). An assessmentof datapooling andsome alternatives. - Anim. Behav. 48, p. 823-832. Lindburg,D.G. (1971). The rhesus monkey in North India: An ecological and behavioral study. In: Primatebehavior:Developmentsin field research(L.A. Rosenblum,ed.). Academic Press, New York,p. 1-106. Machlis, L., Dodd, P.W.D. & Fentress,J.C. (1985). The pooling fallacy: Problems arising when individualscontributemore than one observationto the data set. - Z. Tierpsychol. 68, p. 201-214. Maestripieri, D. (1997). Gestural communication in macaques: Usage and meaning of nonvocalsignals. Evol. Communic. 1, p. 193-222. & Wallen, K. (1997). Affiliative and submissivecommunicationin rhesus macaques. - Primates38, p. 127-138. Marler,P. (1965). Communicationin monkeys and apes. - In: Primate behavior: Field studies of monkeys and apes (I. DeVore,ed.). Holt, Rinehardtand Winston,New York, p. 544-584. (1982). Avian and primate communication:The problem of naturalcategories. Neurosci. & Biobehav.Rev. 6, p. 87-94. - (1992). Functions of arousaland emotion in primatecommunication:a semiotic approach.- In: Topics in primatology(T. Nishida,W.C.McGrew,P. Marler,M. Pickford & F.B. deWaal,eds). Univ. of Tokyo Press, Tokyo,p. 225-233. - & Hamilton,W.J. III (1966). Mechanismof animalbehavior.- John Wiley & Sons, NY. Martin, P. & Bateson, P. (1993). Measuring behaviour.- CambridgeUniversity Press, Cambridge,UK. Mason, W.A. (1985). Experientialinfluenceson the developmentof expressivebehaviorsin rhesusmonkeys.- In: The developmentof expressivebehavior:Biology-environment interactions(G. Zivin, ed.). Academic Press, New York,p. 117-152. 1026 PARTAN Massaro, D.W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. The MIT Press, Cambridge,Massachusetts. Maxim, P.E. (1982). Contexts and messages in macaque social communication.- Am. J. Primatol.2, p. 63-85. (1985). Multidimensionalscaling of macaquesocial interaction.- Am. J. Primatol.8, p. 279-288. McGurk,H. & Macdonald,J. (1976). Hearinglips and seeing voices. - Nature264, p. 746748. Mitchell, G. (1972). Looking behaviorin the rhesus monkey. J. Phenomenol.Psychol. 3, p. 53-67. rules in Morton, E.S. (1977). On the occurrenceand significance of motivation-structural some bird and mammalsounds. - Am. Nat. 111, p. 855-869. Ohala,J.J. (1984). An ethologicalperspectiveon common cross-languageutilizationof Fo of voice. - Phonetica40, p. 1-18. Partan,S.R. (1998). Multimodalcommunication:The integrationof visual and vocal signals by rhesusmacaques.- Dissertation,Universityof California. - & Marler,P. (1999). Communicationgoes multimodal.- Science 283, p. 1272-1273. - & (2002). The Umweltand its relevanceto animalcommunication.Introduction to special issue. -J. Comp. Psychol. 116, p. 116-119. Rawlins, R.G. & Kessler, M.J. (1985). Climate and seasonal reproductionin the Cayo Santiagomacaques.- Am. J. Primatol.9, p. 87-99. - & - (1986). The history of the Cayo Santiago colony. - In: The Cayo Santiago macaques: History, behavior and biology (R.G. Rawlins & M.J. Kessler, eds). State Universityof New YorkPress, Albany,p. 13-45. Redican, W.K. (1975). Facial expressions in nonhumanprimates.- In: Primatebehavior: Developmentin field and laboratoryresearch(L.A. Rosenblum,ed.). Academic Press, New York,p. 103-194. Reynolds, V. (1976). The origins of a behavioralvocabulary:the case of the rhesusmonkey. J. Theory Social Behav. 6, p. 105-142. Rowe, C. (1999). Receiver psychology and the evolution of multicomponentsignals. Anim. Behav. 58, p. 921-931. & Guilford, T. (1996). Hidden colour aversions in domestic chicks triggered by pyrazineodoursof insect warningdisplays. - Nature383, p. 520-522. Rowell, T.E. (1962). Agonistic noises of the rhesus monkey (Macaca mulatta). - Symp. Zool. Soc. Lond. 8, p. 91-96. & Hinde, R.A. (1962). Vocalcommunicationby the rhesusmonkey (Macaca mulatta). - Proc. Zool. Soc. Lond. 138, p. 279-294. Sade, D.S. (1973). An ethogramfor rhesus monkeys:I. Antitheticalcontrastsin postureand movement.- Am. J. Phys. Anthrop.38, p. 537-542. de Waal, F.B.M., van Hooff, J.A.R.A.M. & Netto, W.J. (1976). An ethological analysis of types of agonisticinteractionin a captivegroupof Java-monkeys(Macacafascicularis). - Primates17, p. 257-290. Zeller, A.C. (1986). Comparisonof componentpatternsin threateningand friendly gestures in Macaca sylvanusof Gibraltar.- In: Currentperspectivesin primatesocial dynamics (D.M. Taub& F.A. King, eds). VanNostrandReinholdCo., New York,p. 487-504. SIGNALCOMPOSITION - - 1027 (1996). The inter-playof organisationandfacial communicationin the macaques.- In: Evolutionandecology of macaquesocieties (J.E.Fa & D.G. Lindburg,eds). Cambridge UniversityPress, Cambridge.