Slide show

Various approaches to the automatic
analysis of social behavior
Paweł Boguszewski
Nencki Institute of Experimental Biology
Warsaw, Poland
Measuring Social Behavior: Scientific Challenges, Test Paradigms and Tools
SfN 2009 Satellite Symposium, Chicago, October 20, 2009
Various approaches to the automatic analysis of social behavior
Social behavior as a research tool
Social Behavior is behavior directed towards or taking place
between members of the same species.
Application
•
•
•
•
•
•
•
•
Social withdrawal / sociability
Anxiety - Social Interaction Test
Aggression – Resident vs. Intruder paradigm
Domination / submission
Cognition - social recognition
Novelty seeking.
Maternal behavior
Sexual behavior / receptivity
Various approaches to the automatic analysis of social behavior
Social behavior as a research tool
• Ethologically relevant stimulus (source of anxiety, novelty)
• Dependent measure - natural form of behavior (character of interaction
and time spent by pair of animals in social contact)
• Flexible –experimental parameters:
• Chamber size and familiarity
• Group vs. single housing
• Social background (history)
• Light level, noise
• Duration and frequency
Concerns
• Complex behavior – resultant of multiple factors.
• Behavior of one anima influences that of the other – separation or
standardization of 2nd animal behavior.
Various approaches to the automatic analysis of social behavior
Automatic analysis of behavior
• Faster.
• No human observer bias:
• Objectivity
• Systematic vs. variable error
• No need for trained human observers
• Easier to replicate and compare results between laboratories
Concerns
• Data reductions
• Superficial analysis.
Various approaches to the automatic analysis of social behavior
Automatic analysis of social behavior
1. Computer aided behavior recognition
2. Test customization – simplified measure
3. Computer behavior recognition - machine vision
Various approaches to the automatic analysis of social behavior
1. Computer aided behavior recognition
Video footage encoding:
• Marking events based on digital movie - „multitrack stopwatch”.
• Precise timing, recording of overlapping states and events.
• Multiple subjects analysis.
• Time, sequence, patterns analysis.
• Easy way to supervise behavior recognition.
• Behaviors: Play behavior (wrestling, crawling), agonistic and aversive
postures, sniffing, biting,
http://www.pmbogusz.net/software/
Fot. Noldus Information Technology website
Various approaches to the automatic analysis of social behavior
2. Test customization
- Modified arena to facilitate various aspects of social behavior
(aggression, sociability, play).
- Analysis of single animal behavior – object tracking.
- Simple, spatiotemporal, parameter scoring.
- Easier analysis both for humans and computers.
- Not able to detect specific kinds of social behavior.
- Good in scoring non-social animal activity
Video
Detection
Trajectory
Various approaches to the automatic analysis of social behavior
2. Test customization
Sociability and social recognition test.
Testing social cognition – significant deficit
central to neurodevelopmental disorders.
- Withdrawal from social interaction
- Natural tendency to investigate novel individuals
Automatic analysis:
- Detection:
- IR beams
- video tracking
- vibration analysis
-Variables
- Number of entrances
- Time spent in each compartment
- Distance to the object
- Distance Nadler JJ et al. (2004) Genes, Brain and Behavior
Various approaches to the automatic analysis of social behavior
2. Test customization
Dominant – submissive behavior test
- Competition between two animals: development of dominant – submissive
relationship
- Pharmacologically validated for animal model of bipolar disorders.
- Video tracking system to detect time spent at the feeder
- In mice – tube test of social dominance
Liquid reward – sweetened milk
Malatynska E, Knap JR (2005) Neuroscience and Biobehavioral Reviews
Various approaches to the automatic analysis of social behavior
2. Test customization
Complex tests analysis – Modified Hole Board Test:
All in one: Social + open field + hole board
•
•
•
•
Anxiety
Exploration
Locomotor activity
Social affiliation
Ohl et. al. Behav Res Methods Instrum Comput , 2001.
Various approaches to the automatic analysis of social behavior
3. Computer behavior recognition
Fully automated analysis:
- Long distance:
- precise analysis of spatiotemporal parameters.
- relative position and movement
- Short distance behavior
- position and direction of animal body
- chasing, nose-to-nose, following
- Close contact
- play behavior: wrestling, crawling
- agonistic and aversive postures,
- sniffing, grooming
- direct aggression, biting
Various approaches to the automatic analysis of social behavior
3. Computer behavior recognition
CAPTCHA:
Completely
Automated
Public
Turing test to tell
Computers and
Humans
Apart
http://network-security-research.blogspot.com/2008_08_01_archive.html
Various approaches to the automatic analysis of social behavior
3. Computer behavior recognition
Animal recognition.
- Size of animal - BW image
- One animal has to be painted with background color, problems with
identification
- Long distance: Precise analysis of spatiotemporal parameters ,relative
position and movement, proximity, approach, avoidance.
Boguszewski P, Zagrodzka J. 2002 Behav Brain Res.
Various approaches to the automatic analysis of social behavior
3. Computer behavior recognition
Animal recognition.
- Color markers - color video image.
- Position and direction of animal body chasing, sniffing, nose-to-nose,
following.
Knapska E, et al (2006) Proc Natl Acad Sci U S A.
Various approaches to the automatic analysis of social behavior
3. Computer behavior recognition
Animal recognition.
- Good light condition –strong, white, minimum shadows
- Tracking spots, not animals - missing some short distance behavior
Various approaches to the automatic analysis of social behavior
3. Computer behavior recognition
Animal recognition.
- Whole contour and nose-tail recognition + color identification = virtual
animal body representation
- Better long and short behavior recognition
- Possible limited close contact recognition
Various approaches to the automatic analysis of social behavior
Automatic analysis of social behavior
1. Computer aided behavior recognition
2. Test customization – simplified measure
3. Computer behavior recognition - machine vision
Various approaches to the automatic analysis of social behavior
Statistical Analysis
Nagasaka Y et al. PNAS 2005;102:14854-14859
Various approaches to the automatic analysis of social behavior
Exploratory statistics – data mining
Principal Components Analysis PCA
Structure detection
Var
Var
Var
Var
Var
Var
1
2
3
4
5
6
Var 1
Var 3
Factor 1
Var 2
Var 4
Var 5
Factor 2
Exploration
VAR 1
VAR 2
Anxiety
VAR 3
VAR 4
PCA
Var 6
Factor 3
Factor 1
Factor 2
Various approaches to the automatic analysis of social behavior
SOCIAL INTERACTION
Number
100
30
20
6
4
50
10
2
0
0
0
Young
Old
***
140
120
Numbers
150
160
***
8
40
Numbers
200
Time [s]
10
50
250
Time
Number
Time [s]
Time
100
80
60
40
20
Young
Old
Active pro-social episodes
Boguszewski P, Zagrodzka J. 2002 Behav Brain Res.
0
Young
Old
Young
Old
Passive pro-social episodes
Various approaches to the automatic analysis of social behavior
SOCIAL INTERACTION
Young
Active
Social
Behavior
(44%)
Number of active prosocial episodes
Time of active prosocial episodes
Number of
withdrawal episodes
(22%)
Motor
Activity
(18%)
+
+
Social
Behavior
(41%)
+
+
+
+
_
Number of passive
pro-social episodes
Time of passive prosocial episodes
Passive
Social
Behavior
Old
Motor
Activity
(30%)
Boguszewski P, Zagrodzka J. 2002 Behav Brain Res.
(15%)
_
_
Ambulation of non
social character
Unsociable
Behavior
+
+
+
Various approaches to the automatic analysis of social behavior
Conclusion and “The Future”
- Contemporary automatic analysis of animal behavior is useful and
powerful tool to investigate many aspects of social behavior.
What we need:
- Smart algorithms that will detect close contact behavior.
- Light condition independent systems.
- Statistical tools for better behavior understanding.
Various approaches to the automatic analysis of social behavior
Thank you