Basenet Keynote - Stanford InfoLab

Transcription

Event Sensing on Distributed
Video Sensor Networks
Edward Chang
Associate Professor
Department of Electrical and Computer Engineering
University of California
Acknowledgement

Collaborator
Prof.

Yuan-Fang Wang
Industry Partners
Proximex
ArgusSenses
Intel

Students
Ankur
Jain (lead), Diana Han, Dan Koppel,
Kyle Kakligian, Arun Qamra, David Trainor,
Tomy Tsai
Berkeley smart dust project
Stanford multi-camera array
Desired Features [refs #1,7,8,12]

Independent Cameras
Multiple Cameras
Static, Wired Cameras
Mobile, Wireless, PTZ cameras
Manual Event Recognition
Automatic or Semi-automatic Event Recognition
Tape-based Storage
RAID-based Storage
No Protocol Issues
Smart Semantic-based Protocols
Potential Applications and Needs

Applications
Emergency
search and rescue in natural disaster
Deterrence of cross-border illegal activities
Reconnaissance and intelligence gathering in
digital battlefields

Needs
Rapid
deployment, dynamic configuration, and
continuous operations
Robust and real-time data fusion and analysis
Intelligent event modeling and recognition
Wide-area Surveillance
advertisement of objectvideo.com
Surveillance Scenarios
(1)
(1) Intrusion
Intrusion Detection
Detection
Monitor and alert on tailgating,
loitering, exit/closed entry, other
unauthorized access
(3)
(3) Perimeter
Perimeter Monitoring
Monitoring
(2)
(2)Passenger
PassengerScreening
Screening
Use biometric facial recognition to
identify individuals of interest through
existing closed circuit TV surveillance
(4)
(4) Unattended
Unattended Baggage
Baggage
Z
Z
Object tracking and biometric facial
recognition to determine vehicles and
humans exhibiting suspicious behavior
Identify unattended baggage (or
other objects) left for long periods of
time
Copyright © 2004 Proximex Corp.
Intrusion Detection – Loitering, Tailgating
2
Raise
Raiselow
low
severity
severity alert
alert !!
0
1
Secure
Secureentrance
entrance
Suspect
Suspectloiters
loitersXXseconds,
seconds,
Loitering
pattern
recognized
Loitering pattern recognized
8
Correlate
Correlatealerts
alertsand
and
raise
critical
severity
raise critical severity
alert
alertififit’s
it’sthe
thesame
same
person!
person!
3
6
Capture
Capturehigh
highresolution
resolutionfacial
facial
image
&
soft
biometric
data
image & soft biometric data
Suspect
Suspecttailgates
tailgatesauthorized
authorized
person
personin
into
tosecure
securearea.
area.
System
captures
suspect’s
System captures suspect’s
high
highresolution
resolutionfacial
facialimage
image&&
soft
biometric
data
soft biometric data
5
4
Match
Match watch-List
watch-List
7
Raise
Raisemedium
medium
severity
severity alert
alert !!
Raise
Raisecritical
criticalseverity
severityalert
alertifif
there’s
there’saamatch
match!!
Copyright © 2004 Proximex Corp.
Problem Statement

Video Surveillance with

Multiple cameras
Mobile, wireless networks
Intelligent content analysis
Real-time query processing
Focus of Our Current Work

Event sensing including
¾
¾
¾

Motion detection
Data representation, and
Event recognition
Sensor network management for
¾
¾
Bandwidth and power resource conservation
Dynamic routing
Research and Development Framework

Event detection
Far-field
coordination and update
Near-field sensor data fusion

Event representation
Hierarchical
– multiple levels of detail
Invariant – insensitive to incidental changes

Event recognition
Temporally
correlated event signature
Imbalanced training set
System Architecture
s1
s2
Multi-camera
Data Fusion
s
Event
Recognition
Multi-level
Sequence
Descriptor
.
.
.
.
.
sn
!
Security
Alert
Validation Scenario
p 1 ( t ) = ( x1 ( t ), y 1 ( t )) T
z1
y1
Slave
station
x1
ym
xm
Internet
zm
P ( t ) = ( X ( t ), Y ( t ), Z ( t )) T
Z
Master
station
y2
z2
x2
Y
X
Event Detection: Coordination and Update
[Refs #5, 6]

Dual Kalman filters
Minimizing Bandwidth and power
consumption under pre-specified
accuracy constraints
Update necessary only when
predications diverge
Cache dynamic algorithms instead of
static data
Event Detection: Sensor Data Fusion [Refs #2, 9]

Sensing coordination and
intelligent data fusion
Master fusion
station
x(0) = Treal(0) ←worldX

Two-level hierarchy of Kalman
filter

Bottom level (feed forward)

Internet
p 
 
x(i) = p& (i) 
(i) 
p
&
&
 
p 
 
x(0) = p& (0) 
(0) 
p
&
&
 
Top level (feed backward)

X = Tworld←real( m−1) x(m−1)
Summarize trajectories in local
state vectors
X = Tworld←real(0) x(0)
Merge state vectors from multiple
cameras through registration
Slave station
Slave station
parameters
(i)
(0)

x(m−1) = Treal(m−1) ←worldX
P
&
X= P

&&
P

Fill in missing or occluded
trajectory pieces
Camera pose & frame
rate control
z(0)(t)
z(i)(t)
z(m−1)(t)
Slave station
p(m−1)
 
x(m−1) =p&(m−1)
(m−1) 
p
&
&
 
Sequence Data Learning [Ref #3]
Sequence
Sequence
Representation
Numeric-valued
Representation
Discrete Wavelet
Fourier
Transform
Piece-wise Singular Value
Linear
Decomposition
Symbolic-valued
Representation
Natural
Language
Strings
Requirements

Different Event Queries of Different Granularities
Coarse: temporal pattern query
Fine: exact match

Same Events under Different Representations
Circling behavior in daytime (normal)
Circling behavior with abrupt turn (suspicious)
Circling behavior during midnight (suspicious)
Multi-level Representation
Select proper resolutions in a concept and context
dependent way
Kernel Design Steps
1.
2.
3.
Design primary kernel based on
symbolic description
Design secondary kernels based on
multi-attribute vector description
Fuse kernels
Kernel Fusion

Definition

x: input vector; y: event label
{K1,…,KD} be a set of D kernels for D representations
Kd is a (dis)similarity matrix of the training instances
Kd should be positive semi-definite!
Kernel Fusion

Kfusion(x,y)= F(K1,…,KD)
F: Tensor product
F: Weighted sum
F: Non-linear super kernel
Event Recognition: Imbalanced Data Set
[Refs #4, 10]

Negative samples significantly
outnumber positive samples
Bayesian risk associated with
false negative significantly
outweighs false positive
Adaptive conformal mapping at
decision boundary
Sensor Network Management

Bit-per-joule metric
Multi-hop routing
Enclosure algorithm for
dynamic neighbor
discovery
[Ref #8]
Experimental Results: Semantic Indexing
[Refs #9, 11]
Experimental Results: Statistical Learning
Concluding Remarks
Multi-sensor registration and fusion
Hierarchical and invariant event
representation
Sequence data learning using imbalanced
data set
Mobile camera network bandwidth and
power consumption management

References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
A collaborative Camera System for Surveillance, A. Jain, Yuan-Fang Wang, E. Y. Chang, et al., UCSB
Technical Report, November 2004.
The Anatomy of a Multi-camera Security Surveillance System, L. Jiao, Y. Wu, G. Wu, E. Y. Chang, Y.-F.
Wang, ACM Multimedia System Journal Special Issue, Volume 10, Number 2, October 2004.
Distance Function Design and Fusion for Sequence Data,
Y. Wu and E. Y. Chang, ACM International Conference on Information and Knowledge Management (CIKM),
DC, November 2004 (19% accepted).
Aligning Boundary in Kernel Space for Learning Imbalanced Dataset, G. Wu and E. Y. Chang, IEEE
International Conference on Data Mining (ICDM), United Kingdom, November 2004 (39/452 or 9% accepted).
Adaptive Sampling for Sensor Networks, A. Jain and E. Y. Chang, International Workshop on Data Management
for Sensor Networks in conjunction with VLDB, Toronto, August 2004.
Adaptive Stream Resource Management Using Kalman Filters, A. Jain, E. Y. Chang, and Y.-F. Wang, ACM
International Conference on Management of Data (SIGMOD), pp.11-22, Paris, June 2004 (16% accepted).
Toward Building a Robust and Intelligent Video Surveillance System: A Case Study (invited paper), E. Y.
Chang, Y.-F. Wang, and I-J. Wang, IEEE International Conference on Multimedia, Taipei, June 2004.
Distributed Video Data Fusion and Mining, E.Y. Chang, Y.-F. Wang, and Volkan Rodoplu, SPIE Conf. on
Defense and Security --- Sensors, and Command, Control, Communications, and Intelligence Technologies for
Homeland Security and Homeland Defense (co-chaired by DARPA, Dod/Doj), Orlando, April 2004.
Multi-camera Spatio-temporal Fusion and Biased Sequence-data Learning for Security Surveillance, G. Wu, Y.
Wu, L. Jiao, Y.-F. Wang, and E. Y. Chang, ACM International Conference on Multimedia, pp. 528-538,
Berkeley, November 2003 (17% accepted).
Adaptive Feature-Space Conformal Transformation for Imbalanced-Data Learning, G. Wu and E.Y. Chang, The
Twentieth International Conference on Machine Learning (ICML), pp.816-823, Washington DC, August 2003.
Invariant Feature Extraction and Biased Statistical Inference for Video Surveillance, Y. Wu, L. Jiao, G. Wu, E.
Y. Chang, and Y.-F. Wang, IEEE International Conference on Advanced Video and Signal Based Surveillance,
pp.284-289, Miami, July 2003.
A Framework for Detecting Hazardous Events, Y. Wu, G. Wu, and E.Y. Chang, IS&T/SPIE International
Conference on Storage and Retrieval for Media Databases, San Jose, January 2003.
More on http://www.db.stanford.edu/~echang/

Basenet Keynote - Stanford InfoLab

Transcription

Similar documents