Arena newsletter - October 2013

Architecture
for the
Recognition
of
Threats
to
Mobile Assets Using Networks
of
Multiple Affordable Sensors
Arena
N ewsletter
Issue No 2, October 2013.
A BRIEF PRESENTATION
The EU FP7 project ARENA addresses the design of a
flexible surveillance system for detection and recognition
of threats towards deployment on mobile critical assets
such as trucks, trains, vessels and oil rigs. The objective
of ARENA is to develop methods for automatic detection
and recognition of threats, based on multisensory data
analysis.
MID-TER M REV IEW
On the 21st of March, in Stockholm Sweden, the ARENA project was under review as part of the EU evaluation process. The review meeting was successful with
many interesting discussions and questions. The project
reviewers concluded that there is good progress within
ARENA and that the project has achieved most of its objectives and technical goals for the period with relatively
minor deviations. The objectives are still relevant and the
mid-term review concluded that the objectives are also
achievable.
■
DISSEMINATION ACTIV ITIES
Truck protected by the ARENA system
The 10th IEEE International Conference on Advanced
Video and Signal-Based Surveillance (AVSS) was held
in Krakow, Poland, on the 27-30th of August 2013.
In this conference, the ARENA project represented itself
with a contribution entitled ‘Activity recognition and localization on a truck parking lot‘. The contribution, jointly prepared among the partners, was presented at one of
the conference’s poster sessions.
For more information on the conference, visit:
www.avss2013.org
The full citation of ARENA’s contribution is:
Andersson, M., Patino, L., Burghouts, G., Flizikowski, A., Evans, M.,
Gustafsson, D., Petersson, H., Schutte, K., Ferryman, J., “Activity recognition and localization on a truck parking lot”, The 10th IEEE International Conference on Advanced Video and Signal-based Surveillance
(AVSS 2013), Krakow, 27-30 August, pp. 263-269, 2013.
■
Vessel protected by the ARENA system.
The final project results will be demonstrated in a live
demo and pre-recorded demos in Paris in April 2014. The
demonstrations will show the principles of the ARENA
system in a truck scenario, and optionally in a maritime
scenario.
ARENA has a stakeholder group with representatives
from both the land and maritime cases. The stakeholder
group has played an important role in the development of
user requirements, specifications and scenario definition.
■
COLLABOR ATIVE ACTIV ITIES
Data collection campaign (no. 2)
A data collection campaign was arranged at the University of Reading, UK, early September this year. The
purpose of this additional campaign was to collect more
video surveillance data to further evaluate all algorithms
developed within ARENA.
Integration Workshop
A two day workshop was held in Paris on the 3-4th of
October. The purpose of this workshop was to gather all
Architecture
for the
Recognition
of
Threats
to
Mobile Assets Using Networks
of
Multiple Affordable Sensors
A few photos from the data collection campaign undertaken at University of Reading, UK, Sepetember 2013.
partners for hands-on work with the integration platform.
The meeting was fruitful and the project made progress
towards the final demonstration.
■
now begin to settle. The system will incorporate all levels
from low-level sensor data processing to high-level decision support and HMI-interfaces. Recent and state-of-art
scientific results are used at all these levels.
TECHNICAL OVERV IEW
Object Detection and Tracking
ARENA integration platform
A backbone of the ARNEA-system and the test bed that
is going to be presented at the end of the project is the
integration platform(IP). It is a challenge to set up a
complete sensor system with effective and modular approaches to communicate information between all system components. The IP will provide an effective mean
to provide all algorithms with sensor data and to make
results easily accessible for operators etc.
Currently, intensive and collaborative work is ongoing to
test and make minor adjustments to the implemented IP.
The finalized IP will make an important outcome of the
ARENA project that can be used as a foundation when
designing other, ARENA-like, systems.
Algorithm Development
Not only the integration platform, but also the technologies and methodologies used within the ARENA system
Object detection and tracking is used to detect and track
interesting objects, e.g. pedestrians, within a scene.
Approach for detection
The background areas within each camera view is adaptively described using a Gaussian Mixture Model. All
model parameters are updated online (Zikovic, ICPR’04).
By using the background description, all foreground elements can be extracted. Pedestrians can then be detected
by using a classifier to classify all foreground elements.
The ‘Fastest Pedestrian Detector in the West’ is used for
this task (Dollar, BMCV’10).
Approach for tracking
Standard methods for organizing the sequence of detections (above) into consecutive tracks are applied:
• Linear Kalman filters
• Constant velocity motion models
• Multi-hypothesis tracker for data associations.
Architecture
for the
Recognition
of
Threats
to
Mobile Assets Using Networks
Trajectory speed
changing points
of
Multiple Affordable Sensors
Initial set of zones
Learned zones
Zone learning methodology: (i) Multi-resolution analysis of
all mobiles speed profile to extract speed changing points. (ii)
Speed changing points are the input to a fast clustering algorithm (Leader, Duda et al. 1995). The clustering results in an
initial set of zones {Zn}. (iii) The partition of clusters is corrected by merging similar zones, Zn, employing soft-computing
relationships.
Early results from algorithms performing object detection and
tracking
Action Recognition
The ARENA-system analyses imagery from detected pedestrians and classifies their actions into simple categories such as {walk, run, turn, check, fight, enter, loiter}.
Approaches for action recognition
The action recognition method is effectively one detector
for each action (Burghouts & Schutte, ICPR’12). Each
action detector quantifies STIP features by a soft-assignment random forest (Burghouts, IJPRAI’13). Locations
of the motion features in a 3D volume are captured by a
Gaussian layout model (Burghouts & Schutte, PRL’13).
The bag-of-words histograms are classified by an actionspecific SVM (Burghouts, Schutte, Bouma & den Hollander, Machine Vision and Applications’13).
Group and Fight Detection
Groups and fights will be detected by analysing how
densely pedestrians are located within a scene.
walk
80
2
4
6
4
2
2
run
21
64
0
7
0
0
7
loiter
4
0
78
12
4
2
0
turn
6
4
16
62
4
4
4
enter
8
0
0
10
82
0
0
Zone Based Events
check
0
0
6
19
6
63
6
fight
13
3
0
10
7
0
67
run
loiter
turn
enter
check
fight
Approaches
The group detection is based on K-means clustering
(Hastie et al, The elements of statistical learning, 2009),
the silhouette value (Rousseeuw, JCAM’87) and the
group density measure. Group detection consists of three
steps (Andersson, Gustafsson, St-Laurent, Prévost, JSTSP’13): (i) Tracking points are clustered for segmenting
people into clusters. (ii) The K-value yielding the highest
silhouette value gets to represent the current number of
clusters. (iii) For that K we calculate the group density
measures to see if there are any dense clusters.
walk
true activities
Classification performance of human activities on a parking lot (in percentages).
Overall performance 70.8%.
Top-view of a parking-lot. User defined areas are indicated with
arrows (T=truck, SZ=smoking area, SA=service area, PW=?).
Learned zones are in red.
Many of the real-world events of interest for the ARENA
system can be described by movements within and between different zones within a scene.
predicted activities
Automatic zone learning
Activity zones of the scene are those areas where people
interact or perform behavioural changes: stop, walk to
meet someone, speed up walking, or stand waiting.
Zone based events
Pedestrians patterns of transitions between activity areas
can be learned automatically and used to define situational events. A typical delivered event can be: “From just
south of zone Truck to just north of service area”.
Architecture
for the
Recognition
of
Threats
to
Mobile Assets Using Networks
True positive rate (TPR) for recognition of zone-based
events using data from the Paris data collection campaign .
Events
Instances
TPR
From smoking area to car
1
100%
From car to smoking area
1
100%
From car to truck
3
0%
From service area to truck
11
82%
From truck to car
2
0%
From truck to service area
11
91%
of
Multiple Affordable Sensors
ON THE AGENDA
Intensive work and efforts will be made during the autumn of 2013 to make progress towards the final implementations and to test the integration platform. New
hands-on workshops are being planned for and the planning of the final demonstration is ongoing.
■
Ontology Support
An ontology is included in the design of the ARENAsystem. The purpose of the ARENA Ontology is (i) to enable automatic configuration of threat recognition algorithms given e.g. current conditions and camera position,
etc. (ii) It is also to support situation assessment of the
parking area – e.g. by indicating threats and properties of
elements in a parking zone.
FURTHER INFOR MATION
Illustration on information that could be provided by the ARENA ontology: parking’s elements (static and dynamic), relations
and dependencies between parking’s elements, threats, properties.
Further work (on algorithm)
Further R&D work include to improve the tracking procedure: With increased continuity in position estimates
the event recognition performance can be further improved. Another area to investigate further is sensor fusion, and event fusion: By introducing fusion, events can
be observed from different perspectives and the event
recognition performance can be further improved.
■
The ARENA project is coordinated by FOI. For inquiries and requests for further information, please
visit the project’s site on internet:
www.arena-fp7.eu