MIPRCV Industry Day Prototypes for Video and Robotics (WP5) yp ( )

Transcription

MIPRCV Industry
y Day
y
Prototypes
yp for Video and Robotics ((WP5))
Angel Sappa
Computer Vision Center
[email protected]
l
@
b
th
October 25 2012
Contents
►
Introduction to the different prototypes
►
Video retrieval
►
C
Cooperative
ti detection
d t ti and
d recognition
iti off h
human actions
ti
►
Ubiquitous
q
robotics
►
Advanced driving assistance system (ADAS)
►
Technology transfer
2
Industry Day, Prototypes for Video and Robotics
Barcelona, 25/10/2012
Contents
►
►
Video retrieval
►
C
Cooperative
ti detection
d t ti and
d recognition
iti off h
human actions
ti
►
Ubiquitous
q
robotics
►
►
Technology transfer
3
Prototypes for Video and Robotics
Goal:
►
T develop
To
d
l an evaluate
l t different
diff
t applications
li ti
using
i the
th MIPRCV proposed
d framework
f
k.
Prototypes:
ototypes
4
►
Vid-ret. MI prototype for video retrieval.
►
Vid-hum. MI prototype for cooperative detection and recognition of human actions.
►
Vid-sur. MI prototype for video surveillance.
►
Vid-veh. MI prototype for advanced driving assistance system (ADAS).
►
Vid-rob. MI prototype for ubiquitous robotics.
Video Retrieval
N. Perez de la Blanca (UGR) & F. Pla (UJI)
Design of an interactive videovideo
retrieval system where the
combination of low-level
i f
information
ti in
i tterms off extracted
t t d
video features together with highlevel information in terms of
human-operator suggestions
allows skipping the semantic gap
presents in this type of problems.
5
Cooperative Det./Rec. of Human Actions
M. Castello (CV-CVC)
Design of a Cognitive
Vision System for human
behaviour understanding,
f ll
followed
db
by communication
i ti
of the system results to
end-users.
VID-Hum illustration with the three paradigms of the MIPRCV project:
adaptation (figure bottom), feedback (figure top) and multimodality (figure right).
6
Video Surveillance
R. Mollineda (UJI)
Hardware and software
prototype
t t
to
t perform
f
video
id
surveillance tasks, based on
image and video analysis
and pattern recognition
techniques.
7
Advanced Driving Assistance Syst.
A. López (CV-CVC)

Supervised
p
on-line learning
g to collect samples
p
p
providing
g sufficient variability
y
in a way that with no much of them a good classifier can be learnt.
Multimodality: features coming from
different sources.
Feedback: a human operator corrects the
errors of the incrementally build classifier.
Adaptation: the classifier is continuously
adapted to correct the errors pointed out
by the human operator.
8
Ubiquitous Robotics
A. Sanfeliu (IRI)
Design of a prototype for
ubiquitous robotics with
an architecture that takes
advantage of all the
i f
information
ti ffrom a large
l
number of ubiquitous
sensors (e.g., fixed
cameras, wireless
sensors, onboard robot
sensors).
)
9
Results
Multimodal Interaction in Image and Video Applications.
Editors: A.Sappa,
pp , J.Vitrià
Springer, Series: Intelligent Systems Reference Library
12 Chapters, 200pp
10
Contents
►
►
Video retrieval
►
C
Cooperative
ti detection
d t ti and
d recognition
iti off h
human actions
ti
►
Ubiquitous
q
robotics
►
►
Technology transfer
11
A Multimodal Interactive
Video Retrieval Prototype
T Arnau,
T.
Arnau I.
I Gracia,
Gracia M
M.A.
A Mendoza
Mendoza, N.
N Pérez de la Blanca
Blanca, F.Pla
F Pla
Universidad de Granada – UGR
Universitat Jaume I – UJI
Approached problem
►
The Problem:
■
13
How to retrieve specific videos from a large video collection when
the only additional information on the videos are very broad labels
on its content.
Collections, Queries & Model
►
The CCV database
■
►
14
More than 8000 videos of 20 broad categories from YouTube
Examples of types of queries:
■
Basketball: partidos de profesionales
■
Sport : deportes en la nieve/ deportes al aire libre/ deportes practicados por niños
■
Animals: animales en hogares
■
Leisure : espectaculos en escenarios / actividades lúdicas al aire libre
■
WeddingCeremony : bodas al aire libre / bodas en iglesias
■
Beach : niños / playas desiertas
■
Parade: multitudes/ carrozas /desfiles nocturnos
■
NonMusicPerformance : niños bailando/bailes en escenarios/ bailes al aire libre/patinaje
artístico/bailes
tí ti /b il en eventos
t deportivos
d
ti
■
MusicPerformance: espectaculos en escenarios/espectaculos al aire libre/ conciertos a piano/
conciertos multitudinarios/ coros/solistas
■
WeddingDance: baile inicial de los novios
■
Birthday : tartas de cumple/ niños
■
Graduation : personas con virrete/auditorio/ niños en coles
■
Bird : pajaros en la playa o medios acuaticos/ pajaros en jaulas/pajaros en la vegetacion
■
Dog : perros en la playa o medios acuatios/ perros en la nieve
Collections, Queries & Model
►
The CCV database
■
►
15
More than 8000 videos of 20 broad categories from YouTube
Retrieval Model : Based on a dense Graph Model where the
accumulate feedback information is propagated on each
iteration to rank the relevant items.
Retrieval Model
Dense graph propagation of feedback
16
Video Retrieval Process
►
The Solution
■
An interactive and iterative framework where the user
feedback guides to the retrieval engine on each iteration.
User’s query: kind of
videos
id
iin user’s
’ mind.
i d
17
Prototype
Videos initially organised
in concepts (possible
text tags annotation)
Database exploring
tool (hierarchical
clustering
g samples)
p )
Selected positive
videos by user so far
Active (on-line) learning
from user
user’s
s iterations
18
Contents
►
►
Video retrieval
►
C
Cooperative
ti detection
d t ti
and
d recognition
iti
off human
h
actions
ti
►
Ubiquitous
q
robotics
►
►
Technology transfer
19
Prototype for Cooperative Detection
and Recognition of Human Actions
Jordi Gonzalez and Marc Castellò
Computer
p
Vision Center
CV-CVC
►
►
21
The prototype platform has been installed on top of the CVC .
The cameras survey a scene in the engineering building square.
►
VID-Hum prototype will adapt to the most common trajectory behaviors
(qualitative data) inferred from tracked trajectories.
►
The result function discriminates between Normal trajectory, erratic,
prowling or forbidden access.
22
23
►
►
24
The reasoning engine receives the temporal information coming from the computer
vision algorithms.
The interpretation
p
relies on the designed
g
models to g
guide the conversion from visual
to semantic information.
Example (1)
►
25
Inferred conceptual
predicates are converted to
Natural Language sentences
to allow the prototype its
communication with endusers in a natural way.
Example (2)
26
Example (3)
27
Example (4)
28
►
29
VID-Hum incorporates a virtual avatar who explains using speech what is
happening
pp
g in the scene,, based on the natural language
g g texts g
generated.
Contents
►
►
Video retrieval
►
C
Cooperative
ti detection
d t ti and
d recognition
iti off h
human actions
ti
►
Ubiquitous
q
robotics
►
►
Technology transfer
30
Ubiquitous Robotics
Robot acompañante
31
Ubiquitous Robotics
Robot acompañante
32
Ubiquitous Robotics
Interacción Robot Humano: Aprendizaje asistido de caras
33
Contents
►
►
Video retrieval
►
C
Cooperative
ti detection
d t ti and
d recognition
iti off h
human actions
ti
►
Ubiquitous
q
robotics
►
►
Technology transfer
34
Vision-based Pedestrian Detection
35
Pedestrian Model Training
DOMAIN ADAPTATION
36
Contents
►
►
Video retrieval
►
C
Cooperative
ti detection
d t ti and
d recognition
iti off h
human actions
ti
►
Ubiquitous
q
robotics
►
►
Technology transfer
37

MIPRCV Industry Day Prototypes for Video and Robotics (WP5) yp ( )

Transcription

Similar documents

Invitation - Demcon 15-10-2015 - LEO Center for Service Robotics

Program RoboCafé LEO Robotics | Demcon 15-10-2015

2014 Project Syncere Summer Camp

The specialists tour operators` perspective on sense of place

PDF - AlbertGood.Com

pdf presentation

Barcelona - outandabouttravel.com.au

DEMCON develops robot for automatic asparagus harvesting

mBot Edison eBot

Holiday Houses Costa Brava Airport bus to Barcelona