Multimedia Information Retrieval

Multimedia Information Retrieval
Toetsing
Master AI/CS, MMIS, 2003, Lecture 1
Lecturer: Theo Gevers
Lab: ISIS
Email: [email protected]
http://www.science.uva.nl/~gevers
http://www.science.uva.nl/~gevers/master2003
De toetsing van het college is 1 schriftelijk tentamen en 1 opdracht.
Het tentamen maakt 50% van het uiteindelijke cijfer uit.
De opdracht de overige 50%.
De opdracht is:
Programmeeropdracht.
Programmeeropdracht. Het implementeren van een beeldopzoek systeem.
EN
Schrijfopdracht.
Schrijfopdracht. Het schrijven van ongeveer 10 pagina's over 1 bepaald
onderwerp.
0 Preview
Trends in Internet, Mobile Phones, Mobile Internet
*source Nokia concept
0 Preview
Trends in Internet, Mobile Phones, Mobile Internet
*source Le Monde
300
Number of users in Europe in millions
250
200
150
100
50
1997
1998
1999
Mobile phone
0 Preview
Trends in Internet, Mobile Phones, Mobile Internet
*source Le Monde
•There will be 200 million physical mobile users in Western Europe by 2005.
32 million of these users in Europe will be mobile multimedia users in 2005.
The total Western European mobile market will be worth 104 billion ECU per
year in 2005.
•The mobile multimedia segment of this Western European market will be
worth 24 billion ECU per year in 2005.
2000 2001
Internet
2002
2004
UMTS phone
0 Preview
Trends in Internet, Mobile Phones, Mobile Internet
Max speed
GSM/WAP 9,6-14,4 kb/s
Average speed Connection
Stand by
9,6-10 kb/s
circuit
no
GPRS
64 kb/s
20-50 kb/s
packet
yes
EDGE
128 kb/s
80-110 kb/s
packet
yes
UMTS
2 Mb/s
50-340 kb/s
packet
yes
1
0 Preview
0 Preview
Trends in Internet, Mobile Phones, Mobile Internet
10Gb/s
UMTS
Speed
1Gb/s
1987
0 Preview
Trends summary
•
•
•
•
1989
1992
SURFnet4 34 Mb/s
100kb/s
SURFnet3 2M b/s
1Mb/s
SURFnet2 64 kb/s
10Mb/s
SURFnet1 9,6 kb/s
100Mb/s
1995
1998
2000
SURFnet5 20 Gb/s
GPRS
SURFnet5 2,5 Gb/s
GSM/WAP
SURFnet4 155 Mb/s
Trends in Internet, Mobile Phones, Mobile Internet
2002
0 Preview
Trends in senses of the Web
SOUND
The increase in the use of Internet
The increase in size of digital photography (e.g. DV, Webcams)
The increase of mobile services (UMTS phones)
Digital TV
0 Preview
Trends in senses of the Web
SMELL
•Shopping: stinking out loud
•Gaming: the smell of victory
•Entertainment: read and sniff
•Television: getting to nose you
•Media: Talking heads [beatnik.com]
•Web browsing: Walls of sound
•Advertising: Nowhere to hide
•Health: Leveling the listening field
0 Preview
What is the future?
Multimedia information is the future
2
0 Preview
Industry
0 Preview
Multimedia information
Visual worlds
Company
Application
Data type
Complexity
Ilse Media
Image retrieval on internet Visual/textual unstructured
ANP Press
Photostock
Visual/category
structured
VNU/Nielsen Image tracking/monitoring Visual
unstructured
Akzo Nobel Coating effect classification Visual
structured
Forensic lab XTC-pill image classification Visual
structured
Philips
People tracking
Visual
structured
Elsevier
Multimodal document retrieval
Visual/textual
unstructured
SBS
Billboard tracking/replacement
Visual
unstructured
0 Preview
Demo1: real-time skin detection for human recognition
0 Preview
Demo3: real-time object recognition and tracking
*Hieu
WWW
Trademark assessment
Trademark in public
Video databases
Product database
general pictures
2D of 2D pictures
2D of 3D pictures
general pictures
limited domain
unknown conditions
known camera
unknown conditions
unknown camera
known camera
0 Preview
Demo2: skin/subtitle/speaker identification
0 Preview
Demo4: real-time object recognition and tracking
*Hieu
3
0 Preview
Demo5: real-time human recognition and tracking
*Hieu
0 Preview
Demo6: real-time human recognition and tracking
[Hieu, IEEE PAMI, 2003]
Robust to background clutter and changing object appearance
0 Preview
Demo7: real-time background detection and removal
*Anuj
0 Preview
Demo8: real-time object classification
video
material
0 Preview
Demo9: real-time object classification
video
classification
material
shadow-shape
classification
shadow-shape
0 Preview
Demo10: real-time object classification
video
classification
4
0 Preview
Demo11: real-time object classification
0 Preview
Demo13: real-time object classification
0 Preview
Demo15: real-time object classification
0 Preview
Demo12: real-time object classification
0 Preview
Demo14: real-time object classification
0 Preview
Demo16: real-time object classification
5
0 Preview
0 Preview
Demo17: Mosaic
Demo18: real-time object classification
Feike Winkelman
Techniques:
• Mosaics.
• Shot and key-frame detection.
• Analysis of camera-motion.
0 Preview
Demo19: real-time object classification: image
serach engines
0 Preview
Image segmentation: Philips Medical Systems
[Ghebreab, IEEE
PAMI]
Interactive image segmentation
to assist the user in outlining
the objects in medical images
• Techniques :
Techniques:
• Genre classification of image and video
• Search and learning strategies in image and video databases
• Interactive methods for image search
0 Preview
Forensic search XTC-pills: Forensic Laboratory
Interactive segmentation
(medical images)
0 Preview
Multimedia information
Text-based image retrieval systems
Convert content to keywords
at insertion?
run-time?
once the query is given?
Questions
Key-words
expressiveness
Key-words
ambiguity in language
Key-words
Digital image store
6
0 Preview
Multimedia information
Image database systems
0 Preview
Example: content-based image retrieval
query-by-example
matching
Query image
Digital image
Content-based image retrieval: query by example
database
query in natural format, unlimited repertoire
Question
computer vision is hard? how to handle in a database?
0 Preview
Multimedia information
Retrieval components
Features
text, colour, shape & texture
Representation
boolean, vector space and probabilistic
Indexing
inverted files, RS-trees
Searching and finding
k-nearest neighbor
Interaction
relevance feedback
7