Slides - cosound.dk

Audio Culture Researcher Requirements, Metadata
Design and Potentials of Automated Feature Extraction
Birger Larsen & the RSLIS LARM/CoSound team
Information Systems and Interaction Design
Royal School of Library and Information Science
University of Copenhagen
Denmark
Haakon Lund
RSLIS
Toine Bogers
RSLIS
Marianne Lykke
AAU
Mette Skov
AAU
www.iva.dk/blar - [email protected]
CoSound, June 21, 2013
Slide 2
CoSound, June 21, 2013
This talk
• Audio culture researcher requirements for an audio
research infrastructure
• User-driven development of a metadata scheme for
radio broadcast archives
• Reflections on the potential of automated feature
extraction
Slide 3
CoSound, June 21, 2013
Audio culture researcher requirements
for an audio research infrastructure
LARM Audio Research Archive
• Research infrastructure for radio and
auditive cultural heritage, 2010-1013
• Funded by the Danish Ministry of Science, Technology
and Innovation
• Consortium of 13 Danish institutions, universities +
Danish Radio + State and University Library
Objective: To further radio and audio research by maturing
research source material
• IT-infrastructure (main architecture and interface)
• Solutions to copyright issues
• National bibliography of radio
• Toolbox: Tools for metadating, annotating, search and
dissemination
Organisation: technical solutions, national bibliography,
research cases
Slide 4
CoSound, June 21, 2013
Two main challenges
• Large heterogeneous collection
• 1 million hours of broadcast radio
• 1937-present (24/7 coverage from 1989)
• Extremely sparse metadata, no funding to index
• Streaming access only because of copyright issues
• Heterogeneous user group
• All audio culture researchers...
• ...but very diverse research interests
• No public access due to copyright > large scale
crowdsourcing not possible
Slide 5
CoSound, June 21, 2013
Audio culture researcher requirements
Initial wish: a common philology / metadata / classification
scheme to be used across researchers
How to design a metadata scheme for audio culture research?
Analysis of Humanities information needs
• Questionnaire to Danish and international audio culture
researchers
Analysis of Humanities research tasks and needs
• Wiki for describing and analysing used concepts and
coding wishes and practises
Slide 6
CoSound, June 21, 2013
Questionnaire
results
Brain storming +
tagging sessions
60%
Respondents
17 Danish
+ 51 international
Preferred search entries
(how do you/would you
like to search for audio
for your research?)
• On average title,
genre and topic are
important, media
specific metadata
less so
• But large individual
differences
Slide 7
48%
45%
48%
CoSound, June 21, 2013
Questionnaire
results
Indexing level:
Which unit of
material is
important??
• Several levels of
indexing is
necessary
Needed more detail:
• Wiki for describing
and analysing used
concepts and
coding wishes and
practises
Slide 8
CoSound, June 21, 2013
Sample research cases
WP5.8 – The auditive syntax of radio
• Jacob Kreutzfeldt: Urban Soundscapes
• Identifying ‘real sound’ from Copenhagen
• Analysing how the ‘experience of space’ is
constructed
• Investigating the correlation between technology
and the portrait of ‘the sound of the city’
• Need for search and annotation
• Sound category (e.g. traffic, shop, car, bell, goat)
• Free tagging
• Mix (sound level (fore/background), fade in/out)
• Authenticity (real/scenographic /effect)
• Anchorage (space, language, rhythm, theme)
• Other
• Torben Sanglid: DR Jingles and Theme Songs
• Investigating changes of focus over time
• Theme songs (intro, separators, outro)
• Station IDs, Spots
CoSound, June 21, 2013
Developing a metadata scheme
Need for
• effective retrieval of radio broadcasts
• adding research-specific annotations
• at both the broadcast level as well as at segments of
broadcasts.
Needs of humanities researchers are so diverse that it is
unlikely that a single unified subject list will suit all
Main metadata requirements
• easy to work with
• easily extensible
• would provide for flexible data exchange
Different sources
• Original archive data
• User generated
• Of a general nature
• Project specific
Slide 10
CoSound, June 21, 2013
Conceptual metadata scheme
Slide 11
CoSound, June 21, 2013
Administrative metadata
Channel, program title, start/end time, narrative,
creators and roles
Archive
metadata
Administrative metadata
Program title (missing or wrong),
persons (participants/subject), genre,
related objects, subject, tags, annotation
LARM
metadata
Administrative metadata
[project specific]
Slide 12
Project
metadata
CoSound, June 21, 2013
Slide 13
CoSound, June 21, 2013
Slide 14
CoSound, June 21, 2013
Conceptual metadata scheme
Consistency across project metadata?
• project metadata scheme encoded in streaming
interface
• Coding manual saved in repository
• Documentation
• Reuse
Current state
• 3-level schema implemented in larm.fm
• In use by some research groups
• Individual project schemas needs to hand coded
Slide 15
CoSound, June 21, 2013
Reflections on the potential
of automated feature extraction
Flexible annotation made possible
• But still relatively few programs can be annotated
Automatic feature extraction has the potential to enrich the
audio files automatically
• First pass to e.g. segment the files
• More manual passes to refine and correct automatic
output
• Data driven humanities
For search in radio programs, automatic speech recognition
holds great potentials
• But not easy to do well for Danish
• On-going tests on DR radio news and the Queen’s New
Year addresses in collaboration with SpeechOp.com
Slide 16
CoSound, June 21, 2013
References and links
LARM – http://www.larm-archive.org
CoSound – http://cosound.dk
Unlocking radio broadcasts : User needs in sound
retrieval / Lykke, Marianne; Skov, Mette. Proceedings of
the 4st international conference on Information interaction
in context. Association for Computing Machinery, 2012. s.
298-301.
CHAOS: User-driven Development of a Metadata Scheme
for Radio Broadcast Archives / Lykke, Marianne; Bogers,
Toine; Larsen, Birger; Lund, Haakon. iConference 2013
Proceedings. 2013. (Poster)
Slide 17
CoSound, June 21, 2013
Search and Interaction in Media Archives
http://bit.ly/DRseminar
Open seminar at DR, 9:00-13:00 Thursday June 27, 2013
Designing the Search Experience
Tony Russell-Rose, Director of UXlabs, UK
larm.fm master class
Andreas Røll Larsen, DR Archive, Research & Rights
Good Practices for Finding Information
Kristian Norling, Enterprise Search Evangelist at Findwise, Sweden
Interaction with Sound Art Installations
Marianne Lykke, Professor at University of Aalborg, Denmark
The Future of Music Interaction
Daniel Boland, PhD student at University of Glasgow, UK