Booklet_ohne Spiegelung

1st Summer School on Quality Assessment
AUGUST 12th–17th 2012
ILMENAU, Germany
ality Multimedia Evaluation Visual Assessment Audio Methodology Good Perception Influences Applica
n Summer School Statistics Trends Participants Measurement PhD Students Experts Tests Quality Multidia Evaluation Visual Assessment Audio Methodology Good Perception Influences Application Summer
ool Statistics Trends Participants Measurement PhD Students Experts Tests Quality Multimedia Evaluatio
ual Assessment Audio Methodology Good Perception Influences Application Summer School Statistics
nds Participants Measurement PhD Students Experts Tests Quality Multimedia Evaluation Visual Assessnt Audio Methodology Good Perception Influences Application Summer School Statistics Trends Partiants Measurement PhD Students Experts Tests Quality Multimedia Evaluation Visual Assessment Audio
thodology Good Perception Influences Application Summer School Statistics Trends Participants Measu
nt PhD Students Experts Tests Quality Multimedia Evaluation Visual Assessment Audio Methodology Go
ception Influences Application Summer School Statistics Trends Participants Measurement PhD Student
erts Tests Quality Multimedia Evaluation Visual Assessment Audio Methodology Good Perception Influ
es Application Summer School Statistics Trends Participants Measurement PhD Students Experts Test Q
Multimedia Evaluation Visual Assessment Audio Methodology Good Perception Influences Application
mmer School Statistics Trends Participants Measurement PhD Students Experts Tests Quality Multimedia
luation Visual Assessment Audio Methodology Good Perception Influences Application Summer Schoo
tistics Trends Participants Measurement PhD Students Experts Tests Quality Multimedia Evaluation Visu
essment Audio Methodology Good Perception Influences Application Summer School Statistics Trends
ticipants Measurement PhD Students Experts Tests Quality Multimeuality Multimedia Evaluation Visual
essment Audio Methodology Good Perception Influences Application Summer School Statistics Trends
ticipants Measurement PhD Students Experts Tests Quality Multimedia Evaluation Visual Assessment
dio Methodology Good Perception Influences Application Summer School Statistics Trends Participants
asurement PhD Students Experts Tests Quality Multimedia Evaluation Visual Assessment Audio Method
y Good Perception Influences Application Summer School Statistics Trends Participants Measurement P
dents Experts Tests Quality Multimedia Evaluation Visual Assessment Audio Methodology Good Percep
Influences Application Summer School Statistics Trends Participants Measurement PhD Students Expert
ts Quality Multimedia Evaluation Visual Assessment Audio Methodology Good Perception Influences
plication Summer School Statistics Trends Participants Measurement PhD Students Experts Tests QualiMultimedia Evaluation Visual Assessment Audio Methodology Good Perception Influences Application
mmer School Statistics Trends Participants Measurement PhD Students Experts Tests Quality Multimedia
luation Visual Assessment Audio Methodology Good Perception Influences Application Summer Schoo
tistics Trends Participants Measurement PhD Students Experts Tests Quality Multimedia Evaluation Visu
essment Audio Methodology Good Perception Influences Application Summer School Statistics Trends
ticipants Measurement PhD Students Experts Tests Quality Multimedia Evaluation Visual Assessmentia
luation Visual Assessment Audio MetApplication Summer School Statistics Trends Participants Measure
nt PhD Students Experts Tests
For Ph.D. STUDENTS and researchers
© istockphoto.com/Ever
LECTURES on Statistics and Test Methods
Preamble
On behalf of the Summer School Committee, it is a pleasure for us to welcome
you in Ilmenau for the 1st Summer School on Quality Assessment for Ph.D.
students and researchers, organised by TU Ilmenau, Qualinet, and Fraunhofer
IDMT.
The program will feature theoretical lectures and hands-on sessions with several
experts in the fields of statistics, test methods, and evaluation design. You will
have access to equipment and standardized laboratories for evaluation
activities assisted by experts. These evaluation activities will be based on both
your own research topics and new ideas that will be developed during the
course of the Summer School.
Ilmenau is surrounded by the Thuringian Forest where already the german artist
Johann Wolfgang von Goethe enjoyed many vacations. After the day's
programs you will have the opportunity to do sightseeing in Ilmenau or to let the
evening start in a relaxing atmosphere at bc-club.
We wish to thank TU Ilmenau, Fraunhofer IDMT and Qualinet for giving us the
opportunity to host this Summer School and for their work selecting the
program. We also thank our sponsors, who have generously supported the
conference, allowing us to offset some of the costs of holding this Summer
School.
We hope you enjoy the various scientific and social events of the next six days,
and that your time with us in Ilmenau is rewarding.
In case of need
0049 (0) 152 044 838 47 (Orga-Team)
Summer School Lectures
"Hot topics in audio and video quality evaluation" - Prof. Sebastian Möller,
Quality and Usability Lab, Telekom Innovation Labs, TU Berlin, Germany
Although standardized assessment and evaluation methods exist for speech,
audio, video and audio-visual services, there are a number of aspects which
are currently neglected in the evaluation process, be it in the area of subjective
metrics be it in the area of instrumental estimation. It is the aim of this talk to
shed a light on such missing aspects, grouping them into three topic areas. First,
dimensions of quality will be analyzed taking speech communication services
as an example, and showing that a dimension-based evaluation approach
can provide both summative as well as diagnostic information about relevant
quality aspects. Second, an approach will be presented to analyze the
processes taking place inside a human when s/he is judging on quality, using
EEG signals which are able to capture both conscious as well as pre-conscious
processes. Third, it will be shown that the process of modality selection is still ill
understood, talking the example of a multimodal human-computer interaction
service offering speech and gesture input as well as audiovisual (avatar)
output. The talk is intended to stimulate the discussion and to identify open
points of research which might be of interest to the attendees of the summer
school.
“Statistical experimental design – statistical analysis” - Dr. Nick Zacharov,
Genelec Oy, Iisalmi, Finland
When developing a test design for your experiments you also need to think
about the variables you want to measure, which scales or instruments you want
to use for it and how you want to analyze your data. Although this sounds
straightforward we will discuss what to take care about in the design, what
errors can occur and how to avoid them. Different statistical measures exist that
tell us something about the data we gathered. We will discuss statistical analysis
of different data types. Within statistical data analysis different errors can occur.
We will see how to deal with these errors. At the end we will also shortly cover
the presentation of your data so that others can also understand the main
message of your results.
“Audio evaluation, traditional and new approaches” - Dr. Nick Zacharov,
Genelec Oy, Iisalmi, Finland
Evaluating audio quality has a long tradition in research and development.
Subjective methods have been developed to evaluate the perceived quality,
human perception thresholds to ensure quality acceptance by users. Often
used methods are defined in standards and widely used. However, with novel
applications also more user-centered methods are developed. We will have a
look at different methods, their advantages and disadvantages and what you
have to consider when choosing one of these approaches for your evaluations.
“Non-Parametric statistics – Dos and don’ts” - Prof. Poppy Crum, Stanford
University, Dolby Laboratories, San Francisco, USA
In cases when parametric statistics cannot be used anymore non-parametric
statistics can be applied. These cases are explained and discussed and basics
of non-parametric statistics are shown. Usually when doing a statistical analysis
you get some nice numbers and significances, but often do not know what
exactly your results mean. We will address this point and learn what the results
tell us about the data and discuss power and effect size of the results.
Furthermore, it is very interesting to combine statistics with signal detection
theory and discuss the importance of the listener’s criterion.
"A closer look into new evaluation methods - Mixed methods approaches OPQ and user centered quality of experience" - Dr. Dominik Strohmeier,
Assessment of IP-based Applications, Telekom Innovation Labs, TU Berlin,
Germany and Prof. Alexander Raake, Asessment of IP-based Applications,
Telekom Innovation Labs, TU Berlin, Germany
In our joint session on new evaluation methods for Quality of Experience, we will
focus on three different aspects that we want to discuss. Quality of Experience
asks for new methodologies which enable researchers to understand the
correlation between Mean Opinion Scores and QoE features which are the
descriptive dimension of QoE. We will have a look into interview-based
approaches and sensory evaluation methods like Open Profiling of Quality. The
second part of this joint session will then dive into evaluations in the context of
use. Which methods do we need to study differences between laboratory and
field conditions? We will show you also results how QoE differs between lab and
field conditions. The third part of the session will then focus on the evaluation of
QoE aspects which go beyond technical influencing factors, especially
interactivity. Does interactivity between two partners in a VoIP call have an
impact on their QoE? As a research problem, we will discuss here which new
metrics are needed to be able to discover the impact of interactivity on the
QoE of your system.
“A brief history of quality assessment in MPEG-standardization” - Prof. Karlheinz
Brandeburg, Fraunhofer IDMT, University of Technology Ilmenau, Germany
Quality assessment in MPEG started in 1988 with the standardization of highquality low bit-rate audio coding and its evaluation. First standards for MPEG-1
and MPEG-2 audio were completed in 1992 and 1994 and are now widely used
and accepted. Technical developments put new demands on the existing
standards and new standards are developed constantly. MPEG standards for
MPEG-2 advanced audio coding were developed and systematically
improved. Current work in MPEG focuses on new standards and test methods
for 3D audio
"Data analysis using multivariate analysis methods" - Dr. Dominik Strohmeier,
Assessment of IP-based Applications, Telekom Innovation Labs, TU Berlin,
Germany
This lecture will focus on multivariate data analysis using methods like Principal
Component Analysis or Multiple Factor Analysis. At the beginning, we will have
a look how these methods work and how a large set of variables can be
analyzed into a small set of principal components or factors. We will also have
a short look into tests for significance for these methods. In the second part of
the lecture, I will show you some examples how we can apply these methods
for our quantitative data analysis. Here, I will especially have a look into Internal
and External Preference Mapping techniques.
"Video/3D evaluation, traditional and new approaches" - Prof. Patrick LeCallet,
Ecole polytechnique de l’université de Nantes, France
S-3DTV (stereoscopic-3DTV) should provide enhanced depth perception to
viewer while it might affect other factors of user experience. Subjective
assessment is the most direct way to assess quality of experience (QoE).
However, conventional assessment methods are not sufficient to evaluate the
QoE of S-3DTV. This course aims first to review the challenge of QoE assessment
related to S-3DTV and investigate different technical issues related to QoE
along the 3DTV broadcasting chain. Secondly, methodologies to evaluate QoE
with observers will be discussed, including environment conditions, scales and
display characterization. For user experience judgment, the key idea relies on
using multidimensional QoE indicators. Visual fatigue, as a particular dimension
of QoE, will be addressed separately under optimal viewing conditions using
questionnaire, vision test and EEG signals. For other QoE indicators, we will
review and discuss subjective QoE experiments to investigate the impact of
content acquisition, 3D representation format, compression and transmission on
QoE of S-3DTV. Additionally, we will illustrate how such experiments can lead to
meaningful technological recommendations like optimal stereoscopic shooting
rules, depth budget for visual comfort, bitrate to broadcast S-3DTV, threshold for
view asymmetries to avoid visual discomfort.
Program
Social Events
Sunday
BBQ
7 pm at Fraunhofer IDMT
Monday
“Amiga Label Evening”
9 pm at bc-Club (free entry*)
Tuesday
Documentary film (spanish with german subtitle)
https://www.facebook.com/por.encima.de.las.nubes
9 pm lawn between House H and I (free entry)
“D’n’B”
9 pm at bc-Club (free entry*)
Wednesday
Thursday
Friday
Guided Tour and Dinner in Erfurt
Departure: 5:10 pm at the bus stop (Mensa)
“Alternative”
9 pm at bc-Club (free entry*)
Additional Hiking Trip to Kickelhahn
Start 4 pm at Hotel Tanne
* For free entry please carry your name badges with you!