here - The Good Recording Project

Sound Recording Techniques
MediaCity, Salford
Wednesday 26th March, 2014
www.goodrecording.net
Perception and automated assessment
of recorded audio quality, focussing on
user generated content.
How distortion affects the perceived
quality of music: Psychoacoustic
experiments
Iain Jackson, Bruno M. Fazenda, Trevor J. Cox, Paul
Kendrick, Francis F. Li, Stephen Groves-Kirkby, & Alex
Wilson
Acoustics Research Centre, University of Salford
• How does clipping affect the perception
of quality in music?
– Are hard clipping and soft clipping
perceived differently in terms of quality?
• How well does HASQI predict subjective
quality ratings of clipped music?
• How robust is HASQI across different
styles of music?
What is HASQI?
• Hearing Aid Speech Quality Index (Kates & Arehart, 2010)
• Models the effect of degradation on quality.
• Measures the combined effect of noise, nonlinear
distortion, and linear filters.
• For both normal-hearing and hearing-impaired
listeners.
• Good performance for speech signals (Kressner et al, 2013)
– What happens when applied to music?
Arehart, Kates & Anderson (2011)
• Wide variety of degradation/processing:
– Additive noise, peak clipping, amplitude quantisation, compression,
compression + babble, spectral sub, high-pass filter, low-pass filter, bandpass
filter, positive spectral tilt, negative spectral tilt, single resonance peak,
multiple peaks, stationary noise...
– ...a total of 112 conditions.
• But...
– Only 3 samples of music.
“Haydn”
“jazz”
“vocalise”
• Quality ratings “reasonably well predicted” by HASQI.
– Were also “significantly affected by genre of music”.
Experiment 1
The effect of hard clipping on perceptions
of quality
• In contrast to previous work, we assess the effect of
a single type of processing – hard clipping – against
a comprehensive range of musical styles.
Sample Selection
• Aim: Select a representative sample of as wide a
range of musical styles as possible.
• Guided by previous work (Rentfrew & Gosling, 2003)
– 25 prototype songs from each of 14 Genres:
• Classical, jazz, blues, folk, alternative, rock, heavy metal, country,
pop, religious, rap/hip-hop, soul, funk, and electronica/dance.
• Final sample library of 140 songs.
– We obtained CD copies of 117 songs on the list.
• How to scale down to a manageable number of
songs for test?
– Sort and cluster by timbre.
Sample Selection
Why select by timbre, not genre?
• Genre
– Intuitively useful but lacking in objectivity.
• Timbre
– Apply objective methods to compare songs.
• Samples clustered using modified version of
technique used by Aucouturier and Pachet
(2002).
– Gaussian Mixture Model (GMM) fitted to Mel
Frequency Cepstrum Coefficients (MFCC) for 3 sections
each song, which are then clustered by similarity.
• Total number of clusters is an emergent feature:
– In this case it was found to be 6.
The test set
• From each of our 6 timbre clusters we draw two
samples.
– One cluster, number 4, however contains only one sample.
• Additionally, we include the three samples used by
Arehart et al (2011) in their previous assessment of
HASQI and music (“jazz”, “Haydn”, “vocalise).
• Thus the final test set consists of 14 samples.
Table 1. The 14 songs the final test samples were taken from, by
cluster number.
1
2
3
4
5
6
Song Name
Artist/Composer
Riverboat Set: Denis Dillon’s Square
Dance Polka, Dancing on the Riverboat
Crazy Train
“Haydn” *
Ave Maria
Packin' Truck
“vocalise” *
Kalifornia
Brown Sugar
The Four Seasons: Spring
For What It's Worth
The Girl From Ipanema
Spoonful
Nobody Loves Me But My Mother
“jazz” *
John Whelan
Ozzy Osbourne
*
Franz Schubert
Leadbelly
Tierney Sutton
Fatboy Slim
The Rolling Stones
Antonio Vivaldi
Buffalo Springfield
Stan Getz
Howlin' Wolf
B.B. King
*
Method
Distortion of samples
• HASQI is continuous between values of 0 to 1.
– HASQI values used to estimate discrete levels.
• 10 Levels per song sample:
– 9 levels of distortion, spread at equal intervals over full
range of (available) HASQI values.
– Plus original, clean sample.
Relationship between HASQI values and
threshold
Crazy Train
Threshold (% of peak level)
100
80
60
For What It's Worth
40
20
0
0
0.2
0.4
0.6
Distortion level (1-HASQI)
0.8
1
Table 1. The 14 songs the final test samples were taken from, by cluster number.
Song Name
Artist/Composer
Example Samples
Clean
1
2
3
4
5
6
Riverboat Set: Denis Dillon’s Square
Dance Polka, Dancing on the Riverboat
Crazy Train
“Haydn” *
Ave Maria
Packin' Truck
“vocalise” *
Kalifornia
Brown Sugar
The Four Seasons: Spring
For What It's Worth
The Girl From Ipanema
Spoonful
Nobody Loves Me But My Mother
“jazz” *
John Whelan
Ozzy Osbourne
*
Franz Schubert
Leadbelly
Tierney Sutton
Fatboy Slim
The Rolling Stones
Antonio Vivaldi
Buffalo Springfield
Stan Getz
Howlin' Wolf
B.B. King
*
Medium
High
• Broadly reproduced method used by Arehart et al.
• 30 participants.
– Mean age 23.7 years (SD: 4.7 years)
– No reported hearing impairments
• Sounds presented over headphones.
– Sennheiser 650 HD
– Stereo, 72dB (linear)
• 140 trials.
– 14 songs x 10 processing conditions
– 7 second samples (randomised presentation order)
• Ratings of overall quality.
– Slider labelled Bad and Excellent at either end (output: 0 100)
Results
Figure 1. Mean quality ratings of each cluster,
as a function of distortion level. (Error bars
show 95% CIs.)
Figure 1. Mean quality ratings of each cluster,
as a function of distortion level. (Error bars
show 95% CIs.)
• Differences in quality between timbre clusters?
– Repeated-measures ANOVA
• Independent variables: Level of distortion, cluster
• Dependent variable: Mean quality ratings
• Significant main effect for distortion level (F(4.97, 144.26) =
458.38, p = <.01, η ² = .94).
• Significant main effect for cluster (F(2.33, 67.48) = 42.43, p = <.01,
p
ηp² = .59).
• Significant interaction of cluster x distortion level
(F(11.91, 345.41) = 6.98, p = <.01, ηp² = .19).
• Each successive level of distortion is associated with
a significant decrease in quality ratings, but the rate
of degradation is not perceived equally across all
timbres.
Table 2. Clusters grouped according to (between group)
significantly different quality ratings.
1
2
3
6
4
3
5
5
6
4
Song Name
Artist/Composer
Riverboat Set: Denis Dillon’s Square
Dance Polka, Dancing on the Riverboat
Crazy Train
“Haydn” *
Ave Maria
Packin' Truck
“vocalise” *
Spoonful
Kalifornia
NobodySugar
Brown
Loves Me But My Mother
“jazz”
The
Four
* Seasons: Spring
Kalifornia
For
What It's Worth
Brown
The
GirlSugar
From Ipanema
For What It's Worth
Spoonful
The Girl From
Nobody
Loves Ipanema
Me But My Mother
The Four
“jazz”
* Seasons: Spring
John Whelan
Ozzy Osbourne
*
Franz Schubert
Leadbelly
Tierney Sutton
Howlin'Slim
Fatboy
Wolf
B.B. Rolling
The
King Stones
*
Antonio
Vivaldi
Fatboy Slim
Buffalo
Springfield
The Rolling
Stan
Getz Stones
Buffalo Springfield
Howlin'
Wolf
Stan King
B.B.
Getz
*Antonio Vivaldi
Results
HASQI performance
Table 3. Correlation coefficients for
quality ratings and values predicted
by HASQI for each timbre cluster.
Cluster
Quality
1
.828
2
.689
3
.693
4
.671
5
.801
6
.755
Mean (SD)
.732 (.065)
HASQI performance: for speech = .942 (Kates & Arehart, 2010)
for music = .838, (range = .770 to .849; Arehart et al, 2011)
Rnonlin performance: for music = .95 (1 music sample, 10 participants; Moore et al, 2004)
Conclusions
• How robust is the HASQI model over a
comprehensive range of musical styles?
– The performance of HASQI was found to be (a little) less
accurate than previous work suggests.
– Overall correlation of predicted vs actual quality ratings =
.73 (compared to equivalent value of .84 in Arehart et al).
• Predictive accuracy of HASQI can be improved by
factoring in timbral features of samples.
Experiment 2
The effect of Hard Vs Soft clipping on
perceptions of quality
Hard versus soft clipping
• Partial replication of Experiment 1.
– Both hard and soft clipping processing conditions included
in test set.
– Equivalent to distortion levels 1 to 5 from Experiment 1 (as
opposed to levels 1 to 9 considered in Experiment 1).
• Samples (original, clean files), experimental set-up,
procedure, and number of participants all as per
Experiment 1.
Hard Clipping Thresholds
Threshold (% of peak level)
70
60
50
40
30
20
10
0
0
0.1
0.2
0.3
0.4
Distortion level (1-HASQI)
0.5
0.6
Soft Clipping Thresholds
Threshold (% of peak level)
70
60
50
40
30
20
10
0
0
0.1
0.2
0.3
0.4
Distortion level (1-HASQI)
0.5
0.6
Hard versus soft clipping
Table 4. Comparison examples of hard and soft clipping at equivalent HASQI levels.
Song Name
Artist/Composer
Hard/Soft Clip
Distortion Level
Clean
Ave Maria
Franz Schubert
Hard
Soft
Packin' Truck
Leadbelly
Hard
Soft
Low
Medium
Hard versus soft clipping
100
100
Cluster 1
Cluster 1
90
Cluster 2
90
Cluster 2
80
Cluster 3
Cluster 4
80
Cluster 3
Cluster 4
Cluster 5
Cluster 6
70
Mean quality rating
70
Mean quality rating
Cluster 5
Cluster 6
60
50
40
60
50
40
30
30
20
20
10
10
0
0
0
1
2
3
4
5
6
7
Distortion Level (0 is clean, 9 is most distorted)
8
9
0
1
2
3
4
5
6
7
8
Distortion Level (0 is clean, 9 is most distorted)
Figure 4. Mean quality ratings for hard (left) and soft (right) distortion conditions,
shown by cluster. Error bars show 95% CIs.
9
Hard versus soft clipping
• Across all samples, no significant difference
between ratings for hard and soft clipping.
• HASQI performance is unaffected by type of
distortion.
Experiment 3
Descriptions of quality attributes in
different distortion categories
Digital audio sample statistics
Since digital audio is encoded as discrete samples of the
audio waveform, much can be said about a recording by
the statistical properties of these samples.
The Probability Mass Function can show the presence of
distortion in mastered audio. Consider three categories:
1. The ‘clean’ distribution, where there is no clipping and
a wide dynamic range.
2. Audio with hard-clipping will feature a PMF with high
values at its extreme values, where the maximum
amplitude has been reached.
3. Where softer distortions are used, there is not one
single large value at extremes but more gentle bumps
in the nearby regions.
Subjective Test (Wilson & Fazenda, submitted)
• 63 samples of music, containing a mix
of ‘clean’, hard-clipping distortion and
soft distortions.
• 22 participants gave quality ratings
for each sample on a 5-point scale
and also provided 2 descriptors.
• Ratings for clean samples were
significantly higher than for the two
distorted categories.
• The two distortion categories did not
significantly differ between
themselves (F(1, 2) = 5.72, p < 0.001,
η2 = 0.008).
Verbal descriptions of distortion categories
• As well as a rating out of 5 participants were also asked to provide two words
which described the attributes on which quality was assessed.
• For example:
• “I gave this sample 5 stars because it was clear and full”
• “I gave this sample 1 star because it was distorted and dull”
Word-clouds of the most common attributes associated with (a) clean samples,
(b) hard clipped samples, (c) soft distortion samples.
Verbal descriptions of distortion categories
•
•
Table shows the five most commonly used descriptor words and their
absolute frequencies for each of the clean, hard-clipped and soft distortion
categories.
Chi-Square analysis shows that there is significant variation in the
distribution of words used to describe each of the three categories (χ2(8, N =
547) = 33.28; p < 0.001).
•
Bold frequencies in the table indicate values significantly greater (>) or less
than (<) the expected counts of the null hypothesis.
Verbal descriptions of distortion categories
• “Distorted” is used less than expected by chance to describe
samples in the ‘clean’ category. The opposite is true for both
other categories, the hard and soft clipped distortion samples.
• Samples in the soft category are more frequently described as
“Distorted” than those in the hard category. This suggests that
small amounts of hard-clipping can go unnoticed.
• “Punchy” used less often when describing the soft distortions,
compared to hard-clipping. This may be due to the lesser influence
of inter-sample peaks in soft distortions compared to hardclipping.
• “Harsh” was not associated with either of the distortion categories
but does appear more often than expected by chance for words
describing the clean samples.
Conclusions
• Overall, HASQI found to predict degradation
in music quality reasonably well.
– Performance across hard and soft clipping is very
good.
• Limitation of HASQI for music - not
developed for stereo.
– Model does not account for stereo width and
panning.
References
•
•
•
•
•
•
•
K.H. Arehart, J.M. Kates and M.C. Anderson. Effects of noise, nonlinear processing, and linear
filtering on perceived music quality. Int. J. Audiol. 50(3):177–190. (2011).
J.J. Aucouturier and F. Pachet. Music similarity measures: What’s the use?. Proc. ISMIR.
(2002).
J.M. Kates and K.H. Arehart. The Hearing-Aid Speech Quality Index (HASQI). J. Audio Eng.
Soc. 58(5): 363–381. (2010).
A. Kressner, D. Anderson, and C. Rozell. Evaluating the generalization of the Hearing Aid
Speech Quality Index (HASQI). IEEE Trans. Audio. Speech. Lang. Processing. 21(2): 407–415.
(2013).
B.C.J. Moore, C-T, Tan, N.Zacharov and V-V. Mattila. Measuring and predicting the perceived
quality of music and speech subjected to combined linear and nonlinear distortion. J. Audio
Eng. Soc. 52(12): 1228–1244. (2004).
J.P. Rentfrow and S.D. Gosling. The Do Re Mi’s of everyday life: The structure and personality
correlates of music preferences. J. Pers. Soc. Psychol. 84(6): 1236-56. (2003).
A. Wilson and B.M. Fazenda. Sonic character: Categorisation of distortion profiles in relation
to audio quality of music recordings. Submitted to 17th Int. Conference on Digital Audio
Effects (DAFx-14).