the PDF article.

2015 HBB4ALL -­‐ Report on subtitle layout E. Perego, O. Gerber Morón, S. Bottiroli HBB4ALL 11/26/2015 CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
The Effects of Personalizing Subtitle Layout: Some Preliminary
Results
E. Perego, O. Gerber-Morón, S. Bottiroli
1. State of the art: brief overview
To our knowledge, literature testing specifically the effect of subtitle layout (i.e. size and presence
of a black box background) is still missing. However, this type of research is important to study in
detail whether changing the layout of subtitles has any effects on viewers' attention, cognitive
performance, future intentions and on their overall attitude towards the film experience, as well as
the perceived usability of the subtitle. In fact, and especially for some specific samples of the
population, varying the subtitle layout might lead to a better performance and to a higher
appreciation of the film experience – which should remain a leisurely (vs. a challenging and
frustrating) activity. This is in fact the primary aim of HBB4ALL: to provide subtitles on multiple
platforms that are tailored to the specific needs of the end-users in terms of channels, platforms and
consumption requirements. Adjusting the font size according is one example of the possible
customization of subtitles (http://www.hbb4all.eu/home/hbb4allpilots/multiplatform-subtitle-services/).
Some studies testing the effect of changing specific technical aspects of the subtitles do exist and
they have been carried out especially in the field of subtitling for the deaf and hard of hearing.
Jensema for instance determined the preferred reading speed of deaf viewers, i.e. 145 words per
minute (Jensema 1998). Szarkowska et al. (2011) concentrated on the effect of text editing on deaf
and hard of hearing comprehension and appreciation.
In a very recent study, Romero Fresco (2015) studied some specific subtitle preferences of deaf and
hard of hearing viewers, also using eye tracking technology. The analysis of several types of
subtitle personalization features, such as font size, background boxes and subtitle position, was
included in this study carried out in different countries across Europe. The results regarding the
subtitle layout personalization for boxes and font size were not conclusive, but contributed to show
some tendencies and preferences. For instance, questions on size showed that the majority of
respondents for the study conducted in Italy preferred a middle size (32), arguing that it represents a
half way between too big (36) and too small (28). Nevertheless, the study carried out in Poland
showed that several deaf participants preferred bigger fonts to read more easily – which we believe
might suggest that cross-country differences should always be taken into account in AVT research
(see Perego et al. forthcoming). Regarding specifically the presence of a black box background, the
eye tracking studies conducted in Poland, Spain and Italy did not offer any clear conclusion and
they failed to show whether there are country or viewer related preferences. Only the study carried
out in France showed that some participants stated that black box background contributes to make
subtitles easier to read.
D3.2 – Pilot-A Solution Integration and Trials
1
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
Varying the subtitle layout could be particularly important for vulnerable groups of users who are
sensitive to what type of subtitles they are offered. Some deaf and hard of hearing viewers, for
instance, seem to be concerned about the absence of a black box background on TV programs. The
following is a 2011 comment by Carl Jensema, researcher in the field and deaf himself, who
intervened on a blog by CaptionMax, which has a dedicated Consumer Advisory Board with
experts in all kinds of accessibility and whose board members function as guest bloggers sharing
their accessibility stories or voicing their concerns (http://captionmax.com/blog/2011/02/dvdsubtitles-are-unreadable/):
I’d like to comment on a very annoying trend that I see in subtitling. I’m deaf and I use captions or
subtitles on all of the video media I watch. I’ve also been involved in television captioning research for
more than 30 years, beginning with my appointment as Director of Research at the National Captioning
Institute back in 1979. Before the first closed caption television decoder was manufactured in 1980, quite
a bit of research was done to determine the font and background characteristics that made the captions
most readable. The first decoders presented captions as white letters in a black box, and 30 years later
television decoders still use that method.
There is a reason for it: a black box with white lettering makes the most readable captions.
In recent years there has been a trend toward subtitling movies and other programs with a white or yellow
font without a background. This often makes the subtitles virtually unreadable. The difference in
readability is obvious; check out the examples below. I and other people who use subtitling find it very
frustrating to have unreadable subtitles spoil what could otherwise be an enjoyable program.
D3.2 – Pilot-A Solution Integration and Trials
2
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
Figure 1. Example of subtitles with black box background.
Figure 2. Example of subtitles without black box background.
D3.2 – Pilot-A Solution Integration and Trials
3
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
The two images show in fact a situation where the absence of a black box background makes the
subtitles unreadable. We must be aware that not all viewing situations are the same. However,
taking the needs of heavy subtitle users into account is very important. In fact, the HBB4ALL
project aims at improving the quality of media accessibility for individuals who are deaf and hardof-hearing, or do not have sufficient language skills to understand the content without textual
support
either
in
the
original
or
foreign
languages
(http://www.hbb4all.eu/home/hbb4allpilots/multiplatform-subtitle-services/). Thus, it is important to
understand how to present subtitles in the most suitable way for the different user profiles.
In the same CaptionMax blog, we can further observe that there are different types of users who
seem not to appreciate specific subtitle layouts: anime lovers for instance complain about the
coloured subtitles in licensed anime, and they point out that the fansubs done by “amateurs” are
much more readable. They ascribe greater readability to the benefit of having someone who actually
cares about the product they’re doing, i.e., fansubbers. Color blind users are another user category
who might benefit from more readable layouts.
2. The layout tests
To test whether different layouts have an effect on regular users we carried out a specific study as
part of the complementary user tests for Pilot A of the Multi-Platform Subtitle Services in the
HBB4ALL project. These tests will deliver metrics for quality of service and prepare the Pilot A,
which will provide advanced HbbTV automatic multilingual subtitling functionalities.
The aim of the present study carried out at the University of Trieste (Italy) was to identify the
possible advantages and disadvantages of personalizing subtitle layout. We wanted to assess
whether different subtitle layouts (i.e., big fonts without black box, standard fonts without black
box and black box background, cf. Fig. 1, 2, 3) could affect the enjoyment and the comprehension
of (linguistically, structurally and narratively) complex audiovisual material, and whether they
could affect the overall viewers’ evaluation of the subtitles. To do so, we carried out an empirical
study using an integrated approach (Perego 2014).
Based on current research, we do not expect to find significant differences in our sample of users
especially regarding the subtitle processing effectiveness. Our participants will be mainly university
students, and we expect them to be young and educated. Therefore, according to the subtitle
effectiveness hypothesis (Perego et al. 2010, 2015), we expect them to manage efficiently with all
types of subtitles. On the other hand, we believe that we might find differences with other types of
users (e.g. deaf, elderly, and people with slight sight impairment) and if we vary the presentation
screen (cinema vs. television vs. PC vs. tablet vs. smartphones).
Furthermore, we expect that preferences for a give layout might depend on what users are exposed
to during the experiment. The preferences of Italian viewers are not determined by tradition Italians are not habitual users of subtitled material because they belong to a traditionally dubbing
country (cf. Chaume 2012, MGC 2007, 2011), even if nowadays the situation is rather fluid (Perego
et al forthcoming).
D3.2 – Pilot-A Solution Integration and Trials
4
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
2.1 Method
2.1.1. Design
Three types of subtitle layouts (black box background, bigger text size and standard size without
black box), displayed on the same video excerpt, were presented to participants. They were
randomly assigned to three experimental conditions according to a three-group between-subject
design:
•
•
•
Group 1 (Standard): Standard text size, no black box background, n = 34
Group 2 (Big): Bigger text size, no black box background, n = 20
Group 3 (Box): Black box background, n = 29
The main dependent variables were measures of cognitive performance, as well as evaluative,
preference and usability measures collected in a written questionnaire filled after viewing the video
excerpt (see the following subsections for detailed descriptions).
2.1.2 Participants
•
•
•
•
Eighty-three undergraduates and postgraduates
73 female, 10 male, age range 18-34 years, M = 20.42, SD = 2.39
University of Trieste (Italy)
Italian native speakers who reported not being habitual viewers of subtitled films
D3.2 – Pilot-A Solution Integration and Trials
5
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
Table 1
Participant data divided by groups
Condition
Standard Age
sex Female
Years education
Ita mother tongue
N
Big
Age
sex Female
Years education
Ita mother tongue
N
Box
Age
sex Female
Years education
Ita mother tongue
N
Mean
20.47
88.2%
14.18
91.2%
Std.
Deviation
3.057
1.828
34
20.30
80.0%
14.25
100%
1.750
.410
1.713
20.45
93.1%
14.41
100%
1.901
20
1.955
29
2.1.3 Material
Video
We used a 22-minute video fragment from the seventh episode of How to Get Away with Murder, a
2014 American TV series created by Peter Nowalk, directed by Bill D’Elia and distributed by ABC
Studios. The video was shown in its dubbed version in Czech with Italian subtitles. We chose the
Czech dubbed version to expose participants to an unknown language, and to prevent them from
relying on information delivered from the original English soundtrack, known by all participants.
The video is fast-paced (cf. Lang et al. 1999, 2000) (16 to 23 camera changes per minute) and its
narrative structure is complex: it includes a large number of characters in interaction (18), several
interweaving story lines and alterations in chronology (3 flashbacks) that might be challenging for
viewers (Barsam, 2007; Murphy, 2007).
D3.2 – Pilot-A Solution Integration and Trials
6
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
Subtitles
We created 367 Italian subtitles (196 one liners, 171 two liners) and used them on the Czech dubbed
version of the video. Subtitles were based on the transcript of the original version in English, as
well the Italian dubbed and subtitled versions of the series. The final choice of subtitles for the
experiment tried to be as accurate as possible with the original version in English and the spotting
of the subtitles was based on the synchronization with the Czech dubbed version. The reading speed
chosen was of 180 words per minute1, which represents a higher rate applied in some companies
(Díaz Cintas, 2007) and contributes to render the video more complex to follow. We used the
freeware VLC Media Player (http://www.videolan.org/vlc/) to personalize the subtitle layout for the
experiment, i.e., to enlarge their size and to add a black box (see also Procedure section).
Questionnaires
General questions
•
•
•
Subtitle-reading check (7-point scale questionnaire on subtitle habits)
Demographic questionnaire
Viewing habits questionnaire
Cognitive measures
•
•
•
•
30-item multiple-choice questionnaire on general comprehension (Ex. 1)
30-item questionnaire on dialogue recognition based on specific words or phrases presented
in the film, (Ex. 2)
8-item face-name association test to check ability of participants to correctly the name and
the face of each character (Ex. 3)
60-item visual scene recognition test to verify if participants could recognize the frames
presented in the video (Ex. 4).
7. Attorney Keating explains to Rebecca that
if she does not change her attitude and start to collaborate, she will be sent to prison
if she continue to act in that way, she will not defend her anymore going
everything is under control and she does not have to worry about anything
!
!
!
Example 1. Multiple choice questionnaire item on general comprehension. The right answer is
emboldened.
1
The equivalence is calculated between seconds/frames and spaces. It is possible to estimate the approximate number
of characters that can be used to translate any dialogue. If a character speaks 01:00, the subtitler can use 17 spaces; 35
spaces for 02:00, 53 spaces for 03:00, 70 spaces for 04:00 and so on. Further information regarding the reading speed
can be found in Díaz Cintas & Remael A. (2007).
D3.2 – Pilot-A Solution Integration and Trials
7
www.hbb4all.eu
CIP-IST-PSP-621014
D3.2 v0.5
13. After the first hearing, Griffin O’Reilly argues with Rebecca Sutter outside the courtroom. Griffin
provokes Rebecca with accusations and insults. Among other things, he says to her that she is a
wretch
ragamuffin
cracker
!
!
!
Example 2. Multiple choice questionnaire item on dialogue recognition. The right answer is
emboldened.
Choose the right name for the characters of the video you just watched by making a circle around the name that
corresponds to the face for each frame.
1.
2.
3.
4.
5.
6.
7.
8.
Darcy
Rebecca
Griffin
Annie Keating
Keegan
Julien
Erik Turner
Mrs. Stangard
1.
2.
3.
4.
5.
6.
7.
8.
Darcy
Rebecca
Griffin
Annie Keating
Keegan
Julien
Erik Turner
Mrs. Stangard
1.
2.
3.
4.
5.
6.
7.
8.
Darcy
Rebecca
Griffin
Annie Keating
Keegan
Julien
Erik Turner
Mrs. Stangard
1.
2.
3.
4.
5.
6.
7.
8.
Darcy
Rebecca
Griffin
Annie Keating
Keegan
Julien
Erik Turner
Mrs. Stangard
Example 3. The 8-item face-name association test. The right answer is emboldened.
D3.2 – Pilot-A Solution Integration and Trials
8
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
YES
YES
NO
NO
YES
YES
NO
NO
Example 4. Four examples of the visual scene recognition test. The right answer is emboldened.
Evaluative and preference measures
•
•
•
•
•
2 general evaluative questions on the degree of film appreciation
1 question on preferred version (subtitled or dubbed) to watch the rest of the episode
2-item questionnaire on the general appreciation of the subtitle layout chosen for the
fragment
5-item general questionnaire on the layout
3-item visual preference questionnaire in which participants had to order the images with the
three different subtitle layouts tested according to their preference (cf. Figures 3, 4 and 5)
D3.2 – Pilot-A Solution Integration and Trials
9
www.hbb4all.eu
CIP-IST-PSP-621014
D3.2 v0.5
Figure 3. Freeze-frame with big fonts.
Figure 4. Freeze-frame with standard fonts.
Figure 5. Freeze-frame with black-box background.
D3.2 – Pilot-A Solution Integration and Trials
10
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
System Usability Scale (SUS) for the subtitle layout personalization
The System Usability Scale (SUS) (Brooke, 1996) is a simple, ten-item scale that provides a global
view of subjective assessments of usability. This scale covers many aspects for measuring usability
(need for support, training and complexity). The questionnaire was adapted for our test to focus the
attention on the analysis of the readability of the subtitles according to the subtitle layout.
Therefore, the questions were related to the subtitle layout usability, their perceived aesthetics, etc.
(e.g. “I found that subtitles of this size were very easy to read”; “I felt comfortable reading subtitles
of this size”).
2.1.4 Procedure
We carried out collective viewing sessions in a special cinema room, in which each group watched
the video fragment with Italian subtitles with one of the three subtitle layouts chosen for this test.
Group 1 (n = 34) watched the video fragment with the standard layout; group 2 (n = 20) watched
the video fragment with the bigger text size layout; group 3 (n = 29) watched the video fragment
with the black box background layout. No mention of the audio language and of the subtitle layout
was made. No participant had any knowledge of the original language of the TV series fragment
used in the experiment (Czech). They were informed that the study was on subtitled-film watching
and that the results would be delivered to the European project Hybrid Broadcast Broadband for All
(HBBTV4ALL). The participants signed a consent form and they were given instructions (“You
will be watching a short video except with Italian subtitles. Watch it as if you were at home or at the
cinema. After watching the video, you will have to fill a questionnaire”).
3. Results
Manipulation checks showed that participants attended to the task: they read the subtitles and
remembered their layout quite well. 100%, 90% and 86.2% of participants in the Standard, Big and
Box group respectively remembered subtitle color, without any difference (p=.10). Interestingly,
more participants in the Box group (41.4%) did not remember the subtitles alignment, whereas in
the Standard and Big group were only 8.8% and 0% respectively (p<.001). Regarding the perceived
subtitle reading difficulty, participants did not differ in how difficult they considered the subtitle
reading task (p = .31). The majority stated that reading the subtitles was easy and quite easy.
3.1. Viewing habits
The 83 participants to our experiments reported that they watch (both intra- and interlingually)
subtitled audiovisual material quite rarely, and they prefer to watch dubbed material, which
they say that they watch quite often (Table 2). This is not surprising in a dubbing country, such as
Italy.
D3.2 – Pilot-A Solution Integration and Trials
11
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
Groups did not differ in any of their habits (ps > .05), even those concerning the preferred device
where audiovisual material is watched. Italian participants (young, educated) prefer to watch
audiovisual material online (71%). Some watch audiovisual material on TV (10%) or on DVD
(20%), but no one seems to go to the cinema (Table 3).
Table 2
Participants’ viewing habits
N
How often do you watch audiovisual material in their
original language with subtitles in Italian?
How often do you watch audiovisual material in their
original language with subtitles in the same language of
the audio (e.g., film in English with English subtitles)?
How often do you watch audiovisual material dubbed in
Italian?
In general, do you often watch films, series or other
audiovisual material at the cinema, on TV or on other
devices?
D3.2 – Pilot-A Solution Integration and Trials
Standard
Big
Box
Total
Standard
Big
Box
Total
Standard
Big
Box
Total
Standard
Big
Box
Total
34
20
29
83
32
20
29
81
34
20
29
83
34
20
29
83
Mean
3,68
3,85
3,66
3,71
3,59
3,75
3,83
3,72
4,56
4,95
4,52
4,64
5,06
5,55
5,48
5,33
Std.
Deviation
2,070
1,899
1,798
1,916
1,932
1,888
1,814
1,859
1,761
1,820
1,785
1,771
1,669
1,234
1,526
1,523
12
www.hbb4all.eu
CIP-IST-PSP-621014
D3.2 v0.5
Table 3
Where do you watch audiovisual material more often?
Total
TV
8,4%
streaming
71,1%
cinema
0%
DVD
20,5%
Total
100,0%
3.2 Cognitive measures
Regarding the cognitive measures, results are in line with our initial hypothesis and with previous
literature on the effectiveness of subtitle processing (d’Ydewalle and de Bruycker 2007; Perego et
al. 2010, 2015; Hinkin et al. 2014): young and educated viewers cope well with subtitle
processing irrespective of the subtitle layout, and even if they are not used to subtitles because
they belong to a dubbing country. In particular, groups did not differ in their global
comprehension (p = .78): they answered correctly an average of 23 (out of 30) general
comprehension questions (Table 4). Groups did not differ in dialogue recognition (p = .87): they
were able to remember the exact words or expressions contained in the subtitles 21 times (out of 30)
(Table 5). Groups did not differ in how many face-name associations they remembered (p = .23):
they were able to associate at least 6 faces (out of 8) to their correct names (Table 6). Groups did
not differ in visual scene recognition (p = .55): they were able to recognize an average of 45 (out of
60) freeze-frames and determine whether they had been shown in the excerpt or not (Table 7).
D3.2 – Pilot-A Solution Integration and Trials
13
www.hbb4all.eu
CIP-IST-PSP-621014
D3.2 v0.5
Table 4
Global comprehension
N
Standard
Big
Box
Total
Mean
32
20
28
80
22,8438
23,1500
23,4643
23,1375
Std.
Deviation
4,04099
2,36810
2,82164
3,24815
Table 5
Dialogue recognition
N
Standard
Big
Box
Total
Mean
34
20
29
83
21,2647
21,7500
21,3103
21,3976
Std.
Deviation
3,71980
2,67296
3,65575
3,43916
Table 6
Face-name recognition test
N
Standard
Big
Box
Total
D3.2 – Pilot-A Solution Integration and Trials
Mean
33
16
26
75
6,1212
6,6875
6,7692
6,4667
Std.
Deviation
1,76348
1,49304
1,24283
1,55384
14
www.hbb4all.eu
CIP-IST-PSP-621014
D3.2 v0.5
Table 7
Visual scene recognition
N
Standard
Big
Box
Total
Mean
33
19
28
80
44,7576
43,8947
45,3214
44,7500
Std.
Deviation
4,57596
3,84267
4,48911
4,36180
3.3. Evaluative and preference measures
3.3.1 Overall evaluation of the film experience and future intentions of the viewers
Results show that groups did not differ for their overall evaluation of the film experience (p = .25)
(Table 8). More specifically, this means that participants found the film quite pleasant to pleasant
and they would probably recommend the film to a friend. Furthermore, groups did not differ in how
they would prefer to see the rest of the film (p = .61) if they were given the possibility of seeing it
dubbed in Italian or in its original version with Italian subtitles. Interestingly, although Italians are
used to dubbing and live in a predominantly dubbing country (cf. Chaume 2012, MGC 2007, 2011),
most respondents claim that they would watch the original version with Italian subtitles (Table
9). This is partially unexpected given the habits of the participants, but it shows that the
preferences of users seem to be partly shaped by their more recent viewing experience.
Table 8
Overall evaluation of the film experience
Standard
Big
Box
Total
D3.2 – Pilot-A Solution Integration and Trials
N
Mean
34
20
29
83
5,4853
5,3000
5,8103
5,5542
Std.
Deviation
1,17089
1,12858
,93936
1,09046
15
www.hbb4all.eu
CIP-IST-PSP-621014
D3.2 v0.5
Table 9
Viewers’ preferences in case they could watch the whole episode
I would prefer…
Subtitles
Dubbing
Total
Condition
Standard
21
13
34
Big
11
9
20
Total
Box
20
9
29
52
31
83
3.3.2 Subtitle layout: evaluation and preferences
Groups were asked specific questions on their general attitude towards the subtitle layout they were
exposed to, i.e., standard letters, bigger letters and subtitles on a black box. Table 10 shows that
groups did not differ in their global evaluation of the subtitle layout they were exposed to (p =
.70) – they found it quite pleasant to pleasant and they would be happy with watching the film with
the same layout. Groups were also asked if they would have preferred a different layout compared
to the layout they were exposed to. Groups did not differ in their choice (p = .78) and they
responded that they would not have chosen a different layout even if they could have.
Table 10
General layout preferences
N
Mean
Std. Deviation
Global evaluation of subtitle layout
Standard
Big
Box
34
20
29
5,53
5,40
5,34
,929
,940
,814
Would have preferred a different layout
Standard
Big
Box
34
20
29
2,94
3,15
3,10
1,229
,933
1,235
D3.2 – Pilot-A Solution Integration and Trials
16
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
As shown in Table 11, groups differed in terms of their overall subtitle evaluation with respect to
the subtitle size (big vs. small) and the presence of the black box (p < .001): the Box group gave a
better evaluation than the other two groups: the Box group in fact found the black box quite
pleasant, whereas both the Standard and the Big groups found the layout with no box neither
pleasant nor unpleasant.
Groups did not differ when asked whether the subtitle letter size (or the presence of the back box)
affected their reading process (p = .91) – participants tended to claim that subtitles were helpful
irrespective of their layout. However, groups differed in their answers when they were asked
whether they would have preferred a different layout (in terms of size or presence/absence of the
black box) (p = .01): the Standard group was the group that was more satisfied with the layout
it had been exposed to. The other groups were unsure about their stance and degree of satisfaction
with the layout they were exposed to.
Table 11
Overall and specific evaluation of the subtitle layout
N
Layout evaluation in terms of size and box
(How do you like the dimension of the subtitles you were
exposed to? /the fact that the subtitles you were exposed to
had a black box background?)
Layout (size/box) affects subtitle reading process?
Would you have preferred a different layout
(size/box)?
Standard
Big
Box
Total
Standard
Big
Box
Total
Standard
Big
Box
Total
Mean
34
20
29
83
34
20
29
83
34
20
29
83
3,62
3,55
5,07
4,11
5,06
5,10
5,14
5,10
2,21
3,25
2,97
2,72
Std.
Deviation
,739
,605
1,163
1,126
,776
,718
,581
,692
1,067
1,209
1,426
1,300
Finally, participants were asked to order the three layouts used in the experiment according to their
preferences (Tables 12, 13 and 14). Results show that overall the preferred layout is for this test is
the one including bigger fonts. The survey of the preference choices per group showed that:
•
The Standard group indicated as favorite layout the standard fonts (p2) and disliked the
black box (p3).
D3.2 – Pilot-A Solution Integration and Trials
17
CIP-IST-PSP-621014
•
•
www.hbb4all.eu
D3.2 v0.5
The Big group indicated as favorite layout the bigger fonts (p1) and disliked the black box
(p3).
Finally the Box group indicated as favorite layout the bigger fonts (p1) and disliked the
standard fonts (p2).
Results on the preference choices per group indicate that in most cases viewers tend to prefer the
layout they have been exposed to.
Table 12
Specific layout preferences for bigger size fonts (p1)
p1
1
Condition
Standard
Big
15
14
44,1%
70,0%
Box
13
44,8%
Total
Count
42
% within
50,6%
Condition
2
Count
18
6
14
38
% within
52,9%
30,0%
48,3%
45,8%
Condition
3
Count
1
0
2
3
% within
2,9%
,0%
6,9%
3,6%
Condition
Total
Count
34
20
29
83
% within
100,0% 100,0% 100,0% 100,0%
Condition
Note: p1 refers to the layout (bigger size fonts); the numbers 1, 2 and 3 to
the more (1) to the least (3) favourite layout
D3.2 – Pilot-A Solution Integration and Trials
18
www.hbb4all.eu
CIP-IST-PSP-621014
D3.2 v0.5
p1 = Freeze-frame with big fonts
Table 13
Specific layout preferences for standard fonts (p2)
p2
1
2
Count
% within
Condition
Count
Condition
Standard
Big
19
6
55,9%
30,0%
13
12
Box
Total
6
20,7%
31
37,3%
6
31
% within
38,2%
60,0%
20,7%
37,3%
Condition
3
Count
2
2
17
21
% within
5,9%
10,0%
58,6%
25,3%
Condition
Total
Count
34
20
29
83
% within
100,0% 100,0% 100,0% 100,0%
Condition
Note: p2 refers to the layout (standard fonts); the numbers 1, 2 and 3 to the
more (1) to the least (3) favourite layout
p 2 = Freeze-frame with standard fonts
D3.2 – Pilot-A Solution Integration and Trials
19
www.hbb4all.eu
CIP-IST-PSP-621014
D3.2 v0.5
Table 14
Specific layout preferences for black box (p3)
p3
1
2
Count
% within
Condition
Count
Condition
Standard
Big
0
0
,0%
,0%
3
Box
10
34,5%
2
9
Total
10
12,0%
14
% within
8,8%
10,0%
31,0%
16,9%
Condition
3
Count
31
18
10
59
% within
91,2%
90,0%
34,5%
71,1%
Condition
Total
Count
34
20
29
83
% within
100,0% 100,0% 100,0% 100,0%
Condition
Note: p3 refers to the layout (black box); the numbers 1, 2 and 3 to the more
(1) to the least (3) favourite layout
p 3 = Freeze-frame with black-box background
D3.2 – Pilot-A Solution Integration and Trials
20
www.hbb4all.eu
CIP-IST-PSP-621014
D3.2 v0.5
3.3.3. Perceived usability of the subtitles
Scoring SUS yields a single number representing a composite measure of the overall usability of the
system being studied.2 We generated a usability index (Table 15). In general terms, subtitles were
considered usable irrespective of the layout. However, the Box group gave less positive
evaluations in term of usability than the other two groups (p=.20).
Table 15
Usability index
N
Standard
Big
Box
Total
Mean
34
20
29
83
436
422
384
415
4. Discussion and conclusions
The HBB4ALL project aims at improving the quality of media accessibility for individuals who are
deaf and hard-of-hearing, or do not have sufficient language skills to understand the content without
textual support either in the original or foreign languages.
The experiment that we carried out contributes to accomplish this aim and they represent an
important step towards a broader awareness of what standard hearing viewers prefer when they
watch audiovisual material. In particular, the tests carried out at the University of Trieste (Italy)
aimed at finding whether personalizing the subtitle layout offers advantages (or rather
disadvantages) to the viewers, i.e., whether specific layouts (varying is size or encompassing a
black box background) can affect the way a film is understood, remembered, and enjoyed.
What we found is that changing the subtitle layout does not affect the viewers' cognitive
processing effectiveness when watching audiovisual material. In fact, all viewers responded well to
all cognitive tests. This is consistent with our initial hypothesis, whereby young and educated
viewers can process subtitles effectively also irrespective of the subtitle layout. This is in line with
2
Note that scores for individual items are not meaningful on their own. To calculate the SUS score, first sum the score
contributions from each item. Each item's score contribution will range from 0 to 4. For items 1,3,5,7,and 9 the score
contribution is the scale position minus 1. For items 2,4,6,8 and 10, the contribution is 5 minus the scale position.
Multiply the sum of the scores by 2.5 to obtain the overall value of SUS.
D3.2 – Pilot-A Solution Integration and Trials
21
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
the subtitle effectiveness hypothesis (Perego et al. 2010, 2015). However, we believe that we might
find differences with other types of users (e.g. deaf, elderly, people with slight sight impairment like
colour blindness) and if we vary the presentation screen. Regarding the latter point, we need to
point out that our experiments were carried out in a special cinema room. However, we think that
varying the layout on much smaller screens (tablet or smartphones) might have a different effect on
users.
Regarding the evaluative and the preference measures, our results also show that in general terms
varying the layout does not affect the overall film viewing experience. All participants gave an
overall positive evaluation of the layout they were exposed to and found they helpful. However, the
Box group (i.e., those who were exposed to the subtitles with a black box background) gave a
slightly better evaluation of the layout they used. These results show that the subtitle layout does
not affect the viewers' evaluations of the film experience and their layout preferences either.
Not only can young and educated viewers process subtitles effectively irrespective of the subtitle
layout, but they can also enjoy the film experience. Although this result is in line with the previous
studies and literature included in the introduction section, and although this result shows that
subtitle processing is overall effective, it also tells us that viewers could actually prefer some
specific layout. This should be taken in due consideration for the HBB4ALL purposes, because we
believe that preferences might be stronger in other types of users. Furthermore, this result seems to
prove the validity and the usability of a black box background for subtitling (as in Ivarsson and
Carroll 1998: 41). At the same time, the Standard group was more satisfied than the others with the
layout it had been exposed to, which is consistent with the recent studies carried out in Italy
(Romero Fresco 2015), in which participants showed a preference for middle size subtitles (32). In
fact, it seems to emerge that the least appreciated layout is the bigger size layout. It would be
interesting to compare this result with research on paper reading and font dimension.
A further noteworthy finding is that participants showed a preference to watch the rest of the film
with subtitles. This result is interesting because it seems to show that viewers easily adapt to what
they are exposed to. In our case, the participants to the experiment are Italian, they therefore belong
to a traditionally dubbing country, and they reported that they watch (both intra- and interlingually)
subtitled audiovisual material quite rarely, and they prefer to watch dubbed material, which they
say that they watch quite often. However, right after the exposition to a subtitled product, they fail
to opt for dubbing as their preferred choice to watch the rest of the film. This has interesting
implications because it suggests that changing viewing habits might be easier than expected.
Furthermore, if participants were willing to watch the rest of the film with subtitles, this may mean
that they were satisfied with the quality of the subtitles and that the subtitle layout did not affect
their decision of watching the film with subtitles.
Even though we did not find significant differences among the three layouts tested, further research
should be undertaken to test the personalization of subtitle layout on devices with different screen
sizes as well as with other types of user profiles that need media accessibility. Moreover, it would
be interesting to carry out experiments in other countries where users read subtitles more frequently.
To conclude, our results show that subtitle layout personalization might not be needed for young
educated viewers with standard vision accessing standard (or even complex) AV material, even
D3.2 – Pilot-A Solution Integration and Trials
22
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
though one specific layout choice might make them feel more satisfied. Since motivation and
involvement are at the basis of a satisfactory activity, however, it is possible to believe that TV
stations or subtitle producers might opt for the layout preferred by viewers to enhance their
satisfaction with a given program or channel or to, to keep a client a faithful customer Moreover,
Since motivation, involvement and satisfaction are at the basis of any successful learning
experience. This is rather relevant when using subtitles for language learning purposes.
6. References
Barsam, R. (2007). Looking at Movies. An Introduction to Film. New York: W. W. Norton &
Company.
Brooke, J. (1996). SUS: a “quick and dirty” usability scale. In P. W. Jordan, B. Thomas, B. A.
Weerdmeester, & A. L. McClelland (Eds.), Usability Evaluation in Industry (pp. 189-194).
London: Taylor and Francis.
Chaume, F. (2012). Audiovisual Translation: Dubbing. Manchester: St. Jerome.
Díaz Cintas & Remael A. (2007). Audiovisual Translation: Subtitling. London and New York:
Routledge.
d’Ydewalle, G., & de Bruycker, W. (2007). Eye movements of children and adults while reading
television subtitles. European Psychologist, 12(3), 196-205. doi: 10.1027/1016-9040.12.3.196
Hinkin, M.P., Harries, R.J., & Miranda A.T. (2014). Verbal Redundancx Aids Memory for Filmed
Entertainment Dialogue. The Journal of Psychology: Interdisciplinary and Applied, 148(2),
161-176. doi: 10.1080/00223980.2013.767774
Ivarsson, J., & Carroll, M. (1998). Subtitling. Simrishamn: TransEdit HB.
Jensema, C.J. (1998). Viewer reaction to different television captioning speeds. American Annals of
the Deaf, 143(4), 318-324.
Jensema, C. (2011, February 10). DVD Subtitles are Unreadable. [Web log comment]. Retrieved
from http://captionmax.com/blog/2011/02/dvd-subtitles-are-unreadable/
Lang, A., Bolls, P., Potter, R., & Kawahara, K. (1999). The effects of production pacing and
arousing content on the information processing of television messages. Journal of
Broadcasting & Electronic Media, 43, 451–475. doi:10.1080/08838159909364504
Lang, A., Zhou, S., Schwartz, N., Bolls, P. D., & Potter, R. F. (2000). The effects of edits on
arousal, attention, and memory for television messages. Journal of Broadcasting & Electronic
Media, 44(1), 94–109. doi:10.1207/s15506878jobem4401_7
MCG (Media Consulting Group). (2007). Study on Dubbing and Subtitling Needs and Practices in
the European Audiovisual Industry (Final report). European Commission, Directorate-General
Education and Culture.
MCG (Media Consulting Group). (2011). Study on the Use of Subtitling. The Potential of Subtitling
to Encourage Foreign Language Learning and Improve the Mastery of Foreign Languages
(Final report). European Commission, Directorate-General Education and Culture.
Murphy, J. J. (2007). Me and you and Memento and Fargo. New York: Continuum.
Perego, E., Del Missier, F., Porta, M., & Mosconi, M. (2010). The cognitive effectiveness of
subtitle processing. Media Psychology, 13(3), 243-272.
D3.2 – Pilot-A Solution Integration and Trials
23
CIP-IST-PSP-621014
www.hbb4all.eu
D3.2 v0.5
Perego, E. (2014). Un nuovo approccio integrato per la valutazione empirica della traduzione
audiovisiva. Rivista Internazionale di Tecnica della Traduzione, 16/2014: 189-206.
Perego, E., Del Missier, F., & Bottiroli, S. (2015). Dubbing and subtitling in young and older
adults: Cognitive and evaluative aspects. Perspectives: Studies in Translatology, 23(1): 1-21.
Perego, E., Laskowska, M., Matamala, A., Remael, A., Robert, Isabelle S., Szarkowska, A., Vilaró,
A., Bottiroli, S. (2016, forthcoming). Is subtitling equally effective everywhere? A first crossnational study on the reception of interlingually subtitled messages. Across languages and
cultures 2016/2.
Romero Fresco, P. (Ed.). (2015). The Reception of Subtitles for the Deaf and Hard of Hearing in
Europe. Bern: Peter Lang AG.
Szarkowska, A., Krejtz, I., Klyszejko, Z., & Wieczorek, A. (2011). Verbatim, standard, or edited?
Reading patterns of different captioning styles among deaf, hard of hearing, and hearing
viewers. American Annals of the Deaf 156(4), 363-378.
D3.2 – Pilot-A Solution Integration and Trials
24