Hearing Voices? Do citizens value voice as much as authority? Fred

Hearing Voices? Do citizens value voice as much as authority?
Fred Cutler, Paul Quirk, Ben Nyblade, and Eric Merkley
Prepared for MEDW meeting, Paris, May 23 2015
WARNING: THIS PAPER PRESENTS NULL EXPERIMENTAL RESULTS
The authors bear no responsibility for reader disappointment of the “this won’t get published so
why bother writing it up?” variety.
INTRODUCTION
Citizens now have high expectations for the democratic system in which they live. Scholarship on
the so-called “democratic deficit” has shown that the many features of democratic systems –
institutional and cultural – mean that citizens will always see shortcomings in their own system and
hope for improvement. Citizens’ judgments of the quality of their democracy are based in these
multiple, at least theoretically independent, and sometimes conflicting considerations. Macrooriented scholars have enumerated many of these features (Diamond 2005) and macromeasurement at the country level is now very rich (Bühlmann, Merkel, Müller, and Weẞels 2011).
At the individual level, however, the focus has been on the influence of institutional and contextual
factors conditioned by one dominant individual level attribute: supporting a party or candidate that
wins or one that ends up on the losing side. In this paper we examine the role of what might be
considered the opposite of winning and losing, or more broadly the relationship between a citizen
and political authority. We use an experimental design to examine the impact of merely hearing
one’s political voice, or not, in an election campaign.
Our experiments provide very naturalistic manipulations of hearing or not hearing political
“voices” about an issue, potentially salient for many voters, in an election campaign. Our previous
experiments showed that the voices present in campaign discourse can affect feelings of efficacy
and satisfaction even in a lab setting where the election campaign consists of 10 newspaper articles
from a provincial election away from subjects’ own province. We carry forward that inquiry by
manipulating the issues discussed in the campaign and whether those issues are discussed by a
2
niche party or by civil society groups. In this paper we only describe the design, summarize the
results, and explain the direction for continuation of this experimental inquiry. With a small subject
pool and weak treatments we found no effects to speak of in this round of experiments. We
conclude this report by suggesting changes to the design that may better reveal “voice” effects if
they exist.
THEORY
We focus on differentiating two key mechanisms through which the context of elections and
political discourse influence voter efficacy and ultimately, perhaps, their judgments of the quality of
their democracy. These are Voice and Authority.
The first mechanism – the one that produces the famous “winner-loser gap” – involves
citizens’ concern with authority and reflects a conventional notion of representation in government:
citizens are satisfied and feel efficacious to the extent that electoral processes and outcomes
provide indication that political actors who share their policy preferences will have authority in the
resulting government. The simplest way this happens is when one chooses a party or
representative who ends up on the government side of the legislature and executive. As Anderson
writes, electoral systems shape representation by “producing systematically different kinds of
governments.” The now rather extensive literature linking winning and losing to citizens
satisfaction, accordingly, mostly seeks to demonstrate that winners are more satisfied than losers
and that losers are more satisfied in consensus systems; and it explains these effects mainly as a
result of distances between citizens’ and governments’ policy positions. The empirical finding that
voting for the winning side promotes short-term satisfaction with democracy no doubt reflects the
simple fact that voters want government to serve their interests. Losers are the flip-side but their
responses may be more complicated: their degree of short-term dissatisfaction may depend on
their perceptions of the distance between their preferences and government policy, their party’s
3
chances of holding office in the future, and the possibility that their views will have influence even
though their party is not in government.1
However, if authority and its policy outputs were all that mattered to citizens, many
institutional features of democracies in and of themselves would not directly affect satisfaction with
democracy. And many normative theorists and social commentators would be out of a job, since
they advocate discourse-based solutions to the democratic deficit that seek to change the relative
power of discursive actors and the content that makes up the totality of political discourse in a
polity. Only those institutions that affected the composition and duration of governments would
affect satisfaction (and perhaps indirectly citizens’ relationships with other actors who may be in a
position to influence governments). While some of the interpretation of the winner-loser gap, or
what might be called the citizen-government authority effect, involves the articulation of voters’
concerns, winning and losing is theoretically distinguishable from the character of the voices that
effectively articulate different priorities and positions in a polity, especially during election
campaigns.
Thus a second mechanism connecting political contexts to satisfaction, largely overlooked in
previous empirical literature, involves citizens’ concern with voice and reflects the idea of
democracy as a dialogue that promotes intelligence and mutual regard and confers legitimacy. [Do
we need a paragraph on normative and deliberative democratic theory here?] We claim that
independent of citizens’ relationship with authority, they are more satisfied and feel more
efficacious to the extent that electoral processes give voice to their views. In short, they want to be
heard. Such voice ensures that their views have received consideration, and promises that their
1 Curini, Luigi, Willy Jou, and Vincenzo Memoli. "Satisfaction with Democracy and the Winner/Loser Debate: The Role of Policy Preferences and Past
Experience." British Journal of Political Science 42.02 (2012): 241-261.
4
views will be incorporated in future discussions—even if political actors who hold the same
preferences will have no significant or reliable authority in the resulting government.
To be sure, some scholars have suggested that voice may matter, but still typically combine
it with authority. Most prominently, the seminal piece in the satisfaction literature, Anderson and
Guillory, argued that the effects they observed were a result of opportunities for voice and policymaking together: “Given that consensual systems provide the political minority with a voice in the
decision-making process, we expect that the more consensual the set of political institutions in a
country, the greater is the extent to which negative consequences of losing elections are muted.”2
Note that here voice is not mere voice, independent of authority, but rather a voice in authoritative
decisions.
However, we believe that regardless of authority, voters are concerned with having their
voice heard. That is, mere voice is a distinct component of feelings of represented-ness and efficacy
that are central to citizens’ judgements of democracy. Such voice is not, of course, literally the
citizen’s direct expression of her own views. Some voters may never state their views aloud, much
less in any public forum. Rather, it is the form of voice feasible for most citizens in a mass,
representative, party democracy: one hears one’s own political voice articulated by significant
political actors, principally through mediated channels. Consistent with this conceptualization of
voice, in the most recent, comprehensive work in this area, Anderson and colleagues3 and Ezrow
and Xezonakis4 point to the importance of opportunities for voice for losers, both electoral and
representational. Anderson (2012) writes that: “a more numerous and differentiated [electoral]
2 Anderson and Guillory. ‘Political institutions and satisfaction with democracy.’
3 Anderson et al. Losers' Consent: Elections and Democratic Legitimacy.
4 Ezrow and Xezonakis. ‘Citizen Satisfaction with Democracy and Parties’ Policy Offerings.’
5
supply reduces the negative impact losing has on system attitudes because it provides the next best
thing to winning outright: having one’s political voice articulated clearly and visibly”. 5
We suggest, then, that citizens will have more positive views of their democracy if their
views are voiced by one or more of the parties than if they are not, and that this effect will occur
even if the parties voicing their views end up shut out of political authority. There is indirect
evidence for this in Aarts and Thomasson’s finding that citizens’ views on the quality of
representation are a more powerful determinant of democractic satisfaction than their views about
whether voting “makes a difference”.6 Ezrow and Xezonakis also suggest that voice is important
even in the absence of authority.7 For them, satisfaction depends on more accurate “policy
representation” and yet they mean by this not representation ultimately through authority, but
rather by voice. According to these authors, “diversity of party alternatives” is positively linked to
democratic satisfaction because parties “voice citizen demands for policy”.8 They quote Sartori to
the effect that: “Parties are channels of expression… They are an instrument, or an agency, for
representing the people by expressing their demands.”9 Our experiments speak specifically to these
propositions.
We share the view that regardless of which party ultimately forms the government, citizens
should prefer a democratic system and culture where their issue priorities and positions are
articulated by prominent political actors such as parties, especially during an election campaign. It
is hard to imagine a citizen feeling that she lives in a democracy in the absence of the effective
expression in the public sphere of views that she shares.
5 Anderson. ‘How Electoral Systems Shape What Voters Think About Democracy.’
6 Aarts, Kees, and Jacques Thomassen. "Satisfaction with democracy: Do institutions matter?." Electoral Studies 27.1 (2008): 5-18.
7 Ezrow and Xezonakis. ‘Citizen Satisfaction with Democracy and Parties’ Policy Offerings.’
8 Ezrow and Xezonakis. ‘Citizen Satisfaction with Democracy and Parties’ Policy Offerings.’ P. 1153
9 Sartori, Giovanni. Sartori, Giovanni. Parties and party systems: A framework for analysis. 1976: p. 27
6
The authority and voice mechanisms have distinct observable implications and lead to
scholars looking at data at different levels. The authority mechanism is simpler to analyze, with its
focus on the congruence of an individual with the parties in government. Indeed, a welter of
evidence bears out this hypothesis (Anderson and Guillory, Anderson et al, Curini Memoli and Jou,
Bernauer and Vatter, etc.). The voice mechanism is more complex because it involves something
less tangible. Voters should feel more efficacious when they hear voices articulating those voters’
concerns, but is the discussion of those concerns enough or does it depend, perhaps, on the depth of
discussion, the force of opposing views, or the identity of the actors articulating those views?
STUDY DESIGN
The initial study reported here used computer-based laboratory experiments with undergraduate
student subjects. Subjects experienced stimuli that simulated the experience of voters in election
campaigns and measured several aspects of their responses. We had intended our subjects to be of
two types: undergraduates enrolled in political science classes (which likely makes them a better
match to the general voting public than other university students who tend to have much less
political socialization and interest) and members of small-scale environmental groups. We
targetted the latter so that we could assume that environmental issues were salient to the votersubject and that all else equal those people would prefer that environmental issues received a
hearing in an election campaign. But we were unable to secure the timely cooperation of an
environmental group and so the current paper presents only results from the student subject pool
where we measure environmental concern rather than holding it constant by targetting subjects.
Towards the end, we describe the research design that uses environmentalist subjects.
The experimental manipulations involve the presentation three versions of an election campaign
with important differences in the issue content and the identity of the actors articulating issues.
Our design has two distinctive and related features that combine to provide, in our view, an
7
exceptional degree of correspondence with the real-world conditions pertinent to the subjects—a
property that should have major benefits with respect to external validity.
First, taking advantage of some special features of Canadian party politics, we presented campaigns
that featured varying, yet plausible subsets of the same parties that the subjects engage with in
real-world political life. While experiments that omit parties or use fictitious party names may tell
us something about voters’ responses to policy positions, they are so far removed from the
symbolic and cue-giving roles of parties that they leave us guessing about the power of analogous
treatments in the real world. Using Canadian subjects then, we presented campaigns that either did
or did not include the Green party and did or did not include discussion of environmental issues.
Canadian party politics offer distinct conveniences for the experimenter. At both the provincial and
federal level, recent elections in English-speaking Canada have included four parties: from left to
right on the ideological scale, the environmentalist Greens, the social-democratic New Democrats,
the centrist Liberals, and the right-leaning Conservatives. The crucial advantage of the Canadian
context for our purposes, however, is that provincial party systems do not closely mirror each other
or the national party system. The lists and respective identities of the active, competitive parties
vary considerably across provinces, across levels within a province, and over time. In particular,
many provincial elections effectively involve only two or three of the four national parties.
In addition, most Canadians in a given province do not follow politics in other provinces, especially
the smaller provinces, closely. Yet they are exposed to occasional news from other provinces, with
enough variation in election line-ups, to make any subset of the four parties plausible as a general
matter. These distinctive conditions give us a powerful research opportunity. We presented our
subjects with a provincial campaign occurring at an unspecified, “recent” time in Manitoba, a
8
province with which we can assume our subjects are by-and-large unfamiliar.10 We were thus able
to manipulate basic features of the party system and election discourse while keeping them, from
subjects’ standpoint, completely realistic.
Second, we assiduously maintained this realism in the presentation of policy positions and other
information from the campaign. Many election experiments present brief, stylized, and highly
simplified representations of political actors’ policy or ideological positions—making a vivid
impression of their differences essentially inescapable for subjects. Although such designs have
advantages, they beg the question of which features of campaign activity actually gain voters’
attention and have an impact in the campaign.
In contrast, our design presents a simulated campaign through a series of full-length newspaper
stories, carefully created by us—including rich detail on campaign events and rigorously realistic
statements of issue positions. Specifically, the simulated campaign consists of subjects reading
eleven constructed newspaper articles in a set sequence. The articles were developed by a research
assistant, formerly a reporter for a big-city newspaper, by modifying actual articles from Manitoba
and other provincial election campaigns to meet the requirements of the three different campaigns
specified in our research design. Articles were carefully formatted to look like actual newspaper
articles. (See the online Appendix for the full set of articles). Eight articles (hereafter, “policy
articles”) covered party leaders’ campaign events or appearances in which party leaders,
candidates, and other actors expressed policy positions on one of four issues (two articles for each
issue)—work-for-welfare, the minimum wage, crime, and, depending on the condition, either the
environment or the arts. Policy positions were quite specific, reflecting the style of rhetoric in the
original articles (for example, on crime: building more prisons versus after-school crime prevention
10 We excluded from analysis the 2 subjects (1% per cent) who reported that they had lived in Manitoba.
9
programs). The two final campaign articles summarized the parties’ policy differences, one
reporting on a party leaders’ debate, the other giving a pre-election summary of the campaign.
Subjects then indicated how they would vote in this election. Finally, they received one of four
randomly assigned results stories, indicating that one of the parties had won and would form either
a majority or a minority government.
We presented the same information—the entire news articles—to all subjects in a given condition.
We did nothing to highlight or emphasize the policy content of the news stories, beyond the
methods of presentation in the original articles. All policy differences between parties, across
multiple issues, were designed to reflect consistent party positions on a single, left-right
dimension—a largely accurate portrayal of the Canadian parties.
This use of simulated campaign materials that closely resemble real media coverage has significant
implications for the study. On one hand, it should promote greater engagement by subjects and
lead them to respond to the simulated campaign much as they would to a real-world campaign.
Thinking of the campaign as real or potentially real should lead them to care more about what
happens. On the other hand, this design dilutes the influence of our treatments by exposing
subjects to extensive material that is identical across treatments—namely, the ordinary, non-policy
content of campaign reporting. As with typical news coverage, our campaign news stories included
locations, logistics, and activities of particular events (ribbon-cuttings, barbecues, rallies); estimates
of attendance; names of politicians; their declarations on non-controversial topics; and sundry
remarks by ordinary citizens, interest groups, and local notables, among other things. The impact
of policy positions depends, as in real-world politics, on the efforts and ability of people to notice
them, amid the complexities of concrete issues and the cacophony of a campaign.
Given our experimental design, our concerns in fact are the opposite of those frequently considered
in political psychology—where the artificial treatments are often so strong as to have dubious
10
external validity. Here, our belief is that the simulated campaign is sufficiently realistic, with so
much information distracting from the treatments, that observing effects from a simulated
campaign to general political attitudes like efficacy would be strong evidence that these attitudes
are indeed affected by the features of party systems.
Our efforts in constructing this design were apparently successful. We demonstrate below that our
subjects engaged quite enthusiastically and conscientiously with the campaign materials. In
addition, we show that the policy positions in our news stories succeeded in spreading the parties
fairly neatly across a left-right dimension. In all, we think there are strong grounds to suppose that
treatment effects in our experiments will also occur in real-world campaigns.
EXPERIMENTAL PROTOCOL
Data is from completed experiments with a total of 232 undergraduate student subjects who took
one of several political science courses at UBC in Vancouver, Canada and participated in
experiments in exchange for course credit in 2015. Working at lab computers running Qualtrics
web software, each subject completed a pre-treatment questionnaire, including demographics and
general political attitudes including measures of issue concern and salience. Subjects were then
given a preamble, explaining that they would read news articles about an election campaign in
Manitoba, and later would be asked some questions about it, and would be able to vote for their
preferred party.
Figure 1 - Protocol Summary
1. Pre-treatment questionnaire
2. Subjects experience simulated Manitoba campaign: 10 articles
Eight articles in random order; two at end; click to advance
3. Post-treatment questionnaire: policy, satisfaction, information.
11
Figure 2
Condition
Left/Enviro
Three party (3p),
no Environment
Party Policy Locations
Right/non-Enviro
NDP
Liberal
Conservative
Three party,
Environment with
Environmental
Groups (3pA)
Interest
Groups
NDP
Liberal
Conservative
Four party,
Environment with
Green Party (4p)
Green
NDP
Liberal
Conservative
Subjects read a series of news articles corresponding to their randomly assigned campaign. (The
articles can be viewed in the online appendix.) A critical feature of the development of these
materials was the formulation of policy positions reflecting the appropriate ideological locations of
the parties for the given treatment. For example, in the four-party treatment, each article needed
an extreme left (Green Party), centre-left (New Democrat), centre-right (Liberal), and extreme right
(Conservative) policy position.
Depending on the condition to which a subject is randomly assigned, we present one of three kinds
of campaigns:
1. a four-party campaign (4p) with the Conservative, Liberal, NDP, and Green parties taking policy
positions that correspond to their real-world locations (see below for validation)
12
2. a three-party campaign with only the Conservatives, Liberals, and NDP but with various interest
groups, especially environmental ones, expressing the same points of view – usually in the same
language – as the Green party in the four-party campaign
3. a three-party campaign (3p) where environmental issues are not discussed and the two
explicitly environmental articles are replaced with two articles on the issue of support for
culture and the arts
(For a schematic representation of the three campaign conditions, see Figure 2, above).
Figure 3 - Example Campaign Article
A complication in constructing the newspaper stories was that going from three to four parties
forced us to choose among three options: making the articles longer, reducing the amount of
information on policy for each party, or devoting a larger proportion of the space to policy and less
13
to other aspects of the campaign (logistics of events, and the like, as noted above). In the end we
opted mostly for larger proportion of non-policy content in the three-party articles.
After reading the articles and voting, subjects completed a post-treatment questionnaire in which
we measured our dependent variables and measured attention to the campaign and the success of
our manipulations. Subjects were assigned to one of four election results stories in either a threeor a four-party version corresponding to their campaign condition. These four results were a
Liberal majority, Liberal minority, NDP majority, and NDP minority. In the minority cases, the
articles emphasized that the government would have to cooperate with other parties, while in the
majority cases the article stated that the government could ignore other parties.
Although we have collected dependent measures for a wide variety of potential effects (e.g.
subjects’ own policy positions, satisfaction and efficacy, information, and correct voting), the
current paper looks only at satisfaction with the election and a measure of voice that has been used
for decades as the central survey measure of “external efficacy”. Subjects also completed a battery
of information items about events and positions in the campaign.
MANIPULATION CHECKS
To generalize from our findings to the effects of party systems in real-world election campaigns, we
must have confidence that subjects attended to the series of news articles in a manner broadly
comparable to that of actual voters in real elections. We must check that our manipulations created
treatments.
Our key manipulation is an attempt to treat one set of subjects with a 3-party campaign with no
discussion of the environment and instead a discussion of funding for arts and culture. After the
campaign we presented subjects with a screen showing 16 issues and asked them to click the issue
if it was discussed in the campaign. By this measure, the manipulations were extremely successful.
14
For all issues that were not varied by condition there are no significant differences across a
comparison of the 3p vs 3pA and 4p conditions. For Arts and Culture, 88% of those in the 3p
condition said it was discussed, as compared with just 1% in the environment conditions. In the
environment conditions, by contrast, 91% said the environment was discussed, as compared with
24% in the non-environment condition. For climate change the difference was 61% to 8%. Based
on these variables, we defined a variable indicating successful treatment, which identified 88% as
being correctly treated.
Second, we used two screener questions as recommended by Berinksy and his colleagues (2014).
These were long-winded questions asking the respondent to ignore the actual question that
appeared at the end. Three-quarters of subjects passed both tests, and only 5% failed both. We
conducted analyses both including and excluding the subjects who failed either and both but found
no differences in our conclusions.
Third, we wanted to know if subjects had noticed the election result and particularly the majority
or minority results. We asked at the end how much influence each party would have on policies
until the next election, measured on a 1-4 scale. The mean Liberal influence across the Liberal
majority, Liberal minority, NDP minority, and NDP majority conditions was 3.4, 3.1, 2.5, and 2.3 and
almost exactly the reverse for the NDP, with the Conservatives having more influence in both
minority conditions, and the Greens estimated to have much more influence in the NDP minority
condition than any other. Clearly, these student subjects were in study mode and noticed, in the
last of ten full-length newspaper articles, which party had won.
And that brings us to pointing out that subjects engaged at a gut level with the campaign, as a way
of showing that they took the treatment of the campaign in general terms. We asked subjects: “How
do you think you would feel if you had been through this campaign and gone out and voted for
real?”, 14% said “very satified”, 51% “somewhat satisfied”, 31% “somewhat dissatisfied”, and 9%
15
“very dissatisfied”. Among the “winners” – those who voted for or identified with the party that
they saw having won the election – only 14% were on the dissatisfied side, while among losers fully
51% were dissatisfied.
We are confident that the subjects took the treatment.
VALIDATION OF POLICY POSITIONS
Figure 4
Mean Party Positions Centred by Article -- Total of Both Questions
Crime1
Crime2
Env1
Env2
MW1
MW2
Workfare1
Workfare2
Debate
Summary
-4
-2
Green
0
NDP
Liberal
2
4
Conservative
Ratings by New Subjects with Fake Party Names
For the sake of internal validity and our expectations of treatment effects, we also must have
confidence that our articles are realistic in terms of party platforms and ideological positions. To
assess perceived policy positions, we conducted a calibration study on a separate set of subjects and
a slightly modified version of the articles. These subjects read our eight main four-party campaign
articles with the party names replaced by uninformative, fake names. Subjects were asked to read
one of the articles to determine the position of just one of the fake-name parties on the policy issue
covered in the article. For the debate and campaign summary articles, subjects were asked to place
the fake-name parties on the zero-to-ten left-right ideology scale. Subjects in this study read either
16
four or five articles, identifying the position of a different party each time. By aggregating over
subjects, we derived average positions of each fake party—then translated back to the real
parties—on each of the ten articles. Results from this study reassure us that the campaign news is
highly naturalistic in terms of the party system.11 Figure 3 shows the mean placement of each party,
centred for each article. There is some variation in the average placements of parties across the 10
articles, but in total the parties’ positions are arrayed as we expected from the Greens’ policies on
the left to the Conservatives’ policies on the Right. For example, the policy positions in the end-ofcampaign summary article were placed at 3.7 (Greens), 4.6 (NDP), 5.2 (Liberals), and 6.4
(Conservatives). Not surprisingly, then, after the campaign when subjects were asked to place the
parties on the same left-right scale, the means were 2.97 (Greens), 3.52 (NDP), 4.75 (Liberals), and
6.95 (Conservatives) with distributions depicted in Figure 5.
Figure 5 – Subjects’ Placement of Parties on Left-Right Scale
Parties Placed on Left-Right Ideology Scale by Subjects
0
2
Green
4
6
Left - Right Ideology
NDP
Liberal
8
10
Conservative
Kernel Density Plots, bandwith 0.8
11 See also the discussion in Lau and Redlawsk , How Voters Decide: Information Processing in Election Campaigns.
17
Unfortunately this global success in creating a realistic campaign obscures the fact that the
environment articles in particular do not discriminate the parties very well. We will return to this
problem after presenting the results.
Summary
This experiment exposed subjects, in a crossed randomization, to three different campaigns and
four different campaign results. Environmental issues are discussed in the campaign either not at
all, by three traditional parties plus interest groups, or by four parties including the Greens. The
government formed after the election is either a Liberal or NDP majority that is described as being
able to work on its own policy priorities without consultation, or a Liberal or NDP minority that will
require collaboration with other parties.
HYPOTHESES
We express our hypotheses and expectations for treatment effects for these two manipulations as
independent, though there is always the possibility that the effects will be interactive.
Our dependent variables measure the degree to which subjects think their voice is heard in politics,
how well represented they feel, whether politics in Manitoba would be addressing their concerns,
and their general satisfaction with democracy in Canada. Our expectations across all these variables
are, in a nutshell, that when subjects hear “their issues” discussed, they will be more positive about
democracy and representation than when their issues are not discussed.
H1: Feelings of Voice and Satisfaction will be increasing among environmentally-conscious subjects
across the three conditions: 3p, 3pA, 4p; and decreasing or constant across these conditions for
subjects who do not think the environment is an important issue.
18
H1a: The effect in H1 distinguishing 3pA from 4p conditions will be conditional on feelings toward
the Green party.
H1b: If there are enough subjects who think Arts and Culture are important, their feelings of voice
should be marginally higher in the 3p condition than the other two conditions.
H2: Feelings of satisfaction but not voice will be higher among “winners” than “losers” and the
effect will be stronger or only visible when the winning party forms a majority.
RESULTS
In short, we found no treatment effects of the three campaigns (3p, 3pA, 4p) whatsoever.
Our analysis looked for conditional effects moderated by partisanship, feelings about the Green
party, attitudes on greenhouse gas policy, pre-treatment feelings of representedness, minutes in the
experiment, correctness on policy and issue recall questions, and general political attention and
information. And we looked for an Arts and Culture effect as well (H1b).
Nothing.
There was certainly no treatment effect on the voice measure, as we had found robustly in previous
experiments where subjects experienced two-party or four-party campaigns with different degrees
of policy extremity. We do not think that the measurement of voice is particularly problematic, as
we have construct validity from observing weaker feelings of voice in politics among those at either
end of the self-reported left-right scale, as seen in Figure 6.
19
Figure 6 – Voice by Ideology
1
0
.5
Voice
1.5
2
Voice by Ideology
0
1
2
3
4
5
6
7
Subject Self-Report Left-Right
8
9
10
n=212
But… we did find an interesting effect of subject behaviour in the experiment on the measures of
voice and, although weaker, on the measures asking if Manitoba politics would be addressing the
subjects’ issues as well as satisfaction with te campaign. It involves a time-in-campaign effect
among losers of the election. We admit that this sounds a bit bizarre, but the effect is clear in the
data: Among those whose preferred party did not win the election, the longer the respondent spent
in the lab reading the campaign materials, the more likely she is to disagree with the statement
“people like me have no say in politics”; while among those who ‘won’ the election, there is no such
effect. We depict this in the left panel of Figure 6. And this holds even when we control for pretreatment measures of interest, knowledge, and representedness. Now, we admit that this might be
the one significant interactive relationship among the many we examined. And it is not a true
treatment effect, although we would point out that the minutes in the campaign is prior to the
measurement of voice. But the fact that this is theoretically expected in line with our previous
experiments’ findings and that the same relationship holds, although more weakly for the
20
Important Issues and Satisfaction-with-the-Campaign variables, gives us confidence that the effect
is real.
Further work is required to determine whether these results might well be produced by learning
about the richness of election campaign discourse. It is suggestive that this effect is twice as strong
among those with lower-than-median knowledge and twice as strong among those who could recall
the issues discussed in the campaign more accurately. And even more suggestive that there is a
robust positive relationship between how much the subject took in during the campaign (measured
by party promise questions) and both Voice and Important Issues, even controlling for prior
political knowledge and feelings of representedness (see Figure 6). A subsequent experiment could
manipulate the length of time reading to establish the causal effect more confidently.
Concerning the winner-loser hypothesis (H2), although we observed a winner-loser gap in subjects’
imagined response to the campaign, there is no gap whatsoever in overall democratic satisfaction
from winning and losing this election. So although subjects could imagine themselves satisfied or
dissatisfied by the experience of the campaign, the experimental campaign did not affect their
Figure 7 – Voice by Engagement in Experiment
Voice by Minutes in Experiment
10
20
30
40
Minutes in Experiment
winner=0
50
60
1
1
1.5
1.5
Voice
Voice
2
2
2.5
2.5
Voice by Recall of Party Promises
0
winner=1
20
40
60
Grade on Promise Recall Test
80
n=174, controlling for winner, knowledge, representedness, minutes in exp
21
100
overall satisfaction, just as we had found in previous experiments.
NEXT STEPS FOR THE RESEARCH DESIGN
Obviously, a failure to find the predicted effect in a small student sample with relatively positive
feelings about their voice and their democracy is no reason to immediately abandon the theory.
Indeed, we had intended to test the theory in a more targetted way by running the experiment on a
captive audience of members of environmental organizations. The logic was that because we could
not easily design campaigns with and without each subject’s particular salient issues and then
randomize these after a pre-treatment query about issue concern, instead we would create an
environment and no-environment campaign and test for effects on voice and satisfaction among
people who we were certain cared deeply about the environment and would have good reason to
think that environmental concerns were not adequately represented in normal political and
especially election discourse. Our next step is to carry out an experiment in this fashion. And
indeed we could do so on the other side of the left-right or old-new politics continuums by creating
a campaign that included articles on loosening regulation of business or on de-nationalizing
industries, or whatever.
But we suspect another reason why we found no treatment effects. On reflection, we admit that the
treatments are very weak in the sense that they only minimally discussed the big environmental
issues and the depth of the discourse was likely rather disappointing to real environmentalists. The
discussion included Kyoto carbon emissions targets but also made mention of lake pollution, tax
rebates for fuel-efficient vehicles, parkland, and public transit. In the two conditions with the
environment discussed, the discourse could be said to be rather scattered and not particularly
encouraging for environmentalists to hear. As we saw above in figure 3, our calibration study
found that in one of the environment stories there is little perceived policy differentiation among
the four parties. In both, the Green party plays a rather acerbic, oppositional role rather than
22
sounding like it owns the issue and would show real leadership if it played a role in a government.
Therefore, in subsequent experiments we will create more sophisticated, positive discussions of the
environment and use them, as well as the existing ones (randomized), as treatments. We should be
able to do this with the power available to us with a pool of 600-1000 environmentalist subjects.
CONCLUSION
It is possible, nevertheless, that citizens’ attitudes about their voice and their democracy are not
affected by the discussion or lack of it, around issues about which citizens’ are concerned. It may be
that only a small subset of disaffected non-partisans are susceptible to this kind of effect. Partisans
of the major parties, in places like Canada anyway, are used to their voices being heard as their
party forms the government at either the federal or provincial levels at least once in a while. And
many citizens are satisfied enough that their voices are heard, so an experimental treatment like
this is unlikely to have an effect on them, much like consensus democracy is thought to have a
positive effect among habitual losers rather than at-least-sometimes-winners (Curini, Memoli, and
Jou 2011).
We are not ready to give up on an experimental demonstration of such an effect, however. The
theoretical rationale for expecting the effect is just too powerful. Feelings of having a political voice
are perhaps the most important determinant of a citizen’s judgment of democracy beyond its ability
to produce acceptable social and economic outputs (Dahlberg and Holmberg 2014). And surely
having a say in government is powerfully affected by hearing one’s issues discussed and one’s
positions put forward in the mass media by politicians and governments and parties and experts.
Can we continue to provide experimental evidence for this?
23