business risk and audit risk - University of Wisconsin

BUSINESS RISK AND AUDIT RISK: AN INTEGRATED MODEL
WITH EXPERIMENTAL BOUNDARY TEST
by
Adam M. Vitalis
A dissertation submitted in partial fulfillment of
the requirements for the degree of
Doctor of Philosophy
(Business)
at the
UNIVERSITY OF WISCONSIN–MADISON
2012
Date of final oral examination: May 15, 2012
The dissertation is approved by the following members of the Final Oral Committee:
Brian Mayhew, Associate Professor, Accounting and Information Systems
Mark Browne, Professor, Actuarial Science, Risk Management and Insurance
Ella Mae Matsumura, Professor, Accounting and Information Systems
Colleen Moore, Professor, Department of Psychology
Larry Rittenberg, Professor Emeritus, Accounting and Information Systems
© Copyright by Adam M. Vitalis 2012
All Rights Reserved
i
Acknowledgements
I would like to thank Brian Mayhew, my dissertation committee chair. His valuable
guidance, mentorship, and friendship have been instrumental to my success as a doctoral student.
I also thank the members of my dissertation committee: Mark Browne, Ella Mae Matsumura,
Colleen Moore, and Larry Rittenberg. Beyond providing valuable comments and suggestions to
improve the quality of this thesis, they have provided friendship and guidance throughout my
tenure as a Ph.D. student. I am grateful for their insights as well as the time and effort they
invested for my benefit. I also thank Laura Swenson, Jodi Gissel, and Tracie Majors for the
feedback they provided on this project. I appreciate the valuable comments of workshop
participants at the University of Wisconsin–Madison. I thank the University of Wisconsin
Business School and Department of Accounting for their financial support. I also thank the Ernst
and Young Foundation for funding the laboratory in which I conducted this research.
I also owe many thanks to my wife and family. It was my wife, Jessica, who opened my
eyes to pursuing my Ph.D., a fact I am sure she has regretted many times in the past five years.
However, she has done so with grace and patience as I worked to achieve this goal. For that and
all the sacrifices she has endured, I am grateful. I also thank my daughters, Jaiden and Sienna,
who remind me constantly to stay humble and make time for family. For without my family’s
love and support, this thesis would not have been possible.
ii
BUSINESS RISK AND AUDIT RISK, INTEGRATED MODEL WITH
EXPERIMENTAL BOUNDARY TEST
Table of Contents
Page
CHAPTER ONE
Introduction
1
CHAPTER TWO
Business Risk and Audit Risk: Proposed Integrated Model and Suggestions
for Future Research
1. Audit Risk Model
1.1 Background
1.2 Audit Risk Model Criticisms
8
2. Business Risk Audit Approach
2.1 Emerging Approach
2.2 Account vs. Business Level Focus Examples
13
3. Integrating the Audit Risk and Business Risk Models
3.1 Need for Audit Risk and Business Risk Model Integration
3.2 Proposed Integrated Model
20
4. Selected Business Risk Audit Literature and Suggestions for Future Research
4.1 General Business Risk Audit Approach Literature
4.2 Business Level Risk Assessment
4.3 Audit Evidence Collection: Application of the Risk Assessment to Risk of
Material Misstatement
4.4 Feedback Loops
4.5 Summary and Experimental Boundary Test Discussion
31
CHAPTER THREE
Experimental Boundary Test
1. Literature Review and Hypothesis Development
1.1 Risk Assessment Process
1.2 Cue Processing
1.3 Cue Costs
45
iii
2. Research Design and Methods
2.1 Participants
2.2 Design Background
2.3 Data Sets
2.4 Risk Cues
2.5 Practice Periods
2.6 Experimental Procedures
56
3. Experimental Results
3.1 Descriptive Statistics
3.2 Hypothesis One Results
3.3 Hypothesis Two (a & b) Results
68
CHAPTER FOUR
Supplementary Experimental Sessions
1. Overview
88
2. Matched Periods
2.1 Data Series Discussion
89
3. Independence of Periods
92
4. Results
4.1 Demographics
4.2 Descriptive Statistics
4.3 Test One Results
4.4 Test Two Results
4.5 Additional Tests
93
5. Discussion
112
6. Professional Skepticism Measure
114
CHAPTER FIVE
Conclusion
119
REFERENCE LIST
124
APPENDICES
Appendix 1: Experimental Instructions
130
Appendix 2: Screen Shots
137
Appendix 3: Experimental Earnings Discussion
141
Appendix 4: Equilibrium Payout Summary
143
Appendix 5: Additional Experimental Sessions Instructions
148
iv
Illustrations
Page
FIGURE 1
The Audit Risk Model
12
FIGURE 2
Proposed Business Risk/Audit Risk Model
25
FIGURE 3
The Risk Assessment Process
47
FIGURE 4
Experimental Design Overview
60
FIGURE 5
Distribution/Cue Patterns
64
FIGURE 6
Cue Selections
70
FIGURE 7
Visual Representation of Experimental Tests
74
FIGURE 8
Matched Periods Subsequent to No-error Series
76
FIGURE 9
Graphical Matched Period Results
77
FIGURE 10
No-Error Series Manipulation between PATTERN and NO-PATTERN
91
FIGURE 11
Additional Sessions Cue Selections
97
FIGURE 12
Periods Subsequent to 25 Treatment Periods
100
FIGURE 13
Difference Between PATTERN and NO-PATTERN, by First and
Second Series Around to 25 Treatment Periods
108
FIGURE 14
Professional Skepticism Values by Period, Cues Selected
116
v
FIGURE 15
Cues Selected by Low, Medium, High Skepticism Groupings, Cues Selected
116
FIGURE 16
Professional Skepticism Values by Period, Error Choice
118
FIGURE 17
Cues Selected by Low, Medium, High Skepticism Groupings, Error Choice
118
vi
Tables
Page
TABLE 1
Participant Demographics
58
TABLE 2
Hypothesis 1 Results: Experience Influence on Risk of Error Prediction
75
TABLE 3
Hypothesis 2a Results: Risk Cue Selection
81
TABLE 4
Hypothesis 2b Results: Cue Cost Interaction with Risk of Error Prediction
85
TABLE 5
Additional Sessions Participant Demographics
95
TABLE 6
Additional Sessions Test 2 Results
101
TABLE 7
Additional Sessions Test 2 Results with Control Variables
105
vii
BUSINESS RISK AND AUDIT RISK: AN INTEGRATED MODEL WITH
EXPERIMENTAL BOUNDARY TEST
Abstract
My thesis proposes the first model for how the business risk audit intersects with the audit risk model. In
addition, I explore one possible limitation of the auditor’s ability to generate risk assessments.
Specifically, I examine whether individuals, when provided with the same risk-cue information, are
influenced (biased) in subsequent risk-assessment decisions by positive experience.
New auditing standards require a formal evaluation and integration of business-level risks into the audit
using the business-level risk assessment. Theory suggests the integration of a business risk audit approach
within the framework of the traditional audit risk model in this top-down risk-based audit approach allows
auditors to better align audit resources to those areas of higher risks. However, to my knowledge, there is
little or no discussion of how these models formally integrate. Rather, auditors simply assume their
intersection. My thesis explores and addresses the challenge of developing a unified model and proposes
such a model. In addition, I review current business risk-audit research, map applicable business riskrelated research into key areas of my proposed model, and provide future research suggestions.
Finally, I explore one possible limitation in an auditor’s ability to generate risk assessments. In particular,
I study whether individuals bias toward positive experience. Risk-based audit theory argues that clientspecific experience increases an auditor’s ability to accurately assess client risks. However, prior research
in psychology suggests otherwise. My design mimics the audit setting, in which auditors gain clientspecific experience while working under continuous pressure to increase audit efficiency. Using this
design, I examine the consequences of prior, positive client-specific experience and efficiency pressure,
which is represented by costly risk cues, on the ability to generate a risk assessment. Further, I frame risk
cues with two business risks to mimic a business risk setting.
My results indicate that some individuals bias toward prior experience, but that efficiency pressures do
not interact to further impact this bias. From an audit perspective, my results indicate a bias in certain
auditors toward positive client-specific experience during the risk-assessment process. Together these
results suggest that auditors should be mindful whether this experience bias impedes the risk-assessment
process.
1
CHAPTER ONE
Introduction
The new Public Company Accounting Oversight Board (PCAOB) suite of risk-related
auditing standards (AS) 8 through 15 (PCAOB 2010) as well as international auditing standards
(ISA) 315 (IAASB 2009a) and ISA 220 (IAASB 2009b) require a formal evaluation and
integration of business-level risks into the audit process through the business-level risk
assessment. Requiring the business-level risk assessment adds a necessary combination of action
and theory. The traditional audit risk model historically allows auditors to increase substantive
testing and minimize the risk-assessment process. These new standards require auditors to
complete a risk assessment. Formally integrating a business-level risk assessment into a topdown risk-based audit approach holds great intuitive appeal, and proponents argue that this
approach is more robust and can better inform the audit efforts (Bell et al. 1997; Bell et al. 2005;
Bell and Solomon 2002; Lemon et al. 2000). Through identifying and evaluating high-level
business risks during their business-level risk assessment, auditors are better able to align audit
resources to those areas of higher risk, thereby creating a more effective audit approach. Thus,
coupling the risk-assessment requirement with a top-down risk-based approach adds a valuable
dimension to the audit.
Utilizing this business risk approach within the framework of the traditional audit risk
model is emerging practice. However, there is little or no guidance on how these models
intersect. Rather, auditors simply assume an intersection of business risk and audit risk, and that
the business risk audit complements and supplements the audit risk model.
2
From a research perspective, an integrated model provides a foundation for researchers to
examine emerging, fragmented business risk-audit research holistically within the overall audit
process. This better equips researchers to examine the disparate research within the overall
context of the model in an effort to determine intersecting research and areas of focus while
providing a basis to offer insight into increasing audit effectiveness. My first contribution in this
thesis is to provide a formal model for how the business risk audit intersects with the more
traditional audit risk model to create this more holistic approach.
In Chapter Two, I argue that the business risk model creates a lens or cognitive process
that encompass three broad additions to the audit risk model: business risk assessment, auditevidence collection, and feedback loop. First, the business risk model adds a holistic businesslevel risk assessment, including internal controls. Second, the business risk model adds the
application of the business-level risk assessment to risk of material misstatement through the
overall financial statements, assertions, and other organizational traits, to financial statement
accounts and classes of accounts. This process influences the nature, extent, and timing of
evidence collection. Finally, the business risk model incorporates feedback throughout the audit
to both update the risk assessment and provide a more robust triangulation of audit evidence
throughout the audit in a feedback loop. After outlining my integrated model, I include and map
onto this model current research from the business risk audit literature and include suggestions
for future research.
While the new risk-related audit standards require a formal evaluation and integration of
business-level risks into the audit (PCAOB 2010), little research examines the boundaries of
auditor effectiveness in assessing risk. After establishing an integrated audit risk and business
3
risk model, I then explore one possible boundary of the ability to generate an accurate risk
assessment. Specifically, I examine whether individuals, when provided with the same risk-cue
information, are influenced (biased) in subsequent risk-assessment decisions by positive
experience. Further, to frame the risk cues in a business risk setting, I include risk cue
descriptions that include two business risk areas.
Auditors must identify, evaluate and assess relevant, current-period risk cues for effective
integration into risk assessment. In addition, auditors gain client-specific experience that builds
relevant knowledge through ongoing engagements (Bell et al. 1997; Bell et al. 2005; Bell and
Solomon 2002). This client-specific knowledge provides auditors a more refined lens with which
to filter information, including risk cues, during the risk-assessment process. However, it is also
possible that prior positive client-specific experiences create a bias whereby auditors forego the
process of identifying current risk cues and simply rely on prior client-specific experience.
In Chapter Three, I include an experimental study to explore this potential conflict
between experience and current descriptive cues. I theorize that a long series of positive clientspecific experience dilutes the impact of the subsequent disconfirming evidence. This directs
auditors toward a belief that a lengthy series of positive client-specific experiences legitimately
predicts current-period results, even when faced with disconfirming descriptive risk cues.
Specifically, my first hypothesis explores whether participant risk-assessment decisions are
negatively influenced by client-specific experience by measuring whether participants choose
no-error risk-assessment predictions to a greater extent after a long series of no-error results.
4
In addition, by manipulating costly cue conditions as a proxy for the effort of generating
a risk assessment, I examine the impact efficiency pressures have on the risk-assessment process.
Monetizing cue costs increases the salience of cue-cost decisions on participants. Logic suggests
that increasing risk-cue costs leads to a decrease in the number of cues selected. However, cueprocessing research suggests that individuals become more effective at cue selection without
losing effectiveness in light of rising cue costs (Rakow et al. 2005). In an audit setting, this
would translate into auditors becoming more effective at creating a risk assessments despite
increased efficiency pressures.
My second set of hypotheses examines the impact of costly cues as a proxy for auditefficiency pressure on an auditor’s risk-assessment decisions. Specifically, I examine whether
increased cue costs (efficiency pressure) (1) lead to a decrease in participants’ ability to select
the number of cues that maximize their expected earnings in each period, and (2) interact to
diminish the ability to generate risk assessments.
To mimic the natural audit setting, whereby auditors gain client-specific experience and
continually face pressure to increase audit efficiency, I cross experience (NO-PATTERN or
PATTERN) with cue cost (LOW or HIGH) in a 2 x 2 between-subject design. I operationalize
positive client-specific experience with a long series of no-error period results. In each period,
participants select 0, 1, or 2 costly (LOW vs. HIGH) risk cues, which they use to generate a
prediction (risk assessment) as to whether an error will occur in the current-period. The system
then provides feedback regarding the accuracy of participants’ error risk prediction each period.
Participants repeat this task in each period of the session, thus mimicking the general flow of a
typical audit.
5
To examine my primary hypothesis, I measure negative influence on risk-assessment
decisions by looking at whether participants choose no-error risk-assessment predictions more
after the long no-error result series. Results indicate that experiencing a long series of positive
(no-error) periods influences individuals, but that this bias dissipates after six matched periods
following the no-error treatment periods.
To examine my second hypotheses, I first look at whether efficiency pressures (higher
cue costs) lead participants to choose risk cues that do not maximize their expected earnings in
each period. I further examine whether cue costs influences risk-assessment decisions. In order
to accomplish this, I use a measure of whether subjects select the number of cues that maximize
their expected earnings in each period based on the experimental payout structure and cue
accuracy / error result assumptions. I find that participants are less effective in selecting risk cues
under higher efficiency pressure. However, varying cue costs does not seem to impact riskassessment decisions differently between cue-cost conditions. My results differ from prior
research, which finds that individuals improve at selecting cues with higher cue costs. In
addition, I do not find support that efficiency pressures in the risk-assessment process harm or
impede the ability of individuals to generate accurate risk assessments.
I also include a set of supplemental sessions designed to examine the persistence of the
experience bias results found in the main sessions. Results of these supplemental sessions do not
support statistically significant inferences; however, I do find directionally consistent results with
tests including various control variables. It is likely that this design does not provide enough
contrast between conditions to observe a robust effect. However, it is also possible that
experience bias is more isolated than believed. In all sessions, while the differences are generally
6
statistically significant, the majority of subjects selected the correct error value in both conditions
with small differences between conditions. Thus, taken with the marginally significant results in
the supplemental sessions, the difference in participant error choices suggest that the experience
bias, while significant, is isolated to a small number of individuals. However, as I do find results
in these supplemental sessions that are directionally consistent with those in the main sessions, it
suggests that experience biases certain individuals.
In conclusion, my thesis presents the first academic model for how the business risk audit
intersects with the more traditional audit risk model. With this model, I include current research
generally fitting under the category of business risk audit, map applicable business risk-related
research into key areas of my model, and include suggestions for future research. I then explore
one possible impediment to creating effective risk assessments by exploring whether individuals,
when provided with the same risk-cue information, are influenced (biased) by prior positive
experience in subsequent risk assessment decisions.
My results indicate that some participants bias toward prior experience when generating a
current-period risk assessment. In addition, from an efficiency perspective, my results indicate
that individuals select risk cues that maximize their expected earnings less with higher efficiency
pressure than without higher efficiency pressure, but that efficiency pressure does not negatively
influence risk-assessment decisions. My study provides initial insights and indicates that further
research into the boundaries of risk assessment is necessary.
This thesis proceeds as follows. In Chapter Two, I explore audit risk and business risk,
with a proposed integrated model and suggestions for future research. In Chapter Three, I build
7
on this model to experimentally examine experience with costly risk cues on risk-assessment
decisions. In Chapter Four, I explore the results of a supplemental experiment. Chapter Five
concludes.
8
CHAPTER TWO
Business Risk and Audit Risk: Proposed Integrated Model and Suggestions
for Future Research
Research into the top-down risk-based business risk audit approach is only beginning to
emerge. One contribution of my thesis is providing an integrated audit risk and business riskaudit model. Such a model provides a foundation for a holistic evaluation of emerging business
risk-audit research. With this model, researchers can better examine the disparate research within
the overall context of the model to determine intersecting research and areas of focus. Research
can better provide insight into how to use these combined approaches to increase audit
effectiveness.
In this chapter, I describe the need for an integrated audit risk and business risk model,
provide such a model, include current literature and, finally, offer suggestions for future
research. In Sections 1 and 2, I include brief backgrounds on the audit risk model and business
risk-audit approach. I demonstrate the need for an integrated model, which I include in Section 3.
Finally, in Section 4, I include current research generally fitting under the category of business
risk with suggestions for future research.
1. Audit Risk Model
This section provides a discussion on the audit risk model, including a brief summary of
the history and key attributes of the model, and outlines relevant criticisms of this approach.
9
1.1 Background
In December 1983, the Auditing Standards Board of the American Institute of Certified
Public Accountants (AICPA) issued Statement on Auditing Standard 47 (SFAS 47), “Audit Risk
and Materiality in Conducting an Audit.” The standard provides guidance on audit risk and
materiality and instituted the use of the audit risk model that provides a formal method to
evaluate audit risks (AICPA 1983). With SFAS 47, audit planning is a two-stage process that
includes a risk assessment and evidential planning. In the first stage, the auditor evaluates
important risk factors impacting the likelihood of material misstatements and then designates the
perceived level of risk in the audit. In the second stage, the auditor determines the nature, extent,
and timing of tests necessary to provide conclusions based on this perceived risk level. Together,
these two stages evaluate unique client risks and tailor evidence collection (Mock and Wright
1999). The standard anticipates that this process allows the auditor to “detect errors that he
believes could be large enough, individually or in the aggregate, to be quantitatively material to
the financial statements” (AICPA 1983).
Formally, the audit risk model is stated as: audit risk (“AR”) = inherent risk (“IR”) x
control risk (“CR”) x detection risk (“DR”), or: AR = IR x CR x DR; alternately, AR = risk of
material misstatement (“RMM”) x DR (where RMM = IR x CR), or: AR = RMM x DR.
In this model, audit risk is the probability that the auditor unknowingly issues an
unqualified opinion on financial statements that are materially misstated. Inherent risk is the
probability that material misstatements exist. Control risk is the probability that the internal
control system will fail to detect material misstatements. The risk of material misstatements is
10
the combination of inherent risk and control risk and relates to the risk that a material
misstatement exists in the financial statements. Finally, detection risk is the probability that audit
procedures will fail to identify material misstatements (PCAOB 2010).1
SFAS 47 observes that auditors can report appropriately after performing the audit in
accordance with generally accepted auditing standards, but still be “exposed to loss or injury to
his professional practice from litigation, adverse publicity, or other events arising in connection
with financial statements that he has examined and reported on” (AICPA 1983). This
“engagement risk” is not formally included in the audit risk model and is traditionally assessed
as part of the audit firm’s client acceptance and continuance procedures (Public Oversight Board
2000).
Following SFAS 47, the AICPA’s Professional Issues Task Force issued Practice Alert
94-3 (AICPA 1994). This document outlines three components making up engagement risk: the
entity’s business risk, the auditor’s business risk, and the auditor’s audit risk. They observe that
the “auditor’s business risk may be controlled in part through policies and procedures established
for deciding whether to accept or continue a client.” However, they define entity’s (client’s)
business risk as the “risk that the entity will not survive or will not be profitable.” In this way,
the entity’s business risk factor is focused on the client. In evaluating engagement risk, the
auditor’s audit risk dimension relates to the financial reporting of risk attributes, as outlined in
1
An original iteration of this equation form comes from the appendix in Statement on Auditing Standard 39 (SFAS
39), Audit Sampling. This standard provides a model for required audit sampling to obtain an acceptable level of
“ultimate risk” (AICPA 1981). While the audit risk model is in the same general form as the audit-sampling model
in SFAS 39, the variables relate to different attributes of the audit (audit sampling and overall audit risk). Therefore,
as this document is focused on overall audit risk, I refer to the model set forth in SFAS 47 throughout this document.
Further, while SFAS 47 has been superseded by the recent AS 8 through12 standards, it has still been maintained as
outlined in AS 8: “Audit risk is a function of the risk of material misstatement and detection risk” (PCAOB 2010).
11
SFAS 47, that might increase the chance that an auditor “may unknowingly fail to appropriately
modify his or her [audit] opinion” (AICPA 1994).
Through assessing engagement risk, auditors gain a deeper understanding of the client’s
overall risk profile. Engagement risk both supports the decision whether to accept or maintain a
client relationship, as well as provides a basis for setting audit risk. 2
Through this engagement risk assessment and planning stage, the auditor obtains
necessary client background to define the audit risk level as defined in the audit risk model. Once
audit risk is set, the auditor assesses control and inherent risks for each account or class of
accounts in the financial statements and determines the resulting detection risk (AICPA 1983).
The auditor then plans the audit to determine the nature, timing and extent of auditing procedures
to specific account balances or class of transactions. This way, the auditor is able to focus on
collecting substantive, sampling-based evidence to achieve the required detection risk level, thus
reducing the risk of a material misstatement to a “negligible level” (as envisioned by Statement
on Auditing Standard (SFAS) 39 (AICPA 1981)) and provide an overall assessment as to
whether the financial statements are fairly presented.3 Figure 1 provides a graphical example of
the audit risk process.
2
For a useful discussion of engagement risk, see Rittenberg et al. (2012). The authors claim that engagement risk is
made up of those factors that affect the business risk and those factors affecting financial reporting risk (similar to
audit risk from Practice Alert 94-3).
3
SFAS 39 differentiates between sampling and nonsampling risk. Sampling risk relates to the possibility that
conclusions drawn for a selected sample might differ from those if the entire population was examined.
Nonsampling risk includes all aspects of the overall audit risk not encompassed in sampling risk. Nonsampling risk
might include internal control weaknesses, choosing sampling procedures which are inappropriate for the stated
objective, and failure to recognize errors in selected items (AICPA 1981).
12
FIGURE 1
The Audit Risk Model
In the simplest sense, the auditor obtains sufficient client background to set the
engagement and audit risk levels, and then assesses risk of material misstatement for each
material account or class of accounts. The resulting detection risk level influences the nature,
extent and timing of the fieldwork and audit evidence collection, which then drives the auditor’s
ability to issue an opinion.
1.2 Audit Risk Model Criticisms
While the audit risk model provides a risk-based approach, one recurring criticism of the
model is that “anecdotal and other evidence indicates that many (but by no means all) audits
13
continued to be performed using substantive testing approaches with little or no attention paid to
the results of the risk assessments called for by the model. 4 This phenomenon is perhaps
facilitated by the fact that the model permits ‘defaulting’ to an assumption that risks are at a
maximum level” (Public Oversight Board 2000). Because the audit risk model does not
specifically require a risk assessment, it effectively allows the auditor to increase substantive
tests at the account level without regard to a risk assessment that helps the auditor tailor the audit
approach to unique client risks. In fact, research suggests that audit work papers include only a
weak relation between audit programs and client risks. Further, audit programs change little over
time or across clients (Mock and Wright 1993; 1999; Waller 1993).
2. Business Risk Audit Approach
This section starts with a historical summary of the emerging business risk audit
approach and then discusses how the approach is envisioned to support the audit process.
2.1 Emerging Approach
Business risk-audit theory argues that a lack of focus on client business risk may have a
detrimental impact on audit quality (Bell et al. 2005; Peecher et al. 2007; Knechel 2007). In
response, another audit risk view has emerged over the past couple of decades (Lemon et al.
2000). In this view, auditors begin with a more holistic understanding of the overall client
organization and its environment so as to better align audit resources to those areas of higher
4
In 2006, the AICPA set forth an updated set of “Audit Risk” standards (SFAS 106-11), which, while reiterating the
requirement to complete a risk-assessment procedure including the entity and its environment that includes internal
controls, still provides that in “some cases, the auditor may determine that performing only substantive procedures is
appropriate for specific relevant assertions and risks. In those circumstances, the auditor may exclude the effect of
controls from the relevant risk assessment” (Auditing Standards Board (ASB) 2006a; Auditing Standards Board
(ASB) 2006b).
14
business risk (Knechel 2007). In this view, client business risk is “the risk that an entity’s
business objectives will not be attained as a result of the external and internal factors, pressures,
and forces brought to bear in the entity and, ultimately, the risk associated with the entity’s
survival and profitability” (Bell et al. 1997). This “business risk audit” approach differs from the
audit risk approach in that the auditor more fully engages in an understanding of the company’s
business strategies and barriers to achieving them. 5 Business risk audit “focuses upon the
modeling of business risk processes of the client’s company, and using this knowledge as the
basis for establishing financial statement risk and, accordingly, the main focus of subsequent
audit testing.” This process creates an environment in which the “client company’s strategy is
examined and tied to its business processes” (Robson et al. 2007).
Curtis and Turley (2007) argue that by identifying sources of business risk and evaluating
the systems to monitor and control those systems, auditors can decrease substantive testing. This
business risk audit approach changes the audit from one of testing “large volumes of low-level
transaction controls to high-level monitoring or supervisory controls.” Proponents argue that the
traditional balance-sheet/account-level approach to a financial statement-level (bottom-up) audit
focus “can inhibit the auditor’s development of the level of business understanding needed to
effectively judge financial-statement assertions” (Bell et al. 1997). Rather, a more holistic
approach, which allows the auditor a chance to more fully understand the client-business model,
“assists the auditor’s evaluation of the validity of the client’s accounting transactions, and of the
5
This holistic approach is called many different terms. Following the discussion in Lemon et al. (2000), I use the
term “business risk audit” to encompass audit approaches that consider client business (or strategic) risks in a
holistic, top-down risk-assessment process, including the requirements in current auditing standards. I realize that
the “strategic-systems audit” (SSA) methodology discussions might draw a distinction from this general term;
however, the SSA approach is founded on the holistic, top-down theory, and, as such, I believe the SSA is properly
included in the generally defined business risk audit. See Bell et al. (1997), Bell and Solomon (2002), and Bell et al.
(2005) for discussion on the SSA model.
15
financial statements taken as a whole” (Bell et al. 1997). By encompassing a holistic view of the
organization, the business risk assessment is more robust and can better inform the audit efforts
from a top-down approach rather than a bottom-up approach. Proponents argue that this creates a
more efficient and effective audit.6 In fact, business risk audit has moved beyond a theoretical
discussion and is becoming incorporated into auditing firm methodology (Lemon et al. 2000). In
addition, new auditing standards now require a more formal evaluation and integration of
business-level risks into the audit-planning process (including internal control testing and
analysis of high-risk accounts).7
In essence, the business risk audit approach requires the auditor to gain a broader level of
knowledge in the client’s business than in the traditional audit risk model. It is in this “forest,”
instead of “tree-by-tree,” view that auditors are able to build a mental model that accounts for a
realization that the whole entity may differ from the sum of its constituent parts (Bell and
Solomon 2002). By doing so, auditors are able to refine their lenses—their mental orientations or
perspectives—that influence their judgments and actions during the process of evaluating
business risk and assessing audit risk. Utilizing the business risk audit approach the auditor
decreases the chance of drawing incorrect inferences by losing sight of the fact that auditors
effectively audit the broader economic system by examining the strengths of its interrelationships
and interactions (Bell et al. 1997).
Bell et al. (2005) illustrates this concept with the following simple hypothetical example:
The lead partner on the audit of a Brazilian manufacturer understands that the company sells most of its
products in Mexican markets. While updating her understanding of the entity and its environment, she
6
7
See for example Bell et al. (2005); Bell and Solomon (2002); Lemon et al. (2000); and, Bell et al. (2008).
See ISA 315 (IAASB 2009a) and ISA 220 (IAASB 2009b), as well as the recently released PCAOB suite of riskrelated standards, AS 8 through 15 (PCAOB 2010).
16
learns that the Mexican economy is in deep recession. She naturally, then, expects that the entity’s sales
will have declined and their bad debt expense will have increased. She reviews the most recent quarter’s
financial results and observes that sales have grown in line with the forecasts the management had
provided earlier to the capital market and that the ratio of bad debt expense to sales remained constant.
Consequently, she assesses RMM for both sales and bad debt expense as high.
What exactly happened here? The auditor’s earlier mental model of the organization included the
understanding that the entity operated predominantly in Mexican markets. As the current audit
commenced, she set the objective—update my understanding of the entity and its environment to assess
RMM. While investigating critical components of the entity’s business model, as now required by ISA 315,
she obtained evidence through the news media and by inspecting relevant economic reports that the
Mexican economy was in a state of crisis. This new information was integrated into her mental model of the
organization. She then developed the expectation that this unfortunate turn of events would slow down
entity sales and that a higher percentage of customers would not be able to pay their accounts (i.e., she ran
her mental model to develop expectations about the financial consequences to the entity of the downturn in
the Mexican economy). Upon observing that the entity’s most recent quarterly financial reports presented a
significant growth in sales, and a stable percentage of bad debt expense, she became concerned that this
was too good to be true (i.e., she decided that these accounts had a high RMM). In light of high assessed
RMM, she will further manage DR by designing and applying additional responsive risk-assessment
procedures.
2.2 Account vs. Business Level Focus Examples
The Lincoln Savings and Loan case provides one example of how an account
(transaction)-level sampling focus without a proper holistic view of the business can lead to an
audit failure.8 Lincoln Savings and Loan illustrates the importance of focusing on business risks
instead of simply following a more traditionally focused substantive-evidence testing approach.
In the mid-1980s, the retail real estate markets in Arizona, including Phoenix, were
highly competitive. Lincoln Savings and Loan had recently expanded its business objectives to
include real estate, and was now required to compete with other developers in this increasingly
competitive market. As such, its financial success was heavily tied to demand. While there was a
8
See Erickson et al. (2000) for full discussion of Lincoln Savings and Loan case. Material (including quotations) in
this section taken from Erickson et al. (2000), except where noted.
17
consistent downward trend in the Arizona real estate market from mid-1986 through late 1987,
Lincoln Savings and Loan recognized substantial accounting profits on sales of undeveloped
land. Erickson et al. (2000) outlines transactions, a land sale and land trade of Hidden Valley
property, questioning these results. The nature of these transactions was such that the
transactions were made to parties that had ties to Lincoln Savings and Loan and likely did not
have economic ability to sustain the required payments outlined in the sales. Further, while the
Hidden Valley land was restricted to four-wheeled-drive vehicles, as there were no paved roads
in the area, Lincoln Savings and Loan was able to sell the Hidden Valley land for prices ranging
between $14,000 and $17,500 per acre for land purchased only a couple years earlier for an
average price of $3,100 per acre. The authors argue that these transactions generated over $80
million in arguably fraudulent pretax profits for Lincoln Savings and Loan.
According to Erickson et al. (2000), this was possible because the auditors examined
evidence in isolation and did not adequately understand the business environment of the client.
Therefore, their procedures were inadequate to identify that Lincoln Savings and Loan’s
financial results were simply “too good to be true” given the current state of the industry and
Lincoln Savings and Loan’s business. In this case, the auditors focused on the fact that the
transaction's structure complied technically with the requirements of SFAS No. 66. However, the
substance of the transactions did not make sense given: “(i) the condition of the Arizona real
estate market in 1987, (ii) the location of this property, (iii) the financial resources of the buyer,
and (iv) the lack of a legitimate business motivation for [the parties to] complete this
transaction.” The authors argue that the auditors would have realized Lincoln Savings and Loan
valued the land inconsistently in terms of overall industry trends had they gained an in-depth
18
understanding of the economic climate of the Arizona real estate industry. Because Lincoln
Savings and Loan’s auditors did not review this external industry trend information to
substantiate the substance of the transactions, they were unable to correctly identify that the
established transaction prices were not representative of economic value. In fact, during her
deposition, the audit principal acknowledged that not only did she not consider the Arizona real
estate environment, but also the audit team did not evaluate the business purpose of the real
estate transactions. In fact, the audit principal did not believe it was necessary for auditors to
obtain or use this business-level knowledge.
Erickson et al. (2000) argue that by applying an evaluation of the sale of undeveloped
land, the substance for Lincoln Savings and Loan’s main profit sources during this period,
through a more robust business understanding, the auditors would have evaluated the evidence
differently. By focusing on whether Lincoln Savings and Loan had met the technical
requirements of SFAS No. 66, without this more holistic view of the environment Lincoln
Savings and Loan faced, the auditors were unable to identify flaws in the evidence they relied
upon. Had the auditors utilized a business risk audit approach, they likely would have focused
more attention on the broader economic realities and increased their professional skepticism
about the reported revenues, which would have likely lead to different accounting treatment. In
this way, auditors could have identified the tests necessary to make an accurate determination as
to whether the financial statements were fairly presented (Bell et al. 1997; 2005). Instead, by
focusing on the more transactional approach, auditors established that the SFAS 66 rules were
mechanically appropriate, but in so doing, missed the overall business-level risks in the
transactions that ultimately lead to Lincoln Savings and Loan’s failure.
19
Another example comes from Peecher et al. (2007), whereby the authors consider a
hypothetical harvester and bottler of natural water with very high business growth. To meet their
increasing demand, the firm hires new factory employees but does not adequately train them to
“test and ensure that bottled water is sufficiently free of impurities such as perchlorate.
Perchlorate is a key ingredient of solid rocket fuel and a real risk factor for drinking water.” The
authors use accounts receivable and inventory assertions as their illustrative example. In this
example, the authors assume the client’s quality assurance testing for inventory does not identify
unsafe perchlorate levels in the delivered product. Thus, because the inventory is tainted, current
inventory and outstanding accounts receivable balances are impaired. An auditor focusing on a
physical examination of the inventory (or aging reports and related analysis regarding allowance
for doubtful accounts balances) to confirm the accuracy of the accounts receivable balances
would be able to confirm the inventory’s existence, but current balances would still be incorrect
given that the underlying value of the asset is impaired. Thus, the risk to the accuracy of the
valuation assertion in the financial statements, in this situation, is due to operational risk—one
that is not likely to be identified through traditional substantive testing.9
This case illustrates that a more holistic understanding of company risks can illuminate
broader organizational risks, which can have a direct impact on the financial statement
assertions. In this situation, valuation was not impacted by an incorrect transactional balance, but
rather by operational risks. Obtaining a fuller understanding of the organization provides the
9
One could also argue that the auditor could have included an independent sampling of the water to ensure that the
product’s value was not impaired. Either way, the risk valuation would be operational in nature.
20
necessary context to more fully understand the nature of the product and ask additional questions
related to training or quality testing, thus identifying the potential for impaired quality.
In both of these examples, the auditors correctly applied substantive testing related to the
financial statement assertions, yet still missed a material misstatement. By applying a more
holistic approach, including a fuller understanding of the business and risks, the auditors likely
would have identified additional risks that would have prevented errant financial statements.
3. Integrating the Audit Risk and Business Risk Models
The previous two sections of this chapter provide background on the audit risk and
business risk models. In the following section, I argue the need for an integrated model and
provide a proposed model of the same.
3.1 Need for Audit Risk and Business Risk Model Integration
While the business risk audit approach has intuitive appeal, research on the effectiveness
of business risk audit is limited.10 Theory suggests that business risk audit is “consistent with
generally accepted auditing standards in all material respects and thus with the audit risk model”
(Lemon et al. 2000). However, to my knowledge, no one has fully developed a unified model
that provides an understanding of a practical process for the auditor to implement the integration
of business-level risk-based information into the normal audit process (Rittenberg 1998).
The traditional audit risk model provides a method to integrate risks during audit
planning. However, the audit risk model has become more of a compliance exercise with risk of
10
I include a selection of current business risk audit-related research later in this chapter.
21
material misstatement set as high and audit effort left unadjusted for risk differences. With the
inclusion of a business risk audit focus, auditors should contemplate identifying and assessing
risks at the assertion and financial statement level, instead of simply the individual accountbalance or class-of-transactions level. As discussed earlier, the business risk audit framework
creates a more cognitively rich environment that changes the auditor’s lens from simply
examining transactional evidence in isolation to including those pieces of information in the
overall context of the client’s environment. In essence, it creates a broader lens for the overall
audit, one that the auditor would not likely generate without the addition of the business risk
audit framework. In this way, the business risk audit moves from a more transactional audit
process to one that examines information in a more holistic context.
This business risk audit approach meets the PCAOB goal to better “identify the areas of
greater risk, appropriately assess those risks, and design and perform further audit procedures to
address risks of material misstatements in the financial statements, whether due to error or fraud”
(PCAOB 2010). The inclusion of a business-level risk assessment, which flows from the
strategic business level of the organization through the financial statements and management
assertions to influencing the account level risk of material misstatement, moves the audit from a
more transactional approach to a more holistic, strategic approach. This approach is still
“primarily directed toward improving the auditor’s assessment and management of detection risk
and thus, audit risk” (Peecher et al. 2007). By combining business risk audit with the existing
audit risk approach, the new model balances the need for transactional testing with a demand that
the auditor must “’do the homework’ necessary to identify and to assess the risk of error or the
risk of fraud in the financial statements” (Harris 2009).
22
In addition to providing a more holistic audit risk assessment, business risk audit
proponents argue that all experiences during the audit refine client experience, which then
enriches the auditor’s knowledge base. The feedback loop generated in a business risk audit
provides a continuous updating of the risk assessment and auditor knowledge. As outlined in the
PCAOB’s new risk standards,
The auditor's assessment of the risks of material misstatement, including fraud risks, should
continue throughout the audit. When the auditor obtains audit evidence during the course of the
audit that contradicts the audit evidence on which the auditor originally based his or her risk
assessment, the auditor should revise the risk assessment and modify planned audit procedures or
perform additional procedures in response to the revised risk assessments (PCAOB 2010).
For example, the re-evaluation of the auditor's risk assessments could result in the identification
of relevant assertions or significant risks that were not identified previously and for which the
auditor should perform additional audit procedures (PCAOB 2010).
In this way, the feedback loop provides a method for “evaluation of its detailed
transactions from a grounding in knowledge about [its] larger systems context” (Bell et al.
1997).11
In addition to supporting a continuous update of the audit risk assessment and resulting
audit plan, knowledge enrichment provides the auditor a contextual lens, which provides for
more effective audit evidence analysis. For example, in the Lincoln Savings and Loan case, had
the auditor applied results of their detail testing against industry trends identified during the
business risk-assessment process, the auditor likely would have questioned why the results
seemed “too good to be true,” thus changing the nature of the testing to better understand the
makeup of client earnings. In such a way, the feedback loop would provide a manner to examine
11
See Bell et al. (1997), Bell and Solomon (2002), and Bell et al. (2005) for a more in depth discussion of the SSA
approach.
23
the overall economic system and provide a framework for adaptive behavior, which “might have
increased his level of skepticism about the reasonableness of the reported gains, and
consequently led to the development of more accurate expectations” (Bell et al. 1997). This
multi-viewed approach to evaluating audit evidence and the overall organizational risks allows
the auditor an opportunity to better triangulate audit evidence with information throughout the
audit, thereby increasing the overall effectiveness of the audit-evidence collection (Bell and
Solomon 2002; Bell et al. 2005).12 For example, had the Lincoln Savings and Loan auditors used
a business risk approach, their lens would have likely suggested a stronger focus on why anyone
would buy land at an inflated price, and that in turn would have highlighted questions regarding
that the financing of the land purchases, which were provided through a different branch of
Lincoln Savings and Loan.
3.2 Proposed Integrated Model
To create this more holistic approach, the business risk audit creates a lens or cognitive
process that encompass three broad additions to the audit risk model: business risk assessment,
audit-evidence collection, and feedback loops. First, the business risk model adds a holistic
business-level risk assessment, including internal controls. Second, the business risk model adds
the application of the business-level risk assessment to risk of material misstatement through the
overall financial statements, assertions, and other organizational traits to financial statement
accounts and classes of accounts. This process influences the nature, extent, and timing of
evidence collection. Finally, the business risk model incorporates feedback throughout the audit
to both update the risk assessment and provide a more robust triangulation of audit evidence
12
For a more complete discussion of audit evidence triangulation, see chapter four of Bell et al. (2005).
24
throughout the audit in feedback loops. Figure 2 contains a graphical representation of this
integrated model, including how these three areas of business risk assessment, audit-evidence
collection, and feedback loop integrate with the current audit risk model, and the remainder of
this section outlines these three areas in more detail.
This integrated model also provides a framework for evaluating emerging business risk
audit research. Without such a model, the fragmented extant research makes it burdensome to
evaluate the influence of the business risk audit on the overall audit process. However, a unified
model allows researchers to identify those areas addressed by current research, identify how such
research converges, and further identify opportunities for future research. In Section 4, I include
current business risk audit research within the context of this integrated model. I also include
suggestions for future research.
25
FIGURE 2
Proposed Business Risk / Audit Risk Model
See text for discussion of model.
26
3.2.1 Business Level Risk Assessment
The discussion section for Auditing Standard (AS) 12 (PCAOB) states that “this standard
describes an approach to identifying and assessing risks of material misstatement that begins at
the financial statement level and with the auditor's overall understanding of the company and its
environment and works down to the significant accounts and disclosures and their relevant
assertions.”13 The proposed model, Figure 2, includes a high-level risk assessment incorporating
both financial and business risk attributes sufficient for both a financial statement audit and an
internal controls audit.14 The model offers the auditor an overall understanding of the company
and its environment in order to perform an overarching audit risk assessment sufficient to
“provide a reasonable basis for the identification and assessment of the risks of material
misstatement, whether due to error or fraud, and to design further audit procedures ” (PCAOB
2010). The auditor then utilizes this information by evaluating the significant accounts and
disclosures and related, relevant assertions.
I update the original audit risk model (Figure 1) by including a more formal businesslevel risk-assessment process (Figure 2), shown by the box “Create business level (strategic) risk
assessment” and associated diamond-shaped inputs. Specific inputs, which generally follow
business, company and prior experience focuses, make up the first set of inputs, labeled with the
diamond-shaped “Identify and assess various business, strategic, and other relevant company risk
13
In August 2010, the PCAOB released, “Auditing Standards Related to the Auditor’s Assessment of and Response
to Risk” as PCAOB Release No. 2010-004. This release encompasses PCAOB AS 8 through 15. While each
standard is provided in a stand-alone manner, all eight standards are combined in PCAOB (2010) with discussion.
Unless specifically noted, my references are to the full document, and specific standards within.
14
AS 12. Appendix 5, includes a discussion on the integrated internal-control approach, as well as factors necessary
for an effective evaluation of internal controls as they relate to the risk of material misstatements (PCAOB 2010).
27
cues (signals).”15 However, AS 12 also requires an internal-controls audit as outlined in AS 5;
therefore, I include the results of the internal-controls evaluation as a second input into the
business-level risk assessment, as labeled with the diamond-shaped “Identify and assess internal
processes and controls” (PCAOB 2010). 16
My goal in creating this business risk/audit risk model is to provide for future research,
not to serve as an instructional for completing the business-level risk assessment. Therefore,
while I have attempted to include those attributes that I believe are most critical and/or generally
mandated by either business risk audit theories or current standards, I have not attempted to
provide an answer as to how the risk assessment should be completed.17 Furthermore, in a more
nuanced sense, the risk-assessment process includes separate processes for identifying and
assessing company risks, both with distinct skill sets. I have collapsed these two concepts into
the conceptual idea of the risk-assessment process within my business risk/audit risk model.18 I
defer to current risk-assessment literature as well as emerging research more specifically related
15
In addition to business and company-specific risks, both business risk audit theory and current AS 12 guidance
requires auditors to review and evaluate prior client experiences.
16
AS 12 outlines that the risk of a material misstatement may not only exist at the financial-statement and assertion
levels, but also from a variety of other sources, such as external factors (e.g., industry and environment) and
company specific factors (e.g., nature and activities of the company, and internal controls). Therefore, given the
broad potential causes for a material misstatement, the new standard now requires a risk assessment which
specifically includes the following: an understanding of the company and environment, an understanding of the
company’s internal controls, a consideration of the information from client acceptance and retention, audit-planning
activities any other engagements performed, a formal discussion among audit team members, any results from
analytical procedures, and a formal inquiry of the audit committee and management, among others within the
company, regarding possible material misstatement risks (PCAOB 2010).
17
This would include the input for the internal controls. I would anticipate that this would encompass an overall
framework such as the Committee of Sponsoring Organizations of the Treadway Commission (“COSO”) model,
including compliance and operational risk; however, as my objective in this paper is to address the overall relations,
I defer to future research to more fully flush out this input.
18
See Appendix 5, Auditing Standard 12 for a more detailed description of suggested areas to identify audit risks
(PCAOB 2010).
28
to the business risk-assessment process for insights into a more granular understanding of the
procedures for how the risk assessment is completed effectively.
3.2.2 Audit Evidence Collection: Application of the Risk Assessment to Risk of Material
Misstatement
As discussed earlier, business risk audit theory as well as recent PCAOB and ISA
standards envision the business-level risk assessment as a tool for a more robust evaluation
regarding the nature, timing and extent of audit-evidence planning. One reason for the business
risk audit approach is recognition that substantive testing can become inefficient and ineffective
as transaction volume increases and technologically moves away from paper documents (Public
Oversight Board 2000). By completing a high-level risk assessment, the auditor can use the
results of the same to more accurately evaluate risk of material misstatement for significant
accounts, disclosures and management assertions to create a more efficient and effective audit
(PCAOB 2010). In essence, a more formal business-level risk assessment not only provides
substantiated risk of material misstatement values for those identified accounts and classes of
accounts, but also provides a qualitative factor utilized during the evaluation of material accounts
and classes of accounts during audit planning. To address this linkage between the business-level
risk assessment, through the financial statements and management assertions, to the various risk
of material misstatement values assigned to those identified account balances, I include the
linking box in the business risk/audit risk model at Figure 2 entitled “Applied to” for financial
statements and management assertions.
29
The auditor gains a richer understanding of the various processes, geographic and
divisional areas, as well as other company attributes that provide relevant insights during the
audit through evaluating risks at a business level. Therefore, while the business-level risk
assessment is specifically applied to the financial statements and assertions, I also reference the
need to further evaluate risk of material misstatement not only through financial statements and
assertions, but also geographically (or divisionally), and by process or other relevant company
attributes, with the box on Figure 2 titled, “with regards to.”19
For example, in the Lincoln Savings and Loan illustration, the auditor not only gains an
understanding of the revenue process, but also a formal understanding of how related entities
impact revenue to better evaluate the validity of the asserted revenue amounts. Including the
“with regards to” box acknowledges that the financial statements do not work in isolation; rather,
many different client attributes and processes impact their creation.
The business risk/audit risk model provides a manner for the risk-assessment results to
filter through applicable financial statement and organizational factors to inform risk of material
misstatement at the requisite financial statement accounts. This creates an environment where
substantive testing is less in “a blanket-like fashion” but rather in a “diagnostic detailed testing in
a surgical fashion” (Bell and Solomon 2002).
19
The new PCAOB standards do require that when “a company has multiple locations or business units, the auditor
should identify significant accounts and disclosures and their relevant assertions based on the consolidated financial
statements” (PCAOB 2010).
30
3.2.3 Feedback Loops
As discussed earlier, the audit risk assessment is not meant to be static. Rather, a
knowledge-based feedback loop provides for continual updating of the risk assessment due to
client knowledge. This loop also provides the foundation for the triangulation-of-audit-evidence
concept outlined in the Bell et al. (2005) monograph. In this concept, the auditor not only
examines substantive evidence collected during audit testing, but also examines such evidence in
relation to knowledge gained in other areas of the audit. To reflect these concepts, I include a
feedback loop in Figure 2 titled, “Audit evidence and conclusions feedback,” which reaches from
the business-level risk-assessment process to the evidence and related-conclusion processes.
A final feedback loop is included in Figure 2 between the engagement risk process and
business-level risk-assessment process. As discussed in section 1.1, evaluating engagement risk
under the audit risk model provides insight into the overall nature of the client risk. As such, this
part of the process is most aligned with a high-level risk assessment. I have maintained
engagement risk as a separate process box on this model to reflect the evaluation of client
acceptance and retention as an ongoing requirement. However, the feedback loop is now
included to indicate the close relation between the business risk-assessment and engagement
risk-assessment processes.
31
4. Selected Business Risk Audit Literature and Suggestions for Future Research
The business risk audit approach is still in its infancy. Therefore, there is a dearth of
research as to its efficacy. To that end, while the preceding sections address the question of how
business risk audit integrates with the current audit risk model; this section includes select
business risk audit related research. I focus this research within the business risk/audit risk model
framework outlined in the previous section.
My goal in this section is to provide a summary of relevant research related to the
business risk audit with ideas for future research. As discussed in the prior section, the business
risk audit approach expands on the current audit risk methodology to include a more robust,
context-rich process. Thus, my primary question for future research relates to examining the
demands the additional dimensions place on the auditor and what limitations there are for an
auditor successfully implementing these requirements. This summary is not meant to be
exhaustive; rather, it is provided to begin a dialogue. I start with a general business risk audit
research subsection, and then include research related to the various sections (as outlined in
Section 3) of the proposed model.20
4.1 General Business Risk Audit Approach Literature
Experimental research finds that subjects have fewer errors and are better able to identify
management representations that are inconsistent with industry evidence when utilizing systemsbased information under a systems approach (Brewster 2011; Choy and King 2001). In contrast,
20
I do not attempt to review all the historical research directly related to the audit risk model. My interest is in
examining what research supports a move to the business risk audit approach. Therefore, I restrict my thesis to a
selection of those research studies that are more directly applicable to the business risk audit model.
32
other experimental research finds that managers strategically exploit low-risk accounts by
overriding them more often than high-risk accounts, but that priming subjects to predict a
manager’s strategic response seems to mitigate the effect (Bowlin 2011). Business risk auditors
are less likely to issue a going-concern opinion for a client that subsequently goes bankrupt. This
research suggests that auditors might be fooled by short-term operating efforts that attempt to
reduce financial risk (Bruynseels et al. 2006). In addition, industry experience, not general or
task-specific experience, contributes to an auditor’s judgment performance in a business risk,
experimental setting (van Nieuw Amerongen 2007).
Bierstaker and Wright (2004) find that auditors have begun to use narratives to a greater
extent when documenting internal controls. Further, they find that the narratives work more
effectively with the business risk audit model. Finally, the extent of controls testing has increased
under the new model. Other research finds that auditors who specialize in information
technology assess enterprise resource planning related risks higher and are better able to identify
security risks (Hunton et al. 2004).
4.1.1 Suggestions for Future Research21
To further understand the business risk/audit risk model, it would be useful to update
prior work related to understanding the current trends in audit-firm implementation of a business
risk audit approach. Of particular interest would be:
21
Within each of the four areas included in this section, I include suggestions for possible future research areas.
These include ideas that are both my own as well as from various readings as cited.
33
•
To what extent auditors adequately understand the client’s real business risks and related
controls, evaluate the risks and controls assessments, and then fully integrate the same into
their audit procedures.
•
How the business risk approach should and does influence auditor judgment regarding risk at
the assertion level.22
o To what extent are auditors effective at linking business risks and management
assertions? In the prior audit risk model, the risk assessment was generally completed
at the assertion level. As the audit risk assessment moves to the business level and the
risks become more general to the organization, are auditors able to effectively apply
the risks to relevant assertions?
•
How firms are applying the allocation of resources based on business-level risk assessments.
•
How firms apply internal control assessment results with the business-level risk assessment.
•
What levels of resources are being utilized in the audit risk-assessment process.
•
How the business-level risk assessment integrates with the requirements for fraud
brainstorming sessions.
•
Use the above information to more fully develop a model of the strategic thinking approach
as it relates to audit.
22
Thank you to Ed O’Donnell for the suggestion. See also Knechel (2007) for additional discussion.
34
•
Building on current literature, what are the strategic implications with clients when auditors
focus audit resources based on business-level risks? How do auditors mitigate potential
strategic client responses?
•
Is there (or should there be) some impact on the process of setting materiality when the
auditor more effectively identifies higher risk areas of the financial statements? What is the
relation between materiality and business risks in relation to the risk of not identifying a
material misstatement in the financial statements?
•
What are investor perceptions of this new audit approach? Do investors believe the top-down
approach to lead to a more effective audit, or simply to increased consulting revenue
detrimental to audit rigor (regardless of the accuracy of the perception)?
•
What are auditor perceptions of this new audit approach? Do auditors believe the top-down
approach to lead to a more effective audit, or simply to increased consulting revenue
detrimental to audit rigor (regardless of the accuracy of the perception)?
•
From an empirical perspective, does the business risk approach lead to a more effective audit,
or simply to increased consulting revenue detrimental to audit rigor?
•
Applying neural networks to the business risk audit. One advantage of the neural-network
method is its ability to classify, simultaneously consider multiple types of evidence in
assisting the auditor’s risk assessment and audit judgments (Calderon and Cheh 2002).
Would insights from the neural-network method improve the business risk audit?
35
4.2 Business Level Risk Assessment
Research is mixed with regards to the effectiveness of auditors in the risk-assessment
processes. In research using audit hours and fees, results suggest that there is a relation between
audit effort and inherent risk, but not internal controls (Felix et al. (2001); Hackenbrack and
Knechel (1997); O’Keefe et al. (1994)). Other research finds that auditors identify more business
risks when the client’s business risk-management process is weak, that auditors increase their
assessed risk when they engage in a more complete risk-assessment process, and that auditors are
better able to find financial risks compared to business risks (but, that business risk assessments
were improved with a stringent budget constraint and decision aids) (Kochetova-Kozloski et al.
2010; Diaz 2005).
Brazel et al. (2010) experimentally find that auditors only include nonfinancial measures
when prompted and under high-fraud risk situations. While the study specifically relates to fraud
risks, the results have implications for the business risk assessment, as business risks generally
contain nonfinancial measures. The results caution that auditors need to provide for explicit
prompting to examine nonfinancial measure risks, particularly for perceived low-risk clients.
Kochetova-Kozloski et al. (2010) finds a number of results from an experimental study
related to the business-process analysis. First, the effectiveness of the client-risk management
system does not impact the number of entity-level business risks identified, but did directly
impact the number identified at the core business level. Second, the greater the number of entitylevel business risks identified, the higher the risk of material misstatement assessed at the entity
level. Third, when a formal business risk assessment is performed, it leads to a higher risk of
36
material misstatement. Finally, process-level risk of material misstatement seemed to relate to
both the process and entity-level risk assessments.
Research finds that auditors seem susceptible to a cascade effect, in which judgments
made in one area influence similar judgments in another area. For example, Bhattacharjee et al.
(2007) finds that experience with a similar, but different, client leads to a cascading judgment
effect where the judgments of the prior client not only influence the judgments related to the
same task, but also of indirect tasks. Knechel et al. (2010) further find that auditors incorporate
more information into their risk assessments when given benchmarked performance measures
that are common to multiple business units, as compared to when only unit-specific performance
measures are benchmarked. Further, they find that risk assessments incorporate a broader set of
information available to the auditor when auditors have an extensive strategic analysis available
to them, regardless of the type of performance measures and benchmarks that are available.
Research finds that documentation of risk assessments has an impact on auditor
judgments. Experimental research finds that auditors who are required to document qualitatively
measured risk assessments will engage in a level of word smithery that allows for rationalizing
more-lenient audit judgments than in auditor’s original evaluation (Piercey 2009a & 2009b). In
another study, Bowlin et al. (2008) finds that managers with “diligent” auditor experience are
more sensitive to aggressive reporting penalties than those managers with only manager
experience. Finally, Kachelmeier et al. (2011) finds that, as the risk level for potential
misreporting decreases, auditor-participants under unintentional (but not intentional) risk
condition significantly lower their verification fees. These results suggest a bias in the risk-
37
assessment process depending on whether or not the auditor believes there is a strategic source to
the risk.23
4.2.1 Suggestions for Future Research
Proponents argue experience gained through ongoing engagements better refines the
auditor’s client knowledge and provides a foundation that supports increased effectiveness of the
business-level risk assessment and a more-refined lens to filter audit evidence. While this may
seem intuitive, there is little research in support of the assertion. Given the importance the new
business risk audit approach imparts the business-level risk assessment on the balance of the
audit, I see a greater need to better understand the boundaries of auditor business-level riskassessment effectiveness. Specifically:
•
What impact does prior experience have on decisions made during the risk-assessment
process? (See Chapter Three of my thesis, for experimental examination of this question.)
•
What impact does experience with one client have on decisions made for other clients during
the risk-assessment process?
•
What expertise do auditors need to generate effective business-level risk assessments, and to
what extent do auditors currently possess those attributes? For example, looking beyond
financial risks requires interviewing personnel in areas disconnected from accounting
functions. A more holistic audit approach requires a set of skills not in the traditional lexicon
23
I have only included those studies related to fraud risk that I believe are more directly related to the overall
business risk assessment. One could argue to include that line of literature in this discussion more fully. However, as
that line of literature is well established, I defer to those discussions and only note the complement they provide
here.
38
of prior experiences. In addition, maintaining legitimacy with the client and effectiveness of
the business-level risk assessment will entail new competencies (Knechel 2007). To what
extent do auditors currently possess these skills? And, what might be the impact of the
business-level risk assessment if these skills are found lacking?
•
More fully understand the cognitive processes involved for identifying and assessing
company risks, both separately and for the successful integration of risk cues into the
business-level risk assessment.
•
O’Donnell and D. Perkins (2011) find that utilizing a systems-thinking tool better focuses
auditors on diagnostic patterns of account fluctuations and leads to different risk conclusions
than those who used business-process categories during analytic procedures. Building on
these results, are there tools that would increase the effectiveness of auditor risk assessments?
If so, what are the limitations of these tools?
•
To what extent should analytical procedures be utilized in the risk-assessment process? How
have analytical procedures changed with the advent of a more holistic approach and what is
the implication on the risk assessment?
o See Trompeter and Wright (2010) for a summary of the current state of analytic
procedure practices.
•
The business risk audit approach requires the auditor to obtain access to individuals
throughout the organization not under the information flow through the finance department.
In some cases, this lack of information control can cause the client discomfort. What is the
39
implication on the effectiveness of the business risk approach if this access is managed or
restricted by the client (Knechel 2007)?
•
Given the in-house knowledge, are auditors better able to generate earnings forecasts than
equity analysts? Are they then able to effectively apply this forecast in the risk-assessment
process (Peecher et al. 2007)?
•
What are the legal implications of not identifying a complete universe of business risks?
Considering Grenier et al. (2012), what implication might the business risk-assessment
process have on jury perceptions?
•
See Carnaghan (2006) for a summary of various major business processes and modeling
conventions as a potential way to document audit risk assessments. The paper provides
insights from international standards related to enterprise modeling and suggestions for
future research. Would these tools improve the business risk audit approach?
4.3 Audit Evidence Collection: Application of the Risk Assessment to Risk of Material
Misstatement
Once the business-level risk assessment is complete, auditors must effectively apply the
risk-assessment results in their risk of material misstatement and audit-evidence-collection
planning. This section provides a summary of current business risk audit related research tying
business-level risk assessment to the audit.
Research utilizing Japanese-firm data generally finds limited association between audit
plans and audit risks, in line with prior research (Fukukawa et al. 2006). However, Fukukawa et
al. (2011) expands the study to specifically examine business risks and find indications that
40
certain business risk and fraud risk factors have a greater impact than others. Results of the
linkage between business risk factors is substantiated with results from Fukukawa et al. (2011),
which finds that certain risk categories impact audit-resource allocations.
Other research finds that the business risk assessment influences auditor’s assessments of
relevant financial statement assertions and audit procedures (Shelton et al. 2009). Further,
research also finds that strategic-systems thinking seems to both integrate the business risk
assessment with the risk of material misstatement and increase participant ability to appropriately
respond to seeded patterns of inconsistency and recognize the diagnostic value of change
patterns in inconsistent accounts (Schultz Jr. et al. 2010; O’Donnell and Perkins 2011).
Finally, Fukukawa and Mock (2010) find that the framing of assertions in the risk
assessment impacts the assessed level of risk of material misstatement.
4.3.1 Suggestions for Future Research
It would be useful to further explore the ability of auditors to utilize the results of the
business risk assessment in risk of material misstatement evaluations.
•
What limitations do auditors face in effectively applying risk-assessment results to the risk of
material misstatement measures?
•
AS 12 (as well as business risk audit theory) now envisions that the auditor will no longer
simply default risk of material misstatement to a high risk and increase substantive testing to
achieve the appropriate DR level. Are auditors following this mandate?
41
4.4 Feedback Loops
In the business risk audit model, knowledge gained through both the business risk
assessment and client experience provides auditors the ability to evaluate audit evidence and
create more effective risk assessments. In line with this theory, O’Donnell and Schultz (2005)
find that a strategic risk assessment causes a “halo” effect on auditor judgment. Specifically,
their experimental results indicate a shift in auditor tolerance for inconsistent fluctuations. This
shift causes the auditor to become more (less) likely to adjust account-level risk assessments for
inconsistent fluctuations with strategic/high-level risk assessments that are unfavorable
(favorable). O’Donnell and Schultz (2005) results’ have two implications. First, the results
support the assertion that auditors are influenced by the results of their risk assessment during
evidence evaluation. Second, their results also suggest that auditors may not fully examine audit
evidence that conflicts with positively assessed risk, an outcome that warrants research into
possible mitigating factors.
Ballou et al. (2004) finds that a strategic-level analysis in which the audit client is typical
(atypical) of those in their industry decreases (increases) the audit’s weighting of small problems
found at the business-process level. Results also indicate that risk assessment influences the
auditor’s judgment.
4.4.1 Suggestions for Future Research
The following areas related to the feedback loop process could profit from research.
•
Extending the discussion on the halo effect, to what extent are auditors able to build
skepticism in evidence review? For example, if the business risk assessment suggests a
42
positive environment, is the auditor able to evaluate contradictory evidence in an unbiased
manner?
o To what extent are auditors influenced by the business risk assessment in the audit
process, and how can auditors create more effective mental models to more fully
examine contradictory evidence?
•
Are auditors able to effectively apply more soft evidence found in the risk-assessment
process with the results of more tangible audit evidence found through more substantive
testing?
•
What is the implication of budget constraint on this process?
4.5 Summary and Experimental Boundary Test Discussion
The traditional audit risk-model approach historically allowed auditors to increase
substantive testing and minimize the risk-assessment process. With the new standards, auditors
are now required to complete a risk assessment. Requiring risk assessment with the business risk
approach adds a necessary combination of action and theory. However, there is little or no
guidance on how these business risk and audit risk models intersect. Rather, auditors simply
assume an intersection of business risk and audit risk, and that the business risk audit
complements and supplements the audit risk model.
My first contribution in this thesis is to propose a unified audit risk and business risk
model. This integrated business risk/audit risk model provides a foundation for researchers to
examine emerging, fragmented business risk-audit research holistically within the audit process.
43
I provide a formal model for how the business risk audit intersects with the more traditional audit
risk model to create this more holistic approach. I argue that the business risk audit creates a lens
or cognitive process that encompasses three broad additions to the audit risk model: business risk
assessment, audit-evidence collection, and feedback loop. After outlining my integrated model, I
then include and map into this model current research generally fitting within the category of
business risk audit and include suggestions for future research. This better equips researchers to
examine disparate research within the overall context of this model. Researchers can then
determine how research intersects areas of auditing focus, mining this convergence for insight
into increasing audit effectiveness through an integrated approach.
While the new risk-related audit standards require a formal evaluation and integration of
business-level risks into the audit (PCAOB 2010), little research has examined the boundaries of
auditor effectiveness in generating risk assessments. Having established an integrated audit risk
and business risk model, this thesis will now explore a possible limitation of an auditor’s ability
to generate an accurate risk assessment. That is, whether individuals, when provided with the
same risk-cue information, are influenced (biased) in subsequent risk-assessment decisions by
positive experience.
Risk-based audit theory argues that client-specific experience increases an auditor’s
ability to assess future client risks accurately. Research in psychology, however, suggests that
individuals tend to overweight experience when faced with current risk cues that conflict with
experience.
44
My design mimics the natural audit setting, whereby auditors gain client-specific
experience while working under continuous pressure to increase audit efficiency. This allows me
to examine the consequences of prior, positive client-specific experience and efficiency pressure,
represented by costly risk cues, on an individual’s choices during the risk-assessment process.
Given that new audit standards require auditors to create a formal business-level risk assessment,
my study provides insights into relying on the risk-assessment process without fully
understanding its potential limitations that may lead to unintended, adverse results.
45
CHAPTER THREE
Experimental Boundary Test
New risk-related audit standards require a formal evaluation and integration of businesslevel risks into the audit (PCAOB 2010). However, little research examines the boundaries of
auditor risk-assessment effectiveness. I build on my audit risk-and-business risk-integrated
model to explore one possible limitation in an auditor’s ability to generate a risk assessment. I
explore whether individuals, when provided with the same risk-cue information, are influenced
(biased) in subsequent risk-assessment decisions by positive experience. My study provides
insights into whether an experience-biased risk assessment can lead to unintended, adverse
results, thus identifying one potential boundary on risk-assessment effectiveness.
Chapter Three begins with a literature review and hypothesis development in Section 1.
In Section 2, I discuss my research design and method. Section 3 outlines my results.
1. Literature Review and Hypothesis Development
The new Public Company Accounting Oversight Board (PCAOB) suite of risk-related
auditing standards (AS) 8 through 15 (PCAOB 2010), as well as international auditing standards
(ISA) 315 (IAASB 2009a) and ISA 220 (IAASB 2009b), require a more formal evaluation and
integration of business-level risks into the audit-planning process through the business-level risk
assessment. Formally integrating a business-level risk assessment into a top-down risk-based
audit approach has great intuitive appeal. Auditors identify and evaluate high-level business risks
for their business-level risk assessment, and they are better able to align audit resources to those
areas of higher risks, thereby creating a more effective audit environment. By utilizing a riskbased approach, proponents argue that the business-level risk assessment is more robust and can
46
better inform the audit efforts from a top-down, rather than a bottom-up, approach (Bell et al.
1997; Bell et al. 2005; Bell and Solomon 2002; Lemon et al. 2000). Despite the appeal, however,
little research speaks to its efficacy. This study examines whether experience creates a focal
point that biases individuals when facing current-period risk cues that contradict positive
experience.
1.1 Risk Assessment Process
A forward-focused risk-assessment process requires current risk cues that predict current
organizational risks. During the risk-assessment process, auditors must continuously identify and
assess current, predictive risk cues for effective integration into their risk assessment.24 Auditors
also gain client-specific experience through ongoing engagements.25 In fact, business risk-audit
theory argues auditor-evidence evaluations during an audit become more effective with clientspecific experience (Bell et al. 1997; Bell et al. 2005; Bell and Solomon 2002). The auditor’s
challenge then becomes how to utilize their client-specific experience in providing a more
refined lens for filtering information, while avoiding the tendency to focus on prior period results
24
I define “risk cues,” “information cues,” or simply “cues” as information utilized to make a decision. During the
risk-assessment process, risk cues are those cues the auditor identifies as predictive of areas of higher risk. For the
purposes of this study, I make no distinction as to their origin. Nor do I make a prediction as to their level of
informativeness or predictive value—cues are simply descriptive information utilized in risk assessment decisions.
25
For the purposes of this study, I define “client-specific experience” or “experience” to be the experience an
auditor gains with a client through their interactions over periods of an ongoing engagement. I operationalize clientspecific experience by asking participants to perform the task of predicting the risk of error/no-error in the currentperiod risk assessment, and then receiving feedback as to whether the period contains an error. In this way, I make a
distinction from “auditor experience” as the experience gained over time by working in the audit profession, or other
experiences gained through specific technical or industry experiences, from client-specific experience. I further
define “experience” to mean the positive experience from a desired outcome. In an audit setting, this would mean
that no material error is found in the financial statements. In my current setting, I only examine the positive
experience condition.
47
as a proxy for current, predictive risk cues. I provide a graphical model of a general riskassessment process in Figure 3.
FIGURE 3
The Risk Assessment Process
This model highlights both client-specific experience and risk cues as inputs into the riskassessment process. The challenge in the second, diamond-shaped box of the model is for the
auditor to determine what inputs (and how) to integrate into their business-level risk assessment,
which is then utilized during the audit.
As noted earlier, auditors learn about their clients and use this knowledge in refining their
audit lens to provide relevant context not only during the evaluation of information during the
audit, but also while assessing risk cues in the “Identify and Assess Risk Cues for Risk
Assessment” box in Figure 3. However, it is possible that auditors will begin to utilize clientspecific experience as a replacement for current, predictive risk cues. For example, consider an
auditor that has a long, positive set of experiences with a client’s inventory process. In the
current period, however, a number of risk cues (e.g., high employee turnover, or changes in
48
compensation structure to include increased compensation on the main product, etc.) suggesting
a possible increase in the risk of inventory valuation. In this situation, it is possible that the
auditor will not utilize their prior experiences with the client’s inventory process as a filter for
these risk cues, but will focus on positive experiences to the exclusion of the currently available
risk cues. In other words, it may be that experience no longer acts as a lens during the auditor’s
decision process, but instead becomes a predictor of current-period risks. This scenario illustrates
the potential risk when auditors rely on client-specific experience, which looks backward, instead
of distinguishing current risk cues from client-specific experience when making risk assessment
evaluations.
In accordance with Figure 3, I create a condition in which participants gain prior
experience, have current risk cues, and must predict the current risk of error in the period. This
allows examination of whether individuals effectively focus on current-period risk cues without
regard to experience. Further, to frame the risk cues in a business risk setting I include two
business risk descriptions for the risk cues on the active screens, as discussed in Section 2.4.
In the following section, I examine cue-processing research for insight into the
intersection of client-specific experience and current-period risk cues as it pertains to risk
assessment. In essence, I am testing how auditors process the “client-specific experience” box
and the “risk cues (signals)” box in the risk-assessment process described by Figure 3.
49
1.2 Cue Processing
As discussed in Section 1.1, business risk-audit theory argues that client-specific
experience refines the lens that the auditor uses to evaluate information during the audit, thus
making the auditor more effective. However, this same client-specific experience can serve as a
focal point for the auditor when evaluating current-period risk cues. In this way, auditors rely on
client-specific experience to the exclusion of forward-focused risk cues.
My primary interest is the impact of positive, client-specific experience on the riskassessment process. While negative-experience cue series’ can adversely influence the auditor, I
focus in this initial study on what impact positive client-specific experience-cue patterns have on
risk assessment.
Positive client-specific experience can strengthen an auditor’s belief that the positive
experience will continue such that disconfirming evidence is ignored. This leads to a primacy of
positive experience over current risk cues as the series of positive client-specific experience
extends. In this case, the auditor does not correctly weigh current-period, descriptive risk cues,
which then leads to an ineffective risk assessment—one biased toward experience. Hogarth and
Einhorn (1992) argue, through their belief-adjustment model, that a long series of consistent cues
causes an attention decrement that drives individuals to seek information that confirms earlier
results. Hogarth and Einhorn (1992) predict that processing a long series of sequential
information creates a primacy trend. They also predict that a short series of mixed cues leads to a
recency bias.
50
In accounting, the seminal study by Ashton and Ashton (1988) suggests that the sequence
of audit evidence influences auditors, an idea that triggered an extensive literature supporting
cue-processing biases (see Kahle et al. (2005) for a review of relevant studies). Anderson and
Maletta (1999) find a primacy effect; however, they only find the primacy effect in the low-risk
audit environment with late positive information. They argue that the effect relates to efficiency
only (i.e., auditors over-weight the early, negative signals and would over-audit the affected
areas). Anderson and Maletta (1999) do not find the primacy effect when negative signals follow
positive ones, but they do not consider a long series of information or combine experience with
descriptive cues.
Pinsker (2011) finds that participants bias toward the most recent disclosure information
and concludes that long series of disclosures lead to recency effects. If the recency bias
influences auditors with positive experience, I argue it leads to two possible outcomes. First,
auditors may focus primarily on the most-recent positive-experience result. In such cases they
will ignore the current descriptive cues and follow the lead of positive experience, producing an
ineffective risk assessment similar to one produced by primacy. Alternatively, auditors may
respond more to disconfirming evidence, as argued in Ashton and Ashton (1990), and the
recency effect will lead to a greater focus on the current-period risk cues when contrasted with a
long series of positive experiences. In these situations, auditors focus to a greater extent on the
current-period risk cues than on a long series of positive results, leading to effective risk
assessment. However, in both Pinsker (2011) and Ashton and Ashton (1990), participants simply
read a descriptive cue item and provide their decision; the participants do not receive any
feedback on their results.
51
Results based solely on auditor experience are mixed. For example, Messier and Tubbs
(1994) find that audit experience seems to mitigate the recency effect. However, both Arnold et
al. (2000) and Krull et al. (1993) find that auditor experience does not seem to mitigate recency
bias. In another set of experiments, Monroe and Ng (2000) find information order does not
influence auditor judgments; rather, auditors hold a conservative bias in judgments of inherent
risk. None of the experience-related studies in accounting specifically examine client-specific
experience—only general experience (i.e., years as an auditor). One experience-related study
does examine whether general, industry, and task experience affects auditor judgments in
performing business-level risk-assessment tasks (van Nieuw Amerongen 2007). In this study,
“task” is defined as completing the risk-assessment process, similar to my operationalized client
experience. However, van Nieuw Amerongen (2007) does not find results for general or taskspecific experience; he only finds that industry experience positively affects the auditor’s
judgment in a business-level risk assessment. In this study, I focus on client-specific experience,
not industry or general experience.26 While results are mixed, auditor-experience related studies
generally indicate experience does not mitigate the impact of cue-processing biases. There is no
clear conclusion from accounting research on whether auditors would rely more or less on
experience when faced with both client-specific experience and current-period descriptive risk
cues.
26
Research into whether industry (or other client) experience increases auditor judgment effectiveness is mixed. For
example, Bruynseels et al. suggests that industry specialists are less likely to issue going concern opinions for
companies that subsequently go bankrupt. On the other hand, Hammersley 2006 finds industry specialist auditors are
better able to identify incomplete patterns of cue patterns to correctly evaluate their implications. However, Ballou
et al. (2004) finds that cues are weighted less when clients are found to be typical of their industry, Bhattacharjee et
al. (2007) find a cascading contrast belief effect from one client to another, and Grenier (2011) finds industry
specialization influences the auditors processing of domain evidence, but not self-critical thinking.
52
Outside accounting, psychology research finds that individuals apparently ignore risk
warnings provided after a series of positive experiences (Barron et al. 2008). Barron et al. (2008)
argue that individuals process both descriptive and experiential information together when
evaluating current risk warnings against positive experience. Barron et al. (2008) then run a
series of experiments examining descriptive warnings against positive experience. Results
indicate that individuals rely more on a single descriptive warning provided before a series of
positive experiences than they do on a single descriptive warning provided after a series of
positive experiences. In a setting in which their subjects receive only one descriptive risk
warning manipulated either early or late in the experience series, Barron et al. (2008)
demonstrate the primacy of an individual’s earlier experience over subsequent descriptive risk
warnings. Their results suggest that auditors are less likely to assess the impact of current-period
risk cues accurately during risk assessment after a series of positive client-specific experiences.
However, their study does not combine periodic experience with periodic descriptive risk cues
like those found in an audit setting. Individuals process information differently when they
receive multiple signals (as auditors do when they have to process both periodic experiences with
periodic risk cues) than when they receive one warning during a long series of experiences.
While the Barron et al. (2008) results provide some indication, it leaves open the question of
what impact periodic experience with periodic cues has on an individual’s ability to complete a
risk assessment.
Yechiam et al. (2006) illustrates the experience-description gap effect. Using the buying
and usage habits of car owners with detachable car stereo-panel faces, Yechiam et al. (2006)
finds that as car buyers evaluate a car stereo purchase, the small possibility of theft loss remains
53
salient; based on the description of the risk, buyers tend to purchase the safety feature. However,
as time goes on, drivers generally only experience the high-probability non-loss event, and a
statistically significant number of drivers stop removing the faceplate. Applied to an audit
setting, this suggests auditors should be diligent in obtaining risk cues for business-level risk
assessments early in their relationship with a client. Later, after a series of positive client-specific
experiences, auditors increasingly rely on experience and either obtain fewer risk cues necessary
for an effective risk assessment, or weight them differently in relation to client-specific
experience.
Lejarraga (2010) finds that as the complexity of the task increases, participants rely on
less-detailed, but easier-to-interpret information sampled from experience, over more-complex
and detailed descriptive information. Research examining the description-experience gap finds
that individuals select risk preferences based on whether they experience or receive descriptive
probabilities. Research finds that describing probabilities causes individuals to over-weight small
probabilities, while experiencing a similar outcome leads them to under-weight small probability
outcomes (Rakow et al. 2008).27
As this section has demonstrated, cue-processing literature provides no clear guidance on
the combined impact of client-specific experience and current descriptive risk cues on currentperiod risk decisions. I theorize that a long series of positive client-specific experience dilutes the
impact of disconfirming evidence. 28 In such a situation, an auditor becomes more and more
committed to the belief that the lengthening series of positive client-specific experience cues is
27
See also Rakow and Newell (2010) for a summary of current research into the description-experience gap.
28
See Hogarth and Einhorn (1992) & Pinsker (2007) for discussion of dilution effect.
54
an accurate predictor for current-period results. As periods accumulate, individuals begin to
weight the prior consistent experience greater than current-period disconfirming information;
their experience outweighs current information. In contrast to the primacy of a long series of
consistent positive experiences, mixed client-specific experience drives individuals to weight the
current-period descriptive cues greater than experience. The following hypothesis tests whether
client-specific experience negatively influences risk-assessment decisions (alternate form):
H1: A long series of prior, positive client-specific experiences will negatively influence
risk-assessment decisions.
1.3 Cue Costs
Audit clients view audits as a service they need to purchase at the most competitive price.
In response, auditors are in the ongoing position of balancing fee and cost pressures with profit
demands. To this end, auditors are under continuous pressure to conduct audits more efficiently
(Rittenberg et al. 2012). From a risk-assessment perspective, auditors must balance the need to
identify and collect the necessary risk cues with this ceaseless demand for audit efficiency. In
essence, the time required to complete a risk assessment is a real engagement cost through the
impact on the engagement economics (i.e., recovery rates). Therefore, in my sessions, I directly
examine the impact efficiency pressures have on risk assessments by manipulating costly cue
condition as a proxy for the effort of assessing risk, and I measure the impact efficiency pressure
has on risk-assessment decisions.
Monetizing cue costs makes them more salient to participants in the experimental setting,
and they realize more directly the impact of their decisions. In contrast, Diaz (2005)
55
operationalizes budget constraint by asking the participants to imagine they had budgeted hours
differently than in a prior year. She finds that this influences the effectiveness of the risk
assessment, but only in conjunction with a decision aid. However, Rakow et al. (2005) monetize
the cue costs and find that as cue costs increase, the overall number of selected cues decreases,
but participants more effectively identify those cues that offer higher information value (Rakow
et al. 2005).29.
Neither Diaz (2005) nor Rakow et al. (2005) evaluate cue costs in relation to task
experience. Given that costly cues should, ceteris paribus, decrease the incentive to acquire the
costly current-period risk cues, it is possible that, instead of becoming more effective in the
identification of correct cues, costly cues will create a bias towards a reliance on client-specific
experience patterns. For example, Lejarraga (2010) finds that participants rely on easier-tointerpret information sampled from experience more than on complex descriptive information.
While complex information differs from costly cues, it may be that increasing the cost of the
descriptive information will discourage individuals from selecting these descriptive risk cues
when positive experience results are available. Participants would in turn fail to select cues, and
fail to learn which cues offer higher informational value. Translated to an audit setting, auditors
would not collect enough cues or spend enough time properly evaluating risk signals to create an
effective risk assessment. There would be no increase in cue-selection efficiency with increased
cue costs and, because participants acquire fewer risk cues while becoming increasingly reliant
29
Rakow et al. (2005) evaluate three methods for establishing cue-search hierarchies. The measure of cue
informational value that I reference in this paper is one of their experimental measures. Results referenced are based
generally on their designs and overall results found in their study, not specifically one of their conditions.
56
on client-specific experience cues, overall efficacy with respect to decision-making would
decrease.
Essentially, as cue costs increase, participants will seek fewer risk cues. If this is the case,
it hinders participants’ ability to select cues that maximize their expected earnings in each period
and participant effectiveness in predicting the risk of error (risk assessment) in the period. I offer
the following hypotheses (stated in the alternate form):
H2a: Higher cue-costs decrease participants’ ability to select the number of risk cues that
maximize their expected earnings in each period relative to participants in the low-cost
condition.
H2b: Higher cue costs negatively affect risk-assessment decisions.
2. Research Design and Methods
2.1 Participants
Table 1 presents demographic characteristics of the study participants. Participants
include 85 undergraduate students completing their fourth year in the accounting program at a
large Midwestern university. They voluntarily participated in the study immediately following
their internship requirement. The mean participant age is 22.12 years and there are 47 females
and 38 males. All participants are accounting majors or have a dual major that includes
accounting. On average, participants have 14.29 months of work experience, have held 1.81
internships, and 15.29% have prior experience with risk assessments.
From a theoretical perspective, auditors must process risk information while balancing
client-specific experience. In this setting, I examine the ability of individuals to identify and
57
assess risk signals in the risk-assessment process. My study relies on the general cognitive
abilities of individuals in a cue-processing task. Thus, I provide insight generally into the
psychological impact of cue-processing demands under costly cue conditions and positive
experience. I also provide insight into the audit ecology, given that within the general cueprocessing demands of the audit-risk setting, my subjects are reasonable representatives of
individual auditors’ ability to process information in risk-assessment settings. This finds support
in the discussion in Libby et al. (2002), which describes student participants as an entirely
appropriate sample group when the experimental task focuses on general cognitive abilities. In
addition, Peecher and Solomon (2001) argue that “students should be thought of as being the
‘default’ condition for experimental participants”.
58
TABLE 1
Participant Demographics
Gender
Male
Female
Total
N
47
38
85
%
55.29
44.71
100.00
First - Third
Fourth
Fifth
Total
82
3
85
96.47
3.53
100.00
Accounting
Accounting (dual major)
Total
78
7
85
91.76
8.24
100.00
13
72
85
15.29
84.71
100.00
Year In School
Major
Prior Experience with Risk Assessments
Yes
No
Age
Mean
Median
Work Experience
Mean
Median
Maximum
Number of Internships
Mean
Median
Maximum
22.12 years
22.00 years
14.29 months
6.00 months
96.00 months
1.81 internships
1.00 internships
5.00 internships
59
2.2 Design Background
My design reflects the key features of a typical audit, in which auditors gain clientspecific experience and face pressure to increase audit efficiency in a 2 x 2 between-subject
design. Where I randomly assign between participants to experience (NO-PATTERN –
PATTERN or PATTERN – NO-PATTERN) or cue cost (HIGH or LOW) conditions. In each
period, participants are able to select 0, 1, or 2 costly (LOW vs. HIGH) risk cues, which they use
to generate a prediction (risk assessment) as to whether or not an error will occur in the current
period. This prediction automatically generates experimental effort cost in the period and affects
the penalty in those periods in which there is an error. The system then provides feedback to the
participants regarding the accuracy of their risk-of-error prediction at each period end.
Participants repeat this task in each period of the session, thus mimicking the general flow of an
audit while focusing on the ability of individuals to identify and assess current-period risk cues
after a long series of no-error results. This is shown graphically in Figure 4.
60
FIGURE 4
Experimental Design Overview
po
Hy
th
o
is
es
ne
DV
H:
2
a
This figure graphically represents the general flow of a typical audit (top row). In the second
row, I include the primary attributes of my experimental design. This provides a visual
illustration of how my experiment generally follows the audit flow. This is discussed in more
detail in the text.
2.3 Data Sets
To examine whether individuals make different decisions when facing differing
experience with a client, I build two data sets with differing patterns of error/no-error
(experience) series. Cue-processing literature generally provides a set of informational cues and
asks participants to make decisions based on these cues. For example, Barron et al. (2008)
61
evaluate the impact of a risk warning that follows a series of client-specific experiences. In each
of 100 periods, participants choose between two buttons where each button contains a different
payoff option (a different risk profile). The authors then manipulate a descriptive warning by
varying the timing of when participants receive this descriptive warning to either before the first
period, or after the fiftieth period. The experimenters then measure the change in participant
choices.
In a typical audit setting, auditors do not receive a single descriptive warning after some n
number of periods with the client. Rather, they generate risk assessments each period and
incorporate currently identified and known information into the current year’s risk assessment
and resulting audit, as discussed in more detail in Section 1.1. To operationalize experience with
descriptive risk cues utilizing a more realistic audit setting, I generate a series of data containing
financial reporting errors (“error”) or no-errors (“no-error”) in each period. I generate the NOPATTERN and PATTERN data series in the following manner:
1. I randomly draw a series of values between zero and one in Excel. I assign numbers below
0.70 a value of “no-error,” and those above (or equal to) 0.70 an “error” value to create the
period results.30
2. For the NO-PATTERN condition, I rerun the random values until obtaining a series of
periods that visually does not contain a long series of no-error or error periods.
30
While I realize that it would be unlikely an auditor would face an error rate of this magnitude, I choose these (and
cue) target values to provide some variability for the participants to learn, while still maintaining a high level of noerror periods.
62
3. For the PATTERN condition, I rerun the random values until I obtain one long no-error
series of periods.
a. To be consistent with prior literature, I draw the results until reaching a series of at
least 20 periods that have no errors (Hogarth and Einhorn 1992; Pinsker 2007). In
fact, I end up with a total no-error series of 25 periods.
4. For each PATTERN/NO-PATTERN experience data set, I then generate risk cues. I
randomly generate two risk-cue values for each period. I code values below 0.80 to match the
period error/no-error results, and those above (or equal to) 0.80 I code to not match (be the
opposite of) the error/no-error results for the period.
a. For example, if the period contains an error and cue one has a random value less than
0.80, then cue one would provide information that the period contains an error.
Alternately, if cue two has a random value greater than 0.80, the cue would provide
(incorrect) information that the period contains no error.
i. As I generate the risk-cue values to match or not match the error/no-error
results in each period, risk cues may incorrectly identify the current period
error/no-error in either direction.
5. I rerun the random values for the cues until there is a dispersion matching or not-matching
the error/no-error risk cues across all periods. In this case, I do not target any specific series
of risk-cue informativeness.
63
The instructions outline that the objective is to correctly predict whether there is a high or
low risk of a financial-reporting error in the current period. In addition, the instructions inform
the participants that they have no control over whether an error occurs in any given period.
Rather, they have the opportunity to obtain risk cues that provide some indication as to whether
the current period contains an error/no-error.31 See Figure 5 for visual representation of the
distribution/cue patterns.
31
See Appendix 1, for a copy of experimental instructions.
64
FIGURE 5
Distribution/Cue Patterns
NO-PATTERN Data Series
Periods
Practice periods
PATTERN Data Series
Practice periods contain the same cue information and error/no-error results for
all participants—included as the first 15 periods of the first session.
First 7 periods of
main session
No pattern in series of error/ no-error
results
Period 8 of main
session
Period contains an error, and both cues
indicate "error." Therefore, all
participants have the same cue
information to use in risk-assessment
prediction of current period error/no-error
results
Periods
9 through 33 of
main session
(25 periods)
No pattern in series of
error/no-error results
25-period series of no-error results; no
selected pattern in risk-cue information
Period 34 of main
session
Period contains an error, and both cues
indicate "error.” Therefore, all
participants have the same cue
information to use in risk-assessment
prediction of current period error/no-error
results
Periods 35—51 of
main session
No pattern in series of error/no-error
results
Cue 1
Cue 2
Results
65
2.4 Risk Cues
In each period, participants choose to review zero, one, or two risk cues that indicate
whether there is an error in the current period. After the initial 15 practice periods, risk cues are
costly. In the LOW (HIGH) cost condition, risk cues cost 50 (550) experimental dollars. I select
these values to provide a salient difference that supports cue-optimization calculations without
being so great as to create a disincentive to select any cues. Appendix 3 contains discussion on
the earnings payout.
Further, in this study, I build on the increased demand for a formal evaluation and
integration of business-level risks into the audit and explore one boundary of individuals
integrating business risks into the risk assessment. To represent business risks in my setting, I
include two descriptive risk cues a company would likely face. They are “Economy” and
“Management”. In this way, I am able to provide relevant context for a business risk setting. As
shown in Appendix 1, I provide the following descriptions related to the two risk cues:
Economy: This cue provides an indication as to how events in the overall economy might impact
Company and the likelihood that the current period will have an error which is not identified.
Management: This cue provides insights into changes within management structure,
compensation, staffing changes, etc. at Company which might impact the likelihood of an error
in the current period.
As shown in Appendix 2, the Economy risk cue precedes the Management risk cue in the
active screen of all periods. It is possible this causes an order effect between the risk cues. I
control for this possibility by manipulating the cue distributions underlying these risk cues
66
between sessions. I find no statistical support for an order effect. Therefore, I do not mention the
order of risk cues again in this thesis.
2.5 Practice Periods
The practice period provides subjects with an impression of the cue attributes without
formally providing the data patterns from the main session.
To provide all participants a similar baseline experience with the system and data
attributes, all participants receive 15 periods of data containing the same error and distribution of
risk cues at the beginning of the first session. In these practice periods, participants can select the
cues without cost. The practice data set contains 12 no-error periods. In addition, each risk cue is
accurate 12 of 15 times.
Participants make statistically similar choices for cue selections and prediction of the risk
of error between the conditions at the end of the practice sessions. This indicates that participant
knowledge and experience is consistent between conditions, and verifies that the main session
measured differences relating to treatment effects and not a difference in participants’
understanding of the experimental material.
67
2.6 Experimental Procedures
I run the experiment over a computer network using a custom program developed using
z-Tree.32 This software enables participants to independently determine their choices, while
providing a flexible platform to manipulate the cost and data conditions.
An experimental session consists of five parts: instructions, first experience session,
second experience session, U.S. dollar payment, and questions. Before starting the session,
participants have 10 minutes to read the instructions.33 After reviewing the instructions,
participants must correctly answer a set of questions designed to confirm their understanding of
experimental instructions before moving on. Subjects participate in two sessions, covering both
data conditions. To control for order effects, participants are randomly assigned to either start
with the NO-PATTERN or PATTERN experience conditions first, with the other experience
condition following. The only other functional difference between sessions is that the first
session includes 15 practice periods.34 Otherwise, both sessions progress the same.
Each session proceeds as follows:35
1. Starting in period two, before moving to the active screen, participants are able to view a
screen including a history of all results from prior periods. The history table includes the
error results, prediction of the risk of error choice made, risk information for cues
selected, penalties (when applicable), and earnings. Once participants have reviewed the
information on the history table, they move on to the active screen.
32
z-Tree designed and created by Urs Fischbacher; see: (Fischbacher 2007).
33
Instructions included at Appendix 1.
34
While in the first session this is actually period 15 + 8 = 23 with the practice periods, I do not use the practice
periods in my analysis and drop the first 15 practice periods in my results section discussion for consistency.
35
Example history and active screen shots are available in Appendix 2.
68
2. Participants receive E$ 8,000 at the beginning of each period. Earnings (losses) carry
over from period to period. In essence, this is equivalent to the auditors’ fee, from which
they cover costs and earn a profit (or loss) in each period. See Appendix 3 for earnings
discussion.
3. Participants select zero, one, or two risk cues to review for E$ 50 (550) per risk cue in the
LOW (HIGH) condition. See Section 2.4, for risk cue discussion.
4. After reviewing selected risk cues, participants predict whether there is a high or low risk
of an error in the current period (risk assessment). This risk assessment directly
determines the incremental costs and effectiveness of doing the audit.
5. The system automatically calculates effort cost, penalty (as applicable), and experimental
earnings for the period. Session earnings are updated, and the period ends. 36
6. In each main session, there are a minimum of 51 periods. Starting at the end of 51
periods, there is an 80% chance that the following period will be the final period. The
earliest end period in all sessions is 51 periods. Therefore, I utilize the data from only the
51 non-practice periods all participants completed.
a. I only utilize the 51 periods of nonpractice data in my analysis. In turn, I only
discuss 51 periods of data in my main result sections.
3. Experimental Results
3.1 Descriptive Statistics
Figure 6 provides a summary of risk cues selected by data set. Data is presented both in
the raw number of cues selected (based on total count of all periods, i.e., 51 periods x 85
participants = 4,335 total risk cues available for each data set) and percentage of total. As
expected, participants select significantly fewer cues in the HIGH cue-cost treatment than in the
LOW cue-cost treatment. Under the high-cost setting, participants selected no cues in 23.5% of
the periods; under low cost, 9.2%. In contrast, under the high-cost setting, participants selected
36
Appendix 3 contains an experimental earnings discussion.
69
two cues in 35.3% of the periods, and 49.2% under the low-cost setting. Overall, participants
selected significantly more cues in the low-cost treatment than in the high-cost treatment (1.401
vs. 1.118, p = 0.000).37 See Section 2.4, for risk cue discussion.
37
I calculate these results using a repeated measure ANOVA model; I code periods as within subjects. As
participants repeat their decisions across the periods within each session, this method controls for individual
variance.
70
FIGURE 6
Cue Selections
Data Set Condition
No cues choosen
One cue choosen
Two cues choosen
Total
Overall Mean
Cue Cost
No cues choosen
One cue choosen
Two cues choosen
Total
Overall Mean
COUNT
NO-PATTERN PATTERN
724
687
1,827
1,760
1,784
1,888
4,335
4,335
Total
1,411
3,587
3,672
8,670
1.245
1.277
High
1,008
1,764
1,512
4,284
COUNT
Low
403
1,823
2,160
4,386
Total
1,411
3,587
3,672
8,670
1.118
1.401
p=0.000
PERCENT
NO-PATTERN PATTERN
16.7%
15.8%
42.1%
40.6%
41.2%
43.6%
100.0%
100.0%
Total
16.3%
41.4%
42.4%
100.0%
PERCENT
Low
9.2%
41.6%
49.2%
100.0%
Total
16.3%
41.4%
42.4%
100.0%
p=0.679
High
23.5%
41.2%
35.3%
100.0%
Data by Count
Cue Cost
No cues choosen
One cue choosen
Two cues choosen
Total
NO-PATTERN Data Set Condition
High
Low
Total
524
200
724
891
936
1,827
727
1,057
1,784
2,142
2,193
4,335
PATTERN Data Set Condition
High
Low
Total
Grand Total
484
203
687
1,411
873
887
1,760
3,587
785
1,103
1,888
3,672
2,142
2,193
4,335
8,670
NO-PATTERN Data Set Condition
High
Low
Total
24.5%
9.1%
16.7%
41.6%
42.7%
42.1%
33.9%
48.2%
41.2%
100.0%
100.0% 100.0%
PATTERN Data Set Condition
High
Low
Total
Grand Total
22.6%
9.3%
15.8%
16.3%
40.8%
40.4%
40.6%
41.4%
36.6%
50.3%
43.6%
42.4%
100.0%
100.0%
100.0%
100.0%
Data in Percent
Cue Cost
No cues choosen
One cue choosen
Two cues choosen
Total
Figure 6 provides a summary of risk cues selected by data set and cue cost. Data is presented
both in the raw number of cues selected (based on total count of all periods, i.e., 51 periods x 85
participants = 4,335 total risk cues available for each data set) and percentage of total.
71
3.2 Hypothesis One Results
My primary question explores whether participant risk-assessment decisions are
negatively influenced by client-specific experience, as measured by whether participants choose
no-error risk-assessment predictions to a greater extent after the long no-error result series.
As shown in Figure 7, I test experience in two ways. First, I examine only the PATTERN
results for periods eight and 34, which immediately surround the 25 period no-error series.
Periods eight and 34 both contain risk cues providing correct information that the period contains
an error; participants have the same risk-cue information to use in predicting the risk of error in
their current risk assessment. The only difference between period eight (“early”) and period 34
(“late”), is participant experience in the intervening 25-period no-error series. If subjects choose
a low risk of error more frequently in the period after experiencing a no-error series, it would
imply that the intervening no-error periods (simulating past client-specific experience) influence
participants in their current assessment of risk cues. Examining the choice difference between
these two periods provides a direct measure of whether participants select the low risk of error
(no-error) more often after experiencing a long series of no-errors. The “Test one” arrow, Figure
7, illustrates this.
My second test utilizes both the NO-PATTERN and PATTERN data sets. To measure
whether there is a difference in the risk of error prediction between these two data sets, I look to
those periods where participants receive the same risk-cue and period-result information (i.e.,
when both risk cues contain the same information and the period has the same error/no-error
result). Because I generate data in each period of PATTERN and NO-PATTERN data sets
72
independently, I cannot simply compare periods 34 through 51 directly between data sets. I must
match attributes between conditions to account for the fact that attributes do not necessarily line
up between periods of the two conditions. For example, in PATTERN period 34, both cues
correctly inform participants that there is an error in the period. However, this same set of
attributes does not appear until period 37 in the NO-PATTERN condition. Therefore, I match the
results of PATTERN period 34 with NO-PATTERN period 37. I match periods subsequent to
the 25 no-error series (i.e., starting with period 34) in a similar manner until I have matched all
periods with the same attributes. I obtain 13 matched periods. The matched periods are less than
the total available periods, as some periods cannot be matched. By matching period attributes
between data sets, I take measures where participants have the same risk-cue information to
assess the risk of error in the period. If experience does not influence participants, they should
make the same risk of error predictions based on the same cue information between PATTERN
and NO-PATTERN. However, if experience with the no-error series influences participants in
the PATTERN setting differently than participants in the NO-PATTERN setting, I should see a
difference in their risk of error predictions. The “Test two” arrow, in Figure 7, illustrates this.
Results of my first test indicate that participants predict an average of 0.952 error risk in
the early period. High risk of error (error) is coded as one, and low risk of error (no-error) is
coded as zero. In the late period, participants predict an average of 0.823 error risk, for a
significant difference of 0.129 (p = 0.004); see Table 2, Panel A. These results suggest that
73
participants bias toward predicting low (no-error) risk of error (risk assessment) after
experiencing a long series of no-errors.38
While the results are significant, it is interesting to note that in the PATTERN condition,
participant average was 0.823, which is only 0.177 different from error variable value of 1.00,
and only 0.129 error choice value difference between conditions. Thus, while the results suggest
that individuals bias towards no error after experiencing a long no-error series of results, the
majority of subjects were able to correctly identify the correct value. Thus, the difference in error
choice suggests that the bias, while significant, is isolated to a small number of individuals.
38
I also examine PATTERN late period against the six other PATTERN periods containing the same (two risk cues
correctly predicting an error result) attributes and find that participants in the late period significantly select no-error
against all similar periods as well (p=0.000). Finally, I examine the only two early periods against the closest two
late periods with the same attributes around the 25 no-error series and find consistent results (p=0.042).
74
FIGURE 7
Visual Representation of Experimental Tests
Periods
Practice periods
NO-PATTERN Data Series
PATTERN Data Series
Practice periods contain the same cue information and error/no-error results for
all participants—included as the first 15 periods of the first session.
First 7 periods of
main session
No pattern in series of error/ no-error
results
Period 8 of main
session
Period contains an error, and both cues
indicate "error." Therefore, all
participants have the same cue
information to use in risk-assessment
prediction of current period error/no-error
results
Periods
9 through 33 of
main session
(25 periods)
Test one:
Period
8 vs. no
25-period series of no-error
results;
selected pattern in risk-cue34
information
of
PATTERN
Period 34 of main
session
Periods 35—51 of
main session
No pattern in series of
error/no-error results
Period contains an error, and both cues
indicate "error.” Therefore, all
participants have the same cue
information to use in risk-assessment
prediction of current period error/no-error
results
Test two: examining between
data sets (after period
33) forin series of error/no-error
No pattern
differences in error
prediction
results
of matched periods
75
TABLE 2
Hypothesis 1 Results
Experience Influence on Risk of Error Prediction
Panel A: Hypothesis 1 Results
Evaluation one
Evaluation Two
Expectation
Early Period > Late Period
NO-PATTERN > PATTERN
Results
Early Period: 0.952
Late Period: 0.823
Difference: 0.129
NO-PATTERN: 0.353
PATTERN: 0.290
Difference: 0.063
p-value
0.004
0.022
Conclusion
Hypothesis 1
supported
NOTES: (1) Evaluation one: error predictions between the period prior to long no-error series and the period after long no-error series of
PATTERN condition. Evaluation two: first six matched periods, subsequent to long no-error series, between PATTERN and NO-PATTERN
conditions. (2) Prediction values coded such that a low-risk prediction of an error (no-error) is zero and a high-risk prediction of error (error) is
one. Therefore, values closer to zero mean that, on average, participants predicted more no-error values in the period. (3) P-value results for the
final seven matched PATTERN vs. NO-PATTERN periods is 0.294, indicating no statistical difference between conditions. (4) Given p-values are
from Panels B and C of Table 4.
75
76
For the second test, I use a repeated-measure ANOVA, with Periods coded within
subjects and Experience coded between subjects.39 Figure 8 reveals that the predictions made
under the PATTERN data remain consistently closer to the no-error value in the NO-PATTERN
data for approximately six matched periods after the no-error series.
FIGURE 8
Matched Periods Subsequent to No-error Series
Figure 8 shows PATTERN data consistently closer to no-error value of one (low-error prediction
(no-error) is coded as zero) in early periods. See discussion in Hypothesis 1 results section.
39
While participants do complete both the NO-PATTERN and PATTERN conditions, I do not code the Experience
variable as within subjects in this draft. For each session, I restart the system, which makes the subjects view each
session separately. While I lose some control over possible subject-specific variances between sessions, I gain the
ability to examine more formally differences between data sets. Further, any uncontrolled subject variance would
increase the overall noise found in the results, and bias against finding results.
77
Results for these six matched periods subsequent to the PATTERN no-error series
indicate that participants in the NO-PATTERN condition predict closer to error (high risk of
error) significantly more than in the PATTERN condition (0.353 vs. 0.290, p=0.022); see Table
2, Panel A, and Figure 9.
FIGURE 9
Graphical Matched Period Results
In the last seven periods, I find no significant results between PATTERN and NOPATTERN conditions (p=0.292, not tabulated). These results provide further support that
participants bias toward the low risk of error prediction after experiencing a long series of noerrors, but not indefinitely.
Looking at the results from the perspective of risk-cue processing, I argue that finding
results biased towards the long, no-error series indicates that participants become more and more
committed to the belief that the lengthening series of positive client-specific experience cues are
an accurate predictor for the current-period results. This result provides support for the primacy
of the experience cues with a long series of positive experience, through the diluting effect
78
experience has on the impact of the subsequent disconfirming evidence found with the currentperiod descriptive risk cues. In contrast, those periods following mixed experience results
indicate that participants do not focus on historical results; rather, they utilize current-period
descriptive cues. This is best illustrated in the matched periods subsequent to the 25 period noerror series. As outlined above, participants in the PATTERN condition are biased toward the
no-error results for six periods. However, as shown in Figure 9, the bias ends after a few periods
without consistent no-error results. In fact, as discussed earlier, there is no statistical difference
in the predictions for the final seven periods. This supports the Hogarth and Einhorn (1992)
prediction that mixed cues would lead to a recency focus, a situation that in my view leads to
individuals focusing on current descriptive cues over prior experience.
Once again, however, while the results are significant, the difference between conditions
is only 0.063 error choice value between conditions. Thus, the difference in error choices
suggests that the bias, while significant, is isolated to a small number of individuals.
Overall, these results indicate that experiencing a long series of positive (no-error) results
influences individuals for some length of time. From an audit perspective, these results suggest
that experiencing a long series of positive client experiences may negatively influence some
auditors when assessing current business-level risk.
79
3.3 Hypothesis Two (a & b) Results
My second hypotheses examine the implication of efficiency pressure (costly cues) on the
risk-assessment process. In Hypothesis 2a, I question whether results of prior literature—
suggesting participants more effectively identify cues with higher information value as cue costs
increase—hold in an audit setting. Specifically, I examine whether higher cue-costs decrease
participants’ ability to select the number of risk cues that maximize their expected earnings in
each period relative to participants in the low-cost condition. In Hypothesis 2b, I extend this
inquiry to examine whether increased cue costs interact to negatively influence risk assessment
decisions.
As discussed in Appendix 4, I generate expected payouts for the LOW and HIGH cue
cost conditions to maximize expected earnings in each period. Earnings are maximized in the
LOW condition, when participants select risk cues until one risk cue indicates a risk of error for
the period, or until both risk cues indicate a no-error expectation for the period. Participants then
predict a high risk of error any time a cue indicates an expectation of error. If both cues show a
no-error expectation, then participants predict low risk of error (no-error) for the period. In the
HIGH condition, participants maximize earnings by selecting one risk cue through the entire
session and use whatever that risk cue indicates as their prediction for that period’s risk of error.
Thus, by having these risk cue choice criteria, I am able to measure cue-selection effectiveness.
For Hypothesis 2a, I use the absolute difference from these earnings maximizing cue selection
criteria as my measure of risk-cues selection effectiveness.
80
To evaluate if one cue cost condition is more likely to deviate from this earnings
maximizing cue choice criteria, I utilize absolute difference from these maximizing choices. The
absolute difference does not differentiate whether participants select too few or too many risk
cues; the measure only counts the absolute difference in the number of risk cues selected. Results
indicate, that, in an absolute sense, participants choose the number of risk cues further from the
earnings maximizing number in the HIGH condition across all periods, than in the LOW
condition (0.588 vs. 0.491, p=0.033); see Table 3, Panel A. In the matched periods of
PATTERN/NO-PATTERN results, participants in the HIGH condition select 0.609 risk cues
further from the earnings maximizing number. In the LOW condition, participants select 0.488
risk cues further from the earnings maximizing number, a significant difference of 0.121
(p=0.028); see Table 3, Panel A. From the perspective of Hypothesis 2a, results indicate that
participants in the HIGH cue cost condition are significantly more likely to select the number of
risk cues further from the earnings maximizing number of risk cues. These results bring into
question whether prior research, which suggests that individuals become more effective at
selecting risk cues as cue costs increase, holds in this setting.
81
TABLE 3
Hypothesis 2a Results
Risk Cue Selection
Panel A: Hypothesis 2a Results
All Periods
(includes both data set conditions)
Matched periods subsequent
to no-error series
(includes both data set conditions)
Expectation
HIGH cue cost condition
>
LOW cue cost condition
HIGH cue cost condition
>
LOW cue cost condition
Results
HIGH Cue Costs: 0.588
LOW Cue Costs: 0.491
Difference:
0.097
HIGH Cue Costs: 0.609
LOW Cue Costs: 0.488
Difference:
0.121
p-value
0.033
0.014
Conclusion
Hypothesis 2a
supported
NOTES: (1) Values coded as the absolute difference from earning maximizing cue selection as outlined in text. Thus, values are the average
difference (absolute) from the predicted earning maximizing cue selections. (2) As the values are absolute, P-values are one tailed. (3) Given pvalues are from Panels B and C, below.
81
82
Panel B: Risk Cue Choice Difference, Repeated Measure ANOVA for All Periods with both Cue Cost and Experience
Within-Participants Effects
Periods
Periods x Experience
Periods x Cue Cost
Periods x Experience x Cue Cost
Error (Periods)
Between-Participants Effects
Intercept (Grand Mean)
Experience
Cue Cost
Experience x Cue Cost
Error
Sum of
Squares
Degree of
Freedom
Mean Square
f value
Two-Tailed
p value *
42
28
15
14
1617
50
50
50
50
8300
1
1
0
0
0
4.265
2.916
1.532
1.396
0.000
0.000
0.072
0.125
2524
0
21
0
994
1
1
1
1
166
2524
0
21
0
6
421.423
0.021
3.430
0.000
0.000
0.885
0.066
0.988
Note: N = 85 subjects x 51 periods x 2 data sets
* Within-effect includes Greenhouse–Geisser p-value adjustment for sphericity as indicated in Mauchly’s Test of Sphericity.
82
83
Panel C: Risk Cue Choice Difference, Repeated Measure ANOVA for 13 Matched Periods Subsequent to No-error Series
with both Cue Cost and Experience
Within-Participants Effects
Periods
Periods x Experience
Periods x Cue Cost
Periods x Experience x Cue Cost
Error (Periods)
Between-Participants Effects
Intercept (Grand Mean)
Experience
Cue Cost
Experience x Cue Cost
Error
Sum of
Squares
Degree of
Freedom
Mean Square
f value
Two-Tailed
p value *
24
1
5
1
403
12
12
12
12
1992
2
0
1
0
0
10.004
0.512
2.124
0.294
0.000
0.797
0.049
0.939
665
2
8
1
270
1
1
1
1
166
665
2
8
1
2
408.653
0.946
4.936
0.881
0.000
0.332
0.028
0.349
Note: N = 85 subjects x 13 matched periods x 2 data sets
* Within-effect includes Greenhouse–Geisser p-value adjustment for sphericity as indicated in Mauchly’s Test of Sphericity.
83
84
My results do not support Hypothesis 2b. In Hypothesis 2b, I hypothesize that increased
cue costs will interact negatively influence risk-assessment decisions. To measure this, I first
examine only the PATTERN results for periods immediately surrounding the long, no-error
series (see Figure 7, test one) by cue cost. I find no significant indication that participants predict
the risk of error differently between the periods before and after the PATTERN long no-error
series, when results interact with cue cost (p=0.817, Table 4, Panel A). I then examine
differences in error prediction between PATTERN and NO-PATTERN matched-period data sets
(see Figure 7, test two) by cue cost. Again, I find no significant difference for the six matched
periods between PATTERN and NO-PATTERN, when results interact with cue cost (p=0.456,
Table 4, Panel A). These results suggest that differences in the risk of error predictions do not
relate to efficiency pressure.
Participants are less effective in selecting risk cues under high-efficiency pressure.
However, varying cue costs does not seem to impact their ability to generate an effective risk
assessment.
85
TABLE 4
Hypothesis 2b Results
Cue Cost Interaction with Risk of Error Prediction
Panel A: Hypothesis 2b Results
Expectation
Cue Cost
Evaluation one
Evaluation Two
Period 8 > Period 34 (by Cue Cost)
NO-PATTERN > PATTERN (by Cue Cost)
HIGH
LOW
HIGH
Conclusion
LOW
Hypothesis 2b
NOT supported
Results
p-value
Period 8: 0.905
Period 34: 0.786
Difference: 0.119
Period 8: 1.000
Period 34: 0.860
Difference: 0.140
0.816
NO-PATTERN: 0.369
PATTERN:
0.286
Difference:
0.083
NO-PATTERN: 0.337
PATTERN:
0.295
Difference:
0.042
0.456
NOTES: (1) Evaluation one: error predictions between the period prior to long no-error series and the period after long no-error
series of PATTERN condition. Evaluation two: first six matched periods, subsequent to long no-error series, between PATTERN
and NO-PATTERN conditions. (2) Prediction values coded such that a low risk prediction of an error (no-error) is zero and a highrisk prediction of error (error) is one. Therefore, values closer to zero mean that, on average, participants predicted more no-error
values in the period. (3) P-value results for the final seven matched PATTERN vs. NO-PATTERN periods is 0.346, indicating no
statistical difference between conditions. (4) Given p-values are from Panels B and C, below.
85
86
Panel B: Risk of Error prediction, Repeated Measure ANOVA with PATTERN Early vs. Late Periods and Cue Cost
Within-Participants Effects
Periods
Periods x Cue Cost
Error (Periods)
Between-Participants Effects
Intercept (Grand Mean)
Cue Cost
Error
f value
Two-Tailed
p value *
1
0
0
8.691
0.000
0.004
0.816
134
0
0
1225.889
2.809
0.000
0.097
Sum of
Squares
Degree of
Freedom
Mean
Square
1
0
7
1
1
83
134
0
9
1
1
83
Note: N = 85 subjects x 2 periods (PATTERN early and late periods, coded as within subjects).
* Within-effect includes Greenhouse–Geisser p-value adjustment for sphericity as indicated in Mauchly’s Test of Sphericity.
86
87
Panel C: Risk of Error Prediction, Repeated Measure ANOVA with First Six Matched Periods Subsequent to No-error
Series, Cue Cost and Experience
Two-Tailed
Sum of
Degree of
Squares
Freedom
Mean Square f value
p value *
Within-Participants Effects
Periods
118
5
24
277.754
0.000
Periods x Experience
1
5
0
1.833
0.137
Periods x Cue Cost
1
5
0
1.243
0.293
Periods x Experience x Cue Cost
0
5
0
0.696
0.561
Error (Periods)
70
830
0
Between-Participants Effects
Intercept (Grand Mean)
106
1
106
558.075
0.000
Experience
1
1
1
5.350
0.022
Cue Cost
0
1
0
0.178
0.674
Experience x Cue Cost
0
1
0
0.558
0.456
Error
31
166
0
Note: N = 85 subjects x 6 matched periods x 2 data sets.
* Within-effect includes Greenhouse–Geisser p-value adjustment for sphericity as indicated in Mauchly’s Test of Sphericity.
87
88
CHAPTER FOUR
Supplementary Experimental Sessions
1. Overview
In Chapter Three, I examine the impact of client-specific experience on an individual’s
risk-assessment decisions. As discussed in Section 2 of Chapter Three, to manipulate experience
I create two data sets providing differing patterns of experience, and measure whether
participants choose no-error risk-assessment predictions to a greater extent after the long noerror-result series. By randomly generating the PATTERN and NO-PATTERN data series
separately, I am unable to match results on a period-by-period basis. Therefore, I match periods
between PATTERN and NO-PATTERN conditions based on risk cue and result attributes. There
is no theoretical or statistical reason that this should impact my results; however, I question how
long the bias results persist subsequent to the long no-error series. In a set of supplemental
experimental sessions, I manipulate only the 25 no-error-pattern treatments (“25 treatment
periods”). All other periods contain the same information between conditions, allowing better
examination of the persistence of the experience bias.
Some have also questioned whether participants correctly understand that the result of
each period is not a function of other periods. My theory assumes that subjects are biased by
prior experience. Assuming subjects believe each period is a function of other periods, their
reliance on experience during the risk-assessment process is rational and could provide an
alternate explanation for my findings.
In this supplementary experiment, I address these two concerns. I combine the original
PATTERN and NO-PATTERN data sets and manipulate the 25-period no-error pattern series
89
directly to create new conditions, allowing measurement of results without having to match
periods. I also update the instructions to clarify that results in each period are not a function of
other periods. The remainder of this chapter discusses these supplementary sessions in more
detail, including their results.
2. Matched Periods
In Chapter Three, Section 2.3, I discuss the process of randomly drawing a series of
values to generate the data sets for the PATTERN and NO-PATTERN conditions in my original
experimental sessions. As discussed in that section, by generating each condition’s data set
separately, the risk cues and results for each period between the PATTERN and NO-PATTERN
conditions do not directly line up period-by-period. Because of this, I am required to match
thirteen periods, subsequent to the 25 treatment periods, between the PATTERN condition and
the NO-PATTERN condition based on cue and result attributes. As seen in Figure 8, my original
results suggest that the bias effect is maintained for approximately the first six of the 13 matched
periods. I have no statistical or theoretical reason to suspect this process leads to incorrect
inferences. While I take care to match periods on cue and result attributes, I am only using 13 out
of the available 18 periods of data subsequent to the 25 treatment periods. It is possible,
however, that the matching process influences inferences regarding the persistence of the bias
effect. Therefore, these additional sessions further explore the persistence of the bias effect.
In these additional sessions, I combine the current PATTERN and NO-PATTERN data
sets into one 117 period (including 15 practice periods) data series. I combine both conditions in
total to preserve the same number of periods as the original experiments in order to mitigate any
possible effects associated with changing the length of the experiment. This permits direct, not
90
matched, comparisons between periods, affording a direct measure for how long subjects bias
toward experience.
2.1 Data Series Discussion
This supplementary experiment includes four data series conditions. In the first two
conditions, I combine the original PATTERN and NON-PATTERN conditions into a total of 117
periods. In the first condition, I maintain the original 25 sequential no-error periods from the
PATTERN condition. For the second condition, I manipulate the 25 sequential no-error periods
in the PATTERN set so that they are no longer a series of no-errors. In essence, the only
difference between conditions becomes the 25 treatment periods, a distinction shown in Figure
10. Because everything before and after the 25 treatment periods across these two different
conditions is identical, I no longer need to match periods after the series. 40
The third and fourth treatments are identical to conditions one and two, with the
exception that I combine the original PATTERN and NO-PATTERN data sets in reverse order
(i.e., instead of original PATTERN after NO-PATTERN, I place original NO-PATTERN after
PATTERN). With these treatments I explore whether the impact of having the 25 treatment
periods early or late in the long series of random periods has any effect. This order change
produces no statistical effect, so I do not mention this order condition again.
40
My focus with these additional sessions is to explore further implications of decisions based on experience
manipulations. As such, I am not manipulating cue costs in these additional sessions.
91
FIGURE 10
No-Error Series Manipulation between PATTERN and NO-PATTERN
PATTERN Data Set
NO-PATTERN Data Set
Series
Periods
Error?
Cue 1
Shows
Cue 2
Shows
Series
Periods
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
ERROR
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
ERROR
No Error
No Error
No Error
No Error
ERROR
No Error
No Error
ERROR
No Error
No Error
No Error
No Error
No Error
No Error
No Error
ERROR
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
No Error
ERROR
No Error
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Error?
No Error
ERROR
No Error
No Error
No Error
No Error
No Error
ERROR
No Error
No Error
No Error
No Error
No Error
No Error
ERROR
No Error
No Error
No Error
ERROR
No Error
No Error
No Error
No Error
No Error
No Error
Cue 1
Shows
No Error
ERROR
No Error
No Error
No Error
No Error
No Error
ERROR
No Error
No Error
No Error
No Error
No Error
No Error
ERROR
ERROR
No Error
No Error
ERROR
No Error
No Error
No Error
ERROR
No Error
No Error
Cue 2
Shows
No Error
ERROR
ERROR
No Error
No Error
ERROR
No Error
ERROR
No Error
No Error
No Error
No Error
No Error
ERROR
ERROR
No Error
No Error
No Error
ERROR
No Error
No Error
No Error
No Error
ERROR
No Error
Figure 10 distinguishes manipulated periods between PATTERN and NO-PATTERN conditions.
Highlighted rows differ from the other uniform rows in the 117 periods of PATTERN and NOPATTERN conditions.
92
3. Independence of Periods
The original experimental instructions (Appendix 1) state:
In both sessions, there is a possibility that any period will contain an error. Errors relate to a
material misstatement in the Company financial statements. You have no control over whether
there is an error in any period. Your objective is to correctly predict the risk of an error in the
current period. Attributes are similar in both sessions.
And
Risk cues provide an indication regarding whether there is an error in the current period, but
provide varying levels of reliability.
Because my interest lies in evaluating the impact of experience on decisions made during
risk assessment, rather than in exploring why participants make differing choices after a long noerror pattern series, the wording in the instructions is deliberately neutral. However, this neutral
language creates an environment whereby individual subjects interpret experience differently,
thus affecting the results. In other words, some subjects may interpret that periods are a function
of other periods. To control for possible differences in interpretation, I modify the additional
session's instructions to be more explicit in stating that the cues and results of each period are not
drawn based on any direct function of the results from other periods (See Appendix 5 for updated
instructions):
In this session, there is a possibility that any period will contain an error. Errors relate to a
material misstatement in the Company financial statements. You have no control over whether
there is an error in any period. In addition, the results of each period are not a function of prior
or future periods. Your objective is to correctly predict the risk of an error in the current period.
And
Risk cues provide an indication regarding whether there is an error in the current period. Each
risk cue provides an indication of the current period error. They are not based on a function of
prior or future periods or the other risk cue.
93
By more explicitly outlining that each period’s results are not dependent on any other
period, I better control for the possibility of differing interpretations while subjects make riskassessment decisions.41 I also include this information as an additional comprehension question
that participants must correctly complete prior to starting the session.
4. Results
4.1 Demographics
Table 5 provides demographic characteristics of the additional session participants.
Participants include 82 undergraduate students all completing their fourth year in the accounting
program at a large Midwestern university. They voluntarily participated in the study immediately
following their internship requirement. The mean participant age is 21.89 years and there are 46
females and 36 males. All participants are accounting majors or have a dual major in accounting.
On average, participants have 13.04 months of work experience, have held 1.83 internships, and
25.6% have prior experience with risk assessments.
Examining these results against the original demographic results from Table 1 reveals a
higher percentage of females in the additional sessions than in the main sessions (56.10% vs.
44.67%). To explore whether this change has an impact on the results, I include Gender as a
covariate in my analysis to control for possible differences in decisions made between males and
females. As I select participants from students with equivalent attributes based on their timing in
41
As I do technically generate a long series of data, the periods are not formally independent. However, I generate
the data randomly and do not have any formal function to generate that order based on the other periods. Therefore,
I am careful in my wording to define the data generation process as not having a formal formula for generating the
data series to not create an environment where the subjects are misinformed.
94
the academic program, demographic differences are minimal. I use subjects with similar
attributes to minimize variance due to academic experiences.
Further, as with the main session, participants make statistically similar choices for cue
selections and prediction of the risk of error between the conditions at the end of the practice
sessions.42 This indicates that participant knowledge and experience is consistent between
conditions. Together, demographic and practice session results suggests that participants have
consistent backgrounds, with the exception of gender, and consistent understanding of the
experimental material, ensuring that any measured differences relate to treatment effects and not
a difference in participant background or understanding of experimental instructions.
42
Experience results are still not significant across conditions with the inclusion of control variables outlined in
Section 4.4.1.
95
TABLE 5
Additional Sessions Participant Demographics
Gender
Male
Female
Total
N
36
46
82
%
43.90%
56.10%
100.00%
Fourth (all participants)
82
100.00%
Accounting
Accounting (dual major)
Total
72
10
82
87.80%
12.20%
100.00%
21
25.61%
61
82
74.39%
100.00%
Year in School
Major
Prior Experience with Risk Assessments
Yes
No
Age
Mean
Median
21.89
22.00
years
years
Mean
Median
Maximum
13.04
6.00
72.00
months
months
months
Mean
Median
Maximum
1.83
2.00
5.00
internships
internships
internships
Work Experience
Number of Internships
96
4.2 Descriptive Statistics
Figure 11 includes a summary of risk cues selected by data set. Data is presented both in
the raw number of cues selected (based on total count of 51 periods x 82 participants = 4,182
total risk cues) and percentage of total. For comparison, I also include select cue selection data
from the main sessions as well. Where in the first session, participants did not significantly select
cues differently between PATTERN and NO-PATTERN conditions, in these supplemental
sessions, participants do marginally select cues more in the PATTERN (mean: 1.55) vs. NOPATTERN (mean: 1.40) condition (p=0.101, one tailed). In addition, subjects select a
significantly higher number of cues in the supplementary sessions (mean: 1.47) than in the main
sessions (mean: 1.22, p=0.001). However, this does not translate into an interactive difference
between experience and session.
It is possible that the higher cue selection will lead to different error choices, as
participants who select more cues may be more accurate in their risk of error predictions. I return
to this possibility in my results and discussion sections.
97
FIGURE 11
Additional Sessions Cue Selections
Additional Sessions Data
No Cues Selected
One Cue Selected
Two Cues Selected
Total
Overall Mean
NO-PATTERN
232
639
965
1,836
COUNT
PATTERN
188
682
1,476
2,346
1.40
1.55
p= 0.101, one tailed
TOTAL
420
1,321
2,441
4,182
PERCENT
NO-PATTERN PATTERN TOTAL
12.64%
8.01% 10.04%
34.80%
29.07% 31.59%
52.56%
62.92% 58.37%
100.00%
100.00% 100.00%
1.47
Main Sessions Data
No cues choosen
One cue choosen
Two cues choosen
Total
Overall Mean
NO-PATTERN
200
936
1,057
2,193
PATTERN
203
887
1,103
2,193
1.20
1.24
p= 0.347, one tailed
TOTAL
403
1,823
2,160
4,386
1.22
Additional Means
Overall Differences
Session
Pattern * Session
NO-PATTERN
PATTERN
Main
Supplementary p value
Sessions
Sessions
(one tailed)
1.22
1.47
0.001
1.20
1.24
1.40
1.55
0.237
PERCENT
NO-PATTERN PATTERN TOTAL
9.12%
9.26%
9.19%
42.68%
40.45% 41.56%
48.20%
50.30% 49.25%
100.00%
100.00% 100.00%
98
4.3 Test One Results
I explore whether participant risk-assessment decisions are negatively influenced by
client-specific experience in these additional sessions. By maintaining the previous data sets, I
am able to return to the tests outlined in Figure 7, to test experience.
First, I examine the PATTERN results of the two periods immediately surrounding the 25
treatment periods. These two periods both contain risk cues providing correct information that
the period contains an error. Therefore, participants have the same risk cue information to use in
predicting the risk of error in their current risk assessment. The only difference between the two
periods is participant experience in the intervening 25 treatment periods. If subjects choose a low
risk of error more frequently in the period after experiencing a no-error series, then it would
imply that the intervening no-error periods (simulating past client-specific experience) influence
participants in their current assessment of risk cues. The “Test one” arrow, Figure 7, illustrates
this test.
Results of this first test indicate that participants predict an average of 0.957 error risk in
the early period. High-risk of error (error) is coded as one, and low risk of error (no-error) is
coded as zero. In the late period, participants predict an average of 0.978 error risk. The results
are not significant (p = 0.323). Thus, the results from these additional sessions do not support my
first test.
99
4.4 Test Two Results
My second test utilizes both the updated NO-PATTERN and PATTERN data sets. In
these supplementary sessions, I create two data sets that no longer require a matching of periods,
as all periods subsequent to the 25 treatment periods are the same across conditions. Therefore, I
directly examine the results between PATTERN and NO-PATTERN conditions for the 18
periods subsequent to the 25 treatment periods.
If experience does not influence participants, they should make the same risk of error
predictions based on the same cue information between PATTERN and NO-PATTERN.
However, if experience with the 25 treatment periods influences participants in the PATTERN
setting differently than participants in the NO-PATTERN setting, I should see a difference in
their risk of error predictions. The “Test two” arrow, in Figure 7, illustrates this.
For the second test, I once again use a repeated measure ANOVA, with Periods coded
within subjects and Experience coded between subjects. As in my main sessions, results for all
18 periods indicate that there is no statistical difference between conditions (p= 0.874, see Table
6, Panel A). I then examine whether results persist for some period after the no-error series.
Again consistent with my main session results, Figure 12 reveals that the predictions made under
the PATTERN data remain consistently closer to the no-error value in the NO-PATTERN data
for approximately six periods subsequent to period 34 (the period following the 25 treatment
periods). Therefore, I further examine the results for these six periods. Results are directionally
consistent with my expectations (PATTERN mean: 0.174, and NO-PATTERN mean: 0.213),
however, they are not significant (p=0.374, see Table 6, Panel B).
100
FIGURE 12
Periods Subsequent to 25 Treatment Periods
101
TABLE 6
Additional Sessions Test 2 Results
Panel A: Risk of Error Prediction, Repeated Measure ANOVA all 18 Periods Subsequent to 25 Treatment Periods
Within-Participants Effects
Periods
Periods x Experience
Error (Periods)
Between-Participants Effects
Intercept (Grand Mean)
Experience
Error
Sum of
Squares
Degree of
Freedom
Mean Square
f value
Two-Tailed
p value *
240
1
86
17
17
1360
14
0
0
222.493
0.993
0.000
0.423
325
0
35
1
1
80
325
0
0
744.188
0.025
0.000
0.874
Note: N = 82 subjects x 18 periods
* Within-effect includes Greenhouse–Geisser p-value adjustment for sphericity as indicated in Mauchly’s Test of Sphericity.
101
102
Panel B: Risk of Error Prediction, Repeated Measure ANOVA for Six Periods Subsequent to Period 34
Sum of
Squares
Within-Participants Effects
Periods
Periods x Experience
Error (Periods)
Between-Participants Effects
Intercept (Grand Mean)
Experience
Error
Degree of
Freedom
Mean Square
f value
Two-Tailed
p value *
41
0
16
5
5
400
8
0
0
212.771
0.622
0.000
0.531
18
0
18
1
1
80
18
0
0
78.330
0.798
0.000
0.374
Note: N = 82 subjects x 6 periods
* Within-effect includes Greenhouse–Geisser p-value adjustment for sphericity as indicated in Mauchly’s Test of Sphericity.
102
103
4.5 Additional Tests
Overall, my results do not support my first test between periods eight and 34, or my
second test of those periods subsequent to the 25 treatment periods. However, the results are
directionally consistent to the results in my main sessions. Therefore, in this section, I explore
whether the inclusion of various control variables or alternate tests provide additional context.
4.5.1 Controlling for Participant Cue Beliefs and Gender
My basic question relates to actual decisions made by participants, separate of their
beliefs. Therefore, I examine the six period results further by controlling for post-experiment
questionnaire reported differences in participant cue beliefs. This provides results more directly
measuring the actual decisions made, separate of any differences in perceived beliefs. I measure
these control variables by taking values from the post-experiment questionnaire relating to:
reliance on cues (Cue Reliance), reliance on cues after a long series of same error/no-error results
(Experience Reliance), and belief the current period is not a function of prior ones (Period not
Reliant on Prior). I then include them as covariates in the model shown in Table 6, Panel B. In
this case, the results are not only directionally consistent with my expectations (PATTERN
mean: 0.172, and NO-PATTERN mean: 0.216), but are, arguably, marginally significant
(p=0.101, one tailed, Table 7, Panel A). This provides qualitative, directional support that the
results persist for six periods subsequent to the 25 treatment periods when the results are
controlled for individual cue beliefs.
Section 4.1, indicates that there is a higher percentage of females in the additional
sessions than in the main sessions (56.10% vs. 44.67%). It is possible that females and males
104
make differing error choices. If this is the case, then this demographic change could impact the
results as well. To explore whether a gender difference impacts results, I include Gender as a
control variable with the above model. I do not find that Gender increases the Experience Main
effect when included individually as a control variable (p=0.349). I do find, however, that it
increases the significance when included with the other control variables outlined in Section
4.1.1. In this case, the main effect of Experience is: PATTERN mean: 0.172, and NO-PATTERN
mean: 0.216 (p=0.095, one tailed, Table 7, Panel B). The p-value moves from 0.101 to 0.905
with the inclusion of Gender, suggesting that Gender has some effect on the Experience results,
but not individually.
105
TABLE 7
Additional Sessions Test 2 Results with Control Variables
Panel a: Risk of Error Prediction, Repeated Measure ANOVA for Six Periods Subsequent to Period 34 and Control Variables
Within-Participants Effects
Periods
Periods x Experience
Periods x Cue Reliance
Periods x Experience Reliance
Periods x Period not Reliant on Prior
Error (Periods)
Between-Participants Effects
Intercept (Grand Mean)
Experience
Cue Reliance
Experience Reliance
Period not Reliant on Prior
Error
Sum of
Squares
Degree of
Freedom
0
0
1
0
0
13
5
5
5
5
5
385
6
0
8
2
0
10
1
1
1
1
1
77
f value
Two-Tailed
p value *
0
0
0
0
0
0
0.324
0.722
7.741
0.463
0.698
0.726
0.489
0.001
0.633
0.501
6
0
8
2
0
0
44.619
1.656
55.564
12.607
1.700
0.000
0.202
0.000
0.001
0.196
Mean Square
Note: N = 82 subjects x 6 periods
* Within-effect includes Greenhouse–Geisser p-value adjustment for sphericity as indicated in Mauchly’s Test of Sphericity.
105
106
Panel b: Risk of Error Prediction, Repeated Measure ANOVA for Six Periods Subsequent to Period 34 and Control
Variables, including Gender
Within-Participants Effects
Periods
Periods x Experience
Periods x Cue Reliance
Periods x Experience Reliance
Periods x Period not Reliant on Prior
Periods x Gender
Error (Periods)
Between-Participants Effects
Intercept (Grand Mean)
Experience
Cue Reliance
Experience Reliance
Period not Reliant on Prior
Gender
Error
Sum of
Squares
Degree of
Freedom
0
0
1
0
0
0
13
5
5
5
5
5
5
380
4
0
8
2
0
0
10
1
1
1
1
1
1
76
f value
Two-Tailed
p value *
0
0
0
0
0
0
0
0.694
0.647
7.661
0.432
0.751
0.755
0.501
0.525
0.001
0.650
0.474
0.472
4
0
8
2
0
0
0
31.840
1.756
54.825
12.620
1.750
0
0.000
0.189
0.000
0.001
0.190
0.642
Mean Square
Note: N = 82 subjects x 6 periods
* Within-effect includes Greenhouse–Geisser p-value adjustment for sphericity as indicated in Mauchly’s Test of Sphericity.
106
107
4.5.2 Additional Test Examining 25 Treatment Periods Results
In this test, I examine participant error choices between conditions surrounding the 25
treatment periods. I look at periods early in the 25 treatment periods against those after the 25
treatment periods, by condition. If subjects have a consistent bias within each condition, then
there would be no change in error selections from these two timeframes. To accomplish this test,
I look at the average of periods three through seven of Figure 10, against the five periods
subsequent to period 34, by condition. My measure is whether there is an interactive effect
between the 25 treatment period conditions across these two periods. If experience in the
PATTERN condition biases participants, then the difference between PATTERN and NOPATTERN conditions in the five periods subsequent to period 34 would be greater than the
average difference between conditions of periods three through seven of Figure 10. The mean
across the first five periods is, PATTERN: 0.411 and NO-PATTERN: 0.408, difference: 0.003;
in the later five periods: PATTERN: 0.045 and NO-PATTERN: 0.081, difference: 0.036 (see
Figure 13).43 Controlling for the same variables as in the prior section, this is a marginally
significant (p=0.068, one tailed) difference. These results provide further support that the pattern
of no-error series affects participants and that they are not simply predisposed to selecting cues
towards error/no-error in the different conditions.
43
Scale of mean differences between first series results and second series results relates to differences in cues
between series. While information between experience conditions contains the same information (results and cues),
the information is not the same between the first and second series. However, if the pattern does not influence
subject decisions, then there should be no statistical difference between the experience conditions in the first series
and between experience conditions in the second series. I find an interactive effect contrary to this requirement.
108
FIGURE 13
Difference Between PATTERN and NO-PATTERN,
by First and Second Series Around to 25 Treatment Periods
Difference between PATTERN and
NO-PATTERN
0.04
0.035
0.03
0.025
0.02
0.015
0.01
0.005
0
First Series
Second Series
While I do find results consistent (albeit marginally) with my prior results, it should be
noted that these results require the inclusion of control variables as outlined in Section 4.1.1.
These variables provide measures for individual differences in participant cue beliefs. Therefore,
including the control variables allows me to more directly measure the Experience results,
separate of any differences in perceived beliefs, which serves the fundamental question in this
study regarding actual behavior. However, these results suggest that individual beliefs relating to
cue information impacts error decisions in the risk-assessment process. In fact, while I explicitly
avoid examining the “why” regarding experience bias in this study, and I am unable to fully
explore this question with my current data, these results pose an interesting question for future
evaluation of the impact of individual differences in beliefs.
109
4.5.3 Cue Selection Difference from Main Session
I find that participants do marginally select cues more in the PATTERN vs. NOPATTERN conditions in these additional sessions. In addition, subjects select a significantly
higher number of cues in the supplementary sessions overall than in the main sessions. However,
this does not translate into an interactive difference between experience and session.
It is possible that the higher cue selection will lead to different error choices, as
participants who select more cues may be more accurate in their risk of error predictions. My
cues provide very similar information, so it also lilely that the differences in cue selection would
not significantly impact decisions in sessions. Given the lower percent of “No Cues Selected” in
the supplementary sessions than in the first sessions, it seems reasonable that cue selections
affect results.
One way to explore this question is to directly examine the change in choices made
between period eight and 34, to identify consistency between number of cues selected and error
choice. Between periods eight and 34 of PATTERN, five subjects changed their cue purchases.
Three bought one less and two bought one more. None of these subjects changed their decisions
between period eight and 34. However, in the NO-PATTERN condition, six subjects selected
difference-cue numbers. Five bought one less (four subjects went from two cues to one cue and
one subject went from one cue to zero), and one subject bought two cues less. Of those subjects,
two made incorrect choices, including the one who went from two cues to zero and one who
went from two to one. However, the person who went from one to zero still answered correctly.
Therefore, these results are inconclusive regarding whether the number of cues influences error
110
choice. Given that two participants decreased their choice to no cues, but only one was then
incorrect in his/her error choice, it would suggest that there is at least some impact. That I still
obtain directionally consistent results in my additional sessions, but without the significance of
the earlier sessions, supports this mixed result.
As I did not include the HIGH cost condition, which decreases the overall number of cues
selected, I am not able to directly examine this question. Adding the HIGH cost condition would
help in future exploration of this possibility. See also the discussion within Section 6, related to
professional skepticism. It may also be that I have a more skeptical group of participants in these
additional sessions; participants who are more skeptical select more cues, which could explain
the difference in cue selection between the main and additional sessions. Unfortunately, I did not
collect the skepticism measures in the main sessions, so I am unable to directly exam the results
between my main and additional sessions.
4.5.4 Independence Information
As discussed in Section 3, I modify the additional session's instructions to explicitly state
that the results of each period are not drawn based on a function of the results from other periods.
In fact, not only do I include this wording in the instructions, but I also add a comprehension
question to the set of questions that participants must complete correctly prior to starting the
session. This amended wording better controls for differing interpretations by subjects making
risk-assessment decisions. However, as my design relates to the processing of risk cues with
experience; it is possible that this added wording creates an environment whereby participants
are primed to focus on the experience pattern to an extent that it overwhelms the differences
111
between conditions. This is particularly likely given that the wording is included both in the
instructions and then very prominently in the comprehension questions. It may be impossible for
participants to not review this information during the experimental instruction phase.
To examine this possibility, I run a pair of PATTERN and NO-PATTERN sessions with
the comprehension questions removed (i.e., the wording included only in the instructions). If
results are adversely affected by this wording, then the results in the PATTERN condition
without the comprehension question should be significantly less biased towards no-error than in
the PATTERN condition with the comprehension question. I find no statistical difference
between the error choice means, for the 18 periods subsequent to the 25 treatment periods, when
including and not including the comprehension question under the PATTERN condition (with
question mean: 0.461, without question mean: 0.489, p=0.543).44 This lack of statistical
difference suggests that the inclusion of the comprehension question does not materially
influence the results.
An alternative explanation for not finding statistical results between including and notincluding the comprehension question is that subjects did not change beliefs after viewing the
question. Assuming this is true, it would be reasonable to expect no difference between the two
conditions. In the post-experiment questionnaire, subjects rate their level of belief that prior and
future periods did not function in current-period results. To examine whether subjects changed
their beliefs depending on the inclusion of the comprehension question, I examine whether they
answer these post-experiment questions differently depending on the inclusion of the
comprehension question. When asked whether they believed the current period was not a
44
Results are also consistent for the six periods after period 34.
112
function of current (future) periods, subjects rated their belief 0.609 (0.505) scale-value higher in
the condition with the comprehension questions (p=0.056 (p=0.052), one tailed). This suggests
that participants did internalize the information related to the additional independence wording
differently due to seeing the comprehension question or simply viewing the information in the
instructions. The fact that inclusion of the comprehension questions did not influence the error
decisions cannot be credited to equal interpretation of independence information.
5. Discussion
I do not find support for my first test between periods eight and 34, or my second test of
those periods subsequent to the 25 treatment periods. To further explore the results, I include
additional tests in Section 4.5.
First, I control for differences in participants beliefs regarding the use of risk cues in the
session. I find directional and marginally significant results for six periods subsequent to period
34. In addition, I find marginally stronger results when controlling for gender differences. Taken
together, these results support findings of the main sessions that experience creates bias.
I then examine participant error choices between conditions surrounding the 25 treatment
periods. In this case, I look at periods early in the 25 treatment periods against those after the 25
treatment periods, by condition. Controlling for the same variables as in the prior section yields a
marginally significant difference (p=0.068, one tailed). These results further support the
argument that a pattern of no-error series affects participants.
Increased selection of risk cues in the additional sessions may explain the muted effects.
Results provide mixed answers to this question. First, participants in the additional sessions have
113
fewer periods without choosing any cues, however, error choice results for periods eight and 34
are mixed for those subjects changing cue selections. Further, the directionally consistent results
in these supplemental sessions suggest that the effect is muted. Certainly, there is some
indication that the number of cues selected may influence the results in the additional session.
However, as I did not include the HIGH cue condition, I cannot directly test this possibility with
the current treatments.
Finally, I examine whether adding to instructions more clarity with respect to the
independence of periods mitigates results. Adding this wording may create an environment in
which participants are primed to focus on the experience pattern to an extent that it overwhelms
the differences between conditions. To examine this possibility, I run a set of PATTERN and
NO-PATTERN sessions with the comprehension questions removed (i.e., the wording was
included only in the instructions). If this wording adversely affects results, then the results in the
PATTERN condition without the comprehension question should be significantly less biased
than in the PATTERN condition with the comprehension question. I find no statistical difference
when including and not including the comprehension question in the PATTERN condition.
Overall, I do not find support for my primary tests in these additional sessions. However,
I am able to find directionally consistent results in additional tests including various control
variables. Figure 10 conveys that I only changed four periods from no-error to error to
distinguish the NO-PATTERN condition from the PATTERN condition. This leaves three runs
of six periods within the 25 NO-PATTERN treatment periods. It is likely that this design does
not provide enough contrast between conditions to observe a robust effect. It is also possible that
the experience bias effect is more isolated than believed. In the main sessions, while the
114
differences are statistically significant, the majority of subjects selected the correct error value in
both conditions (with a small, but significance difference between conditions). Consistent with
the main sessions, in these additional sessions differences between conditions are less than 0.045
between conditions. Thus, taken with the marginally significant results in this chapter, the
difference in error choice suggests that the bias, while significant, is isolated to a small number
of individuals. However, as I do find results in these supplemental sessions that are directionally
consistent with those in the main sessions, it suggests that experience biases certain individuals.
6. Professional Skepticism Measure
Hurtt (2010) creates a measure of an individual’s level of professional skepticism based
on characteristics derived from audit standards, psychology, philosophy, and consumer behavior
research. Hurtt (2010) bases her measure on a 30-item scale, which ranges in value from 30 to
180. Hurtt (2010) indicates that student averages range from 90 to 150. I add these questions to
my post-experiment questions and include the resulting values to explore whether differences in
measured skepticism affect the results. In my study, participants ranged from 116 to 170, with an
overall average of 142. This suggests that my subjects are slightly more skeptical than average.
More skeptical individuals may demand a higher level of evidence before making a
decision. If so, then those who rate higher on the skepticism measure would be less likely to bias
towards experience, as they would obtain a more complete set of risk-cue evidence in the current
period before making decisions.
To examine this possibility, I first compare the number of cues selected in the 18 periods
following the 25 treatment periods against the measured professional skepticism variable. Using
115
professional skepticism values from the post-experiment questionnaire, I find a significant
difference between the number of cues selected at each value of skepticism, as shown in Figure
14 (p=0.061, one tailed). However, this result only highlights that the values differ; it does not
provide clear insight into a consistent pattern. Further examination of Figure 14 appears to
unearth a slight trend in selecting cues from the lower skepticism values to the higher values. To
explore this trend, I split the results into three equal groupings of the measured skepticism values
(i.e., values 116-134, 134-152, and 152-170). This provides a low, medium, and high grouping of
skepticism values. With these, I can better determine if participants with low skepticism choose
fewer cues than those showing medium or high skepticism. Participants in the low grouping
purchase an average of 1.310 cues; those in the middle group purchase an average of 1.392; and
those in the highest grouping purchase 1.688 cues, as shown on Figure 15 (significant difference
at p=0.038, one tailed). It appears, then, that more skeptical participants select more cues.
116
FIGURE 14
Professional Skepticism Values by Period, Cues Selected
FIGURE 15
Cues Selected by Low, Medium, High Skepticism Groupings, Cues Selected
117
We can now ask whether the difference in cue selection impacts error choices. I find no
significant difference (p=0.839) in skepticism values for the 18 periods subsequent to the 25
treatment periods; see Figure 16. Examination of the Low/Medium/High skepticism value
groupings again shows no significant difference in error choice values (p=0.842); see Figure 17.
Further, no interactive effect with the experience (PATTERN vs. NO-PATTERN) conditions
(p=0.246) appears, indicating that skepticism does not differentially influence participant error
choices based on experience.
I conclude that more skeptical participants select a higher number of cues, but this does
not translate into different error choices either individually or in concert with experience.
However, results of error choice differences require qualification. My cues provide very similar
information, so it is possible that the differences in cue selection would have little impact on
decisions in sessions. Therefore, this test may be too insensitive to identify changes in error
choice decisions. Overall, however, results suggest that individuals who are more skeptical select
more cues. From an audit perspective, it indicates that auditors who are more skeptical collect
more information prior to making decisions, as we would expect.
118
FIGURE 16
Professional Skepticism Values by Period, Error Choice
FIGURE 17
Cues Selected by Low, Medium, High Skepticism Groupings, Error Choice
119
CHAPTER FIVE
Conclusion
A reliance on the business-level risk assessment and related business risk-audit approach
has emerged as an update to the traditional audit risk model. Formally integrating a businesslevel risk assessment into a top-down risk-based audit approach has great intuitive appeal. By
requiring auditors to identify and evaluate high-level business risks for their business-level risk
assessment, business risk audit theory argues auditors are better able to align audit resources to
those areas of higher risks, thereby creating a more effective audit environment. While this
concept is appealing, research in this space is only beginning to emerge. Further, what research is
available does not unify around an integrated model, which leaves research fragmented and
makes cumbersome any effort to evaluate the impact of research on the overall audit approach.
Research has not, to my knowledge, detailed how these models formally come together.
Rather, auditors simply assume the intersection of business risk and audit risk, and assume that
the business risk audit complements or supplements the audit risk model. My thesis proposes the
first model for how the business risk audit intersects with the more traditional audit risk model.
With this model, I include current research generally applicable to business risk audit, map
applicable business risk-related research into key areas of my model, and include suggestions for
future research.
I then explore whether individuals, when provided with the same risk-cue information,
are biased by positive experience. To frame the risk cues in a business risk setting, I include risk
cue descriptions that include two likely business risks. My primary question is whether
participant risk-assessment decisions suffer negative influence from client-specific experience, as
120
measured by whether participants choose no-error risk-assessment predictions to a greater extent
after a long no-error result series. I then examine how this measured value differs. Results
indicate that experiencing a long series of positive (no-error) period results influence individuals
for some period after their experience. From an audit perspective, these results provide a possible
warning that auditors bias towards a long series of positive client experiences when generating
current risk assessments.
The set of my second hypotheses examines the implication of efficiency pressure (costly
cues) on risk-assessment decisions. First, I examine if higher cue costs (efficiency pressure)
decrease participants’ ability to select the number of risk cues that maximize their expected
earnings in each period relative to those in the low-cost condition. Second, I extend this question
to examine whether increased cue costs (efficiency pressure) negatively influence participant
risk-assessment decisions (prediction of error). My results indicate that auditors are less effective
in selecting risk cues under high efficiency pressures. However, varying cue costs and
experience does not seem to influence participants’ risk-assessment decisions.
My results do not support Hypothesis 2b. In Hypothesis 2b, I first examine only the
PATTERN results for periods immediately surrounding the long, no-error series (see Figure 7,
test one) by cue cost. I find no significant indication that participants predict the risk of error
differently between the periods before and after the PATTERN long no-error series, when it
interacts with cue cost (p=0.817, Table 4, Panel A). I then examine differences in error
prediction between PATTERN and NO-PATTERN matched period data sets (see Figure 7, test
two), by cue cost. Again, I find no significant difference for the six matched periods between
PATTERN and NO_PATTERN, when interacted with cue cost (p=0.456, Table 4, Panel A).
121
These results suggest that difference in the risk of error predictions does not relate to efficiency
pressure.
My research does not support findings in literature higher cue costs help individuals
effectively select information cues. However, it should be noted that my risk cues provide very
similar levels of informativeness regarding the risk of error in the period. Given that I do not
perform a strong test of differences between levels of informativeness in the risk cues, it is
possible that a participant using the cues interchangeably hinders conclusive results. This
interchangeability would minimize inferences relative to the literature.
I also include a set of supplemental sessions designed to examine the persistence of the
experience bias results found in the main sessions. Results of these supplemental sessions do not
support statistically significant inferences; although, I do find directionally consistent results
with tests including various control variables. It is likely that this design does not provide enough
contrast between conditions to observe a robust effect. However, it is also possible that
experience bias is more isolated than believed. In all sessions, while the differences are generally
statistically significant, the majority of subjects selected the correct error value in both conditions
with small differences between conditions. Thus, taken with the marginally significant results in
the supplemental sessions, the difference in error choice suggests that the experience bias, while
significant, is isolated to a small number of individuals. However, as I do find results in these
supplemental sessions that are directionally consistent with those in the main sessions, it suggests
that experience biases certain individuals.
122
Overall results indicate that some participants bias toward prior experience when
generating current-period risk assessments. In addition, from an efficiency perspective, my
results indicate that individuals do not select the earning maximizing number of risk cues under
higher efficiency pressure, but that efficiency pressure does not negatively influence riskassessment decisions. Finally, my results caution that simply relying on a risk assessment during
the audit process may lead to unintended results. My study provides initial insights into this
space and indicates that further research into the boundaries of risk assessment is necessary.
This study carries implications for professional skepticism research. If auditors condition
on historical experience to the detriment of identifying and assessing current-period risk cues,
this bias limits skepticism in areas deemed lower risk due to positive auditor experiences. My
additional sessions employ (Hurtt 2010) professional skepticism measures to explore the
professional skepticism dimension in detail. More skeptical participants select a higher number
of cues, but this does not translate into different error choices either individually or in concert
with the experience condition. These error difference results, however, warrant qualification. My
cues provide very similar information; it is possible, then, that the differences in cue selection
would not have a strong impact on decisions in the sessions. My test may not be sensitive enough
to identify changes in error-choice decisions. Overall, however, results clearly suggest that
individuals who are more skeptical select more cues. Consistent with expectations from an audit
perspective, this suggests that auditors who are more skeptical collect more information prior to
making decisions.
With this research, I offer a few caveats. First, my goal is to provide a model for future
research, not to present an instructional discourse about business risk assessment. As such, I have
123
limited the scope of this thesis to those attributes I consider most critical or generally mandated
by either business risk-audit theories or current standards; however, I have not attempted to
explain how to complete the risk assessment. I defer to current risk-assessment literature as well
as emerging research more specifically related to the business risk-assessment process for a more
granular understanding of the procedures for the completion of a risk assessment.
Second, by focusing on the business risk-audit literature in my research overview, I defer
to prior literature and do not attempt to summarize all prior work more directly focused on the
audit risk model, engagement economics, etc., unless those items directly relate to the emerging
business risk-audit approach.
Third, I do not explore the basis for decisions made in this study. Rather, I simply explore
whether participants make different decisions between conditions. It may be that individuals are
inferring unknown expectations about experience patterns. Finally, I only examine one necessary
component of the risk-assessment approach. It is possible that some interaction with other
components would mitigate the effects identified in this study.
124
REFERENCE LIST
AICPA Professional Issues Task Force. 1994. Practice Alert No. 94-3 Acceptance and
Continuance of Audit Clients. American Institute of Certified Public Accountants
(AICPA).
Anderson, B. H., and M. J. Maletta. 1999. Primacy effects and the role of risk in auditor beliefrevision processes. Auditing 18 (1): 75–89.
Arnold, V., P. A. Collier, S. A. Leech, and S. G. Sutton. 2000. The effect of experience and
complexity on order and recency bias in decision making by professional accountants.
Accounting and Finance 40 (2): 109–134.
Ashton, Alison Hubbard, and Robert H. Ashton. 1988. Sequential belief revision in auditing.
Accounting Review 63 (4): 623.
Ashton, R. H, and A. H Ashton. 1990. Evidence-responsiveness in professional judgment:
Effects of positive versus negative evidence and presentation mode. Organizational
Behavior and Human Decision Processes 46 (1): 1–19.
Auditing Standards Board (ASB) of the American Institute of Certified Public Accountands
(AICPA). 2006a. Statement on Auditing Standards 109: Understanding the Entity and Its
Environment and Assessing the Risks of Material Misstatement.
———. 2006b. Statement on Auditing Standards 110: Performing Audit Procedures in Response
to Assessed Risks and Evaluating the Audit Evidence Obtained.
Auditing Standards Board of the American Institute of Certified Public Accountants. 1981.
Statement on Auditing Standards 39: Audit Sampling.
———. 1983. Statement on Auditing Standards 47: Audit Risk and Materiality in Conducting an
Audit.
Ballou, B., C. E. Earley, and J. S. Rich. 2004. The impact of strategic-positioning information on
auditor judgments about business-process performance. Auditing: A Journal of Practice
& Theory 23 (2): 73–90.
Barron, G., S. Leider, and J. Stack. 2008. The effect of safe experience on a warnings’ impact:
Sex, drugs, and rock-n-roll. Organizational Behavior and Human Decision Processes
106 (2): 125–142.
Bell, T. B, M. E Peecher, and I. Solomon. 2005. The 21st Century Public Company Audit:
Conceptuals Elements of KPMG’s Global Audit Methodology. KPMG International.
Bell, T., F. Marrs, Ira Solomon, and H. Thomas. 1997. Auditing Organizations Through a
Strategic-Systems Lens: The KPMG Business Measurement Process. KPMG Peat
Marwick LLP.
Bell, Timothy B, and Ira Solomon. 2002. Cases in Strategic - Systems Auditing. KPMG LLP, the
U.S. member firm of KPMG International, a Swiss nonoperating association.
125
Bell, Timothy B., R. Doogar, and Ira Solomon. 2008. Audit Labor Usage and Fees under
Business Risk Auditing. Journal of Accounting Research 46 (4): 729–760.
Bhattacharjee, S., M. J. Maletta, and K. K. Moreno. 2007. The cascading of contrast effects on
auditors’ judgments in multiple client audit environments. The Accounting Review 82 (5):
1097–1117.
Bierstaker, J. L, and A. Wright. 2004. Does the adoption of a business risk audit approach
change internal control documentation and testing practices? International Journal of
Auditing 8 (1): 67–78.
Bowlin, K. 2011. Risk-Based Auditing, Strategic Prompts and Auditor Sensitivity to the
Strategic Risk of Fraud. The Accounting Review 86 (4): 1231–1253.
Bowlin, K. O., J. Hales, and S. J. Kachelmeier. 2008. Experimental evidence of how prior
experience as an auditor influences managers’ strategic reporting decisions. Review of
Accounting Studies 14 (1): 63–87.
Brazel, J. F., K. L. Jones, and D. F. Prawitt. 2010. Improving Fraud Detection: Do Auditors
React to Abnormal Inconsistencies between Financial and Non-financial Measures?
Available at SSRN: http://ssrn.com/paper=1534778.
Brewster, B. E. 2011. How a Systems Perspective Improves Knowledge Acquisition and
Performance in Analytical Procedures. The Accounting Review 86 (3): 915–943.
Bruynseels, L., W. R Knechel, and M. Willekens. 2006. Do industry specialists and business risk
auditors enhance audit reporting accuracy? K.U.Leuven - Faculty of Economics and
Applied Economics.
Calderon, T. G., and J. J. Cheh. 2002. A roadmap for future neural networks research in auditing
and risk assessment. International Journal of Accounting Information Systems 3 (4): 203–
236.
Carnaghan, C. 2006. Business process modeling approaches in the context of process level audit
risk assessment: An analysis and comparison. International Journal of Accounting
Information Systems 7 (2): 170–204.
Choy, A. K., and R. R. King. 2001. An experimental investigation of approaches to audit
decision making: An evaluation using systems-mediated mental models. Contemporary
Accounting Research 22 (2): 311–350.
Curtis, E., and S. Turley. 2007. The business risk audit–a longitudinal case study of an audit
engagement. Accounting, Organizations and Society 32 (4-5): 439–461.
Diaz, M. C. 2005. Risk identification and assessment in a risk based audit environment: the
effects of budget constraints and decision aid use. Texas A&M University.
Erickson, M., B. W. Mayhew, and W. L. Felix. 2000. Why Do Audits Fail? Evidence from
Lincoln Savings and Loan. Journal of Accounting Research 38 (1): 165–194.
126
Felix, W., AA Gramling, and M. Maletta. 2001. The contribution of internal audit as a
determinant of external audit fees and factors influencing this contribution. Journal of
Accounting Research 39 (3): 513–534.
Fischbacher, U. 2007. z-Tree: Zurich toolbox for ready-made economic experiments.
Experimental Economics 10 (2): 171–178.
Fukukawa, H., and T. J Mock. 2010. Audit Risk Assessments Using Belief Versus Probability.
Auditing: A Journal of Practice & Theory.
Fukukawa, Hironori, Theodore J. Mock, and Arnold Wright. 2006. Audit Programs and Audit
Risk: A Study of Japanese Practice. International Journal of Auditing 10 (1): 41–65.
Fukukawa, Hironori, Theodore J Mock, and Arnold Wright. 2011. Client Risk Factors and Audit
Resource Allocation Decisions. Abacus 47 (1): 85–108.
Grenier, J. H. 2011. Encouraging Professional Skepticism in the Industry Specialization Era.
Available at SSRN: http://ssrn.com.ezproxy.library.wisc.edu/abstract=1533622.
Grenier, J. H., B. Pomeroy, and A. B. Reffett. 2012. Speak Up or Shut Up? The Moderating Role
of Credibility on Auditor Remedial Defense Tactics. Available at SSRN:
http://ssrn.com.ezproxy.library.wisc.edu/abstract=1857170.
Hackenbrack, K., and W. R Knechel. 1997. Resource Allocation Decisions in Audit
Engagements. Contemporary Accounting Research 14 (3): 481–499.
Hammersley, J. S. 2006. Pattern identification and industry-specialist auditors. The Accounting
Review 81 (2): 309–336.
Harris, S. 2009. 2009-12-17_Statement_Harris.pdf. Available at:
http://pcaobus.org/Rules/Rulemaking/Docket%20026/2009-12-17_Statement_Harris.pdf.
Hogarth, R., and H. Einhorn. 1992. Order effects in belief updating: The belief-adjustment
model. Cognitive Psychology 24 (1): 1–55.
Hunton, J. E., A. M Wright, and S. Wright. 2004. Are financial auditors overconfident in their
ability to assess risks associated with enterprise resource planning systems. Journal of
Information Systems 18 (2): 7–28.
Hurtt, R. K. 2010. Development of a Scale to Measure Professional Skepticism. Auditing: A
Journal of Practice & Theory 29 (1): 149.
IAASB. 2009a. International Standard on Auditing (ISA) 315, “Identifying and Assessing the
Risks of Material Misstatement through Understanding the Entity and Its Environment.”
———. 2009b. International Standard on Auditing (ISA) 240, “The Auditor’s Responsibilities
Relating to Fraud in an Audit of Financial Statements.”
Kachelmeier, S. J., T. M. Majors, and M. G. Williamson. 2011. Does Willful Intent Modify
Risk-Based Auditing? Available at SSRN:
http://ssrn.com.ezproxy.library.wisc.edu/paper=1950791.
127
Kahle, J., R. Pinsker, and R. Pennington. 2005. Belief Revision in Accounting: A Literature
Review of the Belief-Adjustment Model. In Advances in Accounting Behavioral
Research, 8:1–40. Bingley: Emerald (MCB UP ).
Knechel, W. Robert. 2007. The business risk audit: Origins, obstacles and opportunities.
Accounting, Organizations and Society 32 (4-5): 383–408.
Knechel, W. Robert, S. E. Salterio, and N. Kochetova-Kozloski. 2010. The effect of
benchmarked performance measures and strategic analysis on auditors’ risk assessments
and mental models. Accounting, Organizations and Society 35 (3): 316–333.
Kochetova-Kozloski, N., T. M. Kozloski, and W. F. Messier Jr. 2010. The Effect of an Entity’s
Risk Management Process and Business Process Analysis on Auditor Risk Identification
and Assessment. Available at SSRN:
http://ssrn.com.ezproxy.library.wisc.edu/abstract=1578923.
Krull Jr., G., P. M. J. Reckers, and B. Wong-On-Wing. 1993. The effect of experience,
fraudulent signals and information presentation order on auditors’ beliefs. Auditing: A
Journal of Practice & Theory 12 (2): 143–153.
Lejarraga, T. 2010. When experience is better than description: time delays and complexity.
Journal of Behavioral Decision Making 23 (1): 100–116.
Lemon, W. M., K. W. Tatum, and W. S. Turley. 2000. Developments in the audit methodologies
of large accounting firms. London England: Auditing Practices Board.
Libby, R., R. Bloomfield, and M. W. Nelson. 2002. Experimental research in financial
accounting. Accounting, Organizations and Society 27 (8): 775–810.
Messier Jr., W. F., and R. M. Tubbs. 1994. Recency effects in belief revision: the impact of audit
experience and the review process. Auditing: A Journal of Practice & Theory 13 (1): 57–
72.
Mock, Theodore J., and Arnold Wright. 1993. An Exploratory Study of Auditors’ Evidential
Planning Judgments. Auditing: A Journal of Practice & Theory 12 (2): 39–61.
Mock, Theodore J., and Arnold M. Wright. 1999. Are Audit Program Plans Risk-Adjusted?
Auditing 18 (1): 55–74.
Monroe, G., and J. Ng. 2000. An examination of order effects in auditors’ inherent risk
assessments. Accounting and Finance 40 (2): 153–167.
van Nieuw Amerongen, C. M. 2007. Auditor’s performance in risk and control judgments.
Dissertation - Vrije Universiteit Amsterdam.
O’Donnell, E., and J. D. Perkins. 2011. Assessing Risk with Analytical Procedures: Do SystemsThinking Tools Help Auditors Focus on Diagnostic Patterns? Auditing: A Journal of
Practice & Theory 30 (4): 273–283.
O’Donnell, E., and J. Schultz. 2005. The Halo Effect in Business Risk Audits: Can Strategic
Risk Assessment Bias Auditor Judgment about Accounting Details? The Accounting
Review 80 (3): 921–939.
128
O’Keefe, T. B., D. A. Simunic, and M. T. Stein. 1994. The production of audit services:
Evidence from a major public accounting firm. Journal of Accounting Research 32 (2):
241–261.
PCAOB. 2010. Auditing standards related to the Auditor’s assessment of and response to risk
and related amendments to PCAOB Standards. PCAOB Release No. 2010-004.
Peecher, M., and Ira Solomon. 2001. Theory and experimentation in studies of audit judgments
and decisions: Avoiding common research traps. International Journal of Auditing 5 (3):
193–203.
Peecher, Mark E., R. Schwartz, and Ira Solomon. 2007. It’s all about audit quality: Perspectives
on strategic-systems auditing. Accounting, Organizations and Society 32 (4-5): 463–485.
Piercey, M. D. 2009. Motivated reasoning and verbal vs. numerical probability assessment:
Evidence from an accounting context. Organizational Behavior and Human Decision
Processes 108 (2): 330–341.
———. 2011. Documentation Requirements and Audit Risk Assessment. Auditing: A Journal of
Practice & Theory 30 (4): 223–248.
Pinsker, R. 2007. Long series of information and nonprofessional investors’ belief revision.
Behavioral Research In Accounting 19 (1): 197–214.
———. 2011. Primacy or recency? A study of order effects when nonprofessional investors are
provided a long series of disclosures. Behavioral Research in Accounting 23 (1): 161–
183.
Public Oversight Board. 2000. The Panel on Audit Effectiveness Repor and Recommendations.
Public Oversight Board.
Rakow, T, and B. Newell. 2010. Degrees of uncertainty: An overview and framework for future
research on experience-based choice. Journal of Behavioral Decision Making 23 (1): 1–
14.
Rakow, T., B. R Newell, K. Fayers, and M. Hersby. 2005. Evaluating three criteria for
establishing cue-search hierarchies in inferential judgment. Journal of Experimental
Psychology: Learning, Memory, and Cognition 31 (5): 1088–1104.
Rakow, Tim, K. A. Demes, and Ben R. Newell. 2008. Biased samples not mode of presentation:
Re-examining the apparent underweighting of rare events in experience-based choice.
Organizational Behavior and Human Decision Processes 106 (2): 168–179.
Rittenberg, L. E. 1998. Book Review. The Accounting Review 73 (3): 411–413.
Rittenberg, L. E., K. Johnstone, and Audrey Gramling. 2012. Auditing: A Business Risk
Approach. 8th ed. South-Western Cengage Learning.
Robson, K., C. Humphrey, R. Khalifa, and J. Jones. 2007. Transforming audit technologies:
business risk audit methodologies and the audit field. Accounting, Organizations and
Society 32 (4-5): 409–438.
129
Schultz Jr., J. J., James Lloyd Bierstaker, and E. O’Donnell. 2010. Integrating business risk into
auditor judgment about the risk of material misstatement: The influence of a strategicsystems-audit approach. Accounting, Organizations and Society 35 (2): 238–251.
Shelton, S. W., J. L. Koehn, and D. Sinason. 2009. Influence of Business Risk Assessment on
Auditors’ Planned Audit Procedures. Academy of Accounting and Financial Studies
Journal 13 (2): 69–86.
Trompeter, G., and Arnold Wright. 2010. The World Has Changed-Have Analytical Procedure
Practices? Contemporary Accounting Research 27 (2): 669–700.
Waller, W. S. 1993. Auditors’ assessments of inherent and control risk in field settings. The
Accounting Review: 783–803.
Yechiam, E., I. Erev, and G. Barron. 2006. The effect of experience on using a safety device.
Safety Science 44 (6): 515–522.
130
APPENDIX 1
Experimental Instructions
Thank you for participating. Please read, sign and turn in the consent form you received
to the researcher. Prior to beginning the experiment, you must sign that form to participate. You
may choose not to participate. A copy of that form is available for you to take with you in case
you have future questions.
Instructions
Your objective is to correctly predict whether there is a HIGH or LOW risk of a financial
reporting error (“error”) in the current period. To help you with your decision, you will have the
opportunity to choose to review financial reporting risk (“risk”) cues at a cost. These risk cues
provide signals of whether the current period has an error. Selecting error risk has an impact on
net earnings; accurately predicting error risk will maximize earnings in each period.
We will have two sessions today. Each session will contain a number of periods as
outlined below. The second session will follow immediately after the first session. Your net
earnings in each session will be aggregated and paid to you at the end of the second session.
Session Procedures:
History screen: Starting in period two of each session, and before moving to the active screen,
you are shown a screen with a history of the results from all prior periods. Once you have
reviewed the information, select the “Continue” button to move onto the active screen of the
current period. Please note, there will be a slight delay before the “Continue” button is available
to ensure time to review prior period results.
Active Screen: In each period, you will receive an endowment in experimental dollars. You then
have the opportunity to select, or not select, various risk cues to review. Once you have actively
chosen to review or to not review the risk cues, you are required to predict whether there is a
HIGH or LOW risk of an error in the current period. Once you have made this prediction, the
131
system will automatically calculate your effort cost and penalty, as applicable. Earnings are then
updated, and the period ends.
Periods: In the first session, there will be a minimum of 66 periods. Starting at the end of 65
periods, there will be an eighty percent chance that the following period will be the final period.
In the second session, there will be a minimum of 51 periods. Starting at the end of 50 periods,
there will be an eighty percent chance that the following period will be the final period.
In both sessions, there is a possibility that any period will contain an error. Errors relate
to a material misstatement in the Company financial statements. You have no control over
whether there is an error in any period. Your objective is to correctly predict the risk of an error
in the current period. Attributes are similar in both sessions.
After the final period of the second session, you will be asked to complete a series of
questions, your earnings from each session will be combined, and your cash balance will be
translated to U.S. dollars to be paid out in private. If you have a negative balance, you will
receive nothing from this part of the experiment. However, you will still earn the $5.00 show up
fee.
Earnings:
In each period, you will receive an endowment of (E$) 8,000 experimental dollars. Your
net earnings for each period are based on the following formula:
Net Earnings = E$ 8,000 endowment – (number of risk cues selected to view * risk cue cost)
– effort cost – penalty [if applicable]
Earnings variables are outlined below.
Risk cue cost: Risk cues provide an indication regarding whether there is an error in the
current period, but provide varying levels of reliability. There is a cost to review risk
cues. In the first 15 periods of the first session, cue costs are zero. However, starting in
period 16 of the first session, there is a charge of E$ 50 (550) for each cue you review. In
132
all periods of the second session, there is a charge of E$ 50 (550) for each cue you
review.
Effort Cost: Your objective in both sessions is to correctly predict the level of risk that
there will be an undetected error in the current period. The level of risk selected directly
impacts your effort in the current period. Effort is costly; it costs E$ 1,900 to select the
LOW error risk, corresponding to LOW effort, and E$ 4,400 to select the HIGH error
risk, corresponding to HIGH effort.
Penalty: In periods with an error, you will receive a penalty. The penalty assessed will
vary depending on your level of effort for that period. Selecting LOW error risk for the
period corresponds to low effort. In those periods where there is an error, you will be
assessed a high penalty of E$ 13,750 when you select LOW error risk.
Conversely, selecting HIGH error risk for the period corresponds to high effort. In
those periods where there is an error, you will be assessed a lower penalty of E$ 3,800
when you select HIGH error risk. Therefore, when there is an error in the current period,
selecting HIGH effort, will minimize your penalty assessment. Conversely, when there is
no error in the period, selecting LOW effort will maximize your earnings.
133
Earnings Summary:
The chart below provides a summary of possible net earning amounts in each period based on
HIGH or LOW effort selection and error outcomes. The previously discussed, net earnings
formula is included for reference:
Net Earnings = E$ 8,000 endowment – (# of risk cues selected to view * risk cue cost) –
effort cost – penalty [if applicable]
Endowment
Risk Cue Cost
Effort Cost
Penalty
Net Earnings
High Effort and NO Error High Effort and Error
Low Effort and NO Error
Low Effort and Error
8,000
8,000
8,000
8,000
(# of risk cues selected x (# of risk cues selected x (# of risk cues selected x (# of risk cues selected x
(E$)
risk cue cost)
risk cue cost)
risk cue cost)
risk cue cost)
(E$)
(4,400)
(4,400)
(1,900)
(1,900)
(E$)
(3,800)
(13,750)
(E$)
3,600 - (# of risk cues
(E$) selected x risk cue cost)
(200) - (# of risk cues
selected x risk cue cost)
6,100 - (# of risk cues
selected x risk cue cost)
(7,650) - (# of risk cues
selected x risk cue cost)
134
Active Screen Description:
Figure 1, below, shows the active session screen. Please note: for illustration purposes,
this view contains all sections of the active screen at once and is not representative of the actual
results. During each period, only certain areas, as discussed below, are available at any one time.
Following figure 1, you are provided with various screen section descriptions and how to interact
with them.
Figure 1 – Screenshot of the session screen
Section “A”
Section “B”
Section “D”
Section “C”
Section “E”
135
Section “A”: This section contains a history of all results from all prior periods.
Please note, if you did not choose to select a risk cue in a prior period, the only
information provided here will be, “Not Selected”.
Once the number of periods is greater than the available space, the table will scroll to
reach the earlier periods.
Section “B”: This area contains the available risk cues as discussed earlier.
In each period, you will need to actively select which cues you would like to expend
additional effort (for the listed cost) to review. You can select the cues in any order and you may
choose not to review some or all the cues as well. There is no requirement that you must review
cues in any period. To review a cue, select the RED “Yes” button to the right of the cue; to
choose not to review a cue, select the RED “No” button to the right of the cue. For each risk cue,
you must select either the “Yes” or “No” button.
For reference, the risk cues are defined below and fall into one of the two following
categories:
Economy: This cue provides an indication as to how events in the overall economy might impact
Company and the likelihood that the current period will have an error which is not identified.
Management: This cue provides insights into changes within management structure,
compensation, staffing changes, etc. at Company which might impact the likelihood of an error
in the current period.
Section “C”: These boxes contain the current area’s risk selection and results.
Once you have decided which risk cues to evaluate, this area will become visible. In this
area, you predict the current period’s error risk. You can either select HIGH or LOW risk. Once
you have made this selection, the remaining boxes will become visible, including current period
results.
Section “D”: This section contains your current earnings and your cumulative cash balance.
136
As you make decisions during the period, this area will track your current earnings as
well as a running total of all your earnings. Current information will only become available as
you make decisions during the period.
In the bottom left corner of the screen, you will find an icon which opens a calculator.
This is provided for your convenience. You are not required to utilize the calculator.
Section “E”: “Next Period” button
The “Next Period” button will be visible after all choices have been made. Select this
button to move on to the next period.
Final Instructions
Please do not talk to your fellow participants during the main trading session. Collusion
is strictly forbidden during the session. If you are found colluding, you will be asked to leave.
137
APPENDIX 2
Screen Shots
History Information Screen
138
Main Screen Example 1
139
Main Screen Example 2
140
Main Screen Example 3
141
APPENDIX 3
Experimental Earnings Discussion
My interest in this study is to determine whether participants make different choices
within the four conditions. Using financial incentives provides the opportunity to examine
payouts for each pattern and cost condition. Based on the cue values and experience patterns
outlined in the main text, I create the following session earnings structure:
Net Earnings = E$ 8,000 endowment – (number of risk cues selected to view * risk-cue cost)
– effort cost – penalty [if applicable]
Where:
endowment = E$ 8,000. In each period, participants are provided the endowment amount
at the beginning of each period. This amount is similar to the audit fee an
auditor would receive from their audit client.
risk-cue cost = Risk cues provide an indication regarding whether there is an error in any
current period, but provide varying levels of reliability. There is a cost to
review risk cues. In the first 15 periods of the first session, cue costs are
zero. However, starting in period 16 of the first session, there is a charge
of E$ 50 (550) for each risk cue reviewed, in the LOW (HIGH) condition.
Risk-cue costs are similar to the effort cost an auditor would expend in
identifying and evaluating risk cues for the business-level risk assessment.
effort cost = The objective in both sessions is to correctly predict the level of risk that
there will be an undetected error in the current period. The level of risk
predicted directly impacts effort in the current period. Effort is costly; it
costs E$ 1,900 to select the LOW error risk, corresponding to LOW effort,
and E$ 4,400 to select the HIGH error risk, corresponding to HIGH effort.
This amount is included to operationalize the cost the auditor would
expend on different levels of audit effort during the audit. In essence, it
mimics the decision of the auditor to focus more or less resources
depending on the perceived risk of the audited area.
penalty cost = In all periods with an error, participants receive a penalty. The penalty
assessed varies depending on effort level selected for that period. In those
periods in which there is an error, participants are assessed a high penalty
of E$ 13,750 when they predict a low error risk. Conversely, in those
periods in which there is an error, participants are assessed a lower penalty
142
of E$ 3,800 when they predict a high error risk. No penalty is assigned in
those periods without an error. Therefore, when there is an error in the
current period, selecting HIGH effort minimizes a participant’s penalty
assessment. Alternately, when there is no error in the period, selecting
LOW effort maximizes the participant’s earnings.
Earnings summary:
Endowment
Risk Cue Cost
(E$)
High Effort and NO Error
High Effort and Error
Low Effort and NO Error
8,000
8,000
8,000
(# of risk cues selected x risk (# of risk cues selected x
(E$)
cue cost)
risk cue cost)
(# of risk cues selected x
risk cue cost)
Low Effort and Error
8,000
(# of risk cues selected x
risk cue cost)
Effort Cost
(E$)
(4,400)
(4,400)
(1,900)
(1,900)
Penalty
(E$)
-
(3,800)
-
(13,750)
Net Earnings
(E$)
3,600 - (# of risk cues
selected x risk cue cost)
(200) - (# of risk cues
selected x risk cue cost)
6,100 - (# of risk cues
selected x risk cue cost)
Appendix 4 provides a summary of the equilibrium payouts.
(7,650) - (# of risk cues
selected x risk cue cost)
143
APPENDIX 4
Equilibrium Payout Summary
I utilize the payout schedule shown in Appendix 3 to generate expected payouts for the
LOW and HIGH cue cost conditions. My goal is to find the maximum expected earning in each
cue cost condition to support the following risk cue choice selection criteria. In the LOW
condition, individuals should continue to choose risk cues until there is an indication of an error
(or until both are selected), if either risk cue indicates an error, the participant chooses high risk.
If both risk cues indicate no-error, the participant predicts low risk. However, in the HIGH cue
cost condition, participants only select one risk cue and choose high or low risk based on
whether the cue indicates there is an error or no-error in the period.
Risk cues become costly after the practice periods. Therefore, I base my expected value
calculations as of this point as subjects would begin evaluating the possible earnings in each
period based on cue costs, but also have some experience with the performance of cues and
period results. Participants observe 12/15 (80%) NO-ERROR periods, and 12/15 (80%) cue
accuracy for both cues during the practice periods. Therefore, I use 80% as my basis in the
expected value calculations.
As shown in the tables below, in the LOW cue cost condition, participants earn an
expected payout of E$ 31.60, in each period, more by selecting one or two risk cues, based on
the above criteria. However, in the HIGH cue cost condition, participants earn E$ 468.40, more
by selecting one risk cue.
144
Likewise, selecting more risk cues than provided by the above decision criteria is
inefficient and leads to a less than maximum earnings payout. Further selecting either all error or
no-error in all periods will lead to minimum earnings. In addition, selecting error or no-error in
the current period based on the results of the prior period also lead to less than maximum
earnings.
Summary payoff table (for reference, and no cue costs included in this table)
P(No Error)
P(Error)
High Effort Payoff
3,600
(200)
Low Effort Payoff
6,100
(7,650)
Assumptions:
No Error Rate:
80%
12/15 NO_ERROR practice periods
Cue 1 Accuracy:
80%
Accurate 12/15 practice periods
Cue 2 Accuracy:
80%
Accurate 12/15 practice periods
Note: Assumptions based on 15 practice periods
145
Expected value by cue for E$ 50 cue cost
Result
Posterior Probability
Cue one payoffs
P(Cue shows NO-ERROR|result)
NO-ERROR
0.941
ERROR
0.059
Total
P(Cue shows ERROR|result)
NO-ERROR
0.500
ERROR
0.500
Total
Cue two payoffs
Based on cue 1 showing NO-ERROR
P(Cue shows NO-ERROR|result)
NO-ERROR
0.985
ERROR
0.015
Total
P(Cue shows ERROR|result)
NO-ERROR
0.800
ERROR
0.200
Total
Based on cue 1 showing ERROR
P(Cue shows NO-ERROR|result)
NO-ERROR
0.800
ERROR
0.200
Total
P(Cue shows ERROR|result)
NO-ERROR
0.200
ERROR
0.800
Total
High effort choice
payout
Low effort choice
payout
Optimal effort
choice selection
Join probability
Expected payout
based on optimal
choice selection
Total expected
payout by cue
3,341.18
(14.71)
3,326.47
5,694.12
(452.94)
5,241.18
Low Effort
0.680
3,564.00
1,775.00
(125.00)
1,650.00
3,025.00
(3,850.00)
(825.00)
High Effort
0.320
528.00
3,446.15
(4.62)
3,441.54
5,907.69
(119.23)
5,788.46
Low Effort
0.520
3,010.00
2,800.00
(60.00)
2,740.00
4,800.00
(1,550.00)
3,250.00
Low Effort
0.160
520.00
2,800.00
(60.00)
2,740.00
4,800.00
(1,550.00)
3,250.00
Low Effort
0.160
520.00
700.00
(240.00)
460.00
1,200.00
(6,200.00)
(5,000.00)
High Effort
0.160
73.60
4,123.60
Incremental difference in expected earnings going from one to two cues:
31.60
4,092.00
145
146
Expected value by cue for E$ 550 cue cost
Posterior
Result
Probability
Cue one payoffs
High effort choice
payout
Low effort choice
payout
Optimal effort
choice selection
Join probability
Expected payout
based on optimal
choice selection
P(Cue shows NO-ERROR|result)
NO-ERROR
0.941
ERROR
0.059
Total
2,870.59
(44.12)
2,826.47
5,223.53
(482.35)
4,741.18
Low Effort
0.680
3,224.00
P(Cue shows ERROR|result)
NO-ERROR
ERROR
Total
1,525.00
(375.00)
1,150.00
2,775.00
(4,100.00)
(1,325.00)
High Effort
0.320
368.00
2,461.54
(20.00)
2,441.54
4,923.08
(134.62)
4,788.46
Low Effort
0.520
2,490.00
2,000.00
(260.00)
1,740.00
4,000.00
(1,750.00)
2,250.00
Low Effort
0.160
360.00
2,000.00
(260.00)
1,740.00
4,000.00
(1,750.00)
2,250.00
Low Effort
0.160
360.00
500.00
(1,040.00)
(540.00)
1,000.00
(7,000.00)
(6,000.00)
High Effort
0.160
(86.40)
0.500
0.500
Cue two payoffs
Based on cue 1 showing NO-ERROR
P(Cue shows NO-ERROR|result)
NO-ERROR
0.985
ERROR
0.015
Total
P(Cue shows ERROR|result)
NO-ERROR
0.800
ERROR
0.200
Total
Based on cue 1 showing ERROR
P(Cue shows NO-ERROR|result)
NO-ERROR
0.800
ERROR
0.200
Total
P(Cue shows ERROR|result)
NO-ERROR
0.200
ERROR
0.800
Total
Incremental difference in expected earnings going from one to two cues:
Total expected
payout by cue
3,592.00
3,123.60
(468.40)
146
147
To confirm that the overall ex-post payouts support the risk cue selection criteria outlined
above, I also examine the payouts generated by the session distributions. As shown below, the
maximum expected payout is supported by the choice of selecting one risk cue in the HIGH cue
cost condition, and one or two risk cues in the LOW cue cost condition, as detailed earlier.
To generate these payouts, I utilized an Excel spreadsheet including my distributions to
calculate the expected payouts for each period based on the above criteria, as well as fir selecting
all HIGH effort and selecting all LOW effort. For the HIGH cue cost criteria, I calculated the
expected earnings both for a selection of the first risk cue, as well as the second risk cue.
With practice periods
NO-PATTERN Data Set
All high effort (no cues selected)
All low effort (no cues selected)
Follow last period only
Cue 1 only
Cue 2 only
Decision using Cue 1 start
Decision using Cue 2 start
PATTERN Data Set
All high effort (no cues selected)
All low effort (no cues selected)
Follow last period only
Cue 1 only
Cue 2 only
Decision using Cue 1 start
Decision using Cue 2 start
Both sessions
NO-PATTERN & PATTERN Data Sets
All high effort (no cues selected)
All low effort (no cues selected)
Follow last period only
Cue 1 only
Cue 2 only
Decision using Cue 1 start
Decision using Cue 2 start
PATTERN & NO-PATTERN Data Sets
All high effort (no cues selected)
All low effort (no cues selected)
Follow last period only
Cue 1 only
Cue 2 only
Decision using Cue 1 start
Decision using Cue 2 start
Cue Cost: 50
E$
Cue Cost: 550
169,200
155,100
162,300
226,950
229,400
240,000
240,050
169,200
155,100
162,300
201,450
203,900
197,000
197,550
Cue Cost: 50
Cue Cost: 550
180,600
196,350
216,050
258,300
260,750
266,200
266,250
180,600
196,350
216,050
232,800
235,250
221,200
221,750
Cue Cost: 50
E$
Cue Cost: 550
307,200
301,200
335,600
430,050
430,000
446,050
446,150
307,200
301,200
335,600
379,050
379,000
358,050
359,150
Cue Cost: 50
Cue Cost: 550
307,200
301,200
335,600
430,050
430,000
446,050
446,150
307,200
301,200
335,600
379,050
379,000
358,050
359,150
No practice periods
NO-PATTERN Data Set
All high effort (no cues selected)
All low effort (no cues selected)
Follow last period only
Cue 1 only
Cue 2 only
Decision using Cue 1 start
Decision using Cue 2 start
PATTERN Data Set
All high effort (no cues selected)
All low effort (no cues selected)
Follow last period only
Cue 1 only
Cue 2 only
Decision using Cue 1 start
Decision using Cue 2 start
Spring 2012 Additional sessions
Includes PATTERN series
All high effort (no cues selected)
All low effort (no cues selected)
Follow last period only
Cue 1 only
Cue 2 only
Decision using Cue 1 start
Decision using Cue 2 start
PATTERN series randomized
All high effort (no cues selected)
All low effort (no cues selected)
Follow last period only
Cue 1 only
Cue 2 only
Decision using Cue 1 start
Decision using Cue 2 start
Cue Cost: 50
E$
Cue Cost: 550
126,600
104,850
119,550
171,750
169,250
179,850
179,900
126,600
104,850
119,550
146,250
143,750
136,850
137,400
Cue Cost: 50
Cue Cost: 550
138,000
146,100
173,300
203,100
200,600
206,050
206,100
138,000
146,100
173,300
177,600
175,100
161,050
161,600
Cue Cost: 50
E$
Cue Cost: 550
307,200
301,200
335,600
430,050
430,000
446,050
446,150
Cue Cost: 50
292,000
246,200
270,600
404,850
404,800
421,050
421,150
N/A -- ONLY
E$ 50 CUE
COST
CONDITION
UTILIZED IN
ADDITIONAL
SESSIONS
Cue Cost: 550
N/A -- ONLY
E$ 50 CUE
COST
CONDITION
UTILIZED IN
ADDITIONAL
SESSIONS
148
APPENDIX 5
Additional Experimental Sessions Instructions
Thank you for participating. Please read, sign and turn in the consent form you received
to the researcher. Prior to beginning the experiment, you must sign that form to participate. You
may choose not to participate. A copy of that form is available for you to take with you in case
you have future questions.
Instructions
Your objective is to correctly predict whether there is a HIGH or LOW risk of a financial
reporting error (“error”) in the current period. To help you with your decision, you will have the
opportunity to choose to review financial reporting risk (“risk”) cues at a cost. These risk cues
provide signals of whether the current period has an error. Selecting error risk has an impact on
net earnings; accurately predicting error risk will maximize earnings in each period.
There will be one session today. This session will contain a number of periods as outlined
below. Your net earnings will be paid to you at the end of the session.
Session Procedures:
History screen: Starting in period two of the session, and before moving to the active screen, you
are shown a screen with a history of the results from all prior periods. Once you have reviewed
the information, select the “Continue” button to move onto the active screen of the current
period. Please note, there will be a slight delay before the “Continue” button is available to
ensure time to review prior period results.
Active Screen: In each period, you will receive an endowment in experimental dollars. You then
have the opportunity to select, or not select, various risk cues to review. Once you have actively
chosen to review or to not review the risk cues, you are required to predict whether there is a
HIGH or LOW risk of an error in the current period. Once you have made this prediction, the
system will automatically calculate your effort cost and penalty, as applicable. Earnings are then
updated, and the period ends.
149
Periods: There will be a minimum of 117 periods. Starting at the end of 116 periods, there will
be an eighty percent chance that the following period will be the final period.
In this session, there is a possibility that any period will contain an error. Errors relate to a
material misstatement in the Company financial statements. You have no control over whether
there is an error in any period. In addition, the results of each period are not a function of prior or
future periods. Your objective is to correctly predict the risk of an error in the current period.
After the final period, you will be asked to complete a series of questions, and your cash
balance will be translated to U.S. dollars to be paid out in private. If you have a negative balance,
you will receive nothing from this part of the experiment. However, you will still earn the $5.00
show up fee.
Earnings:
In each period, you will receive an endowment of (E$) 8,000 experimental dollars. Your
net earnings for each period are based on the following formula:
Net Earnings = E$ 8,000 endowment – (number of risk cues selected to view * risk cue cost)
– effort cost – penalty [if applicable]
Earnings variables are outlined below.
Risk cue cost: Risk cues provide an indication regarding whether there is an error in the
current period. Each risk cue provides an indication of the current period error. They are
not based on a function of prior or future periods or the other risk cue. There is a cost to
review risk cues. In the first 15 periods of the session, cue costs are zero. However,
starting in period 16 of the session, there is a charge of E$ 50 for each cue you review.
Effort Cost: Your objective is to correctly predict the level of risk that there will be an
undetected error in the current period. The level of risk selected directly impacts your
effort in the current period. Effort is costly; it costs E$ 1,900 to select the LOW error risk,
corresponding to LOW effort, and E$ 4,400 to select the HIGH error risk, corresponding
to HIGH effort.
150
Penalty: In periods with an error, you will receive a penalty. The penalty assessed will
vary depending on your level of effort for that period. Selecting LOW error risk for the
period corresponds to low effort. In those periods where there is an error, you will be
assessed a high penalty of E$ 13,750 when you select LOW error risk.
Conversely, selecting HIGH error risk for the period corresponds to high effort. In
those periods where there is an error, you will be assessed a lower penalty of E$ 3,800
when you select HIGH error risk. Therefore, when there is an error in the current period,
selecting HIGH effort, will minimize your penalty assessment. Conversely, when there is
no error in the period, selecting LOW effort will maximize your earnings.
Earnings Summary:
The chart below provides a summary of possible net earning amounts in each period based on
HIGH or LOW effort selection and error outcomes. The previously discussed, net earnings
formula is included for reference:
Net Earnings = E$ 8,000 endowment – (# of risk cues selected to view * risk cue cost) –
effort cost – penalty [if applicable]
Endowment
Risk Cue Cost
Effort Cost
Penalty
Net Earnings
High Effort and NO Error High Effort and Error
Low Effort and NO Error
Low Effort and Error
8,000
8,000
8,000
8,000
(# of risk cues selected x (# of risk cues selected x (# of risk cues selected x (# of risk cues selected x
(E$)
risk cue cost)
risk cue cost)
risk cue cost)
risk cue cost)
(E$)
(4,400)
(4,400)
(1,900)
(1,900)
(E$)
(3,800)
(13,750)
(E$)
3,600 - (# of risk cues
(E$) selected x risk cue cost)
(200) - (# of risk cues
selected x risk cue cost)
6,100 - (# of risk cues
selected x risk cue cost)
(7,650) - (# of risk cues
selected x risk cue cost)
151
Active Screen Description:
Figure 1, below, shows the active session screen. Please note: for illustration purposes,
this view contains all sections of the active screen at once and is not representative of the actual
results. During each period, only certain areas, as discussed below, are available at any one time.
Following figure 1, you are provided with various screen section descriptions and how to interact
with them.
Figure 1 – Screenshot of the session screen
Section “A”
Section “B”
Section “D”
Section “C”
Section “E”
152
Section “A”: This section contains a history of all results from all prior periods.
Please note, if you did not choose to select a risk cue in a prior period, the only
information provided here will be, “Not Selected”.
Once the number of periods is greater than the available space, the table will scroll to
reach the earlier periods.
Section “B”: This area contains the available risk cues as discussed earlier.
In each period, you will need to actively select which cues you would like to expend
additional effort (for the listed cost) to review. You can select the cues in any order and you may
choose not to review some or all the cues as well. There is no requirement that you must review
cues in any period. To review a cue, select the RED “Yes” button to the right of the cue; to
choose not to review a cue, select the RED “No” button to the right of the cue. For each risk cue,
you must select either the “Yes” or “No” button.
For reference, the risk cues are defined below and fall into one of the two following
categories:
Economy: This cue provides an indication as to how events in the overall economy might impact
Company and the likelihood that the current period will have an error which is not identified.
Management: This cue provides insights into changes within management structure,
compensation, staffing changes, etc. at Company which might impact the likelihood of an error
in the current period.
Section “C”: These boxes contain the current area’s risk selection and results.
Once you have decided which risk cues to evaluate, this area will become visible. In this
area, you predict the current period’s error risk. You can either select HIGH or LOW risk. Once
you have made this selection, the remaining boxes will become visible, including current period
results.
Section “D”: This section contains your current earnings and your cumulative cash balance.
153
As you make decisions during the period, this area will track your current earnings as
well as a running total of all your earnings. Current information will only become available as
you make decisions during the period.
In the bottom left corner of the screen, you will find an icon which opens a calculator.
This is provided for your convenience. You are not required to utilize the calculator.
Section “E”: “Next Period” button
The “Next Period” button will be visible after all choices have been made. Select this
button to move on to the next period.
Final Instructions
Please do not talk to your fellow participants during the main trading session. Collusion
is strictly forbidden during the session. If you are found colluding, you will be asked to leave.