Building confi dence in IT programs

Insights on IT risk
Business briefing
September 2011
Building confidence
in IT programs
Facilitating success through
program risk management
Key issues to blame for
failures in IT programs
are not being identified
properly until after
they have occurred.
Success in IT programs
translates to success
in business
We believe there has never been a better or more
important time for organizations to review how they plan,
execute and realize benefits from strategic IT programs.
Organizations are looking to IT as a key enabler to help
them realize business strategies, improve productivity and
obtain a competitive advantage through product and service
innovation. IT is a focal point for executives seeking to drive cost
competitiveness and transformation agendas that have become
part of the global economy over the past three years.
But even as IT investments are again set to increase significantly
in the coming years, strategic IT program success rates are
still underperforming and in need of attention by organizations
looking to spend their hard-earned capital wisely. Strategic
IT programs are clearly on the rise, but so are expectations.
Organizations are realizing that they must respond to increasing
pressure to improve the return on their program investments.
Today’s strategic IT programs are not only expected to be
delivered on time and on budget, but also to deliver multiple,
high-level business benefits.
Key issues to blame for failures in IT programs are often not
identified properly until after they have occurred. By the time
issues are identified (often in a crisis), it’s too late to for a positive
outcome, and even the time for damage control is limited and
sometimes missed altogether.
Organizations need to challenge the status quo and ask themselves
how they can better manage risks around underperforming
programs and improve performance rates to deliver sustained
benefits. IT program risk management (PRM) can help to increase
the success of strategic IT initiatives. IT PRM provides the means to
better protect organizations from common IT program pitfalls and
increase the likelihood of delivering successful program outcomes.
Insights on IT risk | September 2011
1
While IT spending is increasing, program
IT spending scheduled to increase
An upsurge in spending in IT projects and programs is expected.
IT research and advisory firm, Gartner, has indicated that
worldwide IT expenditure in 2011 is estimated at $3.6 trillion,
a 7.1% increase from 2010. Gartner predicts an increase in IT
spending will be sustained at an average rate of 5.3% per year
through 2015. Gartner also indicates that approximately 20% to
50% of a company’s IT spending will be focused on programs and
projects — depending on an organization’s initiatives. In 2011, this
represents an expected increase in spending of up to $2.16 trillion
on technology-enabled programs and projects.
In addition, Ernst & Young’s 2011 report, Turn risks and opportunities
into results, has identified investing in IT as a top-three priority (see
Figure 1 below). In fact, across Europe, America and the Middle
East, investing in IT is typically either the top or second-highest
priority for executives.
A key driver for the increase in investment in IT is that organizations
that have failed to invest in IT in the past few years have run the risk
of not keeping pace with business demands for increased efficiency,
improved performance and of legacy systems not being supported by
software vendors as time progresses.
A recent Forrester Trends 2011 enterprise resource planning (ERP)
report highlighted that companies are lagging in investing in ERP
projects but will need to reinvest soon to keep up to date with
upgrades and demand for ERP cloud services. Forrester reported
that the “ERP software market was hit hard by the recession but
rebounded last year. … [Roughly] half the companies surveyed are
running on product releases that are two releases behind current.”
A second driver is the rapid uptake of new technologies, such
as cloud computing and mobile technologies, which offer new
ways of working and opportunities for efficiency and innovation.
Organizations are increasingly competing on innovation in new
products and services, with IT and emerging technologies seen as
both a risk and opportunity for companies wishing to differentiate
in the market and improve productivity and performance.
The research highlights that organizations have little choice
but to invest continuously in IT and IT programs or suffer the
consequences of an aging and underperforming application and
infrastructure landscape, ultimately affecting competitiveness.
Figure 1: Global risks and opportunities
Top five global opportunities for 2011
Cost
cutting
Perceived risk impact
Regulation
and compliance
Managing
talent
Pricing
pressure
Market
risks
Emerging
technologies
Expansion of
government’s role
Slow recovery or
double-dip recession
Social acceptance/CSR
Access to capital
Perceived scale of opportunity
Top five global risks for 2011
Risk mitigation measures needed but
not yet implemented
Source: Turning risks and opportunities into results, Ernst & Young, 2011.
2
Insights on IT risk | September 2011
Improving execution
of strategy across
business functions
Investing in
products, services
and operations
Excellence in
investor relations
Investing
in IT
Investing
in processes,
tools and training
to achieve greater
productivity
Investing in cleantech and
emerging markets demand growth
New market channels
Public-private
partnership
Mergers and
acquisitions
Measures to exploit opportunity needed
but not yet implemented
success rates are not
IT programs continue to underperform
Across all industries, organizations continue poor performance in
successfully delivering IT programs, achieving expected outcomes
and sustaining benefits. Reports by Gartner show that IT program
success rates vary between 30% and 50%, remaining relatively
flat over the years. In addition, a PANORAMA study of 1,600 ERP
projects in 2010 indicated that 70% of Tier I ERP clients fail to
realize at least 50% of business benefits. In addition, 51% of current
implementations, at the time of PANORAMA’s survey, were at risk
of going over budget.
While companies have invested significantly in increasing their
knowledge and capabilities in program and project management,
this is not visible in the success rates. In our opinion, the lack of
improvement is mainly due to increased complexity in business
processes and the emerging technology landscape. Organizations
are still failing to properly adapt their program approaches to this
increased complexity.
IT program risk is a key megatrend to be addressed
An Ernst & Young report from 2011, The evolving IT risk landscape,
identified programs and change management as the most significant
megatrends in IT that need to be managed. The paper highlighted
the risks related to IT programs next to several other risk categories
such as cybercrime, cloud computing and consumerization.
Figure 2: IT risk universe
Business environment
IT pro_am risc mana_ement
Transparency
and conÕdence
g n m e nt
l i ty
A li
a bi
Chan_e a_enda
Physical
environment
Infrastructure
Potentially reduced
cost of operations
Avoidance of
security breaches
Enhanced capability
Protect
brand
Manage
compliance
and expectation
Seamless
customer
experience
tia
a il
Internal threats
Business
objectives
Enable
innovation
and change
Predictability
Security
and privacy
IT risk
universe
Cybercrime
den
y
li t
Economic
recovery and
market volatility
Strategy
Applications
and databases
Resiliency
Co n Ô
New business
models and
technology
Cloud computin_
E
Pro_rams
and chan_e
mana_ement
KtafÕn_
Data
Av
Regulatory
pressure
Consumerization
De_al and
re_ulatory
Third-party
suppliers and
outsourcin_
Outcomes
s
nes
ive
ct
ffe
Globalization
Margin
pressure
EfÔciency
IT me_atrends
Com
pli
an
ce
Market
forces
Drive growth
Operations
Investor
conÕdence
I n te g r i t y
Operational
excellence
Source: The evolving IT risk landscape, Ernst & Young, June 2011
Insights on IT risk | September 2011
3
Delivering on the promise of IT is now a crucial
differentiator for corporate competitiveness.
Program failure prevents benefits delivery
Effective program and project management capabilities are now
one of the most crucial differentiators for competitiveness in the
future market place. Given the focus in 2011 on investing in IT,
getting strategic IT programs right could be the make-or-break
difference between realizing business growth and staying afloat.
Strategic IT programs should deliver sustainable business benefits
to create a competitive advantage. An Ernst & Young survey
from 2008, Managing programs for success, found that only 8%
of respondents viewed their company’s ability to obtain business
value from strategic programs as poor, while 33% would have liked
to see some improvement in this area. One explanation for this
may be that in the past, ensuring that programs deliver business
value was a priority of only 15% of the companies. A majority of the
respondents instead focused on the more traditional measures of
success, namely delivering on time and within budget.
Encouragingly, the survey confirmed that a shift toward benefitsdriven programs had begun. Of the companies who responded, 72%
stated that the primary focus of the future will be to make sure that
tangible business benefits and value are realized as a result of their
strategic programs.
While the failure of strategic IT programs can leave companies
exposed to significant increases in costs, reputational damage, loss
of customers and disruption of day-to-day activities, we believe
that in the future, organizations that fail to manage their strategic
IT programs based on realizing business benefits and contributions
to innovation and competitive advantage will find it increasingly
difficult to be relevant to their workforce, customers and the broader
market in which they operate. Indeed, delivering on the promise of
IT is becoming a crucial differentiator for corporate competitiveness.
Figure 3: Focus on strategic IT programs
Historical focus of respondents’ strategic IT programs
Future focus of respondents’ strategic IT programs
100%
100%
75%
75%
72%
53%
50%
50%
32%
25%
15%
0%
16%
12%
Time to
budget
Quality
0%
Time to
budget
Quality
:efeÕtk
delivery
Source: Managing programs for success, Ernst & Young, 2008.
4
25%
Insights on IT risk | September 2011
:efeÕtk
delivery
The big picture: IT program risk universe
There is not one single reason for IT program failure
In our experience, there are usually many factors that result in an
IT program failing to deliver its intended objectives and benefits.
There are typically a number of risks that, combined, result in
programs failing or underperforming. Management needs to be
aware of the organization’s specific IT program risk universe and
implement strategies up front to manage the most likely risks to
program success. The most common causes of complex IT program
failures are:
IT program/project risk universe
Vision and initiation
• Lack of management support
for the program
• Unresolved or uncertain
project vision or strategy
• Poorly defined business
objectives
• Poorly defined project scope
and requirements analysis
• Inadequate assessment of
business impact or priority
• Poorly defined critical success
factors and risk assessment
• Lack of complexity
measurement
• Unclear governance and
decision framework
• Lack of communication and
user group involvement
Planning
Execution
• Aggressive schedule
commitments restricting
proper planning phase
• Inadequate risk assessment,
quantification and allocation
of project risks
• Inappropriate skills, resources
and processes in place
• Incomplete or unrealistic
cost information
• Inadequate understanding
of complexities and
accounting for factors
necessary to succeed
• Mismatched balance between
time, cost, quality and
benefit attainment
• Ineffective prequalification
process
• Incomplete project charter
• Poorly defined contractual
terms and conditions
• Ineffective governance
mechanisms and inconsistent
decision framework
Business acceptance
• Lack of appropriate
accountability and approvals
• Ineffective deployment
strategy
• Ineffective change
management
• Unresolved problems and
disputes
• Inadequate representation of
the “voice of the customer”
• Incomplete operating and
maintenance information
• Shifting budget, scope and
timetables
• Insufficient user satisfaction
• Lack of accountability
• Scale and volume of defects
(e.g., data, test)
• Adversarial team and supplier
relationships
• Missing warranties and
guarantees
• Lack of skills or resources
in program and project
management
• No project go-live review
• Lack of defining appropriate
performance metrics
Measuring and monitoring
• Governance model fails to manage key project internal and
external stakeholders
• Ineffective control of change orders
• Ineffective project management systems
• Poor quality management and assurance plans
• Ineffective project performance monitoring and reporting
• Incomplete design information and changing design and
scope requirements
• Lack of continuity in project staff
• Ineffective communication with stakeholders
• Lack of situational awareness
• Ineffective decision-making and resolution of issues
• Lack of a risk management framework
• Lack of independent progress monitoring and
executive reporting
• Lack of tracking
6
Insights on IT risk | September 2011
The holistic view: program risk implies
enterprise risk
Increased IT program complexity not only makes program
management more challenging, it is also a major driver of risk.
Complex IT programs are associated with considerable uncertainty
and ambiguity. As the degree of complexity increases, so does
the inherent IT program risk, and therefore the need for diligent
IT program governance, risk management and project control. In
addition, the increasing importance of IT programs for reaching
strategic goals implies that the whole business is put at risk when
IT programs fail to deliver expected business benefits.
Figure 4: Risk/complexity matrix
Risk
There is a direct relationship between program complexity and risk
that needs to be assessed and managed throughout the program
life cycle. The risk/complexity matrix (see Figure 4) outlines how, as
the degree of complexity increases, so does the risk, and therefore,
the need for greater governance, risk management and program
controls to protect the broader enterprise also rises.
Complexity is the key driver of risk
Factors of complexity
Time
Risk as a function
of complexity
Team size
Level of innovation and change
Team maturity
Team proximity
Number of internal/external teams
Capability maturity
Degree of learning
Rapid dependent materials
Regulated requirements
Environmental safety
Security requirements
Level of program
governance
required
Complexity
The increasing importance of IT programs for
reaching strategic goals implies that the whole
business is put at risk when IT programs fail to
deliver expected business benefits.
Insights on IT risk | September 2011
7
Key ingredients for effective IT PRM
Using IT PRM to build additional lines of defense
The poor historical performance of IT programs and the magnitude
of the investments in IT can force organizations to take measures
to enhance control and risk management over their strategic IT
programs. A proven method of achieving this is to create multiple
“lines of defense” against the threat of risk. Organizations are
strengthening control by:
1. Appointing experienced risk managers and a risk committee to
take charge of the management of end-to-end program risk — in
addition to the traditional role that a project management office
(PMO) undertakes to log and report project risks and issues.
2. Enhancing the role of internal audit, compliance and enterprise
risk functions to provide assurance coverage where possible
during the implementation of the program.
3. Appointing an external independent PRM provider who is
charged with bringing experience not readily available inside
the company or that other suppliers (e.g., system integrators)
cannot provide due to conflicts of interest.
In essence, an independent IT PRM approach functions as an
additional line of defense for major IT program initiatives.
Defenses include:
• First line of defense — the most crucial layer of risk management
on a program. It typically includes the executive leadership team,
program steering committee, program risk committee, technical
design authority, the PMO, system integrators (SIs) and the
various project workstream leaders.
8
• Second line of defense — the independent IT PRM role. It can
be provided by one independent (mostly external) party, or it can
include a combination of internal and external providers such as:
an independent (external) program risk/quality assurance provider,
operational risk and compliance functions, external auditors and
even software providers.
• Third line of defense — typically includes the audit committee
and internal audit function. Often seen as the last line of defense
when it comes to detecting error and waste in organizational
activities, these functions benefit from being able to rely
on the outputs of a trusted independent party and brand.
An independent IT PRM may even reduce the need for their
oversight and control in program risk and assurance activities.
Independent IT PRM often communicates the program and project
delivery teams’ activities to executive management (e.g., main
board) and the stakeholders who operate in the third line of defense,
such as the board’s audit committee and internal audit function.
The IT PRM function acts independently of the project delivery
team (first line of defense). It often consists of those experienced
with program risk for other companies, providing a “critical friend” to
provide a fresh, independent perspective, and providing confidencebuilding assurance and review activities. The leader of the IT PRM
function will typically also sit on the steering committee in an
independent capacity to challenge and advise on program progress.
Insights on IT risk | September 2011
Figure 5: Lines of defense
Corporate
First line
Second line
Third line
Main board
Audit committee
Executive
leadership
Internal audit
Portfolio risk
committee
Independent
PRM
Program
Steering
committees
Technical
design
authority
Project
Program
management
ofÕce
Project
workstreams
Internal
audit reviews
Key roles
• Main board (“heartbeat”): Responsible for providing overall portfolio and
program direction. Accountable for corporate-level risk management.
• Executive leadership team: Provides program sponsorship, strategy and
direction. Responsible for sign-off of scope; functional, technical and service
solutions; and changes to spending. Also responsible for monitoring of program
plan, budget, risks, issues and change requests.
• Portfolio risk committee: Responsible and accountable for providing overall
portfolio risk management oversight. Accountable for portfolio and program-level
risk management. Seeks interventions to address any concerns across portfolio.
• Audit committee: Assists the board by setting the agenda for and receiving
reports related to the effectiveness of risk management on the project and the
effectiveness of controls within key business processes.
• Internal audit: Typically will have some responsibility for providing independent
assurance to the audit committee on the effectiveness of internal controls within
key business processes on change programs.
• Steering committees: Responsible for ensuring strong buy-in for the solution
and that all stakeholder groups are represented appropriately. Accountable for
effective governance and planning, sign-off of quality deliverables and ensuring
that the solution and business change meets business and user requirements.
• Technical design authority: Responsible for technical review of solution and
ensuring adherence to technical architecture principles of the organization.
• Program management office: Provides day-to-day management controls over
the project, including management of project plan, budget, risks and issues.
Responsible for communicating effectively with governance groups, raising risks
and issues and required sign-offs.
• Independent PRM: Responsible for independently reviewing and advising on the
effectiveness of risk management at the program level, including effectiveness of
mitigation strategies for key program risks.
• Project workstreams: Responsible for day-to-day project delivery and
management of project risk.
Insights on IT risk | September 2011
9
Building confidence in IT programs through
Risk-based analysis, good information and deep
perspective enable value delivery
Figure 6: Ernst & Young IT Program
Engaging the right IT PRM function is ultimately about building
and sustaining the confidence of key stakeholders and having the
right information at the right time to make well-informed decisions
throughout your journey. Using a proprietary methodology, tools
and templates to help embed an IT PRM function, companies are
able to leverage our experience of major IT change earned working
with the world’s largest companies.
Sources of PRM
Clear, concise reporting within an IT PRM framework process is
essential. Key stakeholders need information and findings so they
can make fact-based decisions to mitigate risks and/or improve
program outcomes. To address this, and based on a risk-based
analysis of the most common success criteria for IT programs, we
created the IT Program Confidence Wheel — a reporting tool to
demonstrate an independent view of IT program confidence and
a point-in-time view of the key risks and issues that may affect a
successful outcome.
Outside the program:
In the program:
• Risk manager
• Quality manager
• PMO
• Independent PRM
• Risk committee
• Internal auditors
• External auditors
• Compliance functions
• Software vendors
The IT Program Confidence Wheel contains seven confidence elements
typically found on IT programs. A typical IT PRM assignment:
• Starts with an assessment of program governance and project
management, and proceeds with:
• Solution integrity
• Data integrity
Immediate intervention required.
High-risk area where the impact of not
resolving the program is significant.
• Business readiness
• Support readiness
• Ends with a post go-live assessment
Close monitoring required. There is
awareness of a material risk or issue, but
appropriate remedial action is in place.
Normal monitoring required.
Risk or issue appears managed
at an appropriate level.
10
Insights on IT risk | September 2011
IT PRM
Confidence Wheel®
Projec
tm
ana
ge
Comm
m
unic
atio
en
ns
ma
t
ce
rnan
e
v
go
tion,
m
za
reali
y
ra
and egulator
g
n
r
g
o
r
esi e and
d
P
c
s
nd
Õt lian
ent a
procResou
ure rce
me an
nt d
mg
mt
.
(sco Plan
pe,
and tim
cos e
t)
]
Program
[ofÔ\]f[]
Accept
ance and cutover
Business readiness
Insights on IT risk | September 2011
t]
gra
L
cu
\
oa
a
te
g
to
v] r
a
rity
t]
at va
a) li\
a
\
st
T]
in
ss
C
t
n\
Train and adopt
r
v]
n
o
(D
an
tr
ea
tr
n
an
a
e
l if
or
Test and validate
(User)
mi
an l
Cl] co
n\
giz
t]
s]
l] a n \
ct
Strategiz
e
Opera
t
valida ional
tion
Stabiliz
atio
n
Sust
and us ainabilit
er ad
y
opt
ion
Outc
beneÕtsomes an
reali d
zat
ion
ma
na
ge
m
en
t
ma
na
ge
me
nt
iz
e
ra
al
Strategize
St
p
s ro
d
rt es ce
)
i g n ss
Org
an
cess
and izational proign
at
c
s
o
e
n
d
trols
e
(Business)
pt
na
ge
m
en
t
tegrity
on in
luti
So
d
do
d ri
sk
over
nd cut
ce a y, Business
lian
it
mp ecur ter Recovery)
s
Co ing s
Disa
d
clu
and
(In ity
nu
nti
idate
d val )
Co
t an
on
Tes (Soluti
an
l y-
pp
ne
ti o
Integ
rati
on
g
te
ra
St
n
t i o ol
i za t r
an on po
c
up
Org
li d
and (S
va t)
d or
an p
st u p
Te (S
ain
Tr
r
Ea
Su
di
si
Quali
ty a
n
al
hnic
e
Tec tructur
ras
inf
gy and
dolo ent
m
tho
Me evelop
d
gem ess
ana ctiven
e m effe
c
an ce
rm nan
nd
rk a t
rfo ver
e
ewo emen
P go
ram anag
f
ion ge m
cis an
De . ch
lity
g
pabi
or
, ca proÕle
y
t
xi ity
ple ur
m mat
o
C nd
a
ase
ss c
ine
s
Bu
ents
uirem
Req d design
an
Post-go
live
ne p
Be com
t
a
D
a
11
The path to achieving IT program confidence
First, we focus on the maturity of program governance and project
management (top two segments of IT Program Confidence Wheel).
An objective assessment of these elements will help understand
the maturity of the organizations current processes to deliver
the program, taking into account the overall complexity of the
program. The analysis provides the foundation for understanding
the likelihood of a successful delivery of the program, and what
issues may result if there are gaps in these two important program
confidence areas. It is important to perform the initial assessment
prior to the actual program kick-off to determine how ready the
organization is to proceed with the initiative. By performing an
initial “readiness to start” assessment, organizations increase the
ability to correct any preplanning gaps and increase the ability to
influence positive outcomes of the program. All elements of the IT
Program Confidence Wheel are commonly assessed throughout the
entire lifecycle of the program.
Second, the quality of the deliverables of the program (remaining
five segments) are assessed, on an interim basis, as they are
delivered throughout the life cycle of the program. In the IT
Program Confidence Wheel, there are a number of layers, with
the initial activities (e.g., strategy) toward the center. Each layer
typically represents a key activity and/or deliverable on an IT
program at a point in time. By assessing the activities within each
12
segment and layer at a given stage gate (e.g., end of design, end
of build) we can assess the overall risk profile of the key confidence
segment and therefore the overall program at a given point in time.
Our approach is simple and is based on the view that there is a path
to achieving IT program confidence in each segment of the IT Program
Confidence Wheel. For example, there is a path to achieving confidence
in data integrity, which begins with having a robust strategy for data
migration, then progresses to effective processes for “cleanse and
collect,” “convert and migrate,” “testing and validation” and finishes
with “load and cutover.” If all of these are achieving program KPIs and
have been independently checked and verified to a high standard,
then — subject to no material risks or issues — management can
have confidence in a “go” decision on data integrity.
By managing the risks within each segment, an organization can
“bank” confidence in each critical element of the program and, over
time, progress toward a more transparent and confident view of risk.
This informs management before it makes important “go/ no go”
decisions. As the program progresses towards go-live, the number
of critical risks and major issues should be declining, and the program
can narrow its concerns to the issues that are most likely to impact
a successful go-live (i.e., potential showstoppers).
IT PRM can play an important role in assessing the outstanding issues
that may impact progression to the next stage or go-live and provide
an independent perspective to management on the business impact
of potentially reduced or accepting outstanding risks and issues.
Insights on IT risk | September 2011
Projec
tm
ana
ge
m
en
ion
sm
t
a
nce
erna
gov
,
m
zation
reali
y
ra
and egulator
r
gn
og
esi and
Pr
s d ce
nd
Õt lian
ent a
Comm
unic
at
Quali
ty a
n
]
t]
gra
Lo
Business readiness
a\
]r
a
rity
ta
te
g
s
T]
Accept
ance and cutover
in
t
at]
mi
\
]r
nv
tov
ea
tr
Co
n
(D \
at va
a) li\
or
an
tr
Train and adopt
ss
cu
an l
Cl] co
Test and validate
(User)
n
n\
giz
t]
ra
s]
a
ct n\
St
l]
Strategize
Org
an
cess
and izational proign
controls des
(Business)
an
Stabiliz
atio
n
Opera
t
valida ional
tion
Strategiz
e
Sust
and us ainabilit
er ad
y
opt
ion
Post-go
live
Program
conÔ\]nc]
lia
ity, B
mp ecur ter Recovery)
s
Co ing s
Disa
d
clu
and
(In uity
n
nti
idate
d val )
Co
t an
on
Tes (Soluti
a
pt
tegrity
on in
luti
utover
So
and c siness
u
nce
ents
uirem
Reqnd design
a
d
pp
e
l if
ne
ti o
na
ge
m
en
t
ma
na
ge
m
en
t
procResou
ure rce
me an
nt d
mg
mt
.
g
te
ra
li d
va t)
d or
an p
st u p
Te (S
an
l y-
Su
r
Ea
di
si
d ri
sk
ma
na
ge
me
nt
(sco Plan
p
and e, tim
cos e
t)
iz
e
a
s lp
rt des roce
)
i g n ss
at
e
do
Integ
rati
on
St
n
t i o ol
i za t r
an on po
Org nd c (Sup
a
in
Tra
Key question: Is the financial and business data to drive business processes and effective management
information and reporting tested, proven and ready?
em
ss
nag vene
ma ecti
ce eff
an ce
rm nan
d
k an nt
rfo ver
e
wor
Pe go
me gem
fra ana
ion ge m
cis an
De . ch
lity
g
pabi le
a
or
c
roÕ
y,
xit ity p
ple atur
m
m
Co nd
a
case
ess
sin
Bu
al
hnic
e
Tec tructur
as
infr
gy and
dolo ent
m
tho
Me evelop
d
Figure 7: Achieving data integrity
Outc
beneÕtsomes an
reali d
zat
ion
ne p
Be com
Da
ta
Cleanse and collect
60%
Convert and migrate
90%
Test and validate
100%
Load and cutover
on \ata [mton]r r]kmdtk!
Insights on IT risk | September 2011
º
º
º
º
º
º
º
º
9[`i]ning
hrogram CHAk
Go
Go
30%
º
Hrogram ]d]m]nt [onÕ\]n[] m]akmr]
Third review:
Hr] go%din] ktag] gat]
r]ni]o Õnad nadi\ation
º
Strategy
An\]h]n\]ntdq [`][c]\ an\ n]riÕ]\
Second review:
Data [onn]rkion an\
nadi\ation r]ni]o
10%
9[`i]ning hrogram CHAk
First review:
Data migration
ktrat]gq r]ni]o
Hjg_jYe]d]e]fl[gfÕ\]f[]e]Ykmj]
Proposed independent
PRM activity
Data
integrity
Go
An\]h]n\]ntdq
[`][c]\ an\ n]riÕ]\
13
Method of providing independent PRM
In order to establish a balanced view of independent PRM, we
recommend a triangulated approach:
1. End-to-end advisory — Independent senior-level challenge,
advice and mitigation strategies for key risk areas throughout
the program life cycle. The extent of the role depends on the
program’s risk profile and the level of assurance required by the
organization. It can range from steering committee attendance
to full independent verification and validation services.
2. Stage-gate reviews — Point-in-time “health check” performed
at specific phases of a program and often conducted at key
transition points between program phases. Typically used as an
input to steering committee stage-gate decision-making and
designed to inform management of major risks prior to making
key go/no go decisions. For programs that are already in-flight
(i.e., partially completed), an IT PRM role would typically start
with a baseline stage-gate assessment that provides an overall
assessment of a program’s health.
3. Targeted assessments — Drill-down reviews of common highrisk areas (e.g., third-party contracting, business change,
test strategy, data migration). Outputs provide confidence to
management that high-risk areas have been independently
checked and verified to follow leading practices.
Figure 8: Ernst & Young’s approach to developing an IT PRM framework
Mobilization
Triangulated
approach
Assess program risk and
complepity proÕle
Communicate and agree
on program risks
Program steering
committee
14
Program risk committee
Targeted
assessments
Develop and agree on
PRM plan
Example of
PRM activities
and reviews
Program team
End-to-end
advisory
Determine level of program
assurance required
What are they?
Reporting
Stage-gate
reviews
Executive management
“helicopter sessions”
End-to-end advisory
Stage gate reviews
Targeted assessments (drill-down reviews)
Independent senior-level
challenge and advice
throughout the program. Role
can be full-time or part-time
depending on the risk profile
of the program and level of
end-to-end advisory required.
Point-in-time health check on
the progress and readiness of
the program to move from one
key program stage to the next.
Informs management prior to
making key go/no go decisions.
Deep-dive reviews on areas commonly identified
as high risk (e.g., third-party contracting; business
change management; business and IT controls;
design and execution; security and access; data
migration and conversion; testing, design and
execution; financial reporting readiness; and
support model).
Steering
group
attendance
Ongoing
program
risk role
Stage gate 1 —
Design review
Stage gate 2 —
Build design
Insights on IT risk | September 2011
Third-party
contracting
Project plan
integrity
Testing
and data
strategy
Business and
IT controls
integrity
a series of stage-gate reviews, targeted assessments and end-to-end
advisory meetings throughout the program’s lifetime.
The output of a program risk assessment is an independent IT PRM
framework. Ultimately, management’s risk appetite will determine
the level of program assurance required on any given program or
project. A properly designed independent IT PRM framework (see
Figure 9) provides management with the opportunity to implement
An independent IT PRM framework should be flexible enough to suit
most program implementation models (e.g., ASAP methodology via
SAP or AIM methodology via Oracle).
Figure 9: Example of an independent IT PRM framework
Oracle **
SAP *
DeÕ nition and analysis
Preparation
TA3
TA5
Final preparation
Final preparation
SG3
E2E
TA2
TA4
Delivery
SG2
E2E
Transition
Realization
Design
SG1
TA1
Build
Requirements
Startup and planning
Generic program
Design
TA7
TA8
TA9
TA10
TA11
TA12
TA13
TA14
TA15
TA17
Go live and support
Go-live and support
SG4
E2E
TA6
Production
SG5
E2E
TA16
TA18
E2E
TA19
TA20
TA21
TA22
TA27
TA23
TA24
TA25
TA26
TA30
TA28
TA29
Stage-gate reviews (SG)
SG1 — Planning stage or
baseline review
SG2 — Design-to-build
stage review
SG3 — Build-to-test
stage review
SG4 — Test to pre-go-live
stage review
(Pre-implementation)
SG5 — Early-life support to
steady-state stabilization
End-to-end advisory role (E2E)
E2E — Independent senior-level challenge and advice throughout the program. Regular update meetings with key project stakeholders and
steering committee meetings.
Possible Targeted Assessments (TA)
TA1 — Program
governance and
fundamentals review
TA2 — Risk profiling or
assessment; program
management approach
and risk profiling
TA3 — Third-party
contractor management
and sourcing review
TA4 — Value and benefits
realization design review
TA5 — Business change
strategy review
TA6 — Business process
and controls design review
TA7 — Data strategy review
TA8 – Security authorization
and SOD design review
TA9 — System change
management and ITGC
review
TA10 — Test strategy review
TA11 — RICEFs
identification and scoping
review
TA12 — Organization or
operating model design
review
TA13 — Global template
“fit for purpose” review
TA14 — Template fit gap
analysis review
TA15 — IT infrastructure
and environments
review
TA16 — Data cleanse
and collect review
TA17 — Testing
execution reviews
(unit, assembly)
TA18 — Business change
management execution
review
TA19 — Testing execution
reviews (integration, user,
performance)
TA20 — Business process
and controls testing review
TA21 — Data load and
conversion and trial
cutover review
TA22 — Support and
operational readiness review
TA23 — Application security
and SoD configuration review
TA24 — Businesspreparedness reviews
(business continuity and
disaster recovery)
TA25 — Key go-live issues
review (business impact
assessment)
TA26 — Early-life support
to stabilization review
TA27 — Outcomes and
benefits realization review
TA28 — Post implementation
operational controls
validation reviews
TA29 — Program lessons
learned review
TA30 — Next release and
user adoption strategic
review
* SAP ASAP methodology **Oracle – AIM methodology, for example, can be used as required
Insights on IT risk | September 2011
15
Questions to build program confidence
The answers to key questions help build
program confidence
There are many questions about key confidence elements of large
IT programs that must be answered in order to build and sustain
confidence in the program’s ability to deliver. These questions are
common to many large IT programs.
In the absence of an independent analysis of program risk,
management’s challenge is to determine how well it can trust
the information that is being provided to it. The question asked
increasingly by executives entrusted with IT programs is: who is
checking and verifying that the information I am getting from the
program team is complete, accurate and considers the key risk?
Confidence element
Key questions for IT programs
Program governance
Does the business case have integrity? How complex is the program and and is our organization capable
of delivering? Are the right governance, change and decision-making processes in place, and are they
performing effectively?
Project management
Are the right processes in place so the program is planned, managed and tracked effectively? Does our
organization have the right resources and quality, risk and communication processes in place?
Solution integrity
Are the technology solution and its supporting infrastructure and interfaces tested, proven and ready?
Data integrity
Is the financial and business data to drive business processes and effective management information and
reporting tested, proven and ready?
Business readiness
Are the new business operating models, processes and controls tested, approved and ready for deployment?
Are the organization and its people trained and ready to use the new solution?
Support readiness
Are the support organization, processes and tools ready to support the new solution?
Post go-live activities
Are the activities to support post go-live and the sustainability and adoption of the solution in place and ready?
IT PRM: high value at relatively low cost
Investments in PRM are typically quite small compared to the
overall program budgets and business revenues put at risk.
In our experience, independent IT PRM roles should account
for approximately 2% to 6% of the overall program budget.
Although this may vary depending on the project’s risk
profile, it is broadly in line with what we see in progressive
organizations. While the costs are relatively low, the benefits
of PRM are significant. These include:
• Improved visibility and transparency of program risks and
performance
• Increased confidence in the integrity of business case and
projected benefits
• Enhanced management control of the program
• Increased likelihood that the program will be delivered on
time, on budget and with projected benefits
16
• Early identification of program-critical risks and issues
• Practical services to address problems as they arise
• More informed decision-making as a result of independent
reporting
• Access to independent professional advice on leading
program practices
• Potentially reduced or eliminated surprises
Insights on IT risk | September 2011
Next steps
The assessment, as discussed in this paper, is the start of a journey
focused on improving risk management in major change programs
and projects. The results of the assessment can highlight a number
of necessary next steps, including:
• Assistance in implementation of program risk management
assessment for all key programs/projects for continuous monitoring
• Improvement of the program risk management tools and enablers
• Predictive analytics and root cause analysis and modeling of key
relationships between key project factors
• Analysis to highlight hidden issues, risk and identification of the
root cause of issues such as a detailed program schedule analysis
• Utilization of analytics simulations to predict program outcomes
to undertake appropriate actions if necessary
To help better understand next steps, the graphic below — what
we call “the cube” — is a detailed framework and facilitator. The
cube focuses on three primary dimensions: program governance,
project management and technical solution. This framework is an
extension of the IT Program Confidence Wheel (shown on page 11).
The two are mutually complementary although the cube is primarily
used for complex or critical organization program initiatives and
facilitates a full life-cycle approach.
Figure 10: Program Risk Management Cube
Capability
and
Program governance
BeneÕt realization and
sustainability
maturity
Complexity
proÕle
Organizational
change
management
Business case
integrity
Decision
framework
Scope
management
Jakckgjakkm]ka\]flaÕ]\l`Yl
j]imaj]eYfY_]e]flYll]flagf
Jakckgjakkm]ka\]flaÕ]\3
fgeYfY_]e]flYll]flagfj]imaj]\
Fgjakckgjakkm]ka\]flaÕ]\
Performance
management
Time
management
Human
resource
management
BeneÕts,
design and
realization
Compliance and
regulatory
Governance
effectiveness
Cost
management
Cutover
and
support
Communications
management
Insights on IT risk | September 2011
Sustainability
model
Data
management
Risk
management
Processes, controls,
and predictability
Controls
Security,
BC and DR
Integration
management
Project management
Methodology
and
development
Requirements,
engineering
and design
Procurement
management
Quality
management
Technical
infrastructure
Testing
and
validation
Technical solution
Requirements development,
quality and transition
17
Ernst & Young
Assurance | Tax | Transactions | Advisory
About Ernst & Young
Ernst & Young is a global leader in assurance, tax,
transaction and advisory services. Worldwide,
our 141,000 people are united by our shared
values and an unwavering commitment to quality.
We make a difference by helping our people,
our clients and our wider communities achieve
their potential.
Ernst & Young refers to the global organization
of member firms of Ernst & Young Global
Limited, each of which is a separate legal entity.
Ernst & Young Global Limited, a UK company
limited by guarantee, does not provide services
to clients. For more information about our
organization, please visit www.ey.com
About Ernst & Young’s Advisory Services
The relationship between risk and performance
improvement is an increasingly complex and
central business challenge, with business
performance directly connected to the
recognition and effective management of risk.
Whether your focus is on business transformation
or sustaining achievement, having the right
advisors on your side can make all the difference.
Our 20,000 Advisory professionals form one
of the broadest global advisory networks
of any professional organization, delivering
seasoned multidisciplinary teams that work with
our clients to deliver a powerful and superior
client experience. We use proven, integrated
methodologies to help you achieve your strategic
priorities and make improvements that are
sustainable for the longer term. We understand
that to achieve your potential as an organization
you require services that respond to your specific
issues, so we bring our broad sector experience
and deep subject-matter knowledge to bear in
a proactive and objective way. Above all, we are
committed to measuring the gains and identifying
where the strategy is delivering the value your
business needs. It’s how Ernst & Young makes
a difference.
© 2011 EYGM Limited.
All Rights Reserved.
EYG no. AU0966
In line with Ernst & Young’s commitment to minimize
its impact on the environment, this document has
been printed on paper with a high recycled content.
This publication contains information in summary form and is
therefore intended for general guidance only. It is not intended
to be a substitute for detailed research or the exercise of
professional judgment. Neither EYGM Limited nor any other
member of the global Ernst & Young organization can accept
any responsibility for loss occasioned to any person acting
or refraining from action as a result of any material in this
publication. On any specific matter, reference should be made
to the appropriate advisor. The opinions of third parties set out
in this publication are not necessarily the opinions of the global
Ernst & Young organization or its member firms. Moreover, they
should be viewed in the context of the time they were expressed.
IT Risk Management at Ernst & Young
At Ernst & Young, our services focus on our individual clients’ specific business needs
and issues because we recognize that each is unique to that business.
IT is a key to allowing modern organizations to compete. It offers the opportunity to become closer
to customers and more focused and faster in responses, and can redefine both the effectiveness
and efficiency of operations. But as opportunity grows, so does risk. Effective IT Risk Management
helps you to improve the competitive advantage of your IT operations, by making these operations
more cost efficient and managing down the risks related to running your systems. Our 6,000 IT
Risk professionals draw on extensive personal experience to give you fresh perspectives and open,
objective advice — wherever you are in the world. We work with you to develop an integrated,
holistic approach to your IT risk or to deal with a specific risk and information security issue.
We understand that to achieve your potential you need tailored services as much as consistent
methodologies. We work to give you the benefit of our broad sector experience, our deep
subject-matter knowledge and the latest insights from our work worldwide. It’s how
Ernst & Young makes a difference.
For more information on how we can make a difference in your organization, contact your local
Ernst & Young professional or a member of our team listed below.
Contacts
Global
Norman Lonergan
(Advisory Services Leader, London)
+44 20 7980 0596
[email protected]
Paul van Kessel
(IT Risk and Assurance
Services Leader, Amsterdam)
+31 88 40 71271
[email protected]
Robert Patton
(Americas Leader, Atlanta)
+1 404 817 5579
[email protected]
Andrew Embury
(Europe, Middle East, India and
Africa Leader, London)
+44 20 7951 1802
[email protected]
Doug Simpson
(Asia-Pacific Leader, Sydney)
+61 2 9248 4923
[email protected]
Naoki Matsumura
(Japan Leader, Tokyo)
+81 3 3503 1100
[email protected]
Bernie Wedge
(Americas Leader, Atlanta)
+1 404 817 5120
[email protected]
Manuel Giralt Herrero
(Europe, Middle East, India and
Africa Leader, Madrid)
+34 91572747
[email protected]
Troy Kelly
(Asia Pacific Leader, Hong Kong)
+85 2 2629 3238
[email protected]
Giovanni Stagno
(Japan Leader, Chiyoda-ku)
+81 3 3506 2411
[email protected]
Advisory Services
IT Risk and Assurance Services