Measuring quality: Nursing Practice Metrics Discussion

Nursing Practice
Discussion
Metrics
Keywords: Quality/Balanced scorecard/
Metrics
●This article has been double-blind
peer reviewed
The balanced scorecard can be used to involve nurses in developing and acting on
quality indicators; it also shows at a glance how well standards are being met
Measuring quality:
how to empower
staff to take control
This article...
What a balanced scorecard is and how it works
Why using balance scorecards can improve the quality of care
How to develop a balanced scorecard system
Why staff should be involved in developing indicators
Author Lisa Grant is deputy director of
nursing and governance, The Walton
Centre Foundation Trust, Fazakerley,
Liverpool; Tony Proctor is professor, Chester
Business School, University of Chester.
Abstract Grant L, Proctor T (2011)
Measuring quality: how to empower staff
to take control. Nursing Times; 107, 7: 22-25.
A vast amount of information relating to
standards of patient care is collated from
hospital wards, yet there is not always
evidence that this information is discussed
or acted upon by ward staff.
Involving ward staff in setting up
systems to monitor performance and then
deciding how to address shortcomings uses
their insights into care provision and gives
them ownership over standards of care.
The balanced scorecard is an effective
tool for monitoring quality that can be
applied to healthcare. This article discusses
how to use it to develop and implement
systems of measuring the quality of care.
Q
uality is the focus of much
activity in the NHS. In the
pursuit of providing excellent
service quality, the NHS next
stage review, undertaken by Lord Darzi, set
the basis for a health service that would
empower staff and give patients choice.
One of the aims of the review was to ensure
healthcare would be personalised and fair
(Department of Health, 2008). In an
interim report, Darzi described the development of a quality framework supported
by metrics – ways of measuring outcomes
of care – that would be collated from a
range of staff groups (DH, 2007).
The standards now in place focus on
patient outcomes, and make the provision
of high-quality services a priority for the
NHS. These standards describe the level of
quality that healthcare providers are
expected to meet in terms of safety, clinical
and cost effectiveness, governance, care
that meets individual patient need, joinedup care and quality of care (Care Quality
Commission, 2010). Failure to achieve these
standards can result in financial penalty,
loss of reputation and closure. It is therefore
vital that quality of care delivered is regularly benchmarked, monitored and
improved to reassure patients and providers
that care meets the highest standards.
Griffiths et al (2008) suggested it was
important to identify metrics that would
have an impact on the delivery of patient
care. Since the idea of quality in healthcare
is multifaceted, there are many opinions
on what actually constitutes quality. In
particular, these opinions concern:
» What quality means to patients and
their families;
22 Nursing Times 22.02.11 / Vol 107 No 7 / www.nursingtimes.net
» How it should be evaluated by doctors;
» What role it plays in patients’ overall
satisfaction;
» How it should be addressed by
healthcare managers (Chilgren, 2008).
Many academics have argued about the
nature of quality in healthcare. Descriptions of quality of care vary depending
on the perspective and role of the observer,
who may be a patient, a clinician, a purchaser, or a manager. Descriptions also
depend on the clinical setting, patients’
expectations, and the severity of illness.
Struder (2003) argued that excellence
is determined by patients’ perceptions that
they should receive extraordinary service
and quality. However, Heinemann et al
(1996) suggested that, as well as patient
satisfaction, specific indicators should be
measured to give an insight into how care
is being delivered, such as patient falls,
medication errors and infection rates.
Such metrics should be presented in a
meaningful way to identify areas for
improvement.
Metrics enable us to understand how
procedures are progressing and how they
can be improved. They can provide a way of
making care providers accountable for the
quality of their services. Accountability for
quality exists at many levels, starting from
the point of care, for example where individual nurses are accountable to clinical
managers and patients.
To ensure a high-quality service, we
need a system of target setting and performance monitoring of nursing care.
An existing problem
A great deal of data on the quality of care is
regularly collected in hospital wards.
5 key
points
1
Performance
monitoring is
essential to ensure
patients receive a
high-quality service
Indicators must
have an impact
on the delivery of
patient care
If performance
monitoring is
to be effective,
staff need to feel
actively involved
The balanced
scorecard
assists in
measuring
performance and
helps to identify
shortcomings
The system
allows staff to
see easily where
improvements are
needed
2
3
4
5
box 1. What is
the balanced
scorecard?
The balanced scorecard is a
tool used extensively in
business and industry,
government and non-profit
organisations worldwide to
monitor organisational
performance.
It provides a framework
that assists in measuring
performance, helps identify
what should be done and
measured, and enables people
to put plans into action.
The balanced scorecard
suggests we view
organisations from four
perspectives, and develop
metrics, collect data and
analyse this relative to each
of these:
● The learning and growth
perspective: this includes the
provision of employee
training, along with the use of
mentors and tutors within the
organisation. It also focuses
However, it is not always clear whether
staff have been given the opportunity to
discuss this information or taken action in
response to any problems or shortcomings
it identifies.
This suggests nursing staff at ward level
do not fully accept the need to monitor and
improve the quality of nursing care provided. However, it may indicate that
nurses do not feel a sense of ownership of
the information collected.
One way to approach this problem is to
use a planning system called the balanced
scorecard as a performance tool to monitor care (Kaplan and Norton, 1992). This
offers a way of displaying nursing indicators in a way that makes it easy to see
quickly any indicators where agreed standards are not being met (Box 1). Fig 1 is an
example of a balanced scorecard designed
to monitor the quality of nursing care on a
hospital ward.
Effective use of the balanced scorecard
involves setting targets and tolerances to
measure performance regularly, and
requiring staff to develop action plans to
address unsatisfactory work.
Further evidence derived from ward
on the ease of
communication between
workers that enables them to
get help with a problem
when it is needed.
● The business process
perspective: this refers to
internal processes. Metrics
here give managers
information on how well their
unit is running, and whether
its products and services
meet customer requirements.
● The customer perspective:
this involves developing
metrics for measuring and
evaluating customer
satisfaction.
● The financial perspective:
this means metrics need to
be established to monitor
income and expenditure
streams or efficiency.
In practice, the headings
chosen will vary with the
nature of the organisation
and the task the scorecard
covers. However, these four
perspectives usually appear
as components of the factors
assessed in the scorecard.
meetings can then be instigated to show
how nursing staff are engaged with, and
encouraged to be part of the action-setting
and improvement process.
The tool also enables staff to recognise
positive outcomes. Performance measurements therefore also help to show how
nurses are driving and highlighting their
improvement priorities.
Developing a monitoring system
To develop an effective monitoring
system, it is essential to ask nurses what
metrics they believe would be meaningful
when examining ward and nursing performance.
These views can be collected via a questionnaire. To gain a real insight into
nurses’ opinions of what indicates the
quality of patient care, it is advisable not
to refer to any data already collected in the
questionnaire.
Questionnaires can be structured
around the three subheadings set out in
the NHS next stage review (DH, 2008) and
by the NHS Information Centre (tinyurl.
com/indicators-quality), namely safe,
effective and personalised care. Nurses
should be asked to list under each of these
headings three nursing indicators they
think are important, and which could be
audited regularly to show the quality of
care patients receive.
This information can then be used with
the routine data collected and collated to
identify the desired nursing metrics.
The next stage is to construct a balanced scorecard using the chosen metrics;
the scorecard is usually completed every
month, presenting the previous two
months of data.
Monthly meetings can then be used as
an opportunity for staff to discuss the
information provided by the balanced
scorecard, encouraging them to share
ideas while reflecting on individual
learning experiences and needs. These
meetings can also be used to develop
action plans to address any shortcomings.
Use of nursing indicators in the
balanced scorecard
It is important that the public, managers
and nurses recognise that each indicator
chosen for the balanced scorecard is
important and an indicator of nursing care
(Lee, 2007).
The indicators must be scientifically
sound, usable and feasible. To ensure that
meaningful indicators are chosen, they
must be measurable using available data at
a reasonable cost. There must also be evidence that the quality or quantity of
nursing substantially contributes to
changes measured by the indicator.
Measures should be chosen that minimise the risk that improved performance
on specific indicators gives a false impression of an overall improvement. For
example, measures that focus on the performance of care process rather than
on outcomes are most vulnerable to creating such a false impression (Griffiths et
al, 2008).
Balanced scorecard action plans
The scorecard in Table 1 (overleaf ) has five
headings:
» Efficiency;
» Patient safety;
» Excellence in care metrics;
» Delivering same-sex accommodation
(DSSA) compliance;
» Patient experience.
Under each heading, a number of indicators are used to reflect aspects of nursing
performance. For example, under “patient
experience”, the percentage of patients
reporting excellent, good/fair, poor and
not applicable are recorded in columns
5 and 6 for the current and previous
www.nursingtimes.net / Vol 107 No 7 / Nursing Times 22.02.11 23
Nursing Practice
Discussion
Metrics
The tolerances
columns
are
This balanced
Ward A nursing metrics
Tolerances
Previous Current
colour coded:
scorecard shows what
month’s month’s
» Green
is acceptable, what
Green =
Amber =
Red =
data
data
Acceptable/no
needs action, and
Efficiency
specific action
how urgently action
is needed
Absence: sickness
<5%
5-5.9%
>6%
required;
Bank and agency usage
<5%
5-8%
>8.1%
2%
2%
» Amber Take
Staff pay
Under budget On budget
Over budget
note, consider
PDP % staff completed
100%
99.9-80%
<80%
what action to
Patient safety
take if necessary
MRSA infection/colonisation
and proceed with
community acquired
0
1
>1
0
0
caution;
MRSA infection/colonisation
» Red Stop and
Walton Centre acquired
0
1
>1
0
0
consider action required immediately.
MRSA bacteraemia
This colour coding reflects whether the
community acquired
0
1
>1
0
0
situation is acceptable and the urgency
MRSA bacteraemia
with which any corrective action should be
Walton Centre acquired
0
1
>1
0
0
taken. Monthly statistics shown in colC. difficile infection
umns 5 and 6 are coloured to match the relcommunity acquired
0
1
>1
0
0
evant tolerance column. For example, the
C. difficile infection
ward received one complaint about staff
Walton Centre acquired
0
1
>1
0
0
attitude in the current month (column 5);
Pressure ulcer communtiy acquired 0
1
>1
0
0
this equated with the “tolerance” set in
Pressure ulcer
column 3 (amber), so the entry in column 5
Walton Centre acquired
0
1
>1
0
0
is also coloured amber.
Hand hygiene audit: nursing
100%
100%
Entries for all the data in the scorecard
Hand hygiene audit: medical/
100%
88%
are made using this approach. The colour
non-nursing
coding helps staff to rapidly identify
Saving Lives audit of the month
n/a
n/a
potential problem areas and to monitor
Environmental score
>95%
90-95%
<90%
69%
trends at a glance.
Drug prescription errors
0
0
Under the heading “efficiency”, the balDrug administration errors
0
1
>1
0
0
anced scorecard shows that in the current
Patient falls
0
1
>1
2
2
and previous month the ward was overPatient falls causing harm
No harm
Harm
0
0
spending on staff pay, but underspending
Resus trolley checks
100%
<100%
100%
100%
on bank and agency use. This indicates a
Excellence in care metrics
need for a review to ensure that safe staffing
Slips, trips and falls
>96%
90-<96%
<90%
71%
100%
levels are maintained across the ward.
DVT
>96%
90-<96%
<90%
71%
100%
Results can be observed with respect to
Nutrition
>96%
90-<96%
<90%
71%
100%
patient safety. In this example, the number
Manual handling
>96%
90-<96%
<90%
71%
100%
of patient falls may be unusually high
MRSA screening
>96%
90-<96%
<90%
71%
100%
because the ward helps to rehabilitate
Waterlow care
>96%
90-<96%
<90%
71%
100%
patients with head injuries, and it is useful
Infection control risk assesment >96%
90-<96%
<90%
71%
100%
to add such explanations to scorecards
DSSA compliance
when reporting the results to senior manSleeping areas
Compliant
Non-compliant 0
0
agement. However, the reasons for the falls
Bathroom and toilet facilities
Compliant
Non-compliant 7
0
should still be investigated since they were
Patient experience
above the target levels set for both the curExcellent
66%
65%
rent and previous month.
Good/fair
18%
19%
For the current month, there is an
Poor
2%
2%
overall improvement across all the indicaNot applicable
tors relating to “excellence in care metrics”.
Complaints: care
0-1
2-3
>3
These are the risk assessments completed
Complaints: staff attitude
0
1
>1
1
on admission, which were highlighted as
Clinical incidents reported
underperforming in the previous month.
within 48 hours
100%
90-99%
<90%
100%
Future action plans set by the ward to
maintain this improvement include ward
audits to ensure that risk assessments are
month. Additional columns can be added clinical incidents reported within 48 hours completed in a timely manner.
to display data from earlier months if are recorded in a similar way. “Need for
required.
action” levels for all performance criteria Conclusion
The number of complaints about care, are shown in columns 2, 3 and 4, under the The data collated under the “effectiveness”
and “patient experience” headings should
staff attitudes and the percentage of “tolerances” heading.
Table 1. Example of a nursing metrics scorecard
24 Nursing Times 22.02.11 / Vol 107 No 7 / www.nursingtimes.net
“I could spend the day doing
anything from helping people to
write letters to playing volleyball”
Carol Rooney
p26
relate mainly to national and internal tar- care providers can be made accountable
gets. The data needs to be relevant to for the quality of nursing services (Grifpatients, clinician, purchaser or manager fiths et al, 2008). However, there is an arguand the nurses (Klint and Long, 1989).
ment that nursing outcome indicators
This involves using indicators that may not always be valid, and that reliable
monitor waiting times for appointments outcome measures can be difficult to idenand surgery, which are important aspects tify within general healthcare (Marek,
of the overall patient experience. They 1989).
should also include patients’ overall length
The balanced scorecard has been critiof stay, which it is hoped will remain low to cised as a means to monitor performance.
indicate that treatment and care was deliv- Wickes et al (2007) and Norrekilt (2003)
ered in a timely and effective way.
considered it to be a top-down means of
Both Struder (2003) and Heinemann performance management founded on
et al (1996) believed that excellence in control-based management. Chang (2007)
relation to quality could be viewed from argued that it has little impact on
patients’ own experiences. However, Hein- improving performance valued by local
emann et al also suggested
managers in the NHS.
it was important to estab- nursingtimes.net/
However, the Departlish and monitor specific for more info on latest
ment of Health (2009) recindicators
alongside research and regular updates ognised the use of the balon nursing management
patients’ experiences to
anced scorecard as a
improve the quality of care
quality control system that
provided.
provides a framework for business planSuch differences of opinion can reflect ning, measuring organisational performdifferent professional perspectives within ance and local target setting. It also recogthe NHS. A profession’s targets and aims nised that, apart from financial
relating to these professional perspectives monitoring, the balanced scorecard can
can affect the way in which quality indica- assist in monitoring customer satisfaction.
tors are defined.
In addition, arguments made against
For example, financial targets are the balanced scorecard have not been
important considerations when trying to reflected in our own experience of applying
ensure hospital trusts run effectively it to monitoring quality of care in an NHS
within the resources available. However, ward. The tool has proved a useful mechawhile nurses appreciate this is an impor- nism for drawing attention to trends
tant part of the provision of a quality reflecting both good practice as well
service, they consider other indicators as undesirable outcomes that merit
relating to nursing performance to essen- urgent attention. This has in turn enabled
tial to monitoring care quality.
staff to maintain a high standard of care
Indicators provide a means by which quality. NT
References
Care Quality Commission (2010) Guidance About
Compliance. Essential Standards Of Quality And
Safety. What Providers Should do to Comply with
the Section 20 Regulations of the Health and Social
Care Act 2008 London: CQC. tinyurl.com/
CQCquality-standards
Chang L (2007) The NHS performance assessment
framework – as a balanced scorecard approach:
limitations and implications. International Journal of
Public Sector Management; 20: 2, 101-117.
Chilgren A (2008) Managers and the new
definition of quality. Journal of Healthcare
Management; July: 1, 51663. 53, 4: 221-229.
Department of Health (2009) The NHS Workforce
Scorecard. London: DH. tinyurl.com/workforcescorecard
Department of Health (2008) High Quality Care for
All: NHS Next Stage Review Final Report. London:
DH. tinyurl.com/care-for-all
Department of Health (2007) Our NHS Our Future:
NHS Next Stage Review – Interim Report. London:
DH. tinyurl.com/next-stage-interim
Griffiths P et al (2008) State of the Art Metrics for
Nursing: a Rapid Appraisal. London: King’s College.
tinyurl.com/metrics-report
Heinemann D et al (1996) Partners in patient care:
measuring the effects on patient satisfaction and
other quality indicators. Nursing Economics; 14: 5,
276-285.
Kaplan R, Norton D (1992) The Balanced
Scorecard: Translating Strategy into Action. Boston:
Harvard Business School Press.
Klint R, Long H (1989) Towards a definition of
quality. Physician Executive; 15: 5, 7-11.
Lee B (2007) Identifying outcomes from the
nursing outcomes classification as indicators of
quality care in Korea: a modified Delphi study.
Nursing Studies; 44: 6, 1021–1028.
Marek KD (1989) Outcome measurement in nursing.
Journal of Nursing Quality Assurance; 4: 1, 1-9.
Norrekilt H (2003) The balanced scorecard: what is
the score? A rhetorical analysis of the balanced
scorecard. Accounting, Organisations and Society;
28: 6 591–619.
Struder Q (2003) Hardwiring Excellence. Gulf
Breeze: Fire Starter Publishing.
Wickes A et al (2007) Competing values in
healthcare: balancing the (un)balanced Scorecard.
Journal of Healthcare Management; 52: 5,
309-23
www.nursingtimes.net / Vol 107 No 7 / Nursing Times 22.02.11 25