Seminar D1

Transcription

Seminar D1

Seminar D1
Identifying New Metrics to
Quantify the Business Value of
Your L&D Initiatives
Investigating the Effect of Learning & Development
Initiatives at Maersk
This session will take the participants through how The Maersk Group is working with ensuring the effect of learning
and development initiatives from a theoretical & methodological perspective and illustrate this with practical concrete
examples.
(c) Copyright 2013, A.P. Moller - Maersk. All rights reserved. Not to be distributed without written consent
Personal data
Name & title
Peter V.W. Hartmann, M.Sc. & Ph.D.
Business Intelligence Expert
Responsibilities in Maersk
Currently:
Responsible for HR Analytics/HR Business Intelligence
in Maersk Drilling
Previously:
Responsible for HR Reporting in Maersk Drilling
Responsible for Assessment Tool & HR Analytics in
Maersk Group HR.
Background
Research in Personality & Cognitive Ability
Psychometrics & development of Assessment Tools
The Maersk Group
What do we do?
The Maersk Group
The A.P. Moller - Maersk Group is a diversified conglomerate with 89,000
employees in 135 countries
We serve customers worldwide mainly in the shipping & energy area operating in five business segments:
 Container shipping
 Terminal operation
 Tankers, supply, towage & logistics
 Oil and gas activities
 Offshore Drilling
Documenting the Effect of Learning &
Development Initiatives: Three case studies
Leadership
Training
Acceleration
Program
On the Job
Training
Evaluating the Value of Leadership Training
The Maersk leadership Model
What we expect from our leaders
BUILD AND
SUSTAIN
OUR
BUSINESS
BUILD AND
SUSTAIN OUR
ORGANISATION
ROLE MODEL OUR
CORE VALUES
Leadership behaviours
Leaders of Others
Leaders of Leaders
ROLE MODEL
OUR CORE
VALUES
Live and promote our company values
Live and promote our company values
BUILD AND
SUSTAIN
OUR
BUSINESS
Drive and monitor your team’s performance while
focusing on what matters – to our customers and
to our profitability
Support and hold your leaders accountable for
delivering results and ensure alignment in
achievements across teams
Facilitate collaboration within and between teams
to drive continuous improvements. Engage
customers/stakeholders in the creation of value
Ensure cross-organisational collaboration and
allocate the necessary resources to enable
innovation, improvements and co-creation of value
with our customers
Select and build engaged, diverse, and highperforming teams
Optimise your organisation to be prepared for the
future
Develop the skills, capabilities and effectiveness of
your direct reports through delegation and
coaching
Develop your leaders’ business and leadership
capabilities
to perform efficiently
BUILD AND
SUSTAIN
OUR
ORGANISATI
ON
Measuring the effect of Leadership Training
through the Kirkpatrick model
•What is the participant´s initial reaction to the training course?
•Can be divided into affective (”I like this”) and utility (”I can use this”)
•Important for “face value” and to avoid scaring people off. The felt utility seems to a better prediction for later effects
Reaction
???
Learning
•What knowledge & skills have been acquired?
•Can be divided into what has been learned, what is initially retained and what can the be reproduced – all in the training situation
•Important for determining the “input” the participants are leaving with and potentially bringing back to the office
Behavior
•What “on-the-job” effect is there on behavior, performance, etc.?
•What “learnings” have been transferred e.g. actually brought back to the office and given the opportunity to use/display
•Important for the actual impact on how the job is done and what can be directly observed
Results
•What impact does the training have on the overall “hard-core business metrics”?
•Ultimately, overall profitability, but can be broken into customer satisfaction, overall productivity, etc.
•Important for the final evaluation of the return on investment. Training cost vs. increased profitability
!!!
$
External research suggest moderate effect sizes (d = 0,6) for all levels. However the
relationship between levels is small and diminishing the more distal they are - so “success”
at one level is a poor predictor of “success” on other levels.
Measuring “Reactions”
94% rate
as Above
average
6% rate as
Satisfactory
No ratings
below
average
The many practice sessions during
The best thing was that the
the course makes it possible to use
learnings I received have been
them right away.
imbedded directly to my strengths or
I started using the tools
weaknesses or experience I made in
within the first week.
the past.
I seriously think this is
We were able to adapt
one of the best trainings
the material to fit our real
I ever experienced.
world working lives.
I will be using everything I
I have definitely changed during those
have learned during the
4 days and learned a lot with the
course in my workday.
group.
Testing leadership training impact through
longitudinal approach with control for sampling
Training course for some
2011
2012
2012
2012
2013
Q3
Q1
Q1-Q2
Q3
Q1
Q2
Collection of
Engagement data
Appraisal scores for
2011 consolidated
Possible attendance
Collection of
Engagement data
Appraisal scores for
2012 consolidated
Data consolidated
and analysed
Gain for trained
(n≈ 150)
Gain for not-trained
(n > 500)
Training effect
2013
Effect of Leadership Training on Engagement
Pre
Gain as result of training
Attendees
Post
Control
Effectiveness study suggest a increase in Engagement as result of training program over and above
what the control group have increased
Effect of Leadership Training on Engagement
compared to control group and benchmark
Gain in
Employee
Engagement
Effect of training program
Attendees Control
Effect
Benchmark
Increase in Engagement is
comparable to external
benchmark used
Calculating the ROI of the Leadership courses
showed a significant potential gain on the EBIT
Potential
gain of
training all
Potential
cost of
training all
ROI of
leadership
training
Managersnot having attended
X (Cost of training course + cost of lost productivity)
(Managersnot having attended / Managerstotal population )
X “Effect of training course on engagement”
X “Effect of employee engagement on financial earning”
“Application of scientific methods to the quality assurance of our
leadership training has not yet demonstrated any large effect
nor any direct causal link of our leadership training to actual
business performance. But despite this fact the participants and
our senior HR management team loves it and endorse this to be
a value adding initiative”
Mads Ingholt
Senior Director
Head of Maersk Group Leadership Development
Evaluating the Value of an Acceleration Program
An Acceleration Program in the offshore drilling
industry
WHY:
Pipeline
position
Our critical positions are scarce because the time to competency
is high and restricted by a narrow pipeline.
It is the aim of the Acceleration Program to ensure efficient
development into these positions.
Pipeline
position
Critical positions
HOW:
Every year we start new trainee teams on a 18-month Acceleration Program.
During this time trainees will receive theoretical training at our training facilities.
Practical training will take place on our drilling units worldwide where the trainees will work
their way up through the different pipeline positions and finally be ready to take on a critical
position.
Estimating the effect of Acceleration Program
Trainees
from
Acceleration
Program
Entry
position in
pipeline
Acceleration Program
was evaluated
through comparison
between groups
Comparable
peer group
Entry
position in
pipeline
Intermediate
position in
pipeline
Goal position
in pipeline
(pre-critical)
On Retention over time
On Time to develop from Entry to Goal
On Cost to develop from Entry to Goal
On Performance during pipeline
Intermediate
position in
pipeline
Goal position
in pipeline
(pre-critical)
Estimating the effect of Acceleration Program
0tan29ar29
0tan9ar9
0tan20ar20
0tan29ar29
0tan9ar9
0tan20ar20
0tan0ar0
Peer group
set to index
100
Peer Group
Trainee
% Value of program
Compared to peer group (index)
5 % better retention
35 % faster development
34 % reduced cost
9 % better performance
“Honestly we do not exactly know whether it is the selection of
trainees, the structured training or the planned job rotation &
coaching that is the “active ingredient”. Or even all of them or
none of them. But our data shows that the initiative as a whole
is adding value by ensuring a robust pipeline for those critical
positions”
Jacob Riise
Head of Maersk Drilling Acceleration Programs
Evaluating the Value of structured “On the
Job” Training
Maersk Drilling’s Competency Strategy enabling business
performance and creating competitive advantages
Strategy
Aspiration
CONSISTENTLY
HIGH TEAM
PERFORMANCE
COMPETENT
PEOPLE
We aspire to have
competent people
delivering consistently
high team performance to
our customers and
together we will win in
the market
People who have the
right, documented
skills and behaviour
Ability to use individual
skills and behaviour in
a team setting
-
Efficient training to
the right level
-
Performance
Enhancement
-
Documented
competencies
-
Strong Crew
Management focus
WINNING
TOGETHER
Together with customers
we will investigate new
competency assurance
possibilities
-
Training with
customers
-
On board training
Investigating the effect of training on performance
& setting training targets based on it
Training completion is on average a strong predictor of performance
Targets to be set at 70-90% for optimal ROI of training on performance
ROI of
training on
performance
= 100%
ROI of
training on
performance
= 50%
ROI of
training on
performance
= 10%
“The purpose of our Competency Strategy is to create an
organisation where competencies are developed and
documented in order to enable our business to perform.”
“Our analytics on the link between competencies and operational
performance tell us that our Competency Strategy is succeeding,
although we cannot say whether it is the actual competencies
that cause performance, or whether these are just a proxy for
other causal mechanism.”
Vibeke Sam
Head of Maersk Drilling Learning
“Take away points”
Lessons learned:
•
Measuring training effects is
difficult. Estimating the ROI is
even more difficult
•
But that is no reason why not to
attempt it
•
Collaboration between SMEs in
training, HRIT & Analytics is
essential
•
It is much easier if you design for
evaluation BEFORE you start any
program, than asking the question
AFTER.
Questions?
Seminar D1
L&D Initiatives
Ian Turner
Sales Academy Manager – TalkTalk Business
Our foundations for delivering great ROI
Training had to add to the
bottom line
All key stakeholders needed to be
aligned on delivery & ROI Measures
Training had to be aligned
to business strategy
Training had to take into account existing
company culture
End to end vision
Focus on ROI
Measuring training effectives – removing the variables…
Aligning ROI to the Kirkpatrick model
Results
We had to be able to
demonstrate tangible
results at all 4 levels
Behaviour
Learning
Reaction
Results
Reaction
Behaviour
Learning
Reaction
• Evaluation Sheets
Overall impact delegates felt the programme had
on their ability to produce valuable and
measurable business results
0%
10% 1%
High Impact
41%
48%
Moderate Impact
Not Sure
Little Impact
No Impact
Results
Learning
Behaviour
Learning
Reaction
Knowledge Test at training + 6 weeks and training + 6 months demonstrates
knowledge retention
Score range
Number of delegates –
(Training +6 wks)
Number of delegates –
(Training +6mts)
0 – 24%
0
0
25 – 49%
2
3
50 – 74%
18
19
75 – 100%
80
78
Total complete
100
100
Results
Behaviour
Behaviour
Learning
Competency Based Assessment
Communicat
ion
Pipeline
Manageme…
Uncovering
needs
Lead
Generation
C-Suite
Engagement
3.5
3
2.5
2
1.5
1
0.5
0
Reaction
Product
Knowledge
Negotiation
2013
2014
Market
Awareness
Commercial
Acumen
Results
Quick and simple process built into our CRM to capture immediate feedback at point of
sale
Value prepopulated by
CRM
What stage
was the deal
at prior to our
help?
YES
Has the Sales Academy
helped you to win this
deal?
CRM
calculates
Value
attributed to
Sales
Academy
NO
Specifically which
interventions
helped and how?
END
END
Results
Results
Behaviour
Learning
Sales Revenue from Level 4 Survey
Notes
Reaction
Total reported size of all deals
before training
£13,000,000
Total reported size of all deals after
use of tools from Training
£14,100,000
Value of net sales attributed to
training techniques
£ 1,100,000
Overall Summary
Total cost of training investment
Net Reported Return
Gross ROI
Net ROI
The total amount of the sales revenue attributed to the training
Notes
£ 250,000
£ 1,100,000
Gross ROI %:
(Total adjusted Return
/Total Cost)
NET ROI %:
((Total adjusted Return Total Cost)
/Total Cost)
440%
340%
£ 3.40 in sales were generated for every £1 invested!
ROI Example
Techniques That Helped Participants Most in Their
Sale/s
Value Map
17%
Value Chain
28%
Stakeholder Map
12%
Qualifying the
Opportunity:
Tool
15%
Profitability –
Leakages & Levers
8%
Assessing The
Account
Partnership
People –
6%
Politics
–
People –
Improving
Mapping
Stakeholder
Relationships
Alignment
9%
5%
Summary
Align to
strategy
Engage all
stakeholders
All 4 levels of
Kirkpatrick
Robust ROI
Seminar D1
L&D Initiatives

Seminar D1

Transcription

Similar documents

Maersk Completer - Maersk Drilling

Maersk Resolute - Maersk Drilling

Maersk Drilling Annual Result 2015

2 June 2010 – draft for comment

Consent for Ghusal and Interment

Mærsk Gallant - Maersk Drilling

How to Set up An Employee for Electronic Tax Form

mærsk laser - remote anchor handling system

Teacher Pages

ME4 Newname, still bestin class

preparing for the future

MAERSK POST

When the whistle blows More flexible, equally ambitious