Beyond CBT: Game-Based Training Efficacy, Efficiency, and

Beyond CBT: Game-Based Training
Efficacy, Efficiency, and Applications
Amanda Bond
Lead Human Performance Engineer
Cubic Global Defense
Orlando, Florida, USA
CUBIC PROPRIETARY
THE CURRENT STATE OF TRAINING
CUBIC PROPRIETARY
First… Some History
Notable events in the past 25 years:
• 1990: Microsoft officially launches PowerPoint, the first MD-11 flight
(Helsinki to Tenerife), and the first in-flight entertainment in seats appears
on Northwest Airlines
• 1993: Ticketless travel first appears, the A321 enters service, and the
1,000th 747 rolls off the production line
• 1995: Internet bookings start
• 1996: The A319 enters service, and McDonnell Douglas merges into The
Boeing Company
• 1998: Smoking is banned on domestic U.S. flights
• 1999: Internet check-in and boarding passes start, and airlines ban mobile
phone use on aircraft in the United States
• 2000: JetBlue starts operations, in-flight connectivity appears, and the first
autonomous flight happens (the RQ-8A Fire Scout)
• 2001: September 11th changes everything, and the TSA is created
• 2003: The last Air France Concorde flight
• 2007: Singapore Airlines flies the first A380, fees for checked bags appear
on U.S. airlines… so competition for overhead bins becomes epic
• 2012: Emirates Airlines offers the first mobile phone service on aircraft
CUBIC PROPRIETARY
Then vs. Now
The Flight Deck…
Now
Then (okay, 1980)
CUBIC PROPRIETARY
Then vs. Now
In Flight Entertainment…
Now
Then
CUBIC PROPRIETARY
Then vs. Now
Security…
Now
Then
(okay, this one is pretty far back)
CUBIC PROPRIETARY
Then vs. Now
Passenger Seats…
Now
Then
CUBIC PROPRIETARY
Then vs. Now
Passenger Technology…
Now
Then
CUBIC PROPRIETARY
Then vs. Now
The Passengers…
Now
Then
Now
CUBIC PROPRIETARY
Now
Then vs. Now
The Passengers…
IATA Joint Position Paper on Unruly Passengers:
http://www.iata.org/policy/Documents/tokyo-revision-position-paper.pdf
CUBIC PROPRIETARY
What Does That Mean?
• Cabin crew no longer just follow procedures, but
must react, decide, communicate
• New, changed, and more drastic concerns for:
– Fires
– Security
– Evacuations
• New, changed, and increased concerns for customer
satisfaction
– Longer flights
– More amenities
CUBIC PROPRIETARY
Why CBT & PPT
• So are computer-based training and
passive learning still the best
teaching methods?
CUBIC PROPRIETARY
A Refresher on Knowledge
•
•
•
•
Bloom’s Taxonomy (2014)
Low level thinking: Recall, Demonstrate
High level thinking: Use, Examine, Change, Evaluate
Today’s tasks require more high-level thinking,
which requires more high-level training
Image from: http://www.educatorstechnology.com/2014/03/new-blooms-taxonomy-planning-kit-for.html
CUBIC PROPRIETARY
Knowledge, Skills, Abilities?
• Knowledge = declarative information applied to
performance of a function
• Skills = observable competency to perform a
psychomotor task
• Ability = competence to perform an observable
behavior or observable product
Example: Ditching task
• How is that assessed?
CUBIC PROPRIETARY
For Example…
• August 1985: Japan Airlines flight 123
– Lost all hydraulic lines and control surfaces due to structural
failure/faulty maintenance
– Pilots able to fly for 32 minutes prior to crash
• July 1989: United flight 232
– Lost all hydraulic lines and control surfaces due to structural
failure
– Pilots (and instructor) able to fly for about 40 minutes and
execute a very controlled landing despite breakup of the
aircraft, contributing to lives saved
• November 2003: OO-DLL DHL
– Lost all hydraulic lines and control surfaces due to a surfaceto-air missile strike
– Pilots and Flight Engineer able to fly for about 16 minutes
including a missed approach and land at Baghdad safely
CUBIC PROPRIETARY
For Example…
• What do these examples have in common?
– Total loss of control surfaces
– There was no checklist or procedure
• What is different about these stories?
– Outcomes
– The UAL check airman who assisted had studied the JAL crash
and learned from it
– The DLL CPT had attended training by the UAL check airman
• Do cabin crew experience critical situations for which
there are no checklists or procedures?
CUBIC PROPRIETARY
For Example…
YES. Cabin crew can and do encounter critical situations
for which there is no precedent
For example…
• Fire:
– Low level thinking:
•
•
•
•
See fire
Recall firefighting procedures and tools required
Identify appropriate procedure
Extinguish & check fire using procedure
CUBIC PROPRIETARY
For Example…
– High level thinking:
•
•
•
•
•
See fire,
evaluate fire to determine type,
choose appropriate action based on fire type
recall firefighting procedures for that fire type
extinguish & check fire using procedure
– OR… High level thinking:
•
•
•
•
•
See fire,
Evaluate fire to determine type
Find no existing procedure based on fire type
Create solution based on knowledge and understanding
Extinguish & check fire using generated procedure
CUBIC PROPRIETARY
What Are We Trying to Teach?
• Declarative knowledge is traditionally taught in the
classroom
– CBT, PPT
• Procedural skills are traditionally taught by an
instructor
– CBT, PPT
• Higher-level skills are learned by doing and
experiencing
– Immersive training
CUBIC PROPRIETARY
HOW GAME-BASED TRAINING TEACHES
CUBIC PROPRIETARY
Experiential Learning
Experiential Learning (Kolb, 1984)
Caveats:
• The learner must be willing to
be actively involved in the
experience
• The learner must be able to
reflect on the experience
• The learner must possess and
use analytical skills to
conceptualize the experience
• The learner must possess
decision making and problem
solving skills in order to use
the new ideas gained from the
experience
Concrete
Experience
Active
Experimentation
CUBIC PROPRIETARY
Reflective
Observation
Abstract
Conceptualization
Game-Based Training
• A game is an interactive, intrinsically-motivating, and
rule-governed experience through which an
individual progresses to achieve a goal (to beat the
level, to progress the storyline, to get the most
points)
• Game-based training (GBT) harnesses the
motivational aspects of games and places
instructional content at the core of gameplay: One
must apply knowledge and skill in order to win
CUBIC PROPRIETARY
Benefits of Game-Based Training
• Stizman (2011) Meta Analysis found
– 20% increase in Self Efficacy
– 11% increase in Declarative knowledge
– 14% increase in Procedural knowledge
– 9% increase in Retention
• Other studies show even larger gains
– E.g., Wouters, van Nimwegen, van Oostendorp, & van der
Spek, 2013
CUBIC PROPRIETARY
Benefits of Game-Based Training
• Players learn in context by interacting with objects in a
complex world.
• Consistent with experiential learning, learners can make
connections among concepts
– Players build sound mental models of a domain.
• Games are excellent model-based environments to foster
complex reasoning.
– Students observe system behavior over time; draw and test hypotheses.
• Games provide the players with constant challenge--many
parallel achievements feed into an overriding goal.
• Goals are concrete and immediate.
CUBIC PROPRIETARY
Benefits of Game-Based Training
• Simultaneously teaches and assesses knowledge, skills,
decision-making, and psychomotor abilities in a single
instance
• Make the time spent in any physical spaces (aircraft) or
simulators more effective (e.g., shorter time to reach
competency) to increase throughput.
• Leverages existing investments
• Expose trainees to scenarios that are difficult--or even
impossible--to train.
• Trainees learn how to handle the unexpected
• Provide in-depth data for analytics.
• Give instructor insight into their trainees
CUBIC PROPRIETARY
Benefits of Game-Based Training
Make the time spent in physical simulators
more effective.
When trainees learn and practice first in the game, the first time in
the simulator, s/he …
• … spends less time struggling with what to do because they
are already familiar with the physical environment.
• … receives instruction tailored to their needs in the
simulator.
This approach is effective because of
schematic processing. Since s/he has already
developed a foundation for the operational
environment and the tasks, they will learn
more quickly.
CUBIC PROPRIETARY
Benefits of Game-Based Training
Expose trainees to scenarios that are
difficult--or even impossible--to train.
When trainees experience dangerous situations in the game, s/he…
• … can practice skills and problem-solving techniques that
they would otherwise not be able to do.
• Fire fighting
• Complex medical emergencies
• Unruly, violent passengers near open doors
• … can drill emergency scenarios over and over until they are
trained to automaticity – then you can throw a curveball to
train critical thinking and resilience.
The highly realistic, immersive environment
encourages transfer of training.
CUBIC PROPRIETARY
Benefits of Game-Based Training
Provide in-depth data for analytics.
The trainee’s performance can be tracked and analyzed based on performance as
well as knowledge
• … to determine gaps in knowledge and skill so that they may be addressed
(either by a virtual or real instructor).
• The instructors will know what they need to spend time on with
trainees, making the most of their time.
• … to determine what training they need and what they can skip to optimize
their time at the training center.
• Returning employees and new employees from other airlines can
reduce their training time.
• Training can become adaptive at a very low level of competency
• Make the unobservable observable with data collection and careful
game construction.
• … to use for optimizing crew composition.
• Know which crews will work together the best before they fly for the
first time.
CUBIC PROPRIETARY
GAME-BASED AND IMMERSIVE
TRAINING: RETURN ON INVESTMENT
CUBIC PROPRIETARY
Return-on-Investment
• Game-based training has higher up-front costs based on the number
of development hours required…
Dev. Hours
for One
Hour of
Instructional
Content
Total Hours of
Instructional
Content
Total Dev.
Hours
Hours
Spent
Using
Dev Hours
/ Using
GP*
Gained
per Hour
Total GP
Gained
Dev.
Hours /
+ 1 GP
CBT
2201
5
1,100
53
220
+ 0.5002
+ 2.5
440.0
GBT
1,0001
5
5,000
1042
48.08
+ 0.1252
+ 13
384.6
1Sitzmann,
2011; 2Fletcher, 2011; 3Austin, 2013
• … But game-based training has a better return-on-investment in the
long-run based on the number of development hours required
divided by the total gain in knowledge.
– In GBT, fewer development hours are needed to achieve comparable
learning gains.
*GP = Grade Placement is a standardized metric for quantifying learning gains and is
based on the work of Suppes, Fletcher, & Zanotti (1975).
CUBIC PROPRIETARY
Training Effectiveness – Return on Investment
Fewer game development hours are needed to achieve comparable learning gains.
Fletcher (2011); Sitzmann (2011); Austin (2013); Suppes, Fletcher, & Zanotti (1975).
CUBIC PROPRIETARY
Training Efficiency – Reduced Training Costs
• Complementary to live simulation/real-time training, not
superseding
• Reduced trainee footfall and instructor resource bill
across Cabin Crew training
• Better understanding and higher first-time pass rate on
expensive, resource-intensive training milestones (e.g.,
mock-up/operational assessments) = less time training
on this equipment
• Realistic and enjoyable training experience
• Maintains a single training standard for large numbers of
trainees, while optimizing instructor resourcing and
expertise
CUBIC PROPRIETARY
Conclusions
• Immersive Game-Based Training is one way to bring
training into the next generation of Cabin Crew
• Game-Based Training is adept at teaching higher-level
cognitive tasks and decision-making as well as
declarative and procedural knowledge
• Game-Based Training also provides numerous returns
on investment both monetarily and with respect to
crew instruction
If you’d like to know more please come to the Cabin
Crew Training Workshop 0900 – 1230 Wednesday M23
CUBIC PROPRIETARY