Why NOTECHS? - RAeS Human Factors Group

BR&TE
From NOTECHS to
LH Behaviour Markers
An Implementation Case Study
Hans-Jürgen Hörmann, PhD
Boeing Research &
Technology Europe
Madrid, Spain
Safety and Human Factors
Cpt. Harry Neb,
Lufthansa German Airlines
FRA PC/L
Human Factors Operational
Input
Royal Aeronautical Society - Human Factors Group
Seminar on Assessment & Accreditation, London April 30, 2004
BR&TE
Overview
Principles of Behaviour Assessment in NOTECHS/JARTEL
Development of Lufthansa‘s New Behaviour Marker System
Crosslinking NOTECHS to LH Interpersonal Competences
The Implementation Case: Lufthansa‘s Evaluation Method
Usability Study
Summary
BR&TE
NOTECHS/JARTEL
Principles and Practice of Behaviour
Research
Technology Europe
Marker&Application
Hans-Juergen Hoermann, PhD
Boeing Research & Technology Europe
RAeS, April 30, 2004
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
Why NOTECHS?
BR&TE
JAR OPS (2001) 1.940, 1.945, 1.955, and 1.965,
asking for an evaluation of flight crews’ CRM skills.
“the flight crew must be assessed on their CRM skills in accordance
with a methodology acceptable to the Authority and published in the
Operations Manual.” (1.965)
“The purpose of such an assessment is to: provide feedback to the
crew collectively and individually and serve to identify retraining; and
be used to improve the CRM training system.” (1.965).
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
NOTECHS/JARTEL Project Team
BR&TE
IMASSA
sofréavia
NOTECHS
1997- -2001
1998
JARTEL 1998
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
Initial Setup of NOTECHS
BR&TE
For the purposes of the project, non-technical skills (NoTechS) were
defined as...
“the cognitive and social skills of flight crew members in the
cockpit, not directly related to aircraft control, system
management, and standard operating procedures (SOPs)”
(Avermaete, 1996)
Requirements for NOTECHS…
ƒ to assess the skills of an individual pilot, rather than a crew
ƒ assessments based on observable behaviours
ƒ precisely defined, culturally robust, reliable, and practicable
ƒ to be suitable for use across Europe by both large and small
operators
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
NOTECHS Development
BR&TE
Information Sources
Flin & Martin (1998, 2001) review of existing BM systems
In-depth literature study on selected NTS concepts
Airlines’ current practices
- Air France
- KLM (WILSC / SHAPE)
- Dutch CAA (RLD)
- University of Texas Line-LOS Checklist
- Lufthansa (Quick Reference List)
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
The NOTECHS System
BR&TE
Non-technical skills
Co-operation
Leadership &
management skills
Team building &
maintaining
Decision
making
Category
Element
Considering
others
Supporting
others
Situation
awareness
Behaviour
Helps other crew members in demanding situations
Offers assistance
Conflict
solving
(Avermaete & Kruijsen, 1998)
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
The NOTECHS Framework
Categories
Co-operation
Leadership & Management
skills
BR&TE
Elements
Team building and maintaining
Consideration of others
Supporting others
Conflict solving
Use of authority/assertiveness
Maintaining standards
Planning and co-operation
Workload management
Situation Awareness
System awareness
Environmental awareness
Anticipation
Decision Making
Problem definition/diagnosis
Option generation
Risk assessment/option choice
Outcome review
(Rhona Flin et al., 2003)
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
JARTEL – The NOTECHS Health Check
BR&TE
Influence of cultural differences on the use of NOTECHS across
Europe
Inter-rater agreement and consistency when assessing the same
scenarios
Applicability and usefulness in real check situations
Study 1: Experimental approach
ƒ N = 105 IPs from all over Europe assessing eight recorded
scenarios with good and bad examples of NTS behaviours after
a half day familiarization course
Study 2: Operational approach
ƒ N = 25 IPs from five airlines participated in the feasibility check
of applying NOTECHS in real training and check situations
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
NOTECHS Principles
BR&TE
The 5 principles of NOTECHS application in practice:
Only observable behaviour is to be assessed
Need for a technical consequence
Repetition required
Rating shall distinguish between acceptable and unacceptable NTS
Explanation required
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
Basics of Behaviour Observation
BR&TE
Behaviour is defined in the dictionary as
action or reaction of something under specified circumstances.
Overt behaviours
• Actions that others can directly observe, like activities, movements,
postures, gestures, positions, facial expressions, speech, or social
interactions
Covert behaviours
• Processes, that cannot be directly observed, like thoughts, cognition,
feelings, physiological responses
Behaviour observation is the planned collection of behavioural data
while subjects are engaged in various activities in certain situations
with the intention to assess the collected behavioural data against
specified standards.
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
NOTECHS Application – An Example
BR&TE
ƒ Results of an airline audit - NOTECHS to identify training needs
(Study conducted by German Aerospace Center (DLR), 02/2000)
5-point Rating Scale
5
4
Average scores of 17
airline pilots
3
2
1
0
.
.
.
.
.
s
.
t.
.
e
g
rs
fin e ra t ss m v iew
e rs ertiv tand rdi n ngm
im
em ronm
t
din the
t
e
h
l
t
i
s
e
o
s
e
s
u
SA e m d gen ass
.o
.o
co ad m A sy en vi
as ing
er
l
mb ider p ort
&
n
&
m
.
k
a
d
b
S
o
t
o
o
s
o
p
ity rovi ning orkl
Te ons
ex
pti
Ri
Pr
utc
Su thor
O
A
P
O
n
C
W
S
Pla
Au
Behavioural Elements of NOTECHS
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
NOTECHS Application – An Example
BR&TE
ƒ Results of an airline audit - NOTECHS to examine training effects
(Study conducted by German Aerospace Center (DLR), 02/2000)
5-point Rating Scale
5
Pre CRM
Post CRM
4
3
2
1
0
.
.
.
.
.
s
.
t.
.
e
g
rs
fin e ra t ss m v iew
e rs ertiv tand rdi n ngm
im
em ronm
t
din the
t
e
h
l
t
i
s
o
s
n
s
u
s e e re
SA e m d
.o
.o
co ad m A sy en vi
as ing
ge
as
mb ider p ort
l
&
n
&
m
.
k
a
d
b
o
S
i
y
tio Ris
xt
ro
tco
Te ons
ing orkl
e
p
ov
rit
up
P
u
r
n
o
S
O
P
O
n
C
W
SA
th
Pla
Au
Behaviour Elements of NOTECHS
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
NOTECHS Application – An Example
BR&TE
ƒ Results of an airline audit - NOTECHS to identify training needs
(Study conducted by German Aerospace Center (DLR), 02/2000)
5-point Rating Scale
5
Pre CRM
Post CRM
4
3
2
1
0
C
n
tio
a
r
pe
o-o
.
rsh
e
ad
Le
&
t.
gm
n
M
A
on
i
t
ua
Sit
s
es
n
e
r
wa
D
g
kin
a
M
on
i
s
i
ec
Behaviour Categories of NOTECHS
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
References
Aberdeen website:
EAAP website:
Sofreavia website:
BR&TE
www.psyc.abdn.ac.uk/serv02.htm
www.eaap.net
www.sofreavia.fr/JARTEL
Avermaete, J. van & Kruijsen, E. (Eds.) (1998). NOTECHS. The Evaluation
of Non-Technical Skills of Multi-Pilot Aircrew in Relation to the JAR-FCL
Requirements. Final Report NLR-CR-98443. Amsterdam: NLR.
Flin, R. et al. (2003). Development of the NOTECHS (non-technical skills)
system for assessing pilots’ CRM skills. Human Factors in Aerospace
Safety, 3(2), 97-119.
Goeters, K.M. (Ed.) (2004). Aviation Psychology – Practice and Research.
Research
Aldershot: Ashgate.
Hoermann, H.J. (2001). Cultural variations in the perception of crew
behaviour in multi-pilot aircraft. Le Travail Humain, 64(3), 247-268.
O’Connor, P. et al. (2002). Developing a method for evaluating CRM skills:
A European perspective. International Journal of Aviation Psychology,
Psychology
12(3), 265-288.
H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr. 2004
BR&TE
What Next?
Principles of Behaviour Assessment in NOTECHS/JARTEL
Development of Lufthansa‘s New Behaviour Marker System
Crosslinking NOTECHS to LH Interpersonal Competences
The Implementation Case: Lufthansa‘s Evaluation Method
Usability Study
Summary