Cabin crew expected safety behaviours

Human Factors and Aerospace Safety 4(3) 153-167
© 2004, Ashgate Publishing
FORMAL PAPERS
Cabin crew expected safety
behaviours
Peter Simpson, Christina Owens* and Graham Edkins**
Cathay Pacific Airways Ltd, Hong Kong
* Qantas Airways Ltd, Australia
** Victorian Department of Infrastructure, Australia
Abstract
Expected Safety Behaviours (ESBs) or non-technical skills are becoming an
accepted aspect of pilot training, however, the concept has not been widely
applied to cabin crew. This situation is changing, as regulatory authorities start to
require the assessment of cabin crew CRM and safety behaviours. Qantas has
developed a set of expected safety behaviours for the training and assessment of
cabin crew. Unlike previous sets of behavioural markers, the cabin markers were
developed using a valid scientific methodology. The ESBs were derived from 80
interviews conducted with experienced cabin crew using the critical decision
method protocol, which uses probing questions to explore critical decision events.
Seven ESB categories were derived; situation awareness, passenger management,
crew management, workplace safety, operational understanding, negotiation and
influencing skills, and information and resource management.
Introduction
Crew Resource Management (CRM) has been considered an integral aspect of
pilot training since the early 1980's, and of cabin crew training since the early
1990's. It is only in the last few years that CRM skills and competencies have
begun to be assessed as a regular part of pilot check and training. The concept of
CRM competency assessment may be relatively new to the cockpit, but it is even
Correspondence: Christina Owens. Qantas Airways Ltd. QCC4, Bourke Rd, Mascot, NSW 2020
AUSTRALIA, or e-mail: [email protected]
154 Peter Simpson. Christina Owens and Graham Edkins
newer in the cabin. This paper describes the method that Qantas has used to
develop CRM competencies (also known as non-technical skills or expected
safety behaviours - ESBs) for cabin crew, and the process Qantas is developing
for their use and assessment.
Expected safety behaviours (ESBs) and CRM competencies
CRM and human factors training have been required by regulatory authorities
(e.g., CASA, FAA, JAA) as part of pilot licensing for several years. Cabin crew in
Australia do not require licensing, but most have been exposed to CRM and
human factors training via company training programs. However, until now, the
Australian Civil Aviation Safety Authority (CASA) had not required the
development or assessment of CRM competencies (ESBs) for cockpit or cabin
crew; Civil Aviation Safety Regulation 121A will require this. CRM is finally
starting to be recognised as being as important as technical skills for the safe and
efficient management of a flight.
Expected safety behaviours (ESBs) are generally thought of as observable, nontechnical behaviours that contribute to effective performance within a specific
work environment. They are usually structured into categories, and the categories
are constructed of several elements or behaviours. A behavioural marker should
describe a specific, observable behaviour, not an attitude or personality trait.
These elements or behavioural
markers can enable CRM performance
measurement and assessment, and can also be used to build performance
databases to identify norms and prioritise training needs, including the evaluation
of training (Klampfer et aI., 2001).
In 2001, Qantas started a two-year research project to develop ESBs for cabin
crew. Like other airlines, Qantas already had some experience in observing and
scoring pilot CRM for check and training purposes, although this was not jeopardy
assessment. Methodological
guidelines for the development of behavioural
markers and CRM competencies are lacking. Programs such as NOTECHS (van
Avermeate and Krujisen, 1998; Flin et aI., 2003), LOSA (HeImreich, Klinect and
Wilhelm, in press), and GIHRE (Klampfer et aI., 2001) have developed their lists
of pilot CRM skills and behaviours from training workshops and meetings, and
they are based on opinion and experience, using a variety of informal methods and
techniques, such as amalgamating existing marker systems. A number of airlines
have developed their own behavioural markers systems based on these programs
(Flin and Martin, 2001), or using their methods and processes (e.g., Qantas).
Furthermore, there does not appear to be any published research into the
development of behavioural markers for cabin crew, despite cabin crew CRM
training being mandated in many countries. Numerous investigation reports
reinforce the need for cabin crew to take appropriate action to manage threats and
errors (for example, Air Ontario at Dryden, American Airlines AA132 at
Nashville, British Air Tours runway fire at Manchester, Qantas QFl at Bangkok,
Cabin crew expected safety behaviours 155
Singapore Airlines SQ6 at Taipei). Handling many of these threats and errors calls
for knowledge, skills and abilities quite different from those associated with the
provision of normal service duties, yet there is no consensus on which skills are
needed for effective cabin crew CRM (lCAO, 2002).
The cabin crew ESB project aimed to develop a valid and reliable method of
identifying and developing ESBs, based on established human factors and
behavioural science methods, the primary method being the Critical Decision
Method (CDM) (Klein, Calderwood and McGregor, 1989). The CDM is a
retrospective interview strategy that applies a set of cognitive probes (see table 1)
to non-routine incidents that requires expert judgement or decision making (Klein
et aI., 1989). A semi-structured interview format is used to probe different aspects
of the decision process and situation assessment. The CDM is a variant of
Flanagan's (1954) critical incident technique. The critical incident technique can
be applied for a variety of situations and uses in aviation, including: developing
critical requirements for evaluating the typical performance and safety behaviour
of an operator, evaluating operator proficiency in a check and training situation,
and providing an indication of the effectiveness of training programs (Flanagan,
1954). These are three areas that the ESB project will cover at Qantas, and all
three areas are useful in meeting the up coming Australian regulatory
requirements.
The CDM is effective in revealing experts' knowledge, especially tacit
knowledge, reasoning and decision strategies. Compared to other methods of
knowledge elicitation, the CDM yields more information, including a wider variety
of specific cognitive details, more information about underlying causal linkages
among core subjects, and the revelation of tacit knowledge (Hoffman, Shadbolt,
Burton and Klein, 1995). The reliability of the procedure is based on the idea that
experts have clear memories of salient or unusual safety-related incidents (Hoffman
et aI., 1995). This method of knowledge elicitation has been used successfully in
naturalistic environments such as fire fighting, para-medicine, nursing, helicopter
flying, and military command and control (see Simpson, 2001 for a review). The
CDM is now established as a valid and reliable method of cognitive task analysis
and knowledge elicitation (Hoffman, Crandall and Shadbolt, 1998; Taynor,
Crandall and Wiggins, 1987). However, the limitations of this method are discussed
by O'Hare, Wiggins, Batt and Morrison (1994), and Hamm (2004).
Method for developing the ESBs
Participants
The participants for the critical decision interviews consisted of eighty Customer
Service Managers (CSMs) (known as Pursers or Cabin Supervisors in some
airlines). The CSMs were a mixture of short haul (54%) and long haul (46%) from
156 Peter Simpsoll. Christilla Owells alld Graham Edkills
various bases across Australia. They had spent an average of 7.5 years operating
as a CSM (range = 1-25yrs, sd = 6.2yrs), and their mean age was 42 years (sd =
7.0yrs).
The criteria for participation were that the person must be a CSM, and that they
could discuss, in detail, a recent (within 18 months) safety-related event that was
challenging. CS Ms were paid for the ground duty time they spent being interviewed
(usually 1.5 - 2 hours). Six people, who were also CSMs, conducted the interviews.
Although their main role was interviewing, these six team members also helped
with data analysis and development of the safety behaviours because they are
subject matter experts, with the domain knowledge area of cabin safety.
Desigll
The variables of home base location (Sydney, Melbourne, Brisbane, Perth) and
operation type (short or long haul) were controlled to ensure the CSMs
proportionally represented the Qantas operation. The interview process was based
on the Critical Decision Method (CDM) described in the Introduction.
Procedure (stage J) - critical decisioll method (CDM) The procedure for
employing CDM is well documented (Klein et aI., 1989; Hoffman et aI., 1998),
but the basic steps used in the interviewing process included:
1. Incident selection - CS Ms select a recent, non-routine incident that was
challenging;
2. The interviewer obtains an unstructured free recall of the event;
3. Both participant and the interviewer establishes the sequence of decision
events and constructs a time line;
4. Decision point identification - the interviewer identifies specific decisions
that were made;
5. Decision point probing - probing/questioning techniques are used to identify
effective decisions and resultant behaviours (see Table 1 for examples of the
probes);
6. Hypotheticals and 'what ifs?' The interviewer chooses several decision
points, and asks hypothetical questions based upon different event outcomes
(eg., what would you do if the Captain didn't take your request seriously?).
These queries serve to identify potential errors, alternative decision-action
paths, and expert/novice differences (Hoffman et aI., 1998); and
7. Standard case study - CSMs are provided with a standard case studyl and
repeat steps 3 to 6 as if they were the CSM onboard that aircraft.
I
The standard case study was a real Qantas incident on-board a B737. It involved smpke and
fumes in the cabin. with many CRM issues and problems.
Cabin crew expected safety behaviours 157
Table 1
A sample of CDM probe questions
1998)
Probe type
Cues
(adapted
from Hoffman
et aI.,
Probe content
What were you seeing, hearing, smelling?
Knowledge
What info did you use in decision making and how was it
obtained?
Analogues
Were you reminded of any previous experience?
Standard
scenarios
Does this case reflect a typical scenario or a scenario you
were trained in?
Goals
What were your specific goals and objectives at the time?
Options
What other courses
available?
Mental
models
Did you imagine the possible consequences of your action?
Experience
What specific
necessary?
Decision
Was there time pressure? How long did it take to make the
decision?
making
of action were considered
training,
experience
or
or were
knowledge
was
Aiding
What training, knowledge or experience could have helped
you?
Errors
What mistakes
novice act?
are common at this point? How might a
Participants were informed that the interviews were anonymous. Interviews
were conducted in a quiet office and tape-recorded (if permission was granted).
Most interviews lasted for 1.5-2 hours; requiring at least one hour for the first
critical incident, and half to one hour for the repeated case-study incident.
Verbatim transcripts were made from the tapes.
Procedure (stage 2) - development of expected safety behaviours (ESBs) The
second stage of the study involved the coding of the interviews and development
of the ESBs. The procedure consisted of a number of steps:
1. Initial review and coding of the transcripts to develop behavioural markers.
This was conducted by the research team in a workshop situation. Ten
interviews were used to develop the initial ESB list. This critical step is
expanded in Appendix A.
158 Peter Simpson, Christina Owens and Graham Edkins
2. Improve code structure and markers with feedback from cabin crew subject
matter experts (the six CSMs within the project team).
3. Test the improved ESB list by coding a further 10 interviews for refinement.
The aim of this step was to determine whether the list of ESBs was able to
cover a multitude of cabin situations that required both technical and nontechnical skills.
4. Analysis of behavioural markers by general CSM population, including
written feedback requested from the entire CSM population to gain general
comment and feedback on the list of ESBs, and brief interviews (20 mins)
with random CSMs (n=20) in crew lounges to attain more detailed
responses. Both methods included rating the relative importance of each
behavioural marker to the safe, efficient conduct of the flight.
5. Produce a master list of ESBs.
6. Code remainder of interviews (approximately 60).
7. Attain a frequency count of behavioural marker elements occurring in each
critical incident.
8. Construct a 'training focus matrix' (frequency ESB occurring versus
perceived importance ESB) for the training of expected safety behaviours
(see figure I). This is based upon the results from steps 4 and 7 and allows
objective prioritisation of training.
High
I
M
P
o
R
T
A
N
C
E
Low
Priority
Low
High
FREQUENCY
Figure 1 Expected safety behaviour
Low
training focus matrix
Achieving a high degree of inter-rater reliability is one of the challenges of
coding transcripts. To avoid this problem completely, once this final ESB coding
structure was developed, a single person coded all the remaining interviews.
Cabin crew expected safety behaviours 159
Results and discussion
Critical decision method
The majority of critical incidents recalled by CSMs could be categorised under
seven headings (see figure 2). Almost half of the. incidents were related to
disruptive or drunk passengers (20%), in-flight medical emergencies (17%), or
security/terrorism threats (12%). The large number of security/terrorism threats
has only occurred since Sept I I th terrorist attack. It is interesting to note that many
safety issues involve the aircraft door and slides (10%). Aircraft technical issues
(8%) refer to problems such as aborted take-offs, engine problems, and cockpit
and cabin equipment malfunctions. The break down of all Qantas reported cabin
safety incidents for the 13 month period Jan 200 I to Feb 2002 is proportionally
similar to those of this study (Qantas, 2002). This validates that the ESBs were
developed from critical incidents that reflected the real cabin incidents and events
during that time period.
drunk/disruptive
pax 20%
turbulence 4%
----
aircraft technical is
smoke/fumes
Figure 2
medical
17%
9% /
/
security/terrorism
door/slide issue 10%
12%
Type of critical incidents recalled in the CSM interviews
A final list of seven ESBs was derived (see table 2). The behavioural markers
(or elements) are the CRM competencies that can be used for training and
checking, and they can also be used to satisfy Australian regulatory requirements.
The ESB list is similar to other existing, 'workshop-derived' lists. However, this
research appears to be the only published work on the topic, and it now validates
some of the CRM competencies that are being trained. It should be stressed that
the ESBs are specifically for Qantas international and domestic cabin crew, and
they may not necessarily transfer to other airlines or cultures without
modification. Other Australian carriers have started to modify the Qantas ESBs to
suit their own operations.
160 Peter Simpson, Christina Dwens and Graham Edkins
Table 2
Expected safety behaviour categories and elements
Expected safety
Expected safety behaviour element/behaviour
behaviour category
Situation
• Demonstrates awareness of flight phase or situation
awareness
• Considers political and cultural context
• Considers time constraint
• Recognises higher safety goals and priorities
• Anticipates decision consequences
• Develops contingency plans
Passenger
management
• Assesses passengers (boarding or in-flight)
• Monitors potentially threatening pax behaviour/condition
• Acts decisively to modifY passenger
behaviours/condition
•
•
•
•
Considers passengers well-being
Presents a calm, controlled image to passengers
Diffuses situation in a non-confronting manner
Minimises cabin disruption
Workplace safety
•
•
•
•
•
Pro actively manages OHandS situations
Reactively manages OHandS situations
Follows-up OHandS situations
Communicates OH andS importance
Displays role model OHandS behaviours
Crew
• Assesses crew
management
•
•
•
•
Operational
understanding
• Demonstrates Basic Aeronautical Knowledge (BAK)
• Understands authority/duty of CSM
• Understands authority/duty of others
Negotiation and
influencing skills
•
•
•
•
•
Information and
resource
• Identifies and utilises all resources
• Gathers information
management
•
•
•
•
Provides onboard coaching and training to modifY behaviour
Considers crew well-being
Considers impact of non-routine events on crew performance
Allows and provides crew debrief
Consults
Manages
Manages
Manages
Manages
with others to develop a common strategy
upwards 1 - identifies problem
upwards 2 - expresses concern
upwards 3 - provides options
upwards 4 - uses emergency language
Confirms common understanding of information
Critically analyses information
Provides timely feedback to those who need to know
Prioritises tasks
Cabin crew expected safety behaviours 161
As a learning point from the study, other airlines would be advised to carefully
select their interviewers. Although senior cabin supervisors are experienced in
cabin safety, they are not necessarily experienced in interviewing. Several
interviews were not as detailed as they could have been, due to a focus on physical
events rather than decisions, and many were 'war stories' rather than cognitive
analysis. However, all the interviews were useful and have proven highly valuable
as examples and support for training development.
The frequency of ESB elements was summed for the 80 critical incidents,
yielding over 1,500 discrete occurrences of the elements. This allowed the
determination of the safety behaviours most frequently used by CSMs in normal
operational events and incidents. Figure 3 shows a selection of the frequency
results. The category of Information and Resource Management was the most
frequently occurring safety behaviour. It is thought that the high frequency of this
behaviour is unique to CSMs, because they are the cabin manager, and are
responsible for the control of information and resources within the cabin, and
between the cockpit and cabin. The CSM is also the cabin member most often
making decisions and solving problems for all the cabin crew.
The second most frequently displayed safety behaviour was the category of
Situation Awareness. Again, as the cabin manager, it is not surprising that the
CSMs behaviour and decisions were very closely related with aspects of being
situationally aware. All three levels or stages of situation awareness - perception,
comprehension, and projection (Endsley, 1995) - were represented in the safety
behaviour elements. Projection was evident when the CSMs determined the
consequences of their decisions before implementing them, and their development
of contingency plans in case situations and circumstances changed.
With the frequency count of the ESBs, and the rating of perceived importance by
the CSM population, Qantas now has information on what the most commonly
displayed cabin safety behaviours were, as well as which were perceived to be the
most important. It is interesting to note that the most frequent category, Information
and Resource Management, was considered to be the least important category.
Situation Awareness was ranked highly both in frequency and perceived importance.
This indicates that what crew perceive to be important safety behaviours, may not
reflect reality. This may have an impact on the validity of behavioural marker lists
derived from workshop and 'round-table' discussions that are based on experience
and perception, rather than valid and reliable scientific methodology.
Application of the ESBs
Use in training
Qantas now has 80 real, recent, and highly detailed safety events that can be used
as case studies and examples. They all detail actual threats encountered and errors
162
Peter SimpSOll, Clzristilla QwellS alld Gralzam Edkins
made on Qantas aircraft, and most are very good examples of threat and error
management. This is an excellent resource the company can use for many years.
CREW
debrief
CREW
impact
CREW coaching'
CREW assess
wellbe.;ng
F:"~"";'
,-'
,
." ,'"
I
;::
PAX disruption
PAX diffuse
PAXcalm
f.:.:.:~
PAX wellbe;ng
~
f
.-~
~:~
::~~~:~
~t~:
~~::
f,:::t;:;:::.J.
r:+~--::
~~~i.~;:~:~~
~l:::'E:d~=:~
SA time,
political
~~:=-~"~:.',m'-~.
.,~:.
. ~_ ·I
;i~~~~~el
r--'''' ",.".,..-=,~=.".~-.".....,.,..=~
..
!
_.="'"'~'-~'=====~
:~~~
~~~o~~:~k
r~::~':
':,:~.:.,:::::.~.-~,.._-=.J
INFO
INFO
INFO
INFO
analyse
common
gather
resources
.~
I
"".,.~"""'.""'""
o
20
40
60
80
100
120
140
frequency
Figure 3 Frequency
count of selected expected safety behaviours
The ESBs are currently being used in cabin crew training as part of the annual
Procedures refresher training (a two-day course that all cabin
CRM/Emergency
Cabin crew expected safety behaviours 163
crew undergo annually). During a classroom session, cabin crew listened to three
of the original incident summaries that had been recorded onto audio file. Each
was a few minutes long, and was recorded in the first person, as if the voice on the
recording was that of the CSM recalling his/her own safety event. At the end of
each incident summary, the cabin crew were asked to assess a specific category of
CRM from the ESB list (e.g., 'assess the passenger management competencies').
A mock-up CRM assessment form was used along with the ESBs in table I. The
purpose of this exercise was to introduce cabin crew to the concept of CRM
assessment, and to give them some experience of the forms and categories that
may be used to train and assess them in the future. Unlike pilots, CRM assessment
and CRM competencies are a new concept for cabin crew.
Currently the ESBs are not being used for assessment purposes in the cabin.
Although, this situation will change in the next few years. It is expected that
initially, the ESBs will be used for training and checking, but not necessarily for
jeopardy assessment. This type of assessment will not occur until a level of
acceptance amongst crew and checker/trainer skill has been attained in assessing
CRM competencies.
Development of ESB application guidelines
Broad guidelines for the appropriate application and use of ESBs and their
assessment are being developed across the Qantas group (i.e., international,
domestic and regional operations) for both cockpit and cabin crew. These
guidelines are based on the NOTECHS guidelines (van Avermaete and Krujisen,
1998) and have been adapted to suit the operational and regulatory requirements
faced by Qantas. The guidelines have been developed in conjunction with
stakeholders (mainly Human Factors representatives) from the various flight and
cabin operations areas of the Qantas group. The guidelines describe such issues as;
1. Introduction and definitions of ESB (e.g., categories, elements, behaviours).
2. Philosophy behind the use of the ESBs (e.g., 'to train, reinforce and support
the skills and behaviours considered necessary to provide safe, effective
outcomes').
3. Preferred methods of assessment (e.g., 'Ratings should be done at the
'category' level. An overall rating is given for the category, rather than for
individual components, but that category rating is based upon the
performance outcomes of the elementslbehaviours. ').
4. Process for failed competencies (e.g., Ineffective behaviour should be
observed repeatedly to conclude that it is ineffective. It is not the goal to fail
someone for failing to discuss a decision on a single occasion ... ').
5. Recommended evaluator training and qualifications (e.g., 'Complete initial
training on ESB system, Formal assessment as competent and calibrated
following ESB training in classroom,
Calibration
in operational
environment, and Periodic re-calibration').
164 Peter Simpson, Christina Owens and Graham Edkins
Cabin crew LOSA
While behavioural marker systems can provide a focus for training goals and
needs, they are limited in that they cannot capture every aspect of normal
performance because of the infrequent occurrence of some behaviours (e.g.,
sporadic behaviours such as aggression), especially during an assessment
situation, such as a check flight. Hence, the University of Texas has developed a
method of normal operations, non-jeopardy, Line Operations Safety Audits
(LOSA) in the cockpit. LOSA enable the collection of data on threats and errors
and their management (Helmreich, Klinect and Wilhelm, in press).
LOSA utilises trained observers to collect data about flight crew behaviour on
normal flights under non-jeopardy conditions. Observers record information based
on three worksheets. The first worksheet records a description of external threats
(e.g., weather or ATC) that may influence crew performance and how these
threats are managed. The second worksheet describes the errors made by the flight
crew and what strategies were used to detect and recover from these errors. The
final worksheet records the behavioural markers based on three categories;
Planning, Execution and Review/Modify Plans. The behavioural markers are used
repeatedly for every flight phase and rated on a four-point scale (poor outstanding) (Helmreich, Klinect and Wilhelm, in press).
LOSA is currently used only in cockpit situations. In the longer term, the cabin
crew ESBs can be used in a similar manner to the way in which they are used for
cockpit crew; the evaluation of CRM and non-technical skills in training and
normal operations. This could feedback into the training system, allowing the
refinement and development of ESBs and the training program. The greatest
challenge is moving the evaluation and observation of safety behaviours out of the
training environment and into normal line operations, in the form of a cabin crew
LOSA program. There are many issues and problems to overcome before in-flight
cabin observations can take place. Such problems include:
•
Cabin environment is not as contained as a cockpit.
•
Double deck aircraft (B747 and A380).
• All information goes through CSM.
•
Observers are more obtrusive in cabin.
•
Errors tend to be less consequential in the cabin.
•
Impact on customers and service.
•
Multiple crew to observe, who cannot all be observed at once.
• Less external threats to inflight safety in the cabin.
Check and training for technical skills has been an accepted practice for
decades. Further, CRM and non-technical skills audit and evaluation is gaining
acceptance in the cockpit, and a cockpit LOSA program is running at Qantas. As
yet, no airline has committed to a LOSA-style program for cabin crew. Qantas is
considering incorporating LOSA style threat and error management within its
Cabin crew expected safety behaviours 165
inflight cabin observation program, which is currently being developed and due
for implementation mid-2004. The observation protocol has been developed in
line with general LOSA principles, as far as possible taking into account the issues
above. The protocol is currently under review, and the final version should be
available January 2004. To our knowledge, the proposed Qantas project is the first
attempt to apply this program within the cabin environment.
Summary
As far as is known, the Qantas Cabin Crew ESB study is the first attempt to
develop behavioural markers in aviation using a robust, scientific method. The
CDM was a practical method for this purpose, as has been demonstrated in other
naturalistic environments. It is hoped that other authors will use similar, formal
methods for developing behavioural markers, rather than using the traditional,
informal process of workshops and round-table discussions. The behavioural
markers developed from these informal processes are based on experience,
opinion and perception, and as this current study indicated, the behaviours that
experienced operators perceive to be important to safety may not reflect
operational reality. Hence, any of the current lists of safety behaviours and
behavioural markers being used in aviation and other industries require scientific
validation.
The study has also provided a useful and extensive database of positiveoutcome safety scenarios for use in cabin crew training. The case studies are
specific to the operating environments of Qantas, and are a good resource for the
current CRM focus of threat and error management. For these reasons, it is highly
recommended that other airlines implement a similar program.
The LOSA program has proven successful in the cockpit, and its use in the
cabin and other environments (e.g., ATC) appears to be a good concept in theory,
yet logistical issues and other challenges continue to impede its implementation.
No conclusion can be drawn from this current study regarding the use of the cabin
crew LOSA. However, some form of non-jeopardy, normal operations observation
is required to further understand the nature of threat and error management, and
effective safety behaviours in the cabin. Implementing such a program is now a
major challenge facing the aviation industry.
References
van Amermaete, J. and Krujisen, E. (Eds) (1998). NOTECHS: The evaluation of
non-technical skills ofmulti-pilot aircrew in relation to JAR-FCL requirements.
National Aerospace Laboratory NLR.
166 Peter Simpson. Christina Owens and Graham Edkins
Endsley, M. (1995). Towards a theory of SA in dynamic systems. Human Factors,
37, 32-64.
Flanagan, J. (1954). The critical incident technique. Psychological Bulletin, 51,
327-358.
Flin, R. and Martin, L. (2001). Behavioural markers for Crew Resource
Management: A survey of current practice. International Journal of Aviation
Psychology, 11,95-118.
Flin, R., Martin, L., Goeters, K., Hormann, H., Amalberti, R., Valot, C. and
Nijhuis, H. (2003). Development of the NOTECHS (non-technical skills) for
assessing pilots' CRM skills. Human Factors and Aerospace Safety, 3,97-119.
Hamm, R. (2004). Protocol analysis: The emperor needs a shave. Medical
Decision Making, 24, 681-686.
Helmreich, R., Klinect, J. and Wilhelm, J. (in press). System safety and threat and
error management: The Line Operations Safety Audit (LOSA). In, Proceedings
of the Eleventh International Symposium in Aviation Psychology. Columbus,
OH: The Ohio State University.
Hoffman, R., Crandall, B. and Shadbolt, N. (1998). Use of the CDM to elicit
expert knowledge: A case study in the methodology of cognitive task analysis.
Human Factors, 40,254-276.
Hoffman, R., Shadbolt, N., Burton, A. and Klein, G. (1995). Eliciting knowledge
from experts: A methodological analysis. Organisational Behaviour and
Human Decision Processes. 62,129-158.
International Civil Aviation Organisation (ICAO). (2002). Draft Human factors
digest: Humanfactors in cabin safety. Montreal,9anada.
Klampfer, et al. (2001). Behavioural markers workshop. Swissair Training Centre,
Zurich, 5-6 July.
Klein, G., Calderwood, R. and McGregor, D. (1989). Critical decision method for
eliciting knowledge. IEEE Transactions on Systems, Man and Cybernetics, 19,
462-467.
O'Hare, D., Wiggins, M., Batt, R. and Morrison, D. (1994) Cognitive failure
analysis for aircraft accident investigation. Ergonomics. 37, 1855-1869.
Qantas (2002). Safety analysis paper: Cabin safety incidents - causal factors (SAP
06/1). Sydney, Australia: Qantas, Corporate Safety Dept.
Simpson, P. (2001). Naturalistic decision making in aviation environments.
DSTO-GD-0279. Melbourne, Australia: DSTO, AMRL.
Taynor, J., Crandall, B. and Wiggins, S. (1987). The reliability of the critical
decision method. KA-TR-863(b)-8707F. Yellow Springs, OH: Klein Associates.
Cabin crew expected safety behaviours 167
Appendix A
A list of safety behaviours and non-technical skills could be very lengthy.
Developing the list is a compromise between having a practical, useable list for
operational purposes (training, assessing, checking, auditing, etc) and having a
long, exhaustive list to cover every behaviour and situation. The goal was to
account for the majority of safety-related behaviours, not every individual
behaviour.
Despite the body of literature on verbal protocol analysis and CDM, there is
very little information explaining the practical aspects of coding the transcripts.
The procedure developed for this project used a team of two psychologists, a
human factors specialist and a cabin safety specialist to develop the ESB codes.
The process was:
1. Ten interviews were selected that covered ten different situations.
2. All team members read each transcript, focussing on the decision points.
The actions and behaviours surrounding the decisions were noted, and an
attempt made to attach a recognised human factors / psychological term to
describe the behaviour.
3. After each transcript was worked on in isolation, the team met to discuss
their ideas and findings. As a group, they developed relevant descriptors for
the behavioural elements.
4. This process of coding in isolation then meeting together as a team occurred
for each interview transcript until the initial 10 transcripts were complete.
5. 'Diminishing returns' occurred, where the initial interviews yielded the
largest array of behaviours and elements, and by the tenth transcript, the
codes were being refined rather than new behaviours being uncovered.