Toulouse A-SMGCS verification and validation results

Transcription

Contract No. TREN/04/FP6AE/SI2.374991/503192
Toulouse A-SMGCS verification and validation
results
MONGENIE Olivier & PAUL Stéphane
DSNA & Thales ATM
Document No:
Version No.
Classification:
Number of pages:
D6.4.1
1.0
Public
115
Project Funded by
European Commission, DG TREN
The Sixth Framework Programme
Strengthening the competitiveness
Contract No. TREN/04/FP6AE/SI2.374991/503192
Project Manager
M. Röder
Deutsches Zentrum für Luft und Raumfahrt
Lilienthalplatz 7, D-38108 Braunschweig, Germany
Phone: +49 (0) 531 295 3026, Fax: +49 (0) 531 295 2180
email: [email protected]
Web page: http://www.dlr.de/emma
© 2007, - All rights reserved - EMMA Project Partners
The reproduction, distribution and utilization of this document as well as the communication of its contents to other without explicit
authorization is prohibited. This document and the information contained herein is the property of Deutsches Zentrum für Luft- und
Raumfahrt and the EMMA project partners. Offenders will be held liable for the payment of damages. All rights reserved in the event of the
grant of a patent, utility model or design.
The results and findings described in this document have been elaborated under a contract awarded by the European Commission.
EMMA
Distribution list
Member Type
No.
Web
Contractor
Sub-Contractor
1
Name
POC
Internet
http://www.dlr.de/emma
Intranet
1
DLR
https://extsites.dlr.de/fl/emma
Joern Jakobi
2
AENA
Mario Parra Martínez
3
AIF
Marianne Moller
4
SELEX
Giuliano D'Auria
5
ANS_CR
Miroslav Tykal
6
BAES
Stephen Broatch
7
STAR
Max Koerte
8
DSNA
Thierry Laurent
9
ENAV
Antonio Nuzzo
10
NLR
Luc de Nijs
11
PAS
Alan Gilbert
12
TATM
Stéphane Paul
13
THAV
Alain Tabard
14
AHA
David Gleave
15
AUEB
Konstantinos G. Zografos
16
CSL
Libor Kurzweil
17
DAV
Rolf Schroeder
18
DFS
Klaus-Ruediger Täglich
19
EEC
Stéphane Dubuisson
20
ERA
Jan Hrabanek
21
ETG
Thomas Wittig
22
MD
Phil Mccarthy
23
SICTA
Claudio Vaccaro
24
TUD
Christoph Vernaleken
CSA
Karel Muendel
Distributed1
N.N.
Customer
EC
Morten Jensen
Additional
EUROCONTROL
Paul Adamson
Please insert an X, when the PoC of a company receives this document. Do not use the date of issue!
Save date: 2007-03-08
File Name: D641_Results-TLS_V1.0.doc
Public
Page 2
Version: 1.0
EMMA
Document control sheet
Project Manager
Responsible Authors
Additional Authors
Michael Roeder
MONGENIE Olivier, PAUL Stéphane
SAINI Luca
MARCOU Nicolas
MONTEBELLO Philippe
Subject / Title of Document:
Related Task('s):
Deliverable No.
Save Date of File:
Document Version:
Reference / File Name
Number of Pages
Dissemination Level
Target Date
WP6.4
D6.4.1
2007-03-06
1.0
D641_Results-TLS_V1.0.doc
115
Public
2006-01-31
DSNA, TATM
TATM
DSNA
DSNA
Change control list
Date
Release
Changed Items/Chapters
2005-06-23
2005-09-02
0.01
0.02
§2.3.2
2005-09-12
0.03
§2.3.2 and §2.3.3
2005-12-07
2006-01-16
0.04
0.05
New §2.4
New §2.5
2006-01-20
2006-03-01
0.06
0.07
New §2.5.2
New §2.6
2006-03-31
-
2006-04-04
2006-04-14
0.08
-
2006-04-21
-
Parallel D641 document created by
DSNA for validation results.
New §2.7
The TATM editorial comments to the
parallel D641 document created by
DSNA, release v0.01, are
implemented.
The results of the shadow-mode trials
are completed.
A fast-time simulations chapter is
added.
Editorial and content improvement.
Save date: 2007-03-08
Public
Comment
Initial draft.
Comments by Holger Neufeldt
and by Marianne Moller.
Update of Airbus analysis by
Marianne Moller.
Contribution by Holger Neufeldt.
Contribution by Luca Saini and
Paolo Gervasoni.
Answer of FAA on RTCA issue.
Temporary internal results, to be
updated.
Initial draft validation results by
DSNA. Release v0.01 is delivered
to TATM for review.
Time stamp issue.
By DSNA. Release v0.02 is
delivered to TATM for review.
By DSNA. Release v0.03 is
Page 3
Version: 1.0
EMMA
2006-04-27
0.09
2006-05-09
0.10
2006-12-07
0.11
2007-02-26
0.12
2007-03-06
0.13
2007-03-08
1.0
delivered to DLR for review and
to TATM for integration of the
verification results.
Submission to DSNA and Luca
Integrated parallel D641 document
created by DSNA for validation results Saini for review.
in its release v0.03.
Transfer of the analysis tables for the
shadow-mode validation trials in an
annex.
Editorial improvement.
Check of acronym list.
Suppression of figure 15.
New verification tests, re-organisation
of chapters and consolidation with
V&V test plan.
Small comment about MOGADOR.
Submission to DLR for general
Editorial improvements.
assembly review and/or
submission to EC.
Update of all the document by DSNA Submission by DSNA to TATM.
following EC review.
Update of all the document by Luca
Saini following EC review.
Processing of remaining comments
from EC by Philippe Montebello and
Stéphane Paul.
Approval by EC.
Save date: 2007-03-08
Public
Page 4
Version: 1.0
EMMA
Table of contents
Distribution list........................................................................................................................................ 2
Document control sheet........................................................................................................................... 3
Change control list .................................................................................................................................. 3
Table of contents ..................................................................................................................................... 5
1 Scope .................................................................................................................................................... 7
1.1 Identification ................................................................................................................................. 7
1.2 Project overview............................................................................................................................ 7
1.3 Document overview ...................................................................................................................... 7
1.3.1 Purpose ................................................................................................................................... 7
1.3.2 Intended audience................................................................................................................... 8
1.3.3 Document structure ................................................................................................................ 8
1.4 Relationships to other EMMA documents .................................................................................... 8
2 Verification........................................................................................................................................... 9
2.1 Mosquito position precision .......................................................................................................... 9
2.1.1 System configuration.............................................................................................................. 9
2.1.2 Results .................................................................................................................................... 9
2.1.3 On-the-fly analysis ................................................................................................................. 9
2.1.4 Conclusion............................................................................................................................ 10
2.1.5 RTCA DO-260A vol-1 bug on ADS-B surface position reporting ...................................... 10
2.2 ADS-B ground station coverage.................................................................................................. 11
2.2.1 Results .................................................................................................................................. 12
2.2.2 On-the-fly analysis ............................................................................................................... 12
2.3 Unstable track positions for some aircraft................................................................................... 12
2.3.1 Results .................................................................................................................................. 12
2.3.3 Conclusion............................................................................................................................ 14
2.4 MAGS preliminary verification results ....................................................................................... 14
2.4.1 System configuration............................................................................................................ 14
2.4.2 Result of step (e) – system calibration ................................................................................. 15
2.4.3 Result of step (f) – localisation of test transmitter................................................................ 16
2.4.4 Results of step (g) – localisation of real targets ................................................................... 16
2.5 Surveillance latency and position precision ................................................................................ 18
2.5.1 First test: latency & position verification with a vehicle...................................................... 18
2.5.2 Second test: latency verification with an aircraft ................................................................. 20
2.5.3 Third test: dispersion verification......................................................................................... 21
2.5.4 On-the-fly analysis of the 3 tests .......................................................................................... 22
2.5.5 Conclusion............................................................................................................................ 22
2.6 ADS-B stations synchronisation: time drift test .......................................................................... 23
2.6.1 The test set-up ...................................................................................................................... 23
2.6.3 Detailed analysis and issue solving ...................................................................................... 24
2.6.4 Conclusion and more (interesting) results ............................................................................ 24
2.7 ADS-B latency test...................................................................................................................... 24
2.8 Test of the probability of detection of a stationary vehicle ......................................................... 24
2.8.2 Findings ................................................................................................................................ 25
2.9 ADS-B cost-benefit test............................................................................................................... 25
2.9.2 Findings ................................................................................................................................ 26
2.10 Consolidation of verification results ......................................................................................... 26
2.10.1 Surveillance ........................................................................................................................ 26
2.10.2 Control................................................................................................................................ 27
Save date: 2007-03-08
Public
Page 5
Version: 1.0
EMMA
2.10.3 Human machine interface................................................................................................... 27
3 Validation session – shadow mode trials............................................................................................ 28
3.1 Introduction ................................................................................................................................. 28
3.2 Data description and data collection methods ............................................................................. 28
3.2.1 Raw data ............................................................................................................................... 28
3.2.2 Additional data ..................................................................................................................... 29
3.3 Data analysis................................................................................................................................ 30
3.3.1 Data analysis method............................................................................................................ 30
3.3.2 Analysis tables for the shadow-mode validation trials......................................................... 31
3.4 Results ......................................................................................................................................... 31
3.4.1 Operational feasibility: acceptance of technical performances ............................................ 31
3.4.2 Operational feasibility: procedures....................................................................................... 43
3.4.3 Operational improvements ................................................................................................... 46
4 Validation session – fast-time simulations ......................................................................................... 52
4.1 Introduction ................................................................................................................................. 52
4.2 Data description and data collection methods ............................................................................. 52
4.2.1 Raw data ............................................................................................................................... 52
4.2.2 Additional data ..................................................................................................................... 53
4.3 Data analysis................................................................................................................................ 54
4.3.1 Data analysis method............................................................................................................ 54
4.3.2 Measured indicators ............................................................................................................. 56
4.4 Results ......................................................................................................................................... 60
4.4.1 Operational improvements ................................................................................................... 60
5 Digest and conclusions....................................................................................................................... 68
6 Annexes.............................................................................................................................................. 69
6.1 Annex A: analysis tables for the shadow-mode validation trials ................................................ 69
6.1.1 Operational feasibility: acceptance of technical performances ............................................ 70
6.1.2 Operational feasibility: procedures....................................................................................... 91
6.1.3 Operational improvements ................................................................................................. 110
6.2 Annex B: references .................................................................................................................. 113
6.2.1 Applicable documents ........................................................................................................ 113
6.2.2 Referenced documents ....................................................................................................... 113
6.3 Annex C: abbreviations ............................................................................................................. 113
Save date: 2007-03-08
Public
Page 6
Version: 1.0
EMMA
1 Scope
1.1 Identification
This document provides the Toulouse A-SMGCS verification and validation results.
• Document Name:
• EMMA No.:
D6.4.1
• Revision:
0.13
• File Name:
d641v013.doc
1.2 Project overview
The project is named “European Airport Movement Management by A-SMGCS” with the acronym
EMMA. The duration of the project is 2 years, with a follow-up in EMMA-2 (another 2 years).
The project is organised in six different sub-projects. There are three ground-related sub-projects and
one onboard-related sub-project. Based on an advanced operational concept, three functional level III
advanced surface movement, guidance and control systems (A-SMGCS) will be implemented at three
European airports: Prague-Ruzynĕ, Toulouse-Blagnac and Milano-Malpensa. The systems are to be
tested operationally (i.e. with live traffic). The three ground-related sub-projects and the onboardrelated sub-project are autonomous, but are inter-linked with the sub-projects ‘concept’ and
‘validation’ to guarantee that the different systems are based on a common A-SMGCS interoperable
air-ground co-operation concept and that all are validated with the same criteria.
On-site long-term trials (i.e. 6 months to one year) are to ensure the assessment of benefit estimations.
The results of the test phase shall feed back to the concept of operations, and are intended to set
standards for future implementation in terms of: (a) common operational procedures, (b) common
technical and operational system performance, (c) common safety requirements, and (d) common
standards of interoperability with other ATM systems. These standards shall feed the relevant
documents of international organisations involved in the specification of A-SMGCS, i.e. mainly
ICAO, EUROCAE, and EUROCONTROL.
This document is produced in the scope of work package 6.4 entitled "Verification and Validation at
Toulouse". The activities performed in this work package 6.4 include:
• preparation of the infrastructure at Toulouse-Blagnac airport for verification and validation;
• technical tests (i.e. verification) of the A-SMGCS installed in Toulouse-Blagnac;
• training of Toulouse-Blagnac controllers to the A-SMGCS;
• shadow-mode trials with trained controllers on the A-SMGCS HMI installed in the ToulouseBlagnac tower to evaluate the operational feasibility of the A-SMGCS and assess the operational
improvements brought by the system;
• fast-time simulations of several traffic and environment conditions in Toulouse-Blagnac to
quantify the operational improvements potentially brought by the A-SMGCS.
1.3 Document overview
1.3.1 Purpose
The ICAO manual [15] on Advanced Surface Movement Guidance and Control Systems (A-SMGCS)
and the EUROCAE WG-41 MASPS [16] for A-SMGCS contain operational and performance
requirements that are considered to be necessary in the process of selection, development and
introduction of A-SMGCS. These manuals have been defined for those aerodromes where current
SMGCS needs to be upgraded, or for aerodromes which currently have no SMGCS, but where the
traffic density and/or aerodrome layout requires so.
The objective of this document is to report on the collected data aimed at supporting the assessment of
the surveillance, surface conflict alerting and routing functions as implemented in Toulouse-Blagnac
Save date: 2007-03-08
Public
Page 7
Version: 1.0
EMMA
against the aforementioned ICAO and EUROCAE requirements, in conformance to [3], [4], [7] and
[14].
Due to some delays in the implementation of the Toulouse-Blagnac A-SMGCS test bed (cf. [2]) it was
not possible to run all the tests foreseen in [5]. The corresponding implications for the overall project
plan, as well as the plans foreseen by the project, in general, to cope with such a non-conformance are
described within each test description. However, many tests were run, and some significant results
were obtained, as documented within.
1.3.2 Intended audience
The dissemination of this document is public.
1.3.3 Document structure
Chapter 1 defines the scope of this document, the intended audience, and the document structure.
Chapter 2 presents the verification results. Verification is testing against predefined technical
specifications, i.e. technical functional testing: "did we build the system right?" Each section of this
chapter presents a short description of the experiment set-up, an overview of the raw data that has been
gathered, and different stages of analysis by the different partners involved.
Chapter 3 and 4 present the validation results obtained respectively through shadow-mode trials and
fast-time simulations. Validation is testing against operational requirements as defined by stakeholders
and written down in [6], i.e. "did we build the right system?" The chapters describe the data, data
collection methods, data analysis and results measured in terms of operational feasibility and
operational improvements.
Chapter 5 provides the conclusions drawn from the V&V activities in Toulouse-Blagnac airport.
Chapter 6 provides details on referenced documents and some definitions.
1.4 Relationships to other EMMA documents
Apart from the EMMA documents already referenced above, the author would like to bring the
reader's attention to:
• similar result collection reports for Prague-Ruzynĕ [8] and Milano-Malpensa airports [9],
• a similar result collection report for airborne systems [10],
• the verification and validation analysis report [11],
• the verification and validation recommendation report [12].
Public EMMA documents can be found on www.dlr.de/emma.
Save date: 2007-03-08
Public
Page 8
Version: 1.0
EMMA
2 Verification
Due to delays in the A-SMGCS implementation at Toulouse-Blagnac, it was not possible to
thoroughly follow the V&V test plan, as documented in D6.1.3 [5]. However, some of the verification
tests provided below in §2.1 to §2.9 and the site acceptance tests performed for each of the A-SMGCS
sub-system elements (cf. D4.1.1, D4.12, D4.1.3, D4.2.1, D4.3.1, D4.4.1 and D4.4.2) allow us to take a
stand on a number of technical hypotheses, as described below in §2.10, in particular where the
surveillance, control and human-machine interface hypotheses are concerned.
The D6.1.3 Toulouse-Blagnac V&V Test Plan [5] planned that the MOGADOR tool would be used in
order to assess detection and identification indicators on a long-term basis from EMMA A-SMGCS
recorded data.
The recorded EMMA A-SMGCS SDF data (i.e. one week sample recorded in February 2006) was
input into MOGADOR. However, the results obtained were abnormally bad and a very low number of
tracks could be actually analysed.
This is explained by the important number of lack of identifier and the poor continuity of the
correlation function (i.e. MOGADOR uses the flight call sign as a key element to associate and
identify tracks) and the need for tuning of MOGADOR to Toulouse-Blagnac platform that would
necessitate additional time and effort.
Consequently, since the results of the MOGADOR analysis do not reflect the reality, it was decided
not to publish them in this document.
The impossibility to perform a long-term analysis has a significant impact on the verification phase. In
particular, the detection and identification performance of EMMA A-SMGCS will not be fully verified
in Toulouse-Blagnac.
Therefore, an additional campaign of measures will have to be performed as a preliminary to EMMA2
validation activity.
2.1 Mosquito position precision
In June 2005, Laurent Volkmann (DSNA) calculated the A-SMGCS reported position accuracy
(RPA), by comparing the Mosquito reported position to the D-GPS reported position.
2.1.1 System configuration
For the test, the following set-up was prepared:
• full automatic dependant surveillance broadcast (ADS-B) system installed, up and running;
• ADS-B ground station (GS) configured to emit ASTERIX reports as soon as they receive 1090ES
message updates from ADS-B targets;
• Mosquito installed on a vehicle, up and running;
• the same vehicle mounted with a D-GPS independent receiver, used as a reference;
• ELVIRA recording system connected to the system LAN.
2.1.2 Results
When both Mosquito and D-GPS track trajectories are superimposed on the ELVIRA display (cf.
Figure 1), the reported position accuracy (RPA) appeared to be very good, but the calculation gave bad
results because of the ADS-B GS time stamp. See picture below.
2.1.3 On-the-fly analysis
Fact 1: The D-GPS has an update rate of 5 per second. The GPS inside Mosquito has an update rate of
1 per second. When the mobile is moving, it is assumed that the position sent by the GPS inside
Mosquito is an average position over the last second. This introduces some latency.
Save date: 2007-03-08
Public
Page 9
Version: 1.0
EMMA
Fact 2: Mosquito does not time stamp as no time stamp can be sent in ADS-B position messages. The
first time stamp is performed during the ADS-B ASTERIX message formatting by the ADS-B ground
station and reflects the ASTERIX message emission date.
Fact 3: During the tests and in accordance with the ADS-B standard, each Mosquito/GPS position data
was sent twice per second (with 0.5s interval), whilst the odd and even coding/decoding of the position
data differed. This explains on Figure 1: Mosquito RPA on ELVIRA display the presentation of
Mosquito positions as tight-couples (i.e. in fact the same data) separated by 0.5 seconds.
Fact 4: In June 2005, the Mosquito equipment did not perform any position extrapolation to
compensate the latency.
2.1.4 Conclusion
Thales ATM implemented a position prediction algorithm in order to extrapolate the 1st message
position and the 2nd message position by different factors. The position prediction algorithm was
derived from the ADS-B standard (RTCA-260A).
Figure 1: Mosquito RPA on ELVIRA display
2.1.5 RTCA DO-260A vol-1 bug on ADS-B surface position reporting
Thales ATM implemented inside the Mosquito vehicle ADS-B transmitter an algorithm suggested by
RTCA D0-260A for GPS position prediction by ADS-B transmitting mobiles. This algorithm aims at
compensating some latency effects that can affect the reported position accuracy. During the
implementation activities, Luca Saini and Paolo Gervasoni identified a bug on a formula in the RTCA
D0-260A volume 1 document. The bug details are reported below.
Document reference: Minimum operational performance standards for 1090 MHz extended squitter,
automatic dependant surveillance broadcast (ADS-B) and traffic information service - broadcast (TISB), RTCA DO-260A volume 1, 10th April 2003, Section: 2.2.3.2.4: ADS-B Surface Position Message,
Save date: 2007-03-08
Public
Page 10
Version: 1.0
EMMA
Sub-sections: 2.2.3.2.4.8.2 and 2.2.3.2.4.8.3 Surface Longitude Position, Extrapolation/Estimation,
precision and non-precision case.
In the above mentioned sections there is twice reported a prediction formula of a longitude value in
order to compensate for the time latency between the time of last received update position fix and the
time of the applicability of the ADS-B surface position message.
In such a formula the variable called "approximate longitude" is defined as the "longitude at the time
of the fix". The mistake is found in that this must be the latitude at the time of the fix and not the
longitude.
Note that the corresponding formulas, as reported in the sections for the airborne position messages
longitude extrapolation/estimation are correctly typed.
Two simple examples can describe some effects of the mistake.
Case 1: a target moving along the equator in the E-W direction at a constant velocity. In such a case
it's expected to obtain a constant longitude predicted delta for the same delta time. Applying the
formula would result in delta longitude predicted values that strongly depend on the target longitude
position. In world areas close to plus or minus 90° this delta becomes… infinite!
Case 2: a target is moving in world areas with longitude > + 90° or < - 90°. In such a case the formula
is predicting a negative deltas of longitude (i.e. longitude decreasing) when the target is moving
towards east (longitude increasing) and vice-versa.
RTCA and EUROCAE were contacted to communicate the finding.
The following message was received from FAA, acknowledging our finding.
-----Original Message----From: [email protected] [mailto:[email protected]]
Sent: Thursday, January 19, 2006 6:11 PM
To: […]
Subject: Re: DO-260A bug on ADS-B Surface Position Reporting
Gentlemen,
After a review of the problem that was initially reported below, we agree that there is a typographical
error in the text of RTCA/DO-260A in the two places indicated for Surface Position reporting in the
"Commentary" sections of 2.2.3.2.4.8.2.1 and 2.2.3.2.4.8.3.1, where longitude is indicated in the
definition of the symbol "Phi" when in fact the latitude should have been indicated. I deeply regret that
these errors were entered into the DO-260A document, and cannot understand how a simple cut-andpaste from DO-260 to DO-260A could have occurred, but we are pleased that the error has been
found and can be easily corrected.
To the end of making the correction to these, and other errors identified in DO-260A, I have added the
corrected text for these two errors into the proposed "Change 1 to DO-260A" document that was
initially reviewed during Meeting #19 of the RTCA SC-186 Working Group 3 (WG3) in December
2005 as Working Paper 1090-WP19-06. After discussions of how the proposed changes to both DO260 and DO-260A might effect the 1090ES SARPs Technical Manual, during the ICAO SCRSP TSG
meeting in Fort Lauderdale 2 - 10 February 2006, WG3 will finalize and submit "Change 1"
documents for both DO-260 and DO-260A to RTCA SC-186 membership for their Final Review and
Comment period, which ends with the SC-186 Plenary meeting in Washington on 20 April, where
WG3 expects that both "Change 1" documents will be approved for publication on the RTCA web site.
[…]
Gary Furr
L-3 Communications / The Titan Group
FAA Technical Center
2.2 ADS-B ground station coverage
In June 2005, Luca Saini performed some ADS-B ground station output recordings. At that time, the
equipment configuration was as follows: two ADS-B ground stations, one at the GBAS location and
one at the top of the new tower.
Save date: 2007-03-08
Public
Page 11
Version: 1.0
EMMA
2.2.1 Results
The main results were as follows:
• Both ADS-B ground stations showed good surveillance coverage relating to aircraft;
• The new tower ADS-B ground station showed good performance, with a few systematic holes
(probably due to reflections), concerning vehicle tracking with Mosquito;
• The GBAS ADS-B ground station showed very poor coverage performance concerning vehicle
tracking with Mosquito.
The poor performance of the GBAS ground station relating to vehicle tracking is difficult to
understand. Indeed the GBAS ground station is more powerful and has an additional low noise
amplifier compared to the new tower ADS-B ground station. It is currently thought that the low height
of the antenna makes it more subject to reflection problems.
2.3 Unstable track positions for some aircraft
In June 2005, Luca Saini performed some ADS-B ground station output recordings. At the time, the
equipment configuration was as follows: two ADS-B ground stations, one at the GBAS location and
one at the top of the new tower.
2.3.1 Results
A surprising result was the unstable track positions reported for some aircraft, mainly Air France
aircraft, whilst the EMMA Mosquito track showed very stable trajectories, as well as some other
aircraft (EasyJet, Airbus A380, etc.) – see figures below.
Figure 2: Mosquito (stable) vehicle tracking
Save date: 2007-03-08
Public
Page 12
Version: 1.0
EMMA
Figure 3: Unstable aircraft tracks during pushback
Figure 4: Unstable aircraft tracks during taxi
For aircraft with unstable target reports, there seemed to be a systematic jump between odd and even
target position reports. This is thought to be related to the encoding of the positions, using different
algorithms for odd and even reports. But why is this phenomenon related only to some aircraft?
2.3.2.1 Comments by Thales ATM
Regarding the accuracy of aircraft, it is to be noted that within the ADS-B ASTERIX cat 21 message,
the ground stations forward a Figure of Merit / Position Accuracy indicator (FOM/PA). The higher
this number, the better the accuracy. A FOM/PA of 7 or above is good, 6 or less is poor. The reason
Save date: 2007-03-08
Public
Page 13
Version: 1.0
EMMA
for poor quality is usually the current GPS reception quality or constellation as perceived by the
onboard GPS receiver. Some older types do not deliver adequate position quality.
It is also to be noted that there is a number of other misbehaving equipment sets flying around
(including sudden jumps backwards along track, jumps to the north pole or another distant position
etc.) These are known issues in some manufacturers equipment that are caused by, for example:
• onboard equipment that – whilst outputting ADS-B formats – is not certified for ADS-B use,
• a problem in the internal time synchronisation function of the transponder.
When analysing ADS-B reports one must therefore be extremely careful in describing what actually
happened. ADS-B is just a data link mechanism. The positions are measured by avionics and the ADSB receiver depends on this information (hence the name "dependant surveillance").
2.3.2.2 Comments by Airbus
The above results were provided to Airbus, who gave the following preliminary analysis. Air France
has not yet fully equipped all its airliners with the package enabling GPS data transmission via ADSB. In such a case, the resolution of the position transmitted via ADS-B can lead to jumps around the
aircraft trajectory. The observed jumps should progressively disappear2 whilst Air France completes
the installation.
2.3.3 Conclusion
Thales ATM decided to suppress all ADS-B tracks whose Figure of Merit / Position Accuracy
indicator (FOM/PA) was strictly lower than 7.
In addition, this result is interesting on a safety assessment point of view. The observed behaviour is
linked to an intermediate configuration in the frame of the ADS-B deployment. It would be interesting
to know if an on-board equipment failure of the final configuration might lead to a similar fallback
behaviour. If so, it might be interesting to analyse it in future releases of the EMMA functional hazard
assessment [13].
2.4 MAGS preliminary verification results
An important part of the EMMA installation in Toulouse is the MAGS multilateration. Multilateration
is a complex technology whose implementation on an actual airport requires a great deal of tuning. For
the installation of MAGS at Toulouse-Blagnac airport within the frame of the EMMA programme, the
initial tuning phase has yielded some preliminary verification results that are presented in this section.
The system installation and tuning consists of the following steps:
(a) installation of equipment;
(b) setting up of communication links;
(c) tuning of ground station sensitivity for the intended coverage area;
(d) verification of probability of detection of ground stations;
(e) configuration of calibration mechanism to establish a common time base for all ground stations;
(f) parameter tuning to locate the fixed test transmitter;
(g) parameter tuning to locate targets;
(h) definition of areas and their properties;
(i) optimisation of plot validation and tracking parameters;
(j) integration with sensor data fusion.
The results presented correspond to steps (e), (f) and (g). For this purpose – after an initial on-site
tuning session - data were recorded from the MAGS and taken to the lab. These data were then
replayed in real time to the reference system in the lab and its parameters were configured to obtain
the best possible performance. The following sections present some intermediate results.
For the MAGS system in Toulouse, a set of five ADS-B/MLAT ground stations were installed as
depicted in Figure 5. One ground station was additionally equipped with a calibrator module to
2
If it were not the case, a new analysis would be required.
Save date: 2007-03-08
Public
Page 14
Version: 1.0
EMMA
provide system synchronisation, and another ground station was additionally equipped with an
interrogator module to allow interrogation of aircraft. A fixed test transmitter based on a Mosquito
ADS-B transmitter was installed as a permanent, fixed test target.
GSR
(Old TWR)
GSC
(TWR)
GSR
(PC)
GSR
(GBAS)
1
5
4
2
6
3
Test Transmitter
(DF)
GSI
(SMR)
Figure 5: MAGS Configuration at Toulouse Blagnac
2.4.2 Result of step (e) – system calibration
Calibration results are best illustrated by locating the calibrator transmitter using multilateration. The
result is depicted in Figure 6. The filled red square indicates the ground station calibrator (GSC)
position, while the outlined red squares are 10m, 20m, and 30m, respectively, around the GSC
position.
Figure 6: Measured position of MAGS calibrator station (GSC) at control tower
The GSC position measured by multilateration is shown as sequence of target plots on the MAGS
technical display. Due to ASTERIX cat-10 quantisation, the position resolution is limited to 1m (i.e.
distance between adjacent plots). All position reports are within 3 metres of the exact GSC position.
Save date: 2007-03-08
Public
Page 15
Version: 1.0
EMMA
2.4.3 Result of step (f) – localisation of test transmitter
Figure 7 shows the resulting plot sequence of locating the fixed test transmitter (TT) by
multilateration. Again the minimum distance between adjacent plots is determined by the ASTERIX
position resolution of 1m. All position reports are within 4 metres of the exact TT position.
Figure 7: Measured position sequence of fixed test transmitter
2.4.4 Results of step (g) – localisation of real targets
The following two screenshots in Figure 8 and Figure 9 show real targets detected via multilateration
together with the GSC and the test transmitter. Track continuity was not ensured. Indeed, no tracker
was yet configured (as this is a task within the definition of area properties). Therefore tracks were
neither smoothed nor fully regular in their update rate. It should be noted that, at that time, MAGS did
not interrogate aircraft, so that multilateration relies purely on passive reception. Active interrogation
of targets can be used to enhance update rates and to fill gaps.
Save date: 2007-03-08
Public
Page 16
Version: 1.0
EMMA
Figure 8: Targets, GSC (label “3C3D0C”) and test transmitter (label “3C3D0D”) detected by MAGS
Missing detections can be caused by coverage gaps. It is clear that five ground stations in this
configuration are not ideal in terms of redundancy. For example, if one ground station (e.g. the one
installed at the SMR, cf. Figure 5) does not contribute to a plot data set, the geometrical constellation
is not good enough for parts of the coverage area.
Figure 9: Targets, GSC (label “3C3D0C”) and test transmitter (label “3C3D0D”) detected by MAGS
The next screenshot, in Figure 10, shows one target that taxies onto the apron area. It should be noted
that no tracker was yet configured to smoothen the track and to provide regular updates.
Save date: 2007-03-08
Public
Page 17
Version: 1.0
EMMA
Figure 10: One target picked out
2.5 Surveillance latency and position precision
On Thursday 23rd February 2006, a DSNA/DTI team (headed by Mr. Philippe Soleilhac) performed a
surveillance latency and reported position accuracy verification session. The test scope included the
vehicle tracking system (Mosquito), the automatic dependant surveillance broadcast sensor (AS-680)
and the sensor data fusion (SDF).
2.5.1 First test: latency & position verification with a vehicle
The verification was performed using the latest Mosquito equipment delivered week 7 of year 2006.
Tests were performed by driving a Mosquito-equipped car on service roads. The car was detected only
by the automatic dependant surveillance broadcast (ADS-B) sensor and the surface movement radar
(because the multilateration system was shut down). A very precise differential global positioning
system (D-GPS) was used as time and position reference.
Save date: 2007-03-08
Public
Page 18
Version: 1.0
EMMA
Figure 11: Mosquito latency & precision verification test at ADS-B sensor output
The comparison between the D-GPS reports and the Mosquito reports as output directly by the ADS-B
sensor located at the new control tower (cf. Figure 11) showed a 1.2s latency and a position error of 12
metres.
Save date: 2007-03-08
Public
Page 19
Version: 1.0
EMMA
Figure 12: Mosquito latency & precision verification test at sensor data fusion output
The comparison between the differential global positioning system (D-GPS) reports and the Mosquito
reports as output by the sensor data fusion (cf. Figure 12) showed a 2s latency and a position error of 6
metres.
2.5.2 Second test: latency verification with an aircraft
This test was performed with a flying aircraft. The approach secondary surveillance radar (called
DACOTA) was used as reference system.
Save date: 2007-03-08
Public
Page 20
Version: 1.0
EMMA
Figure 13: Air surveillance latency & precision verification test
The comparison between the DACOTA target reports and the EMMA target reports as output by the
ADS-B sensor located at the new control tower and the sensor data fusion showed a 0.3s latency.
2.5.3 Third test: dispersion verification
The EMMA sensor data fusion (SDF) introduced some target report dispersion, which is not
originating from any sensor.
Save date: 2007-03-08
Public
Page 21
Version: 1.0
EMMA
Figure 14: Dispersion test with flying aircraft
2.5.4 On-the-fly analysis of the 3 tests
The offset of 12 m between the ADS-B output trajectory of Mosquito and the D-GPS recording is due
to a bug inside the Mosquito software. The latency of 1.2 seconds is affected by the same error. A new
software release has been produced.
The "improvement" performed by the SDF output on the trajectory (from 12 m to 6 m) is only due to
the fact that the SMR output is much more precise than the ADS-B output for Mosquito. Fixing the
Mosquito problem should allow to reduce this offset.
The crazy behaviour of the SDF output for the flying aircraft has an explanation and is due to two
probable contributions:
1) The SDF does not fuse the ADS-B output (which is very good in this case) because the ADS-B
emitting transponder does not deliver a figure of merit better than 6 (i.e. 7 or 8 or 9).
2) The EMMA SDF, as far as the DACOTA GTW tracker (alfa, beta and gamma filter) is concerned,
is configured in such a way that aircraft entering pre-defined geometrical areas (cones) associated to
each RWY are kept with a stronger weight on the current direction, at the time the aircraft enters the
cone. This setting is performed in order, for a landing airborne aircraft, to "follow" the RWY
centreline. The case described above was a test flight entering the cone in a perpendicular direction
from what could be expected. As this setting is also creating problems in case aircraft land in
"baïonnette" mode, the tracker's filter values were changed. The new proposed values are not giving
any "preferred" direction on the tracker.
2.5.5 Conclusion
In case a test aircraft will be used for EMMA verification, it should be better that this aircraft is
delivering a good Figure of Merit, otherwise the SDF will not fuse the ADS-B output.
Save date: 2007-03-08
Public
Page 22
Version: 1.0
EMMA
2.6 ADS-B stations synchronisation: time drift test
On Thursday 9th March 2006, a DSNA/DTI team (headed by M Philippe Soleilhac) performed
timestamp tests related to Mosquito and ADS-B stations.
With Mosquito, the position is elaborated onboard the vehicle (using the GPS), but the time is stamped
by the ADS-B ground station. The ADS-B standard does not allow for the transmission of the time
together with the position.
2.6.1 The test set-up
Tests were performed by driving a D-GPS and Mosquito-equipped car on service roads. Positioning
was performed via the D-GPS, the ADS-B ground stations and the mode S multilateration.
GPS
1090ES ADS-B
Precision
GPS RX
Vehicle TX
SMR
OTWR
PCBLD
NTWR
GBAS
GPS Position & Time
ADS-B GTW
Asterix 21
multilateration
Asterix 21
ELVIRA
Asterix 62
Asterix 10
NTP Synchro
SDF
Asterix 62
SAF Clock
Figure 15: Position and timestamp test set-up
ASTERIX-21 messages are time-stamped by ELVIRA with a tenth of a second accuracy. The
following table shows timestamp discrepancies between the ADS-B ground stations.
ADS-B ground
station
New TWR
PC 2
SMR
Old TWR
Time in the ASTERIX messages stamped by
the ADS-B stations
12.34.02
12.15.40
12.14.67
12.15.26
Time stamped by ELVIRA upon
message reception
12.14.68
12.14.77
12.14.68
12.14.75
Table 2-1: Timestamp measurements
Times in the ASTERIX-21 messages, stamped by the ADS-B stations, are in advance, with a dramatic
20 seconds for the ADS-B ground station located at the new control tower. Time in the ASTERIX-62
messages, stamped by the SDF, is also in advance, but with a smaller amount (between 0.7s and 1s).
Save date: 2007-03-08
Public
Page 23
Version: 1.0
EMMA
The ADS-B ground stations are normally synchronised, between themselves and with the real time,
using NTP. It seems that the implementation of NTP in this case was not effective, with a detrimental
effect on latency compensation (at ADS-B gateway level) and position accuracy.
2.6.3 Detailed analysis and issue solving
According to Luca Saini, the poor latency performances are essentially related to three sub-systems:
• Mosquito;
• the ADS-B GS;
• the SDF (at ADS-B GTW level).
With respect to Mosquito, the main problem is due to the fact that the internal processing cycle is not
synchronised with the GPS receiver output. TATM is working on this feature. However, to solve
definitively the full latency problem, it is required to introduce additional features that at least
compensate each known delay. Specific requirements have been added to the software requirements
specification (SRS). See below for more details.
With respect to the ADS-B GS, the main problem is the synchronisation using NTP (SDF being the
master). After investigating the network time protocol (NTP) effect, it was discovered that the issue
was in fact a human error: the NTP server was not reachable on the network. The NTP service,
initially located on the ADS-B gateway PC had been transferred to the SDF PC. Thus, the NTP server
IP address had changed. The ground stations were re-configured to synchronise with the current NTP
server.
With respect to the ADS-B GTW, one additional source of latency has been identified, i.e. the ADS-B
GTW stamped anew the ASTERIX messages without making any change on the reported position.
The proposed correction consists in forwarding the original time stamp given by the ADS-B ground
station to the SDF and letting the SDF extrapolate the position at the time of SDF output as done for
any other sensor track.
2.6.4 Conclusion and more (interesting) results
Beyond the aforementioned test, which reflects human error more than system performance, one
should not discard the results so easily.
First, it is to be noted that the built-in test equipment did not make it evident, neither to the end-user
nor to technical staff, that the ground stations were not synchronised. This is a typical case of slightly
corrupted surveillance data, which unnoticed, results in a severe "misuse of surveillance data" hazard
(cf. [13]).
Secondly, the aforementioned test results provide us with the opportunity to validate another two
requirements, which were not initially in our test plan:
• When not synchronised on external time signal, the automatic dependant surveillance broadcast
(ADS-B) ground subsystem time shall have a maximum drift of 4.4 sec/day.
• When not synchronised on the time reference system, the system time shall have a maximum drift
of 20 ms per day.
2.7 ADS-B latency test
On Wednesday 5th April 2006, Holger Neufeldt, Volker Seidelmann and Bernd Doleschal tested the
internal latency of ADS-B processing under real load conditions at Toulouse-Blagnac. They found that
the total latency between the reception of a radio frequency (RF) signal and the output of the
corresponding ADS-B report on the network (in ADS-B pipeline mode) was between 6 and 35 ms.
2.8 Test of the probability of detection of a stationary vehicle
The objective of this verification test was to measure the probability of detection of a stationary
vehicle at the output of the following components:
• MLAT;
Save date: 2007-03-08
Public
Page 24
Version: 1.0
EMMA
• ADS-B GTW;
• SDF.
This test was performed in January 2006 on the fixed test transmitter located at the direction finder
shelter.
The initial equipment conditions were as follows:
• full ADS-B system installed, up and running;
• ADS-B system synchronised with the airport time reference;
• Mosquito unit, installed within the direction finder shelter, up and running;
In a stationary condition the Mosquito emits the following 1090ES messages with associated emission
rates:
• surface position: once every 5 seconds;
• identity and type: once every 10 seconds.
The ADS-B ground stations emit ASTERIX CAT 021 target reports as soon as they receive updates
from ADS-B targets. The ADS-B GTW (only for stationary targets) adapts the received input rate
from ground stations to a one second output rate (ASTERIX CAT 062) towards the SDF. The SDF
emits target reports (ASTERIX CAT 062) towards the CWP at one-second output-rate.
2.8.2 Findings
The results are provided in Table 2-2.
Toulouse ADS-B MLAT
System
Total measure time
Total detected events
Detection Probability
Unit
sec
1 sec
%
MLAT
5127
5114
99,75
X-pos average
Y-pos average
X-pos stdev
Y-pos stdev
2D average deviation
2D standard deviation
2D deviation 50%-tile
2D deviation 99.99%-tile
2D max deviation
m
m
m
m
m
m
m
m
m
m
m
-456,7
-478,3
0,85
1,25
1,30
1,51
1,31
2,81
3,78
5,74
5,76
Test trasmitter
ADS-B GTW
SDF
5141
5140
5141
5140
100,00
100,00
-479,0
-470,4
0,27
0,77
0,63
0,82
0,37
1,63
2,01
2,01
2,01
-478,5
-470,4
0,27
0,77
0,63
0,82
0,37
1,63
2,00
2,00
2,00
Table 2-2: Test transmitter measured probability of detection
The detection probability at the output of the sensor data fusion is 100%. It has to be noted that the
Toulouse-Blagnac implementation of the ADS-B sub-system (made of 5 ADS-B ground stations) is
over-redundant. Please refer to §2.9 for more details.
Note: the reported accuracies in Table 2-2 are referred to the average measured position of the test
transmitter GPS receiver. They are not referred to any other external reference and therefore they
represent more an indication of the position sample dispersions rather than an absolute accuracy
measure.
2.9 ADS-B cost-benefit test
The objective of this verification test was to assess the number of ADS-B ground stations required at
Toulouse-Blagnac to provide adequate vehicle surveillance coverage and probability of detection.
Save date: 2007-03-08
Public
Page 25
Version: 1.0
EMMA
The test was performed by measuring the probability of detection of moving vehicles and aircraft
along taxiways and runways centrelines (and runways borders) at the output of the following
components:
• each ADS-B ground station;
• ADS-B GTW;
• SDF.
This test was performed in January 2006. The initial equipment conditions were as follows:
• full ADS-B system installed, up and running;
• ADS-B system synchronised with the airport time reference;
• Mosquito units installed, up and running;
• some real life traffic;
Considering the Mosquito emission rate, for a recording period of N seconds of each Mosquito
equipped target and 100% of detection probability the following maximum detected events can be
obtained:
• each ground station: 2*N+N/5;
• ADS-B GTW: N;
• SDF: N.
2.9.2 Findings
The overall probability of detection at SDF output was 100%. The 5 ground stations installed at
Toulouse-Blagnac define an over redundant system, but one single ground station is not enough to
have a complete and reliable coverage. For a single ground station, the detection probability (with
exception of the one located at GBAS shelter) ranges from 70 to 90%, depending on the airport area.
Table 2-3: Detection probability for triples and couples of ADS-B ground stations
As shown in Table 2-3, a combination of 3 ground stations provides an adequate 99.88% probability
of detection of moving vehicles and aircraft along taxiways and runways centrelines (and runways
borders). Combined with other sensors (e.g. SMR), this performance is sufficient to reach a 100%
probability of detection at the output of the sensor data fusion.
2.10 Consolidation of verification results
In the tables below, the hypothesis numbers and hypothesis titles as documented in the ToulouseBlagnac verification and validation test plan (D6.1.3) are recalled, followed by the verification result.
The remaining hypotheses will be verified in the scope of EMMA-2, when the proposed system
improvements will have been implemented. Please refer to the verification and validation results
analysis document (D6.7.1) for more details on the proposed improvements.
2.10.1 Surveillance
Save date: 2007-03-08
Public
Page 26
Version: 1.0
EMMA
Hypothesis
N°
Ver.sur.2
Hypothesis title
Verification result
Coverage
Verified.
Ver.sur.3
Target position
accuracy
Ver.sur.5
Ver.sur.6
Update rate
Latency
Ver.sur.7
Efficiency
Partially verified, cf. D6.7.1 for an in-depth analysis. At the current date (i.e. March 2007),
the vehicle position accuracy issue has been solved and the test can be considered
"verified".
Verified.
Requirements not met, cf. D6.7.1 for an in-depth analysis. At the current date (i.e. March
2007), the vehicle position latency issue has been solved and the test can be considered
"verified".
Partially verified, cf. D6.7.1 for an in-depth analysis. At the current date (i.e. March 2007),
the efficiency issue has been solved and the test can be considered "verified".
2.10.2 Control
Hypothesis
N°
Ver.con.1
Hypothesis
title
Capability
Ver.con.2
Ver.con.5
Boundary
Latency
Verification result
Verified as part of D4.2.1 Site acceptance procedure for the Surface Conflict Alert (SCA)
system at Toulouse airport.
Verified as part of D4.2.1.
Verified as part of D4.2.1 to range between 0 ms and 2 ms (depending on the number of
conflicts).
2.10.3 Human machine interface
Hypothesis N°
Ver.hmi.1
Ver.hmi.2
Ver.hmi.3
Hypothesis title
Surveillance
Control
Update capability
Save date: 2007-03-08
Verification result
Verified.
Verified.
Verified.
Public
Page 27
Version: 1.0
EMMA
3 Validation session – shadow mode trials
3.1 Introduction
The validation session subject of the current chapter aims at assessing both the operational feasibility
of A-SMGCS in Toulouse-Blagnac airport and the operational improvements brought by this system.
Although the operational feasibility and operational improvements are defined as two different stages
of the validation activities, they have been evaluated during the same validation session.
The validation session consisted in a series of shadow-mode trials performed with controllers of
Toulouse-Blagnac airport. They were asked to fill in a questionnaire while assessing the A-SMGCS
HMI installed in the tower.
The questionnaire was derived from the one developed by the DLR for the Prague test site and adapted
to Toulouse-Blagnac local specificities. It addressed two aspects of the operational feasibility (i.e. the
acceptance of technical performance and the procedures) and the operational improvements. For each
topic, a series of statements were presented to the reader who was asked to point out his / her level of
agreement over 6 possibilities ranging from ‘strongly disagree’ to ‘strongly agree’.
All the controllers of Toulouse-Blagnac airport (i.e. 65) have followed a 45 minutes training course on
the SMGCS HMI installed in the tower for operational trials. Since the installation of the SMGCS in
September 2005, the controllers had many opportunities to use and evaluate the HMI during
operations.
The A-SMGCS HMI has the same ‘look-and-feel’ as the SMGCS HMI. However, in order to inform
the controllers working at Toulouse-Blagnac about the differences between both systems, 44 of them
have followed a one-hour briefing about the A-SMGCS between the end of January and the beginning
of February 2006.
Then a questionnaire was distributed to all the controllers. They were asked to fill it in by themselves
whenever they wanted and, if possible, in front of the A-SMGCS HMI in the tower.
Some of them were also asked to participate to interviews during shadow-mode trials on a voluntary
basis. The interviews took place in front of the A-SMGCS HMI in the tower and lasted 1h30 on
average. The questionnaire was filled in with each controller individually.
In all cases, emphasis was put on the importance of collecting controllers’ feedback and remarks in
order to better understand their answers.
The shadow-mode trials took place between the beginning of February and mid March 2006.
3.2 Data description and data collection methods
3.2.1 Raw data
Thirty-five questionnaires in paper format and in French have been collected, representing more than
50 percent of the 65 controllers in Toulouse-Blagnac. Amongst them, 17 were filled in during
interviews with controllers in 4 different sessions and 18 controllers filled them in on their own.
The initial version of the questionnaire, used for Prague test site validation activities, had to be adapted
to the trial constraints and local particularities. Unlike the Prague airport A-SMGCS, Toulouse ASMGCS was not used in operations. The questionnaire was adapted to take this into account. A human
factor expert reviewed the questionnaire in order to delete some minor inconsistencies and
redundancies. However, in order to allow comparison between the different test sites, it was decided to
minimize the differences between the questionnaires.
Three different versions of the questionnaire have been used during shadow-mode trials:
• The first version (V003) was too long (116 statements). After the first questionnaires had been
filled in, it appeared that some questions were not relevant for Toulouse-Blagnac or were
redundant, and therefore needed to be deleted or modified. That version was not distributed to all
the controllers and only 6 V003 questionnaires were filled in, both during interviews and directly
by controllers.
Save date: 2007-03-08
Public
Page 28
Version: 1.0
EMMA
•
The number of statements was reduced to 92 in version V005. A more direct grammatical style
was also adopted in order to ease the understanding of the statements and make the questionnaire
faster to fill in. That version was only used for 6 interviews.
• Version V006 only differs from V005 by one statement. The controllers did not understand the
statement ‘the EMMA display enables to detect a loss of accuracy of the surveillance’ (number 90
of version V005). Therefore, it was decided to delete it in version V006. This last version contains
91 statements. It was distributed to all the controllers that had not filled a questionnaire in yet.
Twenty-three V006 questionnaires were filled in both during interviews and directly by
controllers.
Version V005 is more complete than V006 but as the two versions are very similar, it has been
decided to keep version V005 as the reference questionnaire for the data analysis. In the following,
questionnaire V005 will be referred to as ‘the questionnaire’.
Each paper version has been converted to an individual sheet in an Excel file, allowing a
correspondence between the different versions of the questionnaire and facilitating the data analysis.
All the comments were translated from French to English before they were reported in the Data
Analysis section.
3.2.2 Additional data
Before starting the analysis of the data collected through the questionnaires and in order to better
understand some controllers’ answers and comments, it is worth highlighting that because of delays in
the technical implementation of the A-SMGCS, the controllers had a limited time period to discover
and master the A-SMGCS.
As a consequence, they often replied to the statements with a conservative eye. They sometimes even
preferred not to give any opinion, not knowing the system enough to evaluate the statement correctly.
Another issue was the lack of reliability of some of the functions of the system. In particular, the
identification of the aircraft was not continuous because of technical (e.g.: no identification coverage
on the apron, important latency time for the identification of departing aircraft, label swap, loss of
identification) and operational (e.g.: depending on each airline’s procedures, some pilots do not switch
the transponder on before they have reached the runway threshold) problems.
An important number of false plots or track duplication were regularly observed during all the
shadow-mode trials, especially when a wide body aircraft taxied in the vicinity of the SMR or during
heavy rain. Some of the false reports triggered false alarms, for example when they appeared on the
runway while an aircraft was approaching. These technical limitations have been addressed in
§3.4.3.1.4.
In this context, the controllers had a very limited trust in the system, which was reinforced by their
lack of experience with the A-SMGCS.
The A-SMGCS HMI has been installed on the V2 position in the tower (see Figure 1), on the left-hand
side of the pre-flight working position. The location was convenient to perform shadow-mode trials
and interviews without disturbing the normal operations but it is a bit too far from the working
positions used in normal operations to allow the controllers to look at the HMI from time to time
during periods of reduced traffic.
Save date: 2007-03-08
Public
Page 29
Version: 1.0
EMMA
A-SMGCS
HMI
Figure 16: Layout of the Working Positions in the Toulouse-Blagnac Tower
3.3 Data analysis
3.3.1 Data analysis method
In order to perform the data analysis on the answers given to the 92 statements of the questionnaire, a
figure has been associated to each level of agreement as follows:
• 1 – strongly disagree;
• 2 – disagree;
• 3 – slightly disagree;
• 4 – slightly agree;
• 5 – agree;
• 6 – strongly agree.
Then, a statistical analysis using the indicators described hereafter has been performed on the 33
collected questionnaires after they have been converted in MS-Excel sheets. The results have been
gathered in result sheets of the same file.
For each statement, the total number of answers has been counted. A low number of answers may
indicate that the controllers did not clearly understand the statement but they may also consider that
the system is not mature enough or they are not experienced enough with A-SMGCS to evaluate the
statement correctly. Comments made by the controllers often give helpful information on the reasons
for a low number of answers to a statement.
For each statement, the average of the answers has been calculated. The result, a number between 1
and 6, reflects the global opinion of the controllers on the statement.
For each statement, the standard deviation of the answers has been calculated. A low number for the
standard deviation indicates that the controllers’ opinions are close the ones from the others and
consequently close to the average while a high number reveals the disparity of the controllers’
opinions.
For each statement, the P parameter of the one sample t-test has been calculated. The one sample ttest is a statistical hypothesis test, which compares the mean score of a sample to a known value,
Save date: 2007-03-08
Public
Page 30
Version: 1.0
EMMA
usually the population mean. It allows the determination of the significance of the mean score of the
tested sample.
In the particular case of the questionnaire, the aim was to know if the hypothesis ‘the average of the
answers to each statement is significant’ was true. This hypothesis was tested with the one sample ttest against the ‘null hypothesis’ (i.e.: ‘the average of the total population of controllers is 3.5’). If the
P parameter of the one sample t-test for the tested hypothesis was smaller than 0.05 then the
hypothesis was accepted, which means that the average of the answers of the tested statement was
significant. If the P parameter was greater than 0.05, the hypothesis was rejected, which means that the
average of the answers to the tested statement was not significant.
3.3.2 Analysis tables for the shadow-mode validation trials
Annex A presents the results of the analysis in tables ordered according to the three topics assessed in
the questionnaire: acceptance of technical performances for operational feasibility, procedures for
operational feasibility and operational improvements.
3.4 Results
This section presents the results of the validation session in terms of operational feasibility and
operational improvements. For each stage the results of the metrics primarily defined in the
Verification and Validation Test Plan Document [5] are detailed. The validation of the hypotheses and
high-level objectives specified in that document are derived from the measured metrics.
3.4.1 Operational feasibility: acceptance of technical performances
3.4.1.1 Metrics
The results of the individual metrics related to the operational feasibility and acceptance of technical
performance listed in this section were measured during the shadow-mode trials.
3.4.1.1.1 General
3.4.1.1.1.1 Level of automation acceptance
The controllers are positive in principle about the level of automation of the A-SMGCS. Through the
detection and identification functions, the system provides assistance for surveillance activities on the
airport, even in good visibility conditions.
Automation of the tracking of the aircraft is well accepted when there is no need for high position
accuracy (e.g.: surveillance of an aircraft at the runway threshold or at the beginning of its takeoff).
There is even a demand for an adaptation of the procedures to allow more automation in non-critical
areas (e.g.: procedures requesting pilots to switch the aircraft transponder before pushing back,
reduction of the number of position reports by pilots on taxiways). See section 3.4.2 Operational
feasibility: procedures for further details.
The controllers consider that some functions like identification and alerts are not reliable enough yet
and they do not trust the system enough to exercise control with it.
3.4.1.1.1.2 Modularity acceptance
The modularity of the A-SMGCS is well accepted by the controllers. They consider that, after a short
adaptation period, the HMI can easily be customised to the user’s preferences.
3.4.1.1.1.3 Assignment of responsibilities acceptance
The A-SMGCS has only been implemented for shadow-mode trials in Toulouse-Blagnac.
However, the controllers have not foreseen any major modifications of the current responsibilities of
the different partners already present in Toulouse-Blagnac airport. For instance, the controllers noted
that the pilot should still be responsible for ensuring a safe push back and taxi, even when an ASMGCS is implemented.
Save date: 2007-03-08
Public
Page 31
Version: 1.0
EMMA
The main modifications foreseen by the controllers would be the share of responsibility between the
controller and the A-SMGCS. As indicated in the questionnaires, the controllers seem to be ready to
trust the system to determine the position and identification of mobiles in low visibility conditions.
3.4.1.1.1.4 Visibility transition procedures acceptance
The A-SMGCS has been implemented for shadow-mode trials in Toulouse-Blagnac. No A-SMGCS
procedures have been defined. Therefore, the acceptance of visibility transition procedures could not
be measured.
Nevertheless, the controllers consider that the transition from normal operations to low visibility
operations would be easier with the A-SMGCS and that the system helps them to reduce their
workload during low visibility operations.
Even if several controllers suggested that the current visibility limits could be reduced, the controllers
did not agree on modifying the visibility limits when using the A-SMGCS.
It must be stressed that the current visibility limit for the transition from normal operations to low
visibility operations (i.e. 1000 m) is very high in Toulouse-Blagnac. A reduction of this limit to the
standard 550 m limit, for instance, would bring a significant gain in terms of capacity and efficiency.
See section 3.4.2.10 Visibility transition for further details.
3.4.1.1.1.5 Fallback procedures acceptance
procedures have been defined. Therefore the acceptance of fallback procedures could not be measured.
Nevertheless, the controllers consider that if A-SMGCS procedures were defined, they would not have
to return to SMGCS procedures for all the traffic if only one mobile could not comply with the
technical requirements for A-SMGCS operations.
See section 3.4.2.12 Contingency procedures for further details.
3.4.1.1.1.6 Movement area acceptance
The definition of the manoeuvring and movement areas were agreed prior to the experimentation
sessions, as recommended by the ICAO manual on A-SMGCS.
It was ensured that no inactive or closed areas were to be excluded from the manoeuvring and
movement areas. It was also ensured that a common understanding of the different areas was reached
prior to the assessment of other technical performance indicators, such as detection and identification
coverage.
3.4.1.1.1.7 Movement rate acceptance
procedures have been defined and the movement rate has not been modified. Therefore, the acceptance
of increased movement rates could not be measured operationally.
Nevertheless, the controllers considered that the use of A-SMGCS could increase the airport capacity.
3.4.1.1.2 Surveillance
3.4.1.1.2.1 Controller acceptance of detection coverage
Despite their lack of experience with the A-SMGCS, the controllers consider that the detection
coverage is rather acceptable on the manoeuvring area.
It must be noted that on the apron, aircraft are only detected by the primary radar, which may create
confusion between the different types of mobiles.
3.4.1.1.2.2 Controller acceptance of identification coverage
No consensus was reached on the identification coverage. Several controllers considered that because
of their lack of experience with the A-SMGCS and the poor continuity of the identification function
they could not take a stand on the identification coverage.
Save date: 2007-03-08
Public
Page 32
Version: 1.0
EMMA
Nevertheless, some of the controllers regretted that identification was not possible on the apron (in
order provide traffic information).
3.4.1.1.2.3 Controller acceptance of position report accuracy
Despite their lack of experience, the controllers consider that the accuracy of the position report is
rather acceptable.
However, some of the controllers noted that it was not the responsibility of the controller to take into
account the accuracy of the system in order to perform control with the A-SMGCS (e.g. to ensure
separation between aircraft on taxiways, to check that an aircraft has crossed a stop bar).
Therefore, accuracy margins, taking into account the static accuracy of the system and aircraft latency
and speed should be defined and directly integrated in the procedures. The controllers would accept,
for instance, a procedure stating that an aircraft can be considered as having crossed a stop bar when
its report is displayed 50 m in front of the stop bar on the HMI.
3.4.1.1.2.4 Controller acceptance of heading and velocity accuracy
When the SMGCS was implemented for operational trials, the controllers requested that the heading
and velocity were not displayed on the HMI.
For consistency reasons, it has been decided to reproduce this characteristic on the A-SMGCS HMI.
Therefore the acceptance of heading and velocity accuracy could not be measured.
3.4.1.1.2.5 Controller acceptance of report delay
The controllers could not easily evaluate the system latency because of their lack of experience.
However, they consider that the global consistency between displayed and actual traffic is good.
Some controllers experienced an important latency with the mode S equipped vehicles. Therefore,
these controllers did not accept the report delay for the vehicles concerned.
3.4.1.1.2.6 Controller acceptance of probability of detection
The controllers could not easily evaluate the probability of detection because of their lack of
experience. However, they consider that there are too many missing reports to control the traffic in a
safe and efficient way.
3.4.1.1.2.7 Controller acceptance of probability of identification
No consensus was reached on the probability of identification. Several controllers considered that
because of their lack of experience with the A-SMGCS and the poor quality of the identification
function (poor continuity of service and correlation with flight plan data), they could not take a stand
on the probability of identification.
3.4.1.1.2.8 Controller acceptance of probability of false detection
The controllers consider that the probability of false detection is not acceptable. They had difficulties
to evaluate this metric because of their lack of experience.
3.4.1.1.2.9 Controller acceptance of probability of false identification
The controllers consider that the probability of false identification is not acceptable. They had
difficulties to evaluate this metric because of their lack of experience.
3.4.1.1.3 Routing
The routing function has not been implemented on the A-SMGCS HMI installed in the tower.
Therefore, the related routing metrics could not be measured.
It should be noted that the routing system installed could only be used for technical testing with the
objective of further developing it during the EMMA 2 project. The routing function and corresponding
metrics will be assessed in EMMA 2.
Save date: 2007-03-08
Public
Page 33
Version: 1.0
EMMA
3.4.1.1.4 Control
3.4.1.1.4.1 Controller acceptance of the short-term conflict alert
The controllers could not easily evaluate the short-term conflict alert because of their lack of
experience. Nevertheless, they are interested in principle in an efficient A-SMGCS alert service. They
consider that the A-SMGCS can already help them to detect and prevent incursions on the runway and
into restricted areas and stop bar crossings.
3.4.1.1.4.2 Controller acceptance of the alert response time
No consensus was reached on the runway alert response time. Many controllers considered that
because of their lack of experience with the A-SMGCS they could not take a stand on the runway
incursion alert response time.
Several controllers noted that the runway incursion alert is triggered too late to let them take the
appropriate action.
A few controllers consider that information alerts are often popping up too late to take an appropriate
action before an alarm is triggered.
3.4.1.1.4.3 Controller acceptance of the number of false and nuisance alerts
No consensus was reached on the number of false and nuisance alerts. Many controllers considered
that because of their lack of experience with the A-SMGCS they could not take a stand on the number
of false and nuisance alerts.
However, it should be noted that, during the shadow-mode trials, many false alerts due to false
detections were observed.
3.4.1.1.5 Human Machine Interface
3.4.1.1.5.1 HMI efficiency
The controllers have a globally positive opinion of the HMI efficiency. In particular:
• They consider that the zoom function is easy to use, even if a short adaptation period was
necessary for some of them.
• They consider that it is easy to display several windows.
• They have a rather positive opinion of the size of the pop up windows and the place where they
appear. However, several controllers noted that the alert texts were too small.
• They consider that the windows are rather conveniently arranged on the A-SMGCS HMI.
• They consider that the HMI is rather easy to customise to everyone’s own preferences.
• They consider that the level of interaction with the HMI is satisfactory.
No consensus was reached on whether pop-up windows sometimes obscure aircraft that should be
visible and whether the manual modification of the content of mobile reports on the HMI can be
performed quickly and efficiently. Many controllers considered that because of their lack of
experience with the A-SMGCS they could not take a stand on these issues. Nevertheless, some of the
controllers noted that the manual modification of the label content was not easy to use because of the
not intuitive label selection mode (i.e. click on the mobile instead of the label) and the difficulty to
‘catch’ a quickly moving mobile.
Note: most of the time, the controllers preferred to disable the automatic label anti-overlapping
function. Indeed, when there is a risk of overlap between two labels, the function systematically makes
the leader line of the concerned labels longer. As a result the HMI, quickly full of very long leader
lines (some of the labels even falling outside the display), looses its readability and usability.
Save date: 2007-03-08
Public
Page 34
Version: 1.0
EMMA
3.4.1.1.5.2 HMI information usability
The controllers have a globally positive opinion of the HMI information usability. In particular:
• They are rather satisfied with the amount of information about airborne traffic in the vicinity of
the airport. However, several controllers noted that this kind of information being available in
other tools, it is not useful in a ground tool like A-SMGCS.
• They generally consider that no important function is missing but several of them found
unfortunate that the reliability of the identification function and correlation with flight plan data
was so poor. Others asked for an automatic tracking of a selected aircraft in a dedicated window
and the possibility to delete the third line in a departure label while keeping the aircraft type.
• They consider that the colours used in the HMI are appropriate.
• They consider that the contrast between the windows and their background is sufficient.
• They consider that the height and width of the characters on the HMI are sufficient.
• They consider that the texts in the HMI are rather easy to read, except for the alert text that is too
small.
• They consider that the labels, signs and symbols in the HMI are easy to interpret.
• They consider that information is conveniently arranged in the HMI.
• They consider that symbols can easily be read under different angles of view in the HMI.
• They consider that the screen size is appropriate for a daily work but several of them are
concerned with the integration of the screen in almost full working positions.
Only three controllers evaluated the statement ‘the EMMA display enables to detect a loss of accuracy
of the surveillance’. The statement was deleted in version V006 of the questionnaire, the controllers
not being able to evaluate the loss of accuracy of the surveillance during the shadow-mode trials.
3.4.1.1.5.3 HMI information usefulness
The controllers have a globally positive opinion of the HMI information usefulness. In particular:
• They consider that it is useful to represent the closed taxiways, runways and apron areas on the
HMI.
• They consider that the actions that can be performed on the windows are relevant.
• They consider that the manual anti-overlapping of labels is rather useful, even if they would prefer
a more efficient automatic anti-overlapping function (cf. note in the HMI efficiency metric
section).
• They consider that the use of colour characterisation for the different types of mobiles is useful.
• They consider that the raw video is useful.
• They consider that the function allowing the manual modification of label content is useful, even
if many of them expected an improvement of its usability (Cf. HMI information usability metric).
3.4.1.2 Hypotheses
In this section, the operational feasibility hypotheses described in the Verification and Validation Test
Plan Document [5] are validated or invalidated according to the results of the metrics measured during
the shadow-mode trials. The characters between parentheses in each hypothesis title indicate the
reference of the indicator used in [5].
3.4.1.2.1 General
3.4.1.2.1.1 Level of automation (Opf.pre.1)
Hypothesis:
EMMA A-SMGCS should clearly identify the level of automation of each of these functions at level
I/level II.
Validation:
The hypothesis has been validated.
Save date: 2007-03-08
Public
Page 35
Version: 1.0
EMMA
3.4.1.2.1.2 Modularity (Opf.pre.2)
Hypothesis:
An A-SMGCS shall be composed of different modules required for particular user needs or
technological choices.
The design principle of an A-SMGCS should permit modular enhancements. A-SMGCS for each
aerodrome will comprise a different mix of modular components dependent on operational factors as
categorized in Appendix A. For example, some modules of an A-SMGCS will be required when one or
more of the following conditions exist: a) heavy traffic density; b) visibility condition 2, 3 or 4; and c)
complex aerodrome layout.
The required modules should be defined for each set of conditions (visibility, traffic density).
Validation:
3.4.1.2.1.3 Assignment of responsibilities (Opf.pre.3)
Hypothesis:
Although the responsibilities and functions may vary, they should be clearly defined for all users of ASMGCS. An A-SMGCS should be designed so that the responsibilities and functions may be assigned
to the following:
a) The automated system;
b) Controllers;
c) Pilots;
d) Vehicle drivers;
e) Marshallers;
f) Emergency services;
g) Airport authorities;
h) Regulatory authorities; and
i) Security services.
With the automation of safety-critical functions, responsibilities should be carefully assigned and
shared between equipment and human operators, at level I&II and for visibility conditions 1, 2 and 3.
Procedures should reflect this share of responsibilities.
Validation:
3.4.1.2.1.4 Visibility transition procedure (Opf.pre.4)
Hypothesis:
Transition limits between visibility conditions 1, 2, 3 and 4 should be carefully defined in terms of
RVR and ceiling. Transition procedures should be defined in order to ensure smooth transitions
between different visibility conditions. Taking into account the evolution of visibility, the use of
transition buffers may help to prepare the traffic for operations in the next visibility conditions.
Validation:
The hypothesis could not be tested during the shadow-mode trials.
A-SMGCS visibility transition procedures had not been defined when the validation sessions took
place.
This hypothesis should be tested during operational trials, when the system is mature enough to define
experimental visibility transition procedures adapted to A-SMGCS.
3.4.1.2.1.5 Fallback procedure (Opf.pre.5)
Hypothesis:
Fallback procedures should be defined to recover from the failure of one or more A-SMGCS
equipment. In EMMA and EMMA2, in Toulouse airport, these fallback procedures will be defined to
Save date: 2007-03-08
Public
Page 36
Version: 1.0
EMMA
prevent the hazards due to failure of equipment, identified in D1.3.9 ‘Functional Hazard Analysis and
Very Preliminary System Safety Assessment’.
Validation:
A-SMGCS fallback procedures had not been defined when the validation sessions took place.
experimental fallback procedures adapted to A-SMGCS.
3.4.1.2.1.6 Definition of Movement rate (Opf.pre.6)
Hypothesis:
An A-SMGCS should be capable of operating at a specified movement rate in visibility conditions
down to the aerodrome visibility operational level (AVOL). When visibility conditions are reduced to
below AVOL an A-SMGCS should provide for a reduction of surface movements of aircraft and
vehicles to a level acceptable for the new situation.
The acceptable maximum movement rate should be defined in level I & II and for visibility conditions
1, 2 and 3.
Validation:
Movement rates had not been modified when the validation sessions took place.
experimental movement rates with A-SMGCS.
3.4.1.2.1.7 Definition of movement area (Opf.pre.7)
Hypothesis:
In order to resolve the problem of vehicle control/segregation on a specific stand, the concept is
introduced whereby the role of that stand may change from active to passive and vice versa. Hence,
the use of movement area in this manual excludes passive stands, empty stands and those areas of the
apron(s) which are exclusively designated to vehicle movements.
The movement area should be clearly defined, in terms of included/excluded taxiways, runways,
vehicle routes, stands and other specific areas. The protected areas should also be fully defined.
Validation:
The movement area has been fully defined prior to the experimentations.
3.4.1.2.2 Surveillance
3.4.1.2.2.1 Target reference point (Opf.sur.1)
Hypothesis:
The target reference point of aircraft and vehicle should be carefully defined. For operations in
Toulouse-Blagnac airport, the target reference point is defined as the mid-point of longitudinal axis of
the aircraft or vehicle.
Validation:
Note: the target reference point verification test had not been performed when the validation sessions
took place.
3.4.1.2.2.2 Acceptable detection coverage (Opf.sur.2)
Hypothesis:
Save date: 2007-03-08
Public
Page 37
Version: 1.0
EMMA
Although all the movement area should be covered, the verification phase may reveal some shadow
detection areas. The impact of these shadow areas on ground traffic control should be carefully
determined, depending on the criticality of the area (final approach, initial departure, runway, runway
entrance/exit, taxiway, intersection, apron, protected area, etc.), on the visibility conditions (the area
is visible/not visible from the tower) and on the traffic density. Procedures related to these areas
should be adapted (for instance, by limiting/forbidding all traffic through these areas in visibility
condition 2).
Validation:
The detection coverage is considered as acceptable. Therefore, no modification of A-SMGCS
procedures is required.
The apron is only covered by the primary radar and procedures have not been adapted.
3.4.1.2.2.3 Acceptable identification coverage (Opf.sur.3)
Hypothesis:
Although all the movement area should be covered, the verification phase may reveal some shadow
identifications areas. The impact of these shadow areas on ground traffic control should be carefully
entrance/exit, taxiway, intersection, apron, protected area, etc.), on the visibility conditions (the area
is visible/not visible from the tower) and on the traffic density. Procedures related to these areas
should be adapted (for instance, by limiting/forbidding all traffic through these areas in visibility
condition 2).
Validation:
The hypothesis has not been validated.
The controllers did not reach any consensus on the acceptance of the identification coverage.
The apron is only covered by the primary radar and procedures have not been adapted.
3.4.1.2.2.4 Acceptable target position accuracy (Opf.sur.4)
Hypothesis:
Although the static and dynamic accuracy should be inferior to 7.5m, the verification phase may
reveal some problems:
•
The report is located near the aircraft MLAT antennas, different from the reference point,
(decrease of static accuracy),
•
The system extrapolates the trajectory when it loses a track (decrease of dynamic accuracy),
•
The resolution is too low, leading to a systematic bias for the positioning of a mobile.
The impact of this loss of static and dynamic accuracy should be carefully determined, depending on
the criticality of the concerned area (final approach, initial departure, runway, runway entrance/exit,
taxiway, intersection, apron, protected area, etc.), on the visibility conditions (the mobile is visible/not
visible from the tower) and on the traffic density.
The impact of this loss of accuracy on the performances of the control function should also be
determined.
Validation:
The hypothesis has been partly validated.
The controllers consider that the target position accuracy is acceptable. Nevertheless, many controllers
require an adaptation of the procedures to take into account accuracy margins. Due to the lack of
technical data, this adaptation could not be performed prior to the validation activities.
3.4.1.2.2.5 Acceptable reported velocity and heading accuracy (Opf.sur.5)
Hypothesis:
The reported velocity and heading accuracy will have been measured during the verification phase.
Too low velocity and heading accuracies may have some unwanted technical and operational effects,
Save date: 2007-03-08
Public
Page 38
Version: 1.0
EMMA
in level II, but especially in higher levels of A-SMGCS, where more automated functions will rely on
the determination of correct velocity and headings:
•
Nuisance short term conflict alerts,
•
Nuisance route conformance monitoring alerts,
•
Wrong route definition by the routing function, due to wrong heading of aircraft.
The impact of these lacks of accuracy should be determined, in different visibility and traffic density
conditions. A distinction should be made between direct effects (display of wrong velocities and
headings on the HMI) and indirect effects (nuisance alerts).
Validation:
The hypothesis could not been tested.
It has been chosen not to display the heading and velocity on the A-SMGCS HMI.
This hypothesis can only be tested in Toulouse-Blagnac if the need for heading and velocity
information on the HMI is identified.
3.4.1.2.2.6 Acceptable discrimination accuracy (Opf.sur.6)
Hypothesis:
The discrimination between two mobiles will have been measured during the verification phase. A too
low discrimination accuracy may reduce the controller’s situation awareness, by leading to a
confusion between mobiles or by considering only one mobile, when several mobiles are near the
same location. This is especially true when two vehicles or when an aircraft and a vehicle are
involved.
The impact of a too high discrimination distance should be carefully determined, depending on the
visibility conditions and on the traffic density (aircraft + vehicles).
Validation:
The hypothesis could not be tested.
The position discrimination verification test had not been performed when the validation sessions took
place. This hypothesis should be tested during shadow mode trials, when the discrimination accuracy
has been measured (e.g. in EMMA2).
3.4.1.2.2.7 Acceptable accuracy margins (Opf.sur.7)
Hypothesis:
In order to increase movement efficiency, accuracy margins on the HMI may be defined to allow a
controller to check (if the pilot still reports) or to state (if he does not) that a mobile has cleared the
runway, crossed an intersection, pushed back, etc. These accuracy margins should take into account
the reported size of the mobile on the HMI, and the dynamic accuracy.
The measurement of these values during the verification phase will help to define acceptable accuracy
margins with the controller.
Also, a loss of accuracy, if the controller is not informed, may decrease safety, by encouraging him to
use procedures based on accuracy margins when these margins are not applicable anymore. Fallback
procedures or technical requirements should then be defined to prevent this case.
Validation:
Accuracy margins on the HMI had not been defined when the validation sessions took place.
3.4.1.2.2.8 Acceptable report delay (Opf.sur.8)
Hypothesis:
The update rate, its variations, and the latency will have been measured during the verification phase.
This will lead to the determination of the maximum report delay, i.e. the maximum delay between a
given situation and its display on the controller’s HMI. This delay has a negative effect on the
controller’s situation awareness, especially when no outside view is available from the tower, and may
lead to late reactions in case of safety critical situations.
Save date: 2007-03-08
Public
Page 39
Version: 1.0
EMMA
The impact of this report delay should be carefully examined, depending on the visibility conditions
and on the traffic density.
Validation:
The controllers consider that the consistency between displayed and actual traffic is good.
3.4.1.2.2.9 Acceptable probability of detection (Opf.sur.9)
Hypothesis:
The definition of acceptable detection coverage will address the issue related to fixed and permanent
shadow areas. However, the verification phase may reveal some other dynamic, not fixed, losses of
detection. The impact of these losses on ground traffic control should be carefully determined,
depending on the criticality of the area (final approach, initial departure, runway, runway
entrance/exit, taxiway, intersection, apron, protected area, etc.), on the visibility conditions and on the
traffic density.
Quantitative and qualitative criteria, determined with the help of controllers, such as, the acceptable
number of detection gaps in a track, the size of these gaps, the acceptable number of tracks with a gap,
etc. will help compute acceptable probabilities of detection for each visibility conditions and possibly
different for the different areas of the airport.
Validation:
The controllers consider that there are too many missing reports to control the traffic in a safe and
efficient way.
3.4.1.2.2.10 Acceptable probability of identification (Opf.sur.10)
Hypothesis:
The definition of acceptable identification coverage will address the issue related to fixed and
permanent shadow areas. However, the verification phase may reveal some other dynamic, not fixed,
losses of identification. The impact of these losses on ground traffic control should be carefully
entrance/exit, taxiway, intersection, apron, protected area, etc.), on the visibility conditions and on the
traffic density.
Quantitative and qualitative criteria, determined with the help of controllers, such as, the acceptable
number of identification gaps in a track, the size of these gaps, the acceptable number of tracks with a
gap, etc. will help compute acceptable probabilities of identification for each visibility conditions and
possibly different for the different areas of the airport.
Validation:
The controllers did not reach any consensus on the acceptance of the probability of identification.
3.4.1.2.2.11 Acceptable probability of false detection (Opf.sur.11)
Hypothesis:
The verification phase may reveal some fixed and not fixed false target reports. False target reports
are reports of not existing solid targets (rain, grass, reflection, etc.). These target reports have
negative on the controller’s situation awareness, as they create some additional workload to
determinate their non-significance.
The impact of these false reports on ground traffic control should be carefully determined, depending
on the criticality of the area (especially on or near the runway, where they can cause false short term
conflict alerts), on the visibility conditions (the determination of the significance of the target is more
difficult in visibility 2 or more) and on the traffic density (high traffic densities may cause more
confusion between false and real targets).
Save date: 2007-03-08
Public
Page 40
Version: 1.0
EMMA
Validation:
The controllers consider that the probability of false detection is not acceptable.
3.4.1.2.2.12 Acceptable probability of false identification (Opf.sur.12)
Hypothesis:
The verification phase may reveal some target reports with incorrect identification. These target
reports, labelled with incorrect identification, have negative on the controller’s situation awareness,
as they create some additional workload to re-create a correct mental association between the
position of the mobiles and their identification.
The impact of these reports on ground traffic control should be carefully determined, depending on the
criticality of the area (especially on or near the runway, where they can cause false short term conflict
alerts, due to confusion between mobile types), on the visibility conditions (a correct mental
association is more difficult to re-create in visibility 2 or more) and on the traffic density (high traffic
densities may cause more confusion between mobiles).
Validation:
The controllers consider that the probability of false identification is not acceptable.
3.4.1.2.3 Routing
The routing function has not been implemented on the A-SMGCS HMI installed in the tower.
Therefore the related routing hypotheses could not be tested.
These hypotheses should be tested during shadow mode trials, when the routing function has been
implemented and routing performance has been measured (e.g. in EMMA2).
3.4.1.2.4 Control
3.4.1.2.4.1 Short-term conflict alert validation (Opf.con.1)
Hypothesis:
The short-term conflict alerts produced by the SCA should help to prevent safety critical events. The
SCA rule implementation and parameters should be consistent with this requirement.
Validation:
The hypothesis has been validated for the incursion rules but not for the conflict rules.
The controllers consider that the A-SMGCS helps them to detect and prevent incursions on the runway
and into restricted areas and stop bar crossings.
3.4.1.2.4.2 Acceptable alert response time (Opf.con.2)
Hypothesis:
The short-term conflict alert function should allocate sufficient time to the controller to take the
necessary actions to solve the safety critical situation. For the validation of this hypothesis, it should
be considered that:
•
The short term conflict alert delay should be sufficiently low,
•
The SCA rule implementation and parameters should take into account the response time, possibly
by providing several levels of alerts.
Validation:
The controllers did not reach any consensus on the acceptance of the alert response time.
Save date: 2007-03-08
Public
Page 41
Version: 1.0
EMMA
3.4.1.2.4.3 Acceptable probability of false or nuisance alert (Opf.con.3)
Hypothesis:
The short-term conflict alert system should provide an acceptable number of false and nuisance alerts.
False alerts are alerts caused by incorrect surveillance data (false reports, etc.). Nuisance alerts are
alerts caused by an inappropriate definition of alert rules and parameters. False and nuisance alerts,
not related to a safety critical event, have a negative impact on the situation awareness of the
controller and on the trust he/she has in the SCA system.
Validation:
The controllers did not reach any consensus on the acceptance of the number of false and nuisance
alerts.
3.4.1.2.5 Human Machine Interface
3.4.1.2.5.1 Efficiency (Opf.hmi.1)
Hypothesis:
A-SMGCS shall enable users to interface efficiently.
Validation:
3.4.1.2.5.2 Usefulness of information (Opf.hmi.2)
Hypothesis:
The information provided to the controller though the A-SMGCS display is useful for the operations.
Validation:
3.4.1.2.5.3 Usability of information (Opf.hmi.3)
Hypothesis:
The presentation of the information provided through the A-SMGCS display is adapted to user needs.
Validation:
3.4.1.3 High-level objectives
In this section, the operational feasibility high-level objectives described in the Verification and
Validation Test Plan Document [5] are validated or invalidated according to the results of the
hypotheses validation.
3.4.1.3.1 High-level objective 1
Objective:
EMMA A-SMGCS meets the operational requirements expressed in the ICAO manual on A-SMGCS
[15] for each set of conditions.
Validation:
The objective has been partly validated.
This is mainly explained by the lack of maturity of the surveillance function of EMMA A-SMGCS
installed in Toulouse-Blagnac. It does not allow the operational requirements for the other functions to
be fulfilled.
Please refer to the D6.7.1 Verification and Validation Analysis Report [11] for a detailed review of
each operational requirement defined in EMMA.
Save date: 2007-03-08
Public
Page 42
Version: 1.0
EMMA
Objective:
EMMA A-SMGCS meets the operational requirements expressed by local end-users for each set of
conditions.
Validation:
Objective:
For each set of conditions, a full set of performance requirements is defined for each automated
service.
Validation:
This objective has not been validated.
Some performance requirements have not been fulfilled by the system.
Objective:
EMMA A-SMGCS supports adequate procedures to recover from the possible failures of A-SMGCS
equipment.
Validation:
This objective has not been validated.
3.4.2 Operational feasibility: procedures
No metrics were primarily defined in the Verification and Validation Test Plan Document [5] for the
measurement of the operational feasibility of A-SMGCS procedures.
Nevertheless, this section presents the results of the controllers’ evaluation of statements related to the
operational feasibility of A-SMGCS procedures.
3.4.2.1 Start-up clearance
No consensus was reached on whether the A-SMGCS allows the controllers to set up a more efficient
start up sequence. It appeared from the comments made by several controllers that the A-SMGCS has
no direct impact on the setting up of the start up sequence. It is rather considered as a support tool that
helps the controllers by offering a global synthetic picture of the platform.
3.4.2.2 Push back clearance
The controllers consider that the information on the A-SMGCS HMI helps them to decide whether a
push back clearance should be delayed. It was noted that it would be even more helpful if the
identification function were more reliable and if identification was possible on the apron.
The controllers slightly agree with the statement that the A-SMGCS is sufficient to determine that an
aircraft is on the stand or has left the stand. They generally agree that it can be used to determine that
an aircraft has left the stand but not to check that it is at the stand. Too many permanent false reports
are observed on the apron, in particular on the Echo area where the terminal building creates
interferences on the contact stands, to be able to know whether a stand is free or occupied.
Until the system is more accurate, the controllers consider that it cannot be used as the unique
information source to deliver a push back clearance (i.e. they need the pilot’s confirmation and / or the
external view).
Save date: 2007-03-08
Public
Page 43
Version: 1.0
EMMA
3.4.2.3 Taxi clearance
The controllers consider that when they want to give a taxi clearance they can rely on the information
provided by the A-SMGCS, even when visual reference is not possible on the manoeuvring area
provided that the system has been improved in terms of reliability.
The controllers consider that the surveillance of the longitudinal spacing between aircraft on taxiways
is easier with the A-SMGCS when visual reference is not possible on the manoeuvring area. Several
controllers explained that they are missing the actual size of each aircraft, accuracy of displayed
aircraft position and accuracy margins in order to use the A-SMGCS for the surveillance of the
longitudinal spacing.
Mainly because that information is missing, no consensus was reach on whether the longitudinal
spacing between aircraft on taxiways could be reduced with the A-SMGCS when visual reference is
not possible on the manoeuvring area. Moreover, the controllers often claimed that ground traffic on
the taxiways is not the limiting factor during low visibility operations, the departing aircraft having to
queue at the runway holding point anyway. They do not feel the need to modify the current procedures
(i.e. one aircraft per taxiway) in order to reduce longitudinal spacing on taxiways during low visibility
operations.
The controllers consider that the A-SMGCS will not necessarily prevent them from making records of
the vehicles on the manoeuvring area. As for the use of paper strips, it depends on each controller’s
working habit. Nevertheless, even if some controllers consider that it may not be necessary to maintain
a list of the vehicles on the manoeuvring area during low visibility operations if the reliability of the
A-SMGCS is improved, many of them insisted on the importance of having a paper backup in case of
system failure.
3.4.2.4 Co-ordination
The controllers consider that the coordination and the hand over of a taxiing aircraft between the
control working positions in the tower would be more efficient with the A-SMGCS. Some controllers
noted that the ground and tower working positions should be equipped with A-SMGCS HMIs and that
it could be useful for the hand over, for example when the activation sequence is different from the
off-block sequence.
3.4.2.5 Taxiing on the runway
The controllers consider that the number of pilot position reports could be reduced when using the ASMGCS. Nevertheless, they do not agree on suppressing the pilot position report to confirm that the
aircraft has vacated the runway because it is safety critical.
The controllers consider that with the A-SMGCS an aircraft may be permitted to taxi on the runway in
use even when visual reference is not possible on the manoeuvring area. However, the interest of
making aircraft taxi on the runway in use is limited in Toulouse because the taxiway network is
developed enough to avoid it.
3.4.2.6 Line-up clearance
The controllers consider that line-up of an aircraft at the runway threshold is easier to control with the
A-SMGCS whether visual reference is possible or not on the manoeuvring area. In normal operations
it is particularly true at night and when runway 32R is used for departures because the threshold is far
from the tower.
No consensus was reached on whether multiple line-ups could be performed with the A-SMGCS when
visual reference is not possible on the manoeuvring area. Many controllers noted that the A-SMGCS is
not mature enough (in terms of reliability and accuracy) to perform multiple line-ups in low visibility
conditions. Several of them were reluctant on the idea of modifying the current working habits to
encourage multiple line-ups in low visibility conditions.
3.4.2.7 Takeoff clearance
The controllers consider that the A-SMGCS is rather helpful for better monitoring an aircraft
commencing the takeoff roll.
Save date: 2007-03-08
Public
Page 44
Version: 1.0
EMMA
No consensus was reached on whether rolling takeoff clearances could be given more frequently with
the A-SMGCS. Many controllers consider that the A-SMGCS is not a key tool when making the
decision about a rolling takeoff clearance.
The controllers consider that the A-SMGCS is not sufficient to decide when a takeoff clearance should
be issued. Several of them explained that other parameters have to be considered (e.g. inbound traffic
in final approach, readiness of the pilot).
3.4.2.8 Landing clearance
No consensus was reached on whether the A-SMGCS is sufficient to decide if the runway in use has
been vacated in order to deliver a landing clearance when visual reference is not possible on the
manoeuvring area. Many controllers noted that the A-SMGCS is not mature enough (in terms of
reliability and accuracy) to use it as the unique decision tool to give landing clearances.
3.4.2.9 Conditional clearances
The controllers consider that when visual reference is possible on the manoeuvring area, the ASMGCS is rather helpful to deliver conditional clearances on the runway in use and it is helpful to
deliver conditional clearances on taxiways in a safe and efficient way.
They also consider that it is rather helpful to deliver conditional clearances on taxiways in a safe and
efficient way even when visual reference is not possible on the manoeuvring area.
No consensus was reached on whether the A-SMGCS may help them to deliver conditional clearances
on the runway in use in a safe and efficient way when visual reference is not possible. Several
controllers were reluctant to modify there working habits (i.e.: they do not give conditional clearances
in low visibility conditions) in such a safety critical case, because they do not trust the system enough.
3.4.2.10 Visibility transition
The controllers consider that the transition from normal operations to low visibility operations is easier
with the A-SMGCS.
They also consider that when low visibility procedures are activated, the A-SMGCS helps them to
reduce their workload.
No consensus was reached on whether the visibility limits for the transition to low visibility
procedures could be redefined for the A-SMGCS. Several controllers noted that the A-SMGCS has no
influence on the definition of the visibility limits.
3.4.2.11 Transponder procedures
The controllers consider that it would be appropriate that pilots switch the transponder on before they
request a push back clearance, in order to automatically associate a label to a departing aircraft.
3.4.2.12 Contingency procedures
The controllers consider that if only one mobile fails to comply with the technical requirements for ASMGCS operations, they do not have to return to SMGCS procedures for all the traffic. They
generally agree that if only one mobile cannot be automatically identified, they can easily manage it
either by identifying it manually or by asking for position reports to the pilot.
3.4.2.13 Conclusion on procedures
The controllers consider that the A-SMGCS helps them whether visual reference is possible or not on
the manoeuvring area.
They consider that they can rely on the A-SMGCS instead of the outside view when visual reference is
not possible on the manoeuvring area provided that the reliability of the current system is improved.
They consider that there are not too many inconsistencies between the A-SMGCS HMI and actual
traffic.
They slightly agree with the fact that the mix of cooperative and non-cooperative mobiles could
severely restrict the use of A-SMGCS when visual reference is not possible on the manoeuvring area.
Save date: 2007-03-08
Public
Page 45
Version: 1.0
EMMA
Nevertheless, some of them argued that the severity of the restriction depends on the number of noncooperative mobiles.
3.4.3 Operational improvements
3.4.3.1 Metrics
The results of the individual metrics related to operational improvements listed in this section were
measured during the shadow-mode trials.
The characters between parentheses in each metric title indicate the reference3 of the indicator used in
D6.1.3: Verification and validation test plan for Toulouse-Blagnac airport, version 1.02, issued on the
15th February 2006 [5].
3.4.3.1.1 Capacity
3.4.3.1.1.1 Assessment of departure and arrival runway capacity in visibility conditions 2 and 3
No direct measurement of the runway capacity could be performed during the shadow-mode trials.
Nevertheless, the results of the evaluation of the operational feasibility of A-SMGCS procedures seem
to show that the potential gain in runway capacity would be limited. Indeed, the controllers do not
want to suppress the position report of the pilot when the aircraft has vacated the runway. There is
consensus neither on the use of multiple line-up, rolling takeoff and conditional clearances on the
runway in use in low visibility conditions, nor on the use of A-SMGCS only to decide to give a
landing clearance in low visibility conditions, nor on the redefinition of the visibility limits.
See section 3.4.2 Operational feasibility: procedures for further details.
Therefore, the controllers do not see the introduction of the A-SMGCS as an opportunity to modify
deeply the procedures and their working habits during low visibility operations in order to increase the
runway capacity. This is explained mainly by the immaturity of the system (in terms of reliability and
accuracy) and, as a consequence, by the lack of confidence of the controllers to perform control with
it. The controllers also seem to consider that the interest of introducing changes during low visibility
operations and increasing the runway capacity is very limited because most of the time the traffic
demand can be absorbed without high delays during low visibility operations.
Nevertheless, several controllers stressed that a validated A-SMGCS and an adapted lighting on the
platform could allow both runways to be used during low visibility operations, which would increase
the runway throughput and the capacity of the airport.
3.4.3.1.1.2 Assessment of departure and arrival runway throughput in visibility conditions 2 and
3 (CA01 & CA02)
The controllers have a positive feeling about the increase of departure and arrival throughput during
low visibility operations. The A-SMGCS may also help them to manage more traffic when visual
reference is not possible on the manoeuvring area.
As suggested by some controllers, if both runways are used in low visibility operations with the ASMGCS, the throughput of each runway may be increased because the time between two movements
could be reduced.
A quantitative assessment of the runway throughput has been performed during the fast-time
simulation. See section 4.4.1.1.1.1 Assessment of departure and arrival runway throughput in visibility
conditions 2 and 3 (CA01 & CA02) for further details.
3.4.3.1.1.3 Assessment of mean and maximum number of aircraft simultaneously taxiing in
visibility conditions 2 and 3 (CA09 & CA10)
No direct measurement of the taxiway capacity could be performed during the shadow-mode trials.
3
These figures are unfortunately not always consistent with D6.2.2: Indicators and Metrics for A-SMGCS [7].
Save date: 2007-03-08
Public
Page 46
Version: 1.0
EMMA
to show that there would be a gain in taxiway capacity if the A-SMGCS were used during low
visibility operations. The gain would be mainly due to the use of A-SMGCS by the controllers to
deliver taxi clearances and the reduction of the number of position reports by the pilots on the
taxiways. Conditional clearances on taxiways could also be authorised in low visibility operations,
making the ground traffic more fluid.
However, the controllers did not agree to reduce the longitudinal spacing between aircraft on taxiways
in low visibility conditions because of their lack of trust in the A-SMGCS and the limited interest of
such a modification of the current procedures compared with the traffic demand during low visibility
operations.
A quantitative assessment of the maximum number of aircraft simultaneously taxiing has been
performed during the fast-time simulation. See §4.4.1.1.1.2, Assessment of mean and maximum
number of aircraft simultaneously taxiing in visibility conditions 2 and 3 (CA09 & CA10), for further
details.
3.4.3.1.1.4 Assessment of mean and maximum number of push back clearances in visibility
conditions 2 and 3 (CA07 & CA08)
No direct measurement of the apron capacity could be performed during the shadow-mode trials.
to show that there would be a gain in apron capacity if A-SMGCS were used during low visibility
operations.
However, the controllers do not seem to consider that A-SMGCS would be of any help in the
preparation of the start up sequence. They believe that A-SMGCS could help them in the decision to
deliver a push back clearance and to check that an aircraft has left a stand. Then, A-SMGCS could
have a slightly positive impact on the apron capacity but this impact is restrained by the lack of
identification coverage on the apron.
A quantitative assessment of the maximum number of aircraft pushing back per 10 minutes time slot
has been performed during the fast-time simulation. See section 4.4.1.1.1.3 Assessment of mean and
maximum number of push back clearances in visibility conditions 2 and 3 (CA07 & CA08) for further
details.
3.4.3.1.2 Efficiency
3.4.3.1.2.1 Minimum, mean and maximum departure and arrival taxi times in visibility
conditions 2 and 3 (EF10, EF11 & EF12)
No direct measurement of the taxi time could be performed during the shadow-mode trials.
Nevertheless, from the results of the evaluation of the operational feasibility of A-SMGCS procedures,
a reduction of the taxi time on average during low visibility operations can be expected for the same
reasons as for a potential increase of the taxiway capacity (i.e. use of A-SMGCS for taxi clearance
delivery, reduced number of position reports by pilots, possibility of giving conditional clearances).
See sections 3.4.2 Operational feasibility: procedures and 3.4.3.1.1.3 Assessment of mean and
maximum number of aircraft simultaneously taxiing in visibility conditions 2 and 3 for further details.
A quantitative assessment of the mean departure and arrival taxi times has been performed during the
fast-time simulation. See section 4.4.1.1.2.8 Minimum, mean and maximum departure and arrival taxi
times in visibility conditions 2 and 3 (EF10, EF11 & EF12) for further details.
3.4.3.1.2.2 Assessment of number of communications in visibility conditions 2 and 3 (EF20)
No direct measurement of the number of communications could be performed during the shadowmode trials.
to show that the average number of pilot / controller communications could decrease if the A-SMGCS
Save date: 2007-03-08
Public
Page 47
Version: 1.0
EMMA
was used during low visibility operations. The controllers consider indeed that the number of position
reports by pilots could be reduced in low visibility conditions.
3.4.3.1.3 Human Factors
3.4.3.1.3.1 Controller situational awareness (HF02)
The controllers consider that their situational awareness is improved with the A-SMGCS. In particular,
they consider that the A-SMGCS HMI allows them to have a better awareness of the respective
position and identification of each mobile, a better understanding of the traffic situation and to better
anticipate the evolution of traffic. Several controllers noted that an A-SMGCS HMI should be
installed at the approach control working position to help the controller to better anticipate the
evolution of traffic.
3.4.3.1.3.2 Controller acceptance of the A-SMGCS HMI (HF03)
The controllers have a good global acceptance of the A-SMGCS HMI. In particular, they consider that
it provides the right information at the right time without being unnecessarily complex. The HMI helps
them to reduce their mental workload.
See section 3.4.1.1.5 Human Machine Interface for further details.
3.4.3.1.3.3 Usability of the A-SMGCS HMI (HF05)
The controllers consider that the global usability of the A-SMGCS HMI is satisfactory.
See section 3.4.1.1.5.2 HMI information usability for further details.
3.4.3.1.4 Safety
3.4.3.1.4.1 Detected incorrect display (SA07)
The opinion of the controllers concerning the detected incorrect display events is not obvious. On the
one hand they consider that the A-SMGCS HMI and the actual traffic are quite consistent and that the
A-SMGCS does not endanger safety at the airport. On the other hand, they think that the amount of
incorrectly identified mobiles on the A-SMGCS HMI is a problem to control the traffic in a safe and
efficient way.
Many incorrect display events were observed during the shadow-mode trials. The main categories of
incorrect display event are described hereafter:
• In most cases, false plots appeared somewhere on the airport, sometimes on the runway while an
aircraft was about to land. This triggered false runway incursion alerts. Interferences caused by a
wide-body aircraft (e.g. Beluga, A380) taxiing on taxiway Whisky in the vicinity of the SMR
triggered some of the observed false plots.
• Since the system was used in shadow-mode trials when these events happened, there was no safety
impact on the operations.
• This kind of event may have a very negative impact on the acceptance of the A-SMGCS by the
controllers, even as an information system.
• During one of the shadow-mode trial sessions, a storm brought heavy rain in the southeast part of
the airport, over runway 32L and 32R thresholds. It caused a lot of false plots in this area on the
HMI, making it unusable during the event while visibility suddenly reduced.
• This kind of event may have a negative impact on the acceptance of the A-SMGCS by the
• Several cases of inaccurate line-up on the runway axis occurred for aircraft in final approach. A
discontinuity was observed in the trajectory of approaching flights, suddenly passing from lined up
on the runway axis to shifted several metres to the right or to the left of the runway axis just before
landing. This inconsistency is probably due to a gap between the track detected by the approach
radar and the track detected by the SMR. In some cases the two trajectory were displayed at the
same time on the HMI, causing false runway incursion alerts.
Save date: 2007-03-08
Public
Page 48
Version: 1.0
EMMA
•
This kind of event may have a very negative impact on the acceptance of the A-SMGCS by the
Finally, despite these problems that need to be addressed, the controllers seem to accept the ASMGCS as an additional information tool.
However, major improvements, mainly in terms of reliability of detection and identification, will have
to be made for the controllers to trust the A-SMGCS enough and accept it as a control tool.
3.4.3.1.4.2 Detected loss of display (SA08)
No total loss of A-SMGCS HMI display was observed during the shadow-mode trials.
Only a few cases of mobiles loosing their synthetic plots for a very limited time were observed. The
severity of the losses was mitigated by the continuity of the primary radar track.
Although the controllers consider that there are too many missing reports to control the traffic in a safe
and efficient way, they seem to accept the level of detected loss of display for the A-SMGCS as an
additional information tool.
3.4.3.1.4.3 Detected lack of identifier (SA09)
Many cases of lack of identifier were observed during the shadow-mode trials. The main categories of
lack of identifier events are described hereafter:
• In many cases for departing flights, pilots did not switch the transponder on before they lined up
on the runway threshold for takeoff. As a consequence the aircraft could not be identified before
takeoff. As mitigation, the controller in charge of the flight could ask for the pilot to switch the
transponder on. Nevertheless, it was chosen not to do it systematically not to disturb the
controllers and pilots. Another solution would be a manual identification of the unidentified
mobile. This kind of event could easily be avoided by modifying the transponder procedures for
pilots.
• Several cases of lack of identifier were observed for departing flights where although the
transponder was on (cross-check with the pilot), the aircraft was identified several tenths of
seconds later. A latency time of several tenths of seconds was also observed on mode S equipped
vehicles. As mitigation, the controller could identify the mobile manually.
• In many cases for departing and arriving flights, temporary or permanent losses of identification
were observed. As mitigation, the controller could identify the mobile manually.
• In many cases, mainly for departing aircraft, the transponder code was the only information
displayed on labels because flight plan data were not available. No mitigation was possible during
the shadow-mode trials.
All the observed cases of lack of identifier may have a very negative impact on the acceptance of the
A-SMGCS by the controllers, even as an information system.
In summary, the controllers do not seem to accept the level of detected lack of identifier.
3.4.3.1.4.4 Detected incorrect identifier (SA10)
Several cases of incorrect identifier were observed during the shadow-mode trials. They concerned
label swaps between mobiles. More precisely, the label of an aircraft was ‘caught’ by an unidentified
mobile taxiing in its vicinity (e.g. bus, push back truck).
The controllers who witnessed these cases tried to manually dissociate the label from the incorrectly
identified mobiles but most of the time this action was not possible.
The controllers do not accept the level of detected incorrect identifier for the A-SMGCS as a control
tool but it seems to be more acceptable if it is seen as an additional information tool.
3.4.3.1.4.5 Controller errors induced by A-SMGCS in visibility conditions 2
The ‘controller errors induced by A-SMGCS in visibility conditions 2’ metric could not be measured
during the shadow-mode trials.
Save date: 2007-03-08
Public
Page 49
Version: 1.0
EMMA
3.4.3.2 Hypotheses
In this section, the operational improvement hypotheses described in the Verification and Validation
Test Plan Document [5] are validated or invalidated according to the results of the metrics measured
3.4.3.2.1 Capacity
3.4.3.2.1.1 Capacity hypothesis 1 (Opi.cap.1)
Hypothesis:
Using A-SMGCS, the runway departure and arrival capacity will be maintained or increased,
especially in visibility conditions 2 and 3.
Validation:
The runway departure and arrival capacity should at least be maintained.
3.4.3.2.2.1 Efficiency hypothesis 1 (Opi.eff.1)
Hypothesis:
The number of pilot reports will be maintained or decreased, especially in visibility conditions 2 and
3.
Validation:
The number of pilot reports should be decreased.
3.4.3.2.3 Human Factors
3.4.3.2.3.1 Human factors hypothesis 1 (Opi.hf.1)
Hypothesis:
Using A-SMGCS, the situation awareness of the controller will be improved, in particular the mental
association between position and identification of mobiles.
Validation:
Hypothesis:
The A-SMGCS display will be accepted by the controller.
Validation:
Hypothesis:
The controller will state that A-SMGCS is usable.
Validation:
Save date: 2007-03-08
Public
Page 50
Version: 1.0
EMMA
3.4.3.2.4 Safety
3.4.3.2.4.1 Safety hypothesis 1 (Opi.saf.1)
Hypothesis:
The detected corruption of the A-SMGCS HMI won't have a significant negative impact on the safety
of operations. The following events should be considered:
•
Detected incorrect displays,
•
Detected loss of display,
•
Detected lack of identifier,
•
Detected incorrect identifier.
Validation:
The controllers consider that the detected incorrect displays, detected loss of display and detected
incorrect identifier are acceptable to use A-SMGCS as an additional information tool but not as a
control tool.
They consider that the detected lack of identifier is not acceptable.
3.4.3.2.4.2 Safety hypothesis 2 (Opi.saf.2)
Hypothesis:
When using A-SMGCS, the number of controller errors will be reduced, especially under adverse
weather conditions.
Validation:
This hypothesis should be tested during shadow-mode trials, when all the experimental A-SMGCS
procedures have been performed.
In this section, the operational improvement high-level objectives described in the Verification and
Objective:
With use of EMMA A-SMGCS, the level of safety of airports will be maintained or even increased,
especially under adverse weather conditions and in congested traffic situations.
A full safety assessment of the system installed in Toulouse will be performed in EMMA2. A FHA and
a very preliminary safety assessment are available in D1.3.9 of EMMA.
Validation:
The controllers consider that the A-SMGCS is acceptable from the safety point of view as an
additional information source but not as a control tool.
Objective:
With use of EMMA A-SMGCS, the Human Factors situation will be improved, especially under
adverse weather conditions and in congested traffic situations.
Section 2.6.14 and 2.6.15 of the ICAO manual on A-SMGCS [15] refer to operational requirements on
the HMI and state that ‘where automation is available the automated systems should demonstrate an
acceptable level of HMI efficiency’. It is expected that the assessment of these requirements will
demonstrate that the Human Factors situation has been improved.
Save date: 2007-03-08
Public
Page 51
Version: 1.0
EMMA
Validation:
The objective has been validated.
4 Validation session – fast-time simulations
4.1 Introduction
The validation session subject of the current chapter provides a quantitative assessment of the potential
operational improvements induced by the use of A-SMGCS at Toulouse-Blagnac airport. Since the
system was not used operationally, no direct measurement of operational improvement indicators
could be performed.
Two methods were used in order to assess capacity and efficiency indicators. First, the evolution of
some indicators could directly be derived from the results of the questionnaires and the conclusions of
the use of A-SMGCS procedures, provided in §3, Validation session – shadow mode trials. For
instance, the taxi time in low visibility conditions could be directly derived from the current
measurement of the taxi time in low visibility operations, from the average time length of a position
report by a pilot, and from the fact that the controllers were ready to reduce the number of pilot reports
using A-SMGCS.
The other indicators were assessed using a very simple fast-time simulator.
Due to the fact that the capacity and efficiency indicators were not directly measured, the results given
in this section should be considered with great caution. They can only be seen as ‘potential’
improvements induced by the use of A-SMGCS. Furthermore, as the fast-time simulation only
partially reflects the complexity of traffic management on the Toulouse platform, the results provided
by the simulation should not be considered as ‘absolute’ values: only the relative evolution of an
indicator can give an indication of the corresponding operational improvement.
4.2 Data description and data collection methods
4.2.1 Raw data
The fast-time simulation has been performed with two traffic scenarios, derived from a day with high
traffic on Toulouse-Blagnac airport (25th June 2004):
• The first hour (i.e. from 07:00 to 08:00 UT) is a period of very high traffic with 14 departing and
11 arriving flights,
• The second hour (i.e. from 16:00 to 17:00 UT) is a period of intermediate traffic, with 9 departing
and 6 arriving flights.
During these hours, runways 32R and 32L were active.
Each of these traffic scenarios has been run with four different configuration files, each corresponding
to different operational conditions:
• Normal visibility operations (with or without A-SMGCS, two runways),
• Low visibility operations without A-SMGCS (one runway),
• Low visibility operations with A-SMGCS (one runway),
• Low visibility operations with A-SMGCS (two runways).
A detailed description of these four operational conditions and assumptions made for each of them is
provided in the next section.
Save date: 2007-03-08
Public
Page 52
Version: 1.0
EMMA
4.2.2 Additional data
4.2.2.1 Description of the operational conditions
4.2.2.1.1 Normal visibility operations
During normal visibility conditions, both runways can be used. Runway 32R is dedicated to
departures. There is on average a two-minute waiting time between two takeoffs. Runway 32L is
dedicated to arrivals. The time separations between two landings are dependent on the aircraft types
and ICAO separation standards. Based on observation of real traffic during peak hours, it can be
reasonably assumed that the average time separation between two landing aircraft is two minutes.
The runways are dependent. However, it should be noted that:
• there is no delay between a landing and a following takeoff;
• there is a two minutes time separation between a takeoff and the following landing.
The runway capacity is 46 movements/hour.
The average unimpeded taxi time is 4 minutes, for departures and arrivals.
4.2.2.1.2 Low visibility operations without A-SMGCS
Currently, no A-SMGCS is used in low visibility operations at Toulouse-Blagnac. Only runway 32R,
equipped with ILS cat. III, is used. For this reason, compared to normal visibility conditions, a
supplementary constraint is added: takeoffs become dependent on landings. The average time
separation is about two minutes.
In low visibility operations, only one aircraft at a time is allowed on the main taxiway, which
generates a strong constraint on the ground traffic flow.
The average unimpeded taxi time is 5 minutes, due to the fact that a pilot report is necessary at each
taxiway intersection. Between three and four taxiway intersections are crossed from the stand to the
runway threshold, or from the runway exit to the stand. The airport capacity reduces to 12 aircraft per
hour (i.e. one movement every five minutes).
The controllers make records of every vehicle on the manoeuvring area.
4.2.2.1.3 Low visibility operations with A-SMGCS (one runway)
Following the results and analysis of the questionnaires filled in by the controllers, several conclusions
can be made about the use of A-SMGCS procedures during low visibility operations:
• The controllers were not ready to trust A-SMGCS in order to reduce horizontal separations. They
considered that the rule ‘one aircraft on the main taxiway at a time’ would still be valid when
using the A-SMGCS. They would also continue to make records of every vehicle on the
manoeuvring area.
• The controllers do not demand to be able to perform multiple line-up or rolling takeoffs in low
visibility operations. They also insist on still having a pilot report at runway exit for arrivals. No
significant increase of the unique runway capacity is expected (i.e. still 12 aircraft per hour).
• However, they agree on the fact that the number of pilot position reports could be reduced. If only
one aircraft is present on the main taxiway, it could be assumed that the pilot would not be forced
to give a position report at each runway intersection. In the current scenario, it was assumed that
the taxi time (without holding time) could be reduced to 4 minutes, i.e. the estimated average
unimpeded value in normal visibility conditions.
The current visibility transition limit is 1000m. A lot of controllers claim that the limit could be
reduced because there are situations when low visibility procedures are activated while pilots can still
see the other aircraft. Some controllers suggested a reduction to 800m or 600m. It could result in a
great increase of taxiway capacity in visibility 2 conditions. However, the results of the questionnaires
are not statistically significant, so this potential improvement was not considered for the fast-time
simulations scenarios.
Save date: 2007-03-08
Public
Page 53
Version: 1.0
EMMA
4.2.2.1.4 Low visibility operations with A-SMGCS (two runways)
Several controllers suggested that during low visibility operations, both runways could be used
simultaneously, as in normal visibility operations. The reason why the second runway is not activated
in low visibility procedures is related to the absence of adapted ground lighting and stop bars. Some
controllers suggest that an A-SMGCS HMI could make up for this lack.
A safety case should be performed in order to confirm this hypothesis. However, considering the fact
that a full implementation of A-SMGCS would imply the installation of appropriate ground lighting
and stop bars, this scenario has been chosen in order to assess the potential capacity and efficiency
improvements in low visibility operations when both runways are active.
The separation rules on the runways are the same as in normal visibility operations.
The other parameters are the same as in the ‘low visibility operations with A-SMGCS (one runway)’
scenario.
4.2.2.2 Fast-time simulation assumptions
Two types of fast-time simulators have been developed, following two different types of traffic
management rules:
• The first version is based on the ‘first come, first served’ rule. If two pilots (e.g. of a departing
flight and an arriving flight) request a taxi clearance at the same time, priority is given to the one
with the earliest estimated off-block time (EOBT) for departures or estimated landing time
(ELDT) for arrivals.
• The second version gives priority to arrivals whether the conflict concerns the runway or the
request for taxi clearance.
The first version, which is more equitable, has been tested first. In low visibility operations, the results
lead to a high waiting time of arriving aircraft after landing and before taxi to stand. On ToulouseBlagnac airport it is not realistic to assume that several aircraft would be held back near runway exits
while waiting for a taxi clearance.
For this reason, it has been assumed that the controller would give priority to arriving aircraft and that
he would try to expedite the arriving traffic to the parking stands. The second version has then been
chosen for the fast-time simulations.
However, this assumption should also be qualified, because this traffic management method could
easily lead to a saturation of the apron. In the fast-time simulator, no management of stand allocation
has been performed. It is likely that the controller confronted to low visibility conditions would choose
a more subtle traffic management method, taking into account stand allocations, departures requesting
push back and arrivals requesting taxi clearance to the stands. However, this subtle management
method could not be easily simulated. This is one of the limits of these fast-time simulations.
The first simulation sessions were performed on a whole day of traffic. It appeared that during low
visibility operations, the departure and arrival delays quickly became unacceptable. It is likely that, if
low visibility conditions were to last several hours, Toulouse control would request a regulation to the
Central Flow Management Unit (CFMU). For this reason, it has been decided to focus only on one
hour of traffic for each traffic scenario. This simulates the case of a sudden reduction of visibility
conditions, when the traffic situation can only be solved tactically, using the A-SMGCS and
procedural control. This is also consistent with the fact that priority is given to arriving flights, since
they could not have been regulated earlier.
4.3 Data analysis
4.3.1 Data analysis method
A description of the fast-time simulation algorithm is provided in the D6.1.3 Toulouse V&V Test Plan
[5]. The algorithm that was described in this document was the ‘first come, first served’ algorithm.
The ‘arrivals first’ algorithm is slightly different: it systematically gives priority to arrivals, if an
arriving aircraft is about to land or request a taxi clearance.
The fast-time simulator is run:
•
For each traffic scenario described in section 4.2.1 Raw data;
Save date: 2007-03-08
Public
Page 54
Version: 1.0
EMMA
For each set of environmental conditions described in section 4.2.2.1 Description of the
operational conditions.
The values for each considered capacity and efficiency indicator are directly provided by the fast-time
simulator. A comparative analysis is then performed on these values.
For each scenario and environmental condition, the following indicators have been evaluated:
•
The departure runway throughput. It is stated in number of aircraft taking off from the airport
per hour.
•
The arrival runway throughput. It is stated in number of aircraft landing at the airport per hour.
•
The maximum number of aircraft simultaneously taxiing per hour. Both arriving and
departing aircraft have been considered. A departing aircraft is considered as taxiing between its
actual off-block time (AOBT) and actual takeoff time (ATOT – i.e. AOBT plus actual taxi-out
time). An arriving aircraft is considered as taxiing between its actual landing time (ALDT) and
actual in-block time (AIBT – i.e. ALDT plus actual taxi-in time).
•
The maximum number of aircraft simultaneously waiting in the departure queue. A
departing aircraft is considered as waiting in the departure queue when it has left the stand (i.e.
AOBT has been recorded), the average unimpeded taxi time has been exceeded and it has not
taken off yet (i.e. ATOT has not been recorded yet).
•
The maximum number of aircraft pushing back per 10 minutes time slot. A departing aircraft
is considered as pushing back in a given 10 minutes time slot if its AOBT is recorded during that
time period.
•
The minimum, mean and maximum departure taxi times. They are stated in minutes.
•
The minimum, mean and maximum arrival taxi times. They are stated in minutes.
•
The mean push back delay. The push back delay of an aircraft is calculated as the difference, in
minutes, between its Actual Off-Block Time (AOBT) and its EOBT.
•
The mean departure queuing delay. The departure queuing delay of an aircraft is calculated as
the difference, in minutes, between its actual taxi time (i.e. ATOT minus AOBT) and the average
unimpeded taxi time.
•
The mean departure delay. The departure delay of an aircraft is calculated as the difference, in
minutes, between its ATOT and its estimated takeoff time (ETOT – i.e. EOBT plus average taxiout time). This is only the ground ATC delay. No other delay has been taken into consideration.
•
The mean arrival sequencing delay. The arrival queuing delay of an aircraft is calculated as the
difference, in minutes, between its ALDT and ELDT.
•
The mean arrival queuing delay. The arrival queuing delay of an aircraft is calculated as the
difference, in minutes, between its actual taxi time (i.e. AIBT minus ALDT) and the average
unimpeded taxi time.
•
The mean arrival delay. The arrival delay of an aircraft is calculated as the difference, in
minutes, between its ALDT and ELDT. This delay is only composed of the sequencing and ground
ATC delays. No other delay has been taken into consideration.
•
The number of aircraft departing on time. A departing aircraft is considered as ‘on time’ if its
ATOT is equal to its ETOT.
•
The departure punctuality index. It is calculated as the ratio between the number of aircraft
taking off on time or with a less than 10 minutes delay and the total number of departing aircraft.
•
The number of aircraft arriving on time. An arriving aircraft is considered as ‘on time’ if its
ALDT is equal to its ELDT.
•
The arrival punctuality index. It is calculated as the ratio between the number of aircraft landing
on time or with a less than 10 minutes delay and the total number of arriving aircraft.
The following section presents the raw results of the fast-time simulation. The results of each
measured indicator are presented in tables. All the tables are organised as follows:
•
Each ‘Environmental Conditions’ column corresponds to a set of environmental condition where
NVO stands for normal visibility operations, LVO stands for low visibility operations and RWY
stands for runway.
•
The line entitled ‘High Traffic’ corresponds to the first scenario (i.e. one hour of high traffic from
07:00 to 08:00 UT on the 25th June 2004).
•
Save date: 2007-03-08
Public
Page 55
Version: 1.0
EMMA
•
The line entitled ‘Intermediate Traffic’ corresponds to the second scenario (i.e. one hour of
intermediate traffic from 16:00 to 17:00 UT on the 25th June 2004).
4.3.2 Measured indicators
4.3.2.1 General data
4.3.2.1.1 Departure and arrival runway throughput
Indicator
Scenario
NVO
2 RWYs
Environmental Conditions
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
2 DEP / 11
3 DEP / 11
ARR
ARR
LVO with ASMGCS
2 RWYs
4 DEP / 11
ARR
11 DEP / 10
Departure / Arrival
High Traffic
ARR
Runway Throughput
(in Aircraft per
Intermediate Traffic 9 DEP / 6 ARR 5 DEP / 6 ARR 5 DEP / 6 ARR 5 DEP / 6 ARR
Hour)
Table 4: Fast-Time Simulations Indicators – Departure and Arrival Runway Throughput
4.3.2.1.2 Maximum number of aircraft simultaneously taxiing
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
Indicator
Scenario
NVO
2 RWYs
Maximum Number
of Aircraft
Simultaneously
Taxiing per Hour
High Traffic
5
1
1
1
Intermediate Traffic
4
1
1
1
Table 5: Fast-Time Simulations Indicators – Maximum Number of Aircraft Simultaneously Taxiing
4.3.2.1.3 Maximum number of aircraft simultaneously waiting in the departure
queue
Indicator
Maximum Number
of Aircraft
Simultaneously
Waiting in the
Departure Queue
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
Scenario
NVO
2 RWYs
High Traffic
3
1
1
0
0
1
2
0
Table 6: Fast-Time Simulations Indicators – Maximum Number of Aircraft Simultaneously Waiting in
the Departure Queue
Save date: 2007-03-08
Public
Page 56
Version: 1.0
EMMA
4.3.2.1.4 Maximum number of aircraft pushing back per 10 minutes time slot
Indicator
Scenario
Maximum Number
High Traffic
of Aircraft Pushing
Back per 10 Minutes
Time Slot
NVO
2 RWYs
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
5
1
1
1
3
1
1
1
Table 7: Fast-Time Simulations Indicators – Maximum Number of Aircraft Pushing Back per 10 Minutes
Time Slot
4.3.2.1.5 Minimum departure taxi time
Indicator
Scenario
Minimum Departure
High Traffic
Taxi Time (in
Minutes)
NVO
2 RWYs
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
4
5
4
4
4
5
4
4
Table 8: Fast-Time Simulations Indicators – Minimum Departure Taxi Time
4.3.2.1.6 Mean departure taxi time
Indicator
Scenario
High Traffic
Mean Departure Taxi
Time (in Minutes) Intermediate Traffic
NVO
2 RWYs
6.57
4.44
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
6.07
5.29
5.11
4.67
LVO with ASMGCS
2 RWYs
4.14
4.11
Table 9: Fast-Time Simulations Indicators – Mean Departure Taxi Time
4.3.2.1.7 Maximum departure taxi time
Indicator
Scenario
Maximum Departure
High Traffic
Taxi Time (in
Minutes)
NVO
2 RWYs
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
14
20
15
5
7
6
9
5
Table 10: Fast-Time Simulations Indicators – Maximum Departure Taxi Time
4.3.2.1.8 Minimum arrival taxi time
Indicator
Scenario
NVO
2 RWYs
Minimum Arrival
Taxi Time (in
Minutes)
High Traffic
4
4
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
5
4
7
4
LVO with ASMGCS
2 RWYs
4
4
Table 11: Fast-Time Simulations Indicators – Minimum Arrival Taxi Time
Save date: 2007-03-08
Public
Page 57
Version: 1.0
EMMA
4.3.2.1.9 Mean arrival taxi time
Indicator
Scenario
NVO
2 RWYs
Mean Arrival Taxi
Time (in Minutes)
High Traffic
4
4
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
12.09
12.36
9
6.5
LVO with ASMGCS
2 RWYs
12.36
4.83
Table 12: Fast-Time Simulations Indicators – Mean Arrival Taxi Time
4.3.2.1.10 Maximum arrival taxi time
Indicator
Scenario
NVO
2 RWYs
Maximum Arrival
Taxi Time (in
Minutes)
High Traffic
4
4
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
22
20
14
12
LVO with ASMGCS
2 RWYs
20
7
Table 13: Fast-Time Simulations Indicators – Maximum Arrival Taxi Time
4.3.2.2 Departure delays
4.3.2.2.1 Mean push back delay
Indicator
Scenario
NVO
2 RWYs
Mean Push Back
Delay (in Minutes)
High Traffic
0
0
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
67.21
46.43
16.22
10.67
LVO with ASMGCS
2 RWYs
46.43
11.22
Table 14: Fast-Time Simulations Indicators – Mean Push Back Delay
4.3.2.2.2 Mean departure queuing delay
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
Indicator
Scenario
NVO
2 RWYs
Mean Departure
Queuing Delay (in
Minutes)
High Traffic
2.57
1.07
1.29
0.14
0.44
0.11
0.67
0.11
Table 15: Fast-Time Simulations Indicators – Mean Departure Queuing Delay
4.3.2.2.3 Mean departure delay
Indicator
Scenario
NVO
2 RWYs
Mean Departure
Delay (in Minutes)
High Traffic
2.57
0.44
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
68.29
47.71
16.33
11.33
LVO with ASMGCS
2 RWYs
46.57
11.33
Table 16: Fast-Time Simulations Indicators – Mean Departure Delay
Save date: 2007-03-08
Public
Page 58
Version: 1.0
EMMA
4.3.2.3 Arrival delays
4.3.2.3.1 Mean arrival sequencing delay
Indicator
Scenario
NVO
2 RWYs
Mean Arrival
High Traffic
Sequencing Delay (in
Minutes)
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
1.27
0.36
0.36
0.36
0
0
0
0
Table 17: Fast-Time Simulations Indicators – Mean Arrival Sequencing Delay
4.3.2.3.2 Mean arrival queuing delay
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
Indicator
Scenario
NVO
2 RWYs
Mean Arrival
Queuing Delay (in
Minutes)
High Traffic
0
7.09
8.36
8.36
0
4
2.50
0.83
Table 18: Fast-Time Simulations Indicators – Mean Arrival Queuing Delay
4.3.2.3.3 Mean arrival delay
Indicator
Scenario
NVO
2 RWYs
Mean Arrival Delay
(in Minutes)
High Traffic
1.27
0
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
7.45
8.73
4
2.50
LVO with ASMGCS
2 RWYs
8.73
0.83
Table 19: Fast-Time Simulations Indicators – Mean Arrival Delay
4.3.2.4 Departure punctuality
4.3.2.4.1 Number of aircraft departing on time
Indicator
Number of Aircraft
Departing On Time
vs. Total Number of
Departing Aircraft
per Hour
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
Scenario
NVO
2 RWYs
High Traffic
5/14
1/14
1/14
1/14
7/9
1/9
2/9
3/9
Table 20: Fast-Time Simulations Indicators – Number of Aircraft Departing On Time
Save date: 2007-03-08
Public
Page 59
Version: 1.0
EMMA
4.3.2.4.2 Departure punctuality index
Indicator
Scenario
NVO
2 RWYs
Departure
Punctuality Index
High Traffic
1
1
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
0.07
0.14
0.44
0.44
LVO with ASMGCS
2 RWYs
0.21
0.55
Table 21: Fast-Time Simulations Indicators – Departure Punctuality Index
4.3.2.5 Arrival punctuality
4.3.2.5.1 Number of aircraft arriving on time
Indicator
Scenario
NVO
2 RWYs
Number of Aircraft
High Traffic
Arriving On Time vs.
Total Number of
Arriving Aircraft per Intermediate Traffic
Hour
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
LVO with ASMGCS
2 RWYs
4/11
1/11
1/11
1/11
6/6
0/6
2/6
4/6
Table 22: Fast-Time Simulations Indicators – Number of Aircraft Arriving On Time
4.3.2.5.2 Arrival punctuality index
Indicator
Scenario
NVO
2 RWYs
Arrival Punctuality
Index
High Traffic
1
1
LVO without
LVO with AA-SMGCS
SMGCS
1 RWY
1 RWY
0.73
0.64
1
1
LVO with ASMGCS
2 RWYs
0.64
1
Table 23: Fast-Time Simulations Indicators – Arrival Punctuality Index
4.4 Results
This section presents the results of the validation session in terms of operational improvements. For
each stage, the results of the metrics primarily defined in the Verification and Validation Test Plan
Document [5] are detailed. The validation of the hypotheses and high-level objectives specified in that
document are derived from the measured metrics.
4.4.1.1 Metrics
The results of the individual metrics related to operational improvements listed in this section were
measured during the fast-time simulation.
The characters between parentheses in each metric title indicate the reference of the indicator used in
D6.1.3: Verification and validation test plan for Toulouse-Blagnac airport, version 1.02, issued on the
15th February 2006 [5]. These figures are unfortunately not always consistent with D6.2.2: Indicators
and Metrics for A-SMGCS [7].
Save date: 2007-03-08
Public
Page 60
Version: 1.0
EMMA
4.4.1.1.1 Capacity
4.4.1.1.1.1 Assessment of departure and arrival runway throughput in visibility conditions 2 and
3 (CA01 & CA02)
The fast-time simulation only partially reflects the complexity of traffic management on the ToulouseBlagnac platform. Therefore, the departure and runway throughput measured during the simulation
cannot be used in absolute terms.
Nevertheless, the relative evolutions of the departure and arrival runway throughputs from normal
operations to the various scenarios in low visibility conditions show that:
• The departure throughput is slightly improved (i.e. one more aircraft per hour, representing a 7%
increase) in low visibility conditions, high traffic situation and with one runway activated when
the A-SMGCS used, compared to the same situation when the A-SMGCS is not used.
• The departure throughput is slightly improved (i.e. one more aircraft per hour, representing a 7%
increase) in low visibility conditions, high traffic situation and when A-SMGCS is used with two
runways activated, compared to the same situation with only one runway active.
• The departure throughput is significantly reduced in low visibility conditions, high traffic
situation, whether the A-SMGCS is used or not and whether the number of active runways.
• The departure throughput is unchanged in low visibility conditions, intermediate traffic situation,
whether the A-SMGCS is used or not and whether the number of active runways.
• The arrival throughput is unchanged in low visibility conditions, whether the traffic situation,
whether the A-SMGCS is used or not and whether the number of active runways. The arrival
throughput is even slightly increased (i.e. one more aircraft per hour) in low visibility conditions
and high traffic situation compared to normal operations. This is due to the fact that arrivals are
given the priority in the simulator’s algorithm.
The use of the A-SMGCS may improve the runway throughput by increasing the departure runway
throughput during periods of high traffic. It seems that it will have an actual operational impact only if
both runways can be use in low visibility conditions. This is consistent with the controllers’ opinion.
See section 3.4.3.1.1.2 Assessment of departure and arrival runway throughput in visibility conditions
2 and 3 (CA01 & CA02) for further details.
It must also be noted that the gain in the departure runway throughput obtained by the use of the ASMGCS is limited compared to the dramatic decrease of the departure runway throughput in low
visibility conditions. It shows the limits of the benefits brought by the A-SMGCS when it is not used
as a replacement of the outside view.
4.4.1.1.1.2 Assessment of mean and maximum number of aircraft simultaneously taxiing in
visibility conditions 2 and 3 (CA09 & CA10)
The mean number of aircraft simultaneously taxiing in visibility conditions 2 and 3 has not been
assessed during the fast time simulation.
The relative evolutions of the maximum number of aircraft simultaneously taxiing from normal
operations to the various scenarios in low visibility conditions show that:
• The maximum number of aircraft simultaneously taxiing is strongly reduced during low visibility
conditions compared to normal operations.
• The use of the A-SMGCS does not bring any improvement in the number of aircraft
simultaneously taxiing.
These results are explained by ‘one aircraft on the main taxiway at a time’ rule, which is a strong
constraint on the ground traffic in low visibility conditions. While this rule will be applied when the
A-SMGCS is used (i.e. while the controllers’ trust in the system will be limited), the number of
aircraft simultaneously taxiing will not be increased. See section 3.4.3.1.1.3 Assessment of mean and
maximum number of aircraft simultaneously taxiing in visibility conditions 2 and 3 (CA09 & CA10)
for further details.
Save date: 2007-03-08
Public
Page 61
Version: 1.0
EMMA
4.4.1.1.1.3 Assessment of mean and maximum number of push back clearances in visibility
conditions 2 and 3 (CA07 & CA08)
The mean number of push back clearances in visibility conditions 2 and 3 has not been assessed
during the fast time simulation.
The relative evolutions of the maximum number of aircraft pushing back per 10 minutes time slot
from normal operations to the various scenarios in low visibility conditions show that:
• The maximum number of aircraft pushing back per 10 minutes time slot is strongly reduced during
low visibility conditions compared to normal operations.
• The use of the A-SMGCS does not bring any improvement in the maximum number of aircraft
pushing back per 10 minutes time slot.
The ‘one aircraft on the main taxiway at a time’ rule has a direct impact on the number of aircraft
authorised to push back. Therefore, while the rule will be applied, the number of push back clearances
will not be increased.
4.4.1.1.2.1 Taxiing delays (EF01)
The metrics entitled ‘taxiing delays’ (EF10) corresponds to the indicators ‘mean departure queuing
delay’ and ‘mean arrival queuing delay’ measured during the fast time simulation.
The relative evolutions of the mean departure and arrival queuing delays from normal operations to
the various scenarios in low visibility conditions show that:
• The mean departure queuing delay is decreased by 84 to 89 % in low visibility conditions when
the A-SMGCS is used and two runways are active, compared to the same situation with only one
runway active.
• When only one runway is active in low visibility conditions, the A-SMGCS does not help reduce
the mean departure queuing delay.
• The A-SMGCS does not help reduce the mean arrival queuing delay.
Note: when examining the mean departure and arrival queuing delays, it must be kept in mind that the
average unimpeded taxi time is 5 minutes in low visibility conditions without A-SMGCS against 4 in
all the other scenarios and environmental conditions.
The ‘one aircraft on the main taxiway at a time’ rule is a strong constraint on the ground traffic in low
visibility conditions. It is reflected in the mean departure and arrival queuing delays that should not be
improved with the use of the A-SMGCS while the rule applies, except for departures if both runways
were active.
4.4.1.1.2.2 Gate delays (EF02)
The metrics entitled ‘gate delays’ (EF02) corresponds to the indicator ‘mean push back delay’
measured during the fast time simulation.
The relative evolutions of the mean push back delay from normal operations to the various scenarios
in low visibility conditions show that:
• Important mean push back delays are observed in low visibility conditions (i.e. from 16 to 67
min.), while no push back delays were observed in normal operations.
• The use of the A-SMGCS allows the mean push back delay to be reduced by 31 to 34% in low
visibility conditions.
• The number of active runways has no impact on the mean push back delay in low visibility
conditions.
The A-SMGCS has a positive impact on the mean push back delays in low visibility conditions, even
in an intermediate traffic situation.
Save date: 2007-03-08
Public
Page 62
Version: 1.0
EMMA
4.4.1.1.2.3 Departure queuing delays (EF05)
The relative evolutions of the mean departure queuing delay from normal operations to the various
scenarios in low visibility conditions show that:
• The mean departure queuing delay is lower in low visibility conditions than during normal
operations. This is most probably explained by the ‘one aircraft on the main taxiway at a time’
rule, which limits the ground traffic in low visibility conditions while several aircraft can taxi at
the same time and then queue at the runway takeoff in normal conditions.
• The mean departure queuing delay is improved by 2 to 32 % in low visibility conditions, high
traffic situation when the A-SMGCS is used and both runways are active compared to the other
scenarios in low visibility conditions.
The A-SMGCS should not have a significant impact on the departure queuing delays in low visibility
conditions (except if both runways were used). This is mainly due to the ‘one aircraft on the main
taxiway at a time’ rule, which is a strong constraint on the ground traffic.
4.4.1.1.2.4 Arrival sequencing delays (EF06)
The relative evolutions of the mean arrival sequencing delay from normal operations to the various
scenarios in low visibility conditions show that:
• In an intermediate traffic situation, no arrival sequencing delay has been measured in any of the
scenarios and environmental conditions.
• In a high traffic situation, the mean arrival sequencing delay is less important by 72% in low
visibility conditions than during normal operations. This is most probably due to the choice made
to give priority to arriving flights in low visibility conditions.
• In a high traffic situation and low visibility conditions, the use of the A-SMGCS has no influence
on the mean arrival sequencing delay.
The A-SMGCS does not seem to improve the arrival sequencing delays in low visibility conditions.
Nevertheless, it is mitigated by the deliberate choice to run the simulation in a context where priority
is given to arriving flights. See section 4.2.2.2 Fast-time simulation assumptions for further details.
4.4.1.1.2.5 Mean departure delays (EF07 & EF204)
The relative evolutions of the mean departure delays from normal operations to the various scenarios
in low visibility conditions show that:
• The mean departure delay is strongly increased (i.e. multiplied by 18 to 37) in low visibility
conditions compared to normal operations.
• The use of the A-SMGCS improves the mean departure delay by 30 to 32 % in low visibility
conditions in both traffic situations and whether one or two runways are active.
The mean departure delay is a key metrics for evaluating the efficiency of the A-SMGCS. The results
obtained show that the A-SMGCS could help significantly reduce the departure delays in low visibility
conditions, especially during periods of high traffic. The metrics also show that the use of both
runways would not change the results. This is most probably due to the ‘one aircraft on the main
taxiway at a time’ rule, which is very limitative for the ground traffic in low visibility conditions.
It must also be noted that the gain in the mean departure delay obtained by the use of the A-SMGCS is
limited compared to the dramatic increase of the mean departure delay in low visibility conditions. It
shows the limits of the benefits brought by the A-SMGCS when it is not used as a replacement of the
outside view.
4.4.1.1.2.6 Mean arrival delays (EF08 & EF205)
The relative evolutions of the mean arrival delays from normal operations to the various scenarios in
low visibility conditions show that:
• The mean arrival delay increases in low visibility conditions compared to normal operations.
4
5
The number of communications (EF20) could not be evaluated during the validation sessions.
The number of communications (EF20) could not be evaluated during the validation sessions.
Save date: 2007-03-08
Public
Page 63
Version: 1.0
EMMA
•
The mean arrival delay increases in low visibility conditions, high traffic situation whether one or
two runways are active when the A-SMGCS is used, compared to the same situation without ASMGCS. The increase happens when aircraft have already landed (Cf. mean arrival queuing
delay). The constraint of the ‘one aircraft on the main taxiway at a time’ rule being stronger than
the benefit brought by the use of the A-SMGCS (i.e. reduction of the number of position reports
by pilots), leading to a similar taxi time with or without A-SMGCS.
• The mean arrival delay is improved in low visibility conditions, intermediate traffic situation when
one runway is active and A-SMGCS is used, compared to the same situation without A-SMGCS.
• The mean arrival delay is improved in low visibility conditions, intermediate traffic situation when
the A-SMGCS is used and both runways are active, compared to the same situation with only one
runway active.
The fast-time simulation shows that the A-SMGCS could help reduce arrival delays at least during
intermediate traffic periods. The efficiency of the system is not proven when the traffic is high.
Note: the computed mean departure and mean arrival delays are influenced by the priority given to
arrivals in the simulator's algorithm. An algorithm, based on the ‘first come, first served’ principle,
was also tested. This algorithm gives priority to the most delayed aircraft, whether it is a departure or
an arrival. The results obtained were more balanced between departure and arrival delays than those
obtained with the ‘arrivals first’ algorithm.
However, it is interesting to notice that the global reduction of delays (i.e. departure and arrival
delays) is equivalent with both algorithms. This tends to suggest that this global reduction of ground
ATC delay could be expected, whatever the traffic control method. Only the share between departure
and arrival delays would change.
4.4.1.1.2.7 Punctuality index – number of mobiles on time (EF09 & EF19)
The relative evolutions of the departure and arrival punctuality indexes from normal operations to the
various scenarios in low visibility conditions show that:
• The departure punctuality index strongly decreases in low visibility conditions compared to
normal operations.
• The departure punctuality index is improved in low visibility conditions, high traffic situation
when one runway is active and A-SMGCS is used, compared to the same situation without ASMGCS.
• Whatever the traffic situation, the departure punctuality index is improved in low visibility
conditions when the A-SMGCS is used and both runways are active, compared to the same
situation with only one active runway.
• The departure punctuality is unchanged in low visibility conditions, intermediate traffic situation
and with one active runway whether the A-SMGCS is used or not.
• The arrival punctuality is slightly decreased in low visibility conditions, high traffic situation
whether the A-SMGCS is use or not and whether one or two runways are activated, compared to
normal operations.
• The arrival punctuality is unchanged in intermediate traffic situation whatever the scenario and
environment conditions.
The A-SMGCS seems to have a positive impact on the departure punctuality index in low visibility
conditions. This is mainly perceptible during periods of high traffic and / or if both runways can be
used.
However, the gain in the departure punctuality obtained by the use of the A-SMGCS is limited
compared to the dramatic decrease of the departure punctuality in low visibility conditions. It shows
the limits of the benefits brought by the A-SMGCS when it is not used as a replacement of the outside
view.
The impact of the A-SMGCS on the arrival punctuality index is not proven, keeping in mind that it has
been deliberately chosen to give priority to arriving traffic in the simulation algorithm.
Save date: 2007-03-08
Public
Page 64
Version: 1.0
EMMA
4.4.1.1.2.8 Minimum, mean and maximum departure and arrival taxi times in visibility
conditions 2 and 3 (EF10, EF11 & EF12)
The relative evolutions of the minimum, mean and maximum departure and arrival taxi times from
normal operations to the various scenarios in low visibility conditions show that:
• In all the scenarios and environmental conditions, the minimum departure taxi time is equal to the
average unimpeded taxi time.
• Whatever the traffic situation, the mean departure taxi time is improved in low visibility
conditions when one runway is active and the A-SMGCS is used, compared to the same situation
without A-SMGCS.
• Whatever the traffic situation, the mean departure taxi time is improved in low visibility
conditions when the A-SMGCS is used and both runways are active, compared to the same
• The maximum departure taxi time is improved in low visibility conditions, high traffic situation
when one runway is active and A-SMGCS is used, compared to normal operations.
• Whatever the traffic situation, the maximum departure taxi time is improved in low visibility
conditions when the A-SMGCS is used and two runways are active, compared to the same
• The maximum departure taxi time increases in low visibility conditions, intermediate traffic
situation when one runway is active and A-SMGCS is used compared to the same situation
without A-SMGCS.
• The minimum arrival taxi time increases in low visibility conditions, intermediate traffic situation
when one runway is active and without A-SMGCS, compared to normal operations. In all the
other scenarios and environmental conditions, it is equal to the average unimpeded taxi time.
• The mean and maximum arrival taxi times strongly increase in low visibility conditions compared
to normal operations.
• In high traffic situation, the mean and maximum arrival taxi times are unchanged in low visibility
conditions whether the A-SMGCS is used or not and whatever the number of active runways.
• In intermediate traffic situation, the mean and maximum arrival taxi times are improved in low
visibility conditions when one runway is active and the A-SMGCS is used, compared to the same
situation without A-SMGCS.
• In intermediate traffic situation, the mean and maximum arrival taxi times are improved in low
visibility conditions when the A-SMGCS is used and both runways are active, compared to the
same situation with only one active runway.
The A-SMGCS may help reduce the departure taxi time in low visibility conditions, even more if both
runways may be used. However; the improvement of arrival taxi time seems to be limited to
intermediate traffic situations. These results are consistent with the controllers’ expectations. See
section 3.4.3.1.2.1 Minimum, mean and maximum departure and arrival taxi times in visibility
conditions 2 and 3 (EF10, EF11 & EF12) for further details.
4.4.1.1.2.9 Mean and maximum number of aircraft in the departure queue (EF13 & EF14)
The mean number of aircraft waiting in the departure queue has not been assessed during the fast time
simulation.
The relative evolutions of the maximum number of aircraft simultaneously waiting in the departure
queue from normal operations to the various scenarios in low visibility conditions show that:
• In high traffic situation, the maximum number of aircraft simultaneously waiting in the departure
queue is lower in low visibility conditions than in normal operations. This is most probably due to
the ‘one aircraft on the main taxiway at a time’ rule, which limits the number of aircraft on the
taxiways in low visibility conditions and consequently the number of aircraft in the departure
queue.
• In intermediate traffic situation, the maximum number of aircraft simultaneously waiting in the
departure queue does not significantly vary whatever the scenarios and environmental conditions.
Save date: 2007-03-08
Public
Page 65
Version: 1.0
EMMA
The A-SMGCS does not seem to have a significant influence on the number of aircraft in the
departure queue. The ‘one aircraft on the main taxiway at a time’ rule may bias the result for this
metrics since it strongly limits the number of aircraft on the taxiing simultaneously.
4.4.1.2 Hypotheses
In this section, the operational improvements hypotheses described in the Verification and Validation
Test Plan Document [5] are validated or invalidated according to the results of the metrics measured
4.4.1.2.1 Capacity
Hypothesis:
Using A-SMGCS, the taxiway capacity will be maintained or will increase, especially in visibility
conditions 2 and 3. The following parameters will be considered:
•
Mean number of simultaneously taxiing aircraft,
•
Maximum number of simultaneously taxiing aircraft.
Validation:
The taxiway capacity should at least be maintained.
Hypothesis:
Using A-SMGCS, the apron and gate capacity will be maintained or will increase, especially in
visibility conditions 2 and 3. The following parameters will be considered:
•
Mean number of push back clearances,
•
Maximum number of push back clearances.
Validation:
The apron and gate capacity should at least be maintained.
Hypothesis:
Using A-SMGCS, the runway departure and arrival throughput will be maintained or increased,
especially in visibility conditions 2 and 3.
Validation:
The departure runway throughput should be maintained in intermediate traffic situations and increased
in high traffic situations. The arrival runway throughput should be maintained.
Hypothesis:
The taxi time will be maintained or decreased, especially in visibility conditions 2 and 3. The
following parameters will be considered:
•
Mean departure taxi time,
•
Mean arrival taxi time,
•
Minimum and maximum taxi times.
Save date: 2007-03-08
Public
Page 66
Version: 1.0
EMMA
Validation:
The departure taxi time should be decreased. The arrival taxi time should be maintained in high traffic
situations and decreased in intermediate traffic situations.
Hypothesis:
Related to the reduction of taxi time, the delays due to ATC ground operations will be maintained or
decreased, especially in visibility conditions 2 and 3. The following delays will be considered:
•
Taxiing delays,
•
Gate delays,
•
Departure queuing delays,
•
Arrival sequencing delays,
•
Mean departure delays,
•
Mean arrival delays.
Validation:
In this section, the operational improvements high-level objectives described in the Verification and
Objective:
With use of EMMA A-SMGCS, the level of capacity of airports will be maintained or even increased,
especially under adverse weather conditions and in congested traffic situations.
Section 2.2.3 of the ICAO manual on A-SMGCS states that ‘an A-SMGCS should be capable of
operating at a specified movement rate in visibility conditions down to the aerodrome visibility
operational level (AVOL). When visibility conditions are reduced to below AVOL an A-SMGCS should
provide for a reduction of surface movements of aircraft and vehicles to a level acceptable for the new
situation’. It is expected that the assessment of this requirement will help define levels of acceptable
traffic in different visibility conditions, and that these levels will be higher than the current ones.
Validation:
The level of capacity of airports should be maintained and maybe increased but levels of acceptable
traffic in different visibility conditions could not be defined.
Objective:
With use of EMMA A-SMGCS, the efficiency of traffic movements will be increased, especially under
adverse weather conditions and in congested traffic situations.
Along with capacity assessment, it is expected to demonstrate that the efficiency will also be
increased, especially in Visibility 2 conditions.
Validation:
The objective has been validated.
Save date: 2007-03-08
Public
Page 67
Version: 1.0
EMMA
5 Digest and conclusions
Both shadow-mode trials and fast-time simulations performed during the validation sessions of the ASMGCS implemented in Toulouse-Blagnac airport permitted to evaluate the operational feasibility of
the system. The potential operational improvements brought by the A-SMGCS were also assessed
during the validation sessions. The main findings are summarised in this conclusion section.
The controllers are convinced that an A-SMGCS would be useful in Toulouse-Blagnac airport. This is
why they are supporting the operational implementation of an A-SMGCS in Toulouse-Blagnac.
The controllers are satisfied with the A-SMGCS HMI and the utility of the information displayed.
However, the controllers are expecting improvements in terms of reliability for the current installed ASMGCS to be used as a control tool and not only as an additional information source.
The controllers would be ready to adapt some procedures to take the A-SMGCS into account (e.g.
reduction of the number of position reports by the pilots in low visibility conditions) provided that
some minor technical malfunctions are solved.
The controllers seem to be conservative regarding the A-SMGCS in Toulouse-Blagnac. This is due to
both their lack of experience with this new system and its lack of technical maturity. It prevents the ASMGCS in Toulouse-Blagnac to be fully validated. However, the feelings expressed by the controllers
of Toulouse-Blagnac airport about the A-SMGCS are close to the findings of the validation sessions
performed in the other test-sites of the EMMA project. Therefore, they participate to the global
validation of the A-SMGCS concept.
The controllers are expecting the A-SMGCS to bring significant benefits in terms of safety and human
factors. In particular, the controllers are very interested in the surface conflict alert function of the ASMGCS even if they are expecting some improvements in its implementation before operational use.
The shadow-mode trials revealed that the controllers did not generally express expectations for the
potential benefits brought by the A-SMGCS in terms of capacity and efficiency. In normal visibility
conditions, no benefits can be expected from the A-SMGCS in terms of capacity and efficiency.
The fast-time simulations confirmed that the impact of low visibility conditions on a low or even an
intermediate traffic situation is rather low. Therefore, minor benefits can be expected from the ASMGCS in terms of capacity and efficiency in these situations.
The fast-time simulations showed that the impact of low visibility conditions on a heavy traffic is
much more significant. In Toulouse-Blagnac airport, periods of high traffic in low visibility conditions
are currently managed by issuing CFMU regulations and creating stacks for the incoming traffic. The
use of the A-SMGCS would help reduce delays in this kind of situation.
Many controllers considered that the improvement would be even more significant if both runways
could be used in low visibility conditions instead of one nowadays. This was confirmed by the results
of the fast-time simulations. However, such a change would necessitate strong modifications of the
infrastructures and procedures at Toulouse-Blagnac airport.
The potential benefits that the A-SMGCS would bring if it were used in operations in ToulouseBlagnac airport would remain limited while the system is seen as an additional information tool and
not as a control tool by the controllers.
In order to measure actual benefits of the A-SMGCS in low visibility conditions and high traffic
situations, a lot of effort is still needed. In particular, the reliability of the current system has to be
improved and the current procedures and working habits have to be adapted to the improved system.
However, the frequency of such traffic situations should be assessed before deciding to go a step
further in the development of the A-SMGCS in Toulouse-Blagnac.
Save date: 2007-03-08
Public
Page 68
Version: 1.0
EMMA
6 Annexes
6.1 Annex A: analysis tables for the shadow-mode validation trials
The following sections present the results of the analysis in tables ordered according to the three topics
assessed in the questionnaire: acceptance of technical performances for operational feasibility,
procedures for operational feasibility and operational improvements.
For each statement, the reference number and the English version of the statement are recapped. The
total number of answers, the average and standard deviation of the answers are presented and all the
comments made by the controllers are listed with a reference to the originator by using the coding
‘ATCOn’, where ATCO stands for air traffic controller and n is a number between 1 and 33 allowing
anonymous reference to the author of the comment. Lines with a grey background correspond to
statements where the controllers’ answers are not significant according to the P parameter of the one
sample t-test.
Save date: 2007-03-08
Public
Page 69
Version: 1.0
EMMA
6.1.1 Operational feasibility: acceptance of technical performances
6.1.1.1 Surveillance
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
34
When visual reference is not possible on the
manoeuvring area, the detection coverage
(area in which the mobiles are detected) of the
EMMA display is sufficient to control the
traffic in a safe and efficient way
•
•
28
4.36
1.31
0.002
•
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: Replace ‘manoeuvring area’ with ‘movement area’ in the statement.
Totally disagree on the movement area because the apron is missing (in order
to provide traffic information). But coverage is OK on the manoeuvring area.
ATCO8: To be confirmed (thresholds, arrival on the runway, parking stands:
OK).
ATCO9: In principle, yes.
ATCO11: I do not know.
ATCO20: If the accuracy is guaranteed.
ATCO23: Too early to take a stand.
ATCO30: Not tested.
ATCO33: Technically OK but we do not use EMMA to exercise control.
ATCO35: If all the mobile are surely detected on the global view of the
airport: yes.
Page 70
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
35
manoeuvring area, the identification coverage
(area in which the mobiles are identified) of
the EMMA display is sufficient to control the
traffic in a safe and efficient way.
•
•
25
3.96
1.46
0.110
•
•
•
•
•
•
•
•
•
•
•
36
manoeuvring area, the displayed position of a
mobile on the manoeuvring area is accurate
enough to control the traffic in a safe and
efficient way.
•
•
30
4.21
1.24
0.004
•
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO7: Not enough experience to judge.
ATCO8: The identification coverage is not OK.
ATCO9: Not possible to answer.
ATCO10: Not possible to test because no flight plan correlation for
departures. But arrivals OK.
ATCO12: Even if there are losses of identification, OK (comforted by the
pilot).
ATCO21: Not enough experience.
ATCO22: The departures are not often enough identified.
ATCO23: Not enough experience to take a stand.
ATCO29: I cannot answer.
ATCO30: Not tested.
ATCO33: We do not use EMMA to exercise control.
ATCO35: No but it is not a problem. Yes if improvements.
ATCO1: No precise knowledge.
It is not pertinent to separate the runway sensitive area and manoeuvring area
because there is no difference in the accuracy.
ATCO2: Associated with specific use instructions.
ATCO8: Pilot’s help is necessary (not 100% confidence in the system).
ATCO10: To be tested.
ATCO11: Visibly, yes.
ATCO13: Except if there is a bug!
ATCO18: Not tested enough. It should be close to a control position.
ATCO20: It is too early to answer. I do not know the accuracy of the image.
ATCO23: Cf. question on the runway vacancy.
ATCO29: In principle the ground radar is OK (the analog looks accurate). But
EMMA is not reliable enough to work with (e.g.: problems of label jumps).
ATCO34: No for vacating the runway (3). Yes for the rest of the manoeuvring
area.
ATCO35: It depends on the protection volume decided.
If inaccuracy = 15m. then strongly agree.
Page 71
Version: 1.0
EMMA
Ref.
Statement
37
The current update rate of the EMMA display
is sufficient to control the traffic in a safe and
efficient way when visual reference is not
possible on the manoeuvring area.
Tot. Nb
Average
Answers
SD
P
Comments
•
•
31
5.10
0.79
0.000
•
•
•
•
•
•
•
•
•
•
•
•
38
manoeuvring area, the current amount of
missing reports on the EMMA display is not a
problem to control the traffic in a safe and
efficient way.
•
21
2.67
1.56
0.044
•
•
•
•
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: But the latency has to be known (approx. 5m. delay). Could be better.
ATCO1: Problem with the SMR.
ATCO7: Too early.
ATCO8: Rephrase the question. If there are missing reports, it is a problem.
ATCO10: Not experienced.
ATCO12: The primary radar is infallible.
ATCO15: I did not observed the missing reports and their amount, therefore I
have difficulty in answering the question. Thus I will be number 1.
ATCO22: Lack of practical experience.
ATCO23: Too early as things stand at the moment.
ATCO29: As soon as a report is missing, it is a problem.
There is always a problem when the system is not reliable.
ATCO31: ?
ATCO33: Because we do not use EMMA to exercise control.
ATCO34: We have to be sure to see everybody.
ATCO35: Knowing the comfort it brings, I am already very happy.
Any improvement of the accuracy is positive.
Page 72
Version: 1.0
EMMA
Ref.
Statement
39
manoeuvring area, the current amount of
unidentified mobiles on the EMMA display is
not a problem to control the traffic in a safe
and efficient way.
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
29
2.86
1.53
0.080
•
•
•
•
•
40
manoeuvring area, identifying a mobile is
more efficient when using the EMMA display.
•
34
5.18
0.72
0.000
•
•
•
•
•
•
41
Even when visual reference is possible on the
manoeuvring area, identifying a mobile is
more efficient when using the EMMA display.
•
34
4.59
1.21
0.000
•
•
•
•
•
•
•
•
42
Recognition of the aircraft type is more
efficient with the EMMA display.
33
2.85
1.75
0.099
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO7: Lack of experience.
ATCO8: It is not a problem even without EMMA but it is less efficient.
ATCO23: Too early as things stand at the moment.
ATCO34: We have to be sure to see everybody.
ATCO1: Not enough trust in the system currently.
ATCO8: If EMMA works properly, yes.
ATCO22: It depends on the performance of the system.
ATCO28: If correlated, yes.
ATCO34: If we cannot see anything, EMMA is obviously more efficient.
ATCO1: When it works.
ATCO3: That depends on its position related to the tower.
ATCO19: The information provided by EMMA is a confirmation.
ATCO20: Except night time. Useful when visual reference is not possible.
ATCO22: Because we can look outside.
ATCO28: If correlated, yes.
ATCO29: In non-visible areas in particular.
ATCO32: EMMA is a good confirmation mean. OK for the non-visible areas.
ATCO35: To remove a doubt if there are several vehicles.
ATCO1: Nothing guarantees the correctness of the information in the label.
ATCO9: If the aircraft has a label and if there is no error in the flight plan.
ATCO12: I hope so. Looking forwards to improved flight plan correlation.
ATCO20: Except if the aircraft type is incorrect.
ATCO22: At night or when the aircraft is not visible.
ATCO23: I am not expecting that from EMMA.
ATCO28: If the information comes from the transponder in the aircraft.
ATCO29: Problem if the aircraft type is incorrect.
Page 73
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
43
manoeuvring area, the amount of false reports
on the EMMA display is not a problem to
control the traffic in a safe and efficient way.
•
32
2.34
1.33
0.000
•
•
•
•
•
•
•
44
manoeuvring area, the amount of incorrectly
identified mobiles on the EMMA display is
not a problem to control the traffic in a safe
and efficient way.
•
•
29
2.10
1.29
0.000
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: Lots of false reports (problems SCA and SMR).
ATCO7: If experienced.
ATCO8: False reports are a problem.
ATCO10: Depends on the safety limit.
ATCO12: It may be annoying.
ATCO20: False reports are more annoying on the runway than on the
taxiways.
ATCO29: It is an important problem.
ATCO35: Providing that the tool is known and real false reports are identified.
ATCO1: Already experienced before.
ATCO7: If experienced.
ATCO10: Problems if too many mobiles are not identified.
ATCO12: Very annoying.
ATCO30: Assessment on the ground radar. Not enough experience with
EMMA.
ATCO35: Personally not observed.
If exists: big problem.
Page 74
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
45
Very frequently, I experienced swapping of
labels between two mobiles.
•
18
3
1.19
0.072
•
•
•
•
•
•
•
•
ATCO1: Problem when two mobiles pass each other (no continuous
correlation).
ATCO5: Not tested.
ATCO9: Already experienced (not very frequently).
ATCO10: Not tested.
ATCO12: Experienced on this day only. I do not know.
ATCO18: Experienced, but not enough.
ATCO20: Already experienced but not frequently enough.
ATCO29: Already experienced on the ground radar.
It should not happen.
ATCO30: Not tested.
ATCO31: Experienced once.
ATCO34: And even dangerous ones!
Table 24: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – Surveillance
Save date: 2007-03-08
Public
Page 75
Version: 1.0
EMMA
6.1.1.2 Alerts
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
•
46
When using the EMMA display, I experienced
too many false alerts to use the conflict alert
system.
•
16
3
1.41
0.207
•
•
•
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO5: Not tested.
ATCO7: In principle interested in the alerts but not experienced.
ATCO10: False alerts not experienced.
ATCO12: Not seen.
ATCO15: Not tested.
ATCO17: No opinion.
ATCO20: To be fine-tuned.
ATCO26: Not enough experience time.
ATCO29: Not seen…
ATCO30: Not tested.
ATCO31: Not seen.
ATCO34: One observed.
Page 76
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
•
•
47
While using the EMMA display, I experienced
traffic situations when an alarm should have
been triggered but it has not been.
•
13
2,15
0,99
0,000
•
•
•
•
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: I did not experience this situation.
ATCO5: Not tested.
ATCO12: Not experienced enough.
ATCO15: Not tested.
ATCO17: No opinion.
ATCO20: Not experienced (but I have not used EMMA enough).
ATCO26: Not enough experience time.
ATCO29: Not seen…
ATCO30: Not tested.
ATCO31: Not seen.
ATCO34: Not enough experienced on EMMA to answer.
Page 77
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
•
48
The runway incursion alerts are triggered at
the right moment.
13
3
1.68
0.422
•
•
•
•
•
•
•
•
•
•
•
•
•
49
The EMMA display can help me to detect and
prevent runway incursions
•
27
4.81
0.83
0.000
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO4: Too late. They should appear when the aircraft passes the holding
point (setting up of a virtual holding point).
ATCO5: Not tested.
ATCO8: The alert is triggered too late (to be triggered at the holding point at
the latest).
ATCO9: Too late.
ATCO12: Not seen.
ATCO15: Not seen.
ATCO20: Not enough time. The experienced incursion alert is not much
relevant (arrival / departure distance too short).
ATCO22: A bit late (only one case experienced).
ATCO28: There are still errors remaining.
ATCO31: Not seen.
ATCO32: Not tested enough.
ATCO34: Alert triggered too late (it is already on the runway).
ATCO35: Not observed.
ATCO5: Not tested.
ATCO8: EMMA would be better if the alerts were triggered earlier.
ATCO10: The presentation of the alerts is not good enough.
ATCO18: In LVP, yes.
ATCO21: I hope so.
ATCO29: True even without the alerts.
ATCO31: General case.
Page 78
Version: 1.0
EMMA
Ref.
Statement
50
The EMMA display can help me to detect and
prevent incursions into restricted areas (e.g.:
ILS area in LVP).
Tot. Nb
Average
Answers
SD
P
Comments
•
•
28
4.86
0.97
0.000
•
•
•
•
•
•
51
The EMMA display can help me to detect lit
stop bar crossings.
25
5.04
1.02
0.000
•
•
•
•
•
•
•
•
•
•
•
•
•
52
Information alerts are often popping up too
late to take an appropriate action before an
alarm is triggered.
•
13
4.23
0.83
0.018
•
•
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO21: I hope so.
ATCO30: A bit thin.
ATCO1: Because the obstacle free zone depends on the stop bars in LVP.
ATCO19: And especially to know which stop bar has been crossed.
ATCO22: EMMA helps only.
ATCO29: Yes for permanent stop bars not for remote controlled ones.
ATCO30: Not tested.
ATCO4: Especially for runway incursions.
ATCO5: ?
ATCO7: In principle interested in alerts but not experienced.
ATCO8: Not seen.
ATCO10: Not tested.
ATCO12: To be confirmed after it has been used.
ATCO14: For an opposite runway heading.
ATCO19: In case of opposite runway heading, an alert should be triggered as
soon as the aircraft is on taxiway P20 if it taxis to runway 32R and if the
runway in use is 14L.
ATCO20: Problems of setting: late short final alert.
ATCO30: Not tested.
ATCO31: Not seen.
Page 79
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
•
•
53
Too many unnecessary information alerts are
triggered.
•
9
3.11
1.27
0.460
•
•
•
•
•
•
•
•
•
ATCO1: I did not observe any information alert.
ATCO5: ?
ATCO8: Not seen.
ATCO10: Not tested.
ATCO15: Not seen.
ATCO19: I would like to be allowed to acknowledge manually some of the
alarms.
ATCO30: Not tested.
ATCO31: Not seen.
ATCO32: No direct practical experience.
Table 25: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – Alerts
6.1.1.3 HMI – efficiency
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
65
The image scaling function (zoom in / zoom
out) is easy to use.
•
32
4.69
0.82
0.000
•
•
•
Save date: 2007-03-08
Public
ATCO1: The HMI is the same as the SMR one.
ATCO7: Problem to find the zoom function.
ATCO8: Once one has become used to this function, OK.
ATCO24: Not really tested.
ATCO28: Yes but the pitch is too high!
ATCO29: A priori efficient HMI that can surely be improved after operational
use.
Page 80
Version: 1.0
EMMA
Ref.
66
Statement
It is easy to display several windows.
Tot. Nb
Average
Answers
32
5
SD
P
0.67
0.000
Comments
•
•
•
•
•
•
•
•
67
The pop-up windows appear at the expected
place and size.
26
4.19
1.02
0.002
•
•
•
•
•
•
•
•
•
•
•
68
The windows are conveniently arranged on the
EMMA display.
•
28
4.46
0.96
0.000
•
•
•
•
Save date: 2007-03-08
Public
ATCO5: To be improved.
ATCO7: Not experimented.
ATCO14: Text is too small in the alert windows.
ATCO15: The alert windows are too small.
ATCO19: The alert windows are a bit too big.
ATCO22: The alert texts should be bigger.
ATCO29: Except the size of the alert windows.
ATCO32: Size is OK. Place is OK, especially because the preferred place may
be chosen the first time the windows appear.
ATCO33: Not tested so much.
ATCO34: The alert does not catch the attention enough.
ATCO9: The windows are neither at the right place nor at the right size.
ATCO10: There is too much overlap between the windows. Could be better.
ATCO14: Is not it customisable?
ATCO29: Anyway the HMI is customisable.
ATCO34: I believe that the air window cannot replace the IRMA display at the
ground position.
Page 81
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
69
Pop-up windows sometimes obscure aircraft
that should be visible.
•
20
4.2
1.44
0.119
•
•
•
•
•
•
•
•
•
•
70
The manual modification of the content of
mobile reports on the EMMA display can be
performed quickly and efficiently.
30
2.93
1.55
0.087
•
•
•
•
•
•
71
The EMMA display layout is easy to
customise to my own preferences.
30
4.5
0.82
0.000
•
•
•
Save date: 2007-03-08
Public
ATCO12: Not seen.
ATCO20: Not seen.
ATCO23: See user rules.
ATCO28: The aircraft labels are too big. Smaller characters would be better.
ATCO29: An area should be dedicated to alerts / information.
ATCO30: Not tested.
ATCO1: This function is useless because the menu follows the moving mobile.
ATCO9: I did not manage to modify the content.
ATCO19: It would be easier to click on the labels instead of the position
reports.
ATCO29: You have to click on the position report while you have to click on
the label in IRMA 2000.
The buffer zone (where you remain in the menu even if the cursor moves) is not
wild enough.
ATCO35: It at least has the merit to exist but it is a bit heavy.
ATCO10: Not tested.
ATCO19: A reset function of the controller preferences is missing.
ATCO25: Not needed.
ATCO29: It is a bit long to customise the HMI. Preset customised
configurations should be implemented.
Page 82
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
72
Too much interaction with the EMMA display
is needed.
28
2.71
0.90
0.000
•
•
•
•
ATCO8: I do not know (in principle no but to be checked).
ATCO9: If EMMA has been initially set up correctly, no.
ATCO10: A lot of interaction is need.
ATCO11: Not enough time.
ATCO14: There is no other solution (as in IRMA).
ATCO21: In principle no but lack of practical experience.
ATCO29: In principle no. The HMI is a bit long to customise but OK once it
has been adjusted (except if identification problems occur).
Table 26: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – HMI – Efficiency
6.1.1.4 HMI – usefulness
Ref.
73
74
Statement
It is useful to represent the closed taxiways,
runways and apron areas on the EMMA
display.
The actions that can be performed on the
windows (display / hide, manipulation,
scaling) are relevant.
Tot. Nb
Average
Answers
SD
P
32
4.94
1.24
0.000
32
4.69
0.69
0.000
Comments
•
ATCO30: Problem: it has to be kept up to date.
•
•
ATCO7: Little practical experience.
ATCO9: There should be less choice left (preset windows).
ATCO1: Because the automatic anti-overlapping does not work.
ATCO7: In principle.
ATCO14: Could it be automatic?
ATCO20: We may not have time to use the manual anti-overlapping.
ATCO28: I would have preferred an automatic change…
ATCO29: It should be used as few as possible.
ATCO35: It is a positive thing.
•
ATCO30: Not tested.
•
•
•
•
75
The manual anti-overlapping of labels
(possibility to change the label position
manually) is useful.
•
32
4.53
1.14
0.000
•
•
•
76
The use of colour characterisation for the
different types of mobiles is useful.
Save date: 2007-03-08
32
5.06
0.76
0.000
Public
Page 83
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
77
The raw video display (provided by the
primary surface radar) is useful.
•
30
5.1
0.88
0.000
•
•
•
•
•
•
•
•
78
The
function
allowing
the
manual
modification of label content is useful.
•
30
4.7
1.06
0.000
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: Support of the information.
ATCO3: In order to know that a parking stand is occupied.
ATCO5: In case of failure of the secondary radar yes, but not useful otherwise.
ATCO9: Ideally no (if everybody is identified).
ATCO20: Good backup in case of failure but not useful in normal operations.
ATCO22: Until a better solution is implemented.
ATCO24: Too rough.
ATCO1: But the function is not implemented correctly.
ATCO8: The modification should be automatic.
ATCO14: May be useful but not handy at all.
ATCO15: Useful but not easy to use.
ATCO20: Yes if exists.
ATCO26: Not easy to use.
ATCO27: Answer between 3 and 4.
ATCO28: Yes but beware of erroneous identifications. To be handled with
care.
ATCO30: Useful if everything is not identified. But it is heavy to use.
Page 84
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
•
•
•
•
•
•
•
•
79
The EMMA display gives me information that
I missed before. If yes, which ones?
•
27
5.04
1.06
0.000
•
•
•
•
•
•
•
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: Flight plan information and parking stands.
ATCO3: Identification.
ATCO4: SID. Type.
ATCO5: A lot of information.
ATCO7: Callsign, position.
ATCO8: CANARI, flight plan information.
ATCO9: Labelled mobiles, dynamic airfield map (+ flight plan when the
information is available).
ATCO10: Not seen.
ATCO12: Strip information.
ATCO13: Identification.
ATCO15: Conflict alert.
ATCO16: Hidden parking stands.
ATCO18: Information in the labels.
ATCO19: No but EMMA links information coming from different sources.
ATCO20: Better identification of the vehicles.
ATCO21: Flight plan correlation.
ATCO22: Alarms, helicopter area, parking stands F.
ATCO23: The point is to gather scattered information together and present it
in an ergonomic way.
ATCO24: Critical / sensitive areas, etc.
ATCO26: A lot!
ATCO27: Before, we did not have anything!!!
ATCO28: Obvious answer in LVP conditions!!!
ATCO29: Information in labels and alerts.
Equipped vehicles (but it is already available on the ground radar).
ATCO30: Note tested.
ATCO32: Information is already available somewhere else (strip).
ATCO33: Alerts (e.g.: runway incursion, stop bar crossing).
ATCO34: Closed taxiways and runways.
Alerts.
ATCO35: Already an additional safety net.
Accuracy of the information at t time, without request.
Page 85
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
80
The EMMA display contains too much
information that I regard as not useful.
31
2.29
0.90
0.000
•
•
•
•
•
•
•
81
The EMMA display provides the right
information at the right time.
27
4.58
0.80
0.000
•
•
•
•
•
ATCO9: Except perhaps the EMMATT label.
ATCO12: To be confirmed.
ATCO14: One can choose to keep only what one regards as to be useful.
ATCO23: Selective choice + profile reset function.
ATCO28: The vehicle information in the apron is not useful because it is not
usable (image polluted by tugs, etc.).
ATCO30: Not tested.
ATCO34: Fix runways.
Mobiles around the aircraft (refuellers, etc.).
ATCO35: Except some two or three fix reference reports.
ATCO1: Problems of latency before identification (up to 2 min. 37 sec. this
day) and loss of identification.
ATCO7: To be checked.
ATCO8: In the end, yes.
ATCO12: Yes, when it works…
ATCO30: Not tested.
ATCO34: E.g.: the alert is triggered too late.
ATCO35: I hope so.
According to what I have observed: yes.
Table 27: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – HMI – Usefulness
Save date: 2007-03-08
Public
Page 86
Version: 1.0
EMMA
6.1.1.5 HMI – usability
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
82
The EMMA display gives me sufficient
information about airborne traffic in the
vicinity of the airport.
•
29
4.24
1.24
0.002
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO7: E.g.: no information about a tail wind VFR.
ATCO14: Except for short finals runway and FATO.
ATCO16: Not useful.
ATCO17: Useless.
ATCO18: It is not the purpose of EMMA.
ATCO19: To be used on the ground.
ATCO22: But not the appropriate way (e.g.: colours are missing in the
approach window).
ATCO29: There is even too much information (e.g.: information about transit
flights).
ATCO30: The arrivals are visible.
ATCO33: The approach is not that useful.
ATCO34: No colours.
A background is missing on the map.
Page 87
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
•
•
•
83
Some important functions are missing on the
EMMA display. If yes, which ones?
23
2.48
0.95
0.000
•
•
•
•
•
•
84
The colours used in the EMMA display are
appropriate.
30
4.8
0.76
0.000
85
The contrast between the windows and their
background is sufficient.
31
4.90
0.54
0.000
Save date: 2007-03-08
Public
•
ATCO1: The type of information is OK but MOSQUITO is missing.
ATCO8: It would be OK if each flight was identified and correlated to its flight
plan.
ATCO9: No for the time being.
ATCO12: To be checked in a few months.
ATCO14: The aircraft type.
ATCO19: Information about the closed taxiways.
Runway in use.
ATCO22: In principle no.
ATCO23: ?
ATCO26: TOTAL identification of all the mobiles.
The automatic tracking of a selected aircraft in a dedicated window.
ATCO28: Being able to delete the third line of the departing aircraft
(clearance) while keeping the aircraft type.
ATCO29: A link to dynamic maps (dynamic information on closed runways
and taxiways).
A direct display of the ground lighting, or even a remote control of the ground
lighting from EMMA.
The goniometric information.
The identification feedback (sending of identification by the pilot in reply to a
controller request).
ATCO34: No for the information (yellow).
Perhaps too many colours.
Why not making the alarms flash?
Page 88
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
86
The height and width of the characters on the
EMMA display are sufficient.
30
4.63
0.89
0.000
•
•
•
•
•
•
•
•
•
•
87
All the texts in the EMMA display are easy to
read.
32
4.19
1.15
0.001
•
•
•
•
•
•
•
88
Labels, signs and symbols in the EMMA
display are easy to interpret.
89
Information is conveniently arranged in the
EMMA display.
32
31
4.75
4.81
0.76
0.65
0.000
0.000
•
ATCO18: Except for the ‘Ge SF’ button.
•
ATCO10: Except for the alert window.
ATCO11: A bit messy.
ATCO19: Except for the runway in use for example. True in most cases.
ATCO34: Except label and window overlapping.
•
•
•
Save date: 2007-03-08
Public
ATCO12: Smaller police would be as convenient.
Labels OK. Windows NO.
ATCO22: Texts should be bigger (alarm texts mainly).
ATCO23: Except for the alert windows.
ATCO27: Almost too big.
ATCO28: Divide at least by two the height of the characters in the aircraft
labels.
ATCO29: A bit big (labels).
ATCO1: The alerts are too small.
ATCO4: Alert window.
ATCO7: The names of the taxiways are small in large-scale views but OK if
zoom.
ATCO11: The names of the taxiways are not easy to read.
ATCO13: Yes except for the alarms.
ATCO14: Not the alert windows. Not the pull-down menus.
ATCO21: The names of the taxiways are too small.
ATCO22: The texts should be bigger (especially the alarm ones).
ATCO34: Alert window.
ATCO35: Alert windows to make bigger.
Perhaps make text in the windows thicker.
Page 89
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
90
The EMMA display enables to detect a loss of
accuracy of the surveillance.
•
3
2
0
N/A
•
•
•
•
91
Symbols can easily be read under different
angles of view in the EMMA display.
29
4.76
0.64
0.000
•
•
•
•
•
92
The EMMA display size is appropriate for
daily work.
•
31
4.65
0.88
0.000
•
•
•
ATCO5: What is this gibberish (what is the meaning)?
ATCO7: No information provided on the loss of accuracy.
ATCO8: Unclear.
ATCO10: I do not know. I do not understand the question.
ATCO12: I do not understand the statement.
ATCO11: It is better to stand in front of the screen.
ATCO13: Not applicable.
ATCO1: A bigger screen is needed.
ATCO5: A slightly bigger screen would be great.
ATCO7: I would like a slightly bigger screen.
ATCO8: Bigger, if possible (but that feats in the working positions).
ATCO10: A bigger screen would be good but it has to feat in the working
position.
ATCO21: It depends on where it is. This size means that it has to be close to
the controller.
ATCO22: A bit too big to be integrated easily.
Table 28: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Acceptance of Technical Performance – HMI – Usability
Save date: 2007-03-08
Public
Page 90
Version: 1.0
EMMA
6.1.2 Operational feasibility: procedures
6.1.2.1 Start-up clearance
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
01
The EMMA display allows me to set up a
more efficient start up sequence when visual
reference is possible on the manoeuvring area
and the apron.
•
35
3.23
1.50
0.291
•
•
•
•
•
02
The EMMA display allows me to set up a
more efficient start up sequence when visual
reference is not possible on the manoeuvring
area and the apron.
•
•
35
3.57
1.74
0.438
•
•
•
•
ATCO1: EMMA is not of any use for the start up.
ATCO7: Yes if the apron is visible.
ATCO8: EMMA has no influence on the start up sequence. But it is useful for
push back approvals.
ATCO17: Excellent plan.
ATCO19: EMMA is a confirmation.
ATCO23: General remark: I assume that the system has been validated and it
assumed to be in operations a t time for the whole questionnaire.
After the interview it is clear that for the moment EMMA is not reliable enough
to be operational.
ATCO24: General remark: as EMMA has been installed on a remote position,
it has been tested just a little bit. In the end, excellent tool.
ATCO29: There is no added value compared with the ground radar. It is easier
to set up a start up sequence with EMMA but it is not more efficient.
ATCO31: EMMA does not enable that.
The map in EMMA enables that.
ATCO1: There is no need to have a view of the airfield for the start up.
ATCO8: EMMA has no influence on the start up sequence. But it is useful for
push back approvals.
ATCO19: EMMA is a confirmation.
ATCO20: There is no need for the view.
ATCO22: EMMA is not useful to set up a start up clearance.
ATCO29: There is no added value compared with the ground radar. It is easier
to set up a start up sequence with EMMA but it is not more efficient.
ATCO31: There is no correlation at the parking stands but there is a map of
the parking stands. It prevents from having scattered tools.
ATCO35: In LVP if the sequence is long, if need be…
Table 29: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Start Up Clearance
Save date: 2007-03-08
Public
Page 91
Version: 1.0
EMMA
6.1.2.2 Push back clearance
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
03
The traffic information on the EMMA display
helps me to decide whether a push back
clearance should be delayed.
•
35
5.03
1.15
0.000
•
•
•
•
•
•
•
•
04
The EMMA display is sufficient to determine
that an aircraft is on the stand or has left the
stand.
•
33
4.30
1.26
0.000
•
•
•
•
•
•
•
ATCO1: Yes if the mobiles are identified on the taxiways.
ATCO7: Yes if the apron is visible.
ATCO12: In theory we do not have to exercise control on the apron.
ATCO18: In particular on parking stands F and E10/E20.
ATCO19: EMMA helps.
ATCO21: In principle yes, if the system is reliable.
ATCO31: EMMA helps but it must not be use systematically.
OK for the arrivals but not for the departures because they are not correlated.
ATCO35: Only on hidden parts of the manoeuvring area.
ATCO1: Yes to determine that an aircraft has left the parking stand, no to
determine that it is at the parking stand.
ATCO2: It depends on the accuracy. Is if it is accurate.
ATCO7: I do not trust it enough for the time being, to be discussed (if trust,
yes).
ATCO8: Yes if it is identified.
ATCO18: In particular on parking stands F, E10/E20, G and in the CEV area.
ATCO19: EMMA is not sufficient.
ATCO20: Yes to know that it has left the parking stand.
ATCO21: The external view is needed.
ATCO29: That depends on the accuracy of the system.
ATCO31: EMMA helps but it is not reliable enough.
DISCUSS and EMMA are complementary.
ATCO32: EMMA is not sufficient; there is a need for the external view or the
contact with the pilot.
ATCO35: It is a confirmation.
Table 30: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Push Back Clearance
Save date: 2007-03-08
Public
Page 92
Version: 1.0
EMMA
6.1.2.3 Taxi clearance
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
•
•
•
05
When I want to give a taxi clearance, I can
rely on the information provided by the
EMMA display, even when visual reference is
not possible on the manoeuvring area.
•
•
31
4.68
1.11
0.000
•
•
•
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: Yes if there is a (continuous) identification.
ATCO2: Depends on the accuracy.
ATCO7: Yes if collective trust. Yes if put in operations.
ATCO8: In the end, it would be good (especially on the Fox area, which is not
visible from the tower).
ATCO11: If EMMA is reliable.
ATCO12: Let’s be careful.
ATCO14: If the system is validated.
ATCO15: If the system is validated.
ATCO18: Yes if there are no false reports and all the aircraft have a label.
ATCO19: On moving aircraft.
ATCO21: If the system is reliable.
ATCO22: Depends on the reliability of the system.
ATCO23: If operational.
ATCO28: To be checked.
ATCO29: Depends on the accuracy of the system.
In principle yes, if it works.
ATCO30: We hope so…
ATCO31: I do not know if I can trust the tool, so today no.
It is not the most convenient tool to manage the apron (especially in non visible
areas like in F, E10, E20, E21…).
ATCO32: And by asking for a confirmation to the pilot or by informing him
that we do not know all the traffic.
ATCO34: Yes if it is reliable (e.g.: label swapping…).
ATCO35: Yes if validated!
Page 93
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
06
Surveillance of longitudinal spacing between
aircraft on taxiways is easier with the EMMA
display when visual reference is not possible
on the manoeuvring area.
•
•
35
4.6
1.46
0.000
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: No because the aircraft size and position of the synthetic plot against
the aircraft are unknown.
ATCO5: Is there a definition of longitudinal spacing on taxiways in the official
regulation? I do not think so…
ATCO8: No without any identification or correlation with the flight plan.
ATCO12: It depends on the standards.
ATCO19: Easier.
ATCO20: The actual position of the aircraft is not guaranteed.
ATCO29: It is done more through the use of the raw video than EMMA.
EMMA does not bring anything more.
ATCO31: Yes, even without correlation.
ATCO32: EMMA is not enough, visual confirmation of the pilot is needed.
To be confirmed following practical experience and once trust in EMMA has
been reached.
ATCO34: Yes if reliable.
ATCO35: It depends on the spacing margin e.g. between two taxiways.
Page 94
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
07
manoeuvring area, longitudinal spacing
between aircraft on taxiways could be reduced
with the EMMA display.
•
32
3.72
1.61
0.206
•
•
•
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: No because the aircraft size and position of the synthetic plot against
the aircraft are unknown.
ATCO5: This is not the purpose of the ground radar! I will not ask a pilot to
follow another aircraft close behind just because I have a ground radar.
ATCO8: If the aircraft are identified.
ATCO11: OK if EMMA is reliable.
ATCO20: The actual position of the aircraft is not guaranteed. The size of the
aircraft is not indicated in EMMA.
To be confirmed.
ATCO25: ? No spacing standard.
ATCO28: Not reliable when taxiing near the radar.
ATCO29: Standard to be defined.
ATCO30: It depends on the official regulation.
ATCO31: There is no problem with longitudinal spacing in LVP because most
of the time the pilots can see each other. Anyway, there is no need to make
them queue on taxiways since they will have to wait at the holding point.
ATCO32: EMMA is not enough, visual confirmation of the pilot is needed.
To be confirmed following practical experience and once trust in EMMA has
been reached.
ATCO33: We do not perform any separation during the taxi phase.
ATCO34: Precautions taken in LVP must not be reduced by this tool.
Page 95
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
•
•
08
With the EMMA display, it is no longer
necessary to make records of the vehicles on
•
35
2.8
1.45
0.013
•
•
•
•
•
•
•
•
ATCO1: Yes if all the vehicles are equipped with MOSQUITO.
ATCO4: If all the vehicles are equipped.
ATCO5: Good question! I do not have any opinion yet…
ATCO8: OK if there is only one vehicle but no if there are several ones.
ATCO10: In case of failure of EMMA, there may be a problem.
ATCO11: It is safer to keep the list.
ATCO18: It depends on each controller. Some use a strip to list all the vehicles
present. EMMA is not sufficient.
ATCO19: Problem of trust in the HMI: need for a paper backup in case of
failure or loss of correlation or detection of the vehicles.
ATCO22: If not all the vehicles are equipped, a list is needed.
ATCO27: Without any technical failure!
ATCO29: Lack of reliability of the system and trust in the system.
Problem in case of failure / missing update / additional information on the
paper strip not included in the system.
ATCO30: Depends on the reliability.
ATCO31: If the system is reliable and if there is a correlation with the flight
plan, OK But in case of failure an emergency solution is needed (e.g.: include
the vehicles in DISCUSS).
ATCO32: If the system is reliable for the identification.
ATCO34: Precautions taken in LVP must not be reduced by this tool.
ATCO35: It is the same as for DISCUSS.
Table 31: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Taxi Clearance
Save date: 2007-03-08
Public
Page 96
Version: 1.0
EMMA
6.1.2.4 Co-ordination
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
09
Coordination between the control working
positions in the tower would be more efficient
with the EMMA display.
35
4.97
0.95
0.000
•
•
•
•
10
With the EMMA display, hand over of a
taxiing aircraft between the different control
working positions would be more efficient.
•
35
4.89
1.05
0.000
•
•
ATCO1: The ground and tower working positions should be equipped.
ATCO5: Including for the evaluation of the taxi phase and departure
sequencing.
ATCO8: No observed cases for the moment. It would be useful in LVP mainly.
ATCO19: Confirmation of the sequence of taxiing aircraft, if not activated in
the right sequence. E.g.: an aircraft leaving from B1 will be number one at the
holding point even if the taxi clearance has been delivered to an aircraft in B2
first.
ATCO21: If the tower controller has an EMMA display too.
ATCO29: It must be kept in mind not to simplify too much the communications
for the coordination, but OK.
ATCO31: If the traffic is correctly seen, there is no need for any coordination.
EMMA would enable more tacit coordination.
ATCO33: E.g.: for the vehicles.
ATCO1: Replace ‘flight’ by ‘taxiing aircraft’ in the statement.
ATCO5: In order to know the situation of the departures, an A-SMGCS display
should be implemented at the approach working position.
ATCO22: EMMA helps to be more confident about the position.
Table 32: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Coordination
Save date: 2007-03-08
Public
Page 97
Version: 1.0
EMMA
6.1.2.5 Taxiing on the runway
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
11
The number of position reports can be reduced
when using the EMMA display (e.g.: to make
sure that an aircraft has vacated the runway in
use).
•
•
34
4.62
1.52
0.000
•
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: Yes with an accuracy tolerance margin on the runway (not necessary
at the parking stand).
ATCO8: Especially in LVP.
ATCO11: The confirmation of the pilot is needed.
ATCO18: If there is no bug and if I can trust the system.
ATCO20: The exact position of the aircraft is not known precisely. It depends
on the position of the aircraft on the airfield.
ATCO22: Not for the given example, but OK for other ones.
ATCO28: No in the particular case of an aircraft vacating the runway in use.
ATCO29: OK to reduce the number of reports on the taxiways but not on the
runways in use.
ATCO30: It helps but it does not replace pilot reports.
ATCO31: The example is not appropriate because the pilot has to report in
that particular case.
ATCO32: OK for the reports on taxiways, but the pilot confirmation is needed
in the case of the runway.
ATCO33: The example is not appropriate because the controller has to contact
the pilot to instruct him to change of frequency.
ATCO34: Are we sure that the runway has been actually vacated?
Page 98
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
12
With the EMMA display, an aircraft may be
permitted to taxi on the runway in use even
when visual reference on the manoeuvring
area is not possible.
•
•
31
4.68
1.22
0.000
•
•
•
•
•
ATCO2: Depends on the procedures.
ATCO5: To be considered…
ATCO7: Ambiguous question: EMMA or not, there is no difference.
ATCO8: Unclear.
ATCO19: Not relevant in Blagnac if all the taxiways are opened. Useful if M11
is not accessible, EMMA could allow the aircraft to taxi on the runway to the
threshold from M10.
ATCO20: This is already in use but EMMA brings more comfort.
ATCO22: Yes if the system is reliable.
ATCO27: The runway is under the responsibility of the tower controller.
ATCO29: The ground radar may help, it is easier with EMMA but the tool is
not necessary.
ATCO31: It is not relevant in Blagnac because the whole runway is used in
LVP.
But EMMA would help to increase the capacity in LVP by instructing the
aircraft to vacate the runway earlier and by using runway 14L.
ATCO32: E.g.: taxi on runway 32R from taxiway N2 to the threshold while an
aircraft is at the holding point in N1.
ATCO34: In LVP, no taxi on the runway in use is a vital precaution.
Table 33: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Taxiing on the Runway
6.1.2.6 Line-up clearance
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
13
Even when visual reference on the
manoeuvring area is possible, line-up of an
aircraft at the runway threshold is easier to
control with the EMMA display.
•
34
4.68
1.49
0.000
•
•
•
Save date: 2007-03-08
Public
ATCO1: Yes because the parallax is disturbing (e.g.: holding point at runway
32R is hardly visible from the tower).
ATCO5: It is better to look outside in good visibility conditions!!!
ATCO8: Yes because, for example, the runway 32R threshold is hard to see at
nightfall.
ATCO9: Yes, in 32.
ATCO11: E.g.: at night.
Page 99
Version: 1.0
EMMA
Ref.
Statement
14
When the runway threshold is not visible lineup of an aircraft is easier to control with the
EMMA display.
Tot. Nb
Average
Answers
SD
P
Comments
•
•
34
5.41
0.82
0.000
•
•
•
•
•
•
•
•
•
•
15
Even when visual reference is not possible,
multiple line-ups could be performed with the
EMMA display.
32
3.59
1.72
0.379
•
•
•
•
•
•
•
•
•
•
ATCO5: In order to know where it is in the takeoff phase.
ATCO8: Yes because, for example, the runway 32R threshold is hard to see at
nightfall.
ATCO9: Yes, on every runway.
ATCO34: Does not replace the confirmation of the pilot.
ATCO1: Yes if the distance between the taxiway is long enough (e.g.: 500m on
14R: OK; 150m on 32L: NOK).
ATCO5: I believe that such a use of the radar should not be promoted.
ATCO8: I do not know. No specific case in mind. EMMA may help but it is
tricky.
ATCO10: It is risky. To be considered.
ATCO14: I do not usually do multiple line-up in LVP.
ATCO15: I do not do multiple line-up in LVP.
ATCO18: Technically OK but I would not do it for safety reasons. It would
necessitate an appropriate ground lighting (i.e. not all the runway entry points
are marked out today). It also raises procedure issues.
ATCO19: Considering today’s operational rules.
ATCO20: The tool is not accurate enough (accuracy of the position).
ATCO22: It is not possible from the safety point of view.
ATCO23: Double-check with phraseology.
ATCO26: In accordance with official rules.
ATCO29: It depends on the reliability of the system: if the system is reliable,
OK. But today it is not feasible because there are only two stop bars.
ATCO31: I can do it but it is not very interesting. Better not to do when the
weather is not good. Not used often here.
It depends on the working methods. If the pilots can see each other, OK.
ATCO33: It is better to do multiple line-up if the weather conditions are good.
ATCO34: Surely not.
ATCO35: Reactionary again… Old reflex.
Table 34: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Line-Up Clearance
Save date: 2007-03-08
Public
Page 100
Version: 1.0
EMMA
6.1.2.7 Takeoff clearance
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
16
The EMMA display is helpful for better
monitoring an aircraft commencing the takeoff
roll.
•
33
4.27
1.35
0.003
•
•
•
•
•
•
•
•
•
•
•
17
I can give rolling takeoff clearances more
frequently with the EMMA display.
•
34
3.47
1.40
0.898
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: Yes, at the beginning of the rolling for takeoff.
ATCO11: EMMA is mainly a ground tool.
ATCO12: But not decisive.
ATCO18: No if the weather is good. Yes in LVP.
ATCO22: Because we look outside.
ATCO29: The ground radar is enough, EMMA does not bring anything more.
In case of unauthorised takeoff, the alert system helps a lot.
ATCO33: I would rather look outside.
ATCO1: The exact distance of the closer approaching aircraft to the runway
threshold has to be known.
ATCO8: EMMA is not decisive in that particular case.
ATCO10: Because it depends on the arrivals.
ATCO12: But not only EMMA.
ATCO18: EMMA may help (in addition to the camera and external view).
ATCO20: It is not the purpose of EMMA.
ATCO29: Rolling takeoff clearances can be delivered more easily (thanks to
the identification function) but not more frequently.
The criteria for a rolling takeoff are unchanged with EMMA.
ATCO31: Not relevant.
ATCO32: EMMA is not that helpful for the rolling takeoff clearance (in
principle when a controller gives a rolling takeoff clearance it implies that the
pilot has to hurry because of the incoming traffic).
ATCO33: There is no link with EMMA.
ATCO34: On runway 32 we can see that it is near the holding point (not
related to the view of the arrival).
ATCO35: We can do it without EMMA.
Page 101
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
18
The EMMA display is sufficient to decide
when a takeoff clearance should be issued.
•
35
2.8
1.32
0.008
•
•
•
•
•
ATCO2: Depends on the situation.
ATCO8: In the case separation with approaching traffic has to be performed,
additional information is needed.
ATCO14: About ground constraints only.
ATCO18: In good weather conditions EMMA is not used. Yes in LVP.
ATCO20: Not sure (yes and no). We have to trust the pilot.
ATCO29: EMMA is not decisive, does not bring anything in comparison to the
ground radar.
ATCO30: EMMA is not enough.
ATCO31: EMMA helps but it is not enough (one of the parameters considered
to give the clearance).
ATCO32: The takeoff clearance also depends on the final approaches on the
other runway. Therefore, EMMA cannot be the only tool used.
ATCO35: We can do it without EMMA.
Table 35: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Takeoff Clearance
Save date: 2007-03-08
Public
Page 102
Version: 1.0
EMMA
6.1.2.8 Landing clearance
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
19
manoeuvring area, The EMMA display is
sufficient to decide if the runway in use has
been vacated in order to deliver a landing
clearance.
•
•
34
3.74
1.76
0.413
•
•
•
•
•
•
•
ATCO1: Depends on the accuracy tolerance margin.
ATCO5: Yes if the ground radar is validated as reliable.
ATCO8: In LVP the pilot confirmation is needed.
ATCO17: Plus pilot confirmation.
ATCO20: The accuracy of the system is not known (position).
ATCO21: Not for the moment because the system reliability has not been
proved.
ATCO22: Need for pilot report.
ATCO29: It depends on the trust in the system. Today EMMA is not sufficient.
ATCO30: EMMA is not sufficient.
ATCO32: Pilot acknowledgement is needed.
ATCO35: Yes if validated.
Table 36: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Landing Clearance
6.1.2.9 Conditional clearances
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
20
When visual reference is possible on the
manoeuvring area, the EMMA display may
help me to deliver conditional clearances on
the runway in use in a safe and efficient way.
34
4.41
1.08
0.000
•
•
•
•
•
Save date: 2007-03-08
Public
ATCO1: Nothing guarantees that what the pilot sees is the same as what we
see. EMMA is not needed to give conditional clearances.
ATCO5: I think that a systematic or too frequent use of the ground radar can
be dangerous.
ATCO12: It is a support.
ATCO18: EMMA is not used for conditional clearances (=> answer between 3
and 4).
ATCO22: EMMA helps only.
ATCO29: It helps.
ATCO: Not applicable.
Page 103
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
21
When visual reference is possible on the
taxiways in a safe and efficient way
34
4.53
1.08
0.000
•
•
•
•
•
•
•
•
22
the runway in use in a safe and efficient way.
•
35
3.86
1.44
0.088
•
•
•
•
•
•
•
•
23
taxiways in a safe and efficient way
•
35
4.31
0.96
0.000
•
•
•
•
•
ATCO1: No because I do not look at the HMI if I can see outside. EMMA is not
needed to give conditional clearances.
ATCO5: I consider it as a help when the manoeuvring area is visible but not as
the only local control means.
ATCO12: It is a support.
ATCO18: EMMA is not used for conditional clearances (=> answer between 3
and 4).
ATCO22: EMMA only helps.
ATCO29: The identification function helps a lot in that case.
ATCO35: In some specific cases, e.g. T10 and P yes.
ATCO5: Is it to be desired from the safety point of view? I do not think so.
ATCO8: If the controller cannot see the aircraft, then no conditional
clearance.
ATCO10: To be assessed.
ATCO11: Not enough.
ATCO18: A pilot request is needed anyway.
ATCO20: This is not possible in LVP today (only one entry point and one exit
point on the runway).
ATCO22: Yes, but we need the pilot.
ATCO28: Lacks of reliability when the aircraft is located near the antenna.
ATCO34: Because uncertainty on the aircraft position.
ATCO1: Not sure because not tested.
ATCO5: Yes but only at the intersections of the taxiways.
ATCO18: A pilot request is needed anyway.
ATCO20: It depends on the accuracy of the system.
ATCO22: Yes, but we need the pilot.
ATCO34: If reliable.
Table 37: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Conditional Clearances
Save date: 2007-03-08
Public
Page 104
Version: 1.0
EMMA
6.1.2.10 Visibility transition
Ref.
Statement
24
The transition from normal operations to low
visibility operations is easier with the EMMA
display.
Tot. Nb
Average
Answers
SD
P
0.75
0.000
Comments
•
35
5.29
•
•
•
25
When low visibility procedures (LVP) are
activated, the EMMA display helps me to
reduce my workload.
•
35
5.29
0.89
0.000
•
•
•
Save date: 2007-03-08
Public
ATCO1: Not sure because not tested.
ATCO35: It is obviously more comfortable.
ATCO1: If everything is identified, then I strongly agree.
ATCO5: The level of stress is also reduced because the respective positions of
the aircraft are known.
ATCO35: And my level of stress!
Page 105
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
•
•
26
With the EMMA display, it would make sense
to redefine the visibility limits for the
transition to low visibility procedures (if yes,
please indicate your suggestions).
30
4.03
1.73
0.101
•
•
•
•
•
•
•
•
•
•
•
ATCO1: Today the limit is 1000m. I hope to reduce it to 800m. with the ground
radar.
ATCO2: Depends on the accuracy of EMMA: if it is validated at the technical
level, yes.
ATCO5: They have changed at the CHEA but not yet at LFBO. Why?
ATCO8: There is no direct relation between the use of EMMA and visibility
limits. Moreover, modifying visibility limits is a heavy process.
ATCO9: Today the LVP limit is given by the meteo and not by a tool in the
tower. The use of runway 14L could be authorised in LVP.
ATCO10: Yes in the particular case of a low ceiling and good horizontal
visibility conditions.
ATCO17: I think that these limits are not defined locally and there is no
possible justification for the 1000m. limit this day.
ATCO18: The visibility limit, 1000m. today, could be reduced to 600m. with
EMMA.
ATCO19: Both runways could be used in LVP with EMMA.
ATCO21: Reduction of spacing only.
ATCO22: Capacities could be increased instead of visibility limits.
ATCO23: E.g.: lowest visibility accepted by pilots in CAT I.
ATCO26: 550m.
ATCO29:It has got nothing to do with it! OK to redefine airport capacities
(use of both runways in LVP).
ATCO30: It could be changed to 800m.
ATCO32: today’s limit at 1000m. is a bit too high (in particular to forbid all
the non authorised vehicles to circulate). In principle, it could be lowered to
800 to 600 m.
ATCO34: Slightly agree if EMMA is reliable to replace the external view.
ATCO35: It would be very interesting but I do not have all the clues to give a
value (see what happens in other places).
Table 38: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Visibility Transition
Save date: 2007-03-08
Public
Page 106
Version: 1.0
EMMA
6.1.2.11 Transponder procedures
Ref.
Statement
27
It would be appropriate that pilots switch the
transponder on before they request a push
back clearance (condition to automatically
associate a label to a departing aircraft).
Tot. Nb
Average
Answers
35
5.6
SD
P
0.81
0.000
Comments
•
ATCO29: At start up.
Table 39: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Transponder Procedures
6.1.2.12 Contingency procedures
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
28
If only one mobile fails to comply with the
technical requirements for A-SMGCS
operations (e.g.: on-board Mode S transponder
failure), I have to return to SMGCS
procedures (e.g.: position report by the pilot)
for all the traffic.
•
•
33
2.52
1.25
0.000
•
•
•
•
•
•
ATCO9: Yes if the mobile declares it.
ATCO14: We do neither know the methods to be used nor their limits.
ATCO15: The decision lies on the technical side.
ATCO18: The mobile would be identified with a label including digits anyway.
ATCO19: Manual identification of the mobile.
ATCO21: It depends on the traffic.
ATCO22: Only the unidentified mobile needs to report.
ATCO28: Yes in LVP, no in normal operations.
ATCO29: Not for all the traffic.
ATCO32: In this particular case, we request position reports to the
unidentified mobile.
ATCO35: With only one vehicle, we can cope with it, by characterizing it.
Table 40: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Contingency Procedures
Save date: 2007-03-08
Public
Page 107
Version: 1.0
EMMA
6.1.2.13 Conclusion on procedures
Ref.
Statement
29
The EMMA display helps me when visual
reference is not possible on the manoeuvring
area.
Tot. Nb
Average
Answers
35
5.6
SD
P
0.60
0.000
Comments
•
•
•
•
•
30
I can rely on the EMMA display instead of the
outside view when visual reference is not
possible on the manoeuvring area.
30
4.5
1.61
0.002
•
•
•
•
•
•
•
31
The EMMA display helps me when visual
reference on the manoeuvring area is possible.
Save date: 2007-03-08
•
29
5.34
0.77
0.000
•
•
Public
ATCO1: There is an issue with the current technical limits of the system. Only
a few things are missing in order to become a control tool.
ATCO2: Depends on the accuracy of the system.
ATCO8: EMMA may help but it does not take over from the outside view.
ATCO11: Not as a substitution.
ATCO20: EMMA does not take over from the outside view. There is no
comparison.
ATCO32: I would like to, if the system is certified as being reliable. It also
depends on the official texts.
Visual or pilot confirmation is still needed.
ATCO33: I would rather give the responsibility to the pilot.
ATCO34: Yes if reliable.
ATCO35: It depends on the validation.
ATCO10: For example at night, on remote areas and for small aircraft.
ATCO20: At night mainly.
ATCO35: It MAY help.
Page 108
Version: 1.0
EMMA
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
•
•
•
•
32
I think that there are too many inconsistencies
between the EMMA display and the actual
traffic.
•
27
2.37
1.01
0.000
•
•
•
•
•
•
•
•
•
•
33
The mix of cooperative and non-cooperative
mobiles could severely restrict the use of ASMGCS when visual reference is not possible
on the manoeuvring area.
•
•
35
4.14
1.52
0.042
•
•
•
ATCO8: If the transponders would work, yes.
ATCO9: No opinion (not enough traffic).
ATCO10: If EMMA works, OK.
ATCO18: Today there are too many inconsistencies.
ATCO19: Some false reports.
ATCO20: Lack of time of practical experience to be able to answer.
ATCO22: The primary radar plots are disturbing.
ATCO25: ?
ATCO29: Not experienced enough.
ATCO30: Not tested.
ATCO34: For the time being!
ATCO35: I have never observed big inconsistencies (only 1 or 2 false reports).
ATCO9: Every mobile has to be equipped.
ATCO14: The false reports are more disturbing.
ATCO17: !
ATCO20: Anyway, in LVP the traffic of vehicles is restricted. It is better if all
the vehicles are cooperative.
ATCO25: It depends on the amount of non-cooperative vehicles.
ATCO28: Depends on the procedures applied.
ATCO29: It depends on the amount of non-cooperative mobiles, but it does not
severely restrict the use of A-SMGCS.
Table 41: Results of the Shadow-Mode Trials Questionnaires – Operational Feasibility: Procedures – Conclusion on Procedures
Save date: 2007-03-08
Public
Page 109
Version: 1.0
EMMA
6.1.3.1 Capacity
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
54
The EMMA display may enable to increase
traffic throughput at the airport.
34
4.82
1.11
0.000
•
•
•
55
The EMMA display helps me to manage more
traffic when visual reference is not possible on
•
34
4.82
1.19
0.000
•
ATCO5: May enable a bit more taxi in LVP.
ATCO17: By using runway 14L for departures in LVP as in former times (a
200m. visibility is sufficient.
ATCO19: By using both runways in LVP.
ATCO32: Yes in LVP. No in normal operations.
ATCO35: In LVP yes. Or in normal operations for the hidden parts.
ATCO17: By using runway 14L for departures in LVP as in former times (a
200m. visibility is sufficient.
ATCO19: Improvement of the instructions in LVP: it will enable us to check
that an arriving aircraft is at the parking stand, does not get lost.
Table 42: Results of the Shadow-Mode Trials Questionnaires – Operational Improvements – Capacity
6.1.3.2 Efficiency
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
56
The EMMA display allows me to provide the
pilots with a better level of service.
•
34
5.09
0.83
0.000
•
•
ATCO5: Above all a better level of safety in LVP.
ATCO12: Also for the parking stand.
ATCO22: For example during the push back phase.
ATCO30: Not obvious.
ATCO32: Not obvious (e.g.: if the frequency occupation rate is high or the
controller has to deal with another issue, the provided service will not be
better).
Table 43: Results of the Shadow-Mode Trials Questionnaires – Operational Improvements – Efficiency
Save date: 2007-03-08
Public
Page 110
Version: 1.0
EMMA
6.1.3.3 Human factors
Ref.
57
58
Statement
The EMMA display allows me to have a better
awareness of the respective position and
identification of each mobile.
The EMMA display allows me to have a better
understanding of the traffic situation.
Tot. Nb
Average
Answers
SD
P
Comments
27
5.41
0.64
0.000
•
ATCO34: 4 for the position and 3 for the identification.
33
5.15
0.87
0.000
•
ATCO5: ‘Understanding’ = improper word.
ATCO14: On the ground.
ATCO1: Yes if EMMA is implemented on the approach working positions so
then the controller could see the number of aircraft waiting at the holding
point.
ATCO2: Yes if there is an HMI available in the IFR room.
ATCO3: Yes if HMI in the IFR room.
ATCO4: If HMI in the IFR room.
ATCO6: Yes if the HMI is in the IFR room.
ATCO7: It is hard to answer (rephrase). Lack of time.
ATCO9: Yes in the short term (3 min.).
ATCO20: Real time image but not anticipation.
ATCO34: For areas that we cannot see from the tower.
ATCO35: It is an additional thing but not vital in normal operations.
ATCO5: ?
ATCO18: Not tested enough. But some information is not immediately
accessible.
ATCO22: A bit complex.
ATCO29: The different HMIs should be harmonised.
ATCO32: May seem complex because it is new but it is OK after initial handle.
ATCO12: If everything works.
ATCO30: Reassures.
ATCO35: It relieves it!
•
•
•
•
59
The EMMA display allows me to better
predict the evolution of traffic.
•
32
4.53
1.46
0.001
•
•
•
•
•
•
•
•
•
60
The EMMA display is unnecessarily overcomplicated.
•
31
2.26
0.86
0.000
•
•
•
•
61
The EMMA display helps me to reduce my
mental workload.
35
4.77
1.06
0.000
•
•
•
Table 44: Results of the Shadow-Mode Trials Questionnaires – Operational Improvements – Human Factors
Save date: 2007-03-08
Public
Page 111
Version: 1.0
EMMA
6.1.3.4 Safety
Ref.
Statement
Tot. Nb
Average
Answers
SD
P
Comments
•
•
62
The information in the EMMA display helps
me to avoid conflicts.
35
4.91
0.82
0.000
•
•
•
•
63
The EMMA display endangers safety at the
airport.
34
1.68
0.84
0.000
•
•
•
64
The EMMA display makes the detection of
pilot errors easier.
35
5.09
1.09
0.000
•
•
ATCO6: Taxi parking F – facing South and facing North.
ATCO30: Especially the names of the mobiles.
ATCO33: I do not use EMMA to avoid conflicts.
ATCO34: If reliable.
ATCO35: Any additional equipment is useful!
We could do our job without it, but it is an additional help.
ATCO13: Cf. reliability!
ATCO23: If it has been validated and put in operation at t time.
ATCO29: Yes as it is. No when EMMA is finalised!
ATCO34: Yes if it is reliable!
ATCO16: Yes in LVP
ATCO34: E.g.: on areas St Martin when St Martin is closed.
Table 45: Results of the Shadow-Mode Trials Questionnaires – Operational Improvements – Safety
Save date: 2007-03-08
Public
Page 112
Version: 1.0
EMMA
6.2 Annex B: references
6.2.1 Applicable documents
[1]
European Airport Movement Management by A-SMGCS, EU FP6 contract
6.2.2 Referenced documents
6.2.2.1 Emma deliverables
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
D4.0.2: Sub-project 4 installation and test plan
Convention n° 02 034 Aéroport de Toulouse Blagnac Installation des matériels relatifs à
l’expérimentation EMMA ref. 455/JJD/OP/TE/4, annexed to D4.0.2.
D6.1.1: Generic verification and validation master plan
D6.1.3: Verification and validation test plan for Toulouse-Blagnac airport, version 1.02,
issued on the 15th February 2006
D1.3.5: Operational requirements document
D6.2.2: Indicators and Metrics for A-SMGCS
D6.3.1: Test results Prague
D6.5.1: Milano-Malpensa A-SMGCS Verification and Validation report
D6.6.1: Airborne verification and validation results
D6.7.1: V&V analysis report
D6.8.1: V&V recommendation report
D1.3.9: Functional hazard assessment and very preliminary system safety assessment
6.2.2.2 Other documents
[14]
[15]
[16]
Quality Manual, Thales ATM, QM-03, Commercial-in-Confidence
ICAO manual on Advanced Surface Movement, Guidance And Control System (A-SMGCS),
doc 9830 AN 452 - 2004.
EUROCAE WG-41, MASPS for A-SMGCS, Edition ED-87A, Jan. 2001.
6.3 Annex C: abbreviations
ADS-B
AENA
AIBT
AIF
ALDT
ANS_CR
AOBT
ARR
A-SMGCS
ASTERIX
ATC
ATCO
ATM
ATOT
AUEB
AVOL
CFMU
CWP
DACOTA
DEP
Automatic Dependent Surveillance – Broadcast
ente público Aeropuertos Españoles y Navegación Aérea
Actual In-Block Time
Airbus France
Actual Landing Time
Air Navigation Services Czech Republic
Actual Off-Block Time
Arrival
Advanced Surface Movement Guidance and Control System
All purpose STructured EUROCONTROL Radar Information eXchange
Air Traffic Control
Air Traffic Controller
Air Traffic Management
Actual Takeoff Time
Athens University of Economics and Business
Aerodrome Visibility Operational Level
Central Flow Management Unit
Controller Working Position
Dispositif d'Association, de COrrélation et de Traitement radar pour les
Approches
Departure
Save date: 2007-03-08
Public
Page 113
Version: 1.0
EMMA
DFS
D-GPS
DG-TREN
DLR
German ATC Corporation (Deutsche Flugsicherung GmbH)
Differential Global Positioning System
Directorate General Transport and Energy
Deutsches Zentrum für Luft- und Raumfahrt e.V. / German Aerospace Centre
(German Aerospace Research Institute)
DSNA
Direction des Services de la Navigation Aérienne
DSNA/DTI
Direction des Services de la Navigation Aérienne / Direction de la Technique et
de l’Innovation
EC
European Commission
ELDT
Estimated Landing Time
EMMA
European Airport Movement
ENAV
Ente Nazionale di Assistenza al Volo (Italian ANSP)
EOBT
Estimated Off-Block Time
ERC
EUROCONTROL
ETG
EuroTelematik AG
ETOT
Estimated Takeoff Time
EUROCAE
European Organisation for Civil Aviation Equipment manufacturers
EUROCONTROL European Organisation for the Safety of Air Navigation
FAA
Federal Aviation Administration
FHA
Functional Hazard Assessment
FOM/PA
Figure of Merit / Position Accuracy
GBAS
Ground Based Augmentation System
GS
Ground Station
GPS
Global Positioning System
GSC
Ground Station Calibrator
GTW
Gateway
ICAO
International Civil Aviation Organisation
LAN
Local Area Network
LFBO
Toulouse-Blagnac
LVO
Low Visibility Operation
LVP
Low Visibility Procedure
MAGS
Mode S Airport Ground Sensor
MASPS
Minimum Aviation System Performance Standards
MLAT
Multilateration
MXP
Milano-Malpensa
Nb
Number
NLR
Nationaal Lucht- en Ruimtevaart Laboratorium (NL)
NTP
Network Time Protocol
NVO
Normal Visibility Operations
POC
Point of Contact
R&D
Research and Development
RF
Radio Frequency
RPA
Reported Position Accuracy
RTCA
Requirements and Technical Concepts for Aviation
RWY
Runway
SCA
Surface Conflict Alert
SD
Standard Deviation
SDF
Sensor Data Fusion
SICTA
Sistemi Innovativi per il Controllo del Traffico Aereo
SMR
Surface Movement Radar
SRS
Software Requirements Specification
TATM
Thales ATM
THAV
Thales Avionics
TIS-B
Traffic Information Service - Broadcast
Save date: 2007-03-08
Public
Page 114
Version: 1.0
EMMA
TT
TUD
TWR
UT(C)
V&V
WG
WP
Test Transmitter
Technische Universitaet Darmstadt
Tower
Universal Time (Co-ordinated)
Verification and Validation
Working Group
Work Package
Save date: 2007-03-08
Public
Page 115
Version: 1.0

Toulouse A-SMGCS verification and validation results

Transcription

Similar documents

EMMA ROBERTS

the factsheet with all specifications

appetite for destruction

Emma Blyth WS cat 2016

Wherever you are. Whenever you need it.

Emma Kernohan, 28, And Simon Ferres, 33

BREGUET Br.XIX-2A/B

National French Exam - Northeastern School District

Decks In The City

in pdf