Multi-Sensor Data Integration for Autonomous Sense and Avoid

AIAA 2011-1479
Infotech@Aerospace 2011
29 - 31 March 2011, St. Louis, Missouri
Multi-Sensor Data Integration for
Autonomous Sense and Avoid
Robert H. Chen1, Arthur Gevorkian2, Alex Fung3, Won-Zon Chen4
Northrop Grumman Aerospace Systems, Redondo Beach, California, 90278
and
Vincent Raska5
U.S. Air Force Research Laboratory, Wright-Patterson Air Force Base, Dayton, Ohio, 45433
To operate Unmanned Aerial Systems (UAS) freely in the National Airspace System
(NAS), an on-board sense-and-avoid (SAA) capability equivalent to or better than manned
aircraft will be required. Under U.S. Air Force Research Laboratory (AFRL) sponsorship,
Northrop Grumman Corporation (NGC) has been developing a scalable autonomous SAA
system using a comprehensive sensor suite comprising Traffic Alert and Collision Avoidance
System (TCAS) and Automatic Dependent Surveillance - Broadcast (ADS-B) for detecting
cooperative intruders as well as radar and electro-optical (EO) sensors for detecting noncooperative intruders. This paper focuses on the sensor data integration (SDI) portion of the
autonomous SAA system in the areas of design objective, architectural and algorithmic
approach, and flight test results.
I. Introduction
T
HE demand to use Unmanned Aerial Systems (UAS) in the government and private sectors has been increasing
steadily in recent years. UAS from the small Raven to the large Global Hawk are being used to perform what
were once piloted missions. Expectations are that such current and future UAS will enable precision and persistent
air vehicle operations not even possible with piloted aircraft.
To achieve such potential, UAS must be able to gain access to all classes of airspace, domestically or
internationally, with the same degree of access as piloted aircraft. However, UAS are not currently authorized by the
Federal Aviation Administration (FAA) to operate freely within the National Airspace System (NAS) due to safety
and compliance considerations.
The extraordinary safety record in airspace systems worldwide results from a combination of training, flight
procedures, aircraft equipage, and air traffic control (ATC). Before UAS are granted approval to “file and fly” there
are many issues which need to be resolved: airworthiness, command and control, operator training and
qualifications, equipage, and compliance with all applicable regulatory requirements are just a few. Foremost among
these requirements is a “see-and-avoid” capability, the basic and essential safety principle of a pilot’s ability to "see
and avoid” by first remaining well clear and ultimately avoiding mid-air collisions and other potential conflicts with
other aircraft. Accordingly, the goal of any sense-and-avoid (SAA) system is to perform those conflict- and
collision-avoidance functions normally provided by a pilot in a piloted aircraft. In the meantime, the absence of
pilots onboard UAS is a major confidence barrier while the lack of an onboard SAA system is one of the most
critical technology gaps.
1
Senior Research Engineer, GN&C Technology, One Space Park, M/S R11-2796, Senior Member AIAA.
Software Engineer, VMS Technology, One Space Park, M/S R11-2796.
3
Senior System Engineer, Sensor Modeling & Analysis, One Hornet Way, El Segundo, California, 90245, M/S W1202/W6.
4
Technical Fellow, VMS Design & Integration, One Space Park, M/S R11-1057, Senior Member AIAA.
5
Program Manager, Air Vehicles Directorate, AFRL/RBCC, 2130 Eighth St.
2
1
American Institute of Aeronautics and Astronautics
Approved for Public Release, Distribution Unlimited: 88ABW-2011-1065, 2 March 2011
Copyright © 2011 by the American Institute of Aeronautics and Astronautics, Inc. The U.S. Government has a royalty-free license to exercise all rights under the copyright claimed herein for Go
Under U.S. Air Force Research Laboratory (AFRL) sponsorship, Northrop Grumman Corporation (NGC) has
been developing a scalable autonomous SAA system. The approach has been to integrate various sensors with a
conflict- and collision-avoidance algorithm that selects the optimum trajectory observing right-of-way rules and
ATC clearance, preserving communication and control links, and so forth. These criteria are selected so that SAA
maneuvers are predictable and appropriately model human pilot behavior to the greatest extent possible. Figure 1
depicts the top-level objectives for the SAA technology currently in development to avoid multiple cooperative and
non-cooperative traffic autonomously. The autonomous SAA system includes a comprehensive sensor suite
comprising Traffic Alert and Collision Avoidance System (TCAS) and Automatic Dependent Surveillance Broadcast (ADS-B) for detecting cooperative traffic as well as radar and electro-optical (EO) sensors for detecting
non-cooperative traffic.
Figure 1. Conceptual view of SAA system.
Note that there is no single sensor that can adequately address SAA sensing requirements for cooperative and
non-cooperative traffic. Non-cooperative means no data about the conflicting traffic is transmitted to the UAS from
the conflicting aircraft or from ATC. Non-cooperative aircraft must then be detected by sensors completely
unassisted by external inputs. Radar and EO are two examples of non-cooperative sensors. Cooperative traffic
includes aircraft equipped with systems providing information when queried (e.g., transponders) or providing
information automatically and continually (e.g., ADS-B). Currently, the EO system is the only passive system used,
but Infra Red (IR) sensors could also be used, or both EO and IR together. AFRL is exploring the use of IR and
other candidate sensors for SAA under separate efforts.
AFRL and NGC are also collaborating with industry, Federally Funded Research and Development Centers, the
FAA, RTCA SC-203, and others, towards SAA system development and SAA performance requirements. The
technical results and lessons learned during the progress of this AFRL and NGC development effort have been
shared at various forums and with many groups/stakeholders in the UAS community.
The autonomous SAA system being developed comprises a sensor data integration (SDI) algorithm and a
collision-avoidance algorithm which correspond to the “sense” and “avoid” aspects of SAA, respectively. This paper
focuses on the SDI portion of the SAA system. The SDI function is to generate the most accurate and reliable tracks
for nearby traffic by integrating information from multiple, dissimilar SAA sensors. It enables the collisionavoidance algorithm to assess the potential collision threats and generate the optimal collision avoidance maneuver
when necessary. More details about the overall SAA system and collision-avoidance algorithm are provided in Ref
1.
The rest of this paper is organized as follows. In Section II, the design objectives and architecture are discussed.
Section III describes the algorithmic approach. Some flight test results and future improvements are presented in
Section IV. A brief conclusion is included in Section V.
II. Sensor Data Integration System Overview
Figure 2 shows the autonomous SAA system being developed. It is called Multi-Sensor Integrated Conflict
Avoidance (MuSICA) and consists of four major modules: Sensor Input Management (SIM), Sensor Data
2
American Institute of Aeronautics and Astronautics
Integration (SDI), Jointly Optimal Conflict Avoidance (JOCA), and Flight Control Interface (FCI). Basically, SIM
pre-processes the data from SAA sensors while SDI uses the processed sensor data to generate integrated intruder
tracks. JOCA uses the output from SDI to determine the optimal collision avoidance command when necessary. FCI
further processes generic JOCA commands to be compatible with the specific flight control system of a UAS as well
as integrating the interface with remote UAS pilots.
Figure 2. SDI in the overall SAA architecture.
Note that SIM and SDI together address the “sense” part while JOCA and FCI together address the “avoid” part.
The partitioning of SIM and SDI is chosen so as to have SIM deal with particularities of each dissimilar sensor. For
example, SIM converts the angular measurements of radar and EO from the body frame to the inertial frame used by
SDI. Another example is that SIM uses the latitude, longitude, and altitude of the ownship and intruder provided by
ADS-B to derive the relative position used by SDI. In this way, SDI is preserved to be a sensor agnostic module and
easily adaptable to different SAA sensors. Only SDI is discussed in more detail in this paper.
Characteristics of the SAA sensors used, SDI design objectives, and SDI architecture are discussed below. SAA
sensor characteristics are discussed first as they have a significant influence on the SDI design objective and hence
its resulting architecture.
A. SAA Sensor Characteristics
Figure 3 shows the suite of SAA sensors used. They include different types, i.e., cooperative and noncooperative as well as active and passive. This is to ensure an extensive coverage for detecting different intruder
types. For example, larger commercial transport aircraft must carry transponders and can be detected by TCAS and
possibly ADS-B, but smaller general aviation aircraft may only be detected by radar or EO. Furthermore, there are
strengths and weaknesses with each individual sensor. Use of “smart” data fusion to combine their best features can
produce integrated intruder tracks more accurate and reliable than what can be achieved with a single sensor. For
example, the high quality of the angular measurements from EO can compensate for the poor bearing measurement
from TCAS. The principle of operation and performance characteristics of each SAA sensor type are described
briefly below.
3
American Institute of Aeronautics and Astronautics
Figure 3. SAA sensor types.
TCAS is an airborne secondary surveillance radar (SSR) system with surveillance and collision avoidance
functions.2 It interrogates nearby aircraft that are equipped with Mode A, C, or S transponders to obtain range,
bearing, and relative altitude information. The range is derived from the time it takes to get a response from the
interrogated aircraft. The bearing is determined by a directional antenna which measures the angle of return signal
from the interrogated aircraft. The relative altitude is available through the interrogated aircraft based on its onboard
pressure altitude equipment.
An ADS-B-equipped aircraft broadcasts its own position and associated accuracy and integrity information to
other ADS-B-equipped aircraft and ground receivers.3 Different from TCAS, ADS-B does not expect an
acknowledgment or response from other aircraft receiving the message. Since the position information is typically
derived from the Global Positioning System (GPS) and pressure altimeter, its accuracy is excellent and independent
from the range between broadcasting and receiving aircraft. However, ADS-B will not be mandatory for all airspace
in the United States.
Airborne radar suitable for conflict and collision avoidance is not yet commercially available. This is because
highly accurate angular measurements and fast update rates are required along with sufficiently long detection
ranges for the SAA system to generate safe separation. AFRL and other government agencies (e.g., U.S. Navy) are
developing new radars specifically designed for conflict and collision avoidance. One design approach is to use
advanced phased array and signal processing techniques to meet these demanding requirements by tracking high
priority intruders at higher update rates while searching for new targets at lower update rates.4
Contrary to radar, EO is a passive sensor without active transmission. This allows EO to be used in certain
operational environments where radar may be undesirable (e.g., ground operation or in battle space). Furthermore,
EO provides highly accurate angular measurements at fast update rates. However, EO (monoscopic) does not
inherently generate range information and the detection range is typically poor.
Typical accuracy, update rate, and detection range for each of the sensor types discussed above are listed in
Table 1.
Table 1. SAA sensor characteristics
Accuracy
Update rate
Detection range
TCAS
Range: 175 – 300 ft
Bearing: 9 – 15 deg
Altitude: 50 – 100 ft
1 Hz
≥ 14 nm
ADS-B
Horizontal position:
25 – 250 ft
Vertical position:
50 – 100 ft
1 Hz
≥ 20 nm
Radar
Azimuth: 0.5 – 2 deg
Elevation: 0.5 – 2 deg
Range: 10 – 200 ft
Range rate: 1 – 10 ft/s
0.2 to 5 Hz
5 – 10 nm
EO
Azimuth: 0.1 – 0.5
deg
Elevation: 0.1 – 0.5
deg
20 Hz
2 – 5 nm
B. Design Objective
The design objective of SDI is to integrate multiple dissimilar sensors in order to provide accurate and robust
intruder tracks for conflict and collision avoidance assessment. This overall design objective is further decomposed
into several more specific objectives:
• Integrate the entire sensor suite or a subset thereof.
• Generate one integrated track for one intruder aircraft (i.e., no duplicated tracks or false tracks).
• Obtain best features of dissimilar sensors in terms of accuracy, integrity, continuity, and availability (AICA)
in the integrated tracks.
• Achieve modular and efficient real-time software implementation.
4
American Institute of Aeronautics and Astronautics
The first objective ensures that every combination of SAA sensors can be handled. It is inevitable that the
available sensor data will be different for different intruders. Furthermore, the available sensor data will change over
the course of an encounter due to different sensor characteristics (e.g., detection range and field of view). Therefore,
it is important that SDI can handle all possible combinations of SAA sensors effectively.
The second objective ensures that each intruder is represented by one track and each track represents an actual
intruder. Since an intruder may be detected by multiple sensors, SDI needs to be able to associate and fuse these
multiple sensor data to one integrated track to minimize data communication and downstream data processing (i.e.,
JOCA) needs. Also, since both EO and radar may generate false detections in challenging environments, SDI needs
to be able to suppress false tracks caused by these false detections to prevent unnecessary downstream processing or
an even worse consequence, unwanted maneuvers by JOCA.
The third objective ensures that the strengths of every sensor are combined to form the best possible intruder
tracks. The track quality is defined by using four criteria: 1) track accuracy refers to the estimation error of the
intruder’s position and velocity, 2) track integrity refers to the detection/declaration range and false alarms, 3) track
continuity refers to maintaining a track without interruptions (i.e., no dropped tracks); and 4) track availability refers
to the ability to operate in challenging environments such as rain and clouds.
The fourth objective ensures that the algorithms developed have reasonable computational requirements for
affordable real-time implementation and are easily adaptable to different sensor suites by other UAS platforms.
C. Architecture
To achieve the aforementioned design objectives, SDI was architected to comprise four modules as shown
previously in Fig. 2: Extended Kalman Filter (EKF), Data Association, False Track Filtering (FTF), and Track
Manager. Figure 4 shows a typical SDI data flow that illustrates the functions and interplays of these four modules:
1. Track Manager receives new measurements from one or multiple sensors.
2. EKF propagates the existing tracks to the current time (i.e., time corresponding to the new measurements), so
called predicted tracks.
3. Data Association determines which measurement corresponds to which predicted track.
4. EKF updates the predicted tracks by incorporating their associated measurements, so called updated track.
5. Track Manager updates the track database with the updated tracks and determines if there are any new tracks
to be initiated and old tracks to be terminated.
6. FTF determines if a track is good enough for declaration as a true track or bad enough for termination as a
false track.
7. Track Manager outputs the declared tracks to the collision avoidance function (i.e., JOCA).
Figure 4. SDI Data flow and execution.
It is important to note that the data association method used is measurement-to-track association, and is
performed independently for each sensor with respect to central EKF tracks as shown in Fig. 5. This approach
enables SDI to handle SAA sensors with different coordinates and update rates in a straightforward and effective
manner. That is, SDI immediately processes any sensor measurements when available. To achieve this, SDI is run at
5
American Institute of Aeronautics and Astronautics
a rate matching the highest of the four sensors (i.e., 20 Hz, as driven by EO) such that maximum latency is limited to
50 ms.
Figure 5. SDI framework for handling multiple sensor inputs.
III. Algorithmic Approach
SDI is a real-time multi-target / multi-sensor tracking problem. Target tracking is the task of estimating the states
of a target both at the current time (filtering) and at any point in the future (prediction). The target states include
both relative position and velocity in three-dimensional space. An estimate of the accuracy of these state estimates,
called covariance, is also generated. The state estimates and covariance together form the track that represents the
knowledge about an intruder. Ideally one, and only one, track is generated for each and every intruder. However,
target tracking is performed in the presence of measurement uncertainty because the origin of measurements cannot
always be correctly ascertained and measurements contain noise. As a consequence, errors can be made in the
association of new measurements to existing tracks and these data association errors can result in rather significant
errors on the state estimates of the intruders.
While SDI is a classic target tracking problem, there are several challenges that are unique to the SAA
application including:
• Multiple dissimilar sensors with some significant measurement errors (e.g., large bearing error for TCAS
and lack of range information for EO),
• False target data from both radar and EO,
• High accuracy tracks required for collision avoidance maneuver selection,
• Good computational efficiency for real-time operation onboard UAS.
In this section, more details of the SDI process are first discussed. Potential algorithms for each module in the
SDI architecture are reviewed. Then, the algorithm that best addresses the design objectives is chosen. The
algorithm selection processes for EKF, Data Association, and FTF are discussed in more detail.
A. SDI Process
SDI performs three essential functions: data association, data fusion, and false track filtering. When a set of new
measurements is received from a sensor, these measurements have to be matched to a set of existing tracks before
they can be used to update these tracks. Therefore, data association is to solve the measurement-to-track association
problem to determine which measurement corresponds to which track. After data association has been accomplished
and new measurements have been assigned to existing tracks, data fusion is to correct the state estimates and
covariance with the new measurement. In addition to measurements that are associated with the existing tracks,
there may be some measurements that are not associated with any tracks. They could be first detections of new
intruders or false detections of intruders that do not exist. Therefore, FTF detects and removes false detections while
declaring new tracks only for the true detections.
The interactive process of EKF and Data Association is illustrated in Fig. 6 in four steps. (1) EKF State
Propagation determines the state estimates and covariance at the current time step using the state estimates and
covariance from the previous time step and the ownship’s acceleration (or velocity) from GPS/INS, as shown by the
blue lines. This is done by propagating the state estimates and covariance based on the system dynamics that defines
6
American Institute of Aeronautics and Astronautics
the relative motion between the intruder and ownship. (2) An association gate is applied to each predicted track to
disregard any measurements outside the gate (i.e., detections whose measurements are too “far” away from the
track), as shown by the blue circles. This gating operation is performed to reduce potential errors in the next step. (3)
Data Association determines which measurement corresponds to which track so that the measurement assigned to a
track is the “closest” one in an optimal and global sense, as shown by the red lines. To take into account track
uncertainty and measurement noise, a statistical distance is defined by using the residual weighted by the covariance
and sensor noise variance. (4) EKF Measurement Update corrects the state estimates by using their associated
measurements and updates the covariance based on the expected measurement accuracy, as shown by the green
lines.
Figure 6. Functions Performed by EKF and Data Association.
In addition to the measurements that are associated with the existing tracks, there may be some measurements
that are not associated with any tracks. They could be first detections of new intruders or false detections of
intruders that do not exist. Since one cannot be distinguished from the other immediately, the new tracks initiated for
these measurements will remain internal (i.e., not part of the SDI output) until they are confirmed as true tracks for
new intruders. Therefore, FTF is to detect and remove false detections while declaring internal tracks only for the
true detections. Basically, this is done by checking the consistency of the detections. That is, it is expected that a true
track for a new intruder will receive subsequent detections that are “close” to the track while a false track initiated
by a false detection will not receive any subsequent detections. Therefore, FTF determines when and which internal
tracks are confident enough to be declared as the output of SDI.
Finally, because of possible missed detections and intruders leaving the sensor’s field of view (FOV) or
detection range, some tracks may not have any associated measurement. Track Manager will terminate an existing
track if there is no new associated measurement for a certain period of time (coasting time). Basically, Track
Manager performs various housekeeping tasks of maintaining, initiating, and terminating intruder tracks as shown in
Fig. 7. It also manages the inputs and outputs of SDI, and controls the data flow and execution of EKF, Data
Association, and FTF at the top level as shown in Fig. 4. Note that Track Manager’s function is bookkeeping only
and no mathematical algorithms are involved.
7
American Institute of Aeronautics and Astronautics
Figure 7. Functions performed by False Track Filtering and Track Manager.
B. Extended Kalman Filter
The Kalman filter is a well-known optimal filtering technique.5 The state propagation step uses a vehicle
dynamic model to propagate both state estimates and covariance from the previous time step to the current time step.
This requires knowing the intruder’s acceleration. Since the intruder’s acceleration is not known, it is assumed to be
a zero-mean white Gaussian process which implies that the intruder is effectively non-maneuvering (i.e., flying in a
straight line) but with uncertainty. Then, the process noise variance can be used as a design parameter to trade
estimation accuracy for robustness to maneuvering intruders. Note that a more sophisticated intruder model can be
used to estimate the intruder’s maneuver, thus generating better tracking performance. However, this requires
sufficient sensor accuracy and update rate in order to support the more demanding estimation process which is also
more computationally intensive.
The measurement update step uses the measurements and their expected accuracy (i.e., sensor noise variance) to
update the conditional probability density function (pdf) of the states given the measurement history. Because the
measurement noise is considered, the Kalman filter takes into account the different strengths of dissimilar sensors
(e.g., radar has better range accuracy, while EO has better bearing accuracy). Since the state estimate is the
conditional mean, the Kalman filter generates the minimum variance estimate because the covariance is minimized.
Since the conditional probability is maximized, the state estimate is also the maximum likelihood estimate. This
two-step recursive process lends itself conveniently to handling multiple asynchronous sensors inputs because the
measurement update step can be performed whenever a measurement becomes available.
Since the vehicle dynamic model is expressed in an inertial Cartesian coordinate frame, some measurements are
nonlinear functions of the states. Therefore, a nonlinear optimal filtering technique is required. EKF is chosen due to
the combination of its good tracking performance and reasonable computation requirements.5 It can easily
outperform most ad hoc non-optimal filters (e.g., α-β filter). However, its linear approximation of nonlinear
measurements leaves some room for potential improvements by using more elaborate nonlinear filtering techniques
such as unscented Kalman filter and particle filter. Unscented Kalman filter and particle filter may generate better
tracking performance but at the expense of increased computational burden.
The unscented Kalman filter uses a deterministic sampling approach to overcome the limitation of the EKF due
to its linearization process.6,7 A set of sample points around the conditional mean are selected and propagated
through the nonlinear system. Then, these sample points are approximated as a Gaussian distribution to recover the
conditional mean and covariance. This technique (unscented transformation) captures the conditional mean and
covariance more accurately to the third-order Taylor series expansion compared to the first-order linear
approximation by EKF.
The particle filter is based on the Monte-Carlo simulation technique.8 A weighted set of particles are used to
approximate the conditional pdf. Re-sampling is used to prevent that all but a few of the importance weights are
close to zero. Since the particle filter does not require any assumption about the system dynamics and uncertainty
distributions, it generates better results for highly nonlinear systems with non-Gaussian uncertainty if enough
particles are used. However, the particle filter is highly computationally intensive compared to both the EKF and
unscented Kalman filter.
8
American Institute of Aeronautics and Astronautics
C. Data Association
Data Association is formulated as a classic linear assignment problem (LAP). While LAP is a combinatorial
optimization problem, there are efficient numerical algorithms specially designed to exploit its special structure. The
two most notable algorithms are the Hungarian algorithm9,10 and the Jonker-Volgenant-Castanon (JVC)
algorithm11,12 which have polynomial time computation of O(n4) and O(n3), respectively. This is a significant
improvement over the brute-force search which has exponential time computation of O(n!·n) because there are n!
permutations and each permutation requires n computation operations. For example, if there are 10 measurements
and 10 tracks, then the brute-force search has O(n!·n) = 36,288,000 while Hungarian and JVC algorithms have only
O(n4) = 10,000 and O(n3) = 1,000, respectively.
Data Association currently uses the JVC algorithm considering global optimum and computational
efficiency.13,14 Like the Hungarian algorithm, the JVC algorithm generates the global optimum but is more
computationally efficient. On the other hand, the nearest neighbor algorithm requires less computation but it cannot
guarantee the global optimum. Specifically, the nearest neighbor algorithm assigns tracks one at a time by selecting
the measurement that is “closest” to the track among the measurements that are within the association gate and not
assigned yet. Once a measurement is assigned to a track, it becomes unavailable in the subsequent assignment for
the remaining tracks. Therefore, the solution generated by the nearest neighbor algorithm is not the global optimum
and the solution depends on the order of the tracks being assigned.
Instead of LAP, Data Association can also be solved by using the Joint Probabilistic Data Association (JPDA)
algorithm or the Multiple Hypothesis Tracking (MHT) algorithm.15 Since the JVC algorithm specifically assigns one
measurement to one track, its main disadvantage is that the choice of association sometimes may be ambiguous
especially in dense target or clutter environments with measurements of poor quality. Furthermore, an incorrect
association in the JVC algorithm may lead to further errors on subsequent association attempts. Because JPDA and
MHT are based on the Bayesian probability approach, they are more robust than the JVC algorithm for these
situations.
The JPDA algorithm avoids the ambiguous association decisions by “averaging” over the measurements. It
allows a track to be updated using multiple measurements and a measurement to be used by multiple tracks.
Specifically, the JPDA algorithm updates a track by using all the measurements that are within its association gate
based on the conditional probabilities of the measurements. The weighting for each measurement is calculated using
its residuals relative to the track under consideration and other tracks for which the measurement falls within their
association gates. Clearly, the computation of the conditional probabilities becomes very complex when the number
of tracks increases. Since a track updated by using multiple measurements will not be as accurate as the track
updated by using the correct single measurement, the tradeoff of the JPDA algorithm is that the track accuracy is
reduced as compared to LAP, assuming correct association
The MHT algorithm avoids the ambiguous association decisions by delaying the decision making until future
measurements are incorporated. For each measurement, the MHT algorithm may simultaneously assume that it is a
false alarm, the start of a new track, and a potential update for multiple existing tracks. Each one of these
possibilities generates a separate hypothesis and an associated probability. As more measurements are subsequently
incorporated, these hypotheses and their associated probabilities are updated. The idea is that the probability of the
correct hypotheses will continue increasing and eventually win out over the less likely (and presumably incorrect)
hypotheses. Clearly, a very large number of hypotheses can be generated in realistic operating environments. The
tradeoff of MHT is that the number of tracks would grow exponentially and slow down processing time. Hypothesis
pruning is needed, but very carefully, to prevent removing a statistically unlikely hypothesis which in fact represents
a true intruder.
For SAA application, the aforementioned ambiguous situation in dense target or clutter environments for Data
Association would probably not occur due to relatively low air traffic density. However, further investigation of
denser traffic areas such as terminal areas may be needed.
D. False Track Filtering
To detect and remove inherent false detections, FTF is formulated as a hypothesis testing problem in which there
are two hypotheses for an internal track. One hypothesis is that the internal track is a true track and the other
hypothesis is that the internal track is a false track. A Sequential Probability Ratio Test (SPRT)16,17 is used to
determine which hypothesis is correct by processing observations about the internal track. SPRT calculates the
likelihood ratio of the two hypotheses (called track score) which is the ratio of the probability of the internal track
being a true track over the probability of the internal track being a false track. When an internal track has an
associated measurement, the track score will increase. On the other hand, the track score will decrease when the
internal track has no associated measurement or the associated measurement is far away from the expected value of
9
American Institute of Aeronautics and Astronautics
the internal track. SPRT makes the decision to declare or delete an internal track based on the miss alarm (false
negative) and false alarm (false positive) rates specified by the designer. Specifically, when the track score becomes
larger than a high threshold T1, the internal track is declared and output by SDI. On the other hand, if the track score
becomes smaller than a low threshold T2, the internal track is deleted from the track database. If the track score is
between the two thresholds, it is maintained as an internal track. Figure 8 illustrates this track declaration/deletion
process by using track score.
Figure 8. Track score of SPRT.
The thresholds for declaration and deletion are determined based on the missed and false alarm rates specified by
the designer. If the false and miss alarm rates are chosen smaller, the threshold for declaration becomes higher and
the threshold for termination becomes lower. Therefore, SPRT needs to take more observations and time in order to
reach a decision. On the other hand, if the false and miss alarm rates are chosen larger, the threshold for declaration
becomes lower and the threshold for termination becomes higher. Therefore, SPRT does not need to take as many
observations and time in order to reach a decision. As expected, the duration of the delay in reaching a decision
depends on the quality/confidence of the track required by the user. This delay contributes to the difference between
the detection range and declaration range.
IV. Flight Test Results
The autonomous SAA system, MuSICA including SDI and JOCA, was tested first in a Simulink desktop
simulation, then in a hardware-in-the-loop (HWIL) simulation laboratory, and finally in real flight. A set of
encounter scenarios that are consistent with typical air traffic in the NAS were developed. In these scenarios, the
ownship is setup to fly level, ascend, or descend while on course to a near mid-air collision with one or two intruders
flying straight and level from a head-on or abeam direction. In the HWIL simulation laboratory, MuSICA was
implemented on a Smiths VMC and tested using real hardware of TCAS (Honeywell TPA-100A) and ADS-B
(Garmin GDL-90 UAT) along with simulations of radar, EO, and UAS vehicle models. This allowed MuSICA to be
evaluated on the same hardware used in flight with the same real-time software executive and interface. In 2008 and
2009, three rounds of flight tests were conducted using a Learjet as a UAS surrogate. In these flight tests, the
ownship flew autonomously under the control of MuSICA to avoid collision with one or two intruders successfully.
In this section, the flight test results of SDI from the last round of flight tests is discussed. More details about the
flight tests and overall SAA system performance are provided in Ref 1.
The flight test results for a scenario of one non-cooperative intruder are shown in Fig. 9. The left column shows
the radar detections where there were some false detections early in the encounter and the first true detection
occurred at 2.27 nm. Note that the black lines representing the true position of the intruder are included for the
purpose of comparison. The middle column shows the EO detections where all detections were false. Note that this
particular scenario is a special night flight designed to gather data for future EO night capability development. SAA
EO night capability is currently under development and was not expected to be present during flight test. The right
column shows the track results of SDI where the track started at 2.22 nm. In this case, it took 1.3 seconds for SDI to
initiate the track and the difference between the detection and declaration ranges is 0.05 nm. This example shows
10
American Institute of Aeronautics and Astronautics
that SDI suppressed all the false detections from radar and EO and did not generate any false track. Furthermore,
SDI generated only one track without any dropped track until the intruder left the sensors’ FOV.
Figure 9. Example of radar and EO integration.
The flight test results for a scenario of one cooperative intruder and one non-cooperative intruder are shown in
Fig. 10 and 11. In Fig. 10, the left column shows the ADS-B detections of the cooperative intruder where the blue
line represents the ADS-B detections while the black and pink lines represent the true positions of the cooperative
and non-cooperative intruders, respectively. Note that the blue line overlaps with the black line. The middle column
shows the radar detections where there was no false detection. The first detections occurred at 5.87 and 5.58 nm for
the cooperative and non-cooperative intruders, respectively. The right column shows the EO detections where the
blue and red lines represent the true and false detections, respectively. The first true detections occurred at 2.23 and
0.92 nm for the cooperative and non-cooperative intruders, respectively. Note that TCAS detections were not
processed by SDI in this example.
Figure 11 shows the track results of SDI where the green and red lines represent the cooperative and noncooperative intruders, respectively. Track type is used to indicate the types of sensors being integrated into the track.
For the cooperative intruder, the track started at 7.49 nm with track type 5 indicating that the track was generated
using only the ADS-B detections. Then, ADS-B and radar detections were integrated as indicated by the track type
13. Finally, ADS-B, radar, and EO detections were integrated as indicated by the track type 14. For the noncooperative intruder, the track started at 4.74 nm with track type 9 indicating that the track was generated using only
the radar detections. In this case, it took 12.6 seconds for SDI to initiate the track and the difference between the
detection and declaration ranges is 0.84 nm. Then, radar and EO detections were integrated into the track as
indicated by the track type 10. Finally, when the intruders left the sensors’ FOV, track type became 1 indicating that
the track is on coasting (i.e., no sensor is being integrated). This example shows that SDI suppressed all the false
detections from EO and did not generate any false track. Furthermore, SDI generated only one track for each
intruder without any dropped track.
11
American Institute of Aeronautics and Astronautics
Figure 10. Example of ADS-B, radar, and EO integration: sensor detections.
In a total of 34 test runs regarding the non-cooperative intruder, SDI was able to integrate detections from radar
and EO without any false tracks in every test run, and experienced dropped tracks only once. The average difference
between detection and declaration ranges was 0.33 nm which is the cost (tradeoff) for suppressing false detections.
In a total of 22 test runs regarding the cooperative intruder, SDI was able to integrate detections from ADS-B, radar,
and EO without any false tracks in every test run, but generated duplicated tracks for 4 test runs. While a duplicated
track is not desirable, it is expected that it will only increase the computation load of JOCA but will not significantly
affect the collision avoidance maneuver generated by JOCA. Furthermore, post flight analysis indicated that the
unsuccessful integration is due to the unexpectedly large ADS-B measurement errors which were caused by delay
compensation not being implemented properly. By running the SDI offline to post process the flight test data with
correct ADS-B delay compensation, it was shown that SDI was able to integrate detections from ADS-B, radar, and
EO without any duplicated tracks.
Since the flight tests were focused on the integration of ADS-B, radar, and EO, SDI was tested to integrate
TCAS only in a limited number of test runs. Unfortunately, SDI was not able to integrate detections from all four
sensors due to a bug in the way SDI handled TCAS detections. By running the SDI offline to post process the flight
test data with the fix in place, it was shown that SDI was able to integrate detections from all four sensors. However,
more testing is required for further confirmation. This testing will be conducted first in the HWIL simulation with
real hardware of TCAS and ADS-B and then in the next around of flight testing.
12
American Institute of Aeronautics and Astronautics
Figure 11. Example of ADS-B, radar, and EO integration: SDI tracks.
V. Conclusion
The SDI design for an autonomous SAA system was discussed in detail in terms of its design objective,
architecture, and algorithmic approaches. The test and evaluation of SDI involved a systematic build-up from
Simulink desktop simulation, HWIL simulation, to flight test. The flight test results presented show that SDI
performed very well in achieving its design objective of integrating different combinations of ADS-B, radar, and
EO. Specifically, it demonstrates the improvements in track integrity and continuity over what could be achieved by
any individual sensor. While the SDI performance is promising, more extensive testing will be needed especially in
more challenging operating conditions. For example, sun glares, night operation, and adverse weather could
introduce significant challenges to EO while radar would be susceptible to rain and ground clutters. Finally, further
SDI improvements in robustness and its ability to integrate all four sensors (i.e., ADS-B, radar, EO, and TCAS) will
be developed and demonstrated as part of the coming additional flight test rounds.
Acknowledgments
The authors would like to thank the funding support by AFRL. Acknowledgments are also due to the teammates,
Defense Research Associates, ICx Technologies, Bihrle Applied Research, Calspan Corporation, and FAA
Technical Center for their significant roles in making the AFRL SAA system development program and its flight
demonstration effort a success.
13
American Institute of Aeronautics and Astronautics
References
1
Graham, S., Chen, W. Z., De Luca, J., Kay, J., Deschenes, M., Weingarten, N., Raska, V., and Lee, X., “Multiple-Intruder
Autonomous Avoidance Flight Test,” AIAA Infotech@Aerospace Conference, 2011.
2
RTCA Document No. DO-185A, “Minimum Operational Performance Standards for Traffic Alert and Collision Avoidance
System II (TCAS II) Airborne Equipment,” 1997.
3
RTCA Document No. DO-260, “Minimum Operational Performance Standards for 1090 MHz Automatic Dependent
Surveillance – Broadcast (ADS-B),” 2000.
4
Bonato, M. and Scally, L. J., “Unmanned Sense and Track Radar (USTAR),” AIAA Infotech@Aerospace Conference, 2011.
5
Jazwinski, A., Stochastic Processes and Filtering Theory, Dover, 2007.
6
Julier, S. J. and Uhlmann, J. K., “A New Extension of the Kalman Filter to Nonlinear Systems,” International Symposium on
Aerospace/Defense Sensing, Simulation and Control, 1997.
7
Wan, E. A. and Merwe, R. van der, “The Unscented Kalman Filter for Nonlinear Estimation,” IEEE Symposium 2000.
8
Arulampalam, M. S., Maskell, S., Gordon, N. and Clapp, T., “A Tutorial on Particle Filters for Online Nonlinear/NonGaussian Bayesian Tracking,” IEEE Transactions on Signal Processing, vol. 50, no. 2, 2002, p. 174-188.
9
Kuhn, H., “The Hungarian Method for the Assignment Problem,” Naval Research Logistical Quarterly, vol. 2, 1955, p. 8397.
10
Munkres, J., “Algorithms for the Assignment and Transportation Problems,” Journal of the Society for Industrial and
Applied Mathematics, vol. 5, 1957, p. 32-38.
11
Jonker, R. and Volgenant, A., “A Shortest Augmenting Path Algorithm for Dense and Sparse Linear Assignment
Problems,” Journal of Computing, vol. 38, 1987, p. 325-340.
12
Drummond, O., Castanon, D., and Bellovin, M., “Comparison of 2-D Assignment Algorithms for Sparse, Rectangular,
Floating Point, Cost Matrices,” Journal of the SDI Panels on Tracking (Institute for Defense Analyses, Alexandria, VA), no. 4,
1990, p. 81-97.
13
Levedahl, M., “Performance Comparison of 2-D Assignment Algorithms for Assigning Truth Objects to Measured Tracks,”
Signal and Data Processing of Small Targets, Proceedings of SPIE, vol. 4048, 2000.
14
Malkoff, D. B., “Evaluation of the Jonker-Volgenant-Castanon (JVC) Assignment Algorithm for Track Association,” SPIE,
vol. 3068, 1997.
15
Bar-Shalom, Y., Multitarget/Multisensor Tracking: Applications and Advances, Artech, 2000.
16
Wald, A., “Sequential Tests of Statistical Hypotheses,” Annals of Mathematical Statistics, vol. 16, no. 2, 1945, p. 117-186.
17
Wald, A. and Wolfowitz, J., “Optimum Character of the Sequential Probability Ratio Test,” Annals of Mathematical
Statistics, vol. 19, no. 3, 1948, p. 326-339.
14
American Institute of Aeronautics and Astronautics