FEMA`s Romance with LiDAR

Example comparison between LiDAR-derived
stream cross section and field surveyed cross
section. Many such NCFMP examples demonstrated
that the number of surveyed cross sections could
be significantly reduced by the use of LiDAR.
FEMA’s Romance with LiDAR
I
n 1994, Brigadier General Gerald E.
Galloway chaired the Interagency
Floodplain Management Review
Committee following the devastating
floods of 1993. His report, widely known
as the Galloway Report, recommended
that FEMA evaluate newer technologies that could yield improved digital
elevation models (DEMs) for floodplain
modeling. Recognizing USGS as the
nation’s elevation expert, The Federal
Emergency Management Agency
(FEMA) asked USGS to evaluate
accuracies achievable from LiDAR and
IFSAR. The USGS Open-File Report
96-401 entitled: “Digital Elevation
Model Test for LIDAR and IFSARE
Sensors,” documented USGS’ evaluation
of LiDAR data from a LiDAR system
developed by the Houston Advanced
Research Center (HARC) in cooperation
with FEMA and the NASA Goddard
Space Flight Center. The LiDAR was
flown from an elevation of 3,000’ above
ground level (AGL) over a 3 km2 area
near Glasgow, Missouri which had
been hard hit by the 1993 floods. The
tested RMSE was 37 cm in open ground,
2.65 meters in low cover, 2.0 meters
in scrub, and 3.8 meters in trees—all
non-impressive by today’s standards.
Subsequently, FEMA became an
early LiDAR pioneer and, to the best of
my knowledge, the first federal agency
to adopt LiDAR as its technology for
the future with published guidelines
BY DR DAVID F. MAUNE
Displayed with permission • LiDAR Magazine • Vol. 6 No. 4 • Copyright 2016 Spatial Media • www.lidarmag.com
“ EMA became an early LiDAR pioneer and, to the
F
best of my knowledge, the first federal agency to
adopt LiDAR as its technology for the future with
published guidelines and specifications.
”
and specifications for Flood Insurance
Studies (FIS). To this day floodplain
managers continue to rely more on
LiDAR than any other user group. I
am proud to have been part of FEMA’s
engagement with LiDAR from its
beginnings. My company (Dewberry)
has performed independent QA/QC
of more than 400,000 square miles
of LiDAR, and a good chunk of that
was for FEMA to ensure that the
LiDAR data satisfied requirements
for the National Flood Insurance
Program (NFIP), which produces
Flood Insurance Rate Maps (FIRMs)
that delineate a community’s flood risk
zones. The NFIP’s value and credibility
is largely based on accurate mapping
of flood risks, and LiDAR is best for
accurate hydrologic and hydraulic
(H&H) modeling, at affordable costs.
FEMA has progressed with the times
regarding its accuracy requirements
for topographic data to be used for
floodplain modeling and mapping.
Historically, FEMA’s elevation data
requirements were based on 4’
contours from photogrammetry and
“best available data.” This article will
walk through the timeline of LiDAR
guidelines and specifications used for
FEMA flood risk mapping projects, as
best I can recall from happenings nearly
two decades ago. FEMA has adopted
higher standards as they have become
technically feasible and is committed to
using the best topographic data for new
mapping projects.
Figures 1A and 1B could be interchangeable, the point being that LiDAR
data may either show that structures
previously outside FEMA’s Special Flood
Hazard Area (SFHA) should be inside
the SFHA—or vice versa. Without
accurate LiDAR data and H&H models,
homeowners will invariably question
whether they really need flood insurance. In this example, many properties
were removed from the SFHA, but
in the many instances, LiDAR data of
improved accuracy and currency shows
more properties at risk of flooding than
previously mapped.
Schoharie County, NY
FEMA’s actual use of LiDAR did not
begin until 1998 as I can best recall. In
1998, Dewberry was tasked to prepare
a LiDAR DEM Accuracy Verification
report for FEMA’s first use of LiDAR,
in Schoharie County, NY. We were
specifically tasked to independently
determine if LiDAR data, acquired
from 6,000’ AGL, was equivalent to
topographic maps with 1- or 2-foot
contours as desired for future FEMA
accuracy standards. FEMA desired
DEMs equivalent to either 1- or 2-foot
contours (depending on what proves
achievable from this LiDAR evaluation)
for complete automated hydrologic and
hydraulic (H&H) modeling and analysis.
After flooding in 1996, Schoharie
County planners and emergency
management officials approached
the New York State Department
of Environmental Conservation
(NYSDEC) wanting new flood maps
that more accurately represented their
threat from flooding. Schoharie County
was planning a new GIS-based E-911
system and wanted to incorporate
updated flood hazard mapping data into
the system. Concurrently, FEMA was
preparing a plan to modernize the flood
hazard maps across the United States
using the latest technologies and collaborating with state and local partners
to instill local ownership of the program.
NYSDEC’s needs were compatible with
FEMA’s goals, and the pilot project
was funded. PAR Government Systems
was the engineering firm for the flood
studies and EagleScan was the LiDAR
provider. Dewberry developed FEMA’s
LiDAR accuracy validation procedures
and proposed we use the new National
Standard for Spatial Data Accuracy
(NSSDA) published in 1998 by the
Federal Geographic Data Committee
(FGDC) that required accuracy testing
against checkpoints of higher accuracy
to determine vertical accuracy at
the 95% confidence level, computed
by multiplying RMSEz by 1.9600.
Unfortunately, the NSSDA’s statistics
assumed that all errors followed a
normal error distribution, and that
proved to be an erroneous assumption
for testing the accuracy of LiDAR
Displayed with permission • LiDAR Magazine • Vol. 6 No. 4 • Copyright 2016 Spatial Media • www.lidarmag.com
Digital Terrain Models (DTMs) in
vegetated terrain.
Dewberry’s Special Project Report,
dated December 31, 1998, documented
systematic issues pertaining to LiDAR
sensor calibration, relative accuracy
(inaccuracy) between overlapping
swaths, data processing errors, and
large data voids in forested areas where
most points had been filtered out.
Concurrently, we determined that it
was unfair to the LiDAR data provider
to use QA/QC checkpoints on steep
slopes or bridge abutments where
LiDAR interpolation would unfairly
make the LiDAR look worse than
it really was. After removing unfair
checkpoints, the data initially tested
at 4.44 feet at the 95% confidence level
(approximately equivalent to 7.5 ft
contour accuracy). EagleScan deter-
Figure 1A: Old SFHA (note homes shown under water), pre-LiDAR. This image shows homes
that were previously required to purchase flood insurance.
Images courtesy of the Georgia Geospatial Advisory Council (GGAC).
“ …we determined that it was unfair to the LiDAR data
provider to use QA/QC checkpoints on steep slopes
or bridge abutments where LiDAR interpolation would
unfairly make the LiDAR look worse than it really was.
”
mined that systematic below-ground
errors required reprocessing of LiDAR
data with new atmospheric refraction
models, and they revised procedures
for vegetation removal.
Dewberry’s first conclusion/
recommendation stated: “For the
Schoharie FIS, the reprocessed LiDAR
DEMs should be adequate to support
automated hydrologic modeling of
watersheds, but they should not be used
alone for automated hydraulic modeling
of floodplains. As already planned by
PAR, detailed cross section surveys
should be performed to augment the
DEMs for hydraulic modeling.”
Pinellas County, FL
Dewberry’s second accuracy testing of
LiDAR was in 1999 when we evaluated
LiDAR data of Pinellas County, FL.
FEMA wanted to determine if LiDAR
DEMs have the resolution and accuracy
necessary for automated hydraulic
modeling, without the need for expensive, ground-surveyed cross sections.
At that time, our goal was RMSEz of
15 cm within floodplains because we
thought this accuracy was achievable.
The county provided 677 checkpoints
in five major land cover categories
determined to represent the floodplain
area. The tested RMSEz values for these
five categories were as follows:
1. Woods: RMSEz = 25.3 cm
2. Tall weeds and agricultural fields:
RMSEz = 14.0 cm
3. Short grass or weeds, or bare earth,
sand or rocks: RMSEz = 10.6 cm
4. Mangrove: RMSEz = 57.8 cm
5. Sawgrass: RMSEz = 66.3 cm
We learned that the mangrove and
sawgrass was so dense that the LiDAR
last returns were not penetrating to the
ground. Because the surrounding terrain
was very flat, we recommended that
the LiDAR points within mangrove and
Displayed with permission • LiDAR Magazine • Vol. 6 No. 4 • Copyright 2016 Spatial Media • www.lidarmag.com
with DEM accuracy reported for
up to five representative land cover
categories. Contractors were required
to select a minimum of 20 test points
for each major vegetation category,
with a minimum of 60 test points for
a minimum of three major vegetation
categories. In addition to performance
standards, it provided guidelines for
system calibration, flight planning, GPS
base stations, LiDAR post-processing,
QA/QC, and deliverables.
The North Carolina Floodplain
Mapping Program (NCFMP)
Figure 1B: New SFHA, post-LiDAR. This image shows homes (outlined in red) that are no
longer required to purchase flood insurance, though still recommended. Other LiDAR
datasets demonstrate more homes require insurance.
Images courtesy of the Georgia Geospatial Advisory Council (GGAC).
sawgrass polygons be removed as ground
points and filled in with interpolation
from surrounding ground points; today,
those would be called Low Confidence
Areas. As with Schoharie County, the
wooded areas in Pinellas County proved
that LiDAR elevation errors in forested
areas do not follow a normal error
distribution, that a few outliers cause the
RMSEz values to exaggerate the inaccuracies, and a better way needed to be
found to assess the accuracy of LiDAR in
densely vegetated terrain. Yet, after correction for systematic errors, the LiDAR
data nearly satisfied FEMA’s traditional
requirement for 4-foot contour accuracy
or better in most vegetation categories
(with the exception of mangrove and
sawgrass), and it was clear to me that
photogrammetry could not have done
any better. We concluded that LiDAR
could not be used for automated
hydraulic modeling without the need
for surveyed cross sections to accurately
determine stream channel geometry both
above and below the water level.
FEMA 37, Appendix 4B,
Airborne Light Detection and
Ranging Systems
I worked with Karl Mohr and Mary
Jean Pajak at FEMA headquarters,
and Stan Hovey of Michael Baker
Jr. Inc., to publish Appendix 4B,
Airborne Light Detection and Ranging
Systems, to FEMA 37, “Guidelines and
Specifications for Study Contractors”
in 1999, with minor revisions in 2000.
Based on what was then believed to
be achievable, it specified a maximum
5-meter post spacing and 15 centimeter
RMSEz requirement for all major
vegetation categories that predominate
within the floodplain being studied,
In 2000, twenty-two Federal and local
community entities joined North
Carolina (as FEMA’s first Cooperating
Technical State) in an agreement to work
together to produce accurate, up-to-date
flood hazard data for the State of North
Carolina, using statewide LiDAR and
automated H&H modeling to the degree
possible. Dewberry worked closely with
John Dorman, Gary Thompson and
others at the state (and FEMA) to help
ensure the program would be successful.
Various issue papers developed LiDAR
specifications for data acquisition, calibration, required accuracy and nominal
point spacing, generation of bare-earth
ASCII files, generation of TINs and
breaklines, tile sizes, and development
of DEMs in four different file formats.
Quality control (QC) procedures were
developed for accuracy testing, visual
QC of cleanliness, QC of LiDAR-derived
cross sections, evaluation of different
methods for generating breaklines,
comparison of datasets between different
LiDAR and H&H contractors, and
comparison with other existing LiDAR
datasets. Another issue paper addressed
maintenance, archiving and dissemination of the NCFMP LiDAR data.
Displayed with permission • LiDAR Magazine • Vol. 6 No. 4 • Copyright 2016 Spatial Media • www.lidarmag.com
In 2000-2001, Dewberry and
NCFMP participants were struggling
to determine what LiDAR accuracies
were achievable, especially in forests
where we knew (and later confirmed)
that DTM errors would not follow a
normal error distribution. We required
calibration data to be collected during
each flight over a calibration course
established at each airport. The NCFMP
required a vertical RMSEz of 20-cm for
coastal counties and 25-cm for inland
counties, computed after discarding
the worst 5% of the checkpoints as
there was no written guidance from
anyone as to what to do when errors do
not follow a normal distribution. The
NCFMP chose five land cover categories
(bare-earth and low grass, weeds and
crops, scrub, forested, and built-up) and
surveyed 120 QA/QC checkpoints per
county, i.e., 20 per county in each of four
land cover categories plus 40 per county
in forested areas.
Based on the 120 checkpoints per
county provided by Gary Thompson of
the North Carolina Geodetic Survey,
Dewberry performed the accuracy
assessments and submitted 100 LiDAR
Accuracy Assessment Reports, one for
each of the 100 counties in the state.
Some counties were better than others,
but when we aggregated all 12,000
checkpoints into a single spreadsheet,
the average RMSEz was 18.5-cm
which satisfied the criteria for 2-foot
contour accuracy. However, this statistic
only pertained to the best 95% of the
checkpoints, after the worst 5% had
been removed from the calculations.
Although not statistically defensible in
terms of NSSDA requirements (that
erroneously assumed all errors followed
a normal distribution), this was still
better than the original National Map
Accuracy Standards (NMAS) of 1947
which essentially reported the accuracy
for the best 90% of points tested. Using
2 ft contours as an example, the NMAS
would require that no more than 10
percent of the elevations tested be in
error by more than 1 ft (one half the
contour interval), but the 10% outliers
had no limitations at all. Thus, the
NCFMP’s LiDAR would easily have
passed NMAS requirements for 2-ft
contour accuracy.
Appendix A: Guidance for
Aerial Mapping and Surveying,
to FEMA’s “Guidelines and
Specifications for Flood Hazard
Mapping Partners”
Working with Paul Rooney at FEMA
headquarters, Dewberry provided the
primary authors for FEMA’s Appendix
A approved in February of 2002 and
revised in April of 2003. Appendix A set
FEMA’s requirements in terms of the
NSSDA though it also provided contour
interval equivalents, i.e., specifying
FEMA’s requirements for elevation data
to have 2-ft equivalent contour accuracy
for flat terrain and 4-ft equivalent contour accuracy in rolling to hilly terrain—
regardless of the technology used. It also
included guidelines and specifications
for base maps; horizontal accuracy;
data requirements for different forms
of flood studies; different data models
(mass points, breaklines, TINs, DEMs
and/or contours); file size, tile size and
buffers; mapping areas; cross sections;
treatment of hydraulic structures
(bridges, culverts, dams, weirs); datums,
projections and coordinate systems;
data formats; hydro-enforced elevation
data; and ground surveys for photogrammetric control, cross sections,
hydraulic structures and checkpoints. In
addition to FEMA’s extensive guidance
for photogrammetric surveys, it also
included section A.8 on LiDAR surveys.
The LiDAR portion of Appendix A
addressed LiDAR system definitions;
general guidelines for use; performance
standards (including data voids, artifacts,
outliers, system calibration, flight planning, GPS base stations); post-processing
of LiDAR data to include breakline
requirements; QA/QC of LiDAR to
include accuracy testing and analysis
of errors by land cover categories,
locations, dates, and sensors; verification
of airborne GPS, inertial measurement units (IMUs), and laser ranges;
correction of systematic errors; cross
flight verification; and various forms
of deliverables. The LiDAR portion
The LiDAR portion of these FEMA guidelines
remained the LiDAR industry’s de facto
standard for seven years.
Displayed with permission • LiDAR Magazine • Vol. 6 No. 4 • Copyright 2016 Spatial Media • www.lidarmag.com
of Appendix A remained the LiDAR
industry’s de facto standard until Karl
Heidemann presented the draft USGS
LiDAR Guidelines and Base Specification
Version 13 at the International LiDAR
Mapping Forum (ILMF) in 2010.
Appendix A required field surveyed
cross sections immediately upstream and
downstream of bridges and culverts, to
include channel invert elevations for the
deepest part of the channel. It directed
Mapping Partners to survey intermediate
cross sections where bridges and culverts
“ documented procedures for using the
95th percentile to estimate the accuracy
of LiDAR data in vegetated and forested
areas at the 95% confidence level, based
largely on additional lessons learned
from the NCFMP. In 2010, Harold
Rempel and I were the principal authors
of FEMA’s Procedure Memorandum No.
61—Standards for LiDAR and Other
High Quality Digital Topography that
adopted the 95th percentile methodology and aligned FEMA’s LiDAR
specifications with USGS’ draft v13
Appendix A remained the LiDAR industry’s de facto
standard until Karl Heidemann presented the draft USGS
LiDAR Guidelines and Base Specification Version 13 at
the International LiDAR Mapping Forum (ILMF) in 2010.
”
are more than 1,000 feet apart, especially
where a significant change in conveyance
occurs between cross sections. These
intermediate cross sections could be “cut”
from stereo-photogrammetric or LiDAR
datasets so long as there was no significant change in stream bed geometry
below the water level. With automatic
H&H and LiDAR datasets, the cross
sections could be more numerous and
truly representative of shorter reaches.
Other LiDAR Standards,
Guidelines and Specifications
In 2004, I was the primary author
of the NDEP Guidelines for Digital
Elevation Data, version 1.0, and the
ASPRS Guidelines: Vertical Accuracy
Reporting for Lidar Data, both of which
specifications, but with vertical accuracy
requirements linked to the level of flood
risk identified by FEMA.
National Enhanced Elevation
Assessment (NEEA)
In 2012, Dewberry authored the
National Enhanced Elevation
Assessment (NEEA) sponsored by
USGS, FEMA, NRCS, NOAA, NGA
and others, that directly led to USGS’
current 3D Elevation Program (3DEP)
based on QL2 LiDAR or better (2 or
more points/m2 with RMSEz ≤ 10-cm).
With input from 34 Federal agencies, 13
private and non-profit organizations and
all 50 states, the NEEA estimated annual
flood risk management benefits between
$295M and $500M from nationwide
coverage of QL2 LiDAR. FEMA had
identified many areas throughout the
country where the flood hazards shown
on the older FIRMs understated the
true risk of flooding; those who built to
standards on those older maps were in
fact subject to a higher probability of
flooding. FEMA also found that in some
areas flood hazards on the older maps
overstated the true risk (Figure 1A)
which means those properties insured
under the NFIP were paying more
than they should be paying. Overall,
FEMA’s experience indicates that map
updates tend to be fairly even between
adding and removing homes from the
SFHA. Older data do not consistently
overstate or understate flood risk; there
is some of both. Figure 1B illustrates
the improvements in map accuracy
attained by the use of LiDAR data; more
than 300 structures, some of which are
outlined in red, were removed from the
SFHA in Towns County, GA, through
the use of higher accuracy LiDAR data.
In other locations nationwide, additional
structures are added to SFHAs as a result
of more-accurate data and modeling. It
works both ways. Whether flood risks
are currently understated or overstated,
higher accuracy LiDAR yields the
following benefits: (1) structures
insured at appropriate levels; (2) more
consistent insurance ratings through
better information about risk; and (3)
more insurance purchased because of
improved understanding of risk.
FEMA’s Elevation Guidance
(November, 2015)
FEMA’s latest Elevation Guidance
for Flood Risk Analysis and Mapping
(November 2015) retired FEMA’s
Displayed with permission • LiDAR Magazine • Vol. 6 No. 4 • Copyright 2016 Spatial Media • www.lidarmag.com
Appendix A (2003) and PM 61 (2010)
while aligning FEMA’s requirements with the latest USGS LiDAR
Base Specification, V1.2 from Karl
Heidemann at USGS and the ASPRS
Positional Accuracy Standards for
Digital Geospatial Data (Nov, 2014) for
which Karl Heidemann and I authored
the LiDAR portions. This requires
LiDAR point density of at least 2 points/
m2 with RMSEz ≤ 10-cm, meaning that
FEMA’s requirements have improved
from 4 ft contour accuracy in 1998
(from photogrammetry) to the current
1 ft contour accuracy (from LiDAR). I
am proud to have been part of FEMA’s
romance with LiDAR for nearly 20 years
and am thrilled that FEMA is actively
supporting the 3DEP which is vital to all
of us in the LiDAR profession.
Dr. David Maune is an Associate Vice President at Dewberry where he is an elevation
specialist and manages LiDAR, IFSAR and photogrammetric projects for USGS, NOAA, FEMA,
USACE, and other federal, state and county
governments. He specializes in independent
QA/QC of LiDAR data produced by others.
He is a retired Army Colonel, last serving as
Commander and Director of the U.S. Army
Topographic Engineering Center (TEC), now
the Army Geospatial Center (AGC).
In 1998, he authored NOAA’s National
Height Modernization Study on how to
modernize the National Height System in
the U.S. based on Continuously Operating
Reference Stations (CORS), differential GPS,
LiDAR and IFSAR. Between 1998 and 2010,
he authored all major FEMA guidelines for
LiDAR. In 2004, he authored the Guidelines
for Digital Elevation Data published by the
National Digital Elevation Program (NDEP).
In 2001 and 2007, he was the editor and
principal author of the 1st and 2nd editions
of Digital Elevation Model Technologies
and Applications: The DEM Users Manual,
published by ASPRS, with the 3rd edition
planned for 2017. In 2012, he authored the
National Enhanced Elevation Assessment
(NEEA) report that provided the blueprint
for today’s 3D Elevation Program (3DEP). In
2014, he co-authored the ASPRS Positional
Accuracy Standards for Digital Geospatial
Data. In 2015, he was the editor and
principal author of USACE EM 1110-1-1000,
Photogrammetric and LiDAR Mapping. In
2016 he won the ASPRS Photogrammetric
Award for outstanding achievement in the
field of photogrammetry. Dr. Maune earned
his PhD in Geodesy and Photogrammetry
from The Ohio State University in 1973.
Displayed with permission • LiDAR Magazine • Vol. 6 No. 4 • Copyright 2016 Spatial Media • www.lidarmag.com