Defining and Orienting Surfaces in Complex Point Cloud Datasets

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, TEXAS A&M UNIVERSITY, TECHNICAL REPORT TAMU-CS-TR-2009-4-1
Breaking the Fog: Defining and Orienting Surfaces in
Complex Point Cloud Datasets
Keith E. Biggers, Caleb G. Wells, and Glen N. Williams
Abstract— We present a vertex clustering algorithm for the purposes of surface determination and normal estimation that can
help provide detailed visualizations of complex point cloud datasets. The proposed method combines a novel bucket and layer
spatial partitioning scheme, along with an iterative process for surface subdivision based on the optimization of quality-of-fit
statistics. Our approach can efficiently decompose and approximate a dataset through the local classification and fitting of surface
regions. The algorithm uses a standard least squares approach combined with Delaunay-based triangulation for developing these
approximated surfaces. To demonstrate the effectiveness of our approach, we execute the algorithm on several real-world
datasets scanned from complex environments. We perform an analysis of the various techniques presented and provide a
comparison of our approach with the standard k-nearest neighbors method commonly used for solving this problem. Through this
performance analysis we show that as the complexity of the datasets increase, the performance and accuracy of our proposed
approach continues to function at an effective level.
Index Terms—Vertex clustering, surface determination, normal estimation, layered surfaces, uncertain environments.
1
INTRODUCTION
With the portability and increasing use of imaging sensors, efficient
and accurate reconstruction from detailed scans of complex
environments has introduced a new set of challenges for developing
both visualizations and virtual environments. As the size and
underlying surface complexity contained in these datasets increases,
correctly clustering point samples into distinct surfaces becomes an
increasingly more difficult and expensive task. Figure 1 shows two
example environments containing such complexity. Note the number
of distinct surfaces and the proximity in which they exist in this
environment. Many different challenges can arise and interfere with
accurately distinguishing between different surface elements.
However, performing an analysis of the data in an organized fashion
to produce accurate approximations is a vital component of the
overall surface reconstruction process. Correctly defining and
distinguishing between different surface components is important,
because without this identification, algorithms cannot construct a
correct surface representation.
Decomposing a dataset into a set of surfaces, provided only the
spatial information a set of discrete point samples offers, requires a
methodology that can numerically distinguish between different
surface components. In addition, the algorithm must also be able to
develop an accurate continuous approximation of these identified
surface samples. One commonly used approach relies on a k-nearest
neighbors search combined with a least squares fitting [19]. While
this approach works very well in practice, it is time consuming and
can encounter difficulty under certain conditions. In this paper we
provide an alternative solution that attempts to address these
concerns. The primary contributions of this paper are:
• A systematic methodology for incrementally decomposing
complex point cloud datasets consisting of many surfaces
• An alternative approach for efficiently defining and
orienting these disparate surfaces
• A technique for dealing with uncertainty in areas of high
surface complexity
In the following sections we begin by describing the complexity
of the problem in detail along with the related work. We present our
solution and implementation details, example results, and a
comparative analysis showing the effectiveness of our proposed
K. Biggers and G. Williams are with the Department of Computer Science
and Engineering, Texas A&M University, College Station, TX, 77843-3112.
E-mail: [email protected]; [email protected].
approach. Finally, we conclude with the current research state and
intended future direction for this work.
2
BACKGROUND
Surface reconstruction is the classical problem where, given a set P
of unstructured three-dimensional points (i.e., a point cloud assumed
to sample an unknown surface or set of surfaces), generate an
approximated surface representation S that characterizes these point
samples. This representation may involve a single continuous surface
or many disparate surface patches depending on the underlying
dataset and reconstruction algorithms chosen. Researchers have
performed extensive work in this area resulting in many different
approaches. Some of the popular methods include Delaunay
triangulation [3, 4, 6, 12, 15] and implicit surfaces [1, 9, 11, 19, 20,
23]. In addition, there are many algorithms that support the surface
reconstruction process such as surface segmentation [5, 17] and
feature recognition [18, 21]. Aside from surface reconstruction, there
are also other alternatives for visualizing surfaces in a continuous
manner [25, 30]. Whether it is for surface generation or simply
rendering, one element that many of these approaches require is an
accurate normal estimation describing the general orientation of the
point samples. While some speciality scanners are capable of
automatically generating these normal approximations, this is an
exception and not the rule. Thus, these algorithms must either
implement their own approaches or depend on commonly used
methods performed as part of a pre-processing step for generating
this necessary information.
There are several approaches researchers typically rely on for
defining and orienting surfaces. The most widely used process
establishes the k-nearest neighbors for a point of interest p, and then
performs a least squares fitting of this neighborhood (i.e., typically
through the use of principal component analysis) [19]. The resulting
values characterize a least squares plane approximating the
representative points. These approximations are collected across the
entire dataset and then used to form a neighbor graph. A general
assumption about the model (i.e., that all points belong to a
continuous surface and a normal directed upwards from the highest
point will always orient outwards from the surface), followed by a
graph traversal can correctly orient the normal approximations (i.e.,
determine inside or outside directionality). This approach uses the
normal of the least squares plane to approximate the orientation of
each data point.
DEPARTM
MENT OF COMPUT
TER SCIENCE AND
D ENGINEERING, TEXAS A&M UNIV
VERSITY, TECHNIICAL REPORT TAMU-CS-TR-2009-4
4-1
As thee underlying suurfaces the poinnt clouds represent become
more invoolved (e.g., due to multiple objeects, partially vissible/hidden
objects, etc.)
e
and the unnderlying surfacces become moore complex
(e.g., due to high variatiion in curvaturee, fine grained detail,
d
etc.),
these apprroaches begin too break down duee to the sheer coomplexity of
the enviroonment and unnderlying assum
mptions that the algorithms
must makke to accuratelyy capture the detail.
d
Figure 1 shows two
example environments
e
coontaining these various complexxities. They
can resultt in voids and varying point resolutions,
r
and ambiguous
regions within the dataset where an algoriithm cannot easily segregate
surfaces. Many
M
of the preeviously discusssed approaches can
c run into
difficulty under these uncertain conditionss.
3
Fig. 1. Examples of complex environ
nments.
Pauly et al. proovide an extensio
on to this approaach that weighs all
of the
t points in the neighborhood based
b
on their diistance to the point
of interest
i
p to alloow closer neighb
bors to provide more influence to
the approximation [24]. Mitra et all. also provide ann extension to thhis
appproach that takes into account the effects of neighborhood
n
sizze,
curvvature, samplinng density and noise, and atteempts to find an
optimal neighborhoood size [22].
Amenta et al. present a Voro
onoi-based methhod that finds the
t
furtthest Voronoi vertex (called the pole and approximates the
t
norrmal) and this appproach works for “noise-free”” datasets [2]. Dey
D
andd Goswami provvide an extension
n to this Voronooi approach, callled
the Big Delaunay Ball
B algorithm, which
w
works for noisy samples and
a
c
sphere
s
of the Dellaunay tetrahedrron
uses the bounding circumscribing
whiich must meet a criterion regard
ding the overall size
s of the radii (so
(
it caan adequately haandle noise) [13]].
Alexa et al. prresent a Moving
g Least Squares approach whiich
locaally approximates the surface with
w polynomials and a projectiion
procedure [1]. Theeir approach adeequately handles noisy input, but
b
has been extended by Fleishman ett al. to incorporaate robust statistiics
[16]. Fleishman’s extension iterratively identiffies and discarrds
outliers (which cann alter the overall fit of the surfacce), and then reffits
the surface for a better overall apprroximation.
Boubekeur et al.
a present Volum
me-Surface Treees [8] which usee a
com
mbination of octtrees and quadtrrees for hierarchhically clusteringg a
poinnt cloud, and appproximates a surface
s
using a Partition of Unnity
schheme combined with
w a Bloomen
nthal polygonizattion [7]. While the
t
oveerall theme of Booubekeur’s papeer is similar to ouurs, the underlyiing
techhniques we pressent function qu
uite differently. Finally,
F
Dey et al.
provide a good coomparison study
y using several of the previoussly
disccussed approacches executed on various daatasets and undder
diffferent noise condditions [14].
These approachhes all work veery well in pracctice, but typicaally
makke the assumpttion that the su
urface is both closed
c
and clearrly
deffined (i.e., easilyy distinguishablee by simply traciing the surface). In
adddition, general k-nearest
k
neighb
bor approaches can be very tim
me
connsuming on largeer datasets. Whille these approachhes can be sped up
usinng a spatial daata structure [26
6] or technique such as the one
o
described by Sankkaranarayanan et al. [27], as the dataset siize
incrreases, this cann become a tim
me consuming process since the
t
algoorithm must exaamine each pointt and its neighboors found. Regioons
of varying
v
point deensities or areas where multiplee surfaces lie cloose
together can fail to develop an accu
urate approximattion as well. Theese
appproaches rely heavily on acccurate underlyying point clooud
reprresentations, annd noise or otheer complexities can greatly affe
fect
resuults. Finally, ann accurate valuee for k must bee defined and thhis
valuue relies heavilyy on the underly
ying details of thhe dataset itself (as
(
disccussed in [22]).
RFACE DEFINITIO
ON AND ORIENTA
ATION
SUR
Our approoach removes the
t single closeed surface assuumption and
attempts to
t address the ovverall problem inn a very differennt manner. It
assumes great surface complexity andd attempts to provide an
organizedd and iterative process
p
for idenntifying and ressolving this
complexitty. In the follow
wing sections, we
w explain our approach,
a
its
underlyingg data structuress, and the overaall iterative proceess used for
defining the
t surfaces. Wee developed thiss process in succh a fashion
that it cann perform as a pre-processing
p
s
step
and supportt any of the
reconstrucction and renderiing algorithms presented
p
in Secttion 2.
3.1
Approach Overrview
Our approoach begins wiith a fixed view
wpoint and view direction
(either useer-defined or it defaults to paraallel to the prim
mary vertical
axis) whicch defines a plaane that can be fit with a two-ddimensional
grid. This bucket grid alloows for organizeed and efficient data access,
and through an adapttive refinementt process, ourr algorithm
decomposses these bucketts around the complexity
c
of thhe data. By
breaking down
d
buckets innto smaller sub--buckets, the alggorithm can
perform a more precise annalysis on smalller regions contaaining fewer
data poinnts. In addition, each grid squaare gets subdivvided in the
vertical direction
d
throughh the clustering of points into layers (i.e.,
slices of an
a object’s surfaace) and each laayer gets fit withh a series of
planes (i.e., the surface approximationss). The collectioon of these
locally classified and fit elements charaacterizes an oveerall surface
approximaation. Similar to
t buckets, the algorithm alsoo iteratively
refines theese layers to achhieve a more exaact approximation over time.
As the alggorithm developss and refines theese layers, certainn layers will
represent “hidden surfacess” from the initiaally defined view
wpoint.
The algorithm
a
perforrms this refinem
ment process in a stepwise
fashion annd continues unntil certain connditions are met. We have
experimennted with seeveral methodds for perforrming the
decompossition and we will
w present each shortly. The ouutput of this
algorithm is a set of orieented point sampples for use as input
i
to any
approach that requires suuch data. In the early stages thee output is a
coarse appproximation, butt after several reefinement steps, the result is
a more fine-grained appproximation that
t
better fitss the data.
Ultimatelyy the user decidees the resulting resolution
r
since they control
how long the refinement executes and whhen the process has reached
an acceptaable level througgh the definition of a fitting thresshold.
3.2
Data Structures
s
Our approoach uses a fouur level hierarchhy for organizinng data (see
Figure 2 for a simple example). The most primitivee geometric
elements are the points thhemselves whichh get passed as input to the
algorithm (as shown in thhe top left of Figgure 2). Each diiscrete point
contains standard spatial informattion with no defined
interconneectivity. In orderr to allow for orgganized and effiicient access
across all of the dataset we
w use the aforeementioned buckketized grid.
mn that extends inn an infinite
Each buckket can be visuallized as a colum
direction above
a
and below
w the defined grid square and alll points that
reside insside a column are stored in that particular bucket. By
breaking the
t dataset down into a hierarchhy of buckets (aas shown in
the top rigght of Figure 2), it allows quick access
a
to the sett of points in
any given region of the dataset.
d
The oveerall size and reesolution of
DEPARTMENT OF COMP
PUTER SCIENCE AND
A
ENGINEERIN
NG, TEXAS A&M UNIVERSITY,
U
TECH
HNICAL REPORT TAMU-CS-TR-2009-4-1
2
F 2. Data hiera
Fig.
archy (points, buc
ckets, layers, pla
anes and triangle
es).
Fig. 4. Iterative fitting process (tolerance
e colors based on
n R =0.75).
thhis bucket gridd is adaptive and
a
adjusts durring the refineement
p
procedure.
Within eachh bucket, the algorithm ordeers the points and
e
establishes
various logical group
pings (i.e., vertex clusters). We refer
too these groupinngs as layers. Eaach layer attemppts to characterrize a
s
slice
of the real-w
world surface fro
om which the pooints sample. Finnally,
thhe algorithm brreaks layers do
own into a seriees of fitting plaanes.
T
These
fitting plannes attempt to accurately
a
approoximate the geom
metry
thhe point sampless represent (as sh
hown in the botttom left of Figurre 2).
D
Decomposing
a layer into a serries of fitting planes allows a more
m
a
accurate
approxiimation of areass of high curvatture and along sharp
s
e
edges.
These plaanes are easily triangulated to provide an acccurate
representation off the actual apprroximated surfacces (as shown inn the
b
bottom
right of Figure
F
2).
Thee decompositionn centers on thee notion of a defined
d
delta-z
value. For
F each buckett, the algorithm performs a lineear search and
breaks the
t points into layers
l
by findingg vertical gaps (i.e.,
(
through a
simple distance calculation) between diifferent groupinggs greater than
the valuue of delta-z (seee Figure 3 for a two-dimensioonal example).
This process assumes that
t
the user also provides a value for delta-z
m, but similar too the grid size, we
w will present
as inputt to the algorithm
an exam
mple automated method
m
for apprroximating a ‘good’ value later
in Sectiion 5. The algoorithm begins byy performing this initial layer
decompposition for eachh bucket and all points
p
are groupped into layers.
Each laayer is then fit with
w a single leaast squares plane to provide a
startingg approximationn of the surfacce. Our method uses linear
regressiion, but an apprroach based on principal compoonent analysis
would work
w
equally weell. Various statiistics are determ
mined to gauge
the oveerall fitting quaality of each pllane to the datta points (i.e.,
standardd error, correlatiion coefficient, and
a R2). This iniitial step in the
algorithhm results in a coarse
c
approximation of the dataaset (as shown
in the top of Figure 4).. Note that at thhis stage only a single
s
plane is
a buckets are
fit to eaach layer (repressented with a uniique color) and all
of equaal size. Thus, thee resulting approximation is veryy coarse and in
most caases does not acccurately fit the detail of the datasset.
Givven this coarse approximation
a
annd the various fitting
fi
statistics
determiined, the algorithhm can then inccrementally refinne the surfaces
to betteer meet user speccifications. Withh each step, it suubdivides areas
of poorr fitting qualityy into smaller regions
r
and theen refits those
regions to improve the overall approxim
mation. We havee chosen to use
the R2 value
v
of the fittiing planes as thee determining faactor for when
to split planes because it provides a goood approximatioon of how well
the model fits its dataa values. The algorithm
a
could rely on other
fitting statistics equally as well. Thhe user can deefine a fitting
tolerancce; otherwise, the
t algorithm uses
u
a standard default value
represenntative of a ‘goood’ fit. The refinnement of both a bucket and a
layer coontinues until thhe algorithm hass met this toleraance condition,
and at that point the bucket
b
and layeer refinement sttops. Figure 4
shows the
t difference beetween the initiaal coarse approxximation using
the fittiing planes and thhe approximatioon after a series of refinement
steps. Inn the left side of
o the figure, eaach layer is assiggned a unique
color annd a layer may consist of multiiple fitting planees all rendered
with thee same color. Noote that in the boottom diagrams,, the algorithm
has subbdivided both thee buckets and layyers around the complexity of
the dataa resulting in muuch smaller and more precise fitttings. We will
present the refinement approaches
a
the algorithm
a
uses next.
We have investiigated several different approaches for
subdividing a layer annd we illustratee each in Figuree 5. The first
approacch breaks each laayer down into a set of fitting pllanes based on
a defineed number of pllanes and by deccomposing the layer into a set
of equaally sized spatiall intervals. A plaane is then fit too each of these
3.3
3
Determination Proces
ss
T process beggins by organizin
The
ng the data baseed on the previoously
d
defined
grid. Thhis process assum
mes that the useer provides an innitial
resolution of thiss grid as an inpu
ut parameter, buut we will presennt an
e
example
automatted method for approximating
a
‘ggood’ starting values
laater in Section 5. The algorithm
m then places each
e
point inside the
b
bucket
in which its projected po
osition lies (i.e., the point lies innside
thhe column). It then sorts the points
p
within eaach bucket baseed on
thheir vertical position to provid
de an efficient top to bottom seearch
s
strategy.
At thiss point the iniitial organizatioon of the datasset is
c
complete
and thee decomposition process can beggin.
F 3. Two-dimen
Fig.
nsional example of delta-z and fittting elements.
DEPARTM
MENT OF COMPUT
TER SCIENCE AND
D ENGINEERING, TEXAS A&M UNIV
VERSITY, TECHNIICAL REPORT TAMU-CS-TR-2009-4
4-1
inteervals. We will refer to this ap
pproach as the ‘distance’
‘
methood.
Thee second apprroach works in
n a similar fashion
f
except it
deccomposes the layyer into equal in
ntervals based onn the total numbber
of point
p
samples. We will refer to
t this approachh as the ‘densitty’
metthod. The final approach functtions in a very different mannner.
Thiis approach breaaks the planes down
d
based on maximizing
m
the R2
valuue. For each layyer, the algorithm
m uses the pointt furthest from the
t
fittiing plane as a diividing line betw
ween two new fiitting planes in the
t
atteempt to optimizee the average R2 value across the
t layer. We will
w
refeer to this apprroach as the ‘R
‘ 2’ method. In
I each of theese
appproaches, the alggorithm adds on
ne additional plaane per refinemeent
stepp and completelly refits the layeer with each steep taken. While in
manny cases these thhree approachess will function very similarly theere
are instances wheere, depending on data poinnt distribution or
undderlying surface detail, one approach has advanttages over anothher.
Thuus, we perform a comparative an
nalysis of the different methods in
Secction 4.
Fig. 5. Layer refinem
ment approaches
s.
At each step inn the process a layer’s overalll fitting quality is
w
averagee of the R2 valuues
deteermined throughh the use of a weighted
for a layer’s planes (referred as WR
RS). We use the following
f
equatiion
for finding this valuue:
W
WRS
=
∑
n
∑
i
⋅ R
n
i
i
(
(1)
In this
t equation Ri is the R2 value for a particular plane and ni is the
t
num
mber of points thhe plane fits. Ussing this weighteed average ensurres
thatt the algorithm takes into accou
unt both the quaality of fit and the
t
undderlying numberr of points bein
ng fit before deeciding whether to
deccompose a layerr further. This layer refinementt continues untill a
layeer’s WRS value is
i above the defiined tolerance.
To subdivide a bucket, the algorithm usess a similar WRS
WR
calcculation over alll planes containeed within a buckket. If the averaage
is greater
g
than the tolerance, all refinement
r
stopss on layers withhin
thatt bucket and onn the bucket itsself. If the valuue is less than the
t
toleerance, then the algorithm breaaks the bucket down
d
into a set of
subb-buckets, reorgaanizes point disttributions, deterrmines new layeers,
andd then refits all elements. The user can definne the subdivisiion
dim
mension (i.e., thhe decompositiion size of thee bucket) or the
t
algoorithm uses a quadtree
q
approacch by default. Thus
T
as the buckket
getss subdivided, layers and fittin
ng planes are regenerated
r
to fit
smaaller regions, ressulting in more precise
p
fittings.
The greatest complexity of th
his overall proccess is in how to
deteermine the orderr of refining bucckets and layers after
a
the algorithhm
devvelops this initiaal approximation
n. Our current appproach allows for
f
refiinement in one of
o two fashions.. First, it allowss for an interactiive
refinemennt where a user can control thee refinement inn a stepwise
fashion. At
A each step the user can decide whether to subddivide all of
the buckeets or refine all of the layers below the deffined fitting
tolerance. The other alternnative is a semi--automated apprroach where
the algoriithm refines thee layers in seveeral stages, folllowed by a
bucket reffinement for anyy region where the
t collective fittting quality
is below the
t tolerance. Thhis approach alllows the algorithhm to make
several atttempts at subdivviding and fittinng layers, and if an accurate
solution is not found durring these steps, the algorithm decomposes
d
maller spatial regions and re-evaaluates. The
the bucket into a set of sm
user also has
h the ability too control the fittting value at anyy given time
in the proocess and adjust it up or down depending
d
on thhe quality of
the resultts they desire. It is importantt to note that the defined
tolerance value is an impportant element to
t the overall prrocess. This
value deppends greatly onn the details of the
t underlying dataset
d
(i.e.,
point denssity, noise, and surface complexxity). Setting thhis value too
high or loow could greatlly affect both thhe results obtainned and the
number of refinement steeps required to find
f
an acceptabble solution.
Based on our experience we make the geeneral assumptioon that a R2
value of 0.70
0
characterizzes a ‘good’ fit, but this value will depend
heavily onn the dataset dettail and the user’’s desired level of
o accuracy.
The algoriithm uses this vaalue as the standdard default, but the user can
alter the value
v
the algorithhm uses.
The leeast squares fittiing algorithm geenerates a best fit plane by
minimizinng the sum of the squared error
e
for the daata samples
provided. It disregards surface orientaation, and thuss once the
refinemennt step is compleete, the algorithhm needs to corrrectly orient
the develooped approximattions. All of the datasets we usee contain the
original sccan information which helps wiith this orientatioon problem.
The algorrithm can easilyy determine a close
c
approximaation of the
original sccanner location from
f
this inform
mation, and can use
u a simple
dot produuct between the normal approxim
mation and the vector from
the scanneer to the data point to determinee if the general orientation
o
is
correct orr not. If this vaalue is greater than zero then the normal
orientationn is correct, othherwise the algoorithm needs to reverse the
vector. Iff the original sccan information is not availablle, then this
mes much moree complex. We will
w discuss
orientationn process becom
this issue in more detail in Section 5. At the completion of this step,
all normals should be corrrectly oriented outwards from the surfaces
they repreesent.
At thee end of a refineement step, the algorithm trianggulates each
fitting plaane to provide an
a approximate surface visualizzation to the
user. A usser can use this triangulation
t
as a means of validdation and it
can help drive the user interaction
i
proceess. To triangulate a fitting
plane, thee algorithm rotaates it (along with
w
its accompaanying data
points) suuch that its norm
mal is parallel to the vertical axiss and it then
projects thhe points onto the
t plane passinng through the origin. This
reduces thhe complexity of the problem byy allowing the algorithm
a
to
perform a simple two-dim
mensional Delauunay triangulatiion of those
points. Byy reducing the triangulation to
t only two-dim
mensions, it
allows foor very efficiennt construction. However, if the points
representeed by the planne do not consstitute a heightt map, this
approach will generate an
a invalid trianggulation. Once triangulated,
t
the algorithm lifts and rootates the trianguulation back to the original
orientationn resulting in a three-dimennsional triangullation. This
triangulatiion represents thhe defined surfacce and the algoriithm can use
it to help adjust
a
the normaal values for bettter visual qualityy. Similar to
a standardd Gouraud shadding, the algorithhm can adjust each
e
point’s
normal baased on the averrage of the norm
mal values contaained by the
surroundinng triangles.
The overall
o
refinemennt process contiinues until it meets
m
one of
two condiitions: first, all layers
l
are abovee the user defineed tolerance
value, or second, if the usser is satisfied with
w the results and
a chooses
to stop thhe process. Thee right hand side of Figure 4 shows the
progress of
o the refinement process afterr both a single and several
steps (green implies the planes
p
are above the tolerance vaalue and red
1 and 11 show some examples of the final
implies beelow). Figures 10
output of this
t process run on two real-worrld datasets.
DEPARTMENT OF COMP
PUTER SCIENCE AND
A
ENGINEERIN
NG, TEXAS A&M UNIVERSITY,
U
TECH
HNICAL REPORT TAMU-CS-TR-2009-4-1
3.4
3
Implementation
W implementedd the previously described approoach entirely in C++
We
a
and
used the CGAL library [10] for connstructing the twod
dimensional
Delaaunay triangulattions. All effortss were made to make
m
thhis code as opttimal as possiblle. As discussedd, the process iss not
e
entirely
automateed and can invo
olve some user innteraction. How
wever,
w have structurred this interacttion such that thhe user is helpinng to
we
g
guide
the processs and is not requ
uired to control all execution deetails.
T interaction allows for a mo
This
ore correct solutiion upon terminnation
o the process. We
of
W assume the user is generallly familiar withh the
u
underlying
surfaaces the point cloud represennts and capablle of
d
determining
wheen a close approx
ximation exists.
4
RESULTS
Inn this section we
w present som
me sample resuults of our apprroach
e
executed
on a vaariety of datasetts (in both size and complexity)) and
u
using
a variety of parameters. Th
he datasets choseen provide a range of
s
simple
to compllex, as well as both closed annd open surfacees. In
g
general,
we willl evaluate two primary factorrs across all off the
d
datasets.
These factors
f
include efficiency
e
and accuracy with respect
too the underlyingg dataset compllexity. The overrall goal is to obtain
o
thhe most accuratee approximation
n in as few steps and fitting plannes as
p
possible.
We executedd the proposed method on an Intel
I
Pentium D 3.2
G
GHz
CPU withh 2GB of mem
mory. The algoorithm performs all
c
computations
on the CPU and usses the GPU soleely for the purpoose of
rendering.
4.1
4
Example Runs
W chose four datasets
We
d
to run through
t
the algoorithm. We obtaained
thhe first two daatasets (the Bun
nny and Buddhha models) from
m the
S
Stanford
3D Scanning Reposito
ory [28] and both
b
represent single
s
c
closed
surfaces. These providee easier test caases and providde an
a
adequate
baselinne for the algorithm. The thirdd and fourth dattasets
represent real-w
world complex environments consisting
c
of many
m
d
different
objectss with no guaraantees that surfaaces will be cloosed.
T
These
datasets are
a LIDAR scan
ns of real-world simulated dissaster
a
areas
(shown in Figure 1) capturred at Texas A&
&M’s Disaster City®
C
[29]. The latter two
t
datasets pro
ovide a much more
m
challengingg test
f
for
the algorithhm consisting of many diffeerent surface tyypes,
o
occluded
surfacees, and varying point densities. Thus, the colleection
o these various datasets provid
of
des a full spectrrum of potentiaal use
c
cases
for this alggorithm, and wee feel representss an accurate tesst for
e
evaluation
purpooses. Table 1 sho
ows the general statistics for eacch of
thhe datasets usedd.
Table 1. Example Datase
ets
Dataset
Bunnny
Budddha
Wood Pile
Rubblee Pile
Scan
ns
10
0
53
3
3
3
Total Points
P
362,2272
2,944,,912
314,8863
2,307,,174
Each dataset connsists of a series of scans and ourr algorithm inferrs the
E
o
original
scanner locations (for orientation purpposes) based onn the
inncluded registration information
n.
For testing each
e
of the daatasets we chose a standard seet of
p
parameters
to usse across all three methods. Thhe parameters chhosen
a as follows:
are
Table 2.
2 Initial Parametters
Parameterr
Start Grid Size
Delta-z
Algorithm Iteraations
Layer Stepps
Subdivision Size
S
Tolerancee
Grid Rotation (X
X,Y,Z)
Bunny
10
0.1
6
6
2
0.7
(0,0,0)
Buddha
35
0.1
6
8
2
0.7
(π/4, π/4, 0)
Wood
10
0.05
6
12
2
0.7
(0,0,0)
Rubble
50
0.05
6
6
2
0.7
(0,0,0)
In order to evaluaate the effectivveness of each approach we
determiined the total WR
RS (as defined inn Section 3.3) accross the entire
dataset. This value takees into account both the qualityy of the fit, as
well as the number of points includedd in the fit. Havving a high R2
value when
w
fitting a plane
p
to four pooints is a triviaal process, but
obtaininng a high value when fitting a plane
p
to 50 poinnts implies the
algorithhm found a muuch more suitabble fit. This weighted
w
value
allows us
u to better gaugge the overall fitt quality while also
a taking into
accountt the amount off data being fit.. In addition, we
w provide the
numberr of data pointss that get discarrded at each step due to the
inabilityy to fit a planne. The rationalle behind this issue will be
discusseed in more detaiil in Section 5.
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
1
Bunny
2
3
Buddha
4
5
Wood
6
Rubble
W
at each ste
ep using the dista
ance method.
Fig. 6. WRS
Table 3. Discarded Pointss Using Distance
e Method
Iteration
1
2
3
4
5
6
Bunny
B
5
53
129
385
3
3,686
1
12,060
Budddha
222
2622
7066
3,4999
42,005
423,6669
Wood
63
287
945
4,007
17,322
43,975
Rubble
546
2,390
7,481
21,736
52,940
92,513
g
of the WRSS values over eaach step in the
Figuure 6 shows a graph
refinem
ment procedure using the distaance method. Note
N
that the
overall fitting quality generally increaases with each step. Table 3
t number of points discarded at the completioon of each step
shows the
and thiss quantity also inncreases with eacch step. This is a side effect of
the subddivision process and varies betw
ween datasets. Noote in the sixth
iterationn of the rubble pile examplee, the total poiints discarded
represennts only 4% of the entire dataseet. However, in the wood pile
this perrcentage is moree substantial. Thhis could be atttributed to the
overall complexity off this particularr dataset combiined with the
fashion in which the disstance method divides
d
points witthin a layer.
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
1
Bunny
2
3
Buddha
4
5
Wood
W
at each ste
ep using the denssity method.
Fig. 7. WRS
6
Rubble
DEPARTM
MENT OF COMPUT
TER SCIENCE AND
D ENGINEERING, TEXAS A&M UNIV
VERSITY, TECHNIICAL REPORT TAMU-CS-TR-2009-4
4-1
Table 4. Discarded Points
P
Using Den
nsity Method
Iteration
1
2
3
4
5
6
Bunny
5
11
25
76
1,101
3,328
Buddha
B
22
96
385
2,770
19,141
66,250
Woood
63
2733
8677
3,2883
6,2772
7,3999
Rubble
546
1,586
4,393
10,993
20,381
32,090
Figure 7 showss a graph of the WRS values oveer each step in the
t
refiinement proceduure using the density
d
method. These results are
a
veryy similar to thoose obtained wiith the distance method. Tablee 4
shoows the number of
o points discard
ded at the complletion of each steep.
Notte that the densiity approach disccards significanttly fewer points as
com
mpared to the distance
d
approacch and their fittiing accuracies are
a
veryy similar. This can be attribu
uted to the factt that the density
metthod divides poiints into equal sized
s
groupings based on the tootal
num
mber of points inn a layer. This en
nsures a balanceed split resultingg in
few
wer discarded poiints.
1.0
0
0.9
9
0.8
8
0.7
7
0.6
6
0.5
5
0.4
4
0.3
3
0.2
2
0.1
1
0.0
0
4.2
M
Method
Compa
arison
In order to
t gauge the efffectiveness of our
o proposed appproach, we
now provvide a comparattive analysis wiith the k-nearesst neighbors
(referred as KNN) technnique. As previiously discussedd, k-nearest
neighbors represents one of the most coommonly used methods
m
for
solving thhis problem. Thuus, we feel this iss an adequate evvaluation for
judging ouur approach.
We im
mplemented our k-nearest neighbbors approach based
b
on the
efficient octree
o
implemenntation describedd by Samet et al. [26]. After
some expperimentation we
w chose a value
v
of 10 for
f
k. Our
implemenntation iterates thhrough each point in the dataseet, finds the
closest neeighbors, and thhen performs a least
l
square fittiing of those
points. We
W did not im
mplement cachinng or any othher speedup
mechanism
ms.
Table 7 shows a coomparison betw
ween the runtim
mes of our
proposed approaches witth the k-nearestt neighbors appproach. The
runtimes on our three approaches
a
include the time required
r
for
triangulatiing the fitting pllanes, whereas the
t k-nearest neighbor times
simply reeflect the time to query withh no triangulattion of the
elements. These results clearly demonstrate the increased efficiency
our approoach provides. Note
N
that for the rubble pile exxample, our
approach is on average foour times faster than the k-neareest neighbor
approach.
Tablle 7. Total Runtim
me Comparison
Dataset
Bunny
Buddha
W
Wood
Pile
R
Rubble
Pile
1
Bunny
2
3
Buddha
4
5
Wood
6
Rubble
2
Fig. 8. WRS at each
h step using the R method.
2
T
Table
5. Discarde
ed Points Using R Method
Iteration
1
2
3
4
5
6
Bunny
5
35
116
444
3,783
12,545
Buddha
B
22
194
728
3,872
48,345
353,760
3
Woood
63
2800
9411
4,1111
18,3666
41,5000
Table 6. Total Runtimess
Dataset
Bunny
Buddha
Wood Pilee
Rubble Pile
Distance
85s
705s
57s
426s
Density
76s
710s
58s
429s
R2
78s
723s
52s
406s
Finally, Table 6 shows the total runtimes for eaach method. Theese
runntimes are baseed on performin
ng six iterationns and using the
t
parameters as definned in Table 2. Each
E
iteration inncludes a series of
a bucket subd
divisions, and thhe triangulation of
layeer refinements and
all new/altered
n
elem
ments. We feel th
hese runtimes arre very respectabble
in regards
r
to the ovverall fitting vallues obtained. Overall,
O
the density
appproach seems too function bestt due to its higgh quality resuults
com
mbined with the fact that it discaards the fewest nuumber of points.
Density
76s
710s
58s
429s
R2
78s
723s
52s
406s
KNN
K
8
87s
1,4459s
159s
1,7719s
Next we will
w evaluate the quality of fit. Fiigure 9 shows a graph
comparingg the final accurracies of our approach with thoose from the
k-nearest neighbors
n
approoach. This graph also uses the WRS
W value. A
plane wass fit to each k-nearest neighborhhood and WRS values
v
were
measured across the dataset similar to thhe approach usedd in Section
4.1.
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
Rubble
546
2,317
7,206
21,474
52,316
93,250
Figure 8 showss a graph of the WRS values oveer each step in the
t
a
refiinement proceduure using the R2 method. Thesee results again are
veryy similar to thhose obtained using the distaance and density
metthods. Table 5 shhows the numbeer of points discaarded at each steep.
Sim
milar to the disttance method, this
t
approach also discards moore
poinnts due to the fashion
f
in whicch layers are divvided into unevven
poinnt groupings, and
a the inability
y to fit a planee to these isolatted
poinnts.
Distance
8
85s
7005s
5
57s
4226s
Distance
Density
R2
KNN
Bunny Budd
dha Wood
Rubble
Fig. 9. Finnal WRS comparrison for each daataset.
It is im
mportant to poinnt out that for thhe test cases the accuracy of
our approoach is at least on
o par with that of the k-nearest neighbors.
The overaall runtimes of ouur approach are significantly bettter. Figures
10 and 11 show some of the
t results of ouur approach and it should be
q
approxim
mations and
clear that it does indeed generate good quality
does so inn a timely manneer.
5
DISC
CUSSION
As we have
h
shown thrrough the previious results, ouur proposed
approach works well on both
b
simple and complex
c
environnments. The
approach has several advvantages over exxisting techniquees. First and
foremost is the efficiencyy of the algorithhm. As shown, it
i can break
down andd reconstruct a laarge and compleex dataset in veryy little time.
The seconnd advantage is the manner in which
w
the algorrithm adapts
and decom
mposes around the
t complexity of
o the dataset. This
T helps to
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, TEXAS A&M UNIVERSITY, TECHNICAL REPORT TAMU-CS-TR-2009-4-1
improve efficiency as well as reduce the memory footprint by
allowing areas capable of being fit with a single larger fitting plane
to avoid being fit with a number of smaller patches. Finally, the
coarse to fine refinement fashion in which this method functions
allows an initial approximation, and then increased fidelity is
incorporated with each step in the refinement process. The proposed
approach could provide a ‘preview’ capability for larger datasets.
Our approach also has several disadvantages worth noting. The
first disadvantage is the process of discarding point samples due to
the inability to fit a plane. As the algorithm subdivides both buckets
and layers, cases can occur where there are not enough points to fit a
plane (i.e., at least three points). This typically occurs as a result of
dividing the space such that there are isolated point samples. This
includes both the layer methods as well as bucket subdivision. There
are two alternatives for handling these points, discard them or merge
them with the closest layer. We chose to discard the points because
at the end of each iteration step an orientation is assigned to each
data point, so if a point gets discarded it will simply persist its
orientation value from the previous step. The second disadvantage is
that areas near a bucket’s border may result in lesser quality
approximations due to the full detail not getting incorporated into the
model. This is one area where the k-nearest neighbors method
functions much better because it can incorporate data in a full sphere
around the data point. One possible solution to address this problem
is to perform multiple cycles of our algorithm, where buckets
overlap, and then merge the results together. This would be a more
time consuming process, but would result in better quality
approximations. A third disadvantage involves cases where the deltaz value cannot distinguish between different surfaces due to noise or
just general complexity in the dataset. A possible solution for this
issue is to perform our surface fitting approach from each of the
three orthogonal axis directions centered at the defined viewpoint.
The results could then be merged into a single approximation. This
would ensure that the detail of the surface would be captured from
multiple different directions, one of which should be able to
distinguish between the surfaces.
As previously discussed, the user must provide the grid size and
delta-z values as input since these values heavily relate to the
underlying dataset details. We propose the following methods for
automatically determining initial values for these parameters. Since
the grid size is adaptive this value is less critical as the algorithm will
subdivide around the data. However, a possible method could
perform a statistical analysis of a dataset taking into account the
density of point samples and could estimate a ‘good’ starting value
based on these characteristics. The delta-z value is the more critical
value where a close approximation is needed. One potential method
could break the dataset into buckets based on the previous approach,
sort the points and then find the n largest gaps in each bucket. The
algorithm could execute this across all buckets and average these
values to find a ‘good’ starting delta-z value for the specific dataset.
We discussed three methods for layer refinement (i.e., distance,
density and R2), but there are other alternatives that the algorithm
could use as well. One example could take into account both the
fitting statistic and the number of points being fit to ensure that the
focus remains on fitting larger areas consisting of a greater number
of points (i.e., an extension of the WRS value). This approach would
try to limit the refinement of areas containing a small number of
points (e.g., 4-10 points). It would also prevent unneeded
refinements that simply result in the discarding of points. Finally, it
could perform these splits such that it maximizes the number of
points in each new element. Each of these would help to better the
overall approximation generated.
Earlier, we mentioned the problem of orienting the
approximations without use of the original scan information. This is
a very difficult problem because there is little information that the
algorithm can use for making assumptions regarding a surface’s
orientation. Thus, the algorithm must use a simple heuristic, and
unless the surface is closed, there is no guarantee that the normals
will be correct. For example, one such heuristic is the approach
proposed by Hoppe et al. [19] which we presented in Section 2. This
approach uses an assumption about the highest data point along with
surface tracing for determining correct orientation. This heuristic
works if the surface is both continuous and closed, but in the rubble
pile and wood pile datasets we cannot make these assumptions due to
various complexities in the environment (e.g., hidden surfaces,
shadowed areas void of data, etc.). Thus, orientation would at best be
a guessing process based on the surrounding elements. User
interaction could possibly play a role in helping to solve this
problem, but would most likely involve a very manual process.
6
CONCLUSION AND FUTURE WORK
In this paper we presented an alternative solution for defining and
properly orienting surfaces in point cloud datasets. We demonstrated
that our approach is both efficient and generates accurate results. We
presented several alternatives for refinement and each has various
pros and cons. Finally, we demonstrated the effectiveness of our
approach on both simple and complex datasets and showed that it is
effective on environments containing great uncertainty and surface
complexity. Overall this approach provides both a good methodology
for visualizing point cloud data and supports the overall surface
reconstruction process.
In the future, we will continue to investigate different refinement
techniques and hybrid methods to allow the algorithm to better fit the
dataset over time while minimizing the discarding of point samples.
We intend to explore methods that can minimize any user interaction
required through the process. We will also investigate the extension
of these techniques to the overall reconstruction process. For
example, it would be beneficial if the algorithm could easily stitch
together the triangulated surface patches to form continuous surfaces.
Finally, this process lends itself well to parallelization and we intend
to investigate the distribution of computation further. It is our hope
that once we incorporate these various elements, the results of the
algorithm will further improve and will provide for an overall
solution for constructing and visualizing complex virtual
environments from point cloud data.
ACKNOWLEDGEMENTS
We would like to thank the National Institute of Standards and
Technology for providing the two Disaster City® datasets that
propelled this research. We would also like to thank the Stanford 3D
Scanning Repository for providing the Bunny and Buddha datasets
that were used for testing and evaluation purposes.
REFERENCES
[1] M. Alexa, J. Behr, D. Cohen-Or, S. Fleishman, D. Levin, and C.
T. Silva, Computing and rendering point set surfaces. IEEE
Transactions on Visualization and Computer Graphics, 9(1),
pp. 3-15, 2003.
[2] N. Amenta and M. Bern, Surface reconstruction by Voronoi
filtering. Discrete and Computational Geometry, vol. 22, pp.
481-504, 1999.
[3] N. Amenta, M. Bern, and M. Kamvysselis, A new Voronoibased surface reconstruction algorithm. In Proc. of the 25th
Annual Conference on Computer Graphics and Interactive
Techniques, pp. 415-421, 1998.
[4] N. Amenta, S. Choi, T. K. Dey, and N. Leekha, A simple
algorithm for homeomorphic surface reconstruction. In Proc. of
the 16th Annual Symposium on Computational Geometry, pp.
213-222, 2000.
[5] M. Attene, B. Falcidieno, and M. Spagnuolo, Hierarchical mesh
segmentation based on fitting primitives. The Visual Computer:
International Journal of Computer Graphics, 22(3), pp. 181193, 2006.
[6] C. L. Bajaj, F. Bernardini, and G. Xu, Automatic reconstruction
of surfaces and scalar fields from 3D scans. In Proc. of the 22nd
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, TEXAS A&M UNIVERSITY, TECHNICAL REPORT TAMU-CS-TR-2009-4-1
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
Annual Conference on Computer Graphics and Interactive
Techniques, pp. 109-118, 1995.
J. Bloomenthal, An implicit surface polygonizer. In Graphics
Gems IV, Academic Press Professional, Inc., pp. 324-349, 1994.
T. Boubekeur, W. Heidrich, X. Granier, and C. Schlick,
Volume-surface trees. In Computer Graphics Forum (Proc. of
Eurographics 2006), 25(3), pp. 399-406, 2006.
J. C. Carr, R. K. Beatson, J. B. Cherrie, T. J. Mitchell, W. R.
Fright, B. C. McCallum, and T. R. Evans, Reconstruction and
representation of 3D objects with radial basis functions. In
Proc. of the 28th Annual Conference on Computer Graphics
and Interactive Techniques, pp. 67-76, 2001.
CGAL - Computational Geometry Algorithms Library.
http://www.cgal.org, 2008.
B. Curless and M. Levoy, A volumetric method for building
complex models from range images. In Proc. of the 23rd
Annual Conference on Computer Graphics and Interactive
Techniques, pp. 303-312, 1996.
T. K. Dey, J. Giesen, and J. Hudson, Delaunay based shape
reconstruction from large data. In Proc. of the IEEE 2001
Symposium on Parallel and Large-data Visualization and
Graphics, pp. 19-27, 2001.
T. K. Dey and S. Goswami, Provable surface reconstruction
from noisy samples. Computational Geometry: Theory and
Applications, 35(1), pp. 124-141, 2006.
T. K. Dey, G., Li, and J. Sun, Normal estimation for point
clouds: A comparison study for a Voronoi based method. In
Proc. of Eurographics Symposium on Point-Based Graphics,
pp. 39-46, 2005.
H. Edelsbrunner and E. P. Mücke, Three-dimensional alpha
shapes. ACM Transactions on Graphics, 13(1), pp. 43-72, 1994.
S. Fleishman, D. Cohen-Or, and C. T. Silva, Robust moving
least-squares fitting with sharp features. ACM Transactions on
Graphics, 24(3), pp. 544-552, 2005.
M. Garland, A. Willmott, and P. S. Heckbert, Hierarchical face
clustering on polygonal surfaces. In Proc. of the 2001
Symposium on Interactive 3D Graphics, pp. 49-58, 2001.
S. Gumhold, X. Wang, and R. Macleod, Feature extraction from
point clouds. In Proc. of the 10th International Meshing
Roundtable, pp. 293-305, 2001.
H. Hoppe, T. DeRose, T. Duchamp, J. McDonald, and W.
Stuetzle, Surface reconstruction from unorganized points. In
Proc. of the 19th Annual Conference on Computer Graphics
and Interactive Techniques, pp. 71-78, 1992.
M. Kazhdan, M. Bolitho, and H. Hoppe, Poisson surface
reconstruction. In Proc. of the 4th Eurographics Symposium on
Geometry Processing, pp. 61-70, 2006.
Y. Lee, S. Park, Y. Jun, and W. C. Choi, A robust approach to
edge detection of scanned point data. The International Journal
of Advanced Manufacturing Technology, 23(3-4), pp. 263-271,
2004.
N. J. Mitra, A. Nguyen, and L. Guibas, Estimating surface
normals in noisy point cloud data. International Journal of
Computational Geometry and Applications, 14(4-5), pp. 261276, 2004.
Y. Ohtake, A. Belyaev, M. Alexa, G. Turk, and H.-P. Seidel,
Multi-level partition of unity implicits. ACM Transactions on
Graphics, 22(3), pp. 463-470, 2003.
M. Pauly, R. Keiser, L. P. Kobbelt, and M. Gross, Shape
modeling with point-sampled geometry. ACM Transactions on
Graphics, 22(3), pp. 641-650, 2003.
S. Rusinkiewicz and M. Levoy, QSplat: a multiresolution point
rendering system for large meshes. In Proc. of the 27th Annual
Conference on Computer Graphics and Interactive Techniques,
pp. 343-352, 2000.
H. Samet, Neighbor finding in images represented by octrees.
Computer Vision, Graphics, and Image Processing, 46(3), pp.
367-386, 1989.
[27] J. Sankaranarayanan, H. Samet, and A. Varshney, A fast all
nearest neighbor algorithm for applications involving large
point-clouds. Computers and Graphics, 31(2), pp. 157-174,
2007.
[28] Stanford Computer Graphics Lab., The Stanford 3D Scanning
Repository. http://graphics.stanford.edu/data/3Dscanrep/, 2008.
[29] Texas Engineering Extension Service, Disaster City®.
http://teex.com/teex.cfm?pageid=USARprog&area=usar&templ
ateid=1117, 2008.
[30] M. Zwicker, H. Pfister, J. van Baar, and M. Gross, Surface
splatting. In Proc. of the 28th Annual Conference on Computer
Graphics and Interactive Techniques, pp. 371-378, 2001.
Fig. 10. Final approximation of the rubble pile dataset.
Fig. 11. Final approximation of the wood pile dataset.