LS Models - Computer Science

Level Set Models for
Computer Graphics
David E. Breen
Department of Computer Science
Drexel University
Ross T. Whitaker
School of Computing
University of Utah
Outline
• Level Set Model Introduction
APPLICATIONS
• 3D Volumetric Metamorphosis
• Volume Dataset Segmentation
• Surface Editing
Ken Museth
Department of Science and Technology
Linkoeping University, Sweden
What is a Level Set Model?
• A deformable implicit model, φ(X,t) = 0
What is a Level Set Model?
• A deformable implicit model, φ(X,t) = 0
• Sampled representation of dimension n+1
– Images (2D) represent curves
– Volumes (3D) represent surfaces
What is a Level Set Model?
• A deformable implicit model, φ(X,t) = 0
• Sampled representation of dimension n+1
– Images (2D) represent curves
– Volumes (3D) represent surfaces
• Change level set by modifying samples
What is a Level Set Model?
• A deformable implicit model, φ(X,t) = 0
• Sampled representation of dimension n+1
– Images (2D) represent curves
– Volumes (3D) represent surfaces
• Change level set by modifying samples
• Change sample values by solving a PDE
1
This is a Level Set Model!
Changing the image moves
the curve
• Red curve is defined by φ(x,y) = 127
of a
curve!
What is a Level Set Model?
• φ(X) is not defined by a specific equation
What is a Level Set Model?
• φ(X) is not defined by a specific equation
• φ(X) is represented by a regular 3D sampling
– Signed distance volume dataset
What is a Level Set Model?
What is a Level Set Model?
• φ(X) is not defined by a specific equation
• φ(X) is represented by a regular 3D sampling
– Signed distance volume dataset
• Level Set model is deformed by evolving the
Level Set equation on the sampling
– Osher & Sethian 1988
r
r
r
• ∂ φ(X) /∂ t = ∇φ (X) F (X, ...)
• φ(X) is not defined by a specific equation
• φ(X) is represented by a regular 3D sampling
– Signed distance volume dataset
• Level Set model is deformed by evolving the
Level Set equation on the sampling
– Osher & Sethian 1988
r
r
r
• ∂ φ(X) /∂ t = ∇φ (X) F (X, ...)
• Connects changes in sample values to changes of
the level set curve/surface
2
Level Set Speed Term
Advantages of LS Models
• LS model deformation is controlled by F( )
• F(X,…) defines the speed of the LS surface
in direction of the surface normal N at each
point X on the surface
• F( ) is defined by a user for each
application in order to achieve a
specific goal
• We have defined many different speed
terms for our segmentation applications
• Always produce closed, non-selfintersecting (simple) surfaces
Deforming Mesh May
Self-Intersect
Important for GCAD, CAM
and Analysis
No Self-Intersection with
Level Set Deformations
Advantages of LS Models
• Easily change topological genus
– Holes may close or open
– Separate pieces come together/split apart
• Ideal for complex deformable models of
unknown genus
Figurine automatically manufactured from
a level set model
3
Mug-to-Chain Morph
Advantages of LS Models
• Concise interface for control → F( )
Advantages of LS Models
Advantages of LS Models
• Concise interface for control → F( )
• Free of mesh connectivity and
quality issues
• Concise interface for control → F( )
• Free of mesh connectivity and
quality issues
• No need to reparameterize during
deformation
Disadvantages of LS Models
Using Level Set Models
in Computer Graphics
• No inherent parameterization
– Problem for texture mapping & some deformations
• Computationally expensive (?)
• High memory requirements (?)
• Involved in three level set modeling projects
– Narrow Band methods are O(surface area)! Adalsteinsson &
Sethian 1995, Whitaker 1998, Peng et al. 1999
• Cannot control genus (?)
– Han et al. 2001
• Cannot represent fine or sharp features (?)
– Losasso et al. 2004, Nielsen & Museth 2006
4
Using Level Set Models
in Computer Graphics
Using Level Set Models
in Computer Graphics
• Involved in three level set modeling projects
• Involved in three level set modeling projects
• 3D Volumetric Metamorphosis
• 3D Volumetric Metamorphosis
– Make one solid model become another
– Make one solid model become another
• Volume Dataset Segmentation
– Extract complex structures from volumes
Using Level Set Models
in Computer Graphics
• Involved in three level set modeling projects
3D Volumetric
Metamorphosis
• 3D Volumetric Metamorphosis
– Make one solid model become another
• Volume Dataset Segmentation
– Extract complex structures from volumes
• Surface Editing
– Cut, paste, blend, smooth arbitrary 3D objects
Level Set Morphing Movements
• Each point on surface moves in the direction of local normal.
Step-size proportional to signed distance to target γ B
Level Set Morphing Movements
• Each point on surface moves in the direction of local normal.
Step-size proportional to signed distance to target γ B
r
"# X
( ) = $# Xr % Xr
( ) ( )
"t
B
5
Level Set Morphing Movements
• Each point on surface moves in the direction of local normal.
Step-size proportional to signed distance to target γ B
r
"# X
( ) = $# Xr % Xr
( ) ( )
"t
B
• Regions inside
expand
• Regions outside
contract
• Guaranteed
convergence
Different Alignments
Produce Different Morphs
Level Set Morphing Movements
• Each point on surface moves in the direction of local normal.
Step-size proportional to signed distance to target γ B
r
"# X
( ) = $# Xr % Xr
( ) ( )
"t
B
• Regions inside
expand
• Regions outside
contract
• Guaranteed
convergence
• Not moving
points!
Morphing Challenge
• How to morph between these three models?
!
Polygonal
Mesh
CSG
Model
MRI
Scan
Morphing Between
Different Types of Models
• Combining
– A variety of scan conversion algorithms
– A flexible metamorphosis (morphing)
technique based on level set models
• Produces
– A technique for morphing between
different types of geometric models
6
LS Segmentation Framework
• Extract closed, smooth models from volumes
Volume Dataset
Segmentation
LS Segmentation Framework
•
•
•
•
Extract closed, smooth models from volumes
Rough estimate
Initialization
Initial conditions
Frog Embryo Example
• Interactively fit sphere/toroid to input data
LS Segmentation Framework
•
•
•
•
Extract closed, smooth models from volumes
Rough estimate + Fine tuning (sub-voxel)
Initialization + Level Set Deformation
Initial conditions + Evolve Level Set PDE
Frog Embryo Example
• Interactively fit sphere/toroid to input data
• Scan convert models → Initialization
7
Frog Embryo Example
Results - MRI Frog Embryo
• Interactively fit sphere/toroid to input data
• Scan convert models → Initialization
• Level Set Deformation
– Attracted to maximum gradient magnitude
– Curvature-based smoothing
Mouse Embryo Segmentation
Ventricles
Results - Diffusion Tensor
• Scalar fields derived from a diffusion tensor
MRI scan (isotropic & anisotropic regions)
Liver
Results - Diffusion Tensor
Results - Diffusion Tensor
Brain Ventricles (Isotropic Diffusion)
White Matter (Anisotropic Diffusion)
8
Same Dataset Different Models
Different Dataset
Same Parameters
Exploded View
Segmentation from Multiple
Non-uniform Datasets
• Input → Several scans of the same object
– Different orientations
– Different, non-uniform resolutions
Isosurface
Initialization
After Level Set
Deformation
Segmentation from Multiple
Non-uniform Datasets
• Input → Several scans of the same object
– Different orientations
– Different, non-uniform resolutions
• Unable to extract satisfactory model from
any one dataset
Segmentation from Multiple
Non-uniform Datasets
• Input → Several scans of the same object
– Different orientations
– Different, non-uniform resolutions
• Unable to extract satisfactory model from
any one dataset
• Approach → Drive level set segmentation
with a moving least-squares polynomial
approximation based on all datasets
9
Segmentation Results
CT
Surface Editing
MRI
Laser
Scan
Problem Statement
?
Problem Statement
+
?
Level Set Surface Editing
Level Set Surface Editing
• Perform surface editing operations within a
level set framework to gain LS benefits
• Perform surface editing operations within a
level set framework to gain LS benefits
– Guaranteed to produce closed,
non-self-intersecting objects
– Easily handle change of genus
– No mesh-connectivity/triangle-quality issues
– Single framework for editing different types of
complex models
– Unified interface to surface deformation
10
Level Set Surface Editing
LS Surface Editing Pipeline
• Perform surface editing operations within a
level set framework to gain LS benefits
– Guaranteed to produce closed,
non-self-intersecting objects
– Easily handle change of genus
– No mesh-connectivity/triangle-quality issues
– Single framework for editing different types of
complex models
– Unified interface to surface deformation
– Many objects are volumetric anyway!
LS Surface Editing Pipeline
Input data
Input data
LS Surface Editing Pipeline
← Loops back!
Input data Conversions
Input data
Conversions
Input data
LS Surface Editing Pipeline
← Loops back!
Input data
Conversions
Operators
Conversions
Operators
Operator Speed Function
Building Blocks
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
Viewing
11
Operator Speed Function
Building Blocks
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Cut-off function based on a
distance d to geometric object q
– Regionally constrains level set deformation and
computation
Operator Speed Function
Building Blocks
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Cut-off function based on a
distance d to geometric object q
– Regionally constrains level set deformation and
computation
• G(γ) → Function of geometric measure γ
– e.g. curvature2
Operator Speed Function
Building Blocks
Automatic Blending
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Cut-off function based on a
distance d to geometric object q
– Regionally constrains level set deformation and
computation
• G(γ) → Function of geometric measure γ
– e.g. curvature2
• C(γ) → Filters geometric measure γ
– Controls local geometric properties
Automatic Blending
Position
Paste
Paste objects together to make a single object
(CSG union operation)
Position
Position red object relative to gray object
Automatic Blending
Position
Paste
Find intersection “curve” (voxels close to
both objects)
12
Automatic Blending
Blending Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
Position
Paste
Blend
Blending only occurs near intersection curve
after a paste operation
Blending Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero at a user-specified
distance from the intersection curve
Blending Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero at a user-specified
distance from the intersection curve
• G(γ) → Mean curvature
• C(γ) → Goes to zero as curvature
approaches a user-specified value
Blending Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero at a user-specified
distance from the intersection curve
• G(γ) → Mean curvature
Blending Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero at a user-specified
distance from the intersection curve
• G(γ) → Mean curvature
• C(γ) → Goes to zero as curvature
approaches a user-specified value
• Motion constrained to only move outwards
13
Localized Smoothing
• Spikes in eyes
(reconstruction
error)
Localized Smoothing
• Spikes in eyes
(reconstruction
error)
• Position
superellipsoids
Localized Smoothing
• Spikes in eyes
(reconstruction
error)
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Position
superellipsoids
• Region is
smoothed
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero as X approaches
surface of superellipsoid
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero as X approaches
surface of superellipsoid
• G(γ) → Mean curvature
14
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero as X approaches
surface of superellipsoid
• G(γ) → Mean curvature
• C(γ) → Goes to zero as curvature
approaches a user-specified value
Fixing a Teapot Model
Model errors
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero as X approaches
surface of superellipsoid
• G(γ) → Mean curvature
• C(γ) → Goes to zero as curvature
approaches a user-specified value
• Motion constrained to only move inwards
Fixing a Teapot Model
Model errors
Fixing a Teapot Model
Position
superellipsoid
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
Model errors
Position
superellipsoid
Region is
smoothed
15
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero as X approaches
surface of superellipsoid
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero as X approaches
surface of superellipsoid
• G(γ) → Mean curvature
• C(γ) → Goes to zero as curvature
approaches a user-specified value
Topology Simplification
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero as X approaches
surface of superellipsoid
• G(γ) → Mean curvature
Smoothing Speed Function
• F(X, N, φ, …) = Dq(d)G(γ)C(γ)
• Dq(d) → Goes to zero as X approaches
surface of superellipsoid
• G(γ) → Mean curvature
• C(γ) → Goes to zero as curvature
approaches a user-specified value
• Motion constrained to only move outwards
Embossing a Teapot Model
• Surface attracted to point set
• Convert image into 3D points
16
Point Embossing
• U → Unit vector from
X to attraction point P
• γ→N•U
r
Pi
r
Ui
Embossing Speed Function
• F(X, N, φ, …) = Dq(d)G(P)C(γ)
– measure of angle
v
Ni
!
r
Xi
!
!
!
Embossing Speed Function
• F(X, N, φ, …) = Dq(d)G(P)C(γ)
• U → Unit vector from X to attraction point P
• γ→N•U
– measure of angle between surface normal and direction to
attraction point
Embossing Speed Function
• F(X, N, φ, …) = Dq(d)G(P)C(γ)
• U → Unit vector from X to attraction point P
• γ→N•U
– measure of angle between surface normal and direction to
attraction point
• Dq(d) → Goes to zero as X approaches surface of
superellipsoid
Embossing Speed Function
• F(X, N, φ, …) = Dq(d)G(P)C(γ)
• U → Unit vector from X to attraction point P
• γ→N•U
Embossing Speed Function
• F(X, N, φ, …) = Dq(d)G(P)C(γ)
• U → Unit vector from X to attraction point P
• γ→N•U
– measure of angle between surface normal and direction to
attraction point
– measure of angle between surface normal and direction to
attraction point
• Dq(d) → Goes to zero as X approaches surface of
superellipsoid
• G(P) → φ(P) (Deformation stops at attraction point)
• Dq(d) → Goes to zero as X approaches surface of
superellipsoid
• G(P) → φ(P) (Deformation stops at attraction point)
• C(γ) → Non-zero for positive values of γ
– Motion constrained to only move in one direction
17
Global Smoothing with a
Morphological Opening
Point Set Embossing
• Opening → Erosion followed by a Dilation
Global Smoothing with a
Morphological Opening
• Opening → Erosion followed by a Dilation
• Erosion → Offset surface inwards by σ
• Dilation → Offset surface outwards by σ
Global Smoothing with a
Morphological Opening
Laser scan reconstruction with spiky errors
Global Smoothing with a
Morphological Opening
•
•
•
•
Opening → Erosion followed by a Dilation
Erosion → Offset surface inwards by σ
Dilation → Offset surface outwards by σ
Quickly calculate distance field from level
set model with Sethian’s Fast Marching
Method
• Faster than solving LS equation over
complete surface
Global Smoothing with a
Morphological Opening
Offset surface inwards
18
Global Smoothing with a
Morphological Opening
How to Repair a Greek Bust?
Offset surface outwards to create smoothed surface
Copy Right Chin
Reflect and Position on Left
Paste onto Face
Local Smoothing
19
Copy a Nose
Paste on New Nose
Local Smoothing
Different Hair Style?
Sharpen Hair
How to make a doubleheaded, winged dragon?
20
Load in Dragon Model
Enclose the Head
Cut Off the Head
Make Two Copies of Head
Paste and Blend Heads Together
Load in Body
21
Paste and Blend Together
Heads and Body
Enclose and Copy the Wing
Load in Double-headed Dragon
Paste and Blend the Wing
Position a Mirror-copy of Wing
Load in a Supertoroid
22
Paste Supertoroid to Body
Interactive Cut, Paste & Blend
Closed Surface Ideal for
Automated Manufacturing
Blend Along Intersection Curve
Two-Headed Winged
Dragon Charm
Editing Operators Summary
• Design LS surface editing framework
Model produced
by Utah’s
Geometric
Design and
Computation
Group
– Single representation and numerous tools
• Definition of new LS surface editing operators
– Automatic blending along intersection curves
– Spatially constrained smoothing and embossing
• User control over local surface properties
• User control of surface movement direction
23
Level Set Model Summary
Acknowledgments
• Level set models may be utilized in a wide
variety of computer graphics applications
• Thanks to Al Barr, Sean Mauch, Leonid Zhukov,
Jason Wood (Leeds), Jia Mao, Neha Soni, Erica
Stanley, David Johnson (Utah), Mathieu Desbrun
(USC), Khrysaundt Koenig, and Katrine Museth
• Supported by NSF, NIH, ONR, NLM, Caltech
• Datasets provided by CIT Biological Imaging
Center & Multires Modeling Group, Utah’s SCI
Institute & Geometric Design and Computation
Group, Stanford CG Lab, U. of Edinburgh,
Microsoft, Cyberware, City of Hope Hospital
– 3D metamorphosis
– Volume segmentation
– Surface editing
• Have some shortcomings, being addressed
• Offer unique and powerful capabilities and
outstanding benefits
Related Papers
•
•
•
•
•
•
•
D.E. Breen and R.T. Whitaker, “A Level-Set Approach for the Metamorphosis of Solid
Models,” IEEE Transactions on Visualization and Computer Graphics, Vol. 7, No. 2,
pp. 173-192, April-June 2001
D.E. Breen, S. Mauch, R.T. Whitaker and J. Mao, “3D Metamorphosis Between Different
Types of Geometric Models,” Proc. Eurographics, pp. 36-48, 2001
R.T. Whitaker, D.E. Breen, K. Museth and N. Soni, “Segmentation of Biological Volume
Datasets Using a Level-Set Framework,” Volume Graphics 2001, Springer, pp. 249-263,
2001
K. Museth, D. Breen, L. Zhukov and R. Whitaker, ``Level Set Segmentation From
Multiple Nonuniform Volume Datasets,'' Proc. IEEE Visualization, pp. 179-186, 2002
L. Zhukov, K. Museth, D.E. Breen, R.T. Whitaker, A.H. Barr, “Level Set Segmentation
and Modeling of DT-MRI Human Brain Data,” Journal of Electronic Imaging, Vol. 12,
No. 1, pp. 125-133, January 2003
K. Museth, D.E. Breen, R.T. Whitaker and A.H. Barr, “Level Set Surface Editing
Operators,” Proc. SIGGRAPH, pp. 330-338, 2002
D.E. Breen, S. Mauch and R.T. Whitaker, “3D Scan Conversion of CSG Models into
Distance, Closest-Point and Colour Volumes,” Volume Graphics, Springer, pp. 135-158,
2000
24