tutorial notes as PDF

Transcription

Forschungszentrum Karlsruhe
Technik und Umwelt
Institut für Angewandte Informatik (IAI)
Models for simulating
instrument-tissue interactions
Uwe G. Kühnapfel,
H.K. Çakmak, H. Maaß, S. Waldhausen
MMVR 2001
Newport Beach, January 27, 2001
Dr. U. Kühnapfel
23.01.01
Technik und Umwelt
Overview
Introduction
About M.I.S. Simulation
Karlsruhe VEST System Overview
Modelling of Soft Tissue
Simulation of Surgical Interactions
Visual Effects for enhanced Realism
Dr. U. Kühnapfel
23.01.01
Terminology
• Virtual Reality (VR): Real-Time, Navigation, Interaction
• Minimally Invasive Surgery (MIS)
• Virtual Endoscopic Surgery Trainer (VEST)
Motivation
Patients benefits
+ less traumatic
+ fast recovery
+ short hospital stay
Ü
Surgeons disatvantages
- limited view
- limited mobility
- limited haptics
Training is essential
„Classic“ Training Systems
Pelvi-Trainer
Training with plastic Models and Animal Tissue
Training with living animals / cadavers
Difficult and expensive to setup, ethic problems
Ü
Ü
Setup is time- and cost consuming
Limited „realism“ (anatomy, physiology)
Dr. U. Kühnapfel
23.01.01
VEST Applications
Arthroscopy
Logan et al. (1996), Univ. of Hull, UK
Ziegler, Müller et al. (1995), FhG-IGD, D
Bronchoscopy
BroBro-Nielsen et al.
al. (1999), HT-Medical
HT-Medical,, USA
Cardiac-Surgery / Anastomosis
Playter et al. (1997), BDI, USA
Realistic
Simulation
Scenarios
Training
Environment,
Environment,
User interface
Craniofacial Surgery
Keeve (1996), Uni-Erlangen, D
Eye-Surgery
Soft-Tissue
Simulation
Sinclair et al. (1998), Georgia-Tech, USA
Gynaecology
Szekely et al. (1998), ETH-Zürich, CH
Kühnapfel et al. (1998), FZK, D
Laparoscopy
Cover et al.
al. (1993), Georgia-Tech, USA
Kühnapfel et al. (1995), FZK, D
Interactions
and Visual
Effects
Physiology
Simulation
Trauma Surgery (mil.)
Basdogan et al. (1997), Musculographics, USA
Tumor Diagnosis and Palpation
VEST-System Requirements
Burdea et al. (1998), CAIP, USA
Ü
Ü
Ü
Limited surgical Interactions
Limited anatomical „Realism“
Limited modelling of „Physiology“
Dr. U. Kühnapfel
VR / computer graphics in medicine [ 1 ]
23.01.01
Knowledge
Database
Model Database
Virtual endoscopic
Image
KISMET
SGI-IRIX / WinNT
Graphics
Workstation
Forces
Position
Data
Position
Sensoring
System
Instrument Positions
Binary Switches
Dr. U. Kühnapfel
VEST system functional diagram
23.01.01
Technik und Umwelt
The “Karlsruhe Endoscopic Surgery Trainer”
„Phantom Box“with user interface:
2 (3) MIS-instruments, 1 Endoscope,
6 footswitches
PC based position measuring system
Rendering and Simulation with KISMET software on:
• SGI-workstations (Onyx, Octane) and
• NT-workstation (Intergraph, SGI-Visual PC)
Force-Feedback with commercial haptic devices:
• Impulse Engine (Immersion Corp.)
• PHANToM (SensAble Technologies)
Ø PhD-thesis Dr.-Ing. Kuhn Ch., FZK - IAI, ´97
Dr. U. Kühnapfel
Structure of the M.I.S. Training System
23.01.01
Videoclip
Dr. U. Kühnapfel
VEST-System Overview
23.01.01
Technik und Umwelt
Integration of two commercially available ForceFeedback Input Devices with KISMET
• „Laparoscopic Impulse-Engine“ (Immersion)
• „Phantom“(Sensable Technologies)
Research on „Feeling of Tissue Elasticities“
in MRI-Volume Datasets
Dr. U. Kühnapfel
VR Force-Feedback Devices
23.01.01
VEST System components
Interactions
Interactions
Grasping
Grasping
Irrigation
Irrigation
Cutting
Cutting
Suction
Suction
Clipping
Clipping
Suturing
Suturing
Coagulation
Coagulation Ligation
Ligation
Visual
VisualEffects
Effects
Fluids
Fluids
Jet of water
Jet of water
Fluid accumulation
Fluid accumulation
Wave effects
Wave effects
Air bubbles
Air bubbles
Bleeding
Bleeding
Particle systems
Particle systems
Texture animation
Texture animation
Stopping
of
Stopping
ofbleeding
bleeding
Coagulation
Coagulation
Clipping
Clipping
Smoke/Steam
Smoke/Steam
Steaming up of endoscopic lense
Cakmak / Kühnapfel
Modelling
Modelling
Spline-Modelling
Spline-Modelling
Splinecurves/2D-Cross-sections
Splinecurves/2D-Cross-sections
Spline-surfaces
Spline-surfaces
Volume
Rendering
Volume
Rendering
Image slice rendering
Simulation
Simulation
of
of
Soft
SoftTissue
Tissue
Elastodynamics
Elastodynamics
Image slice rendering
2D/3D Textures
2D/3D Textures
Procedural
Textures
Procedural
Textures
Turbulence functions
Turbulence functions
L-Trees
L-Trees
Properties
Propertiesof
oftissue
tissue
Tissue measurement in vivo/vitro
Tissue measurement in vivo/vitro
Physiology
Physiology
Hemodynamics
Hemodynamics
Simulation of Pulse
Simulation of Pulse
Arterial bleeding
Arterial bleeding
Organ bleeding
Organ bleeding
Morphodynamics
Morphodynamics
Actively Deforming Objects
Actively Deforming Objects
Peristalsis
Peristalsis
Dilatation
Dilatation
Contraction
Contraction
23.01.01
Technik und Umwelt
Elastodynamic tissue model
1. Modelling-Approach: Particle Simulation
• Nonlinear, visco-elastic, mass-spring model
• System of coupled differential equations
• Numerical solution of ODEs
Lagrange Equation (ODE 2nd Order) :
Newton-Euler Integration
dx
d 2 xi
mi ⋅ 2 + γi ⋅ i + g i (t , x i ) = f i (t )
dt
dt
t + ∆t
vi
mi
γ
i
gi
fi
Mass of knot
Damping
Inner forces
External forces
Dr. U. Kühnapfel
(
)
 t ∆t
t
t 
= k i ⋅v i +
⋅ f i − gi 
mi


Bi
ki =
∆t ⋅γi
Stability ?!?
1+
mi
Modelling of soft tissue
23.01.01
Technik und Umwelt
Tissue-Deformation:
2. Elastostatic solution of ODE-system
Fast Finite Element Modeling (FFEM)
pre-calculated stiffness-matrix:
stiffness-matrix:
- very fast solver possible (++)
- recalculation required for topology
changes (e.g. cutting)
- size of stiffness matrix (--)
 −1
K o− 1

K io
K
K
−1
oi
−1
i
f o  u 
  =  o 

f i  ui 
3. Deformable
Volume-Models
- import of patient data (++)
- elasticity Data from tables
- size of stiffness matrix (--)
Dr. U. Kühnapfel
23.01.01
Technik und Umwelt
Mechanical Properties of soft tissue
Experiments
in-vivo and post-mortem
uniaxial tensile tests
compression tests
Tissue model
Polynomial 4th order
Results
Different properties in-vivo and p.m.
Organ specific stress / strain curves
Linear slope for compression: 0% - 15%
Ø PhD-thesis Dr.-Ing. Maaß H., FZK - IAI, ´99
Maaß, Çakmak, Kühnapfel
σ = E z ⋅ε z ⋅(1 + a 2 ε z + a 3ε z2 + a 4 ε z3 )
23.01.01
Technik und Umwelt
Speed-of-Sound and E-Modulus Range Tables
E-Modulus in MPa
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
Fat (soft)
Fat (not soft)
Liver
Spleen
Heart-Muscle
Kidney
post
mortem
Fat
Liver
Spleen
Kidney
intra
vitam
1400 1420 1440 1460 1480 1500 1520 1540 1560 1580 1600
Speed of Sound in m/s
Dr.-Ing. Heiko Maaß / Kü
Modelling of soft tissue: Elasticity Parameters
23.01.01
Technik und Umwelt
Modelling-Tool: KisMo (KISMET-Modeller)
• Interactive design of model geometry and
spatial nodal net
• Scene editor for model connectivity
• Spline modelling
• Multilayer functionality for volumetric
cell definition
• Volume rendering of CT/MR-data
H. Çakmak / Kü
23.01.01
Technik und Umwelt
Elasticity
parameters
KisMo
3D-Geometry
3D-Geometry
Splinemodelling
Elasticity
measurement
Procedurale
Textures
Model-/Material-/
Model-/Material-/
and TextureDatabase
Internal
Format
3D-Mass
-Spring-Net
Net
3D-Mass-SpringModelling
aid
Object connections
Simulation
scenario
Actively deforming Objects
Medical image
data
CT,MR,VF,VM
H. Çakmak / Kü
Volume Rendering
Moveable image slices
Modelling of simulation scenarios
KISMET
Elastodynamics
simulation
23.01.01
Visualisation of medical image data
Methods: Image slices
2D-/3D-Textures
Modelling of deformable objects
Methods: Spline curves and revolving
spline surfaces
Object connection:
Nearest neighbour approach
syp
zc
Ci
sxn
y
T
nc
sxp
di
Pi
yc
xc
syn
H. Çakmak / Kü
z
Modelling of soft tissue with KisMo
x
23.01.01
Technik und Umwelt
Modelling with KisMo
Training simulation with KISMET
Model complexity
Eladyn.Models
Objects
Eladyn.Models
Objects Mass-knots
Mass-knots Springs
Springs
Uterus,Tubes,Lig.
7
1.335
5.030
Uterus,Tubes,Lig.
7
1.335
5.030
Arterial
tree
6
594
2.508
Arterial tree
6
594
2.508
Venous
tree
7
675
2.844
Venous tree
7
675
2.844
Intestines
1
147
560
Intestines
1
147
560
Total
21
2.751
10.942
Total
21
2.751
10.942
Vertex/Polygons
15.182
6.966
Vertex/Polygons
15.182
6.966
H. Çakmak / Kü
Frame rates
• SGI Octane, 2xR10000, 250MHz
Ü 11 fps
• SGI Visual PC, 2xPIII, 400MHz
9 fps
• Intergraph PC, 2xPII, 400MHz
Ü 6 fps
Ü
Scene Example: Laparoscopic Gynaecology
23.01.01
Active Deformable Objects
F1,ado
• Superposition of forces
4Inflating Forces F0,ado
0,ado
4Axial Forces F1,ado
1,ado
• Mathematical Functions
4periodical
4position-dependent
4time-dependent
F0,ado
Videoclip
Fado (t , k ) =
1
∑F
i =0
Ai =
i , ado
(t , k )
1
⋅(∆p i , mean + sgn rand ⋅∆p i , var ⋅rand )
h



ki

Fi,ado (t , k i ) = u i ⋅Ai ⋅sin 2π ⋅
t
⋅
wS
+
wD
⋅
⋅
(
fr
+
frD
⋅
rand
)
i
i
i
i



ni



Dr. U. Kühnapfel
Modelling: Texture, Motility (Morphodynamics)
23.01.01
Technik und Umwelt
Basic Surgical Interactions
Grasping
Application of clips
Coagulation
Cutting
Kühnapfel / Çakmak
Surgical Interactions with deformable objects
23.01.01
Videoclip
Basic Surgical Interactions: Grasping - Clipping - Cutting
23.01.01
Modelling with Mass-Spring-Systems (Particle systems)
Mass-Spring-System
Set of mass-points K and linkage elements V : FMS={K,V}
Mass-point
Point in 3d with final mass and infinite small volume
Linkage elements:
Elastic (Spring)
Elastic Bending (Spring)
F0 = − F1 = k ⋅( u − l0 )
u
u
p0
p1
Viscous (Damping)
F0 = − F1 = η ⋅u&
p0
p1
FB , 0 = −
u0
u 0 + u1
p0
p2
α
FB,0
⋅FB ,1
FB,1
p1
FB,2
FB , 2 = − (FB ,1 + FB ,0 )
Equation of motion
Plastic
κ
F0 = − F1 = 
− κ
(α − α 0 ) u 0 + u1
FB,1 = k ⋅
⋅
αo
u 0 + u1
u ≥ l0
sonst
p0
p1
&&i + d i u&i = Fext −
mi u
ni
∑k
j =0
j≠i
Basic interactions: application of particle systems
i, j
u i − FB ,i − mi g
23.01.01
Technik und Umwelt
Collision recognition
Rigid Instruments
Deformable Objects
INT_P1
INT_P3
• Creation of new particles
• Change of velocity and direction
Pold
Organ
n
Pipe
G
INT_P2
a'
INT_P4
Collision response
Particle System
Friction coefficient µp
Pnew
v
n
v'
Z
d
(nž
a‘)<0
d'
Flat
Bounding volumes
Range test
Distance test
a'
a
Pold
S
Pnew
(nž
a)ž
(nž
a‘)<0
vT⋅(1-µp)
Elasticity coefficient εp
n
G
vN⋅εp
Deformable Object
Object deformation after impact
Fext = m ⋅k ⋅( v 2 − v' 2 ) ⋅
Basic interactions: application of particle systems
v
v
23.01.01
Technik und Umwelt
Simulation of Suturing and Slings
• Suture material modelled as Spring-Mass-System
• Collision management
H. Çakmak / Kü
23.01.01
Modelling of surgical suture material
Classification: Material and Absorbtion
Material
Material
anorganic
anorganic
organic
organic
synthetic
synthetic
dissolvable
dissolvable
absorbale
absorbale
——
Kollagen,
tA=30d
Kollagen,Catgut:
Catgut:tRtR=8d,
=8d, tA=30d
Polyglykolsäure:
tA=120d
Polyglykolsäure:tR=15d,
tR=15d, tA=120d
Polydioxanon:
tA=180d
Polydioxanon:tRtR=35d,
=35d, tA=180d
Suture types
Modelling
Monofil Pseudo-monofil Multifil
H. Çakmak / Kü
not
notdissolvable
dissolvable
not
permanent
permanent
not permanent
permanent
——
Steel,Titanium:
Steel,Titanium:tZ=0
tZ=0
Silk,
Yarn
:
tR=0, tZ=?-4 a
—
Silk, Yarn : tR=0, tZ=?-4 a
—
Polyamide:
tR=0, tZ=1-2Ja
Polyester:
Polyamide: tR=0, tZ=1-2Ja
Polyester:tZ=0
tZ=0
Polypropylen:
tR=0, tZ=1-5a
Polypropylen: tR=0, tZ=1-5a
Stiff
Suture Material Modelling
Soft
Curls
23.01.01
Simulation of Suturing Procedure & Knots
Self-collision of suture material
Hierarchical Collision Recognition
• Intersection of bounding spheres
• Intersection of cylinders (springs)
Translation
Collision Response
• Definition of active and passive element
• Shifting of passive cylinder
VEj
Pnew
Sj=P
f
f
f
h
pr
VEi,0
VE(i+1),
0
H. Çakmak / Kü
iF = 2f / h
iR = pr → P
Pnew = (1 − iF ) ⋅pr + iF ⋅P
23.01.01
Technik und Umwelt
Collision Needle - Suture - Organ models
Suture-Deformable object:
Needle - Deformable object:
• Mutual influence
• Register interacting object knots
• Suture constrained
• Check force treshold
• Object deformation by suture motion
• Find intersection
Ü
Definition of intersection as a constraint (a)
(b)
Representation: Barycentric coordinates
P = w0 P0 + w1P1 + w2 P2 w0 + w1 + w2 = 1 ∧ wi ]0;1[
P1 P2
P
P3
(c)
H. Çakmak / Kü
Simulation of surgical suturing procedure
(d)
23.01.01
Technik und Umwelt
Sling Mechanism and Interaction
Sling mechanism
H. Çakmak / Kü
23.01.01
Technik und Umwelt
Modelling of surgical slings
Deformable effector
(a) Linkage
(b) Dilatation
(c) Central knot for stabilisation
Functionality : Shortening of Sling
• Caculate new knot positions
Shortening factor sf
Simulation time step ∆t
• Definition of new spring length
H. Çakmak / Kü
23.01.01
Technik und Umwelt
Morphodynamics Simulation
Pulse Animation
Propagation of pressure
waves in an arterial tree
T i m e
Animation of bleeding
after arterial injuries with
physiological correct data
e.g. Aorta: Q = 83 ml/s
r = 12 mm
v = 29 cm/s
Stopping of bleeding with clips
Knowledge database
H.K. Çakmak / Kü
„Special effects“ for enhanced realism
23.01.01
Videoclip
H.K. Çakmak / Kü
„Special effects“ for enhanced realism: morpodynamics (pulse)
23.01.01
Technik und Umwelt
Arterial Bleeding
• Particle System Simulation
• Coupled with Pulse Simulation
• Application of clips to stop bleeding
• Accumulation of blood
• Parameters: Blood loss per vessel
Rendering settings
H. Çakmak / Kü
Visual Effects for Simulation Realism
23.01.01
Videoclip
H.K. Çakmak / Kü
„Special effects“ for enhanced realism: arterial bleeding
23.01.01
Animation of organ surface bleeding (non-arterial)
Method
Texture animation
Algorithm
• Doubling of knot P/P‘after cut
• Texture projection
• Time dependent texture scaling
Feature
Definition of supply of blood at
each object knot
Application
Ü Styptic POR-8 injection
Ü Coagulation stopping of bleeding
Ü Irrigation of wound:
Increase of texture transparency
Videoclip
„Special effects“ for enhanced realism: surface bleeding
23.01.01
Technik und Umwelt
Irrigation and Suction
• Implementation of Particle Systems
• Motion blurred particle rendering
• Tissue deformation: Particle impact
• Fluid accumulation
• Effects: Splashing
Reflection
Ripples
H.K. Çakmak / Kü
Surgical Interactions with deformable objects (enhanced Realism)
23.01.01
Technik und Umwelt
Hydromechanics: Continuity equation
dV A ⋅du r 2 ⋅π ⋅du
Q=
=
=
= r 2 ⋅π ⋅v
dt
dt
dt
Modelling of water-jet
Instrument specification:
Fa. AESCULAP
Qmax=3 l/min, d=5 mm Ü vmax=2.55 m/s
Videoclip
Visual Effects
Fluid accumulation
Wave effects
Air bubbles
H.K. Çakmak / Kü
Simulation of Irrigation and Suction
23.01.01
Technik und Umwelt
Steam / Smoke for Coagulation
• 3D-Procedural Textures
• Billboard-technique
• Animation: Texture shifting
Spline Key-Frame
Color / Transparency
• Steaming up of endoscopic lense
H. Çakmak / Kü
Visual Effects for Simulation Realism: Coagulation Steam / Smoke
23.01.01
Animation von coagulation smoke
Basis
3D-Turbulence function(Perlin): Density distribution
Method 1: 3D-Textures (a)
Visualisation: 3D-Texture Mapping
Animation:
Modification of colours and transparencies
Shifting of Texture coordinates
Method 2: 2D-Textures (b)
Visualisation:
...
t
Billboard-Technique
t
t
Viewpoint dependent scaling
t
...
Animation:
Texture circulation
Spline-Key-Frame Technique
Coordination with instrument activation
1
0
m-1
m-2
(a)
(b)
y
kn-1
v
A
...
k1
k0
x
z
23.01.01
Animation of coagulation smoke - Method 2
Videoclip
23.01.01
Direct haptic Interaction with Voxel-Volumes
Application:
• Training in interventinal radiology
• Needle injections (spine disks)
Method(s):
• use CT data for tissue-stiffness
• gray-value encoding
• DICOM interface
• use colour lookup-table (RGBA)
for direct volume-rendering of
CT-dataset (3D texture emulation)
• Phantom as haptic display
Haptic Interaction with Voxel-Volumes
23.01.01
Technik und Umwelt
Result(s):
• Training System for Minimally Invasive Surgery with realistic User-Interface
• Methods and SW-Tools for realistic Modelling of deformable Objects
• Realtime Simulation of Surgical Interactions:
Grasp, Clip, Cut, Coagulation, Irrigation, Slings, Suturing
• Active deformable Objects: Organ motility
• Particle Systems for Fluid Simulation
Conclusions:
• VR-based Surgical Simulation Systems will
become much more realistic in the future
• They will be integrated into multimedia teaching and training environments
• All surgical disciplines will be covered
more and actual info’s:
Dr. U. Kühnapfel
http://www-kismet.iai.fzk.de
23.01.01

tutorial notes as PDF

Transcription

Similar documents

NK Kempo Karate