A COMPUTER-BASED SIMULATION FOR TRAINING DENTAL

A COMPUTER-BASED SIMULATION FOR
TRAINING DENTAL PREPARATION
By
WILLIAM CHRISTOPHER HUNT
A THESIS PRESENTED TO THE GRADUATE SCHOOL
OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
MASTER OF SCIENCE
UNIVERSITY OF FLORIDA
2011
c 2011 William Christopher Hunt
⃝
2
I dedicate this work to my mother:
”Well you need to at least get a Masters...
It’s what a Bachelors meant when I was your age!”
3
ACKNOWLEDGMENTS
Dr. Jörg Peters has been the ideal thesis supervisor. His frank criticism, confidence,
and patient encouragement aided the creation of this project and the writing of this
thesis in innumerable ways.
4
TABLE OF CONTENTS
page
ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
CHAPTER
1
INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
1.1 Statement of Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2 Early Work at UF SurfLab . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3 Definition of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2
RELATED WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3
SYSTEM IMPLEMENTATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
Overview . . . .
Data Model . . .
Initialization . . .
View and Control
Configuration . .
Haptics . . . . .
Sound . . . . . .
Ergonomics . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
14
15
17
18
22
22
26
26
4
TESTING AND EVALUATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5
FUTURE WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
APPENDIX
A
USER TESTING INSTRUCTIONS . . . . . . . . . . . . . . . . . . . . . . . . . 35
A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
A.2 Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
B
SURVEY RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
BIOGRAPHICAL SKETCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5
LIST OF TABLES
Table
page
2-1 Existing academic dental systems . . . . . . . . . . . . . . . . . . . . . . . . . 13
2-2 Existing commercial dental systems . . . . . . . . . . . . . . . . . . . . . . . . 13
B-1 Quantitative survey responses . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6
LIST OF FIGURES
Figure
page
3-1 Basic high-level CEDS UML Class Diagram showing state and behavior . . . . 14
3-2 Overcoming disconnected height maps and steep normal aliasing on tooth 36
(FDI numbering) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A
Before sewing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B
Inherent edge aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . .
C
Modified edge normals . . . . . . . . . . . . . . . . . . . . . . . . . .
18
18
18
18
3-3 Examples of generated carries on teeth 26-28 and 35-38 (FDI numbering)
A
Tooth 26 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B
Tooth 27 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C
Tooth 28 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
D
Tooth 35 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
E
Tooth 36 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
F
Tooth 37 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
G
Tooth 38 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
18
18
18
18
18
18
18
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3-4 CEDS display loop Sequence Diagram . . . . . . . . . . . . . . . . . . . . . . . 19
3-5 Jagged edges on tooth 36 (FDI numbering) . . . . . . . . . . . . . . . . . . . . 20
3-6 Demonstration of handpiece orientation features
A
Handpiece in front of tooth . . . . . . .
B
Handpiece above tooth . . . . . . . . .
C
Handpiece behind tooth . . . . . . . . .
D
Handpiece with arrow . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
21
21
21
21
21
3-7 GLConsole Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3-8 CEDS haptic loop Sequence Diagram . . . . . . . . . . . . . . . . . . . . . . . 23
3-9 Demonstration of drilling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
A
Drilling handpiece . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
B
Removed caries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3-10 CEDS haptic loop Activity Diagram . . . . . . . . . . . . . . . . . . . . . . . . . 27
3-11 An AR marker attached to a headband . . . . . . . . . . . . . . . . . . . . . . . 28
3-12 A dental handpiece attached to a Novint Falcon grip . . . . . . . . . . . . . . . 28
3-13 The Novint Falcon with attached handpiece oriented vertically . . . . . . . . . . 29
3-14 Students at the UF College of Dentistry trying CEDS . . . . . . . . . . . . . . . 29
5-1 Anaglyph 3D red/blue glasses with the CEDS ’UF’ AR Marker Attached . . . . 32
7
Abstract of Thesis Presented to the Graduate School
of the University of Florida in Partial Fulfillment of the
Requirements for the Degree of Master of Science
A COMPUTER-BASED SIMULATION FOR
TRAINING DENTAL PREPARATION
By
William Christopher Hunt
December 2011
Chair: Jörg Peters
Major: Computer and Information Science and Engineering
We present a low-cost VR simulation environment for dental students that
allows trainees to practice dental preparation. The trainee haptically interacts with a
virtual tooth, drilling into its enamel and dentin layers. When trainings students, the
environment can compute metrics, saving educators’ time. The system was tested and
evaluated by dental students and professors of dentistry.
8
CHAPTER 1
INTRODUCTION
1.1 Statement of Problem
Traditional dental training methods are costly, both in material and time. For
example dental students destroy their plastic practice teeth or drilling practice pads,
so material must constantly be purchased. Additionally dental instructors must spend
time to individually evaluate students’ work.
Students entering into the University of Florida dental program are introduced to
working with a dental handpiece by drilling on a practice pad called the Learn-A-Prep II
[1], but this has been shown to ”... not significantly improve the likelihood of receiving an
A or B on the Class II practical (p=0.53) or on the Complex practical (p=0.37)” [2]. This
form of training serves to acclimate dental students to the drill, but does not feel exactly
like the real thing. The surface of the practice pad shows common tooth decay shapes
that can be traced, and different tooth layers are modeled with different colors.
Typodonts and plastic training teeth have other downsides. The surface of a
dentoform is not as hard as a real tooth’s enamel surface. Furthermore training teeth
are usually homogeneous, with no simulation of the various tooth layers. When moving
through a real tooth, the drill passes through layers of differing hardness (decay, enamel,
dentin, and pulp). Being familiar with the feel of moving through these different material
types is necessary to avoid making mistakes on real patients, but plastic teeth fail to
prepare students fully.
A computer-based replacement could be reusable and low-cost. Simulation
software has no inherit limit on the number of attempts that can be made at a removal
procedure. This translates into a high savings in material costs over time. Additionally an
instructor would only need to set the metrics that the students’ work is graded against.
This saves teachers from repeating the same generic evaluations, leaving time for more
9
specialized one-on-one attention. As a bonus, evaluation would be instant, accurate,
and objective.
These sentiments are reflected by UF Dental Professor Karl-Johan Soderholm:
The development of an inexpensive dental simulator, capable of assisting
dental students in developing and improving their psychomotor skill level is
very challenging but also very worthwhile. Today, dental students have to
spend months in a laboratory environment to develop those skill levels, a
training that is both teacher demanding as well as expensive. By use of a
software based dental simulator, the students would be able to practice at
any place they so prefer and receive feedback directly from the computer.
Such a training would be much more efficient and inexpensive than the
training that is offered in the laboratory.
1.2 Early Work at UF SurfLab
The UF Surflab has a history working with medical applications. The Toolkit
for Illustration of Procedures in Surgery (TIPS) is a collaborative partnership of
surgeons and IT researchers leveraging low-cost hardware and advanced software.
Its goal is creating and broadly disseminating a low-cost, computer-based, 3D
interactive multimedia authoring and learning environment including force feedback
for communication of surgical procedures. The Computer Enhanced Dental Simulation
(CEDS) project follows in these footsteps.
Our original implementation of CEDS was focused on emulating and ultimately
replacing the Learn-A-Prep II practice device [1]. This version did not use any force-feedback
devices and was instead controlled using a drawing tablet. Students would practice
tracing the shapes from the surface of the practice drilling pad.
1.3 Definition of Terms
•
ARToolkit: A software library for building Augmented Reality (AR) applications.
Markers are identified in live video and the position and orientation of those
markers are used to superimpose virtual objects [3]. In CEDS, ARToolKit markers
10
are used to create a Virtual Reality effect by allowing users to navigate the 3D
scene as though they were really there.
•
Burr: The small cutter attached to the handpiece that can remove tooth material
when rotated at high speeds. Dental burrs are generally cylindrical with varying
bevel.
•
Caries: A progressive decay of bone structure. Specifically in dentistry, a
demineralization of hard tissue and destruction of organic matter.
•
Dentin: One of the major components of a tooth. It is yellow and found just
below the top enamel. Dentin is less hard, but also less brittle than the enamel it
supports.
•
Dentoform: The leading dental simulation brand, synonymous with artificial
simulation teeth [4]. Working on Dentoform simulation teeth is generally the last
training step before beginning work on real patients.
•
Enamel: The hard, brittle, whitish top layer of tooth material.
•
FDI World Dental Federation Notation: Notation developed by the FDI World
Dental Federation used internationally by dentists to a specify teeth (ISO 3950).
•
Handpiece: The pen-like drill held by the dentist. Burrs can be switched out on the
handpiece. High speed handpieces can spin as high as 800,000 RPM while low
speed handpieces operate at up to 40,000 RPM but provide higher torque.
•
Learn-A-Prep II: A dental training device used to familiarize dental students with
operating the handpiece. It is a rectangular pad with shapes on the surface to trace
and different colors at different depths to represent different layers of material [1].
•
Preparation: The drilling necessary to remove unwanted material and insert
restorative filling. Preparations are broken down into 6 preparation classes. When
a dental student practices a preparation on a dentoform tooth, they are said to be
performing an ”ideal preparation”.
•
Pulp: The soft organic material found inside the pulp chamber located below the
dentin.
•
Typodont: A model mouth which holds plastic teeth for training.
11
CHAPTER 2
RELATED WORK
Several attempts have been made to create haptic dental training systems, but most
projects rely on expensive Phantom devices or specialized 3D stereoscopic screens.
Furthermore, out of the projects covered below, only Thomas et al. have tested their
simulation and provided results. Additionally, none have discussed the ergonomics of
their simulation setup, nor techniques to automatically evaluate the drilled tooth [5].
Kim et al. propose an extensive setup with stereoscopic 3D, but their tooth models
appear rather rounded and unrealistic [6]. Noborio et al. attempt to improve on Kim et
al. with a rich user interface and various burr shapes [7]. Rhienmora et al. provide a
method for recording and replaying haptic motion for training, but they only modify the
display mesh itself without altering the number of triangles (limiting the scale of work
that can be done in a preparation) [8]. Liu et al. provide detailed analysis of raw forces
involved in dental preparation, but admit to haptic instability with random oscillations.
Liu et al. also achieve material removal by using a deformable mesh [9]. Wu et al.
render surface voxels directly, requiring bilinear interpolation over normals for smooth
appearance, however even with normal filtering the shape of the underlying voxel
structure is visible [10]. Yau et al. run the marching cubes algorithm on an adaptive
octree-based data model that can handle fine detail, but can be complicated to update
[11]. William et al. describe a robust surface dragging approach that we implement, but
they encounter a ”blocky” feeling when working directly with the voxel data structure,
which we have managed to mostly avoid [12]. Finally Thomas et al. only visualize a 2D
cross-section [5].
Several commercial dental training systems exist, but many of them are either still in
development or do not provide detailed technical information.
12
Table 2-1. Existing academic dental systems
Author
Overview
Noborio et al. [7]
Phantom device; Quickhull and Gilbert, Johnson and
Keerthi (GJK) algorithms on voxel model
Kim et al. [6]
Phantom device; 3D display; Adaptive polygonization
method and Mauch’s closest point transform (CPT)
algorithm on voxel model
Rhienmora et al. [8]
Phantom device; 3D display; Forces from surface
normals; Deformable polygon mesh; Haptic
recording, playback for training
Liu et al. [9]
Phantom device; Deformable polygon mesh; Haptic
instability
Wu et al. [10]
Phantom device; Half-silvered mirror; Direct
rendering of boundary voxels; Filtering of surface
normals
Yau et al. [11]
Phantom device; 3D display; Marching cubes on
adaptive octree voxel model
William et al. [12]
Falcon device; Ball pivot algorithm (BPA) on voxel
model
Thomas et al. [5]
Impulse device; 2D cross-section display
Table 2-2. Existing commercial dental systems
Project
Virtual Reality Dental Training System (VRDTS) [13]
Sensable Dental [14]
VOXEL-MAN Dental [15]
Moog Simodont Dental Trainer [16]
DentSim [17]
13
Overview
Falcon
Phantom
Phantom devices; 3D
display
Other
Other
CHAPTER 3
SYSTEM IMPLEMENTATION
3.1
Overview
Figure 3-1. Basic high-level CEDS UML Class Diagram showing state and behavior
CEDS is a Microsoft Windows application written in C++. The simulation is broken
into modules representing the data model, initializer, view and control, dynamic
configuration, haptics, and sound. The modules communicate with each other via a
central simulation object which permits access to the modules’ various interfaces using
the singleton pattern.
Two threads run simultaneously during the simulation. The display thread checks for
any new drilling since the last display loop execution and updates the surface normals
for viewing if necessary. The display thread also checks for visible Augmented Reality
(AR) markers to orient the viewing angle. Display updates must be efficient enough not
to significantly impede the frame-rate. The display loop is summarized in Figure 3-4.
14
The haptic thread provides force feedback, plays sound if necessary, and updates
the data model when material should be removed. The haptic thread must execute
within 1ms to avoid a choppy feeling. This loop is described in Figure 3-8.
Figure 3-1 is a UML Class Diagram describing at a high-level the various encapsulated
modules in the CEDS simulation. The following sections describe each module in
greater detail. We also discuss the ergonomics of our physical setup.
3.2 Data Model
The CEDS data model extends the original tablet-based CEDS implementation
in which only downward drilling was possible. Because the Falcon is restricted to
3 degrees of freedom, and because a class-1 dental preparation is characterized
mainly by firm vertical drilling and lateral sweeping rather than rotation, the tooth
data representation is a simple piecewise-linear height function that does not support
undercutting. The finishing touches of a real-life, ideal preparation involve some slight
undercutting (to ensure that filling material stays locked in place), but the bulk of basic
material removal occurs through translation of the handpiece, not rotation. The Falcon’s
restrictions may help newer students understand how to firmly hold the handpiece
steady and properly cut into a tooth. A study at Ottawa University suggested that
utilizing a tooth preparation support system that limited a student’s degrees of freedom
in this manner was useful to achieve a greater competency in tooth preparation [18].
CEDS employs a simple height map approach rather than a voxel approach to
represent the tooth topography. We are able to represent the entire tooth by using two
height maps. One height function models the top of the tooth, and another models the
bottom. This simple design uses minimal space and enables direct visualization with
high surface resolution and accurate surface normals on the top of the tooth. The height
map however, does not enforce the semantics of the haptic interaction. We found a
height map of size 150x150 to be detailed enough to create a realistic rendering and
a believable feeling, while small enough for interactive work. Data structure resolution
15
could be raised to support even more detailed haptic interaction, but we chose to keep
the size small enough to easily represent the entire tooth surface in a single 2D float
array without requiring any extra optimizations. Increasing the resolution of traditional
3D voxel representations can quickly become too computationally limiting in comparison
[12].
The height map data structure stores surface normals for display. Its public object
interface provides two methods for working with normals. First, a point on the surface
may be specified as a future normal update point, and second, a command may be sent
to recalculate the normals surrounding the most recent update point.
When the tooth surface becomes sufficiently steep aliasing can occur and the
resolution can become visually apparent on tooth edges (where the topography of the
tooth appears vertical). On a freshly initialized tooth model this aliasing is problematic
near the edges of the height maps (on the sides of the tooth). We are able to fix these
normals in initialization without modifying the aliased underlying geometry. This normal
modification process will be explained in the initialization section.
When drilling reduces a top height to be lower than a corresponding bottom height,
the height at that coordinate is removed from both the top and bottom functions, and all
adjacent height map points are set to the average of their corresponding top and bottom
heights. This process sews together the opening in the tooth geometry created when a
height map point is removed from display. The moment after a height map coordinate
is removed, but before its surrounding edge is sewn together, is the only time when the
model exists in an inconsistent state for rendering, but in practice this step happens so
quickly it cannot be observed. Finally the normals corresponding to the new edges on
the top and bottom height maps are averaged for smoother appearance.
After the data structure is initialized, the top height map is saved to later determine
depth. When depths are added together they represent total material removed. Decay
is stored by depth, extending down from the original tooth height. After drilling has
16
occurred, the percent of decay remaining can be determined easily by comparing depth
removed with decay depth. Similarly, the amount removed of any other tooth material
can be computed.
The tooth is separated into layers of increasingly soft material: enamel, dentin,
and pulp. We currently model the borders between tooth layers by depth from the
original height of the tooth and distance laterally from the edge of the tooth. In reality the
distance from the top of the tooth into the subsequent layers is not as uniform and would
ideally be configured by a 3D model provided by a dental expert.
3.3 Initialization
The initialization module parses standard OBJ files. The OBJ models used in CEDS
were created from STL files representing 3D scans of real teeth. The geometry of the
original 3D tooth scans were manually modified to be completely enclosed, a necessary
condition to fully initialize the height maps.
To initialize the top and bottom heigh maps, each triangle in the input tooth model
is examined, and height map coordinates are compared to overlapping triangles. The
top height map maintains the maximum height corresponding to any triangles at that
particular coordinate, while the bottom height map maintains the minimum triangle
heights.
Simply initializing the data model coordinates with maximum and minimum triangle
heights is insufficient to create a complete tooth representation, because the edges
of the top and bottom functions are not guaranteed to meet. After initializing the data
structure the edge points are identified and sewn together by averaging edge points.
Special care must be taken to create proper normals at the edges. The top and bottom
normals at such points are averaged.
After sculpting the tooth surface, we fix aliased surface normals near the edges.
For each degenerate edge point on the height map for which a normal cannot be
directly computed, we search radially outward for a nearby non-steep height map point.
17
We borrow the x and y components of the nearby normal and combine it with the z
component of the original aliased normal and renormalize it to create a convincing
modified normal.
A Before sewing
B Inherent edge aliasing
C Modified edge normals
Figure 3-2. Overcoming disconnected height maps and steep normal aliasing on tooth
36 (FDI numbering)
We programmatically initialize decay using a depth-first search for low points on the
top of the tooth starting from the data model center. The decay will fill up valleys on the
tooth surface.
A Tooth 26
D Tooth 35
B Tooth 27
E Tooth 36
C Tooth 28
F Tooth 37
G Tooth 38
Figure 3-3. Examples of generated carries on teeth 26-28 and 35-38 (FDI numbering)
3.4 View and Control
CEDS uses FreeGLUT to drive the OpenGL display loop and collect keyboard and
mouse input.
18
Figure 3-4. CEDS display loop Sequence Diagram
The display loop traverses the top and bottom tooth height maps once to directly
render them. For any location on the data model with a top height associated with it, a
bottom height must also exist. The height maps are rendered by drawing every possibly
quad between existing height map points. For any coordinate without a height value,
no adjacent quads will be drawn. This direct representation of the height map creates
a jagged edge geometry, but this edge can only be noticed when viewing directly from
above, or from very close. Figure 3-5 shows an example of this jagged edge (along the
bottom of the top two teeth), and how it quickly becomes unapparent when not viewed
from very close.
CEDS requires no special thread synchronization for display. Kim et al. point out
that many existing sculpting systems rely on uniform polygonizations like the marching
cubes algorithm, which can be computationally limiting [6].
The scene is viewed using a virtual camera mounted on a spherical coordinate
system with the tooth top at the sphere center. The default camera orientation assumes
that the handpiece is being manipulated by the right hand, however a left-handed user
can choose to rotate the scene 180 degrees to compensate for this. A student can
navigate the scene using an ARToolKit marker attached to their head, giving a Virtual
Reality effect when the viewer moves the marker relative to the camera [3].
19
Figure 3-5. Jagged edges on tooth 36 (FDI numbering)
Drawing the handpiece on the tooth surface rather than at the virtual device tip is
important to make the surface appear hard to the user. When the virtual handpiece is
dragging on the tooth surface the physical device location dips into the virtual tooth. If
the handpiece model were rendered at this actual device location it would appear to be
sinking into the tooth surface. Seeing this penetration would betray the user’s notion
of how a metallic handpiece should interact with a hard tooth. To avoid this problem
the virtual handpiece is rendered at what we call the haptic anchor point, which will be
explained thoroughly in the haptic section below. This should be the point where the
user would expect a physical handpiece to appear.
Colors for decay and tooth layers are written to a single 3D texture after the tooth
is loaded. Decay depths are read from the initialized height map, and tooth layers are
derived using a constant depth from the tooth surface.
20
A Handpiece in front of tooth
B Handpiece above tooth
C Handpiece behind tooth
D Handpiece with arrow
Figure 3-6. Demonstration of handpiece orientation features
Initially it can be difficult to position the virtual handpiece on the virtual tooth.
Humans rely on stereoscopic vision to navigate real 3D space. We have used some
tricks to help the user orient the virtual handpiece relative to the virtual tooth. First
we project a transparent line down vertically from the burr. Based on how this line
intersects, obscures, or is obscured by the tooth, the user can intuitively place the virtual
burr on the tooth surface. The second orientation feature is a red, transparent arrow that
points to the handpiece from the burr. If the handpiece leaves the screen the user can
easily find it and bring it back into view. Finally we added a light source above the burr
to simulate the small light that is sometimes located at the bottom of a handpiece. This
light creates realistic shadows and highlights on the tooth surface that provide useful
21
orientation information to the user. These orientation features are overviewed in Figure
3-9.
When the view of the tooth surface is obscured by the handpiece, or when the tooth
is otherwise not in direct view of the practitioner, a mirror is used. CEDS has a mirror
that can be toggled on and off.
3.5
Configuration
Figure 3-7. GLConsole Interface
We use the GLConsole library to provide simple dynamic configuration of
parameters. These parameters can be saved and loaded, to help with customizing
variables using domain experts. This console is implemented as a transparent overlay
over the scene. Any variables registered with the configuration module can be edited in
real-time to allow quick testing of new settings.
3.6
Haptics
Our original naive haptic approach was to use surface normals to directly specify
device forces. While this approach could conceivably work for soft, deformable objects,
it does not translate well to hard objects since it does not impose constraints on the
positions to which the handpiece can move. It operates under the assumption that
22
Figure 3-8. CEDS haptic loop Sequence Diagram
exerting forces away from the surface will prohibit the virtual device from passing
through illegal positions. While this approach might work with a very high resolution data
structure and an extremely powerful device, the Falcon is not powerful or precise enough
to assume that such a naive approach could create a realistic feeling when dragging
over complicated topography. William et al. similarly found that such techniques perform
erratically in tight corners or inside pits [12].
To achieve a realistic, hard feel we use the anchor approach [12]. The anchor is
the point in the virtual scene where the viewer would expect the virtual burr to end up
according to real physics, rather than the actual device position. The handpiece will
always be pulled towards the anchor with a spring force. When the virtual burr is judged
to be within the virtual tooth, the anchor must be dragged over the tooth surface, rather
than in to it.
When the haptic loop executes a new anchor point is chosen from among 27 anchor
candidates (the 26 adjacent points plus the current point). Our algorithm chooses the
candidate closest to the virtual device position that does not intersect with the virtual
tooth. Intersection of the virtual burr with the virtual tooth is evaluated using a burr mask
data structure. While William et al. identified the anchor approach when used directly
on the voxel model as ”blocky”, our approach feels very smooth on the surface, and only
exhibits some ”blocky” tendencies where the model becomes nearly vertical.
The forces pulling the device towards the anchor were calibrated to be as hard as
possible without causing any thrashing effects. The outward force (in the z direction)
23
must be scaled to prevent lateral thrashing, because the haptic arms of the Falcon are
not mechanically capable of working as hard horizontally as vertically.
For convenience we introduce a centering state to automatically bring the device
to the center of the work area. While in centering state the anchor point will move
incrementally closer to the center point. Centering stops when the handpiece is pulled
sufficiently far from the centering anchor.
A friction effect is vital for a realistic, hard haptic response. Without friction the user
pushing with sufficient force can trigger a thrashing effect created by an underdamped
spring force. The friction effect creates an over-damped spring force that keeps the
device stable even when pushed very hard. This force is created by tracking the most
recent location during the haptic loop. The frictional force is the difference between
the current device position and the previous position, scaled by a frictional factor and
clamped so that the friction cannot overcome the force produced by the user’s hand and
devolve into uncontrollable thrashing.
Sudden discontinuities in force occur based on insufficient haptic refresh speed or
data structure resolution. These can be smoothed out using a force filter. The filter or
damper clamps the magnitude of change in force exerted between consecutive calls to
the haptic loop [11].
We do not allow the handpiece to exit the work area. This helps keep the user from
losing track of the virtual handpiece on the screen. If the virtual handpiece is moved
sufficiently far out of this area centering mode is enabled.
When the handpiece is on but not in contact with the tooth, the user should be able
to feel the spinning burr. This is accomplished by adding tiny randomized forces, giving
the user a tactile sense of the handpiece being switched on.
Conventionally removal using a spinning burr is measured in terms of the feed
speed, the speed at which the spinning burr is moved into the material to be removed.
If the burr is moved into the material steadily at or below the acceptable rate, the user
24
should feel little resistance. Pushing harder against the tooth with the moving burr
should allow more removal, up to some limit depending on the torque of the burr. The
ability to remove material is also limited by the area and type of material in contact with
the burr. Finally, the drilling direction needs to be considered. For instance a typical
dental drill can remove more material more easily when moved laterally rather than
vertically.
The following are basic drilling requirements that we gleaned from dental students
and professionals:
•
When holding a moving high speed handpiece, if the burr accidentally brushes
a tooth, no noticeable damage should occur. Opening up a preparation requires
sustained pressure.
•
Consistent removal with a high speed burr should be possible with a relatively low
force (on the order of one Newton).
•
Tooth material should be naturally removed by brushing the burr back and forth.
Holding the burr lightly on the tooth surface and brushing it back and forth should
create a trench that the burr cannot slide out of within 10 strokes.
We measure the ability of a spinning burr to cut into the tooth surface by examining
anchor candidates that collide with the tooth and weighting them proportionally to
their hardness. The total value that would be decremented from all tooth height map
points intersecting with the potential burr is calculated, scaled by the distance of the
virtual device tip from the anchor point (which describes the force exerted by the user),
and compared to a removal threshold corresponding to the direction the virtual burr
is moving. Anchor candidates that would translate the anchor sideways have a lower
removal threshold than vertical anchor candidates, because a spinning burr tip is more
capable of removing material laterally than vertically. Notice that the removal threshold
for anchor points translating the virtual burr upward is irrelevant.
While we can smoothly push the height map surface down to create a satisfying,
gradual removal effect, lateral removal on our height map can be problematic. To
properly manage the lateral removal rate we must introduce a removal timer. Even if the
25
drilling constraints seem to be met, the timer must fire before removal may occur again
to achieve a smoother lateral removal.
A Drilling handpiece
B Removed caries
Figure 3-9. Demonstration of drilling
Collision detection is achieved by comparing a collection of height offsets representing
the shape of the burr to the tooth surface around the virtual device tip. This pre-computed
burr mask avoids the need to constantly recompute the geometry of the burr shape.
The implementation of this mask is a combination of two height functions which is
essentially identical to the implementation of the underlying tooth data model. The burr
is considered to be colliding with tooth if the burr top is above the tooth bottom, or if the
burr bottom is below the tooth top.
3.7 Sound
A realistic handpiece sound plays when the handpiece is on. Improving the
sound module will be important in creating a more effective multi-modal dental training
experience.
3.8 Ergonomics
It was important for us to create a convincing ergonomic simulation using low-cost
supplies. Our ultimate goal is to minimize the amount of time from when an experienced
dental practitioner first sits down with CEDS, and when the practitioner is able to work
26
Figure 3-10. CEDS haptic loop Activity Diagram
with it naturally. A dental professional should be an instant expert at operating a proper
dental simulation program.
We originally attached our Augmented Reality (AR) marker to a mask (similar
to what a practicing dentist would wear during work on a patient), but this was too
cumbersome for our simulation and introduced sanitation issues (we needed to prepare
another mask for every new user). Our current solution is to affix the marker to a
headband. The marker is only detected if it is completely unobscured, which can be
troublesome for users with longer hair.
27
Figure 3-11. An AR marker attached to a headband
The ball grip and pistol grip that come with the Novint falcon do not provide a
realistic interface to a dental simulation. We were able to acquire a broken dental
handpiece that we attached to a ball grip. While this is an excellent way to recycle
inoperable drills, if no handpiece can be obtained a pen or toothbrush can serve this
purpose.
Figure 3-12. A dental handpiece attached to a Novint Falcon grip
We first chose to reorient the device so that the device arms point upward.
Considering that the prevailing force from the user will be downward for drilling, we
wanted the device to be oriented to provide the most powerful and stable response. The
working area is haptically rendered approximately in the middle of the device coordinate
system to make sure the response is most effective.
We lift the device slightly above the floor so that when the device arms are
outstretched to the centering position, the hands of a seated user land comfortably
on the handpiece. The device is in a box so that a flat surface can be used as a fulcrum
point for the hand. Having a spot to rest the hand while operating the handpiece is very
28
Figure 3-13. The Novint Falcon with attached handpiece oriented vertically
important for dental preparation. It avoids requiring the user to suspend their arm in
place for long periods of time. During a real preparation, a dentist rests an outstretched
finger on another tooth.
To switch between right-handed and left-handed users, the device can be turned
and the view can be flipped.
Figure 3-14. Students at the UF College of Dentistry trying CEDS
29
Figure 3-14 shows a complete experimental setup. The AR marker, fulcrum surface,
handpiece, and display window are highlighted in red.
While 3D screens or wearable Virtual Reality displays are ideal for immersive
simulation, the current goal of CEDS is an effective training simulation using ubiquitous
off-the-counter technology. We place a conventional screen in between the eyes of the
user and the handpiece in an attempt to render the virtual scene in approximately the
same place where the real tooth would be expected. This setup can be accomplished
with most flat screens, including laptops (if the screen is flipped upside-down).
30
CHAPTER 4
TESTING AND EVALUATION
CEDS was tested on UF dental students. They were given a brief set of instructions
and were instructed to try all the features of the system. After using the system the
students were asked to respond to a brief survey. The instructions and a summary of the
survey results are included in the Appendix.
In general students and teachers receive the system enthusiastically, but agree that
it is not quite ready to replace conventional training methods. This sentiment is shared
by UF Dental Professor Karl-Johan Soderholm.
The key weakness is a true 3D perception, which makes it difficult to
initially orient the burr in space. The tactile feeling is now there, but even that
needs to be refined until the simulator has reached its fullest potential.
When asked how much a student would pay for the software (separate from any
hardware costs) the average response was $36.36 (taking not sure/not much as $0.00).
Though only a minority of respondents found that CEDS could replace conventional
training methods, a majority imagined the system, as it is now, useful for beginning
students.
Testers rated orientation features like the red line extending from the burr, the light
on the end of the handpiece, and the AR marker navigation highly. They agreed that
finding the virtual tooth with the handpiece is easy, even without a real 3D screen.
Additional feature requests included a mode for a low speed handpiece, special
features to view exactly how deep the user is in the tooth (how close to the pulp
chamber), sound that dynamically changes with drill speed and removal, alternative
burr options, real 3D, and a foot pedal for control of speed.
31
CHAPTER 5
FUTURE WORK
As 3D display devices become more ubiquitous, it might no longer too expensive
to use a small glasses-free 3D screen (perhaps a cellular phone or portable gaming
device). We may also test out the effectiveness of simple red/blue 3D (which does not
require a special screen), and in particular the combination of the AR marker input
with 3D glasses. If the screen could be attached directly to the haptic device, then the
simulation setup would become truly portable. We imagine students being able to check
out a haptic device from their training lab and practice on their computer at home.
Figure 5-1. Anaglyph 3D red/blue glasses with the CEDS ’UF’ AR Marker Attached
Because the data structure is rendered directly as a height map it is efficient and
can be further optimized (perhaps through vertex arrays or GPU programming). The
entire application could then be executed on a portable device. The next version of
CEDS might be made to run on a Nintendo 3DS or an Android phone.
It should be easy to add undo capabilities so that users can step back to a previous
version of their preparation and try again. We can keep as many snapshots of the data
model as memory would allow and swap between them when desired. Undo semantics
would go hand-in-hand with an infraction system that would detect when the user makes
a major mistake and give the user the option of going back and trying again.
A weakness of the current data model is that there is no consideration of the tooth’s
structural integrity. In reality when dentists prepare a tooth they must be very careful not
to leave the drilled area brittle. Furthermore, any small protrusions would likely snap off
32
because they are too brittle. In the current implementation it would be possible to create
tall, thin columns that would be impossible to construct by drilling on a real tooth. In the
future we can add a smoothing step that would check the local drilling area for such
impossible structures and remove them.
Tooth decay and tooth layer generation can be slightly randomized. Instead of
simply always filling up the lowest parts of the tooth surface with decay, decay would be
implemented as a semi-random walk with a possible tendency to follow the various tooth
surface grooves. The depths and hardness of the decay and tooth layers could also be
slightly randomized to make sure that students are prepared for the unexpected.
Sound pitch should be adjusted to react to the burr speed. Sound should also react
to material contact.
We would like to find a foot pedal to control burr speed. This has proven difficult
because currently available commercial foot pedals are either binary switches, flight
pedals, or attached to a massive steering wheel assembly.
We would like to be able to provide more interesting graphical feedback to the
user to enhance the learning experience for novices. This could include generating a
simulation of an x-ray image of the tooth so that the student could practice understanding
how to drill a tooth using the same information a dentist might receive in a real work
environment. Beginning students might benefit from seeing a real-time cross section of
their drilling work to get a better sense of how far down they can drill before going too
deep and penetrating the pulp chamber.
We would like to draw a full mouth of teeth at once to help orient the user and
simply the tooth choosing process. We would also like to render the rest of the oral
cavity and external mouth to fully surpass the realism of the typodont. Additionally, we
want to allow the orientation of the mouth to change relative to the handpiece.
Finally, in the future we will allow dental experts to calibrate the preparation grading.
Similar to how we can test how close the user is to removing all decay, we could test to
33
see how close the user is to matching the ideal preparation. Allowing a user to easily
compare their work to the ideal preparation and retry as much as possibly would create
the ultimate tool for dental training.
34
APPENDIX A
USER TESTING INSTRUCTIONS
A.1
Introduction
CEDS is a simple and cost-effective dental drilling simulation that uses common
off-the-shelf technology in an attempt to replace traditional dental preparation training
methods.
When using CEDS try to sit and grasp the handpiece naturally. Moving the real
handpiece will move the handpiece on the screen. You will actually feel a real force
when you touch the virtual tooth with the virtual handpiece.
The hardness of the virtual tooth material and the speed of removal can be adjusted
at any time.
A.2 Controls
•
Grade (g): Press the ’g’ key to display the percentage of decay remaining.
•
Reload (r): Press the ’r’ key to reload the tooth (undo all drilling).
•
Drilling (d): Press the ’d’ key or the space bar to toggle drilling on/off (compare
drilling mode to a high-speed 330 carbide bur).
•
Hand (h): Press the ’h’ key to switch between a right-handed view and a
left-handed view.
•
Viewing (with Augmented Reality): Wear a headband with the blocky ’UF’ logo.
This is an Augmented Reality (AR) marker. If the camera that is on the top of the
computer screen has a good view of this marker, then the virtual scene will rotate
relative to the position of your head. Use this tool to help you naturally navigate the
scene.
•
Viewing (with the mouse): Click and drag on the computer track pad to rotate the
virtual scene.
•
Zooming: Put two fingers on the track pad and drag up or down to zoom in or out
of the virtual scene.
•
Switching Teeth: The virtual teeth are all loaded from 3D scans of real teeth. 21 28 and 31 - 38 are available (FDI numbering). The default is 36.
35
•
Decay: The black decay is automatically generated based on the 3D model of the
tooth. The decay is placed by finding the lowest point on the tooth closest to the
tooth center (when looking from above). The decay will extend downward through
the enamel. In the future the decay would be randomized to enhance training
effectiveness.
•
Dentin: Dentin is colored yellow and located at a constant depth below the
enamel surface. In the future the dentin would be randomized to enhance training
effectiveness.
36
APPENDIX B
SURVEY RESULTS
These results are from 11 UF Dental Students (4 Juniors, 7 Seniors).
•
•
•
•
A force feedback system costs approximately $150. How much in addition to the
cost of the force feedback system, would you be willing to spend to own a copy of
CEDS?
–
$0 / Not Sure / Not Much: 8
–
$50: 1
–
$150: 1
–
$200: 1
Where do you see CEDS fitting into the dental curriculum?
–
First year or pre-dental: 8
–
I don’t see it fitting into the dental curriculum / not sure: 2
–
Practicing before cutting a plastic tooth / for orientation only, not simulation: 3
What extra features would be useful? (optional)
–
Low speed handpiece: 3
–
Cross-section/X-ray view: 1
–
Drill sound pitch modulation: 2
–
More burrs: 2
–
Real 3D: 1
–
Foot Pedal: 1
Does CEDS have major shortcomings? How can they be improved? (optional)
–
Difficult to orient: 1
37
Table B-1. Quantitative survey responses
Strongly
Disagree
Disagree
0.0% (0)
0.0% (0)
The virtual tooth
looks like a real
tooth.
The virtual tooth
27.3% (3) 36.4% (4)
feels like a real tooth.
Material removal
0.0% (0)
54.5% (6)
from the tooth feels
plausible.
36.4% (4)
CEDS could be used 9.1% (1)
in lieu of the dental
practice pad.
CEDS could be
27.3% (3) 63.6% (7)
used in lieu of plastic
practice teeth.
CEDS could be used 81.8% (9) 18.2% (2)
in lieu of real teeth.
Finding the virtual
0.0% (0)
27.3% (3)
tooth with the
handpiece is easy.
The vertical red line
0.0% (0)
0.0% (0)
helps position the
burr on the virtual
tooth.
The light coming
0.0% (0)
0.0% (0)
from the handpiece
helps orientate the
burr around the
virtual tooth.
The ’UF’ augmented 9.1% (1)
9.1% (1)
reality marker helps
orient the view.
CEDS should be
18.2% (2) 0.0% (0)
part of every dental
student’s training.
38
Don’t
Know
0.0% (0)
Agree
54.5% (6)
Strongly
Agree
45.5% (5)
0.0% (0)
36.4% (4)
0.0% (0)
9.1% (1)
36.4% (4)
0.0% (0)
27.3% (3)
27.3% (3)
0.0% (0)
0.0% (0)
9.1% (1)
0.0% (0)
0.0% (0)
0.0% (0)
0.0% (0)
9.1% (1)
54.5% (6)
9.1% (1)
0.0% (0)
45.5% (5)
54.5% (6)
0.0% (0)
72.7% (8)
27.3% (3)
9.1% (1)
45.5% (5)
27.3% (3)
72.7% (8)
9.1% (1)
0.0% (0)
REFERENCES
[1] W. Mix, “Learn-a-prep ii.” http://whipmix.com/product/learn-a-prep-ii/.
[2] L. Boushell, R. Walter, and C. Phillips, “Learn-a-prep ii as a predictor of
psychomotor performance in a restorative dentistry course,” Journal of Dental
Education, vol. 75, no. 10, pp. 1362–1369, 2011.
[3] K. H., “Artoolkit.” http://www.hitl.washington.edu/artoolkit/.
[4] C. Dentoform, “Simulation teeth.” http://www.columbiadentoform.com/main.asp?
CAT=5.
[5] G. Thomas, L. Johnson, S. Dow, and C. Stanford, “The design and testing of a
force feedback dental simulator,” Computer Methods and Programs in Biomedicine,
vol. 64, no. 1, pp. 53–64, 2001.
[6] L. Kim and S. Park, “Haptic interaction and volume modeling techniques for realistic
dental simulation,” The Visual Computer: International Journal of Computer
Graphics, vol. 22, pp. 90–98, 2006.
[7] H. Noborio, D. Sasaki, Y. Kawamoto, T. Tatsumi, and T. Sohmura, “Mixed reality
software for dental simulation system,” in Haptic Audio visual Environments and
Games, International Workshop on, pp. 19–24, 2008.
[8] P. Rhienmora, P. Haddawy, P. Khanal, S. Suebnukarn, and M. Dailey, “A virtual
reality simulator for teaching and evaluating dental procedures,” Methods of
Information in Medicine, vol. 49, pp. 396–405, 2010.
[9] G. Liu, Y. Zhang, and W. Townsend, “Force modeling for tooth preparation in a
dental training system,” Virtual Reality, vol. 12, pp. 125–136, 2008.
[10] J. Wu, G. Yu, D. Wang, Y. Zhang, and C. Wang, “Voxel-based interactive haptic
simulation of dental drilling,” in Proceedings of the ASME 2009 International Design
Engineering Technical Conferences Computers and Information in Engineering
Conference, pp. 1–10, 2009.
[11] H. Yau, L. Tsou, and M. Tsai, “Octree-based virtual dental training system with a
haptic device,” Computer-Aided Design Applications, vol. 3, pp. 415–424, 2006.
[12] J. William, O. Telles, and L. W.S., “Interactive 3d haptic carving using combined
voxels and mesh,” in Haptic Audio Visual Environments and Games, International
Workshop on, pp. 108–113, 2008.
[13] Novint, “Virtual reality dental training system (vrdts).” http://www.novint.com/
index.php/medicaldental.
[14] Sensable, “Sensable dental.” http://www.sensabledental.com/dental-home.htm.
[15] VOXEL-MAN, “Voxel-man dental.” http://www.voxel-man.de/simulator/dental/.
39
[16] Moog, “Haptic technology in the moog simodont dental trainer.”
http://www.moog.com/markets/medical-dental-simulation/
haptic-technology-in-the-moog-simodont-dental-trainer/.
[17] I. Navigation, “Dentsim.” http://www.denx.com/DentSim/overview.html.
[18] M. Nishida, T. Sohmura, and J. Takahashi, “Training in tooth preparation utilizing a
support system,” Journal of Oral Rehabilitation, vol. 31, pp. 149–154, 2004.
40
BIOGRAPHICAL SKETCH
William Christopher Hunt received his B.S. in Digital Arts and Sciences and his B.A.
in East Asian Languages and Literatures from the University of Florida in 2010, where
he graduating with honors. He spent the 2008-2009 academic year as an undergraduate
in Tokyo at Aoyama Gakuin University through a UF exchange program.
William began research with Jörg Peters in the UF SurfLab in 2010 as a graduate
student, where he focused on the CEDS dental project.
During his studies William worked as a software developer at the Gainesville
test preparation company Gleim Publications starting in 2008. In 2011 he accepted a
position as a Systems Engineer with Mitsubishi Heavy Industries in Yokohama, Japan.
41