slides

TACTILE SENSING & FEEDBACK
Jukka Raisamo
Multimodal Interaction Research Group
Tampere Unit for Computer-Human Interaction
Department of Computer Sciences
University of Tampere, Finland
Contents
• Tactile sensing in detail
• Tactile feedback
• Feedback technologies & displays
1
Tactile sensing
2
Tactile sensing
• There’s two different types of receptors
responsible for tactile sensing found in the skin
– free nerve endings
– encapsulated nerve endings (i.e., mechanoreceptors)
• Most tactile information is delivered via
mechanoreceptors but, e.g., hair receptors also
affect the sensations
Bent hair
Skin
Indented
skin
Bent hair
RA
receptor
3
Indented
skin
RA
receptor
Sustained
pressure
SA receptor
Remember from Lecture 1:
RA = rapidly adapting
SA = slowly adapting
Mechanoreceptors 1/3
• Mechanoreceptors are sensitive to mechanical
pressure or deformation of the skin
– differ in size, receptive fields, rate of adaptation,
location in the skin, and physiological properties
– four types: Meissner’s corpuscles, Pacinian corpuscles,
Merkel’s disks and Ruffini endings
4
Mechanoreceptors 2/3
• Thresholds of different receptors overlap
– in the brains the sensation is determined by the
combined inputs from different types of
receptors
– operating range for the perception of vibration
about 0.04 to 500 Hz (for hearing about 20 – 20000
Hz)
– frequencies over 500 Hz are felt more as
textures, not vibration
– skin surface temperature affects perceiving
tactile sensations (inhibites or excites individual
receptors)
5
Mechanoreceptors 3/3
Receptor
Merkel’s
disks
Ruffini
endings
Meissner’s
corpuscles
Pacinian
corpuscles
Rate of
Location
adaptation
SA-I
Shallow
Receptive
field
2— 3 mm
Stimulus
frequency
0— 30 Hz
0— 15 Hz
SA-II
Deep
>10 mm
RA-I
Shallow
3— 5 mm
PC (RA-II)
Deep
>20 mm
Function
Pressure; edges and intensity
Directional skin stretch,
tension
10— 60 Hz Local skin deformation, low
frequency vibratory sensations
80— 400 Hz Unlocalized high frequency
vibration; tool use
• Mechanoreceptors are generally specialized to
certain stimuli
– contact forces are detected by Merkel’s discs and Ruffini
endings
– vibration primarily stimulates the Meissner’s corpuscles
and Pacinian corpuscles
6
Hairy vs. hairless skin
• Hairy skin is generally less sensitive to
vibration compared to glabrous skin
– there seems to be no Pacinian receptors in the
hairy skin (however, they are present in the deeper
underlying tissue surrounding joints and bones)
• Hairy skin is poorer to detect both vibration
& pressure
– yet has about the same capacity for
discriminating vibrotactile frequencies
7
Tactile dimensions
• Tactile acuity (vibration & pressure)
• Spatial acuity
• Temporal acuity
8
About thresholds
• Threshold = the point at which an effect is
consciously experienced
– detection threshold (the smallest detectable level of
stimulus; a.k.a. absolute threshold)
– difference threshold (the smallest detectable
difference between stimuli; a.k.a. just noticeable
difference (JND))
• To reduce the detection threshold:
– increase the duration of the tactile stimulation
– increase the area of stimulation
– increase the temporal interval of two
consecutive stimuli (within certain limits)
9
Tactile acuity for vibration
• Vibration primarily stimulates Pacinian
corpuscles and Meissner’s corpuscles
• pacinian channel (high frequency, from about 60Hz)
• non-pacinian channel (low frequency, below 60Hz)
• Human sensitivity for vibration:
– sensitivity for mechanical vibration increases
above 100 Hz and decreases above 320 Hz (250
Hz said to be the optimum)
• An absolute threshold of 0.2 m (2/1000 of a
millimetre) in amplitude has been reported
on the palm of a hand for 250 Hz vibration
10
Tactile acuity for pressure
• Pressure primarily stimulates the Merkel’s
disks
• Sensitivity for pressure is largely dependant
on the location of stimulation
– discrimination has higher resolution at those parts
of the body with a low threshold (e.g., fingertips)
• The face is being reported to have the
smallest detection threshold of about 5 mg
(5/1000 of a gram) in weight (equals dropping a
wing of a fly from 3 cm onto the skin)
11
Tactile acuity
• Threshold responses for
pressure (bars) and 200
Hz vibration (dots) for
15 body sites
• Noteworthy:
– human body is highly
sensitive for vibration
– vibration thresholds
correlate with the
density of cutaneous
mechanoreceptors
12
Age and tactile acuity
• There appears to be no significant
reduction in vibrotactile detection at the
fingertips in older subjects.
• Pressure sensitivity reduces as a function of
age
• Training can be used to improve sensimotor
performance
13
Spatial acuity 1/2
• Fingertips are the most sensitive part of the
human hand in texture & vibrotactile perception
– reported to have the largest density of PC receptors
– the more there is spatial distance between two stimuli,
the more difficult it is to discriminate them
• Tactile texture perception is mediated more by
vibrational cues for fine textures, and by spatial
cues for coarse textures
– discrimination of spatial interval is considerably more
accurate than temporal interval
– when using hand, exploration of spatially varying surfaces
is done with larger area of skin (increased sensitivity by
active touch)
14
Spatial acuity 2/2
• Spatial dimensions for touch
– two-point discrimination (two simultaneous points
of stimulation)
– point localization (two consecutive point of
stimulation)
– grating discrimination (detectable difference
between two gratings)
• Why do people do better with gratings than
two-point discrimination?
– active vs. passive touch
15
Spatial acuity for pressure 1/2
• Spatial thresholds (in
mm) for two-point
discrimination (bars)
and point localization
(dots) for 14 body sites
• Noteworthy:
– smallest threshold in
facial area & hands
– threshold for point
localization lower
throughout the body
16
Spatial acuity for pressure 2/2
• Pressure thresholds (in
mg) for two-point
discrimination applied
on the left (dots) and
right (bars) side of the
body
• Noteworthy:
– no major difference
between the sides of
the body
– smallest in facial area,
fingers have about the
same acuity as trunk
17
Temporal acuity
• Temporal resolution for touch with two
successive stimuli is about 5 ms
– To compare:
for audition: about 0.01 ms for clicks
for vision: about 25 ms
• Resolution for tactile numerosity is
reported to lie between 1 to 5 pulses (with
intervals of 20 and 100 ms)
– at brief intervals two pulses presented to the
same location may mask one another
18
Thermotactile interactions
• Eventhough being separate modalities,
temperature and touch have interactions
– thermal adaptation
• cooling degrades tactile sensitivity
• warming sometimes enhances
– thermal intensification
• cold objects feel heavier
• warm objects feel heavier but less than cold ones
– thermal sharpening
• the warmer or colder the two points are, the easier they
are to discriminate
• Thermal cues are very important in the
identification of objects
19
Touch is not an absolute sense
• Several factors affect the sensitivity
–
–
–
–
–
–
–
age
sex
individual differences
attention, fatigue, mood, stress
diseases, disabilities
training
...
ð scalability is important factor for tactile
interfaces
20
Tactile feedback technologies
21
Methods for tactile stimulation
• The methods include:
–
–
–
–
–
–
skin deformation
vibration
electric stimulation
skin stretch
friction (micro skin-stretch)
temperature
22
Tactile actuators
• There are several different technologies
used in tactile interfaces
–
–
–
–
–
–
vibrating motors
linear motors
solenoids
piezoelectric actuators
pneumatic systems
...whatever causes an effect can be used
• Possible actuator configurations
– single element
– multiple elements (an array/matrix)
23
Actuators: vibrating motors
24
Vibrating motors
• How they work:
– provides relatively small-amplitude vibration
(linear or rotary)
– applies motion either directly to the skin or
through mediating structure
– used singly or in arrays
• Most common types
– DC-motors with eccentric rotating mass
– voice coils
25
Vibrating motors: eccentric rotating mass
• DC-motor rotates an offcenter spinning mass
– inexpensive & exsisting
technology
– poor resolution: it takes time
to start and stop
• Frequency control only
(amplitude ~= freq2)
– amplitude fixed by the size &
the weight of the rotating
mass and the speed of
rotation
• Used in various devices
– mobile phones, pagers,
gaming devices, etc.
26
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart y our computer, and then open the file again. If the
red x still appears, y ou may have to delete the image and then insert it again.
Vibrating motors: voice coils
• Voice coil basics
– current driven through the
movable coil
– created magnetic field interacts
with the field of the permanent
magnet (one-way movement)
– vibrations created by switching
the current on/off
• Both frequency and amplitude
can be controlled somewhat
independently
– however, the motor has always a
peak at certain frequencies (e.g.,
250 Hz)
27
The image cannot be displayed. Your computer may not hav e enough memory to open the image, or the image may hav e been corrupted. Restart y our computer, and then open the file again. If the red x still appears, y ou may hav e to delete the image and then
insert it again.
Vibrating motors: overview
• Advantages:
–
–
–
–
simple, existing technology
relatively inexpensive
easily powered and controlled
quite small power consumption
• Disadvantages:
– not very expressive feedback
– vibration can be irritating
– sometimes hard to miniaturize efficiently
28
Actuators: linear motors
29
Linear motors: pin displays
• How they work:
– pins in an array are actuated independently
– the actuated pins contact the surface of the skin
• Advantages:
– simple, readily available
– continuously positionable
– versatile: static pressure, vibration; shapes or force
display
– relatively fast
• Disadvantages:
– very difficult to pack tightly
– relatively high cost (lots of motors/device)
30
Example: tactile array
• Mimics complex tactile
sensations
– stimulate the fingertips
– each pin has
piezoelectric actuator
– Array 1: 100 pins over 1
cm2, frequency range
25-400 Hz
– Array 2: 24 pins with 2
mm spacing, 25-500Hz
31
Example: Braille displays
• Braille = tactile
language for sensory
substitution
• Traditionally Braille
displays used solenoids
to push up the pins
(nowadays mostly
piezoelectric actuators
are used)
32
Example: tactile arrays in a mouse
• Allows the user to scan
the of an image
– the pins rise and fall
dynamically delivering a
tactile stimuli to the
fingertips
– can be used to code
patterns and colours into
tactile data
• VTMouse (2001)
– three 4x8 matrix (32 pins)
put in the place of the
buttons
• VTPlayer (2003)
– two 4x4 matrix with 16
pins
(http://www.virtouch2.com/)
33
Actuators: solenoids
34
Solenoids
• Multi-modal mouse by
Akamatsu & MacKenzie
(1996)
– solenoid driven pin under
the left index finger that
moves up & down to
generate vibration
• Haptic Pen by Lee et al.
(2004)
– solenoid shakes the pen by
moving up and down at top
of the pen
35
Actuators: piezoelectric actuators
36
Piezoelectric actuators 1/2
• How they work:
– single or multilayer ceramic elements
– an element expands/bends when voltage is
applied
– multiple layers can be used to amplify the effect
• Properties:
– very large forces but small motions
– one element typically around 0.2-1.0 mm thick
– resolution for frequencies ~0.01 Hz
37
Piezoelectric actuators 2/2
• Electromechanical device
that converts electrical
energy into mechanical
motion
• Typically very compact as
only few components are
used in a complete system
– actuator itself can be very
small
38
Example: STReSS & Virtual Braille Display
• 2D tactile display with
an array of miniature
actuators
– stimulate the fingertip
at about 1 cm2 in area
– elements can be
bended in two
directions to increase
the forces applied to
the fingertip
(http://www.laterotactile.com/)
39
Example: Tactile Handheld Miniature
Bidirectional (THMB)
• THMB is an improved
version of VBD miniaturized
to fit inside a PDA-size case
• The handheld device
comprises an LCD screen
that allows combining
tactile and visual feedback
• THMB stimulates the user's
thumb and is mounted on a
vertical slider so that it can
be dragged up and down
along the left side of the
case
(http://www.laterotactile.com/)
40
Piezoelectric actuators: overview
• Advantages:
–
–
–
–
–
small in size
potentially inexpensive in large volumes
high frequency and static modes
very fast response time
low power consumption
• Disadvantages:
– dynamics: small displacements require accurate
amplification
– high driving voltage
41
Actuators: pneumatic systems
42
Pneumatic systems
• Two possible output modes based on skin
indentation (and vibration)
– suction
– air-pressure
• How it works:
– technologies: fillable air-pockets, air jets,
suction holes
– vibratory rates: typically 20-300 Hz
– static pressure with sealed pockets
43
Pneumatic systems: suction
• Draws air from a suction
hole creating an illusion
that the skin is pushed
• Very low spatial resolution
(only appropriate for the
palm)
– two basic patterns of
stimulation (large holes and
small holes)
• Need for regulation of air
pressure (=lots of
equipment)
44
Pneumatic systems: air-pressure
DataGlove with pneumatics
(Sato et al., 1991)
Teletact II (Stone, 1992)
• DataGlove
– bandwidth of 5 Hz, amplitude & frequency modulated
• Teletact II
– 29+1 air pockets (40 tubes to control the air-pressure)
– object slippage (fingers) + force feedback (palm)
45
Pneumatic systems: overview
• Advantages:
– tubing make it possibly to take the bulky part away from
point of application
– pressure can be more appropriate for some applications
than pins or vibrating motors
– can mimic skin-slip (with multiple adjacent inflated
pockets)
• Disadvantages:
– requires bulky parts (air compressor or motor-driven
pistons)
– not really portable
– can be very noisy
– difficult to display sharp edges or discontinuities
46
Actuators: shape-memory alloys
47
Shape-memory alloys
• Metals that "remembers" their geometry
– restores its original geometry when heated
– usually temperature change of about 10°C is
necessary to initiate the phase change
• How it works:
– expands (and heats up) when current runs
through it
– contracts when cools down
– stimulates the skin when vibrates (expandcontract cycles)
48
Shape memory alloys
Wearable Tactile Displays (MIT Touchlab)
Tactile Display based on
Shape Memory Alloy
Tactile Display based on
Elastomer Actuators
49
Tactile displays: skin stretch
50
Skin-stretch
• Two main methods:
– rotational skin stretch
– lateral skin stretch
• What happens:
– forces are applied to skin
for displacement
– contact forces are
perceived as stretching of
the skin
• Applying skin stretch is
being investigated as an
alternative method to
vibrotactile feedback
51
Friction: skin-slip display
• Micro skin-stretch
– motor driven smooth
cylinder strapped against
finger
– when rotates, stimulates the
mechanoreceptors
(Chen and Marcus, 1994)
• Felt as a sensation of slip
– grasp simulations: causes
the user to increase grip
force
– often used to append force
feedback displays
52
Tactile displays: electrotactile
stimulation
53
Electrotactile stimulation
• Electrical stimulation is not widely
accepted to consumer use
– often sudden bursts give an "invasive"
impression
– “square waves” can be easily felt as too strong
stimuli and they keep tickling the nerves
– the sensitivity to electrical stimulation varies
greatly between and within individuals (e.g.,
sweating & pressure affect the sensation)
• Used mostly in research prototypes and for
rehabilitation purposes
54
Example: SmartTouch
• Tactile display to present
realistic skin sensation
– a thin electrotactile display
and a sensor mounted
• Two layers
– top layer: 4x4 array of
stimulating electrodes
– bottom layer: optical sensors
• Visual information is
captured by the sensors and
displayed through electrical
stimulation
– e.g., the black stripes on a
paper are perceived as bumps
(http://www.star.t.u-tokyo.ac.jp/projects/smarttouch/)
55
Tactile displays: Dielectric elastomer
actuators
56
Example: Dielectric polymer
• Uses a dielectric polymer
film (c) between two
electrodes (b & d)
– voltage causes the electrodes
to attract each other
– the film contracts in thickness
and expands in area
• Runs at around 1000 V
(DC) at very low current
– require less power
compared to traditional
vibration motors and
piezo actuators
57
Actuators: Ultrasonic transducers
58
Example: Ultrasonic transducer
• Based on acoustic
radiation pressure
– a prototype consists of
324 airborne ultrasound
transducers controlled
individually
– the feedback can be felt
about 20 cm over the
surface
• Although the produced
force is weak to feel
constant pressure, it
was sufficient for
vibratory sensation
59
Actuators: Electrorheological fluids
60
Example: Electrorheological fluids
(a)
(b)
• Liquid which viscosity changes
into semi-solid when electric
current is applied (pic. a b)
– change in viscosity feels as more
resistive surface
– can change from liquid to gel, and
back, within milliseconds
• The change in viscosity is
proportional to the applied
current
• Can be used to simulate
different surface frictions
61