First Evaluation of A Novel Tactile Display Exerting

First Evaluation of A Novel Tactile Display Exerting Shear
Force via Lateral Displacement
KNUT DREWING
Max Planck Institute for Biological Cybernetics
MICHAEL FRITSCHI
Technische Universität München
REGINE ZOPF, and MARC O. ERNST
Max Planck Institute for Biological Cybernetics
and
MARTIN BUSS
Technische Universität München
Based on existing knowledge on human tactile movement perception, we constructed a prototype of a novel tactile multipin
display that controls lateral pin displacement and, thus produces shear force. Two experiments focus on the question of whether
the prototype display generates tactile stimulation that is appropriate for the sensitivity of human tactile perception. In
particular, Experiment I studied human resolution for distinguishing between different directions of pin displacement and
Experiment II explored the perceptual integration of information resulting from the displacement of multiple pins. Both experiments demonstrated that humans can discriminate between directions of the displacements, and also that the technically
realized resolution of the display exceeds the perceptual resolution (>14◦ ). Experiment II demonstrated that the human brain
does not process stimulation from the different pins of the display independent of one another at least concerning direction. The
acquired psychophysical knowledge based on this new technology will in return be used to improve the design of the display.
Categories and Subject Descriptors: H1.2 [Models and Principles]: User/Machine Systems—Human information processing;
H5.2 [Information Interfaces and Presentation]: User Interfaces—Evaluation/methodology; haptic I/O; theory and methods
General Terms: Design, Experimentation, Human Factors
Additional Key Words and Phrases: Haptic interfaces, psychophysics, tactile movement perception, shear force, tangential
displacement
This work is part of the TOUCH-HapSys project financially supported by the 5th Framework IST Program of the European Union,
action line IST-2002-6.1.1, contract number IST-2002-38040. For the content of this paper the authors are solely resposible for,
it does not necessarily represent the opinion of the European Community.
Knut Drewing is now at the Institute of Psychology, Giessen University, Otto-Behaghel-Str. 10F, 35393 Gießen, Germany; email:
[email protected].
Authors’ addresses: Knut Drewing, Regine Zopf, and Marc Ernst, Max Planck Institute for Biological Cybernetics, Spemannstraße
38, 72076 Tübingen, Germany; email: {regine.zopf, marc.ernst}@tuebingen.mpg.de; Michael Fritschi and Martin Buss, Institute
of Automatic Control Engineering (LSR), Technische Universität München, 80290 München, Germany; email: {michael.fritschi,
martin.buss}@ei.tum.de.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided
that copies are not made or distributed for profit or direct commercial advantage and that copies show this notice on the first
page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists,
or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested
from Publications Dept., ACM, Inc., 1515 Broadway, New York, NY 10036 USA, fax: +1 (212) 869-0481, or [email protected].
c 2005 ACM 1544-3558/05/0400-0001 $5.00
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005, Pages 1–14.
2
1.
•
K. Drewing et al.
INTRODUCTION
Human tactile perception is a complex integration of various sensations evoked by forces acting on our
skin. Tactile displays are the fundamental technical component of a virtual tactile environment that
tries to recreate these sensations by generating forces. For a long time, tactile displays have usually
been constructed either as shape or as vibrotactile displays. Shape displays follow the idea to render
the 3D-shape of an object to the skin. They produce displacements which are quasistatic, have large
amplitude, and indent the skin [Lee et al. 1999; Shinohara et al. 1998]. This goal in many cases is
achieved with an array of small pins that are mutually independent. Vibrotactile displays also use a
pin array to produce displacements. But in contrast to shape displays, displacements in vibrotactile
displays have a small amplitude and are vibratory (frequency range of about 25 to 500 Hz; e.g., Essick
[1998], Ikei [2002], Ikei et al. [1997], Summers et al. [2001], Summers and Chanter [2002]). Vibrotactile
displays create 2D rather than 3D patterns on the skin. Both types of displays are usually based on
forces that are normal to the contact surface. However, in order to generate a realistic impression of the
environment, it is probably as important to provide forces lateral to the human skin, the so-called shear
forces. At present, there are a few prototypes or experimental displays that provide lateral stimulation
in differing ways to the human finger [Hayward and Cruz-Hernandez 2000; Salada et al. 2002, 2004].
The display of lateral stimulation is particularly reasonable when considering movements of the
finger relative to the environment. The tactile perception of to-be-expected or actual movement plays
an important role in haptically guided action and haptics. To give an example, imagine gripping, lifting,
and manipulating a fragile object like an egg: Grip forces should be sufficiently large to avoid slip of the
object, but also sufficiently small to avoid damage to the object. Immediate tactile sensation of skin slip
has been demonstrated to be crucial for a precise grip force control [Johannson and Westling, 1990].
And, in dexterous manipulation the perception of the direction of lateral force, that is, the direction in
which slip is about to occur, seems to be crucial to stably maintain the desired orientation of the object
[e.g., Flanagan et al. 1999]. As importantly, the perception of actual movement is an always present
effect of every active exploration of our environment with our finger. While stroking with the finger
across a surface, the surface stretches our skin and slips beneath the finger.
We are specifically interested in tactile perception of actual movement and the development of an
appropriate multipin device to display lateral effects of finger movements relative to the environment.
Based on existing knowledge on human movement perception, we constructed a prototype of a novel
multipin device that controls lateral pin displacement and, thus produces shear force. However, there
are still many unanswered questions related to tactile perception because the technology has not previously been available. Our approach is to use the prototype device to obtain additional psychophysical
knowledge that in turn will be used to improve the design of the display.
In this paper, we first present the existing psychophysical background and, then, the design of our
multipin display. The two present experiments focus on the question of whether the current display
generates tactile stimulation that is appropriate for the sensitivity of human tactile perception. In
particular, Experiment I studied human sensitivity for distinguishing between different directions
of lateral 2D pin displacement and Experiment II explored the perceptual effect resulting from the
integration of information from multiple pins.
2.
PSYCHOPHYSICS OF TACTILE MOVEMENT PERCEPTION
The tactile perception of movement relative to the finger has been suggested to rely mainly upon two
distinguishable cues: the spatio-temporally ordered translation of a stimulation across the skin and
lateral stretch of the skin (for an overview, see Essick [1998]). In most situations the two cues co-occur:
Srinivasan and colleagues [1990] pressed a plain glass plate with a small protrusion on the finger.
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
Evaluation of Lateral Displacement Display
•
3
Then, the glass plate started to move laterally. For longer movement (14 mm) the plate first sticks
to the skin stretching it, but, then, slipped leading to a translation of the protrusion across the skin.
In their study, shorter movement (5.5 mm) was used to investigate the effects of “pure” stretch without
slip; for the same purpose in other studies, a probe was glued to the skin [Gould et al. 1979]. In contrast,
translation without stretch—inter alia—has been investigated and can be mimicked by pin displays
when sequentially producing indentation at neighboring skin areas [Gardner and Sklar 1994, 1996;
Olausson 1994]. Research suggests that human movement perception is noticeably more sensitive to the
stretch than to the translation component, for example, when comparing the minimal path traversed
to detect relative movement (e.g., on the forearm the minimal path is shorter by factor 7 for stretch as
compared to translation; Gould et al. [1979]). This holds as long as surrounding skin is not prevented
from the spreading influence of stretch [Essick 1998]. Likewise, in display design we focused on stretch
cues and assured the spread of stretch.
The neural base for encoding movement is under debate. It has been suggested that translation
stimuli result in a pattern of sequential activation of adjacent receptors with small receptive field areas
(probably SA I and, particularly FA I [Essick 1998; Srinivasan et al. 1990]). For pure stretch stimuli,
an initial pattern of response in fast adapting receptors and a persisting directional-sensitive one in
slow adapting receptors was observed [cf. Edin 1992; LaMotte and Srinivasan 1991; Srinivasan et al.
1990]. A recent physiological study on SA I, SA II, and FA I receptors in the human fingertip suggests
that most of the corresponding afferents are broadly tuned to a certain preferred direction of stretch
force. In each receptor population preferred directions were distributed in all angular directions, but
with a certain population-specific bias (SA I: in distal direction, SA II: proximal; FA I: proximal and
radial [Birznieks et al. 2001]). Note that these results do not perforce suggest a mechanism of how
receptors encode direction neither for stretch nor for translation. However, a couple of characteristics
of the potentially involved receptors have been reported [Edin 1992; Greenspan and Bolanowski 1996;
Johnson 2002; Johansson and Vallbo, 1979a; Mountcastle et al. 1972]: FA I receptors are very sensitive
to single rapid indentations, SA I receptors provide sustained response to larger indentation (up to 1.5
mm), and SA II receptors are particularly sensitive to stretch. SA I and SA II receptors respond on
frequencies up to 100 Hz, FA I up to 200 Hz. Receptive field areas are 25–60 mm2 for SA II and about
5 mm2 for FA I and SA I receptors; physiological estimates of mean spacings between receptors are
3.2 mm for SA II and 0.8 mm for FA I and SA I [Essick, 1998; Johnson 2002; Johansson and Vallbo
1979b; Johnson et al. 2000]. Recent histology reports even smaller spacings between cells that have
been associated with the receptor types (e.g., 0.2 mm for FA I and 0.5 mm for SA I Nolano et al. [2003]).
In display design we took into account the above basic knowledge on human neural bandwidths.
On a higher perceptual level, psychophysical studies have demonstrated that tactile movement
detection and discrimination between movements in opposing directions depend on various factors—
such as the length of the movement path, or the innervation density of the stimulated skin area (e.g.,
Loomis and Collins [1978], Whitsel et al. [1979]). However, to our knowledge, there is only a single study
on human resolution of movement direction, which is an important parameter for display design. Keyson
and Houtsma [1995] displayed movements on the distal phalanx of the finger via a 0.8 mm diameter
Braille point that was fixed to a trackball. The trackball could be moved in any horizontal direction. A
single stimulus consisted of a forward–backward movement traversing twice a straight path of 3.25 mm
with a velocity of 41 mm/s. For these stimuli, Keyson and Houtsma [1995] report discrimination thresholds for movement direction of about 14◦ with the lowest thresholds for paths that went down to the
wrist. However, these results are not entirely relevant to our device. Our device is designed to maximize
skin stretch, whereas in the Keyson and Houtsma device skin stretch was minimized by minimizing skin
friction. Moreover, our device is designed as a multipin display and by the requirement to achive high
pin density it is constrained to smaller displacements than investigated by Keyson and Houtsma [1995].
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
4
•
K. Drewing et al.
Fig. 1. Mechanical design of the display for two pins in one axis. The human finger is in contact with the pins through a gap
in the base plate covering the device. The depicted optional rubber layer between the pins and the finger was not used in the
present study.
Consequently, we tested in Experiment I human directional resolution for stimuli provided by our
display. The questions were whether, on one hand, our display is able to provide stimuli that humans
can discriminate and, on the other hand, whether its directional resolution satisfies human perceptual resolution. Moreover, the display is a multipin device. Because the technology has not previously
been available, so far no knowledge exists on the perceptual aspects of multiple movement signals.
Experiment II, hence, explored the perceptual integration of information from multipin stimulation.
3.
3.1
TECHNICAL REALIZATION OF THE MULTIPIN DISPLAY
Design and Hardware Setup
The fundamental concept of the display is based on a square 2 × 2 pin array (Figure 1). The pins
move—independently from another—tangentially to the skin along both horizontal directions. A pin
diameter of 1 mm, a center-to-center pin spacing of 3 mm (zero position of lateral movement), and a
lateral movement of 2 mm along each axis and for each pin are realized.
The four pins are in direct contact with the finger (optional filtered by an elastic rubber layer).
Two rods orthogonally attached to the upper region of each pin transmit two-dimensional movement
of the pins. To allow for this movement, the four pin bodies are attached with universal-joint shafts
that are screwed to the ground plate of the chassis. Tuning screws between the base plate and the
joints enable a justification along the z-axis in a small range to vary the indentation offset normal to
the finger. The rods are connected over reduction rocker arms to the servomotor levers. To reduce pin
motion normal to the skin down to a tolerable value, a relatively long pin (69 mm) was used. With a pin
diameter of 1 mm, a realization of a pin with these dimensions would be impossible because of bending.
Furthermore, it would be difficult to attach the two steering rods providing the tangential motion.
To avoid this problem, the length of the pin has been reduced to 6 mm and attached to a thicker,
square-section rod (Figure 2, left). The pins are fitted to the inner edge of the rods (black points) to
provide a large motion workspace for the pins (Figure 2, right). On the actuator side, we use off-theshelf servomotors, selected on criteria such as high performance, small displacements, and light weight.
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
Evaluation of Lateral Displacement Display
•
5
Fig. 2. Pin-body design in side view and pin arrangement in top view. The pins are attached to a thicker, square-section rod and
fitted to the inner edge of the rods.
Fig. 3. Hardware setup of the display. The left picture depicts the device in use and a close-up view of the pin area; the right
picture shows the mechanical details.
The servomotors come with a position control circuit, accessed by a pulse-width modulated signal, which
contains the specified position information.
The left picture in Figure 3 shows the display in use with a close-up view of the pin area that is in contact with the finger through a gap in the cover plate. A more detailed view in the right picture of Figure
3 shows the mechanical details of the display: the pin bodies, the control rods, and the servomotors.
3.2
Computer Interface and Technical Data
A real-time task generates the required signals for the eight servomotors at the printer-port of the
computer, whose data lines are connected to the corresponding servomotors (Figure 4).
In the application process the position information for the servomotors is calculated and communicated to the real-time task. To provide sufficient power for the servomotors an external power supply is
used. The display reaches a positioning resolution of 10 µm in a work space of 2 × 2 mm. It works with
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
6
•
K. Drewing et al.
Fig. 4. The display is connected to a computer via the parallel port, and controlled via a real-time task.
a maximum velocity of 22.8 mm/s applying maximum forces of 4.23 N per pin axis. The display weighs
1100 g and has a total size of 150 × 150 × 90 mm.
4.
EXPERIMENT I: DIRECTION DISCRIMINATION FOR PIN DISPLACEMENT
In Experiment I, we studied human discrimination performance for different directions of tactile
movement cues displayed with the device. The device was optimized to allow for full skin stretch.
Note though that the elasticity of the user’s skin and the exerted pressure also influence how much of
the stimulation is taken up by stretch and slip, respectively. We measured direction thresholds (JNDs)
for single-pin displacement for eight different directions (distal direction: “up,” proximal: “down,” to the
“right” (radial) and the “left” (ulnar) side of the phalanx when looking at the nail and all four directions
in between: “up-right,” “down-right,” “up-left,” “down-left”). We used a two-interval forced choice task
and the method of constant stimuli.
4.1
Method
4.1.1 Participants. Fourteen right-handed participants (nine female and five male) took part for
pay. Their age ranged from 21 to 40 years (average 26 years). None of them reported any known tactile
deficit due to accident or illness concerning the left index finger. With the exception of two staff members
of the laboratory, the participants were not familiar with the device and naı̈ve to the purpose of the
experiment.
4.1.2 Apparatus and Stimuli. In a quiet room participants sat at a table with their left elbow
resting comfortably on a custom-made support. By using robust tape, their left hand was attached to
the device, so that the distal phalanx of their left index finger was reliably centered at the midpoint
of the pin display. Adjustable paperboard stabilizers on the left and right sides of the finger further
established the central finger position. We used the device with a single pin only (for this experiment
the other three pins were removed) and a particular cover plate that allowed for individual adaptation
of gap size along the finger and, thus, maximal spread of skin stretch (Figure 5). The rectangular gap in
the cover plate was individually adjusted so that it was beneath the entire distal phalanx of the index
finger. Just at the very ends the phalanx was supported by the metal plate. The device was placed inside
a box to prevent possible visual cues on pin movement, and white noise via headphones masked the
sounds of the mechanics. A custom-made program on an IBM-compatible PC controlled the stimulus
presentation and collected responses, which were entered via a keyboard.
Tactile stimuli were unidirectional single strokes of the single pin with a length of 1 mm and a velocity
of 10 mm/s. Each stroke started at the center. Stroke directions are defined by their circular deviation
in degree from the direction center-to-distal part of the phalanx (0, cf. Figure 6, left).
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
Evaluation of Lateral Displacement Display
•
7
Fig. 5. Adaptable plates. The plates can be moved along one axis (red arrows) and allow adjustment of the gap in the cover plate
to the individual finger size. In the present experiments, the adaptable plates were used to maximize the skin area stretched by
the pin.
Fig. 6. The left part of the figure depicts the eight standard directions for pin movements in Experiment I (orange arrows) and
their length (1 mm). The right part depicts the comparison directions (gray arrows including the orange standard) with respect
to the standard direction.
4.1.3 Design and Procedure. The design comprised one within-participant variable: stroke direction
realized by eight standard stimuli of different directions (separated by 45◦ , Figure 6, left). For each
standard stimulus, we measured the discrimination threshold (JNDs) for direction—using the method
of constant stimuli in a two-interval forced choice paradigm:
Each standard was paired with 19 comparison strokes, the directions of which were distributed in
10◦ steps around the corresponding standard within a range of ± 90◦ (Figure 6, right). In each of two
experimental sessions, each pair of standard and comparison stroke was presented six times (order of
pairs randomized). The sessions were on different days, lasted 2 h each (including practice trials in the
first and three breaks in each session) and included 1824 trials overall.
A single trial consisted of the sequential presentation of a standard and a comparison stimulus (order
balanced between repetitions). Participants self-initiated a trial via a key press, 1600 ms later the first
stimulus was presented, 2800 ms thereafter the second stimulus. Immediately after each stimulus, a
100 ms high pitch tone signaled the participant to lift his finger until 1.1 s later another 100 ms low pitch
tone signaled to lower the finger back onto the device. In this time, the pin was driven back to the center
of the display without stimulating the finger. Starting 100 ms before each stimulus presentation and
ending with the low pitch tone white noise was displayed on the earphones. 1600 ms after the second
stimulus, participants had to decide by a key press if the direction of the second stimulus differed in
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
8
•
K. Drewing et al.
Fig. 7. Medians and quartiles of individual 84% thresholds for direction discrimination by stroke direction conditions of
Experiment I.
clockwise or counterclockwise direction with respect to the first stimulus. Participants were instructed
to envision the different directions as being drawn on a clock. No error feedback was given.
4.1.4 Data Analysis. Using the psignifit toolbox for Matlab [Wichmann and Hill 2001a, 2001b], we
fitted individual psychometric functions (cumulative Gaussians, maximum-likelihood procedure) to the
proportion of trials in which the comparison stroke was perceived in a more clockwise direction than
the standard stroke—against the direction of the comparison. In the fit the point of subjective equality
was fixed to the direction of the standard. We estimated the 84% discrimination threshold (equaling
the estimate of standard deviation in the fitted Gaussian) and a percentage of stimulus-independent
errors (lapse rate, constrained between 0 and 10%), the additional estimation of which reduces biases in
the threshold estimate [Wichmann and Hill 2001a]. The individual values per stroke direction entered
further analyses. For further statistics, we used nonparametric methods. The used methods base on
rank orders of the measured values rather than on the absolute values and, thus, allow to include
threshold estimates that fell outside the reliably measured range of the present task (1% estimates
<10◦ and 4.5% > 90◦ , that is, the maximal deviation of comparison stimuli from the standard).
4.2
Results and Discussion
Figure 7 depicts medians and quartiles of the individual discrimination thresholds (JNDs) by stroke
direction. Median thresholds ranged from 23◦ (for direction 0◦ ) to 35◦ (for direction 270◦ ). The individual
thresholds per stroke direction entered a Friedman test. The test reached significance, χ 2 (7) = 20.7,
p < .01, indicating a slight perceptual anisotropy. Bonferroni-adjusted post hoc comparisons (Wilcoxon
tests) between pairs of directions in no case reached significance. But, the descriptive data (cf. Figure 7)
suggest that discrimination performance is somewhat better for upward as compared to other directions.
At first glance, the direction of the observed perceptual anisotropy seems to be in contrast with
results of Keyson and Houtsma [1995], who observed that the lowest thresholds for movements are in
the down direction. They argued that their downward movements caused tension in the fingertip where
the finger is anchored to the fingernail and where humans are particularly sensitive. However, their
movements are hardly comparable with ours. Albeit they reduced stretch, it may well be that by the long
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
Evaluation of Lateral Displacement Display
•
9
movement path in their study (3.25 mm in one direction) a tension component came into play, which with
the present shorter path (1 mm) did not considerably contribute to movement perception. It is highly
speculative to relate our results to the direction-encoding bias observed in SA I populations [Birznieks
et al. 2001]. However, it is an interesting question for future research of how such anisotropies relate
to receptor populations and what this may mean for tactile direction encoding.
Between participants median discrimination thresholds (across stroke direction) ranged from 21◦ to
78◦ . A Friedman test confirmed that these individual differences were reliable, χ 2 (13) = 52.4, p < .001.
These differences may point to pronounced individual differences in human resolution of direction.
But they may also reflect individual difficulties with the use of the display, particularly given that the
thresholds of most participants (11 out of 14) fell in the—almost halved—range between 21◦ and 40◦ .
One may ask whether the worst participants (thresholds of 46◦ , 59◦ , and 78◦ ) simply failed to detect
the movement. However, being asked all participants reported that they perceived movement and,
more importantly, even the worst participants displayed discrimination between directions. Moreover,
Gould et al.[1979] states a skin displacement threshold of 0.7 mm for discrimination between opposing
directions at the forearm, that is, a threshold below the present displacement at a less innervate body
site. One may also ask whether the lateral stimulation did stretch the skin of all participants. Indeed
a minority of participants (5 out 14) did not report to have experienced stretch. However, whereas
displacement thresholds for pure translations are considerably larger than for pure stretch and, in
general, exceed the displacements used here [Essick, 1992], the (median) discrimination threshold for
participants reporting no feeling of stretch (26◦ ) did not exceed that of the participants who clearly
experienced stretch (26◦ ). Thus, it is likely that stretch played a role for all participants. Still it is an
open question whether and in how far also slip occurred.
In general, we observed larger thresholds for direction discrimination than Keyson and Houtsma
[1995]. But this difference was to be expected for various reasons: In Keyson and Houtsma’s study,
70.7% correct responses defined the discrimination threshold, whereas we used a less liberal 84%
criterion. Moreover, their study minimized stretch, while ours maximized stretch. Moreover, in their
study the movements were considerably longer and faster than in the present experiment (6.5 versus
1 mm, and 41 versus 10 mm/s, respectively). Tactile movement detection is known to improve both
with increasing path length and—up to a point—with increasing velocity [Whitsel et al. 1979]. It is
reasonable that accuracy in the perception of movement direction as well improves with path length
and velocity.
Most importantly here, the magnitude of the present discrimination thresholds demonstrated that
almost all participants were able to discriminate between the stimuli displayed by our device. It is
also important to note that the technical directional resolution of the device (about 0.6◦ at 1 mm
displacement) clearly exceeds the perceptual thresholds we observed (individual median >20◦ ).
Further development of the device may take advantage of the knowledge obtained on the range and
the optimum of human directional resolution. Of rather theoretical than technological interest are the
reliable, but small differences between stroke directions.
5.
EXPERIMENT II: PERCEPTUAL INTEGRATION OF MULTIPIN PATTERNS
Experiment I demonstrated that the device is appropriate for the perception of different directions
of movement signals on the human finger. In a second step, Experiment II studied the integration
of perceived displacement direction displayed by multiple pins. We applied a thoroughly tested and
well-established model on human signal integration, the maximum-likelihood-estimate (MLE) model
[Ernst and Banks 2002; Ernst and Bülthoff 2004]. According to the MLE model, the human brain
integrates redundant signals from the same physical property such that the integrated signal is less
noisy and thus more reliable than each individual signal. This is because of a noise reduction due to
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
10
•
K. Drewing et al.
Fig. 8. The left part of the figure depicts the two standard directions for pin movements in Experiment II (orange arrows)
and their length (1 mm). The right part depicts the spatial arrangement of pins in the different pin number conditions (3 mm
center-to-center distance).
averaging of the signals. In effect we therefore hypothesize that an increase in the number of pins
increases the signal to noise ratio. However, an increase of the signal with such an averaging is only
beneficial and leads to a better signal to noise ratio as long as the errors (distribution of noise) in the
individual signals are at least partly independent [Oruc et al. 2003]. Errors in the signal can stem from
physical as well as neural sources.
More reliable (i.e., less noisy) signals, of course, can be better discriminated than less reliable signals.
In the present experiment, we tested whether discrimination performance improves from one to two
and four pins (all moving in the same direction) due to the integration of information across the pins. We
predict that the JNDs for direction should profit from multipin as compared to single-pin displacement—
if more pins evoke more direction-relevant neural response than one pin, and if the neural signals are
at least partially independently processed (i.e., a correlation between the noise distribution of the
signals less than 1). We measured 84% discrimination thresholds (JNDs) for movements in up and
down-direction—using, again, a two-interval forced choice task and the method of constant stimuli.
5.1
Method
5.1.1 Participants. Twelve right-handed participants (six female, six male) took part for pay. Their
age ranged from 19 to 40 years (average 27 years). None of them reported any known tactile deficit due
to accident or illness concerning the left index finger. Eight of the participants were familiar with the
device from Experiment I. With the exception of two staff members of the laboratory, participants were
naive to the purpose of the experiment.
5.1.2 Apparatus and Stimuli. Apparatus and stimuli were similar to Experiment I. However, we
provided the tactile stimuli with one, two, or four pins, in parallel (other pins removed). That is, in
multipin conditions all pins moved simultaneously and with identical velocity and direction; the pins
were separated by 3 mm center-to-center distance (Figure 8, left).
5.1.3 Design and Procedure. The design comprised two within-participant variables: stroke direction realized by two standard stimuli (0◦ (up) versus 180◦ (down)), and pin number (1, 2, versus 4 pins).
For each condition, we measured the 84% discrimination threshold for movement direction—using the
identical procedure as in Experiment I.
Thus, the experiment consisted of 1368 trials (= 2 directions × 3 pin numbers × 19 comparisons × 12
repetitions). Trials with identical pin number were blocked in one of three 1-h sessions (on different days
within one week); otherwise trials were randomized. The six possible orders of blocks were randomly
assigned to the six female as well as to the six male participants.
5.1.4 Data Analysis. Data were analyzed analog to Experiment I.
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
Evaluation of Lateral Displacement Display
•
11
Fig. 9. Medians of individual 84% thresholds for direction discrimination by pin number and stroke direction; error bars indicate
25% and 75% percentiles, respectively.
5.2
Results and Discussion
Analyzed by stroke direction × pin number conditions, median direction-discrimination thresholds
(JNDs, see Figure 9) in this experiment were rather uniform, ranging from 19◦ to 23◦ . Threshold
quartiles ranged from a maximal 75% percentile of 33◦ down to a minimal 25% percentile of 16◦ .
The individual thresholds per condition entered a Friedman test. The test clearly failed to reach
significance, χ 2 (5) = 6.7, p > .20, demonstrating that participants’ discrimination performance did
not differ between any combinations of movement direction and pin number. This means we were not
able to replicate the advantage of up-directions observed in Experiment I. Further, it means that we
did not find an effect of pin number on discrimination performance.
Estimates of individual discrimination thresholds per condition ranged from 8◦ to 57◦ in the present
experiment. Individual median thresholds across conditions ranged between 14◦ and 34◦ . A Friedmantest between individuals, again, reached significance, χ 2 (11) = 38.0, p < .001. However, as compared to
Experiment I, individual differences were less pronounced and thresholds tended to be lower. One reason
for both observations may be that most participants were already familiar with the device and individual
difficulties reduced with use. For the lower thresholds also, memory effects may have played a role: There
were substantially less different standard stroke directions in the present experiment as compared
to Experiment I (2 as compared to 8). Discrimination performance in the present experiment, thus,
may have benefited from a more stable memory representation of the standard strokes. However, the
magnitude of discrimination thresholds in the present experiment confirmed that humans are able to
discriminate between the directions of the movements displayed and that the perceptual resolution
(individual median >14◦ ) did not exceed the technical resolution of the display.
Importantly, we did not find an effect of pin number on human discrimination performance. The MLE
model on human signal integration [Ernst and Bülthoff 2004] states that the human brain integrates—
at least partially—independent redundant signals on the same physical property such that the integrated signal is less noisy. Applying the model, the lack of an integration benefit in the present study,
then, indicates that the displacements of the different pins of the device did not evoke more relevant
neural response than the displacement of a single pin or that these were not independently processed
in the perceptual system. In other words, these constant discrimination thresholds mean that increasing
the number of pins did not increase the signal to noise ratio.
In terms of receptor density, it seemed unlikely that four pins did not evoke more response than one
pin. The 3 mm center-to-center distance of the pins clearly exceeds the spacing of the FA I and SA I
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
12
•
K. Drewing et al.
receptors (0.8 mm or below) and fits with the spacing of SA II receptors (3.2 mm or below) [Essick, 1998;
Johnson 2002; Johnson et al. 2000; Nolano et al. 2003]. However, possibly discrimination by stretch did
not benefit from the additional pins because a single pin was sufficient to stretch a maximal skin area
and, in addition, translation cues did not play a role for discrimination, because slip across the skin
did not occur or slip paths were too short. Alternatively, the brain may rather use a complex integrated
direction code instead of averaging single direction-specific signals.
However, the lack of integration benefit points in a clear direction for future research: In order to
achieve stronger and more reliable signals by the different pins, it may help if future development of
the device will include mechanisms that also allow for an increased pin distance. In contrast, it is an
interesting question as to what degree non-simultaneous, differential multipin patterns of displacement
provided by the current prototype are able to evoke discriminable percepts.
6.
CONCLUSION
In this paper, we presented the construction and evaluation of a novel multipin display that controls
lateral pin displacement and, thus produces shear force. The design profited from basic knowledge
on human psychophysics. But because the technology has not previously been available, there are
unanswered questions on the perceptual side. In two experiments we evaluated whether the display
produces tactile stimuli that are appropriate for human sensitivity in tactile movement perception.
Both experiments demonstrated that humans can discriminate well between directions of the
lateral stimuli displayed, and also that their perceptual resolution does not exceed the already realized technical resolution of the device. In this sense, we demonstrated that the device is able to produce
tactile movement signals that are appropriate for human perception, and that shear forces can be used to
mediate a differentiated impression of at least one environmental aspect. Future development of the
device may take advantage of the obtained magnitude for the human direction threshold (JND), which
for no individual in any experiment was better than 14◦ (median across directions). In Experiment II,
we further found evidence that the human brain does not process stimulation from the different pins
of the display independent of one another, at least concerning movement direction, so that estimates
were no more reliable whether one, two, or four pins were used for stimulation. The constant direction
thresholds indicate that the signal to noise ratio was independent of the number of pins used. From
this evidence, we concluded that—in order to obtain independent signals from the different pins and
therefore a better and more reliable direction estimate—it may help to increase the interpin distance.
An increased pin distance provides also the option to implement an increased pin excursion. Beyond
evaluation, we made observations that might be interesting from a psychophysical point of view and can
be followed up using the device. Most importantly, in Experiment I, we observed a perceptual anisotropy
for direction discrimination, namely an advantage for the perception of displacement in up-directions,
which thorough investigations may relate to characteristics of receptor populations.
Taken together, the approach of coupling display design tightly with psychophysics proved successful
from a theoretical as well as a technological point of view. After further development and evaluation
of the display, it will be combined with a high-force kinesthetic device [Ueberle et al. 2004]. In this
combination, lateral stimulation might markedly improve the realism of active haptic exploration and
as well provide tactile cues that are important for dexterous teleoperation.
ACKNOWLEDGMENTS
We wish to thank Christoph Lange for his assistance in computer programming.
REFERENCES
BIRZNIEKS, I., JENMALM, P., GOODWIN, A. W., AND JOHANSSON, R. S.
afferents. J. Neurosci. 21, 8222–8237.
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
2001. Encoding of direction of fingertip forces by human tactile
Evaluation of Lateral Displacement Display
•
13
EDIN, B. B. 1992. Quantitative analysis of static strain sensitivity in human mechanoreceptors from hairy skin. J. Neurophysiol.
67, 1105–1113.
ERNST, M. O. AND BANKS, M. S. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature
415, 429–433.
ERNST, M. O. AND BÜLTHOFF, H. H. 2004. Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162–169.
ESSICK, G. K. 1998. Factors affecting direction discrimination of moving tactile stimuli. In Neural Aspects of tactile sensation,
J. W. Morley, Ed. Elsevier, Amsterdam, 1–54.
FLANGAN, J. R., BURSTEDT, M. K. O., AND JOHANSSON, R. S. 1999. Control of fingertip forces in multi-digit manipulation. J.
Neurophysiol. 81, 1706–1717.
GARDNER, E. P. AND SKLAR, B. F. 1994. Discrimination of the direction of motion on the human hand: A psychophysical study of
stimulation parameters. J. Neurophysiol. 71, 2414–2429.
GARDNER, E. P. AND SKLAR, B. F. 1996. Factors influencing discrimination of direction of motion on the human hand. Soc.
Neurosci. Abstr. 12, 798.
GOULD, W. R., VIERCK, C. J., AND LUCK, M. M. 1979. Cues supporting recognition of the orientation or direction of movement
of tactile stimuli. In Sensory Function of the Skin of Humans. Proceedings of the 2nd International Symposium on the Skin
Senses, D. R. Kenshalo, Ed. Plenum, New York, 63–73.
GREENSPAN, J. D. AND BOLANOWSKI, S. J. 1996. The psychophysics of tactile perception and its peripheral physiological basis. In
Pain and Touch, L. Kruger, Ed. Academic, San Diego, CA, 25–103.
HAYWARD, V. AND CRUZ-HERNANDEZ, J. M. 2000. Tactile display device using distributed lateral skin stretch. In Proceedings of
the Haptic Interfaces for Virtual Environment and Teleoperator Systems Symposium, ASME IMECE2000, Orlando, FL., Nov.
2000, Proc. ASME vol. DSC-69-2, 1309–1314.
IKEI, Y. 2002. TextureExplorer: A tactile and force display for virtual textures. In Proceedings of the 10th International
HAPTICS’02 Symposium, Orlando, FL., Mar. 2002, IEEE, Piscataway, NJ, 327.
IKEI, Y., WAKAMATSU, K., AND FUKUDA, S. 1997. Texture presentation by vibratory tactile display-image based presentation of
a tactile texture. In Virtual Reality Annual International Symposium, Albuquerque, NM, March 1997, IEEE, Piscataway, NJ,
199–205.
JOHNSON, K. O. 2002. Neural basis of haptic perception. In Stevens Handbook of Experimental Psychology, vol. 1: Sensation
and Perception, H. Pashler and S. Yantis, Eds. Wiley, New York, 537–583.
JOHNSON, K. O., YOSHIOKA, T., AND VEGA-BERMUDEZ, F., 2000. Tactile functions of mechanoreceptive afferents innervating the
hand. J. Clin. Neurophysiol. 17, 539–558.
JOHANSSON, R. S. AND VALLBO, A. B. 1979a. Detection of tactile stimuli. Thresholds of afferent units related to psychophysical
thresholds in the human hand. J. Physiol. 297, 405–422.
JOHANSSON, R. S. AND VALLBO, A. B. 1979b. Tactile sensibility in the human hand: Relative and absolute densities of four types
of mechanoreceptive units in glabrous skin. J. Physiol. 286, 283–300.
JOHANSSON, R. S. AND WESTLING, G. 1990. Tactile afferent signals in control of precision grip. In Attention and Performance XIII,
M. Jeannerod, Ed. Erlbaum, Mahwah, NJ, 677–713.
KEYSON, D. V. AND HOUTSMA, A. J. M. 1995. Directional sensitivity to a tactile point stimulus moving across the fingerpad.
Percept. Psychophys. 57, 738–744.
LAMOTTE, R. H. AND SRINIVASAN, M. A. 1991. Surface microgeometry: Tactile perception and neural encoding. In Information
Processing in the Somatosensory System, O. Franzen and J. Westman, Eds. Macmillan, London, 49–58.
LEE, J.-H., AHN, I.-S., AND PARK, J.-O. 1999. Design and implementation of tactile feedback device using electromagnetic type.
In Intelligent Robots and Systems IROS ’99, Korea, Oct. 1999, vol. 3, 1549–1554.
LOOMIS, J. M. AND COLLINS, C. C. 1978. Sensitivity to shifts of a point stimulus: An instance of tactile hyperacuity. Percept.
Psychophys. 24, 487–492.
NOLANO, M., PROVITERA, V., CRISCI, C., STANCANELLI, A., WENDELSCHAFER-CRABB, G., KENNEDY, W. R., AND SANTORO, L. 2003. Quantification of myelinated nerve endings and mechanoreceptors in human digital skin. Ann. Neurol. 54, 197–205.
MOUNTCASTLE, V. B., LAMOTTE, R. H., AND CARLI, G. 1972. Detection thresholds for stimuli in humans and monkeys: comparison
with threshold events in mechanoreceptive afferent nerve fibers innervating the monkey hand. J. Neurophysiol. 35, 122–136.
OLAUSSON, H. 1994. The influence of spatial summation on human tactile directional sensibility. Somatosens. Mot. Res. 11,
305–310.
ORUC, I., MALONEY, T. M., AND LANDY, M. S. 2003. Weighted linear cue combination with possibly correlated error. Vision Res.
43, 2451–2468.
SALADA, M., COLGATE, J., LEE, M., AND VISHTON, P. 2002. Validating a novel approach to rendering fingertip contact sensations.
In Proceedings of the 10th International HAPTICS’02 Symposium, Orlando, FL., Mar. 2002, IEEE, Piscataway, NJ, 217–224.
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.
14
•
K. Drewing et al.
SALADA, M., COLGATE, J. E., VISHTON, P., AND FRANKEL, E. 2004. Two experiments on the perception of slip at the fingertip. In
Proceedings of the 12th International HAPTICS’04 Symposium, Chicago, IL., Mar. 2004, IEEE, Piscataway, NJ, 146–153.
SHINOHARA, M., SHIMIZU, Y., AND MOCHIZUKI, A. 1998. Three-dimensional tactile display for the blind. IEEE Trans. Rehabil. Eng.
6, 249–256.
SRINIVASAN, M. A., WHITEHOUSE, J. M., AND LAMOTTE, R. H. 1990. Tactile detection of slip: Surface microgeometry and peripheral
neural codes. J. Neurophysiol. 63, 1323–1332.
SUMMERS, I. R. AND CHANTER, C. M., 2002. A broadband tactile array on the fingertip. J. Acoust. Soc. Am. 112, 2118–2126.
SUMMERS, I. R., CHANTER, C. M., SOUTHALL, A., AND BRADY, A. 2001. Results from a tactile array on the fingertip, In Eurohaptics
2001: Conference Proceedings, Birmingham, UK, July 2001, C. Baber, M. Faint, S. Wall, and A. M. Wing, Eds., University of
Birmingham, 26–28.
UEBERLE, M., MOCK, N., AND BUSS, M. 2004. ViSHaRD10, A novel hyper-redundant haptic interface. In Proceedings of the 12th
International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2004), Chicago,
IL, USA, March 2004, IEEE Computer Society, Press, Los Alamitos, CA, 58–65.
WHITSEL, B. L., DREYER, D. A., HOLLINS, M., AND YOUNG, M. G. 1979. The coding of direction of tactile stimulus movement:
Correlative psychophysical and electrophysiological data. In Sensory Functions of the Skin of Humans, J. R. Kenshalo, Ed.
Plenum, New York, 79–107.
WICHMANN, F. A. AND HILL, N. J. 2001a. The psychometric function: I. Fitting, sampling, and goodness of fit. Percept. Psychophys.
63, 1293–1313.
WICHMANN, F. A. AND HILL, N. J. 2001b. The psychometric function: II. Bootstrap-based confidence intervals and sampling.
Percept. Psychophys. 63, 1314–1329.
Received October 2004; revised December 2004; accepted January 2005
ACM Transactions on Applied Perceptions, Vol. 2, No. 2, April 2005.