supporting mobile microinteractions

Transcription

SUPPORTING MOBILE MICROINTERACTIONS
A Thesis Proposal
Presented to
The Academic Faculty
by
Daniel L. Ashbrook
In Partial Fulfillment
of the Requirements for the Degree
Doctor of Philosophy in the
College of Computing
Georgia Institute of Technology
December 2007
SUPPORTING MOBILE MICROINTERACTIONS
Approved by:
Thad E. Starner, Advisor
Blair MacIntyre
Gregory D. Abowd
James A. Landay
Computer Science & Engineering
University of Washington
Charles L. Isbell, Jr.
Date Approved:
TABLE OF CONTENTS
LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
v
LIST OF FIGURES
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vi
INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
1.1
Thesis Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
1.2
Two Interaction Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.2.1
Touchscreen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.2.2
Gestural . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.3
Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
1.4
Proposal Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
RELATED WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
2.1
Mobility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
2.2
Touchscreen Watch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
2.3
Gesture Watch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
MICROINTERACTIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
3.1
Calendar Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
3.2
Quickdraw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
3.2.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
3.2.2
Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
3.2.3
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
3.2.4
Quickdraw Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
TOUCHWATCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
4.1
Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
4.1.1
Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
4.1.2
Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
4.1.3
Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
4.1.4
Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
4.1.5
Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
I
II
III
3.3
IV
iii
4.1.6
Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
4.1.7
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24
Rim Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
4.2.1
Card Dragging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
4.2.2
Calendar Dialing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26
Proposed Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
WRIST-BASED GESTURE INTERACTION . . . . . . . . . . . . . . . . . . . . . .
29
5.1
Gesture Triggering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
29
5.2
Proposed work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
5.2.1
Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
5.2.2
Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
30
5.2.3
Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
PLAN FOR COMPLETION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
32
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33
4.2
4.3
V
VI
iv
LIST OF TABLES
1
In seconds, average time for events during scheduling appointments, per commonly
used device. Navigation time for scrap paper is effectively zero. . . . . . . . . . .
11
Claimed (columns) versus actual (rows) usage of devices for scheduling task. The
rows are sorted by retrieval + navigation time, from Table 1, shown in parentheses.
12
Devices carried by participants and their location. “Body” refers to on-body placement of a device, such as clipping to clothing or using a wrist strap. “Bag” includes
purses and backpacks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
4
Statistics for mean (SD) hand, pocket and access times in seconds. . . . . . . . . .
17
5
Mean (SD) effective width We in pixels for each movement condition, parallel (k)
and perpendicular (⊥) to the direction of movement. . . . . . . . . . . . . . . . .
23
2
3
v
LIST OF FIGURES
1
The Tissot T-Touch (a) and Silen-T (b) touch-sensitive watches. The Silen-T’s touch
mechanism is activated by touching the crown (next to the E on the bezel) for longer
than one second; the areas on the crystal labeled THERMO, METEO, ALTIMETER ,
CHRONO, COMPASS , and ALARM then become sensitive, activating various functionality when touched. The Silen-T also features a touch-sensitive crystal, which is
activated by pressing the crown. The watch then vibrates when the user’s finger is
run clockwise around the face, with a long vibration when the finger is at the current
hour position and short vibrations at the minute position. . . . . . . . . . . . . .
7
The IBM WatchPad (a) and Blaskó’s interface for a variation with nine touchsensitive areas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
The Fossil/Abacus WristPDA. (a) shows the application menu, with icons large
enough to push with a fingertip; (b) shows the calculator application with tiny buttons; (c) illustrates the scale of the included stylus that is normally tucked into
the watch band. It is likely that the stylus or another pointed instrument would be
needed to effectively use the calculator application. . . . . . . . . . . . . . . . . .
9
Interfaces using a center/rim style of interaction. (a) is an example of a pie menu
[5], (d) illustrates Cirrin [17], (b) is a conceptual visualization of the earPod menu
system [38], and (c) is a pie menu-like selection system designed for see-through
head-mounted displays (the central black area will appear to be transparent to the
wearer). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
Three on-body command-gesture devices. Device (a) is the Gesture Pendant [1], (b)
is an example of FreeDigiter [20] in use, and (c) illustrates the Gesture Watch [12].
10
The path participants walked, starting at flag 1 and proceeding either clockwise or
counterclockwise. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
7
Placement: pocket (a), hip (b), and wrist (c).
. . . . . . . . . . . . . . . . . . . .
14
8
Screen (a) is shown when an alert occurs. The participant must mentally note the
displayed number and slide the blue box from the left to the target box on the right.
Screen (b) is shown after the slide is completed; from the displayed four numbers,
the participant must touch the number that was displayed on screen (a). . . . . . .
15
Timeline of events and status of brightness detector during one notification-response
cycle. The top line is the percentage of light detected from the camera (complete
darkness, 0% on the bottom), and the bottom is the timeline of events recorded by
the phone’s logging software. See the text for further details. . . . . . . . . . . . .
16
2
3
4
5
6
9
vi
10
Images (a), (b) and (c) show the targets (short green lines at rim of circle) and
movement guides (yellow line between targets) displayed to study participants. Images (d), (e) and (f) show actual movements for one session; blue lines indicate
movement by participant’s finger and small yellow circles show inflection points.
Movement images were displayed only to researcher. Images (a) and (d) are for tap
condition; images (b) and (e) are for through condition; images (c) and (f) are for
rim condition. Images (a)–(f) are printed at actual size. . . . . . . . . . . . . . .
21
11
The simulated touchscreen watch. . . . . . . . . . . . . . . . . . . . . . . . . . .
22
12
Effective index of difficulty (IDe ) versus movement time (MT ) for the three movement conditions. The regression line is dotted in each graph. Regression statistics
are: tap slope: 40ms/bit, intercept: 228ms, R2 : 0.04); through slope: 54ms/bit,
intercept: 106ms, R2 : 0.09); rim slope: 158ms/bit, intercept: −18ms, R2 : 0.45). .
24
13
Possible watch face layouts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
14
Blaskó’s card dragging technique [9]. The default watch face is illustrated in (a),
with the asterisk denoting the desired card. The motion made by the user’s finger is
illustrated in (b), and the resulting screen with the activated content is shown in (c).
25
Proposed “card dragging” interaction using icons on rim of watch. (a): view of
watch with no applications active; (b): user touches face to show application icons;
(c): user selects desired application icon; (d)–(f): user drags application “card”
out of “stack”. Images are shown at actual size for hardware used during Fitts’
study and conform to design recommendations discovered therein. . . . . . . . . .
26
Mockup of a possible dial-metaphor interface for calendar navigation. Dark gray
segments in (a)–(c) show “now”, thick red borders indicate area selected, and bands
around rim in (d) indicate appointments. User touches red-banded area and drags
finger clockwise to zoom in to next closer detail view; to zoom out, user drags finger
counterclockwise. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
Candidate gesture performed by user (a) and partial gesture database (b) with
match percentage (bottom) and database contents (top) illustrated. . . . . . . . .
30
Prototype watch hardware. Features on the side of the board illustrated in (a), from
the top left and in clockwise direction are the Bluetooth module, the power/charge
switch, battery wires, reset and action buttons, display connector, and mini-USB
connector (for charging and programming). (b) shows (in the same order), the
microSD card slot, unpopulated pads for the accelerometer chip, and the processor.
Also visible are programmable LEDs and Bluetooth status LEDs. . . . . . . . . .
31
15
16
17
18
vii
ABSTRACT
Many interactions with mobile devices could be more like checking the time on a wristwatch—short
and almost without conscious thought. I term activities like these microinteractions.
For my dissertation work, I propose to investigate and design interfaces to facilitate microinteractions. I will do this through developing interfaces for small, on-body devices. Many interactions
that might otherwise be micro- are stymied by too-long “setup” times—the amount of time it takes
to retrieve the device and navigate to the appropriate application—that must take place before the
desired interaction can be accomplished. My proposed work will shorten or remove altogether the
setup time
The lack of support for microinteractions in current technology leads to a failure to use the
provided functionality, to the detriment of the user. To better enable microinteractions, I propose
to develop two new techniques for wrist-based interactions. I choose the wrist because it is a socially acceptable platform for wearable devices, and because it enables very fast access. The two
interactions are touchscreen-based and gesture-based.
One issue with mobile interactions is how to avoid false activations, when the device is unintentionally activated by some natural action of the user. Many current devices attempt to solve the
problem by having “push-to-activate” mechanisms; however, this practice results in added complexity and, in some cases, the loss of single-handed functionality. The methods that I explore in this
proposal remove the need for push-to-activate functionality, allowing the user to initiate microinteractions with no prelude.
For the touchscreen-based interactions, I propose to use a round-faced touchscreen watch. Historically, most clocks and watches have been circular, while all commercially available touchscreen
watches thus far have been rectangular. I present results of a study on the best method for placing
buttons around the rim of a round touchscreen watch, and I propose two related interaction techniques that simultaneously encourage users to utilize the most efficient method of interaction and
prevent undesired interaction due to accidental touches of the screen.
Mobile gesture-based systems in particular suffer from the issue of unintentional activation:
the normal movements of the user can be confused for intentional gestures. This problem is most
often solved with push-to-activate; however, in the case of wrist-based gesture systems, pushing
requires the use of the non-wristwatch hand, meaning that the gesture functionality could easily be
replaced by buttons. I propose a solution to the unintentional activation problem that, rather than
involving pushing-to-activate, uses an “Everyday Gesture Library” to help interaction designers
choose gestures that are unlikely to be confused with normal gestures made in the course of daily
life.
1
CHAPTER I
INTRODUCTION
I propose to investigate the space of microinteractions with small, on-body devices. The category of
small, on-body devices is reasonably self-descriptive. I define microinteractions as single-purpose
interactions with a device that take a very short amount of time, such as reading a text message,
dialing a phone number, or checking upcoming appointments.
Often, would-be microinteractions are surrounded by “setup” and “teardown” interactions that
serve only to prepare the device for the microinteraction. An example of this is reading a text
message, or SMS, on a mobile phone. When the SMS is received, the user is alerted via a sound
or vibration. To read the SMS, the user has to remove the phone from its storage place (a bag or
pocket, for example), open or unlock the phone, and navigate through one or more menus to get
to the message. After reading the message, a similar sequence must be followed by closing the
application, re-locking or closing the phone, and returning it to the original storage location.
One way of better enabling such interactions to be more “micro” is by reducing the setup and
teardown times surrounding the interactions. Of these two times, the setup time is more important,
as it represents the delay between the desire by the user to perform some action, and the actual time
at which the user is able to start the action. The teardown time, on the other hand, usually only
delays the user in returning to the “natural state” that he was in before the interaction started.
I propose the use of on-body devices to reduce both setup and teardown times. As Chapter 3
reveals, removing the need to extract a device from a pocket or other storage area greatly decreases
the setup time for an interaction. Making a device small and placing it in an easily accessible
location on the body enables the user to quickly access the device, potentially eliminating the need
to perform any teardown actions at all.
As an example of an on-body device with which one might have microinteractions, I specifically
propose to investigate the wristwatch. Watches are socially accepted as wearable technology, and
judging by the consumer market for “gadget” watches, there is an appetite for watches that support
microinteractions. For example, Sony recently released a watch/phone combo that allows the audio
player functionality of the phone to be controlled by the watch.
Both input and output are important when considering the space of interaction with small, onbody devices such as wristwatches. In this proposal, I focus on the problem of input. I approach
this problem with two input techniques: touchscreen-based and gesture-based.
1.1 Thesis Statement
My hypothesis is that mobile microinteractions can be supported with two techniques:
• Rim-based interactions on a round touchscreen wristwatch will allow slower but complex
interaction while avoiding false positives; and
• Push-to-activate mechanisms can be avoided for fast wrist-based gestural interactions by creating and using an Everyday Gesture Library to help in the selection of appropriate gestures.
2
1.2 Two Interaction Techniques
Although I am concentrating on the wristwatch as an exemplar of small on-body devices, there is
the question of how to interact with the watch. Consider these potential applications: checking the
time, skipping to the next track on a digital audio player, sending a call to voicemail, reading a text
message, checking the arrival time of the next bus, taking down a phone number, getting an idea
of appointment times for the day, dialing a phone number from an address book, calculating a tip,
checking the weather, updating status information in a social notification service such as “Twitter”,
and setting or responding to a reminder alarm.
There are two categories these applications can be divided into:
1. User initiating: The user wants to perform an action, and can decide when to do it. Examples include dialing a phone number, updating social notification services, or reading a text
message.
2. User responding: The user responds to an externally-imposed event such as an incoming call
or a reminder alarm going off. The event is time-sensitive, so the user has little latitude in
deciding when to respond.
To complement each of these interaction styles, I propose to investigate two interaction techniques:
1.2.1
Touchscreen
For user-initiated tasks such as inputting a phone number, it makes sense to assume that the user can
choose somewhat when and where to initiate that task; for example, by stepping out of the flow of
people on the sidewalk to briefly interact. In this case, the ability to have more complex input and
output are more important than just speed of access. Touch is one well-understood way to interact
with a small device; however, many touchscreen-based watches require the use of a tiny stylus due
to the small screen size.
In Chapter 4 I discuss the problem of using the finger to interact with a tiny touchscreen, present
results on the accuracy of several types of fingertip-based interaction on a round touchscreen watch,
and propose an interaction technique that allows the avoidance of accidental activation by a sleeve
or brush of the hand.
1.2.2
Gestural
When an event occurs to which a user must respond quickly, a gesture is a faster option than having
to manually manipulate a device. To send a call to voicemail, silence an alarm, or to skip to the next
audio track, the user can use a quick and simple “twitch” motion. Gesture recognition can augment
or replace context sensitivity in some cases: rather than the phone deciding not to ring based on
context, it can first perform a short vibration at the wrist and allow the user to silence it very quickly
and unobtrusively with a short gesture, allowing the user to continue the current activity without
interruption.
One problem with this idea is which gestures to use. To avoid having to use the other hand to
push a button for “triggering,” it is desirable for the system to perform continuous recognition. Doing so, however, raises the possibility of the system mistaking a normal gesture made in conversation
for a command to the system. I propose a solution to this problem in Chapter 5.
1.3 Contributions
The contributions of the proposed work are several. Supporting mobile microinteractions by reducing access time and avoiding push-to-activate will affect everyday interactions with mobile devices.
By allowing more efficient usage, users will be more likely to use the functionality that the device
offers, and speeding up access time for the many daily uses that devices are put to will have a large
overall effect.
My proposed work has several general contributions:
1. Decreasing access time by removing the “push to activate” mechanism present in many systems [1, 19, 28]. Doing this not only removes a time-consuming motion, but can reduce the
overall complexity of interaction by allowing one-handed use. Chapter 4.2 discusses this
with respect to round touchscreen watches, and while Chapter 5.1 concentrates on wristbased gesture interfaces, the techniques discussed are applicable to a wide range of gesture
interfaces for everyday use.
2. The introduction of new interaction techniques for the wrist. The wristwatch is a piece of
technology that is well-understood by the population at large, and that is starting to garner
interest as a platform for interaction. Most extant methods for interacting with the wristwatch
are, however, button-based, often leading to unnecessary complexity. In Chapters 4.2 and
5.1, I propose two new methods for interacting with a wristwatch that could lead to simpler
or more efficient interactions with multifunctional wristwatches.
3. A study quantifying the access time properties of mobile devices stored in the pocket, in a belt
holster, and mounted on the wrist. Although one would expect that a watch is faster to access
than a pocket or holster, I quantify how much faster it is and break down the access time into
several measurable steps. This study, presented in Chapter 3.2, also indicates that people
are generally capable of simple interactions with a touchscreen while performing pedestrian
navigation.
My next set of contributions relates to touchscreen wristwatches with round faces:
4. A study determining the optimal on-screen button size for three interaction types on a round
touchscreen watch. As discussed later, a round display suggests an interaction region around
the edge and a separate center area. Given the edge-based interaction area, what type of
touch interaction should be done, and how many buttons may be accommodated? This study,
presented in Chapter 4.1, compares three types of touches—tapping, sliding in a straight
line, and sliding along the rim—and determines the optimal button size for each.
5. A low-false-positive interaction technique. To remove the need for a “push-to-touch” mechanism while avoiding false positives, there must be a way to distinguish intentional touches
of the screen from accidental impacts, such as by a sleeve or accidental brush of the hand. In
Chapter 4.2 I introduce an interaction technique that avoids false detection of user actions by
using a dial metaphor.
6. Experimental validation of the study results. In Chapter 4.3, I propose a study to experimentally validate the results of the button-size study. The results of the first study were found
using a Fitts-style reciprocal pointing task with very small targets; thus, I plan to re-run the
study to determine user performance with actual buttons of sizes that were found.
The final set of contributions is related to gesture-based interactions:
7. A method for interaction designers to design low-false-positive gestures. To allow users to
initiate actions using gestures while avoiding a push-to-gesture mechanism, in Chapter 5.1 I
propose a system to allow interaction designers to test potential gestures against a database of
everyday movements made by typical users. This system will allow designers to pick gestures
that have a low probability of being falsely recognized.
8. Prototype gesture-sensitive watch hardware. Experimentation with wearable sensing is often
difficult, requiring the carrying of extra hardware such as large batteries, palmtop computers
and various cables. I will publish schematics and software for the gesture-sensing watch
platform described in Chapter 5.2.1 to enable future progress by other researchers.
1.4 Proposal Overview
First, in Chapter 2 I outline some related work in mobile interaction, small on-body devices, and
research relevant to the proposed interaction techniques. Chapter 3 presents and discusses some
quantitative data that help to explain microinteractions. In Chapter 4 I discuss my progress so far
with touchscreen wristwatch interactions and propose further work, and in Chapter 5 I do the same
for gestural interaction. Finally, Chapter 6 presents my plan for completion of my dissertation.
CHAPTER II
RELATED WORK
This chapter presents previous research in areas related to this proposal. Some of the cited work is
only generally related, as little work has been done in some of the specific areas that I am proposing
to investigate.
2.1 Mobility
The use of mobile technology while on the go is an area of research that has been getting more attention recently. This is unsurprising: as electronics get smaller and more power-efficient, they lend
themselves to easily being carried. At the same time, the increasing prevalence of wireless services
means that communication has become supported in places that were previously only “transition
zones” such as sidewalks, subways and highways. These developments mean that people now have
the capability to interact and communicate in places where they previously could only walk, ride or
drive. For better or for worse, on-the-go device usage is an activity that will need to be taken into
account when designing interaction.
Kristoffersen and Ljungberg noted a variety of problems with portable devices that combined to
prevent activity from simply “taking place”—instead, users of the technology had to “make place”
to use the device [13]. The biggest problems they found appeared to be related to the form-factor
of the devices used, which were PDAs: users needed two hands to use the device, but often only
one or no hands were free; and frequently they needed to put down the computer to use it—either
to have a surface to type or write on, or to refer back to the display—but no surface was available
for this purpose. These results suggest that, for some purposes, a wrist-based platform may be more
practical.
In contrast to the physical limitations of devices, the Resource Competition Framework proposed by Oulasvirta et al. [23] provides a way to think about the cognitive demands placed on the
user of a mobile device while in various situations. The authors performed an experiment, asking
users to perform various mobile Internet tasks in a variety of situations. They found that visual
attention paid to the environment versus the device varied from 51% (on a busy street), to 22% (in
a café), to 14% (on the metro), implying that interfaces allowing eyes-free interaction may be more
usable in situations involving high competition for the user’s attention.
Patel et al. examined people’s perceptions about how often they have their mobile phone nearby.
Their data show that people routinely overestimate the physical availability of their mobile phones
[24]. Even when the mobile phone is with its user, it may not be quickly accessible; Cui et al. found
that 40% of women and 30% of men miss phone calls simply due to the manner in which they
carry the mobile phone on their person [8]. Similarly, Starner et al. found correlations between an
individual’s decision to use or not use a mobile scheduling device (such as a day planner or PDA)
and the amount of time and effort required to access and make ready the device [34]. Together, these
studies suggest that the time required to access a device may be an important property affecting
mobile use.
6
2.2 Touchscreen Watch
Despite the increasing functionality in wrist-worn technology, there has not been much academic
work on wristwatch interfaces, and I have been unable to find any work at all on round touchscreen
watches, either commercially or in the literature. Two commercial examples of round-faced watches
with touch capability—though not with round touchscreens—are the Tissot T-Touch and Silen-T
wristwatches [28], shown in Figure 1. Both watches feature a “push-to-touch” mechanism, designed
to avoid accidentally activating functionality. The T-Touch is activated by holding a finger to the
touch-sensitive crown for longer than one second, and the Silen-T is activated by pressing in the
crown (which is a button). This push-to-activate mechanism is a recurring feature of mobile systems
in much of the literature.
(a)
(b)
Figure 1: The Tissot T-Touch (a) and Silen-T (b) touch-sensitive watches. The Silen-T’s touch mechanism is activated by touching the crown (next to the E on the bezel) for longer than one second;
the areas on the crystal labeled THERMO, METEO, ALTIMETER , CHRONO, COMPASS , and ALARM then
become sensitive, activating various functionality when touched. The Silen-T also features a touchsensitive crystal, which is activated by pressing the crown. The watch then vibrates when the user’s
finger is run clockwise around the face, with a long vibration when the finger is at the current hour
position and short vibrations at the minute position.
Although round touchscreen watches are absent in the literature, there has been work on rectangular touchscreen watches. IBM’s Linux-based WatchPad prototype (Figure 2(a)) incorporates a
four-zone touchscreen, and Raghunath et al. implemented some simple interfaces for it [26]. However, because of the low resolution of the touchscreen, most of the applications depended upon the
scrollwheel included on the watch (visible in Figure 2(a) on the left side of the watch).
Blaskó further extended the IBM work, creating a rectangular prototype—although non-watchbased—with nine touch-sensitive zones (Figure 2(b)) that allowed for virtual scroll wheels [10],
sliding, and gestures [9]. In his interfaces, Blaskó used the bezel at the edges of the display as a “tactile landmark” to inform the user in an eyes-free manner of their current location on the touchscreen.
Unfortunately, the interfaces were only evaluated in stationary situations, and no consideration was
given to the potential issue of accidental activation.
One commercial (although not commercially successful) example of a touchscreen watch was
the Fossil/Abacus WristPDA, which was based on PalmOS and had a 160x160 touchscreen. While
its main menu showed only four icons (Figure 3(a)) and could conceivably be operated with a
fingertip, other application screens were tiny enough (Figure 3(b)) to require the use of a stylus built
into the watch band (Figure 3(c)).
While not wristwatch-based, there have been a number of publications on the general topic of
(a)
(b)
Figure 2: The IBM WatchPad (a) and Blaskó’s interface for a variation with nine touch-sensitive
areas.
circular interactions. Several techniques investigate menu selection; the most well-known are the
Pie Menu [5], which uses a round menu with wedge-shaped buttons (Figure 4(a)), and its extension,
Marking Menus [14]. These works have been extended in numerous ways: FlowMenu [11], for
example, allows in-place hierarchical menu selection, command parameter entry and text input;
earPod [38] used a pie-menu like design for an auditory menu (Figure 4(b)); and Schmidt et al.
[29] developed a pie menu-like interface for wearable computers (Figure 4(c)). Scrolling is another
common interaction using circular interfaces. The Radial Scroll Tool [30] allows a user to scroll
through a document by making a “dial turning” motion. The Virtual Scroll Ring [21] is a similar
idea. As well as moving through documents, scrolling widgets can be used for parameter selection:
both FlowMenus and the Curve Dial [31]—the successor to the Radial Scroll Tool—allow for eyesfree parameter entry.
Text entry has also been implemented using circular interfaces. FlowMenu uses an adaptation of
the Quikwrite technique [25]. Cirrin [17] uses a circular key layout and a continuous gliding-style
interaction to enter text (Figure 4(d)).
Much of the literature on circular-style interfaces defines, whether explicitly or implicitly, two
regions of interaction: the rim and the center. While these areas also make sense for a round
touchscreen watch, I have encountered a lack of data on appropriate widget sizes for wristwatchsized screens.
2.3 Gesture Watch
Activity recognition in general, and gesture recognition specifically, have been areas of research
for a number of years. With the advent of inexpensive accelerometer chips, gesture recognition
has moved from primarily computer vision-based to acceleration-based, with some systems [4] also
incorporating MEMS-based gyroscopes.
Gesture recognition may be differentiated from activity recognition by looking at the assumed
intent of the input. Activity recognition is, in general, concerned with automatically determining
what a user is doing at any given moment. Gesture recognition, on the other hand, assumes intentional input by the user with the explicit understanding that the movements made by the user will
be used to perform some action. In this proposal, therefore, I will discuss only gesture recognition.
Projects in gesture recognition can be divided into a spectrum based on the type of gestures
(a)
(b)
(c)
Figure 3: The Fossil/Abacus WristPDA. (a) shows the application menu, with icons large enough
to push with a fingertip; (b) shows the calculator application with tiny buttons; (c) illustrates the
scale of the included stylus that is normally tucked into the watch band. It is likely that the stylus or
another pointed instrument would be needed to effectively use the calculator application.
to be recognized. At one end of the spectrum lies sign language recognition. Starner et al. used
computer vision to recognize a fixed set of American Sign Language (ASL) [33], and that work
has been continued by McGuire et al., who incorporate accelerometers and recognize a much wider
range of ASL [19].
The other end of the gesture recognition spectrum is occupied by fast, simple command gesture
systems. Perhaps the simplest example is that of tapping or hitting a device in order to perform
some action [27]. More complicated interactions can be achieved by moving towards the language
end of the spectrum. One of the earliest of this type of interfaces was the Gesture Pendant (Figure
5(a)), a project I worked on in 1999 [1]. The Gesture Pendant was a computer-vision based system
that allowed broad gestures to be used to control home appliances. The functions controlled by
the gestures were largely parameter-adjustments, such as changing the volume on a stereo, the
brightness of a light, or the channel on a television.
The FreeDigiter [20] system was a derivative of the Gesture Pendant, using infrared proximity
sensors to recognize splayed fingers passing in front of the mounting position on headphones (Figure
5(b)); this could be used to control a mobile phone, mobile audio player, or other device. Amongst
the improvements made on the Gesture Pendant was that the proximity sensors allowed for a much
smaller size and unobtrusive design.
Proximity sensors were also used for another Gesture Pendant derivative, the Gesture Watch
[12]. The Gesture Watch placed the proximity sensors on the wrist (Figure 5(c)), allowing the user
to make gestures in the air above the device.
One issue shared by the Gesture Pendant and its derivatives is how to tell when the user has
intended to initiate a gesture. The approach taken by these projects has generally been to take
a “push-to-gesture” approach. Similar to “push-to-talk” for speech recognition systems, push-togesture has the user push a button (Gesture Pendant) or make a known gesture (FreeDigiter, Gesture
Watch) to initiate and end gesture recognition. In Chapter 5, I propose a solution to this commonlyencountered problem.
(a)
(b)
(c)
(d)
Figure 4: Interfaces using a center/rim style of interaction. (a) is an example of a pie menu [5], (d)
illustrates Cirrin [17], (b) is a conceptual visualization of the earPod menu system [38], and (c) is
a pie menu-like selection system designed for see-through head-mounted displays (the central black
area will appear to be transparent to the wearer).
(a)
(b)
(c)
Figure 5: Three on-body command-gesture devices. Device (a) is the Gesture Pendant [1], (b) is
an example of FreeDigiter [20] in use, and (c) illustrates the Gesture Watch [12].
CHAPTER III
MICROINTERACTIONS
Microinteractions, as discussed briefly in Chapter 1, are interactions with a device that take a short
time to complete. I gave a number of examples in that Chapter, that for convenience I will replicate
here: checking the time, skipping to the next track on a digital audio player, sending a call to
voicemail, reading a text message, checking the arrival time of the next bus, taking down a phone
number, getting an idea of appointment times for the day, dialing a phone number from an address
book, calculating a tip, checking the weather, updating status information in a social notification
service such as “Twitter”, and setting or responding to a reminder alarm.
These functions are clearly desirable—they serve to make everyday life easier and reduce the
cognitive burden of many activities, and all are supported on many current devices in one form
or another. Most of these tasks, however, are hidden behind layers of interface that the user must
navigate through in order to reach the requisite application. In addition, devices are often carried in
a pocket, purse, or bag, making access a cumbersome task at best. I define setup time as the time
taken to get to the point in use of a device that the desired task can be accomplished. As will be
shown, setup time can be divided into access time—the time taken to access and make ready the
device itself—and navigation time—the time taken to navigate through the interface of the device
until the desired function can be performed.
To show that in microinteractions the setup time is important, I will present two pieces of evidence. The first is from a calendaring device study performed by several members of our lab in
2004 [35]. The second is a study I recently completed and that has been accepted for publication to
CHI [2]. The first study (Section 3.1) indicates that if a device takes too long to access, people may
simply not use it at all, preferring instead to rely on a faster method. The second study (Section 3.2)
quantifies access time for three device placements in two different mobility conditions. Section 3.3
offers some discussion on how these two studies impact my proposed thesis work of reducing both
access and navigation time.
3.1 Calendar Study
This section uses data from a study conducted in early 2004 by Thad Starner, Neils Snoeck, Ben
Wong, and Marty McGuire [35]. The study surveyed 138 participants (88% age 18-25, 70% male,
90% students) passing through Georgia Tech’s Student Center as to their preferred method of
scheduling appointments when away from their desks. Participants were then asked to schedule
an appointment with the researcher, and the method actually used was noted.
Device
Scrap
Planner
PDA
Retr
17.8
11.8
11.0
Nav
—
7.6
12.7
Retr + Nav
17.8
19.4
23.7
Entry
18.1
12.5
14.0
Total
35.9
31.9
37.7
Table 1: In seconds, average time for events during scheduling appointments, per commonly used
device. Navigation time for scrap paper is effectively zero.
11
Memory
Scrap
Planner
PDA
(0s)
(17.8s)
(19.4s)
(23.7s)
Memory
24
1
—
—
Scrap
9
13
—
—
Planner
16
13
14
—
PDA
4
1
1
8
Table 2: Claimed (columns) versus actual (rows) usage of devices for scheduling task. The rows
are sorted by retrieval + navigation time, from Table 1, shown in parentheses.
Four devices were commonly used to perform scheduling: the participant’s own memory, a scrap
of paper, a planner, and a PDA. Note that at the time the study was conducted, smartphones were still
uncommon. Table 1, adapted directly from the paper, shows the mean time per device for retrieving
the device, navigating through the interface to the proper place to schedule the appointment, and the
time to enter the appointment.
Table 2, also adapted directly from the paper, sorts the four devices plus memory by retrieval
plus navigation time (I have elided the “other” column and row from the table, as no timing information is given in the paper for this category). It shows the device that the participant claimed
that they used for scheduling versus the device that was actually used when the researcher tried to
schedule an appointment with the participant. The table shows that when a participant failed to
use the claimed device, they almost always picked a method of performing scheduling that allowed
them to get to the device faster.
3.2 Quickdraw
This Section is adapted from [2], accepted to CHI 2008. It provides quantitative data about the
amount of time required to access a device in three positions on the body, and in two mobility
conditions.
3.2.1
Introduction
Mobile phones have become ubiquitous, with the number of subscribers worldwide predicted to
pass three billion in late 20071 . It is not uncommon for mobile phones to be the first object with
which people interact in the morning and one of the last things with which people interact in the
evening [7]. Despite the importance of these mobile devices in everyday life, little empirical data
has been published on many fundamental usage properties.
One of these underreported-upon properties is access time—the amount of time it takes a user
to get to a device for the purpose of interacting with it. To take the mobile phone as a familiar
example, the access time is how long it takes to get the phone from its storage place—such as a
pocket, holster, or bag—and become ready to interact with the device by orienting it properly.
In this work, we examine and quantify two important factors that could impact access time: onbody placement and user mobility. Specifically, we focus on the effects of walking versus standing
when accessing a touch screen interface mounted on the wrist, on the hip, and in the pocket. This
work is not intended to influence users to change their behavior with respect to mobility or device
storage, but to allow designers a better understanding of how mobility and placement affect access
time.
1 http://www.portioresearch.com/WWMM
stats07.html
3.2.2
Experiment
To explore how body placement and mobility influence the time needed to respond to and access
a mobile device, we examined two independent variables with a 2x3 Latin-square within-subjects
study design. The first variable is the mobility of the participant and the second is the placement of
a device on the body. Although our interest in mobile devices is broader than telephones, we used a
mobile phone throughout the study in order to keep the interaction consistent between conditions.
Our two mobility conditions are standing and walking, chosen because people on-the-go are
likely to be in one of these two states much of the time. Our three on-body placement conditions
were in the pocket, on the hip in a holster, and on the wrist, reflecting common placements for
current mobile electronics.
During the standing condition we instructed participants to stand in a corner of our lab to minimize visual distractions that might interfere with access time.
For the walking condition, participants were instructed to walk at a normal pace around a track
constructed in our laboratory (Figure 6). The track was approximately 26 meters long and was
denoted with flags hanging from the ceiling with the tips 0.75 meters apart. Each flag was hung so
the tip was approximately 1.6 meters above the floor. We chose to use this flag arrangement rather
than floor-based track markers (as in [36]) because walking is in general a head-up task (that is,
people usually look ahead some distance while walking rather than looking directly at the floor).
The walking direction was counterbalanced between clockwise and counter-clockwise directions.
Figure 6: The path participants walked, starting at flag 1 and proceeding either clockwise or
counterclockwise.
For the device placement condition, we instructed participants to put the phone into a pants
or skirt pocket (after removing other items), into the manufacturer-provided holster clipped to the
top of the pants or to the belt, or to attach it to the wrist with a velcro strap (Figure 7). While
using an actual touchscreen watch—such as SMS Technology’s M500 mobile phone watch—would
have allowed for a more natural experience, we opted instead to maintain internal validity by using
the same device (and therefore retaining the same display and touch input properties) for all three
conditions.
To determine the amount of time required to access the phone, we asked each participant to perform a simple task. Periodically, the phone generated an alert in the form of a sound and vibration.
Each participant was requested to respond to these alerts as quickly as possible. When the alert
occurred, the participant retrieved the device and looked at the screen, which showed a blue box and
a large number (Figure 8(a)). The participant made a mental note of the number on the display and
slid the blue box to the right to unlock the phone. The participant then chose the number they had
just seen from a list of four numbers 8(b). After choosing the number, the participant returned the
phone to its original position and waited for the next alert.
(a)
(b)
(c)
Figure 7: Placement: pocket (a), hip (b), and wrist (c).
3.2.2.1
Software and Equipment
The software for our study was implemented in Python on a Motorola E680i cameraphone running
the GNU/Linux operating system. The E680i has a 320x240 color touchscreen, stereo speakers,
and a number of buttons. For this study, we used only the touchscreen and deactivated all of the
hardware buttons so they would not be pushed accidentally. Pygame, a Python interface to the
graphics library SDL, was used to create the user interface and to log user actions.
During each condition, the operation of the software was the same. At random intervals between
20 and 40 seconds (selected from a uniform random distribution), the software generated alerts. An
alert consisted of a loud mobile phone ringing sound and vibration that lasted up to 20 seconds.
During the alert the software also displayed the prompt number and unlock mechanism on screen
(Figure 8(a)).
To respond to the alert, participants first unlocked the phone. Unlocking was accomplished by
sliding the blue box to the right edge of the screen (similar to the Apple iPhone unlock gesture).
This mechanism was implemented to prevent accidental responses while retrieving the phone from
the pocket or holster. Next the phone displayed a screen consisting of a two by two grid of numbers
(Figure 8(b)). The software waited for the user to select a number and logged the response. This interaction emulated common interruptions on mobile devices (for example: receiving a call, reading
the caller ID, and sending the caller to voicemail). At this point the trial was complete. The phone
relocked itself, and a timer was set to generate the next alert. The software logged the timestamps
of each alert, the movements of the slider, and the selection of the numbers.
Because the phone had no mechanism for determining whether it was in a pocket or holster, we
implemented an extremely simple light sensor using the built-in camera. Approximately four times
per second, the pixel values from the camera were summed and stored. With the assumption that
the holster and participants’ pockets would be dark, we could detect when the phone was removed
from, and replaced in, the pocket or holster. Figure 9 shows the output of the light sensor and other
events logged by the software.
(a)
(b)
Figure 8: Screen (a) is shown when an alert occurs. The participant must mentally note the displayed number and slide the blue box from the left to the target box on the right. Screen (b) is shown
after the slide is completed; from the displayed four numbers, the participant must touch the number
that was displayed on screen (a).
3.2.2.2
Dependent Measures
We are most interested in how device placement and mobility influences access time—the amount
of time it takes for a participant to retrieve the device and respond to an alert. Figure 9 shows a
timeline of a typical notification-response cycle. The points in the timeline are as follows:
1.
2.
3.
4.
5.
6.
7.
Blank screen; participant walking track or standing.
Alarm starts ringing.
Participant pulls phone from pocket or out of holster. Light level increases to nearly 100%.
Participant starts to move slider.
Screen with four numbers is displayed.
Participant has picked a number; screen returns to blank.
Participant returns phone to pocket or holster. Light level falls to 0%.
We extracted several measurements from the timeline (numbers refer to the timeline in Figure
9):
• Access time: Alarm start (2) to user acknowledgment of the alarm by moving the slider (4).
• Pocket time: Alarm start (2) to retrieval of the device from the pocket or holster (3) (does not
apply to wrist).
• Hand time: End of phone removal from the pocket or holster (3) to the beginning of participant’s response (4) (does not apply to wrist).
• Slide time: Moving the slider (4 to start of 5).
• Answer time: Picking a number from the set of four (5).
• Replacement time: Replacing the phone to the pocket or holster (start of 6 to 7—does not
apply to wrist).
Figure 9: Timeline of events and status of brightness detector during one notification-response
cycle. The top line is the percentage of light detected from the camera (complete darkness, 0% on
the bottom), and the bottom is the timeline of events recorded by the phone’s logging software. See
the text for further details.
We define access time in this context as the time required for the participant to react to the alarm,
acquire the device (either looking at the wrist or pulling it from a pocket or the holster), note the
number on the screen, and touch the slider to begin sliding it.
3.2.2.3
Procedure
The evaluation for each participant began with the researcher presenting an overview of the study.
Participants completed a consent form and a short survey about their use of mobile technology.
The researcher then explained the experimental software and tasks. Each participant practiced responding to alerts three times on the phone for each of the placement conditions (pocket, hip, wrist)
resulting in a total of nine practice responses. Next, the researcher familiarized the participant with
the track by walking it twice in each direction; on the first lap the participant followed the researcher,
while on the second, the researcher followed the participant. Finally, the researcher answered any
questions from the participant, and started data collection.
Each condition consisted of a set of trials. The number of trials per condition was either five or
seven, averaging to six per condition per participant. This design was selected to prevent participants
from anticipating the end of a set of trials. For the walking conditions, participants were told they
could slow down or stop if needed to respond to the alert quickly, but to keep walking if possible.
After the completion of the required number of trials, the phone displayed “STOP” in a large
font on the screen. Participants were requested to stop where they were (if in the mobile condition)
to allow the researcher to measure how far they had walked around the track.
3.2.2.4
Participants
We recruited fifteen participants (5 females) for our study from our academic institution. The average participant age was 24.87 years (SD = 2.99). Fourteen of our participants were right handed.
Each of our participants owned a mobile phone and all but three had that phone with them on arrival. Six of our participants wore a wristwatch when they arrived to participate in the study. We
also asked about several other devices and where the participants carried them. Table 3 provides a
summary.
Mobile phone
Phone headset
Audio player
Camera
Total
Pocket
7
4
4
6
21
Bag
4
—
5
5
14
Hip
1
—
1
—
2
Body
—
—
1
1
2
Total
12
4
11
12
39
Table 3: Devices carried by participants and their location. “Body” refers to on-body placement of
a device, such as clipping to clothing or using a wrist strap. “Bag” includes purses and backpacks.
3.2.3
Results
One participant was discarded as an outlier—having taken up to 8.5 standard deviations longer
than average to respond to alerts—leaving 14 participants. Each participant performed all six of
the conditions. Each condition averaged six alerts per condition, for a total of 504 alert-response
cycles. Table 4 shows a summary of data for each condition. For this paper, we consider p < .05 to
be significant and will only report p values for non-significant results.
A multi-way ANOVA reveals that the placement of the device has a significant effect on access
time, but mobility is not significant (p = .14). There is also no significant interaction between mobility and placement (p = .81). Post-hoc analysis of the access times using a paired Student’s T-test
reveals a significant difference between all three combinations of variables: hip/pocket, hip/wrist
and pocket/wrist. Therefore, we have a total ordering of access time for the placement condition:
wrist (M = 2.8) < pocket (M = 4.6) < hip (M = 5.5); by comparing the mean access times, we can
see that the pocket condition yields a 66% longer access time than the wrist while the hip requires
98% more access time than the wrist.
For non-watch conditions, where the light sensor was used, pocket time was significantly affected by placement, but not mobility (p = .128), and there was a significant interaction between
placement and mobility. There was no significant effect of placement for hand time (M = 1.145,
SD = .628), but there was a significant effect for mobility as well as an interaction.
Some, although not all, of the other measures described earlier were significant (refer to Figure
9). The answer time was found to be significantly affected by mobility, but not placement (walking:
M = 1.227, SD = .304; standing: M = 1.479, SD = .583). Measures non-significantly impacted
by placement or mobility were slide time (M = .071, SD = .72) and replacement time (M = 3.195,
SD = 1.787).
Time
Hand
Pocket
Access
Time
Placement
Hip
Pocket
Hip
Pocket
Hip
Pocket
Wrist
Walking
1.047 (.656)
1.109 (.647)
4.355 (.906)
3.427 (.770)
5.377 (.947)
4.414 (.904)
2.728 (.291)
Standing
1.339 (.728)
1.093 (.396)
4.312 (.770)
3.829 (.788)
5.660 (1.155)
4.817 (.875)
2.846 (.420)
Average
1.199 (.707)
1.092 (.536)
4.333 (.836)
3.625 (.868)
5.518 (1.047)
4.616 (.897)
2.787 (.360)
Table 4: Statistics for mean (SD) hand, pocket and access times in seconds.
3.2.4
Quickdraw Discussion
The watch placement condition resulted in much faster access to the device. This finding is unsurprising, because participants did not have to remove the phone from the pocket or holster in order
to use it.
In Figure 9, access time (from 2 to 4 in the timeline) is divided into two segments for non-wrist
conditions: pocket time (from 2 to 3) and hand time (from 3 to 4). The statistics in Table 4 reveal
that the majority of access time is consumed by pocket time2 : on average, 78% of the time from
the alarm until the participant started moving the slider was involved in getting the device out of the
pocket or holster!
The watch condition also resulted in much more consistent access time to the device; the standard deviation of access time for the pocket and hip conditions is 2.5 and 2.9 times more than the
wrist condition, respectively. The long time required to retrieve the phone from its holder may help
explain why the participants in the Cui et al. study reported missing phone calls due to how they
carried their phones [8].
While we anticipated the superior performance of the watch condition, we did not expect the
holster to perform worse than the pocket condition for access time. Reviewing user comments
made during the study, however, it becomes clear that poor holster design may account for this
result. Despite our use of the manufacturer’s holster, several participants complained during practice
trials that the holster was difficult to use, and that difficulty may have persisted through the course
of the study. An additional possibility is familiarity; while participants were presumably wellpracticed with putting items into and removing them from the pockets of their own pants, none of
the participants reported using a holster to carry their phone (Table 3) and therefore may have been
slower with the holster than would have otherwise been expected. Given these factors, our results
for the hip condition should probably be viewed as a worst case for holster access.
Finally, it was interesting to find that, in contrast to previous work [36], walking did not significantly impede user performance relative to standing still. In contrast, our data was trending to show
that standing resulted in slower access. One possible explanation for this difference is our inability
to separate reaction time and access time with our current experimental design. It is possible that
the walking conditions kept the participants more engaged in the experiment relative to the standing
conditions, and therefore the reaction time was slower while standing.
3.3 Discussion
The results from the calendar study in Section 3.1 imply that the setup time involved in retrieving
and navigating within the scheduling device, if long enough, can dissuade a person from using
the device at all. This conclusion is borne out by other work as well [6, 15]. This suggests that
decreasing the access and navigation times may improve interaction with the device.
The calendar study directly influenced further work that resulted in a project called “DualPurpose Speech” (DPS) [16]. DPS greatly reduced in-application navigation time for scheduling
appointments by utilizing the user’s regular conversational speech as input to a wearable appointment system. When, in a conversation, the user said something like “Okay, Monday at 3:30 sounds
good,” the system would automatically navigate to the correct day and time in the appointment
software.
Techniques such as DPS to reduce in-application navigation time are one way to increase the
2 Note
that in Table 4 pocket + hand 6= access. Two participants had light sensor problems, making hand and pocket
time impossible to recover. Thus, these subjects were not included in this subsection of the table, making the quantities
not exactly sum.
usability of devices, but have the weakness of being domain-dependent. To use DPS with a different
application would require training the system for a new set of vocabulary, and figuring out how to
unobtrusively mesh commands to the system with everyday speech for that application. Reducing
access time for a task is an alternate approach that may be easier to implement.
The “Quickdraw” study described in Section 3.2 looked at how different on-body placements
of a device might affect access time for the device. By placing a device on the wrist, access times
were greatly reduced.
Good placement is one component to reducing overall setup time, and DPS-like techniques
reduce in-application navigation time. However, the remaining issue is that of out-of-application
navigation time: getting to the application or function that is desired. This thesis proposal focuses
on two methods for decreasing this time.
CHAPTER IV
TOUCHWATCH
As discussed in the Introduction, touchscreens are a well-understood and common method for interacting with mobile devices. Several touchscreen watches have existed (see Chapter 2) but most have
very simplistic input capability—such as the IBM WatchPad—or require the use of a stylus for more
complex input, as with the Fossil WristPDA. Neither of these methods is appropriate for on-the-go
microinteractions. If a user wants to quickly dial a phone number for a call on a Bluetooth headset,
a stylus will take a frustratingly long time to access, and trying to input a phone number with only
four soft buttons is likely to be time consuming as well. For these reasons, I want to explore how a
finger-only touchscreen can be used for quick interactions with a touchscreen-based watch.
As discussed in Chapter 2, Blaskó has performed some research in this area, especially with
respect to manipulating numeric parameters. However, his touchscreen watch is rectangular—as
have been all other touchscreen watches I have been able to find thus far. This rectangular form is
somewhat at odds with standard wristwatch design. The convention of circular watch displays has
a long history tracing to some of the first wristwatches, pocket watches, and the first analog clocks
[18]. This design has become so ingrained in our culture that even many digital watches still have
round faces surrounding the blocky digital time readout. Given this apparent cultural preference,
I propose to investigate what touchscreen interactions should be like for a watch with a circular
display.
One approach to round touchscreen interaction is to follow the convention of other circular
interfaces and put buttons around the edge. I conducted an experiment to determine what the best
way for a user to interact with such a rim-based interface might be, and to find out with what density
buttons could be placed around the edge. The results are presented in Section 4.1, which is partially
excerpted from a paper in preparation for submission to MobileHCI 2008: D. Ashbrook, K. Lyons,
and T. Starner, “My Watch is on the Fitts: an Investigation into Round Touchscreen Wristwatches”.
Section 4.2 follows with some discussion on how to use the results to design an interaction
technique, and Section 4.3 proposes a study to evaluate the effectiveness of this technique.
4.1 Experiment
We conducted a Fitts’ Law experiment [32] to investigate the feasibility of interactions on a small,
round touchscreen watch. We are particularly interested in the properties of an interaction area
around the rim of the watch face.
4.1.1
Variables
Movement type is our primary independent variable. Considering the kinds of movements that
might be used for selection on the face of a touchscreen watch, we decided to test three methods.
A tap to select is the simplest method, much like using a mouse to click on icons. This is the
tap condition. Another possibility is sliding the finger from one place on the surface to another;
this interaction is used in variety of interfaces such as Cirrin [17] and FlowMenu [11]. There are
several possibilities for sliding on a watch surface. A natural method is to pick the shortest distance
between the finger and the target, sliding the finger through the center region of the face; this forms
20
the through condition. Another option is sliding the finger using the bezel as a guide, avoiding the
center region of the watch. This is the rim condition.
The main dependent variable of interest is the effective target width, We [32]. This measure
indicates the size of a target that would accommodate 96% of the attempts to select it under Fitts’
Law conditions. Calculating this number will give insight into how many target regions, i.e. buttons,
could be placed around the rim of the watch, and how much space will be left for a display in the
center. The calculation of We is discussed in Section 4.1.6.
4.1.2
Design
Our experiment is a repeated measures within-subjects design. The primary independent variable
of interest is movement type (tap, through, rim). We also experimentally manipulated the distance
between targets to obtain data needed for our Fitts’ Law calculations. Because our surface of interaction is circular, we used degrees to specify the target placements and distances. In order to
ensure that our data covers the face of the watch, we first randomly chose a start position from a
uniform random distribution from 0◦ –360◦ . Using a time metaphor, we next divided the face of the
display into twelve segments of 30◦ each. We further divided each segment in half to get a minimum distance between targets of 15◦ . We used all of the distances in 15◦ increments between the
minimum of 15◦ and a maximum distance of 180◦ , resulting in a total of twelve different distances.
Note that for the tap and through conditions, movement are straight-line, so a 180◦ distance will
yield a movement equal to the diameter of the watch—1.25in (30mm). The rim condition, however,
will yield an arc length, so a 180◦ distance will actually yield a movement equal to πr, or 1.96in
(47mm).
(a) tap
(b) through
(c) rim
(d) tap
(e) through
(f) rim
(g) key
Figure 10: Images (a), (b) and (c) show the targets (short green lines at rim of circle) and movement
guides (yellow line between targets) displayed to study participants. Images (d), (e) and (f) show
actual movements for one session; blue lines indicate movement by participant’s finger and small
yellow circles show inflection points. Movement images were displayed only to researcher. Images
(a) and (d) are for tap condition; images (b) and (e) are for through condition; images (c) and (f)
are for rim condition. Images (a)–(f) are printed at actual size.
Figure 11: The simulated touchscreen watch.
The experiment was composed of numerous trials. For each trial, the participant moved back
and forth between the pair of targets 15 times, resulting in 30 movement end points. A block is
a set of twelve trials (one per distance) with one movement type (e.g. rim). The blocks were
repeated three times for each of the three movement types. Therefore we have 12 distances × 3
repetitions × 3 movement types = 108 trials per participant. The movement types and distances
were ordered according to a Latin square to minimize potential ordering effects. In between each
block, participants were given an enforced 30-second break to avoid fatigue; if desired, they could
break for longer than 30 seconds.
In order to avoid biasing participants with any particular size of target, and lacking recommendations from the literature, we selected targets as close to zero width as we could display. These
were simple lines extending from the outside of the watch towards the center (Figure 10). Each line
was one pixel wide, or approximately 0.58◦ at the inner end of the target. While these targets are unrealistically small, they will allow us to determine the effective width We of a target by considering
the distribution of nearby hits. Using the Shannon formulation for the index of difficulty,
D
+1
(1)
ID = log2
W
this small target size yields a maximum range of ID values from 4.64 to 8.3 bits.
4.1.3
Apparatus
Lacking a programmable round touchscreen watch, we simulated one. We removed the case from
a Motorola E680i touchscreen mobile phone and placed it into a custom-made plastic case (Figure
11). In order to simulate a round watch face with a bezel, we placed over the screen a customcut plate of .06in (1.6mm) thick steel with a hole approximately 1.25in (30mm) in diameter, with
a sloping bevel around the hole. The “watch” was mounted on each participant’s arm using a
velcro strap, with the visible portion of the screen centered on the arm approximately where a
normal wristwatch face would be. Software was written for the phone using Python and Pygame
(an interface to the SDL graphics library) to present trials to participants and collect data.
4.1.4
Procedure
Participants performed a Fitts-style reciprocal 2D pointing task using their finger as the pointer.
Two targets were displayed on the watch face and participants were requested to select the targets as
quickly and accurately as possible. For sliding conditions, a line was drawn to remind the participant
what kind of motion to make (Figure 10). Because we were trying to determine We in addition to
standard Fitts metrics, we did not force participants to hit the targets—instead, our software simply
detected the endpoints of the movements.
In the tap condition, the participant lifted their finger in between each tap of a target. In the
through condition, the participants were instructed to slide their finger along the shortest distance
between the targets. The rim condition was designed with bezel-based interactions in mind, and
required participants to slide their finger along the rim of the watch between the targets, making an
arc.
4.1.5
Participants
Fifteen volunteers were recruited to participate in the experiment. Fourteen volunteers were male
and one was female; ages ranged from 20 to 28 (SD = 1.8). Participants were paid US $10 per hour
for participation; no participant took more than one hour to complete the experiment. All participants were either right-handed or primarily used their right hand for control tasks such as mouse
movement. Eight participants reported normally wearing a watch, and all stated they normally wore
the watch on the left wrist.
4.1.6
Results
The standard deviation σ of each cluster was calculated, and points falling greater than 3σ away
from the cluster mean were discarded. We calculated the effective target width We [32]. This metric
is defined using the standard deviation of the endpoints of movements: We = 4.133σ. Statistically,
for a button of width We , 96% of attempts to hit the button will be successful. We calculated the
effective width both parallel and—while not a traditional measure—perpendicular to the direction
of movement. For tap and through, this meant along and perpendicular to the line between the
cluster means; for rim, this was around the circumference of the circle and on the line from the
center of the circle to the center of each mean. The mean value of We (in pixels) for each condition
is listed in Table 5.
Condition
tap
through
rim
We k
25.1 (5.0)
22.9 (5.2)
22.9 (7.8)
We ⊥
25.3 (5.5)
22.4 (5.9)
10.1 (3.8)
Table 5: Mean (SD) effective width We in pixels for each movement condition, parallel (k) and
perpendicular (⊥) to the direction of movement.
The effective distance De was calculated as the mean of the distances between each pair of
movement end points, which in turn allowed the effective index of difficulty IDe to be calculated.
This is similar to the index of difficulty ID, except that the effective distance De and effective width
We are used instead (see Equation 1). The range of IDe values for each condition were: 0.90–3.66
(M = 2.32) bits for tap; 0.63–4.44 (M = 2.40) bits for through; and .74–3.97 (M = 2.26) bits for rim.
Figure 12 shows a plot of the effective index of difficulty IDe versus the movement time MT , and
shows corresponding regressions for each movement condition. Finally, the throughput values were
calculated. Throughput combines both speed and accuracy, and is calculated as T P = IDe /MT . For
tap, the mean T P was found to be 7.7 (SD = .98) bits/sec; for through, 11.8 (SD = 1.0) bits/sec;
and for rim, 7.2 (SD = .72) bits/sec.
(a) tap
(b) rim
(c) through
Figure 12: Effective index of difficulty (IDe ) versus movement time (MT ) for the three movement
conditions. The regression line is dotted in each graph. Regression statistics are: tap slope:
40ms/bit, intercept: 228ms, R2 : 0.04); through slope: 54ms/bit, intercept: 106ms, R2 : 0.09);
rim slope: 158ms/bit, intercept: −18ms, R2 : 0.45).
4.1.7
Discussion
Our tap and through conditions are typical Fitts’ tasks, though rim is not. However, Figure 12(c)
shows that the correlation between effective index of difficulty IDe and movement time MT is
reasonably linear (R2 = .45) for rim. The R2 values for tap and through—.04 and .09, respectively—
do not show as close a fit; we believe this is due to the small range of IDe values. Fitts’ original
paper found a range of IDe values between 1 and 6.02 bits [32], while we have the smaller range of
0.63—4.44 bits.
Our effective indices of difficulty can be attributed to several factors. The first is the small
size of the watch face—only 1.25in (30mm) in diameter, giving only a relatively small range of
possible movement distances. The second factor is our small target size. By giving only thin lines
for targets and not enforcing via software the participants hitting the targets, we essentially allowed
participants to choose their own widths and distances, growing We and shrinking De , therefore
decreasing the range of IDe values. Another possible influence is that for some of the smaller
distances, a participant’s finger could occlude the targets, removing the visual guidance necessary
for a Fitts’ task.
Given the effective width values for the three conditions, we can now think about what a round
touchscreen watch interface might look like. Our initial approach was to divide the circumference
of the watch face by the effective width for a particular movement type and put that many buttons
around the edge. However, we discovered that participants, possibly due to the width of their fingers,
did not touch the touchscreen close to the edge: the mean distance Dµ from the rim for tap was 24.3
(SD = 10.8) pixels; for through 21.8 (SD = 10.8) pixels; and for rim 18.3 (SD = 8.7) pixels.
To determine the optimal number of buttons to place around the rim, we can use Dµ , the mean
distance from the rim. Dividing the circumference of this circle by the effective width gives the
maximum number of buttons:
2π(r − Dµ )
N=
We
(a) tap
(b) through
(c) rim
(d) key
Figure 13: Possible watch face layouts.
Where r is the watch radius and the We used is parallel to the movement direction for rim and
perpendicular for tap and through. As illustrated in Figure 13, this calculation yields 15 buttons
for tap, 18 buttons for through, and 20 buttons for rim. In Figure 13, images (a)–(c) are printed
at the actual size of the physical watch face. In the key, for tap and through, We1 represents We
perpendicular to the direction of movement and We2 is parallel; for rim these are switched. The
number in the center of each layout is the percentage of screen area that is left for the center region
after the buttons have been placed.
4.2 Rim Interfaces
How may the results of the experiment be used to design interfaces? Although the rim method
resulted in the best performance, it is bound to be unfamiliar to novice users. If simply presented
with 20 buttons around the watch rim, it is likely that novices would use something similar to the
tap method to interact, causing errors and frustration. Therefore, some method must be used to
encourage users to utilize the most effective interaction.
A second potential problem that may be encountered when using a touchscreen watch is erroneous activation. An accidental brush of the watch against an object or even a finger touch while
adjusting the watch on the wrist could activate unwanted functionality.
Both of these problems may be addressed by considering the round face of the watch. A round
interface naturally lends itself to circular movements. By creating the interface such that users
naturally use circular, rim-based movements, accidental activations will be reduced. However, how
can such interfaces be created without feeling unnaturally forced to the user?
(a)
(b)
(c)
Figure 14: Blaskó’s card dragging technique [9]. The default watch face is illustrated in (a), with
the asterisk denoting the desired card. The motion made by the user’s finger is illustrated in (b),
and the resulting screen with the activated content is shown in (c).
4.2.1
Card Dragging
In his thesis [9], Blaskó proposed the idea of pullable “content cards”, illustrated in Figure 14. Each
card contains information or an applications, and “protrudes” slightly into the face of the watch
from the edge. When the user wants to access a particular card, they drag the card from the edge
down, “pulling” application or information into view. The benefit of this method is that it trained
users in Blaskó’s on-screen gesture system.
I propose to use a similar method to encourage users to use the rim method of interaction. Each
button will represent a “card” containing information or an application. Users will, starting with
the button, drag around the rim 360◦ (or so) to bring up the application. This interaction has a
similar function to the iPhone’s slide-to-activate mechanism: it is exceedingly unlikely that a full
360◦ movement will be made accidentally, so any such movements can be treated as intentional on
the part of the user.
Figure 15 illustrates a mock-up of this idea, showing a theoretical interaction in which a user
selects the weather application from a list and drags its “card” out of the “stack” of applications.
(a)
(b)
(c)
(d)
(e)
(f)
Figure 15: Proposed “card dragging” interaction using icons on rim of watch. (a): view of watch
with no applications active; (b): user touches face to show application icons; (c): user selects desired application icon; (d)–(f): user drags application “card” out of “stack”. Images are shown at
actual size for hardware used during Fitts’ study and conform to design recommendations discovered therein.
4.2.2
Calendar Dialing
Opening applications using rim dragging avoids false detections, but using other interaction techniques within the applications violates the consistency principle of HCI. In addition, having up to
20 possible buttons might be useful within applications. Therefore, I am interested in how the rim
paradigm of interaction may be used in other applications.
In the real world, one common interface that uses circular movements is the knob. This suggests
that a knob metaphor may make sense on a round touchscreen watch as well. An already-familiar
use of this metaphor is the Apple iPod’s touchwheel interface: the touchwheel is used for scrolling
in lists and for changing the volume of playing audio. One commonality among most knobs is that
of directionality—turning the knob clockwise increases a value or moves in one direction while
turning it counterclockwise generally has the opposite effect.
This idea is illustrated in Figure 16 for a calendaring application. Clockwise and counterclockwise turns are often mentally associated with increasing and decreasing; for example, clockwise
turns of dials increase volume on stereos while counterclockwise turns decrease volume. The
mockup uses this same metaphor of increasing with clockwise dialing and decreasing with counterclockwise dialing, but to zoom in and out of a calendar.
(a)
(b)
(c)
(d)
Figure 16: Mockup of a possible dial-metaphor interface for calendar navigation. Dark gray
segments in (a)–(c) show “now”, thick red borders indicate area selected, and bands around rim in
(d) indicate appointments. User touches red-banded area and drags finger clockwise to zoom in to
next closer detail view; to zoom out, user drags finger counterclockwise.
The lowest detail view, of the months of the year, is illustrated in Figure 16(a). The current
month, June, is indicated by coloring that segment in dark gray. To zoom into a view of June, the
user puts a finger on the “Jun” segment (this touch is indicated by a red border in the Figure) and
quickly moves it 360◦ around the rim in a clockwise direction. The calendar zooms into a view of
the weeks of June, shown in Figure 16(b). Although 28 days are shown (possibly with coloring or
other indicators of schedules for that day), we learned in Section 4.1 that 20 is the maximum number
of buttons that should be placed around the edge. Therefore, as an intermediate step, the user selects
the entire week. This is accomplished in the same way as zooming into the month: the user quickly
rotates a finger 360◦ clockwise around the rim. The display changes to a week view in Figure 16(c),
and the same interaction can take place. Finally, 16(d) shows the day view with bands indicating
schedule items. A user could further zoom into the calendar at this point—perhaps by selecting
an appointment band—to edit information or view more detail. One can imagine extra information
shown at each of the zoom levels to indicate number of appointments, upcoming schedule items,
priorities, or other data.
4.3 Proposed Work
The experiment described in Section 4.1 gives recommendations for the numbers of buttons for a
round touchscreen watch interface, but the results have not been empirically validated. Therefore,
I intend to perform a second study to validate the results. The procedure will be similar to the first
study, but will be in the traditional Fitts vein, with targets of specific sizes and locations.
Once the second study has been completed and the results from the first have either been validated or modified appropriately, I will create several prototype interfaces using the rim method of
interaction to determine how effective it is. I will implement the application selection and calendar zooming interfaces described in Section 4.2 and compare performance and user preference to a
standard-style interface such as on the Palm-based Fossil WristPDA.
CHAPTER V
WRIST-BASED GESTURE INTERACTION
I propose to create and study a wristwatch prototype that performs basic gesture recognition. Rather
than a complex, many-movement language such as American Sign Language, my proposed system
will recognize a small collection of quick, unobtrusive gestures used for control rather than communication. I call my overall system MAGIC: Mobile Action Gesture Interface Control.
5.1 Gesture Triggering
One issue with on-body gesture recognition is how to trigger the start of the recognition process.
A common solution is to have a “push to gesture” mechanism, either a physical button [1] or other
intentional action [3] that signals to the computer that a gesture is about to be performed. This
increases the accuracy of recognition because there is much less chance of random movements being
erroneously recognized as intentional gestures. This is especially important in a system intended
for everyday use.
The drawback to the trigger approach is that an extra motion must be made. In the case of
a wristwatch—such as the Tissot (Figure 1)—it is not physically possible to push buttons on the
watch with the hand belonging to the wrist that the watch is attached to. This necessitates the use of
the other hand to push a button, which obviates the purpose of having control gestures rather than
buttons in the first place.
An alternative to using a button could be to use a starting gesture, after which the intended
gesture is performed. This solution is superficially similar to wordspotting in speech recognition
[37] (“Computer, save file”); however, such speech recognition systems are usually constrained
to quiet office environments. In the context of everyday use, it becomes much more difficult to
find a start gesture or word that is unobtrusive, easy-to-perform, and will have a low likelihood of
mis-recognition.
There is a solution to the triggering issue, however. By having each user create a database
of movements during a typical day—an “Everyday Gesture Library” (EGL)—new gestures can be
rated based on how likely they are to be mis-recognized. A user could purchase a device, wear it
for two or three days, and then train it on a set of new gestures. For each candidate gesture, the
system would then run through the EGL to see if it recognizes any of that data as the gesture under
consideration. If it does, it would warn the user to choose a different gesture.
Figure 17 illustrates the idea. A portion of an EGL recorded for one user is displayed in the top
portion of (b), showing the accelerometer trace of some normal movements made during the course
of a day. A candidate gesture—perhaps for a “play/pause” gesture for a digital audio player—is
illustrated in (a). This candidate gesture is similar to a movement made by the user during a normal
day; the similarity is indicated by the match percentage line shown in the bottom half of (b). The
match percentage indicates there is a high likelihood of accidental triggering, so a different gesture
should be chosen.
This technique could be used in end-user devices, where the user would wear the device for
several days to generate an EGL before training the system with a chosen gesture set, or could
be used by interface designers using a larger EGL collected from a variety of representative users.
A wide variety of interfaces that now use push-to-activate could benefit, from hand gesture-based
29
(a)
(b)
Figure 17: Candidate gesture performed by user (a) and partial gesture database (b) with match
percentage (bottom) and database contents (top) illustrated.
devices such as the Gesture Pendant [1] to touch-sensing interfaces like the Tissot T-Touch [28].
5.2 Proposed work
I propose to validate the idea of using an EGL to help pick the best gestures for an interface. While
the EGL concept should apply to any gesture system intended for everyday use, in keeping with the
theme of this proposal I intend to implement and test the concept with a wristwatch-based gesture
system.
5.2.1
Hardware
I am currently in the final stages of creating a wristwatch prototype with which to test this interaction
concept. The watch will consist of a 1.58”/4cm (diagonal measure of visible area) organic LED
(OLED) screen, with an ARM9 processor, 3-axis accelerometer, microSD card slot, and Bluetooth
radio. By itself, the watch should be able to collect and store a day’s worth of movement data, but
probably will not have the computational power to perform realtime gesture recognition. Instead,
it will communicate over Bluetooth to a more powerful device, such as a mobile phone, that is
nearby—perhaps in the user’s pocket.
Figure 18 shows the two sides of the prototype hardware board. The board is 1.4”/3.56cm on a
side. The screen is not shown in the photo, but sits on the processor side of the board with the cable
wrapping around to the other side.
5.2.2
Software
To perform the gesture database tasks discussed above, I have implemented proof-of-concept software. Server code runs on a Motorola e680i GNU/Linux-based mobile phone and records data from
a wrist-mounted Bluetooth accelerometer. Separate software, written in Python and pyGame (a
Python wrapper for the SDL graphics library), uses Dynamic Time Warping (DTW) [22] to provide
visualization like that illustrated in Figure 17 and to perform both database searches and gesture
recognition.
Currently, the Python search and recognition code is somewhat slow: with graphics turned on
(for feedback), it is slower than realtime; with graphics off, it is 4–5 times faster than realtime, but
(a)
(b)
Figure 18: Prototype watch hardware. Features on the side of the board illustrated in (a), from the
top left and in clockwise direction are the Bluetooth module, the power/charge switch, battery wires,
reset and action buttons, display connector, and mini-USB connector (for charging and programming). (b) shows (in the same order), the microSD card slot, unpopulated pads for the accelerometer
chip, and the processor. Also visible are programmable LEDs and Bluetooth status LEDs.
this is still very slow when searching a 40 hour long database. I am working with JungSoo Kim, a
MS student in our group, to convert the codebase to C, and to incorporate additional measures to
improve performance, such as coarse-to-fine search using Gaussian pyramids.
5.2.3
Study
In order to validate that creating and searching through an EGL is an effective way to determine a
good set of gestures to use for a mobile interface, I want to conduct a study with interface designers.
Several users will be given wristwatch hardware to record an aggregate data set over a period of
several days. The data will be subdivided into two segments, one to form an EGL and one for
testing purposes.
A number of HCI students will be recruited to design gesture-based interfaces using the system.
A variety of possible mobile interface functions—such as play/pause, volume up/down, or send to
voicemail—will be selected, and participants will be asked to construct a series of gestures to activate these functions. The designers will be divided into a control group and an experimental group;
the control group will design the gesture interfaces without the EGL system, while the experimental
group will use the system.
The gesture interfaces from each group will be evaluated for performance using the non-EGL
segment of the collected data. The data will be used to simulate a user wearing the device in normal
life, without trying to activate any gestures. The number of times that each of the designer’s gestures
are erroneously recognized will be calculated to determine if using the EGL had a positive effect on
the performance of the simulated device.
CHAPTER VI
PLAN FOR COMPLETION
E XPECTED C OMPLETION
TASK
MAGIC (Chapter 5)
Gesture hardware development
Gesture software development
Everyday Gesture Library study
Writeup of paper for publication
TouchWatch (Chapter 4)
Mobile phone-based hardware mockup
Button study software development
Button size validation study
Card dragging software development
“Dialing” software development
Interface study
Writeup of paper for publication
Thesis
Thesis draft
Defense
32
January 2008
January 2008
February 2008
March 2008
March 2008
March 2008
April 2008
May 2008
June 2008
July 2008
August 2008
September 2008
November 2008
REFERENCES
[1] A SHBROOK , D., AUXIER , J., G ANDY, M., and S TARNER , T., “Experiments in interaction
between wearable and environmental infrastructure using the gesture pendant,” in Proc. of
HCII Workshop on Wearable Computing, (New Orleans, LA), 2001.
[2] A SHBROOK , D., C LAWSON , J., LYONS , K., PATEL , N., and S TARNER , T., “Quickdraw:
The impact of mobility and on-body placement on device access time,” in Human Factors in
Computing Systems (CHI) 2008, 2008.
[3] A SHBROOK , D., LYONS , K., and S TARNER , T., “My watch is on the fitts: an investigation
into round touchscreen wristwatches,” in Submitted to Human Factors in Computing Systems
(CHI) 2008, 2008.
[4] B.V., X. T., “Xsens inertial motion sensor.” http://xsens.com, December 2007. http:
//xsens.com.
[5] C ALLAHAN , J., H OPKINS , D., W EISER , M., and S HNEIDERMAN , B., “An empirical comparison of pie vs. linear menus,” in Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems (CHI), 1988.
[6] C AMPBELL , C. and M AGLIO , P., “Supporting notable information in office work,” in CHI
’03: CHI ’03 extended abstracts on Human factors in computing systems, (New York, NY,
USA), pp. 902–903, ACM, 2003.
[7] C HIPCHASE , J., P ERSSON , P., P IIPPO , P., A ARRAS , M., and YAMAMOTO , T., “Mobile
essentials: field study and concepting,” in Designing for User eXperience, 2005.
[8] C UI , Y., C HIPCHASE , J., and I CHIKAWA , F., “A cross culture study on phone carrying and
physical personalization,” in HCI International, 2007.
[9] G ÁBOR B LASK Ó, Cursorless Interaction Techniques for Wearable and Mobile Computing.
PhD thesis, Columbia University, 2007.
[10] G ÁBOR B LASK Ó and F EINER , S., “Evaluation of an eyes-free cursorless numeric entry system for wearable computers,” in Proceedings of 10th IEEE International Symposium on Wearable Computers (ISWC 2006), (Montreux, Switzerland), pp. 21–28, 2006.
[11] G UIMBRETI ÉRE , F. and W INOGRAD , T., “Flowmenu: combining command, text, and data
entry,” in Proceedings of ACM Symposium on User Interface Software and Technology (UIST),
2000.
[12] K IM , J., H E , J., LYONS , K., and S TARNER , T., “The gesture watch: A wireless contactfree gesture based wrist interface,” in Proceedings of 11th IEEE International Symposium on
Wearable Computers (ISWC 2007), 2007.
33
[13] K RISTOFFERSEN , S. and L JUNGBERG , F., ““making place” to make it work: empirical explorations of hci for mobile cscw,” in GROUP ’99: Proceedings of the international ACM SIGGROUP conference on Supporting group work, (New York, NY, USA), pp. 276–285, ACM,
1999.
[14] K URTENBACH , G., The Design and Evaluation of Marking Menus. PhD thesis, Univ. Toronto,
1993.
[15] L IN , M., L UTTERS , W. G., and K IM , T. S., “Understanding the micronote lifecycle: improving mobile support for informal note taking,” in CHI ’04: Proceedings of the SIGCHI
conference on Human factors in computing systems, (New York, NY, USA), pp. 687–694,
ACM Press, 2004.
[16] LYONS , K., S KEES , C., S TARNER , T., S NOECK , C. M., W ONG , B., and A SHBROOK , D.,
“Augmenting conversations using dual-purpose speech,” in Proc. of ACM ACM Symposium on
User Interface Software and Technology (UIST) 2004, (Santa Fe, NM), 2004.
[17] M ANKOFF , J. and A BOWD , G. D., “Cirrin: a word-level unistroke keyboard for pen input,”
in Proceedings of ACM Symposium on User Interface Software and Technology (UIST), 1998.
[18] M ARTIN , T., “Time and time again: parallels in the development of the watch and the wearable
computer,” in Proceedings of ISWC, 2002.
[19] M C G UIRE , R., H ERNANDEZ -R EBOLLAR , J., S TARNER , T., H ENDERSON , V., B RASHEAR ,
H., and ROSS , D., “Towards a one-way american sign language translator,” in Proceedings.
Sixth IEEE International Conference on Automatic Face and Gesture Recognition, (Washington, DC, USA), pp. 620–625, IEEE Computer Society, May 2004.
[20] M ETZGER , C., A NDERSON , M., and S TARNER , T., “Freedigiter: A contact-free device for
gesture control,” in Proceedings of 8th IEEE International Symposium on Wearable Computers
(ISWC 2004), 2004.
[21] M OSCOVICH , T. and H UGHES , J., “Navigating documents with the virtual scroll ring,” in
Proceedings of ACM Symposium on User Interface Software and Technology (UIST), 2004.
[22] M YERS , C. S. and R ABINER , L. R., “A comparative study of several dynamic time-warping
algorithms for connected word recognition,” The Bell System Technical Journal, vol. 60,
pp. 1389–1409, September 1981.
[23] O ULASVIRTA , A., TAMMINEN , S., ROTO , V., and K UORELAHTI , J., “Interaction in 4-second
bursts: the fragmented nature of attentional resources in mobile hci,” in CHI ’05: Proceedings
of the SIGCHI conference on Human factors in computing systems, (New York, NY, USA),
pp. 919–928, ACM, 2005.
[24] PATEL , S., K IENTZ , J., H AYES , G., B HAT, S., and A BOWD , G., “Farther than you may think:
An empirical investigation of the proximity of users to their mobile phones.,” in Ubicomp,
pp. 123–140, 2006.
[25] P ERLIN , K., “Quikwriting: continuous stylus-based text entry,” in Proceedings of ACM Symposium on User Interface Software and Technology (UIST), 1998.
[26] R AGHUNATH , M. T. and NARAYANASWAMI , C., “User interfaces for applications on a wrist
watch,” Personal and Ubiquitous Computing, vol. 6, pp. 17–30, February 2002.
34
[27] RONKAINEN , S., H ÄKKIL Ä , J., K ALEVA , S., C OLLEY, A., and L INJAMA , J., “Tap input as
an embedded interaction method for mobile devices,” in TEI ’07: Proceedings of the 1st international conference on Tangible and embedded interaction, (New York, NY, USA), pp. 263–
270, ACM, 2007.
[28] S.A., T., “Tissot wristwatches.” http://tissot.ch, December 2007. http://tissot.ch.
[29] S CHMIDT, A., G ELLERSEN , H.-W., B EIGL , M., and T HATE , O., “Developing user interfaces
for wearable computers: Don’t stop to point and click,” in Intelligent Interactive Assistance &
Mobile Multimedia Computing, 2000.
[30] S MITH , G. M. and M . C . SCHRAEFEL, “The radial scroll tool: scrolling support for stylusor touch-based document navigation,” in Proceedings of ACM Symposium on User Interface
Software and Technology (UIST), 2004.
[31] S MITH , G., M . C . SCHRAEFEL, and BAUDISCH , P., “Curve dial: eyes-free parameter entry
for guis,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
(CHI), 2005.
[32] S OUKOREFF , R. and M AC K ENZIE , I., “Towards a standard for pointing device evaluation,
perspectives on 27 years of fitts’ law research in hci,” Intl. Journal of Human-Computer Studies, vol. 61, no. 6, 2004.
[33] S TARNER , T., “Visual recognition of american sign language using hidden markov models,”
Master’s thesis, MIT Media Laboratory, Cambridge, MA, February 1995.
[34] S TARNER , T. E., S NOECK , C. M., W ONG , B. A., , and M C G UIRE , R. M., “Use of mobile
appointment scheduling devices,” in Proceedings of the SIGCHI conference on Human factors
in computing systems, ACM Press, 2004.
[35] S TARNER , T. E., S NOECK , C. M., W ONG , B. A., and M C G UIRE , R. M., “Use of mobile
appointment scheduling devices,” in CHI ’04: CHI ’04 extended abstracts on Human factors
in computing systems, (New York, NY, USA), pp. 1501–1504, ACM, 2004.
[36] VADAS , K., PATEL , N., LYONS , K., S TARNER , T., and JACKO , J., “Reading on-the-go: a
comparison of audio and hand-held displays,” in MobileHCI, 2006.
[37] W ILCOX , L., S MITH , I., and B USH , M., “Wordspotting for voice editing and audio indexing,”
in CHI ’92: Proceedings of the SIGCHI conference on Human factors in computing systems,
(New York, NY, USA), pp. 655–656, ACM, 1992.
[38] Z HAO , S., D RAGICEVIC , P., C HIGNELL , M., BALAKRISHNAN , R., and BAUDISCH , P., “Earpod: eyes-free menu selection using touch input and reactive audio feedback,” in Proceedings
of the SIGCHI Conference on Human Factors in Computing Systems (CHI), 2007.

supporting mobile microinteractions

Transcription

Similar documents

Miroku`s Gesture - Phoebus Media Group

VRSCDX V-Rod® 10th Anniversary Edition GASSER Wheel

Biometric Rich Gestures

UNO-TS2 Data Sheet

Slides of the talk - Centre for Digital Music

classic solid auto shirt

Acorn Designs Pocket Journals and Lokta Cards

WinAir 95 - Freudenberg Filtration Technologies

motorsports shirt

technician shirt