Tactile: keyboard, mice, stylus / tablets (chord keyboards,... Visual: monitors, printers (helmets, see-through, microscreens) Interaction Devices

Tactile: keyboard, mice, stylus / tablets (chord keyboards,... Visual: monitors, printers (helmets, see-through, microscreens) Interaction Devices

Interaction Devices
devices
Tactile: keyboard, mice, stylus / tablets (chord keyboards, 3D gloves)
Visual:
monitors, printers (helmets, see-through, microscreens)
Acoustic: speech synthesis & recognition, music
Smell: burn chemicals, control air flow, in movies across head rest
Taste ?
Neurological Implants, “R U wired?”, Sci-Fi
cochlear implants
Virtual Retinal Display (VRD)
Laser-based projected image
onto the retina
Visual Cortical Implant
1
devices
Keyboards
QWERTY
Dvoark
designed for manual (slow) typewriters
minimizes finger travel distance
QWERTY
Dvoark
Words / minute
150
200
Error
more
less
Dvoark takes a week or so to learn:
Benefits not greater than effort to learn?
Resistance to change?
Chord Keyboards
Several keys are pressed to enter a symbol.
Speed up to 300w/m with high accuracy.
Steep learning curve, constant practice.
E.g.: court recorders, piano keyboards...
2
devices
task performance
.68
18.4
.95
7.9
.45
8.2
q
a
z
w
s
x
1.17
7.2 .41
.54 .54
12.6
e 22.9 21.6 i
.60
d rt
1.3
uy k o
c fg jh , l p
. ;
vb nb
/
.42
space
3
QUERTY
Work Load and Errors
% Error
% Work Load
symbols
common device performance
metrics / analysis
Homing Position
devices
4
Special keys often require "homing" or positioning of hands off
keyboard's "home row".
Homing is a source of error (description) and often requires a change in
visual focus from task to keyboard.
Function keys require homing. Side function keys were operated by
single hand.
Control sequence (shortcuts / accelerator) commands require least
homing like function key use.
Cursor positioning keys vary both home position and key layout across
keyboards.
Pointing Devices
3rd hand - homing problem
devices
5
Direct Pointers: fingers, stylus / pen (lightpen, touchscreen)
+ User selects and manipulates objects on display directly with pointer
device. More natural (touchscreen).
+ Fast positioning
-
Pointing on vertical display causes fatigue.
-
Less accurate, pointing (hand) can block view of display. In-Out
touchscreen positioning can help accuracy.
Touchscreens useful in novice user and environmental critical areas.
automated tellers, information kiosk, factory floor computers
controlling manufacturing, civil engineers on construction sites,
survey workers in the field.
devices
Indirect Pointers (mice, joystick)
-
User positions a "pointer" icon on screen with a pointer device.
The pointer device does not contact the display. After indirectly
positioning icon the user can select and manipulate displayed
objects.
-
Positioning less "natural", a learned skill
+ View not blocked, less fatigue
Mouse is dominant indirect pointer.
Card et.al. showed arrow key faster
than mouse for very short distances only.
Seconds
Indirect pointers have a resolution – sensitivity to movement
("pixels"/inch, ballistic mode)
Distance
4
3
2
1
0
1
2
4
Centimeters
Fitt's law applies to the study of pointing tasks.
Arrow
Mouse
8
6
Keyboard VS. Pointer Devices
devices
7
Direct pointers are faster than keyboard cursor keys in most cases.
Indirect pointers excel at fine resolution position of on screen targets and
far positioning tasks.
Cursor keys excel at low resolution position of on screen targets (form
fill in tasks) and close position tasks.
Cursor keys preferred on tasks that mix text processing (keyboard
required) and positioning
Less 3rd hand homing problem with cursor keys and keyboard
commands.
Indirect pointer are often "preferred" position and selection devices in
"easy to use" oriented software.
Are small mobile devices “easy” or “convenient” to use?
User satisfaction due to closure, control, direct manipulation interface.
Uncommon pointers
foot pointer
devices
8
Feet based pointer devices (no 3rd hand)
Head (eye) based pointers -- infrared headbands, glasses
IR head tracker
Trackir
3D Manipulators
Data Glove: manipulates objects, has sense of contact with object. No
"mass/resistance" feedback.
Joystring: manipulates objects, no sense
of contact with object, "mass/resistance" feedback.
3D “mouse”: eg. 3DConnexion.
Dimensions = (x, y ,z, pitch, yaw, and roll).
devices
Joystring
hand grabs "inverted T"
wires attached to
stepper motors provide
resistance.
9
tactile
feedback
flexing
sensors
tracking
sensor
Dataglove
Interface
board
foot pointer
Pen I/O
devices
10
Natural I/O interface: selection, input
Metaphor to existing tools
Expressive and portable
Smart Paper & Electronic Ink
Smart paper (use awareness in the media)
text OCR convert, auto complete, spell (grammar) check, justify,
evaluate math
draw CAD conversion, snap, stretch...
Electronic writing - pen strokes (vs bitmaps) can be edited, layered
(“post-its”), attached to objects in smart paper.
PenPoint -- eg. pen commands, common edit cmds
devices
11
brackets, selects pieces of text
caret, insert text
check, display options for selected text, objects, icons, tools, etc
circle, opens edit pad to modify text fields and labels
cross out deletes object beneath it
scratch out deletes sfsff
flick, scrolls document in flick direction {up, down, right, left}
pigtail deletes a character
tap selects or invokes
press - hold initiates drag (move)
tap - press initiates drag (copy)
devices
Palm Pilot's graffiti reduced stroke input language
write numbers here
write letters here
Graffiti
Help screen
Division marks
Start
stroke at
heavy dot
Lift
Stylus
here
12
devices
13
devices
Color Vision
3 interacting variables of color vision:
Hue
color
Brightness
intensity (bright - dull)
Saturation
% color in field
14
blue / yellow
green / red
white / black
Opponent process theory of color vision
These colors can't be seen in same patch of light.
They produce shadows and edges.
Avoid use of opponent (opposite) colors.
Color is very useful to have user selected items stand out in a display.
Spatial and temporal representations of blue colors is worse than
other colors.
Selection / Applicability
Color can also be used to indicate whether a menu option is valid in
the current state or not ("greyed options").
Color Usage
devices
Alert / Attention.
Change of color represents change in state (green, yellow, red).
•
•
•
Use few colors that are easy to discriminate
Use warning colors sparingly.
Consistent system wide analysis of color use.
Element Discrimination.
Color provides contrast and improves discrimination. Need high
contrast difference. Contrast a function of luminance or hue.
Category grouping & field definition. Color can help group display
elements and facilitate visual search. Visual search is affected by:
•
•
•
•
number of items
color separation of categories
legibility of coded symbols
relationship between color coding and targets
15
As screen density increases color effect increase.
devices
Color can define visual fields on display - weather maps
Size & Visual Acuity
As number of colors increase size of text should also increase.
Color can't be assumed! Redundantly code display.
Designer's color perception != user's color perception
color & text codes (categorization)
color, size & text
color, size, text & icon ....
Color Memory:
5 - 7 color memory for codes.
Don't tax Working Memory use around 4 colors!
16
Strong color connotations:
red
danger, error, hot, revolution
green
OK, go on, well, alive, healing
yellow
slow down, caution, sun light
Weak color connotations:
blue
fluid / liquid, wet, calm, hidden
black
empty, death, anarchy
brown
earth, warm
Color Preferences
•
•
•
Children prefer warm colors: reds, browns
Adults prefer cool colors: blues, greens
Occupations (degrees) have color associations:
green health
orange engineering
devices
17
Rules of Thumb for Visual Displays
devices
18
No more than 6 colors (including black) should be used on one screen.
Backgrounds should not be brighter than foregrounds. Grid lines should
be half intensity
Do not have extreme color contrasts between foreground and
background colors -- causes afterimages due to rod fatigue.
Use white for critical / important dynamic information (color gun fails)
A monochrome flash of twice intensity is as effective as color use.
Separate significant information on display by size, distance, intensity
or highlighting.
Use screen position consistently. Time / Date info or page number in
same place.
Groups screen elements. Provide title for elements. Hard to title than
grouping is poor.
devices
19
US Government suggested color use in (critical) monitoring systems
Color
State
Result
Flashing red
Emergency
immediate action
red
Alert
corrective action
magenta
Emergency
warning
out of limit indicator
yellow
Advise
Caution, recheck
Blue
Advisory
( use only as
background / filler )
Resolution
devices
Pixels (picture elements) / inch - density measure. For text higher
resolution implies larger font dimensions. (Text same visual size)
Pixel shape (saturation)
Display Fields
Layout (tiled)
center in fields
uncluttered fields
20
Windowed Displays
devices
21
Tiled vs Overlapping Windows.
Titled: Applications with static window sizes and little or no window
manipulation. E.g.: form entry, message windows, popups.
Overlapped: Applications with dynamically sized windows with user
window manipulation.More robust - let user control display rather than
application. E.g.: text or graphic editors.
Experts tend to prefer overlapped (control ,customization).
Novices tend to prefer tiled (full screen / switch apps) when applicable.
Preference
full screen Vs overlapped
mobile devices ?
tiles Vs full screen
Wearable displays
devices
22
Display glasses user sees a full-sized
display floating in front of their eyes.
Projected Displays - displays projected
transparent surfaces: glass windows, cockpits, visors
opaque surfaces: SixthSense, Cave
Enable user to view displays w/o moving eyes in critical environments.
Proposed for displays in cars. (dashboard controls, in car TV / movies
Head Mounted Displays
devices
23
Visual display can appear 360 degrees, or at least always be present
regardless of head position.
HMD w/ trackers can update display based on head movement (inside a
virtual world).
Sound (and possibly smell) can also be presented (3D sound effects).
Complete control of user's visual and acoustic interface. No external
environment interference.
Comfort?
Acoustic Interfaces
devices
24
Speech Synthesis (generation)
+ Good quality - issue of digitized sampling resolution and complexity.
+ Basic speech phonemes can be edited to make comprehensible
speech.
?
Speaking devices: cars, computers, houses...
devices
Speech Recognition
hearing & understanding limited w/o training
Trained systems. User speech commands are sampled and stored for
pattern matching.
Production Systems:
spoken cmd pattern1 :
spoken cmd pattern2 :
:
action1
action2
:
-
Sensitive to surrounding noise
-
Low transference across users
-
High storage overhead for representing spoken commands and
searching command pattern set. (Can be hierarchical in nature).
25
Digital Sound Processing
+ High Quality, actual sound
?
capacity function of sampling
rate / compression
CD R/W provides large capacity
Using Recorded (or Digitized) Sound
Sampling & manipulations (music -rap)
Sound Bites (commercials, film, TV) HCI Applications:
Voice Mail -- Computer Phone surveys - digitized voice
production with user input by touch tones.
Acoustic menu system...
Acoustic Desktop
Sonic Icons: pilots, blind
Desktop becomes a spherical environment.
devices
26
Virtual Reality
devices
27
(Reality) = = the product of our 3 lb. universe (brain)
Reality consists of a person's perception of the world around them. That
perception is the product of their sensory inputs and expectations.
Illusions, "magic" are the interactions of our sensations and
expectations. We see and hear what we want, what we expect.
Current computer controlled interactive devices have the rudimentary
capability to generate a truly artificial reality for the user:
Vision and Acoustic control of environment via helmets, goggles,
headphones, ...
Tactile interaction and feedback via data glove like devices for
hands, feet, body glove ...
Existing, accepted artificial realities: movies, books, television, games,
simulators for training, sensory deprivation, dreams...
Uses of VR, AR (augmented realityu)
devices
28
Multimedia interfaces like interactive CD-ROM for graphics,
animations, and sound, helmets, and data gloves enable complex
adventure/fantasy environments (caves) for entertainment.
Will books and movies become interactive ?
Scientific Visualization - engineers and scientist can move through
dangerous or theoretical worlds.
Boeing Dreamliner first virtual (VR assisted) designed aircraft
Medicine - surgeons walk through accurate 3D representations of
patients gathered w/ CAT or MRI scans. (Robotic/Waldo surgery)
Waldo enhancements for workers in dangerous environments.
Programming as a visual/intuitive process - looking at data and processes
aka Neuromancer. Software walkthrough gets a new meaning!
Is software modeling 2D (UML...) or can it be 3D
3D program visualization
I. Goldman's 3D program visualizer (2002)
devices
29
3D programming language
M.A. Najork S.M. Kaplan, 1991, The CUBE Language
devices
30
Wearable Computers
devices
31
"We try to move our designs closer to the
human being and blend the machine with
the body." - Hideji Takemasa, NEC
Wearable Data Terminal an optical scanner
worn on forearm enables reading bar codes,
OCR, graphics, with writable optical disk and
CD-ROM database computer located in
terminal worn on neck.
The MIThril hardware platform combines
body-worn computation, sensing, and
networking in a clothing-integrated design.
The MIThril software platform is a
combination of user interface elements and
machine learning tools built on the Linux
operating system.
devices
32
"A person's computer should be worn, much
as eyeglasses or clothing are worn, and
interact with the user based on the context of
the situation. With heads-up displays,
unobtrusive input devices, personal wireless
local area networks, and a host of other
context sensing and communication tools, the
wearable computer can act as an intelligent
assistant, whether it be through a
Remembrance Agent, augmented reality, or
intellectual collectives." -- MIThril
annual wearable computing conference urls
http://iswc.gatech.edu/archives.htm
http://www.media.mit.edu/wearables/
http://wearables.gatech.edu/
I, Cyborg
devices
In 1998 Porfessor K. Warwick surgically implant a silicon chip
transponder surgically implanted into his forearm.
http://www.kevinwarwick.com/index.asp
He can be monitored using a signals
emitted by the chip. He could operate
doors, lights, heaters and other computers
without lifting a finger...
In 2002 a 100 electrode array was
implanted into the median nerve fibres of
his left arm. The implant can send signals
back and forth between Warwick's
nervous system and a computer.
This bi-directional functionality was
demonstrated with the aid of Kevin’s
wife Irena and a second, less complex
implant with her nervous system...
33