History of Touch Interface

History
of
Touch
Interface
Shiori
Saito
|
Assistant
Researcher
–
CIRO
Project
Creative
Environment
for
Emerging
Electronic
Culture
(CE3C)
Alberta
College
of
Art
and
Design
June
10,
2010
With
the
introduction
of
the
iPhone,
iPod
Touch
and
now
the
iPad
we
might
say
that
we
have
finally
arrived
at
the
multi‐touch
revolution
in
technology.
As
with
most
developments
this
was
not
new.
Since
the
announcement
of
the
iPhone
in
2007,
many
have
asked
Bill
Baxton
about
multi‐touch.
The
reason
is
largely
because
they
assume
that
he
has
been
involved
in
the
topic
for
a
number
of
years.
Yet,
he
responded,
"[m]ulti‐touch
technologies
have
a
long
history.
To
put
it
in
perspective,
the
original
work
undertaken
by
my
team
was
done
in
1984,
the
same
year
that
the
first
Macintosh
computer
was
released,
and
we
were
not
the
first."
1
Indeed
it
has
been
30
years
since
the
invention
of
mouse
by
Engelbart
and
English
in
1965,
and
finally
it
became
ubiquitous
by
the
touch
technology.
History
of
touch
technology
started
to
be
developed
in
the
second
half
of
the
1960s,
but
multi‐touch
technology
was
not
born
until
1982.
Following
is
a
brief
history
of
multi‐touch
technology:
1960s
Early
touch
screen
technology
was
under
development
in
the
second
half
of
the
1960s
at
IBM,
the
University
of
Illinois,
and
Ottawa
Canada.
All
were
single‐touch
with
no
pressure‐sensitivity.2
1972
PLATO
IV
Touch
Screen
Terminal
(Computer‐based
Education
Research
Laboratory,
University
of
Illinois,
Urbana‐Champain).
Source:http://www.billbuxton.com
/PLATO%20IV%20Touch.jpg
1982
In
addition
to
its
touch
features,
PLATO
IV
was
remarkable
for
its
use
of
real‐time
random‐access
audio
playback,
and
the
invention
of
the
flat
panel
2
plasma
display. Flexible
Machine
Interface2
(Nimish
Mehta
,
University
of
Toronto).
The
first
multi‐touch
system
developed.
Consisted
of
a
frosted‐glass
panel
with
local
optical
properties
that
when
viewed
from
behind
with
a
camera
a
black
spot
appeared
whose
size
reflected
amount
of
finger
pressure.
Not
only
used
to
capture
the
shadows
of
fingers,
but
to
capture
and
recognize
shapes
of
objects
as
well.
4
1
Buxton,
Bill.
(2009)
,
Multi‐Touch
Systems
that
I
Have
Known
and
Loved.
http://www.billbuxton.com/multitouchOverview.html
http://en.wikipedia.org/wiki/Plato_computer
3
Mehta,
Nimish
(1982),
A
Flexible
Machine
Interface,
M.A.Sc.
Thesis,
Department
of
Electrical
Engineering,
University
of
Toronto
supervised
by
Professor
K.C.
Smith.
4
Buxton,
Bill.
(2010),
A
Touching
Story:
A
Personal
Perspective
on
the
History
of
Touch
Interfaces
Past
and
Future
2
1983
"Pinching"
Gesture
Technique
Introduced
–
Video
Place
/
Video
Desk
(Myron
Krueger).
Emerged
as
an
early
example
of
augmented
reality
art,
done
by
an
artist
mostly
interested
in
VR.
Source:
http://www.billbuxton.com/myronOval.jpg
1984
1985
1991
A
vision
based
environment
that
responded
to
the
movement
and
gestures
of
people;
therefore,
it
did
not
sense
the
touch,
per
se,
but
this
became
a
significant
inspiration
towards
the
gestural
interface
for
multi‐touch
system
today
since
this
was
implemented
in
a
number
of
configurations,
including
table
and
wall.
Birth
of
Multi‐Touch
Screen
(Developed
by
Bob
Boie,
Bell
Labs,
Murray
Hill
NJ).
The
first
multi‐touch
display
–
the
first
sensor
capable
of
simultaneously
capturing
multiple
touch‐points
on
a
display.
A
multi‐touch
touch
screen,
not
a
table,
which
used
a
transparent
capacitive
array
of
touch
sensors
overlaid
on
a
CRT.
Could
manipulate
graphical
objects
with
fingers
with
excellent
response
time.
Multi‐Touch
Tablet
1
(Input
Research
Group,
University
of
Toronto) Developed
a
touch
tablet
capable
of
sensing
an
arbitrary
number
of
simultaneous
touch
inputs,
reporting
both
location
and
degree
of
touch
for
each.
The
Tablet
used
capacitance,
rather
than
optical
sensing
so
was
thinner
and
Source:
http://www.billbuxton.com/DGPtouch.jpg
much
simpler
than
camera‐based
systems.
Digital
Desk
(Pierre
Wellner,
Rank
Xerox
EuroPARC,
Cambridge)
An
early
front
projection
tablet
top
system
that
used
optical
and
acoustic
techniques
to
sense
both
hands/fingers
as
well
as
paper‐
based
controls
and
data.
Clearly
demonstrated
multi‐touch
concepts
such
as
two
finger
scaling
and
translation
of
graphical
objects.
Source:
http://www.billbuxton.com/ubiVid13sm.jpg
1992
Simon
–
The
world's
First
Smart
Phone
(released
by
IBM
and
Bell
South)
Shown
as
a
concept
product
in
1992.
Released
to
the
public
in
1993.
While
only
a
single‐touch
device,
Simon,
like
the
iPhone,
relied
on
a
touch‐screen
driven
“soft
machine”
user
interface.
5
Source:
http://www.billbuxton.com/simon.jpg
~1998
Fingerworks
(Newark,
Delaware)
http://www.fingerworks.com/
Source:
http://images.appleinsider.com/fingerworks01
2210.003.png
1999
Portfolio
Wall
(Alias|Wavefront,
Toronto
On,
Canada)
Source
http://www.billbuxton.com/PWflick.jpg
2002
Produced
a
range
of
multi‐touch
tablets
with
multi‐touch
sensing
capabilities,
including
the
iGesture
Pad
and
the
TouchStream
keyboard.
Products
were
immediately
discontinued
when
the
company's
assets
were
acquired
by
Apple
Inc.
in
early
2005.
Allowed
images
to
be
sorted,
annotated,
and
presented
in
sequence.
Not
multi‐touch;
however,
its
interface
was
entirely
based
on
finger
touch
gestures,
and
which
are
now
introduced
on
some
touch‐
based
mobile
devices
as
well.
Smart
Skin
Jun
Rekimoto
Sony
Computer
Science
Laboratories
http://www.csl.sony.co.jp/person/rekimoto/smartskin
Adopted
mutual
capacitive
sensing
that
is
also
used
in
iPhone.
Source:
http://www.billbuxton.com/jun.jpg
5
Recognizes
multiple
hand
positions
and
their
shapes
as
well
as
calculates
the
distances
between
the
hands
and
the
surface
by
using
capacitive
sensing
and
a
mesh‐shaped
antenna.
J.R.
Lewis.
(1996),
Reaping
the
benefits
of
modern
usability
evaluation:
The
Simon
Story
2006
Jeff
Han
demonstrated
the
first
FTIR
multi‐touch
at
TED
2006.
Souce:
http://www.gadgetell.com/images/2006/12/te
d.jpg
Introduced
Frustrated
Total
Internal
Reflection
(FTIR)
and
Diffused
Illumination
(DI),
which
have
allowed
for
the
low
cost
development
of
such
surfaces,
have
led
to
a
number
of
technological
and
application
innovations.
InnoVis
Visualization
Laboratory
at
the
University
of
Calgary,
Canada.
Image
from
memory
[en]
code
project
at
InnoVis.
Source:
http://www.utahinrichs.de/memoryencode/im
ages/overview_gallery.jpg
2007
The
lab
is
equipped
with
touch‐sensitive
SMART
boards
and
a
diamond
touch
display.
In
the
context
of
HCI,
lab
members
worked
collaboratively
–
even
interdisciplinary
(art
+
science)
projects.
Apple
unveiled
iPhone
http://www.apple.com/iphone/technology/
Apple
re‐entered
the
mobile‐computing
market
for
the
first
time
in
14
years
(since
Newton
in
1993).
Like
the
Simon
(1992),
a
mobile
phone
with
a
soft
touch‐based
interface.
Source:
http://www.mapds.com.au/newsletters/0807/i
phone_home.gif
Pinching
technique
introduced
by
Krueger
(1983)
was
used
to
zoom
in
or
out
of
a
map
or
photo.
Alias
Portfolio
Wall
(1999)
type
flicking
gestures
were
employed
to
go
through
a
sequence
of
images.
Supported
third‐party
"apps”
development
by
releasing
iPhone
SDK
for
multi‐touch
mobile
contents
including
games,
reference,
GPS
navigation,
social
networking,
and
ads
for
television
shows,
films,
and
celebrities.
Microsoft
unveiled
Microsoft
Surface
http://www.surface.com
Source:
http://www.billbuxton.com/MS_SC_CollabPhot
oApp.jpg
Source:
http://news.cnet.com/i/bto/20081105/percepti
ve_ted4_441x211.jpg
Jeff
Han
Presented
Large
Interactive
Media
Wall
(8x3ft)
Jeff
Han
presented
8
x
3
foot‐long
large
multi‐
touch
wall
at
TED
2007.
Designed
for
presentations
and
broadcast
uses.
World’s
Largest
Multi‐Touch
Wall
(16x8ft)
Obscura
Digital,
Goodby
Silverstein
&
partners
and
HP
@
WSJ
D5
conference
2007
Source:
http://technabob.com/blog/wp‐
content/uploads/2007/06/obscura_multitouch.
jpg
2009
Interactive
table
surface
that
is
capable
of
sensing
multiple
fingers
and
hands
and
identifying
various
objects
and
their
position
on
the
surface.
A
key
indication
of
this
technology
making
the
transition
from
research,
development
and
demo
to
mainstream
commercial
applications.
Interactive
multi‐touch
16'x8'
wall,
produced
by
Obscura
Digital,
Goodby
Silverstein
&
partners
and
HP
for
the
WSJ
D5
conference
2007.
Comes
with
contents
such
as
videos,
photos,
music,
games
and
real
time
RSS
feeds.
Razorfish
Touch
Framework
http://emergingexperiences.com/2009/05/razorfish‐
multi‐touch‐framework/
Designed
to
simplify
the
creation
of
touch‐based
advertising
experiences.
(native
in
Windows
XP,
Vista
and
7)
and
built
on
Windows
Presentation
Foundation
(WPF).
Source:
http://emergingexperiences.com/wp‐
content/uploads/2009/05/razorfashion_teaser1
.jpg.jpg
Works
on
Microsoft
Surface,
touch
walls,
in‐store
kiosks
and
mobile
phones
(for
both
commercial
+
TUIO‐based
custom
hardware).
DIY
Touch
Screen:
TouchKit
from
NOR_/D
Labs
and
Eyebeam
http://labs.nortd.com/touchkit/
Source:
http://zedomax.com/blog/wp‐
content/uploads/2008/08/multi‐touch‐kit.jpg
2010
http://www.apple.com/ipad/
Source:
http://images.smh.com.au/2010/01/28/107476
9/apple‐ipad‐tablet‐device‐420x0.jpg
A
modular
multi‐touch
development
solution
with
the
aim
to
make
multi‐touch
readily
available
in
an
open
source
fashion.
They
provide
source
files
and
allows
the
use
and
appropriation
of
the
code
for
projects.
Apple
released
iPad
Established
a
new
class
of
devices
between
smartphones
and
laptops.
Supports
screen
rotation
in
all
four
orientations
unlike
the
iPhone
and
iPod
touch,
which
work
in
three
orientations.
To
sum
up,
as
we
have
seen
above,
touch
technologies
have
a
long
history
since
1960’s.
The
history
started
with
integration
and
virtualization
of
physical
devices
such
as
mice
and
keyboards.
This
led
to
the
intuitive,
gesture‐based
interface
as
Myron
Krueger
introduced
in
1983.
As
functionalities
were
integrated,
the
types
of
system
vary
from
large‐scale
screen
to
mobile
devices.
Although
the
major
wave
of
multi‐touch
technologies
was
in
2007
from
consumers’
perspective
with
the
iPhone,
2006
was
the
pivotal
year
for
open‐source
developers
when
Jeff
Han
presented
his
low‐cost
multi‐touch
system.
This
shifted
the
status
of
multi‐touch
development
from
closed,
hi‐end
research
to
open,
low‐cost
research,
largely
encouraged
by
a
number
of
technological
and
application
innovations.
The
element
of
touch
is
powerful
and
touch‐based
interfaces
will
continue
to
evolve.
However,
as
Bill
Buxton
states,
“everything—including
touch—is
best
for
something
and
worst
for
something
else”
4,
indeed,
during
the
process
of
evolution,
we
will
witness
more
types
of
sensing
interfaces
wherever
the
senses
are
naturally
situated
and
we
have
use
of
them.