*K. MATSUDA , A. TAKEMURA , K. MIURA , T. OGAWA , K

An advanced real-time monocular/binocular eye tracking system using a high frame-rate digital camera. Output
DA Converter
TCP/IP
File
Output
Windows
Camera
Windows
IEEE1394b/USB3.0
Camera
Infrared illumination
USB3.0
Lens
mount
Windows
eye
Camera
Infrared illumination
Stimulus display
Stimulus display
monocular monkey/ human
Grasshopper3 GS3U3-41C6NIRC(USB3)
Grasshopper
GRAS-03K2MC(IEEE1394b)
Ai AF Micro-Nikkor
60mm f/2.8D
Lens
eye
2
OGAWA ,
2
KAWANO IR filter
C-Mount ADAPTER
for Nikon
62S PRO1D R-72
Experiments
1. Evalua.on of the System's Accuracy We evaluated system accuracy by using the synthe.c eye. We used 8 calibra.on points (-­‐10,10), (0,10), (10,10), (-­‐10,0), (0,0), (10,0), (-­‐10,-­‐10), (10,-­‐10). We set the eye 25 points, horizontal angle -­‐10 to +10 degrees and ver.cal angle -­‐10 to 10 degrees at intervals of 5degrees. We measured the eyes’ gaze angles for 1 second at each semng. Camera
z
y
+10 [deg]
-10[deg]
+10 [deg]
synthe.c eyes
z
eye
x
-10[deg]
Camera
2. Measuring of Mouse's Eye Movements 3. Measuring of Monkey's Eye Movements We measured monkey's eye movements by our method and the search coil method at the same .me. binocular human
Methods
3. An example of monkey eye movements 6ms
( 2 frames@334Hz)
f (z) =
s(t)v(t + z)
Shift 6ms
Delay of our method.
Left Horizontal angle plot
actual measurement= 0.996 theoretical value+(−0.022)
R^2= 1.000
−1
c
p
o
2
2
p
c
p
o
2
2
The view axis vector in the camera coordinate system There are two way to measure the view axis. The first method [A] treats only the pupil center posi.on. This method is used when the subject's head is fixed .ghtly. The second method [B] treats the pupil center posi.on and the center posi.on of illumina.on's reflec.on on the cornea. This method is used when subject's head is fairly sta.onary. Method A
Method B
(Cx , Cy , Cz ) =
Xp
R
X o Yp
,
Yo
R
,
(Xp
R2
Xo )2
R
(Yp
Yo )2
(Cx , Cy , Cz ) =
Xp Xc Yp Yc
,
,
Rpc
Rpc
Rpc 2
(Xp
Xc ) 2
Rpc
(Yp
Yc ) 2
(Xangle , Yangle ) =
arctan
Cy
Cx
, arctan
Cz
Cz
The ac.ve calibra.on and the target coordinate system When the subject fixates small targets (more than 3 points) that appear on the computer display, the system provides a transi.on matrix for the eye posi.on from the camera coordinate system to the target coordinate system. The target coordinate system
Y
Camera
T T
T T T
T
X
(Tx,Ty,Tz)
T T
T
We ask the subject to fixate each of 9 (more than 3 points)
z
eye
targets displayed on a screen. We measure the subject's view axis
vectors (C1-9) in the camera coordinate system and calculate
transition matrix from these vectors and the respective required
x
view axis vectors (T1-9) in the target coordinate system .
display
n
1
2
3
4
5
6
7
8
9
a
d
g
b
e
h
c
f
i
=
n
Cix 2
Cix Ciy
i=1
n
Cix Ciy
i=1
n
Ciz Cix
i=1
Tx
Ty
Tz
=
a b c
d e f
g h i
1
n
Ciz Cix
i=1
n
i=1
n
i=1
n
i=1
n
Ciy 2
i=1
Cix Tix
i=1
n
Ciy Ciz
Ciy Ciz
Ciz
i=1
n
n
2
Cix Tiy
i=1
n
Ciy Tix
i=1
n
Cix Tiz
i=1
n
Ciy Tiy
i=1
n
Ciz Tix
i=1
n
Ciy Tiz
i=1
n
Ciz Tiy
i=1
Ciz Tiz
i=1
Cx
Cy
Cz
The gaze point in 3D space We calculated the horizontal/ver.cal gaze angle of each eye by processing its video-­‐image. The origin was set on the center of eyes. “HL” is the horizontal gaze angle of the le\ eye and “HR” is that of the right eye. “HL” is posi.ve and “HR” is nega.ve in the figure. C corresponds to the inter-­‐ocular distance, ~60mm, though differences can be detected among individuals. According to the law of sines, the target posi.on of the gaze in X-­‐Z plane can be described as follows. Then we calculated the ver.cal posi.on. “VL” and “VR” is the ver.cal gaze angle of each eye. We used average of them . depth Z
⇡
a=
HL
vertical Y
gaze position
2
⇡
gaze position
b = + HR
y
2
+
+
c
c=⇡ a b
B
A
VL+VR
A
B
C
y
=
z
tan(
)
=
=
HL
2
HR
sin a sin b sin c
b
a
C
Horizontal X
(x,
z)
=
(B
cos
a
, B sin a)
VL+VR
C (0,0)
2
(0,-C/2)
2
(C/2,0)
+
cos a sin b C
sin a sin b
depth Z
z
=
(C
,
C
)
a, b, c: the three angle of the triangle.
sin c
2
sin c
(0,0)
A, B, C: the three sides of the triangle.
Depth [Z position]
actual measurement= 0.966 theoretical value+(11.438)
R^2= 0.999
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−2
●
●
●
●
1500
●
●
●
●
●
●
actual mesurements [deg]
actual mesurements [deg]
●
●
●
●
●
●
●
●
7
6
●
●
●
●
●
5
●
●
●
●
●
4
●
●
●
●
●
●
●
●
3
●
●
●
●
−3
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−4
●
●
●
●
●
−5
●
●
●
●
−6
●
●
●
●
●
●
●
−7
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
1000
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
2
−8
●
●
●
●
500
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
1
−9
●
●
●
●
●
●
●
●
●
●
Results
1.  System’s accuracy 0
1
2
3
4
5
6
7
8
9
●
●
●
●
●
●
●
10
−10
−9
−8
−7
theoretical values [deg]
−6
−5
−4
XZ plot
5
4
4
3
3
2
2
1
●
0
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−1
−2
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●● ●
●
●
● ●
●●
● ●●●●
●●
●● ●
●
●
● ●●●
●
●●
● ●
●
●● ●
●
●●●●
●
●●●
●●
●
●
●●
● ●●
●
●●●
●
●
●●
●
●●●●
●●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●●●
●●● ●
●
●
● ●●
●●●
●●
●
●●●
●
●
●●
●
●● ●
●
●
●
●
●
●
●●●
●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●●
●●
●●
●
● ●●
●
●
●
●
●●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
● ●●●
●●
●
●●●
●
●
●
●
●
●
●
●● ●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
● ●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●
●
●
●●●
●
●
●●●
●
●
●●
●
●
●●
●●●●
●
●
●
●
●●●●
● ●
●●● ●●●●●●●● ●●
●●●●●
●
●●●●●
●
●●
●●
●● ● ●
●
●
●●●
●●
● ● ●
●●●
●●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●●●●●●
●
●●
●
● ●●
● ●
●
●● ●
●●
●
●
●● ●
●
●
●
●●
●● ●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−2
−1
0
500
1000
1500
theoretical values [mm]
YZ plot
5
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−3
theoretical values [deg]
Vertical [Y position] [mm]
5. Measuring of Human Gaze Points in 3D We measured gaze posi.ons in 3-­‐D space, sampling frequency was 500Hz. The subject moved his gaze 5 ways. 1)The center to le\, le\ to center. 2)The center to up-­‐le\, up-­‐le\ to center. 3)The center to up, and up to center. 4)The center to up-­‐right, up-­‐right to center. 5) The center to right, right to center. 100mm
100mm
600mm
100mm
300mm
The target positions from the view point of the human subject.
The positions of the visual objects.
−10
0
●
●
−3
●
1
0
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−1
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●●
●
−2
●●
●
−4
−4
−5
−5
●
●●
●
●●
●
●
●
●
●
●
● ●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●●●
●
●●●
●●●
−3
●
●
●●
●●
●
●
●
●
● ●●
●● ●
●●●
●●
●
●
●●
●
●●
● ●●
●●
●
●
●●
● ● ●
● ●● ●
●● ●
●
●●●●●
● ●● ●
●
●●
●
●
●●●
●
●
● ●●●
●●●
●●
●
●
●●
● ●●●●
●
●
●●
●●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●●
●
●
●
● ●
●
●
●●
●
●●●
●
●●●
●●●●●●●●
●●
●●
●
●●
●
●●●
●
●
●●
●
●
●
●
●●
●●
●●
● ●
●
● ●●●
●
●
●
●●●●●●
●
●
●
●
●●
●
●●
●●
●
● ●
● ●●
●
●●●●
●
●
●
●
●
●
●
●● ●
●●●
●●
●
●
●
●
●
● ●●●
●●●
●
●
●
●
●
●
●
●●●
●●
●
●
●●●
●
●
●
●
●
●●
●●
●
●●●
●
●
●
●●●●●●
●
● ●●●
●●
●
●
●●
●
● ●●
●
●
●
● ●
●
●●●
●●● ●
●
●
●●●●●● ●
●●●●
●
●●
●● ●
●●
●
●●
●●
●
●
● ●
●
●●●
●
●●
●
● ●●●●●
●
●● ●
●
● ● ●
● ●
●● ● ●
●
●●●
● ●●●
●●
●●
●●
●
●●●
●
●●
●
●●
● ● ●●
●●● ●
● ● ●
● ●●● ●●●
●
● ●●● ● ●
●● ●
●●
● ● ●●
●●●
●
● ●
●
●● ●
●●
●
0
500
1000
1500
0
500
Depth [Z position] [mm]
1000
1500
Depth [Z position] [mm]
5. An example of human gaze posi.ons. Gaze posi.ons of a subject (with a 10 points (20ms) moving average filter). Eye movements according to Yarbus (1957)
near to far
far to near
left eye right eye
left eye right eye
100
80
Method A
Head fixed tightly
Horizontal
mean error 0.028 [deg]
sd
0.098 [deg]
60
Vertical
40
B
Method
Head is fairly stationary
Camera
Vertical
mean error 0.096 [deg]
sd
0.071 [deg]
20
0
-150
700
-100
500
0
400
50
Horizontal
Horizontal
mean error -0.042 [deg]
sd
0.138 [deg]
300
100
150
200
Depth
700
●●
●●
●
●
●
●
●
●
●
●
Vertical
mean error 0.047 [deg]
sd
0.139 [deg]
500
coordinate
Target coordinate
Superimposed image
Errors in target coordinate
2. An example of mouse eye movements (OKNs) in the camera coordinate system by method A 30ms
Close up
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●
●
●●●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
● ●
●
●●
● ●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●●
●●
●
●
● ● ●
●● ●
●
●
●
●
●
●●
●●
●
●
●●
●●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
● ●●●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●●
● ●
●
●
●
●●
●
●
●
● ●● ● ● ●
●●
●
●● ●
●
●
●
●
●● ●
●● ●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●● ●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●● ●
●
●
●
●
●
●●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
● ● ●●
●
●
●
●
●
●
●
●
●
● ●●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●● ●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
● ●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●● ●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●●
● ●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●
●
●
●
●
●
●
●●●
●
●
● ●
●
●
●
●●● ●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
● ● ●
● ● ●
●
●
●
●
● ●
●
●
●
●
●
● ●
●
●●
●
●
●
●
●
● ● ●●
●●
●
●
●
●●
●
●
● ●
●
●
●
●●●
●
● ●●●
●
● ● ●● ● ●
●
●
●
●
●●
● ●
●
●
●
●
●
●
● ●●
●
●
●●
● ● ●●
●●
● ● ● ●●
●
●●
● ●
● ● ●●
●
●●
● ●
●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
● ● ●● ●
●
●
● ●
●●
●
●
●●
● ●
●
● ● ●● ●
●
●
●
●
●●
● ●●
●●
●
●
●
●
●● ● ●●
●
●
●
● ●●
●
●
● ● ●●
●●
●
●
●
●●
●●
●
● ●
● ●●●
●
●
●
●
●
●●
●
●● ● ●
●
●●
●●
● ●
●
● ●
●
●
●●
●●
●
● ●●
● ●●●
● ●●
●
●●
●●●
●●
●
●●
●
●
●●
●
●
●
●
●
●● ●
●●
●
●
●●
●
●●
●
●●
●●
●●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●●●
●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
600
400
300
Reference:
Enright JT, Changes in vergence mediated by saccades. J. Physiol. 350: 9-31, 1984
Yarbus AL, Eye movements during changes of the stationary points of fixation. Biophysics 2:
679-683, 1957
600
-50
Depth [mm]
pc
p
Method B.
●
●
●
●
●
●
8
Horizontal [X position] [mm]
2
Method A.
Right Horizontal angle plot
actual measurement= 0.999 theoretical value+(−0.083)
R^2= 1.000
0
●
●
●
●
●
9
4. Evalua.on of the System's Accuracy of Gaze Points in 3D We set le\ and right synthe.c eyes horizontal angle (1,-­‐1), (1.5,-­‐1.5), (2.0,-­‐2.0), (2.5, -­‐2.5), (3.0,-­‐3.0), (3.5,-­‐3.5), (4.0,-­‐4.0), (5.0,-­‐5.0), (6.0, -­‐6.0), (8.0, -­‐8.0), (10.0, 10.0) [deg], ver.cal angle 0 [deg]. We measured le\ and right eyes’ gaze angles and posi.ons for 1 second at each semng. Search coil.
4. Gaze angles and posi.on calculated by the data measured from the synthe.c eyes Le\ and right horizontal angle mean errors are -­‐0.037, -­‐0.084 [deg]. Their standard devia.ons are 0.036,0.030 [deg]. Maximum error was 15.1mm at depth 687.1mm, a\er applied a 10 points (20ms) moving average filter, maximum error decreased. 10
The passive calibra.on When the subject spontaneously moves its eye, the system defines the rota.on center of the pupil, the rota.on radius of the pupil center, the cornea curvature center and the length between the pupil center and the cornea curvature center. α
d y
(Xr,Yr)
y
Pupil
The pupil center (Xp,Yp)
A reflection
β
Rpc
center
pupil
L
R
x
y
(Xc,Yc)
=(Xr-α,Yr-β)
Rpc：The length between
S
x
the pupil center and the
Camera
Infrared
z
z
cornea curvature center
illumination
Pupil
(Xo, Yo)
R：The rotation radius of
pupil
The cornea
The rotation center
R
θ
the pupil center
A reflection
curvature center
x
The rotation center of the pupil center
(Xo, Yo)
(Xc,Yc)
d
Suppose that a cornea is a part of a sphere, we
S
R=
The cornea curvature center(Xc, Yc)
refer to the center of that sphere as a cornea
Mouse eye sample
cos =
S
L
curvature center (Xc,Yc). If the illumination and
The rotation center projected to xy plane exists
The pupil center, the cornea curvature center and rotation
1
camera is located farther enough from the
on
the
extended
lines
of
ellipses’
minor
axes.
L
center of the pupil exist on the same line.
R sin = d
cornea curvature center shifts, the position
between the cornea curvature center and the
(X
X )
(Y
Y )
(Xo,Yo)
reflection
is
always
the
same.
In
this
figure
α
R
=
R
The rotation center of the pupil center on
R
(X
X )
(Y
Y )
and β are fixed.
The rotation radius of the pupil center
captured image.
626.09
actual mesurements [mm]
model
DA Converter
TCP/IP
File
2
MIURA ,
K.
T.
K.
1AIST, Tsukuba, Japan; 2Kyoto Univ., Kyoto, Japan
Introduc.on We developed a new eye tracking system by adop.ng an IEEE-­‐1394b or an USB3.0 digital camera that provides high sensi.vity, high resolu.on, and high frame rate. The system is non-­‐invasive and inexpensive and can be used for mice, monkeys, and humans. Infrared light illuminates the eye and the reflected image of the iris and the black image of the pupil are captured by the camera. The center of the pupil is calculated and tracked over .me. The movement of the head is compensated by using the reflected image of the infrared light.
System outline Stimulus display
Stimulus display
eye
Hot mirror
Infrared illumination
mouse
Camera
IEEE1394b
mouse
1
TAKEMURA ,
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●●●
●●●●
●
●
●
●●
●
●
●
●●
●●
●●
●●
●
●
●●
● ●●●●
●
●●
●
●
●
●
● ●● ● ●
●
●
●●
●
●● ●●
●● ●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●● ●
●
●
●
●●
●
●●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●●
●
●●●
●
●
● ●
●●
●
●● ●
●
●
●
●●
●●●●
●●●
●
●●●
●●●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●● ●
●
●
●● ●
●● ●
●●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
● ●●
● ●
● ●
● ●
● ●
● ●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
100
●
●
center to up−left
●
center to up−right
●
up−left to center
●
up−right to center
●
center to left
●
left to center
●
center to right
●
right to center
Vertical [mm]
*K.
1
MATSUDA , A.
50
0
●
●
●
center to up−left
center to up−right
up−left to center
up−right to center
center to up
up to center
200
−150
−100
−50
0
Horizontal [mm]
50
100
150
200
300
400
500
600
700
Depth [mm]
Conclusion　By using the data measured from the synthe.c eyes, we found that the system can measure the eye movement beder than 1 degree accuracy and delay .me is 6ms at 334Hz (depends on camera capturing sampling rate). This system can be used for mice, monkeys and humans. By using this system, we succeeded in characterizing vergence eye movements of humans when ocular fixa.on shi\ed between two targets placed at different distances in 3-­‐D space. Supported by KAKENHI (24650105).