Modelling saccades slides

Introduction
Why move the eyes?

What we see
What we think we see
Lena, Playboy centerfold, Nov 1972
(standard test image in the field of image processing since 1973)
2
CoSMo 2016 - G. Blohm
Aug 3, 2016
Introduction
Why move the eyes?


Functional role of eye movements: optimize visual processing!

Image stabilization




Image exploration


Saccades re-orient gaze
Resulting constraints for the controller(s)



3
Smooth pursuit of moving objects (low-pass)
Vestibulo-ocular reflex (VOR) compensates for head movements (high-pass)
Opto-kinetic nystagmus (OKN) stabilizes wide-field motion
Minimize retinal error (position)
Minimize retinal image slip (velocity)
Be as fast and accurate as possible (role of predictions/anticipations…)
CoSMo 2016 - G. Blohm
Aug 3, 2016
Introduction
Example: saccade model

Alfred Yarbus, 1967
Adapted from:
Jürgens et al. 1981
Scudder 1988
4
CoSMo 2016 - G. Blohm
Aug 3, 2016
Where to start?
2 potential approaches


Blind approach




Informed approach



5
Straight use of systems identification tools
Model the system based on the measured input-output relationship
= Engineering approach
Model systems components (not the system as a whole)
Incorporate detailed knowledge about physiology, biomechanics etc
= Neuroscience approach
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye/head plants
Eye and neck muscles
properties


Damped spring-mass system
equivalent
Equation of motion of eye ball:
𝑑2𝜃
𝐽 ∙ 2 + 𝐹𝑝 = 𝐹𝑚
𝑑𝑡
Muscle force applied:
𝑑𝑑 𝑅𝑚 𝑑𝐹𝑚
𝐹𝑚 = 𝐹0 − 𝑅𝑚 ∙
−
∙
𝑑𝑑 𝐾𝑠𝑠 𝑑𝑑
6
Robinson (1964), Scudder (2009)
J: moment of inertia
Fp: passive force (muscle tissue)
Fm: active muscle force
Passive muscle/tissue force:
𝑑𝐹𝑝
𝑑2𝜃
𝑑𝑑
𝑅1 𝑅2 ∙ 2 + 𝑅1 𝐾2 + 𝑅2 𝐾1 ∙
+ 𝐾1 𝐾2 ∙ 𝜃 − 𝑅1 + 𝑅2 ∙
𝑑𝑑
𝑑𝑑
𝑑𝑡
𝐹𝑝 =
𝐾1 + 𝐾2
CoSMo 2016 - G. Blohm
Aug 3, 2016
Modelling the eye
Low inertia
Viscous muscles (like elastic bands)
 second-order linear system approximation


x(t )
h1 (τ )
h2 (τ )
y (t )
(leads to terrible expressions!)



7
T1 = 175ms, T2 = 13ms
T1: viscosity of the plant
T2: inertial properties of eye
y (t ) = h1 (t ) ∗ h2 (t ) ∗ x(t )
CoSMo 2016 - G. Blohm
1
(T1s + 1) ⋅ (T2 s + 1)
Aug 3, 2016
Modelling the eye
Testing the model’s impulse response

1
(T1s + 1) ⋅ (T2 s + 1)
angular position (deg)
20
15
10
5
angular velocity (deg/s)
0
8
0
0.1
0.2
0.3
0.4
0
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1
0.6
0.5
time (s)
0.7
0.8
0.9
1
0.5
1000
500
0
-500
CoSMo 2016 - G. Blohm
Aug 3, 2016
Moving the eyes
We need a motor command (phasic) to move the eyes

Eye moves, but gaze cannot be held

angular position (deg)
20
15
10
5
angular velocity (deg/s)
0

9
1
(T1s + 1) ⋅ (T2 s + 1)
T1
0
0.1
0.2
0.3
0.4
0
0.1
0.2
0.3
0.4
0.6
0.7
0.8
0.9
1
0.5
0.6
time (s)
0.7
0.8
0.9
1
0.5
200
100
0
-100
A tonic activation needed to maintain gaze
CoSMo 2016 - G. Blohm
Aug 3, 2016
Gaze holding: the neural integrator
How do we get the tonic portion?

Neural integrator
1
angular position (deg)
20
+
1
(T1s + 1) ⋅ (T2 s + 1)
15
10
5
0
10
s
T1
25
angular velocity (deg/s)

0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0
0.1
0.2
0.4
0.3
time (s)
0.5
0.6
0.7
200
150
100
50
0
CoSMo 2016 - G. Blohm
Aug 3, 2016
Motor command generation
Problem: movement is not monitored...
angular position (deg)
2500
2000
1500
1000
500
0
angular velocity (deg/s)

0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0
0
0.05
0.1
0.15
0.2
time (s)
0.25
0.3
0.35
0.4
15000
10000
5000
1
∆G
11
PG
E *
s
T1
+
1
(T1s + 1) ⋅ (T2 s + 1)
CoSMo 2016 - G. Blohm
Aug 3, 2016
Motor command generation
Solution: resettable integrator
angular position (deg)

Only remaining error e drives
the eyes
15
10
5
0
angular velocity (deg/s)

20
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0
0.05
0.1
0.15
0.2
time (s)
0.25
0.3
0.35
0.4
1500
1000
500
0
RI
1
∆G
_
e
PG
E *
s
T1
+
1
(T1s + 1) ⋅ (T2 s + 1)
Jürgens et al. 1981
Scudder 1988
12
CoSMo 2016 - G. Blohm
Aug 3, 2016
Control modes
13
CoSMo 2016 - G. Blohm
Aug 3, 2016
Control modes

Open-loop control models




control only possible when system is well-defined, i.e. there is a
precise mathematical relationship between input and output
No knowledge of perturbations
No learning possible
Unstable!
Wikipedia
14
CoSMo 2016 - G. Blohm
Aug 3, 2016
Control modes

Feed-forward control models (model predictive control)



Uses an inverse model of the plant physics
Control output only depends on internal (inverse) model of
the plant physics, independently of actual plant reaction
Feed-forward control without feedback = ballistic



Anticipatory regulation required for optimal behaviour
Based on learning
Neural network analogy / implementation: Perceptron
Wikipedia
15
CoSMo 2016 - G. Blohm
Aug 3, 2016
Control modes

Feedback control models (closed-loop)


Uses output history to adjust future commands
Negative feedback



Helps maintaining stability in the system despite of external changes
Reduction of the input that caused the output
Positive feedback


Amplifies the event that caused the output
Auto-excitation: self-enforcing loop
Wikipedia
16
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movement models
17
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Eye and neck muscles
Superior and
inferior recti
Medical-look.com
Medial and lateral recti
Blohm 2004
18
Superior and
inferior obliques
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Vestibulo-ocular reflex
Leigh & Zee 2006
Wikipedia
19
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Vestibulo-ocular reflex
Green & Angelaki 2004




20
Angular velocity from semi-circular canals: ω
Linear acceleration senses by otolith organs: α
VO: vestibular-only neurons
EM: eye movement sensitive neurons
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Saccades:

Brainstem pathways






22
MN: motoneurons
AM: abducens motoneurons
AI: abducens interneurons
EBN: excitatory burst neurons
IBN: inhibitory burst neurons
OPN: omnipause neurons
CoSMo 2016 - G. Blohm
Ramat et al. 2005
Aug 3, 2016
Eye movements

Saccades:

Executive control


Ballistic: Robinson model
Feedback control
Adapted from:
Jürgens et al. 1981
Scudder 1988
Leigh & Zee 2006
23
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Saccades:

Eye-head coordination: first models
Galiana, Lefèvre, Guitton (1992)

24
Freedman 2001, 2008
Cannot model separate gaze and head goals
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Saccades:

Eye-head coordination: new nested control model

25
Daye, Optican, Blohm, Lefèvre (2014)
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Smooth pursuit

Visually-guided pursuit
Blohm 2004
Adapted from Churchland et al. 2003
26
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Smooth pursuit

Anticipatory pursuit
Barnes & Collins 2008
27
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Smooth pursuit

Stochastic pursuit model
Orban de Xivry, Coppe, Blohm, Lefevre 2013
28
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Saccade-pursuit interaction
De Brower et al. 2002
De Brouwer et al. 2001
29
CoSMo 2016 - G. Blohm
Aug 3, 2016
Eye movements

Saccade-pursuit interaction
De Brower et al. 2002
30
CoSMo 2016 - G. Blohm
Aug 3, 2016
Next lecture: Bayesian processes
Next JC: Goossens & Van Opstal, J Neurophysiol 2006
http://www.compneurosci.com/NSCI850.html
31
CoSMo 2016 - G. Blohm
Aug 3, 2016