Heuristic Optimization Methods: Simulated Annealing

Transcription

Heuristic Optimization Methods: Simulated Annealing
Zavod za telekomunikacije
Sveučilište u Zagrebu, Hrvatska
Department of Telecommunications
University of Zagreb, Croatia
Heuristic Optimization Methods:
Simulated Annealing
Nina Skorin-Kapov
Masters program
Information and communication technologies
Outline

Simmulated Annealing concept
 Main
idea
 Role of temperature parameter


Pseudocode
Main design issue: Cooling schedule
 Initial
Temperature
 Schedule type ((in)homogeneous)
 Decrement function
 Termination criterion (Final temperature)



SA design elements
Example
Zavod za telekomunikacije
2
Optimization methods
Exact
methods
A*
Approximate
methods
Constraint
programming
Simplexbased for LP
Branch and Bound,
Cut , Price (B&B,
B&C, B&P)
Approximation
algorithms
Heuristic
algorithms
Dynamic
programming
Improvement (meta)heuristics
Constructive heuristics
• Local search
• Greedy algorithms
•Nature inspired
Hybrid methods
• GRASP
•Problem specific
heuristics
Zavod za telekomunikacije
• Tabu search,
•Genetic algorithms,
•Simmulated Annealing,
•Ant colony
3
Simmulated Annealing (SA)




Iterative, single-solution based metaheuristic
Developed in the 1980‟s through the works of S.
Kirkpatrick and V. Cerny
Applies principles of statistical mechanics
Considers the annealing process where a metal is heated
and then slowly cooled to obtain a strong crystalline
structure



the strength depends on the cooling rate;
imperfections appear if the initial temperature is too high or the
cooling rate is too fast
SA is based on the Monte Carlo model which simulates the
energy changes in the system during the cooling process
until it converges to an equilibrium state developed by
Metropolis et al. in 1953
Zavod za telekomunikacije
4
Analogy
Physical system
Optimization problem
System state
Solution
Molecular positions
Decision variables
Energy
Objective function
Ground state
Global optimum
Metastable state
Local optimum
Rapid quenching
Local search
Temperature
Control parameter T
Careful annealing
Simmulated annealing
From: El-Ghazali Talbi, “Metaheuristics: From Design to Implementation”, Wiley 2009
Zavod za telekomunikacije
5
Main idea

Main idea: stochastic search the search can
accept non-improving solutions and moves with
a certain probability and in that way escape
from local optima and delay convergence
 Under


Memoryless (only records incumbent solution)
In contrast to LS and TS, SA does not generate
an entire neighborhood but creates one
neighbor at random and evaluates it
Zavod za telekomunikacije
6
Neighbor selection – most common method

In each iteration, a random neighbor is
generated:
 If it
improves the current solution  accept as the
new current solution
 Improving moves always selected
 If it
does not improve the current solution accept
with a certain probability which depends on:
 Control parameter T
(temperature)
 Their relative fitness: Δf = |f(current solution)-f(neighbor)|
Usually follows
a Boltzman
distribution for
non-improving
neighbors.
Zavod za telekomunikacije
1, for improving neighbor

P(T , f )   f
T

e , for non - improving neighbor
7
Move acceptance

The search escapes local optima by probabilistic
acceptance of non-improving moves

P, proportional to control parameter T

At high temperatures the probability of accepting non-improving
moves is high


At low temperatures the probability of accepting non-improving moves
is low


T=∞, all moves are accepted; random local search
T=0, no non-improving moves are accepted; local search (with first
improving neighbor selection)
P, inversley proportional to the relative fitness Δf=|f(current
solution)-f(neighbor)|

The worse the neighbor, the smaller the selection probability
Law of thermodynamics: at temperature T, the
probability of an increase in energy of magnitude ΔE is:
(k is Boltzmann’s constant)
Zavod za telekomunikacije
P(T , E ) 
 f
e kT
8
Alternative method

Accept improving neighbors also with a certain
probability
 Δf
is not absolute value but:
 For maximization: Δf =
f(current solution)-f(neighbor)
 For minimization: Δf = f(neighbor)-f(current solution)
 Improving
neighbors give negative Δf while nonimproving neighbor give positive Δf
 Acceptance probability:
P(T , f ) 
Zavod za telekomunikacije
1
f
1 e T
9
Solution quality influence
Example: Consider a maximization problem
The current solution x has a fitness f(x)=100
The current temperature T is T=10
2 selection
strategies:
1, improving neighbor

P1    f
T

e , non - improving neighbor
P2 
1
f
1 e T
f(x’)
Δf
P1
P2
10
50
90
100
110
150
200
90
50
10
0
-10
-50
-100
0.0001234098
0.0067379470
0.3678794412
1
1
1
1
0.0001233946
0.0066928509
0.2689414214
0.5000000000
0.7310585786
0.9933071491
0.9999546021
Zavod za telekomunikacije
10
Control parameter T

Control parameter T determines the probability of
accepting non-improving neighbors
 For
larger values of T, the relative fitness of the current
solution and neighbor (Δf) is not as influential
acceptance if a non-improving neighbor does not
depend so much on its quality but on chance, random
Local Search (choose random neighbor)
 If T is very small, the stochasticity of the search
decreases, i..e for e.g. T=0, behaves like first
improving Local Search (never accept a nonimproving
neighbor)

T is analogous to the temperature in the cooling
process
Zavod za telekomunikacije
11
Dependency of selection probability on T

Example:



Consider a minimization problem
The current solution x has a fitness f(x)=13
The neighbor generated in the next iteration x‟ has a fitness of
f(x‟)=23
P(T , f ) 
 f
e T

 ( 2313)
e T
Temperature T
P(T, 10)=e(-10/T)
1010
0.9999999990
100
0.9048374180
20
0.6065306597
10
0.3678794412
5
0.1353352832
1
0.0000453999
Zavod za telekomunikacije

10
eT
12
Role of parameter T

T is decreased during the search (according to a
cooling schedule)i.e. as the search progresses, the
probability of accepting non-improving moves
decreases

Motivation: at the beginning, the search is almost
random (diversify!) and never stays too long in a local
neighborhood (around a local optimum). Eventually,
when we are in a good area, possibly near a/the global
optimum, converge towards local search (intensify!)

There is a guarantee of asymptotic convergence to the global
optimum for infinite iterations
Zavod za telekomunikacije
13
Pseudocode (Min; nbr. selection method 1)
Start Simmulated Annealing
Generate initial solution s ;
0
Select initial temperatu re T ;
0
s  s0 ; s best  s0 ; T  T0
while (! terminatio n criterion)
Select a neighbor s' N(s)
if ( f ( s' )  f ( s )) then s  s'
 f
elseif [( f ( s' )  f ( s )) and ( random[0,1)  e T )] then s  s'
endif
if ( f ( s )  f ( s best )) then s best  s
Update T ,
endwhile
return s best ;
End
Zavod za telekomunikacije
14
Main design issue: Cooling schedule
Cooling schedule
Initial
Temperature
Schedule
type ((in)homogeneous)
Decrement
function
Termination
Zavod za telekomunikacije
criterion (Final temperature)
15
Initial Temperature

Ideally, the search should start so that many, but not
all, non-improving neighbors are accepted (usually
around 40-60%),
First improving
Local search
T0
Initial
temperature T0
Random Local
search
T∞
3 main strategies for setting the Initial Temperature:
 Set T0 high enough to accept all neighbors during initial
phase of search; -: high computational cost; +:easiest.
 Determined experimentally based on the deviation of the
objective function of obtained solutions; several runs
 Initialize T0 to achieve a predetermined acceptance rate
e.g. 40-60%
Zavod za telekomunikacije
16
Cooling schedule types


2 basic types with analogies to (in)homogeneous
Markov chains
Homogeneous cooling schedule:
T
is decreased in each iteration (move) by a very small
amount according to the decrement function
 Simple; most commonly used in practice

Inhomogeneous cooling schedule:
 At
each level of temperature T, multiple solutions are
(can be) explored until an equilibrium is reached, after
which the temperature is reduced according to the
decrement function
Zavod za telekomunikacije
17
Reaching equilibrium state

For inhomogeneous cooling schedules,
reaching the equilibrium state must be defined
 Usually
a sufficient number of moves is defined
according to the size of the problem instance and
neighborhood size


Static strategy: the number of moves to
equilibrium is predetermined (a given
proportion of the neighborhood)
Adaptive strategy: the number of moves
depends on the characteristics of the search
 E.g.
it may not be necessary to reach equilibrium at
all temperatures; equilibrium may be when we reach
an improving move
Zavod za telekomunikacije
18
Decrement function

Trade-off: between solution quality and cooling
speed (faster cooling usually gives poor
solutions but less computational time)
β is a specified constant
T T  
Linear:
Geometric:
T  T ,  [0,1]
Very slow
decrease:
T
T
,
1  T
Logarithmic:
Zavod za telekomunikacije
Titer
T0

log(iter )
Most commonly used.
Usually, α is between
0.5 and 0.99
β is a constant near 0
Too slow for practical
purposes, but has a
convergence proof
19
Termination criteria (final temperature)

Reaching a final temperature Tf (most common)
 In
theory, the search should continue until Tf=0, but in
practice it is sufficient to terminate the search when the
acceptance probability is negligible, e.g. Tf =0.01
 Note: Tf, T0, and the decrement function determine the
number of iterations
 Inhomogeneous case: depends how equlibrium
 Homogeneous case:


Geometric decrement:

Very slow decrease:
N it 
is reached
log T f  log T0
N it 
log
T0  T f
T0T f
Alternative termination criterion: The number of
iterations without improvement of the incumbent
solution
Zavod za telekomunikacije
20
SA design elements









Solution representation (e.g. binary vector)
Initial solution (e.g. random, greedy)
Objective function
Fitness function
Neighborhood definition („move‟ operator)
Termination criterion (final temperature or other)
Initial temperature
Decrement function
(in)homogeneous cooling
 Equilibrium
Zavod za telekomunikacije
state for inhomogeneous
21
Example

Objective and fitness function:
3
2
max f ( x)  x  60 x  900 x  50
x  [0,31], integer


Representation: x in binary form with 5 bits
Initial solution:
1 0 0 1 1

f(s)=2349
Neighborhood (move operator): “1-flip”
Zavod za telekomunikacije
22
Example: Cooling schedule

Initial temperature: T0=100

Homogeneous cooling: reduce T after each
iteration

Termination criterion: Final temperature Tf =
0.01 (NOTE: we will show only the first few
iterations)

Decrement function: geometric
T  T ,   0.9
Zavod za telekomunikacije
23
Example:

Start SA:
Current
solution s
1 0 0 1 1
f(s)=2349
Incumbent
solution sbest
1 0 0 1 1
f(sbest)=2349
Current
temperature T
Zavod za telekomunikacije
T=100
24
Example: T0=100 , the first 6 iterations
Current
temp T
Nbr. x’
flipped f(x’)
bit
(move)
Δf
(or
0)
P(T, Δf)=
e(-Δf/T) or 1
Accept
move?
New x
f(xbest)
100
00011
1
2237
112 0,3262797946
NO
10011
2349
90
10111
3
1177
1172 0,0000022107
NO
10011
2349
81
10010
5
2642
YES
10010
2642
72.9
11010
2
466
10010
2642
65.61
1000
4
3186
10000
3186
59.0
10100
3
2050
10000
3186
Zavod za telekomunikacije
0
1,0000000000
2176 1,08817E-13
0
1,0000000000
1136 0,0000000043
NO
YES
NO
25
Example: T0=100
After 6
iterations:


1 0 0 0 0
f(sbest)=3186
The initial temperature T0=100 seems too small  nonimproving moves are very unlikely to be chosen (the
acceptance probability for non-improving moves is very
small, none have been chosen)


Incumbent
solution sbest
It seems that with this temperature, for many more iterations we
would be stuck in this local optimum s=10000, f(s)=3186
Note: This example has an objective function with 3 local
optima, with a global optimum of 4050 for solution x=10
(binary: 010101)
Let‟s try with T0=500
Zavod za telekomunikacije
26
Example: T0=500
Current
temp T
Nbr. x’
flipped f(x’)
bit
(move)
Δf
P(T, Δf)=
e(-Δf/T) or 1
Accept
move?
New x
f(xbest)
500
00011
1
2237
112
0,7993151344
YES
00011
2349
450
00111
3
3753
0
1,0000000000
YES
00111
3753
405
00110
5
3506
247
0,5434179536
YES
00110
3753
364,5
01110
2
3634
0
1,0000000000
YES
01110
3753
328
01100
4
3938
0
1,0000000000
YES
01100
3938
295,2
01000
3
3922
16
0,9472421256
YES
01000
3938
265,7
01010
4
4050
0
1,0000000000
YES
01010
4050
239,1
01011
5
4021
29
0,8857786733
YES
01011
4050
215,2
11011
1
293
3728 0,0000000300
NO
01011
4050
Zavod za telekomunikacije
27
Example: T0=500
After 9
iterations:
Current
solution s
Incumbent
solution sbest
Current
temperature T

0 1 0 1 1
f(s)=4050
0 1 0 1 1
f(sbest)=4050
T=215.2
Better! The initial temperature T0=500 was high
enough to leave the local optimum
 However,
if the search continued now for another cca
90 iterations (the search continues until T reaches Tf),
all the visited solutions would probably be near the
found optimum (01011) or in its local neighborhood
Zavod za telekomunikacije
28

Re-heating:
 The

search records the temperature at which the
incumbent solution was obtained
 Repeat the search from the recorded temperature until
the final temperature is reached
 Repeat multiple times (like a multi-start algorithm). Since
the search is stochastic, its trajectory in the search space
is different each time

Restricted neighborhoods


Some moves are so unlikely to be accepted that they are ignored to
save computational time
Order of searching

Sampling the neighborhood in a systematic manner (instead of
randomly generating a neighbor in each iteration)
Zavod za telekomunikacije
29
TS vs. SA
Simmulated Annealing
Tabu search
Mechanism to escape local optima:
nonimproving move acceptance ratio
Mechanism to escape local optima:
accept non-improving best neighbor, no
backtracking, tabu list
Intesifies the search around „good‟
solutions
The search always moves forward
(except when specifically applying the
quality/influence dimensions)
Move selection depends on the
quality of the neighbor (and T)
Move selection depends on the
neighborhood and history of the search
Can accept a non-improving move at
any time
Accepts non-improving moves ONLY
when the search is in a local optimum
Stochastic
Deterministic
Iterative single-solution local searchbased with additional parameters:
Temperature , decrement function,…
Iterative single-solution local searchbased with additional parameters: Tabu
list, other memory structures,…
Zavod za telekomunikacije
30