Dynamic Programming

CPS 616
DYNAMIC PROGRAMMING
8-1
DEFINITION
Dynamic Programming is a general algorithm design technique for solving
problems defined by recurrences with overlapping subproblems
Invented by American mathematician Richard Bellman in the 1950s to solve
optimization problems and later assimilated by CS
Main idea:
• set up a recurrence relating a solution to a larger instance to solutions of
some smaller instances
• solve smaller instances once
• record solutions in a table
• extract solution to the initial instance from that table
EXAMPLE 1: FIBONACCI NUMBERS
•
Recall definition of Fibonacci numbers:
− F(n) = F(n-1) + F(n-2)
− F(0) = 0
− F(1) = 1
• Computing the nth Fibonacci number recursively (top-down):
F(n)
F(n-1)
+
F(n-2)
F(n-2) + F(n-3) + F(n-3) + F(n-4)
Let
DP:
C(n) = # of additions to calculate F(n)
C(0)=0, C(1)=0, C(n) = C(n-1)+C(n-2) + 1, not O(n)!
0
1
1
2
3
5
8
13
21
0
1
2
3
4
5
6
7
8
CPS 616
DYNAMIC PROGRAMMING
8-2
EXAMPLE 2: COIN-ROW PROBLEM
Puzzle:
• There is a row of n coins whose values are some positive integers c₁, c₂,...,cn,
not necessarily distinct. The goal is to pick up the maximum amount of
money subject to the constraint that no two coins adjacent in the initial row
can be picked up.
• E.g.: 5, 1, 2, 10, 6, 2. What is the best selection?
Recurrence Relation
• Let F(n) be the maximum amount that can be picked up from the row of n
coins. To derive a recurrence for F(n), we partition all the allowed coin
selections into two groups:
− those without last coin – the max amount is ?
− those with the last coin -- the max amount is
• Thus we have the following recurrence
− F(n) = max{cn + F(n-2), F(n-1)} for n > 1,
− F(0) = 0, F(1)=c₁
Index i
Coin ci
F(i)
Solution(i)
0
-0
∅
1
5
5
{1}
2
1
3
2
Algorithm:
F(0) = 0
Solution(0) = {}
F(1) = ci
Solution = {1}
for i from 2 to n do
addcoin = ci + F(i-2)
addcoin > F(i-1)
F(i) = addcoin
Solution(i) = Solution(i-2) ∪ {i}
else
F(i) = F(i-1)
Solution(i) = Solution(i-1)
4
10
5
6
6
2
CPS 616
DYNAMIC PROGRAMMING
8-3
EXAMPLE 3: COIN-COLLECTING PROBLEM
Problem
• Several coins are placed
in cells of an n×m board.
• A robot, located in the
upper left cell of the
board, needs to collect as
many of the coins as
possible and bring them
to the bottom right cell.
• On each step, the robot
can move either one cell
to the right or one cell
down from its current
location.
Approach
• Let F(i,j) be the largest number of coins the robot can collect and bring to cell
(i,j) in the ith row and jth column.
• Let P(i,j) be the path that the robot took to collect the maximum coins to bring
to cell (i,j) in the ith row and jth column.
F
P
1
2
3
4
5
6
1
1
1
2
2
3
3
4
4
5
5
2
3
4
5
6
CPS 616
DYNAMIC PROGRAMMING
Recurrence
Encode the distribution of coins on the board into a table C, where
C(i,j) =
1 if there is a coin in cell (i,j)
0 otherwise
If the coins are coming from the left, F(i,j) = F(i,j-1) + C(i,j)
If the coins are coming from above, F(i,j) = F(i-1,j) + C(i,j)
Therefore
F(i, j) = max{F(i-1, j), F(i, j-1)} + C(i,j)
F(0, j) = 0, F(i, 0) = 0
for 1 ≤ i ≤ n, 1 ≤ j ≤ m
Algorithm
F(1,1) =C(1,1)
P(1,1) = [(1,1)]
for j from 2 to m do
F(1,j) = F(1,j-1) + C(1,j)
P(1,j) = P(1,j-1) + (1,j)
for i from 2 to n do
F(i,1) = F(i-1,1) + C(i,1)
P(i,1) = P(i-1,1) + (i,1)
for j from 2 to m do
if F(i-1,j) > F(i,j-1)
F(i,j) = F(i-1,j) + C(i,j)
P(i,j) = P(i-1,j) + (i,j)
else
F(i,j) = F(i,j-1) + C(i,j)
P(i,j) = P(i,j-1) + (i,j)
8-4
CPS 616
DYNAMIC PROGRAMMING
8-5
EXAMPLE 4: KNAPSACK PROBLEM
Problem
• Knapsack of capacity W
• n items of weights w1, w2, ... wn, and values v1, v2, ..vn
• All weights are positive integers.
• Find most valuable subset of items which fit into knapsack
Notation
• Let V(i,w) = value of the most valuable combination of the first i items that fit
into knapsack of capacity w
• Let C(i,w) = contents of the knapsack that results in V(i,w)
• We want to find V(n,W) and C(n,W)
Recursive Analysis
To determine V(i,w), need to decide whether the ith item, which weighs wi, should
be inserted.
There are 3 cases:
• If wi>v then it cannot be inserted: V(i,w)=V(i-1,w), and C(i,w)=C(i-1,w)
• Otherwise, decide whether it is worth adding it: compare
− the best way of adding it: V(i-1,w-wi) +vi
− with the value if it is not added V(i-1,w)
and pick the best
− If it is added, then
C(i,w) = C(i-1,w-wi) ∪ {wi}
− Otherwise,
C(i,w) = C(i-1,w)
Recurrence
𝑉(𝑖 − 1, 𝑗)
𝑖𝑓 𝑗 − 𝑤𝑖 < 0
𝑉(𝑖, 𝑗) = �
max [𝑉(𝑖 − 1, 𝑗), V(i − 1, w − 𝑤𝑖 )] + 𝑣𝑖 otherwise
Set initial conditions as:
V(i,0) =0 V(0,j) = 0, for i,j≥0
CPS 616
DYNAMIC PROGRAMMING
8-6
Solution
0
...
i-1
i
...
n
0
0
...
0
0
j-wi
0
...
j
0
V(i-1,j-wi)
...
W
0
V(i-1,j)
V(i,j)
0
goal
Memory Functions Method
• Comparison of top-down with bottom-up:
− Bottom-up does not repeat work
− Top-down guides calculation of only necessary sub-cases
• Memory Functions method:
− Algorithm is top-down and recursive
− When calculating a sub-case for the first time, save it in table
− At each function call, check table to see if value calculated already
Memory Functions Algorithm
// Assume answers will be stored in array MFK(n,W)
// initialized to -1except for row 0 and column 0 initialised with 0s
Function MFKnapsack(i, j)
if MFK(i, j) ≥ 0 return MFK(i, j)
MFK(i, j) = MFKnapsack(i-1, j)
if wi > j
else
MFK(i, j) = max(MFKnapsack(i-1, j) , MFKnapsack(i-1, j- wi)+ vi )
return MFK(i, j)
Example:
Knapsack of capacity W=5, n=4.
Item Weight Value
i
wi
vi
1
2
$12
2
1
$10
3
3
$20
4
2
$15
Calculate MFKnapsack(4,5)
0
1
2
3
4
0
$0
$0
$0
$0
$0
1
$0
2
$0
3
$0
4
$0
5
$0
CPS 616
DYNAMIC PROGRAMMING
8-7
EXAMPLE 5: WARSHALL'S ALGORITHM
Definitions: Transitive Closure
• Graph Definition: The transitive closure of a directed graph with n vertices is
the graph which shows, for every vertex, all the vertices it can eventually
reach
• Adjacency Matrix Definition: The transitive closure of a directed graph with
n vertices is the n × n boolean matrix T in which element T(i,j) is 1 if there
exists a non-trivial path (i.e. directed path of positive length) from the ith
vertex to the jth vertex, and 0 otherwise.
Example
Original Graph
Transitive Closure
Brute Force Construction
• Algorithm:
− For each vertex i in the graph:

Perform a DFS or BFS traversal of the graph starting at vertex i

For each vertex j encountered during that traversal, set T(i,j) = 1
• Efficiency: n vertices × whole tree traversal starting at each vertex

∈ O(n3) if adjacency matrices are used

∈ O(n×(n+edges)) if adjacency lists are used
CPS 616
DYNAMIC PROGRAMMING
Warshall's Algorithm
• Construct transitive closure T as the last matrix in the sequence of n-by-n
matrices R(0), … , R(k), … , R(n) where
• R(k)[i,j] = 1 iff there is nontrivial path from i to j with only first k vertices
allowed as intermediate
• Note that R(0) = A (adjacency matrix), R(n) = T (transitive closure)
Example
8-8
CPS 616
DYNAMIC PROGRAMMING
8-9
Mechanism
Recurrence
On the k-th iteration, the algorithm determines for every pair of vertices i, j if a
path exists from i and j with just vertices 1,…,k allowed as intermediate
R(k)[i,j] =
R(k-1)[i,j]
or
R(k-1)[i,k] and R(k-1)[k,j] ]
(path using just 1 ,…,k-1)
(paths i →k and k → j
using just 1 ,…,k-1
Matrix Generation
Recurrence implies the following rules for generating R(k) from R(k-1):
• Rule 1 If an element in row i and column j is 1 in R(k-1), it remains 1 in R(k)
• Rule 2 If an element in row i and column j is 0 in R(k-1), it must be changed to
1 in R(k) if and only if the element in its row i and column k and the element in
its column j and row k are both 1’s in R(k-1)
Pseudocode
Analysis
• Time efficiency: Θ(n3)
• Space efficiency: Matrices can be written over their predecessors
CPS 616
DYNAMIC PROGRAMMING
8 - 10
EXAMPLE 6: FLOYD'S ALGORITHM
Problem
In a weighted (di)graph, find shortest paths between every pair of vertices
Example
a
b
c
d
a
b
c
d
Representation
Floyd's Algorithm
• Same idea as Marshall’s algorithm:
• construct solution through series of matrices D(0), …, D(n) using increasing
subsets of the vertices allowed as intermediate
• At the k-th iteration, the algorithm determines shortest paths between every
pair of vertices i, j that use only vertices among 1,…,k as intermediate
• D(k)[i,j] = min {D(k-1)[i,j], D(k-1)[i,k] + D(k-1)[k,j]}
CPS 616
Example
DYNAMIC PROGRAMMING
8 - 11
CPS 616
DYNAMIC PROGRAMMING
8 - 12
Pseudocode
Analysis
• Time efficiency: Θ(n3)
• Space efficiency: Matrices can be written over their predecessors
Actual path
a
b
c
d
a
∅
{
{
{
b
{
c
d
} {
} {
}
}
{
} {
}
∅
} {
}
{
}
∅
} {
} {
}
∅