What is Optimal Control Theory?

What is Optimal Control Theory?


Dynamic Systems: Evolving over time.
Time:
Discrete or continuous.
Optimal way to control a dynamic
system.

Prerequisites:
Calculus, Vectors and Matrices,
ODE&PDE

Applications:
Production, Finance/Economics,
Marketing and others.
Basic Concepts and Definitions

A dynamic system is described by state equation:
where x(t) is state variable, u(t) is control variable.
 The control aim is to maximize the objective function:
Usually the control variable u(t) will be constrained as
follows:


Sometimes, we consider the following constraints:
(1) Inequality constraint
(2) Constraints involving only state variables
(3) Terminal state
where X(T) is reachable set of the state variables at time T.
Formulations of Simple Control Models

Example 1.1
A Production-Inventory Model.
We consider the production and inventory storage of a
given good in order to meet an exogenous demand at
minimum cost.
Table 1.1 The Production-Inventory Model of Example 1.1

Example 1.2
An Advertising Model.
We consider a special case of the Nerlove-Arrow
advertising model.
Table 1.2 The Advertising Model of Example 1.2

Example 1.3 A Consumption Model.
This model is summarized in Table 1.3:
Table 1.3 The Consumption Model of Example 1.3
History of Optimal Control Theory




Calculus of Variations.
Brachistochrone problem: path of least time
Newton, Leibniz, Bernoulli brothers, Jacobi, Bolza.
Pontryagin et al.(1958): Maximum Principle.
Figure 1.1 The Brachistochrone problem
Notation and Concepts Used






= “is equal to” or “is defined to be equal to” or “is
identically equal to.”
:= “is defined to be equal to.”
 “is identically equal to.”
 “is approximately equal to.”
 “implies.”
 “is a member of.”
Let y be an n-component column vector and z be an
m-component row vector, i.e.,
when n = m, we can define the inner product
If
is an m x k matrix and B={bij} is a k x n matrix, C={cij}=AB,
which is an m x n matrix with components:
Differentiating Vectors and Matrices with
respect to Scalars


Let f : E1Ek be a k-dimensional function of a scalar
variable t. If f is a row vector, then
If f is a column vector, then
Differentiating Scalars with respect to
Vectors

If F: En x Em  E1, n 2, m 2, then the gradients Fy
and Fz are defined, respectively as;
Differentiating Vectors with respect to
Vectors

If F: En x Em  Ek, is a k – dimensional vector function, f
either row or column, k 2; i,e;
where fi = fi (y,z), y  En is column vector and z  Em is row
vector, n 2, m 2, then fz will denote the k x m matrix;
fy will denote the k x n matrix

Matrices fz and fy are known as Jacobian matrices.

Applying the rule (1.11) to Fy in (1.9) and the rule (1.12)
to Fz in (1.10), respectively, we obtain Fyz=(Fy)z to be the
n x m matrix
and Fzy = (Fz)y to be the m x n matrix
Product Rule for Differentiation

Let x  En be a column vector and g(x)  En be a row
vector and f(x)  En be a column vector, then
Vector Norm

The norm of an m-component row or column vector z is
defined to be

Neighborhood Nzo of a point is
where  > 0 is a small positive real number.
 A function F(z): Em  E1 is said to be of the order o(z), if

The norm of an m-dimensional row or column vector
function z(t), t  [0, T], is defined to be
Some Special Notation


left and right limits
discrete time (employed in Chapters 8-9)
xk: state variable at time k.
uk: control variable at time k.
k : adjoint variable at time k, k=0,1,2,…,T.
xk:= xk+1- xk.: difference operator.
xk*, uk*, and k, are quantities along an optimal path.


bang function
sat function
impulse control
If the impulse is applied at time t, then we calculate the
objective function J as
Convex Set and Convex Hull



A set D  En is a convex set if  y, z  D,
py +(1-p)z  D, for each p  [0,1].
Given xi  En, i=1,2,…,l, we define y  En to be a convex
combination of xi  En, if  pi  0 such that
The convex hull of a set D  En is
Concave and Convex Function



 : D  E1 defined on a convex set D  En is concave
if for each pair y,z  D and for all p  [0,1],
If  is changed to > for all y,z  D with y  z, and
0<p<1, then  is called a strictly concave function.
If (x) is a differentiable function on the interval
[a,b], then it is concave, if for each pair y,z [a,b],
(z)  (y)+  x (y)(z-y).


If the function  is twice differentiable, then it is concave
if at each point in [a,b],
 xx 0 .
In case x is a vector,  x x needs to be a negative
definite matrix.
If  : D  E1 defined on a convex set D  En is a
concave function, then -  : D  E1 is a convex
function.
Figure 1.2 A Concave Function
Affine Function and Homogeneous Function of
Degree One


: En  E1 is said to be affine, if (x) - (0) is linear.
 : En  E1 is said to be homogeneous of degree one, if
(bx) = b (x), where b is a scalar constant.
Saddle Point


 : En x Em  E1, a point
point of (x,y), if
Note also that
 En x Em is called a saddle
Figure 1.3 An Illustration of a Saddle Point
Linear Independence and Rank of a Matrix




A set of vectors a1,a2,…,an  En is said to be linearly
dependent if  pi,, not all zero, such that
If the only set of pi for which (1.25) holds is p1= p2=
….= pn= 0, then the vectors are said to be linearly
independent.
The rank of an m x n matrix A is the maximum number of
linearly independent columns in A, written as rank (A).
An m x n matrix is of full rank if rank (A) = n.