HW 4

Math 565
Fritz Keinert
Homework 4
due Thursday, Apr 9, 2015
For each problem that uses Matlab or some other tool, you should hand in a printout
of the relevant script or function file(s), or a transcript of your interactive session, plus
whatever outputs or plots are requested. Put the problems in the proper order, and
label all printouts clearly. The final output should have full accuracy (format long);
intermediate results can be shorter, if you want.
1. The function
f (x) = (x2 − x21 )2 + (1 − x1 )2
has its minimum at (1, 1). Starting with the simplex (0, 0), (2, 0), (0, 2), do three steps
of the Nelder-Mead method (algorithm 9.5). Hand calculation may be easier than programming the algorithm.
Hint: There are 5 possibilities at each step (reflection, expansion, inside contraction,
outside contraction, shrinking). A different one of these will occur at each step.
(10)
2. The Rosenbrock function
f (x) =
1
100(x2 − x21 )2 + (1 − x1 )2
2
has the correct form f = 21 krk2 for a least squares problem.
(a) Find r and J. Verify that ∇f = J T r.
(b) Find J T J and compare to ∇2 f . Verify that close to the solution, J T J and ∇2 f
are almost the same.
(c) Run the Gauss-Newton method (with α = 1, no line search) with initial guesses
(1.2, 1.2) and (−1.2, 1).
Hint: Just take your Newton method from HW 2, and replace ∇2 f by J T J.
(d) (2 points extra credit). For some reason this algorithm converges in 2 steps, no
matter what the initial guess is. Explain why.
Note: I haven’t figured this out myself yet. It may be pretty hard.
(10)
3. (Nocedal, problem 12.15) Consider the constrained optimization problem
2
3
minimize x1 −
+ (x2 − t)4
2
subject to 1 − x1 − x2 ≥ 0
1 − x1 + x2 ≥ 0
1 + x1 − x2 ≥ 0
1 + x1 + x2 ≥ 0
(a) For what values of the parameter t does the point x∗ = (1, 0)T satisfy the KKT
conditions?
(b) Show that when t = 1 there is a point x∗ where only the first constraint is active,
and which satisfies the KKT conditions. Find the coordinates of this point, and the
corresponding value of λ1 .
(10)
2
Math 565
—
Homework 4
—
due Thursday, Apr 9, 2015
4. (Nocedal, problem 12.19) Consider the constrained optimization problem
minimize − 2x1 + x2
subject to (1 − x1 )3 − x2 ≥ 0,
x2 + 0.25x21 − 1 ≥ 0.
The optimal solution is x∗ = (0, 1)T , where both constraints are active.
(a) Do the LICQ hold at this point?
(b) Are the KKT conditions satisfied at this point?
(c) Write down the sets F(x∗ ) and C(x∗ , λ∗ ).
(d) Are the necessary second-order conditions satisfied? Are the sufficient secondorder conditions satisfied?
(10)
5. (a) Use Matlab routine fmincon to solve problem 3, with t = 1, numerically.
Start with x0 = (0, 0). Print out the optimal x, f (x) and λ.
(b) Same for problem 4, with starting guess x0 = (−2, 2).
Hint: you should get the same results that you got by hand in problems 3 and 4.
(10)
6. (Extra Credit) (Nocedal, problem 12.3) Does the optimization problem
minimize (x2 + 100)2 + 0.01x21
subject to x2 − cos x1 ≥ 0
have a finite or infinite set of local solutions?
(2)