EN530.603.F2014 Applied Optimal Control


  • Course Title: Applied Optimal Control
  • Course Number: EN530.603, Fall 2014
  • Time: MW 3:00-4:15
  • Location: Shriver Hall 104
  • Professor: Marin Kobilarov
    • Contact: marin(at)jhu.edu
    • Office Hours: M 2:00-3:00, Hackerman 117
  • Teaching Assistant: Gowtham Garimella
    • Contact: ggarime1(at)jhu.edu
    • TA Office Hours: M 1:00-2:00, Location: 306 Hackerman

The course focuses on the optimal control of dynamical systems subject to constraints and uncertainty by studying analytical and computational methods leading to practical algorithms. Topics include nonlinear optimization, calculus of variations, dynamic programming, linear quadratic (Gaussian) control, numerical trajectory optimization, optimal estimation (e.g. Kalman filtering, batch estimation), learning-based stochas- tic control. The methods and algorithms will be illustrated through implementation of various simulated examples. A class project will involve optimal control and estimation implementation using robotic systems simulated using the Gazebo robotics virtual reality environment. Recommended Course Background: linear algebra, differential equations, basic probability theory; experience with control systems; programming in MATLAB.


  • Nonlinear Optimization Basics
  • Calculus of Variations
  • Dynamic Programming
  • Implementation and Numerical methods
  • Optimal filtering for linear and nonlinear systems
  • Optimal feedback control with uncertainty
  • Learning-based stochastic control


There are a number of good textbooks on optimal control and nonlinear optimization. The suggested primary books are especially useful since they contain almost all of course material and are good self-contained references. The only exception are the last few lectures related to learning-based control and numerical methods. Bertsekas provides a thorough study of nonlinear programming with detailed proofs but also guidelines for implementation with provable performance; Kirk’s book is one of the best introduction to optimal control and calculus of variations, but does not deal with uncertainty; Stengel’s book includes a comprehensive study of both control and estimation but lacks some of the details found in Kirk; Gill, Murray, and Wright book is an excellent reference targeted to more practical implementation and numerical issues. There are, of course, many more great texts on the subject.

  • Primary Recommended Textbooks:
    • D. Kirk. “Optimal Control Theory: An Introduction, 2004. ISBN: 9780486434841
    • Stengel, “Optimal Control and Estimation”, edition: 94, ISBN: 9780486682006
    • D. Bertsekas. “Nonlinear Programming”, 1999. ISBN: 9781886529007.
  • Additional Recommended Textbooks:
    • Bryson and Ho, “Applied Optimal Control”, edition: REV 75, ISBN: 9780891162285
    • P. Gill, W. Murray, and M. Wright. “Practical Optimization. Academic Press, 1982. ISBN: 9780122839528
    • J. L. Crassidis, J. L. Junkins, “Optimal Estimation of Dynamic Systems”, Second Edition, Chapman & Hall, ISBN-13: 978-1439839850


WeekTopicLecture NotesCodeAssignments
9/3/2014Course Overview and Matrix Algebra BasicsLecture #1
9/8/2014Unconstrained Optimization BasicsLecture #2lecture2_1.m lecture2_2.m
9/10/2014Constrained Optimization BasicsLecture #3lecture3_1.m lecture3_2.mHomework #1
9/17/2014Calculus of variationsLecture #4
9/22/2014Continuous Optimal Control + Terminal ConstraintsLecture #5lecture5_1.m lecture5_2.m lecture5_3.mHomework #2
9/29/2014LQRLecture #6lecture6_1.m lecture6_2.m lecture6_2.m
10/8/2014Inequality ConstraintsLecture #7Homework #3
10/15/2014Review and Midterm
10/16/2014Dynamic ProgrammingLecture #8
10/22/2014Numerical MethodsLecture #9car_shooting.m arm_sim.m ddp.zip ddp_pnt_obst.m trajopt_sqp.m trajopt_sqp_car.mHomework #4
11/3/2014Optimal EstimationLecture #10
11/10/2014cont.int_kf_test.m uni_ekf_test.m uni_br_nobs.m plotcov2.m shape_fit.mHomework #5
11/17/2014Stochastic ControlLecture #11
11/24/2014Thanksgivings Holiday
12/1/2014Second Midterm on 12/1
12/8/2014reading period
12/15/2014Final exam, 12/16, will be emailed/handed out at 12:00 in Hackerman 117. Due at 4:00pm in Hackerman 117. Project presentations 4-5:30pm on 12/19 in Hackerman B17.Project reports due 12/19

Grading Policy

There will be homeworks given every two weeks and due two weeks later (on Wednesday before class). There will be two midterms covering the first half and second half of the semester, and a final covering the whole material. Grades will be determined according to:

Homeworks 30%
Midterms 30%
Final 30%
Project 10%


You can work together with other students to study the material related to homeworks/tests but the submitted work must be entirely your own. For more information, see the guide on “Academic Ethics for Undergraduates” and the Ethics Board web site (http://ethics.jhu.edu).

Late homework will not be accepted without a prior approval from the instructor.