Description
 Course Title: Applied Optimal Control
 Course Number: EN530.603, Fall 2018
 Time: MW 3:004:15
 Location: Gilman 55
 Professor: Marin Kobilarov
 Contact: marin(at)jhu.edu
 Office Hours: M 1:002:00, Hackerman 117
 Teaching Assistants:
 Gabe Baraban [gbaraba1(at)jhu.edu] and Subhransu Mishra [smishra9(at)jhu.edu]
 TA Office Hours: M 11:0012:00, Location: Hackerman 306
 File Submission (use this link to submit homework code, project presentations, etc…)
The course focuses on the optimal control of dynamical systems subject to constraints and uncertainty by studying analytical and computational methods leading to practical algorithms. Topics include nonlinear optimization, calculus of variations, dynamic programming, linear quadratic (Gaussian) control, numerical trajectory optimization, optimal estimation (e.g. Kalman filtering, batch estimation), stochastic control. The methods and algorithms will be illustrated through implementation of various simulated examples. A class project will involve optimal control and estimation implementation using robotic systems simulated with a physicsbased virtual reality environment. Recommended Course Background: linear algebra, differential equations, basic probability theory; experience with control systems; programming in MATLAB and/or Python.
Objectives
 Nonlinear Optimization Basics
 Calculus of Variations
 Dynamic Programming
 Implementation and Numerical methods
 Optimal filtering for linear and nonlinear systems
 Optimal feedback control with uncertainty
 Stochastic control
 Applications to nontrivial dynamical systems (in simulation)
Reading
There are a number of good textbooks on optimal control and nonlinear optimization. The suggested primary books are especially useful since they contain almost all of course material and are good selfcontained references. The only exception are the last few lectures related to stochastic control and numerical methods. Bertsekas provides a thorough study of nonlinear programming with detailed proofs but also guidelines for implementation with provable performance; Kirk’s book is one of the best introduction to optimal control and calculus of variations, but does not deal with uncertainty; Stengel’s book includes a comprehensive study of both control and estimation but lacks some of the details found in Kirk; Gill, Murray, and Wright book is an excellent reference targeted to more practical implementation and numerical issues. There are, of course, many more great texts on the subject.
 Primary Recommended Textbooks:
 D. Kirk. “Optimal Control Theory: An Introduction, 2004. ISBN: 9780486434841
 Stengel, “Optimal Control and Estimation”, edition: 94, ISBN: 9780486682006
 D. Bertsekas. “Nonlinear Programming”, 1999. ISBN: 9781886529007.
 Additional Recommended Textbooks:
 Bryson and Ho, “Applied Optimal Control”, edition: REV 75, ISBN: 9780891162285
 P. Gill, W. Murray, and M. Wright. “Practical Optimization. Academic Press, 1982. ISBN: 9780122839528
 J. L. Crassidis, J. L. Junkins, “Optimal Estimation of Dynamic Systems”, Second Edition, Chapman & Hall, ISBN13: 9781439839850
Schedule
Week  Topic  Lecture Notes  Code  Assignments 

8/31/2018  Course Overview and Matrix Algebra Basics  Lecture #1  
9/6/2018  Unconstrained Optimization Basics  Lecture #2  lecture2_1.m lecture2_2.m  Homework #1 
9/13/2018  Constrained Optimization Basics  Lecture #3  lecture3_1.m lecture3_2.m  
9/19/2018  Calculus of variations  Lecture #4  Homework #2 

9/20/2018  Continuous Optimal Control + Terminal Constraints  Lecture #5  lecture5_1.m lecture5_2.m lecture5_3.m  
9/27/2018  LQR  Lecture #6  lecture6_1.m lecture6_2.m lecture6_2.m  Homework #3 
10/3/2018  Inequality Constraints  Lecture #7  Homework #4  
10/10/2018  Dynamic Programming Midterm (on 10/17)  Lecture #8  
10/22/2018  Numerical Methods  Lecture #9  car_shooting.m arm_sim.m trajopt_sqp.m trajopt_sqp_car.m  Homework #5 
10/29/2018  cont.  ddp.zip hw7_car_template.m  Homework #6  
10/31/2017  Optimal Estimation  Lecture #10  
11/7/2017  cont.  int_kf_test.m uni_ekf_test.m uni_br_nobs.m plotcov2.m shape_fit.m  Homework #7  
11/26/2017  Stochastic Control  Lecture #11  Project  
12/4/2017  Presentation by Gowtham Garimella  
12/13/2017  Final exam, will be emailed/handed out at 9:00 am in Hackerman 117. Due at 2:00pm in Hackerman 117 or uploaded to website.  
12/14/2017  Project presentations 912 on 12/14 in Gilman 55.  Project reports due 12/17 
Grading Policy
There will be homeworks given every two weeks and due two weeks later (on Wednesday before class). There will be two midterms covering the first half and second half of the semester, and a final covering the whole material. Grades will be determined according to:
Homeworks  30% 
Midterms  30% 
Final  30% 
Project  10% 
Ethics
You can work together with other students to study the material related to homeworks/tests but the submitted work must be entirely your own. For more information, see the guide on “Academic Ethics for Undergraduates” and the Ethics Board web site (http://ethics.jhu.edu).
Late homework will not be accepted without a prior approval from the instructor.