Stochastic Systems (ECEN 689)

Stochastic models
     – Markov chains
     – Classification and class properties
     – Equilibrium distribution
     – Stability
     – Queueing networks

Dynamic programming
     – Shortest path problem
     – The dynamic programming recursion
     – Principle of optimality
     – Controlled Markov chains
     – Finite horizon cost criterion
     – Discounted and average cost criterion
     – Value iteration, policy iteration, and linear programming

Partially observed systems
     – State estimation
     – Unnormalized distribution
     – Information state
     – Dynamic programming for partially observed systems
     – Bandit problems

Linear quadratic Gaussian systems
     – State estimation
     – Kalman filter
     – Linear Gaussian optimization
     – The separation of estimation and control
     – LQG systems
     – Certainty equivalence