This core ECE course covers digital control systems, Markov decision processes and reinforcement learning algorithms. The first half of the course covers the fundamentals of modern digital control systems, including state space models and their analysis, state variable feedback and the basics of system identification. The second half of the course deals with Markov decision processes with applications in social sensing and communication systems. The topics covered include stochastic dynamic programming, simulation based optimization and reinforcement learning algorithms. Throughout the course, applications in discrete event systems and machine learning will be emphasized.