List All Projects   List Graduate Projects   List Senior Design Projects   List REU Projects   Other Undergraduate Projects

Project # 5
Inverse Reinforcement Learning for Discrete-Time Linear Systems with Application to Leader–Follower Robotic Arms

Team Lead: Jianan Zou
Other Team Members: Basanta Adhikari
Faculty Advisor(s): Yijing Xie
Department: EE
Project Type: Graduate Student
Title: Inverse Reinforcement Learning for Discrete-Time Linear Systems with Application to Leader–Follower Robotic Arms
Abstract: Learning control objectives from demonstration is essential for enabling robots to acquire complex behaviors from expert guidance. However, many learning from demonstration and inverse reinforcement learning (IRL) methods lack structured control interpretations, making it difficult to deploy the learned objectives in feedback controllers with performance guarantees. Moreover, accurate dynamic models and analytic policy gradients are often unavailable in real robotic systems, motivating a framework that can infer control objectives directly from data. This project develops a model-free inverse reinforcement learning framework for discrete-time linear systems and validates it on leader–follower robotic arms. A model-based approach is first introduced as a theoretical foundation, where the expert policy is assumed to be optimal for a quadratic cost. By exploiting the discrete-time Riccati equation, exact gradients of the feedback gain with respect to the state cost are derived for efficient cost recovery. Building on this, a data-driven off-policy inverse Q-learning algorithm is developed to learn directly from state–control data, where an equivalent feedback controller is recovered and cost parameters are updated via simultaneous perturbation stochastic approximation. Results show that the proposed IOC-based IRL framework can recover task objectives while preserving interpretability and control performance guarantees.
Poster: View Poster #5 (PDF)
Judges Scoring: Submit Evaluation (judges only)