Publications
21.
Prajapat, Manish; Lahr, Amon; Köhler, Johannes; Krause, Andreas; Zeilinger, Melanie N.
Towards Safe and Tractable Gaussian Process-Based MPC: Efficient Sampling within a Sequential Quadratic Programming Framework Proceedings Article Forthcoming
In: Forthcoming, (Accepted at the 2024 Conference on Decision and Control (CDC)).
@inproceedings{prajapat_towards_2024,
title = {Towards Safe and Tractable Gaussian Process-Based MPC: Efficient Sampling within a Sequential Quadratic Programming Framework},
author = {Manish Prajapat and Amon Lahr and Johannes Köhler and Andreas Krause and Melanie N. Zeilinger},
doi = {10.48550/arXiv.2409.08616},
year = {2024},
date = {2024-09-13},
urldate = {2024-09-13},
abstract = {Learning uncertain dynamics models using Gaussian process~(GP) regression has been demonstrated to enable high-performance and safety-aware control strategies for challenging real-world applications. Yet, for computational tractability, most approaches for Gaussian process-based model predictive control (GP-MPC) are based on approximations of the reachable set that are either overly conservative or impede the controller's safety guarantees. To address these challenges, we propose a robust GP-MPC formulation that guarantees constraint satisfaction with high probability. For its tractable implementation, we propose a sampling-based GP-MPC approach that iteratively generates consistent dynamics samples from the GP within a sequential quadratic programming framework. We highlight the improved reachable set approximation compared to existing methods, as well as real-time feasible computation times, using two numerical examples.},
note = {Accepted at the 2024 Conference on Decision and Control (CDC)},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
Learning uncertain dynamics models using Gaussian process~(GP) regression has been demonstrated to enable high-performance and safety-aware control strategies for challenging real-world applications. Yet, for computational tractability, most approaches for Gaussian process-based model predictive control (GP-MPC) are based on approximations of the reachable set that are either overly conservative or impede the controller's safety guarantees. To address these challenges, we propose a robust GP-MPC formulation that guarantees constraint satisfaction with high probability. For its tractable implementation, we propose a sampling-based GP-MPC approach that iteratively generates consistent dynamics samples from the GP within a sequential quadratic programming framework. We highlight the improved reachable set approximation compared to existing methods, as well as real-time feasible computation times, using two numerical examples.
22.
Simpson, Léo; Xie, Jing; Asprion, Jonas; Scattolini, Riccardo
A Learning-based Model Predictive Control Scheme with Application to Temperature Control Units Proceedings Article
In: 2024 IEEE Conference on Control Technology and Applications (CCTA), pp. 675-680, IEEE, Newcastle upon Tyne, United Kingdom, 2024, ISSN: 2768-0770.
@inproceedings{xie2024learningbased,
title = {A Learning-based Model Predictive Control Scheme with Application to Temperature Control Units},
author = {Léo Simpson and Jing Xie and Jonas Asprion and Riccardo Scattolini
},
url = {https://arxiv.org/abs/2402.05606},
doi = {10.1109/CCTA60707.2024.10666571},
issn = {2768-0770},
year = {2024},
date = {2024-09-11},
urldate = {2024-09-11},
booktitle = {2024 IEEE Conference on Control Technology and Applications (CCTA)},
pages = {675-680},
publisher = {IEEE},
address = {Newcastle upon Tyne, United Kingdom},
abstract = {Temperature control is a complex task due to its often unknown dynamics and disturbances. This paper explores the use of Neural Nonlinear AutoRegressive eXogenous (NNARX) models for nonlinear system identification and model predictive control of a temperature control unit. First, the NNARX model is identified from input-output data collected from the real plant, and a state-space representation with known measurable states consisting of past input and output variables is formulated. Second, a tailored model predictive controller is designed based on the trained NNARX network. The proposed control architecture is experimentally tested on the temperature control units manufactured by Tool-Temp AG. The results achieved are compared with those obtained using a PI controller and a linear MPC. The findings illustrate that the proposed scheme achieves satisfactory tracking performance while incurring the lowest energy cost among the compared controllers.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Temperature control is a complex task due to its often unknown dynamics and disturbances. This paper explores the use of Neural Nonlinear AutoRegressive eXogenous (NNARX) models for nonlinear system identification and model predictive control of a temperature control unit. First, the NNARX model is identified from input-output data collected from the real plant, and a state-space representation with known measurable states consisting of past input and output variables is formulated. Second, a tailored model predictive controller is designed based on the trained NNARX network. The proposed control architecture is experimentally tested on the temperature control units manufactured by Tool-Temp AG. The results achieved are compared with those obtained using a PI controller and a linear MPC. The findings illustrate that the proposed scheme achieves satisfactory tracking performance while incurring the lowest energy cost among the compared controllers.
23.
Gao, Yunfan; Messerer, Florian; van Duijkeren, Niels; Diehl, Moritz
Stochastic Model Predictive Control with Optimal Linear Feedback for Mobile Robots in Dynamic Environments Proceedings Article
In: 8th IFAC Conference on Nonlinear Model Predictive Control NMPC 2024, pp. 153-158, IFAC-PapersOnLine, 2024.
@inproceedings{24_gao_stochasticmpc,
title = {Stochastic Model Predictive Control with Optimal Linear Feedback for Mobile Robots in Dynamic Environments},
author = {Yunfan Gao and Florian Messerer and Niels van Duijkeren and Moritz Diehl},
doi = {https://doi.org/10.1016/j.ifacol.2024.09.024},
year = {2024},
date = {2024-09-03},
urldate = {2024-09-03},
booktitle = {8th IFAC Conference on Nonlinear Model Predictive Control NMPC 2024},
volume = {58},
number = {18},
pages = {153-158},
publisher = {IFAC-PapersOnLine},
abstract = {Robot navigation around humans can be a challenging problem since human movements are hard to predict. Stochastic model predictive control (MPC) can account for such uncertainties and approximately bound the probability of a collision to take place. In this paper, to counteract the rapidly growing human motion uncertainty over time, we incorporate state feedback in the stochastic MPC. This allows the robot to more closely track reference trajectories. To this end the feedback policy is left as a degree of freedom in the optimal control problem. The stochastic MPC with feedback is validated in simulation experiments and is compared against nominal MPC and stochastic MPC without feedback. The added computation time can be limited by reducing the number of additional variables for the feedback law with a small compromise in control performance.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Robot navigation around humans can be a challenging problem since human movements are hard to predict. Stochastic model predictive control (MPC) can account for such uncertainties and approximately bound the probability of a collision to take place. In this paper, to counteract the rapidly growing human motion uncertainty over time, we incorporate state feedback in the stochastic MPC. This allows the robot to more closely track reference trajectories. To this end the feedback policy is left as a degree of freedom in the optimal control problem. The stochastic MPC with feedback is validated in simulation experiments and is compared against nominal MPC and stochastic MPC without feedback. The added computation time can be limited by reducing the number of additional variables for the feedback law with a small compromise in control performance.
24.
Leeman, Antoine P.; Köhler, Johannes; Messerer, Florian; Lahr, Amon; Diehl, Moritz; Zeilinger, Melanie N.
Fast System Level Synthesis: Robust Model Predictive Control Using Riccati Recursions Proceedings Article
In: 8th IFAC Conference on Nonlinear Model Predictive Control NMPC 2024, IFAC-PapersOnLine, 2024.
@inproceedings{leeman_fast_2024,
title = {Fast System Level Synthesis: Robust Model Predictive Control Using Riccati Recursions},
author = {Antoine P. Leeman and Johannes Köhler and Florian Messerer and Amon Lahr and Moritz Diehl and Melanie N. Zeilinger},
url = {https://doi.org/10.48550/arXiv.2401.13762},
doi = {10.1016/j.ifacol.2024.09.027},
year = {2024},
date = {2024-09-01},
urldate = {2024-02-07},
booktitle = {8th IFAC Conference on Nonlinear Model Predictive Control NMPC 2024},
volume = {58},
number = {18},
publisher = {IFAC-PapersOnLine},
abstract = {System Level Synthesis (SLS) enables improved robust MPC formulations by allowing for joint optimization of the nominal trajectory and controller. This paper introduces a tailored algorithm for solving the corresponding disturbance feedback optimization problem. The proposed algorithm builds on a recently proposed joint optimization scheme and iterates between optimizing the controller and the nominal trajectory while converging q-linearly to an optimal solution. We show that the controller optimization can be solved through Riccati recursions leading to a horizon-length, state, and input scalability of O(N2(n3x+n3u)) for each iterate. On a numerical example, the proposed algorithm exhibits computational speedups of order 10 to 10^3 compared to general-purpose commercial solvers.
},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
System Level Synthesis (SLS) enables improved robust MPC formulations by allowing for joint optimization of the nominal trajectory and controller. This paper introduces a tailored algorithm for solving the corresponding disturbance feedback optimization problem. The proposed algorithm builds on a recently proposed joint optimization scheme and iterates between optimizing the controller and the nominal trajectory while converging q-linearly to an optimal solution. We show that the controller optimization can be solved through Riccati recursions leading to a horizon-length, state, and input scalability of O(N2(n3x+n3u)) for each iterate. On a numerical example, the proposed algorithm exhibits computational speedups of order 10 to 10^3 compared to general-purpose commercial solvers.
25.
Allamaa, Jean Pierre; Patrinos, Panagiotis; Auweraer, Herman Van; Son, Tong Duy
Learning-Based NMPC Adaptation for Autonomous Driving Using Parallelized Digital Twin Journal Article
In: IEEE Transactions on Control Systems Technology, vol. Early Access, pp. 1-16, 2024, ISSN: 1063-6536, (Preprint: https://doi.org/10.48550/arXiv.2402.16645).
@article{allamaa2024lbMPC,
title = {Learning-Based NMPC Adaptation for Autonomous Driving Using Parallelized Digital Twin},
author = {Jean Pierre Allamaa and Panagiotis Patrinos and Herman Van Auweraer and Tong Duy Son},
doi = {10.1109/TCST.2024.3437163},
issn = {1063-6536},
year = {2024},
date = {2024-08-14},
urldate = {2024-05-29},
journal = {IEEE Transactions on Control Systems Technology},
volume = {Early Access},
pages = {1-16},
abstract = {In this work, we focus on the challenge of transferring an autonomous driving (AD) controller from simulation to reality (Sim2Real). We propose a data-efficient method for online and on-the-fly adaptation of parametrizable control architectures such that the target closed-loop performance is optimized while accounting for uncertainties such as model mismatches, changes in the environment, and task variations. The novelty of the approach resides in leveraging black-box optimization enabled by executable digital twins (xDTs) for data-driven parameter calibration through derivative-free methods to directly adapt the controller in real time (RT). The xDTs are augmented with domain randomization (DR) for robustness and allow for safe parameter exploration. The proposed method requires a minimal amount of interaction with the real world as it pushes the exploration toward the xDTs. We validate our approach through real-world experiments, demonstrating its effectiveness in transferring and fine-tuning a nonlinear model predictive control (NMPC) with nine parameters, in under 10 min. This eliminates the need for hours-long manual tuning and lengthy machine learning training and data collection phases. Our results show that the online adapted NMPC directly compensates for the Sim2Real gap and avoids overtuning in simulation. Importantly, a 75% improvement in tracking performance is achieved, and the Sim2Real gap over the target performance is reduced from a factor of 876 to 1.033.},
note = {Preprint: https://doi.org/10.48550/arXiv.2402.16645},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In this work, we focus on the challenge of transferring an autonomous driving (AD) controller from simulation to reality (Sim2Real). We propose a data-efficient method for online and on-the-fly adaptation of parametrizable control architectures such that the target closed-loop performance is optimized while accounting for uncertainties such as model mismatches, changes in the environment, and task variations. The novelty of the approach resides in leveraging black-box optimization enabled by executable digital twins (xDTs) for data-driven parameter calibration through derivative-free methods to directly adapt the controller in real time (RT). The xDTs are augmented with domain randomization (DR) for robustness and allow for safe parameter exploration. The proposed method requires a minimal amount of interaction with the real world as it pushes the exploration toward the xDTs. We validate our approach through real-world experiments, demonstrating its effectiveness in transferring and fine-tuning a nonlinear model predictive control (NMPC) with nine parameters, in under 10 min. This eliminates the need for hours-long manual tuning and lengthy machine learning training and data collection phases. Our results show that the online adapted NMPC directly compensates for the Sim2Real gap and avoids overtuning in simulation. Importantly, a 75% improvement in tracking performance is achieved, and the Sim2Real gap over the target performance is reduced from a factor of 876 to 1.033.
26.
Zhang, Shuhao; Bos, Mathis; Vandewal, Bastiaan; Decré, Wilm; Gillis, Joris; Swevers, Jan
Robustified Time-optimal Collision-free Motion Planning for Autonomous Mobile Robots under Disturbance Conditions Proceedings Article
In: 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 14258-14264, IEEE, Yokohama, Japan, 2024, ISBN: 979-8-3503-8457-4.
@inproceedings{lirias4141698,
title = {Robustified Time-optimal Collision-free Motion Planning for Autonomous Mobile Robots under Disturbance Conditions},
author = {Shuhao Zhang and Mathis Bos and Bastiaan Vandewal and Wilm Decré and Joris Gillis and Jan Swevers},
url = {https://kuleuven.limo.libis.be/discovery/fulldisplay?docid=lirias4141698&context=SearchWebhook&vid=32KUL_KUL:Lirias&search_scope=lirias_profile&adaptor=SearchWebhook&tab=LIRIAS&query=any,contains,LIRIAS4141698&offset=0},
doi = {10.1109/ICRA57147.2024.10610134},
isbn = {979-8-3503-8457-4},
year = {2024},
date = {2024-08-08},
urldate = {2024-02-07},
booktitle = {2024 IEEE International Conference on Robotics and Automation (ICRA)},
pages = {14258-14264},
publisher = {IEEE},
address = {Yokohama, Japan},
abstract = {This paper presents a robustified time-optimal motion planning approach for navigating an Autonomous Mobile Robot (AMR) from an initial state to a terminal state without colliding with obstacles, even when subjected to disturbances, which are modeled as random process noise and measurement noise. The approach iteratively solves the robustified problem by incorporating updated state-dependent safety margins for collision avoidance, the evolution of which is derived separately from the robustified problem. Additionally, a strategy for selecting an alternative terminal state to reach is introduced, which comes into play when the desired terminal state becomes infeasible considering the disturbances. Both of these contributions are integrated into a robustified motion planning and control pipeline, the efficacy of which is validated through simulation experiments.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper presents a robustified time-optimal motion planning approach for navigating an Autonomous Mobile Robot (AMR) from an initial state to a terminal state without colliding with obstacles, even when subjected to disturbances, which are modeled as random process noise and measurement noise. The approach iteratively solves the robustified problem by incorporating updated state-dependent safety margins for collision avoidance, the evolution of which is derived separately from the robustified problem. Additionally, a strategy for selecting an alternative terminal state to reach is introduced, which comes into play when the desired terminal state becomes infeasible considering the disturbances. Both of these contributions are integrated into a robustified motion planning and control pipeline, the efficacy of which is validated through simulation experiments.
27.
Schulz, Felix; Hoffman, Jasper; Zhang, Yuan; Boedecker, Joschka
Learning When to Trust the Expert for Guided Exploration in RL Workshop
2024, (ICML 2024 Workshop: Foundations of Reinforcement Learning and Control -- Connections and Perspectives).
@workshop{schulz2024learning,
title = {Learning When to Trust the Expert for Guided Exploration in RL},
author = {Felix Schulz and Jasper Hoffman and Yuan Zhang and Joschka Boedecker },
url = {https://openreview.net/forum?id=QkTANn4mRa},
year = {2024},
date = {2024-08-07},
urldate = {2025-08-07},
abstract = {Reinforcement learning (RL) algorithms often rely on trial and error for exploring environments, leading to local minima and high sample inefficiency during training. In many cases, leveraging prior knowledge can efficiently construct expert policies, e.g. model predictive control (MPC) techniques. However, the expert might not be optimal and thus, when used as a prior, might introduce bias that can harm the control performance. Thus, in this work, we propose a novel RL method based on a simple options framework that only uses the expert to guide the exploration during training. The exploration is controlled by a learned high-level policy that can decide to follow either an expert policy or a learned low-level policy. In that sense, the high-level skip policy learns when to trust the expert for exploration. As we aim at deploying the low-level policy without accessing the expert after training, we increasingly regularize the usage of the expert during training, to reduce the covariate shift problem. Using different environments combined with potentially sub-optimal experts derived from MPC or RL, we find that our method improves over sub-optimal experts and significantly improves the sample efficiency.},
note = {ICML 2024 Workshop: Foundations of Reinforcement Learning and Control -- Connections and Perspectives},
keywords = {},
pubstate = {published},
tppubtype = {workshop}
}
Reinforcement learning (RL) algorithms often rely on trial and error for exploring environments, leading to local minima and high sample inefficiency during training. In many cases, leveraging prior knowledge can efficiently construct expert policies, e.g. model predictive control (MPC) techniques. However, the expert might not be optimal and thus, when used as a prior, might introduce bias that can harm the control performance. Thus, in this work, we propose a novel RL method based on a simple options framework that only uses the expert to guide the exploration during training. The exploration is controlled by a learned high-level policy that can decide to follow either an expert policy or a learned low-level policy. In that sense, the high-level skip policy learns when to trust the expert for exploration. As we aim at deploying the low-level policy without accessing the expert after training, we increasingly regularize the usage of the expert during training, to reduce the covariate shift problem. Using different environments combined with potentially sub-optimal experts derived from MPC or RL, we find that our method improves over sub-optimal experts and significantly improves the sample efficiency.
28.
Messerer, Florian; Baumgärtner, Katrin; Nurkanovic, Armin; Diehl, Moritz
Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems Journal Article
In: Nonlinear Analysis: Hybrid Systems, vol. 53, pp. 101499, 2024, ISSN: 1751-570X.
@article{MESSERER2024101499,
title = {Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems},
author = {Florian Messerer and Katrin Baumgärtner and Armin Nurkanovic and Moritz Diehl},
url = {https://www.sciencedirect.com/science/article/pii/S1751570X24000360},
doi = {https://doi.org/10.1016/j.nahs.2024.101499},
issn = {1751-570X},
year = {2024},
date = {2024-08-01},
urldate = {2023-08-14},
journal = {Nonlinear Analysis: Hybrid Systems},
volume = {53},
pages = {101499},
abstract = {We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinuous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinuous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.
29.
Cecchin, Leonardo; Trachte, Adrian; Fagiano, Lorenzo; Diehl, Moritz
Real-time prediction of human-generated reference signals for advanced digging control Proceedings Article
In: pp. 496–501, IEEE, Bari, IT, 2024.
@inproceedings{cecchin_real-time_2024,
title = {Real-time prediction of human-generated reference signals for advanced digging control},
author = {Leonardo Cecchin and Adrian Trachte and Lorenzo Fagiano and Moritz Diehl},
doi = {10.1109/CASE59546.2024.10711371},
year = {2024},
date = {2024-08-01},
pages = {496–501},
publisher = {IEEE},
address = {Bari, IT},
abstract = {In the realm of excavator control, advanced techniques, such as Model Predictive Control (MPC) and two-degrees-of-freedom structures (feedforward plus feedback), proved to have great potential for enhancing efficiency and performance. These methods rely on the knowledge of future reference, which is often pre-defined, to optimize the system behavior as a function of it. However, this assumption fails in applications where a human operator chooses the reference at runtime, such as in the case of non-autonomous digging operations. To cope with this problem, we study different approaches to use the collected data of human-generated reference signals to learn a predictive model of the operator commands. The considered methods are function approximation techniques based on Kriging, Set-Membership, and LSTM Neural Networks. We summarize the principles and the implementation of each method, and compare their performance using an experimental data-set of operations from a real-world excavator, where four operator-defined reference signals are predicted.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In the realm of excavator control, advanced techniques, such as Model Predictive Control (MPC) and two-degrees-of-freedom structures (feedforward plus feedback), proved to have great potential for enhancing efficiency and performance. These methods rely on the knowledge of future reference, which is often pre-defined, to optimize the system behavior as a function of it. However, this assumption fails in applications where a human operator chooses the reference at runtime, such as in the case of non-autonomous digging operations. To cope with this problem, we study different approaches to use the collected data of human-generated reference signals to learn a predictive model of the operator commands. The considered methods are function approximation techniques based on Kriging, Set-Membership, and LSTM Neural Networks. We summarize the principles and the implementation of each method, and compare their performance using an experimental data-set of operations from a real-world excavator, where four operator-defined reference signals are predicted.
30.
Cecchin, Leonardo; Ohtsuka, Toshiyuki; Trachte, Adrian; Diehl, Moritz
Model Predictive Controller for Hydraulic Cylinders with Independent Metering Control Valves Proceedings Article
In: IFAC-PapersOnLine, pp. 281–287, IFAC, Kyoto, JP, 2024.
@inproceedings{cecchin_model_2024,
title = {Model Predictive Controller for Hydraulic Cylinders with Independent Metering Control Valves},
author = {Leonardo Cecchin and Toshiyuki Ohtsuka and Adrian Trachte and Moritz Diehl},
url = {https://www.sciencedirect.com/science/article/pii/S240589632401423X},
doi = {10.1016/j.ifacol.2024.09.044},
year = {2024},
date = {2024-08-01},
booktitle = {IFAC-PapersOnLine},
volume = {58},
pages = {281–287},
publisher = {IFAC},
address = {Kyoto, JP},
series = {18},
abstract = {Hydraulic cylinders are pivotal components in various industrial, construction, and off-highway applications, where efficient actuation is crucial for reducing energy consumption, minimizing heat generation, and extending components’ lifespan. The integration of Independent Metering Control, a valve topology allowing five valves to independently control the flow, represents a significant advancement in enhancing hydraulic systems’ performance. However, the lack of a reliable and flexible control solution remains a challenge. In this paper, we present the implementation of nonlinear Model Predictive Control, using a favorable model formulation and a state-of-the-art solver (acados). We show how it can deliver close-to-optimal performance with real-time capabilities, addressing the current gap in achieving efficient control for hydraulic cylinders with Independent Metering Control.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Hydraulic cylinders are pivotal components in various industrial, construction, and off-highway applications, where efficient actuation is crucial for reducing energy consumption, minimizing heat generation, and extending components’ lifespan. The integration of Independent Metering Control, a valve topology allowing five valves to independently control the flow, represents a significant advancement in enhancing hydraulic systems’ performance. However, the lack of a reliable and flexible control solution remains a challenge. In this paper, we present the implementation of nonlinear Model Predictive Control, using a favorable model formulation and a state-of-the-art solver (acados). We show how it can deliver close-to-optimal performance with real-time capabilities, addressing the current gap in achieving efficient control for hydraulic cylinders with Independent Metering Control.
31.
Lahr, Amon; Tronarp, Filip; Schmidt, Nathanael Bosch Jonathan; Hennig, Philipp; Zeilinger, Melanie N.
Probabilistic ODE Solvers for Integration Error-Aware Numerical Optimal Control Proceedings Article
In: Proceedings of the 6th Annual Learning for Dynamics & Control Conference (L4DC), pp. 1018–1032, PMLR, 2024.
@inproceedings{lahr_probabilistic_2024,
title = {Probabilistic ODE Solvers for Integration Error-Aware Numerical Optimal Control},
author = {Amon Lahr and Filip Tronarp and Nathanael Bosch Jonathan Schmidt and Philipp Hennig and Melanie N. Zeilinger},
url = {https://proceedings.mlr.press/v242/lahr24a.html
https://proceedings.mlr.press/v242/lahr24a/lahr24a.pdf},
year = {2024},
date = {2024-07-15},
urldate = {2024-02-07},
booktitle = {Proceedings of the 6th Annual Learning for Dynamics & Control Conference (L4DC)},
volume = {242},
pages = {1018--1032},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
abstract = {Appropriate time discretization is crucial for real-time applications of numerical optimal control, such as nonlinear model predictive control. However, if the discretization error strongly depends on the applied control input, meeting accuracy and sampling time requirements simultaneously can be challenging using classical discretization methods. In particular, neither fixed-grid nor adaptive-grid discretizations may be suitable, when they suffer from large integration error or exceed the prescribed sampling time, respectively. In this work, we take a first step at closing this gap by utilizing probabilistic numerical integrators to approximate the solution of the initial value problem, as well as the computational uncertainty associated with it, inside the optimal control problem (OCP). By taking the viewpoint of probabilistic numerics and propagating the numerical uncertainty in the cost, the OCP is reformulated such that the optimal input reduces the computational uncertainty insofar as it is beneficial for the control objective. The proposed approach is illustrated using a numerical example, and potential benefits and limitations are discussed.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Appropriate time discretization is crucial for real-time applications of numerical optimal control, such as nonlinear model predictive control. However, if the discretization error strongly depends on the applied control input, meeting accuracy and sampling time requirements simultaneously can be challenging using classical discretization methods. In particular, neither fixed-grid nor adaptive-grid discretizations may be suitable, when they suffer from large integration error or exceed the prescribed sampling time, respectively. In this work, we take a first step at closing this gap by utilizing probabilistic numerical integrators to approximate the solution of the initial value problem, as well as the computational uncertainty associated with it, inside the optimal control problem (OCP). By taking the viewpoint of probabilistic numerics and propagating the numerical uncertainty in the cost, the OCP is reformulated such that the optimal input reduces the computational uncertainty insofar as it is beneficial for the control objective. The proposed approach is illustrated using a numerical example, and potential benefits and limitations are discussed.
32.
Hoffman, Jasper; Clausen, Diego Fernandez; Brosseit, Julien; Bernhard, Julian; Esterle, Klemens; Werling, Moritz; Karg, Michael; Boedecker, Joschka
PlanNetX: Learning an efficient neural network planner from MPC for longitudinal control Proceedings Article
In: Proceedings of the 6th Annual Learning for Dynamics & Control Conference, pp. 1214-1227, PMLR, 2024.
@inproceedings{hoffmann_plannetx_2024,
title = {PlanNetX: Learning an efficient neural network planner from MPC for longitudinal control},
author = {Jasper Hoffman and Diego Fernandez Clausen and Julien Brosseit and Julian Bernhard and Klemens Esterle and Moritz Werling and Michael Karg and Joschka Boedecker},
url = {https://proceedings.mlr.press/v242/hoffmann24a.html
https://proceedings.mlr.press/v242/hoffmann24a/hoffmann24a.pdf},
year = {2024},
date = {2024-07-15},
urldate = {2024-07-15},
booktitle = {Proceedings of the 6th Annual Learning for Dynamics & Control Conference},
pages = {1214-1227},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
abstract = {Model predictive control (MPC) is a powerful, optimization-based approach for controlling dynamical systems. However, the computational complexity of online optimization can be problematic on embedded devices. Especially, when we need to guarantee fixed control frequencies. Thus, previous work proposed to reduce the computational burden using imitation learning (IL) approximating the MPC policy by a neural network. In this work, we instead learn the whole planned trajectory of the MPC. We introduce a combination of a novel neural network architecture PlanNetX and a simple loss function based on the state trajectory that leverages the parameterized optimal control structure of the MPC. We validate our approach in the context of autonomous driving by learning a longitudinal planner and benchmarking it extensively in the CommonRoad simulator using synthetic scenarios and scenarios derived from real data. Our experimental results show that we can learn the open-loop MPC trajectory with high accuracy while improving the closed-loop performance of the learned control policy over other baselines like behavior cloning.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Model predictive control (MPC) is a powerful, optimization-based approach for controlling dynamical systems. However, the computational complexity of online optimization can be problematic on embedded devices. Especially, when we need to guarantee fixed control frequencies. Thus, previous work proposed to reduce the computational burden using imitation learning (IL) approximating the MPC policy by a neural network. In this work, we instead learn the whole planned trajectory of the MPC. We introduce a combination of a novel neural network architecture PlanNetX and a simple loss function based on the state trajectory that leverages the parameterized optimal control structure of the MPC. We validate our approach in the context of autonomous driving by learning a longitudinal planner and benchmarking it extensively in the CommonRoad simulator using synthetic scenarios and scenarios derived from real data. Our experimental results show that we can learn the open-loop MPC trajectory with high accuracy while improving the closed-loop performance of the learned control policy over other baselines like behavior cloning.
33.
Pas, Pieter; Løwenstein, Kristoffer Fink; Bernardini, Daniele; Patrinos, Panagiotis
Exploiting Parallelism in a QPALM-based Solver for Optimal Control Workshop
2024, (RSS 2024 workshop: Frontiers of optimization for robotics).
@workshop{PasQPALMOCP,
title = {Exploiting Parallelism in a QPALM-based Solver for Optimal Control },
author = {Pieter Pas and Kristoffer Fink Løwenstein and Daniele Bernardini and Panagiotis Patrinos},
year = {2024},
date = {2024-07-14},
urldate = {2024-07-14},
note = {RSS 2024 workshop: Frontiers of optimization for robotics},
keywords = {},
pubstate = {published},
tppubtype = {workshop}
}
34.
Wang, Renzi; Bodard, Alexander; Schuurmans, Mathijs; Patrinos, Panagiotis
EM++: A parameter learning framework for stochastic switching systems Working paper
2024.
@workingpaper{wang2024em++,,
title = {EM++: A parameter learning framework for stochastic switching systems},
author = {Renzi Wang and Alexander Bodard and Mathijs Schuurmans and Panagiotis Patrinos},
url = {https://doi.org/10.48550/arXiv.2407.16359
},
year = {2024},
date = {2024-07-01},
abstract = {This paper proposes a general switching dynamical system model, and a custom majorization-minimization-based algorithm EM++ for identifying its parameters. For certain families of distributions, such as Gaussian distributions, this algorithm reduces to the well-known expectation-maximization method. We prove global convergence of the algorithm under suitable assumptions, thus addressing an important open issue in the switching system identification literature. The effectiveness of both the proposed model and algorithm is validated through extensive numerical experiments.},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
This paper proposes a general switching dynamical system model, and a custom majorization-minimization-based algorithm EM++ for identifying its parameters. For certain families of distributions, such as Gaussian distributions, this algorithm reduces to the well-known expectation-maximization method. We prove global convergence of the algorithm under suitable assumptions, thus addressing an important open issue in the switching system identification literature. The effectiveness of both the proposed model and algorithm is validated through extensive numerical experiments.
35.
Zhang, Yuan; Yang, Shaohui; Ohtsuka, Toshiyuki; Jones, Colin; Boedecker, Joschka
Latent Linear Quadratic Regulator for Robotic Control Tasks Working paper
2024, (RSS 2024 Workshop on Koopman Operators in Robotics).
@workingpaper{zhang2024latent,
title = {Latent Linear Quadratic Regulator for Robotic Control Tasks},
author = {Yuan Zhang and Shaohui Yang and Toshiyuki Ohtsuka and Colin Jones and Joschka Boedecker},
url = {https://arxiv.org/abs/2407.11107},
year = {2024},
date = {2024-07-01},
urldate = {2024-07-01},
booktitle = {RSS 2024 Workshop on Koopman Operators in Robotics},
abstract = {Model predictive control (MPC) has played a more crucial role in various robotic control tasks, but its high computational requirements are concerning, especially for nonlinear dynamical models. This paper presents a latent linear quadratic regulator (LaLQR) that maps the state space into a latent space, on which the dynamical model is linear and the cost function is quadratic, allowing the efficient application of LQR. We jointly learn this alternative system by imitating the original MPC. Experiments show LaLQR's superior efficiency and generalization compared to other baselines.},
note = {RSS 2024 Workshop on Koopman Operators in Robotics},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
Model predictive control (MPC) has played a more crucial role in various robotic control tasks, but its high computational requirements are concerning, especially for nonlinear dynamical models. This paper presents a latent linear quadratic regulator (LaLQR) that maps the state space into a latent space, on which the dynamical model is linear and the cost function is quadratic, allowing the efficient application of LQR. We jointly learn this alternative system by imitating the original MPC. Experiments show LaLQR's superior efficiency and generalization compared to other baselines.
36.
Løwenstein, Kristoffer Fink; Bernardini, Daniele; Patrinos, Panagiotis
QPALM-OCP: A Newton-type Proximal Augmented Lagrangian tailored for Quadratic Programs arising in Model Predictive Control Journal Article
In: IEEE Control Systems Letters, 2024, ISSN: 2475-1456, (Submitted to the 63rd IEEE Conference on Decision and Control 2024 (CDC)).
@article{Lowenstein2024QPALMOCP,
title = {QPALM-OCP: A Newton-type Proximal Augmented Lagrangian tailored for Quadratic Programs arising in Model Predictive Control},
author = {Kristoffer Fink Løwenstein and Daniele Bernardini and Panagiotis Patrinos},
doi = {10.1109/LCSYS.2024.3410638},
issn = {2475-1456},
year = {2024},
date = {2024-06-06},
urldate = {2024-04-02},
journal = {IEEE Control Systems Letters},
abstract = {In Model Predictive Control (MPC) fast and reliable Quadratic Programming (QP) solvers are of fundamental importance. The inherent structure of the subsequent Optimal Control Problems (OCPs) can lead to substantial performance improvements if exploited. Therefore, we present a structure-exploiting proximal augmented Lagrangian based solver extending the general-purpose QP-solver QPALM. Our solver relies on semismooth Newton iterates to solve the inner sub-problem while directly accounting for the OCP structure via efficient and sparse matrix factorizations. The matrices to be factorized depends on the active set and therefore low-rank factorization updates can be employed like in active-set methods resulting in cheap iterates. We benchmark our solver against other state-of-the-art QP-solvers and our algorithm compare favorably against these solvers},
note = {Submitted to the 63rd IEEE Conference on Decision and Control 2024 (CDC)},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In Model Predictive Control (MPC) fast and reliable Quadratic Programming (QP) solvers are of fundamental importance. The inherent structure of the subsequent Optimal Control Problems (OCPs) can lead to substantial performance improvements if exploited. Therefore, we present a structure-exploiting proximal augmented Lagrangian based solver extending the general-purpose QP-solver QPALM. Our solver relies on semismooth Newton iterates to solve the inner sub-problem while directly accounting for the OCP structure via efficient and sparse matrix factorizations. The matrices to be factorized depends on the active set and therefore low-rank factorization updates can be employed like in active-set methods resulting in cheap iterates. We benchmark our solver against other state-of-the-art QP-solvers and our algorithm compare favorably against these solvers
37.
Reiter, Rudolf; Ghezzi, Andrea; Baumgärtner, Katrin; Hoffman, Jasper; McAllister, Robert D; Diehl, Moritz
AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control Working paper
2024.
@workingpaper{reiter2024ac4mpc,
title = {AC4MPC: Actor-Critic Reinforcement Learning for Nonlinear Model Predictive Control},
author = {Rudolf Reiter and Andrea Ghezzi and Katrin Baumgärtner and Jasper Hoffman and Robert D McAllister and Moritz Diehl },
url = {https://doi.org/10.48550/arXiv.2406.03995},
year = {2024},
date = {2024-06-06},
abstract = {Ac{MPC} and ac{RL} are two powerful control strategies with, arguably, complementary advantages. In this work, we show how actor-critic ac{RL} techniques can be leveraged to improve the performance of ac{MPC}. The ac{RL} critic is used as an approximation of the optimal value function, and an actor roll-out provides an initial guess for primal variables of the ac{MPC}. A parallel control architecture is proposed where each ac{MPC} instance is solved twice for different initial guesses. Besides the actor roll-out initialization, a shifted initialization from the previous solution is used. Thereafter, the actor and the critic are again used to approximately evaluate the infinite horizon cost of these trajectories. The control actions from the lowest-cost trajectory are applied to the system at each time step. We establish that the proposed algorithm is guaranteed to outperform the original ac{RL} policy plus an error term that depends on the accuracy of the critic and decays with the horizon length of the ac{MPC} formulation. Moreover, we do not require globally optimal solutions for these guarantees to hold. The approach is demonstrated on an illustrative toy example and an ac{AD} overtaking scenario.},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
Ac{MPC} and ac{RL} are two powerful control strategies with, arguably, complementary advantages. In this work, we show how actor-critic ac{RL} techniques can be leveraged to improve the performance of ac{MPC}. The ac{RL} critic is used as an approximation of the optimal value function, and an actor roll-out provides an initial guess for primal variables of the ac{MPC}. A parallel control architecture is proposed where each ac{MPC} instance is solved twice for different initial guesses. Besides the actor roll-out initialization, a shifted initialization from the previous solution is used. Thereafter, the actor and the critic are again used to approximately evaluate the infinite horizon cost of these trajectories. The control actions from the lowest-cost trajectory are applied to the system at each time step. We establish that the proposed algorithm is guaranteed to outperform the original ac{RL} policy plus an error term that depends on the accuracy of the critic and decays with the horizon length of the ac{MPC} formulation. Moreover, we do not require globally optimal solutions for these guarantees to hold. The approach is demonstrated on an illustrative toy example and an ac{AD} overtaking scenario.
38.
Frey, Jonathan; Gao, Yunfan; Messerer, Florian; Lahr, Amon; Zeilinger, Melanie N.; Diehl, Moritz
Efficient Zero-Order Robust Optimization for Real-Time Model Predictive Control with Acados Proceedings Article
In: 2024 European Control Conference (ECC), IEEE, Stockholm, Sweden, 2024, ISBN: 978-3-9071-4410-7.
@inproceedings{frey_efficient_2023,
title = {Efficient Zero-Order Robust Optimization for Real-Time Model Predictive Control with Acados},
author = {Jonathan Frey and Yunfan Gao and Florian Messerer and Amon Lahr and Melanie N. Zeilinger and Moritz Diehl},
doi = {10.23919/ECC64448.2024.10591148},
isbn = {978-3-9071-4410-7},
year = {2024},
date = {2024-06-03},
urldate = {2023-12-18},
booktitle = {2024 European Control Conference (ECC)},
publisher = {IEEE},
address = {Stockholm, Sweden},
abstract = {Robust and stochastic optimal control problem (OCP) formulations allow a systematic treatment of uncertainty, but are typically associated with a high computational cost. The recently proposed zero-order robust optimization (zoRO) algorithm mitigates the computational cost of uncertainty-aware MPC by propagating the uncertainties separately from the nominal dynamics. This paper details the combination of zoRO with the real-time iteration (RTI) scheme and presents an efficient open-source implementation in acados, utilizing BLASFEO for the linear algebra operations. In addition to the scaling advantages posed by the zoRO algorithm, the efficient implementation drastically reduces the computational overhead, and, combined with an RTI scheme, enables the use of tube-based MPC for a wider range of applications. The flexibility, usability and effectiveness of the proposed implementation is demonstrated on two examples. On the practical example of a differential drive robot, the proposed implementation results in a tenfold reduction of computation time with respect to the previously available zoRO implementation.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Robust and stochastic optimal control problem (OCP) formulations allow a systematic treatment of uncertainty, but are typically associated with a high computational cost. The recently proposed zero-order robust optimization (zoRO) algorithm mitigates the computational cost of uncertainty-aware MPC by propagating the uncertainties separately from the nominal dynamics. This paper details the combination of zoRO with the real-time iteration (RTI) scheme and presents an efficient open-source implementation in acados, utilizing BLASFEO for the linear algebra operations. In addition to the scaling advantages posed by the zoRO algorithm, the efficient implementation drastically reduces the computational overhead, and, combined with an RTI scheme, enables the use of tube-based MPC for a wider range of applications. The flexibility, usability and effectiveness of the proposed implementation is demonstrated on two examples. On the practical example of a differential drive robot, the proposed implementation results in a tenfold reduction of computation time with respect to the previously available zoRO implementation.
39.
Simpson, Léo; Asprion, Jonas; Muntwiler, Simon; Köhler, Johannes; Diehl, Moritz
Parallelizable Parametric Nonlinear System Identification via tuning of a Moving Horizon State Estimator Proceedings Article Forthcoming
In: Forthcoming, (Accepted to the 63rd IEEE Conference on Decision and Control 2024 (CDC)).
@inproceedings{Simpson2024Parallelizable,
title = {Parallelizable Parametric Nonlinear System Identification via tuning of a Moving Horizon State Estimator},
author = {Léo Simpson and Jonas Asprion and Simon Muntwiler and Johannes Köhler and Moritz Diehl},
doi = {https://doi.org/10.48550/arXiv.2403.17858},
year = {2024},
date = {2024-05-29},
urldate = {2024-05-29},
abstract = {This paper introduces a novel optimization-based approach for parametric nonlinear system identification. Building upon the prediction error method framework, traditionally used for linear system identification, we extend its capabilities to nonlinear systems. The predictions are computed using a moving horizon state estimator with a constant arrival cost. Eventually, both the system parameters and the arrival cost are estimated by minimizing the sum of the squared prediction errors. Since the predictions are induced by the state estimator, the method can be viewed as the tuning of a state estimator, based on its predictive capacities. The present extension of the prediction error method not only enhances performance for nonlinear systems but also enables learning from multiple trajectories with unknown initial states, broadening its applicability in practical scenarios. Additionally, the novel formulation leaves room for the design of efficient and parallelizable optimization algorithms, since each output prediction only depends on a fixed window of past actions and measurements. In the special case of linear time-invariant systems, we show an important property of the proposed method which suggests asymptotic consistency under reasonable assumptions. Numerical examples illustrate the effectiveness and practicality of the approach, and one of the examples also highlights the necessity for the arrival cost.},
note = {Accepted to the 63rd IEEE Conference on Decision and Control 2024 (CDC)},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
This paper introduces a novel optimization-based approach for parametric nonlinear system identification. Building upon the prediction error method framework, traditionally used for linear system identification, we extend its capabilities to nonlinear systems. The predictions are computed using a moving horizon state estimator with a constant arrival cost. Eventually, both the system parameters and the arrival cost are estimated by minimizing the sum of the squared prediction errors. Since the predictions are induced by the state estimator, the method can be viewed as the tuning of a state estimator, based on its predictive capacities. The present extension of the prediction error method not only enhances performance for nonlinear systems but also enables learning from multiple trajectories with unknown initial states, broadening its applicability in practical scenarios. Additionally, the novel formulation leaves room for the design of efficient and parallelizable optimization algorithms, since each output prediction only depends on a fixed window of past actions and measurements. In the special case of linear time-invariant systems, we show an important property of the proposed method which suggests asymptotic consistency under reasonable assumptions. Numerical examples illustrate the effectiveness and practicality of the approach, and one of the examples also highlights the necessity for the arrival cost.
40.
Bourkhissi, Lahcen El; Necoara, Ion
Convergence rates for an inexact linearized ADMM for nonsmooth optimization with nonlinear equality constraints Working paper
2024, (Under review).
@workingpaper{bourkhissi2024convergence,
title = {Convergence rates for an inexact linearized ADMM for nonsmooth optimization with nonlinear equality constraints},
author = {Lahcen El Bourkhissi and Ion Necoara},
year = {2024},
date = {2024-05-29},
urldate = {2024-05-29},
note = {Under review},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}