Publications
21.
Lahr, Amon; Tronarp, Filip; Schmidt, Nathanael Bosch Jonathan; Hennig, Philipp; Zeilinger, Melanie N.
Probabilistic ODE Solvers for Integration Error-Aware Model Predictive Control Working paper
2024, (Submitted to the 6th Annual Learning for Dynamics & Control Conference (L4DC 2024)).
@workingpaper{lahr_probabilistic_2024,
title = {Probabilistic ODE Solvers for Integration Error-Aware Model Predictive Control},
author = {Amon Lahr and Filip Tronarp and Nathanael Bosch Jonathan Schmidt and Philipp Hennig and Melanie N. Zeilinger},
url = {https://doi.org/10.48550/arXiv.2401.17731},
year = {2024},
date = {2024-02-07},
urldate = {2024-02-07},
abstract = {Appropriate time discretization is crucial for nonlinear model predictive control. However, in situations where the discretization error strongly depends on the applied control input, meeting accuracy and sampling time requirements simultaneously can be challenging using classical discretization methods. In particular, neither fixed-grid nor adaptive-grid discretizations may be suitable, when they suffer from large integration error or exceed the prescribed sampling time, respectively. In this work, we take a first step at closing this gap by utilizing probabilistic numerical integrators to approximate the solution of the initial value problem, as well as the computational uncertainty associated with it, inside the optimal control problem (OCP). By taking the viewpoint of probabilistic numerics and propagating the numerical uncertainty in the cost, the OCP is reformulated such that the optimal input reduces the computational uncertainty insofar as it is beneficial for the control objective. The proposed approach is illustrated using a numerical example, and potential benefits and limitations are discussed.},
note = {Submitted to the 6th Annual Learning for Dynamics & Control Conference (L4DC 2024)},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
Appropriate time discretization is crucial for nonlinear model predictive control. However, in situations where the discretization error strongly depends on the applied control input, meeting accuracy and sampling time requirements simultaneously can be challenging using classical discretization methods. In particular, neither fixed-grid nor adaptive-grid discretizations may be suitable, when they suffer from large integration error or exceed the prescribed sampling time, respectively. In this work, we take a first step at closing this gap by utilizing probabilistic numerical integrators to approximate the solution of the initial value problem, as well as the computational uncertainty associated with it, inside the optimal control problem (OCP). By taking the viewpoint of probabilistic numerics and propagating the numerical uncertainty in the cost, the OCP is reformulated such that the optimal input reduces the computational uncertainty insofar as it is beneficial for the control objective. The proposed approach is illustrated using a numerical example, and potential benefits and limitations are discussed.
22.
Leeman, Antoine P.; Köhler, Johannes; Messerer, Florian; Lahr, Amon; Diehl, Moritz; Zeilinger, Melanie N.
Fast System Level Synthesis: Robust Model Predictive Control Using Riccati Recursions Working paper
2024, (Submitted to the 2024 IFAC Conference on Nonlinear Model Predictive Control (NMPC)).
@workingpaper{leeman_fast_2024,
title = {Fast System Level Synthesis: Robust Model Predictive Control Using Riccati Recursions},
author = {Antoine P. Leeman and Johannes Köhler and Florian Messerer and Amon Lahr and Moritz Diehl and Melanie N. Zeilinger},
url = {https://doi.org/10.48550/arXiv.2401.13762},
year = {2024},
date = {2024-02-07},
urldate = {2024-02-07},
abstract = {System Level Synthesis (SLS) enables improved robust MPC formulations by allowing for joint optimization of the nominal trajectory and controller. This paper introduces a tailored algorithm for solving the corresponding disturbance feedback optimization problem. The proposed algorithm builds on a recently proposed joint optimization scheme and iterates between optimizing the controller and the nominal trajectory while converging q-linearly to an optimal solution. We show that the controller optimization can be solved through Riccati recursions leading to a horizon-length, state, and input scalability of O(N2(n3x+n3u)) for each iterate. On a numerical example, the proposed algorithm exhibits computational speedups of order 10 to 10^3 compared to general-purpose commercial solvers.
},
note = {Submitted to the 2024 IFAC Conference on Nonlinear Model Predictive Control (NMPC)},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
System Level Synthesis (SLS) enables improved robust MPC formulations by allowing for joint optimization of the nominal trajectory and controller. This paper introduces a tailored algorithm for solving the corresponding disturbance feedback optimization problem. The proposed algorithm builds on a recently proposed joint optimization scheme and iterates between optimizing the controller and the nominal trajectory while converging q-linearly to an optimal solution. We show that the controller optimization can be solved through Riccati recursions leading to a horizon-length, state, and input scalability of O(N2(n3x+n3u)) for each iterate. On a numerical example, the proposed algorithm exhibits computational speedups of order 10 to 10^3 compared to general-purpose commercial solvers.
23.
Bonassi, Fabio; Bella, Alessio La; Farina, Marcello; Scattolini, Riccardo
Nonlinear MPC design for incrementally ISS systems with application to GRU networks Journal Article
In: Automatica, vol. 159, iss. 11381, pp. 111381, 2024.
@article{bonassi2024nonlinear,
title = {Nonlinear MPC design for incrementally ISS systems with application to GRU networks},
author = {Fabio Bonassi and Alessio La Bella and Marcello Farina and Riccardo Scattolini},
doi = {https://doi.org/10.1016/j.automatica.2023.111381},
year = {2024},
date = {2024-01-03},
urldate = {2024-01-03},
journal = {Automatica},
volume = {159},
issue = {11381},
pages = {111381},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
24.
Bourkhissi, Lahcen El; Necoara, Ion
Complexity of linearized quadratic penalty for optimization with nonlinear equality constraints Working paper
2023, (Under review).
@workingpaper{bourkhissi2023complexity,
title = {Complexity of linearized quadratic penalty for optimization with nonlinear equality constraints},
author = {Lahcen El Bourkhissi and Ion Necoara},
doi = {https://doi.org/10.48550/arXiv.2402.15639},
year = {2023},
date = {2023-12-31},
abstract = {In this paper we consider a nonconvex optimization problem with nonlinear equality constraints. We assume that both, the objective function and the functional constraints, are locally smooth. For solving this problem, we propose a linearized quadratic penalty method, i.e., we linearize the objective function and the functional constraints in the penalty formulation at the current iterate and add a quadratic regularization, thus yielding a subproblem that is easy to solve, and whose solution is the next iterate. Under a dynamic regularization parameter choice, we derive convergence guarantees for the iterates of our method to an ϵ first-order optimal solution in O(1/ϵ3) outer iterations. Finally, we show that when the problem data satisfy Kurdyka-Lojasiewicz property, e.g., are semialgebraic, the whole sequence generated by our algorithm converges and we derive convergence rates. We validate the theory and the performance of the proposed algorithm by numerically comparing it with the existing methods from the literature.},
note = {Under review},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
In this paper we consider a nonconvex optimization problem with nonlinear equality constraints. We assume that both, the objective function and the functional constraints, are locally smooth. For solving this problem, we propose a linearized quadratic penalty method, i.e., we linearize the objective function and the functional constraints in the penalty formulation at the current iterate and add a quadratic regularization, thus yielding a subproblem that is easy to solve, and whose solution is the next iterate. Under a dynamic regularization parameter choice, we derive convergence guarantees for the iterates of our method to an ϵ first-order optimal solution in O(1/ϵ3) outer iterations. Finally, we show that when the problem data satisfy Kurdyka-Lojasiewicz property, e.g., are semialgebraic, the whole sequence generated by our algorithm converges and we derive convergence rates. We validate the theory and the performance of the proposed algorithm by numerically comparing it with the existing methods from the literature.
25.
Frey, Jonathan; Gao, Yunfan; Messerer, Florian; Lahr, Amon; Zeilinger, Melanie N.; Diehl, Moritz
Efficient Zero-Order Robust Optimization for Real-Time Model Predictive Control with Acados Working paper
2023.
@workingpaper{frey_efficient_2023,
title = {Efficient Zero-Order Robust Optimization for Real-Time Model Predictive Control with Acados},
author = {Jonathan Frey and Yunfan Gao and Florian Messerer and Amon Lahr and Melanie N. Zeilinger and Moritz Diehl},
doi = {10.48550/arXiv.2311.04557},
year = {2023},
date = {2023-12-18},
abstract = {Robust and stochastic optimal control problem (OCP) formulations allow a systematic treatment of uncertainty, but are typically associated with a high computational cost. The recently proposed zero-order robust optimization (zoRO) algorithm mitigates the computational cost of uncertainty-aware MPC by propagating the uncertainties outside of the MPC problem. This paper details the combination of zoRO with the real-time iteration (RTI) scheme and presents an efficient open-source implementation in acados, utilizing BLASFEO for the linear algebra operations. In addition to the scaling advantages posed by the zoRO algorithm, the efficient implementation drastically reduces the computational overhead, and, combined with an RTI scheme, enables the use of tube-based MPC for a wider range of applications. The flexibility, usability and effectiveness of the proposed implementation is demonstrated on two examples. On the practical example of a differential drive robot, the proposed implementation results in a tenfold reduction of computation time with respect to the previously available zoRO implementation.},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
Robust and stochastic optimal control problem (OCP) formulations allow a systematic treatment of uncertainty, but are typically associated with a high computational cost. The recently proposed zero-order robust optimization (zoRO) algorithm mitigates the computational cost of uncertainty-aware MPC by propagating the uncertainties outside of the MPC problem. This paper details the combination of zoRO with the real-time iteration (RTI) scheme and presents an efficient open-source implementation in acados, utilizing BLASFEO for the linear algebra operations. In addition to the scaling advantages posed by the zoRO algorithm, the efficient implementation drastically reduces the computational overhead, and, combined with an RTI scheme, enables the use of tube-based MPC for a wider range of applications. The flexibility, usability and effectiveness of the proposed implementation is demonstrated on two examples. On the practical example of a differential drive robot, the proposed implementation results in a tenfold reduction of computation time with respect to the previously available zoRO implementation.
26.
Bourkhissi, Lahcen El; Necoara, Ion; Patrinos, Panagiotis
Linearized ADMM for Nonsmooth Nonconvex Optimization with Nonlinear Equality Constraints Proceedings Article
In: 2023 62nd IEEE Conference on Decision and Control (CDC), pp. 7312-7317, IEEE, Singapore, Singapore, 2023, ISSN: 2576-2370.
@inproceedings{Lahcen23LinADMM,
title = {Linearized ADMM for Nonsmooth Nonconvex Optimization with Nonlinear Equality Constraints},
author = {Lahcen El Bourkhissi and Ion Necoara and Panagiotis Patrinos},
doi = {10.1109/CDC49753.2023.10384166},
issn = {2576-2370},
year = {2023},
date = {2023-12-13},
urldate = {2023-12-13},
booktitle = {2023 62nd IEEE Conference on Decision and Control (CDC)},
pages = {7312-7317},
publisher = {IEEE},
address = {Singapore, Singapore},
abstract = {This paper proposes a new approach for solving a structured nonsmooth nonconvex optimization problem with nonlinear equality constraints, where both the objective function and constraints are 2-blocks separable. Our method is based on a 2-block linearized ADMM, where we linearize the smooth part of the cost function and the nonlinear term of the functional constraints in the augmented Lagrangian at each outer iteration. This results in simple subproblems, whose solutions are used to update the iterates of the 2 blocks variables. We prove global convergence for the sequence generated by our method to a stationary point of the original problem. To demonstrate its effectiveness, we apply our proposed algorithm as a solver for the nonlinear model predictive control problem of an inverted pendulum on a cart.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper proposes a new approach for solving a structured nonsmooth nonconvex optimization problem with nonlinear equality constraints, where both the objective function and constraints are 2-blocks separable. Our method is based on a 2-block linearized ADMM, where we linearize the smooth part of the cost function and the nonlinear term of the functional constraints in the augmented Lagrangian at each outer iteration. This results in simple subproblems, whose solutions are used to update the iterates of the 2 blocks variables. We prove global convergence for the sequence generated by our method to a stationary point of the original problem. To demonstrate its effectiveness, we apply our proposed algorithm as a solver for the nonlinear model predictive control problem of an inverted pendulum on a cart.
27.
Roy, Wim Van; Nurkanovic, Armin; Abbasi-Esfeden, Ramin; Frey, Jonathan; Pozharskiy, Anton; Swevers, Jan; Diehl, Moritz
Continuous Optimization for Control of Finite-State Machines with Cascaded Hysteresis Via Time-Freezing Proceedings Article Forthcoming
In: 2023 Conference on Decision and Control (CDC), Forthcoming.
@inproceedings{VanRoy2023CDC,
title = {Continuous Optimization for Control of Finite-State Machines with Cascaded Hysteresis Via Time-Freezing},
author = {Wim Van Roy and Armin Nurkanovic and Ramin Abbasi-Esfeden and Jonathan Frey and Anton Pozharskiy and Jan Swevers and Moritz Diehl},
year = {2023},
date = {2023-12-01},
booktitle = {2023 Conference on Decision and Control (CDC)},
abstract = {Control problems with Finite-State Machines (FSM) are often solved using integer variables, leading to a mixed-integer optimal control problem (MIOCP). This paper proposes analternative method to describe a subclass of FSMs using complementarity constraints and time-freezing. The FSM from this subclass is built up by a sequence of states where a transition between the states is triggered by a single switching function. This can be looked at as a cascade of hysteresis loops where a memory effect is used to maintain the active state of the state machine. Based on the reformulation for hybrid systems with a hysteresis loop, a method is developed to reformulate this subclass in a similar fashion. The approach transforms the original problem into a Piecewise Smooth System (PSS), which can be discretized using the recently developed Finite Elements with Switch Detection, allowing for high-accuracy solutions. The reformation is compared to a mixed-integer formulation from the literature on a time-optimal control problem. This work is a first step towards the general reformulation of FSMs into nonsmooth systems without integer states.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
Control problems with Finite-State Machines (FSM) are often solved using integer variables, leading to a mixed-integer optimal control problem (MIOCP). This paper proposes analternative method to describe a subclass of FSMs using complementarity constraints and time-freezing. The FSM from this subclass is built up by a sequence of states where a transition between the states is triggered by a single switching function. This can be looked at as a cascade of hysteresis loops where a memory effect is used to maintain the active state of the state machine. Based on the reformulation for hybrid systems with a hysteresis loop, a method is developed to reformulate this subclass in a similar fashion. The approach transforms the original problem into a Piecewise Smooth System (PSS), which can be discretized using the recently developed Finite Elements with Switch Detection, allowing for high-accuracy solutions. The reformation is compared to a mixed-integer formulation from the literature on a time-optimal control problem. This work is a first step towards the general reformulation of FSMs into nonsmooth systems without integer states.
28.
Voogd, Kevin; Allamaa, Jean Pierre; Alonso-Mora, Javier; Son, Tong Duy
Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin Proceedings Article
In: 22nd IFAC World Congress 2023, pp. 1510-1515, Elsevier Ltd, Yokohama, Japan, 2023, ISSN: 2405-8963.
@inproceedings{Voogd2023ReinforcementLF,
title = {Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin},
author = {Kevin Voogd and Jean Pierre Allamaa and Javier Alonso-Mora and Tong Duy Son},
url = {https://www.sciencedirect.com/science/article/pii/S2405896323022553},
doi = {https://doi.org/10.1016/j.ifacol.2023.10.1846},
issn = {2405-8963},
year = {2023},
date = {2023-11-30},
urldate = {2023-11-30},
booktitle = {22nd IFAC World Congress 2023},
journal = {IFAC-PapersOnLine},
volume = {56},
number = {2},
pages = {1510-1515},
publisher = {Elsevier Ltd},
address = {Yokohama, Japan},
abstract = {Reinforcement learning (RL) is a promising solution for autonomous vehicles to deal with complex and uncertain traffic environments. The RL training process is however expensive, unsafe, and time-consuming. Algorithms are often developed first in simulation and then transferred to the real-world, leading to a common sim2real challenge where performance decreases when the domain changes. In this paper, we propose a transfer learning process to minimize the gap by exploiting digital twin technology, relying on a systematic and simultaneous combination of virtual and real world data coming from vehicle dynamics and traffic scenarios. The model and testing environment is evolved from model, hardware to vehicle in the loop and proving ground testing stages, similar to standard development cycle in the automotive industry. In particular, we also integrate other transfer learning techniques such as domain randomization and adaptation in each stage. The simulation and real data are gradually incorporated to accelerate and make the transfer learning process more robust. The proposed RL methodology is applied to develop a path-following steering controller for an autonomous electric vehicle. After learning and deploying the real-time RL control policy on the vehicle, we obtained satisfactory and safe control performance already from the first deployment, demonstrating the advantages of the proposed digital twin based learning process.
},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Reinforcement learning (RL) is a promising solution for autonomous vehicles to deal with complex and uncertain traffic environments. The RL training process is however expensive, unsafe, and time-consuming. Algorithms are often developed first in simulation and then transferred to the real-world, leading to a common sim2real challenge where performance decreases when the domain changes. In this paper, we propose a transfer learning process to minimize the gap by exploiting digital twin technology, relying on a systematic and simultaneous combination of virtual and real world data coming from vehicle dynamics and traffic scenarios. The model and testing environment is evolved from model, hardware to vehicle in the loop and proving ground testing stages, similar to standard development cycle in the automotive industry. In particular, we also integrate other transfer learning techniques such as domain randomization and adaptation in each stage. The simulation and real data are gradually incorporated to accelerate and make the transfer learning process more robust. The proposed RL methodology is applied to develop a path-following steering controller for an autonomous electric vehicle. After learning and deploying the real-time RL control policy on the vehicle, we obtained satisfactory and safe control performance already from the first deployment, demonstrating the advantages of the proposed digital twin based learning process.
29.
Acerbo, Flavia Sofia; Swevers, Jan; Tuytelaars, Tinne; Son, Tong Duy
Evaluation of MPC-based Imitation Learning for Human-like Autonomous Driving Proceedings Article
In: 22nd IFAC World Congress, pp. 4871-4876, Elsevier Ltd, Yokohama, Japan, 2023, ISSN: 2405-8963.
@inproceedings{acerboEvaluationMPCbasedImitation2023,
title = {Evaluation of MPC-based Imitation Learning for Human-like Autonomous Driving},
author = {Flavia Sofia Acerbo and Jan Swevers and Tinne Tuytelaars and Tong Duy Son},
url = {https://www.sciencedirect.com/science/article/pii/S2405896323016610},
doi = {https://doi.org/10.1016/j.ifacol.2023.10.1257},
issn = {2405-8963},
year = {2023},
date = {2023-11-30},
urldate = {2023-11-30},
booktitle = {22nd IFAC World Congress},
journal = { IFAC-PapersOnLine},
volume = {56},
number = {2},
pages = {4871-4876},
publisher = {Elsevier Ltd},
address = {Yokohama, Japan},
abstract = {This work evaluates and analyzes the combination of imitation learning (IL) and differentiable model predictive control (MPC) for the application of human-like autonomous driving. We combine MPC with a hierarchical learning-based policy, and measure its performance in open-loop and closed-loop with metrics related to safety, comfort and similarity to human driving characteristics. We also demonstrate the value of augmenting open-loop behavioral cloning with closed-loop training for a more robust learning, approximating the policy gradient through time with the state space model used by the MPC. We perform experimental evaluations on a lane keeping control system, learned from demonstrations collected on a fixed-base driving simulator, and show that our imitative policies approach the human driving style preferences.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This work evaluates and analyzes the combination of imitation learning (IL) and differentiable model predictive control (MPC) for the application of human-like autonomous driving. We combine MPC with a hierarchical learning-based policy, and measure its performance in open-loop and closed-loop with metrics related to safety, comfort and similarity to human driving characteristics. We also demonstrate the value of augmenting open-loop behavioral cloning with closed-loop training for a more robust learning, approximating the policy gradient through time with the state space model used by the MPC. We perform experimental evaluations on a lane keeping control system, learned from demonstrations collected on a fixed-base driving simulator, and show that our imitative policies approach the human driving style preferences.
30.
Reiter, Rudolf; Nurkanovic, Armin; Frey, Jonathan; Diehl, Moritz
Frenet-Cartesian model representations for automotive obstacle avoidance within nonlinear MPC Journal Article
In: European Journal of Control, vol. 74, pp. 100847, 2023, ISSN: 0947-3580, (2023 European Control Conference Special Issue).
@article{REITER2023100847,
title = {Frenet-Cartesian model representations for automotive obstacle avoidance within nonlinear MPC},
author = {Rudolf Reiter and Armin Nurkanovic and Jonathan Frey and Moritz Diehl},
url = {https://www.sciencedirect.com/science/article/pii/S0947358023000766},
doi = {https://doi.org/10.1016/j.ejcon.2023.100847},
issn = {0947-3580},
year = {2023},
date = {2023-11-01},
journal = {European Journal of Control},
volume = {74},
pages = {100847},
abstract = {In recent years, nonlinear model predictive control has been extensively used for solving automotive motion control and planning tasks. In order to formulate the nonlinear model predictive control problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the nonlinear MPC related optimization problem, involving a Cartesian and a Frenet coordinate frame in a single nonlinear program. We specify costs and collision avoidance constraints in the more advantageous coordinate frame, derive appropriate formulations and compare different obstacle constraints. With this approach, we exploit the simpler formulation of opponent vehicle constraints in the Cartesian coordinate frame, as well as road-aligned costs and constraints related to the Frenet coordinate frame. Comparisons to other approaches in a simulation framework highlight the advantages of the proposed methods.},
note = {2023 European Control Conference Special Issue},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In recent years, nonlinear model predictive control has been extensively used for solving automotive motion control and planning tasks. In order to formulate the nonlinear model predictive control problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the nonlinear MPC related optimization problem, involving a Cartesian and a Frenet coordinate frame in a single nonlinear program. We specify costs and collision avoidance constraints in the more advantageous coordinate frame, derive appropriate formulations and compare different obstacle constraints. With this approach, we exploit the simpler formulation of opponent vehicle constraints in the Cartesian coordinate frame, as well as road-aligned costs and constraints related to the Frenet coordinate frame. Comparisons to other approaches in a simulation framework highlight the advantages of the proposed methods.
31.
Zhang, Yuan; Wang, Jianhong; Boedecker, Joschka
Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization Proceedings Article
In: 7th Annual Conference on Robot Learning, 2023.
@inproceedings{zhang2023robust,
title = {Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization},
author = {Yuan Zhang and Jianhong Wang and Joschka Boedecker},
url = {https://openreview.net/forum?id=keAPCON4jHC},
year = {2023},
date = {2023-10-16},
urldate = {2023-10-16},
booktitle = {7th Annual Conference on Robot Learning},
abstract = {Reinforcement learning (RL) is recognized as lacking generalization and robustness under environmental perturbations, which excessively restricts its application for real-world robotics. Prior work claimed that adding regularization to the value function is equivalent to learning a robust policy under uncertain transitions. Although the regularization-robustness transformation is appealing for its simplicity and efficiency, it is still lacking in continuous control tasks. In this paper, we propose a new regularizer named Uncertainty Set Regularizer (USR), to formulate the uncertainty set on the parametric space of a transition function. To deal with unknown uncertainty sets, we further propose a novel adversarial approach to generate them based on the value function. We evaluate USR on the Real-world Reinforcement Learning (RWRL) benchmark and the Unitree A1 Robot, demonstrating improvements in the robust performance of perturbed testing environments and sim-to-real scenarios.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Reinforcement learning (RL) is recognized as lacking generalization and robustness under environmental perturbations, which excessively restricts its application for real-world robotics. Prior work claimed that adding regularization to the value function is equivalent to learning a robust policy under uncertain transitions. Although the regularization-robustness transformation is appealing for its simplicity and efficiency, it is still lacking in continuous control tasks. In this paper, we propose a new regularizer named Uncertainty Set Regularizer (USR), to formulate the uncertainty set on the parametric space of a transition function. To deal with unknown uncertainty sets, we further propose a novel adversarial approach to generate them based on the value function. We evaluate USR on the Real-world Reinforcement Learning (RWRL) benchmark and the Unitree A1 Robot, demonstrating improvements in the robust performance of perturbed testing environments and sim-to-real scenarios.
32.
Mamedov, Shamil; Reiter, Rudolf; Azad, Seyed Mahdi Basiri; Boedecker, Joschka; Diehl, Moritz; Swevers, Jan
Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots Working paper
2023, (Submitted to ICRA 2024).
@workingpaper{mamedov2023safe,
title = {Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots},
author = {Shamil Mamedov and Rudolf Reiter and Seyed Mahdi Basiri Azad and Joschka Boedecker and Moritz Diehl and Jan Swevers},
url = {https://doi.org/10.48550/arXiv.2212.02941},
year = {2023},
date = {2023-09-28},
abstract = {Flexible robots may overcome some of the industry's major challenges, such as enabling intrinsically safe human-robot collaboration and achieving a higher load-to-mass ratio. However, controlling flexible robots is complicated due to their complex dynamics, which include oscillatory behavior and a high-dimensional state space. NMPC offers an effective means to control such robots, but its extensive computational demands often limit its application in real-time scenarios. To enable fast control of flexible robots, we propose a framework for a safe approximation of NMPC using imitation learning and a predictive safety filter. Our framework significantly reduces computation time while incurring a slight loss in performance. Compared to NMPC, our framework shows more than a eightfold improvement in computation time when controlling a three-dimensional flexible robot arm in simulation, all while guaranteeing safety constraints. Notably, our approach outperforms conventional reinforcement learning methods. The development of fast and safe approximate NMPC holds the potential to accelerate the adoption of flexible robots in industry.},
note = {Submitted to ICRA 2024},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
Flexible robots may overcome some of the industry's major challenges, such as enabling intrinsically safe human-robot collaboration and achieving a higher load-to-mass ratio. However, controlling flexible robots is complicated due to their complex dynamics, which include oscillatory behavior and a high-dimensional state space. NMPC offers an effective means to control such robots, but its extensive computational demands often limit its application in real-time scenarios. To enable fast control of flexible robots, we propose a framework for a safe approximation of NMPC using imitation learning and a predictive safety filter. Our framework significantly reduces computation time while incurring a slight loss in performance. Compared to NMPC, our framework shows more than a eightfold improvement in computation time when controlling a three-dimensional flexible robot arm in simulation, all while guaranteeing safety constraints. Notably, our approach outperforms conventional reinforcement learning methods. The development of fast and safe approximate NMPC holds the potential to accelerate the adoption of flexible robots in industry.
33.
Ghezzi, Andrea; Hoffman, Jasper; Frey, Jonathan; Boedecker, Joschka; Diehl, Moritz
Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation Proceedings Article Forthcoming
In: 2023 Conference on Decision and Control (CDC), Forthcoming.
@inproceedings{Ghezzi2023b,
title = {Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation},
author = {Andrea Ghezzi and Jasper Hoffman and Jonathan Frey and Joschka Boedecker and Moritz Diehl},
doi = {https://doi.org/10.48550/arXiv.2304.01782},
year = {2023},
date = {2023-08-17},
booktitle = {2023 Conference on Decision and Control (CDC)},
abstract = {This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs.
34.
Bonassi, Fabio; Bella, Alessio La; Panzani, Giulio; Farina, Marcello; Scattolini, Riccardo
Deep Long-Short Term Memory networks: Stability properties and Experimental validation Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{bonassi2023deep,
title = {Deep Long-Short Term Memory networks: Stability properties and Experimental validation},
author = {Fabio Bonassi and Alessio La Bella and Giulio Panzani and Marcello Farina and Riccardo Scattolini},
url = {https://ieeexplore.ieee.org/document/10178405
http://arxiv.org/abs/2304.02975},
doi = {https://doi.org/10.23919/ECC57647.2023.10178405},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-25},
urldate = {2023-07-25},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {The aim of this work is to investigate the use of Incrementally Input-to-State Stable (δISS) deep Long Short Term Memory networks (LSTMs) for the identification of nonlinear dynamical systems. We show that suitable sufficient conditions on the weights of the network can be leveraged to setup a training procedure able to learn provenly-δISS LSTM models from data. The proposed approach is tested on a real brake-by-wire apparatus to identify a model of the system from input-output experimentally collected data. Results show satisfactory modeling performances.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The aim of this work is to investigate the use of Incrementally Input-to-State Stable (δISS) deep Long Short Term Memory networks (LSTMs) for the identification of nonlinear dynamical systems. We show that suitable sufficient conditions on the weights of the network can be leveraged to setup a training procedure able to learn provenly-δISS LSTM models from data. The proposed approach is tested on a real brake-by-wire apparatus to identify a model of the system from input-output experimentally collected data. Results show satisfactory modeling performances.
35.
Allamaa, Jean Pierre; Patrinos, Panagiotis; Auweraer, Herman Van; Son, Tong Duy
Safety Envelope for Orthogonal Collocation Methods in Embedded Optimal Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Allamaa2022SafetyEF,
title = {Safety Envelope for Orthogonal Collocation Methods in Embedded Optimal Control},
author = {Jean Pierre Allamaa and Panagiotis Patrinos and Herman Van Auweraer and Tong Duy Son},
url = {https://ieeexplore.ieee.org/document/10178116
https://arxiv.org/abs/2211.14853},
doi = {https://doi.org/10.23919/ECC57647.2023.10178116},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Orthogonal collocation methods are direct approaches for solving optimal control problems (OCP). A high solution accuracy is achieved with few optimization variables, making it more favorable for embedded and real-time NMPC applications. However, collocation approaches lack a guarantee about the safety of the resulting trajectory as inequality constraints are only set on a finite number of collocation points. In this paper we propose a method to efficiently create a convex safety envelope containing the trajectory such that the solution fully satisfies the OCP constraints. We make use of Bernstein approximations of a polynomial’s extrema and span the solution over an orthogonal basis using Legendre polynomials. The tightness of the safety envelope estimation, high accuracy in solving the underlying differential equations, fast rate of convergence and little conservatism are properties of the presented approach making it a suitable method for safe real-time NMPC deployment. We show that our method has comparable computational performance to pseudospectral approaches and can accurately approximate the original OCP up to 9 times more quickly than standard multiple-shooting method in autonomous driving applications, without adding complexity to the formulation.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Orthogonal collocation methods are direct approaches for solving optimal control problems (OCP). A high solution accuracy is achieved with few optimization variables, making it more favorable for embedded and real-time NMPC applications. However, collocation approaches lack a guarantee about the safety of the resulting trajectory as inequality constraints are only set on a finite number of collocation points. In this paper we propose a method to efficiently create a convex safety envelope containing the trajectory such that the solution fully satisfies the OCP constraints. We make use of Bernstein approximations of a polynomial’s extrema and span the solution over an orthogonal basis using Legendre polynomials. The tightness of the safety envelope estimation, high accuracy in solving the underlying differential equations, fast rate of convergence and little conservatism are properties of the presented approach making it a suitable method for safe real-time NMPC deployment. We show that our method has comparable computational performance to pseudospectral approaches and can accurately approximate the original OCP up to 9 times more quickly than standard multiple-shooting method in autonomous driving applications, without adding complexity to the formulation.
36.
Wang, Renzi; Schuurmans, Mathijs; Patrinos, Panagiotis
Interaction-aware Model Predictive Control for Autonomous Driving Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Wang2023,
title = {Interaction-aware Model Predictive Control for Autonomous Driving},
author = {Renzi Wang and Mathijs Schuurmans and Panagiotis Patrinos},
url = {https://ieeexplore.ieee.org/document/10178332
https://arxiv.org/abs/2211.17053},
doi = {https://doi.org/10.23919/ECC57647.2023.10178332},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We propose an interaction-aware stochastic model predictive control (MPC) strategy for lane merging tasks in automated driving. The MPC strategy is integrated with an online learning framework, which models a given driver’s cooperation level as an unknown parameter in a state-dependent probability distribution. The online learning framework adaptively estimates the surrounding vehicle’s cooperation level with the vehicle’s past state trajectory and combines this with a kinematic vehicle model to predict the distribution of a multimodal future state trajectory. Learning is conducted using logistic regression, enabling fast online computations. The multi-future prediction is used in the MPC algorithm to compute the optimal control input while satisfying safety constraints. We demonstrate our algorithm in an interactive lane changing scenario with drivers in different randomly selected cooperation levels.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We propose an interaction-aware stochastic model predictive control (MPC) strategy for lane merging tasks in automated driving. The MPC strategy is integrated with an online learning framework, which models a given driver’s cooperation level as an unknown parameter in a state-dependent probability distribution. The online learning framework adaptively estimates the surrounding vehicle’s cooperation level with the vehicle’s past state trajectory and combines this with a kinematic vehicle model to predict the distribution of a multimodal future state trajectory. Learning is conducted using logistic regression, enabling fast online computations. The multi-future prediction is used in the MPC algorithm to compute the optimal control input while satisfying safety constraints. We demonstrate our algorithm in an interactive lane changing scenario with drivers in different randomly selected cooperation levels.
37.
Roy, Wim Van; Abbasi-Esfeden, Ramin; Swevers, Jan
A Dynamic Programming-based Heuristic Approach for Unit Commitment Problems Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-8, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{vanRoy2023ADP,
title = {A Dynamic Programming-based Heuristic Approach for Unit Commitment Problems},
author = {Wim Van Roy and Ramin Abbasi-Esfeden and Jan Swevers},
url = {https://ieeexplore.ieee.org/document/10178216},
doi = {https://doi.org/10.23919/ECC57647.2023.10178216},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-8},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Unit Commitment (UC) problems are an essential set of problems in the power industry with applications in energy grid or heating systems management and control. The engineering goal is to balance the demand with the production of a network of production units, called generators, by providing a schedule and operating points for each generator cost-effectively while considering constraints. The constraints are caused by the dynamics of the system, the limits on the reserves, and possible robustness requirements. Due to the appearance of the on/off states from the generators, the resulting problems are NP-hard to solve. Thus, existing techniques to achieve a cost-efficient solution are computationally expensive. This paper proposes a dynamic programming-based heuristic to solve a UC problem. The heuristic focuses on finding a feasible and cost-effective solution for systems with a limited number of generators where a long time horizon is important. This method is compared to a Mixed Integer Linear Program (MILP) implementation for a micro-grid where it achieves a computation time that is an order of magnitude smaller than MILP programs for problems with a limited number of generators but a long time horizon.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Unit Commitment (UC) problems are an essential set of problems in the power industry with applications in energy grid or heating systems management and control. The engineering goal is to balance the demand with the production of a network of production units, called generators, by providing a schedule and operating points for each generator cost-effectively while considering constraints. The constraints are caused by the dynamics of the system, the limits on the reserves, and possible robustness requirements. Due to the appearance of the on/off states from the generators, the resulting problems are NP-hard to solve. Thus, existing techniques to achieve a cost-efficient solution are computationally expensive. This paper proposes a dynamic programming-based heuristic to solve a UC problem. The heuristic focuses on finding a feasible and cost-effective solution for systems with a limited number of generators where a long time horizon is important. This method is compared to a Mixed Integer Linear Program (MILP) implementation for a micro-grid where it achieves a computation time that is an order of magnitude smaller than MILP programs for problems with a limited number of generators but a long time horizon.
38.
Abbasi-Esfeden, Ramin; Roy, Wim Van; Swevers, Jan
Iterative Switching Time Optimization for Mixed-integer Optimal Control Problems Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{AbbasiEsfeden2023STO,
title = {Iterative Switching Time Optimization for Mixed-integer Optimal Control Problems},
author = {Ramin Abbasi-Esfeden and Wim Van Roy and Jan Swevers},
url = {https://ieeexplore.ieee.org/document/10178419},
doi = {https://doi.org/10.23919/ECC57647.2023.10178419},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {This paper proposes an iterative method to solve Mixed-Integer Optimal Control Problems arising from systems with switched dynamics. The so-called relaxed problem plays a central role within this context. Through a numerical example, it is shown why relying on the relaxed problem can lead the solution astray. As an alternative, an iterative Switching Time optimization method is proposed. The method consists of two components that iteratively interact: a Switching Time optimization (STO) problem and a sequence optimization. Each component is explained in detail, and the numerical example is resolved, the results of which shows the efficiency of the proposed algorithm. Finally, the advantages and disadvantages of the method are discussed and future lines of research are sketched.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper proposes an iterative method to solve Mixed-Integer Optimal Control Problems arising from systems with switched dynamics. The so-called relaxed problem plays a central role within this context. Through a numerical example, it is shown why relying on the relaxed problem can lead the solution astray. As an alternative, an iterative Switching Time optimization method is proposed. The method consists of two components that iteratively interact: a Switching Time optimization (STO) problem and a sequence optimization. Each component is explained in detail, and the numerical example is resolved, the results of which shows the efficiency of the proposed algorithm. Finally, the advantages and disadvantages of the method are discussed and future lines of research are sketched.
39.
Simpson, Léo; Nurkanovic, Armin; Diehl, Moritz
Direct Collocation for Numerical Optimal Control of Second-Order ODE Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Simpson2023DCSODE,
title = {Direct Collocation for Numerical Optimal Control of Second-Order ODE},
author = {Léo Simpson and Armin Nurkanovic and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178181},
doi = {https://doi.org/10.23919/ECC57647.2023.10178181},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Mechanical systems are usually modeled by second-order Ordinary Differential Equations (ODE) which take the form q¨=f(t, q, q˙). While simulation methods tailored to these equations have been studied, using them in direct optimal control methods is rare. Indeed, the standard approach is to perform a state augmentation, adding the velocities to the state. The main drawback of this approach is that the number of decision variables is doubled, which could harm the performance of the resulting optimization problem. In this paper, we present an approach tailored to second-order ODE. We compare it with the standard one, both on theoretical aspects and in a numerical example. Notably, we show that the tailored formulation is likely to improve the performance of a direct collocation method, for solving optimal control problems with second-order ODE of the more restrictive form q¨=f(t, q).},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Mechanical systems are usually modeled by second-order Ordinary Differential Equations (ODE) which take the form q¨=f(t, q, q˙). While simulation methods tailored to these equations have been studied, using them in direct optimal control methods is rare. Indeed, the standard approach is to perform a state augmentation, adding the velocities to the state. The main drawback of this approach is that the number of decision variables is doubled, which could harm the performance of the resulting optimization problem. In this paper, we present an approach tailored to second-order ODE. We compare it with the standard one, both on theoretical aspects and in a numerical example. Notably, we show that the tailored formulation is likely to improve the performance of a direct collocation method, for solving optimal control problems with second-order ODE of the more restrictive form q¨=f(t, q).
40.
Gao, Yunfan; Messerer, Florian; Frey, Jonathan; Duijkeren, Niels; Diehl, Moritz
Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{GaoCFMPECC23,
title = {Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC},
author = {Yunfan Gao and Florian Messerer and Jonathan Frey and Niels Duijkeren and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178311},
doi = {https://doi.org/10.23919/ECC57647.2023.10178311},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied [1]. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied [1]. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.