Publications
61.
Kessler, Nicolas; Fagiano, Lorenzo
On the stabilization of forking and cyclic trajectories for nonlinear systems Proceedings Article
In: 3rd Modeling, Estimation and Control Conference MECC 2023, pp. 199–204, IFAC-PapersOnLine, Lake Tahoe, NV, USA, 2023, ISSN: 2405-8963.
@inproceedings{kessler2023stabilization,
title = {On the stabilization of forking and cyclic trajectories for nonlinear systems},
author = {Nicolas Kessler and Lorenzo Fagiano },
url = {https://www.sciencedirect.com/science/article/pii/S2405896323023571},
doi = {https://doi.org/10.1016/j.ifacol.2023.12.024},
issn = {2405-8963},
year = {2023},
date = {2023-12-03},
urldate = {2023-12-03},
booktitle = {3rd Modeling, Estimation and Control Conference MECC 2023},
volume = {56},
number = {3},
pages = {199--204},
publisher = {IFAC-PapersOnLine},
address = {Lake Tahoe, NV, USA},
abstract = {Stabilizing a reference trajectory for a nonlinear system is a common, non-trivial task in control theory. An approach to solve this problem is to approximate the nonlinear system along the trajectory as an uncertain linear time-varying one, and to solve an optimization problem featuring Linear Matrix Inequality (LMI) constraints to derive a stabilizing, smooth, gain-scheduled control law. Such an approach is extended here by considering a set of reference trajectories instead of a single one, such that switching among them is permitted. These switching events are commonly encountered in industrial plants, such as energy generation systems, and are of high relevance in practice. The approach allows one to derive a gain-scheduled control law guaranteeing asymptotic stability also during the switching and accounting for the linearization errors. Simulation results on a chemical system highlight the effectiveness of the method.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Stabilizing a reference trajectory for a nonlinear system is a common, non-trivial task in control theory. An approach to solve this problem is to approximate the nonlinear system along the trajectory as an uncertain linear time-varying one, and to solve an optimization problem featuring Linear Matrix Inequality (LMI) constraints to derive a stabilizing, smooth, gain-scheduled control law. Such an approach is extended here by considering a set of reference trajectories instead of a single one, such that switching among them is permitted. These switching events are commonly encountered in industrial plants, such as energy generation systems, and are of high relevance in practice. The approach allows one to derive a gain-scheduled control law guaranteeing asymptotic stability also during the switching and accounting for the linearization errors. Simulation results on a chemical system highlight the effectiveness of the method.
62.
Voogd, Kevin; Allamaa, Jean Pierre; Alonso-Mora, Javier; Son, Tong Duy
Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin Proceedings Article
In: 22nd IFAC World Congress 2023, pp. 1510-1515, Elsevier Ltd, Yokohama, Japan, 2023, ISSN: 2405-8963.
@inproceedings{Voogd2023ReinforcementLF,
title = {Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin},
author = {Kevin Voogd and Jean Pierre Allamaa and Javier Alonso-Mora and Tong Duy Son},
url = {https://www.sciencedirect.com/science/article/pii/S2405896323022553},
doi = {https://doi.org/10.1016/j.ifacol.2023.10.1846},
issn = {2405-8963},
year = {2023},
date = {2023-11-30},
urldate = {2023-11-30},
booktitle = {22nd IFAC World Congress 2023},
journal = {IFAC-PapersOnLine},
volume = {56},
number = {2},
pages = {1510-1515},
publisher = {Elsevier Ltd},
address = {Yokohama, Japan},
abstract = {Reinforcement learning (RL) is a promising solution for autonomous vehicles to deal with complex and uncertain traffic environments. The RL training process is however expensive, unsafe, and time-consuming. Algorithms are often developed first in simulation and then transferred to the real-world, leading to a common sim2real challenge where performance decreases when the domain changes. In this paper, we propose a transfer learning process to minimize the gap by exploiting digital twin technology, relying on a systematic and simultaneous combination of virtual and real world data coming from vehicle dynamics and traffic scenarios. The model and testing environment is evolved from model, hardware to vehicle in the loop and proving ground testing stages, similar to standard development cycle in the automotive industry. In particular, we also integrate other transfer learning techniques such as domain randomization and adaptation in each stage. The simulation and real data are gradually incorporated to accelerate and make the transfer learning process more robust. The proposed RL methodology is applied to develop a path-following steering controller for an autonomous electric vehicle. After learning and deploying the real-time RL control policy on the vehicle, we obtained satisfactory and safe control performance already from the first deployment, demonstrating the advantages of the proposed digital twin based learning process.
},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Reinforcement learning (RL) is a promising solution for autonomous vehicles to deal with complex and uncertain traffic environments. The RL training process is however expensive, unsafe, and time-consuming. Algorithms are often developed first in simulation and then transferred to the real-world, leading to a common sim2real challenge where performance decreases when the domain changes. In this paper, we propose a transfer learning process to minimize the gap by exploiting digital twin technology, relying on a systematic and simultaneous combination of virtual and real world data coming from vehicle dynamics and traffic scenarios. The model and testing environment is evolved from model, hardware to vehicle in the loop and proving ground testing stages, similar to standard development cycle in the automotive industry. In particular, we also integrate other transfer learning techniques such as domain randomization and adaptation in each stage. The simulation and real data are gradually incorporated to accelerate and make the transfer learning process more robust. The proposed RL methodology is applied to develop a path-following steering controller for an autonomous electric vehicle. After learning and deploying the real-time RL control policy on the vehicle, we obtained satisfactory and safe control performance already from the first deployment, demonstrating the advantages of the proposed digital twin based learning process.
63.
Acerbo, Flavia Sofia; Swevers, Jan; Tuytelaars, Tinne; Son, Tong Duy
Evaluation of MPC-based Imitation Learning for Human-like Autonomous Driving Proceedings Article
In: 22nd IFAC World Congress, pp. 4871-4876, Elsevier Ltd, Yokohama, Japan, 2023, ISSN: 2405-8963.
@inproceedings{acerboEvaluationMPCbasedImitation2023,
title = {Evaluation of MPC-based Imitation Learning for Human-like Autonomous Driving},
author = {Flavia Sofia Acerbo and Jan Swevers and Tinne Tuytelaars and Tong Duy Son},
url = {https://www.sciencedirect.com/science/article/pii/S2405896323016610},
doi = {https://doi.org/10.1016/j.ifacol.2023.10.1257},
issn = {2405-8963},
year = {2023},
date = {2023-11-30},
urldate = {2023-11-30},
booktitle = {22nd IFAC World Congress},
journal = { IFAC-PapersOnLine},
volume = {56},
number = {2},
pages = {4871-4876},
publisher = {Elsevier Ltd},
address = {Yokohama, Japan},
abstract = {This work evaluates and analyzes the combination of imitation learning (IL) and differentiable model predictive control (MPC) for the application of human-like autonomous driving. We combine MPC with a hierarchical learning-based policy, and measure its performance in open-loop and closed-loop with metrics related to safety, comfort and similarity to human driving characteristics. We also demonstrate the value of augmenting open-loop behavioral cloning with closed-loop training for a more robust learning, approximating the policy gradient through time with the state space model used by the MPC. We perform experimental evaluations on a lane keeping control system, learned from demonstrations collected on a fixed-base driving simulator, and show that our imitative policies approach the human driving style preferences.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This work evaluates and analyzes the combination of imitation learning (IL) and differentiable model predictive control (MPC) for the application of human-like autonomous driving. We combine MPC with a hierarchical learning-based policy, and measure its performance in open-loop and closed-loop with metrics related to safety, comfort and similarity to human driving characteristics. We also demonstrate the value of augmenting open-loop behavioral cloning with closed-loop training for a more robust learning, approximating the policy gradient through time with the state space model used by the MPC. We perform experimental evaluations on a lane keeping control system, learned from demonstrations collected on a fixed-base driving simulator, and show that our imitative policies approach the human driving style preferences.
64.
Reiter, Rudolf; Nurkanovic, Armin; Frey, Jonathan; Diehl, Moritz
Frenet-Cartesian model representations for automotive obstacle avoidance within nonlinear MPC Journal Article
In: European Journal of Control, vol. 74, pp. 100847, 2023, ISSN: 0947-3580, (2023 European Control Conference Special Issue).
@article{REITER2023100847,
title = {Frenet-Cartesian model representations for automotive obstacle avoidance within nonlinear MPC},
author = {Rudolf Reiter and Armin Nurkanovic and Jonathan Frey and Moritz Diehl},
url = {https://www.sciencedirect.com/science/article/pii/S0947358023000766},
doi = {https://doi.org/10.1016/j.ejcon.2023.100847},
issn = {0947-3580},
year = {2023},
date = {2023-11-01},
journal = {European Journal of Control},
volume = {74},
pages = {100847},
abstract = {In recent years, nonlinear model predictive control has been extensively used for solving automotive motion control and planning tasks. In order to formulate the nonlinear model predictive control problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the nonlinear MPC related optimization problem, involving a Cartesian and a Frenet coordinate frame in a single nonlinear program. We specify costs and collision avoidance constraints in the more advantageous coordinate frame, derive appropriate formulations and compare different obstacle constraints. With this approach, we exploit the simpler formulation of opponent vehicle constraints in the Cartesian coordinate frame, as well as road-aligned costs and constraints related to the Frenet coordinate frame. Comparisons to other approaches in a simulation framework highlight the advantages of the proposed methods.},
note = {2023 European Control Conference Special Issue},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In recent years, nonlinear model predictive control has been extensively used for solving automotive motion control and planning tasks. In order to formulate the nonlinear model predictive control problem, different coordinate systems can be used with different advantages. We propose and compare formulations for the nonlinear MPC related optimization problem, involving a Cartesian and a Frenet coordinate frame in a single nonlinear program. We specify costs and collision avoidance constraints in the more advantageous coordinate frame, derive appropriate formulations and compare different obstacle constraints. With this approach, we exploit the simpler formulation of opponent vehicle constraints in the Cartesian coordinate frame, as well as road-aligned costs and constraints related to the Frenet coordinate frame. Comparisons to other approaches in a simulation framework highlight the advantages of the proposed methods.
65.
Zhang, Yuan; Wang, Jianhong; Boedecker, Joschka
Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization Proceedings Article
In: 7th Annual Conference on Robot Learning, 2023.
@inproceedings{zhang2023robust,
title = {Robust Reinforcement Learning in Continuous Control Tasks with Uncertainty Set Regularization},
author = {Yuan Zhang and Jianhong Wang and Joschka Boedecker},
url = {https://openreview.net/forum?id=keAPCON4jHC},
year = {2023},
date = {2023-10-16},
urldate = {2023-10-16},
booktitle = {7th Annual Conference on Robot Learning},
abstract = {Reinforcement learning (RL) is recognized as lacking generalization and robustness under environmental perturbations, which excessively restricts its application for real-world robotics. Prior work claimed that adding regularization to the value function is equivalent to learning a robust policy under uncertain transitions. Although the regularization-robustness transformation is appealing for its simplicity and efficiency, it is still lacking in continuous control tasks. In this paper, we propose a new regularizer named Uncertainty Set Regularizer (USR), to formulate the uncertainty set on the parametric space of a transition function. To deal with unknown uncertainty sets, we further propose a novel adversarial approach to generate them based on the value function. We evaluate USR on the Real-world Reinforcement Learning (RWRL) benchmark and the Unitree A1 Robot, demonstrating improvements in the robust performance of perturbed testing environments and sim-to-real scenarios.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Reinforcement learning (RL) is recognized as lacking generalization and robustness under environmental perturbations, which excessively restricts its application for real-world robotics. Prior work claimed that adding regularization to the value function is equivalent to learning a robust policy under uncertain transitions. Although the regularization-robustness transformation is appealing for its simplicity and efficiency, it is still lacking in continuous control tasks. In this paper, we propose a new regularizer named Uncertainty Set Regularizer (USR), to formulate the uncertainty set on the parametric space of a transition function. To deal with unknown uncertainty sets, we further propose a novel adversarial approach to generate them based on the value function. We evaluate USR on the Real-world Reinforcement Learning (RWRL) benchmark and the Unitree A1 Robot, demonstrating improvements in the robust performance of perturbed testing environments and sim-to-real scenarios.
66.
Mamedov, Shamil; Reiter, Rudolf; Azad, Seyed Mahdi Basiri; Boedecker, Joschka; Diehl, Moritz; Swevers, Jan
Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots Working paper
2023, (Submitted to ICRA 2024).
@workingpaper{mamedov2023safe,
title = {Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots},
author = {Shamil Mamedov and Rudolf Reiter and Seyed Mahdi Basiri Azad and Joschka Boedecker and Moritz Diehl and Jan Swevers},
url = {https://doi.org/10.48550/arXiv.2212.02941},
year = {2023},
date = {2023-09-28},
abstract = {Flexible robots may overcome some of the industry's major challenges, such as enabling intrinsically safe human-robot collaboration and achieving a higher load-to-mass ratio. However, controlling flexible robots is complicated due to their complex dynamics, which include oscillatory behavior and a high-dimensional state space. NMPC offers an effective means to control such robots, but its extensive computational demands often limit its application in real-time scenarios. To enable fast control of flexible robots, we propose a framework for a safe approximation of NMPC using imitation learning and a predictive safety filter. Our framework significantly reduces computation time while incurring a slight loss in performance. Compared to NMPC, our framework shows more than a eightfold improvement in computation time when controlling a three-dimensional flexible robot arm in simulation, all while guaranteeing safety constraints. Notably, our approach outperforms conventional reinforcement learning methods. The development of fast and safe approximate NMPC holds the potential to accelerate the adoption of flexible robots in industry.},
note = {Submitted to ICRA 2024},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
Flexible robots may overcome some of the industry's major challenges, such as enabling intrinsically safe human-robot collaboration and achieving a higher load-to-mass ratio. However, controlling flexible robots is complicated due to their complex dynamics, which include oscillatory behavior and a high-dimensional state space. NMPC offers an effective means to control such robots, but its extensive computational demands often limit its application in real-time scenarios. To enable fast control of flexible robots, we propose a framework for a safe approximation of NMPC using imitation learning and a predictive safety filter. Our framework significantly reduces computation time while incurring a slight loss in performance. Compared to NMPC, our framework shows more than a eightfold improvement in computation time when controlling a three-dimensional flexible robot arm in simulation, all while guaranteeing safety constraints. Notably, our approach outperforms conventional reinforcement learning methods. The development of fast and safe approximate NMPC holds the potential to accelerate the adoption of flexible robots in industry.
67.
Bonassi, Fabio; Bella, Alessio La; Panzani, Giulio; Farina, Marcello; Scattolini, Riccardo
Deep Long-Short Term Memory networks: Stability properties and Experimental validation Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{bonassi2023deep,
title = {Deep Long-Short Term Memory networks: Stability properties and Experimental validation},
author = {Fabio Bonassi and Alessio La Bella and Giulio Panzani and Marcello Farina and Riccardo Scattolini},
url = {https://ieeexplore.ieee.org/document/10178405
http://arxiv.org/abs/2304.02975},
doi = {https://doi.org/10.23919/ECC57647.2023.10178405},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-25},
urldate = {2023-07-25},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {The aim of this work is to investigate the use of Incrementally Input-to-State Stable (δISS) deep Long Short Term Memory networks (LSTMs) for the identification of nonlinear dynamical systems. We show that suitable sufficient conditions on the weights of the network can be leveraged to setup a training procedure able to learn provenly-δISS LSTM models from data. The proposed approach is tested on a real brake-by-wire apparatus to identify a model of the system from input-output experimentally collected data. Results show satisfactory modeling performances.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The aim of this work is to investigate the use of Incrementally Input-to-State Stable (δISS) deep Long Short Term Memory networks (LSTMs) for the identification of nonlinear dynamical systems. We show that suitable sufficient conditions on the weights of the network can be leveraged to setup a training procedure able to learn provenly-δISS LSTM models from data. The proposed approach is tested on a real brake-by-wire apparatus to identify a model of the system from input-output experimentally collected data. Results show satisfactory modeling performances.
68.
Allamaa, Jean Pierre; Patrinos, Panagiotis; Auweraer, Herman Van; Son, Tong Duy
Safety Envelope for Orthogonal Collocation Methods in Embedded Optimal Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Allamaa2022SafetyEF,
title = {Safety Envelope for Orthogonal Collocation Methods in Embedded Optimal Control},
author = {Jean Pierre Allamaa and Panagiotis Patrinos and Herman Van Auweraer and Tong Duy Son},
url = {https://ieeexplore.ieee.org/document/10178116
https://arxiv.org/abs/2211.14853},
doi = {https://doi.org/10.23919/ECC57647.2023.10178116},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Orthogonal collocation methods are direct approaches for solving optimal control problems (OCP). A high solution accuracy is achieved with few optimization variables, making it more favorable for embedded and real-time NMPC applications. However, collocation approaches lack a guarantee about the safety of the resulting trajectory as inequality constraints are only set on a finite number of collocation points. In this paper we propose a method to efficiently create a convex safety envelope containing the trajectory such that the solution fully satisfies the OCP constraints. We make use of Bernstein approximations of a polynomial’s extrema and span the solution over an orthogonal basis using Legendre polynomials. The tightness of the safety envelope estimation, high accuracy in solving the underlying differential equations, fast rate of convergence and little conservatism are properties of the presented approach making it a suitable method for safe real-time NMPC deployment. We show that our method has comparable computational performance to pseudospectral approaches and can accurately approximate the original OCP up to 9 times more quickly than standard multiple-shooting method in autonomous driving applications, without adding complexity to the formulation.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Orthogonal collocation methods are direct approaches for solving optimal control problems (OCP). A high solution accuracy is achieved with few optimization variables, making it more favorable for embedded and real-time NMPC applications. However, collocation approaches lack a guarantee about the safety of the resulting trajectory as inequality constraints are only set on a finite number of collocation points. In this paper we propose a method to efficiently create a convex safety envelope containing the trajectory such that the solution fully satisfies the OCP constraints. We make use of Bernstein approximations of a polynomial’s extrema and span the solution over an orthogonal basis using Legendre polynomials. The tightness of the safety envelope estimation, high accuracy in solving the underlying differential equations, fast rate of convergence and little conservatism are properties of the presented approach making it a suitable method for safe real-time NMPC deployment. We show that our method has comparable computational performance to pseudospectral approaches and can accurately approximate the original OCP up to 9 times more quickly than standard multiple-shooting method in autonomous driving applications, without adding complexity to the formulation.
69.
Wang, Renzi; Schuurmans, Mathijs; Patrinos, Panagiotis
Interaction-aware Model Predictive Control for Autonomous Driving Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Wang2023,
title = {Interaction-aware Model Predictive Control for Autonomous Driving},
author = {Renzi Wang and Mathijs Schuurmans and Panagiotis Patrinos},
url = {https://ieeexplore.ieee.org/document/10178332
https://arxiv.org/abs/2211.17053},
doi = {https://doi.org/10.23919/ECC57647.2023.10178332},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We propose an interaction-aware stochastic model predictive control (MPC) strategy for lane merging tasks in automated driving. The MPC strategy is integrated with an online learning framework, which models a given driver’s cooperation level as an unknown parameter in a state-dependent probability distribution. The online learning framework adaptively estimates the surrounding vehicle’s cooperation level with the vehicle’s past state trajectory and combines this with a kinematic vehicle model to predict the distribution of a multimodal future state trajectory. Learning is conducted using logistic regression, enabling fast online computations. The multi-future prediction is used in the MPC algorithm to compute the optimal control input while satisfying safety constraints. We demonstrate our algorithm in an interactive lane changing scenario with drivers in different randomly selected cooperation levels.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We propose an interaction-aware stochastic model predictive control (MPC) strategy for lane merging tasks in automated driving. The MPC strategy is integrated with an online learning framework, which models a given driver’s cooperation level as an unknown parameter in a state-dependent probability distribution. The online learning framework adaptively estimates the surrounding vehicle’s cooperation level with the vehicle’s past state trajectory and combines this with a kinematic vehicle model to predict the distribution of a multimodal future state trajectory. Learning is conducted using logistic regression, enabling fast online computations. The multi-future prediction is used in the MPC algorithm to compute the optimal control input while satisfying safety constraints. We demonstrate our algorithm in an interactive lane changing scenario with drivers in different randomly selected cooperation levels.
70.
Roy, Wim Van; Abbasi-Esfeden, Ramin; Swevers, Jan
A Dynamic Programming-based Heuristic Approach for Unit Commitment Problems Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-8, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{vanRoy2023ADP,
title = {A Dynamic Programming-based Heuristic Approach for Unit Commitment Problems},
author = {Wim Van Roy and Ramin Abbasi-Esfeden and Jan Swevers},
url = {https://ieeexplore.ieee.org/document/10178216},
doi = {https://doi.org/10.23919/ECC57647.2023.10178216},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-8},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Unit Commitment (UC) problems are an essential set of problems in the power industry with applications in energy grid or heating systems management and control. The engineering goal is to balance the demand with the production of a network of production units, called generators, by providing a schedule and operating points for each generator cost-effectively while considering constraints. The constraints are caused by the dynamics of the system, the limits on the reserves, and possible robustness requirements. Due to the appearance of the on/off states from the generators, the resulting problems are NP-hard to solve. Thus, existing techniques to achieve a cost-efficient solution are computationally expensive. This paper proposes a dynamic programming-based heuristic to solve a UC problem. The heuristic focuses on finding a feasible and cost-effective solution for systems with a limited number of generators where a long time horizon is important. This method is compared to a Mixed Integer Linear Program (MILP) implementation for a micro-grid where it achieves a computation time that is an order of magnitude smaller than MILP programs for problems with a limited number of generators but a long time horizon.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Unit Commitment (UC) problems are an essential set of problems in the power industry with applications in energy grid or heating systems management and control. The engineering goal is to balance the demand with the production of a network of production units, called generators, by providing a schedule and operating points for each generator cost-effectively while considering constraints. The constraints are caused by the dynamics of the system, the limits on the reserves, and possible robustness requirements. Due to the appearance of the on/off states from the generators, the resulting problems are NP-hard to solve. Thus, existing techniques to achieve a cost-efficient solution are computationally expensive. This paper proposes a dynamic programming-based heuristic to solve a UC problem. The heuristic focuses on finding a feasible and cost-effective solution for systems with a limited number of generators where a long time horizon is important. This method is compared to a Mixed Integer Linear Program (MILP) implementation for a micro-grid where it achieves a computation time that is an order of magnitude smaller than MILP programs for problems with a limited number of generators but a long time horizon.
71.
Abbasi-Esfeden, Ramin; Roy, Wim Van; Swevers, Jan
Iterative Switching Time Optimization for Mixed-integer Optimal Control Problems Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{AbbasiEsfeden2023STO,
title = {Iterative Switching Time Optimization for Mixed-integer Optimal Control Problems},
author = {Ramin Abbasi-Esfeden and Wim Van Roy and Jan Swevers},
url = {https://ieeexplore.ieee.org/document/10178419},
doi = {https://doi.org/10.23919/ECC57647.2023.10178419},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {This paper proposes an iterative method to solve Mixed-Integer Optimal Control Problems arising from systems with switched dynamics. The so-called relaxed problem plays a central role within this context. Through a numerical example, it is shown why relying on the relaxed problem can lead the solution astray. As an alternative, an iterative Switching Time optimization method is proposed. The method consists of two components that iteratively interact: a Switching Time optimization (STO) problem and a sequence optimization. Each component is explained in detail, and the numerical example is resolved, the results of which shows the efficiency of the proposed algorithm. Finally, the advantages and disadvantages of the method are discussed and future lines of research are sketched.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper proposes an iterative method to solve Mixed-Integer Optimal Control Problems arising from systems with switched dynamics. The so-called relaxed problem plays a central role within this context. Through a numerical example, it is shown why relying on the relaxed problem can lead the solution astray. As an alternative, an iterative Switching Time optimization method is proposed. The method consists of two components that iteratively interact: a Switching Time optimization (STO) problem and a sequence optimization. Each component is explained in detail, and the numerical example is resolved, the results of which shows the efficiency of the proposed algorithm. Finally, the advantages and disadvantages of the method are discussed and future lines of research are sketched.
72.
Simpson, Léo; Nurkanovic, Armin; Diehl, Moritz
Direct Collocation for Numerical Optimal Control of Second-Order ODE Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Simpson2023DCSODE,
title = {Direct Collocation for Numerical Optimal Control of Second-Order ODE},
author = {Léo Simpson and Armin Nurkanovic and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178181},
doi = {https://doi.org/10.23919/ECC57647.2023.10178181},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Mechanical systems are usually modeled by second-order Ordinary Differential Equations (ODE) which take the form q¨=f(t, q, q˙). While simulation methods tailored to these equations have been studied, using them in direct optimal control methods is rare. Indeed, the standard approach is to perform a state augmentation, adding the velocities to the state. The main drawback of this approach is that the number of decision variables is doubled, which could harm the performance of the resulting optimization problem. In this paper, we present an approach tailored to second-order ODE. We compare it with the standard one, both on theoretical aspects and in a numerical example. Notably, we show that the tailored formulation is likely to improve the performance of a direct collocation method, for solving optimal control problems with second-order ODE of the more restrictive form q¨=f(t, q).},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Mechanical systems are usually modeled by second-order Ordinary Differential Equations (ODE) which take the form q¨=f(t, q, q˙). While simulation methods tailored to these equations have been studied, using them in direct optimal control methods is rare. Indeed, the standard approach is to perform a state augmentation, adding the velocities to the state. The main drawback of this approach is that the number of decision variables is doubled, which could harm the performance of the resulting optimization problem. In this paper, we present an approach tailored to second-order ODE. We compare it with the standard one, both on theoretical aspects and in a numerical example. Notably, we show that the tailored formulation is likely to improve the performance of a direct collocation method, for solving optimal control problems with second-order ODE of the more restrictive form q¨=f(t, q).
73.
Gao, Yunfan; Messerer, Florian; Frey, Jonathan; Duijkeren, Niels; Diehl, Moritz
Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{GaoCFMPECC23,
title = {Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC},
author = {Yunfan Gao and Florian Messerer and Jonathan Frey and Niels Duijkeren and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178311},
doi = {https://doi.org/10.23919/ECC57647.2023.10178311},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied [1]. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied [1]. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.
74.
Baumgärtner, Katrin; Diehl, Moritz
Local Convergence Analysis of Damping for Zero-Order Optimization-Based Iterative Learning Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Baumgaertner2023,
title = {Local Convergence Analysis of Damping for Zero-Order Optimization-Based Iterative Learning Control},
author = {Katrin Baumgärtner and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178225},
doi = {https://doi.org/10.23919/ECC57647.2023.10178225},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Within the Iterative Learning Control (ILC) framework, damping is often introduced as a heuristic to facilitate convergence of the ILC iterates. We analyze how two simple damping approaches affect the local convergence behaviour of a zero-order optimization-based ILC method introduced in [1] and prove that the condition for local convergence, which is given in terms of the eigenvalues of an iteration matrix, can be relaxed if damping is introduced. Leveraging a simple example, we illustrate the effects of damping, which might be (1) convergence of an initially diverging iteration or (2) acceleration or deceleration of a converging iteration.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Within the Iterative Learning Control (ILC) framework, damping is often introduced as a heuristic to facilitate convergence of the ILC iterates. We analyze how two simple damping approaches affect the local convergence behaviour of a zero-order optimization-based ILC method introduced in [1] and prove that the condition for local convergence, which is given in terms of the eigenvalues of an iteration matrix, can be relaxed if damping is introduced. Leveraging a simple example, we illustrate the effects of damping, which might be (1) convergence of an initially diverging iteration or (2) acceleration or deceleration of a converging iteration.
75.
Baumgärtner, Katrin; Messerer, Florian; Diehl, Moritz
A Unified Local Convergence Analysis of Differential Dynamic Programming, Direct Single Shooting, and Direct Multiple Shooting Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Baumgaertner2023a,
title = {A Unified Local Convergence Analysis of Differential Dynamic Programming, Direct Single Shooting, and Direct Multiple Shooting},
author = {Katrin Baumgärtner and Florian Messerer and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178367},
doi = {https://doi.org/10.23919/ECC57647.2023.10178367},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We revisit three classical numerical methods for solving unconstrained optimal control problems - differential dynamic programming, direct single shooting, and direct multiple shooting - and examine their local convergence behaviour. In particular, we show that all three methods converge with the same linear rate if a Gauss-Newton (GN) - or Generalized Gauss-Newton (GGN) - Hessian approximation is used, which is the case in widely used implementations such as iLQR.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We revisit three classical numerical methods for solving unconstrained optimal control problems - differential dynamic programming, direct single shooting, and direct multiple shooting - and examine their local convergence behaviour. In particular, we show that all three methods converge with the same linear rate if a Gauss-Newton (GN) - or Generalized Gauss-Newton (GGN) - Hessian approximation is used, which is the case in widely used implementations such as iLQR.
76.
Ghezzi, Andrea; Simpson, Léo; Bürger, Adrian; Zeile, Clemens; Sager, Sebastian; Diehl, Moritz
A Voronoi-Based Mixed-Integer Gauss-Newton Algorithm for MINLP Arising in Optimal Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Ghezzi2023a,
title = { A Voronoi-Based Mixed-Integer Gauss-Newton Algorithm for MINLP Arising in Optimal Control},
author = {Andrea Ghezzi and Léo Simpson and Adrian Bürger and Clemens Zeile and Sebastian Sager and Moritz Diehl},
doi = {https://doi.org/10.23919/ECC57647.2023.10178130},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We present a new algorithm for addressing nonconvex Mixed-Integer Nonlinear Programs (MINLPs) where the cost function is of nonlinear least squares form. We exploit this structure by leveraging a Gauss-Newton quadratic approximation of the original MINLP, leading to the formulation of a Mixed-Integer Quadratic Program (MIQP), which can be solved efficiently. The integer solution of the MIQP is used to fix the integer variables of the original MINLP, resulting in a standard Nonlinear Program. We introduce an iterative procedure to repeat the optimization of the two programs in order to improve the solution. To guide the iterations towards unexplored regions, we devise a strategy to partition the integer solution space based on Voronoi diagrams. Finally, we first illustrate the algorithm on a simple example of MINLP and then test it on an example of real-world complexity concerning the optimal control of an energy system. Here, the new algorithm outperforms state-of-the-art methods, finding a solution with a lower objective value, at the cost of requiring an increased runtime compared to other approximate methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present a new algorithm for addressing nonconvex Mixed-Integer Nonlinear Programs (MINLPs) where the cost function is of nonlinear least squares form. We exploit this structure by leveraging a Gauss-Newton quadratic approximation of the original MINLP, leading to the formulation of a Mixed-Integer Quadratic Program (MIQP), which can be solved efficiently. The integer solution of the MIQP is used to fix the integer variables of the original MINLP, resulting in a standard Nonlinear Program. We introduce an iterative procedure to repeat the optimization of the two programs in order to improve the solution. To guide the iterations towards unexplored regions, we devise a strategy to partition the integer solution space based on Voronoi diagrams. Finally, we first illustrate the algorithm on a simple example of MINLP and then test it on an example of real-world complexity concerning the optimal control of an energy system. Here, the new algorithm outperforms state-of-the-art methods, finding a solution with a lower objective value, at the cost of requiring an increased runtime compared to other approximate methods.
77.
Reiter, Rudolf; Hoffman, Jasper; Boedecker, Joschka; Diehl, Moritz
A Hierarchical Approach for Strategic Motion Planning in Autonomous Racing Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-8, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Reiter2023Hierarchical,
title = {A Hierarchical Approach for Strategic Motion Planning in Autonomous Racing},
author = {Rudolf Reiter and Jasper Hoffman and Joschka Boedecker and Moritz Diehl},
doi = {10.23919/ECC57647.2023.10178143},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-8},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We present an approach for safe trajectory planning, where a strategic task related to autonomous racing is learned sample efficiently within a simulation environment. A high-level policy, represented as a neural network, outputs a reward specification that is used within the function of a parametric nonlinear model predictive controller. By including constraints and vehicle kinematics in the nonlinear program, we can guarantee safe and feasible trajectories related to the used model. Compared to classical reinforcement learning, our approach restricts the exploration to safe trajectories, starts with an excellent prior performance and yields complete trajectories that can be passed to a tracking lowest-level controller. We do not address the lowest-level controller in this work and assume perfect tracking of feasible trajectories. We show the superior performance of our algorithm on simulated racing tasks that include high-level decision-making. The vehicle learns to efficiently overtake slower vehicles and avoids getting overtaken by blocking faster ones.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present an approach for safe trajectory planning, where a strategic task related to autonomous racing is learned sample efficiently within a simulation environment. A high-level policy, represented as a neural network, outputs a reward specification that is used within the function of a parametric nonlinear model predictive controller. By including constraints and vehicle kinematics in the nonlinear program, we can guarantee safe and feasible trajectories related to the used model. Compared to classical reinforcement learning, our approach restricts the exploration to safe trajectories, starts with an excellent prior performance and yields complete trajectories that can be passed to a tracking lowest-level controller. We do not address the lowest-level controller in this work and assume perfect tracking of feasible trajectories. We show the superior performance of our algorithm on simulated racing tasks that include high-level decision-making. The vehicle learns to efficiently overtake slower vehicles and avoids getting overtaken by blocking faster ones.
78.
Xie, Jing; Bonassi, Fabio; Farina, Marcello; Scattolini, Riccardo
Robust offset-free nonlinear model predictive control for systems learned by neural nonlinear autoregressive exogenous models Journal Article
In: International Journal of Robust and Nonlinear Control, vol. 33, no. 16, pp. 9992-10009, 2023.
@article{xie2023robust,
title = {Robust offset-free nonlinear model predictive control for systems learned by neural nonlinear autoregressive exogenous models},
author = {Jing Xie and Fabio Bonassi and Marcello Farina and Riccardo Scattolini},
url = {https://onlinelibrary.wiley.com/doi/full/10.1002/rnc.6883},
doi = {10.1002/rnc.6883},
year = {2023},
date = {2023-07-13},
urldate = {2023-07-13},
journal = {International Journal of Robust and Nonlinear Control},
volume = {33},
number = {16},
pages = {9992-10009},
publisher = {arXiv},
abstract = {This paper presents a robust Model Predictive Control (MPC) scheme that provides offset-free setpoint tracking for systems described by Neural Nonlinear AutoRegressive eXogenous (NNARX) models. The NNARX model learns the dynamics of the plant from input-output data, and during the training the Incremental Input-to-State Stability (𝛿ISS) property is forced to guarantee stability. The trained NNARX model is then augmented with an explicit integral action on the output tracking error, which allows the control scheme to enjoy offset-free tracking ability. A tube-based MPC is finally designed, leveraging the unique structure of the model, to ensure robust stability and robust asymptotic zero error regulation for constant reference signals in the presence of model-plant mismatch or unknown disturbances. Numerical simulations on a water heating system show the effectiveness of the proposed control algorithm.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This paper presents a robust Model Predictive Control (MPC) scheme that provides offset-free setpoint tracking for systems described by Neural Nonlinear AutoRegressive eXogenous (NNARX) models. The NNARX model learns the dynamics of the plant from input-output data, and during the training the Incremental Input-to-State Stability (𝛿ISS) property is forced to guarantee stability. The trained NNARX model is then augmented with an explicit integral action on the output tracking error, which allows the control scheme to enjoy offset-free tracking ability. A tube-based MPC is finally designed, leveraging the unique structure of the model, to ensure robust stability and robust asymptotic zero error regulation for constant reference signals in the presence of model-plant mismatch or unknown disturbances. Numerical simulations on a water heating system show the effectiveness of the proposed control algorithm.
79.
Løwenstein, Kristoffer Fink; Fagiano, Lorenzo; Bernardini, Daniele; Bemporad, Alberto
Physics-Informed Online Learning of Gray-box Models by Moving Horizon Estimation Journal Article
In: European Journal of Control, pp. 100861, 2023, ISSN: 0947-3580.
@article{Lowenstein2023PhysicsInformed,
title = {Physics-Informed Online Learning of Gray-box Models by Moving Horizon Estimation},
author = {Kristoffer Fink Løwenstein and Lorenzo Fagiano and Daniele Bernardini and Alberto Bemporad},
url = {https://www.sciencedirect.com/science/article/pii/S0947358023000900},
doi = {10.1016/j.ejcon.2023.100861},
issn = {0947-3580},
year = {2023},
date = {2023-07-03},
urldate = {2023-07-03},
journal = {European Journal of Control},
pages = {100861},
abstract = {A simple yet expressive prediction model is an essential ingredient in model-based control and estimation. Models derived from fundamental physical principles may fail to capture the complexity of the actual system dynamics. A potential solution is the use of a physics-informed, or gray-box model that extends a physics-based model with a data-driven part. Learning the latter might be challenging, due to noisy measurements and lack of full state information. This work presents a method based on Moving Horizon Estimation (MHE) for simultaneous state estimation and training of a black-box submodel, such as a neural network. The method can be used in offline training or applied online for adaptation without any prior knowledge than the white-box submodel. We analyze the capabilities of the method in a two degree of freedom robotic manipulator case study, also showing how it can be used for online adaptation to cope with a time-varying model mismatch.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
A simple yet expressive prediction model is an essential ingredient in model-based control and estimation. Models derived from fundamental physical principles may fail to capture the complexity of the actual system dynamics. A potential solution is the use of a physics-informed, or gray-box model that extends a physics-based model with a data-driven part. Learning the latter might be challenging, due to noisy measurements and lack of full state information. This work presents a method based on Moving Horizon Estimation (MHE) for simultaneous state estimation and training of a black-box submodel, such as a neural network. The method can be used in offline training or applied online for adaptation without any prior knowledge than the white-box submodel. We analyze the capabilities of the method in a two degree of freedom robotic manipulator case study, also showing how it can be used for online adaptation to cope with a time-varying model mismatch.
80.
Schuurmans, Mathijs; Katriniok, Alexander; Meissen, Christopher; Tseng, H. Eric; Patrinos, Panagiotis
Safe, Learning-Based MPC for Highway Driving under Lane-Change Uncertainty: A Distributionally Robust Approach Journal Article
In: Artificial Intelligence, vol. 320, pp. 103920, 2023, ISSN: 0004-3702.
@article{schuurmansSafeLearningBasedMPC2022,
title = {Safe, Learning-Based MPC for Highway Driving under Lane-Change Uncertainty: A Distributionally Robust Approach},
author = {Mathijs Schuurmans and Alexander Katriniok and Christopher Meissen and H. Eric Tseng and Panagiotis Patrinos},
url = {https://www.sciencedirect.com/science/article/pii/S0004370223000668},
doi = {https://doi.org/10.1016/j.artint.2023.103920},
issn = {0004-3702},
year = {2023},
date = {2023-07-01},
urldate = {2022-01-01},
journal = {Artificial Intelligence},
volume = {320},
pages = {103920},
publisher = {arXiv},
abstract = {We present a case study applying learning-based distributionally robust model predictive control to highway motion planning under stochastic uncertainty of the lane change behavior of surrounding road users. The dynamics of road users are modeled using Markov jump systems, in which the switching variable describes the desired lane of the vehicle under consideration and the continuous state describes the pose and velocity of the vehicles. We assume the switching probabilities of the underlying Markov chain to be unknown. As the vehicle is observed and thus, samples from the Markov chain are drawn, the transition probabilities are estimated along with an ambiguity set which accounts for misestimations of these probabilities. Correspondingly, a distributionally robust optimal control problem is formulated over a scenario tree, and solved in receding horizon. As a result, a motion planning procedure is obtained which through observation of the target vehicle gradually becomes less conservative while avoiding overconfidence in estimates obtained from small sample sizes. We present an extensive numerical case study, comparing the effects of several different design aspects on the controller performance and safety.
},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present a case study applying learning-based distributionally robust model predictive control to highway motion planning under stochastic uncertainty of the lane change behavior of surrounding road users. The dynamics of road users are modeled using Markov jump systems, in which the switching variable describes the desired lane of the vehicle under consideration and the continuous state describes the pose and velocity of the vehicles. We assume the switching probabilities of the underlying Markov chain to be unknown. As the vehicle is observed and thus, samples from the Markov chain are drawn, the transition probabilities are estimated along with an ambiguity set which accounts for misestimations of these probabilities. Correspondingly, a distributionally robust optimal control problem is formulated over a scenario tree, and solved in receding horizon. As a result, a motion planning procedure is obtained which through observation of the target vehicle gradually becomes less conservative while avoiding overconfidence in estimates obtained from small sample sizes. We present an extensive numerical case study, comparing the effects of several different design aspects on the controller performance and safety.