Publications
21.
Mamedov, Shamil; Reiter, Rudolf; Azad, Seyed Mahdi Basiri; Boedecker, Joschka; Diehl, Moritz; Swevers, Jan
Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots Working paper
2023, (Submitted to ICRA 2024).
@workingpaper{mamedov2023safe,
title = {Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots},
author = {Shamil Mamedov and Rudolf Reiter and Seyed Mahdi Basiri Azad and Joschka Boedecker and Moritz Diehl and Jan Swevers},
url = {https://doi.org/10.48550/arXiv.2212.02941},
year = {2023},
date = {2023-09-28},
abstract = {Flexible robots may overcome some of the industry's major challenges, such as enabling intrinsically safe human-robot collaboration and achieving a higher load-to-mass ratio. However, controlling flexible robots is complicated due to their complex dynamics, which include oscillatory behavior and a high-dimensional state space. NMPC offers an effective means to control such robots, but its extensive computational demands often limit its application in real-time scenarios. To enable fast control of flexible robots, we propose a framework for a safe approximation of NMPC using imitation learning and a predictive safety filter. Our framework significantly reduces computation time while incurring a slight loss in performance. Compared to NMPC, our framework shows more than a eightfold improvement in computation time when controlling a three-dimensional flexible robot arm in simulation, all while guaranteeing safety constraints. Notably, our approach outperforms conventional reinforcement learning methods. The development of fast and safe approximate NMPC holds the potential to accelerate the adoption of flexible robots in industry.},
note = {Submitted to ICRA 2024},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
Flexible robots may overcome some of the industry's major challenges, such as enabling intrinsically safe human-robot collaboration and achieving a higher load-to-mass ratio. However, controlling flexible robots is complicated due to their complex dynamics, which include oscillatory behavior and a high-dimensional state space. NMPC offers an effective means to control such robots, but its extensive computational demands often limit its application in real-time scenarios. To enable fast control of flexible robots, we propose a framework for a safe approximation of NMPC using imitation learning and a predictive safety filter. Our framework significantly reduces computation time while incurring a slight loss in performance. Compared to NMPC, our framework shows more than a eightfold improvement in computation time when controlling a three-dimensional flexible robot arm in simulation, all while guaranteeing safety constraints. Notably, our approach outperforms conventional reinforcement learning methods. The development of fast and safe approximate NMPC holds the potential to accelerate the adoption of flexible robots in industry.
22.
Ghezzi, Andrea; Hoffman, Jasper; Frey, Jonathan; Boedecker, Joschka; Diehl, Moritz
Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation Proceedings Article Forthcoming
In: 2023 Conference on Decision and Control (CDC), Forthcoming.
@inproceedings{Ghezzi2023b,
title = {Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation},
author = {Andrea Ghezzi and Jasper Hoffman and Jonathan Frey and Joschka Boedecker and Moritz Diehl},
doi = {https://doi.org/10.48550/arXiv.2304.01782},
year = {2023},
date = {2023-08-17},
booktitle = {2023 Conference on Decision and Control (CDC)},
abstract = {This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs.
23.
Messerer, Florian; Baumgärtner, Katrin; Nurkanovic, Armin; Diehl, Moritz
Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems Working paper
2023.
@workingpaper{messerer2023approximate,
title = {Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems},
author = {Florian Messerer and Katrin Baumgärtner and Armin Nurkanovic and Moritz Diehl},
url = {https://arxiv.org/abs/2308.03431},
doi = {https://doi.org/10.48550/arXiv.2308.03431},
year = {2023},
date = {2023-08-14},
abstract = {We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.
24.
Bonassi, Fabio; Bella, Alessio La; Panzani, Giulio; Farina, Marcello; Scattolini, Riccardo
Deep Long-Short Term Memory networks: Stability properties and Experimental validation Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{bonassi2023deep,
title = {Deep Long-Short Term Memory networks: Stability properties and Experimental validation},
author = {Fabio Bonassi and Alessio La Bella and Giulio Panzani and Marcello Farina and Riccardo Scattolini},
url = {https://ieeexplore.ieee.org/document/10178405
http://arxiv.org/abs/2304.02975},
doi = {https://doi.org/10.23919/ECC57647.2023.10178405},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-25},
urldate = {2023-07-25},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {The aim of this work is to investigate the use of Incrementally Input-to-State Stable (δISS) deep Long Short Term Memory networks (LSTMs) for the identification of nonlinear dynamical systems. We show that suitable sufficient conditions on the weights of the network can be leveraged to setup a training procedure able to learn provenly-δISS LSTM models from data. The proposed approach is tested on a real brake-by-wire apparatus to identify a model of the system from input-output experimentally collected data. Results show satisfactory modeling performances.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The aim of this work is to investigate the use of Incrementally Input-to-State Stable (δISS) deep Long Short Term Memory networks (LSTMs) for the identification of nonlinear dynamical systems. We show that suitable sufficient conditions on the weights of the network can be leveraged to setup a training procedure able to learn provenly-δISS LSTM models from data. The proposed approach is tested on a real brake-by-wire apparatus to identify a model of the system from input-output experimentally collected data. Results show satisfactory modeling performances.
25.
Allamaa, Jean Pierre; Patrinos, Panagiotis; Auweraer, Herman Van; Son, Tong Duy
Safety Envelope for Orthogonal Collocation Methods in Embedded Optimal Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Allamaa2022SafetyEF,
title = {Safety Envelope for Orthogonal Collocation Methods in Embedded Optimal Control},
author = {Jean Pierre Allamaa and Panagiotis Patrinos and Herman Van Auweraer and Tong Duy Son},
url = {https://ieeexplore.ieee.org/document/10178116
https://arxiv.org/abs/2211.14853},
doi = {https://doi.org/10.23919/ECC57647.2023.10178116},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Orthogonal collocation methods are direct approaches for solving optimal control problems (OCP). A high solution accuracy is achieved with few optimization variables, making it more favorable for embedded and real-time NMPC applications. However, collocation approaches lack a guarantee about the safety of the resulting trajectory as inequality constraints are only set on a finite number of collocation points. In this paper we propose a method to efficiently create a convex safety envelope containing the trajectory such that the solution fully satisfies the OCP constraints. We make use of Bernstein approximations of a polynomial’s extrema and span the solution over an orthogonal basis using Legendre polynomials. The tightness of the safety envelope estimation, high accuracy in solving the underlying differential equations, fast rate of convergence and little conservatism are properties of the presented approach making it a suitable method for safe real-time NMPC deployment. We show that our method has comparable computational performance to pseudospectral approaches and can accurately approximate the original OCP up to 9 times more quickly than standard multiple-shooting method in autonomous driving applications, without adding complexity to the formulation.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Orthogonal collocation methods are direct approaches for solving optimal control problems (OCP). A high solution accuracy is achieved with few optimization variables, making it more favorable for embedded and real-time NMPC applications. However, collocation approaches lack a guarantee about the safety of the resulting trajectory as inequality constraints are only set on a finite number of collocation points. In this paper we propose a method to efficiently create a convex safety envelope containing the trajectory such that the solution fully satisfies the OCP constraints. We make use of Bernstein approximations of a polynomial’s extrema and span the solution over an orthogonal basis using Legendre polynomials. The tightness of the safety envelope estimation, high accuracy in solving the underlying differential equations, fast rate of convergence and little conservatism are properties of the presented approach making it a suitable method for safe real-time NMPC deployment. We show that our method has comparable computational performance to pseudospectral approaches and can accurately approximate the original OCP up to 9 times more quickly than standard multiple-shooting method in autonomous driving applications, without adding complexity to the formulation.
26.
Wang, Renzi; Schuurmans, Mathijs; Patrinos, Panagiotis
Interaction-aware Model Predictive Control for Autonomous Driving Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Wang2023,
title = {Interaction-aware Model Predictive Control for Autonomous Driving},
author = {Renzi Wang and Mathijs Schuurmans and Panagiotis Patrinos},
url = {https://ieeexplore.ieee.org/document/10178332
https://arxiv.org/abs/2211.17053},
doi = {https://doi.org/10.23919/ECC57647.2023.10178332},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We propose an interaction-aware stochastic model predictive control (MPC) strategy for lane merging tasks in automated driving. The MPC strategy is integrated with an online learning framework, which models a given driver’s cooperation level as an unknown parameter in a state-dependent probability distribution. The online learning framework adaptively estimates the surrounding vehicle’s cooperation level with the vehicle’s past state trajectory and combines this with a kinematic vehicle model to predict the distribution of a multimodal future state trajectory. Learning is conducted using logistic regression, enabling fast online computations. The multi-future prediction is used in the MPC algorithm to compute the optimal control input while satisfying safety constraints. We demonstrate our algorithm in an interactive lane changing scenario with drivers in different randomly selected cooperation levels.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We propose an interaction-aware stochastic model predictive control (MPC) strategy for lane merging tasks in automated driving. The MPC strategy is integrated with an online learning framework, which models a given driver’s cooperation level as an unknown parameter in a state-dependent probability distribution. The online learning framework adaptively estimates the surrounding vehicle’s cooperation level with the vehicle’s past state trajectory and combines this with a kinematic vehicle model to predict the distribution of a multimodal future state trajectory. Learning is conducted using logistic regression, enabling fast online computations. The multi-future prediction is used in the MPC algorithm to compute the optimal control input while satisfying safety constraints. We demonstrate our algorithm in an interactive lane changing scenario with drivers in different randomly selected cooperation levels.
27.
Roy, Wim Van; Abbasi-Esfeden, Ramin; Swevers, Jan
A Dynamic Programming-based Heuristic Approach for Unit Commitment Problems Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-8, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{vanRoy2023ADP,
title = {A Dynamic Programming-based Heuristic Approach for Unit Commitment Problems},
author = {Wim Van Roy and Ramin Abbasi-Esfeden and Jan Swevers},
url = {https://ieeexplore.ieee.org/document/10178216},
doi = {https://doi.org/10.23919/ECC57647.2023.10178216},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-8},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Unit Commitment (UC) problems are an essential set of problems in the power industry with applications in energy grid or heating systems management and control. The engineering goal is to balance the demand with the production of a network of production units, called generators, by providing a schedule and operating points for each generator cost-effectively while considering constraints. The constraints are caused by the dynamics of the system, the limits on the reserves, and possible robustness requirements. Due to the appearance of the on/off states from the generators, the resulting problems are NP-hard to solve. Thus, existing techniques to achieve a cost-efficient solution are computationally expensive. This paper proposes a dynamic programming-based heuristic to solve a UC problem. The heuristic focuses on finding a feasible and cost-effective solution for systems with a limited number of generators where a long time horizon is important. This method is compared to a Mixed Integer Linear Program (MILP) implementation for a micro-grid where it achieves a computation time that is an order of magnitude smaller than MILP programs for problems with a limited number of generators but a long time horizon.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Unit Commitment (UC) problems are an essential set of problems in the power industry with applications in energy grid or heating systems management and control. The engineering goal is to balance the demand with the production of a network of production units, called generators, by providing a schedule and operating points for each generator cost-effectively while considering constraints. The constraints are caused by the dynamics of the system, the limits on the reserves, and possible robustness requirements. Due to the appearance of the on/off states from the generators, the resulting problems are NP-hard to solve. Thus, existing techniques to achieve a cost-efficient solution are computationally expensive. This paper proposes a dynamic programming-based heuristic to solve a UC problem. The heuristic focuses on finding a feasible and cost-effective solution for systems with a limited number of generators where a long time horizon is important. This method is compared to a Mixed Integer Linear Program (MILP) implementation for a micro-grid where it achieves a computation time that is an order of magnitude smaller than MILP programs for problems with a limited number of generators but a long time horizon.
28.
Abbasi-Esfeden, Ramin; Roy, Wim Van; Swevers, Jan
Iterative Switching Time Optimization for Mixed-integer Optimal Control Problems Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{AbbasiEsfeden2023STO,
title = {Iterative Switching Time Optimization for Mixed-integer Optimal Control Problems},
author = {Ramin Abbasi-Esfeden and Wim Van Roy and Jan Swevers},
url = {https://ieeexplore.ieee.org/document/10178419},
doi = {https://doi.org/10.23919/ECC57647.2023.10178419},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {This paper proposes an iterative method to solve Mixed-Integer Optimal Control Problems arising from systems with switched dynamics. The so-called relaxed problem plays a central role within this context. Through a numerical example, it is shown why relying on the relaxed problem can lead the solution astray. As an alternative, an iterative Switching Time optimization method is proposed. The method consists of two components that iteratively interact: a Switching Time optimization (STO) problem and a sequence optimization. Each component is explained in detail, and the numerical example is resolved, the results of which shows the efficiency of the proposed algorithm. Finally, the advantages and disadvantages of the method are discussed and future lines of research are sketched.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper proposes an iterative method to solve Mixed-Integer Optimal Control Problems arising from systems with switched dynamics. The so-called relaxed problem plays a central role within this context. Through a numerical example, it is shown why relying on the relaxed problem can lead the solution astray. As an alternative, an iterative Switching Time optimization method is proposed. The method consists of two components that iteratively interact: a Switching Time optimization (STO) problem and a sequence optimization. Each component is explained in detail, and the numerical example is resolved, the results of which shows the efficiency of the proposed algorithm. Finally, the advantages and disadvantages of the method are discussed and future lines of research are sketched.
29.
Simpson, Léo; Nurkanovic, Armin; Diehl, Moritz
Direct Collocation for Numerical Optimal Control of Second-Order ODE Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Simpson2023DCSODE,
title = {Direct Collocation for Numerical Optimal Control of Second-Order ODE},
author = {Léo Simpson and Armin Nurkanovic and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178181},
doi = {https://doi.org/10.23919/ECC57647.2023.10178181},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Mechanical systems are usually modeled by second-order Ordinary Differential Equations (ODE) which take the form q¨=f(t, q, q˙). While simulation methods tailored to these equations have been studied, using them in direct optimal control methods is rare. Indeed, the standard approach is to perform a state augmentation, adding the velocities to the state. The main drawback of this approach is that the number of decision variables is doubled, which could harm the performance of the resulting optimization problem. In this paper, we present an approach tailored to second-order ODE. We compare it with the standard one, both on theoretical aspects and in a numerical example. Notably, we show that the tailored formulation is likely to improve the performance of a direct collocation method, for solving optimal control problems with second-order ODE of the more restrictive form q¨=f(t, q).},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Mechanical systems are usually modeled by second-order Ordinary Differential Equations (ODE) which take the form q¨=f(t, q, q˙). While simulation methods tailored to these equations have been studied, using them in direct optimal control methods is rare. Indeed, the standard approach is to perform a state augmentation, adding the velocities to the state. The main drawback of this approach is that the number of decision variables is doubled, which could harm the performance of the resulting optimization problem. In this paper, we present an approach tailored to second-order ODE. We compare it with the standard one, both on theoretical aspects and in a numerical example. Notably, we show that the tailored formulation is likely to improve the performance of a direct collocation method, for solving optimal control problems with second-order ODE of the more restrictive form q¨=f(t, q).
30.
Gao, Yunfan; Messerer, Florian; Frey, Jonathan; Duijkeren, Niels; Diehl, Moritz
Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{GaoCFMPECC23,
title = {Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC},
author = {Yunfan Gao and Florian Messerer and Jonathan Frey and Niels Duijkeren and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178311},
doi = {https://doi.org/10.23919/ECC57647.2023.10178311},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied [1]. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied [1]. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.
31.
Baumgärtner, Katrin; Diehl, Moritz
Local Convergence Analysis of Damping for Zero-Order Optimization-Based Iterative Learning Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Baumgaertner2023,
title = {Local Convergence Analysis of Damping for Zero-Order Optimization-Based Iterative Learning Control},
author = {Katrin Baumgärtner and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178225},
doi = {https://doi.org/10.23919/ECC57647.2023.10178225},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Within the Iterative Learning Control (ILC) framework, damping is often introduced as a heuristic to facilitate convergence of the ILC iterates. We analyze how two simple damping approaches affect the local convergence behaviour of a zero-order optimization-based ILC method introduced in [1] and prove that the condition for local convergence, which is given in terms of the eigenvalues of an iteration matrix, can be relaxed if damping is introduced. Leveraging a simple example, we illustrate the effects of damping, which might be (1) convergence of an initially diverging iteration or (2) acceleration or deceleration of a converging iteration.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Within the Iterative Learning Control (ILC) framework, damping is often introduced as a heuristic to facilitate convergence of the ILC iterates. We analyze how two simple damping approaches affect the local convergence behaviour of a zero-order optimization-based ILC method introduced in [1] and prove that the condition for local convergence, which is given in terms of the eigenvalues of an iteration matrix, can be relaxed if damping is introduced. Leveraging a simple example, we illustrate the effects of damping, which might be (1) convergence of an initially diverging iteration or (2) acceleration or deceleration of a converging iteration.
32.
Baumgärtner, Katrin; Messerer, Florian; Diehl, Moritz
A Unified Local Convergence Analysis of Differential Dynamic Programming, Direct Single Shooting, and Direct Multiple Shooting Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Baumgaertner2023a,
title = {A Unified Local Convergence Analysis of Differential Dynamic Programming, Direct Single Shooting, and Direct Multiple Shooting},
author = {Katrin Baumgärtner and Florian Messerer and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178367},
doi = {https://doi.org/10.23919/ECC57647.2023.10178367},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We revisit three classical numerical methods for solving unconstrained optimal control problems - differential dynamic programming, direct single shooting, and direct multiple shooting - and examine their local convergence behaviour. In particular, we show that all three methods converge with the same linear rate if a Gauss-Newton (GN) - or Generalized Gauss-Newton (GGN) - Hessian approximation is used, which is the case in widely used implementations such as iLQR.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We revisit three classical numerical methods for solving unconstrained optimal control problems - differential dynamic programming, direct single shooting, and direct multiple shooting - and examine their local convergence behaviour. In particular, we show that all three methods converge with the same linear rate if a Gauss-Newton (GN) - or Generalized Gauss-Newton (GGN) - Hessian approximation is used, which is the case in widely used implementations such as iLQR.
33.
Ghezzi, Andrea; Simpson, Léo; Bürger, Adrian; Zeile, Clemens; Sager, Sebastian; Diehl, Moritz
A Voronoi-Based Mixed-Integer Gauss-Newton Algorithm for MINLP Arising in Optimal Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Ghezzi2023a,
title = { A Voronoi-Based Mixed-Integer Gauss-Newton Algorithm for MINLP Arising in Optimal Control},
author = {Andrea Ghezzi and Léo Simpson and Adrian Bürger and Clemens Zeile and Sebastian Sager and Moritz Diehl},
doi = {https://doi.org/10.23919/ECC57647.2023.10178130},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We present a new algorithm for addressing nonconvex Mixed-Integer Nonlinear Programs (MINLPs) where the cost function is of nonlinear least squares form. We exploit this structure by leveraging a Gauss-Newton quadratic approximation of the original MINLP, leading to the formulation of a Mixed-Integer Quadratic Program (MIQP), which can be solved efficiently. The integer solution of the MIQP is used to fix the integer variables of the original MINLP, resulting in a standard Nonlinear Program. We introduce an iterative procedure to repeat the optimization of the two programs in order to improve the solution. To guide the iterations towards unexplored regions, we devise a strategy to partition the integer solution space based on Voronoi diagrams. Finally, we first illustrate the algorithm on a simple example of MINLP and then test it on an example of real-world complexity concerning the optimal control of an energy system. Here, the new algorithm outperforms state-of-the-art methods, finding a solution with a lower objective value, at the cost of requiring an increased runtime compared to other approximate methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present a new algorithm for addressing nonconvex Mixed-Integer Nonlinear Programs (MINLPs) where the cost function is of nonlinear least squares form. We exploit this structure by leveraging a Gauss-Newton quadratic approximation of the original MINLP, leading to the formulation of a Mixed-Integer Quadratic Program (MIQP), which can be solved efficiently. The integer solution of the MIQP is used to fix the integer variables of the original MINLP, resulting in a standard Nonlinear Program. We introduce an iterative procedure to repeat the optimization of the two programs in order to improve the solution. To guide the iterations towards unexplored regions, we devise a strategy to partition the integer solution space based on Voronoi diagrams. Finally, we first illustrate the algorithm on a simple example of MINLP and then test it on an example of real-world complexity concerning the optimal control of an energy system. Here, the new algorithm outperforms state-of-the-art methods, finding a solution with a lower objective value, at the cost of requiring an increased runtime compared to other approximate methods.
34.
Reiter, Rudolf; Hoffman, Jasper; Boedecker, Joschka; Diehl, Moritz
A Hierarchical Approach for Strategic Motion Planning in Autonomous Racing Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-8, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Reiter2023Hierarchical,
title = {A Hierarchical Approach for Strategic Motion Planning in Autonomous Racing},
author = {Rudolf Reiter and Jasper Hoffman and Joschka Boedecker and Moritz Diehl},
doi = {10.23919/ECC57647.2023.10178143},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-8},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We present an approach for safe trajectory planning, where a strategic task related to autonomous racing is learned sample efficiently within a simulation environment. A high-level policy, represented as a neural network, outputs a reward specification that is used within the function of a parametric nonlinear model predictive controller. By including constraints and vehicle kinematics in the nonlinear program, we can guarantee safe and feasible trajectories related to the used model. Compared to classical reinforcement learning, our approach restricts the exploration to safe trajectories, starts with an excellent prior performance and yields complete trajectories that can be passed to a tracking lowest-level controller. We do not address the lowest-level controller in this work and assume perfect tracking of feasible trajectories. We show the superior performance of our algorithm on simulated racing tasks that include high-level decision-making. The vehicle learns to efficiently overtake slower vehicles and avoids getting overtaken by blocking faster ones.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present an approach for safe trajectory planning, where a strategic task related to autonomous racing is learned sample efficiently within a simulation environment. A high-level policy, represented as a neural network, outputs a reward specification that is used within the function of a parametric nonlinear model predictive controller. By including constraints and vehicle kinematics in the nonlinear program, we can guarantee safe and feasible trajectories related to the used model. Compared to classical reinforcement learning, our approach restricts the exploration to safe trajectories, starts with an excellent prior performance and yields complete trajectories that can be passed to a tracking lowest-level controller. We do not address the lowest-level controller in this work and assume perfect tracking of feasible trajectories. We show the superior performance of our algorithm on simulated racing tasks that include high-level decision-making. The vehicle learns to efficiently overtake slower vehicles and avoids getting overtaken by blocking faster ones.
35.
Xie, Jing; Bonassi, Fabio; Farina, Marcello; Scattolini, Riccardo
Robust offset-free nonlinear model predictive control for systems learned by neural nonlinear autoregressive exogenous models Journal Article
In: International Journal of Robust and Nonlinear Control, 2023.
@article{xie2022robust,
title = {Robust offset-free nonlinear model predictive control for systems learned by neural nonlinear autoregressive exogenous models},
author = {Jing Xie and Fabio Bonassi and Marcello Farina and Riccardo Scattolini},
url = {https://doi.org/10.1002/rnc.6883
http://arxiv.org/abs/2210.06801},
doi = {10.1002/rnc.6883},
year = {2023},
date = {2023-07-13},
urldate = {2023-07-13},
journal = {International Journal of Robust and Nonlinear Control},
publisher = {arXiv},
abstract = {This paper presents a robust Model Predictive Control (MPC) scheme that provides offset-free setpoint tracking for systems described by Neural Nonlinear AutoRegressive eXogenous (NNARX) models. The NNARX model learns the dynamics of the plant from input-output data, and during the training the Incremental Input-to-State Stability (𝛿ISS) property is forced to guarantee stability. The trained NNARX model is then augmented with an explicit integral action on the output tracking error, which allows the control scheme to enjoy offset-free tracking ability. A tube-based MPC is finally designed, leveraging the unique structure of the model, to ensure robust stability and robust asymptotic zero error regulation for constant reference signals in the presence of model-plant mismatch or unknown disturbances. Numerical simulations on a water heating system show the effectiveness of the proposed control algorithm.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This paper presents a robust Model Predictive Control (MPC) scheme that provides offset-free setpoint tracking for systems described by Neural Nonlinear AutoRegressive eXogenous (NNARX) models. The NNARX model learns the dynamics of the plant from input-output data, and during the training the Incremental Input-to-State Stability (𝛿ISS) property is forced to guarantee stability. The trained NNARX model is then augmented with an explicit integral action on the output tracking error, which allows the control scheme to enjoy offset-free tracking ability. A tube-based MPC is finally designed, leveraging the unique structure of the model, to ensure robust stability and robust asymptotic zero error regulation for constant reference signals in the presence of model-plant mismatch or unknown disturbances. Numerical simulations on a water heating system show the effectiveness of the proposed control algorithm.
36.
Løwenstein, Kristoffer Fink; Fagiano, Lorenzo; Bernardini, Daniele; Bemporad, Alberto
Physics-Informed Online Learning of Gray-box Models by Moving Horizon Estimation Journal Article
In: European Journal of Control, pp. 100861, 2023, ISSN: 0947-3580.
@article{Lowenstein2023PhysicsInformed,
title = {Physics-Informed Online Learning of Gray-box Models by Moving Horizon Estimation},
author = {Kristoffer Fink Løwenstein and Lorenzo Fagiano and Daniele Bernardini and Alberto Bemporad},
url = {https://www.sciencedirect.com/science/article/pii/S0947358023000900},
doi = {10.1016/j.ejcon.2023.100861},
issn = {0947-3580},
year = {2023},
date = {2023-07-03},
urldate = {2023-07-03},
journal = {European Journal of Control},
pages = {100861},
abstract = {A simple yet expressive prediction model is an essential ingredient in model-based control and estimation. Models derived from fundamental physical principles may fail to capture the complexity of the actual system dynamics. A potential solution is the use of a physics-informed, or gray-box model that extends a physics-based model with a data-driven part. Learning the latter might be challenging, due to noisy measurements and lack of full state information. This work presents a method based on Moving Horizon Estimation (MHE) for simultaneous state estimation and training of a black-box submodel, such as a neural network. The method can be used in offline training or applied online for adaptation without any prior knowledge than the white-box submodel. We analyze the capabilities of the method in a two degree of freedom robotic manipulator case study, also showing how it can be used for online adaptation to cope with a time-varying model mismatch.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
A simple yet expressive prediction model is an essential ingredient in model-based control and estimation. Models derived from fundamental physical principles may fail to capture the complexity of the actual system dynamics. A potential solution is the use of a physics-informed, or gray-box model that extends a physics-based model with a data-driven part. Learning the latter might be challenging, due to noisy measurements and lack of full state information. This work presents a method based on Moving Horizon Estimation (MHE) for simultaneous state estimation and training of a black-box submodel, such as a neural network. The method can be used in offline training or applied online for adaptation without any prior knowledge than the white-box submodel. We analyze the capabilities of the method in a two degree of freedom robotic manipulator case study, also showing how it can be used for online adaptation to cope with a time-varying model mismatch.
37.
Schuurmans, Mathijs; Katriniok, Alexander; Meissen, Christopher; Tseng, H. Eric; Patrinos, Panagiotis
Safe, Learning-Based MPC for Highway Driving under Lane-Change Uncertainty: A Distributionally Robust Approach Journal Article
In: Artificial Intelligence, vol. 320, pp. 103920, 2023, ISSN: 0004-3702.
@article{schuurmansSafeLearningBasedMPC2022,
title = {Safe, Learning-Based MPC for Highway Driving under Lane-Change Uncertainty: A Distributionally Robust Approach},
author = {Mathijs Schuurmans and Alexander Katriniok and Christopher Meissen and H. Eric Tseng and Panagiotis Patrinos},
url = {https://www.sciencedirect.com/science/article/pii/S0004370223000668},
doi = {https://doi.org/10.1016/j.artint.2023.103920},
issn = {0004-3702},
year = {2023},
date = {2023-07-01},
urldate = {2022-01-01},
journal = {Artificial Intelligence},
volume = {320},
pages = {103920},
publisher = {arXiv},
abstract = {We present a case study applying learning-based distributionally robust model predictive control to highway motion planning under stochastic uncertainty of the lane change behavior of surrounding road users. The dynamics of road users are modeled using Markov jump systems, in which the switching variable describes the desired lane of the vehicle under consideration and the continuous state describes the pose and velocity of the vehicles. We assume the switching probabilities of the underlying Markov chain to be unknown. As the vehicle is observed and thus, samples from the Markov chain are drawn, the transition probabilities are estimated along with an ambiguity set which accounts for misestimations of these probabilities. Correspondingly, a distributionally robust optimal control problem is formulated over a scenario tree, and solved in receding horizon. As a result, a motion planning procedure is obtained which through observation of the target vehicle gradually becomes less conservative while avoiding overconfidence in estimates obtained from small sample sizes. We present an extensive numerical case study, comparing the effects of several different design aspects on the controller performance and safety.
},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present a case study applying learning-based distributionally robust model predictive control to highway motion planning under stochastic uncertainty of the lane change behavior of surrounding road users. The dynamics of road users are modeled using Markov jump systems, in which the switching variable describes the desired lane of the vehicle under consideration and the continuous state describes the pose and velocity of the vehicles. We assume the switching probabilities of the underlying Markov chain to be unknown. As the vehicle is observed and thus, samples from the Markov chain are drawn, the transition probabilities are estimated along with an ambiguity set which accounts for misestimations of these probabilities. Correspondingly, a distributionally robust optimal control problem is formulated over a scenario tree, and solved in receding horizon. As a result, a motion planning procedure is obtained which through observation of the target vehicle gradually becomes less conservative while avoiding overconfidence in estimates obtained from small sample sizes. We present an extensive numerical case study, comparing the effects of several different design aspects on the controller performance and safety.
38.
Yan, Schengchao; Zhang, Yuan; Zhang, Baohe; Boedecker, Joschka; Burgard, Wolfram
Geometric Regularity with Robot Intrinsic Symmetry in Reinforcement Learning Proceedings Article
In: RSS 2023 Workshop on Symmetries in Robot Learning, 2023.
@inproceedings{yan2023geometric,
title = {Geometric Regularity with Robot Intrinsic Symmetry in Reinforcement Learning},
author = {Schengchao Yan and Yuan Zhang and Baohe Zhang and Joschka Boedecker and Wolfram Burgard},
url = {https://doi.org/10.48550/arXiv.2306.16316},
year = {2023},
date = {2023-06-28},
urldate = {2023-06-28},
booktitle = {RSS 2023 Workshop on Symmetries in Robot Learning},
abstract = {Geometric regularity, which leverages data symmetry, has been successfully incorporated into deep learning architectures such as CNNs, RNNs, GNNs, and Transformers. While this concept has been widely applied in robotics to address the curse of dimensionality when learning from high-dimensional data, the inherent reflectional and rotational symmetry of robot structures has not been adequately explored. Drawing inspiration from cooperative multi-agent reinforcement learning, we introduce novel network structures for deep learning algorithms that explicitly capture this geometric regularity. Moreover, we investigate the relationship between the geometric prior and the concept of Parameter Sharing in multi-agent reinforcement learning. Through experiments conducted on various challenging continuous control tasks, we demonstrate the significant potential of the proposed geometric regularity in enhancing robot learning capabilities.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Geometric regularity, which leverages data symmetry, has been successfully incorporated into deep learning architectures such as CNNs, RNNs, GNNs, and Transformers. While this concept has been widely applied in robotics to address the curse of dimensionality when learning from high-dimensional data, the inherent reflectional and rotational symmetry of robot structures has not been adequately explored. Drawing inspiration from cooperative multi-agent reinforcement learning, we introduce novel network structures for deep learning algorithms that explicitly capture this geometric regularity. Moreover, we investigate the relationship between the geometric prior and the concept of Parameter Sharing in multi-agent reinforcement learning. Through experiments conducted on various challenging continuous control tasks, we demonstrate the significant potential of the proposed geometric regularity in enhancing robot learning capabilities.
39.
Baumgärtner, Katrin; Zanelli, Andrea; Diehl, Moritz
Stability Analysis of Nonlinear Model Predictive Control with Progressive Tightening of Stage Costs and Constraints Journal Article
In: IEEE Control Systems Letters, vol. 7, pp. 3018-3023, 2023, ISSN: 2475-1456.
@article{Baumgaertner2023b,
title = {Stability Analysis of Nonlinear Model Predictive Control with Progressive Tightening of Stage Costs and Constraints},
author = {Katrin Baumgärtner and Andrea Zanelli and Moritz Diehl},
doi = {https://doi.org/10.1109/LCSYS.2023.3289707},
issn = {2475-1456},
year = {2023},
date = {2023-06-26},
journal = {IEEE Control Systems Letters},
volume = {7},
pages = {3018-3023},
abstract = {We consider a stage-varying nonlinear model predictive control (NMPC) formulation and provide a stability result for the corresponding closed-loop system under the assumption that cost and constraints are progressively tightening. We illustrate the generality of the stage-varying formulation pointing out various approaches proposed in the literature that can be cast as stage-varying and progressively tightening optimal control problems.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We consider a stage-varying nonlinear model predictive control (NMPC) formulation and provide a stability result for the corresponding closed-loop system under the assumption that cost and constraints are progressively tightening. We illustrate the generality of the stage-varying formulation pointing out various approaches proposed in the literature that can be cast as stage-varying and progressively tightening optimal control problems.
40.
Lahr, Amon; Zanelli, Andrea; Carron, Andrea; Zeilinger, Melanie N.
Zero-Order Optimization for Gaussian Process-based Model Predictive Control Journal Article
In: European Journal of Control, pp. 100862, 2023, ISSN: 0947-3580.
@article{lahrZeroOrderOptimizationGaussian2023,
title = {Zero-Order Optimization for Gaussian Process-based Model Predictive Control},
author = {Amon Lahr and Andrea Zanelli and Andrea Carron and Melanie N. Zeilinger},
url = {https://www.sciencedirect.com/science/article/pii/S0947358023000912},
doi = {10.1016/j.ejcon.2023.100862},
issn = {0947-3580},
year = {2023},
date = {2023-06-15},
urldate = {2023-07-18},
journal = {European Journal of Control},
pages = {100862},
abstract = {By enabling constraint-aware online model adaptation, model predictive control using Gaussian process (GP) regression has exhibited impressive performance in real-world applications and received considerable attention in the learning-based control community. Yet, solving the resulting optimal control problem in real-time generally remains a major challenge, due to (i) the increased number of augmented states in the optimization problem, as well as (ii) computationally expensive evaluations of the posterior mean and covariance and their respective derivatives. To tackle these challenges, we employ (i) a tailored Jacobian approximation in a sequential quadratic programming (SQP) approach and combine it with (ii) a parallelizable GP inference and automatic differentiation framework. Reducing the numerical complexity with respect to the state dimension nx for each SQP iteration from O(nx6) to O(nx3), and accelerating GP evaluations on a graphical processing unit, the proposed algorithm computes suboptimal, yet feasible, solutions at drastically reduced computation times and exhibits favorable local convergence properties. Numerical experiments verify the scaling properties and investigate the runtime distribution across different parts of the algorithm.
},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
By enabling constraint-aware online model adaptation, model predictive control using Gaussian process (GP) regression has exhibited impressive performance in real-world applications and received considerable attention in the learning-based control community. Yet, solving the resulting optimal control problem in real-time generally remains a major challenge, due to (i) the increased number of augmented states in the optimization problem, as well as (ii) computationally expensive evaluations of the posterior mean and covariance and their respective derivatives. To tackle these challenges, we employ (i) a tailored Jacobian approximation in a sequential quadratic programming (SQP) approach and combine it with (ii) a parallelizable GP inference and automatic differentiation framework. Reducing the numerical complexity with respect to the state dimension nx for each SQP iteration from O(nx6) to O(nx3), and accelerating GP evaluations on a graphical processing unit, the proposed algorithm computes suboptimal, yet feasible, solutions at drastically reduced computation times and exhibits favorable local convergence properties. Numerical experiments verify the scaling properties and investigate the runtime distribution across different parts of the algorithm.