Publications
1.
Roy, Wim Van; Nurkanovic, Armin; Abbasi-Esfeden, Ramin; Frey, Jonathan; Pozharskiy, Anton; Swevers, Jan; Diehl, Moritz
Continuous Optimization for Control of Finite-State Machines with Cascaded Hysteresis Via Time-Freezing Proceedings Article Forthcoming
In: 2023 Conference on Decision and Control (CDC), Forthcoming.
@inproceedings{VanRoy2023CDC,
title = {Continuous Optimization for Control of Finite-State Machines with Cascaded Hysteresis Via Time-Freezing},
author = {Wim Van Roy and Armin Nurkanovic and Ramin Abbasi-Esfeden and Jonathan Frey and Anton Pozharskiy and Jan Swevers and Moritz Diehl},
year = {2023},
date = {2023-12-01},
booktitle = {2023 Conference on Decision and Control (CDC)},
abstract = {Control problems with Finite-State Machines (FSM) are often solved using integer variables, leading to a mixed-integer optimal control problem (MIOCP). This paper proposes analternative method to describe a subclass of FSMs using complementarity constraints and time-freezing. The FSM from this subclass is built up by a sequence of states where a transition between the states is triggered by a single switching function. This can be looked at as a cascade of hysteresis loops where a memory effect is used to maintain the active state of the state machine. Based on the reformulation for hybrid systems with a hysteresis loop, a method is developed to reformulate this subclass in a similar fashion. The approach transforms the original problem into a Piecewise Smooth System (PSS), which can be discretized using the recently developed Finite Elements with Switch Detection, allowing for high-accuracy solutions. The reformation is compared to a mixed-integer formulation from the literature on a time-optimal control problem. This work is a first step towards the general reformulation of FSMs into nonsmooth systems without integer states.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
Control problems with Finite-State Machines (FSM) are often solved using integer variables, leading to a mixed-integer optimal control problem (MIOCP). This paper proposes analternative method to describe a subclass of FSMs using complementarity constraints and time-freezing. The FSM from this subclass is built up by a sequence of states where a transition between the states is triggered by a single switching function. This can be looked at as a cascade of hysteresis loops where a memory effect is used to maintain the active state of the state machine. Based on the reformulation for hybrid systems with a hysteresis loop, a method is developed to reformulate this subclass in a similar fashion. The approach transforms the original problem into a Piecewise Smooth System (PSS), which can be discretized using the recently developed Finite Elements with Switch Detection, allowing for high-accuracy solutions. The reformation is compared to a mixed-integer formulation from the literature on a time-optimal control problem. This work is a first step towards the general reformulation of FSMs into nonsmooth systems without integer states.
2.
Ghezzi, Andrea; Hoffman, Jasper; Frey, Jonathan; Boedecker, Joschka; Diehl, Moritz
Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation Proceedings Article Forthcoming
In: 2023 Conference on Decision and Control (CDC), Forthcoming.
@inproceedings{Ghezzi2023b,
title = {Imitation Learning from Nonlinear MPC via the Exact Q-Loss and its Gauss-Newton Approximation},
author = {Andrea Ghezzi and Jasper Hoffman and Jonathan Frey and Joschka Boedecker and Moritz Diehl},
doi = {https://doi.org/10.48550/arXiv.2304.01782},
year = {2023},
date = {2023-08-17},
booktitle = {2023 Conference on Decision and Control (CDC)},
abstract = {This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
This work presents a novel loss function for learning nonlinear Model Predictive Control policies via Imitation Learning. Standard approaches to Imitation Learning neglect information about the expert and generally adopt a loss function based on the distance between expert and learned controls. In this work, we present a loss based on the Q-function directly embedding the performance objectives and constraint satisfaction of the associated Optimal Control Problem (OCP). However, training a Neural Network with the Q-loss requires solving the associated OCP for each new sample. To alleviate the computational burden, we derive a second Q-loss based on the Gauss-Newton approximation of the OCP resulting in a faster training time. We validate our losses against Behavioral Cloning, the standard approach to Imitation Learning, on the control of a nonlinear system with constraints. The final results show that the Q-function-based losses significantly reduce the amount of constraint violations while achieving comparable or better closed-loop costs.
3.
Messerer, Florian; Baumgärtner, Katrin; Nurkanovic, Armin; Diehl, Moritz
Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems Working paper
2023.
@workingpaper{messerer2023approximate,
title = {Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems},
author = {Florian Messerer and Katrin Baumgärtner and Armin Nurkanovic and Moritz Diehl},
url = {https://arxiv.org/abs/2308.03431},
doi = {https://doi.org/10.48550/arXiv.2308.03431},
year = {2023},
date = {2023-08-14},
abstract = {We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.},
keywords = {},
pubstate = {published},
tppubtype = {workingpaper}
}
We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.
4.
Bonassi, Fabio; Bella, Alessio La; Panzani, Giulio; Farina, Marcello; Scattolini, Riccardo
Deep Long-Short Term Memory networks: Stability properties and Experimental validation Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{bonassi2023deep,
title = {Deep Long-Short Term Memory networks: Stability properties and Experimental validation},
author = {Fabio Bonassi and Alessio La Bella and Giulio Panzani and Marcello Farina and Riccardo Scattolini},
url = {https://ieeexplore.ieee.org/document/10178405
http://arxiv.org/abs/2304.02975},
doi = {https://doi.org/10.23919/ECC57647.2023.10178405},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-25},
urldate = {2023-07-25},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {The aim of this work is to investigate the use of Incrementally Input-to-State Stable (δISS) deep Long Short Term Memory networks (LSTMs) for the identification of nonlinear dynamical systems. We show that suitable sufficient conditions on the weights of the network can be leveraged to setup a training procedure able to learn provenly-δISS LSTM models from data. The proposed approach is tested on a real brake-by-wire apparatus to identify a model of the system from input-output experimentally collected data. Results show satisfactory modeling performances.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The aim of this work is to investigate the use of Incrementally Input-to-State Stable (δISS) deep Long Short Term Memory networks (LSTMs) for the identification of nonlinear dynamical systems. We show that suitable sufficient conditions on the weights of the network can be leveraged to setup a training procedure able to learn provenly-δISS LSTM models from data. The proposed approach is tested on a real brake-by-wire apparatus to identify a model of the system from input-output experimentally collected data. Results show satisfactory modeling performances.
5.
Allamaa, Jean Pierre; Patrinos, Panagiotis; Auweraer, Herman Van; Son, Tong Duy
Safety Envelope for Orthogonal Collocation Methods in Embedded Optimal Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Allamaa2022SafetyEF,
title = {Safety Envelope for Orthogonal Collocation Methods in Embedded Optimal Control},
author = {Jean Pierre Allamaa and Panagiotis Patrinos and Herman Van Auweraer and Tong Duy Son},
url = {https://ieeexplore.ieee.org/document/10178116
https://arxiv.org/abs/2211.14853},
doi = {https://doi.org/10.23919/ECC57647.2023.10178116},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Orthogonal collocation methods are direct approaches for solving optimal control problems (OCP). A high solution accuracy is achieved with few optimization variables, making it more favorable for embedded and real-time NMPC applications. However, collocation approaches lack a guarantee about the safety of the resulting trajectory as inequality constraints are only set on a finite number of collocation points. In this paper we propose a method to efficiently create a convex safety envelope containing the trajectory such that the solution fully satisfies the OCP constraints. We make use of Bernstein approximations of a polynomial’s extrema and span the solution over an orthogonal basis using Legendre polynomials. The tightness of the safety envelope estimation, high accuracy in solving the underlying differential equations, fast rate of convergence and little conservatism are properties of the presented approach making it a suitable method for safe real-time NMPC deployment. We show that our method has comparable computational performance to pseudospectral approaches and can accurately approximate the original OCP up to 9 times more quickly than standard multiple-shooting method in autonomous driving applications, without adding complexity to the formulation.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Orthogonal collocation methods are direct approaches for solving optimal control problems (OCP). A high solution accuracy is achieved with few optimization variables, making it more favorable for embedded and real-time NMPC applications. However, collocation approaches lack a guarantee about the safety of the resulting trajectory as inequality constraints are only set on a finite number of collocation points. In this paper we propose a method to efficiently create a convex safety envelope containing the trajectory such that the solution fully satisfies the OCP constraints. We make use of Bernstein approximations of a polynomial’s extrema and span the solution over an orthogonal basis using Legendre polynomials. The tightness of the safety envelope estimation, high accuracy in solving the underlying differential equations, fast rate of convergence and little conservatism are properties of the presented approach making it a suitable method for safe real-time NMPC deployment. We show that our method has comparable computational performance to pseudospectral approaches and can accurately approximate the original OCP up to 9 times more quickly than standard multiple-shooting method in autonomous driving applications, without adding complexity to the formulation.
6.
Wang, Renzi; Schuurmans, Mathijs; Patrinos, Panagiotis
Interaction-aware Model Predictive Control for Autonomous Driving Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Wang2023,
title = {Interaction-aware Model Predictive Control for Autonomous Driving},
author = {Renzi Wang and Mathijs Schuurmans and Panagiotis Patrinos},
url = {https://ieeexplore.ieee.org/document/10178332
https://arxiv.org/abs/2211.17053},
doi = {https://doi.org/10.23919/ECC57647.2023.10178332},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We propose an interaction-aware stochastic model predictive control (MPC) strategy for lane merging tasks in automated driving. The MPC strategy is integrated with an online learning framework, which models a given driver’s cooperation level as an unknown parameter in a state-dependent probability distribution. The online learning framework adaptively estimates the surrounding vehicle’s cooperation level with the vehicle’s past state trajectory and combines this with a kinematic vehicle model to predict the distribution of a multimodal future state trajectory. Learning is conducted using logistic regression, enabling fast online computations. The multi-future prediction is used in the MPC algorithm to compute the optimal control input while satisfying safety constraints. We demonstrate our algorithm in an interactive lane changing scenario with drivers in different randomly selected cooperation levels.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We propose an interaction-aware stochastic model predictive control (MPC) strategy for lane merging tasks in automated driving. The MPC strategy is integrated with an online learning framework, which models a given driver’s cooperation level as an unknown parameter in a state-dependent probability distribution. The online learning framework adaptively estimates the surrounding vehicle’s cooperation level with the vehicle’s past state trajectory and combines this with a kinematic vehicle model to predict the distribution of a multimodal future state trajectory. Learning is conducted using logistic regression, enabling fast online computations. The multi-future prediction is used in the MPC algorithm to compute the optimal control input while satisfying safety constraints. We demonstrate our algorithm in an interactive lane changing scenario with drivers in different randomly selected cooperation levels.
7.
Roy, Wim Van; Abbasi-Esfeden, Ramin; Swevers, Jan
A Dynamic Programming-based Heuristic Approach for Unit Commitment Problems Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-8, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{vanRoy2023ADP,
title = {A Dynamic Programming-based Heuristic Approach for Unit Commitment Problems},
author = {Wim Van Roy and Ramin Abbasi-Esfeden and Jan Swevers},
url = {https://ieeexplore.ieee.org/document/10178216},
doi = {https://doi.org/10.23919/ECC57647.2023.10178216},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-8},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Unit Commitment (UC) problems are an essential set of problems in the power industry with applications in energy grid or heating systems management and control. The engineering goal is to balance the demand with the production of a network of production units, called generators, by providing a schedule and operating points for each generator cost-effectively while considering constraints. The constraints are caused by the dynamics of the system, the limits on the reserves, and possible robustness requirements. Due to the appearance of the on/off states from the generators, the resulting problems are NP-hard to solve. Thus, existing techniques to achieve a cost-efficient solution are computationally expensive. This paper proposes a dynamic programming-based heuristic to solve a UC problem. The heuristic focuses on finding a feasible and cost-effective solution for systems with a limited number of generators where a long time horizon is important. This method is compared to a Mixed Integer Linear Program (MILP) implementation for a micro-grid where it achieves a computation time that is an order of magnitude smaller than MILP programs for problems with a limited number of generators but a long time horizon.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Unit Commitment (UC) problems are an essential set of problems in the power industry with applications in energy grid or heating systems management and control. The engineering goal is to balance the demand with the production of a network of production units, called generators, by providing a schedule and operating points for each generator cost-effectively while considering constraints. The constraints are caused by the dynamics of the system, the limits on the reserves, and possible robustness requirements. Due to the appearance of the on/off states from the generators, the resulting problems are NP-hard to solve. Thus, existing techniques to achieve a cost-efficient solution are computationally expensive. This paper proposes a dynamic programming-based heuristic to solve a UC problem. The heuristic focuses on finding a feasible and cost-effective solution for systems with a limited number of generators where a long time horizon is important. This method is compared to a Mixed Integer Linear Program (MILP) implementation for a micro-grid where it achieves a computation time that is an order of magnitude smaller than MILP programs for problems with a limited number of generators but a long time horizon.
8.
Abbasi-Esfeden, Ramin; Roy, Wim Van; Swevers, Jan
Iterative Switching Time Optimization for Mixed-integer Optimal Control Problems Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{AbbasiEsfeden2023STO,
title = {Iterative Switching Time Optimization for Mixed-integer Optimal Control Problems},
author = {Ramin Abbasi-Esfeden and Wim Van Roy and Jan Swevers},
url = {https://ieeexplore.ieee.org/document/10178419},
doi = {https://doi.org/10.23919/ECC57647.2023.10178419},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {This paper proposes an iterative method to solve Mixed-Integer Optimal Control Problems arising from systems with switched dynamics. The so-called relaxed problem plays a central role within this context. Through a numerical example, it is shown why relying on the relaxed problem can lead the solution astray. As an alternative, an iterative Switching Time optimization method is proposed. The method consists of two components that iteratively interact: a Switching Time optimization (STO) problem and a sequence optimization. Each component is explained in detail, and the numerical example is resolved, the results of which shows the efficiency of the proposed algorithm. Finally, the advantages and disadvantages of the method are discussed and future lines of research are sketched.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper proposes an iterative method to solve Mixed-Integer Optimal Control Problems arising from systems with switched dynamics. The so-called relaxed problem plays a central role within this context. Through a numerical example, it is shown why relying on the relaxed problem can lead the solution astray. As an alternative, an iterative Switching Time optimization method is proposed. The method consists of two components that iteratively interact: a Switching Time optimization (STO) problem and a sequence optimization. Each component is explained in detail, and the numerical example is resolved, the results of which shows the efficiency of the proposed algorithm. Finally, the advantages and disadvantages of the method are discussed and future lines of research are sketched.
9.
Simpson, Léo; Nurkanovic, Armin; Diehl, Moritz
Direct Collocation for Numerical Optimal Control of Second-Order ODE Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Simpson2023DCSODE,
title = {Direct Collocation for Numerical Optimal Control of Second-Order ODE},
author = {Léo Simpson and Armin Nurkanovic and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178181},
doi = {https://doi.org/10.23919/ECC57647.2023.10178181},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Mechanical systems are usually modeled by second-order Ordinary Differential Equations (ODE) which take the form q¨=f(t, q, q˙). While simulation methods tailored to these equations have been studied, using them in direct optimal control methods is rare. Indeed, the standard approach is to perform a state augmentation, adding the velocities to the state. The main drawback of this approach is that the number of decision variables is doubled, which could harm the performance of the resulting optimization problem. In this paper, we present an approach tailored to second-order ODE. We compare it with the standard one, both on theoretical aspects and in a numerical example. Notably, we show that the tailored formulation is likely to improve the performance of a direct collocation method, for solving optimal control problems with second-order ODE of the more restrictive form q¨=f(t, q).},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Mechanical systems are usually modeled by second-order Ordinary Differential Equations (ODE) which take the form q¨=f(t, q, q˙). While simulation methods tailored to these equations have been studied, using them in direct optimal control methods is rare. Indeed, the standard approach is to perform a state augmentation, adding the velocities to the state. The main drawback of this approach is that the number of decision variables is doubled, which could harm the performance of the resulting optimization problem. In this paper, we present an approach tailored to second-order ODE. We compare it with the standard one, both on theoretical aspects and in a numerical example. Notably, we show that the tailored formulation is likely to improve the performance of a direct collocation method, for solving optimal control problems with second-order ODE of the more restrictive form q¨=f(t, q).
10.
Gao, Yunfan; Messerer, Florian; Frey, Jonathan; Duijkeren, Niels; Diehl, Moritz
Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{GaoCFMPECC23,
title = {Collision-free Motion Planning for Mobile Robots by Zero-order Robust Optimization-based MPC},
author = {Yunfan Gao and Florian Messerer and Jonathan Frey and Niels Duijkeren and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178311},
doi = {https://doi.org/10.23919/ECC57647.2023.10178311},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied [1]. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper presents an implementation of robust model predictive control (MPC) for collision-free reference trajectory tracking for mobile robots. The presented approach considers the robot motion to be subject to process noise bounded by ellipsoidal sets. In order to efficiently handle the evolution of the disturbance ellipsoids within the MPC, the zero-order robust optimization (zoRO) scheme is applied [1]. The idea is to fix the disturbance ellipsoids within one optimization iteration and solve the problem repeatedly with updated disturbance ellipsoid trajectories. The zero-order approach is suboptimal in general. However, we show that it does not impair convergence to the reference trajectory in the absence of obstacles. The experiments on an industrial mobile robot prototype demonstrate the performance of the controller.
11.
Baumgärtner, Katrin; Diehl, Moritz
Local Convergence Analysis of Damping for Zero-Order Optimization-Based Iterative Learning Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-6, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Baumgaertner2023,
title = {Local Convergence Analysis of Damping for Zero-Order Optimization-Based Iterative Learning Control},
author = {Katrin Baumgärtner and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178225},
doi = {https://doi.org/10.23919/ECC57647.2023.10178225},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-6},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {Within the Iterative Learning Control (ILC) framework, damping is often introduced as a heuristic to facilitate convergence of the ILC iterates. We analyze how two simple damping approaches affect the local convergence behaviour of a zero-order optimization-based ILC method introduced in [1] and prove that the condition for local convergence, which is given in terms of the eigenvalues of an iteration matrix, can be relaxed if damping is introduced. Leveraging a simple example, we illustrate the effects of damping, which might be (1) convergence of an initially diverging iteration or (2) acceleration or deceleration of a converging iteration.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Within the Iterative Learning Control (ILC) framework, damping is often introduced as a heuristic to facilitate convergence of the ILC iterates. We analyze how two simple damping approaches affect the local convergence behaviour of a zero-order optimization-based ILC method introduced in [1] and prove that the condition for local convergence, which is given in terms of the eigenvalues of an iteration matrix, can be relaxed if damping is introduced. Leveraging a simple example, we illustrate the effects of damping, which might be (1) convergence of an initially diverging iteration or (2) acceleration or deceleration of a converging iteration.
12.
Baumgärtner, Katrin; Messerer, Florian; Diehl, Moritz
A Unified Local Convergence Analysis of Differential Dynamic Programming, Direct Single Shooting, and Direct Multiple Shooting Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Baumgaertner2023a,
title = {A Unified Local Convergence Analysis of Differential Dynamic Programming, Direct Single Shooting, and Direct Multiple Shooting},
author = {Katrin Baumgärtner and Florian Messerer and Moritz Diehl},
url = {https://ieeexplore.ieee.org/document/10178367},
doi = {https://doi.org/10.23919/ECC57647.2023.10178367},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We revisit three classical numerical methods for solving unconstrained optimal control problems - differential dynamic programming, direct single shooting, and direct multiple shooting - and examine their local convergence behaviour. In particular, we show that all three methods converge with the same linear rate if a Gauss-Newton (GN) - or Generalized Gauss-Newton (GGN) - Hessian approximation is used, which is the case in widely used implementations such as iLQR.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We revisit three classical numerical methods for solving unconstrained optimal control problems - differential dynamic programming, direct single shooting, and direct multiple shooting - and examine their local convergence behaviour. In particular, we show that all three methods converge with the same linear rate if a Gauss-Newton (GN) - or Generalized Gauss-Newton (GGN) - Hessian approximation is used, which is the case in widely used implementations such as iLQR.
13.
Ghezzi, Andrea; Simpson, Léo; Bürger, Adrian; Zeile, Clemens; Sager, Sebastian; Diehl, Moritz
A Voronoi-Based Mixed-Integer Gauss-Newton Algorithm for MINLP Arising in Optimal Control Proceedings Article
In: 2023 European Control Conference (ECC), pp. 1-7, IEEE, Bucharest, Romania, 2023, ISBN: 978-3-907144-08-4.
@inproceedings{Ghezzi2023a,
title = { A Voronoi-Based Mixed-Integer Gauss-Newton Algorithm for MINLP Arising in Optimal Control},
author = {Andrea Ghezzi and Léo Simpson and Adrian Bürger and Clemens Zeile and Sebastian Sager and Moritz Diehl},
doi = {https://doi.org/10.23919/ECC57647.2023.10178130},
isbn = {978-3-907144-08-4},
year = {2023},
date = {2023-07-17},
urldate = {2023-07-17},
booktitle = {2023 European Control Conference (ECC)},
pages = {1-7},
publisher = {IEEE},
address = {Bucharest, Romania},
abstract = {We present a new algorithm for addressing nonconvex Mixed-Integer Nonlinear Programs (MINLPs) where the cost function is of nonlinear least squares form. We exploit this structure by leveraging a Gauss-Newton quadratic approximation of the original MINLP, leading to the formulation of a Mixed-Integer Quadratic Program (MIQP), which can be solved efficiently. The integer solution of the MIQP is used to fix the integer variables of the original MINLP, resulting in a standard Nonlinear Program. We introduce an iterative procedure to repeat the optimization of the two programs in order to improve the solution. To guide the iterations towards unexplored regions, we devise a strategy to partition the integer solution space based on Voronoi diagrams. Finally, we first illustrate the algorithm on a simple example of MINLP and then test it on an example of real-world complexity concerning the optimal control of an energy system. Here, the new algorithm outperforms state-of-the-art methods, finding a solution with a lower objective value, at the cost of requiring an increased runtime compared to other approximate methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present a new algorithm for addressing nonconvex Mixed-Integer Nonlinear Programs (MINLPs) where the cost function is of nonlinear least squares form. We exploit this structure by leveraging a Gauss-Newton quadratic approximation of the original MINLP, leading to the formulation of a Mixed-Integer Quadratic Program (MIQP), which can be solved efficiently. The integer solution of the MIQP is used to fix the integer variables of the original MINLP, resulting in a standard Nonlinear Program. We introduce an iterative procedure to repeat the optimization of the two programs in order to improve the solution. To guide the iterations towards unexplored regions, we devise a strategy to partition the integer solution space based on Voronoi diagrams. Finally, we first illustrate the algorithm on a simple example of MINLP and then test it on an example of real-world complexity concerning the optimal control of an energy system. Here, the new algorithm outperforms state-of-the-art methods, finding a solution with a lower objective value, at the cost of requiring an increased runtime compared to other approximate methods.
14.
Xie, Jing; Bonassi, Fabio; Farina, Marcello; Scattolini, Riccardo
Robust offset-free nonlinear model predictive control for systems learned by neural nonlinear autoregressive exogenous models Journal Article
In: International Journal of Robust and Nonlinear Control, 2023.
@article{xie2022robust,
title = {Robust offset-free nonlinear model predictive control for systems learned by neural nonlinear autoregressive exogenous models},
author = {Jing Xie and Fabio Bonassi and Marcello Farina and Riccardo Scattolini},
url = {https://doi.org/10.1002/rnc.6883
http://arxiv.org/abs/2210.06801},
doi = {10.1002/rnc.6883},
year = {2023},
date = {2023-07-13},
urldate = {2023-07-13},
journal = {International Journal of Robust and Nonlinear Control},
publisher = {arXiv},
abstract = {This paper presents a robust Model Predictive Control (MPC) scheme that provides offset-free setpoint tracking for systems described by Neural Nonlinear AutoRegressive eXogenous (NNARX) models. The NNARX model learns the dynamics of the plant from input-output data, and during the training the Incremental Input-to-State Stability (𝛿ISS) property is forced to guarantee stability. The trained NNARX model is then augmented with an explicit integral action on the output tracking error, which allows the control scheme to enjoy offset-free tracking ability. A tube-based MPC is finally designed, leveraging the unique structure of the model, to ensure robust stability and robust asymptotic zero error regulation for constant reference signals in the presence of model-plant mismatch or unknown disturbances. Numerical simulations on a water heating system show the effectiveness of the proposed control algorithm.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This paper presents a robust Model Predictive Control (MPC) scheme that provides offset-free setpoint tracking for systems described by Neural Nonlinear AutoRegressive eXogenous (NNARX) models. The NNARX model learns the dynamics of the plant from input-output data, and during the training the Incremental Input-to-State Stability (𝛿ISS) property is forced to guarantee stability. The trained NNARX model is then augmented with an explicit integral action on the output tracking error, which allows the control scheme to enjoy offset-free tracking ability. A tube-based MPC is finally designed, leveraging the unique structure of the model, to ensure robust stability and robust asymptotic zero error regulation for constant reference signals in the presence of model-plant mismatch or unknown disturbances. Numerical simulations on a water heating system show the effectiveness of the proposed control algorithm.
15.
Acerbo, Flavia Sofia; Swevers, Jan; Tuytelaars, Tinne; Son, Tong Duy
Evaluation of MPC-based Imitation Learning for Human-like Autonomous Driving Proceedings Article Forthcoming
In: 22nd IFAC World Congress 2023, arXiv, Yokohama, Japan, Forthcoming.
@inproceedings{acerboEvaluationMPCbasedImitation2022,
title = {Evaluation of MPC-based Imitation Learning for Human-like Autonomous Driving},
author = {Flavia Sofia Acerbo and Jan Swevers and Tinne Tuytelaars and Tong Duy Son},
doi = {10.48550/arXiv.2211.12111},
year = {2023},
date = {2023-07-12},
urldate = {2023-07-12},
booktitle = {22nd IFAC World Congress 2023},
number = {arXiv:2211.12111},
publisher = {arXiv},
address = {Yokohama, Japan},
abstract = { This work evaluates and analyzes the combination of imitation learning (IL) and differentiable model predictive control (MPC) for the application of human-like autonomous driving. We combine MPC with a hierarchical learning-based policy, and measure its performance in open-loop and closed-loop with metrics related to safety, comfort and similarity to human driving characteristics. We also demonstrate the value of augmenting open-loop behavioral cloning with closed-loop training for a more robust learning, approximating the policy gradient through time with the state space model used by the MPC. We perform experimental evaluations on a lane keeping control system, learned from demonstrations collected on a fixed base driving simulator, and show that our imitative policies approach the human driving style preferences.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
This work evaluates and analyzes the combination of imitation learning (IL) and differentiable model predictive control (MPC) for the application of human-like autonomous driving. We combine MPC with a hierarchical learning-based policy, and measure its performance in open-loop and closed-loop with metrics related to safety, comfort and similarity to human driving characteristics. We also demonstrate the value of augmenting open-loop behavioral cloning with closed-loop training for a more robust learning, approximating the policy gradient through time with the state space model used by the MPC. We perform experimental evaluations on a lane keeping control system, learned from demonstrations collected on a fixed base driving simulator, and show that our imitative policies approach the human driving style preferences.
16.
Voogd, Kevin; Allamaa, Jean Pierre; Alonso-Mora, Javier; Son, Tong Duy
Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin Proceedings Article Forthcoming
In: 22nd IFAC World Congress 2023, arXiv, Yokohama, Japan, Forthcoming.
@inproceedings{Voogd2022ReinforcementLF,
title = {Reinforcement Learning from Simulation to Real World Autonomous Driving using Digital Twin},
author = {Kevin Voogd and Jean Pierre Allamaa and Javier Alonso-Mora and Tong Duy Son},
url = {https://arxiv.org/abs/2211.14874},
doi = {10.48550/ARXIV.2211.14874},
year = {2023},
date = {2023-07-10},
urldate = {2023-07-10},
booktitle = {22nd IFAC World Congress 2023},
publisher = {arXiv},
address = {Yokohama, Japan},
abstract = {Reinforcement learning (RL) is a promising solution for autonomous vehicles to deal with complex and uncertain traffic environments. The RL training process is however expensive, unsafe, and time consuming. Algorithms are often developed first in simulation and then transferred to the real world, leading to a common sim2real challenge that performance decreases when the domain changes. In this paper, we propose a transfer learning process to minimize the gap by exploiting digital twin technology, relying on a systematic and simultaneous combination of virtual and real world data coming from vehicle dynamics and traffic scenarios. The model and testing environment are evolved from model, hardware to vehicle in the loop and proving ground testing stages, similar to standard development cycle in automotive industry. In particular, we also integrate other transfer learning techniques such as domain randomization and adaptation in each stage. The simulation and real data are gradually incorporated to accelerate and make the transfer learning process more robust. The proposed RL methodology is applied to develop a path following steering controller for an autonomous electric vehicle. After learning and deploying the real-time RL control policy on the vehicle, we obtained satisfactory and safe control performance already from the first deployment, demonstrating the advantages of the proposed digital twin based learning process.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {inproceedings}
}
Reinforcement learning (RL) is a promising solution for autonomous vehicles to deal with complex and uncertain traffic environments. The RL training process is however expensive, unsafe, and time consuming. Algorithms are often developed first in simulation and then transferred to the real world, leading to a common sim2real challenge that performance decreases when the domain changes. In this paper, we propose a transfer learning process to minimize the gap by exploiting digital twin technology, relying on a systematic and simultaneous combination of virtual and real world data coming from vehicle dynamics and traffic scenarios. The model and testing environment are evolved from model, hardware to vehicle in the loop and proving ground testing stages, similar to standard development cycle in automotive industry. In particular, we also integrate other transfer learning techniques such as domain randomization and adaptation in each stage. The simulation and real data are gradually incorporated to accelerate and make the transfer learning process more robust. The proposed RL methodology is applied to develop a path following steering controller for an autonomous electric vehicle. After learning and deploying the real-time RL control policy on the vehicle, we obtained satisfactory and safe control performance already from the first deployment, demonstrating the advantages of the proposed digital twin based learning process.
17.
Løwenstein, Kristoffer Fink; Fagiano, Lorenzo; Bernardini, Daniele; Bemporad, Alberto
Physics-Informed Online Learning of Gray-box Models by Moving Horizon Estimation Journal Article
In: European Journal of Control, pp. 100861, 2023, ISSN: 0947-3580.
@article{Lowenstein2022PhysicsInformed,
title = {Physics-Informed Online Learning of Gray-box Models by Moving Horizon Estimation},
author = {Kristoffer Fink Løwenstein and Lorenzo Fagiano and Daniele Bernardini and Alberto Bemporad},
url = {https://www.sciencedirect.com/science/article/pii/S0947358023000900},
doi = {10.1016/j.ejcon.2023.100861},
issn = {0947-3580},
year = {2023},
date = {2023-07-03},
urldate = {2023-07-03},
journal = {European Journal of Control},
pages = {100861},
abstract = {A simple yet expressive prediction model is an essential ingredient in model-based control and estimation. Models derived from fundamental physical principles may fail to capture the complexity of the actual system dynamics. A potential solution is the use of a physics-informed, or gray-box model that extends a physics-based model with a data-driven part. Learning the latter might be challenging, due to noisy measurements and lack of full state information. This work presents a method based on Moving Horizon Estimation (MHE) for simultaneous state estimation and training of a black-box submodel, such as a neural network. The method can be used in offline training or applied online for adaptation without any prior knowledge than the white-box submodel. We analyze the capabilities of the method in a two degree of freedom robotic manipulator case study, also showing how it can be used for online adaptation to cope with a time-varying model mismatch.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
A simple yet expressive prediction model is an essential ingredient in model-based control and estimation. Models derived from fundamental physical principles may fail to capture the complexity of the actual system dynamics. A potential solution is the use of a physics-informed, or gray-box model that extends a physics-based model with a data-driven part. Learning the latter might be challenging, due to noisy measurements and lack of full state information. This work presents a method based on Moving Horizon Estimation (MHE) for simultaneous state estimation and training of a black-box submodel, such as a neural network. The method can be used in offline training or applied online for adaptation without any prior knowledge than the white-box submodel. We analyze the capabilities of the method in a two degree of freedom robotic manipulator case study, also showing how it can be used for online adaptation to cope with a time-varying model mismatch.
18.
Schuurmans, Mathijs; Katriniok, Alexander; Meissen, Christopher; Tseng, H. Eric; Patrinos, Panagiotis
Safe, Learning-Based MPC for Highway Driving under Lane-Change Uncertainty: A Distributionally Robust Approach Journal Article
In: Artificial Intelligence, vol. 320, pp. 103920, 2023, ISSN: 0004-3702.
@article{schuurmansSafeLearningBasedMPC2022,
title = {Safe, Learning-Based MPC for Highway Driving under Lane-Change Uncertainty: A Distributionally Robust Approach},
author = {Mathijs Schuurmans and Alexander Katriniok and Christopher Meissen and H. Eric Tseng and Panagiotis Patrinos},
url = {https://www.sciencedirect.com/science/article/pii/S0004370223000668},
doi = {https://doi.org/10.1016/j.artint.2023.103920},
issn = {0004-3702},
year = {2023},
date = {2023-07-01},
urldate = {2022-01-01},
journal = {Artificial Intelligence},
volume = {320},
pages = {103920},
publisher = {arXiv},
abstract = {We present a case study applying learning-based distributionally robust model predictive control to highway motion planning under stochastic uncertainty of the lane change behavior of surrounding road users. The dynamics of road users are modeled using Markov jump systems, in which the switching variable describes the desired lane of the vehicle under consideration and the continuous state describes the pose and velocity of the vehicles. We assume the switching probabilities of the underlying Markov chain to be unknown. As the vehicle is observed and thus, samples from the Markov chain are drawn, the transition probabilities are estimated along with an ambiguity set which accounts for misestimations of these probabilities. Correspondingly, a distributionally robust optimal control problem is formulated over a scenario tree, and solved in receding horizon. As a result, a motion planning procedure is obtained which through observation of the target vehicle gradually becomes less conservative while avoiding overconfidence in estimates obtained from small sample sizes. We present an extensive numerical case study, comparing the effects of several different design aspects on the controller performance and safety.
},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present a case study applying learning-based distributionally robust model predictive control to highway motion planning under stochastic uncertainty of the lane change behavior of surrounding road users. The dynamics of road users are modeled using Markov jump systems, in which the switching variable describes the desired lane of the vehicle under consideration and the continuous state describes the pose and velocity of the vehicles. We assume the switching probabilities of the underlying Markov chain to be unknown. As the vehicle is observed and thus, samples from the Markov chain are drawn, the transition probabilities are estimated along with an ambiguity set which accounts for misestimations of these probabilities. Correspondingly, a distributionally robust optimal control problem is formulated over a scenario tree, and solved in receding horizon. As a result, a motion planning procedure is obtained which through observation of the target vehicle gradually becomes less conservative while avoiding overconfidence in estimates obtained from small sample sizes. We present an extensive numerical case study, comparing the effects of several different design aspects on the controller performance and safety.
19.
Yan, Schengchao; Zhang, Yuan; Zhang, Baohe; Boedecker, Joschka; Burgard, Wolfram
Geometric Regularity with Robot Intrinsic Symmetry in Reinforcement Learning Proceedings Article
In: RSS 2023 Workshop on Symmetries in Robot Learning, 2023.
@inproceedings{yan2023geometric,
title = {Geometric Regularity with Robot Intrinsic Symmetry in Reinforcement Learning},
author = {Schengchao Yan and Yuan Zhang and Baohe Zhang and Joschka Boedecker and Wolfram Burgard},
url = {https://doi.org/10.48550/arXiv.2306.16316},
year = {2023},
date = {2023-06-28},
urldate = {2023-06-28},
booktitle = {RSS 2023 Workshop on Symmetries in Robot Learning},
abstract = {Geometric regularity, which leverages data symmetry, has been successfully incorporated into deep learning architectures such as CNNs, RNNs, GNNs, and Transformers. While this concept has been widely applied in robotics to address the curse of dimensionality when learning from high-dimensional data, the inherent reflectional and rotational symmetry of robot structures has not been adequately explored. Drawing inspiration from cooperative multi-agent reinforcement learning, we introduce novel network structures for deep learning algorithms that explicitly capture this geometric regularity. Moreover, we investigate the relationship between the geometric prior and the concept of Parameter Sharing in multi-agent reinforcement learning. Through experiments conducted on various challenging continuous control tasks, we demonstrate the significant potential of the proposed geometric regularity in enhancing robot learning capabilities.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Geometric regularity, which leverages data symmetry, has been successfully incorporated into deep learning architectures such as CNNs, RNNs, GNNs, and Transformers. While this concept has been widely applied in robotics to address the curse of dimensionality when learning from high-dimensional data, the inherent reflectional and rotational symmetry of robot structures has not been adequately explored. Drawing inspiration from cooperative multi-agent reinforcement learning, we introduce novel network structures for deep learning algorithms that explicitly capture this geometric regularity. Moreover, we investigate the relationship between the geometric prior and the concept of Parameter Sharing in multi-agent reinforcement learning. Through experiments conducted on various challenging continuous control tasks, we demonstrate the significant potential of the proposed geometric regularity in enhancing robot learning capabilities.
20.
Baumgärtner, Katrin; Zanelli, Andrea; Diehl, Moritz
Stability Analysis of Nonlinear Model Predictive Control with Progressive Tightening of Stage Costs and Constraints Journal Article
In: IEEE Control Systems Letters, vol. 7, pp. 3018-3023, 2023, ISSN: 2475-1456.
@article{Baumgaertner2023b,
title = {Stability Analysis of Nonlinear Model Predictive Control with Progressive Tightening of Stage Costs and Constraints},
author = {Katrin Baumgärtner and Andrea Zanelli and Moritz Diehl},
doi = {https://doi.org/10.1109/LCSYS.2023.3289707},
issn = {2475-1456},
year = {2023},
date = {2023-06-26},
journal = {IEEE Control Systems Letters},
volume = {7},
pages = {3018-3023},
abstract = {We consider a stage-varying nonlinear model predictive control (NMPC) formulation and provide a stability result for the corresponding closed-loop system under the assumption that cost and constraints are progressively tightening. We illustrate the generality of the stage-varying formulation pointing out various approaches proposed in the literature that can be cast as stage-varying and progressively tightening optimal control problems.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We consider a stage-varying nonlinear model predictive control (NMPC) formulation and provide a stability result for the corresponding closed-loop system under the assumption that cost and constraints are progressively tightening. We illustrate the generality of the stage-varying formulation pointing out various approaches proposed in the literature that can be cast as stage-varying and progressively tightening optimal control problems.