economic-mpc.tex

\chapter{Economic MPC}
\label{chap:economic-mpc}

Many modern control problems present complex performance metrics, such as economic aspects, regulatory specifications and use of resources. Consider an iron furnace, where the inputs are its temperature, the air flow, the inflow of iron ore, limestone and coke, and the products includes the iron yield, its quality, the CO$_2$ emission and the amount of slag. The stage cost $\ell(x,u)$ represents economic losses, energy use, cost of materials, yield\ldots. $\ell(x,u)$ is a continuous lower-bounded (otherwise the problem is not well defined) function.

One approach is to split the problem into the offline computation of the steady-state, \textit{e.g.} the optimal furnace temperature, from the economic cost to go
\begin{align*}
  \min_{x_s,u_s} &\ \ell(x_s,u_s) \\
  \text{subject to} &\ x_s = f(x_s,u_s) \\
                 & x\in \mathcal{X},\ u\in \mathcal{U}
\end{align*}
and an online MPC to track the steady state $(x_s,u_s)$ using a quadratic cost function $g_k(x_k-x_s,u_k-u_s)$ that has the minimum at $(x_s,u_s)$
\begin{align*}
  \min_{\bsu} & \sum_{k=0}^{K-1} g(x_k-x_s,u_k-u_s) \\
  \text{subject to} &\ x_{k+1} = f(x_k,u_k) \\
              & x_0 = x,\ x_K=x_s \\
              & x_k\in \mathcal{X}_k\\
              & u_k\in \mathcal{U}_k
\end{align*}
This two-stage approach is simpler to analyze but suffers from suboptimal cost of the system trajectory.

Economic MPC uses instead the loss function $\ell$ directly in the dynamic optimization problem
\begin{align*}
  \min_{\bsu} & \sum_{k=0}^{K-1} \ell(x_k,u_k) \\
  \text{subject to} &\ x_{k+1} = f(x_k,u_k) \\
              & x_0 = x,\ x_K=x_s \\
              & x_k\in \mathcal{X}_k \\
              & u_k\in \mathcal{U}_k
\end{align*}
eliminating the cascaded optimization by replacing it with a single time-scale problem for which the optimization can compute an efficient trajectory, at the cost of a more computationally difficult problem to solve in real-time, since often times the problem is not convex.

Economic MPC is computationally more difficult compared to standard MPC where $g$ is zero at the equilibrium point because there are operating points $(x,u)$ which are not steady states but are economically more convenient:
\begin{equation*}
  \ell(x,u) < \ell(x_s,u_s).
\end{equation*}
To illustrate the situation, consider the furnace example above, starting from an operation point where the furnace is hot and producing molten iron. The controller will stop the input of coke, thereby reducing the costs but the furnace will keep producing iron while its temperature remains above the melting point of ore, if the MPC horizon $K$ is too short. In other words, we want the furnace to be economical while at the same time being in a operating state for the future.

The economic MPC is naturally an infinite horizon problem because we request long-term profits to be maximised. Moreover an optimal trajectory may not drive the system to a single point equilibrium but rather follow a semi-periodic trajectory (see for instance \cite[page~158]{MPC-diehl}). The boundedness is guaranteed by the constraint over the infinite trajectory because by construction, the controller produces an acceptable future trajectory. Such infinite horizon problem problems are computationally very hard.

Solving the problem with a finite horizon is more tractable but because the system is not necessarily driven towards the equilibrium, we cannot assume that the tail, after the horizon $K$, is small as we did with tracking MPC. Instead low cost unstable trajectories may emerge (cheap trajectory now, expensive to fix in the future). This is the reason why eMPC are solved by constraining the terminal state $x_K=x_s$.

Imposing $x_K=x_s$ induces a final stage cost $g_K(x_s)$ but ensures that the system is still operational at the end of the horizon and not in an unrecoverable state (unfeasible? high cost?). The increased terminal cost $g_K(x_K)$ affects minimally the closed-loop solution (but if it affects minimally, why adding it?).

\subsection{Stability and Performance of Economic MPC}
\label{sec:stability-performance-economic-MPC}

In tracking MPC, the terminal constraint $x_K=0$ was essential in proving that eq.~\eqref{eq:lyapunov-tracking-MPC} was a Lyapunov function for the plant. The key idea was that a valid new trajectory would keep the state close to equilibrium at the tail. However for economic MPC, the same manipulation does not work, since
\begin{equation}
  \label{eq:eMPC-Lyapunov-like-inequality}
  W(x_{i+1}) \le W(x_i) + \ell(x_s,u_s) - \ell(x_i,u_0^\star(x_i))
\end{equation}
is not in general smaller than $W(x_i)$: to keep the system at steady-state, the cost may exceed the cost of the state after applying control input $u_0^\star(x)$. This is because in contrast with tracking MPC where the stage cost $g_k$ had the property of being zero at the equilibrium point, in economic MPC, $\ell$ is not minimized at the steady state; $W(x)$ as defined in eq.~\eqref{eq:lyapunov-tracking-MPC} is no longer a Lyapunov function and stronger conditions are required to guarantee asymptotic stability. This is outside the scope of this introduction.

Since eMPC can generate semi-periodic trajectories, are these trajectories better than tracking a point? The following theorem guarantees that trajectories created started from an arbitrary initial point are more efficient that tracking a steady-state provided the number of time steps $N$ is sufficiently large:
\begin{theorem}
  Let $\{x_k,\ k=0,\ldots,N-1\}$ be a closed-loop trajectory generated by the economic MPC controller with terminal constraint $x_s$. Then, for any initial condition $x_0$
  \begin{equation}
    \label{eq:eMPC-inequality-trajectory-steadystate}
    \limsup_{N\rightarrow \infty} \frac{1}{N} \sum_{k=0}^{N-1} \ell(x_k,u_k) \le \ell(x_s,u_s).
  \end{equation}
\end{theorem}
\begin{proof}
  The proof relies on the identity eq.~\eqref{eq:eMPC-Lyapunov-like-inequality}, rewritten as
  \begin{equation*}
    W(x_{k+1}) - W(x_k) \le \ell(x_s,u_s) - \ell(x_k,u_0^\star(x_k))
  \end{equation*}
  Summing the first $N$ terms of the series and dividing by $N$ gives
  \begin{equation*}
    \frac{1}{N}\sum_{k=0}^{N-1} \left[W(x_{k+1}) - W(x_k)\right] = \underbrace{\frac{W(x_N) - W(x_0)}{N}}_{\rightarrow 0 \text{ when } N\rightarrow 0} \le \frac{1}{N}\sum_{k=0}^{N-1}  \left[\ell(x_s,u_s) - \ell(x_k,u_0^\star(x_k))\right]
  \end{equation*}
  since $W(x_N)$ is lower bounded\footnote{The assumption is that $\ell$ is lower bounded and that $W(x)$ is the sum of $K$ terms $\ell(x,u)$: one concludes that also $W$ is bounded.}. After rearranging the terms, we obtain the expression eq.~\eqref{eq:eMPC-inequality-trajectory-steadystate}~\cite[Sect.~2.8.1]{MPC-diehl}.
\end{proof}

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "notes"
%%% End: