Description

Description:

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.

Share

Transcript

IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 18, NO. 2, MARCH 2010 267
Fast Model Predictive Control UsingOnline Optimization
Yang Wang and Stephen Boyd
, Fellow, IEEE
Abstract—
A widely recognized shortcoming of model predictivecontrol (MPC) is that it can usually only be used in applicationswith slow dynamics, where the sample time is measured in secondsor minutes. A well-known technique for implementing fast MPCis to compute the entire control law ofﬂine, in which case the on-line controller can be implemented as a lookup table. This methodworks well for systems with small state and input dimensions (say,nomorethanﬁve),fewconstraints,andshorttimehorizons.Inthispaper, we describe a collection of methods for improving the speedof MPC, using online optimization. These custom methods, whichexploit the particular structure of the MPC problem, can computethe control action on the order of 100 times faster than a methodthat uses a generic optimizer. As an example, our method com-putes the control actions for a problem with 12 states, 3 controls,andhorizonof30timesteps(whichentailssolvingaquadraticpro-gram with 450 variables and 1284 constraints) in around 5 ms, al-lowing MPC to be carried out at 200 Hz.
IndexTerms—
Modelpredictivecontrol(MPC),real-timeconvexoptimization.
I. I
NTRODUCTION
I
N CLASSICAL model predictive control (MPC), the con-trol action at each time step is obtained by solving an on-lineoptimizationproblem.Witha linearmodel, polyhedralcon-straints,andaquadraticcost,theresultingoptimizationproblemis a quadratic program (QP). Solving the QP using general pur-pose methods can be slow, and this has traditionally limitedMPC to applications with slow dynamics, with sample timesmeasured in seconds or minutes. One method for implementingfast MPC is to compute the solution of the QP explicitly as afunction of the initial state [1], [2]; the control action is thenimplemented online in the form of a lookup table. The majordrawbackhereisthatthenumberofentriesinthetablecangrowexponentially with the horizon, state, and input dimensions, sothat “explicit MPC” can only be applied reliably to small prob-lems (where the state dimension is no more than around ﬁve).In this paper, we describe a collection of methods that can beused togreatly speedupthecomputation of thecontrolactioninMPC,usingonlineoptimization.Someoftheideashavealreadybeen noted inliterature,and here we willdemonstrate that when
ManuscriptreceivedSeptember13,2008;revisedNovember11,2008.Manu-script received in ﬁnal form March 10, 2009. Firstpublished June 30, 2009;cur-rent version published February 24, 2010. Recommended by Associate EditorJ. Lee. The work of Y. Wang was supported by a Rambus Corporation StanfordGraduate Fellowship. This work was supported in part by the Precourt InstituteonEnergyEfﬁciency,byNSFAward0529426,byNASAAwardNNX07AEIIA,and by AFOSR Award FA9550-06-1-0514.Y. Wang and S. Boyd are with the Department of Electrical Engineering,Stanford University, Stanford, CA 94305 USA (e-mail: yw224@stanford.edu;boyd@stanford.edu).Digital Object Identiﬁer 10.1109/TCST.2009.2017934
used in combination, they allow MPC to be implemented ordersof magnitude faster than with generic optimizers.Our main strategy is to exploit the structure of the QPs thatarise in MPC [3], [4]. It has already been noted that with an ap-propriate variable reordering, the interior-point search directionat each step can be found by solving a block tridiagonal systemof linear equations. Exploiting this special structure, a problemwithstatedimension ,inputdimension ,andhorizon takesoperations per step in an interior-point method,as opposed to if the special structure were notexploited. Since interior-point methods require only a constant(and modest) number of steps, it follows that the complexityof MPC is therefore
linear
rather than cubic in the problemhorizon.We should mention here, that a popular method for reducingthe complexity of the MPC QP is by reformulating the QP en-tirely in terms of the control inputs. In this case, the QP be-comes dense (the structure of the problem is lost), and requiresoperations per step in an interior point method. Thistechnique is often combined with a strategy known as
moveblocking
, where the input is assumed to be constant for ﬁxedportions of the horizon, and so the number of optimization vari-ables is further reduced (see, e.g., [5] and [6]). This can work well when the horizon is small, but we will see that even formodest values of , this method will be slower compared witha method that fully exploits the problem structure.Another important technique that can be used in online MPCis
warm-starting
[7], [8], in which the calculations for each stepare initialized using the predictions made in the previous step.Warm-start techniques are usually not used in general interior-point methods (in part because these methods are already soefﬁcient)buttheycanworkverywellwithanappropriatechoiceofinterior-pointmethod,cuttingthenumberofstepsrequiredbya factor of ﬁve or more.The ﬁnal method we introduce is (very) early terminationof an appropriate interior-point method. It is not surprisingthat high quality control is obtained even when the associatedQPs are not solved to full accuracy; after all, the optimizationproblem solved at each MPC step is really a planning exercise,meant to ensure that the current action does not neglect thefuture. We have found, however, that after only surprisinglyfew iterations (typically between 3 and 5), the quality of controlobtained is very high, even when the QP solution obtained ispoor.We will illustrate our methods on several examples: a me-chanical control system, a supply chain problem, and a ran-domly generated example. The mechanical control system ex-amplehas 12states, 3controls,andahorizon ;
1063-6536/$26.00 © 2009 IEEE
268 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 18, NO. 2, MARCH 2010
each MPC step requires the solution of a QP with 450 vari-ables, 924 inequality constraints, and 360 equality constraints.A simple, non-optimized C implementation of our method al-lows each MPC step to be carried out in around 5 ms, on a 3GHz PC, which would allow MPC to be carried out at around200 Hz. For a larger example, with 30 states, 8 con-trols, and horizon , each step requires solution of aQP with over a thousand variables, and several thousand con-straints; our method can be compute each MPC step in around25 ms, allowing an update rate of 40 Hz. (Details of the perfor-mance obtained will be given later.)MPC can be used for several types of control (and also,estimation) problems, including tracking problems, regulatorproblems, and stochastic control problems; moreover, thetime-varying and ﬁnite-horizon versions of each of these canbe handled. In this paper we will focus on the inﬁnite-horizontime-invariant stochastic control problem, mentioning at theend of the paper how the methods extend or apply to othertypes of problems.
A. Related Work
Model predictive control goes by several other names, suchas rolling-horizon planning, receding-horizon control, dynamicmatrixcontrol,anddynamiclinearprogramming.Ithasbeenap-pliedinawiderangeofapplications,includingchemicalprocessand industrial control [5], [9]–[12], control of queuing systems[13], supply chain management [14], [15], stochastic controlproblemsin economicsand ﬁnance[16],[17], dynamichedging[18], and revenue management [19], [20].FordiscussionofMPCfromthepointofviewofoptimalcon-trol, see [21]–[23]; for a survey of stability and optimality re-sults, see [24]. For closed-form (ofﬂine) methods, see [1], [2],and [25]. For dynamic trajectory optimization using optimiza-tion methods, see [26].PreviousworkthataddressesefﬁcientsolutionoftheQPsthatarise in MPC includes [27]–[29],and [63]; for efﬁcient methodsfor large-scale MPC problems arising in chemical process con-trol, see [30]–[32]. Efﬁcient methods for robust MPC are ad-dressed by [33]–[35]. The idea of using Newton’s method, ap-plied to a smooth non-quadratic cost function can be found in[36], [37].II. T
IME
-I
NVARIANT
S
TOCHASTIC
C
ONTROL
A. System Dynamics and Control
Inthissection,wedescribethebasictime-invariantstochasticcontrol problem with linear dynamics. The state dynamics aregiven by(1)where denotes time, is the state, is theinputorcontrol,and isthedisturbance.Thematricesand are (known) data. We assumethat , for different values of , are independent identicallydistributed (IID) with known distribution. We letdenote the mean of (which is independent of ).Thecontrolpolicymustdeterminethecurrentinput fromthe current and previous states , i.e., we will havea causal state feedback policy. We letdenote the control policy, so thatThis is equivalent to the statement that the random variableis measurable on the random variable . Animportant special case is a time-invariant static state feedback control, for which , where iscalled the control function.
B. Objective and Constraints
We deﬁne the following objective:(2)where isaconvexquadraticstagecostfunction,with the formHere,, and are parameters, and we will assumewhere denotes matrix inequality.We also have state and control constraints, deﬁned as a set of linear inequalities(3)where , and are problem data,and denotes vector (componentwise) inequality.In many problems the objective and constraints are separablein the state and controls. This means that , and that thestate and control constraints can be written separately, aswhere, here, .A further specialization is where, in addition, and are diag-onal, and the state and control constraints consist of lower andupper boundswhere and . We refer tothese as box constraints.
C. Stochastic Control Problem
The stochastic control problem is to ﬁnd a control policy thatsatisﬁes the state and control constraints (3), and minimizesthe objective (2). Several pathologies can occur. The sto-chastic control problem can be infeasible: there exists no causalstate feedback policy for which the contraints hold for all
WANG AND BOYD: FAST MODEL PREDICTIVE CONTROL USING ONLINE OPTIMIZATION 269
(with probability one), and is ﬁnite. Our stage cost can be un-bounded below (since it has linear terms and the quadratic partneed not be positive deﬁnite), so the stochastic control problemcanalsobeunboundedbelow,i.e.,thereexistsapolicyforwhich. Our interest in this paper is to describe a methodfor efﬁciently computing an MPC control policy (which is itself only a heuristic suboptimal policy) and not the technical detailsof stochastic control, so we will not consider these issues fur-ther, and simply assume that the problem is feasible, with ﬁniteoptimal cost. See, e.g., [43] for more on the technical aspects of linear stochastic control.It can be shown that there is an optimal policy which hasthe form of a time-invariant static state feedback control, i.e.,. Unfortunately, the state feedack functioncan be effectively computed in only a few special cases,such as when there are no constraints and is Gaussian.
D. Steady-State Certainty Equivalent Problem
For future reference we describe the steady-state certainty-equivalent problem, in which we assume that and have theconstant values and , and the process noise has constantvalue equal to its mean . The dynamics equation becomesthe constraint becomes , and the objective be-comes .The steady-state certainty-equivalent problem is thenminimizesubject towith variables and . This problem is a convexQP and is readily solved. We will let denote an optimal valueof in the certainty equivalent problem.In many cases the steady-state certainty-equivalent problemhas a simple analytic solution. For example, when , theobjective is purely quadratic (i.e., and are both zero), and(which means satisfy the constraints), wehave .
E. Model Predictive Control
MPC is a heuristic for ﬁnding a good, if not optimal, con-trol policy for the stochastic control problem. In MPC the con-trol is found at each step by ﬁrst solving the optimizationproblemminimizesubject to(4)withvariables and .Here, is the (planning) horizon, the function isthe terminal cost function, which we assume is quadraticwith , and is the terminal state con-straint. There are many methods for choosing the MPC param-eters , and ; see, e.g., [44]–[48].The problem (4) is a convex QP with problem dataLet beoptimalfor the QP (4). The MPC policy takes . The MPCinput is evidently a (complicated) function of the currentstate , so the MPC policy has a static state-feedback form. It can be shown that is a piecewise-afﬁne function, when the quadratic cost term is positive deﬁnite(see, e.g., [1]).We can give a simple interpretation of the MPC QP (4). Inthis QP, we truncate the true cost function to a horizon stepsin the future, and we replace the current and future disturbances(which are not yet known) with their mean values. We representthe truncated portion of the cost function with an approximatevalue function . We then compute an optimal trajectory forthissimpliﬁedproblem.Wecanthinkof as a plan for what we would do, if the disturbance over the nextsteps were to take on its mean value. We use only the ﬁrstcontrol action in this plan, , as our actual control. At time, the actual value of becomes available to us, sowe carry out the planning process again, over the time period. This is repeated for each time step.
F. Explicit MPC
In explicit model predictive control, an explicit form of thepiecewise-afﬁnecontrollaw iscomputedofﬂine.Wecom-pute, ofﬂine, the polyhedral regions over which the control isafﬁne, as well as the offset and control gain for each region[1], [2], [49]. The online control algorithm is then reduced toa lookup table: the region associated with the current stateis ﬁrst determined, and then the control law associated with thatregion is applied.This method is very attractive for problems with a smallnumber of states and controls, simple constraints, and a modesthorizon, for which the number of regions is manageable. Forother problems (say, with , and ) thenumber of regions can be very large, so explicit MPC is nolonger practically feasible. (There is some hope that a goodcontrol law can be obtained by simplifying the piecewise-afﬁnefunction , e.g., using the methods described in [50]–[52],replacing it with a piecewise-afﬁne function with a manageablenumber of regions.)Even when the number of regions is manageable (say, a fewthousand), it can still be faster to solve the QP (4), using themethods described in this paper, rather than implementing thelookup table required in explicit MPC. Furthermore, explicitMPC cannot handle cases where the system, cost function,or constraints are time-varying. On the other hand, the QP(4) can be easily modiﬁed for the case where the matricesand vary over time (we onlyneed to keep track of the time index for these matrices).
270 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 18, NO. 2, MARCH 2010
III. P
RIMAL
B
ARRIER
I
NTERIOR
-P
OINT
M
ETHOD
In this section, we describe a basic primal barrier interior-point for solving the QP (4), that exploits its special structure.Much of this material has been reported elsewhere, possibly ina different form or context (but seems to not be widely enoughappreciated among those who use or study MPC); we collect ithere in one place, using a uniﬁed notation. In Section IV, wedescribe variations on the method described here, which giveeven faster methods.WeﬁrstrewritetheQP(4)inamorecompactform.Wedeﬁnean overall optimization variableand express the QP (4) asminimizesubject to (5)where......... ... .................. ... ..................... ... ..................Evidently these matrices have considerable structure; for ex-ample, isblocktridiagonal.Inspecialcaseswecanhaveevenmore structure. When the problem is state control separable, forexample, is block diagonal.
A. Primal Barrier Method
We will use an infeasible start primal barrier method to solvethe QP [3, Ch. 11], [53]. We replace the inequality constraintsin the QP (5) with a barrier term in the objective, to get theapproximate problemminimizesubject to (6)where is a barrier parameter, and is the log barrierassociated with the inequality constraints, deﬁned aswhere are the rows of . (We takeif .) The problem (6) is a convex optimization problemwith smooth objective and linear equality constraints, and canbe solved by Newton’s method, for example.As approaches zero, the solution of (6) converges to a so-lution of the QP (5): it can be shown that the solution of (6) isno more than suboptimal for the QP (5) (see, e.g.,[3, Sect. 11.2.2]). In a basic primal barrier method, we solve asequence of problems of the form (6), using Newton’s methodstarting from the previously computed point, for a decreasingsequence of values of . A typical method is to reduce bya factor of 10 each time a solution of (6) is computed (withinsome accuracy). Such a method can obtain an accurate solu-tion of the srcinal QP with a total effort of around 50 or soNewton steps. (Using the theory of self-concordance [3, Sect.11.5], [54], one can obtain a rigorous upper bound on the totalnumber of Newton steps required to solve the QP to a givenaccuracy. But this bound is far larger than the number of stepsalways observed in practice.)
B. Infeasible Start Newton Method
We now focus on solving the problem (6) using an infeasiblestart Newton method [3, Sect. 10.3.2]. We associate a dual vari-able with the equality constraint . The opti-mality conditions for (6) are then(7)where , and denotes the th row of . Theterm is the gradient of . We also have the implicitconstraint here that . We call the primal residual, andthedualresidual.Thestackedvector iscalledtheresidual; the optimality conditions for (6) can then be expressedas .

We Need Your Support

Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...Sign Now!

We are very appreciated for your Prompt Action!

x