Description

A Limited Memory Quasi-Newton Preconditioner for Large Scale Optimization

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.

Related Documents

Share

Transcript

III EuropeanConference on Computational MechanicsSolids, Structures and Coupled Problems in EngineeringC.A. Mota Soares et.al. (eds.)Lisbon, Portugal, 5–8 June 2006
A LIMITED MEMORY QUASI-NEWTON PRECONDITIONER FORLARGE SCALE OPTIMIZATION
Veranise Dubeux
1
, Jos´e Herskovits
1
, and SandroRodrigues Mazorche
21
COPPE - Federal University of Rio de Janeiro - MechanicalEngineering ProgramCaixa Postal 68503, 21945 970, Rio de Janeiro - Brazil.e-mail: veranise@optimize.ufrj.br , jose@optimize.ufrj.br
2
Departamentode Matem´atica, ICE - UniversidadeFederal de Juiz de Fora.Cidade Universitria, Cep 36036-330- Juiz de Fora - MG - Brazil.e-mail: mazorche@ice.ufjf.br
Keywords:
Preconditionerforthe conjugategradientemethod,Limited Memoryquasi-Newtontechniqueand interior point algorithm.
Abstract.
We study the appication of the conjugated gradient method preconditioned by alimited memory quasi-Newton matrix in the resolution of the FAIPA’s internal linear systems.FAIPA, the Feasible Arc Interior Point Algorithm, is an interior-point algorithm that solvesnonlinear optimization problems. It makes iterations in the primal and dual variables of theoptimization problem to solve the Karush-Kuhn-Tucker optimality conditions. Given an initialinterior point, it deﬁnes a sequence of interior points with the objetive function monotonicallyreduced.FAIPA requires the solution of three linear systems whit the same coefﬁcient matrix at each it-eration. These systems, when solved in terms of the Lagrange’s multipliers, are in general full,symmetric and positive deﬁnite. The conjugated gradient is largely employed for the iterativesolution of symmetric and positive deﬁnite linear systems.When the system is badly conditioned, a preconditioner matrix con be employed. The precon-ditioner’s choice is fundamental for the technique’s efﬁciency.We present a preconditioner for the FAIPA’s linear systems based on the limited memoy BFGS technique and consider problems with full and sparse matrices. Numerical examples show the performance of our methodology.
1
Veranise Dubeux, Jos´e Herskovits and Sandro Rodrigues Mazorche
1 INTRODUCTION
The engineeringoptimization task consists in ﬁnding the designvariables
x
1
,x
2
,...,x
n
thatminimize
f
(
x
)
subject to
g
(
x
)
≤
0
and
h
(
x
) = 0
,
(1)where
x
≡
[
x
1
,x
2
,...,x
n
]
t
, the scalar function
f
(
x
)
is the objective function and
g
(
x
)
≡
[
g
1
(
x
)
,g
2
(
x
)
,...,g
m
(
x
)]
t
and
h
(
x
)
≡
[
h
1
(
x
)
,h
2
(
x
)
,...,h
p
(
x
)]
t
representinequalityandequal-ity constraints. We assume that
f
(
x
)
,
g
(
x
)
and
h
(
x
)
are continuousin
ℜ
n
as well as their ﬁrstderivatives. In engineering applications most of these functions are nonlinear. Then, (1) is asmooth nonlinear constrainedmathematical programmingproblem.Reallife engineeringsystemsinvolveaverylargenumberofdesignvariablesandconstraints.Evaluationof functionsandof derivativescomingfrom engineeringmodelsis very expensive interms of computer time. In practical applications, calculation and storage of second derivativesare impossibleto be carried out. Then, numericaltechniquesfor engineeringoptimization mustbe capable to solve very large problems with a reasonable number of function evaluations andwithout needing second derivatives. Robustness is also a crucial point for industrial applica-tions.Quasi-Newton method creates an approximation matrix of second derivatives. [13, 25, 43,48, 50] With this method large problems can be solved in a reasonable number of iterations.Employing rank two updatingrules, like BFGS or DFP, it is possible to obtain positive deﬁniteapproximation matrices. This is a requirement of optimization algorithms that include a linesearch procedure [24, 25, 26, 43, 48, 50] to ensure global convergence. However, the classicquasi-Newton method cannot be applied for large problems since it requires the calculus andstorage of approximationmatrices, which are always full.Limited memory quasi-Newton method avoids the storage of the approximation matrix.[11, 42, 47, 48] Positive deﬁnite matrices can also be obtained with this technique. It was ﬁrstdeveloped for unconstrainedoptimization and then extended to problems with side constraints.Employingthe Feasible Arc Interior Point Algorithm (FAIPA), the limited memory methodcanalso be applied for constrainedoptimization problems. [19, 31, 32, 44]Anotherapproachto solve largeproblemswith a quasi-Newton techniqueconsists in obtain-ingsparseapproximationmatrices. This ideawasﬁrst exploitedbyToint in the 70th[56, 57,58]and by Fletcher et. al. in the 90th .[17, 18] In both cases sparse matrices were obtained in avery efﬁcient way. However, those methods cannot be appliedfor optimization algorithms witha line search, since it is not guaranteedthat the approximationmatrices are positive deﬁnite. Inbooth cases, the authors worked with a trust region algorithm, but the numerical results werepoor.The numerical techniques described in this chapter are based on the Feasible Arc InteriorPoint Algorithm (FAIPA)[34] for nonlinear constrained optimization. FAIPA, that is an exten-sion of the Feasible Directions Interior Point Algorithm, [22, 23, 24, 26, 49] integrates ideascoming from the modern Interior Point Algorithms for Linear Programming with Feasible Di-2
Veranise Dubeux, Jos´e Herskovits and Sandro Rodrigues Mazorche
rection Methods. At each point, FAIPA deﬁnes a ”Feasible Descent Arc”. Then, it ﬁnds a newinterior pointon thearc, with a lower objective. Newton,quasi - Newtonandﬁrst orderversionsof FAIPA can be obtained.FAIPA is supported by strong theoretical results. Global convergence to a local minimumof the problem is proved with relatively weak assumptions. The search along an arc ensuressuperlinear convergence for the quasi - Newton version, even when there are highly nonlinearconstraints, avoiding the so called ”Maratos’ effect” [45]. FAIPA, that is simple to code, doesnot require the solution of quadratic programs and it is not a penalty or a barrier method. Itmerely requiresthe solution of three linear systems with the samematrix periteration. This oneincludes the second derivative of the Lagrangian, or a quasi - Newton approximation. Severalpractical applicationsof the present and previous versions of FAIPA, as well as several numer-ical results, show that FAIPA constitutes a very strong and efﬁcient technique for engineeringdesign optimization, [1, 2, 4, 5, 6, 7, 8, 27, 28, 29, 30, 38, 39, 40, 41, 54] and also for structuralanalysis problems with variational inequalities. [3, 60, 62]The main difﬁculty to solve large problems with FAIPA comes from the size and sparsity of the internal linear systems of equations. Since the quasi-Newton matrix is included in the sys-tems, limited memory and sparse quasi - Newton techniques can produceimportant reductionsof computer calculusand memory requirements.In this chapterwe presenta new sparsequasi - Newtonmethodthat workswith diagonalpos-itive deﬁnitematrices andemploythis techniquefor constrainedoptimization with FAIPA. Thisapproachcan beemployedalso in the well known sequential quadraticprogrammingalgorithm(SQP)[25, 51, 52] or in the interior point methods for nonlinear programming, as primal-dualor pathfollowingalgorithms.[20, 48, 61] We also describe numericaltechniques,to solve largeproblemswith FAIPA, employingexact or iterative linear solvers andsparse or limited memoryquasi-Newton formulations.Quasi - Newton method is describedin the next section, including limited memory formula-tion and our proposal for sparse quasi - Newton matrices. FAIPA is described in Sec. 3 and thestructure of the internal systems and some numerical techniques to solve them are discussed inSec. 4. Numerical experiments with a set of test problemsare reported in Sec. 5, followedwithsome results in structural optimization. Finally, we present our conclusionsin the last section.
2 Quasi - NewtonMethod for NonlinearOptimization
We consider now the unconstrainedoptimization problemminimize
f
(
x
);
x
∈ℜ
n
(2)Modern iterative algorithms deﬁne, at each point, a descent direction of
f
(
x
)
and make aline search looking for a better solution. The quasi - Newton method works with a matrix thatapproximates the Hessian of the objective function, or of its inverse. The basic idea is to buildthe quasi - Newton matrix with information gathered while the iterations progress.Let the symmetric matrix
B
k
∈ ℜ
n
×
n
be the current approximation of
∇
2
f
(
x
k
)
. An im-3
Veranise Dubeux, Jos´e Herskovits and Sandro Rodrigues Mazorche
proved approximation
B
k
+1
is obtainedfrom
B
k
+1
=
B
k
+ ∆
B
k
.
(3)Since
∇
f
(
x
k
+1
)
−∇
f
(
x
k
)
≈
[
∇
2
f
(
x
k
)](
x
k
+1
−
x
k
)
,
the basic idea of quasi - Newton method consist in taking
∆
B
k
in such way that
∇
f
(
x
k
+1
)
−∇
f
(
x
k
) = [
B
k
+1
](
x
k
+1
−
x
k
)
,
(4)called
“secant condition”
, is true.Let
δ
=
x
k
+1
−
x
k
and
γ
=
∇
f
(
x
k
+1
)
−∇
f
(
x
k
)
.
Then, (4) is equivalent to
γ
=
B
k
+1
δ.
(5)The substitution of (3) into (5) gives us
n
conditions to be satisﬁed by
∆
B
k
. Since
∆
B
k
∈ℜ
n
×
n
, the secant condition is not enough to determine
B
k
+1
. Several updating rules for
B
k
+1
were proposed. [13, 43, 52] The most successful is the BFGS (Broyden, Fletcher, Shanno,Goldfarb) formula
B
k
+1
=
B
k
+
γγ
t
δ
t
γ
−
B
k
δδ
t
B
k
δ
t
B
k
δ.
(6)If
B
k
is positive deﬁnite, it can be proved that
δ
t
γ >
0
(7)is a sufﬁcient condition to have
B
k
+1
positive deﬁnite. Under certain assumptions about
f
(
x
)
,(7) is satisﬁed if an appropriate line search procedureis employed. [25]A quasi-Newton algorithm can then be stated as follows:
ALGORITHM 1. Data
. Initial
x
0
∈ℜ
n
and
B
0
∈ℜ
n
×
n
symmetric and positive deﬁnite. Set
k
= 0
.
Step 1
. Computation of the search direction
d
k
∈ℜ
n
, by solving the linear system
B
k
d
k
=
−∇
f
(
x
k
)
(8)
Step 2
.
Line search
Find a step length
t
k
that reduces
f
(
x
)
, accordingto a given line search criterium.
Step 3
.
Updates
Take
x
k
+1
:=
x
k
+
t
k
d
k
4
Veranise Dubeux, Jos´e Herskovits and Sandro Rodrigues Mazorche
B
k
+1
:=
B
k
+ ∆
B
k
k
:=
k
+ 1
Step 4
.
Go back to Step 1.
Working with an approximation of the inverse,
H
k
≈
[
∇
2
f
(
x
)]
−
1
, is advantageoussince itallows the search direction
d
k
to be calculatedwith a simple matrix-vector multiplication.We have that the secant condition (5) is equivalent to
δ
=
H
k
+1
γ
. Thus, an updating rulefor
H
can be easily obtainedby interchanging
B
and
H
as well as
δ
and
γ
in (6). We have
H
k
+1
=
H
k
+
δδ
t
δ
t
γ
−
H
k
γγ
t
H
k
γ
t
H
k
γ ,
(9)called DFP ( Davidon, Fletcher, Powell) updating rule. In general, the approximation matrix
H
k
+1
that is obtained with this rule is not the inverse of
B
k
+1
given by BFGS rule. An ex-pression for
H
k
+1
correspondingto the BFGS rule can be obtained from (6) by computingtheinverse of
B
k
+1
employingthe Sherman - Morrison - Woodbury formula, [13]
H
k
+1
=
H
k
+
1 +
γ
t
H
k
γ γ
t
δ
δδ
t
δ
t
γ
−
δγ
t
H
k
+
H
k
γδ
t
γ
t
δ .
(10)
2.1 Limited Memory Quasi - Newton Method
With the limited memory formulation, the product of the quasi-Newton Matrix
H
k
+1
timesa vector
v
∈ ℜ
n
, or a matrix, can be efﬁciently computed without the explicit assembly andstorage of
H
k
+1
. It is only required the storage of the
q
last pairs of vectors
δ
and
γ
. Inparticular, this techniquecan beemployedfor the computationof the search directionin a quasi- Newton algorithm for unconstrainedoptimization.The updatingrule (10) for
H
can be expressed as follows:
H
k
+1
=
H
k
−
q
+ [∆
H
k
−
q
Γ]
E
[∆
H
k
−
q
(11)where
∆ = [
δ
k
−
q
,δ
k
−
q
+1
,δ
k
−
q
+2
,...,δ
k
−
1
]; ∆
∈ℜ
n
×
q
Γ = [
γ
k
−
q
,γ
k
−
q
+1
,γ
k
−
q
+2
,...,γ
k
−
1
]; Γ
∈ℜ
n
×
q
E
=
R
−
t
(
D
+ Γ
t
H
k
−
q
Γ)
R
−
1
−
R
−
t
−
R
−
1
0
;
E
∈ℜ
2
q
×
2
q
R
=
upper
(∆
t
Γ);
R
∈ℜ
q
×
q
D
=
diag
(
R
)
We write
A
=
upper
(
B
)
when
A
ij
=
B
ij
for
j
≥
i
and
A
ij
= 0
for
j < i
.The limited memory methodtakes
H
k
−
q
=
I
. Then, the following expressionfor
H
k
+1
v
isobtained:
H
k
+1
v
=
v
+ [∆ Γ]
E
[
∆
Γ]
t
v
.
(12)5

Search

Similar documents

Related Search

Decomposition methods for large-scale optimizLarge Scale OptimizationNeed a super Memory? Read this how can you deEvolving A New Model (SDLC Model-2010) For SoA simple rapid GC-FID method for the determinA. K. (2011) Some Herbal Remedies for Common Experimental Study of a Shape Memory Alloy AcVery Large Scale IntegrationLate Hellenistic large-scale coinagesLarge Scale Data Processing

We Need Your Support

Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks