Health & Medicine

A Limited Memory Quasi-Newton Preconditioner for Large Scale Optimization

Description
A Limited Memory Quasi-Newton Preconditioner for Large Scale Optimization
Published
of 21
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  III EuropeanConference on Computational MechanicsSolids, Structures and Coupled Problems in EngineeringC.A. Mota Soares et.al. (eds.)Lisbon, Portugal, 5–8 June 2006 A LIMITED MEMORY QUASI-NEWTON PRECONDITIONER FORLARGE SCALE OPTIMIZATION Veranise Dubeux 1 , Jos´e Herskovits 1 , and SandroRodrigues Mazorche 21 COPPE - Federal University of Rio de Janeiro - MechanicalEngineering ProgramCaixa Postal 68503, 21945 970, Rio de Janeiro - Brazil.e-mail: veranise@optimize.ufrj.br , jose@optimize.ufrj.br 2 Departamentode Matem´atica, ICE - UniversidadeFederal de Juiz de Fora.Cidade Universitria, Cep 36036-330- Juiz de Fora - MG - Brazil.e-mail: mazorche@ice.ufjf.br Keywords:  Preconditionerforthe conjugategradientemethod,Limited Memoryquasi-Newtontechniqueand interior point algorithm. Abstract.  We study the appication of the conjugated gradient method preconditioned by alimited memory quasi-Newton matrix in the resolution of the FAIPA’s internal linear systems.FAIPA, the Feasible Arc Interior Point Algorithm, is an interior-point algorithm that solvesnonlinear optimization problems. It makes iterations in the primal and dual variables of theoptimization problem to solve the Karush-Kuhn-Tucker optimality conditions. Given an initialinterior point, it defines a sequence of interior points with the objetive function monotonicallyreduced.FAIPA requires the solution of three linear systems whit the same coefficient matrix at each it-eration. These systems, when solved in terms of the Lagrange’s multipliers, are in general full,symmetric and positive definite. The conjugated gradient is largely employed for the iterativesolution of symmetric and positive definite linear systems.When the system is badly conditioned, a preconditioner matrix con be employed. The precon-ditioner’s choice is fundamental for the technique’s efficiency.We present a preconditioner for the FAIPA’s linear systems based on the limited memoy BFGS technique and consider problems with full and sparse matrices. Numerical examples show the performance of our methodology. 1  Veranise Dubeux, Jos´e Herskovits and Sandro Rodrigues Mazorche 1 INTRODUCTION The engineeringoptimization task consists in finding the designvariables  x 1 ,x 2 ,...,x n  thatminimize  f  ( x ) subject to  g ( x ) ≤ 0 and  h ( x ) = 0 , (1)where  x  ≡  [ x 1 ,x 2 ,...,x n ] t , the scalar function  f  ( x )  is the objective function and  g ( x )  ≡ [ g  1 ( x ) ,g  2 ( x ) ,...,g  m ( x )] t and h ( x ) ≡ [ h 1 ( x ) ,h 2 ( x ) ,...,h  p ( x )] t representinequalityandequal-ity constraints. We assume that  f  ( x ) ,  g ( x )  and  h ( x )  are continuousin  ℜ n as well as their firstderivatives. In engineering applications most of these functions are nonlinear. Then, (1) is asmooth nonlinear constrainedmathematical programmingproblem.Reallife engineeringsystemsinvolveaverylargenumberofdesignvariablesandconstraints.Evaluationof functionsandof derivativescomingfrom engineeringmodelsis very expensive interms of computer time. In practical applications, calculation and storage of second derivativesare impossibleto be carried out. Then, numericaltechniquesfor engineeringoptimization mustbe capable to solve very large problems with a reasonable number of function evaluations andwithout needing second derivatives. Robustness is also a crucial point for industrial applica-tions.Quasi-Newton method creates an approximation matrix of second derivatives. [13, 25, 43,48, 50] With this method large problems can be solved in a reasonable number of iterations.Employing rank two updatingrules, like BFGS or DFP, it is possible to obtain positive definiteapproximation matrices. This is a requirement of optimization algorithms that include a linesearch procedure [24, 25, 26, 43, 48, 50] to ensure global convergence. However, the classicquasi-Newton method cannot be applied for large problems since it requires the calculus andstorage of approximationmatrices, which are always full.Limited memory quasi-Newton method avoids the storage of the approximation matrix.[11, 42, 47, 48] Positive definite matrices can also be obtained with this technique. It was firstdeveloped for unconstrainedoptimization and then extended to problems with side constraints.Employingthe Feasible Arc Interior Point Algorithm (FAIPA), the limited memory methodcanalso be applied for constrainedoptimization problems. [19, 31, 32, 44]Anotherapproachto solve largeproblemswith a quasi-Newton techniqueconsists in obtain-ingsparseapproximationmatrices. This ideawasfirst exploitedbyToint in the 70th[56, 57,58]and by Fletcher et. al. in the 90th .[17, 18] In both cases sparse matrices were obtained in avery efficient way. However, those methods cannot be appliedfor optimization algorithms witha line search, since it is not guaranteedthat the approximationmatrices are positive definite. Inbooth cases, the authors worked with a trust region algorithm, but the numerical results werepoor.The numerical techniques described in this chapter are based on the Feasible Arc InteriorPoint Algorithm (FAIPA)[34] for nonlinear constrained optimization. FAIPA, that is an exten-sion of the Feasible Directions Interior Point Algorithm, [22, 23, 24, 26, 49] integrates ideascoming from the modern Interior Point Algorithms for Linear Programming with Feasible Di-2  Veranise Dubeux, Jos´e Herskovits and Sandro Rodrigues Mazorche rection Methods. At each point, FAIPA defines a ”Feasible Descent Arc”. Then, it finds a newinterior pointon thearc, with a lower objective. Newton,quasi - Newtonandfirst orderversionsof FAIPA can be obtained.FAIPA is supported by strong theoretical results. Global convergence to a local minimumof the problem is proved with relatively weak assumptions. The search along an arc ensuressuperlinear convergence for the quasi - Newton version, even when there are highly nonlinearconstraints, avoiding the so called ”Maratos’ effect” [45]. FAIPA, that is simple to code, doesnot require the solution of quadratic programs and it is not a penalty or a barrier method. Itmerely requiresthe solution of three linear systems with the samematrix periteration. This oneincludes the second derivative of the Lagrangian, or a quasi - Newton approximation. Severalpractical applicationsof the present and previous versions of FAIPA, as well as several numer-ical results, show that FAIPA constitutes a very strong and efficient technique for engineeringdesign optimization, [1, 2, 4, 5, 6, 7, 8, 27, 28, 29, 30, 38, 39, 40, 41, 54] and also for structuralanalysis problems with variational inequalities. [3, 60, 62]The main difficulty to solve large problems with FAIPA comes from the size and sparsity of the internal linear systems of equations. Since the quasi-Newton matrix is included in the sys-tems, limited memory and sparse quasi - Newton techniques can produceimportant reductionsof computer calculusand memory requirements.In this chapterwe presenta new sparsequasi - Newtonmethodthat workswith diagonalpos-itive definitematrices andemploythis techniquefor constrainedoptimization with FAIPA. Thisapproachcan beemployedalso in the well known sequential quadraticprogrammingalgorithm(SQP)[25, 51, 52] or in the interior point methods for nonlinear programming, as primal-dualor pathfollowingalgorithms.[20, 48, 61] We also describe numericaltechniques,to solve largeproblemswith FAIPA, employingexact or iterative linear solvers andsparse or limited memoryquasi-Newton formulations.Quasi - Newton method is describedin the next section, including limited memory formula-tion and our proposal for sparse quasi - Newton matrices. FAIPA is described in Sec. 3 and thestructure of the internal systems and some numerical techniques to solve them are discussed inSec. 4. Numerical experiments with a set of test problemsare reported in Sec. 5, followedwithsome results in structural optimization. Finally, we present our conclusionsin the last section. 2 Quasi - NewtonMethod for NonlinearOptimization We consider now the unconstrainedoptimization problemminimize  f  ( x ); x ∈ℜ n (2)Modern iterative algorithms define, at each point, a descent direction of   f  ( x )  and make aline search looking for a better solution. The quasi - Newton method works with a matrix thatapproximates the Hessian of the objective function, or of its inverse. The basic idea is to buildthe quasi - Newton matrix with information gathered while the iterations progress.Let the symmetric matrix  B k ∈ ℜ n × n be the current approximation of   ∇ 2 f  ( x k  ) . An im-3  Veranise Dubeux, Jos´e Herskovits and Sandro Rodrigues Mazorche proved approximation  B k +1 is obtainedfrom B k +1 =  B k + ∆ B k .  (3)Since ∇ f  ( x k +1 ) −∇ f  ( x k ) ≈ [ ∇ 2 f  ( x k )]( x k +1 − x k ) , the basic idea of quasi - Newton method consist in taking  ∆ B k in such way that ∇ f  ( x k +1 ) −∇ f  ( x k ) = [ B k +1 ]( x k +1 − x k ) ,  (4)called  “secant condition” , is true.Let δ  =  x k +1 − x k and  γ   = ∇ f  ( x k +1 ) −∇ f  ( x k ) . Then, (4) is equivalent to γ   =  B k +1 δ.  (5)The substitution of (3) into (5) gives us  n  conditions to be satisfied by  ∆ B k . Since  ∆ B k ∈ℜ n × n , the secant condition is not enough to determine  B k +1 . Several updating rules for  B k +1 were proposed. [13, 43, 52] The most successful is the BFGS (Broyden, Fletcher, Shanno,Goldfarb) formula B k +1 =  B k +  γγ  t δ t γ   −  B k δδ t B k δ t B k δ.  (6)If   B k is positive definite, it can be proved that δ t γ >  0  (7)is a sufficient condition to have  B k +1 positive definite. Under certain assumptions about  f  ( x ) ,(7) is satisfied if an appropriate line search procedureis employed. [25]A quasi-Newton algorithm can then be stated as follows:  ALGORITHM 1. Data . Initial x 0 ∈ℜ n and  B 0 ∈ℜ n × n symmetric and positive definite. Set  k  = 0 . Step 1 . Computation of the search direction  d k ∈ℜ n , by solving the linear system B k d k = −∇ f  ( x k )  (8) Step 2 .  Line search Find a step length  t k that reduces  f  ( x ) , accordingto a given line search criterium. Step 3 .  Updates Take x k +1 :=  x k +  t k d k 4  Veranise Dubeux, Jos´e Herskovits and Sandro Rodrigues Mazorche B k +1 :=  B k + ∆ B k k  :=  k  + 1 Step 4 .  Go back to Step 1.   Working with an approximation of the inverse,  H  k ≈  [ ∇ 2 f  ( x )] − 1 , is advantageoussince itallows the search direction  d k to be calculatedwith a simple matrix-vector multiplication.We have that the secant condition (5) is equivalent to  δ  =  H  k +1 γ  . Thus, an updating rulefor  H   can be easily obtainedby interchanging  B  and  H   as well as  δ  and  γ   in (6). We have H  k +1 =  H  k +  δδ t δ t γ   −  H  k γγ  t H  k γ  t H  k γ ,  (9)called DFP ( Davidon, Fletcher, Powell) updating rule. In general, the approximation matrix H  k +1 that is obtained with this rule is not the inverse of   B k +1 given by BFGS rule. An ex-pression for  H  k +1 correspondingto the BFGS rule can be obtained from (6) by computingtheinverse of   B k +1 employingthe Sherman - Morrison - Woodbury formula, [13] H  k +1 =  H  k +  1 +  γ  t H  k γ γ  t δ   δδ t δ t γ   −  δγ  t H  k + H  k γδ t γ  t δ .  (10) 2.1 Limited Memory Quasi - Newton Method With the limited memory formulation, the product of the quasi-Newton Matrix  H  k +1 timesa vector  v  ∈ ℜ n , or a matrix, can be efficiently computed without the explicit assembly andstorage of   H  k +1 . It is only required the storage of the  q  last pairs of vectors  δ  and  γ  . Inparticular, this techniquecan beemployedfor the computationof the search directionin a quasi- Newton algorithm for unconstrainedoptimization.The updatingrule (10) for  H   can be expressed as follows: H  k +1 =  H  k − q + [∆  H  k − q Γ] E  [∆  H  k − q (11)where ∆ = [ δ k − q ,δ k − q +1 ,δ k − q +2 ,...,δ k − 1 ]; ∆ ∈ℜ n × q Γ = [ γ  k − q ,γ  k − q +1 ,γ  k − q +2 ,...,γ  k − 1 ]; Γ ∈ℜ n × q E   =   R  − t ( D + Γ t H  k − q Γ) R  − 1 − R  − t − R  − 1 0  ;  E   ∈ℜ 2 q × 2 q R   =  upper (∆ t Γ);  R   ∈ℜ q × q D  =  diag ( R  ) We write  A  =  upper ( B )  when  A ij  =  B ij  for  j  ≥ i  and  A ij  = 0  for  j < i .The limited memory methodtakes  H  k − q =  I  . Then, the following expressionfor  H  k +1 v  isobtained: H  k +1 v  =  v  + [∆ Γ] E  [ ∆  Γ] t v .  (12)5
Search
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks