Environment

A semidefinite framework for trust region subproblems with applications to large scale minimization

Description
A semidefinite framework for trust region subproblems with applications to large scale minimization
Categories
Published
of 35
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  ASemideniteFrameworkforTrustRegion SubproblemswithApplicationstoLargeScale Minimization    FranzRendl y HenryWolkowicz  z August29,1996  RevisionofCORRReport94-32. Abstract Primal-dualpairsofsemideniteprogramsprovideageneralframe-workforthetheoryandalgorithmsforthetrustregionsubproblem (TRS).Thislatterproblemconsistsinminimizingageneralquadraticfunctionsubjecttoaconvexquadraticconstraintand,therefore,itisageneralizationoftheminimumeigenvalueproblem.Theimportanceof(TRS)isduetothefactthatitprovidesthestepintrustregion minimizationalgorithms.Thesemideniteframeworkisstudiedasan interestinginstanceofsemideniteprogrammingaswellasatoolforviewingknownalgorithmsandderivingnewalgorithmsfor(TRS).In particular,adualsimplextypemethodisstudiedthatsolves(TRS)asaparametriceigenvalueproblem.ThismethodusestheLanczosalgorithmforthesmallesteigenvalueasablackbox.Therefore,theessentialcostofthealgorithmisthematrix-vectormultiplicationand,thus,sparsitycanbeexploited.Aprimalsimplextypemethodpro-videsstepsfortheso-calledhardcase.Extensivenumericaltestsfor  ThisisanabbreviatedrevisionoftheUniversityofWaterlooresearchreportCORR 94-32. y TechnischeUniversitatGraz,InstitutfurMathematik,Kopernikusgasse24,A-8010Graz,Austria.ResearchsupportbyChristianDopplerLaboratoriumfurDiskreteOpti-mierung(rendl@fmatbds01.tu-graz.ac.at). z UniversityofWaterloo,DepartmentofCombinatoricsandOptimization,Wa-terloo,OntarioN2L3G1,Canada(E-mail:hwolkowi@orion.uwaterloo.caandURL:http://orion.uwaterloo.ca/~hwolkowi).ResearchsupportbytheNationalScienceand EngineeringResearchCouncilCanada. 1   largesparseproblemsarediscussed.Thesetestsshowthatthecostofthealgorithmis1+    ( n  )timesthecostofndingaminimumeigen-valueusingtheLanczosalgorithm,where0 <  ( n  ) <  1isafraction whichdecreasesasthedimensionincreases. Keywords:trustregionsubproblems,parametricprogramming,semidenite programming,min-maxeigenvalueproblems,largescaleminimization.AMS1991SubjectClassication:Primary:49M37,90C30;Secondary:90C06,65K10. 1Introduction  Thetrustregionsubproblem,(TRS),consistsinminimizingaquadratic(possiblynonconvex)functionsubjecttoaquadratic(ornorm)constraint.Wepresentanecientalgorithmforthisproblemthatcanexploitsparsity.Thealgorithmisbasedonaparametriceigenvalueproblemwithinasemidef-initeprogramming,SDP,framework.Weincludetwopairsofprimal-dualSDPs.Theseprogramsprovideatransparentframeworkforouralgorithm aswellasforcurrentalgorithmsfor(TRS).Let q  ( x  ):=  x  t Ax  ?  2  a  t x; where A  =  A  t isasymmetricreal n    n  matrix and  a  2<  n  .Andlet s>  0  : Computationofthestepbetweeniterates,in trustregionalgorithmsforminimization,requiressolutionofthe trustregion subproblem  ( TRS  )    :=min  q  ( x  )(1.1)subjectto  x  t x  =  s  2 (   s  2 ) : (1.2)(Forsimplicityofexposition,(TRS)referstotheequalityconstrainedcase,=  s  2 : Numericaltestsareprovidedfortheinequalitycase,   s  2 : Wein-cludetheoreticaldetailsfortheinequalitycasewhentheyaresubstantially dierent.)Itiswellknownthatavector x  yieldstheglobalminimumof(TRS)ifandonlyifthereexists   2<  suchthatthefollowingrelationshold:( A  ?  I  ) x  =  a  ;(stationarity)(1.3) x  t x  =  s  2 ;(feasibility)(1.4) A  ?  I    0;(strengthenedsecondorder)(1.5)2   where   0denotespositivesemideniteness.Moreover,if A  ?  I  ispositivedenite,thentheoptimizer x  isunique.Morerecently,ithasbeenshown thatstrongLagrangiandualityholds,seeTheorem2.1.Currently,mostmethodsforsolving(TRS)arebasedonapplyingNew-ton'smethodtothe secularequation  in  ; whichisessentially(1.4)aftereliminating  x  using(1.3).TheNewtonmethodis safeguarded  tomaintain positivedenitenessinordertosatisfy(1.5).EachiterationusuallyrequiresCholeskyfactorizationsof A  ?  I  ,bothforsolving(1.3)andforsafeguarding (1.5).Thiscanbetooexpensiveforlargescaleoptimizationifsparsityislost.Thegeneral(TRS)liessomewherebetweenthepurequadratic(eigen-valuecase, a  =0)andthepurelinear(norm, A  =0)case.Bothofthesecasescanbesolvedquicklyandeasily.Dene g  ( s  ):=min  f  x  t Ax  ?  2  a  t x  : x  t x  =  s  2 g  : (1.6)Thus g  describestheoptimalsolutionofproblem(1.1),(1.2)dependingon thenorm  s  of x  .Furtherdene G  (   ):=min  f  y  t Ay  ?  2  a  t y  : y  t y  =1  g  : (1.7)Setting  x  :=  sy  wehave k x  k =  s  ()k y  k =1.Therefore,for s>  0,weget g  ( s  )=  s  2 G  ( 1 s ) : Itisclearthat G  (   )andtherefore g  ( s  )describe,inlimiting behaviour,thepurelyquadraticproblem(   !  0)aswellasthepurelylinearobjectivefunction(   !1  ).Wewillshowbelowthat g  ( s  )iscloselyrelated toaparametric(orperturbed)eigenvalueproblem. 1.1Background  TrustregionsubproblemsappearinthecontextofnonlinearleastsquaresinworkbyLevenberg1944,20]andinworkbyMarquadt1963,24].Theseauthorsworkedonthecasewhere A  ispositivedenite.Applicationsto generalminimizationappearsinworkbyGoldfeld,QuandtandTrotter1966 11].Earlytheoreticalresultson(TRS)appearinForsytheandGolub1965 8].Inparticular,theystudypropertiesofthesecularfunction,whichisessentialinalgorithmicderivations.Hebden197314]proposedanalgorithm whichexploitsthestructureofthesecularfunction.HemadeuseofearlierworkonthestructurebyReinsch1967and197135,36].Gay198110]improvedonthisalgorithmandhandledthehardcase,i.e.thecasewheretheoptimalLagrangemultiplierisequaltothesmallesteigenvalueof A  .3   Otheralgorithmsatthistimewereproposedin26,40].Amoreecienttreatmentofthehardcasewasthecentralpointoftheseminalworkby MoreandSorensenin1983,29].Theiralgorithmhasremainedasthestandardfor(TRS).Ittypicallyyieldsanapproximateoptimalsolutionin underteniterationsofaNewtontypemethod.Inparticular,thealgorithmisparticularlyecientinthehardcaseandtypicallytakesonly2-3iterations,seealso27].Otheralgorithmsarepresentedine.g.9,12]andmorerecently,usingDC(dierenceofconvexfunctions)optimization,in46].The(TRS)hasappearedelsewhereintheliteratureunderdierentguises.Itisequivalenttotheproblemofridgeregressioninestimation problems,e.g.16];anditisalsoequivalenttotheproblemofregularization forill-posedproblems,e.g.47].Recently,therehasbeenarevivalofinterestwithnewdualityresults,re-lationstoeigenvalueperturbations,andextensionstononconvexconstraintfunctions,seee.g.7,42,43,28,3,2].The(TRS)problemhasbeenshown tobesolvableinpolynomialtime,see52,18].Thepolynomialityisde-rivedusingdetailedestimatesin49].In43],itisshownthatstrongduality holdsfortheLagrangiandualof(TRS);thus,(TRS)isequivalenttoacon-cavemaximizationproblemand,therefore,itisatractablepolynomialtimeproblem,bytheresultsforgeneralconvexprogramspresentedin30].Appli-cationstosolvingNP-hardproblemsaregivenin13,53].Arecentstudyofthesubgradientsandstabilityof g  ( s  )ispresentedin44].Asurprisingresultthatthereisatmostonelocal-nonglobaloptimumfor(TRS)ispresentedin 25].Thispaperusesaparametriceigenvalueproblemtosolve(TRS).Pre-viouscharacterizationsofsolutionsof(TRS)asaparametriceigenvalueproblemappearin43,Theorem3.2].Inaddition,independentworkona parametriceigenvalueapproachsimilartoourworkispresentedbySorensen 41],andiscontinuedbySantosandSorensen38].(ComparisonswithourworkisgivenintheconcludingSection5.1.)Also,recently,therehasbeenalotofinterestinproblemswithmulti-plequadraticconstraints.Thetwotrustregionproblemaroseinsequen-tialquadraticprogrammingtechniquesforconstrainednonlinearminimiza-tion,5].Otherapplicationstoconstrainedoptimizationappearine.g.6].MultiplequadraticconstraintsariseincombinatorialoptimizationandtheirLagrangianrelaxationscanbeshowntobeequivalenttosemidenitere-laxations,e.g.39,33,34,32,19].In32],(TRS)isthetoolthatisused toprovetheequivalenceofseveralboundsforquadratic0,1optimization.Theseproblemsandrelaxationsalsoappearinsystemscontrole.g.4].4   Thus,(TRS)canbeseentobeanimportantsteppingstonebetween convexprogramsononehand,whichyieldnecessaryandsucientoptimal-ityconditions,andNP-hardproblemsontheotherhand,suchasmultiplequadraticconstrainedproblemsand,equivalently,hardcombinatorialop-timizationproblems.Moreover,theprimal-dualpairofSDPprogramswestudyillustratemanyoftheimportantgeometricandalgebraicpropertiesofgeneralSDPprograms.ThispairofSDPprogramsprovidetheoptimalso-lutionfor(TRS);thisisincontrasttogeneralSDPprogramswhichusually ariseasrelaxationsandprovideboundsfortheunderlyingoriginalproblem. 1.2Outline  Themainresultinthispaperisanalgorithmthatsolves(TRS)usingmatrix-vectormultiplicationandnoexplicitsolutionofasystemofequations,seeSection4.(Thereaderwhoisonlyinterestedinthealgorithmcanskip directlytothissection.)Thealgorithmisbasedonmaximizing(uncon-strained)arealvaluedconcavefunction, k  ( t ) ; basedonaparametriceigen-valueproblem,i.e. k  ( t )=( s  2 +1)   1 ( D  ( t )) ?  t; where D  ( t )=  "  t ?  a  t ?  aA  # : Wealsoprovideageneralframeworkfor(TRS)basedontwoprimal-dualpairsofSDPs.TheseSDPframeworkscanbeusedtoderivevariousal-gorithms.Inaddition,theSDPsareofinterestinthemselvessincethey illustratemanyinterestingpropertiesofgeneralsemideniteprogramming.Thepaperisorganizedasfollows.Webeginwithanonlinearprimal-dualpairofSDPsthatsolve(TRS).ThispaircanbeusedtodescribethestepsofthestateoftheartMoreandSorensenalgorithm,seeRemark2.4.WethenpresentalinearSDPprimal-dualpair.WeshowthatthestepsfrombeforecanbedoneherewithoutCholeskyfactorizations;thuswecan exploitstructureandsparsity.BothpairsofSDPsareequivalentto(TRS).Theprimal-dualSDPswithlinearconstraintsprovidetheframeworkforouralgorithmandareessentialfordealingwiththehardcase.Section3providesdetailedanalysisofthefunctionsthatariseintheaboveSDPprimal-dualpairs.Aparametriceigenvalueproblem,equivalentto(TRS),isderivedinSection3.1.Theorem3.7providestherelationshipsthatformthebasisofouralgorithm.Section3.2discussesthehardcaseinsidetheSDPframework;whileSection3.3describesthevariousfunctionsassociatedwith(TRS).5 
Search
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks