Health & Fitness

A CUTTING PLANE METHOD FOR SOLVING CONVEX OPTIMIZATION PROBLEMS OVER THE CONE OF NONNEGATIVE POLYNOMIALS

Description
A CUTTING PLANE METHOD FOR SOLVING CONVEX OPTIMIZATION PROBLEMS OVER THE CONE OF NONNEGATIVE POLYNOMIALS Iurie Caraus Moldova State University Fac. o Mathematics and Inormatics Mateevici 60 str., Chisinau
Published
of 10
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
A CUTTING PLANE METHOD FOR SOLVING CONVEX OPTIMIZATION PROBLEMS OVER THE CONE OF NONNEGATIVE POLYNOMIALS Iurie Caraus Moldova State University Fac. o Mathematics and Inormatics Mateevici 60 str., Chisinau Republic o Moldova, MD-009 Ion Necoara University Politehnica Bucharest Fac. o Automation and Computers Automation and System Engineering Dept. Spl. Independentei Bucharest, Romania Abstract: Many practical problems can be ormulated as convex optimization problems over the cone o nonnegative univariate polynomials. We use a cutting plane method or solving this type o optimization problems in primal orm. Thereore, we must be able to veriy whether a polynomial is nonnegative, i.e. i it does not have real roots or all real roots are multiple o even order. In this paper an eicient method is derived to determine a scalar value or which the polynomial is negative and in the case that such a value exists a easible cut is constructed. Our method is based on Sturm theorem, which allows to determine the number o distinct roots o a polynomial on a given interval, in combination with the bisection method. For numerical stability we construct the associated Sturm sequence using Chebyshev basis, and thus we can work with high degree polynomials, up to hundreds. Numerical results show the eiciency o our new approach. Key Words: convex optimization problems over the cone o nonnegative polynomials, cutting plane method, Chebyshev polynomials, Sturm sequence, easible cut. 1 Introduction Nonnegative polynomials on the real line play a undamental role in systems and control theory: e.g., robust control [11], ilter design problems [10], etc. The set o nonnegative univariate polynomials orm a convex set, more precisely a convex cone and was recently studied in the area o convex optimization [8]. Convex optimization techniques allow us to eiciently treat the cone o nonnegative polynomials, which can be parameterized by semideinite matrices [9, 15]. In [8] it was shown that or a convex optimization problem over this particular cone, the corresponding dual problem leads to a special semideinite matrix structure, more speciically a dual Hankel-matrix structure. Thereore, various interior-point and cutting plane algorithms exploiting this dual semideinite Hankel-structure were proposed in [8]. However, or large-scale problems the existing semideinite interior-point based algorithms are ineicient since we must solve linear systems with large dimension matrices. Moreover, the methods mentioned above utilize the classical basis or polynomials which limit the polynomial degree. This originates rom the act that the polynomials are described using the natural powers 1, x, x,, which lead to ill-conditioned Hankel matrices. In this article we also consider convex problems over the cone o nonnegative polynomials on the real line, subject to linear constraints. We restrict ourselves to irst-order methods, i.e. methods that use only irst-order inormation (unction evaluation and gradient evaluation). In particular we propose a cutting plane algorithm [5] or solving this type o problems in primal orm without embedding it into a semideinite ormat corresponding to the dual ormulation. Tackling the problem directly in the primal orm permits us to remove some o the disadvantages o the methods mentioned above, in particular it allows us to take into account the structure o the ISSN: Issue 7, Volume 8, July 009 given problem such as sparsity. Our contribution is twoold. First, or the numerical stability o our method it is important that the polynomials are represented with respect to some orthogonal basis, e.g. the Chebyshev basis. This approach permits to work with high degree polynomials, up to hundreds. The explanation is derived rom the observation that the Chebyshev polynomials have Sturm sequences with small coeicients and thus our method is numerically stable. Moreover, using Chebyshev basis or representing nonegative polynomials, the corresponding dual cone leads to a semideinite Toeplitz-plus-Hankel matrix structure, i.e. with well-behaved matrices. Second, or using a cutting plane algorithm we need an eicient method to construct a easible cut. In our case this reduces to checking nonnegativity o a polynomial. I the polynomial does not have real roots or all real roots are multiple o even order, then it is nonnegative. I the polynomial is negative then, we propose an eicient method or constructing a easible cut using bisection in combination with Sturm theorem or determining some scalar or which the polynomial value is negative. We construct the Sturm sequence in the Chebyshev basis and with Sturm theorem we can determine the number o distinct real zeros o a polynomial in a given interval. We determine a scalar value or which the polynomial is negative by Sturm theorem with bisection and with this value a easible hyperplane is constructed and used in the cutting plane method. The layout o the paper is as ollows. Section gives the problem ormulation, the main ingredients o a cutting plane method and the mathematical deinitions which are used to derive Sturm theorem. In Section 3 we derive a method or counting the real zeros o a given polynomial within an interval. The method uses the notion o Cauchy index, replacing the problem o counting zeros lying in an interval, by the evaluation o a Cauchy index o a rational unction associated with that polynomial in the Chebyshev orm. In Section 4 we propose an eicient method or constructing a easible cut based on bisection and Sturm theorem. We conclude with an application in Section 5. Preliminaries In this section we introduce the problem that we are going to solve and the mathematical tools that will be used in the paper..1 Problem ormulation In this paper the ollowing convex problem is considered: min G() s.t. A b (1) K R, where G : R n+1 R is a convex unction, A is a R m (n+1) matrix and b R m. We consider that n is even and in general m n + 1. We assume that the set { R n+1 : A b} is bounded. Moreover, K R is the convex cone o coeicients o polynomials nonnegative on the interval (a, b), i.e. or = [ 0 n ] T K R = { R n+1 : x+ + n x n 0, x (a, b)}. There exist dierent methods or solving convex problems o the orm (1), such as interior-point based methods in primal and dual orm [8, 9], irst-order methods [7], etc. In this paper we use a cutting plane method or solving (1). The basic scheme o a cutting plane method or solving a general convex problem min x Q g(x), where g is a convex unction and Q is a convex set in some vector space endowed with a scalar product,, consists in the ollowing steps: Algorithm 1 (cutting plane method or solving convex problem g = min x Q g(x)) 1. choose an accuracy ε 0 and a starting point x 0. k th iteration (k 0):.1 i x k Q (easibility cut) a k, b k s.t. a k, x x k +b k 0. i x k Q (optimality cut) g(x) g(x k ) + g(x k ), x x k x dom(g) x Q ISSN: Issue 7, Volume 8, July 009 3. compute x k+1 4. test g(x k+1 ) g ε. Here, g(x) denotes the (sub)gradient o the unction g and ε is the required accuracy o the approximate solution. There are dierent strategies to update x k+1 in step 3 o a cutting plane method (Algorithm 1): e.g. center o gravity [9], ellipsoid [6], analytic center [7], etc.. Sturm theorem Note that in order to solve the optimization problem (1) using a cutting plane method we should check i a polynomial belongs to the convex cone K R. Sturm theorem allows us to decide i our polynomial has real roots in the interval (a, b). In the sequel we introduce the basic ingredients or deriving Sturm theorem (see [3, 4] or more details). Let (x) be a real polynomial, i.e. in the classical basis it can be written as: (x) = n k x k, where k are real numbers. In the next deinition we introduce the so-called Cauchy index: Deinition 1 The Cauchy index o a real rational unction R(x) between the limits a and b (notation I b ar(x); a and b are real numbers or ± ) is the dierence between the number o jumps o R(x) rom to + and that o jumps rom + to as the argument changes rom a to b. In counting the number o jumps, the extreme values a and b are not included. One o the methods o computing the Cauchy index I b ar(x) is based on the classical Sturm sequence: Deinition We consider a sequence o polynomials: 0 (x), 1 (x),..., m (x). () Such a sequence is called a Sturm sequence on an interval (a, b), where either a or b may be ininite, i (i) m (x) does not vanish in (a, b); (ii) at any zero o k (x), k = 1,, m 1, the two adjacent unctions are nonzero in (a, b) and have opposite signs; that is, k 1 (x) k+1 (x) 0. Deinition 3 Let { i (x)} i=0,...,m be a Sturm sequence on (a, b), and let x 0 be a point o (a, b) at which 0 (x 0 ) 0. We deine V (x 0 ) to be the number o changes o sign o { i (x 0 )} i, zero values being ignored. I (i) a is inite, then V (a) is deined as V (a + ε), where ε is such that no i (x) vanishes in (a, a+ε) and similarly or b when b is inite; (ii) a =, then V (a) is deined to be the number o changes o sign o { lim x i(x)} i and similarly or V (b) when b =. The next theorem provides a way to compute the Cauchy index: Theorem 4 (Sturm theorem) [3] I { i (x)} i is a Sturm sequence in (a, b) and V (x) is the number o variations o sign in the sequence, then Ia b 1 (x) 0 (x) = V (a) V (b). (3) It is easy to see that by means o Sturm theorem the number o distinct real roots o the polynomial (x) in the interval (a, b) is Ia b (x), where (x) (x) denotes the derivative o (x). 3 Sturm sequence in Chebyshev basis Any algorithm or devising the Sturm sequence (e.g. Euclidian algorithm) is numerically unstable when the polynomials are represented in the classical basis since in general we have to perorm polynomial division. Thereore, in this section we derive the Sturm sequence associated to a given polynomial using the shited Chebyshev basis. This type o polynomials are orthogonal on the given interval and satisies a three term recurrence relation. ISSN: Issue 7, Volume 8, July 009 3.1 Constructing Sturm sequence using shited Chebyshev polynomials We deine shited Chebyshev polynomials on a given interval (a, b) using the ollowing linear transormation [] s(x) = x (a + b), x (a, b). b a The shited Chebyshev polynomials o the irst kind o degree i on (a, b) denoted T i (x) satisy the ollowing three-recurrence relation: T 0 (x) = 1 T 1 (x) = s(x) T i+1 (x) = s(x)t i (x) T i 1 (x), i 1. Shited Chebyshev polynomials o the second kind U i (x) are deined as ollows: U 0 (x) = 1 U 1 (x) = s(x) U i+1 (x) = s(x)u i (x) U i 1 (x), i 1. Given a polynomial (x) using the shited Chebyshev basis, a Sturm sequence may be ormed using Euclid algorithm or computing the greatest common divisor between 0 (x) = (x) and 1 (x) = (x). Divide 0 (x) by 1 (x) and take (x) to be the negative o the resulting remainder. Proceeding in this ashion, the degree o i (x) is always less than i 1 (x). Let us assume that our polynomial can be represented in the shited Chebyshev basis o the irst kind as: n (x) = a k T k (x), x (a, b). It is easy to show the ollowing relation between the irst and second kind Chebyshev polynomials: [T k (x)] = k b a U k 1(x). Based on this relation we ind that n 1 1 (x) = b k U k (x), where b k = (k + 1) b a a k+1, k = 0,..., n 1. (4) The remainder (x) can be computed as (x) = (α 1 1T 1 (x) + α 1 ) 1 (x) 0 (x), i.e. takes the ollowing orm: (x) =[ 1 a a α1 1b 1 + α 1 b 0 ]U 0 (x)+ n [ 1 a k+ + 1 α1 1b k+1 + k=1 a n 1 α1 1b k 1 + α 1 b k 1 a k]u k (x), where α1 1 = and α 1 = a n 1 b n α1 1. b n 1 b n 1 For the general case (i.e. j ) we assume that j 1 (x) = n j+1 c k U k (x) and j (x) = n j d k U k (x). Writing the remainder j+1 (x) = (α j 1U 1 (x) + α j ) j (x) j 1 (x) we ind that n j 1 + where k=1 j+1 (x) = [α j 1d 1 + α j d 0 c 0 ]U 0 (x) [α j 1d k 1 + α j 1d k+1 + α j d k c k ]U k (x), α j 1 = c n j+1 d n j, α j = c n j d n j 1 α j 1 d n j. (5) It is important to see that or any scalar β (a, b), the evaluation o the Sturm sequence at this value can be done recursively, without explicitly evaluating j (x) at β. Indeed, given 0 (β) and 1 (β) (these values can be computed since we know explicitly the polynomial (x)), we have the ollowing recurrent relation: j+1 (β) = (α j 1U 1 (β) + α j ) j (β) j 1 (β) or all j. Thereore, or inding the number o changes o sign o the sequence { i (β)} i 0 we need to know the ollowing two sequences {α i 1} i 0 and {α i } i 0. From the above discussion we can conclude that the computational complexity or inding the Sturm sequence is O(n ). ISSN: Issue 7, Volume 8, July 009 Remark 5 Note that we can also have the case when the dierence in the degree o two consecutive polynomials in the Sturm sequence is larger than one. In this case it is obvious that the quotient is not linear. However, the derivation o the coeicients o the quotient are computed similarly as above or the linear case. From the previous discussion we can derive the ollowing theorem: Theorem 6 The number N o distinct real roots inside the interval (a, b) o a given polynomial (x) written in the Chebyshev basis is given by N = V (a) V (b). 3. Destroying multiplicity In this section we derive the Sturm sequence in the case when the polynomial has multiple roots. I we have multiple roots, we add a small positive constant ε to our polynomial in order to destroy its multiplicity. We construct the Sturm sequence as in the previous section: let ε 0(x) be the perturbed polynomial, i.e. ε 0(x) = n a k T k (x) + [ε + a 0 ]T 0 (x). k=1 Note that 1(x) ε = n 1 b k U k (x), where b k is deined in (4). As in the previous section we obtain (x) ε =[ 1 a 1 (a 0 + ε) + α 1 b 1 + α b 0 ]U 0 (x) n + [ 1 a 1 k+ + α 1 b k+1 + α 1 1b k 1 + k=1 a n α b k 1 a k]u k (x), where α 1 = and α = a n 1 b n α 1. b n 1 b n 1 By induction we can show the ollowing: (i) i j = 1 n/ j 1 j+1(x) ε = r k (ε)u k (x) + n j 1 k=j r k U k (x), where or the coeicients r k (ε) the dependence rom ε is linear. (ii) i j = n/ + 1 n, taking into account (5) we obtain n j+1(x) ε = r n k (ε)u n k (x), k=j+1 where or the coeicients r k (ε) the dependence rom ε is rational, i.e. r k (ε) = P (ε)/q(ε), with P (ε) and Q(ε) are real polynomials, with deg(p ) = deg(q) + 1. Theorem 6 can also be applied in this case, which allow us to determine the number o distinct real roots o our original polynomial. 3.3 Comrade Matrix Approach Using the Sturm sequence method developed in Section 3 we can check i a given polynomial has real roots or not on some interval. I the associated Cauchy index Ia b = 0 then the polynomial is nonnegative on the interval (a, b). An- other method to decide i the polynomials is nonnegative or not on the interval (a, b) is given by the comrade matrix approach. For an orthogonal basis o polynomials satisying a 3-term recurrence relation 0 (x) = α 0, 1 (x) = α 1 x + β 1, i (x) = (α i x + β i ) i 1 (x) γ i i (x), i (with γ i 0), and a polynomial represented in this basis as (x) = δ( n (x) + a 1 n 1 (x) +... a n 0 (x)), we deine the comrade matrix β 1 1 α 1 α a n α n γ α β 1 α α a n 1 α n γ 0 3 C = α 3 β 3 1 α 3 α a 3 α n a +γ n α n γ n 1 α n 1 a 1+β n α n It is known that the zeros o the polynomial are the eigenvalues o the above matrix C. Recently methods with O(n ) complexity were developed to solve the comrade eigenvalue problem [17], [18]. The comrade matrix approach consists in the ollowing steps: ISSN: Issue 7, Volume 8, July 009 compute the eigenvalues o the matrix C associated to our polynomial take the middle points and evaluate in these points i all these values are nonnegative then belongs to K R. 3.4 Example We consider the polynomial = n a k T k (x) on the interval (a, b). We take a polynomial o degree n = 4 on the interval (1, 10). The corresponding Chebyshev coeicients or this polynomial are given in the ollowing table: a 0 =.63 a 1 = 1.70 a = 0.8 a 3 = 0.7 a 4 = 0.9 a 5 = 0.37 a 6 = 0.5 a 7 = 0.47 a 8 = 0.18 a 9 = 0.5 a 10 = 0.15 a 11 = 0.53 a 1 = 0.4 a 13 = 0.51 a 14 = 0.47 a 15 = 0.47 a 16 = 0.69 a 17 = 0.39 a 18 = 0.7 a 19 = 0.6 a 0 = 0.50 a 1 = 0.1 a = 0. a 3 = 0.03 a 4 = 0.04 For a = 1 the values o the Sturm sequence are displayed in the ollowing table and the number o sign changes is 13: 0 = 1 1 = = = = = = = = = = = = = = = = = = = = = = = = 0.06 For b = 10 the values o the Sturm sequence are given below and the number o sign changes are 11: 0 = = = = = = =.01 7 = = = = = = = = = = = = = = = = = = 0.0 In conclusion, I1 10 = V (1) V (10) = = and so our polynomial has real roots on this interval. n=4 interval [1 10] Figure 1: A polynomial o degree n = 4 From Figure 1 we can see that our polynomial has indeed two real roots in the interval (1, 10), in the neighborhood o x = 1. For the same polynomial, represented using shited Chebyshev basis on the interval (a, b), we calculated the zeros o this polynomial using comrade matrix approach and we obtain explicitly the ollowing two roots: and An eicient method or constructing a easible cut In the introduction we have described the main steps o a cutting plane method (Algorithm 1) or solving convex problems. As we have seen, the main diiculty in application o the cutting plane ISSN: Issue 7, Volume 8, July 009 method is to derive an eicient method or constructing the easible cut. In this section we provide such a method in the case when we solve convex problems over the cone o nonnegative polynomials o the orm (1). Our method combines bisection with the Sturm theorem (see Theorem 6) in order to ind a scalar value or which the polynomial takes a negative value. Once this scalar is determined, we can derive a easible hyperplane in an straightorward manner. We also discuss the complexity o our method. 4.1 Checking nonnegativity o a polynomial In the sequel we present an algorithm or veriying the nonnegativity o a given polynomial (x) 1. The output o this algorithm is either a scalar value α or which the polynomial value is negative or our polynomial is nonnegative. Algorithm 1. Given a polynomial (x) written in the shited Chebyshev basis on the interval (a, b) as (x) = n c kt k (x), where c 0 c n are real coeicients.. Find N = V (a) V (b) the number o distinct real roots o (x) in (a, b), using Theorem 6..1 i N = 0 then our polynomial is nonnegative and stop.. i N = n then go to step 4..3 else use Theorem 6 or the perturbed polynomial (x) + ε and ind N ε = V ε (a) V ε (b) (where ε 0 is suiciently small)..3.1 i N ε = 0 then (x) is nonnegative but has multiple roots and stop..3. i N ε = N go to step else (i.e. 0 N ε N) go to step 4 or (x) = ε (x). 3. i this interval contains more than one root, we divide it into two subintervals 1 Note that any polynomial written in the classical basis can be also represented using shited Chebyshev basis or a given interval. ( ] ( ] (a + b) (a + b) a, and, b. We evaluate the given polynomial in. I (a + b) ( ) (a + b) (a + b) 0, then α =. We interrupt this process and we construct the (a + b) easibility cut using α =. 4. Obtain an interval (a i, b i ]. 4.1 i the interval has more than one root we evaluate (x) in a i + b i and proceed as in step i the interval has one root we will veriy the unction values in a i and b i. I the unction values in a i and b i are positive, then we will discard this interval. Otherwise α is either a i or b i. 4.3 else continue this process discarding intervals that contain no roots. I we did not receive a negative unction value during this algorithm, then it means that we have a nonnegative polynomial. Note that our Algorithm is dierent rom the bisection method or isolation o the roots o a polynomial since we interrupt the iteration at the irst time when an α, or which our polynomial has a negative value, is ound. 4. Feasibility/Optimality cuts or the cone o nonnegative polynomials We assume that the convex optimization problem (1) has the ollowing reormulation when using shited Chebyshev basis: min where p = [c 0 c
Search
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks