Journal of Combinatorial Optimization, 4, 197–215, 2000c
2000 Kluwer Academic Publishers. Manufactured in The Netherlands.
A Semideﬁnite Programming Approachto the Quadratic Knapsack Problem
∗
C. HELMBERG
Konrad Zuse Zentrum f ¨ ur Informationstechnik Berlin, Heilbronnerstra
ß
e 10, D10711 Berlin, Germany
F. RENDL
Technische Universit ¨ at Graz, Institut f ¨ ur Mathematik, Steyrergasse 30, A8010 Graz, Austria
R. WEISMANTEL
Universit ¨ at Magdeburg, Institut f ¨ ur Mathematische Optimierung, Universitrits Platz 2, 039106 Magdeburg,Germany Received December 22, 1998; Revised January 18, 2000; Accepted January 19, 2000
Abstract.
In order to gain insight into the quality of semideﬁnite relaxations of constrained quadratic 0
/
1programming problems we study the quadratic knapsack problem. We investigate several basic semideﬁnite relaxationsofthisproblemandcomparetheirstrengthintheoryandinpractice. Variouspossibilitiestoimprovethesebasic relaxations by cutting planes are discussed. The cutting planes either arise from quadratic representations of linear inequalities or from linear inequalities in the quadratic model. In particular, a large family of combinatorialcuts is introduced for the linear formulation of the knapsack problem in quadratic space. Computational resultson a small class of practical problems illustrate the quality of these relaxations and cutting planes.
Keywords:
semideﬁnite programming, quadratic knapsack problem, cutting planes, 0
/
1 polytopes, relaxations
1. Introduction
The quadratic knapsack problem(QK) Maximize
x
t
Cx
subject to
a
t
x
≤
b x
∈ {
0
,
1
}
n
.
with
a
∈
N
n
,
b
∈
N
is one of the standard combinatorial optimization problems (in someworks, the term quadratic knapsack problem has also been used for continuous versions of (QK)(Pardolosetal., 1991)). Typicalapplicationsofthediscreteproblemsuchasportfolioselection (Laughhunn, 1970) or site selection (Gallo et al., 1980) are of the following form.Afewitemshavetobeselectedfromagivengroundsetsubjecttobudgetaryconstraints. Theobjective, proﬁt/quality maximization or risk minimization, is strongly inﬂuenced by thepairwiseinterrelationbetweentheitemsoftheselectedset. Besidesthesedirectapplications
∗
A preliminary version of this paper appeared in the Proceedings of IPCO ’96 (Helmberg et al., 1996).
198
HELMBERG, RENDL AND WEISMANTEL
the quadratic knapsack problem also appears as a subproblem in several applications of constrained quadratic 0
/
1 programming, e.g. in VLSI and compiler design (Ferreira et al.,1996; Johnson et al., 1993).Current practical approaches for solving (QK) are branch and bound algorithms. Thebounds are specially designed for the quadratic knapsack problem and often require specialproperties of the objective function such as nonnegativity of
c
ij
(Hammer and Rader, 1997;Caprara et al., 1999). These approaches are of little use if (QK) appears as a subproblemwithin general constrained quadratic 0
/
1 programming. This drawback can be overcomeby resorting to a polyhedral approach. Typically, polyhedral methods work with linearrelaxations of (QK). However, these often exhibit a large gap even for very small instances(
n
=
20). We design a polyhedral approach using semideﬁnite methods and show that thisleads to reasonable bounds.In recent years it became clear that for unconstrained quadratic 0
/
1 programming problemssemideﬁniteprogrammingyieldsstrongbounds, intheory(GoemansandWilliamson,1995; Nesterov, 1998) as well as in practice (Helmberg and Rendl, 1998). From a theoretical point of view there is a well established technique for including linear constraints inthis semideﬁnite relaxation (Balas, 1975a; Sherali and Adams, 1990; Lov´asz and Schrijver,1991; Balas et al., 1993). The main ingredients of this technique are the pairwise multiplication of feasible inequalities and the substitution of products of variables by linearvariables (the ﬁnal projection step back into linear space is not needed in our context).Inthispaperwegotowardsapracticalrealizationofthistheoreticalapproach.InSection2wesetupvariousinitialsemideﬁniterelaxationsof(QK)inthelineofBalas(1975a),Sheraliand Adams (1990), Lov´asz and Schrijver (1991) and Balas et al. (1993) and compare themwithrespecttoqualityandefﬁciency.Inparticularweintroducethesquarerepresentationof the knapsack inequality. For the practical examples tested, the semideﬁnte relaxation basedon this single constraint achieves almost the bound of the standard semideﬁnite relaxation.The latter bound is much more expensive to compute, because it involves
n
additionalinequalities. We show that in the special case of a linear cost function the semideﬁniterelaxation formed by the square representation of the knapsack inequality still sufﬁces tobeat the linear relaxation.In order to improve an initial semideﬁnite relaxation it is necessary to add further inequalities that are valid for the linear formulation of the srcinal program in quadraticspace. Here there are two possibilities. One can derive valid inequalities that are valid forthepolyhedron associated with thefeasibleregion of thesrcinal problem and lift them intothe quadratic model (by the technique of Lov´asz and Schrijver or alternatives) or developa polyhedral understanding of the quadratic model itself. Some aspects of the quadraticknapsack polyhedron have been studied in Ferreira et al. (1996). In Section 3 we introducea new large family of combinatorial cuts for the linear formulation of the knapsack problemin quadratic space and analyze their strength in comparison to quadratic representations of certain linear cutting planes.Aclassofconstraintsthatappearsfrequentlywithinquadratic0
/
1programmingproblemsis the family of generalized upper bound constraints of the form
x
i
≤
k
. For constraintsof this form we construct a special quadratic representation, analyze its strength, and relateit to the hypermetric inequalities of the maxcut polytope. This is the topic of Section 4.
A SEMIDEFINITE PROGRAMMING APPROACH
199We discuss implementational issues and numerical results of the semideﬁnite programmingcodeforthequadraticknapsackprobleminSection5.Itwillturnoutthatthesemidefinite relaxations are of good quality. Computationally they are too expensive for solvinglargerinstancesof(QK)tooptimality. However, theapproachgivessomeinsightonimportant classes of inequalities that may help to set up good initial relaxations of more complexproblems.Thisisofparticularimportanceincombinationwithnewmethods(Bensonetal.,1997; Helmberg and Rendl, 1997) that can compute approximate solutions to semideﬁniterelaxations for large problem instances as well.
2. Modeling linear constraints for semideﬁnite relaxations
A common approach for designing relaxations for quadratic 0
/
1 programs is to linearizethe quadratic cost function by switching to “quadratic space”. Variables
y
ij
are introducedto model the products
x
i
x
j
for
i
≤
j
. Stated differently, the dyadic product
xx
t
is replacedby a (symmetric) matrix variable
Y
. We denote the diagonal of this matrix by
y
. Usingthis notation the feasible set of matrices can be restricted to those satisfying
Y
−
yy
t
0,i.e.
Y
−
yy
t
must be positive semideﬁnite (Lov´asz and Schrijver, 1991; Balas et al., 1994).This condition is equivalent to
Y y y
t
1
0
.
(1)The diagonal elements
y
i
are obviously bounded by 0 and 1 and correspond to
x
i
. Lookingat the determinant of a 3
×
3 principal minor containing the last row we get
y
i
y
j
−
y
i
y
j
(
1
+
y
i
y
j
−
y
i
−
y
j
)
≤
y
ij
≤
y
i
y
j
+
y
i
y
j
(
1
+
y
i
y
j
−
y
i
−
y
j
)
(2)which bounds the offdiagonal variables by
−
18
≤
y
ij
≤
1. Note, that the nonnegativityconstraints
y
ij
≥
0 are not implied by this semideﬁnite relaxation.The easiest way to model a linear constraint
a
t
x
≤
b
, with
a
∈
N
n
and
b
∈
N
on
Y
isto restrict the diagonal elements of
Y
, yielding the
diagonal representation
of
a
t
x
≤
b
andour ﬁrst semideﬁnite relaxation,(SQK1) Maximize
C
,
Y
subject to
Diag
(
a
),
Y
≤
bY
−
yy
t
0
.
Can we do better than (SQK1) by choosing some other representation of the knapsack inequality? Let us ﬁrst consider a particular case of the generic approach of Sherali andAdams (1990) and Lov´asz and Schrijver (1991).
b
−
a
t
x
≥
0 implies
(
b
−
a
t
x
)(
b
−
a
t
x
)
=
b
2
−
2
ba
t
x
+
a
t
xx
t
a
≥
0
.
200
HELMBERG, RENDL AND WEISMANTEL
So a possible representation for the knapsack inequality could read
b
2
−
2
ba
t
y
+
a
t
Ya
≥
0
.
However, this inequality is already implied by the semideﬁnite constraint
Y
−
yy
t
0.Because of the integrality of the coefﬁcients
a
i
and
b
we can employ a combinatorialargument to sharpen this inequality. Observe that

(
2
b
−
1
)
−
2
a
t
x

is at least one for all0
/
1 vectors
x
and therefore the quadratic representation of
(
2
b
−
1
−
2
a
t
x
)
2
≥
1 yieldsa valid inequality for the boolean quadric polytope which is best formulated with respectto (1),
(
−
2
a
t
2
b
−
1
)
Y y y
t
1
−
2
a
2
b
−
1
≥
1
.
(3)Thisinequalityisnomoreimpliedbythecondition
Y
−
yy
t
0. Indeed,itcanbeworkedoutthat(3)belongstotheclassofhypermetricinequalities
1
andthatforspecialchoicesof
a
and
b
it deﬁnes a facet of the boolean quadric polytope (for deﬁnition see Section 3). Notice,that this inequality does not exclude 0
/
1 solutions that violate the knapsack constraint.However, it is tight for all
x
which satisfy
a
t
x
=
b
or
a
t
x
=
b
−
1 and might therefore turnout to be a useful cutting plane.Toachieveourgoalofcuttingoffalargerpartofthebooleanquadricpolytopeweexploitthe fact that
a
t
x
≥ −
b
on the feasible set. Squaring both sides of
a
t
x
≤
b
yields
a
t
xx
t
a
≤
b
2
.
Replacing
xx
t
by
Y
we call this the
square representation
of the inequality
a
t
x
≤
b
and useit to form a second relaxation(SQK2) Maximize
C
,
Y
subject to
aa
t
,
Y
≤
b
2
Y
−
yy
t
0
.
Lemma 2.1.
(
SQK
2
)
is tighter than
(
SQK
1
)
.
Proof:
With
Z
=
Y
−
yy
t
we get
a
t
Za
+
(
a
t
y
)
2
≤
b
2
(4)which implies
a
t
y
≤
b
by the positive semideﬁniteness of
Z
.
This proof suggests the following corollary.
Corollary 2.2.
If a
t
y
=
b for some Y satisfying
aa
t
,
Y
≤
b
2
and Y
−
yy
t
0
,
then ais in the null space of Z
=
Y
−
yy
t
.
A SEMIDEFINITE PROGRAMMING APPROACH
201Another possibility to construct quadratic representations is to multiply the inequality byeither
x
i
or
(
1
−
x
i
)
(Balas, 1975a; Sherali and Adams, 1990; Lov´asz and Schrijver, 1991;Balas et al., 1993). If, for some ﬁxed
i
, we sum up the two inequalities
n
j
=
1
a
j
y
ij
≤
by
i
(5)
n
j
=
1
a
j
(
y
j
−
y
ij
)
≤
b
(
1
−
y
i
)
(6)we get
a
t
y
≤
b
.
Lemma 2.3
(Balas, 1975a; Sherali and Adams, 1990; Lov´asz and Schrijver, 1991)
.
Therelaxation obtained by replacing
Diag
(
a
),
Y
≤
b of
(
SQK
1
)
with a pair of inequalities
(
5
)
and
(
6
)
for some i is tighter than
(
SQK
1
)
.
By including all
n
inequalities of type (5) and one additional inequality of type (6) weget(SQK3) Maximize
c
,
r
subject to
n
j
=
1
a
j
y
ij
≤
by
i
i
=
1
...
n
n
j
=
1
a
j
(
y
jj
−
y
1
j
)
≤
b
(
1
−
y
1
)
Y
−
yy
t
0
.
Lemma 2.4.
(
SQK
3
)
is tighter than
(
SQK
2
)
.
Proof:
By multiplying inequality
i
of type (5) with
a
in
j
=
1
a
i
a
j
y
ij
≤
ba
i
y
i
and summing up over all
n
inequalities, we obtain
a
t
Ya
≤
ba
t
y
≤
b
2
. The right hand sideinequality follows from Lemma 2.3.
Itisclearthat(SQK3)canbeimprovedattheexpenseofaddingtheremaininginequalitiesof the form (6) to the relaxation. The improvement of the latter relaxation towards (SQK3)is, in all our test examples, neglectable. This is the reason why we work with (SQK3).Our computational experiments have also revealed that among the relaxations (SQK2) and(SQK3), the ﬁrst one has the best tradeoff between running time and quality. In fact, it isusually more efﬁcient to start with (SQK2) and to add Inequalities (5) and (6) in case of violation only, see Section 5.