Applied Numerical Mathematics 57 (2007) 304–319www.elsevier.com/locate/apnum
A matrix decomposition RBF algorithm: Approximationof functions and their derivatives
Andreas Karageorghis
a
, C.S. Chen
b
, YiorgosSokratis Smyrlis
a
,
∗
a
Department of Mathematics and Statistics, University of Cyprus/
Π
ANE
Π
I
Σ
THMIO KY
Π
POY,PO Box 20537, 1678 Nicosia/
Λ
EYK
ΩΣ
IA, Cyprus/KY
Π
PO
Σ
b
Department of Mathematics, University of Southern Mississippi, Hattiesburg, MS 39406, USA
Available online 18 May 2006
Abstract
We propose an efﬁcient algorithm for the approximation of functions and their derivatives using radial basis functions (RBFs).The interpolation points are placed on concentric circles and the resulting matrix has a block circulant structure. We exploit thiscirculant structure to develop an efﬁcient algorithm for the solution of the resulting system using RBFs. As a result, extremely highaccuracy in approximating the given function and its derivatives can be achieved. The given algorithm is also capable of solvinglargescale problems with more than 100000 interpolation points in two dimensions.
©
2006 IMACS. Published by Elsevier B.V. All rights reserved.
Keywords:
Radial basis functions; Boundary meshless methods; Circulant matrices; Elliptic boundary value problems
1. Introduction
During the last three decades, methods using radial basis functions (RBFs) have been gaining in popularity invarious areas of scientiﬁc computing such as neural networks, computer graphics, Computer Aided Design (CAD) andsurface reconstruction. RBFs are particularly effective for multivariate approximation [8,7,10,11,28,29]. The theoryof RBFs was not well understood until Franke [14] published a survey paper on the evaluation of 29 interpolatingmethods. This was followed by a paper by Micchelli [25] who established ﬁrm theoretical results on the invertibilityof the RBF interpolation matrix. Further details of these results can be found in [8].In contrast to the many attractive features of RBFs, one disadvantage is that most of the commonly used RBFs areglobally supported. This implies that the resulting interpolation matrix is dense. It appears that other than possiblysymmetry, the interpolation matrices have no obvious structure that could be exploited in the solution of the interpolation system. As a result, when using a direct symmetric solver the cost of solving the linear system is
N
3
/
6
+
O
(N
2
)
ﬂops. Further, when the number of interpolation points used is large and they are densely distributed the matrix becomes illconditioned. In practice, direct methods for solving the systems resulting from RBFs are inappropriate forproblems with more than 2000 interpolation points. To alleviate these difﬁculties, compactly supported radial basisfunctions (CSRBFs) were developed in the last decade [6,29,33,34]. The use of CSRBFs leads to sparse matrix
*
Corresponding author.
Email address:
smyrlis@ucy.ac.cy (Y.S. Smyrlis).01689274/$30.00
©
2006 IMACS. Published by Elsevier B.V. All rights reserved.doi:10.1016/j.apnum.2006.03.028
A. Karageorghis et al. / Applied Numerical Mathematics 57 (2007) 304–319
305
systems which is desirable for largescale problems, and these basis functions have been used in various applications.However, the convergence rate of these newly constructed CSRBFs is slow and it is thus difﬁcult to achieve highaccuracy. Also, the way that the scaling factor appearing in CSRBFs needs to be chosen is not well understood.Most of the work to date on RBFs is related to interpolation of scattered data. To tackle largescale interpolationproblemsusingRBFs,variousfastﬁttingalgorithmshavebeendeveloped[2–5].Theseapproachesareveryeffectiveinthe reconstruction and representation of threedimensional objects. However, it is not clear how these fast algorithmsfor solving largescale interpolation problems can be extended to solving science and engineering problems whichinvolve solving a series of partial differential equations (PDEs). During the past decade, there has been an increasinginterest in applying RBFs for solving such problems involving the solution of PDEs [15,20]. Methods using RBFs are
meshless
and have therefore recently attracted much attention in the engineering community and RBFs have becomean important tool in scientiﬁc computing. Recently, preconditioning techniques [1,21] and domain decompositionmethods (DDM) [18,36] have been proposed to alleviate the conditioning and storage difﬁculties associated withsolving PDEs using RBFs when using a large number of degrees of freedom. One disadvantage of preconditioningand DDM techniques is the fact that their implementation is highly nontrivial.In general, when using RBFs interpolation schemes, randomly scattered points or uniform grid points are chosen.In the solution of PDEs using RBFs we have the freedom of selecting the collocation (interpolation) points. If thesecollocation points are chosen in a particular fashion, the resulting coefﬁcient matrix has a certain structure which canbe exploited for the efﬁcient solution of the corresponding system. For regularly spaced points, an RBF formulationcan lead to Toeplitz matrices, for which effective preconditioning techniques have been developed [1]. In this paper,by placing the interpolation points on concentric circles, we develop an efﬁcient matrix decomposition algorithm byexploiting the properties of circulant matrices. Recently, efﬁcient algorithms using the properties of circulant matriceshave been employed for solving homogeneous differential equations in the context of the Method of FundamentalSolutions (MFS) [12,13,31,32]. These algorithms make extensive use of Fast Fourier Transforms (FFTs). The basicidea of using FFTs for RBFs is not new as it has been used, albeit in a different context, by Jetter and Stöckler [19].Motivated by this simple and elegant circulant matrix approach, in this paper we extend these ideas for the reconstruction of function surfaces and their derivatives. The technique developed here will be later extended to solving a largeclass of partial differential equations in a subsequent paper. In this paper, we focus on the approximation of functionsand their derivatives in the twodimensional case.
2. Radial basis function interpolation
We consider the interpolation of a multivariate function
f
:
Ω
→
R
, where
Ω
⊂
R
2
, from a set of sample values
{
f(
x
j
)
}
N j
=
1
on a discrete set
X
={
x
j
}
N j
=
1
⊂
Ω
. Such multivariate functions can be efﬁciently reconstructed if theyare approximated by linear combinations of univariate interpolation functions with Euclidean norm
·
. This can beachieved by using translates
Φ(
x
−
x
j
)
of a single continuous real valued function
Φ
deﬁned on
R
, and by letting
Φ
be radially symmetric; i.e.,
Φ(
x
)
:=
ϕ
x
,
with a continuous function
ϕ
on
R
+
0
.
In the mathematical literature,
ϕ
is often called a radial basis function withcenters
{
x
j
}
N j
=
1
and
Φ
is the associated kernel.Interpolants
ˆ
f
N
to
f
can be constructed as
ˆ
f
N
(
x
)
=
N
j
=
1
a
j
ϕ
x
−
x
j
(2.1)with real coefﬁcients
{
a
j
}
N j
=
1
. The coefﬁcients on the righthand side of (2.1) can be determined by interpolation; i.e.,
ˆ
f
N
(
x
i
)
=
f(
x
i
),
1
i
N,
if the linear system
N
j
=
1
a
j
ϕ
x
i
−
x
j
=
f(
x
i
),
1
i
N,
(2.2)
306
A. Karageorghis et al. / Applied Numerical Mathematics 57 (2007) 304–319
is uniquely solvable. This is true if the symmetric
N
×
N
matrix
A
ϕ
=
ϕ(
x
1
−
x
1
) ... ϕ(
x
1
−
x
n
).........ϕ(
x
n
−
x
1
) ... ϕ(
x
n
−
x
n
)
(2.3)is nonsingular. In the RBF literature, it has been observed that, for certain choices of RBFs, (2.3) could be singular,for some conﬁgurations of interpolation points. As a result, a lot of interest arose in ﬁnding sufﬁcient conditions toensure the existence of
A
−
1
ϕ
[8]. Most of the globally deﬁned RBFs are only conditionally positive deﬁnite [25], thusnot guaranteeing the invertibility of the matrix
A
ϕ
. In order to guarantee the unique solvability of the interpolationproblem, one needs to add a polynomial term to the interpolant (2.1) giving
ˆ
f
N
(
x
)
=
N
j
=
1
a
j
ϕ
x
−
x
j
+
t
k
=
1
b
k
p
k
(
x
),
(2.4)along with the constraints
N
j
=
1
a
j
p
k
(
x
j
)
=
0
,
1
k
t,
(2.5)where
{
p
k
}
t k
=
1
is a basis for
P
m
−
1
, the set of polynomials in two variables of degree
m
−
1, and
t
=
m
+
12
is the dimension of
P
m
−
1
. Let
P
T
=
p
1
(
x
1
) ... p
1
(
x
N
).........p
t
(
x
1
) ... p
t
(
x
N
)
.
The interpolation conditions
f(
x
i
)
=
N
j
=
1
a
j
ϕ
x
i
−
x
j
+
t
k
=
1
b
k
p
k
(
x
i
),
1
i
N,
subject to (2.5) can be rewritten as the linear system
A
ϕ
P P
T
0
ab
=
f
0
,
(2.6)where
a
=[
a
1
,...,a
N
]
T
,
b
=[
b
1
,...,b
t
]
T
and
f
=[
f(
x
1
),...,f(
x
N
)
]
T
.For convenience, we often replace the argument of
ϕ
in (2.1) by
r
:=
x
−
x
j
. In Table 1, we present a list of popular choices of basis functions
ϕ
. In Table 1, polynomials up to degree
m
−
1 are required to guarantee the reconstruction of a given function. Despite the many attractive features of these RBFs, it is known that all the RBFsin Table 1 are globally supported. Thus, the direct solution of the system resulting from the interpolation equationsrequires
O
(N
3
)
operations and
O
(N
2
)
storage, and is thus impractical for large
N
. As stated earlier, in order to overcome these difﬁculties, CSRBFs were developed [6,29,33,34]. The most popular CSRBFs are the ones constructedby Wendland [33] as shown in Table 2 where the cutoff function
(r)
+
is deﬁned to be
r
if 0
r
1 and zero elsewhere. The application of CSRBFs leads to a sparse interpolation matrices which are positive deﬁnite and which canbe reconstructed without additional polynomial terms. Despite the effectiveness of CSRBFs, there are some issuesassociated with their slow rate of convergence, as mentioned earlier.Apart from the problems of storage and computer running time, RBF systems suffer from illconditioning whena large number of interpolation points is used. In general, the condition number is directly linked to the order of thebasis functions and density of the interpolation points. Schaback [30] has established the following
A. Karageorghis et al. / Applied Numerical Mathematics 57 (2007) 304–319
307Table 1Globally deﬁned radial basis functionsGaussian
ϕ(r)
=
e
−
cr
2
for
c >
0Inverse multiquadrics
ϕ(r)
=
(r
2
+
c
2
)
β/
2
,
c >
0
>β
Sobolev splines
ϕ(r)
=
K
ν
(r)r
ν
, where
K
ν
is the spherical Bessel function,
ν >
0Linear
ϕ(r)
=
r
,
m
=
1Cubic
ϕ(r)
=
r
3
,
m
=
2Polyharmonic splines
ϕ(r)
=
r
β
log
r
,
β
∈
2
Z
,
m>β/
2Polyharmonic splines
ϕ(r)
=
r
β
,
β
∈
R
>
0
\
2
Z
,
m>
[
β/
2
]
Multiquadrics
ϕ(r)
=
(r
2
+
c
2
)
β/
2
,
β
∈
R
>
0
\
2
Z
,
c >
0
,m>
[
β/
2
]
Table 2Wendland’s CSPDRBFs [33]
d
=
1
ϕ
=
(
1
−
r)
+
C
0
ϕ
=
(
1
−
r)
3
+
(
3
r
+
1
) C
2
ϕ
=
(
1
−
r)
5
+
(
8
r
2
+
5
r
+
1
) C
4
d
=
3
ϕ
=
(
1
−
r)
2
+
C
0
ϕ
=
(
1
−
r)
4
+
(
4
r
+
1
) C
2
ϕ
=
(
1
−
r)
6
+
(
35
r
2
+
18
r
+
3
) C
4
ϕ
=
(
1
−
r)
8
+
(
32
r
3
+
25
r
2
+
8
r
+
1
) C
6
Uncertainty Principle.
Either one goes for a small error and gets a bad sensitivity, or one wants a stable algorithmand has to take a comparably large error.The higher the order of the basis function, the worse the condition number, and the better the accuracy. If thenumber and positions of the interpolation points (data density) are ﬁxed, the Uncertainty Principle suggests that theorder of the basis functions used should be chosen with great care. This order should be as low as the applicationpermits, and any excessive order will have negative effects on stability. Furthermore, for low density interpolationpoints, one can use highorder basis functions, and for high density interpolation points, one can use loworder basisfunctions to avoid numerical problems.In the next section, we propose an algorithm which will enable us to solve (2.3) for large
N
.
3. Matrix decomposition algorithm
Our goal is to develop an efﬁcient algorithm for the solution of the system arising from equations (2.2). Usually,the collocation points are uniformly distributed in a region containing
Ω
. The resulting system (2.2) can be solvedusing a direct solver at a cost of
O
(N
3
)
operations.For the new approach, we assume that the given function to be approximated can be extended outside its domain
Ω
.We then place the collocation points on the boundaries of concentric disks deﬁned by
Ω
R
i
=
x
∈
R
2
:

x

<R
i
,
1
i
m,
(3.1)where
R
1
<R
2
<
···
<R
m
and
Ω
⊂
Ω
R
m
.On the circles
∂Ω
R
i
, 1
i
m
, we deﬁne the
mn
collocation points
x
i,j
={
(x
i,j
,y
i,j
)
}
m,ni
=
1
,j
=
1
by
x
i,j
=
R
i
cos
2
(j
−
1
)πn
+
2
α
i
πn
, y
i,j
=
R
i
sin
2
(j
−
1
)πn
+
2
α
i
πn
, j
=
1
,...,n,
where the position of the points on the circle of radius
R
i
is shifted by an angle 2
α
i
π/n
with 0
α
i
1. For
α
i
=
0,we have a rotation for each circle. To ensure a uniform distribution of the collocation points, we choose the radii of the concentric circles as follows
R
i
=
imr
max
, i
=
1
,...,m,
308
A. Karageorghis et al. / Applied Numerical Mathematics 57 (2007) 304–319
Fig. 1. Typical distribution of circulant collocation points.
where
r
max
is the radius of the largest circle. In Fig. 1 we present a typical distribution of collocation points with
n
=
20,
m
=
15,
r
max
=
1, and
α
i
=
0 for
i
=
1
,
3
,...,
15,
α
i
=
0
.
5, for
i
=
2
,
4
,...,
14.The collocation equations (2.2) yield a system of the form
A
ϕ
a
=
f
,
(3.2)where the
mn
×
mn
matrix
A
ϕ
, has the structure
A
ϕ
=
A
11
A
12
... A
1
,m
A
21
A
22
... A
2
,m
............A
m
1
A
m
2
... A
m,m
(3.3)where each of the
n
×
n
submatrices
A
k,ℓ
is
circulant
, and
A
k,ℓ
=
circ
ϕ
x
k,
1
−
x
ℓ,
1
,ϕ
x
k,
1
−
x
ℓ,
2
,...,ϕ
x
k,
1
−
x
ℓ,n
, k,ℓ
=
1
,...,m.
Also, we have
f
(i
−
1
)n
+
j
=
f(
x
i,j
), i
=
1
,...,m, j
=
1
,...,n.
We shall exploit the property that these submatrices
A
k,ℓ
are circulant by using the fact that circulant matricesare diagonalized in the following way. If
C
=
circ
(c
1
,...,c
n
)
, then
C
=
U
∗
DU
where
D
=
diag
(d
1
,...,d
n
)
,
d
j
=
nk
=
1
c
k
ω
(k
−
1
)(j
−
1
)
and the
n
×
n
matrix
U
is the Fourier matrix which is the conjugate of the matrix (see[9,31])
U
∗
=
1
n
1
/
2
1 1 1
...
11
ω ω
2
... ω
n
−
1
1
ω
2
ω
4
... ω
2
(n
−
1
)
............
1
ω
n
−
1
ω
2
(n
−
1
)
... ω
(n
−
1
)(n
−
1
)
,