504
zyxwvutsrqpo
EEE
zyxwvuts
RANSACTIONS
ON
SPEECH
zyxwv
ND
AUDIO
PROCESSING,
VOL.
3,
NO
6,
NOVEMBER
1995
alysis
of
the
Filtere
X
LMS
Algorit
Elias
Bjamason,
zyxwvu
ssociate
Member,
zyxwv
EEE
AbstractThe presence of a transfer function
zyxwvu
n
the
auxiliary
path following the adaptive filter and/or in the errorpath,
as
in the case of active noise control, has been shown to generally degrade the performance of the
LMS
algorithm.
Thus,
he con vergence rate
is
lowered, the residual power
is
increased, and the algorithm can even become unstable. To ensure convergene of the allgorithm, the input to the error correlator
has
to
be
filtered by a copy of the auxiliaryerrorpath transfer
fundon.
This paper presents an analysis of the FilteredX
LMS
zyxwvut
lgorithm
using stochastic methods. The influence
of
offline
and
online
estimation of the errorpath filter on the algorithm
is
also
nves tigated. Some derived bounds and predicted dynamic behavior are found to correspond very well to simulation
results.
I. INTRODUCTION
HE active control of sound and vibration involves the
zyxwv
n
troduction of a number of controlled “secondary” sources driven such that the field generated by these sources interferes destructively with the field caused by the srcinal “primary” source. The extent
to
which such destructive interference
is
possible depends on the geometric arrangement
zyxwvut
f
the primay and secondary sources and their environment, and on the spectrum of the field produced by the primary source
[SI.
Due to the timevariance of the acoustic path the system
has
to be adaptive. Fig. 1 shows a simple active noise cancellation system without feedback, using a loudspeaker
as
a
secondary source. Here,
zyxwvutsrq
describes the adaptive controller and
I
the interpolation filter used to reconstruct the estimated signd
y(t).
In
Fig. 2(a) a model of the active cancellation system is shown. Here,
h,
describes the system in the auxiliarypath, modeling antialiasing filter, loudspeaker, and acoustic path from loudspeaker to microphone. The system in the errorpath
he
models the path common to desired and estimated
signd,
including error microphone and antialiasing filter. Because of the system in the auxiliarypath,
w
has to include the inverse system of
h,
Wi z)/H, z)
s the system to be identified). Measurements show that
h,
can
be
a nonminimum phase system. This results in making
w
noncausal, and therefore, degrading the cancellation effect. The drawback
of
the noncausal parts
is
decreased due to the inherent delay in acoustic system. The filter
he
does not affect
the
required
model
wo,
but it does affect the behavior of the adaptive algorithm as does
h,.
primary
source
(noise) secondary source
Fig. 1. Active noise cancellahon system without feedback (b)
Fig.
2.
(a)
h,
can
be
moved through the summing junction
to
form an equivalent system.
(b)
h,
and
he
can be combined to form
a
new errorpath system €unction
h.
Ln
she same way,
wb
and the inverse system
of
the auxiliary
path
hgl
can
be
combined to
wo
which
is
the system to be identified.
the
LMS
algorithm. Thus, the convergence rate is lowered, the residual power is increased, and the algorithm can even become unstable.
In
order to stabilize the algorithm a filter identical to the filter
h
in the auxiliaryerrorpath (the com bined
auxiliary
and errorpath filter will hereafter simply be referred
to
as
errorpath filter) is used to filter the reference input
a
(n)
[24],
[16]
(see Fig.
1)
to the algorithm. The recursive relation for updating the tapweight vector in this case
is
(1)
n
+
1)
=
pu n)
+
PEf(++)
where
gf
and
.ru
are vectors
of
dimension
N,
and
uf(n)
=
~(n)
h, he
=
u(n)
*
h,
11.
THE
FILTEREDX
LMS
&GO“
y(n)
=
zoT(n) &(n),
The existence
of
a filter in the auxiliary and/or the error
Yf(n)
=
Y(n)
*
hz
*
he
=
Y(n)
*h,
path has been shown to generally degrade the performance
of
df n)
=
zyx
‘ n)
*he
=
d(n)
*
h,
Manuscnpt received July 18, 1993; revised February 21, 1995.
=
ddn)

Yf(4
+
4n).
The author was with the hstitut
fur
Netzwerk und Signdtheorie, Technis che Hochschule Darmstadt, Merckstr. 25, 64283 Darmstadt, Germany. He is now with Rockwell, Vesturbrun 16, 1s107 Reykjavik, Iceland.
Normalization
of
LMStype algorithms has been shown to greatly improve their convergence properties for colored
IEEE
Log
Number 9414956.
inputs, as well as making the algorithms independent of
10636676/95 04.00
0
1995
IEEE
BJARNASON ANALYSIS OF
THE FILTEREDX
LMS ALGORITHM
zyxwvutsr
05
zy
the energy of the input signal. In the case of the FXLMS algorithm, a normalization rule can be given
(2)
zyxwv
f
zyxwvutsr
n)gb(n)
p=
zyxwvutsrqpo
T
Stochastic Analysis
In the early
1960 s
Widrow and Hoff
[23]
introduced the LMS algorithm. They showed that with a stationary stochas tical input the expected value of the weight vector converged to the optimal Wiener solution. Starting with the firstorder moments by Gersho, in
1969 [13],
and with the secondorder moments by Ungerbock, in
1974 [22],
the analysis of the LMS algorithm for a stochastic input has been a long and slow moving process. The real breakthrough was the introduction of the independence theory for an independent, identically distributed (IID) whiteinput process by Gardner, in
1984 [12].
In
1985,
Feuer and Weinstein
[l
11
carried out the calculation of the covariance matrix of the weighterror vector for a Gaussian input. Nitzberg
[17]
and
[18],
and later, Bershad
[2],
extended the analysis on the NLMS algorithm assuming a Gaussian input. Further extension of the analysis to a large group of input processes called spherically invariant random processes (SIRP) was done by Rupp
[19],
in
1993.
In the following, the classical stochastical analysis methods will be introduced and applied to the FXLMS algorithm. In
1981,
Widrow
et
al.
[24]
introduced the FXLMS algo rithm, and showed that with a stationary stochastical input the expected value of the weight vector converges to the Wiener solution. This can be shown by minimizing the expected value of the squared filtered error
~[e;(n)]
)
min.
(3)
leading to the Wiener solution
RUfllfWO
=
T(df+e?,f)Uf'
(4)
Here,
zyxwvutsrqp
df
eqf)q
is the expected value of the product of the filtered input vector
gf(n)
and the filtered output
h
(d(n)
+
e,(n))
=
df n)
+
e,f(n),
with
d(n)
being that part of the error that is predictable while the disturbance term
e,(n)
is the part of the error
e(.)
that is not predictable by the adaptive filter. The firstorder moment of
e,(n)
is assumed zero and the secondorder moment is called the minimum meansquared error of the Wiener solution
E[e,(n)l
=
0,
(5)
E[e:
zyxwvuts
=
Jmin.
(6)
This can be done by minimizing the covariance matrix
E
[g(n

)gT(n
)]
of
the weighterror vector. It will be assumed that the input signals belong to a certain class of processes called SIRP's. For an introduction to SIRP's
[7]
and
[21]
are a good reading. The firstorder moments of the processes are all assumed to be zero. Furthermore, it is assumed that the correlated random process
~(n)
s a linear transformation
A
of an uncorrelated process
~(n)
f the same order
N
The autocorrelation matrix (ACF) matrix
&
can then be obtained Let
Q
be an unitary orthogonal matrix that diagonalizes
&.
Then the
unitary similarity transformation
is given by The elements of the diagonal matrix
A
are the eigenvalues of the ACF matrix. Furthermore,
QT
can also diagonalize
z

It can be shown for SIRP's that the elements of the vector
~(n)
re statistically independent with respect to each other, with the usage of spherical coordinates.
A
further implication of this, due to the symmetry of the joint density function, is that all moments containing at least one uneven order equal zero. A further important element of the analysis is the independence theory. This allows one to assume that the vectors
g(i)
and
g j)
are independent of each other for
zy
#
j.
This assumption was introduced by Gersho, in
1969 [13]
and later shown to be true for certain conditions by Mazo, in
1979 [15].
Clearly, there are many practical problems for which the independence theory is not satisfied. Nevertheless, experience with the LMS algorithm has shown that results gained by the application of the independence theory retain sufficient information about the structure of the adaptive process to serve as reliable design guidelines, even for some problems with highly dependent data samples
[l].
For convenience, the algorithm for updating the weights of Furthermore, it is assumed that the disturbance term
e,(n)
is uncorrelated with the input process
u(n).
Also, it is assumed the FXLMS adaptive filter is given again that
e,(n)
is a white process. Modeling the errorpath with a transversal filter
h
of order
Mh,
the meansquare of the
filtered
zyxwvutsr
rror
can
be calculated
~[e;(n)]
=E
f(tr(R,,E[z(n

)gT(n

]
+
Jmin).
(7)
This relation is derived in Appendix Al. In order to minimize
E[e?(n)],
he trace term has to be minimized.
Mh1
a O
(14) (15)
506
zyxwvutsrqponmlkj
EEE
TRANSACTIONS
ON
SPEECH
zyxwv
ND
zyxwv
UDIO
PROCESSING,
VOL
3,
NO
6,
NOVEMBER
1995
FirstOrderMoments:
Building a weighterror vector for
zyxwvu
zyxwv
>
zyxw
)
and a simple delay
D
with a weight
hD
in the
z
(n)
=
~(n)
zoo
at time
n
and modeling the errorpa& error path the stability bounds
for
the stepsize are given by with a transversal filter
h
of order
hfh
the following vector equation results
Mh1 Mh1
c(n
+
1)
=
c(n)

P
hZh&
j)
z=o
j=o
T(,

)c(n

zy
Mh1 Mh1
+P
hah,E(n
)eo n

).
z=o
3=0
(14)
Using the independence theory [13] and 1151 and the ex pectation operator leads
to
The term
E[gf
n)eof
TI)]
is neglected since its expectation
is
zero. Diagonalizing
Ruu
with the unitary matrix using the abbreviation
y(n)
=

'E[g(n>],
he following equation is obtained Normalizing results in exactly the same bounds that are calculated
in161
for a sinusoidal input at
R
=
0
and
T
0
<
Q:
<
2sin (2(2;+
1)).
There, it
is
further shown that these bounds are the minimum
stability
bounds for
a
sinusoidal input. The stability of the firstorder moments
is,
therefore, a sufficient requirement for a sinusoidal input.
In
general, this is not the case. Thus, convergence of the secondorder moments of the weighterror vector or the error signal
is
desired.
SecondOrder
Moments:
When investigating the behavior
of
the secondorder moments, the system mismatch
S(n)
=
E[gT(n)g(n)]
ives a good insight into the properties
of
the algorithm. For the study
of
S(n)
the trace of the covariance matrix
&,(n)
=
E[4n)gT(n)]
as to be evaluated. The mean squared error
E[e2(n)]
an also be derived from the covariance matrix by using the identity
E[e2(n)]
tr(KOO(n)&,,)
+
Jmin.
For the covariance matrix the following expression is obtained in
(22)
at the bottom of this page. Using the abbreviation
(n)
=
E[g(
where
Go
n)
epresents the covariance m error vector, leads to For each element of the vector
g(n)
a characteristic equation exists
Mh1 h'h1Mh1Mh1Mh1
2

1
+
pxk
h:ZFz
=
0
;for
k
=
1..N.
(19)
P2
hahjhkhl
2=0
i=O
j=o
k=O
kO
To
ensure stability of the firstorder moments, the roots Therefore, attention
is
paid
to
the roots due to
pAmax,
where
.
E[g(n

)gT(n )gzl(n)g(n
E)gT(n

k)]
of the characteristic equation have to remain inside the
zyxwv
nit
Mh1 Mh1
circle. Increasing
PAk
moves the roots out
of
the unit circle.
+
P JA,
h?h;BUU.
(23)
z=O
j=o
A
is
the largest
of
the eigenvalues
A,
of the ACF matrix
&
For a sinusoidal input of an amplitude
A
Amax
=
A2/2
Again, let
A
be the matrix that transforms the uncorrelated process
~(n)
o a correlated process
g(n)
and be the orthog
BJAFNASON:
ANALYSIS
zyxwvutsrqponml
F THE
FILTEREDX
zyxwvutsrq
MS
ALGORITHM
zyxwvutsrq
07
onal matrix that diagonalizes
zyxwvutsr
Then
E
=
zyxwv
;UT
and

TBUuQ
A.
By using these identities and the abbreviation

TKij(n)Q
=
zyxwvut
tJ(n)
he following presentation of (23) is obtained in(24) and (25) at the bottom of this page. Considering the term with
z(n)
=
h1/2Q21 n)h1/2
ith respect to spherically invariant processes
[
191, defining as the joint fourth order moment of the input process results in
E[g(n
)gT(n

)Z(n)g(n

Z)gT(n

zyx
)lrS
=
(26)
mi2
2)
zyxwvutsrqpo
n)
I
j
=
i
#
=
k,
mi2’2)ZsT(n); j
=
1
fa
=
k,
mi2’2
E
mm(n);
0;
for
r
=
s,
for
r
#
s,
j
=
k
#
i
=
zyxw
else.
;
Because all of the main diagonal elements of
cij(n
1)
depend only on the main diagonal elements of
Ckz(n)
nd due to the invariance
of
the trace to diagonalization, only the elements of the main diagonal of
Coo(n)
re of interest in calculating the system mismatch
S n).
By using the result of (28) and the abbreviation
i2’2)
mk2’2)/a:
he following simpler representation for the diagonal elements of
C,,(n
+
1)
and
Cok(n
1)
is
obtained
Mh1
coo(n
+
1)
=
Goo 4

2P
h?hcOi(4
2=0
MI.
1
i=O
j>i
ML
1
Mh
1
i=o
j=o
i=O
The vector is filled by the eigenvalues from the diagonal matrix
A.
Delayed
LMS
Algorithm:
For the special case of a simple delay
D
in the error path with weight
hD,
(29)
and
(30) can be reduced to Equation (33) can be rewritten to (see Appendix A2) The function fDLMs depends on the delay
D
and the characteristics of the input signal such as eigenvalue ratio
z
etc. For a stable algorithm the maximum possible value of the sum in (34) is
Dc,,(n

D
+
1).
Therefore, the following inequality holds The higher the eigenvalue ratio the slower the convergence. Correspondingly the value of fDLMS increases. Substituting the results of (34) into (31) and using the
ztransformation
and the matrix inversion lemma results in the following equation for the secondorder moments (see Appendix A3)
508
zyxwvutsrqpon
EE
zyxwvutsrqpon
RANSACTIONS
ON
SPEECH
zyxwvu
ND
AUDIO
PROCESSWG,
VOL.
3,
NO.
6,
NOVEMBER
1995
Fig.
3.
Active system without feedback,
using
the
F2teredX
zyxwvuts
XS
algorithm.
where The entries of
B(z)
are where The elements of the matrix
D(z)
are defined with
(40)
at The characteristic equation of the discretetime system the bottom of this page. described with
(37)
can be split into two polynomials
N1
1

p2h$Gpi2)zD
X:dZz
=
0,
(41)
dil
=
0.
(42)
zyxwv
=0
For a white process with identical eigenvalues, i.e.,&
=
X
(for
zyxwvutsrq
=
1..N),
the characteristic equation can be reduced to zD+l

D
+
2ph3

p2
h4,X2(6p(N
+
2)
+
~~DLMS)
0,
143)
+
2
2p
4
DX
zyxwvutsrq
(6p’2)
fDLMS)
=
0.
(44) zDtl

Dph&X
1
0.5
0
0.5

I

I
1.
ii 5
i
zy
015
zyxw
0’5
i S
Re(z.1
Fig.
4.
Root loci
of
the
Characteristic equatton for
a
whtteinput process
wiW
=
10,X
=
1,a:
=
m?”)
=
1
and
D
=
8.
The convergence behavior of the
DLMS
algorithm depends
on
the
roots of
the characteristic equation. For
N
2
2 (43)
is
the dominant one. Fig. 4 depicts the root locus of the characteristic equation for
D
=
8.
It can be seen that
D
roots
start in
&e srcin. With increasing
p
they move radially away
from
the srcin under an angle of
27rilD;
i
=
0..D

1.
The initial root at the point
1,O)
moves inwards along the positive real
axis
in
the
direction of the srcin. This root dominates the convergence and the stability of the algorithm.
As
p
increases further, the negative square terms in
p
become dominant. The movement
of the
roots
is
reversed and they move back on the sane locus
as
before. When the negative and positive
parts
in
,U
of
the
chaxacteristic equation cancel each other, the dominant root retums
to
the point
(1,
0)
and the other
D
roots return to the srcin. Increasing
p
further would migrate the dominant
root
outside the unit circle making the algorithm unstable. The
stability
condition for the algorithm can then be given by the following expression: From Fig. 5 it can be observed that the magnitude of the
dominant
root
has
a
minimum
in
the
middle
of
the
stable
interval.
This
indicates that the convergence will be fastest if the stepsize equals half of the upper bound. This can
also
be shown because
this
is exactly the point were the
Oth
order term
in
the characteristic equation reaches its highest positive value.