A NOVEL FRAMEWORK FOR ACCURATE LUNG SEGMENTATION USING GRAPH CUTS
Asem M. Ali
1
Ayman S. ElBaz
2
Aly A. Farag
11
Computer Vision and Image Processing Laboratory (CVIP), University of Louisville.
2
Bioengineering Department, University of Louisville
∗
ABSTRACT
The closeness of the gray levels between lung tissues and the chesttissues makes lung segmentation based only on image signals difﬁcult. This work proposes an automatic segmentation framework of lung consisting of three stages. First is the image signal modelling, and initial labelling stage. The labelling algorithm is basedon a probabilistic model, which models the image signal with a linear combination of Gaussian distributions with positive and negativecomponents. In the second stage a new analytically estimated potentials for Potts model parameter is used to identify the spatial interaction between the neighboring pixels. Finally the third stage, wherean energy function using the previous models is formulated, and isglobally minimized using
s/t
graph cuts. Experiments show that thedeveloped technique segments CT lung images more accurately thanother known algorithms.
1. INTRODUCTION
Due to the fast and high technology progress in high resolution Xraycomputed tomography (CT) scanners, we have multislice spiral CTscanners. This encourages many studies of CT screening for lungcancer to appear. Lung segmentation is a vital step in many applications such as detection and quantiﬁcation of interstitial disease, andthe detection and/or characterization of lung cancer nodules. Formore details, see Sluimer’s et al. survey paper [1]. But CT lungdensity depends on many factors such as image acquisition protocol,subject tissue volume, volume air, and physical material properties of the lung parenchyma. These factors make lung segmentation basedon threshold technique difﬁcult. So developing new accurate algorithms with no human interaction, which depends on gray level difference between lung and its background, to precisely segment thelung is important.In the literature, there are many techniques were developed forlung segmentation in CT images. Hu et al. [2], proposed an optimal gray level thresholding technique which is used to select athreshold value based on the unique characteristics of the data set.A segmentationbyregistration scheme was proposed by Sluimer etal. [3] for automated segmentation of the pathological lung in CT.In this scheme a scan with normal lungs is registered to a scan containing pathology. When the resulting transformation is applied toa mask of the normal lungs, a segmentation is found for the pathological lungs. Our proposed framework for lung segmentation usesgraph cuts as a powerful optimization technique to get the optimalsegmentation. Greig et al. [4] discovered the power of graph cuts algorithms from combinatorial optimization, and showed that graphcuts can be used for binary image restoration. The problem was
∗
Acknowledgement: This work is supported by the Kentucky Lung Cancer Program. Corresponding author: Aly Farag –
farag@cairo.spd.louisville.edu
formulated as Maximum A Posterior estimation of a Markov Random Field (MAPMRF). Boykovand Jolly [5] proposedaframework that used
s/t
graph cuts to get a globally optimal object extractionmethod for Ndimensional images. They minimized cost function,which combined region and boundary properties of segments as wellas topological constraints. That work illustrated the effectiveness of formulating the object segmentation problem via graph cuts.Many publications extended Boykov and Jolly [5] technique indifferent directions. Blake et al. [6] used a mixture of Markov Gibbsrandom ﬁeld (MGRF) to approximate the regional properties of segments and the spatial interaction between segments. Geocuts combines geometric cues with energy function. Objcuts integrates highlevel contextual information. Multilevel and banded methods wereproposed in many publications. For more details and references see[7]. These works showed the power of graph cuts as a tool for image segmentation; since it optimizes energy functions that can integrate regions, boundary, and shape information. Also the
s/t
graphcuts technique offers a reliable and robust globally optimal objectsegmentation method. Most of these works are interactive segmentation. Although interactive segmentation imposes some topologicalconstraints reﬂecting certain highlevel contextual information aboutthe object, it depends on the user input. The user inputs haveto be accurately positioned. Otherwise the segmentation results are changed.This paper proposes an automatic lung segmentation approachwhere the lung tissues are separated from the surrounding anatomical structures appearing in the chest CT scans. Due to the closenessof the gray levels between lung tissues and the chest tissues, the proposed approach does not depend only on image signals, but also ittakes into account the spatial relationships between the region labelsin order to preserve the details. To model the low level information inthe CT lung image, the gray level distribution of CT data is modelledusing a new probabilistic model. This model is based on modellingthe gray level with a linear combination of Gaussian distributionswith positive and negative components. The spatial interaction between the neighboring pixels is modelled using Potts model with anew analytically estimated potentials. We use these models to formulate an energy function, which is globally minimized using
s/t
graph cuts.
2. THE PROPOSED FRAMEWORK
Our segmentation framework of the lung consists of three stages.First is the initial labelling stage, shown in Fig.1. The labelling algorithm is based on a probabilistic model, which models the gray levelwith a linear combination of Gaussian (LCG) distributions with positive and negative components. Details of this probabilistic modelare presented in [8]. This work focuses only on the second and thirdstages. In Sec. 2.1 the graph construction and energy formulationare described. Sec. 3 gives the details of the analytical estimation of potentials for Potts model parameter.
9081424406722/07/$20.00 ©2007 IEEE ISBI 2007
(a) (b)(c) (d)
Fig.1
. Summery of LCG Model. (a) LCG approximation of the lungandchesttissues, (b)ThecomponetsoftheLCGmodel, (c)TheLCGmodels of each class with the best separation threshold
t
= 109
, and(d) Segmentation results using the threshold.
2.1. Graph Cuts Approach
The weighted undirected graph
G
=
V
,
E
is a set of vertices
V
,and a set of edges
E
connecting the vertices. Each edge is assigneda nonnegative weight. The set of vertices
V
corresponds to the set of image pixels
P
, and two specially terminal vertices
s
(source/object),and
t
(sink/background). The set of edges
E
consists of two subsets.The ﬁrst subset is the edges connecting the neighboring pixels in theimage, and these are called
nlinks
, and the other subset is the edgesconnecting the pixels with the terminals , and these are called
tlinks
.An
s/t
cut is a set of edges
C ⊂ E
such that terminals are separatedin the induced graph
G
(
C
) =
V
,
E − C
. The cut divides the setof image pixels into two subsets, background and object. The sumof weights of edges, which belong to cut, is the cut cost
C
. MinCut/MaxFlow algorithms in combinatorial optimization show that aglobally minimum
s/t
cut can be computed efﬁciently in loworderpolynomial time [9]. Boykov and Kolmogorov, in [10] described a
new maxﬂow algorithm that signiﬁcantly outperformed the standardtechniques. In this work we use this algorithm to ﬁnd the minimumcut, which corresponds to optimal segmentation, among all the cutsin the graph.
2.2. Segmentation Energy and Optimal Solution
Consider a neighborhood system in
P
, which is represented by a set
N
of all unordered pairs
{
p,q
}
of neighboring pixels in
P
. Let
L
the set of labels
{
“1”, “2”
}
, correspond to lung and backgroundrespectively. Labelling is a mapping from
P
to
L
, and we denote theset of labelling by
f
=
{
f
1
,...,f
p
,...,f
P
}
. In other words, thelabel
f
p
, which is assigned to the pixel
p
∈ P
, classiﬁes it to lungor background. Now the goal is to ﬁnd the best labelling
f
, optimalsegmentation, by minimizing the following function
E
(
f
) =
p
∈P
D
p
(
f
p
) +
{
p,q
}∈N
V
(
f
p
,f
q
)
(1)where
D
p
(
f
p
)
, measures how much assigning a label
f
p
to pixel
p
disagrees with the pixel intensity,
I
p
. A good example for
D
p
(
f
p
)
Fig. 2
. Example of undirected Graph: Image’s pixels (ai) are thegraph’s nodes. nlinks is constructed for 4neighborhood system.Source node represents object and sink represents background.represents the regional properties of segments [4, 7]
D
p
(
f
p
) =
−
ln P
(
I
p

f
p
)
(2)To compute this regional term, we use a new probabilistic model toestimate the probabilities of the object and background. This probabilistic model models the gray level with a linear combination of Gaussian distributions with positive and negative components. Thesecond term is the Potts energy, which represents the penalty for adiscontinuity between pixels
p
and
q
. Since discontinuity happens atthe boundary, [7] denoted this term by boundary term. The potentialfunction,
V
(
.,.
)
, is deﬁned as follow
V
(
f
p
,f
q
) =
γ
if
f
p
=
f
q
;
0
if
f
p
=
f
q
(3)In this work we use a new analytical approach to estimate the spatial interaction between the neighboring pixels,
γ
, in Potts model, asshown in Sec. 3.For image segmentation we construct the graph,
G
=
V
,
E
asfollows: Each image pixel,
p
∈ P
represents a vertex in the graph.The two terminals of the graph represent the lung (
s
\
“1”
) , and thebackground(
t
\
“2”
). Eachimagepixel,
p
∈ P
hastwotlinks,
{
s,p
}
,and
{
p,t
}
. And four, three, or two nlinks,denoted by
N
. Thenumber of nlinks for each pixel depends on the position of this pixelin the image lattice. The tlinks weights:
{
s,p
}
is
−
ln P
(
I
p

“2”)
,and
{
p,t
}
is
−
ln P
(
I
p

“1”)
. Each nlink,
{
p,q
} ∈ N
has theweight
γ
if
f
p
=
f
q
or 0 otherwise.An example for the graph that we used in image segmentation isshown in Fig.2. After we constructed the graph
G
, and deﬁned theweight of each edge in
E
, we compute the minimum cut
C
on
G
inpolynomial time via
s/t
MinCut/MaxFlow algorithm described in[10]. [7] proved that the min
s/t
cut of a similar graph correspondsto the optimal segmentation of the image.
3. SPATIAL INTERACTION MODEL
To model the spatial interaction between neighboring pixels, we usetheMarkovGibbsrandomﬁeld(MGRF).ToconstructaMGRFmodelfor the observed image, the image labelling
f
=
{
f
1
,...,f
P
}
ispresented as realizations of random variables, and the probabilitymeasure representing the joint distribution of all pixel labels on animage grid is called a random ﬁeld
F
=
{
F
1
,...,F
P
}
.
P
(
f
)
is theprobability of a particular labelling or conﬁguration
f
∈ F
, where
F
is the set of all possible labelling.
F
is a Markov random ﬁeld (MRF)with respect to a neighborhood system
N
if
P
(
f
p

f
{P−{
p
}}
) =
909
P
(
f
p

f
{
N
p
}
)
.
F
is a MGRF if and only if
P
(
F
=
f
)
>
0
, for all
f
∈ F
, and therefore
P
(
f
)
is a Gibbs Probability distribution
P
(
f
) = 1
Z
N
e
E
(
f
)
(4)where
Z
N
is a normalizing constant called the partition function and
E
(
f
)
is the energy function. The selection of
E
(
f
)
determines thetype of Gibbs Markov random ﬁled model.Let
V
=
{
V
(
l
1
,l
2
) = 0
if
l
1
=
l
2
and
V
(
l
1
,l
2
) =
γ
if
l
1
=
l
2
:
l
1
,l
2
∈ L}
denote the Gibbs potential governing symmetric pairwisecooccurrences of the region labels. Then the MGRF model of regionmapsisspeciﬁedbythefollowingGibbsprobabilitydistribution(GPD) where
Z
N
is the normalizing factor (the partition function):
P
(
f
) =
1
Z
N
exp
{
p,q
}∈N
V
(
f
p
,f
q
)
=
1
Z
N
exp(
γ

T
N

(2
f
neq
(
f
)
−
1))
(5)Here,
T
N
=
{{
p,q
}
:
p,q
∈ P
;
{
p,q
} ∈ N}
is the family of theneighboring pixel pairs supporting the Gibbs potentials,

T
N

is thecardinality of that family, and
f
neq
(
f
)
denotes the relative frequencyof the not equal labels in the pixel pairs of that family:
f
neq
(
f
) = 1

T
N

{
p,q
}∈
T
N
T
(
f
p
=
f
q
)
(6)where, the indicator function,
T
(
A
)
equals 1 when the condition
A
is true, and zero otherwise. To identify the Potts model that describethe label image
f
, we have to estimate only the potential value
γ
.
3.1. The proposed approach for Parameter Estimation
To estimate the model parameter
γ
that speciﬁes the Gibbs potential,we identify the MGRF model using a reasonably close ﬁrst approximation of the Maximum Likelihood Estimation (MLE) of
γ
. It isderived in accord with [11] from the unconditional loglikelihood
L
(
f

γ
) = 1
P
log
P
(
f
)
(7)To compute,
1
P
log
P
(
f
)
, in Eq. (7) we use the approximate partition function
Z
N
given in [12]. It is reduced in our case as follows:
Z
N
=
f
∈F
exp(
γ

T
N

(2
f
neq
(
f
)
−
1))
(8)Then the loglikelihood of Eq. (7) can be written in as follows:
L
(
f

γ
) =
γρ
(2
f
neq
(
f
)
−
1)
−
1
P
log
f
∈F
exp(
γ

T
N

(2
f
neq
(
f
)
−
1))
(9)
where
ρ
=

T
N
P
. The approximation is obtained by truncating theTaylor’sseriesexpansionof
L
(
f

γ
)
totheﬁrstthreetermsintheclosevicinity of the zero potential,
γ
= 0
:
L
(
f

γ
)
≈
L
(
f

0) +
γ dL
(
f

γ
)
dγ
γ
=0
+ 12
γ
2
d
2
L
(
f

γ
)
dγ
2
γ
=0
(10)Because zero potential produces an independent random ﬁeld(IRF) of equiprobable region labels
l
∈ L
, the relative frequencyof the not equal pairs of labels over
T
N
has in this case the mean(
a
)
0
.
26%
(
b
)
7
.
44%
(
c
)
5
.
07%
Fig. 3
. Phantom Segmentation Results of: (a) proposed algorithm ,(b) ICM , (c) IT. (The misclassiﬁed pixels are shown in red color)value
1
K
and the variance
K
−
1
K
2
, where
K
is number of labels. Sincethe Gibbs distribution is an exponential family, so the ﬁrst derivativeis the expected value, and the second derivative is the variance, so itis easy to show:
dL
(
f

γ
)
dγ
γ
=0
= 2
ρ
f
neq
(
f
)
−
1
K
;
d
2
L
(
f

γ
)
dγ
2
γ
=0
=
−
4
ρ
K
−
1
K
2
where
f
neq
(
f
)
is the relative frequency of the not equal label pairs inthe region map
f
speciﬁed in Eq. (6). The approximate likelihood of Eq. (10) results in the following MLE of
γ
:
γ
=
K
2
K
−
1
f
neq
(
f
)
−
1
K
(11)ThisrelationshipallowsforcomputingthepotentialsoftheMGRFmodel for any label image
f
.
4. EXPERIMENTS AND DISCUSSION
In order to measure the accuracy of our approach, we have created ageometric phantom with the same gray level distribution in regionsas in the lung CT images at hand using the inverse mapping approach[8]. The error 0.26% between our results and ground truth conﬁrms the high accuracy of the proposed segmentation framework.For comparison, Fig. 3 shows the binary results obtained with ourtechnique, Iterative Threshold (IT) [2] approach, and Iterative Conditional Modes (ICM) [13] technique. Also the proposed , ICM, andIT approaches are run on 60 axial human chest slices obtained byspiralscan lowdose computer tomography (LDCT), the 8mmthick LDCT slices were reconstructed every 4 mm with the scanning pitchof 1.5 mm. Due to space limitation the results of seven of them areshown in Fig.4. In this experiment, the errors are evaluated with respect to the ground truth produced by an expert (a radiologist). Thepercentage error of the misclassiﬁed pixels is shown for each approach. In contrast to IT approach, our segmentation does not loseabnormal lung tissues, and bronchi as well as bronchioles. Although,for some results, the error from ICM technique is higher than errorfromITapproach, themisclassiﬁcationinICMislessdangerousthanIT. The majority of the misclassiﬁed pixels are located at the boundary in ICM results. But the misclassiﬁed pixels in IT results loseabnormal lung tissues, bronchi and bronchioles. These tissues areimportant if lung segmentation is a prestep in a detection of lungnodules system. To highlight the advantage of the optimization using the graph cuts than ICM or IT, we calculate the energies, Eq.(1),of 15 segmented images of the previous three approaches. As shownin Fig. 5 our proposed approach outperforms others and gives minimum, optimal segmentation, energy for all segmented images.
5. CONCLUSION
In this paper, we have presented a novel framework for automaticlung segmentation using the graph cuts approach. First, the graylevel distribution of CT data is modelled using a new probabilisticmodel. This model is based on modelling the gray level with a linear combination of Gaussian distributions with positive and negative
910
(
a
1
)
0
.
88%
(
a
2
)
2
.
44%
(
a
3
)
2
.
29%
(
b
1
)
2
.
66%
(
b
2
)
6
.
03%
(
b
3
)
7
.
29%
(
c
1
)
1
.
5%
(
c
2
)
8
.
29%
(
c
3
)
2
.
54%
(
d
1
)
1
.
56%
(
d
2
)
7
.
95%
(
d
3
)
2
.
27%
(
e
1
)
1
.
74%
(
e
2
)
2
.
61%
(
e
3
)
2
.
82%
(
f
1
)
4
.
36%
(
f
2
)
8%
(
f
3
)
8
.
52%
(
g
1
)
1
.
63%
(
g
2
)
7
.
55%
(
g
3
)
5
.
61%
Fig. 4
. Segmentation Results of: proposed algorithm (
1
st
column) ,ICM (
2
nd
column), and IT (last column). (The misclassiﬁed pixelsare shown in red color)
Fig. 5
. The Energy of different techniques for segmented imagescomponents. Due to space limitation, we did not cover the detailsof this approach. Then the spatial interaction between the neighboring pixels is modelled using the MGRF model with a new analytically estimated potential. These models are combined to formulate an energy function, which is globally minimized using
s/t
graphcuts. The experimental results show that the proposed approach outperforms thestateoftheart approaches for lung segmentation. Infuture work, we will generalize the proposed approach to work with3D as well as 2D spaces.
6. REFERENCES
[1] I.C. Sluimer, A.M.R. Schilham, M. Prokop, and B. van Ginneken, “Computer analysis of computed tomography scans of the lung: a survey,”
IEEE Trans. Med. Imag.
, vol. 25, no. 4, pp.385–405, 2006.[2] S. Hu, E. A. Hoffman, and J. M. Reinhardt, “Automatic lungsegmentation for accurate quantitation of volumetric Xray CTimages,”
IEEE Trans. Med. Imag.
, vol. 20, no. 6, pp. 490–498,June 2001.[3] I.C. Sluimer, M. Prokop, and B. van Ginneken, “Toward automated segmentation of the pathological lung in ct.,”
IEEE Trans. Med. Imag.
, vol. 24, no. 8, pp. 1025–1038, 2005.[4] D. M. Greig, B. T. Porteous, and A. H. Seheult, “Exact maximum a posteriori estimation for binary images,”
J. Roy. Stat.Soc. B
, vol. 51, no. 2, pp. 271–279, 1989.[5] Y. Y. Boykov and M. P. Jolly, “Interactive graph cuts for optimal boundary & region segmentation of objects in ND images,” in
Proc. of ICCV
, 2001, vol. 1, pp. 105–112.[6] A. Blake, C. Rother, M. Brown, P. P´erez, and P. H. S. Torr,“Interactive image segmentation using an adaptive GMMRFmodel.,” in
Proce. of ECCV
, 2004, vol. 1, pp. 428–441.[7] Y. Boykov and Garath F. L., “Graph cuts and efﬁcient NDimage segmentation,”
Int. J. of Comp. Vis.
, vol. 70, no. 2, pp.109–131, 2006.[8] A.A. Farag, A. ElBaz, and G. Gimelfarb, “Density estimationusing modiﬁed expectation maximization for a linear combination of gaussians,” in
Proc. of ICIP
, 2004, vol. 3, pp. 1871 –1874.[9] L. Ford and D. Fulkerson, “Flows in networks,” in
PrincetonUniversity Press
, 1962.[10] Y. Boykov and V. Kolmogorov, “An experimental comparisonof mincut/maxﬂowalgorithms for energy minimization in vision,”
IEEE TPAMI
, vol. 26, no. 9, pp. 1124–1137, 2004.[11] G. L. Gimelfarb, “Image textures and gibbs random ﬁelds,” in
Kluwer Academic
, 1999.[12] J. E. Besag, “Spatial interaction and the statistical analysis of lattice systems,”
J. Roy. Stat. Soc. B
, vol. 36, pp. 192–236,1974.[13] J. E. Besag, “On the statistical analysis of dirty pictures,”
J. Roy. Stat. Soc. B
, vol. 48, pp. 259–302, 1986.
911