Entertainment & Humor

Subjective Surfaces: A Geometric Model for Boundary Completion

Subjective Surfaces: A Geometric Model for Boundary Completion
of 21
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  International Journal of Computer Vision 46(3), 201–221, 2002c  2002 Kluwer Academic Publishers. Manufactured in The Netherlands. Subjective Surfaces: A Geometric Model for Boundary Completion A. SARTI  DEIS, University of Bologna, Italy asarti@deis.unibo.it R. MALLADI AND J.A. SETHIAN  Department of Mathematics, Lawrence Berkeley National Laboratory, University of California, Berkeley, CA 94720, USA malladi@euphrates.lbl.govsethian@math.berkeley.edu  Received June 14, 2000; Revised August 22, 2001; Accepted August 22, 2001 Abstract.  We present a geometric model and a computational method for segmentation of images with missingboundaries. In many situations, the human visual system fills in missing gaps in edges and boundaries, building andcompleting information that is not present. Boundary completion presents a considerable challenge in computervision, since most algorithms attempt to exploit existing data. A large body of work concerns completion models,which postulate how to construct missing data; these models are often trained and specific to particular images. Inthis paper, we take the following, alternative perspective: we consider a given reference point within the image,and then develop an algorithm which tries to build missing information on the basis of the given point of viewand the available information as boundary data to the algorithm. Starting from this point of view, a surface isconstructed. It is then evolved with the mean curvature flow in the metric induced by the image until a piecewiseconstant solution is reached. We test the computational model on modal completion, amodal completion, andtexture segmentation. We extend the geometric model and the algorithm to 3D in order to extract shapes from lowsignal/noise ratio ultrasound image volumes. Results in 3D echocardiography and 3D fetal echography are alsopresented. Keywords:  subjective surfaces, segmentation, perceptual contours, level sets, differential geometry, Riemanniangeometry, surface evolution 1. Introduction Consider the images in Fig. 1. The human visual sys-tem completes the internal objects exploiting the exist-ingsparsedata.Intheimageontheleft,asolidtrianglein the center of the figure appears to have well-definedcontoursevenincompletelyhomogeneousareas;inthecenter figure, a large rectangle is perceived, and, in thefigureontheright,awhitesquarepartiallyoccludedbya gray disk is perceived. These contours, which are notcharacterized by image gradients, may be thought of as “apparent” or “subjective” contours. Following theusualconvention,wedistinguishbetween“modalcom-pletion”, in which the goal is to construct a perceivedboundary (as in Fig. 1, left and center) and “amodalcompletion”, in which one reconstructs the shape of a partially occluded objects (as in Fig. 1, right); seeKanizsa (1976, 1979).Ourgoalistoextract,i.e.segment,theseinternalob- jects. Since some information is missing, one class of algorithms typically complete the segmentation pro-cess by building models to postulate what happens  202  Sarti, Malladi and Sethian Figure 1 . Three images with subjective contours. between the available data. In this paper we presenta method for segmentation of images with missingboundaries.We take the following perspective: •  The observer is drawn to a reference point withinthe image, and this is a natural point from which toconstruct a completion process. •  Starting from this given reference point, we candevise an algorithm which takes advantage of theavailable boundary data to construct a complete seg-mentation. Thus, the computed segmentation is afunction of the reference point.As in Sarti et al. (2000), we view a segmentationas the construction of a piecewise constant surfacethat varies rapidly across the boundary between dif-ferent objects and stays  fl at within it. In Mumfordand Shah (1989) the segmentation is a piecewisesmooth/constant approximation of the image. In ourapproach the segmentation is a piecewise constant ap-proximation of the point-of-view or the reference sur-face. To obtain it we de fi ne the following steps:1. Select a  fi xation point and build an initial surfacecorresponding to this point-of-view.2. Detect existing local features in the image.3. Evolve the point of view surface with a  fl ow thatdepends both on the geometry of the surface andon the image features. The  fl ow evolves the initialconditiontowardsapiecewiseconstantsurface.Thecompleted object is the  fl at manifold at the top of the surface.4. (Automatically) Pick the level set that describes thedesired object (that is a closed curve).During the evolution, the point-of-view surface isattracted by the existing boundaries and steepens. Thesurface evolves towards a piecewise constant solutionthrough the closing of the boundary fragments and the fi lling in the homogeneous regions. A solid object isdelineated as a constant surface bounded by existingand recovered shape boundaries. With this method, theimage completion process depends both on the pointof view and on the geometric properties of the image(Sarti et al., 2000).Humanperceptualorganizationhasbeenextensivelystudied, and the resulting theories of vision are oftenused to construct computational models for image seg-mentation. For example, properties like boundary con-tinuity of direction, regularity, proximity constraints,maximum homogeneity, closure, and symmetry havebeenappliedasguidelinesformanycomputervisional-gorithms. The perspective and algorithms in this paperare connected to some experimental results on percep-tual organization of the human visual system outlinedbyKanizsa(1976,1979,1980).Inparticular,theemer-gence of the subjective contours depends on the posi-tion of the point of view:  “ if you  fi x your gaze on anapparent contour, it disappears, yet if you direct yourgazetotheentire fi gure,thecontoursappeartobereal, ” see Kanizsa (1976).In the following, we construct a geometrical modelforboundarycompletionbykeepinginmindtheafore-mentioned characteristics of perceptual organization.Both the mathematical and algorithmic approach inour method relies on a considerable body of recentworkbasedonapartialdifferentialequationsapproachto both front propagation and to image segmentation.Level set methods, introduced by Osher and Sethian(1988), track the evolution of curves and surfaces, us-ing the theory of curve and surface evolution and thelink between front propagation and hyperbolic conser-vation laws developed by Sethian (1985, 1987). These  Subjective Surfaces: A Geometric Model for Boundary Completion 203methods embed the desired interface as the zero levelset of an implicit function, and then employ  fi nite dif-ferences to approximate the solution of the resultinginitial value partial differential equation. Malladi et al.(1993) and Caselles et al. (1993) used this technologyto segment images by growing trial shapes inside a re-gion with a propagation velocity which depends on theimage gradient and hence stops when the boundary isreached; thus, image segmentation is transformed intoan initial value partial differential equation in which apropagating front stops at the desired edge.Sochen et al. (1998) view image processing as theevolution of an image manifold embedded in a higherdimensional Riemannian space towards a minimal sur-face. This framework has been applied to processingboth single- and vector-valued images de fi ned in twoand higher dimensions (see Kimmel et al., 2000). Inthis paper we view segmentation as the evolution of aninitial reference manifold under the in fl uence of localimage features.Our approach takes a more general view of the seg-mentation problem. Rather than follow a particularfront or level curve which one attempts to steer to thedesired edge (as in Malladi et al., 1993, 1995; MalladiandSethian,1998;Casellesetal.,1993,1997a,1997b),we begin with an initial surface, chosen on the basisof a user-supplied reference  fi xation point. We then fl ow this  entire surface  under a speed law dependenton the image gradient, without regard to any particularlevelset.Suitablychosen,this fl owsharpensthesurfacearound the edges and connects segmented boundariesacross the missing information. On the basis of thissurface sharpening, we can identify a level set corre-sponding to an appropriate segmented edge.InSartietal.(2000),thetheoreticalunderpinningsof this approach were presented. In this paper we concen-trate on results for a variety of 2D images (cognitive,medical, photographic, texture images) and we extendthemethodologytotheextractionof3Dshapes.Inpar-ticular we test our algorithm on 2D and 3D ultrasoundimages in which the signal to noise ratio is truly smalland the most part of the edge is missing.Thepaperisorganizedasthefollowing.InSection2we review past work in segmentation of images withmissing boundaries. In Section 3 we discuss the math-ematical problem and in Section 4 we present a nu-merical method to solve it. In Section 5 the extensionof the 2D subjective surface model to 3D manifolds ispresented.InSection6,weshowresultsoftheapplica-tion of the method to different 2D images and discussboth the modal and amodal completion scenarios. InSection7,resultsoftheapplicationofsubjectivemani-folds to 3D image completion are presented and a vali-dation of the method on 3D echocardiographic imagesis provided. In Section 8, a discussion about the accu-racy and the ef  fi ciency of the computational model isalso provided. 2. Past Work and Background Inthissection,wereviewsomepreviousworkaimedatrecoveringsubjectivecontours.Weareinterestedinre-covering explicit shape representations that reproducethat of the human visual perception, especially in re-gions with no image-based constraints such as gradi-ent jump or variation in texture. In Mumford (1994),the distribution of subjective contours are modeledand computed by particles traveling at constant speedsbut moving in directions given by Brownian motion.Williams and Jacobs (1997a, 1997b) introduce the no-tion of a stochastic completion  fi eld, the distribution of particle trajectories joining pairs of position and direc-tion constraints, and show how it could be computed.In this approach, a distribution of particles is beingcomputed, rather than an explicit contour/surface,closed or otherwise.A combinatorial approach is considered in Saund(1999). A sparse graph is constructed whose nodes aresalient visual events such as contrast edges and L-typeand T-type junctions of contrast edges, and whose arcsare coincidence and geometric con fi gurational rela-tions among node elements. An interpretation of thescene consists of choices among a small set of labelsfor graph elements. Any given labeling induces an en-ergy, or cost, associated with physical consistency and fi gural interpretation.A common feature of both completion  fi elds, com-binatorialmethods,aswellasvariationalsegmentationmethods(MumfordandShah,1989)istopostulatethatthe segmentation process is independent of observer ’ spoint of focus. On the other hand, methods based onactive contours perform a segmentation strongly de-pendent on the user/observer interaction. Since theirintroduction in Kass et al. (1988), deformable mod-els have been extensively used to integrate boundariesand extract features from images. An implicit shapemodeling approach with topological adaptability andsigni fi cant computational advantages has been intro-duced in Malladi et al. (1993), Caselles et al. (1993)and Malladi et al. (1995). In these papers, the level set  204  Sarti, Malladi and Sethian approach (Osher and Sethian, 1988; Sethian, 1999) isusedtoframecurvemotionwithacurvature-dependentspeed. These and a host of other related works rely onedge information to construct a shape representationof an object. In the presence of large gaps and miss-ing edge data, the models can go astray and away fromthe required shape boundary. This behavior is due toa constant speed component in the governing equa-tion that helps the curve from getting trapped by iso-lated spurious edges. On the other hand, if the con-stant in fl ation term is switched off, as in Caselles et al.(1997a) and Sarti and Malladi (2001), the curve hasto be initialized close to the  fi nal shape for reasonableresults. We also note a more recent segmentation ap-proach introduced in Chan and Vese (1998), in whichtheauthorsusegeometriccurveevolutionforsegment-ing shapes without gradient by imposing a homogene-ity constraint. An exhaustive review of algorithms forgrouping is outside the purpose of this paper; we referthe interested reader to Malik et al. (2000) for a largerperspective as well as for the classi fi cation of groupingalgorithms in the broad families of region-based andcontour-based approaches.The approach in this paper rests on view thatsegmentation, regardless of dimensionality, is a  ‘ view-point ’  dependent computation. 1 The  “ view-point ”  orthe user-de fi ned initial guess to the segmentation al-gorithm serves as input to the algorithm, and is usedto construct a point-of-view surface. Next, this refer-encesurfaceisevolvedaccordingtoafeature-indicatorfunction. The shape completion aspect relies on twocomponents: (1) the evolution of a surface and (2) a fl ow that combines the effects of level set curve evo-lution with that of surface evolution. In what follows,wepresentageometricframeworkthatmakesthispos-sible. Computing the  fi nal segmentation (a contour orsurface) is accomplished by choosing and plotting aparticularlevelsetofahigherdimensionalfunction;aswe shall show, this particular level set may be chosenautomatically. 3. Theory 3.1. Local Feature Detection Consider an image  I   :  (  x  ,  y )  →  I  (  x  ,  y )  as a real pos-itive function de fi ned in some domain  M   ⊂  R 2 . Oneinitial task in image understanding is to build a repre-sentation of the local structure of the image. This ofteninvolves detection of intensity changes, orientation of structures, T-junctions and texture. The result of thisstage is a representation corresponding to  the raw pri-mal sketch  (Marr, 1982). Several methods have beenproposed to compute the raw primal sketch, includingmultiscale/multiorientation image decomposition withGabor  fi ltering (Gabor, 1946), wavelet transform (Lee,1996), deformable  fi lter banks (Perona, 1995), textons(Julesz,1981;Maliketal.,1999)etc.Forthepurposeof the present paper we consider a simple edge indicator,namely g (  x  ,  y )  =  11 + ( |∇  G σ  (  x  ,  y )   I  (  x  ,  y ) | /β) 2  where G σ  (ξ)  =  exp ( − (ξ/σ) 2 )σ  √  π (1)The edge indicator function  g (  x  ,  y )  is a non-increasing function of   |∇  G σ  (  x  ,  y )   I  (  x  ,  y ) | , where G σ  (  x  ,  y )  is a Gaussian kernel and  ()  denotes the con-volution.Thedenominatoristhegradientmagnitudeof asmoothedversionoftheinitialimage.Thus,thevalueof   g  is closer to 1 in  fl at areas ( |∇   I  |→ 0) and closerto 0 in areas with large changes in image intensity, i.e.the local edge features. The minimal size of the detailsthat are detected is related to the size of the kernel,which acts like a scale parameter. By viewing  g  as apotential function, we note that its minima denotes theposition of edges. Also, the gradient of this potentialfunction is a force  fi eld that always points in the localedge direction; see Fig. 2.To compute  ∇  G σ  (  x  ,  y )   I  (  x  ,  y ) , we use the con-volution derivative property  ∇  G σ  (  x  ,  y )   I  (  x  ,  y )  =∇  ( G σ  (  x  ,  y )  I  (  x  ,  y ))  and perform the convolution bysolving the linear heat equation ∂ϕ∂ t  = ∇ · ( ∇  ϕ)  (2)in the time interval [0 ,σ  ] with the initial condition ϕ(  x  ,  y , 0 )  =  I  (  x  ,  y ) . We conclude this subsection byobserving that there are other ways of both smoothingan image as well as computing an edge indicator func-tion or in general a feature indicator function, see, forexample, Caselles et al. (1997a), Malladi et al. (1995),Malladi and Sethian (1996), Perona and Malik (1987),Sarti et al. (1999), Sochen et al. (1998) and Kimmelet al. (2000). 3.2. Global Boundary Integration Now,considerasurface  S   :  (  x  ,  y ) → (  x  ,  y ,) de fi nedin the same domain  M   of the image  I  . The differential  Subjective Surfaces: A Geometric Model for Boundary Completion 205 Figure 2 . Local edge detection: The edge map  g  and its spatial gradient −∇  g . area of the graph  S   in the Euclidean space is given by: dA  =   1 +  2  x   +  2  y  dx dy  (3)We will use the edge indicator  g  to stretch and shrink a metric appropriately chosen so that the edges act asattractors under a particular  fl ow. With the metric  g applied to the space, we have dA g  =  g (  x  ,  y )   1 +  2  x   +  2  y  dx dy .  (4)Now, consider the area of the surface  A g  =    M  g (  x  ,  y )   1 +  2  x   +  2  y  dx dy ,  (5)(see Appendix A for derivation) and evolve the surfacein order to reduce it. The steepest descent of Eq. (5)is obtained with usual multivariate calculus techniquesand results in the following  fl ow ∂∂ t  =  g  1 +  2  x     yy  − 2   x    y   xy  +  1 +  2  y    xx  1 +  2  x   +  2  y + ( g  x    x   + g  y   y )  (6)(see Appendix B for derivation). 2 The  fi rst term onthe right hand side is a parabolic term that evolvesthe surface in the normal direction under its mean cur-vature weighted by the edge indicator  g . The surfacemotion is slowed down in the vicinity of edges (thatis, where  g → 0). The second term on the right corre-sponds to pure passive advection of the surface alongtheunderlyingvelocity fi eld −∇  g  whosedirectionandstrength depend on position. This term pushes/attractsthe surface in the direction of the image edges. Notethat  g (  I  (  x  ,  y ))  is not a function of the third coordi-nate, therefore the vector  fi eld −∇  g  lies entirely on the (  x  ,  y )  plane.The following characterizes the behavior of the  fl ow(Eq. (6)):1. In regions of the image where edge information ex-ists,theedgeindicator g  →  0andthemainequation(Eq. (6)) reduces to a simple advection equation: ∂∂ t  ≈  g  x    x   + g  y   y  (7)drivingthesurfacetowardstheedges.Thelevelsetsof the surface are attracted to the edge and accu-mulate.Consequently,thespatialgradientincreasesand the surface begins to develop a discontinuity.2. Inside homogeneous regions of the image the edgeindicator g = 1,andEq.(6)reducestotheEuclideanmean curvature  fl ow: ∂∂ t  =  1 +  2  x     yy  − 2   x    y   xy  +  1 +  2  y    xx  1 +  2  x   +  2  y (8)whosesolutionsarethehorizontalplanescharacter-izing the inside of the  fi gures and the background.3. We now address the regions in the image corre-sponding to subjective contours. We take the viewthat appropriate subjective contours are continu-ation of existing edge fragments. As discussedabove, in regions with well de fi ned edge informa-tion, Eq. (6) causes the level curves to accumulate,therebycausinganincreaseinthespatialgradientof   . Since continuity for the surface is required dur-ing the evolution, the edge fragment information
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks