Taxes & Accounting

Multiple Mechanisms of Top-Down Processing in Vision

Multiple Mechanisms of Top-Down Processing in Vision
of 25
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
  21 Mutpe Mecansms of Top-Down Processing in Vision Giorgio Ganis 1,2,3 and  Stephen M. Kosslyn 3,4 1 Department of Radiology, Harvard Medical School, Boston, MA 02115, USA 2 Massachusetts General Hospital, Martinos Center, Charlestown, MA 02129, USA 3 Department of Psychology, Harvard University, Cambridge, MA 02138, USA 4 Department of Neurology, Massachusetts General Hospital, Boston, MA 02142, USA 1. Introduction No animal could survive for long without perception. We must perceive the worl, not only to n foo, shelter, an mates, ut also to avo preators.Perception will fail if an animal does not register what is actually in the world. However, this simple observation does not imply that all processing during per-ception is “bottom up”—driven purely by the sensory input. Rather, bottom-uprocessing can be usefully supplemented by using stored information, engagingn processing that is “top down”—driven by stored knowledge, goals, or expecta-tions. In this chapter we explore the nature of top-down processing and its inti-mate dance with bottom-up processing. We begin by considering basic facts about the primate visual system, and then consider a theory of its functional organization, followed by novel proposals regarding the nature of different sorts of top-down processing. . The Structure of Visual Processing in the Brain An enormous amount has been learned about visual processing by studying animal models. In particular, the macaque monkey has very similar visual abili-tes to those of humans, an the anatomy of ts vsual system appears very smlarto ours. Studies of the monkey brain have revealed key aspects of the organiza-tion of the visual system, namely, its hierarchical structure and the reciprocalnature of most connections between different visual areas of the brain. We brie y review key aspects of both characteristics of the brain next.  22 G. Ganis an S.M. Kosslyn .1. Hierarchical Organization Over the past several decades, researchers have provided much evidence thatthe primate visual system is organized hierarchically. In the early 1960s and 970s, Hubel and Wiesel’s electrophysiological ndings, rst in cats and then n nonhuman primates, strongly suggested a hierarchical relationship amongearly areas in the visual system; this inference was based on the increasing size and complexity of the receptive elds as one goes from striate cortex to areasarther along in the processing stream (Hubel and Wiesel 1962, 1965, 1968, 974). The earliest areas of the visual system are organized topographically;space on the cortex represents space in the world, much as space on the retinaepresents space in the world (Felleman and Van Essen 1991; Fox et al. 1986;eeger 1999; Sereno et al. 1995; Tootell et al. 1998; Van Essen et al. 2001). The higher-level areas are not organized topographically, but often representnformation using population codes (Fujita et al. 1992; Miyashita and Chang 988; Tanaka et al. 1991). In such codes, different neurons respond to complexsual propertes, an shape s coe y the specc combination of neurons that are activated.The work by Felleman and Van Essen (1991) charted the hierarchical organi-ation of the entire visual system. They compiled a matrix of known anatomicalconnections among areas and showed that the pattern of connectivity could beste accounte for y a herarchcal structure wth multple parallel streams. Thestriate cortex (V1) was at the bottom of the entire hierarchy, and the inferotem-poral (area TE) and parahippocampal (areas TH and TF) cortex were at the top of the ventral stream (which is specialized for object vision, registering propertiessuch as shape and color) (Desimone and Ungerleider 1989).The big picture of cortical organization provided by Felleman and Van Essenhas een generally con rme y computatonal analyses of the same ataset Hilgetag et al. 1996), as well as by additional empirical approaches, such as thosebased on measuring the proportion of projecting supragranular layer neurons labeled by a suitable retrograde tracer (Vezoli et al. 2004). This neuroanatomical icture of a hierarchically organized visual system has also been con rmed by ata from single unit recording studies of higher-level visual areas. For instance,area TE in the inferotemporal cortex has been shown to contain neurons with extremely large receptive elds (often encompassing the entire visual eld),hich are tuned to complex combinations of visual features (such as combination of shape fragments and textures); in contrast, neurons in lower-level areas, such as V4 (fourth visual area), have smaller receptive elds and are tuned to simplereature combinations (Tanaka 1996). .2. Connections Among Areas A considerable amount is now known about the connections among visual brainareas, and the evidence suggests that different connections are used in bottom-upand top-down processing.  2. Top-Down Processing in Vision 23 2.2.1. Feed-Forward Connections and Bottom-Up Processing Contiguous neurons in area V1 have contiguous receptive elds (i.e., regions of space in which they will respond to stimuli). A set of contiguous neurons in area V1 in turn projects to a single neuron in area V2 (secondary visual area), andthis neuron has a larger receptive eld than any of those neurons that feed into it. This “many-to-one” mapping continues up the hierarchy until the receptiveelds become so large that the areas are no longer topographically organized.The neuroanatomical ndings and the properties of the receptive elds have given rise to numerous models that emphasize the feed forward nature of theventral stream (Fukushima 1988; Riesenhuber and Poggio 1999; VanRullenet al. 2001; Wallis and Rolls 1997; Serre et al. 2007). Electrophysiological nd-ings that document the fast onset of neural responses to visual stimuli at alllevels in the ventral stream (i.e., the mean latency of neurons in area TE atthe highest level of the hierarchy is just over 100 ms after stimulus onset) pro-vided additional impetus for these models (Lamme and Roelfsema 2000). In these models, objects are identi ed during a feed forward pass throughout the ventral stream herarchy, wth ncreasngly complex nformaton’s eng extracted at higher levels in the system. For instance, in Riesenhuber and Pog-gio’s model of the ventral stream, the units farther upstream (from area V1 toE) are tuned to increasingly complex features, all the way to units that are tuned to specic views of objects. 2.2.2. Feedback Connections and Top-Down Processing Crucially for the topic at hand, and consistent with the connectivity patternreported in earlier work by Rockland and Pandya (1979) and others, Fellemanand Van Essen described not only feed-forward connections among areas in the rmate vsual system ut also wesprea feeack connectons. They foun a striking regularity in the pattern of laminar srcin of feed-forward and feedback connections: although feed-forward connections srcinate in the supragranular layers (often layer III) and terminate in layer IV in the target area, feedbackconnections srcinate from neurons in layer VI and IIIA of the projecting area and end in layer I of the target area. Indeed, numerous other anatomical studies n nonhuman prmates have conrme that there are massve feeack connec-tions at many levels in the visual system, including pathways from areas that are not traditionally considered visual areas (Barone et al. 2000; Budd 1998; Clavagnier et al. 2004; Rockland and Pandya 1979; Salin and Bullier 1995). Forexample, area V1 has been shown to receive direct feedback connections from many extrastriate regions including V2, V3, V4, TEO (temporo–occipital), TE, as well as from nonvisual areas, including the frontal eye elds, area 36, areas H/TF, STP (superior temporal polysensory), and even the auditory cortex.The feedback connections are not simply the inverse of feed-forward con-nections. The feed-forward connections display a lovely many-to-one mappingas they ascend the hierarchy, but there is nothing of the sort for the feedback connections. Instead, the feedback connections do not appear to be precisely  24 G. Ganis an S.M. Kosslyn targeted, but rather often appear to meander (Budd 1998). Evidently, the feed-back connections are not simply “replaying” information sent downstream. .2.3. Neuroanatomically Inspired Models of Top-Down Processing A class of models of object vision has incorporated the nding of feedback con-nections in the visual system (Grossberg and Mingolla 1985; Li 1998; Mumford 992; Ullman 1989, 1995). Generally, these models assume that feedback con-nections provide a mechanism by which top-down processing can occur, allowing elatively abstract information stored in higher-level visual areas to inuence and constrain processing in lower-level visual areas. To illustrate the basic idea of hy top-down processing is needed, researchers have created binarized photo-graphs. In such photographs, gray-scale pixels are replaced with white if theirbrightness value is above a chosen threshold, or replaced with black if it is below this value. Because binarized images are highly degraded, pure bottom-up pro-cesses typically cannot organize them correctly into their constituent parts, and often one needs to use previously acquired knowledge about objects to identifythe objects in them (Fig. 1).The moels ust mentone rest on algorthms that allow an nterplay etween stored information and online input. For instance, Mumford posits that higher-level visual areas try to nd the best t with the information they receive fromlower-level visual areas by using the more abstract knowledge they store (e.g., a epresentation of a shape). The feedback connections allow higher-level visual areas to reconstruct the visual input in lower-level visual areas, based on such a est t. The msmatch etween the reconstructe vsual nput an the orgnal nput in lower-level visual areas (i.e., information not explained by the current ig . 1. This binarized picture illustrates the problems encountered by purely bottom-upapproaches to vision. It is very if cult to parse correctly the fox at the center of thepicture using purely bottom-up processing. Using top-down processing to exploit con-straints imposed by knowledge of the shape of foxes makes the task much easier  2. Top-Down Processing in Vision 25 t in higher-level areas) is then sent forward, which can trigger another top-down rocessing cycle.This class of models of top-down processing in the ventral stream has typically ignored the role of areas outside the ventral stream or has only postulated nspeci ed extravisual inputs. However, many nonvisual areas in the frontal andarietal lobe are connected to areas in the ventral stream (Petrides 2005).Another class of models, in contrast, focuses on top-down inuences that these nonvisual areas exert on areas in the ventral stream. For instance, the model of refrontal function by Miller and Cohen (2001) has focused on the role of therefrontal cortex in biasing processing in areas in the ventral stream.Traditionally, the different classes of models have been pursued indepen-dently, although some of the terminology has overlapped. Unfortunately, theterm “top-down processing” in vision has been used loosely in the neuroscientic literature to refer to a disparate range of phenomena. For instance, it has beensed in the context of the neural effects of visual attention (Hopnger et al. 000), memory retrieval (Tomita et al. 1999), and phenomena such as illusorycontours (Halgren et al. 2003).In the remainder of the present chapter, we develop explicit distinctions between different types of visual top-down processes; these distinctions are cast wthn the context of a roa theory of the vsual system that ncorporates othbottom-up and top-down processes (Kosslyn 1994) as well as the role of non-visual areas. Our aim is to make explicit some of the assumptions regarding top-own processes that are mplct n the lterature an propose a rst-orer taxonomy, rather than to provide an exhaustive review of the top-down process-ng literature. In the following section we briey summarize our theory of visual rocessng urng vsual oect entcaton, relyng on the ackgroun alreay rovided, and then we proceed to describe how different types of top-down rocessing may operate within this system. 3. A Theory of the Functional Organization of Late VisualProcessing in the Primate Brain We propose that there are two general kinds of visual processes, “early” and“late.” Early visual processes rely entirely on information coming from the eyeswhereas late processes rely on information stored in memory to direct processing.We must distinguish between early and late processes and the specic brain areas nvolved in vision: late processes can occur even in areas that are involved in therst stages of bottom-up processing (Lamme and Roelfsema 2000). Low-level visual areas are involved in both early (bottom-up) and late (top-down rocessing).Vson, an more spec cally oect ent caton, s not a untary an unf-ferentiated process. Indeed, similarly to memory operations such as encoding and recall, which are carried out by many subprocesses (Schacter 1996; Squire1987), object identication is carried out by numerous subprocesses (for example,
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks