Comprehensive Physiology offers downloadable PowerPoint presentations of figures for non-profit, educational use, provided the content is not modified and full credit is given to the author and publication.

Download a PowerPoint presentation of all images

	Figure 1. The abstract painting Interchanged (1955) by Willem de Kooning. At a reported purchase price of US $300 million, it is one of the most expensive paintings in the world. See text for additional details.
	Figure 2. The importance of recurrent processing in visual perception. When viewed for the first time, this two‐tone “Mooney” image appears to be an unrecognizable pattern of black and white blobs. That is, it is hard to interpret this image based on sensory information (i.e., bottom‐up or feed‐forward processing) alone. However, after viewing a full grayscale or color counterpart (for which see Figure 3A—which, being smaller in size, has no pixel‐to‐pixel correspondence with the above image), the Mooney image becomes easy to interpret. Note that viewing the disambiguating image rapidly, drastically, and enduringly alters our perception of the Mooney image, although the Mooney image itself remains physically unchanged. Feed‐forward theories of vision cannot help explain such phenomena. Recurrent (or “reentrant”) neural signals bring to bear such top‐down influences as prior knowledge and the behavioral context to help constrain the interpretation of the visual image. Learning to interpret Mooney images can be understood as an extreme case of knowledge‐mediated disambiguation that is part and parcel of normal visual perception (51,82,125,142).
	Figure 3. Some complexities of natural images and the information processing required to making sense of them. Panels A through D show real‐world scenes that help illustrate some of the complexities of such scenes. For instance, there are multiple objects of the same type in each picture. They all vary greatly in image size, illumination, shadows, occlusion, viewpoint, etc. But the visual system recognizes them as objects of the same kind. In other words, the brain must be able to discard a variety of image features as irrelevant to recognizing the images. Note that, in case of each scene in this figure, our understanding of the scene evolves over time. Note also that the objects, their spatial relationships, and even the semantic similarities and differences among them are such that the percept that each scene elicits is more than just the sum of the parts. In addition, each picture contains implicit cues to motion and/or depth that static, 2D pictures such as these do not do justice to. In fact, natural scenes differ from each other and from the relatively simple stimuli used in many a vision study, such as a sinusoidal grating on a neutral gray background, in myriad ways. Studies have shown that the visual system is adapted to contend with the statistics of natural scenes (307), so that neural responses to “artificial” stimuli may provide a substantially different and potentially misleading picture of how the brain works. On the other hand, note that our visual system performs very well in recognizing scenes with “unnatural” statistics, such as the street scene in panel C and the indoor scene in panel D, even though it evidently did not encounter the “unnatural” statistics of human‐made objects until quite recently on the evolutionary time scale. This is because the brain is quite good at adapting to a variety of nonoptimal inputs.
	Figure 4. Visual system can often recognize objects with great precision with very little information. A veridical description of the object, even if it were possible, is not necessary. (A) In the picture on the left, very little of the dog is visible. But we have little trouble recognizing the dog. Many probably can even readily name the breed of the dog. (B) Few would have trouble recognizing the person from the picture. (C) The tell‐tale hat. For most readers, this picture simply shows a hat hung on some plumbing. But this famous picture, on the cover of the very first issue of Physics Today in May 1948, showed the famous “pork pie” hat of J. Robert Oppenheimer, the father of the atomic bomb. Therefore, the picture was highly topical and was readily understood at the time. But for most of us living today, this picture requires either some historical knowledge or explanation. Thus, our perception of a visual image is influenced by a great many nonvisual factors. Neurophysiological understanding of such influences on cognition represents a major challenge.
	Figure 5. There is more to vision than feed‐forward processing. This picture of an unknown little girl was taken by South African photojournalist Kevin Carter in the famine‐stricken South Sudan in March 1993. The girl had reportedly collapsed from weakness on her way to a United Nations food center. (Reproduced, with permission, from The Vulture and the Little Girl by Kevin Carter. Pulitzer Prize for Feature Photography, 1994.) Note that feed‐forward processing by itself would completely miss the import of the picture.
	Figure 6. (A) A self‐driving car. (B) Two instances of the CAPTCHA (or Completely Automated Public Turing test to tell Computers and Humans Apart) internet device designed to prevent automated logins (121,326). In the example on the left, the website asks the user to perform a straightforward object categorization task, namely distinguish pictures of people wearing glasses from pictures of people without glasses. This tends to be quite successful in preventing machines from logging on to the website. However, this success is not so much because it would be all that difficult nowadays to train a suitably designed computer program to distinguish the two categories of human faces (see, e.g., (2,191)), but essentially because, at present, such programs tend to be hyperspecialized, and tend not to generalize beyond their training sets to other visual objects or tasks (121,199). That is, such an “intelligent” program, once trained to tell aforementioned types of faces apart, can be readily stumped at present by rather slight changes in the underlying categories (panel B, right), or the task (not shown) (2,191). However, recent research suggests that the problem of overspecialization of intelligent machines is likely to straightforwardly surmountable, so that CAPTCHA strategies such as the one illustrated in this panel are unlikely to be effective for long. That is, machine vision is getting ever better at mimicking human high‐level vision (199).
	Figure 7. An outline of Marr's theory of visual processing. See text for details.
	Figure 8. Visual anatomical hierarchy in the macaque monkey described by Felleman and Van Essen, 1991 (98). Each colored rectangle represents a distinct cortical visual area. The open rectangles at top denote higher cortical areas that are not considered primarily visual area. The two gray rectangles at bottom denote the retinal ganglion cells (RGC) and the lateral geniculate nucleus (LGN). Lines connecting the cortical areas denote interconnections, usually reciprocal, between a given pairs of areas. The various brain regions are represented in a tiered, or hierarchical, fashion based on objective anatomical criteria, most important of which are the laminar patterns of feed‐forward and feedback connections (also see (98,216,343)). For additional details and abbreviations, see Felleman and Van Essen, 1991 (98). An alternative formulation of the hierarchy, originally formulated by Mishkin, Gross, and their colleagues (130) (also see (78,129)) is largely similar, but it parcels the cortex into many fewer areas and recognizes fewer interconnections. Also, some the visual area names are different in this scheme. For instance, areas AIT and CIT (anterior and central inferotemporal areas, respectively) in the Felleman and Van Essen scheme are equivalent to area TE (temporal area) in the Gross et al. scheme, and area PIT (posterior inferotemporal area) in the Van Essen scheme is equivalent to area TEO (temporooccipital area) in the Gross et al. scheme. Both schemes are used in this review, based on the scheme used by the study in question. Reproduced, with permission, from (98).
	Figure 9. Our evolving understanding of the functional organization of the primate visual system. Panels A and B depict our view of the two main visual processing streams in the macaque brain in the early 1980s. (A) The dorsal and ventral visual pathways as originally formulated by Mishkin and colleagues in 1993 is denoted by solid arrows (OB → OA → PG pathway being the anatomically dorsal pathway, and OB → OA → TEO → TE being the ventral pathway) (225). The continuation of these pathways to FDΔ and FDv, respectively (dashed arrows) represented the collective outcome of many additional studies. (B) Anatomical locations of the various regions, some renamed according to a more modern naming convention (188). (C) Key changes in the receptive field properties of neurons along the ventral pathway. Areas are color‐coded as in panel B. Panel D summarizes our understanding of the same pathways some 30 years later, as summarized by Kravitz and colleagues in 2013 (188). Note that it is clear that the two pathways have turned out to be far more interconnected than previously envisioned. What is lost in terms of pedagogical simplicity is more than made up for by the nuance and granularity of this network picture, presaged decades ago by Kerrigan and Maunsell (221). Self‐evidently, far more remains to be learned, including how these networks interact with other networks in the brain and subserve behavior. Adapted, with permission, from (188). Human brain (not shown) is also purported to have two evolutionarily homologous processing pathways, although there is even less empirical information to support this notion.
	Figure 10. What's to infer? Is not it all there in the image? The answer is no. Retinal image is simply a 2D pattern of image intensities. Image intensities of a particular image are represented in panel A as a color‐coded surface plot, where the height and color of a point denotes the image intensity at that point. Note that when the image is represented in this fashion, it makes no sense to us. But when the same image is represented as corresponding variations in image intensities (panel B), we readily recognize it as an image of a brook in the woods. However, the information in the two representations is exactly the same. The difference is that, in panel B, the image is in an input “format” that our eyes can process. Beyond that, the inferential processes to “make sense” of the pattern of intensities are exactly the same.
	Figure 11. Visual illusions help illustrate the inferential nature of visual perception. They also demonstrate that the inference need not be a deliberate, volitional, or conscious process. (A) Ames room. Two people stand in opposite corners of the room. One appears to be much taller than the other, even though they are roughly the same height. Instead, it is the room that is distorted to produce this illusion. Picture courtesy of Ian Stannard, Flickr. Reproduced with permission. (B) Hollow‐mask illusion. This picture shows the front of the mask (right) and the hollow back of the mask (left). Nonetheless, both look like normal, convex faces. For a video demonstration of the hollow mask illusion, see https://www.youtube.com/watch?v=sKa0eaKsdA0. Note that, in this video, the mask appears to flip its direction of rotation at the same time the other side of the mask begins appearing. Theoretical studies show that visual illusions are often perfectly rational inferences given the evidence (see, e.g., (108,357)).
	Figure 12. The spatiotemporal domain of the methods available for the study of the functional organization of nervous system in 2014, compared to the methods available in 1988 (inset). Each colored region represents a range of spatial and temporal resolutions for a given method. Open regions represent measurement techniques; filled regions, perturbation techniques; EEG, electroencephalography; MEG, magnetoencephalography; PET, positron emission tomography; VSD, voltage‐sensitive dye; TMS, transcranial magnetic stimulation; and 2‐DG, 2‐deoxyglucose. Redrawn, with permission, from (290).
	Figure 13. Neuronal responses in monkey visual area V1 during binocular rivalry. See text for details. Adapted, with permission, from (33).
	Figure 14. Figure‐ground segregation and perceptual organization. (A) How many circles can you see in this image? In this image, referred to as the Coffer Illusion, you should be able to see 16 circles. Figure courtesy of Dr. Anthony Norcia, Stanford University. Reproduced with permission. (B) Camouflage is an extreme case of figure‐ground segregation, where the object of interest is hard to recognize even when “in plain view.” This image shows two variants of the pepper moth Biston betularia, one black and the other with light peppered coloring. The black variant is effectively camouflaged against colored tree bark whereas the light variant is easy to recognize (i.e., it “pops out”). The opposite is true when the same two variants are seen against a background of light bark and lichens. The black variant emerged for the first time in the industrial midlands of Britain in the 19th century, where tree barks were turning black with industrial soot. Soon the black variants became the more common variant, because the predators of the moths had much greater success breaking the camouflage of the lighter variants because it was harder for the lighter variants to find light backgrounds to camouflage themselves against. This was an instance of the prey “gaming” the predators’ high‐level visual faculties to enhance its own survival (179,334). Figure from Ford, E.G. (1977) Ecological Genetics. Springer. Used with permission.
	Figure 15. Selectivity for faces in the macaque superior temporal polysensory area (STP). (A) Anatomical location of area STP (yellow highlight). (B) Responses of a single STP neuron to various visual objects, including variations in face stimuli. Note that, among the stimuli tested, the neuron responds best to a face with all the key facial features (left column, second stimulus from top). Cutting this stimulus into 16 pieces and showing the pieces in a shuffled order essentially eliminated the response (right column, third stimulus form top). The icon at bottom right denotes the size and visual field location of the receptive field. C, contralateral visual field (i.e., contralateral to the recording location). I, ipsilateral. Such neurons with large receptive fields that span both the visual hemifields are common in the visually responsive areas of the central and anterior temporal cortex. Adapted, with permission, from (40).
	Figure 16. Coarse‐to‐fine tuning of shape categories in the macaque IT. The stimulus set consisted of 38 stimuli, a subset of which is shown in the inset. The global shape categories (inset, vertical axis) consisted of human faces, monkey faces, and geometric shapes. The fine categories (inset, horizontal axis) consisted of the various facial identities and expressions. The plots show the cumulative information transmission rate of a sample of IT neurons about both the global and fine categories (red and blue lines, respectively). The thick horizontal line along the x axis denotes the stimulus duration. Adapted, with permission, from (140,323).
	Figure 17. Grid cells help represent the external visual space. Grid cells were originally reported in rats, but have since been found in many species, including monkeys. (A) Responses of a single grid cell in the entorhinal cortex of the rat. Left, black lines denote the trajectory of a rat freely moving in a box. The cell fired spikes (red dots) when the rat was at specific locations within the box. These locations were organized in a grid‐like fashion that spanned the box. Right, the firing rates of the cell represented as a heatmap, where “warmer” colors denote higher firing rates. Adapted, with permission, from (134). (B) Responses of a single grid cell in the entorhinal cortex of the macaque monkey. Unlike the rat referred to in panel A, the monkey was stationary. It sat in a primate chair with its head held steady, but freely moved its eyes as it looked at real‐world pictures (not shown). Left, red dots denote the locations in a picture (not shown) that the monkey fixated, or gazed steadily at for a brief period. Center, firing rate of the grid cell shown in heatmap format. The scale bars at bottom each denote 6° of visual angle. Adapted, with permission, from (180).
	Figure 18. Coding of visual context. The classical receptive field (CRF) of a given neuron is the portion of the visual field in which the neuron is most responsive to visual stimuli. The surrounding region in which visual stimulation modulates the responses to CRF stimulation is referred to as the nonclassical receptive field (nCRF) or nonclassical surround. This figure shows one of the earliest demonstrations of the modulatory effect of nCRF on CRF, in this case the response of a single neuron in macaque MT. Individual neurons in MT often response best when the stimulus moves in a particular direction, often referred to as its preferred or optimal direction. The neuron shown responded best when the dots in the CRF moved horizontally from left to right. CRF is denoted by the dashed rectangle in the icons at top. (Left panel) Modulatory effect of stationary surround on motion stimuli in the CRF. The direction of the movement of dots in the CRF was systematically varied, while the dots in the nCRF were held stationary. (Right panel) Modulatory effect of surround motion on motion stimuli in the CRF. The responses of the same neuron when the dots moved in its optimal direction, while the direction of the dots in the nCRF was systematically varied. Note that the neuron's responses vary systematically (i.e., the responses are “tuned”18) with respect to both CRF motion, and motion in both the center and surround. Thus, neuron can convey information of the motion “context,” or motion of a given moving object relative to nearby stationary or moving objects. Adapted, with permission, from (5).
	Figure 19. Corticostriatal loops in human brain. (A) Four distinguishable but mutually overlapping loops are usually recognized (colored labeled arrows), based primarily on the types of tasks in which they play prominent roles. Cortical inputs arrive largely via the striatum and ultimately are directed back into the cortex via the thalamus. The (ultimate) cortical output of the basal ganglia reaches largely to the same cortical areas that give rise to the initial inputs to the basal ganglia. The visual loop is known to play a prominent role in the learning of visual object categories, but during object categorization tasks using learned categories, the executive loop also plays a prominent role. Corticostriatal loops in the nonhuman primate brain are largely similar (not shown). Adapted, with permission, from (287); also see (222,250). (B) A more detailed circuit map of the visual loop shows the flow of information within the loop. GPe: Globus pallidus, external portion. GPi: Globus pallidus, internal portion. SNr: Substantia nigra pars reticulata. SNc: Substantia nigra pars compacta. STN: Subthalamic nucleus. VTA: Ventral tegmental area. Adapted, with permission, from (286).
	Figure 20. An illustration of Granger causality. This figure shows cause and effect relationship between two hypothetical neural responses (top and bottom rows, respectively). The cause‐and‐effect relationship exists throughout the responses, but is most readily apparent by visual inspection of those portions of the responses where the response is prominently modulated (arrows). Granger causality uses the entire length of both responses to quantitatively measure the cause‐and‐effect relationship even when the relationship may be too subtle or complex to be visually evident. In the present case, the response shown in the top row is said to “Granger‐cause” the response in the bottom row. Note that the term “Granger causation” denotes an inferred cause‐and‐effect relationship, which may or not include direct causation. Thus, the concept of Granger causality is in some respects narrower, and in some other respects broader, than the concept of direct causation (13,29,224). But in either case, the cause must necessarily precede the effect.
	Figure 21. Resting state brain networks are very similar to brain networks active during tasks. (A) Smith and colleagues (309) extracted 20 mutually independent patterns of activation in the resting state networks (RSNs) from a database of 36 adult human subjects using the independent components analysis (ICA) (319). Brain regions identified by each of these ICAs can be thought of as an independent network. Smith and colleagues then compared RSN ICAs with ICAs of task‐activated networks from nearly 30,000 subjects from the BrainMap (BM) database (brainmap.org). Comparisons for ten most informative ICAs are shown side by side in this panel. For each ICA, activation is shown in a color‐coded format for a coronal, sagittal, and horizontal section (top, middle, bottom row, respectively). (B) Smith and colleagues (309) then analyzed the extent to which the top ten of the RSN ICAs play a role in various types of behavioral tasks (or “behavioral task domains” defined by the BrainMap database). Higher color values denote a correspondingly larger role by the given network in a given task. Note that each given type of task recruits different RSNs to different extents. Conversely, each RSN is active during multiple, different task paradigms, with the degree of participation varying according to the task. Thus, the RSNs represent a repertoire that the brain recruits and employs to various degrees depending on the task at hand. For details and some important caveats, see (309). Adapted, with permission, from (309).
	Figure 22. Networks in human brain at rest. Nodes denote the center of mass of the corresponding brain regions, and the edges (i.e., colored lines) represent intrinsic connections between a pair of regions identifiable in the resting brain. Blue nodes represent the “rich club” brain regions, which are well‐connected brain regions that are well‐connected with other well‐connected brain regions. Gray dots denote nonrich club regions. Red lines denote connections between rich club regions. Orange lines denote “feeder” connections that connect a rich club region with a nonrich club region. Yellow lines denote “feeder” connections that connect a nonrich club region with another nonrich club region. Adapted, with permission, from (339). Note that this figure does not show subcortical or cerebellar networks, which decidedly play crucial roles in brain function.
	Figure 23. Neuronal responses in monkey middle temporal area (MT) reflect perceptual decisions in awake, behaving monkeys. (A) Stimuli and task paradigm. The studies by Newsome and colleagues (37,236) (278) used dots moving either in the preferred direction of the cell under study, or in the opposite (“null”) direction, depending on the trial. The investigators “tuned” the strength of the motion information by changing the proportion of dots that moved in the same direction (i.e., percentage correlation of motion). (B) The “neurometric function” (solid dots and solid fitted curve) an individual neuron that closely paralleled the “psychometric function” of the animal's behavioral responses (open dots and dashed fitted curve). (C) The close parallels between the neurometric versus psychometric functions held across different animal subjects. The dashed line notwithstanding indicates the best fitting linear trend. (D) Demonstration of a causal relationship between the responses of individual neurons and the animal's percepts using microstimulation. The psychometric function of an animal with or without microstimulation (solid dots and fitted line and open and dashed fitted line, respectively). See text and (37,236) for details. Adapted, with permission, from (37).
	Figure 24. A schematic illustration of correlation, decorrelation, and sparsening of the responses at the population level during the initial rapid transient responses (panel A), or at later stages (panels B‐E). Each panel shows a highly idealized “population” consisting of four neurons (circles). Each quadrant of a given circle denotes the response of the neuron to a given stimulus, color‐coded according to the color scale at bottom left. See text for details. Adapted, with permission, from (140).
	Figure 25. Visual cliff demonstrates development of depth perception. (A) The visual cliff is a laboratory apparatus that helps test depth perception in human infants and animals. It consists of an actual cliff covered with a sturdy but transparent plexiglass (112). The cliff is textured with a high‐contrast checkerboard pattern, so that the cliff is clearly visible through the plexiglass. An infant called by his mother from the opaque side of the apparatus readily crawls to her (112). On the other hand, he is reluctant to venture over the perceived cliff (panel B). Even when the infants know by patting the glass that it is solid, they still tend to be reluctant to cross. Infants’ decision as to whether or not to cross the visual cliff are also influenced by whether the gestures of the parent are encouraging, neutral, or discouraging (268). Such behaviors show sophisticated inferences based on a joint evaluation of various depth cues, risks, and rewards. Studies show that healthy human infants have such depth perception even before they are able to crawl (49). The visual cliff effect has been reported in many mammalian species (94). For a video of visual cliff effect, see https://www.youtube.com/watch?v=p6cqNhHrMJA.
	Figure 26. fMRI in young human infants. (A) Renderings of what infants at various ages are likely to see when they view a teddy bear. (B) Visual responses in the neonate. Visually responsive regions are located in the anterior aspect of the calcarine sulcus in either hemisphere. Moreover, the visually evoked responses are lower compared to the periods of rest. (C) Visual responses in a different 5‐month old infant. Visual stimulation activates a much posterior aspect of the calcarine sulcus. Also, the visually evoked responses are higher than the responses during rest. Panels B and C are courtesy of Dr. Ernst Martin (215) and reproduced with permission.
	Figure 27. Preferential responses in the human cerebellum during high‐level cognitive task. Panel A shows the differential PET responses in a heatmap format, where brighter colors represent greater response. Panel B schematically summarizes the regions (red squares) that showed the task‐dependent preferential response. Note that the responses are highly lateralized. Adapted, with permission, from (256).
	Figure 28. Hemineglect. This figure shows the results from a drawing test (119,135) from a single patient with left hemineglect, resulting from a localized lesion in the right temporal lobe. The patient was asked the draw the dial of a clock. In most clinical cases, lesions tend to be less circumscribed and more widespread than in the patient whose drawing is shown here. Hence, drawings by most hemineglect patients tend to be much more complex, and less clear‐cut, than the “text book” case shown in this figure (see (119,135) for reviews). Figure courtesy of Scholarpedia.
	Figure 29. Our evolving understanding of multimodal anatomical connections with the visual system. (A) Traditional view of the cortical anatomy of the primate brain recognized very few areas with multimodal anatomical connections (colored areas). (B) A more modern scheme of the cortical anatomy of multisensory areas. Colored areas represent regions where anatomical and/or electrophysiological studies have demonstrated multisensory interactions. Dashed gray outlines represent opened sulci. See (109) for details, including the criteria used for determining multimodal connectivity at the anatomical level. Adapted, with permission, from (109).
	Figure 30. Visual‐haptic object processing activates lateral occipital complex (LOC) in the occipitotemporal pathway. Ahmedi and colleagues (8) compared BOLD responses to four conditions: visual objects, somatosensory (or haptic) objects, visual textures, and haptic textures. Statistical map of the contralateral hemisphere from a single subject are shown in panel A (3D folded view), panel B (inflated view of the same hemisphere), and panel C (flattened view of the same hemisphere). Bottom, BOLD responses to the four conditions are shown in the somatosensory cortex (bottom left) and LOC in the occipitotemporal junction (bottom right). Col S, collateral sulcus; Cal S, calcarine sulcus; CS, central sulcus; IPS, intraparietal sulcus; lateral S, lateral sulcus; STS, superior temporal sulcus. Adapted, with permission, from (8).
	Figure 31. Color‐graphemic synesthesia. (A) (left) A stimulus that can elicit synesthesia in color‐graphemic synesthetes. (A) (right) A rendition of what the synesthete likely to have perceived. Note that, since color‐graphemic synesthetes tend to perceive different numbers as different colors, the triangle made up of 2’s stands out, or “pops out,” perceptually for them. By contrast, nonsynesthetes perceive all the numbers to be of the same color, so that for them, the triangle is not readily distinguishable from the background. (B) Neural responses during color‐graphemic synesthesia as measured by fMRI. BOLD responses to graphemic stimuli were contrasted against the responses to nongraphemic stimuli in synesthetes (left) and control subjects and results are rendered on inflated, bottom‐up views of brains of representative subjects. Both control subjects and synesthetes showed common activation of the “grapheme region” (Gr). In addition to this common activation, graphemes activated the color selective areas of the retinotopic region V4 (hV4) in synesthetes but not in nonsynesthetes. See (156) for details. Adapted, with permission, from (156).

Further Reading

Li Z. Understanding vision: Theory, models, and data. New York, NY: Oxford University Press, 2014.

Mitra P and Bokil H. Observed brain dynamics. Oxford; New York: Oxford University Press, 2008.

Nelson CA and Collins ML. Handbook of developmental cognitive neuroscience. Cambridge, Mass.: MIT Press, 2008.

Sporns O. Discovering the human connectome. Cambridge, Mass.: MIT Press, 2012.

Palmer SE. Vision science: photons to phenomenology. Cambridge, Mass.: MIT Press, 1999.

Rolls ET and Deco G. The noisy brain: stochastic dynamics as a principle of brain function. Oxford; New York: Oxford University Press, 2010.

Seung S. Connectome: how the brain's wiring makes us who we are. Boston: Houghton Mifflin

Harcourt, 2012.

Sporns O. Discovering the human connectome. Cambridge, Mass.: MIT Press, 2012.

Werner JS and Chalupa LM. The new visual neurosciences. Cambridge, MA: MIT Press, 2013.

Wolfe JM, Kluender KR, Levi DM, Bartoshuk LM, Herz RS, Klatzky RL, Lederman SJ, Merfeld DM. Sensation & Perception. Sunderland, MA: Sinauer Associates, 2014.

Teaching Material

J. Hegdé. Neural Mechanisms of High-Level Vision. Compr Physiol 8: 2018, 903-953.

Didactic Synopsis

Major Teaching Points:

• Visual perception is essentially an inferential process, in that the visual system infers the likely interpretation of a given image by evaluating the various underlying statistical (i.e., probabilistic) factors. This is not to say that the inferences are necessarily optimal or that they are made consciously.
• For last several decades, neurophysiological research has been guided by the implicit assumption that the goal of visual processing is to help construct a veridical internal representation of the external visual world. This, along with earlier methodological difficulties in monitoring the activity of large number of neurons in multiple areas, led to a couple of decades of neurophysiological research focused on delineating functional specialization of individual visual areas, which has to do with figuring out “which visual area does what” in representing the external visual world.
• More recent research, however, suggests that a “goal-oriented connectomic” view is a better framework for understanding visual processing as a sensory process wherein large number of different brain areas act, not as individual areas but as parts of as a larger network, to implement the animal's behavioral goal at hand.
• The networks that implement a given behavior vary dynamically and probabilistically depending on various factors, including the sensory information, behavioral goal, possible rewards or punishments, etc.
• Vision is not a single process, but a collection of processes.
• Even though vision is the dominant sensory modality for humans and other primates, it does not function in isolation, but interacts actively with other senses and the brain systems that implement behavior.

Didactic Legends

The figures—in a freely downloadable PowerPoint format—can be found on the Images tab along with the formal legends published in the article. The following legends to the same figures are written to be useful for teaching.

Figure 1 The abstract painting Interchanged (1955) by Willem de Kooning. At a reported purchase price of US $300 million, it is one of the most expensive paintings in the world. See text for additional details.

Teaching points: Low-level features of an image refer to basic properties of image elements, such as the color or brightness of the various image regions, the orientation of the various lines and surfaces, etc. High-level properties of the image refer to the larger, more holistic import of the image. High-level understanding of an image is more than the sum of understanding all of its low-level features. Abstract paintings such as this help illustrate the distinction between low- and high-level visions especially well, because the painting is deliberately constructed so as to maximally dissociate the two sets of visual processes. However, in most real-world images, the processing of low-level features leads to a high-level level understanding so readily that it can be hard to grasp the distinction between the two.

Figure 2 The importance of recurrent processing in visual perception. When viewed for the first time, this two-tone “Mooney” image appears to be an unrecognizable pattern of black and white blobs. That is, it is hard to interpret this image based on sensory information (i.e., bottom-up or feed-forward processing) alone. However, after viewing a full grayscale or color counterpart (for which see Figure 3A—which, being smaller in size, has no pixel-to-pixel correspondence with the above image), the Mooney image becomes easy to interpret. Note that viewing the disambiguating image rapidly, drastically, and enduringly alters our perception of the Mooney image, although the Mooney image itself remains physically unchanged. Feed-forward theories of vision cannot help explain such phenomena. Recurrent (or “reentrant”) neural signals bring to bear such top-down influences as prior knowledge and the behavioral context to help constrain the interpretation of the visual image. Learning to interpret Mooney images can be understood as an extreme case of knowledge-mediated disambiguation that is part and parcel of normal visual perception (51, 82, 125, 142).

Teaching points: Vision is not a deterministic process; the same visual image can produce dramatically different perceptual outcomes depending on the perceptual state of the viewer. Two-tone, “Mooney” images such as this help demonstrate this by showing that perception can change dramatically in the total absence of any changes in the sensory information. Such perceptual changes would not occur if the sensory input solely determined the perceptual outcome.

Figure 3 Some complexities of natural images and the information processing required to making sense of them. Panels A through D show real-world scenes that help illustrate some of the complexities of such scenes. For instance, there are multiple objects of the same type in each picture. They all vary greatly in image size, illumination, shadows, occlusion, viewpoint, etc. But the visual system recognizes them as objects of the same kind. In other words, the brain must be able to discard a variety of image features as irrelevant to recognizing the images. Note that, in case of each scene in this figure, our understanding of the scene evolves over time. Note also that the objects, their spatial relationships, and even the semantic similarities and differences among them are such that the percept that each scene elicits is more than just the sum of the parts. In addition, each picture contains implicit cues to motion and/or depth that static, 2D pictures such as these do not do justice to. In fact, natural scenes differ from each other and from the relatively simple stimuli used in many a vision study, such as a sinusoidal grating on a neutral gray background, in myriad ways. Studies have shown that the visual system is adapted to contend with the statistics of natural scenes (307), so that neural responses to “artificial” stimuli may provide a substantially different and potentially misleading picture of how the brain works. On the other hand, note that our visual system performs very well in recognizing scenes with “unnatural” statistics, such as the street scene in panel C and the indoor scene in panel D, even though it evidently did not encounter the “unnatural” statistics of human-made objects until quite recently on the evolutionary time scale. This is because the brain is quite good at adapting to a variety of nonoptimal inputs.

Teaching points: Given the complexity of the visual world, visual perception cannot be brought about by feedforward processing alone. Feedback and lateral processing bring to bear information about the sensory and behavioral context, prior knowledge, etc. on visual processing. These images also help demonstrate the fact that vision is not a unitary process but a multifaceted one, because the precise nature of the visual processing depends critically, among other things, on the nature of the scene and of the task at hand.

Figure 4 Visual system can often recognize objects with great precision with very little information. A veridical description of the object, even if it were possible, is not necessary. (A) In the picture on the left, very little of the dog is visible. But we have little trouble recognizing the dog. Many probably can even readily name the breed of the dog. (B) Few would have trouble recognizing the person from the picture. (C) The tell-tale hat. For most readers, this picture simply shows a hat hung on some plumbing. But this famous picture, on the cover of the very first issue of Physics Today in May 1948, showed the famous “pork pie” hat of J. Robert Oppenheimer, the father of the atomic bomb. Therefore, the picture was highly topical and was readily understood at the time. But for most of us living today, this picture requires either some historical knowledge or explanation. Thus, our perception of a visual image is influenced by a great many nonvisual factors. Neurophysiological understanding of such influences on cognition represents a major challenge.

Teaching points: Visual system can often recognize objects with great precision with very little information. This is because, as noted earlier, the visual system is adept at coming up with the best possible inference based on all available information. Thus, when there is very little sensory information available in a given visual scene, the visual system “fills in the gaps” using its prior knowledge of the visual world.

Figure 5 There is more to vision than feed-forward processing. This picture of an unknown little girl was taken by South African photojournalist Kevin Carter in the famine-stricken South Sudan in March 1993. The girl had reportedly collapsed from weakness on her way to a United Nations food center. (The Vulture and the Little Girl by Kevin Carter. Pulitzer Prize for Feature Photography, 1994. Reproduced with permission.) Note that feed-forward processing by itself would completely miss the import of the picture.

Teaching points: Feed-forward processing does not fully account for how vision works. For instance, feedforward processing by itself would completely miss the point of this picture.

Figure 6 (A) A self-driving car. (B) Two instances of the CAPTCHA (or Completely Automated Public Turing test to tell Computers and Humans Apart) internet device designed to prevent automated logins (121, 326). In the example on the left, the website asks the user to perform a straightforward object categorization task, namely distinguish pictures of people wearing glasses from pictures of people without glasses. This tends to be quite successful in preventing machines from logging on to the website. However, this success is not so much because it would be all that difficult nowadays to train a suitably designed computer program to distinguish the two categories of human faces (see, e.g., (2, 191)), but essentially because, at present, such programs tend to be hyperspecialized, and tend not to generalize beyond their training sets to other visual objects or tasks (121, 199). That is, such an “intelligent” program, once trained to tell aforementioned types of faces apart, can be readily stumped at present by rather slight changes in the underlying categories (panel B, right), or the task (not shown) (2, 191). However, recent research suggests that the problem of overspecialization of intelligent machines is likely to straightforwardly surmountable, so that CAPTCHA strategies such as the one illustrated in this panel are unlikely to be effective for long. That is, machine vision is getting ever better at mimicking human high-level vision (199).

Teaching points: Visual abilities and limitations of modern computers help illustrate some of the high-level visual faculties needed to operate in the real world, and the underlying computational issues. Most of the current limitations of machine vision systems lie in the realm of high-level vision and semantic understanding of images, that is, understanding the larger meaning of images, such as that in Figure 5, beyond merely recognizing the various objects in the image. But machine vision is rapidly catching up with high-level capabilities of humans.

Figure 7 An outline of Marr's theory of visual processing. See text for details.

Teaching points: Marr's influential model of visual perception, now largely disproven, posits that visual processing proceeds in a feed-forward, modular, and hierarchical manner and results in a veridical internal representation of the visual world.

Figure 8 Visual anatomical hierarchy in the macaque monkey described by Felleman and Van Essen, 1991 (98). Each colored rectangle represents a distinct cortical visual area. The open rectangles at top denote higher cortical areas that are not considered primarily visual area. The two gray rectangles at bottom denote the retinal ganglion cells (RGC) and the lateral geniculate nucleus (LGN). Lines connecting the cortical areas denote interconnections, usually reciprocal, between a given pairs of areas. The various brain regions are represented in a tiered, or hierarchical, fashion based on objective anatomical criteria, most important of which are the laminar patterns of feed-forward and feedback connections (also see (98, 216, 342)). For additional details and abbreviations, see Felleman and Van Essen, 1991 (98). An alternative formulation of the hierarchy, originally formulated by Mishkin, Gross, and their colleagues (130) (also see (78, 129)) is largely similar, but it parcels the cortex into many fewer areas and recognizes fewer interconnections. Also, some the visual area names are different in this scheme. For instance, areas AIT and CIT (anterior and central inferotemporal areas, respectively) in the Felleman and Van Essen scheme are equivalent to area TE (temporal area) in the Gross et al. scheme, and area PIT (posterior inferotemporal area) in the Van Essen scheme is equivalent to area TEO (temporooccipital area) in the Gross et al. scheme. Both schemes are used in this review, based on the scheme used by the study in question. Reproduced, with permission, from (98).

Teaching points: From an anatomical viewpoint, visual areas of the brain can be arranged as a richly interconnected hierarchy. This does not necessarily mean that the visual information processing for which it provides the neural substrate is also hierarchical. This is ultimately because information does not flow in a single, feed-forward direction within the anatomical hierarchy. That is, the information does not always travel “up” the anatomical hierarchy, but instead flows in multiple different directions.

Figure 9 Our evolving understanding of the functional organization of the primate visual system. Panels A and B depict our view of the two main visual processing streams in the macaque brain in the early 1980s. (A) The dorsal and ventral visual pathways as originally formulated by Mishkin and colleagues in 1993 is denoted by solid arrows (OB → OA → PG pathway being the anatomically dorsal pathway, and OB → OA → TEO → TE being the ventral pathway) (225). The continuation of these pathways to FD_Δ and FDv, respectively (dashed arrows) represented the collective outcome of many additional studies. (B) Anatomical locations of the various regions, some renamed according to a more modern naming convention (188). (C) Key changes in the receptive field properties of neurons along the ventral pathway. Areas are color-coded as in panel B. Panel D summarizes our understanding of the same pathways some 30 years later, as summarized by Kravitz and colleagues in 2013 (188). Note that it is clear that the two pathways have turned out to be far more interconnected than previously envisioned. What is lost in terms of pedagogical simplicity is more than made up for by the nuance and granularity of this network picture, presaged decades ago by Kerrigan and Maunsell (220). Self-evidently, far more remains to be learned, including how these networks interact with other networks in the brain and subserve behavior. Adapted, with permission, from (188). Human brain (not shown) is also purported to have two evolutionarily homologous processing pathways, although there is even less empirical information to support this notion.

Teaching points: Our understanding of the functional organization of the visual system has changed drastically in the last few decades. The evolution of our understanding of the dorsal versus ventral visual pathways is a case in point. Initial studies seemed to support the conventional hierarchical, modular view of visual information processing. But subsequent research has revealed extensive anatomical interconnections and functional interactions among the various visual areas within and across the two pathways, and the computational consequences of these interactions have become clearer. The inescapable conclusion is that the visual system works as an intricate and dynamic network, and not as a bottom-to-top conveyer belt.

Figure 10 What's to infer? Isn't it all there in the image? The answer is no. Retinal image is simply a 2D pattern of image intensities. Image intensities of a particular image are represented in panel A as a color-coded surface plot, where the height and color of a point denotes the image intensity at that point. Note that when the image is represented in this fashion, it makes no sense to us. But when the same image is represented as corresponding variations in image intensities (panel B), we readily recognize it as an image of a brook in the woods. However, the information in the two representations is exactly the same. The difference is that, in panel B, the image is in an input “format” that our eyes can process. Beyond that, the inferential processes to “make sense” of the pattern of intensities are exactly the same.

Teaching points: Visual processing is a fundamentally inferential process. This is because the retinal image is inherently ambiguous. The retinal image by itself “makes no sense” unless it is appropriately interpreted by subsequent visual processing.

Figure 11 Visual illusions help illustrate the inferential nature of visual perception. They also demonstrate that the inference need not be a deliberate, volitional, or conscious process. (A) Ames room. Two people stand in opposite corners of the room. One appears to be much taller than the other, even though they are roughly the same height. Instead, it is the room that is distorted to produce this illusion. Picture courtesy of Ian Stannard, Flickr. Reproduced with permission. (B) Hollow-mask illusion. This picture shows the front of the mask (right) and the hollow back of the mask (left). Nonetheless, both look like normal, convex faces. For a video demonstration of the hollow mask illusion, see https://www.youtube.com/watch?v=sKa0eaKsdA0. Note that, in this video, the mask appears to flip its direction of rotation at the same time the other side of the mask begins appearing. Theoretical studies show that visual illusions are often perfectly rational inferences given the evidence (see, e.g., (108, 357).

Teaching points: Visual illusions provide some of the most self-evident demonstrations that visual perception is fundamentally inferential in nature wherein the brain takes in all the available, pertinent information and comes up with an interpretation that fits the information at hand. Note that knowing that the percept is illusory does not make it go away, nor can we easily make the illusory percept disappear by trying to achieve the nonillusory percept, that is, that the persons in panel A are of roughly the same height, or that the face on the right on panel B is really concave. This is because of the brain's proclivity to go with what it deems to be a rational inference given the totality of the evidence. It is often impossible for the mind to veto this inference.

Figure 12 The spatiotemporal domain of the methods available for the study of the functional organization of nervous system in 2014, compared to the methods available in 1988 (inset). Each colored region represents a range of spatial and temporal resolutions for a given method. Open regions represent measurement techniques; filled regions, perturbation techniques; EEG, electroencephalography; MEG, magnetoencephalography; PET, positron emission tomography; VSD, voltage-sensitive dye; TMS, transcranial magnetic stimulation; 2-DG, 2-deoxyglucose. Redrawn, with permission, from (290).

Teaching points: The repertoire of available neurophysiological techniques has improved greatly within the last few decades. The repertoire of techniques for analyzing neurophysiological data also has undergone phenomenal improvements, as has our ability to cross-compare data obtained from different techniques in humans as well as animals.

Figure 13 Neuronal responses in monkey visual area V1 during binocular rivalry. See text for details. Adapted, with permission, from (33, 201, 206).

Teaching points: Visual perception can change even when the visual stimulus itself remains unchanged. Binocular rivalry is one such case. Such perceptual changes that in the total absence of any change in the visual stimulus have often been used as experimental tools for dissociating the confounding effects of sensory stimulation on the visual percept.

Figure 14 Figure-ground segregation and perceptual organization. (A) How many circles can you see in this image? In this image, referred to as the Coffer Illusion, you should be able to see 16 circles. Figure courtesy of Dr. Anthony Norcia, Stanford University. Reproduced with permission. (B) Camouflage is an extreme case of figure-ground segregation, where the object of interest is hard to recognize even when “in plain view.” This image shows two variants of the pepper moth Biston betularia, one black and the other with light peppered coloring. The black variant is effectively camouflaged against colored tree bark whereas the light variant is easy to recognize (i.e., it “pops out”). The opposite is true when the same two variants are seen against a background of light bark and lichens. The black variant emerged for the first time in the industrial midlands of Britain in the 19th century, where tree barks were turning black with industrial soot. Soon the black variants became the more common variant, because the predators of the moths had much greater success breaking the camouflage of the lighter variants because it was harder for the lighter variants to find light backgrounds to camouflage themselves against. This was an instance of the prey “gaming” the predators’ high-level visual faculties to enhance its own survival (179, 334).

Teaching points: Both figure-ground segregation (i.e., the perceptual process whereby an object of behavioral interest is perceptually distinguished from the background) and perceptual grouping (i.e., the complementary of process of mentally grouping related parts of the scene), are important aspects of high-level understanding of visual scenes.

Figure 15 Selectivity for faces in the macaque superior temporal polysensory area (STP). (A) Anatomical location of area STP (yellow highlight). (B) Responses of a single STP neuron to various visual objects, including variations in face stimuli. Note that, among the stimuli tested, the neuron responds best to a face with all the key facial features (left column, second stimulus from top). Cutting this stimulus into 16 pieces and showing the pieces in a shuffled order essentially eliminated the response (right column, third stimulus form top). The icon at bottom right denotes the size and visual field location of the receptive field. C, contralateral visual field (i.e., contralateral to the recording location). I, ipsilateral. Such neurons with large receptive fields that span both the visual hemifields are common in the visually responsive areas of the central and anterior temporal cortex. Adapted, with permission, from (40).

Teaching points: Neurons in the temporal area show remarkable selectivity for faces. This figure shows one of the most striking and widely known demonstrations of such narrow selectivity.

Figure 16 Coarse-to-fine tuning of shape categories in the macaque IT. The stimulus set consisted of 38 stimuli, a subset of which is shown in the inset. The global shape categories (inset, vertical axis) consisted of human faces, monkey faces, and geometric shapes. The fine categories (inset, horizontal axis) consisted of the various facial identities and expressions. The plots show the cumulative information transmission rate of a sample of IT neurons about both the global and fine categories (red and blue lines, respectively). The thick horizontal line along the x axis denotes the stimulus duration. Adapted, with permission, from (140, 323).

Teaching points: Responses of many face-selective neurons in the monkey temporal cortex change in a coarse-to-fine fashion in time. This figure shows one of the most clear-cut demonstrations of this effect. However, it is important to remember that responses of most neurons in most areas tend to change over time in a much more complex fashion that rarely lends itself to pithy slogans.

Figure 17 Grid cells help represent the external visual space. Grid cells were originally reported in rats, but have since been found in many species, including monkeys. (A) Responses of a single grid cell in the entorhinal cortex of the rat. Left, black lines denote the trajectory of a rat freely moving in a box. The cell fired spikes (red dots) when the rat was at specific locations within the box. These locations were organized in a grid-like fashion that spanned the box. Right, the firing rates of the cell represented as a heatmap, where “warmer” colors denote higher firing rates. Adapted, with permission, from (134). (B) Responses of a single grid cell in the entorhinal cortex of the macaque monkey. Unlike the rat referred to in panel A, the monkey was stationary. It sat in a primate chair with its head held steady, but freely moved its eyes as it looked at real-world pictures (not shown). Left, red dots denote the locations in a picture (not shown) that the monkey fixated, or gazed steadily at for a brief period. Center, firing rate of the grid cell shown in heatmap format. The scale bars at bottom each denote 6° of visual angle. Adapted, with permission, from (180).

Teaching points: Grid cells in the entorhinal cortex help represent the external visual space. Such neuronal representations of the external space appear to be quite common across various mammalian species.

Figure 18 Coding of visual context. The classical receptive field (CRF) of a given neuron is the portion of the visual field in which the neuron is most responsive to visual stimuli. The surrounding region in which visual stimulation modulates the responses to CRF stimulation is referred to as the nonclassical receptive field (nCRF) or nonclassical surround. This figure shows one of the earliest demonstrations of the modulatory effect of nCRF on CRF, in this case the response of a single neuron in macaque MT. Individual neurons in MT often response best when the stimulus moves in a particular direction, often referred to as its preferred or optimal direction. The neuron shown responded best when the dots in the CRF moved horizontally from left to right. CRF is denoted by the dashed rectangle in the icons at top. (Left panel) Modulatory effect of stationary surround on motion stimuli in the CRF. The direction of the movement of dots in the CRF was systematically varied, while the dots in the nCRF were held stationary. (Right panel) Modulatory effect of surround motion on motion stimuli in the CRF. The responses of the same neuron when the dots moved in its optimal direction, while the direction of the dots in the nCRF was systematically varied. Note that the neuron's responses vary systematically (i.e., the responses are “tuned”) with respect to both CRF motion, and motion in both the center and surround. Thus, neuron can convey information of the motion “context,” or motion of a given moving object relative to nearby stationary or moving objects. Adapted, with permission, from (5).

Teaching points: The responses of visual neurons are often modulated by the visual context, that is, by visual stimuli outside the neuron's classical receptive field. This figure shows one of the earliest demonstrations of the effects of stimulating the nonclassical surround on the stimuli within the classical receptive field.

Figure 19 Corticostriatal loops in human brain. (A) Four distinguishable but mutually overlapping loops are usually recognized (colored labeled arrows), based primarily on the types of tasks in which they play prominent roles. Cortical inputs arrive largely via the striatum and ultimately are directed back into the cortex via the thalamus. The (ultimate) cortical output of the basal ganglia reaches largely to the same cortical areas that give rise to the initial inputs to the basal ganglia. The visual loop is known to play a prominent role in the learning of visual object categories, but during object categorization tasks using learned categories, the executive loop also plays a prominent role. Corticostriatal loops in the nonhuman primate brain are largely similar (not shown). Adapted, with permission, from (287); also see (222, 250). (B) A more detailed circuit map of the visual loop shows the flow of information within the loop. GPe: Globus pallidus, external portion. GPi: Globus pallidus, internal portion. SNr: Substantia nigra pars reticulata. SNc: Substantia nigra pars compacta. STN: Subthalamic nucleus. VTA: Ventral tegmental area. Adapted, with permission, from (286).

Teaching points: Corticostrial loops refer to a circular pathway in which information flows from the cortex to the various parts of the striatum and back to the cortex. Corticostriatal loops play an important role in perception and perceptual learning. This figure illustrates the four main corticostriatal loops in the human brain and the general connectivity pattern of each loop.

Figure 20 An illustration of Granger causality. This figure shows cause and effect relationship between two hypothetical neural responses (top and bottom rows, respectively). The cause-and-effect relationship exists throughout the responses, but is most readily apparent by visual inspection of those portions of the responses where the response is prominently modulated (arrows). Granger causality uses the entire length of both responses to quantitatively measure the cause-and-effect relationship even when the relationship may be too subtle or complex to be visually evident. In the present case, the response shown in the top row is said to “Granger-cause” the response in the bottom row. Note that the term “Granger causation” denotes an inferred cause-and-effect relationship, which may or not include direct causation. Thus, the concept of Granger causality is in some respects narrower, and in some other respects broader, than the concept of direct causation (13, 29, 224). But in either case, the cause must necessarily precede the effect.

Teaching points: Granger causality is a powerful analytical method which can quantify causal relationship between two or more time-domain data (technically known as time series data). It was originally developed by Clive Granger (1934-2009; Novel Prize in Economics, 2003) to mine for, and measure, cause-and-effect relationships in time series data in economics, such as two or more stock prices. Granger causality is one of the many instances in which neurophysiology has hugely benefited by eagerly embracing experimental and analytical approaches from other fields of research.

Figure 21 Resting state brain networks are very similar to brain networks active during tasks. (A) Smith and colleagues (309) extracted 20 mutually independent patterns of activation in the resting state networks (RSNs) from a database of 36 adult human subjects using the independent components analysis (ICA) (319). Brain regions identified by each of these ICAs can be thought of as an independent network. Smith and colleagues then compared RSN ICAs with ICAs of task-activated networks from nearly 30,000 subjects from the BrainMap (BM) database (brainmap.org). Comparisons for ten most informative ICAs are shown side by side in this panel. For each ICA, activation is shown in a color-coded format for a coronal, sagittal, and horizontal section (top, middle, bottom row, respectively). (B) Smith and colleagues (309) then analyzed the extent to which the top ten of the RSN ICAs play a role in various types of behavioral tasks (or “behavioral task domains” defined by the BrainMap database). Higher color values denote a correspondingly larger role by the given network in a given task. Note that each given type of task recruits different RSNs to different extents. Conversely, each RSN is active during multiple, different task paradigms, with the degree of participation varying according to the task. Thus, the RSNs represent a repertoire that the brain recruits and employs to various degrees depending on the task at hand. For details and some important caveats, see (309). Adapted, with permission, from (309).

Teaching points: The brain networks active during tasks partially overlap resting state networks in nuanced, meaningful ways. This figure illustrates the brain responses during a given active task characterized by the given weighted combination of the resting state networks that are recruited in a task-dependent fashion.

Figure 22 Networks in human brain at rest. Nodes denote the center of mass of the corresponding brain regions, and the edges (i.e., colored lines) represents intrinsic connections between a pair of regions identifiable in the resting brain. Blue nodes represent the “rich club” brain regions, which are well-connected brain regions that are well-connected with other well-connected brain regions. Gray dots denote nonrich club regions. Red lines denote connections between rich club regions. Orange lines denote “feeder” connections that connect a rich club region with a nonrich club region. Yellow lines denote “feeder” connections that connect a nonrich club region with another nonrich club region. Adapted, with permission, from (339). Note that this figure does not show subcortical or cerebellar networks, which decidedly play crucial roles in brain function.

Teaching points: Brain areas are organized as densely interconnected networks. The pattern of interconnections among brain regions tend to follow a “rich club” pattern, where well-connected regions tend to be interconnected among each other, rather than with poorly interconnected regions.

Figure 23 Neuronal responses in monkey middle temporal area (MT) reflect perceptual decisions in awake, behaving monkeys. (A) Stimuli and task paradigm. The studies by Newsome and colleagues (37, 236, 278) used dots moving either in the preferred direction of the cell under study, or in the opposite (“null”) direction, depending on the trial. The investigators “tuned” the strength of the motion information by changing the proportion of dots that moved in the same direction (i.e., percentage correlation of motion). (B) The “neurometric function” (solid dots and solid fitted curve) an individual neuron that closely paralleled the “psychometric function” of the animal's behavioral responses (open dots and dashed fitted curve). (C) The close parallels between the neurometric versus psychometric functions held across different animal subjects. The dashed line notwithstanding indicates the best fitting linear trend. (D) Demonstration of a causal relationship between the responses of individual neurons and the animal's percepts using microstimulation. The psychometric function of an animal with or without microstimulation (solid dots and fitted line and open and dashed fitted line, respectively). See text and (37, 236) for details. Adapted, with permission, from (37, 236) and (37, 236).

Teaching points: Individual neurons in many visual areas reliably represent the perceptual outcome. This figure shows one of the earliest and most influential demonstrations of this principle. Perhaps even more importantly, the various techniques showcased in this study also paved for the study of less striking (and far more common) scenarios where individual neurons only carry limited information about the perceptual outcome.

Figure 24 A schematic illustration of correlation, decorrelation, and sparsening of the responses at the population level during the initial rapid transient responses (panel A), or at later stages (panels B-E). Each panel shows a highly idealized “population” consisting of four neurons (circles). Each quadrant of a given circle denotes the response of the neuron to a given stimulus, color-coded according to the color scale at bottom left. See text for details. Adapted, with permission, from (140).

Teaching points: Sparsening refers to a widespread neurophysiological phenomenon where a given visual stimulus elicits an initial transient response from many neurons in a given population, but most of the responses return to background levels shortly thereafter. Decorrelation is a related temporal dynamic phenomenon where different neurons in a population respond differently to a given stimulus. Sparsening and decorrelation are somewhat complementary phenomena, because increasing sparseness actually increases correlation (or reduces decorrelation). This figure schematically illustrates some of the many different ways in which response sparsening can occur within a neuronal population.

Figure 25 Visual cliff demonstrates development of depth perception. (A) The visual cliff is a laboratory apparatus that helps test depth perception in human infants and animals. It consists of an actual cliff covered with a sturdy but transparent plexiglass (112). The cliff is textured with a high-contrast checkerboard pattern, so that the cliff is clearly visible through the plexiglass. An infant called by his mother from the opaque side of the apparatus readily crawls to her (112). On the other hand, he is reluctant to venture over the perceived cliff (panel B). Even when the infants know by patting the glass that it is solid, they still tend to be reluctant to cross. Infants’ decision as to whether or not to cross the visual cliff are also influenced by whether the gestures of the parent are encouraging, neutral, or discouraging (268). Such behaviors show sophisticated inferences based on a joint evaluation of various depth cues, risks, and rewards. Studies show that healthy human infants have such depth perception even before they are able to crawl (49). The visual cliff effect has been reported in many mammalian species (94). For a video of visual cliff effect, see https://www.youtube.com/watch?v=p6cqNhHrMJA.

Teaching points: Visual cliff is a classic demonstration that depth perception is a mental faculty that develops quite early during postnatal development in mammalian species. Visual cliff studies helped establish the field of systematic characterization of brain function during early development.

Figure 26 fMRI in young human infants. (A) Renderings of what infants at various ages are likely to see when they view a teddy bear. (B) Visual responses in the neonate. Visually responsive regions are located in the anterior aspect of the calcarine sulcus in either hemisphere. Moreover, the visually evoked responses are lower compared to the periods of rest. (C) Visual responses in a different 5-month old infant. Visual stimulation activates a much posterior aspect of the calcarine sulcus. Also, the visually evoked responses are higher than the responses during rest. Panels B and C are courtesy of Dr. Ernst Martin (215) and reproduced with permission.

Teaching points: Visual responses in very young human infants are drastically different from those in human adults. Thus, brain development is not just an elaboration of a functional organizational blueprint that exists at birth.

Figure 27 Preferential responses in the human cerebellum during high-level cognitive task. Panel A shows the differential PET responses in a heatmap format, where brighter colors represent greater response. Panel B schematically summarizes the regions (red squares) that showed the task-dependent preferential response. Note that the responses are highly lateralized. Adapted, with permission, from (256).

Teaching points: It was long thought that the cerebellum is primarily involved in fine-tuning of movements. Recent anatomical and neuroimaging studies have revealed that the cerebellum is involved in computations that involve high-level cognitive tasks, including high-level vision. Of particular import to the students of neurophysiology is that what is currently known about this aspect of cerebellar function is likely to be just the tip of the iceberg.

Figure 28 Hemineglect. This figure shows the results from a drawing test (119, 135) from a single patient with left hemineglect, resulting from a localized lesion in the right temporal lobe. The patient was asked the draw the dial of a clock. In most clinical cases, lesions tend to be less circumscribed and more widespread than in the patient whose drawing is shown here. Hence, drawings by most hemineglect patients tend to be much more complex, and less clear-cut, than the “text book” case shown in this figure (see (119, 135) for reviews).

Teaching points: Lesions in various regions of the brain can produce specific deficits in high-level vision. A standard method of clinically characterizing the deficit in a given patient is to show the patient a (often standardized) set of drawings and ask him or her to reproduce the drawings. Close examination of the patient's drawings can reveal much about the specific visual deficits in the given patient. Such drawing tests are typically only a part of a battery of tests that are administered, since the actual cases tend to be highly complex, and often multimodal, so as to warrant multiple different types of tests.

Figure 29 Our evolving understanding of multimodal anatomical connections with the visual system. (A) Traditional view of the cortical anatomy of the primate brain recognized very few areas with multimodal anatomical connections (colored areas). (B) A more modern scheme of the cortical anatomy of multisensory areas. Colored areas represent regions where anatomical and/or electrophysiological studies have demonstrated multisensory interactions. Dashed gray outlines represent opened sulci. See (109) for details, including the criteria used for determining multimodal connectivity at the anatomical level. Adapted, with permission, from (109).

Teaching points: Brain is multimodal; vision does not act alone. This figure helps illustrate our rapidly developing appreciation of just how multimodal the brain is. Even regions that we once thought were exclusively visual areas are now known to have strong interactions with other sensory modalities.

Figure 30 Visual-haptic object processing activates lateral occipital complex (LOC) in the occipitotemporal pathway. Ahmedi and colleagues (8) compared BOLD responses to four conditions: visual objects, somatosensory (or haptic) objects, visual textures, and haptic textures. Statistical map of the contralateral hemisphere from a single subject are shown in panel A (3D folded view), panel B (inflated view of the same hemisphere), and panel C (flattened view of the same hemisphere). Bottom, BOLD responses to the four conditions are shown in the somatosensory cortex (bottom left) and LOC in the occipitotemporal junction (bottom right). Col S, collateral sulcus; Cal S, calcarine sulcus; CS, central sulcus; IPS, intraparietal sulcus; lateral S, lateral sulcus; STS, superior temporal sulcus. Adapted, with permission, from (8).

Teaching points: Visual-haptic object processing activates lateral occipital complex, a brain region that plays a prominent role in visual processing. This result helps highlight the fact that cross-modal processing is a general principle of brain function, and not a special case scenario.

Figure 31 Color-graphemic synesthesia. (A) (left) A stimulus that can elicit synesthesia in color-graphemic synesthetes. (A) (right) A rendition of what the synesthete likely to have perceived. Note that, since color-graphemic synesthetes tend to perceive different numbers as different colors, the triangle made up of 2’s stands out, or “pops out,” perceptually for them. By contrast, nonsynesthetes perceive all the numbers to be of the same color, so that for them, the triangle is not readily distinguishable from the background. (B) Neural responses during color-graphemic synesthesia as measured by fMRI. BOLD responses to graphemic stimuli were contrasted against the responses to nongraphemic stimuli in synesthetes (left) and control subjects and results are rendered on inflated, bottom-up views of brains of representative subjects. Both control subjects and synesthetes showed common activation of the “grapheme region” (Gr). In addition to this common activation, graphemes activated the color selective areas of the retinotopic region V4 (hV4) in synesthetes but not in nonsynesthetes. See (156) for details. Adapted, with permission, from (156).

Teaching points: Color-graphemic synesthetes perceive numbers as colors. Visual stimuli such as those shown in panel A not only help identify such synesthetes but also help study the brain mechanisms of this type of synesthesia. fMRI results show that color graphemic stimuli activate some brain regions in both synesthetes and controls subjects. But synesthetes show additional activation of the color-selective portion of the retinotopic region V4 (hV4), whereas nonsynesthetes do not show hV4 activation.