What is the concept and function of Iconic Memory in human cognition?

Iconic memory is the visual sensory memory (SM) register pertaining to the visual domain. It is a component of the visual memory system which also includes visual short term memory (VSTM) and long term memory (LTM). Iconic memory is described as a very brief (<1000 ms), pre-categorical, high capacity memory store. It contributes to VSTM by providing a coherent representation of our entire visual perception for a very brief period of time. Iconic memory assists in accounting for phenomenon such as change blindness and continuity of experience during eye saccades. Iconic memory is no longer thought of as a single entity but instead, is composed of at least two distinctive components. Classic experiments including Sperling’s partial report paradigm as well as modern techniques continue to provide insight into the nature of this SM store.

 

Overview

The occurrence of a sustained physiological image of an object after its physical offset has been observed by many individuals throughout history. One of the earliest documented accounts of the phenomenon was by Aristotle who proposed that afterimages were involved in the experience of a dream. Natural observation of the light trail produced by glowing ember at the end of a quickly moving stick sparked the interest of researchers in the 1700s and 1800s. They became the first to begin empirical studies on this phenomenon which later became known as visible persistence. In the 1900s, the role of visible persistence in memory gained considerable attention due to its hypothesized role as a pre-categorical representation of visual information in VSTM. In 1960, George Sperling began his classic partial-report experiments to confirm the existence of visual sensory memory and some of its characteristics including capacity and duration. It was not until 1967 that Ulric Neisser termed this quickly decaying memory store iconic memory. Approximatley 20 years after Sperling’s original experiments, two separate components of visual sensory memory began to emerge: visual persistence and informational persistence. Sperling’s experiments mainly tested the information pertaining to a stimulus, whereas others such as Coltheart performed directs tests of visual persistence. In 1978, Di Lollo proposed a two-state model of visual sensory memory. Although it has been debated throughout history, current understanding of iconic memory makes a clear distinction between visual and informational persistence which are tested differently and have fundamentally different properties. Informational persistence which is the basis behind iconic memory is thought to be the key contributor to visual short term memory as the precategorical sensory store.

 

Components of Iconic Memory

The two main components of iconic memory are visible persistence and informational persistence. The first is a relatively brief (150 ms) pre-categorical visual representation of the physical image created by the sensory system. This would be the “snapshot” of what the individual is looking at and perceiving. The second component is a longer lasting memory store which represents a coded version of the visual image into post-categorical information. This would be the “raw data” that is taken in and processed by the brain. A third component may also be considered which is neural persistence: the physical activity and recordings of the visual system. Neural persistence is generally represented by neuroscientific techniques such as EEG and fMRI.

 

Visible Persistence

Visible persistence is the phenomenonal impression that a visual image remains present after its physical offset. This can be considered a by-product of neural persistence. Visible persistence is more sensitive to the physical parameters of the stimulus than informational persistence which is reflected in its two key properties.

The duration of visible persistence is inversely related to stimulus duration. This means that the longer the physical stimulus is presented for, the faster the visual image decays in memory.

The duration of visible persistence is inversely related to stimulus luminance. When the luminance, or brightness of a stimulus is increased, the duration of visible persistence decreases. Due to the involvement of the neural system, visible persistence is highly dependent on the physiology of the photoreceptors and activation of different cell types in the visual cortex. This visible representation is subject to masking effects whereby the presentation of interfering stimulus during, or immediately after stimulus offset interferes with one’s ability to remember the stimulus.

Different techniques have been used to attempt to indentify the duration of visible persistence. The Duration of Stimulus Technique is one in which a probe stimulus (auditory “click”) is presented simultaneously with the onset, and on a separate trial, with the offset of a visual display. The difference represents the duration of the visible store which was found to be approximately 100-200 ms. Alternatively, the Phenomenal Continuity and Moving Slit Technique estimated visible persistence to be 300 ms. In the first paradigm, an image is presented discontinuously with blank periods in between presentations. If the duration is short enough, the participant will perceive a continuous image. Similarly, the Moving Slit Technique is also based on the participant observing a continuous image. Only instead of flashing the entire stimulus on and off, only a very narrow portion or “slit” of the image is displayed. When the slit is oscillated at the correct speed, a complete image is viewed.

 

Neural Basis of Visible Persistence

Underlying visible persistence is neural persistence of the visual sensory pathway. A prolonged visual representation begins with activation of photoreceptors in the retina. Although activation in both rods and cones has been found to persist beyond the physical offset of a stimulus, the rod system persists longer than cones. Other cells involved in a sustained visible image include M and P retinal ganglion cells. M cells (transient cells), are active only during stimulus onset and stimulus offset. P cells (sustained cells), show continuous activity during stimulus onset, duration, and offset. Cortical persistence of the visual image has been found in the primary visual cortex (V1) in the occipital lobe which is responsible for processing visual information.

 

Informational Persistence

Information persistence represents the information about a stimulus that persists after its physical offset. It is visual in nature, but not visible. Sperling’s experiments were a test of informational persistence. Stimulus duration is the key contributing factor to the duration of informational persistence. As stimulus duration increases, so does the duration of the visual code. The non-visual components represented by informational persistence include the abstract characteristics of the image, as well as its spatial location. Due to the nature of informational persistence, unlike visible persistence, it is immune to masking effects. The characteristics of this component of iconic memory suggest that it plays the key role in representing a post-categorical memory store for which VSTM can access information for consolidation.

 

Neural Basis of Information Persistence

 

The dorsal stream (green) and ventral stream (purple) are shown. They originate from a common source in visual cortex.

 

Although less research exists regarding the neural representation of informational persistence compared to visual persistence, new electrophysiological techniques have begun to reveal cortical areas involved. Unlike visible persistence, informational persistence is thought to rely on higher-level visual areas beyond the visual cortex. The anterior superior temporal sulcus (STS), a part of the ventral stream, was found to be active in macaques during iconic memory tasks. This brain region is associated with object recognition and object identity. Iconic memory’s role in change detection has been related to activation in the middle occipital gyrus (MOG). MOG activation was found to persist for approximately 2000ms suggesting a possibility that iconic memory has a longer duration than what was currently thought. Iconic memory is also influenced by genetics and proteins produced in the brain. Brain-derived neurotrophic factor (BDNF) is a part of the neurotrophin family of nerve growth factors. Individuals with mutations to the BDNF gene which codes for BDNF have been shown to have shortened, less stable informational persistence.

 

Role of Iconic Memory

Iconic memory provides a smooth stream of visual information to the brain which can be extracted over an extended period of time by VSTM for consolidation into more stable forms. One of iconic memory’s key roles is involved with change detection of our visual environment which assists in the perception of motion.

 

Temporal Integration

Iconic memory enables integrating visual information along a continuous stream of images, for example when watching a movie. In the primary visual cortex new stimuli do not erase information about previous stimuli. Instead the responses to the most recent stimulus contain about equal amounts of information about both this and the preceding stimulus. This one-back memory may be the main substrate for both the integration processes in iconic memory and masking effects. The particular outcome depends on whether the two subsequent component images (i.e., the “icons”) are meaningful only when isolated (masking) or only when superimposed (integration).

 

Change Blindness

The brief representation in iconic memory is thought to play a key role in the ability to detect change in a visual scene. The phenomenon of change blindness has provided insight into the nature of the iconic memory store and its role in vision. Change blindness refers to an inability to detect differences in two successive scenes separated by a very brief blank interval, or interstimulus interval (ISS). When scenes are presented without an ISS, the change is easily detectible. It is thought that the detailed memory store of the scene in iconic memory is erased by each ISS, which renders the memory inaccessible. This reduces the ability to make comparisons between successive scenes.

 

Saccadic Eye Movement

It has been suggested that iconic memory plays a role in providing continuity of experience during saccadic eye movements. These rapid eye movements occur in approximately 30 ms and each fixation lasts for approximately 300 ms. Research suggests however, that memory for information between saccades is largely dependent on VSTM and not iconic memory. Instead of contributing to trans-saccadic memory, information stored in iconic memory is thought to actually be erased during saccades. A similar phenomenon occurs during eye-blinks whereby both automatic and intentional blinking disrupts the information stored in iconic memory.

 

Development of Iconic memory

The development of iconic memory begins at birth and continues as development of the primary and secondary visual system occurs. By 5 years of age, children have developed the same unlimited capacity of iconic memory that adults posses. The duration of informational persistence however increases from approximately 200 ms at age 5, to an asymptotic level of 1000 ms as an adult (>11 years). A small decrease in visual persistence occurs with age. A decrease of approximately 20 ms has been observed when comparing individuals in their early 20’s to those in their late 60’s. Throughout one’s lifetime, mild cognitive impairments (MCIs) may develop such as errors in episodic memory (autobiographical memory about people, places, and their contex), and working memory (the active processing component of STM) due to damage in hippocampal and association cortical areas. Episodic memories are autobiographical events that a person can discuss. Individuals with MCIs have be found to show decreased iconic memory capacity and duration. Iconic memory impairment in those with MCIs may be used as a predictor for the development of more severe deficits such as Alzheimer’s disease and dementia later in life.

 

Sperling’s Partial Report Procedure

In 1960, George Sperling became the first to use a partial report paradigm to investigate the bipartite model of VSTM. In Sperling’s initial experiments in 1960, observers were presented with a tachioscopic visual stimulus for a brief period of time (50 ms) consisting of either a 3×3 or 3×4 array of alphanumeric characters such as:

P Y F G
V J S A
D H B U

Recall was based on a cue which followed the offset of the stimulus and directed the subject to recall a specific line of letters from the initial display. Memory performance was compared under two conditions: whole report and partial report.

 

Whole Report

The whole report condition required participants to recall as many elements from the original display in their proper spatial locations as possible. Participants were typically able to recall three to five characters from the twelve character display (~35%). This suggests that whole report is limited by a memory system with a capacity of four-to-five items.

 

Partial Report

 

Sperling’s original partial report paradigm.

 

The partial report condition required participants to identify a subset of the characters from the visual display using cued recall. The cue was a tone which sounded at various time intervals (~50 ms) following the offset of the stimulus. The frequency of the tone (high, medium, or low) indicated which set of characters within the display were to be reported. Due to the fact that participants did not know which row would be cued for recall, performance in the partial report condition can be regarded as a random sample of an observer’s memory for the entire display. This type of sampling revealed that immediately after stimulus offset, participants could recall most letters (9 out of 12 letters) in a given row suggesting that 75% of the entire visual display was accessible to memory. This is a dramatic increase in the hypothesized capacity of iconic memory derived from full-report trials.

 

Variations of the partial report procedure

 

Averbach & Coriell’s partial report paradigm.

 

Visual Bar Cue

A small variation in Sperling’s partial report procedure which yielded similar results was the use of a visual bar marker instead of an auditory tone as the retrieval cue. In this modification, participants were presented with a visual display of 2 rows of 8 letters for 50 ms. The probe was a visual bar placed above or below a letter’s position simultaneously with array offset. Participants had an average accuracy of 65% when asked to recall the designated letter.

 

Temporal Variations

Varying the time between the offset of the display and the auditory cue allowed Sperling to estimate the time course of sensory memory. Sperling deviated from the original procedure by varying tone presentation from immediately after stimulus offset, to 150, 500, or 1000 ms. Using this technique, the initial memory for a stimulus display was found to decay rapidly after display offset. At approximately 1000 ms after stimulus offset, there was no difference in recall between the partial-report and whole report conditions. Overall, experiments using partial report provided evidence for a rapidly decaying sensory trace lasting approximately 1000 ms after the offset of a display.

 

Circle Cue & Masking

The effects of masking were identified by the use of a circle presented around a letter as the cue for recall. When the circle was presented before the visual stimulus onset or simultaneously with stimulus offset, recall matched that found when using a bar or tone. However, if a circle was used as a cue 100 ms after stimulus offset, there was decreased accuracy in recall. As the delay of circle presentation increased, accuracy once again improved. This phenomenon was an example of metacontrast masking. Masking was also observed when images such as random lines were presented immediately after stimulus offset.