Frequently Asked Biological Questions About LISSOM
The long-range connections in LISSOM are inhibitory, yet most
of the long-range horizontal connections connect excitatory neurons to
other excitatory neurons. How can you explain that?
LISSOM models the cortex at a slightly higher level than
individual neurons and their connections, modeling only each vertical
column (as a single unit) and the interactions between the columns.
In addition, the focus is on self-organization, which in a Hebbian
model is driven primarily by high contrast inputs. In this restricted
regime, the lateral interactions have been clearly demonstrated to be
inhibitory, as a result of interactions with local inhibitory
interneurons, even though at low contrasts the effects are excitatory
(Hirsch and Gilbert
Weliky et al 1995). If we start to study
low-contrast inputs in the future, we will need to extend LISSOM to
use a more complicated recurrent local circuit that would have the
correct behavior at low contrasts, but because that would slow down the
already-long run time drastically and make the model more complex, we
have avoided it for now.
The published LISSOM simulations start out with a large
excitatory lateral connection radius and then gradually reduce it to
cover a much smaller area. There is no evidence for this massive
The shrinking radius was inherited from SOM, and is not a
required part of the model. It does help to make sure that the entire
map becomes globally ordered even when the starting point for
self-organization is extremely disorganized. However, there are
several other more-plausible ways to achieve the same effect, such as
gradually increasing the effect of inhibition relative to excitation.
Moreover, there is no evidence that there is ever a time that the
cortex is quite as disorganized as the starting points we use in the
So, why do you use such an implausible starting point?
We are studying computation, and we wish to know the true
capabilities of the algorithms we are investigating. We deliberately
make the task harder for the algorithm than is likely to occur in
nature, to show that it is robust. The state of the model at any
point during training can be chosen instead as a starting point, and
compared to the biological case from that point forward. We do not
take this step explicitly ourselves because we are not modeling any
particular species, and the known "starting point" clearly differs
There is little evidence of any inhibitory interactions in
the very earliest stages of development, yet LISSOM has inhibitory
connections from the start. Why?
Immature cortical neurons appear to be very difficult to
excite already, and thus probably do not require any further
inhibition to prevent runaway (epileptic) responses. We hypothesize
that inhibitory influences arise automatically as soon as the neurons
begin responding more strongly, thus maintaining a balance between
excitation and inhibition. The model does not currently contain any
mechanism for this balancing process, and instead just sets excitation
and inhibition to be approximately equal by fiat, which is probably
much simpler to do in a model than in a real organism. A complete
model would need to incorporate such a control mechanism, but we try
to keep the model as simple as we possibly can unless we are looking
at a phenomenon dominated by a particular mechanism.
Isn't the orientation map already organized at eye-opening?
How can you explain that in an input-driven Hebbian model?
Intrinsic nervous-system activity can probably account for
this, although it is not yet clear which particular source is
responsible. One such possibility is retinal waves, patches of
activity known to exist in the retina before eye opening
(Meister et al 1991,
Wong et al 1993). It has been claimed that retinal
waves are not sufficiently oriented to account for orientation
selectivity (Miller 1994).
However, we have shown that even entirely unoriented patterns can
drive such selectivity if the patterns have sharp edges
(Bednar and Miikkulainen 2003).
It is not known if retinal activity reaches the cortex in time to
drive development, but there are also a number of other candidates,
including spindle waves and the PGO waves of REM sleep. See
(Bednar 2002) for more details.
Work by Chapman,Stryker,and Bonhoeffer (1996) showed that the orientation map
remains fairly static as it becomes sharpened by visual experience,
but in LISSOM the orientation patches can be seen to vary in location
as self-organization progresses. Why?
The initial chaos is due to having a high learning rate and to
having a shrinking lateral excitatory radius; once those conditions
cease, the map is stable. Simulations without a shrinking radius and
with only a short period with a high learning rate (not yet submitted
for publication) show a stable map once orientation selectivity has
My question or critique was not answered by this FAQ. What
should I do?
Send email to email@example.com, or to the corresponding author of
the paper you read! We much appreciate getting feedback.
- James A. Bednar and Risto Miikkulainen. Learning innate face preferences. Neural Computation, 15(7):1525-1557, 2003.
- James A. Bednar. Learning to See: Genetic and Environmental Influences on Visual Development. PhD thesis, Department of Computer Sciences, The University of Texas at Austin, Austin, TX, 2002. Technical Report AI-TR-02-294.
- Barbara Chapman, Michael P. Stryker, and Tobias Bonhoeffer. Development of orientation preference maps in ferret primary visual cortex. The Journal of Neuroscience, 16(20):6443-6453, 1996.
- Judith A. Hirsch and Charles D. Gilbert. Synaptic physiology of horizontal connections in the cat's visual cortex. The Journal of Neuroscience, 11:1800-1809, June 1991.
- M. Meister, R. O. L. Wong, D. A. Baylor, and C. J. Shatz. Synchronous bursts of action-potentials in the ganglion cells of the developing mammalian retina. Science, 252:939-943, 1991.
- Kenneth D. Miller. A model for the development of simple cell receptive fields and the ordered arrangement of orientation columns through activity-dependent competition between ON- and OFF-center inputs. The Journal of Neuroscience, 14:409-441, January 1994.
- Michael Weliky, Karl Kandler, David Fitzpatrick, and Lawrence C. Katz. Patterns of excitation and inhibition evoked by horizontal connections in visual cortex share a common relationship to orientation columns. Neuron, 15:541-552, September 1995.
- R. O. L. Wong, M. Meister, and C. J. Shatz. Transient period of correlated bursting activity during development of the mammalian retina. Neuron, 11(5):923-938, Nov 1993.