Space and Time of Cortical Activity

Moshe Abeles  (Bar Ilan University, Ramat-Gan, Israel)
Thursday, July 06, 2006
We wish to evaluate what are the scales in space and time needed for understanding how the cortex works.
Anatomical and microscopical studies have parceled the cortex on different scales: Cytoarchitectonic Areas, Hyper-Columns, Mini-Column, Single Neuron, Individual Synapses, a Single Channel. I shall provide evidence that surface maps are much too coarse. The individual neurons seem to be the largest unit that can be used to understand the cortical mechanisms for processing information.
Activity may be measured by fMRI (or PET) on a time scale of seconds, by EEG (or MEG) on a Time scale of a small fraction of a second and by microelectrodes on a time scale of milliseconds or less. I shall provide experimental evidence that the firing time of cortical neurons may fire at a precision better then 1 ms.
Some theoretical implications of these findings will be discussed.
Supported in part by Grants from ISF, DIP, and the Rich Center.

Time delays and neural field dynamics

Fatihcan Atay  (Max Planck Institute for Mathematics in the Sciences, Leipzig, Germany)
Friday, July 07, 2006
We consider field models of neural dynamics which include both distance-dependent delays prevalent in intracortical connections, and fixed delays arising in interareal feedback loops. Furthermore, motivated by experimental evidence, we introduce distributions of propagation speed and feedback delays, instead of fixed values. We present an analytical method for studying the stability of equilibria and the bifurcations leading to spatial patterns and traveling waves, which is applicable for general connectivity kernels.

Information Flows in Causal Networks

Nihat Ay  (Max Planck Institute for Mathematics in the Sciences, Leipzig, Germany)
Friday, July 07, 2006
Mathematical information theory provides an important framework for understanding cognitive processes. It has been successfully applied to neural systems displaying feed forward structures. It turns out that the analysis of recurrent structures is more subtle. This is mainly due to the fact that corresponding information-theoretic quantities allow for the ambiguity of their causal and associational interpretations. In order to understand information flows in recurrent networks, one has to make a clear distinction between causal and associational effects. In collaboration with Daniel Polani we addressed this problem using a causality theory developed by Judea Pearl and his coworkers. I will discuss some possible applications of this work to complexity theory.

Psychophysiological Modeling of Spatial Color Vision in Man

Werner Backhaus  (Freie Universität Berlin, Germany)
Friday, July 07, 2006
A psychophysiological model of the elementary color sensations red, green, blue, yellow, black and white, which constitute our color sensations, is presented. The model takes spectral sensitivity functions of human cones into account. The well approved neuronal color opponent coding (COC) model, originally developed for insect vision is incorporated and adapted to light discrimination judgments in humans [1]. In the case of human color vision, the parameters of the COC model could yet only be determined from the data of light discrimination experiments up to arbitrary rotations of the basis of the subjective light discrimination space [2], because of the rotational invariance of the Euclidean metric. In addition to spanning the light-discrimination space, the COC neurons are assumed to steer the amounts of the elementary color sensations piece by piece linearly. A hypothesized interaction-mechanism of the elementary colors normalizes the total amounts to 100%. An advantage is that all the parameters of this model have definite values, i.e. no rotational degrees of freedom. Classical psychophysical measurements of the amounts of the chromatic elementary color sensations stimulated by monochromatic light are simulated with the model. Best fits of the predicted data to the measured data of two observers determined the parameters uniquely. The model precisely describes the common characteristics as well as the measured individual differences by respectively deviating values of the physiological parameters.
Furthermore, the model has been extended by spatial aspects of color vision [3], explaining 1) hyperacuity, 2) the virtual projection of the color sensations to the outside of our head and 3) the visual "enlargement with depth" effect [4] and related 4) size distortion and constancy phenomena caused by 3D drawings.
[1] Backhaus, W., 2004. A physiological model of human color discrimination. In: Proceedings of the 4th Forum of European Neuroscience (FENS), July 10–14, 2004, Lisbon, Portugal, Abstracts, Vol. 2, A, p. 194.5. FENS, Lisbon.
[2] Backhaus, W., 2006. Psychophysical simulations of spatial color vision. In: Fünftes Symposium Licht und Gesundheit, 23.-24.2.2006. Eine Sondertagung der TU Berlin und der DGP mit DAfP und LiTG, Hrsg. H. Kaase & F. Serick, Tagungsband, Hauptvorträge, pp. 8-21. Technische Universtät, Berlin (ISBN 3-9807635-0-2).
[3] Backhaus, W., 2001. Measurement and simulation of colour sensations. In: Neuronal Coding of Perceptual Systems, ed. W. Backhaus. World Scientific, London , pp. 445-474.
[4] Backhaus, W. 2003. Evidence for a spherical geometry of color perception. In: Proceedings of the 29th Göttingen Neurobiology Conference and the 5th Meeting of the German Neuroscience Society 2003. From Basic Research to Therapy, pp.1007-1009. Thieme, Stuttgart - New York.

Fluctuation-driven Neurodynamics: A model of Decision-Making

Gustavo Deco  (Universitat Pompeu Fabra, Barcelona, Spain)
Friday, July 07, 2006
The problem of decision-making has become the center of interest of many neuroscientists aiming to understand the neural basis of intelligent behavior by linking perception and action. Behavioral, neurophysiological, and theoretical studies are converging to a common theory that assumes an underlying diffusion process which integrates both the accumulation of perceptual and cognitive evidence for making the decision and motor choice in one unifying neural network. In particular, a number of recent neurophysiological experiments are providing information on the neural mechanisms underlying decision-making, by analyzing the responses of neurons that correlate with the animal's behavior. In this talk, we analyse computational models of decision-making involving populations of excitatory neurons engaged in competitive interactions mediated by inhibition. Sensory input may bias the competition in favor of one of the populations, resulting in a gradually developing decision in which neurons in the chosen population exhibit increased activity while other populations are inhibited. In this scenario spontaneous, non-selective network activity and the decision state, in which one population has been activated, both represent stable solutions of the underlying equations, i.e. they are multistable. Decision-making is then understood as the fluctuation-driven, probabilistic transition from the spontaneous to the decision state.

Bayesian inference and learning with spiking neurons

Sophie Deneve  (École Normale Supérieure, Paris, France)
Friday, July 07, 2006
We show that the dynamics of spiking neurons can be interpreted as a form of Bayesian inference in time. Neurons that optimally integrates evidence about labile events in the external world exhibit properties similar to leaky integrate and fire neurons with spike-dependant adaptation, and maximally respond to fluctuations of their input. Spikes signals the occurrence of new information, i.e. what cannot be predicted from the past activity. As a result, firing statistics are close to Poisson, albeit providing a deterministic representation of probabilities. We proceed to develop a theory of Bayesian learning in spiking neural networks, where neurons learn to recognize the spatial and temporal dynamics of their synaptic inputs. Meanwhile, successive layers of neurons learn hierarchical causal models for the sensory input. The resulting learning rules are local, spike time dependant, and highly non-linear. This approach provides a principled description of spiking and plasticity rules maximizing the information transfer, while limiting the number of costly spikes, between successive layers of neurons.

Neural Dynamics and Biophysics of Active Memory

Daniel Durstewitz  (University of Plymouth, United Kingdom)
Friday, July 07, 2006
The life of all mammals and birds often involves situations where information has to be integrated across short periods of time. This is for instance the case when consistent temporal relations between events in a sequence are to be exploited for prediction, or when a behavioral choice crucially depends on recently encountered stimuli ('non-Markovian decision problems'). Such situations require active (working) memory, the ability to maintain information within the ongoing neural dynamics (as opposed to, e.g., synaptic long-term plasticity). Based on in-vivo electrophysiological recordings from behaving animals, various neuro-dynamical mechanisms for the active maintenance of information have been suggested. Most of them rest on the idea of multistability in the sense of multiple co-existing attractor states of the population firing rate. More recent models, again fed by physiological observations, employ long effective time constants close to bifurcations, or store information within the phase-relation patterns of neural spike times (weak phase-locking). These ideas and their empirical basis will be discussed, with emphasis on the biophysical mechanisms that could give rise to the various dynamical phenomena. In particular, the special role of NMDA synaptic currents in controlling neural system dynamics in intriguing and computationally important ways will be highlighted.

U-shape in learning faces reveals universal non-grammar

André Grüning  (SISSA Trieste, Italy)
Thursday, July 06, 2006
The ability to extract information that concisely expresses dependencies among language components is fundamental to human language cognition. Among quantitative characterizations of language acquisition processes, the human sensitivity to non-adjacent syllable dependences has been described by a U-shaped curve, as a function of the variability of an intervening irrelevant utterance. In artificial language learning experiments, it was indeed easier to learn pairs of monosyllabic non-words if the middle two-syllable non-word was either constant, or varied among a large set, irrespective of the co-occurring pair, than if its variability was limited. This finding might characterize a possibly uniquely human cross-over between statistical learning and rule extraction. We find, however, that a similar curve describes learning in an unrelated domain, that of facial configurations. When trained on a corpus of face drawings in which the middle component (eyes-nose) was unrelated to the fixed combinations of upper (hair-ears) and lower (mouth-chin) components, subjects were better at rejecting 'ungrammatical' upper-lower combinations if the eyes-nose component was either always the same, or varied independently in a large set. Subjects' performance on a subsequent face grouping task indicates an inability to extract the underlying rule. The U-shape may then universally characterize incidental statistical learning, unrelated to the specific language domain.

GABA Reversal Pontetials Control Synchronization of Neuronal Oscillators

Boris Gutkin  (Institut Pasteur, Paris, France)
Friday, July 07, 2006
Ho Young Jeong(2) and Boris Gutkin(1)
(1) Group for Neural Theory, DEC, ENS-Paris and Department of Neuroscience, Institut Pasteur, Paris (2)Center for Neural Science, New York University, New York, NY USA
GABAergic synapse reversal potential is controlled by the concentration of chloride. This concentration can change significantly during development and as a function of neuronal activity. Thus GABA inhibition can be hyperpolarizing, shunting or partially depolarizing. Previous results pint-pointed the conditions under which hyperpolarizing inhibition (or depolarizing excitation) can lead to synchrony of neural oscillators. Here we examine the role of the GABAergic reversal potential in generation of synchronous oscillations in circuits of neural oscillators. Using weakly coupled oscillator analysis we show when shunting and partially depolarizing inhibition can produce synchrony, asynchrony and co-existence of the two. In particular, we show that this depends critically on such factors as the firing rate, the speed of the synapse, spike frequency adaptation and most importantly on the dynamics of spike generation (type I vs. type II). We back up our analysis with directly simulations of small circuits of conductance based neurons as well as large-scale networks of neural oscillators. The simulation results are compatible with the analysis: e.g. when bistability is predicted analytically the large scale-network shows clustered states.

Neuronal Map Formation & Time-Resolved Learning: Do Different Modalities "Observing" the Same Object Compete?

J. Leo van Hemmen  (Technische Universität München, Germany)
Saturday, July 08, 2006
A map is a neuronal representation of the outside sensory world. Spike-timing-dependent synaptic plasticity (STDP) is a universal means to ``learn' spatio-temporal neuronal activity patterns [1] and explain neuronal map formation [2]. Time resolution may well, and does, happen at a millisecond timesale or better. For example, in the barn owl STDP resolves the Konishi paradox of attaining an accuracy (s) that is two orders of magnitude better than all neurnal time constants involved and explains the full neuronal map in the laminar nucleus, where inputs from left and right ear convene.
Quite often, however, STDP is not completely self-organizing but needs an external ``teacher' to coordinate maps arising from two, or more, different sensory modalities. To this end supervised STDP (SSTDP) may be quite helpful [3]. Here we explain SSTDP through two concrete examples, the clawed frog Xenopus and the barn owl. For the latter it is known since long that the visual system is the teacher, for the former we surmise it is in an early stage of the frog's existence. We also discuss a convergence proof for the algorithm underlying SSTDP.

W. Gerstner, R. Kempter, J.L. van Hemmen, and H. Wagner, A neuronal learning rule for sub-millisecond temporal coding, Nature 383 (1996) 76-78.
C. Leibold and J.L. van Hemmen, Spiking neurons learning phase delays: How mammals may develop auditory time-difference sensitivity, Phys. Rev. Lett. 94 (2005) 168102-1/4
J.-M.P. Franosch, M. Lingenheil, and J.L. van Hemmen, How a frog can learn what is where in the dark, Phys. Rev. Lett. 95 (2005) 078106-1/4

Probing Generic Integration Principles for Cognitive Functions

Helge Ritter  (Universität Bielefeld, Germany)
Saturday, July 08, 2006
While physics gives us a picture how simple laws can drive the self-organization of matter at increasing complex scales of organization, biological evolution led to systems with processes enabling a rapid self-organization of information. Trying to replicate even remotely similar capabilities in artificial intelligent systems, such as robots, poses the challenge to find architectural principles that can integrate large collections of possibly heterogeneous functional elements into a coherently operating whole. We illustrate work on this issue by three concrete examples from our research: (i) an architecture for the self-organized formation of semantic maps by combining ideas from neural self-organization and non-euclidean spaces, (ii) a generic approach for image categorization by information compression motivated from analogies to compression-based structure formation in physics, and (iii) a layered competitive network architecture to decompose complex patterns into simpler constituents. Finally, we discuss how this research fits into the larger picture of developing and integrating elements towards artificial cognitive systems, in particular cognitive robots for human-machine interaction.

Topological self-organization and critical dynamics of input-driven neural threshold networks

Thimo Rohlf  (Santa Fe Institute, USA)
Saturday, July 08, 2006
It has been proposed that computational capabilities and adaptation are optimized by dynamical systems that self-organize into a critical state at the border between chaos and order. Recent work on real-time computation at the 'edge of chaos' in recurrent neural networks strongly supports this conjecture [1]. Here, based on a simple model of network self-organization by a local 'homeostasis rule' coupling rewiring events to a dynamical order parameter (average neural activity) [2,3], we study topological evolution of input-driven neural threshold networks. In addition to the original system, a subset of network nodes is driven by external (stochastic) input signals. Compared to the undriven model, we find a much faster convergence to criticality; if a sufficiently large (but finite) fraction of neurons is driven, networks become critical even for finite system size N. Several dynamical order parameters exhibit pronounced power-law scaling, long-range correlations and 1/f noise, including the average neural activity. Finally, we discuss possibilities to exploit this mechanism for computations (e.g. on time series) in neural systems, e.g. by combining it with dynamical or evolutionary learning rules, and whether similar self-organizing principles might be at work in real neural systems.
[1] Bertschinger, N. and Natschlaeger, T., Neural Computation 16, 1413-1436 (2004)
[2] Bornholdt, S. and Rohlf, T., Phys. Rev. Lett. 84, 6114 (2000)
[3] Rohlf, T. and Bornholdt, S., Physica A 310, 245-259 (2002)

Oscillations, synchrony and temporal codes in cortical processing

Wolf Singer  (Max Planck Institute for Brain Research, Frankfurt, Germany, and Frankfurt Institute for Advanced Studies, Germany)
Thursday, July 06, 2006
In principle, neurons can encode information in at least two ways , i) by varying the amplitude of their response (rate code) and/or ii) by adjusting the precise timing of individual discharges (temporal code). It is proposed that cortical networks use both coding strategies in parallel, exploiting their respective advantages. Evidence is presented that precise coordination of spike timing across neuronal assemblies is frequently associated with an oscillatory patterning of discharge sequences in the beta and gamma frequency range and the synchronisation of distributed responses with millisecond precision. This synchronisation appears to be exploited to i) jointly raise the saliency of responses to sensory stimuli in the context of perceptual grouping (binding), ii) read out grouping criteria residing in the functional architecture of cortical networks during the free exploration of visual scenes , iii) prepare as a function of attention and response anticipation the hand-shaking between distributed cortical regions , iv) determine the occurrence and polarity of use dependent synaptic modifications and v) gate the access of sensory signals to consciousness. Recent data indicate further that cognitive functions requiring dynamic binding are disturbed in schizophrenia patients and that these deficits go along with a reduced ability to precisely synchronize responses across distributed cortical networks. The possibility is suggested that some of the dissociative symptoms characteristic of this disease result from impaired binding functions.

Latching dynamics as a model of recursion in language and beyond

Alessandro Treves  (SISSA Trieste, Italy, and Norwegian University of Science and Technolgy, Trondheim, Norway)
Thursday, July 06, 2006
I will report research by Emilio Kropff and myself.
We study a Potts neural network as a model of the architecture and function of large cortical networks in the mammalian brain [1]. The basic assumption is that hebbian associative plasticity informs connections both within local networks and at long-range synapses [2]. Such large-scale associative networks are proposed to participate in higher order processes including semantic memory and language [3,4,5]. We discuss the modified Hebb rule that allows to deal with catastrophic overload, and estimate the network storage capacity in the presence of strong correlations. We then study the latching phenomenon - the capacity of such a network to spontaneously generate arbitrarily long and complex sequences hopping from one memory retrieval state to the next, based on their correlation [6]. We show that the complexity in the sequence of attractors can be controlled by an effective threshold parameter. The transition between recursive and non recursive systems appears to scale with the number of local attractor states, suggestive of a spontaneous evolution of infinite recursion in human cognition.
[1] Kropff E and Treves A, J. Stat. Mech. P08010 (2005)
[2] Braitenberg V and Schuz A, Anatomy of the Cortex: Statistics and Geometry (Berlin: Springer, 1991)
[3] Pulvermuller F, Prog. Neurobiol. 67:85 (2002) [4] McRae K, de Sa V and Seidemberg M, J. Exp. Psychol. General 126:99 (1997)
[5] Tyler LK et al, The neural representation of nouns and verbs: PET studies, Brain, 124:1619 (2003)
[6] Treves A, Cogn. Neuropsychol. 6:101 (2005)

Probability Collectives and Supervised Learning

David Wolpert  (NASA Ames Research Center, USA)
Friday, July 07, 2006
There are two major fields that analyze distributed systems: statistical physics and game theory. These fields can be re-expressed in a way that makes them mathematically identical. Doing so allows us to combine techniques from them, producing a hybrid formalism. That hybrid is called Probability Collectives (PC).
As borne out by numerous experiments, PC is particularly well-suited to black-box optimization and associated problems in distributed control. The core idea is that rather than directly optimize a variable of interest x, often it is preferable to optimize an associated probability distribution, P(x). That optimization can be done either via Monte Carlo Optimization (MCO) or, under certain circumstances, in closed form. Recently is was realized that one can map MCO into a supervised machine learning problem. This means that all the powerful techniques of supervised learning can be used to improve MCO. As a special case, those techniques can be used to improve the optimization of P(x) in a PC-based optimizer. In this way the techniques of supervised learning can be leveraged to improve black-box optimization and distributed control.
In this talk I review PC. I also illustrate the identity between MCO and supervised learning using PC. In particular, I present results showing how cross-validation can be used to adaptively set an annealing schedule for the optimization of P(x), and to adaptively modify the complexity of P(x). I also illustrate the benefit of bagging in PC.

Predictive Mechanisms in Neuronal Processing for the Generation of simple Semantics in Artificial Agents

Florentin Wörgötter  (Universität Göttingen, Germany)
Thursday, July 06, 2006
Predictive mechanisms and inference processes are known as a means to improve the behavior of artificial systems, like networks or robots. Here we show that it is possible to design predictive control mechanisms with provable convergence properties, which a system can learn through network plasticity. Systems of these kind will arrive at behavioral and synaptic homeostasis at the same time thereby leading to stable synaptic weights. Supporting a "constructivist's perspective", evidence can be provided that during such a process very simple semantic properties arise in the network, which are objectively defined by the structure of the agent's environment and not through subjective interference by the designer of the system. It will be argued that the 'thwarting of predicitons' can possibly be used as one fundamental process to arrive at early cognitive properties in real and artificial systems.