Self-organization and unsupervised learning in recurrent networks

Cortical circuits are shaped by a number of different plasticity mechanisms, but it is still unclear how these endow the cortex with useful information processing abilities. Over the last years we have developed recurrent neural network models that self-organize their connectivity under the influence of different plasticity mechanism including spike-timing dependent plasticity and different forms of homoestatic plasticity. These self-organizing recurrent networks (SORNs) can learn about the temporal structure in input time series in an unsupervised fashion and can greatly outperform non-adaptive networks on challenging prediction tasks. Furthermore, these networks can explain a number of features of cortical circuits including the Poisson-like firing of individual neurons, the overall distribution of synaptic connection strength, and the high degree of synaptic turnover and patterns of synaptic fluctuations. In addition, they make testable predictions regarding the distribution of synaptic life times. Finally, they also explain some psychological results on sequence learning in adult subjects. Overall, our results suggest that cortical circuits are shaped by processes of network self-organization through the combined action of multiple forms of neuronal plasticity.