A maximum entropy approach to unsupervised high-dimensional binary clustering using Hopfield networks

We present here a novel method for clustering and de-noising high-dimensional, noisy, binary data based on Hopfield networks and minimum probability flow (MPF) learning. In this talk I will discuss both theoretical aspects of MPF as well as applications of themethod to synthetic and neural data. We propose the method as a general technique for de-noising and clustering high-dimensional binary data and compare it to other well-known methods such as k-means clustering, locally-linear embedding and multi-dimensional scaling. We show how the method can be applied to the classical task of finding and extracting recurring spatiotemporal patterns in recorded spiking activity of neuronal populations. In contrast to previously proposed methods, the proposed technique (1) does not seek to classify exactly recurring patterns, but rather approximate versions possibly differing by a certain number of missed, shifted or excess spikes, (2) does not suffer of combinatorial explosion when complexity and size of the patterns considered are increased. Modeling furthermore the sequence of occurring Hopfield memories over the original data as a Markov process, we are able to extract low-dimensional representations of neural population activity on longer time scales. We demonstrate the approach on a data set obtained in rat barrel cortex and show that it is able to extract a remarkably low-dimensional, yet accurate representation of average population activity observed during the experiment.

This is joint work with Christopher Hillar, MSRI and Redwood Center for Theoretical Neuroscience, Berkeley, CA, USA.