Recognizing sequences of sequences

The brain’s decoding of fast sensory streams is currently impossible to emulate, even approximately, with artificial agents. For example robust speech recognition is relatively easy for humans but exceptionally difficult for artificial speech recognition systems. In this talk, I propose that recognition can be simplified with an internal model of how sensory input is generated, when formulated in a Bayesian framework. We show that a plausible candidate for an internal or generative model is a hierarchy of ‘stable heteroclinic channels’. This model describes continuous dynamics in the environment as a hierarchy of sequences, where slower sequences cause faster sequences. Under this model, online recognition corresponds to the dynamic decoding of causal states, giving a representation of the environment with predictive power on several timescales. The ensuing decoding or recognition scheme uses synthetic sequences of syllables, where syllables are sequences of phonemes and phonemes are sequences of sound-wave modulations. The resulting recognition dynamics disclose inference at multiple time scales and are reminiscent of neuronal dynamics seen in the real brain.