Topology of Neural Representations

Linking is the topological phenomenon of two manifolds being interlocked, so that no continuous deformation can pull them apart without forcing a collision between them. Under the manifold hypothesis, data classes are supported near low-dimensional manifolds in feature space, and we ask what linking implies for a network trained to classify them. A classifier's final linear readout forces any successful network to place the class representations on opposite sides of a hyperplane, which means it has unlinked them. We show that the linking number is preserved, up to sign, by feedforward networks with monotone activations whose intermediate layers are no wider than the input dimension, an impossibility result for perfect classification of linked classes. Modern architectures escape without widening: skip connections, attention, and non-monotone activations are all capable of performing a geometric operation we call folding. Experiments support both halves, with the predicted fold visible inside a trained network. This goes beyond expressivity, giving the invariant an operational meaning: the linking number becomes a nonlinear, interpretable observable of a representation. I will close by proposing extrinsic topological data analysis as a novel toolkit for neural network interpretability, exemplified by link and knot invariants. The program will develop robust, computable numerical methods to detect further embedding-sensitive topological features.

No background in topology is assumed. Joint work with Lek-Heng Lim.