On the fibers and semialgebraicity of ReLU neuromanifolds

In the first part of the talk we focus on the semialgebraicity of the neuromanifold of a feedforward ReLU neural network and its symmetries. We prove that it is not a semialgebraic quotient of the space of weights of the network. We furthermore argue that it is pointwise semialgebraic and a pro-semialgebraic space (we will discuss both this notions in the talk). In the second part, we introduce and study the notion of honest open subset of the space of weights, where the network does not show any hidden symmetries. We conjecture that the maximal honest open is always semi-algebraic and prove that in the shallow case it is even Zariski.