Ideas on Learnability

Informally learning denotes the ability of a system to improve its (behavioral) performance if necessary. Especially in a complex system this might be very difficult due to complicated interdependencies between the system parameters and its performance. There seems to exist a sever tradeoff between the number of parameters that are adapted (a crude measure of the size of the search space) and the complexity of their interdependencies, as suggested from simulations of recurrent neural networks. To better understand the properties of this learnability tradeoff a more formal description is certainly necessary. Therefore some attempts towards formalizing learnability and related concepts, such as modularity, will be discussed.