Information-theoretic modelling of scaling in communication systems

The principle of least effort in communications has been shown, by Ferrer i Cancho and Sol´e, to explain emergence of power laws (e.g., Zipf’s law) in human languages. This study sharpens the results of Ferrer i Cancho and Sol´e by explicitly solving the problem. The extended model contrasts Zipf’s law found in the vicinity of the transition between referentially useless systems and indexical reference systems, with a logarithmic law found at the transition. Arranging the codes according to a logarithmic law is observed to be the most representative optimal solution for maximising the referential power under the effort constraints. We also extend the principle and the information-theoretic model to multiple coding channels.