Time is limited on the road to asymptopia -- Asking the ergodicity question while validating ABMs

One challenge in the empirical validation of agent-based models (ABMs) is how to infer reliable insights from numerical simulations. Ergodicity (besides stationarity) is a precondition in any estimation-related task, however it has not been systematically explored and is often simply presumed. For non-ergodic observables it remains largely unclear how to deal with the associated uncertainty. Here we show how an understanding of (broken) ergodicity in the convergence of summary statistics (so-called moments) improves the validation and calibration of 15 ABMs. We take two prototype agent-based financial market models and run Monte Carlo experiments to study convergence behaviour of selected moments. We find that for most moments the convergence time it takes to reach asymptopia is infeasibly long, thus leaving us in a pre-asymptopic regime. Choosing an efficient mix of ensemble size and simulated time length can help guiding validation efforts through this jungle of uncertainty.