‘One Size Does Not Fit All’: A Roadmap of Purpose-Driven Mixed-Method Pathways for Sensitivity Analysis of Agent-Based Models

• Published in: Journal of artificial societies and social simulation Vol. 23; no. 1
• Main Authors: Ligmann-Zielinska, Arika, Siebers, Peer-Olaf, Magliocca, Nicholas, Parker, Dawn C., Grimm, Volker, Du, Jing, Cenek, Martin, Radchuk, Viktoriia, Arbab, Nazia N., Li, Sheng, Berger, Uta, Paudel, Rajiv, Robinson, Derek T., Jankowski, Piotr, An, Li, Ye, Xinyue
• Format: Journal Article
• Language: English
• Published: Guildford Department of Sociology, University of Surrey 01.01.2020
• Subjects: Mixed methods research; Policy analysis; Realism; Robustness; Sensitivity analysis; Simulation; Structural models; Uncertainty; Variability
• ISSN: 1460-7425; 1460-7425
• DOI: 10.18564/jasss.4201

Designing, implementing, and applying agent-based models (ABMs) requires a structured approach, part of which is a comprehensive analysis of the output to input variability in the form of uncertainty and sensitivity analysis (SA). The objective of this paper is to assist in choosing, for a given ABM, the most appropriate methods of SA. We argue that no single SA method fits all ABMs and that different methods of SA should be used based on the overarching purpose of the model. For example, abstract exploratory models that focus on deeper understanding of the target system and its properties are fed with only the most critical data representing patterns or stylized facts. For them, simple SA methods may be sufficient in capturing the dependencies between the output-input spaces. In contrast, applied models used in scenario and policy-analysis are usually more complex and data-rich because a higher level of realism is required. Here the choice of a more sophisticated SA may be critical in establishing the robustness of the results before the model (or its results) can be passed on to end-users. Accordingly, we present a roadmap that guides ABM developers through the process of performing SA that best fits the purpose of their ABM. This roadmap covers a wide range of ABM applications and advocates for the routine use of global methods that capture input interactions and are, therefore, mandatory if scientists want to recognize all sensitivities. As part of this roadmap, we report on frontier SA methods emerging in recent years: a) handling temporal and spatial outputs, b) using the whole output distribution of a result rather than its variance, c) looking at topological relationships between input data points rather than their values, and d) looking into the ABM black box âĂŞ– finding behavioral primitives and using them to study complex system characteristics like regime shifts, tipping points, and condensation versus dissipation of collective system behavior.