How do spatial simulation models incorporate and handle real world system uncertainty?

O’Sullivan et al. write that uncertainty is an inescapable element of modeling (194). Uncertainty enters at several locations within the modeling process: those models that are built with less than complete knowledge of the internal dynamics of the system–i.e., all of them–suffer from uncertainty regarding the accuracy with which they represent the target system’s dynamics. Even those parameters which can be empirically determined cannot be determined with perfect accuracy, allowing measurement error to further introduce uncertainty (measurement uncertainty). Finally, even if the target system were perfectly understood and perfectly measured, it itself may exhibit natural variability which the model must then reflect (process uncertainty) (195 & 198).

Simulation models typically deal with these sources of uncertainty through the incorporation of stochastic variability. Including stochasticity in models on the one hand deviates from the Enlightenment ideal of perfect determinism that non-simulation models can offer, but on the other hand arguably makes models more realistic, i.e. reflective of target system dynamics. On the other hand, stochastic elements make models far less tractable analytically, making interpretation and understanding of model behavior much more challenging.

O’Sullivan et al. offer the example of a simple harvesting model: a maximum sustainable yield calculated from a deterministic model should, according to the constraints of that model, be sustainable indefinitely; yet when even a small measure of variability in regrowth rate is introduced, the “sustainable” yield level inevitably leads to extinction (197). Stochasticity, in other words, can radically alter the behavior of models.

It is important to keep in mind a distinction between model stochasticity and target system stochasticity: random processes are used in models to represent uncertainties derived from a multitude of sources which are not themselves interchangeable. A random element in an agent’s foraging behavior may reflect a lack of understanding of the target actor’s decision-making process, or a difficulty in properly parameterizing that decision-making, or even a calculated decision to leave out unnecessary fine-scale processes in an already complicated model.

Given the prevalence of uncertainty in the modeling process, many ways of thinking about and dealing with this uncertainty have arisen. O’Sullivan et al. break down these approaches into three categories (199). The first of these deals with uncertainties inhering to the model itself: how does uncertainty in parameterization or model processes propagate through the model? Many of these approaches attempt to determine the relative importance of the model’s parameters on output: how much does altering parameter A by 10% impact the output relative to a altering parameter B by 10%? Can small errors in initial parameters throw the model out of balance, or are they damped out? Robustness analysis goes a step further and analyzes the impact of plugging entirely different sub-processes into the model. A second category compares model outcomes to “real world” data to evaluate model performance: while prediction is only one possible purpose for modeling, it is nonetheless a meaningful one and this kind of validation has its place (227, 228). A final set of model analysis approaches evaluating a model’s success from a heuristic perspective: what does this model teach us? By comparing a model’s ability to replicate a series of relevant observed patterns, or by assessing a model’s usefulness in creating or attracting popular legitimacy, models can also play a critical role in facilitating greater understanding of the target system among scientists and the public at large.