W2: Sensitivity and Uncertainty Assessment of Integrated Environmental Models

Organised by Holger Maier, Jim Ascough, James Brown and Jaroslav Mysiak


Title: Admitting to Uncertainty Undermines the Decision and other Uncertain Myths

Authors: Katherine von Stackelberg

Abstract: The goal of uncertainty analysis is to make the modeling process more transparent by acknowledging and, to the extent possible, quantifying the inherent uncertainties. Unfortunately, sometimes it has just the opposite effect. Uncertainty analyses, particularly (multi-dimensional) probabilistic analyses can be difficult to communicate and consequently understand. It sometimes appears as though every parameter is more uncertain than not, making it difficult to justify a decision and defend it publicly, and obscuring the real message. However, not acknowledging the uncertainties inherent in any analysis is at best foolish and at worst dangerous by providing a false sense of confidence that this is the number. By identifying sources and magnitude of uncertainties, decisionmakers can determine whether additional information should be obtained prior to making a decision, and provides a quantitative context for specific and particular individual results. This paper presents the advantages of uncertainty analysis, some strategies for overcoming uncertainty analysis paralysis, and identifies specific questions managers and decisionmakers should ask regarding any analysis. We discuss several bioaccumulation modeling tools that we have developed in the context of risk-based tools to support environmental decisionmaking, and how uncertainty is handled, including TrophicTrace (which uses interval methods), FishRand (a second-order probabilistic aquatic bioaccumulation model), and the Spatially Explicit Exposure Model (SEEM). Particular emphasis is also placed on communicating uncertainty to decisionmakers and other interested stakeholders within a risk assessment context.

Title: Integrated Risk Modeling Frameworks: A Case Study of Exposure to Polychlorinated Biphenyls

Authors: Katherine von Stackelberg

Abstract: We present an integrated modeling framework based on tools developed for the Remedial Investigation/Feasibility Study (RI/FS) for the Hudson River Superfund Site. The integrated risk models include modules for fate and transport, bioaccumulation, and human health and ecological effects. In addition, we incorporate economic values for the benefits of risk reductions under proposed management actions using the results of two contingent valuation (CV) surveys developed specifically to elicit values for risk reductions and endpoints associated with exposure to polychlorinated biphenyls (PCBs) via fish ingestion. The integrated risk model explicitly distinguishes between two sources of uncertainty: incertitude and variability. Variability reflects population heterogeneity, and identifies the distribution of risks across the population (e.g., subsistence anglers versus occasional recreational anglers). Uncertainty reflects unknown but measurable quantities, such as measurement error. There is a limit to how much variability can be reduced; in theory, enough data could be collected to eliminate incertitude. These two sources of uncertainty have very different implications for decisionmaking. We will discuss some of the technical issues associated with the disaggregation of incertitude and variability in the integrated risk models. We use the risk assessment process and the tools developed for the Hudson River RI/FS to demonstrate the feasibility of incorporating economic information on the benefits of risk reductions within an integrated modeling framework. In addition, we explore issues related to stated preference methods through the CV surveys and discuss the implications of uncertainty in both the economic and risk aspects of the integrated model. We discuss the difficulties of conveying complex model results and associated probabilities to the general public, and discuss methods for managing uncertainty in the decisionmaking process.

Title: Uncertainty and policy support

Authors: Jaroslav Mysiak, James Brown

Abstract: To understand discourses concerning uncertainty and environmental policies, it is crucial to realise what changes the field underwent in the recent decades. First, policy problems rose up to complex social choices which have to balance divergent beliefs, interests and values. In this situation, different perspectives/ understanding of what the problem is and how it should be tackled are equally legitimate (Rittel, 1977; Ackoff, 1979; Rosness, 1998; Sarewitz, 2004). Second, as a consequence, environmental policies turned to more pro-active, precautionary (Wynne, 1992), non-structural (Faisal et al., 1999) and demand-side (Mohamed and Savenije, 2000) approaches, favouring more cautious exploitation of resources. Third, process of policy making rely more than ever on interdisciplinary, pluralistic, inclusive approaches, with scientists participating alongside other stakeholders in deliberative decision making (Edeelfee, 2000; Robertson and McGee, 2003), participatory assessment (Bacic et al, 2005), or group model building (Vennix, 1999). In this context, uncertainty has become a key topic. Partly, because policy outcomes are predictable only to some extent and associated uncertainty is large enough to sustain persistent conflicts and indecision. Partly, because uncertainty provides a political resource which can sustain own cause or contest opposed beliefs and interests (Weiss, 2002; Stirling, 2005). In this paper we address the large divergence among scientists on how risk and uncertainty ought to be defined and tackled (see Refsgaard et al., 2005; Brown, 2004; Walker et al., 2003; Norton et al., 2005, van Asselt & Rotmans, 2002 for a review). We argue that this persist lack of consensus about the general definition of uncertainty and the theoretical framework in which it is embedded cannot be resolved by harmonising terminology used in different fields. Uncertainty encompasses a group of phenomena, rather than a single one, connected to observation, perception, reflection and prediction of the nature of real world. The differences between competing understanding of uncertainty (e.g. as a feature of real world systems versus state of the mind) are deeply rooted in methodological context in which uncertainty is conceptualised and debated. The scope of uncertainty analysis and the theoretical framework provide lenses which contribute to these differences. Consequently, there is no single theory able to account for all associated matters.

Title: Resolving Model Parameter Values From C and N Stock Measurements in a Wide Range of Tropical Mature Forests Using Nonlinear Inversion

Authors: Shuguang Liu, Pamela Anderson, Guoyi Zhou, Boone Kauffman, Flint Hughes, David Schimel, Vicente Watson and Joseph Tosi

Abstract: Objectively assessing the performance of a model and deriving model parameter values from observations are critical and challenging in landscape to regional modeling. In this paper, we applied a nonlinear inversion technique to calibrate the ecosystem model CENTURY against carbon and nitrogen stock measurements collected from 33 mature tropical forest sites in seven life zones in Costa Rica. Net primary productivity from the MODIS, C and N stocks in aboveground live biomass, litter, coarse woody debris (CWD), and in soils were used to calibrate the model. To investigate the resolution of available observations on the number of adjustable parameters, inversion was performed using nine setups of adjustable parameters. Statistics including observation sensitivity, parameter correlation coefficient, parameter sensitivity, and parameter confidence limits were used to evaluate the information content of observations, resolution of model parameters, and overall model performance. Results indicated that soil organ carbon content, soil nitrogen content, and total aboveground biomass carbon had the highest information contents, while measurements of carbon in litter and nitrogen in CWD contributed little to the parameter estimation processes. The available information could resolve the values of 2 to 4 parameters. Adjusting just one parameter resulted in underfitting and unacceptable model performance, while adjusting five parameters simultaneously led to overfitting. Results further indicated that the MODIS NPP values were compressed as compared with the spatial variability of net primary production (NPP) values inferred from inverse modeling. Using inverse modeling to infer NPP and other model sensitive model parameters from C and N stock observations provides an opportunity to utilize data collected by national to regional forest inventory systems to reduce the uncertainties in the carbon cycle and generate valuable databases to validate and improve MODIS NPP algorithms.

Title: Modelling Support To Improve The Quality Of Environmental Studies

Authors: Huub Scholten

Abstract: Sound decisions in environmental management are often based on modelling, performed by teams consisting of team members with multidisciplinary backgrounds and solving multidisciplinary problems. Modelling in itself is a risky activity, in which many things can go wrong, including miscommunication within the team and between the team and the problem owner, malpractice, lack of process knowledge and a lack of documentation. This kind of problems makes modelling an obscure and arbitrary practice. The multidisciplinary character enhances the problem-solving scope of model studies, but does further increase the problems. Projects that use methods and tools for multidisciplinary model-based problem solving should be based on a sound, explicit methodology, described in a clear definition of the whole modelling process. Such a definition can be used as guidance for the modelling team. This guidance should be filtered for the role of a team member and to the disciplinary characteristics of a project. Next to the guidance on what modelling teams have to do, records should be kept of what they actually do and why they do it in that way. Together, guidance and records, provide transparency of modelling projects, allow audits, improve quality and make model studies reconstructable.