ITPCAS OpenIR  > 图书馆
Relative model score: a scoring rule for evaluating ensemble simulations with application to microbial soil respiration modeling
Elshall, AS (Elshall, Ahmed S.)1,8,9; Ye, M (Ye, Ming)1,2,3; Pei, YZ (Pei, Yongzhen)3; Zhang, F (Zhang, Fan)4; Niu, GY (Niu, Guo-Yue)5,6; Barron-Gafford, GA (Barron-Gafford, Greg A.)5,7
Source PublicationSTOCHASTIC ENVIRONMENTAL RESEARCH AND RISK ASSESSMENT
2018-12-01
Volume32Issue:10Pages:2809-2819
DOI10.1007/s00477-018-1592-3
Abstract

This paper defines a new scoring rule, namely relative model score (RMS), for evaluating ensemble simulations of environmental models. RMS implicitly incorporates the measures of ensemble mean accuracy, prediction interval precision, and prediction interval reliability for evaluating the overall model predictive performance. RMS is numerically evaluated from the probability density functions of ensemble simulations given by individual models or several models via model averaging. We demonstrate the advantages of using RMS through an example of soil respiration modeling. The example considers two alternative models with different fidelity, and for each model Bayesian inverse modeling is conducted using two different likelihood functions. This gives four single-model ensembles of model simulations. For each likelihood function, Bayesian model averaging is applied to the ensemble simulations of the two models, resulting in two multi-model prediction ensembles. Predictive performance for these ensembles is evaluated using various scoring rules. Results show that RMS outperforms the commonly used scoring rules of log-score, pseudo Bayes factor based on Bayesian model evidence (BME), and continuous ranked probability score (CRPS). RMS avoids the problem of rounding error specific to log-score. Being applicable to any likelihood functions, RMS has broader applicability than BME that is only applicable to the same likelihood function of multiple models. By directly considering the relative score of candidate models at each cross-validation datum, RMS results in more plausible model ranking than CRPS. Therefore, RMS is considered as a robust scoring rule for evaluating predictive performance of single-model and multi-model prediction ensembles.

Subject Area地理学
WOS IDWOS:000446064000003
Language英语
Indexed BySCIE
KeywordThermodynamic Integration Marginal Likelihood Decomposition Uncertainty Multimodel Prediction Distributions Reliability Mechanisms Selection
WOS Research AreaEngineering ; Environmental Sciences & Ecology ; Mathematics ; Water Resources
WOS SubjectEngineering, Environmental ; Engineering, Civil ; Environmental Sciences ; Statistics & Probability ; Water Resources
Cooperation Status国际
ISSN1436-3240
Department环境变化与地表过程重点实验室
PublisherSPRINGER
Citation statistics
Document Type期刊论文
Identifierhttp://ir.itpcas.ac.cn/handle/131C11/8535
Collection图书馆
Corresponding AuthorYe, M (Ye, Ming)
Affiliation1.Florida State Univ, Dept Comp Sci, Tallahassee, FL 32306 USA;
2.Florida State Univ, Dept Earth Ocean & Atmospher Sci, Tallahassee, FL 32306 USA;
3.Tianjin Polytech Univ, Sch Comp Sci & Software Engn, Tianjin 300387, Peoples R China;
4.Chinese Acad Sci, Inst Tibetan Plateau Res, Key Lab Tibetan Environm Changes & Land Surface P, Beijing 100101, Peoples R China;
5.Univ Arizona, Biosphere 2, Tucson, AZ USA;
6.Univ Arizona, Dept Hydrol & Water Resources, Tucson, AZ 85721 USA;
7.Univ Arizona, Sch Geog & Dev, Tucson, AZ USA;
8.Univ Hawaii Manoa, Dept Geol & Geophys, Honolulu, HI 96822 USA;
9.Univ Hawaii Manoa, Water Resources Res Ctr, Honolulu, HI 96822 USA.
Recommended Citation
GB/T 7714
Elshall, AS ,Ye, M ,Pei, YZ ,et al. Relative model score: a scoring rule for evaluating ensemble simulations with application to microbial soil respiration modeling[J]. STOCHASTIC ENVIRONMENTAL RESEARCH AND RISK ASSESSMENT,2018,32(10):2809-2819.
APA Elshall, AS ,Ye, M ,Pei, YZ ,Zhang, F ,Niu, GY ,&Barron-Gafford, GA .(2018).Relative model score: a scoring rule for evaluating ensemble simulations with application to microbial soil respiration modeling.STOCHASTIC ENVIRONMENTAL RESEARCH AND RISK ASSESSMENT,32(10),2809-2819.
MLA Elshall, AS ,et al."Relative model score: a scoring rule for evaluating ensemble simulations with application to microbial soil respiration modeling".STOCHASTIC ENVIRONMENTAL RESEARCH AND RISK ASSESSMENT 32.10(2018):2809-2819.
Files in This Item:
File Name/Size DocType Version Access License
2018071.pdf(1345KB)期刊论文出版稿开放获取CC BY-NC-SAView Application Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Elshall, AS (Elshall, Ahmed S.)]'s Articles
[Ye, M (Ye, Ming)]'s Articles
[Pei, YZ (Pei, Yongzhen)]'s Articles
Baidu academic
Similar articles in Baidu academic
[Elshall, AS (Elshall, Ahmed S.)]'s Articles
[Ye, M (Ye, Ming)]'s Articles
[Pei, YZ (Pei, Yongzhen)]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Elshall, AS (Elshall, Ahmed S.)]'s Articles
[Ye, M (Ye, Ming)]'s Articles
[Pei, YZ (Pei, Yongzhen)]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 2018071.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.