Comments on “Calibrated Surface Temperature Forecasts from the Canadian Ensemble Prediction System Using Bayesian Model Averaging”

2007 ◽  
Vol 135 (12) ◽  
pp. 4226-4230 ◽  
Author(s):  
Thomas M. Hamill
Keyword(s):  
Surface Temperature ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Ensemble Prediction System

scholarly journals A Local Ensemble Prediction System for Fog and Low Clouds: Construction, Bayesian Model Averaging Calibration, and Validation

2008 ◽  
Vol 47 (12) ◽  
pp. 3072-3088 ◽  
Author(s):  
Stevie Roquelaure ◽  
Thierry Bergot
Keyword(s):  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Ensemble Prediction System ◽  
Low Clouds ◽  
Wide Range ◽  
Ensemble Strategy ◽  
Low Visibility

Abstract At main international airports, air traffic safety and economic issues related to poor visibility conditions are crucial. Meteorologists face the challenge of supplying airport authorities with accurate forecasts of fog and cloud ceiling. These events are difficult to forecast because conditions evolve on short space and time scales during their life cycle. To obtain accurate forecasts of fog and low clouds, the Code de Brouillard à l’Echelle Locale (the local scale fog code)–Interactions between Soil, Biosphere, and Atmosphere (COBEL–ISBA) local numerical forecast system was implemented at Charles de Gaulle International Airport in Paris. However, even with dedicated observations and initialization, uncertainties remain in both initial conditions and mesoscale forcings. A local ensemble prediction system (LEPS) has been designed around the COBEL–ISBA numerical model and tested to assess the predictability of low visibility procedures events, defined as a visibility less than 600 m and/or a ceiling below 60 m. This work describes and evaluates a local ensemble strategy for the prediction of low visibility procedures. A Bayesian model averaging method has been applied to calibrate the ensemble. The study shows that the use of LEPS for specific local event prediction is well adapted and useful for low visibility prediction in the aeronautic context. Moreover, a wide range of users, especially those with low cost–loss ratios, can expect economic savings with the use of this probabilistic system.

A Comparison between Raw Ensemble Output, (Modified) Bayesian Model Averaging, and Extended Logistic Regression Using ECMWF Ensemble Precipitation Reforecasts

2010 ◽  
Vol 138 (11) ◽  
pp. 4199-4211 ◽  
Author(s):  
Maurice J. Schmeits ◽  
Kees J. Kok
Keyword(s):  
Logistic Regression ◽  
Bias Correction ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Ensemble Prediction ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
The Difference ◽  
Model Output Statistics

Abstract Using a 20-yr ECMWF ensemble reforecast dataset of total precipitation and a 20-yr dataset of a dense precipitation observation network in the Netherlands, a comparison is made between the raw ensemble output, Bayesian model averaging (BMA), and extended logistic regression (LR). A previous study indicated that BMA and conventional LR are successful in calibrating multimodel ensemble forecasts of precipitation for a single forecast projection. However, a more elaborate comparison between these methods has not yet been made. This study compares the raw ensemble output, BMA, and extended LR for single-model ensemble reforecasts of precipitation; namely, from the ECMWF ensemble prediction system (EPS). The raw EPS output turns out to be generally well calibrated up to 6 forecast days, if compared to the area-mean 24-h precipitation sum. Surprisingly, BMA is less skillful than the raw EPS output from forecast day 3 onward. This is due to the bias correction in BMA, which applies model output statistics to individual ensemble members. As a result, the spread of the bias-corrected ensemble members is decreased, especially for the longer forecast projections. Here, an additive bias correction is applied instead and the equation for the probability of precipitation in BMA is also changed. These modifications to BMA are referred to as “modified BMA” and lead to a significant improvement in the skill of BMA for the longer projections. If the area-maximum 24-h precipitation sum is used as a predictand, both modified BMA and extended LR improve the raw EPS output significantly for the first 5 forecast days. However, the difference in skill between modified BMA and extended LR does not seem to be statistically significant. Yet, extended LR might be preferred, because incorporating predictors that are different from the predictand is straightforward, in contrast to BMA.

Calibrated Surface Temperature Forecasts from the Canadian Ensemble Prediction System Using Bayesian Model Averaging

2007 ◽  
Vol 135 (4) ◽  
pp. 1364-1385 ◽  
Author(s):  
Laurence J. Wilson ◽  
Stephane Beauregard ◽  
Adrian E. Raftery ◽  
Richard Verret
Keyword(s):  
Surface Temperature ◽  
Bias Correction ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Training Sample ◽  
Ensemble Prediction ◽  
Ensemble Forecasts ◽  
Modest Improvement ◽  
Component Models

Abstract Bayesian model averaging (BMA) has recently been proposed as a way of correcting underdispersion in ensemble forecasts. BMA is a standard statistical procedure for combining predictive distributions from different sources. The output of BMA is a probability density function (pdf), which is a weighted average of pdfs centered on the bias-corrected forecasts. The BMA weights reflect the relative contributions of the component models to the predictive skill over a training sample. The variance of the BMA pdf is made up of two components, the between-model variance, and the within-model error variance, both estimated from the training sample. This paper describes the results of experiments with BMA to calibrate surface temperature forecasts from the 16-member Canadian ensemble system. Using one year of ensemble forecasts, BMA was applied for different training periods ranging from 25 to 80 days. The method was trained on the most recent forecast period, then applied to the next day’s forecasts as an independent sample. This process was repeated through the year, and forecast quality was evaluated using rank histograms, the continuous rank probability score, and the continuous rank probability skill score. An examination of the BMA weights provided a useful comparative evaluation of the component models, both for the ensemble itself and for the ensemble augmented with the unperturbed control forecast and the higher-resolution deterministic forecast. Training periods around 40 days provided a good calibration of the ensemble dispersion. Both full regression and simple bias-correction methods worked well to correct the bias, except that the full regression failed to completely remove seasonal trend biases in spring and fall. Simple correction of the bias was sufficient to produce positive forecast skill out to 10 days with respect to climatology, which was improved by the BMA. The addition of the control forecast and the full-resolution model forecast to the ensemble produced modest improvement in the forecasts for ranges out to about 7 days. Finally, BMA produced significantly narrower 90% prediction intervals compared to a simple Gaussian bias correction, while achieving similar overall accuracy.

scholarly journals Calibration Of Rainfall Ensemble Prediction Of ECMWF System 4 Using Bayesian Model Averaging

Agromet ◽  
2020 ◽  
Vol 34 (1) ◽  
pp. 20-33
Author(s):  
Robi Muharsyah ◽  
Tri Wahyu Hadi ◽  
Sapto Wahyu Indratno
Keyword(s):  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Skill Score ◽  
Seasonal Forecast ◽  
Probabilistic Forecast ◽  
Ensemble Prediction ◽  
Operating Characteristics ◽  
Ensemble Prediction System ◽  
Medium Range

Bayesian model averaging (BMA) is a statistical post-processing method for producing probabilistic forecasts from ensembles in the form of predictive PDFs. It is known that BMA is able to improve the reliability of probabilistic forecast of short range and medium range rainfall forecast. This study aims to develop the application of BMA to calibrate seasonal forecast (long range) in order to improved quality of seasonal forecast in Indonesia. The seasonal forecast used is monthly rainfall from the output of the ensemble prediction system European Center for Medium-Range Weather Forecasts (ECMWF) system 4 model (ECS4) and it is calibrated against observational data at 26 stations of the Agency for Meteorology Climatology and Geophysics of Republic of Indonesia (BMKG) in Java Island in 1981 – 2018. BMA predictive PDFs is generated with Gamma distribution approach which is obtained based on sequential training windows (JTS) and conditionals training windows (JTC). BMA-JTS approach is done by selecting the width of the 30-month training window as the optimal training period while the BMA-JTC is carried out with a cross-validation scheme for each month. In general, Both of BMA-JTS and BMA-JTC better than RAW models. BMA-JTC calibration results are varying according to spatial and temporal, but in general the result is better in the dry season and during the El Nino phase. BMA is able to improve the distribution characteristics of the RAW model ECS4 prediction which is shown by: a smaller value of Continuous Rank Probability Score (CRPS), a larger value of the Continuous Rank Probability Skill Score (CRPSS) and more flat form of the Verification Rank Histogram (VRH) than the RAW model. BMA also increases the skill, esolution and reliability of prediction of probability Below Normal (BN) and Above Normal (AN), which is known from the increasing Brier Skill Score (BSS), and the increasing area under curve of Relative Operating Characteristics (ROC) compared to the RAW model. Furthermore, the reliability of BN and AN of BMA results also has the category of “still very useful” and “perfect” compared to RAW models that are in the “dangerous”, “not useful” and “marginally useful” categories. The reliability of BMA results with the category “still very useful” and “perfect” show that the probabilistic forecast of BN and AN events can be used in making decisions related to seasonal forecast.

scholarly journals Improving prediction quality of sea surface temperature (SST) in Niño3.4 region using Bayesian Model Averaging

2021 ◽  
Vol 893 (1) ◽  
pp. 012028
Author(s):  
Robi Muharsyah ◽  
Dian Nur Ratri ◽  
Damiana Fitria Kussatiti
Keyword(s):  
Surface Temperature ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Seasonal Forecast ◽  
Sea Surface ◽  
Post Processing ◽  
Enso Events ◽  
Prediction Quality

Abstract Prediction of Sea Surface Temperature (SST) in Niño3.4 region (170 W - 120 W; 5S - 5N) is important as a valuable indicator to identify El Niño Southern Oscillation (ENSO), i.e., El Niño, La Niña, and Neutral condition for coming months. More accurate prediction Niño3.4 SST can be used to determine the response of ENSO phenomenon to rainfall over Indonesia region. SST predictions are routinely released by meteorological institutions such as the European Center for Medium-Range Weather Forecasts (ECMWF). However, SST predictions from the direct output (RAW) of global models such as ECMWF seasonal forecast is suffering from bias that affects the poor quality of SST predictions. As a result, it also increases the potential errors in predicting the ENSO events. This study uses SST from the output Ensemble Prediction System (EPS) of ECMWF seasonal forecast, namely SEAS5. SEAS5 SST is downloaded from The Copernicus Climate Change Service (C3S) for period 1993-2020. One value representing SST over Niño3.4 region is calculated for each lead-time (LT), LT0-LT6. Bayesian Model Averaging (BMA) is selected as one of the post-processing methods to improve the prediction quality of SEAS5-RAW. The advantage of BMA over other post-processing methods is its ability to quantify the uncertainty in EPS, which is expressed as probability density function (PDF) predictive. It was found that the BMA calibration process reaches optimal performance using 160 months training window. The result show, prediction quality of Niño3.4 SST of BMA output is superior to SEAS5-RAW, especially for LT0, LT1, and LT2. In term deterministic prediction, BMA shows a lower Root Mean Square Error (RMSE), higher Proportion of Correct (PC). In term probabilistic prediction, the error rate of BMA, which is showed by the Brier Score is lower than RAW. Moreover, BMA shows a good ability to discriminating ENSO events which indicates by AUC ROC close to a perfect score.

Multi-model ensemble prediction of terrestrial evapotranspiration across north China using Bayesian model averaging

2016 ◽  
Vol 30 (16) ◽  
pp. 2861-2879 ◽  
Author(s):  
Gaofeng Zhu ◽  
Xin Li ◽  
Kun Zhang ◽  
Zhenyu Ding ◽  
Tuo Han ◽  
...  
Keyword(s):  
Bayesian Model ◽  
North China ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Ensemble Prediction ◽  
Model Ensemble

scholarly journals Relative effects of statistical preprocessing and postprocessing on a regional hydrological ensemble prediction system

2017 ◽  
Author(s):  
Sanjib Sharma ◽  
Ridwan Siddique ◽  
Seann Reed ◽  
Peter Ahnert ◽  
Pablo Mendoza ◽  
...  
Keyword(s):  
Surface Temperature ◽  
Warm Season ◽  
Hydrologic Model ◽  
Laboratory Research ◽  
Ensemble Prediction ◽  
Lead Times ◽  
Prediction System ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Near Surface

Abstract. The relative roles of statistical weather preprocessing and streamflow postprocessing in hydrological ensemble forecasting at short- to medium-range forecast lead times (day 1–7) are investigated. For this purpose, a regional hydrologic ensemble prediction system (RHEPS) is developed and implemented. The RHEPS is comprised by the following components: i) hydrometeorological observations (multisensor precipitation estimates, gridded surface temperature, and gauged streamflow); ii) weather ensemble forecasts (precipitation and near-surface temperature) from the National Centers for Environmental Prediction 11-member Global Ensemble Forecast System Reforecast version 2 (GEFSRv2); iii) NOAA’s Hydrology Laboratory-Research Distributed Hydrologic Model (HL-RDHM); iv) heteroscedastic censored logistic regression (HCLR) as the statistical preprocessor; v) two statistical postprocessors, an autoregressive model with a single exogenous variable (ARX(1,1)) and quantile regression (QR); and vi) a comprehensive verification strategy. To implement the RHEPS, 1 to 7 days weather forecasts from the GEFSRv2 are used to force HL-RDHM and generate raw ensemble streamflow forecasts. Forecasting experiments are conducted in four nested basins in the U.S. middle Atlantic region, ranging in size from 381 to 12,362 km2. Results show that the HCLR preprocessed ensemble precipitation forecasts have greater skill than the raw forecasts. These improvements are more noticeable in the warm season at the longer lead times (> 3 days). Both postprocessors, ARX(1,1) and QR, show gains in skill relative to the raw ensemble flood forecasts but QR outperforms ARX(1,1). Preprocessing alone has little effect on improving the skill of the ensemble flood forecasts. Indeed, postprocessing alone performs similar, in terms of the relative mean error, skill, and reliability, to the more involved scenario that includes both preprocessing and postprocessing. We conclude that statistical preprocessing may not always be a necessary component of the ensemble flood forecasting chain.

Sign in / Sign up

Export Citation Format

Share Document