RADAR-BASED STOCHASTIC PRECIPITATION NOWCASTING USING THE SHORT-TERM ENSEMBLE PREDICTION SYSTEM (STEPS) (CASE STUDY: PANGKALAN BUN WEATHER RADAR)

Nowcasting, or the short-term forecasting of precipitation, is urgently needed to support the mitigation circle in hydrometeorological disasters. Pangkalan Bun weather radar is single-polarization radar with a 200 km maximum range and which runs 10 elevation angles in 10 minutes with a 250 meters spatial resolution. There is no terrain blocking around the covered area. The Short-Term Ensemble Prediction System (STEPS) is one of many algorithms that is used to generate precipitation nowcasting, and is already in operational use. STEPS has the advantage of producing ensemble nowcasts, by which nowcast uncertainties can be statistically quantified. This research aims to apply STEPS to generate stochastic nowcasting in Pangkalan Bun weather radar and to analyze its advantages and weaknesses. Accuracy is measured by counting the possibility of detection and false alarms under the 5 dBZ threshold and plotting them in a relative operating characteristic (ROC) curve. The observed frequency and forecast probability is represented by a reliability diagram to evaluate nowcast reliability and sharpness. Qualitative analysis of the results showed that the STEPS ensemble produces smoothed reflectivity fields that cannot capture extreme values in an observed quasi-linear convective system (QLCS), but that the algorithm achieves good accuracy under the threshold used, up to 40 minutes lead time. The ROC shows a curved upper left-hand corner, and the reliability diagram is an almost perfect nowcast diagonal line.

Inter-Comparison of Ensemble Forecasts for Low Level Wind Shear against Local Analyses Data over Jeju Area

Ensemble verification of low-level wind shear (LLWS) is an important issue in airplane landing operations and management. In this study, we conducted an accuracy and reliability analysis using a rank histogram, Brier score, and reliability diagram to verify LLWS ensemble member forecasts based on grid points over the Jeju area of the Republic of Korea. Thirteen LLWS ensemble member forecasts derived from a limited area ensemble prediction system (LENS) were obtained between 1 July 2016 and 30 May 2018, and 3-h LLWS forecasts for lead times up to 72 h (three days) were issued twice a day at 0000 UTC (9 am local time) and 1200 UTC (9 pm local time). We found that LLWS ensemble forecasts have a weak negative bias in summer and autumn and a positive bias in the spring and winter; the forecasts also have under-dispersion for all seasons, which implies that the ensemble spread of an ensemble is smaller than that of the corresponding observations. Additionally, the reliability curve in the associated reliability diagram indicates an over-forecasting of LLWS events bias. The selection of a forecast probability threshold from the LLWS ensemble forecast was confirmed to be one of the most important factors for issuing a severe LLWS warning. A simple method to select a forecast probability threshold without economic factors was conducted. The results showed that the selection of threshold is more useful for issuing a severe LLWS warning than none being selected.

The Calibration Simplex: A Generalization of the Reliability Diagram for Three-Category Probability Forecasts

Full exposition of the performance of a set of forecasts requires examination of the joint frequency distribution of those forecasts and their corresponding observations. In settings involving probability forecasts, this joint distribution has a high dimensionality, and communication of its information content is often best achieved graphically. This paper describes an extension of the well-known reliability diagram, which displays the joint distribution for probability forecasts of dichotomous events, to the case of probability forecasts for three disjoint events, such as “below,” “near,” and “above normal.” The resulting diagram, called the calibration simplex, involves a discretization of the 2-simplex, which is an equilateral triangle. Characteristics and interpretation of the calibration simplex are illustrated using both idealized verification datasets, and the 6–10- and 8–14-day temperature and precipitation forecasts produced by the U.S. Climate Prediction Center.

Some Remarks on the Reliability of Categorical Probability Forecasts

Studies on forecast evaluation often rely on estimating limiting observed frequencies conditioned on specific forecast probabilities (the reliability diagram or calibration function). Obviously, statistical estimates of the calibration function are based on only limited amounts of data and therefore contain residual errors. Although errors and variations of calibration function estimates have been studied previously, either they are often assumed to be small or unimportant, or they are ignored altogether. It is demonstrated how these errors can be described in terms of bias and variance, two concepts well known in the statistics literature. Bias and variance adversely affect estimates of the reliability and sharpness terms of the Brier score, recalibration of forecasts, and the assessment of forecast reliability through reliability diagram plots. Ways to communicate and appreciate these errors are presented. It is argued that these errors can become quite substantial if individual sample points have too large influence on the estimate, which can be avoided by using regularization techniques. As an illustration, it is discussed how to choose an appropriate bin size in the binning and counting method, and an appropriate bandwidth parameter for kernel estimates.

Increasing the Reliability of Reliability Diagrams

The reliability diagram is a common diagnostic graph used to summarize and evaluate probabilistic forecasts. Its strengths lie in the ease with which it is produced and the transparency of its definition. While visually appealing, major long-noted shortcomings lie in the difficulty of interpreting the graph visually; for the most part, ambiguities arise from variations in the distributions of forecast probabilities and from various binning procedures. A resampling method for assigning consistency bars to the observed frequencies is introduced that allows for immediate visual evaluation as to just how likely the observed relative frequencies are under the assumption that the predicted probabilities are reliable. Further, an alternative presentation of the same information on probability paper eases quantitative evaluation and comparison. Both presentations can easily be employed for any method of binning.