Evaluation and Comparison of Global Ensemble Prediction Systems for Probabilistic Predictions of Heavy Rainfalls; Case Study: Kan Basin, Tehran, Iran

  Heavy rainfalls in small basins can lead to devastating flash flood, and causing a number of fatalities and tremendous damages. Thus, forecasting of heavy rainfall is an important step in development of a flood warning system. Various models were used for rainfall forecasting such as artificial neural network (Moustris et al. 2011), time series models (Sapmson et al, 2013), wavelet theory (Partal and Kişi, 2007) and regression tree model (Fallahi et al, 2011). In recent decades, the Numerical Weather Prediction (NWP) models were widely applied for weather prediction. Numerical weather predictions (NWPs) usually have uncertainties in initial conditions and model structures. In recent decades, Ensemble Prediction Systems (EPS) have been increasingly used to capture the uncertainties in NWPs. Several operational centers, including the European Centre for Medium-Range Weather Forecasts (ECMWF), the National Centers for Environmental Prediction (NCEP), the Japan Meteorological Agency (JMA), and the United Kingdom Meteorological Office (UKMO) offer valuable operational numerical predictions at a global scale (Hsiao et al, 2013).
The purpose of the present study is the comparison of the ECMWF, UKMO, and NCEP global ensemble prediction systems for forecasting of heavy rainfalls in Kan watershed, Tehran, Iran. In this paper, the performance of the global ensemble prediction models are evaluated for heavy rainfall forecasting in Kan Basin, Tehran, Iran. This research was conducted for 8 heavy rainfalls (flood producer) occurred in the study area using two different precipitation thresholds including 5 and 10 mm. For this purpose, the UKMO, NCEP and ECMWF ensemble predictions which are archived in the TIGGE database were used. Other forecast centers were not used in this study for various reasons, such as the unavailability ensemble forecasts of some centers in 0000 UTC. It is worth noting that interpolated predictions on 0.125 degree resolution were used in this study.
Then, the heavy rainfalls predicted by UKMO, NCEP and ECMWF were compared to the observed rainfall. Three criteria including the accuracy, reliability and sharpness were applied to assess the predictive efficiency of ensemble forecasters. The Brier Score, reliability diagram and average width of 50% and 90% prediction intervals were respectively used to assess the three mentioned criteria.
The Brier score is widely applied in meteorology to assess the probability and ensemble forecasts. This score is presented as following equation:† BS= 1/N ∑_(i=1)^N▒〖(P_i-O_i)〗^2 (1)
In the above equation, P_i is the forecast probability of the event, O_i is the observational value equal to 1 or 0 depending on whether the event occurred or not and N is the number of forecast-observation pairs. A minimum Brier score is equal to zero for a perfect forecaster.
Reliability diagrams is a graph of the observed frequency plotted against the forecast probability of the event. For perfect reliability, the forecast probability and the frequency of event is equal, thus the closer the reliability curve to the diameter is the higher the reliability.
Sharpness is a feature of the forecasts that refers to the concentration of the predictive distributions.
The more concentrated the predictive distributions are, the sharper the forecasts and thus the better the predictive model. The results showed that for 5 and 10 mm rainfall thresholds, UKMO’s ensembles were the least efficient, reliable and sharp. Thus UKMO’s ensembles are not suitable for heavy rainfall forecasting in the study area. It was also observed that for 5 mm rainfall threshold, there was not a significant difference between the accuracy and reliability of NCEP and ECMWF ensembles but with increasing the level of threshold to 10 mm, NCEP’s ensembles had higher efficiency and were more reliable. In terms of sharpness, NCEP’s ensembles was also the most sharp, followed by ECMWF and UKMO. Since the higher threshold is necessary for heavy rainfall prediction, so the 10 mm rainfall threshold was used in assessment of the predictions by the criteria. Comparing of the results based on three mentioned criteria showed that NCEP’s ensembles had the best performance compared to the other predictions. Thus, it is recommended to study the NCEP’s ensembles for prediction of heavy and flood producer rainfalls in mountainous watersheds like Kan Basin