scholarly journals Understanding the relationship between clouds and surface downward radiation forecast errors with Unsupervised Deep Learning

2021 ◽  
Author(s):  
Matthias Zech ◽  
Lueder von Bremen
Keyword(s):  
Deep Learning ◽  
Unsupervised Learning ◽  
Prediction Models ◽  
Weather Prediction ◽  
Reanalysis Data ◽  
Cloud Formation ◽  
Trade Wind ◽  
Forecast Errors ◽  
Model Learning ◽  
Unsupervised Deep Learning

<p>Cloudiness is a difficult parameter to forecast and has improved relatively little over the last decade in numerical weather prediction models as the EMCWF IFS. However, surface downward solar radiation forecast (ssrd) errors are becoming more important with higher penetration of photovoltaics in Europe as forecasts errors induce power imbalances that might lead to high balancing costs. This study continues recent approaches to better understand clouds using satellite images with Deep Learning. Unlike other studies which focus on shallow trade wind cumulus clouds over the ocean, this study investigates the European land area. To better understand the clouds, we use the daily MODIS optical cloud thickness product which shows both water and ice phase of the cloud. This allows to consider both cloud structure and cloud formation during learning. It is also much easier to distinguish between snow and cloud in contrast to using visible bands. Methodologically, it uses the Unsupervised Learning approach <em>tile2vec</em> to derive a lower dimensional representation of the clouds. Three cloud regions with two similar neighboring tiles and one tile from a different time and location are sampled to learn lower-rank embeddings. In contrast to the initial <em>tile2vec</em> implementation, this study does not sample arbitrarily distant tiles but uses the fractal dimension of the clouds in a pseudo-random sampling fashion to improve model learning.</p><p>The usefulness of the cloud segments is shown by applying them in a case study to investigate statistical properties of ssrd forecast errors over Europe which are derived from hourly ECMWF IFS forecasts and ERA5 reanalysis data. This study shows how Unsupervised Learning has high potential despite its relatively low usage compared to Supervised Learning in academia. It further shows, how the generated land cloud product can be used to better characterize ssrd forecast errors over Europe.</p>

scholarly journals Deep learning for bias correction of MJO prediction

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
H. Kim ◽  
Y. G. Ham ◽  
Y. S. Joo ◽  
S. W. Son
Keyword(s):  
Deep Learning ◽  
Bias Correction ◽  
Numerical Models ◽  
Weather Prediction ◽  
Primary Source ◽  
Correction Method ◽  
Prediction Skill ◽  
Convective System ◽  
Forecast Errors ◽  
Potential Predictability

AbstractProducing accurate weather prediction beyond two weeks is an urgent challenge due to its ever-increasing socioeconomic value. The Madden-Julian Oscillation (MJO), a planetary-scale tropical convective system, serves as a primary source of global subseasonal (i.e., targeting three to four weeks) predictability. During the past decades, operational forecasting systems have improved substantially, while the MJO prediction skill has not yet reached its potential predictability, partly due to the systematic errors caused by imperfect numerical models. Here, to improve the MJO prediction skill, we blend the state-of-the-art dynamical forecasts and observations with a Deep Learning bias correction method. With Deep Learning bias correction, multi-model forecast errors in MJO amplitude and phase averaged over four weeks are significantly reduced by about 90% and 77%, respectively. Most models show the greatest improvement for MJO events starting from the Indian Ocean and crossing the Maritime Continent.

scholarly journals Unsupervised Deep Learning via Affinity Diffusion

2020 ◽  
Vol 34 (07) ◽  
pp. 11029-11036
Author(s):  
Jiabo Huang ◽  
Qi Dong ◽  
Shaogang Gong ◽  
Xiatian Zhu
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
General Purpose ◽  
Training Data ◽  
Learning Approach ◽  
Model Learning ◽  
Feature Representations ◽  
Discriminative Feature ◽  
Training Samples ◽  
Unsupervised Deep Learning

Convolutional neural networks (CNNs) have achieved unprecedented success in a variety of computer vision tasks. However, they usually rely on supervised model learning with the need for massive labelled training data, limiting dramatically their usability and deployability in real-world scenarios without any labelling budget. In this work, we introduce a general-purpose unsupervised deep learning approach to deriving discriminative feature representations. It is based on self-discovering semantically consistent groups of unlabelled training samples with the same class concepts through a progressive affinity diffusion process. Extensive experiments on object image classification and clustering show the performance superiority of the proposed method over the state-of-the-art unsupervised learning models using six common image recognition benchmarks including MNIST, SVHN, STL10, CIFAR10, CIFAR100 and ImageNet.

scholarly journals Spatial Growth of Perturbations in a Turbulent Baroclinic Jet

2011 ◽  
Vol 68 (11) ◽  
pp. 2731-2741
Author(s):  
Rahul B. Mahajan ◽  
Gregory J. Hakim
Keyword(s):  
Prediction Models ◽  
Weather Prediction ◽  
Storm Track ◽  
Spreading Rate ◽  
Forecast Errors ◽  
Far Field ◽  
Quasigeostrophic Model ◽  
Spreading Rates ◽  
Analysis Errors ◽  
Numerical Weather Prediction Models

Abstract The spatial spreading of infinitesimal disturbances superposed on a turbulent baroclinic jet is explored. This configuration is representative of analysis errors in an idealized midlatitude storm track and the insight gained may be helpful to understand the spreading of forecast errors in numerical weather prediction models. This problem is explored through numerical experiments of a turbulent baroclinic jet that is perturbed with spatially localized disturbances. Solutions from a quasigeostrophic model for the disturbance fields are compared with those for a passive tracer to determine whether disturbances propagate faster than the basic-state flow. Results show that the disturbance spreading rate is sensitive to the structure of the initial disturbance. Disturbances that are localized in potential vorticity (PV) have far-field winds that allow the disturbance to travel downstream faster than disturbances that are initially localized in geopotential, which have no far-field wind. Near the jet, the spread of the disturbance field is observed to exceed the tracer field for PV-localized disturbances, but not for the geopotential-localized disturbances. Spreading rates faster than the flow for geopotential-localized disturbances are found to occur only for disturbances located off the jet axis. These results are compared with those for zonal and time-independent jets to qualitatively assess the effects of transience and nonlinearity. This comparison suggests that the average properties of localized perturbations to the turbulent jet can be decomposed into a superposition of dynamics associated with a time-independent parallel flow plus a “diffusion” process.

Towards operational postprocessing of cloud cover at MeteoSwiss

2020 ◽  
Author(s):  
Stephan Hemri ◽  
Christoph Spirig ◽  
Jonas Bhend ◽  
Lionel Moret ◽  
Mark Liniger
Keyword(s):  
Cloud Cover ◽  
Prediction Models ◽  
Weather Prediction ◽  
Weather Forecast ◽  
Forecast Errors ◽  
Observation Data ◽  
Model Output ◽  
Ensemble Forecasts ◽  
Direct Model ◽  
Ensemble Model Output Statistics

<p>Over the last decades ensemble approaches have become state-of-the-art for the quantification of weather forecast uncertainty. Despite ongoing improvements, ensemble forecasts issued by numerical weather prediction models (NWPs) still tend to be biased and underdispersed. Statistical postprocessing has proven to be an appropriate tool to correct biases and underdispersion, and hence to improve forecast skill. Here we focus on multi-model postprocessing of cloud cover forecasts in Switzerland. In order to issue postprocessed forecasts at any point in space, ensemble model output statistics (EMOS) models are trained and verified against EUMETSAT CM SAF satellite data with a spatial resolution of around 2 km over Switzerland. Training with a minimal record length of the past 45 days of forecast and observation data already produced an EMOS model improving direct model output (DMO). Training on a 3 years record of the corresponding season further improved the performance. We evaluate how well postprocessing corrects the most severe forecast errors, like missing fog and low level stratus in winter. For such conditions, postprocessing of cloud cover benefits strongly from incorporating additional predictors into the postprocessing suite. A quasi-operational prototype has been set up and was used to explore meteogram-like visualizations of probabilistic cloud cover forecasts.</p>

Toward Better Understanding of the Contiguous Rain Area (CRA) Method for Spatial Forecast Verification

2009 ◽  
Vol 24 (5) ◽  
pp. 1401-1415 ◽  
Author(s):  
Elizabeth E. Ebert ◽  
William A. Gallus
Keyword(s):  
Correlation Coefficient ◽  
Prediction Models ◽  
Mean Squared Error ◽  
Weather Prediction ◽  
Forecast Errors ◽  
Forecast Verification ◽  
Rain Area ◽  
Squared Error ◽  
Verification Methods ◽  
The Mean

Abstract The contiguous rain area (CRA) method for spatial forecast verification is a features-based approach that evaluates the properties of forecast rain systems, namely, their location, size, intensity, and finescale pattern. It is one of many recently developed spatial verification approaches that are being evaluated as part of a Spatial Forecast Verification Methods Intercomparison Project. To better understand the strengths and weaknesses of the CRA method, it has been tested here on a set of idealized geometric and perturbed forecasts with known errors, as well as nine precipitation forecasts from three high-resolution numerical weather prediction models. The CRA method was able to identify the known errors for the geometric forecasts, but only after a modification was introduced to allow nonoverlapping forecast and observed features to be matched. For the perturbed cases in which a radar rain field was spatially translated and amplified to simulate forecast errors, the CRA method also reproduced the known errors except when a high-intensity threshold was used to define the CRA (≥10 mm h−1) and a large translation error was imposed (>200 km). The decomposition of total error into displacement, volume, and pattern components reflected the source of the error almost all of the time when a mean squared error formulation was used, but not necessarily when a correlation-based formulation was used. When applied to real forecasts, the CRA method gave similar results when either best-fit criteria, minimization of the mean squared error, or maximization of the correlation coefficient, was chosen for matching forecast and observed features. The diagnosed displacement error was somewhat sensitive to the choice of search distance. Of the many diagnostics produced by this method, the errors in the mean and peak rain rate between the forecast and observed features showed the best correspondence with subjective evaluations of the forecasts, while the spatial correlation coefficient (after matching) did not reflect the subjective judgments.

Probabilistic fire-danger forecasting: A framework for week-two forecasts using statistical post-processing techniques and the Global ECMWF Fire Forecast System (GEFF)

2021 ◽  
Author(s):  
Rochelle P. Worsnop ◽  
Michael Scheuerer ◽  
Francesca Di Giuseppe ◽  
Christopher Barnard ◽  
Thomas M. Hamill ◽  
...  
Keyword(s):  
Prediction Models ◽  
Weather Prediction ◽  
Weather Forecast ◽  
Reanalysis Data ◽  
Post Processing ◽  
Medium Range Weather Forecast ◽  
Skill Scores ◽  
Probabilistic Forecasts ◽  
Systematic Biases ◽  
Processing Techniques

AbstractWildfire guidance two weeks ahead is needed for strategic planning of fire mitigation and suppression. However, fire forecasts driven by meteorological forecasts from numerical weather prediction models inherently suffer from systematic biases. This study uses several statistical-postprocessing methods to correct these biases and increase the skill of ensemble fire forecasts over the contiguous United States 8–14 days ahead. We train and validate the post-processing models on 20 years of European Centre for Medium-range Weather Forecast (ECMWF) reforecasts and ERA5 reanalysis data for 11 meteorological variables related to fire, such as surface temperature, wind speed, relative humidity, cloud cover, and precipitation. The calibrated variables are then input to the Global ECMWF Fire Forecast (GEFF) system to produce probabilistic forecasts of daily fire-indicators which characterize the relationships between fuels, weather, and topography. Skill scores show that the post-processed forecasts overall have greater positive skill at Days 8–14 relative to raw and climatological forecasts. It is shown that the post-processed forecasts are more reliable at predicting above- and below-normal probabilities of various fire indicators than the raw forecasts and that the greatest skill for Days 8–14 is achieved by aggregating forecast days together.

Interpretation of clearness day-ahead forecast errors using novel cloud classification

2021 ◽  
Author(s):  
Matthias Zech ◽  
Lueder von Bremen
Keyword(s):  
Solar Radiation ◽  
Water Content ◽  
Radiative Forcing ◽  
Weather Prediction ◽  
Climate Science ◽  
High Water ◽  
High Water Content ◽  
Forecast Errors ◽  
High Solar Radiation ◽  
Unsupervised Deep Learning

<p>        The formation and dissipation of clouds are one of the longest studied and yet least understood phenomenon in nature. This is crucial in atmospheric and climate science as clouds have a significant impact on radiative forcing. In numerical weather prediction, solar radiation forecasts have lower skill than other parameters as temperature forecasts despite recent progresses. This study aims at better understanding cloud situations over Europe and how solar radiation forecast errors are related to these situations. Therefore, an enhanced cloud class algorithm based on unsupervised Deep Learning and hierarchical clustering is introduced. By using the MODIS optical cloud thickness product, the algorithm is able to classify 14 different daily cloud situations which are applied on defined tile regions (approximately 70,000 km²) of Europe. These different classes differ in both optical cloud phase and the overall structure of the cloud shape. The usefulness of the cloud classes is illustrated by showing regional differences of cloud type frequencies over the last 20 years. To better understand solar radiation forecast errors, the cloud classes are assigned to ECMWF IFS clearness day-ahead forecast errors. We show that high-water content and mixed-cloud phase situations lead to highest absolute forecast errors for single sites. Summed up over an area, we observe an accumulation of forecast errors for mixed-cloud phase situations whereas for other cloud situations forecast errors are more likely to cancel each other out (e.g. broken high-water content clouds). This study is useful for researchers and practitioners to better understand situations of high solar radiation errors by using the developed cloud product.</p>

scholarly journals Ensemble prediction of floods – catchment non-linearity and forecast probabilities

2007 ◽  
Vol 7 (4) ◽  
pp. 431-444 ◽  
Author(s):  
J. Komma ◽  
C. Reszler ◽  
G. Blöschl ◽  
T. Haiden
Keyword(s):  
Prediction Models ◽  
Weather Prediction ◽  
Ensemble Methods ◽  
Rain Gauge ◽  
Ensemble Prediction ◽  
Lead Times ◽  
Forecast Errors ◽  
Ensemble Spread ◽  
Relative Operating Characteristic ◽  
Data Set

Abstract. Quantifying the uncertainty of flood forecasts by ensemble methods is becoming increasingly important for operational purposes. The aim of this paper is to examine how the ensemble distribution of precipitation forecasts propagates in the catchment system, and to interpret the flood forecast probabilities relative to the forecast errors. We use the 622 km2 Kamp catchment in Austria as an example where a comprehensive data set, including a 500 yr and a 1000 yr flood, is available. A spatially-distributed continuous rainfall-runoff model is used along with ensemble and deterministic precipitation forecasts that combine rain gauge data, radar data and the forecast fields of the ALADIN and ECMWF numerical weather prediction models. The analyses indicate that, for long lead times, the variability of the precipitation ensemble is amplified as it propagates through the catchment system as a result of non-linear catchment response. In contrast, for lead times shorter than the catchment lag time (e.g. 12 h and less), the variability of the precipitation ensemble is decreased as the forecasts are mainly controlled by observed upstream runoff and observed precipitation. Assuming that all ensemble members are equally likely, the statistical analyses for five flood events at the Kamp showed that the ensemble spread of the flood forecasts is always narrower than the distribution of the forecast errors. This is because the ensemble forecasts focus on the uncertainty in forecast precipitation as the dominant source of uncertainty, and other sources of uncertainty are not accounted for. However, a number of analyses, including Relative Operating Characteristic diagrams, indicate that the ensemble spread is a useful indicator to assess potential forecast errors for lead times larger than 12 h.

Sign in / Sign up

Export Citation Format

Share Document