scholarly journals Treatment of Observation Error due to Unresolved Scales in Atmospheric Data Assimilation

2006 ◽  
Vol 134 (10) ◽  
pp. 2900-2915 ◽  
Author(s):  
Tijana Janjić ◽  
Stephen E. Cohn
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Covariance Matrix ◽  
Covariance Function ◽  
Model Problem ◽  
Numerical Models ◽  
Observation Error ◽  
State Dependent ◽  
Full State ◽  
Assimilation Process

Abstract Observations of the atmospheric state include scales of motion that are not resolved by numerical models into which the observed data are assimilated. The resulting observation error due to unresolved scales, part of the “representativeness error,” is state dependent and correlated in time. A mathematical formalism and algorithmic approach has been developed for treating this error in the data assimilation process, under an assumption that there is no model error. The approach is based on approximating the continuum Kalman filter in such a way as to maintain terms that account for the observation error due to unresolved scales. The two resulting approximate filters resemble the Schmidt–Kalman filter and the traditional discrete Kalman filter. The approach is tested for the model problem of a passive tracer undergoing advection in a shear flow on the sphere. The state contains infinitely many spherical harmonics, with a nonstationary spectrum, and the problem is to estimate the projection of this state onto a finite spherical harmonic expansion, using observations of the full state. Numerical experiments demonstrate that approximate filters work well for the model problem provided that the exact covariance function of the unresolved scales is known. The traditional filter is more convenient in practice since it requires only the covariance matrix obtained by evaluating this covariance function at the observation points. A method for modeling this covariance matrix in the traditional filter is successful for the model problem.

scholarly journals Assimilation of Stratospheric Temperature and Ozone with an Ensemble Kalman Filter in a Chemistry–Climate Model

2011 ◽  
Vol 139 (11) ◽  
pp. 3389-3404 ◽  
Author(s):  
Thomas Milewski ◽  
Michel S. Bourqui
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Climate Model ◽  
Observation Error ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Analysis Error

Abstract A new stratospheric chemical–dynamical data assimilation system was developed, based upon an ensemble Kalman filter coupled with a Chemistry–Climate Model [i.e., the intermediate-complexity general circulation model Fast Stratospheric Ozone Chemistry (IGCM-FASTOC)], with the aim to explore the potential of chemical–dynamical coupling in stratospheric data assimilation. The system is introduced here in a context of a perfect-model, Observing System Simulation Experiment. The system is found to be sensitive to localization parameters, and in the case of temperature (ozone), assimilation yields its best performance with horizontal and vertical decorrelation lengths of 14 000 km (5600 km) and 70 km (14 km). With these localization parameters, the observation space background-error covariance matrix is underinflated by only 5.9% (overinflated by 2.1%) and the observation-error covariance matrix by only 1.6% (0.5%), which makes artificial inflation unnecessary. Using optimal localization parameters, the skills of the system in constraining the ensemble-average analysis error with respect to the true state is tested when assimilating synthetic Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) retrievals of temperature alone and ozone alone. It is found that in most cases background-error covariances produced from ensemble statistics are able to usefully propagate information from the observed variable to other ones. Chemical–dynamical covariances, and in particular ozone–wind covariances, are essential in constraining the dynamical fields when assimilating ozone only, as the radiation in the stratosphere is too slow to transfer ozone analysis increments to the temperature field over the 24-h forecast window. Conversely, when assimilating temperature, the chemical–dynamical covariances are also found to help constrain the ozone field, though to a much lower extent. The uncertainty in forecast/analysis, as defined by the variability in the ensemble, is large compared to the analysis error, which likely indicates some amount of noise in the covariance terms, while also reducing the risk of filter divergence.

scholarly journals Balance and Ensemble Kalman Filter Localization Techniques

2011 ◽  
Vol 139 (2) ◽  
pp. 511-522 ◽  
Author(s):  
Steven J. Greybush ◽  
Eugenia Kalnay ◽  
Takemasa Miyoshi ◽  
Kayo Ide ◽  
Brian R. Hunt
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Weather Prediction ◽  
Length Scale ◽  
Observation Error ◽  
Long Distance ◽  
Error Covariance Matrix ◽  
Error Covariance

Abstract In ensemble Kalman filter (EnKF) data assimilation, localization modifies the error covariance matrices to suppress the influence of distant observations, removing spurious long-distance correlations. In addition to allowing efficient parallel implementation, this takes advantage of the atmosphere’s lower dimensionality in local regions. There are two primary methods for localization. In B localization, the background error covariance matrix elements are reduced by a Schur product so that correlations between grid points that are far apart are removed. In R localization, the observation error covariance matrix is multiplied by a distance-dependent function, so that far away observations are considered to have infinite error. Successful numerical weather prediction depends upon well-balanced initial conditions to avoid spurious propagation of inertial-gravity waves. Previous studies note that B localization can disrupt the relationship between the height gradient and the wind speed of the analysis increments, resulting in an analysis that can be significantly ageostrophic. This study begins with a comparison of the accuracy and geostrophic balance of EnKF analyses using no localization, B localization, and R localization with simple one-dimensional balanced waves derived from the shallow-water equations, indicating that the optimal length scale for R localization is shorter than for B localization, and that for the same length scale R localization is more balanced. The comparison of localization techniques is then expanded to the Simplified Parameterizations, Primitive Equation Dynamics (SPEEDY) global atmospheric model. Here, natural imbalance of the slow manifold must be contrasted with undesired imbalance introduced by data assimilation. Performance of the two techniques is comparable, also with a shorter optimal localization distance for R localization than for B localization.

Evaluation of Surface Analyses and Forecasts with a Multiscale Ensemble Kalman Filter in Regions of Complex Terrain

2011 ◽  
Vol 139 (6) ◽  
pp. 2008-2024 ◽  
Author(s):  
Brian C. Ancell ◽  
Clifford F. Mass ◽  
Gregory J. Hakim
Keyword(s):  
Kalman Filter ◽  
High Resolution ◽  
Data Assimilation ◽  
Ensemble Kalman Filter ◽  
Complex Terrain ◽  
Error Variance ◽  
Small Scale ◽  
Observation Error ◽  
Grid Spacing ◽  
Background Error

Abstract Previous research suggests that an ensemble Kalman filter (EnKF) data assimilation and modeling system can produce accurate atmospheric analyses and forecasts at 30–50-km grid spacing. This study examines the ability of a mesoscale EnKF system using multiscale (36/12 km) Weather Research and Forecasting (WRF) model simulations to produce high-resolution, accurate, regional surface analyses, and 6-h forecasts. This study takes place over the complex terrain of the Pacific Northwest, where the small-scale features of the near-surface flow field make the region particularly attractive for testing an EnKF and its flow-dependent background error covariances. A variety of EnKF experiments are performed over a 5-week period to test the impact of decreasing the grid spacing from 36 to 12 km and to evaluate new approaches for dealing with representativeness error, lack of surface background variance, and low-level bias. All verification in this study is performed with independent, unassimilated observations. Significant surface analysis and 6-h forecast improvements are found when EnKF grid spacing is reduced from 36 to 12 km. Forecast improvements appear to be a consequence of increased resolution during model integration, whereas analysis improvements also benefit from high-resolution ensemble covariances during data assimilation. On the 12-km domain, additional analysis improvements are found by reducing observation error variance in order to address representativeness error. Removing model surface biases prior to assimilation significantly enhances the analysis. Inflating surface wind and temperature background error variance has large impacts on analyses, but only produces small improvements in analysis RMS errors. Both surface and upper-air 6-h forecasts are nearly unchanged in the 12-km experiments. Last, 12-km WRF EnKF surface analyses and 6-h forecasts are shown to generally outperform those of the Global Forecast System (GFS), North American Model (NAM), and the Rapid Update Cycle (RUC) by about 10%–30%, although these improvements do not extend above the surface. Based on these results, future improvements in multiscale EnKF are suggested.

scholarly journals Efficient ensemble data assimilation for coupled models with the Parallel Data Assimilation Framework: Example of AWI-CM

2019 ◽  
Author(s):  
Lars Nerger ◽  
Qi Tang ◽  
Longjiang Mu
Keyword(s):  
Data Assimilation ◽  
Numerical Models ◽  
Computing Time ◽  
Coupled Model ◽  
Ocean Model ◽  
Coupled Models ◽  
Parallel Data ◽  
Efficient Data ◽  
Abstract Data ◽  
Assimilation Process

Abstract. Data assimilation integrates information from observational measurements with numerical models. When used with coupled models of Earth system compartments, e.g. the atmosphere and the ocean, consistent joint states can be estimated. A common approach for data assimilation are ensemble-based methods which use an ensemble of state realizations to estimate the state and its uncertainty. These methods are far more costly to compute than a single coupled model because of the required integration of the ensemble. However, with uncoupled models, the methods also have been shown to exhibit a particularly good scaling behavior. This study discusses an approach to augment a coupled model with data assimilation functionality provided by the Parallel Data Assimilation Framework (PDAF). Using only minimal changes in the codes of the different compartment models, a particularly efficient data assimilation system is generated that utilizes parallelization and in-memory data transfers between the models and the data assimilation functions and hence avoids most of the filter reading and writing and also model restarts during the data assimilation process. The study explains the required modifications of the programs on the example of the coupled atmosphere-sea ice-ocean model AWI-CM. Using the case of the assimilation of oceanic observations shows that the data assimilation leads only small overheads in computing time of about 15 % compared to the model without data assimilation and a very good parallel scalability. The model-agnostic structure of the assimilation software ensures a separation of concerns in that the development of data assimilation methods and be separated from the model application.

scholarly journals Efficient Adaptive Error Parameterizations for Square Root or Ensemble Kalman Filters: Application to the Control of Ocean Mesoscale Signals

2010 ◽  
Vol 138 (3) ◽  
pp. 932-950 ◽  
Author(s):  
Jean-Michel Brankart ◽  
Emmanuel Cosme ◽  
Charles-Emmanuel Testut ◽  
Pierre Brasseur ◽  
Jacques Verron
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
Error Estimates ◽  
Forecast Error ◽  
Observation Error ◽  
Square Root ◽  
Error Statistics ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Optimal Estimates

Abstract In Kalman filter applications, an adaptive parameterization of the error statistics is often necessary to avoid filter divergence, and prevent error estimates from becoming grossly inconsistent with the real error. With the classic formulation of the Kalman filter observational update, optimal estimates of general adaptive parameters can only be obtained at a numerical cost that is several times larger than the cost of the state observational update. In this paper, it is shown that there exists a few types of important parameters for which optimal estimates can be computed at a negligible numerical cost, as soon as the computation is performed using a transformed algorithm that works in the reduced control space defined by the square root or ensemble representation of the forecast error covariance matrix. The set of parameters that can be efficiently controlled includes scaling factors for the forecast error covariance matrix, scaling factors for the observation error covariance matrix, or even a scaling factor for the observation error correlation length scale. As an application, the resulting adaptive filter is used to estimate the time evolution of ocean mesoscale signals using observations of the ocean dynamic topography. To check the behavior of the adaptive mechanism, this is done in the context of idealized experiments, in which model error and observation error statistics are known. This ideal framework is particularly appropriate to explore the ill-conditioned situations (inadequate prior assumptions or uncontrollability of the parameters) in which adaptivity can be misleading. Overall, the experiments show that, if used correctly, the efficient optimal adaptive algorithm proposed in this paper introduces useful supplementary degrees of freedom in the estimation problem, and that the direct control of these statistical parameters by the observations increases the robustness of the error estimates and thus the optimality of the resulting Kalman filter.

scholarly journals Towards variational retrieval of warm rain from passive microwave observations

2018 ◽  
Vol 11 (7) ◽  
pp. 4389-4411 ◽  
Author(s):  
David Ian Duncan ◽  
Christian D. Kummerow ◽  
Brenda Dolan ◽  
Veljko Petković
Keyword(s):  
Data Assimilation ◽  
Passive Microwave ◽  
Observation Error ◽  
Precipitation Frequency ◽  
Light Rain ◽  
Freezing Level ◽  
Warm Rain ◽  
State Dependent ◽  
Potential Synergy ◽  
Microwave Imager

Abstract. An experimental retrieval of oceanic warm rain is presented, extending a previous variational algorithm to provide a suite of retrieved variables spanning non-raining through predominantly warm raining conditions. The warm rain retrieval is underpinned by hydrometeor covariances and drizzle onset data derived from CloudSat. Radiative transfer modelling and analysis of drop size variability from disdrometer observations permit state-dependent observation error covariances that scale with columnar rainwater during iteration. The state-dependent errors and nuanced treatment of drop distributions in precipitating regions are novel and may be applicable for future retrievals and all-sky data assimilation methods. This retrieval method can effectively increase passive microwave sensors' sensitivity to light rainfall that might otherwise be missed. Comparisons with space-borne and ground radar estimates are provided as a proof of concept, demonstrating that a passive-only variational retrieval can be sufficiently constrained from non-raining through warm rain conditions. Significant deviations from forward model assumptions cause non-convergence, usually a result of scattering hydrometeors above the freezing level. However, for cases with liquid-only precipitation, this retrieval displays greater sensitivity than a benchmark operational retrieval. Analysis against passive and active products from the Global Precipitation Measurement (GPM) satellite shows substantial discrepancies in precipitation frequency, with the experimental retrieval observing more frequent light rain. This approach may be complementary to other precipitation retrievals, and its potential synergy with the operational passive GPM retrieval is briefly explored. There are also implications for data assimilation, as all 13 channels on the GPM Microwave Imager (GMI) are simulated over ocean with fidelity in warm raining conditions.

scholarly journals A New Data Assimilation Scheme: The Space-Expanded Ensemble Localization Kalman Filter

2013 ◽  
Vol 2013 ◽  
pp. 1-6 ◽  
Author(s):  
Hongze Leng ◽  
Junqiang Song ◽  
Fengshun Lu ◽  
Xiaoqun Cao
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Forecast Error ◽  
Three Dimensional ◽  
Full Rank ◽  
Observation Error ◽  
Model Framework ◽  
Background Error ◽  
Error Covariance ◽  
Expanded Ensemble

This study considers a new hybrid three-dimensional variational (3D-Var) and ensemble Kalman filter (EnKF) data assimilation (DA) method in a non-perfect-model framework, named space-expanded ensemble localization Kalman filter (SELKF). In this method, the localization operation is directly applied to the ensemble anomalies with a Schur Product, rather than to the full error covariance of the state in the EnKF. Meanwhile, the correction space of analysis increment is expanded to a space with larger dimension, and the rank of the forecast error covariance is significantly increased. This scheme can reduce the spurious correlations in the covariance and approximate the full-rank background error covariance well. Furthermore, a deterministic scheme is used to generate the analysis anomalies. The results show that the SELKF outperforms the perturbed EnKF given a relatively small ensemble size, especially when the length scale is relatively long or the observation error covariance is relatively small.

scholarly journals Data assimilation with multiple types of observation boreholes via the ensemble Kalman filter embedded within stochastic moment equations

2021 ◽  
Vol 25 (4) ◽  
pp. 1689-1709
Author(s):  
Chuan-An Xia ◽  
Xiaodong Luo ◽  
Bill X. Hu ◽  
Monica Riva ◽  
Alberto Guadagnini
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Type A ◽  
Observation Error ◽  
Type B ◽  
Moment Equations ◽  
Error Covariance Matrix ◽  
Monitoring Wells ◽  
Error Covariance

Abstract. We employ an approach based on the ensemble Kalman filter coupled with stochastic moment equations (MEs-EnKF) of groundwater flow to explore the dependence of conductivity estimates on the type of available information about hydraulic heads in a three-dimensional randomly heterogeneous field where convergent flow driven by a pumping well takes place. To this end, we consider three types of observation devices corresponding to (i) multi-node monitoring wells equipped with packers (Type A) and (ii) partially (Type B) and (iii) fully (Type C) screened wells. We ground our analysis on a variety of synthetic test cases associated with various configurations of these observation wells. Moment equations are approximated at second order (in terms of the standard deviation of the natural logarithm, Y, of conductivity) and are solved by an efficient transient numerical scheme proposed in this study. The use of an inflation factor imposed to the observation error covariance matrix is also analyzed to assess the extent at which this can strengthen the ability of the MEs-EnKF to yield appropriate conductivity estimates in the presence of a simplified modeling strategy where flux exchanges between monitoring wells and aquifer are neglected. Our results show that (i) the configuration associated with Type A monitoring wells leads to conductivity estimates with the (overall) best quality, (ii) conductivity estimates anchored on information from Type B and C wells are of similar quality, (iii) inflation of the measurement-error covariance matrix can improve conductivity estimates when a simplified flow model is adopted, and (iv) when compared with the standard Monte Carlo-based EnKF method, the MEs-EnKF can efficiently and accurately estimate conductivity and head fields.

scholarly journals Data assimilation with multiple types of observation boreholes via ensemble Kalman filter embedded within stochastic moment equations

2020 ◽  
Author(s):  
Chuan-An Xia ◽  
Xiaodong Luo ◽  
Bill X. Hu ◽  
Monica Riva ◽  
Alberto Guadagnini
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Type A ◽  
Observation Error ◽  
Type B ◽  
Moment Equations ◽  
Error Covariance Matrix ◽  
Monitoring Wells ◽  
Error Covariance

Abstract. We employ an approach based on ensemble Kalman filter coupled with stochastic moment equations (MEs-EnKF) of groundwater flow to explore the dependence of conductivity estimates on the type of available information about hydraulic heads in a three-dimensional randomly heterogeneous field where convergent flow driven by a pumping well takes place. To this end, we consider three types of observation devices, corresponding to (i) multi-node monitoring wells equipped with packers (Type A), (ii) partially (Type B) and (iii) fully (Type C) screened wells. We ground our analysis on a variety of synthetic test cases associated with various configurations of these observation wells. Moment equations are approximated at second order (in terms of the standard deviation of the natural logarithm, Y, of conductivity) and are solved by an efficient transient numerical scheme proposed in this study. The use of an inflation factor imposed to the observation error covariance matrix is also analyzed to assess the extent at which this can strengthen the ability of the MEs-EnKF to yield appropriate conductivity estimates in the presence of a simplified modeling strategy where flux exchanges between monitoring wells and aquifer are neglected. Our results show that (i) the configuration associated with Type A monitoring wells leads to conductivity estimates with the (overall) best quality; (ii) conductivity estimates anchored on information from Type B and C wells are of similar quality; (iii) inflation of the measurement-error covariance matrix can improve conductivity estimates when an incomplete/simplified flow model is adopted; and (iv) when compared with the standard Monte Carlo -based EnKF method, the MEs-EnKF can efficiently and accurately estimate conductivity and head fields.

The Computational Complexity and Parallel Scalability of Atmospheric Data Assimilation Algorithms

2004 ◽  
Vol 21 (11) ◽  
pp. 1689-1700 ◽  
Author(s):  
P. M. Lyster ◽  
J. Guo ◽  
T. Clune ◽  
J. W. Larson
Keyword(s):  
Kalman Filter ◽  
Computational Complexity ◽  
Data Assimilation ◽  
Covariance Matrix ◽  
General Circulation ◽  
Circulation Model ◽  
Parallel Scalability ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Analysis System

Abstract This paper quantifies the computational complexity and parallel scalability of two algorithms for four-dimensional data assimilation (4DDA) at NASA's Global Modeling and Assimilation Office (GMAO). The first, the Goddard Earth Observing System Data Assimilation System (GEOS DAS), uses an atmospheric general circulation model (GCM) and an observation-space-based analysis system, the Physical-Space Statistical Analysis System (PSAS). GEOS DAS is very similar to global meteorological weather forecasting data assimilation systems but is used at NASA for climate research. The second, the Kalman filter, uses a more consistent algorithm to determine the forecast error covariance matrix than does GEOS DAS. For atmospheric assimilation, the gridded dynamical fields typically have more than 106 variables; therefore, the full error covariance matrix may be in excess of a teraword. For the Kalman filter this problem will require petaflop s−1 computing to achieve effective throughput for scientific research.

Sign in / Sign up

Export Citation Format

Share Document