scholarly journals Model error covariance estimation in particle and ensemble Kalman filters using an online expectation–maximization algorithm

2020 ◽  
Author(s):  
Tadeo J. Cocucci ◽  
Manuel Pulido ◽  
Magdalena Lucini ◽  
Pierre Tandeo
Keyword(s):  
Expectation Maximization ◽  
Kalman Filters ◽  
Expectation Maximization Algorithm ◽  
Model Error ◽  
Covariance Estimation ◽  
Ensemble Kalman Filters ◽  
Error Covariance

scholarly journals Brief communication: An innovation-based estimation method for model error covariance in Kalman filters

2021 ◽  
Author(s):  
Eviatar Bach ◽  
Michael Ghil
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Kalman Filters ◽  
Estimation Method ◽  
Model Error ◽  
High Accuracy ◽  
Covariance Estimation ◽  
Observation Error ◽  
Error Covariance

Abstract. We present a simple innovation-based model error covariance estimation method for Kalman filters. The method is based on Berry and Sauer (2013) and the simplification results from assuming known observation error covariance. We carry out experiments with a prescribed model error covariance using a Lorenz (1996) model and ensemble Kalman filter. The prescribed error covariance matrix is recovered with high accuracy.

scholarly journals On Serial Observation Processing in Localized Ensemble Kalman Filters

2015 ◽  
Vol 143 (5) ◽  
pp. 1554-1567 ◽  
Author(s):  
Lars Nerger
Keyword(s):  
Ensemble Kalman Filter ◽  
Kalman Filters ◽  
Large Scale ◽  
The State ◽  
Ensemble Kalman Filters ◽  
Filter Performance ◽  
Error Covariance ◽  
Analysis Process ◽  
Scale Models ◽  
Serial Observation

Abstract Ensemble square root filters can either assimilate all observations that are available at a given time at once, or assimilate the observations in batches or one at a time. For large-scale models, the filters are typically applied with a localized analysis step. This study demonstrates that the interaction of serial observation processing and localization can destabilize the analysis process, and it examines under which conditions the instability becomes significant. The instability results from a repeated inconsistent update of the state error covariance matrix that is caused by the localization. The inconsistency is present in all ensemble Kalman filters, except for the classical ensemble Kalman filter with perturbed observations. With serial observation processing, its effect is small in cases when the assimilation changes the ensemble of model states only slightly. However, when the assimilation has a strong effect on the state estimates, the interaction of localization and serial observation processing can significantly deteriorate the filter performance. In realistic large-scale applications, when the assimilation changes the states only slightly and when the distribution of the observations is irregular and changing over time, the instability is likely not significant.

scholarly journals Efficient Local Error Parameterizations for Square Root or Ensemble Kalman Filters: Application to a Basin-Scale Ocean Turbulent Flow

2011 ◽  
Vol 139 (2) ◽  
pp. 474-493 ◽  
Author(s):  
Jean-Michel Brankart ◽  
Emmanuel Cosme ◽  
Charles-Emmanuel Testut ◽  
Pierre Brasseur ◽  
Jacques Verron
Keyword(s):  
Turbulent Flow ◽  
Covariance Matrix ◽  
Kalman Filters ◽  
Square Root ◽  
Ensemble Kalman Filters ◽  
Error Covariance Matrix ◽  
Square Roots ◽  
Schur Product ◽  
Error Covariance ◽  
Basin Scale

Abstract In large-sized atmospheric or oceanic applications of square root or ensemble Kalman filters, it is often necessary to introduce the prior assumption that long-range correlations are negligible and force them to zero using a local parameterization, supplementing the ensemble or reduced-rank representation of the covariance. One classic algorithm to perform this operation consists of taking the Schur product of the ensemble covariance matrix by a local support correlation matrix. However, with this parameterization, the square root of the forecast error covariance matrix is no more directly available, so that any observational update algorithm requiring this square root must include an additional step to compute local square roots from the Schur product. This computation generates an additional numerical cost or produces high-rank square roots, which may deprive the observational update from its original efficiency. In this paper, it is shown how efficient local square root parameterizations can be obtained, for use with a specific square root formulation (i.e., eigenbasis algorithm) of the observational update. Comparisons with the classic algorithm are provided, mainly in terms of consistency, accuracy, and computational complexity. As an application, the resulting parameterization is used to estimate maps of dynamic topography characterizing a basin-scale ocean turbulent flow. Even with this moderate-sized system (a 2200-km-wide square basin with 100-km-wide mesoscale eddies), it is observed that more than 1000 ensemble members are necessary to faithfully represent the global correlation patterns, and that a local parameterization is needed to produce correct covariances with moderate-sized ensembles. Comparisons with the exact solution show that the use of local square roots is able to improve the accuracy of the updated ensemble mean and the consistency of the updated ensemble variance. With the eigenbasis algorithm, optimal adaptive estimates of scaling factors for the forecast and observation error covariance matrix can also be obtained locally at negligible additional numerical cost. Finally, a comparison of the overall computational cost illustrates the decisive advantage that efficient local square root parameterizations may have to deal simultaneously with a larger number of observations and avoid data thinning as much as possible.

scholarly journals Efficient Parameterization of the Observation Error Covariance Matrix for Square Root or Ensemble Kalman Filters: Application to Ocean Altimetry

2009 ◽  
Vol 137 (6) ◽  
pp. 1908-1927 ◽  
Author(s):  
Jean-Michel Brankart ◽  
Clément Ubelmann ◽  
Charles-Emmanuel Testut ◽  
Emmanuel Cosme ◽  
Pierre Brasseur ◽  
...  
Keyword(s):  
Covariance Matrix ◽  
Kalman Filters ◽  
Signal Amplitude ◽  
Observation Error ◽  
Square Root ◽  
Ensemble Kalman Filters ◽  
Error Covariance Matrix ◽  
Standard Algorithm ◽  
Error Covariance ◽  
North Brazil

Abstract In the Kalman filter standard algorithm, the computational complexity of the observational update is proportional to the cube of the number y of observations (leading behavior for large y). In realistic atmospheric or oceanic applications, involving an increasing quantity of available observations, this often leads to a prohibitive cost and to the necessity of simplifying the problem by aggregating or dropping observations. If the filter error covariance matrices are in square root form, as in square root or ensemble Kalman filters, the standard algorithm can be transformed to be linear in y, providing that the observation error covariance matrix is diagonal. This is a significant drawback of this transformed algorithm and often leads to an assumption of uncorrelated observation errors for the sake of numerical efficiency. In this paper, it is shown that the linearity of the transformed algorithm in y can be preserved for other forms of the observation error covariance matrix. In particular, quite general correlation structures (with analytic asymptotic expressions) can be simulated simply by augmenting the observation vector with differences of the original observations, such as their discrete gradients. Errors in ocean altimetric observations are spatially correlated, as for instance orbit or atmospheric errors along the satellite track. Adequately parameterizing these correlations can directly improve the quality of observational updates and the accuracy of the associated error estimates. In this paper, the example of the North Brazil Current circulation is used to demonstrate the importance of this effect, which is especially significant in that region of moderate ratio between signal amplitude and observation noise, and to show that the efficient parameterization that is proposed for the observation error correlations is appropriate to take it into account. Adding explicit gradient observations also receives a physical justification. This parameterization is thus proved to be useful to ocean data assimilation systems that are based on square root or ensemble Kalman filters, as soon as the number of observations becomes penalizing, and if a sophisticated parameterization of the observation error correlations is required.

Vertical Covariance Localization for Satellite Radiances in Ensemble Kalman Filters

2010 ◽  
Vol 138 (1) ◽  
pp. 282-290 ◽  
Author(s):  
William F. Campbell ◽  
Craig H. Bishop ◽  
Daniel Hodyss
Keyword(s):  
Kalman Filters ◽  
Forecast Error ◽  
Model Space ◽  
Error Variance ◽  
Observation Error ◽  
Ensemble Kalman Filters ◽  
Weighting Functions ◽  
Error Covariance ◽  
Space Localization ◽  
Satellite Channels

Abstract A widely used observation space covariance localization method is shown to adversely affect satellite radiance assimilation in ensemble Kalman filters (EnKFs) when compared to model space covariance localization. The two principal problems are that distance and location are not well defined for integrated measurements, and that neighboring satellite channels typically have broad, overlapping weighting functions, which produce true, nonzero correlations that localization in radiance space can incorrectly eliminate. The limitations of the method are illustrated in a 1D conceptual model, consisting of three vertical levels and a two-channel satellite instrument. A more realistic 1D model is subsequently tested, using the 30 vertical levels from the Navy Operational Global Atmospheric Prediction System (NOGAPS), the Advanced Microwave Sounding Unit A (AMSU-A) weighting functions for channels 6–11, and the observation error variance and forecast error covariance from the NRL Atmospheric Variational Data Assimilation System (NAVDAS). Analyses from EnKFs using radiance space localization are compared with analyses from raw EnKFs, EnKFs using model space localization, and the optimal analyses using the NAVDAS forecast error covariance as a proxy for the true forecast error covariance. As measured by mean analysis error variance reduction, radiance space localization is inferior to model space localization for every ensemble size and meaningful observation error variance tested. Furthermore, given as many satellite channels as vertical levels, radiance space localization cannot recover the true temperature state with perfect observations, whereas model space localization can.

Sign in / Sign up

Export Citation Format

Share Document