Impact of flow-dependent analysis-error covariance norms on extratropical singular vectors

2006 ◽  
Vol 132 (615) ◽  
pp. 625-646 ◽  
Author(s):  
Mark Buehner ◽  
Ayrton Zadra
Keyword(s):  
Singular Vectors ◽  
Error Covariance ◽  
Analysis Error

scholarly journals Ensemble Forecasts and the Properties of Flow-Dependent Analysis-Error Covariance Singular Vectors

2003 ◽  
Vol 131 (8) ◽  
pp. 1741-1758 ◽  
Author(s):  
Thomas M. Hamill ◽  
Chris Snyder ◽  
Jeffrey S. Whitaker
Keyword(s):  
Total Energy ◽  
Linear Combination ◽  
Singular Vector ◽  
Energy Norm ◽  
Initial Time ◽  
Forecast Errors ◽  
Ensemble Forecasts ◽  
Singular Vectors ◽  
Error Covariance ◽  
Analysis Error

Abstract Approximations to flow-dependent analysis-error covariance singular vectors (AEC SVs) were calculated in a dry, T31 L15 primitive-equation global model. Sets of 400-member ensembles of analyses were generated by an ensemble-based data assimilation system. A sparse network of simulated rawinsonde observations were assimilated, and a perfect model was assumed. Ensembles of 48-h forecasts were also generated from these analyses. The structure of evolved singular vectors was determined by finding the linear combination of the forecast ensemble members that resulted in the largest forecast-error variance, here measured in a total-energy norm north of 20°N latitude. The same linear combination of analyses specifies the initial-time structure that should evolve to the forecast singular vector under assumptions of linearity of error growth. The structures of these AEC SVs are important because they represent the analysis-error structures associated with the largest forecast errors. If singular vectors using other initial norms have very different structures, this indicates that these structures may be statistically unlikely to occur. The European Centre for Medium-Range Weather Forecasts currently uses singular vectors using an initial total-energy norm [“total-energy singular vectors” or (TE SVs)] to generate perturbations to initialize their ensemble forecasts. Approximate TE SVs were also calculated by drawing an initial random ensemble with perturbations that were white in total energy and applying the same approach as for AEC SVs. Comparing AEC SVs and approximate TE SVs, the AEC SVs had maximum amplitude in midlatitudes near the tropopause, both at the initial and evolved times. The AEC SVs were synoptic in scale, deep, and did not appear to be geographically localized nor tilted dramatically upshear. This contrasts with TE SVs, which started off relatively smaller in scale, were tilted upshear, and had amplitudes typically largest in the lower to midtroposphere. The difference between AEC SVs and TE SVs suggests that operational ensemble forecasts based on TE SVs could be improved by changing the type of singular vector used to generate initial perturbations. This is particularly true for short-range ensemble forecasts, where the structure of the forecast ensemble is more closely tied to the analysis ensemble.

Singular vectors and estimates of the analysis-error covariance metric

1998 ◽  
Vol 124 (549) ◽  
pp. 1695-1713 ◽  
Author(s):  
Jan Barkmeijer ◽  
François Bouttier ◽  
Martin Van Gijzen
Keyword(s):  
Singular Vectors ◽  
Error Covariance ◽  
Analysis Error

Maximum Likelihood Ensemble Filter: Theoretical Aspects

2005 ◽  
Vol 133 (6) ◽  
pp. 1710-1726 ◽  
Author(s):  
Milija Zupanski
Keyword(s):  
Maximum Likelihood ◽  
Data Assimilation ◽  
Cost Function ◽  
Ensemble Data Assimilation ◽  
Error Covariance ◽  
Ensemble Data ◽  
Analysis Error ◽  
Nonlinear Observation ◽  
The Cost ◽  
Maximum Likelihood Ensemble Filter

Abstract A new ensemble-based data assimilation method, named the maximum likelihood ensemble filter (MLEF), is presented. The analysis solution maximizes the likelihood of the posterior probability distribution, obtained by minimization of a cost function that depends on a general nonlinear observation operator. The MLEF belongs to the class of deterministic ensemble filters, since no perturbed observations are employed. As in variational and ensemble data assimilation methods, the cost function is derived using a Gaussian probability density function framework. Like other ensemble data assimilation algorithms, the MLEF produces an estimate of the analysis uncertainty (e.g., analysis error covariance). In addition to the common use of ensembles in calculation of the forecast error covariance, the ensembles in MLEF are exploited to efficiently calculate the Hessian preconditioning and the gradient of the cost function. A sufficient number of iterative minimization steps is 2–3, because of superior Hessian preconditioning. The MLEF method is well suited for use with highly nonlinear observation operators, for a small additional computational cost of minimization. The consistent treatment of nonlinear observation operators through optimization is an advantage of the MLEF over other ensemble data assimilation algorithms. The cost of MLEF is comparable to the cost of existing ensemble Kalman filter algorithms. The method is directly applicable to most complex forecast models and observation operators. In this paper, the MLEF method is applied to data assimilation with the one-dimensional Korteweg–de Vries–Burgers equation. The tested observation operator is quadratic, in order to make the assimilation problem more challenging. The results illustrate the stability of the MLEF performance, as well as the benefit of the cost function minimization. The improvement is noted in terms of the rms error, as well as the analysis error covariance. The statistics of innovation vectors (observation minus forecast) also indicate a stable performance of the MLEF algorithm. Additional experiments suggest the amplified benefit of targeted observations in ensemble data assimilation.

Estimates of ocean forecast error covariance derived from Hessian Singular Vectors

2015 ◽  
Vol 89 ◽  
pp. 104-121 ◽  
Author(s):  
Kevin D. Smith ◽  
Andrew M. Moore ◽  
Hernan G. Arango
Keyword(s):  
Forecast Error ◽  
Singular Vectors ◽  
Error Covariance

scholarly journals Assimilation of Stratospheric Temperature and Ozone with an Ensemble Kalman Filter in a Chemistry–Climate Model

2011 ◽  
Vol 139 (11) ◽  
pp. 3389-3404 ◽  
Author(s):  
Thomas Milewski ◽  
Michel S. Bourqui
Keyword(s):  
Kalman Filter ◽  
Data Assimilation ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Climate Model ◽  
Observation Error ◽  
Background Error ◽  
Error Covariance Matrix ◽  
Error Covariance ◽  
Analysis Error

Abstract A new stratospheric chemical–dynamical data assimilation system was developed, based upon an ensemble Kalman filter coupled with a Chemistry–Climate Model [i.e., the intermediate-complexity general circulation model Fast Stratospheric Ozone Chemistry (IGCM-FASTOC)], with the aim to explore the potential of chemical–dynamical coupling in stratospheric data assimilation. The system is introduced here in a context of a perfect-model, Observing System Simulation Experiment. The system is found to be sensitive to localization parameters, and in the case of temperature (ozone), assimilation yields its best performance with horizontal and vertical decorrelation lengths of 14 000 km (5600 km) and 70 km (14 km). With these localization parameters, the observation space background-error covariance matrix is underinflated by only 5.9% (overinflated by 2.1%) and the observation-error covariance matrix by only 1.6% (0.5%), which makes artificial inflation unnecessary. Using optimal localization parameters, the skills of the system in constraining the ensemble-average analysis error with respect to the true state is tested when assimilating synthetic Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) retrievals of temperature alone and ozone alone. It is found that in most cases background-error covariances produced from ensemble statistics are able to usefully propagate information from the observed variable to other ones. Chemical–dynamical covariances, and in particular ozone–wind covariances, are essential in constraining the dynamical fields when assimilating ozone only, as the radiation in the stratosphere is too slow to transfer ozone analysis increments to the temperature field over the 24-h forecast window. Conversely, when assimilating temperature, the chemical–dynamical covariances are also found to help constrain the ozone field, though to a much lower extent. The uncertainty in forecast/analysis, as defined by the variability in the ensemble, is large compared to the analysis error, which likely indicates some amount of noise in the covariance terms, while also reducing the risk of filter divergence.

Development of a nonlinear ensemble data assimilation method with global state-space covariance localization

2020 ◽  
Author(s):  
Milija Zupanski
Keyword(s):  
Data Assimilation ◽  
State Space ◽  
High Dimensional ◽  
Ensemble Data Assimilation ◽  
Error Covariance ◽  
Ensemble Data ◽  
Space Localization ◽  
Initial Ensemble ◽  
Analysis Error ◽  
Observation Space

<p>High-dimensional ensemble data assimilation applications require error covariance localization in order to address the problem of insufficient degrees of freedom, typically accomplished using the observation-space covariance localization. However, this creates a challenge for vertically integrated observations, such as satellite radiances, aerosol optical depth, etc., since the exact observation location in vertical does not exist. For nonlinear problems, there is an implied inconsistency in iterative minimization due to using observation-space localization which effectively prevents finding the optimal global minimizing solution. Using state-space localization, however, in principal resolves both issues associated with observation space localization.</p><p> </p><p>In this work we present a new nonlinear ensemble data assimilation method that employs covariance localization in state space and finds an optimal analysis solution. The new method resembles “modified ensembles” in the sense that ensemble size is increased in the analysis, but it differs in methodology used to create ensemble modifications, calculate the analysis error covariance, and define the initial ensemble perturbations for data assimilation cycling. From a practical point of view, the new method is considerably more efficient and potentially applicable to realistic high-dimensional data assimilation problems. A distinct characteristic of the new algorithm is that the localized error covariance and minimization are global, i.e. explicitly defined over all state points. The presentation will focus on examining feasible options for estimating the analysis error covariance and for defining the initial ensemble perturbations.</p>

scholarly journals An estimate of the inflation factor and analysis sensitivity in the ensemble Kalman filter

2017 ◽  
Vol 24 (3) ◽  
pp. 329-341 ◽  
Author(s):  
Guocan Wu ◽  
Xiaogu Zheng
Keyword(s):  
Kalman Filter ◽  
Covariance Matrix ◽  
Ensemble Kalman Filter ◽  
Forecast Error ◽  
Model Errors ◽  
Error Covariance Matrix ◽  
Spatially Correlated ◽  
Error Covariance ◽  
Inflation Factor ◽  
Analysis Error

Abstract. The ensemble Kalman filter (EnKF) is a widely used ensemble-based assimilation method, which estimates the forecast error covariance matrix using a Monte Carlo approach that involves an ensemble of short-term forecasts. While the accuracy of the forecast error covariance matrix is crucial for achieving accurate forecasts, the estimate given by the EnKF needs to be improved using inflation techniques. Otherwise, the sampling covariance matrix of perturbed forecast states will underestimate the true forecast error covariance matrix because of the limited ensemble size and large model errors, which may eventually result in the divergence of the filter. In this study, the forecast error covariance inflation factor is estimated using a generalized cross-validation technique. The improved EnKF assimilation scheme is tested on the atmosphere-like Lorenz-96 model with spatially correlated observations, and is shown to reduce the analysis error and increase its sensitivity to the observations.

scholarly journals Formulations for Estimating Spatial Variations of Analysis Error Variance to Improve Multiscale and Multistep Variational Data Assimilation

2018 ◽  
Vol 2018 ◽  
pp. 1-17
Author(s):  
Qin Xu ◽  
Li Wei
Keyword(s):  
Data Assimilation ◽  
Spatial Variation ◽  
Variational Analysis ◽  
Variance Reduction ◽  
Error Variance ◽  
Spatial Variations ◽  
Variational Data Assimilation ◽  
Coarse Resolution ◽  
Error Covariance ◽  
Analysis Error

When the coarse-resolution observations used in the first step of multiscale and multistep variational data assimilation become increasingly nonuniform and/or sparse, the error variance of the first-step analysis tends to have increasingly large spatial variations. However, the analysis error variance computed from the previously developed spectral formulations is constant and thus limited to represent only the spatially averaged error variance. To overcome this limitation, analytic formulations are constructed to efficiently estimate the spatial variation of analysis error variance and associated spatial variation in analysis error covariance. First, a suite of formulations is constructed to efficiently estimate the error variance reduction produced by analyzing the coarse-resolution observations in one- and two-dimensional spaces with increased complexity and generality (from uniformly distributed observations with periodic extension to nonuniformly distributed observations without periodic extension). Then, three different formulations are constructed for using the estimated analysis error variance to modify the analysis error covariance computed from the spectral formulations. The successively improved accuracies of these three formulations and their increasingly positive impacts on the two-step variational analysis (or multistep variational analysis in first two steps) are demonstrated by idealized experiments.

scholarly journals Mitigating Observation Perturbation Sampling Errors in the Stochastic EnKF

2015 ◽  
Vol 143 (7) ◽  
pp. 2918-2936 ◽  
Author(s):  
I. Hoteit ◽  
D.-T. Pham ◽  
M. E. Gharamti ◽  
X. Luo
Keyword(s):  
Ensemble Kalman Filter ◽  
Forecast Error ◽  
Sampling Errors ◽  
Observational Error ◽  
Error Covariance ◽  
Atmospheric Data ◽  
Observational Errors ◽  
Analysis Error ◽  
Lorenz 96 ◽  
The Impact

Abstract The stochastic ensemble Kalman filter (EnKF) updates its ensemble members with observations perturbed with noise sampled from the distribution of the observational errors. This was shown to introduce noise into the system and may become pronounced when the ensemble size is smaller than the rank of the observational error covariance, which is often the case in real oceanic and atmospheric data assimilation applications. This work introduces an efficient serial scheme to mitigate the impact of observations’ perturbations sampling in the analysis step of the EnKF, which should provide more accurate ensemble estimates of the analysis error covariance matrices. The new scheme is simple to implement within the serial EnKF algorithm, requiring only the approximation of the EnKF sample forecast error covariance matrix by a matrix with one rank less. The new EnKF scheme is implemented and tested with the Lorenz-96 model. Results from numerical experiments are conducted to compare its performance with the EnKF and two standard deterministic EnKFs. This study shows that the new scheme enhances the behavior of the EnKF and may lead to better performance than the deterministic EnKFs even when implemented with relatively small ensembles.

Sign in / Sign up

Export Citation Format

Share Document