scholarly journals Ensemble Forecasts and the Properties of Flow-Dependent Analysis-Error Covariance Singular Vectors

2003 ◽  
Vol 131 (8) ◽  
pp. 1741-1758 ◽  
Author(s):  
Thomas M. Hamill ◽  
Chris Snyder ◽  
Jeffrey S. Whitaker
Keyword(s):  
Total Energy ◽  
Linear Combination ◽  
Singular Vector ◽  
Energy Norm ◽  
Initial Time ◽  
Forecast Errors ◽  
Ensemble Forecasts ◽  
Singular Vectors ◽  
Error Covariance ◽  
Analysis Error

Abstract Approximations to flow-dependent analysis-error covariance singular vectors (AEC SVs) were calculated in a dry, T31 L15 primitive-equation global model. Sets of 400-member ensembles of analyses were generated by an ensemble-based data assimilation system. A sparse network of simulated rawinsonde observations were assimilated, and a perfect model was assumed. Ensembles of 48-h forecasts were also generated from these analyses. The structure of evolved singular vectors was determined by finding the linear combination of the forecast ensemble members that resulted in the largest forecast-error variance, here measured in a total-energy norm north of 20°N latitude. The same linear combination of analyses specifies the initial-time structure that should evolve to the forecast singular vector under assumptions of linearity of error growth. The structures of these AEC SVs are important because they represent the analysis-error structures associated with the largest forecast errors. If singular vectors using other initial norms have very different structures, this indicates that these structures may be statistically unlikely to occur. The European Centre for Medium-Range Weather Forecasts currently uses singular vectors using an initial total-energy norm [“total-energy singular vectors” or (TE SVs)] to generate perturbations to initialize their ensemble forecasts. Approximate TE SVs were also calculated by drawing an initial random ensemble with perturbations that were white in total energy and applying the same approach as for AEC SVs. Comparing AEC SVs and approximate TE SVs, the AEC SVs had maximum amplitude in midlatitudes near the tropopause, both at the initial and evolved times. The AEC SVs were synoptic in scale, deep, and did not appear to be geographically localized nor tilted dramatically upshear. This contrasts with TE SVs, which started off relatively smaller in scale, were tilted upshear, and had amplitudes typically largest in the lower to midtroposphere. The difference between AEC SVs and TE SVs suggests that operational ensemble forecasts based on TE SVs could be improved by changing the type of singular vector used to generate initial perturbations. This is particularly true for short-range ensemble forecasts, where the structure of the forecast ensemble is more closely tied to the analysis ensemble.

Cyclogenetic Perturbations and Analysis Errors Decomposed into Singular Vectors

2005 ◽  
Vol 62 (7) ◽  
pp. 2234-2247 ◽  
Author(s):  
Chris Snyder ◽  
Gregory J. Hakim
Keyword(s):  
Potential Vorticity ◽  
Vertical Structure ◽  
Energy Norm ◽  
Vertical Shear ◽  
Initial Time ◽  
Forecast Errors ◽  
Ensemble Forecasts ◽  
Singular Vectors ◽  
Analysis Error ◽  
Analysis Errors

Abstract Singular vectors (SVs) have been applied to cyclogenesis, to initializing ensemble forecasts, and in predictability studies. Ideally, the calculation of the SVs would employ the analysis error covariance norm at the initial time or, in the case of cyclogenesis, a norm based on the statistics of initial perturbations, but the energy norm is often used as a more practical substitute. To illustrate the roles of the choice of norm and the vertical structure of initial perturbations, an upper-level wave with no potential vorticity perturbation in the troposphere is considered as a typical cyclogenetic perturbation or analysis error, and this perturbation is then decomposed by its projection onto each energy SV. All calculations are made, for simplicity, in the context of the quasigeostrophic Eady model (i.e., for a background flow with constant vertical shear and horizontal temperature gradient). Viewed in terms of the energy SVs, the smooth vertical structure of the typical perturbation, as well as its evolution, results from strong cancellation between the growing and decaying SVs, most of which are highly structured and tilted in the vertical. A simpler picture, involving less cancellation, follows from decomposition of the typical perturbation into SVs using an alternative initial norm, which is based on the relation between initial norms and the statistics of initial perturbations together with the empirical assumption that the initial perturbations are not dominated by interior potential vorticity. Differences between the energy SVs and those based on the alternative initial norm can be understood by noting that the energy norm implicitly assumes initial perturbations with second-order statistics given by the covariance matrix whose inverse defines the energy norm. Unlike the “typical” perturbation, perturbations with those statistics have large variance of potential vorticity in the troposphere and fine vertical structure. Finally, a brief assessment is presented of the extent to which the upper wave, and more generally the alternative initial norm, is representative of cyclogenetic perturbations and analysis errors. There is substantial evidence supporting deep perturbations with little vertical structure as frequent precursors to cyclogenesis, but surrogates for analysis errors are less conclusive: operational midlatitude analysis differences have vertical structure similar to that of the perturbations implied by the energy norm, while short-range forecast errors and analysis errors from assimilation experiments with simulated observations are more consistent with the alternative norm.

On Ensemble Prediction Using Singular Vectors Started from Forecasts

2005 ◽  
Vol 133 (10) ◽  
pp. 3038-3046 ◽  
Author(s):  
Martin Leutbecher
Keyword(s):  
Singular Vector ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Singular Vectors ◽  
The Tropics ◽  
The Impact ◽  
Very High ◽  
Tropical Cyclone Tracks

Abstract The impact on the ECMWF Ensemble Prediction System of using singular vectors computed from 12-h forecasts instead of analyses has been studied. Results are based on 34 cases in November–December 1999 and 28 cases in September 2003. The similarity between singular vectors started from a 12-h forecast and singular vectors started from an analysis is very high for the extratropical singular vectors in each of the 62 cases and for both hemispheres. Consistently, ensemble scores and spread measures show close to neutral impact on geopotential height in the extratropics. The sensitivity of the singular vectors to the choice of trajectory is larger in the Tropics than in the extratropics. However, the spread in tropical cyclone tracks is not significantly decreased if singular vectors computed from 12-h forecasts are used. The computation of singular vectors from forecasts could be used to disseminate the ensemble forecasts earlier or to allocate more resources to the nonlinear forecasts. Furthermore, it greatly facilitates the implementation of computationally more demanding configurations for the singular-vector-based initial perturbations.

The Norm Dependence of Singular Vectors

2004 ◽  
Vol 61 (23) ◽  
pp. 2943-2949 ◽  
Author(s):  
Zhiming Kuang
Keyword(s):  
Perturbation Theory ◽  
Singular Vector ◽  
Vector Analysis ◽  
Ensemble Forecasts ◽  
Singular Vectors ◽  
General Properties ◽  
Wide Range ◽  
Linearized System ◽  
Targeted Observations

Abstract For a linearized system such as ∂ψ/∂t = 𝗠ψ, singular vector analysis can be used to find patterns that give the largest or smallest ratios between the sizes of 𝗠ψ and ψ. Such analyses have applications to a wide range of atmosphere–ocean problems. The resulting singular vectors, however, depend on the norm used to measure the sizes of 𝗠ψ and ψ, as noted in various applications. This causes complications because the choices of norm are generally nonunique. Based on perturbation theory, a derivation of how singular vectors change with norms typically used in the atmosphere–ocean literature is provided, and it is shown that the norm dependences observed in previous studies can be understood as general properties of singular vectors. This will hopefully clarify the interpretation of these observed norm dependencies, and provide guidance to new studies on how singular vectors would vary for different norms. It is further argued, based on these results, that there may not be as much norm-related ambiguity in problems, such as designing targeted observations or ensemble forecasts, as is often assigned to them.

scholarly journals Estimation and calibration of observation impact signals using the Lanczos method in NOAA/NCEP data assimilation system

2012 ◽  
Vol 19 (5) ◽  
pp. 541-557 ◽  
Author(s):  
M. Wei ◽  
M. S. F. V. De Pondeca ◽  
Z. Toth ◽  
D. Parrish
Keyword(s):  
Data Assimilation ◽  
Weather Prediction ◽  
Lanczos Method ◽  
Forecast Errors ◽  
Observation Data ◽  
Error Covariance ◽  
Observation Impact ◽  
Analysis Error ◽  
The Tropics ◽  
The Impact

Abstract. Despite the tremendous progress that has been made in data assimilation (DA) methodology, observing systems that reduce observation errors, and model improvements that reduce background errors, the analyses produced by the best available DA systems are still different from the truth. Analysis error and error covariance are important since they describe the accuracy of the analyses, and are directly related to the future forecast errors, i.e., the forecast quality. In addition, analysis error covariance is critically important in building an efficient ensemble forecast system (EFS). Estimating analysis error covariance in an ensemble-based Kalman filter DA is straightforward, but it is challenging in variational DA systems, which have been in operation at most NWP (Numerical Weather Prediction) centers. In this study, we use the Lanczos method in the NCEP (the National Centers for Environmental Prediction) Gridpoint Statistical Interpolation (GSI) DA system to look into other important aspects and properties of this method that were not exploited before. We apply this method to estimate the observation impact signals (OIS), which are directly related to the analysis error variances. It is found that the smallest eigenvalue of the transformed Hessian matrix converges to one as the number of minimization iterations increases. When more observations are assimilated, the convergence becomes slower and more eigenvectors are needed to retrieve the observation impacts. It is also found that the OIS over data-rich regions can be represented by the eigenvectors with dominant eigenvalues. Since only a limited number of eigenvectors can be computed due to computational expense, the OIS is severely underestimated, and the analysis error variance is consequently overestimated. It is found that the mean OIS values for temperature and wind components at typical model levels are increased by about 1.5 times when the number of eigenvectors is doubled. We have proposed four different calibration schemes to compensate for the missing trailing eigenvectors. Results show that the method with calibration for a small number of eigenvectors cannot pick up the observation impacts over the regions with fewer observations as well as a benchmark with a large number of eigenvectors, but proper calibrations do enhance and improve the impact signals over regions with more data. When compared with the observation locations, the method generally captures the OIS over regions with more observation data, including satellite data over the southern oceans. Over the tropics, some observation impacts may be missed due to the smaller background errors specified in the GSI, which is not related to the method. It is found that a large number of eigenvectors are needed to retrieve impact signals that resemble the banded structures from satellite observations, particularly over the tropics. Another benefit from the Lanczos method is that the dominant eigenvectors can be used in preconditioning the conjugate gradient algorithm in the GSI to speed up the convergence.

scholarly journals Singular vector decomposition for sensitivity analyses of tropospheric chemical scenarios

2011 ◽  
Vol 11 (6) ◽  
pp. 16745-16799 ◽  
Author(s):  
N. Goris ◽  
H. Elbern
Keyword(s):  
Singular Vector ◽  
Weather Prediction ◽  
Emission Factors ◽  
The State ◽  
Vector Analysis ◽  
Sensitivity Analyses ◽  
Initial Time ◽  
State Estimate ◽  
Singular Vectors ◽  
Initial Values

Abstract. Observations of the chemical state of the atmosphere typically provide only sparse snapshots of the state of the system due to their insufficient temporal and spatial density. Therefore the measurement configurations need to be optimised to get a best possible state estimate. One possibility to optimise the state estimate is provided by observation targeting of sensitive system states, to identify measurement configurations of best value for forecast improvements. In recent years, numerical weather prediction adapted singular vector analysis with respect to initial values as a novel method to identify sensitive states. In the present work, this technique is transferred from meteorological to chemical forecast. Besides initial values, emissions are investigated as controlling variables. More precisely uncertainties in the amplitude of the diurnal profile of emissions are analysed, yielding emission factors as target variables. Singular vector analysis is extended to allow for projected target variables not only at final time but also at initial time. Further, special operators are introduced, which consider the combined influence of groups of chemical species. As a preparation for targeted observation calculations, the concept of adaptive observations is studied with a chemistry box model. For a set of six different scenarios, the VOC versus NOx limitation of the ozone formation is investigated. Results reveal, that the singular vectors are strongly dependent on start time and length of the simulation. As expected, singular vectors with initial values as target variables tend to be more sensitive to initial values, while emission factors as target variables are more sensitive to simulation length. Further, the particular importance of chemical compounds differs strongly between absolute and relative error growth.

scholarly journals The Characteristics of Key Analysis Errors. Part III: A Diagnosis of Their Evolution

2007 ◽  
Vol 135 (7) ◽  
pp. 2754-2777 ◽  
Author(s):  
Jean-François Caron ◽  
M. K. Yau ◽  
Stéphane Laroche
Keyword(s):  
Level Structure ◽  
Potential Temperature ◽  
Initial Time ◽  
Forecast Errors ◽  
Diagnostic Study ◽  
Low Level ◽  
Diabatic Processes ◽  
Upper Level ◽  
Analysis Error ◽  
Analysis Errors

Abstract This paper presents a diagnostic study of the evolution of initial corrections obtained from the key analysis error algorithm that minimizes the short-range (24 h) forecast errors for four specific events poorly forecasted over the eastern part of North America. A potential vorticity (PV) perspective is employed. It is shown that the modification to the low-level structure at the initial time is mainly attributed to the modification of the low-level PV distribution, while changes in the upper-level structure are attributed to the modification of the upper-level PV distribution. The low-level corrections grow mainly through background surface potential temperature advection by the wind corrections attributable to the interior PV corrections. Changes in the diabatic processes and the vertical alignment of low-level PV corrections by differential PV advection also increase the magnitude of the low-level corrections with time. The upper-level corrections grow by advection of background PV from wind corrections. However, the cause of these latter wind corrections responsible for upper-level background PV advection varies from case to case. An investigation of the relative importance of the low-level and of the upper-level initial corrections to produce the final-time corrections also reveals strong variability between cases. Finally, comparison of two cases in which the key analysis errors propagate vertically with two others without significant vertical propagation shows how the relative position of the key analysis errors with respect to the structure of the background flow can influence the evolution of the initial corrections.

Singular vectors and estimates of the analysis-error covariance metric

1998 ◽  
Vol 124 (549) ◽  
pp. 1695-1713 ◽  
Author(s):  
Jan Barkmeijer ◽  
François Bouttier ◽  
Martin Van Gijzen
Keyword(s):  
Singular Vectors ◽  
Error Covariance ◽  
Analysis Error

Maximum Likelihood Ensemble Filter: Theoretical Aspects

2005 ◽  
Vol 133 (6) ◽  
pp. 1710-1726 ◽  
Author(s):  
Milija Zupanski
Keyword(s):  
Maximum Likelihood ◽  
Data Assimilation ◽  
Cost Function ◽  
Ensemble Data Assimilation ◽  
Error Covariance ◽  
Ensemble Data ◽  
Analysis Error ◽  
Nonlinear Observation ◽  
The Cost ◽  
Maximum Likelihood Ensemble Filter

Abstract A new ensemble-based data assimilation method, named the maximum likelihood ensemble filter (MLEF), is presented. The analysis solution maximizes the likelihood of the posterior probability distribution, obtained by minimization of a cost function that depends on a general nonlinear observation operator. The MLEF belongs to the class of deterministic ensemble filters, since no perturbed observations are employed. As in variational and ensemble data assimilation methods, the cost function is derived using a Gaussian probability density function framework. Like other ensemble data assimilation algorithms, the MLEF produces an estimate of the analysis uncertainty (e.g., analysis error covariance). In addition to the common use of ensembles in calculation of the forecast error covariance, the ensembles in MLEF are exploited to efficiently calculate the Hessian preconditioning and the gradient of the cost function. A sufficient number of iterative minimization steps is 2–3, because of superior Hessian preconditioning. The MLEF method is well suited for use with highly nonlinear observation operators, for a small additional computational cost of minimization. The consistent treatment of nonlinear observation operators through optimization is an advantage of the MLEF over other ensemble data assimilation algorithms. The cost of MLEF is comparable to the cost of existing ensemble Kalman filter algorithms. The method is directly applicable to most complex forecast models and observation operators. In this paper, the MLEF method is applied to data assimilation with the one-dimensional Korteweg–de Vries–Burgers equation. The tested observation operator is quadratic, in order to make the assimilation problem more challenging. The results illustrate the stability of the MLEF performance, as well as the benefit of the cost function minimization. The improvement is noted in terms of the rms error, as well as the analysis error covariance. The statistics of innovation vectors (observation minus forecast) also indicate a stable performance of the MLEF algorithm. Additional experiments suggest the amplified benefit of targeted observations in ensemble data assimilation.

scholarly journals Interpretation of Adaptive Observing Guidance for Atlantic Tropical Cyclones

2007 ◽  
Vol 135 (12) ◽  
pp. 4006-4029 ◽  
Author(s):  
C. A. Reynolds ◽  
M. S. Peng ◽  
S. J. Majumdar ◽  
S. D. Aberson ◽  
C. H. Bishop ◽  
...  
Keyword(s):  
Data Assimilation ◽  
Total Energy ◽  
Tropical Cyclones ◽  
Error Variance ◽  
Naval Research ◽  
Error Statistics ◽  
Background Error ◽  
Analysis Error ◽  
The Impact ◽  
Perturbation Growth

Abstract Adaptive observing guidance products for Atlantic tropical cyclones are compared using composite techniques that allow one to quantitatively examine differences in the spatial structures of the guidance maps and relate these differences to the constraints and approximations of the respective techniques. The guidance maps are produced using the ensemble transform Kalman filter (ETKF) based on ensembles from the National Centers for Environmental Prediction and the European Centre for Medium-Range Weather Forecasts (ECMWF), and total-energy singular vectors (TESVs) produced by ECMWF and the Naval Research Laboratory. Systematic structural differences in the guidance products are linked to the fact that TESVs consider the dynamics of perturbation growth only, while the ETKF combines information on perturbation evolution with error statistics from an ensemble-based data assimilation scheme. The impact of constraining the SVs using different estimates of analysis error variance instead of a total-energy norm, in effect bringing the two methods closer together, is also assessed. When the targets are close to the storm, the TESV products are a maximum in an annulus around the storm, whereas the ETKF products are a maximum at the storm location itself. When the targets are remote from the storm, the TESVs almost always indicate targets northwest of the storm, whereas the ETKF targets are more scattered relative to the storm location and often occur over the northern North Atlantic. The ETKF guidance often coincides with locations in which the ensemble-based analysis error variance is large. As the TESV method is not designed to consider spatial differences in the likely analysis errors, it will produce targets over well-observed regions, such as the continental United States. Constraining the SV calculation using analysis error variance values from an operational 3D variational data assimilation system (with stationary, quasi-isotropic background error statistics) results in a modest modulation of the target areas away from the well-observed regions, and a modest reduction of perturbation growth. Constraining the SVs using the ETKF estimate of analysis error variance produces SV targets similar to ETKF targets and results in a significant reduction in perturbation growth, due to the highly localized nature of the analysis error variance estimates. These results illustrate the strong sensitivity of SVs to the norm (and to the analysis error variance estimate used to define it) and confirm that discrepancies between target areas computed using different methods reflect the mathematical and physical differences between the methods themselves.

Sign in / Sign up

Export Citation Format

Share Document