A unifying framework for detecting outliers and change points from time series

2006 ◽  
Vol 18 (4) ◽  
pp. 482-492 ◽  
Author(s):  
J. Takeuchi ◽  
K. Yamanishi
Keyword(s):  
Time Series ◽  
Change Points

scholarly journals Change Point Enhanced Anomaly Detection for IoT Time Series Data

Water ◽  
2021 ◽  
Vol 13 (12) ◽  
pp. 1633
Author(s):  
Elena-Simona Apostol ◽  
Ciprian-Octavian Truică ◽  
Florin Pop ◽  
Christian Esposito
Keyword(s):  
Time Series ◽  
Anomaly Detection ◽  
Change Point ◽  
Time Series Data ◽  
Multivariate Time Series ◽  
Change Point Detection ◽  
Change Points ◽  
Series Data ◽  
Prediction And Forecasting ◽  
Point Detection

Due to the exponential growth of the Internet of Things networks and the massive amount of time series data collected from these networks, it is essential to apply efficient methods for Big Data analysis in order to extract meaningful information and statistics. Anomaly detection is an important part of time series analysis, improving the quality of further analysis, such as prediction and forecasting. Thus, detecting sudden change points with normal behavior and using them to discriminate between abnormal behavior, i.e., outliers, is a crucial step used to minimize the false positive rate and to build accurate machine learning models for prediction and forecasting. In this paper, we propose a rule-based decision system that enhances anomaly detection in multivariate time series using change point detection. Our architecture uses a pipeline that automatically manages to detect real anomalies and remove the false positives introduced by change points. We employ both traditional and deep learning unsupervised algorithms, in total, five anomaly detection and five change point detection algorithms. Additionally, we propose a new confidence metric based on the support for a time series point to be an anomaly and the support for the same point to be a change point. In our experiments, we use a large real-world dataset containing multivariate time series about water consumption collected from smart meters. As an evaluation metric, we use Mean Absolute Error (MAE). The low MAE values show that the algorithms accurately determine anomalies and change points. The experimental results strengthen our assumption that anomaly detection can be improved by determining and removing change points as well as validates the correctness of our proposed rules in real-world scenarios. Furthermore, the proposed rule-based decision support systems enable users to make informed decisions regarding the status of the water distribution network and perform effectively predictive and proactive maintenance.

scholarly journals Changes of China’s agri-food exports to Germany caused by its accession to WTO and the 2008 financial crisis

2015 ◽  
Vol 7 (2) ◽  
pp. 262-279 ◽  
Author(s):  
Zhichao Guo ◽  
Yuanhua Feng ◽  
Thomas Gries
Keyword(s):  
Time Series ◽  
Financial Crisis ◽  
Regression Models ◽  
Structural Breaks ◽  
Global Financial Crisis ◽  
Middle Part ◽  
Change Points ◽  
Content Type ◽  
The World ◽  
Before And After

Purpose – The purpose of this paper is to investigate changes of China’s agri-food exports to Germany caused by China’s accession to WTO and the global financial crisis in a quantitative way. The paper aims to detect structural breaks and compare differences before and after the change points. Design/methodology/approach – The structural breaks detection procedures in this paper can be applied to find out two different types of change points, i.e. in the middle and at the end of one time series. Then time series and regression models are used to compare differences of trade relationship before and after the detected change points. The methods can be employed in any economic series and work well in practice. Findings – The results indicate that structural breaks in 2002 and 2009 are caused by China’s accession to WTO and the financial crisis. Time series and regression models show that the development of China’s exports to Germany in agri-food products has different features in different sub-periods. Before 1999, there is no significant relationship between China’s exports to Germany and Germany’s imports from the world. Between 2002 and 2008 the former depends on the latter very strongly, and China’s exports to Germany developed quickly and stably. It decreased, however suddenly in 2009, caused by the great reduction of Germany’s imports from the world in that year. But China’s market share in Germany still had a small gain. Analysis of two categories in agri-food trade also leads to similar conclusions. Comparing the two events we see rather different patterns even if they both indicate structural breaks in the development of China’s agri-food exports to Germany. Originality/value – This paper partly originally proposes two statistical algorithms for detecting different kinds of structural breaks in the middle part and at the end of a short-time series, respectively.

scholarly journals Closed-Form Estimation of Multiple Change-Point Models

2013 ◽  
Author(s):  
Greg Jensen
Keyword(s):  
Time Series ◽  
Linear Regression ◽  
Change Point ◽  
Marginal Likelihood ◽  
Stationary Time Series ◽  
Change Points ◽  
Model Complexity ◽  
Marginal Model ◽  
General Strategy ◽  
Change Point Models

Identifying discontinuities (or change-points) in otherwise stationary time series is a powerful analytic tool. This paper outlines a general strategy for identifying an unknown number of change-points using elementary principles of Bayesian statistics. Using a strategy of binary partitioning by marginal likelihood, a time series is recursively subdivided on the basis of whether adding divisions (and thus increasing model complexity) yields a justified improvement in the marginal model likelihood. When this approach is combined with the use of conjugate priors, it yields the Conjugate Partitioned Recursion (CPR) algorithm, which identifies change-points without computationally intensive numerical integration. Using the CPR algorithm, methods are described for specifying change-point models drawn from a host of familiar distributions, both discrete (binomial, geometric, Poisson) and continuous (exponential, Gaussian, uniform, and multiple linear regression), as well as multivariate distribution (multinomial, multivariate normal, and multivariate linear regression). Methods by which the CPR algorithm could be extended or modified are discussed, and several detailed applications to data published in psychology and biomedical engineering are described.

Ecological change points: The strength of density dependence and the loss of history

2017 ◽  
Author(s):  
José M. Ponciano ◽  
Mark L. Taper ◽  
Brian Dennis
Keyword(s):  
Time Series ◽  
Population Dynamics ◽  
Density Dependence ◽  
Change Point ◽  
Likelihood Estimation ◽  
Theory And Practice ◽  
Change Points ◽  
Ecological Change ◽  
Proposed Model ◽  
Change Point Model

AbstractChange points in the dynamics of animal abundances have extensively been recorded in historical time series records. Little attention has been paid to the theoretical dynamic consequences of such change-points. Here we propose a change-point model of stochastic population dynamics. This investigation embodies a shift of attention from the problem of detecting when a change will occur, to another non-trivial puzzle: using ecological theory to understand and predict the post-breakpoint behavior of the population dynamics. The proposed model and the explicit expressions derived here predict and quantify how density dependence modulates the influence of the pre-breakpoint parameters into the post-breakpoint dynamics. Time series transitioning from one stationary distribution to another contain information about where the process was before the change-point, where is it heading and how long it will take to transition, and here this information is explicitly stated. Importantly, our results provide a direct connection of the strength of density dependence with theoretical properties of dynamic systems, such as the concept of resilience. Finally, we illustrate how to harness such information through maximum likelihood estimation for state-space models, and test the model robustness to widely different forms of compensatory dynamics. The model can be used to estimate important quantities in the theory and practice of population recovery.

scholarly journals Homogeneity tests and non-parametric analyses of tendencies in precipitation time series in Keszthely, Western Hungary

2019 ◽  
Vol 139 (3-4) ◽  
pp. 849-859 ◽  
Author(s):  
Tímea Kocsis ◽  
Ilona Kovács-Székely ◽  
Angéla Anda
Keyword(s):  
Time Series ◽  
Change Points ◽  
Annual Precipitation ◽  
Trend Test ◽  
Climate Data ◽  
Homogeneity Tests ◽  
Break Points ◽  
The Mean ◽  
Parametric Analyses ◽  
Precipitation Records

Abstract This study aims to investigate the precipitation trends in Keszthely (Western Hungary, Central Europe) through an examination of historical climate data covering the past almost one and a half centuries. Pettitt’s test for homogeneity was employed to detect change points in the time series of monthly, seasonal and annual precipitation records. Change points and monotonic trends were analysed separately in annual, seasonal and monthly time series of precipitation. While no break points could be detected in the annual precipitation series, a significant decreasing trend of 0.2–0.7 mm/year was highlighted statistically using the autocorrelated Mann-Kendall trend test. Significant change points were found in those time series in which significant tendencies had been detected in previous studies. These points fell in spring and winter for the seasonal series, and October for the monthly series. The question therefore arises of whether these trends are the result of a shift in the mean. The downward and upward shift in the mean in the case of spring and winter seasonal amounts, respectively, leads to a suspicion that changes in precipitation are also in progress in these seasons. The study concludes that homogeneity tests are of great importance in such analyses, because they may help to avoid false trend detections.

Sign in / Sign up

Export Citation Format

Share Document