scholarly journals Predicting Stock Movements: Using Multiresolution Wavelet Reconstruction and Deep Learning in Neural Networks

Information ◽  
2021 ◽  
Vol 12 (10) ◽  
pp. 388
Author(s):  
Lifang Peng ◽  
Kefu Chen ◽  
Ning Li
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Deep Learning ◽  
Wavelet Analysis ◽  
Stock Prices ◽  
Time Signal ◽  
Small Scale ◽  
Data Set ◽  
Wide Range ◽  
Stock Behavior

Stock movement prediction is important in the financial world because investors want to observe trends in stock prices before making investment decisions. However, given the non-linear non-stationary financial time series characteristics of stock prices, this remains an extremely challenging task. A wavelet is a mathematical function used to divide a given function or continuous-time signal into different scale components. Wavelet analysis has good time-frequency local characteristics and good zooming capability for non-stationary random signals. However, the application of the wavelet theory is generally limited to a small scale. The neural networks method is a powerful tool to deal with large-scale problems. Therefore, the combination of neural networks and wavelet analysis becomes more applicable for stock behavior prediction. To rebuild the signals in multiple scales, and filter the measurement noise, a forecasting model based on a stock price time series was provided, employing multiresolution analysis (MRA). Then, the deep learning in the neural network method was used to train and test the empirical data. To explain the fundamental concepts, a conceptual analysis of similar algorithms was performed. The data set for the experiment was chosen to capture a wide range of stock movements from 1 January 2009 to 31 December 2017. Comparison analyses between the algorithms and industries were conducted to show that the method is stable and reliable. This study focused on medium-term stock predictions to predict future stock behavior over 11 days of horizons. Our test results showed a 75% hit rate, on average, for all industries, in terms of US stocks on FORTUNE Global 500. We confirmed the effectiveness of our model and method based on the findings of the empirical research. This study’s primary contribution is to demonstrate the reconstruction model of the stock time series and to perform recurrent neural networks using the deep learning method. Our findings fill an academic research gap, by demonstrating that deep learning can be used to predict stock movement.

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 1.91% to 6.69%. <div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

<div>Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 2.38% to 5.27%. The code and the pre-trained model will be available at https://github.com/linlei1214/SITS-BERT upon publication.</div><div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals Study of the impact of the COVID-19 pandemic on international air transportation

2021 ◽  
Vol 29 (1) ◽  
pp. 22-35
Author(s):  
Eugeny Yu. Shchetinin
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Stock Prices ◽  
Short Term Memory ◽  
Air Transportation ◽  
Time Series Forecasting ◽  
The Cost ◽  
The Impact

Time Series Forecasting has always been a very important area of research in many domains because many different types of data are stored as time series. Given the growing availability of data and computing power in the recent years, Deep Learning has become a fundamental part of the new generation of Time Series Forecasting models, obtaining excellent results.As different time series problems are studied in many different fields, a large number of new architectures have been developed in recent years. This has also been simplified by the growing availability of open source frameworks, which make the development of new custom network components easier and faster.In this paper three different Deep Learning Architecture for Time Series Forecasting are presented: Recurrent Neural Networks (RNNs), that are the most classical and used architecture for Time Series Forecasting problems; Long Short-Term Memory (LSTM), that are an evolution of RNNs developed in order to overcome the vanishing gradient problem; Gated Recurrent Unit (GRU), that are another evolution of RNNs, similar to LSTM.The article is devoted to modeling and forecasting the cost of international air transportation in a pandemic using deep learning methods. The author builds time series models of the American Airlines (AAL) stock prices for a selected period using LSTM, GRU, RNN recurrent neural networks models and compare the accuracy forecast results.

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

<div>Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 2.38% to 5.27%. The code and the pre-trained model will be available at https://github.com/linlei1214/SITS-BERT upon publication.</div><div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

<div>Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 2.38% to 5.27%. The code and the pre-trained model will be available at https://github.com/linlei1214/SITS-BERT upon publication.</div><div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals Data augmentation for computed tomography angiography via synthetic image generation and neural domain adaptation

2020 ◽  
Vol 6 (1) ◽  
Author(s):  
Malte Seemann ◽  
Lennart Bargsten ◽  
Alexander Schlaefer
Keyword(s):  
Computed Tomography ◽  
Neural Networks ◽  
Deep Learning ◽  
Medical Imaging ◽  
Computed Tomography Angiography ◽  
Data Augmentation ◽  
Domain Adaptation ◽  
Synthetic Image ◽  
Wide Range ◽  
The Impact

AbstractDeep learning methods produce promising results when applied to a wide range of medical imaging tasks, including segmentation of artery lumen in computed tomography angiography (CTA) data. However, to perform sufficiently, neural networks have to be trained on large amounts of high quality annotated data. In the realm of medical imaging, annotations are not only quite scarce but also often not entirely reliable. To tackle both challenges, we developed a two-step approach for generating realistic synthetic CTA data for the purpose of data augmentation. In the first step moderately realistic images are generated in a purely numerical fashion. In the second step these images are improved by applying neural domain adaptation. We evaluated the impact of synthetic data on lumen segmentation via convolutional neural networks (CNNs) by comparing resulting performances. Improvements of up to 5% in terms of Dice coefficient and 20% for Hausdorff distance represent a proof of concept that the proposed augmentation procedure can be used to enhance deep learning-based segmentation for artery lumen in CTA images.

Deep Learning Based Classification of Time Series of Chaotic Systems over Graphic Images

2021 ◽  
Author(s):  
Süleyman UZUN ◽  
Sezgin KAÇAR ◽  
Burak ARICIOĞLU
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Transfer Learning ◽  
Chaotic Systems ◽  
Initial Conditions ◽  
Step Size ◽  
Learning Methods ◽  
Data Set ◽  
Graphic Images ◽  
Different Chaotic Systems

Abstract In this study, for the first time in the literature, identification of different chaotic systems by classifying graphic images of their time series with deep learning methods is aimed. For this purpose, a data set is generated that consists of the graphic images of time series of the most known three chaotic systems: Lorenz, Chen, and Rossler systems. The time series are obtained for different parameter values, initial conditions, step size and time lengths. After generating the data set, a high-accuracy classification is performed by using transfer learning method. In the study, the most accepted deep learning models of the transfer learning methods are employed. These models are SqueezeNet, VGG-19, AlexNet, ResNet50, ResNet101, DenseNet201, ShuffleNet and GoogLeNet. As a result of the study, classification accuracy is found between 96% and 97% depending on the problem. Thus, this study makes association of real time random signals with a mathematical system possible.

scholarly journals The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?

2021 ◽  
Vol 4 ◽  
Author(s):  
Stefano Markidis
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
High Performance ◽  
Critical Role ◽  
Low Frequency ◽  
Accurate Solution ◽  
Limiting Factor ◽  
Data Set ◽  
Linear Solvers ◽  
Computational Performance

Physics-Informed Neural Networks (PINN) are neural networks encoding the problem governing equations, such as Partial Differential Equations (PDE), as a part of the neural network. PINNs have emerged as a new essential tool to solve various challenging problems, including computing linear systems arising from PDEs, a task for which several traditional methods exist. In this work, we focus first on evaluating the potential of PINNs as linear solvers in the case of the Poisson equation, an omnipresent equation in scientific computing. We characterize PINN linear solvers in terms of accuracy and performance under different network configurations (depth, activation functions, input data set distribution). We highlight the critical role of transfer learning. Our results show that low-frequency components of the solution converge quickly as an effect of the F-principle. In contrast, an accurate solution of the high frequencies requires an exceedingly long time. To address this limitation, we propose integrating PINNs into traditional linear solvers. We show that this integration leads to the development of new solvers whose performance is on par with other high-performance solvers, such as PETSc conjugate gradient linear solvers, in terms of performance and accuracy. Overall, while the accuracy and computational performance are still a limiting factor for the direct use of PINN linear solvers, hybrid strategies combining old traditional linear solver approaches with new emerging deep-learning techniques are among the most promising methods for developing a new class of linear solvers.

scholarly journals Wavelet analysis applied on temporal data sets in order to reveal possible pre-seismic radio anomalies and comparison with the trend of the raw data 

2021 ◽  
Author(s):  
Giovanni Nico ◽  
Pier Francesco Biagi ◽  
Anita Ermini ◽  
Mohammed Yahia Boudjada ◽  
Hans Ulrich Eichelberger ◽  
...  
Keyword(s):  
Time Series ◽  
Wavelet Analysis ◽  
Power Spectrum ◽  
Power Spectra ◽  
Wavelet Spectrum ◽  
Time Signal ◽  
Time Behavior ◽  
Sudden Increase ◽  
Night Time ◽  
Raw Data

&lt;p&gt;Since 2009, several radio receivers have been installed throughout Europe in order to realize the INFREP European radio network for studying the VLF (10-50 kHz) and LF (150-300 kHz) radio precursors of earthquakes. Precursors can be related to &amp;#8220;anomalies&amp;#8221; in the night-time behavior of&amp;#160; VLF signals. A suitable method of analysis is the use of the Wavelet spectra.&amp;#160; Using the &amp;#8220;Morlet function&amp;#8221;, the Wavelet transform of a time signal is a complex series that can be usefully represented by its square amplitude, i.e. considering the so-called Wavelet power spectrum.&lt;/p&gt;&lt;p&gt;The power spectrum is a 2D diagram that, once properly normalized with respect to the power of the white noise, gives information on the strength and precise time of occurrence of the various Fourier components, which are present in the original time series. The main difference between the Wavelet power spectra and the Fourier power spectra for the time series is that the former identifies the frequency content along the operational time, which cannot be done with the latter. Anomalies are identified as regions of the Wavelet spectrogram characterized by a sudden increase in the power strength.&lt;/p&gt;&lt;p&gt;On January 30, 2020 an earthquake with Mw= 6.0 occurred in Dodecanese Islands. The results of the Wavelet analysis carried out on data collected some INFREP receivers is compared with the trends of the raw data. The time series from January 24, 2020 till January 31, 2000 was analyzed. The Wavelet spectrogram shows a peak corresponding to a period of 1 day on the days before January 30. This anomaly was found for signals transmitted at the frequencies 19,58 kHz, 20, 27 kHz, 23,40 kHz with an energy in the peak increasing from 19,58 kHz to 23,40 kHz. In particular, the signal at the frequency 19,58 kHz, shows a peak on January 29, while the frequencies 20,27 kHz and 23,40 kHz are characterized by a peak starting on January 28 and continuing to January 29. The results presented in this work shows the perspective use of the Wavelet spectrum analysis as an operational tool for the detection of anomalies in VLF and LF signal potentially related to EQ precursors.&lt;/p&gt;

scholarly journals Coastal Change Patterns from Time Series Clustering of Permanent Laser Scan Data

2020 ◽  
Author(s):  
Mieke Kuschnerus ◽  
Roderik Lindenbergh ◽  
Sander Vos
Keyword(s):  
Time Series ◽  
Laser Scanning ◽  
Clustering Algorithm ◽  
Coastal Areas ◽  
Small Scale ◽  
Temporal Data ◽  
Agglomerative Clustering ◽  
Data Set ◽  
Deformation Processes ◽  
Spatio Temporal

Abstract. Sandy coasts are constantly changing environments governed by complex interacting processes. Permanent laser scanning is a promising technique to monitor such coastal areas and support analysis of geomorphological deformation processes. This novel technique delivers 3D representations of a part of the coast at hourly temporal and centimetre spatial resolution and allows to observe small scale changes in elevation over extended periods of time. These observations have the potential to improve understanding and modelling of coastal deformation processes. However, to be of use to coastal researchers and coastal management, an efficient way to find and extract deformation processes from the large spatio-temporal data set is needed. In order to allow data mining in an automated way, we extract time series in elevation or range and use unsupervised learning algorithms to derive a partitioning of the observed area according to change patterns. We compare three well known clustering algorithms, k-means, agglomerative clustering and DBSCAN, and identify areas that undergo similar evolution during one month. We test if they fulfil our criteria for a suitable clustering algorithm on our exemplary data set. The three clustering methods are applied to time series of 30 epochs (during one month) extracted from a data set of daily scans covering a part of the coast at Kijkduin, the Netherlands. A small section of the beach, where a pile of sand was accumulated by a bulldozer is used to evaluate the performance of the algorithms against a ground truth. The k-means algorithm and agglomerative clustering deliver similar clusters, and both allow to identify a fixed number of dominant deformation processes in sandy coastal areas, such as sand accumulation by a bulldozer or erosion in the intertidal area. The DBSCAN algorithm finds clusters for only about 44 % of the area and turns out to be more suitable for the detection of outliers, caused for example by temporary objects on the beach. Our study provides a methodology to efficiently mine a spatio-temporal data set for predominant deformation patterns with the associated regions, where they occur.

Sign in / Sign up

Export Citation Format

Share Document