scholarly journals Using Data Augmentation Based Reinforcement Learning for Daily Stock Trading

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1384
Author(s):  
Yuyu Yuan ◽  
Wen Wen ◽  
Jincui Yang
Keyword(s):  
Reinforcement Learning ◽  
Data Augmentation ◽  
Training Data ◽  
Stock Trading ◽  
Data Set ◽  
Stable Algorithm ◽  
Q Learning ◽  
Using Data ◽  
Policy Optimization ◽  
Sharp Ratio

In algorithmic trading, adequate training data set is key to making profits. However, stock trading data in units of a day can not meet the great demand for reinforcement learning. To address this problem, we proposed a framework named data augmentation based reinforcement learning (DARL) which uses minute-candle data (open, high, low, close) to train the agent. The agent is then used to guide daily stock trading. In this way, we can increase the instances of data available for training in hundreds of folds, which can substantially improve the reinforcement learning effect. But not all stocks are suitable for this kind of trading. Therefore, we propose an access mechanism based on skewness and kurtosis to select stocks that can be traded properly using this algorithm. In our experiment, we find proximal policy optimization (PPO) is the most stable algorithm to achieve high risk-adjusted returns. Deep Q-learning (DQN) and soft actor critic (SAC) can beat the market in Sharp Ratio.

scholarly journals Building Damage Detection from Post-Event Aerial Imagery Using Single Shot Multibox Detector

2019 ◽  
Vol 9 (6) ◽  
pp. 1128 ◽  
Author(s):  
Yundong Li ◽  
Wei Hu ◽  
Han Dong ◽  
Xueyan Zhang
Keyword(s):  
Machine Learning ◽  
Data Augmentation ◽  
Hurricane Sandy ◽  
Training Data ◽  
Aerial Images ◽  
Detection Methods ◽  
Single Shot ◽  
Data Set ◽  
Augmentation Strategies ◽  
Post Disaster

Using aerial cameras, satellite remote sensing or unmanned aerial vehicles (UAV) equipped with cameras can facilitate search and rescue tasks after disasters. The traditional manual interpretation of huge aerial images is inefficient and could be replaced by machine learning-based methods combined with image processing techniques. Given the development of machine learning, researchers find that convolutional neural networks can effectively extract features from images. Some target detection methods based on deep learning, such as the single-shot multibox detector (SSD) algorithm, can achieve better results than traditional methods. However, the impressive performance of machine learning-based methods results from the numerous labeled samples. Given the complexity of post-disaster scenarios, obtaining many samples in the aftermath of disasters is difficult. To address this issue, a damaged building assessment method using SSD with pretraining and data augmentation is proposed in the current study and highlights the following aspects. (1) Objects can be detected and classified into undamaged buildings, damaged buildings, and ruins. (2) A convolution auto-encoder (CAE) that consists of VGG16 is constructed and trained using unlabeled post-disaster images. As a transfer learning strategy, the weights of the SSD model are initialized using the weights of the CAE counterpart. (3) Data augmentation strategies, such as image mirroring, rotation, Gaussian blur, and Gaussian noise processing, are utilized to augment the training data set. As a case study, aerial images of Hurricane Sandy in 2012 were maximized to validate the proposed method’s effectiveness. Experiments show that the pretraining strategy can improve of 10% in terms of overall accuracy compared with the SSD trained from scratch. These experiments also demonstrate that using data augmentation strategies can improve mAP and mF1 by 72% and 20%, respectively. Finally, the experiment is further verified by another dataset of Hurricane Irma, and it is concluded that the paper method is feasible.

RF-Identity

2021 ◽  
Vol 5 (1) ◽  
pp. 1-23
Author(s):  
Chao Feng ◽  
Jie Xiong ◽  
Liqiong Chang ◽  
Fuwei Wang ◽  
Ju Wang ◽  
...  
Keyword(s):  
Data Augmentation ◽  
Critical Role ◽  
Training Data ◽  
Identification Accuracy ◽  
Identification System ◽  
Frequency Bandwidth ◽  
Person Identification ◽  
Data Set ◽  
Rf Devices ◽  
Large Frequency

Person identification plays a critical role in a large range of applications. Recently, RF based person identification becomes a hot research topic due to the contact-free nature of RF sensing that is particularly appealing in current COVID-19 pandemic. However, existing systems still have multiple limitations: i) heavily rely on the gait patterns of users for identification; ii) require a large amount of data to train the model and also extensive retraining for new users and iii) require a large frequency bandwidth which is not available on most commodity RF devices for static person identification. This paper proposes RF-Identity, an RFID-based identification system to address the above limitations and the contribution is threefold. First, by integrating walking pattern features with unique body shape features (e.g., height), RF-Identity achieves a high accuracy in person identification. Second, RF-Identity develops a data augmentation scheme to expand the size of the training data set, thus reducing the human effort in data collection. Third, RF-Identity utilizes the tag diversity in spatial domain to identify static users without a need of large frequency bandwidth. Extensive experiments show an identification accuracy of 94.2% and 95.9% for 50 dynamic and static users, respectively.

Underwater Acoustic Target Recognition Based on Generative Adversarial Network Data Augmentation

2021 ◽  
Vol 263 (2) ◽  
pp. 4558-4564
Author(s):  
Minghong Zhang ◽  
Xinwei Luo
Keyword(s):  
Data Augmentation ◽  
Target Recognition ◽  
Training Data ◽  
Small Samples ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Underwater Acoustic ◽  
Adversarial Network ◽  
Acoustic Target ◽  
The Impact

Underwater acoustic target recognition is an important aspect of underwater acoustic research. In recent years, machine learning has been developed continuously, which is widely and effectively applied in underwater acoustic target recognition. In order to acquire good recognition results and reduce the problem of overfitting, Adequate data sets are essential. However, underwater acoustic samples are relatively rare, which has a certain impact on recognition accuracy. In this paper, in addition of the traditional audio data augmentation method, a new method of data augmentation using generative adversarial network is proposed, which uses generator and discriminator to learn the characteristics of underwater acoustic samples, so as to generate reliable underwater acoustic signals to expand the training data set. The expanded data set is input into the deep neural network, and the transfer learning method is applied to further reduce the impact caused by small samples by fixing part of the pre-trained parameters. The experimental results show that the recognition result of this method is better than the general underwater acoustic recognition method, and the effectiveness of this method is verified.

scholarly journals Identification of Leukemia Subtypes from Microscopic Images Using Convolutional Neural Network

Diagnostics ◽  
2019 ◽  
Vol 9 (3) ◽  
pp. 104 ◽  
Author(s):  
Ahmed ◽  
Yigit ◽  
Isik ◽  
Alpkocak
Keyword(s):  
Machine Learning ◽  
Data Augmentation ◽  
Nearest Neighbor ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Support Vector ◽  
K Nearest Neighbor ◽  
Data Set ◽  
Leukemia Data

Leukemia is a fatal cancer and has two main types: Acute and chronic. Each type has two more subtypes: Lymphoid and myeloid. Hence, in total, there are four subtypes of leukemia. This study proposes a new approach for diagnosis of all subtypes of leukemia from microscopic blood cell images using convolutional neural networks (CNN), which requires a large training data set. Therefore, we also investigated the effects of data augmentation for an increasing number of training samples synthetically. We used two publicly available leukemia data sources: ALL-IDB and ASH Image Bank. Next, we applied seven different image transformation techniques as data augmentation. We designed a CNN architecture capable of recognizing all subtypes of leukemia. Besides, we also explored other well-known machine learning algorithms such as naive Bayes, support vector machine, k-nearest neighbor, and decision tree. To evaluate our approach, we set up a set of experiments and used 5-fold cross-validation. The results we obtained from experiments showed that our CNN model performance has 88.25% and 81.74% accuracy, in leukemia versus healthy and multiclass classification of all subtypes, respectively. Finally, we also showed that the CNN model has a better performance than other wellknown machine learning algorithms.

scholarly journals Short-Term Load Forecasting for a Single Household Based on Convolution Neural Networks Using Data Augmentation

Energies ◽  
2019 ◽  
Vol 12 (18) ◽  
pp. 3560 ◽  
Author(s):  
Acharya ◽  
Wi ◽  
Lee
Keyword(s):  
Neural Networks ◽  
Data Augmentation ◽  
Load Forecasting ◽  
Training Data ◽  
Convolution Neural Networks ◽  
Forecasting Method ◽  
High Volatility ◽  
Using Data ◽  
Advanced Metering ◽  
Residential Electric Demand

Advanced metering infrastructure (AMI) is spreading to households in some countries, and could be a source for forecasting the residential electric demand. However, load forecasting of a single household is still a fairly challenging topic because of the high volatility and uncertainty of the electric demand of households. Moreover, there is a limitation in the use of historical load data because of a change in house ownership, change in lifestyle, integration of new electric devices, and so on. The paper proposes a novel method to forecast the electricity loads of single residential households. The proposed forecasting method is based on convolution neural networks (CNNs) combined with a data-augmentation technique, which can artificially enlarge the training data. This method can address issues caused by a lack of historical data and improve the accuracy of residential load forecasting. Simulation results illustrate the validation and efficacy of the proposed method.

scholarly journals Learning to rank with click-through features in a reinforcement learning framework

2016 ◽  
Vol 12 (4) ◽  
pp. 448-476 ◽  
Author(s):  
Amir Hosein Keyhanipour ◽  
Behzad Moshiri ◽  
Maryam Piroozmand ◽  
Farhad Oroumchian ◽  
Ali Moeini
Keyword(s):  
Reinforcement Learning ◽  
Learning To Rank ◽  
Training Data ◽  
High Dimensionality ◽  
Compact Representation ◽  
Second Phase ◽  
Data Sets ◽  
Data Set ◽  
Content Type ◽  
Benchmark Data

Purpose Learning to rank algorithms inherently faces many challenges. The most important challenges could be listed as high-dimensionality of the training data, the dynamic nature of Web information resources and lack of click-through data. High dimensionality of the training data affects effectiveness and efficiency of learning algorithms. Besides, most of learning to rank benchmark datasets do not include click-through data as a very rich source of information about the search behavior of users while dealing with the ranked lists of search results. To deal with these limitations, this paper aims to introduce a novel learning to rank algorithm by using a set of complex click-through features in a reinforcement learning (RL) model. These features are calculated from the existing click-through information in the data set or even from data sets without any explicit click-through information. Design/methodology/approach The proposed ranking algorithm (QRC-Rank) applies RL techniques on a set of calculated click-through features. QRC-Rank is as a two-steps process. In the first step, Transformation phase, a compact benchmark data set is created which contains a set of click-through features. These feature are calculated from the original click-through information available in the data set and constitute a compact representation of click-through information. To find most effective click-through feature, a number of scenarios are investigated. The second phase is Model-Generation, in which a RL model is built to rank the documents. This model is created by applying temporal difference learning methods such as Q-Learning and SARSA. Findings The proposed learning to rank method, QRC-rank, is evaluated on WCL2R and LETOR4.0 data sets. Experimental results demonstrate that QRC-Rank outperforms the state-of-the-art learning to rank methods such as SVMRank, RankBoost, ListNet and AdaRank based on the precision and normalized discount cumulative gain evaluation criteria. The use of the click-through features calculated from the training data set is a major contributor to the performance of the system. Originality/value In this paper, we have demonstrated the viability of the proposed features that provide a compact representation for the click through data in a learning to rank application. These compact click-through features are calculated from the original features of the learning to rank benchmark data set. In addition, a Markov Decision Process model is proposed for the learning to rank problem using RL, including the sets of states, actions, rewarding strategy and the transition function.

scholarly journals Reinforcement Learning for Options Trading

2021 ◽  
Vol 11 (23) ◽  
pp. 11208
Author(s):  
Wen Wen ◽  
Yuyu Yuan ◽  
Jincui Yang
Keyword(s):  
Reinforcement Learning ◽  
Learning Model ◽  
Financial Assets ◽  
Time Intervals ◽  
Stable Algorithm ◽  
Underlying Asset ◽  
Options Trading ◽  
Policy Optimization ◽  
Asset Data ◽  
Reinforcement Learning Model

Reinforcement learning has been applied to various types of financial assets trading, such as stocks, futures, and cryptocurrencies. Options, as a novel kind of derivative, have their characteristics. Because there are too many option contracts for one underlying asset and their price behavior is different. Besides, the validity period of an option contract is relatively short. To apply reinforcement learning to options trading, we propose the options trading reinforcement learning (OTRL) framework. We use options’ underlying asset data to train the reinforcement learning model. Candle data in different time intervals are utilized, respectively. The protective closing strategy is added to the model to prevent unbearable losses. Our experiments demonstrate that the most stable algorithm for obtaining high returns is proximal policy optimization (PPO) with the protective closing strategy. The deep Q network (DQN) can exceed the buy and hold strategy in options trading, as can soft actor critic (SAC). The OTRL framework is verified effectively.

scholarly journals Influence of synchronization within a sensor network on machine learning results

2021 ◽  
Vol 10 (2) ◽  
pp. 233-245
Author(s):  
Tanja Dorst ◽  
Yannick Robin ◽  
Sascha Eichstädt ◽  
Andreas Schütze ◽  
Tizian Schneider
Keyword(s):  
Machine Learning ◽  
Feature Extraction ◽  
Data Augmentation ◽  
Time Synchronization ◽  
Fundamental Problem ◽  
Training Data ◽  
Sensor Data ◽  
Test Bed ◽  
Data Set ◽  
Linear Discriminant

Abstract. Process sensor data allow for not only the control of industrial processes but also an assessment of plant conditions to detect fault conditions and wear by using sensor fusion and machine learning (ML). A fundamental problem is the data quality, which is limited, inter alia, by time synchronization problems. To examine the influence of time synchronization within a distributed sensor system on the prediction performance, a test bed for end-of-line tests, lifetime prediction, and condition monitoring of electromechanical cylinders is considered. The test bed drives the cylinder in a periodic cycle at maximum load, a 1 s period at constant drive speed is used to predict the remaining useful lifetime (RUL). The various sensors for vibration, force, etc. integrated into the test bed are sampled at rates between 10 kHz and 1 MHz. The sensor data are used to train a classification ML model to predict the RUL with a resolution of 1 % based on feature extraction, feature selection, and linear discriminant analysis (LDA) projection. In this contribution, artificial time shifts of up to 50 ms between individual sensors' cycles are introduced, and their influence on the performance of the RUL prediction is investigated. While the ML model achieves good results if no time shifts are introduced, we observed that applying the model trained with unmodified data only to data sets with time shifts results in very poor performance of the RUL prediction even for small time shifts of 0.1 ms. To achieve an acceptable performance also for time-shifted data and thus achieve a more robust model for application, different approaches were investigated. One approach is based on a modified feature extraction approach excluding the phase values after Fourier transformation; a second is based on extending the training data set by including artificially time-shifted data. This latter approach is thus similar to data augmentation used to improve training of neural networks.

scholarly journals Fish Detection Using Deep Learning

2020 ◽  
Vol 2020 ◽  
pp. 1-13 ◽  
Author(s):  
Suxia Cui ◽  
Yu Zhou ◽  
Yonghui Wang ◽  
Lujun Zhai
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Data Augmentation ◽  
Autonomous Underwater Vehicle ◽  
Training Data ◽  
Training Process ◽  
Data Set ◽  
Speed Up ◽  
Fish Detection ◽  
Process Speed

Recently, human being’s curiosity has been expanded from the land to the sky and the sea. Besides sending people to explore the ocean and outer space, robots are designed for some tasks dangerous for living creatures. Take the ocean exploration for an example. There are many projects or competitions on the design of Autonomous Underwater Vehicle (AUV) which attracted many interests. Authors of this article have learned the necessity of platform upgrade from a previous AUV design project, and would like to share the experience of one task extension in the area of fish detection. Because most of the embedded systems have been improved by fast growing computing and sensing technologies, which makes them possible to incorporate more and more complicated algorithms. In an AUV, after acquiring surrounding information from sensors, how to perceive and analyse corresponding information for better judgement is one of the challenges. The processing procedure can mimic human being’s learning routines. An advanced system with more computing power can facilitate deep learning feature, which exploit many neural network algorithms to simulate human brains. In this paper, a convolutional neural network (CNN) based fish detection method was proposed. The training data set was collected from the Gulf of Mexico by a digital camera. To fit into this unique need, three optimization approaches were applied to the CNN: data augmentation, network simplification, and training process speed up. Data augmentation transformation provided more learning samples; the network was simplified to accommodate the artificial neural network; the training process speed up is introduced to make the training process more time efficient. Experimental results showed that the proposed model is promising, and has the potential to be extended to other underwear objects.

Sign in / Sign up

Export Citation Format

Share Document