scholarly journals Hybrid LSTM Self-Attention Mechanism Model for Forecasting the Reform of Scientific Research in Morocco

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Asmaa Fahim ◽  
Qingmei Tan ◽  
Mouna Mazzi ◽  
Md Sahabuddin ◽  
Bushra Naz ◽  
...  
Keyword(s):  
Time Series ◽  
Short Term Memory ◽  
Research Area ◽  
Vital Role ◽  
Attention Mechanism ◽  
Percentage Increase ◽  
Mechanism Model ◽  
Research Outcomes ◽  
Future Outcomes ◽  
The Impact

Education is the cultivation of people to promote and guarantee the development of society. Education reforms can play a vital role in the development of a country. However, it is crucial to continually monitor the educational model’s performance by forecasting the outcome’s progress. Machine learning-based models are currently a hot topic in improving the forecasting research area. Forecasting models can help to analyse the impact of future outcomes by showing yearly trends. For this study, we developed a hybrid, forecasting time-series model by long short-term memory (LSTM) network and self-attention mechanism (SAM) to monitor Morocco’s educational reform. We analysed six universities’ performance and provided a prediction model to evaluate the best-performing university’s performance after implementing the latest reform, i.e., from 2015–2030. We forecasted the six universities’ research outcomes and tested our proposed methodology’s accuracy against other time-series models. Results show that our model performs better for predicting research outcomes. The percentage increase in university performance after nine years is discussed to help predict the best-performing university. Our proposed algorithm accuracy and performance are better than other algorithms like LSTM and RNN.

scholarly journals Emotion-Semantic-Enhanced Bidirectional LSTM with Multi-Head Attention Mechanism for Microblog Sentiment Analysis

Information ◽  
2020 ◽  
Vol 11 (5) ◽  
pp. 280
Author(s):  
Shaoxiu Wang ◽  
Yonghua Zhu ◽  
Wenjing Gao ◽  
Meng Cao ◽  
Mengyao Li
Keyword(s):  
Sentiment Analysis ◽  
Short Term Memory ◽  
Syntactic Structure ◽  
Contextual Information ◽  
Research Field ◽  
Attention Mechanism ◽  
Feature Representation ◽  
Mechanism Model ◽  
Hidden Layer ◽  
The Impact

The sentiment analysis of microblog text has always been a challenging research field due to the limited and complex contextual information. However, most of the existing sentiment analysis methods for microblogs focus on classifying the polarity of emotional keywords while ignoring the transition or progressive impact of words in different positions in the Chinese syntactic structure on global sentiment, as well as the utilization of emojis. To this end, we propose the emotion-semantic-enhanced bidirectional long short-term memory (BiLSTM) network with the multi-head attention mechanism model (EBILSTM-MH) for sentiment analysis. This model uses BiLSTM to learn feature representation of input texts, given the word embedding. Subsequently, the attention mechanism is used to assign the attentive weights of each words to the sentiment analysis based on the impact of emojis. The attentive weights can be combined with the output of the hidden layer to obtain the feature representation of posts. Finally, the sentiment polarity of microblog can be obtained through the dense connection layer. The experimental results show the feasibility of our proposed model on microblog sentiment analysis when compared with other baseline models.

scholarly journals Deep Learning-Based Maximum Temperature Forecasting Assisted with Meta-Learning for Hyperparameter Optimization

Atmosphere ◽  
2020 ◽  
Vol 11 (5) ◽  
pp. 487 ◽  
Author(s):  
Trang Thi Kieu Tran ◽  
Taesam Lee ◽  
Ju-Young Shin ◽  
Jong-Suk Kim ◽  
Mohamad Kamruzzaman
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Human Life ◽  
Time Series Prediction ◽  
Vital Role ◽  
Maximum Temperature ◽  
Hyperparameter Optimization ◽  
Meta Learning

Time series forecasting of meteorological variables such as daily temperature has recently drawn considerable attention from researchers to address the limitations of traditional forecasting models. However, a middle-range (e.g., 5–20 days) forecasting is an extremely challenging task to get reliable forecasting results from a dynamical weather model. Nevertheless, it is challenging to develop and select an accurate time-series prediction model because it involves training various distinct models to find the best among them. In addition, selecting an optimum topology for the selected models is important too. The accurate forecasting of maximum temperature plays a vital role in human life as well as many sectors such as agriculture and industry. The increase in temperature will deteriorate the highland urban heat, especially in summer, and have a significant influence on people’s health. We applied meta-learning principles to optimize the deep learning network structure for hyperparameter optimization. In particular, the genetic algorithm (GA) for meta-learning was used to select the optimum architecture for the network used. The dataset was used to train and test three different models, namely the artificial neural network (ANN), recurrent neural network (RNN), and long short-term memory (LSTM). Our results demonstrate that the hybrid model of an LSTM network and GA outperforms other models for the long lead time forecasting. Specifically, LSTM forecasts have superiority over RNN and ANN for 15-day-ahead in summer with the root mean square error (RMSE) value of 2.719 (°C).

scholarly journals STL-ATTLSTM: Vegetable Price Forecasting Using STL and Attention Mechanism-Based LSTM

Agriculture ◽  
2020 ◽  
Vol 10 (12) ◽  
pp. 612
Author(s):  
Helin Yin ◽  
Dong Jin ◽  
Yeong Hyeon Gu ◽  
Chang Jin Park ◽  
Sang Keun Han ◽  
...  
Keyword(s):  
Time Series ◽  
Prediction Accuracy ◽  
Crop Production ◽  
Time Series Data ◽  
Short Term Memory ◽  
Attention Mechanism ◽  
Series Data ◽  
Percentage Error ◽  
Hot Pepper ◽  
Types Of Information

It is difficult to forecast vegetable prices because they are affected by numerous factors, such as weather and crop production, and the time-series data have strong non-linear and non-stationary characteristics. To address these issues, we propose the STL-ATTLSTM (STL-Attention-based LSTM) model, which integrates the seasonal trend decomposition using the Loess (STL) preprocessing method and attention mechanism based on long short-term memory (LSTM). The proposed STL-ATTLSTM forecasts monthly vegetable prices using various types of information, such as vegetable prices, weather information of the main production areas, and market trading volumes. The STL method decomposes time-series vegetable price data into trend, seasonality, and remainder components. It uses the remainder component by removing the trend and seasonality components. In the model training process, attention weights are assigned to all input variables; thus, the model’s prediction performance is improved by focusing on the variables that affect the prediction results. The proposed STL-ATTLSTM was applied to five crops, namely cabbage, radish, onion, hot pepper, and garlic, and its performance was compared to three benchmark models (i.e., LSTM, attention LSTM, and STL-LSTM). The performance results show that the LSTM model combined with the STL method (STL-LSTM) achieved a 12% higher prediction accuracy than the attention LSTM model that did not use the STL method and solved the prediction lag arising from high seasonality. The attention LSTM model improved the prediction accuracy by approximately 4% to 5% compared to the LSTM model. The STL-ATTLSTM model achieved the best performance, with an average root mean square error (RMSE) of 380, and an average mean absolute percentage error (MAPE) of 7%.

scholarly journals Time-Series Classification Based on Fusion Features of Sequence and Visualization

2020 ◽  
Vol 10 (12) ◽  
pp. 4124
Author(s):  
Baoquan Wang ◽  
Tonghai Jiang ◽  
Xi Zhou ◽  
Bo Ma ◽  
Fan Zhao ◽  
...  
Keyword(s):  
Time Series ◽  
Human Brain ◽  
Time Series Data ◽  
Short Term Memory ◽  
Computational Cost ◽  
Open Data ◽  
Attention Mechanism ◽  
Series Data ◽  
Sequence Features ◽  
Fusion Features

For the task of time-series data classification (TSC), some methods directly classify raw time-series (TS) data. However, certain sequence features are not evident in the time domain and the human brain can extract visual features based on visualization to classify data. Therefore, some researchers have converted TS data to image data and used image processing methods for TSC. While human perceptionconsists of a combination of human senses from different aspects, existing methods only use sequence features or visualization features. Therefore, this paper proposes a framework for TSC based on fusion features (TSC-FF) of sequence features extracted from raw TS and visualization features extracted from Area Graphs converted from TS. Deep learning methods have been proven to be useful tools for automatically learning features from data; therefore, we use long short-term memory with an attention mechanism (LSTM-A) to learn sequence features and a convolutional neural network with an attention mechanism (CNN-A) for visualization features, in order to imitate the human brain. In addition, we use the simplest visualization method of Area Graph for visualization features extraction, avoiding loss of information and additional computational cost. This article aims to prove that using deep neural networks to learn features from different aspects and fusing them can replace complex, artificially constructed features, as well as remove the bias due to manually designed features, in order to avoid the limitations of domain knowledge. Experiments on several open data sets show that the framework achieves promising results, compared with other methods.

scholarly journals Research on Short-Term Load Prediction Based on Seq2seq Model

Energies ◽  
2019 ◽  
Vol 12 (16) ◽  
pp. 3199 ◽  
Author(s):  
Gangjun Gong ◽  
Xiaonan An ◽  
Nawaraj Kumar Mahato ◽  
Shuyan Sun ◽  
Si Chen ◽  
...  
Keyword(s):  
Power System ◽  
Prediction Model ◽  
Short Term Memory ◽  
Predictive Performance ◽  
Weather Conditions ◽  
Attention Mechanism ◽  
Load Prediction ◽  
Short Term ◽  
System Load ◽  
The Impact

Electricity load prediction is the primary basis on which power-related departments to make logical and effective generation plans and scientific scheduling plans for the most effective power utilization. The perpetual evolution of deep learning has recommended advanced and innovative concepts for short-term load prediction. Taking into consideration the time and nonlinear characteristics of power system load data and further considering the impact of historical and future information on the current state, this paper proposes a Seq2seq short-term load prediction model based on a long short-term memory network (LSTM). Firstly, the periodic fluctuation characteristics of users’ load data are analyzed, establishing a correlation of the load data so as to determine the model’s order in the time series. Secondly, the specifications of the Seq2seq model are given preference and a coalescence of the Residual mechanism (Residual) and the two Attention mechanisms (Attention) is developed. Then, comparing the predictive performance of the model under different types of Attention mechanism, this paper finally adopts the Seq2seq short-term load prediction model of Residual LSTM and the Bahdanau Attention mechanism. Eventually, the prediction model obtains better results when merging the actual power system load data of a certain place. In order to validate the developed model, the Seq2seq was compared with recurrent neural network (RNN), LSTM, and gated recurrent unit (GRU) algorithms. Last but not least, the performance indices were calculated. when training and testing the model with power system load data, it was noted that the root mean square error (RMSE) of Seq2seq was decreased by 6.61%, 16.95%, and 7.80% compared with RNN, LSTM, and GRU, respectively. In addition, a supplementary case study was carried out using data for a small power system considering different weather conditions and user behaviors in order to confirm the applicability and stability of the proposed model. The Seq2seq model for short-term load prediction can be reported to demonstrate superiority in all areas, exhibiting better prediction and stable performance.

scholarly journals Evaluation and Comparison of Random Forest and A-LSTM Networks for Large-scale Winter Wheat Identification

2019 ◽  
Vol 11 (14) ◽  
pp. 1665 ◽  
Author(s):  
Tianle He ◽  
Chuanjie Xie ◽  
Qingsheng Liu ◽  
Shiying Guan ◽  
Gaohuan Liu
Keyword(s):  
Time Series ◽  
Random Forest ◽  
Winter Wheat ◽  
Large Scale ◽  
Short Term Memory ◽  
State Of The Art ◽  
Training Sample ◽  
Central China ◽  
Surface Reflectance ◽  
The Impact

Machine learning comprises a group of powerful state-of-the-art techniques for land cover classification and cropland identification. In this paper, we proposed and evaluated two models based on random forest (RF) and attention-based long short-term memory (A-LSTM) networks that can learn directly from the raw surface reflectance of remote sensing (RS) images for large-scale winter wheat identification in Huanghuaihai Region (North-Central China). We used a time series of Moderate Resolution Imaging Spectroradiometer (MODIS) images over one growing season and the corresponding winter wheat distribution map for the experiments. Each training sample was derived from the raw surface reflectance of MODIS time-series images. Both models achieved state-of-the-art performance in identifying winter wheat, and the F1 scores of RF and A-LSTM were 0.72 and 0.71, respectively. We also analyzed the impact of the pixel-mixing effect. Training with pure-mixed-pixel samples (the training set consists of pure and mixed cells and thus retains the original distribution of data) was more precise than training with only pure-pixel samples (the entire pixel area belongs to one class). We also analyzed the variable importance along the temporal series, and the data acquired in March or April contributed more than the data acquired at other times. Both models could predict winter wheat coverage in past years or in other regions with similar winter wheat growing seasons. The experiments in this paper showed the effectiveness and significance of our methods.

scholarly journals Improved ACD-Based Financial Trade Durations Prediction Leveraging LSTM Networks and Attention Mechanism

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Yong Shi ◽  
Wei Dai ◽  
Wen Long ◽  
Bo Li
Keyword(s):  
Large Scale ◽  
Short Term Memory ◽  
Input Sequence ◽  
Parameter Tuning ◽  
Attention Mechanism ◽  
Market Liquidity ◽  
Security Market ◽  
Modelling Framework ◽  
Acd Model ◽  
The Impact

The liquidity risk factor of security market plays an important role in the formulation of trading strategies. A more liquid stock market means that the securities can be bought or sold more easily. As a sound indicator of market liquidity, the transaction duration is the focus of this study. We concentrate on estimating the probability density function p Δ t i + 1 | G i , where Δ t i + 1 represents the duration of the (i + 1)-th transaction and G i represents the historical information at the time when the (i + 1)-th transaction occurs. In this paper, we propose a new ultrahigh-frequency (UHF) duration modelling framework by utilizing long short-term memory (LSTM) networks to extend the conditional mean equation of classic autoregressive conditional duration (ACD) model while retaining the probabilistic inference ability. And then, the attention mechanism is leveraged to unveil the internal mechanism of the constructed model. In order to minimize the impact of manual parameter tuning, we adopt fixed hyperparameters during the training process. The experiments applied to a large-scale dataset prove the superiority of the proposed hybrid models. In the input sequence, the temporal positions which are more important for predicting the next duration can be efficiently highlighted via the added attention mechanism layer.

scholarly journals Study of the impact of the COVID-19 pandemic on international air transportation

2021 ◽  
Vol 29 (1) ◽  
pp. 22-35
Author(s):  
Eugeny Yu. Shchetinin
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Stock Prices ◽  
Short Term Memory ◽  
Air Transportation ◽  
Time Series Forecasting ◽  
The Cost ◽  
The Impact

Time Series Forecasting has always been a very important area of research in many domains because many different types of data are stored as time series. Given the growing availability of data and computing power in the recent years, Deep Learning has become a fundamental part of the new generation of Time Series Forecasting models, obtaining excellent results.As different time series problems are studied in many different fields, a large number of new architectures have been developed in recent years. This has also been simplified by the growing availability of open source frameworks, which make the development of new custom network components easier and faster.In this paper three different Deep Learning Architecture for Time Series Forecasting are presented: Recurrent Neural Networks (RNNs), that are the most classical and used architecture for Time Series Forecasting problems; Long Short-Term Memory (LSTM), that are an evolution of RNNs developed in order to overcome the vanishing gradient problem; Gated Recurrent Unit (GRU), that are another evolution of RNNs, similar to LSTM.The article is devoted to modeling and forecasting the cost of international air transportation in a pandemic using deep learning methods. The author builds time series models of the American Airlines (AAL) stock prices for a selected period using LSTM, GRU, RNN recurrent neural networks models and compare the accuracy forecast results.

Non-linear ADRL estimation of corruption and FDI inflow to Ghana

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Randolph Nsor-Ambala ◽  
Cephas Paa Kwasi Coffie
Keyword(s):  
Time Series ◽  
Rational Expectation ◽  
Efficient Market Hypothesis ◽  
Empirical Examination ◽  
Data Set ◽  
Fdi Inflows ◽  
Content Type ◽  
Non Linear ◽  
Future Outcomes ◽  
The Impact

Purpose This paper aims to examine the effect of corruption on foreign direct investment (FDI) inflow in Ghana. This provides answers to the call for further empirical examination of the contextual impact of corruption on FDI inflow. Design/methodology/approach The study uses a non-linear ADRL time series econometric model to estimate data from the World Bank and the international country risk guide (1984–2019). Findings The study confirms the sand in the wheel and the grabbing hand hypothesis of the impact of control of corruption (CoC) on FDI both in the short and long run. However, degradation on the CoC index has a significant and more than a proportionate constraint on FDI inflows, while an improvement in CoC has no significant impact on improving FDI inflows. An explanation for this outcome was proposed after comparing this finding to a similar prior study with a Nigerian data set (Zangina and Hassan, 2020). The proposed explanation relied mainly on the rational expectation hypothesis and drawing elements of the efficient market hypothesis. FDI inflows do not react to outcomes or trajectories reasonably expected because such rationally expected future outcomes will have been modelled into existing FDI movement decisions. Instead, FDI flows react to “surprises” and often respond in a more than proportional manner. Practical implications Political leadership in Ghana should be conscious of the severe adverse effects of inaction or ineffective action in curbing corruption, leading to slippering in CoC rankings. In the case of Ghana, the dependence of FDI on CoC is even more pronounced as the other variables within the specified model show an insignificant impact on FDI. Additionally, admittedly aggregated cross-country data in econometric modelling is appealing and has some empirical basis, but these must not erode the relevance of country-specific studies as both are needed to support theorization. Originality/value The paper is among the first to test for the asymmetric relationship between corruption or its control thereof and FDI with a time series approach, and hence, the findings offer new insight.

scholarly journals Prediksi Penambahan Kasus Covid-19 di Indonesia Melalui Pendekatan Time Series Menggunakan Metode Exponential Smoothing

2021 ◽  
Vol 6 (1) ◽  
pp. 112
Author(s):  
Calvin Mikhailouzna Gibran ◽  
Sulis Setiyawati ◽  
Febri Liantoni
Keyword(s):  
Time Series ◽  
Exponential Smoothing ◽  
Percentage Increase ◽  
Prediction System ◽  
Accurate Data ◽  
Time Series Approach ◽  
The Government ◽  
The Right ◽  
Exponential Smoothing Method ◽  
The Impact

The Covid-19 pandemic in Indonesia has emerged starting in 2020. To know the development of cases, a good calculation is needed. A prediction system can help in analyzing accurate data on positive causes, cures, and deaths. The right prediction or forecast can be the answer to the question of the impact that will occur, forecasting will provide an overview to the government and the community so that it is hoped that related parties can prepare for future impacts or even reduce the number of cases growth. In this study, the Exponential Smoothing method was used as a prediction calculation. This method is simple but effective in producing accurate predictions. Forecasting data used comes from the Indonesian government with the assumption that the data is valid and reliable. Based on research that has been carried out to predict the increase in new cases of the Indonesian National Covid-19, the best alpha (α) value is 0.33 with an SSE of 1048027,939. This shows that the number of cases is increasing. The results of forecasting in this study using the time series approach and the SES method are more suitable for predicting the percentage increase in cases than knowing the exact number.

Sign in / Sign up

Export Citation Format

Share Document