scholarly journals Short-Term Prediction of Bus Passenger Flow Based on a Hybrid Optimized LSTM Network

2019 ◽  
Vol 8 (9) ◽  
pp. 366 ◽  
Author(s):  
Yong Han ◽  
Cheng Wang ◽  
Yibin Ren ◽  
Shukang Wang ◽  
Huangcheng Zheng ◽  
...  
Keyword(s):  
High Efficiency ◽  
Short Term Memory ◽  
Spatial Scales ◽  
Optimization Procedure ◽  
Stochastic Gradient Descent ◽  
Short Term ◽  
Passenger Flow ◽  
Proposed Model ◽  
Gradient Descent Algorithm ◽  
Lstm Network

The accurate prediction of bus passenger flow is the key to public transport management and the smart city. A long short-term memory network, a deep learning method for modeling sequences, is an efficient way to capture the time dependency of passenger flow. In recent years, an increasing number of researchers have sought to apply the LSTM model to passenger flow prediction. However, few of them pay attention to the optimization procedure during model training. In this article, we propose a hybrid, optimized LSTM network based on Nesterov accelerated adaptive moment estimation (Nadam) and the stochastic gradient descent algorithm (SGD). This method trains the model with high efficiency and accuracy, solving the problems of inefficient training and misconvergence that exist in complex models. We employ a hybrid optimized LSTM network to predict the actual passenger flow in Qingdao, China and compare the prediction results with those obtained by non-hybrid LSTM models and conventional methods. In particular, the proposed model brings about a 4%–20% extra performance improvements compared with those of non-hybrid LSTM models. We have also tried combinations of other optimization algorithms and applications in different models, finding that optimizing LSTM by switching Nadam to SGD is the best choice. The sensitivity of the model to its parameters is also explored, which provides guidance for applying this model to bus passenger flow data modelling. The good performance of the proposed model in different temporal and spatial scales shows that it is more robust and effective, which can provide insightful support and guidance for dynamic bus scheduling and regional coordination scheduling.

scholarly journals BJBN： BERT-JOIN-BiLSTM Networks for Medical Auxiliary Diagnostic

2022 ◽  
Vol 2022 ◽  
pp. 1-7
Author(s):  
Chuanjie Xu ◽  
Feng Yuan ◽  
Shouqiang Chen
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Local Features ◽  
Global Information ◽  
Short Term ◽  
Baseline Model ◽  
Proposed Model ◽  
Global Representation ◽  
Long Short Term Memory ◽  
Lstm Network

This study proposed a medicine auxiliary diagnosis model based on neural network. The model combines a bidirectional long short-term memory（Bi-LSTM）network and bidirectional encoder representations from transformers (BERT), which can well complete the extraction of local features of Chinese medicine texts. BERT can learn the global information of the text, so use BERT to get the global representation of medical text and then use Bi-LSTM to extract local features. We conducted a large number of comparative experiments on datasets. The results show that the proposed model has significant advantages over the state-of-the-art baseline model. The accuracy of the proposed model is 0.75.

scholarly journals Dynamic Displacement Forecasting of Dashuitian Landslide in China Using Variational Mode Decomposition and Stack Long Short-Term Memory Network

2019 ◽  
Vol 9 (15) ◽  
pp. 2951 ◽  
Author(s):  
Yin Xing ◽  
Jianping Yue ◽  
Chuang Chen ◽  
Kanglin Cong ◽  
Shaolin Zhu ◽  
...  
Keyword(s):  
Short Term Memory ◽  
Forecast Accuracy ◽  
Short Term ◽  
Dynamic Displacement ◽  
Term Memory ◽  
Mode Decomposition ◽  
Proposed Model ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Lstm Network

In recent decades, landslide displacement forecasting has received increasing attention due to its ability to reduce landslide hazards. To improve the forecast accuracy of landslide displacement, a dynamic forecasting model based on variational mode decomposition (VMD) and a stack long short-term memory network (SLSTM) is proposed. VMD is used to decompose landslide displacement into different displacement subsequences, and the SLSTM network is used to forecast each displacement subsequence. Then, the forecast values of landslide displacement are obtained by reconstructing the forecast values of all displacement subsequences. On the other hand, the SLSTM networks are updated by adding the forecast values into the training set, realizing the dynamic displacement forecasting. The proposed model was verified on the Dashuitian landslide in China. The results show that compared with the two advanced forecasting models, long short-term memory (LSTM) network, and empirical mode decomposition (EMD)–LSTM network, the proposed model has higher forecast accuracy.

scholarly journals A Hybrid GLM Model for Predicting Citywide Spatio-Temporal Metro Passenger Flow

2021 ◽  
Vol 10 (4) ◽  
pp. 222
Author(s):  
Yong Han ◽  
Tongxin Peng ◽  
Cheng Wang ◽  
Zhihao Zhang ◽  
Ge Chen
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Percentage Error ◽  
Spatial Dependency ◽  
Short Term ◽  
Attention Networks ◽  
Passenger Flow ◽  
Typical Data ◽  
Proposed Model ◽  
Spatio Temporal

Accurate prediction of citywide short-term metro passenger flow is essential to urban management and transport scheduling. Recently, an increasing number of researchers have applied deep learning models to passenger flow prediction. Nevertheless, the task is still challenging due to the complex spatial dependency on the metro network and the time-varying traffic patterns. Therefore, we propose a novel deep learning architecture combining graph attention networks (GAT) with long short-term memory (LSTM) networks, which is called the hybrid GLM (hybrid GAT and LSTM Model). The proposed model captures the spatial dependency via the graph attention layers and learns the temporal dependency via the LSTM layers. Moreover, some external factors are embedded. We tested the hybrid GLM by predicting the metro passenger flow in Shanghai, China. The results are compared with the forecasts from some typical data-driven models. The hybrid GLM gets the smallest root-mean-square error (RMSE) and mean absolute percentage error (MAPE) in different time intervals (TIs), which exhibits the superiority of the proposed model. In particular, in the TI 10 min, the hybrid GLM brings about 6–30% extra improvements in terms of RMSE. We additionally explore the sensitivity of the model to its parameters, which will aid the application of this model.

scholarly journals A Combined Method for MEMS Gyroscope Error Compensation Using a Long Short-Term Memory Network and Kalman Filter in Random Vibration Environments

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1181
Author(s):  
Chenhao Zhu ◽  
Sheng Cai ◽  
Yifan Yang ◽  
Wei Xu ◽  
Honghai Shen ◽  
...  
Keyword(s):  
Kalman Filter ◽  
Standard Deviation ◽  
Error Compensation ◽  
Random Vibration ◽  
Short Term Memory ◽  
Combined Method ◽  
Short Term ◽  
Mems Gyroscope ◽  
Long Short Term Memory ◽  
Lstm Network

In applications such as carrier attitude control and mobile device navigation, a micro-electro-mechanical-system (MEMS) gyroscope will inevitably be affected by random vibration, which significantly affects the performance of the MEMS gyroscope. In order to solve the degradation of MEMS gyroscope performance in random vibration environments, in this paper, a combined method of a long short-term memory (LSTM) network and Kalman filter (KF) is proposed for error compensation, where Kalman filter parameters are iteratively optimized using the Kalman smoother and expectation-maximization (EM) algorithm. In order to verify the effectiveness of the proposed method, we performed a linear random vibration test to acquire MEMS gyroscope data. Subsequently, an analysis of the effects of input data step size and network topology on gyroscope error compensation performance is presented. Furthermore, the autoregressive moving average-Kalman filter (ARMA-KF) model, which is commonly used in gyroscope error compensation, was also combined with the LSTM network as a comparison method. The results show that, for the x-axis data, the proposed combined method reduces the standard deviation (STD) by 51.58% and 31.92% compared to the bidirectional LSTM (BiLSTM) network, and EM-KF method, respectively. For the z-axis data, the proposed combined method reduces the standard deviation by 29.19% and 12.75% compared to the BiLSTM network and EM-KF method, respectively. Furthermore, for x-axis data and z-axis data, the proposed combined method reduces the standard deviation by 46.54% and 22.30% compared to the BiLSTM-ARMA-KF method, respectively, and the output is smoother, proving the effectiveness of the proposed method.

scholarly journals Air pollution forecasting application based on deep learning model and optimization algorithm

2021 ◽  
Author(s):  
Azim Heydari ◽  
Meysam Majidi Nezhad ◽  
Davide Astiaso Garcia ◽  
Farshid Keynia ◽  
Livio De Santoli
Keyword(s):  
Air Pollution ◽  
Wind Speed ◽  
Power Plant ◽  
Air Temperature ◽  
Short Term Memory ◽  
Combined Cycle ◽  
Short Term ◽  
Term Memory ◽  
Proposed Model ◽  
Long Short Term Memory

AbstractAir pollution monitoring is constantly increasing, giving more and more attention to its consequences on human health. Since Nitrogen dioxide (NO2) and sulfur dioxide (SO2) are the major pollutants, various models have been developed on predicting their potential damages. Nevertheless, providing precise predictions is almost impossible. In this study, a new hybrid intelligent model based on long short-term memory (LSTM) and multi-verse optimization algorithm (MVO) has been developed to predict and analysis the air pollution obtained from Combined Cycle Power Plants. In the proposed model, long short-term memory model is a forecaster engine to predict the amount of produced NO2 and SO2 by the Combined Cycle Power Plant, where the MVO algorithm is used to optimize the LSTM parameters in order to achieve a lower forecasting error. In addition, in order to evaluate the proposed model performance, the model has been applied using real data from a Combined Cycle Power Plant in Kerman, Iran. The datasets include wind speed, air temperature, NO2, and SO2 for five months (May–September 2019) with a time step of 3-h. In addition, the model has been tested based on two different types of input parameters: type (1) includes wind speed, air temperature, and different lagged values of the output variables (NO2 and SO2); type (2) includes just lagged values of the output variables (NO2 and SO2). The obtained results show that the proposed model has higher accuracy than other combined forecasting benchmark models (ENN-PSO, ENN-MVO, and LSTM-PSO) considering different network input variables. Graphic abstract

scholarly journals Extraction of local and global features by a convolutional neural network–long short-term memory network for diagnosing bearing faults

2021 ◽  
pp. 095440622110165
Author(s):  
Zhang Chao ◽  
Wang Wei-zhi ◽  
Zhang Chen ◽  
Fan Bin ◽  
Wang Jian-guo ◽  
...  
Keyword(s):  
Neural Network ◽  
Fault Diagnosis ◽  
Condition Monitoring ◽  
Short Term Memory ◽  
Vibration Signal ◽  
Short Term ◽  
Global Features ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Lstm Network

Accurate and reliable fault diagnosis is one of the key and difficult issues in mechanical condition monitoring. In recent years, Convolutional Neural Network (CNN) has been widely used in mechanical condition monitoring, which is also a great breakthrough in the field of bearing fault diagnosis. However, CNN can only extract local features of signals. The model accuracy and generalization of the original vibration signals are very low in the process of vibration signal processing only by CNN. Based on the above problems, this paper improves the traditional convolution layer of CNN, and builds the learning module (local feature learning block, LFLB) of the local characteristics. At the same time, the Long Short-Term Memory (LSTM) is introduced into the network, which is used to extract the global features. This paper proposes the new neural network—improved CNN-LSTM network. The extracted deep feature is used for fault classification. The improved CNN-LSTM network is applied to the processing of the vibration signal of the faulty bearing collected by the bearing failure laboratory of Inner Mongolia University of science and technology. The results show that the accuracy of the improved CNN-LSTM network on the same batch test set is 98.75%, which is about 24% higher than that of the traditional CNN. The proposed network is applied to the bearing data collection of Western Reserve University under the condition that the network parameters remain unchanged. The experiment shows that the improved CNN-LSTM network has better generalization than the traditional CNN.

Sentence similarity evaluation using Sent2Vec and siamese neural network with parallel structure

2021 ◽  
pp. 1-10
Author(s):  
Hye-Jeong Song ◽  
Tak-Sung Heo ◽  
Jong-Dae Kim ◽  
Chan-Young Park ◽  
Yu-Seop Kim
Keyword(s):  
Neural Network ◽  
Language Processing ◽  
Short Term Memory ◽  
Parallel Structure ◽  
Short Term ◽  
Similarity Estimation ◽  
Accurate Judgment ◽  
Proposed Model ◽  
Sentence Similarity ◽  
Long Short Term Memory

Sentence similarity evaluation is a significant task used in machine translation, classification, and information extraction in the field of natural language processing. When two sentences are given, an accurate judgment should be made whether the meaning of the sentences is equivalent even if the words and contexts of the sentences are different. To this end, existing studies have measured the similarity of sentences by focusing on the analysis of words, morphemes, and letters. To measure sentence similarity, this study uses Sent2Vec, a sentence embedding, as well as morpheme word embedding. Vectors representing words are input to the 1-dimension convolutional neural network (1D-CNN) with various sizes of kernels and bidirectional long short-term memory (Bi-LSTM). Self-attention is applied to the features transformed through Bi-LSTM. Subsequently, vectors undergoing 1D-CNN and self-attention are converted through global max pooling and global average pooling to extract specific values, respectively. The vectors generated through the above process are concatenated to the vector generated through Sent2Vec and are represented as a single vector. The vector is input to softmax layer, and finally, the similarity between the two sentences is determined. The proposed model can improve the accuracy by up to 5.42% point compared with the conventional sentence similarity estimation models.

scholarly journals Identifying vulgarity in Bengali social media textual content

2021 ◽  
Vol 7 ◽  
pp. e665
Author(s):  
Salim Sazzed
Keyword(s):  
Social Media ◽  
Gradient Descent ◽  
Short Term Memory ◽  
Stochastic Gradient Descent ◽  
Media Content ◽  
Short Term ◽  
Long Short Term Memory ◽  
Highly Correlated ◽  
Negative Sentiment ◽  
Textual Content

The presence of abusive and vulgar language in social media has become an issue of increasing concern in recent years. However, research pertaining to the prevalence and identification of vulgar language has remained largely unexplored in low-resource languages such as Bengali. In this paper, we provide the first comprehensive analysis on the presence of vulgarity in Bengali social media content. We develop two benchmark corpora consisting of 7,245 reviews collected from YouTube and manually annotate them into vulgar and non-vulgar categories. The manual annotation reveals the ubiquity of vulgar and swear words in Bengali social media content (i.e., in two corpora), ranging from 20% to 34%. To automatically identify vulgarity, we employ various approaches, such as classical machine learning (CML) classifiers, Stochastic Gradient Descent (SGD) optimizer, a deep learning (DL) based architecture, and lexicon-based methods. Although small in size, we find that the swear/vulgar lexicon is effective at identifying the vulgar language due to the high presence of some swear terms in Bengali social media. We observe that the performances of machine leanings (ML) classifiers are affected by the class distribution of the dataset. The DL-based BiLSTM (Bidirectional Long Short Term Memory) model yields the highest recall scores for identifying vulgarity in both datasets (i.e., in both original and class-balanced settings). Besides, the analysis reveals that vulgarity is highly correlated with negative sentiment in social media comments.

scholarly journals Intelligent Islanding Detection of Microgrids Using Long Short-Term Memory Networks

Energies ◽  
2021 ◽  
Vol 14 (18) ◽  
pp. 5762
Author(s):  
Syed Basit Ali Bukhari ◽  
Khawaja Khalid Mehmood ◽  
Abdul Wadood ◽  
Herie Park
Keyword(s):  
Short Term Memory ◽  
Computational Time ◽  
Islanding Detection ◽  
Phase Voltage ◽  
Short Term ◽  
Term Memory ◽  
Three Phase ◽  
Empirical Wavelet Transform ◽  
Long Short Term Memory ◽  
Lstm Network

This paper presents a new intelligent islanding detection scheme (IIDS) based on empirical wavelet transform (EWT) and long short-term memory (LSTM) network to identify islanding events in microgrids. The concept of EWT is extended to extract features from three-phase signals. First, the three-phase voltage signals sampled at the terminal of targeted distributed energy resource (DER) or point of common coupling (PCC) are decomposed into empirical modes/frequency subbands using EWT. Then, instantaneous amplitudes and instantaneous frequencies of the three-phases at different frequency subbands are combined, and various statistical features are calculated. Finally, the EWT-based features along with the three-phase voltage signals are input to the LSTM network to differentiate between non-islanding and islanding events. To assess the efficacy of the proposed IIDS, extensive simulations are performed on an IEC microgrid and an IEEE 34-node system. The simulation results verify the effectiveness of the proposed IIDS in terms of non-detection zone (NDZ), computational time, detection accuracy, and robustness against noisy measurement. Furthermore, comparisons with existing intelligent methods and different LSTM architectures demonstrate that the proposed IIDS offers higher reliability by significantly reducing the NDZ and stands robust against measurements uncertainty.

Production Forecasting with the Interwell Interference by Integrating Graph Convolutional and Long Short-Term Memory Neural Network

2021 ◽  
pp. 1-17
Author(s):  
Enda Du ◽  
Yuetian Liu ◽  
Ziyan Cheng ◽  
Liang Xue ◽  
Jing Ma ◽  
...  
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Short Term ◽  
Term Memory ◽  
Production Forecasting ◽  
Temporal Correlations ◽  
Proposed Model ◽  
The Mean ◽  
Long Short Term Memory ◽  
The Impact

Summary Accurate production forecasting is an essential task and accompanies the entire process of reservoir development. With the limitation of prediction principles and processes, the traditional approaches are difficult to make rapid predictions. With the development of artificial intelligence, the data-driven model provides an alternative approach for production forecasting. To fully take the impact of interwell interference on production into account, this paper proposes a deep learning-based hybrid model (GCN-LSTM), where graph convolutional network (GCN) is used to capture complicated spatial patterns between each well, and long short-term memory (LSTM) neural network is adopted to extract intricate temporal correlations from historical production data. To implement the proposed model more efficiently, two data preprocessing procedures are performed: Outliers in the data set are removed by using a box plot visualization, and measurement noise is reduced by a wavelet transform. The robustness and applicability of the proposed model are evaluated in two scenarios of different data types with the root mean square error (RMSE), the mean absolute error (MAE), and the mean absolute percentage error (MAPE). The results show that the proposed model can effectively capture spatial and temporal correlations to make a rapid and accurate oil production forecast.

Sign in / Sign up

Export Citation Format

Share Document