A Hybrid CNN–LSTM Algorithm for Online Defect Recognition of CO2 Welding

Tianyuan Liu; Jinsong Bao; Junliang Wang; Yiming Zhang

doi:10.3390/s18124369

A Hybrid CNN–LSTM Algorithm for Online Defect Recognition of CO2 Welding

Sensors ◽

10.3390/s18124369 ◽

2018 ◽

Vol 18 (12) ◽

pp. 4369 ◽

Cited By ~ 13

Author(s):

Tianyuan Liu ◽

Jinsong Bao ◽

Junliang Wang ◽

Yiming Zhang

Keyword(s):

Molten Pool ◽

Short Term Memory ◽

Feature Fusion ◽

Automatic Welding ◽

Test Results ◽

Welding Defects ◽

Co2 Welding ◽

Lstm Network ◽

Defect Recognition ◽

Pool Image

At present, realizing high-quality automatic welding through online monitoring is a research focus in engineering applications. In this paper, a CNN–LSTM algorithm is proposed, which combines the advantages of convolutional neural networks (CNNs) and long short-term memory networks (LSTMs). The CNN–LSTM algorithm establishes a shallow CNN to extract the primary features of the molten pool image. Then the feature tensor extracted by the CNN is transformed into the feature matrix. Finally, the rows of the feature matrix are fed into the LSTM network for feature fusion. This process realizes the implicit mapping from molten pool images to welding defects. The test results on the self-made molten pool image dataset show that CNN contributes to the overall feasibility of the CNN–LSTM algorithm and LSTM network is the most superior in the feature hybrid stage. The algorithm converges at 300 epochs and the accuracy of defects detection in CO2 welding molten pool is 94%. The processing time of a single image is 0.067 ms, which fully meets the real-time monitoring requirement based on molten pool image. The experimental results on the MNIST and FashionMNIST datasets show that the algorithm is universal and can be used for similar image recognition and classification tasks.

Download Full-text

Based on the CO2 Gas Shielded Welding Molten Pool Image Edge Detection Algorithm

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.437.840 ◽

2013 ◽

Vol 437 ◽

pp. 840-844 ◽

Cited By ~ 1

Author(s):

Xiao Gang Liu ◽

Bing Zhao

Keyword(s):

Edge Detection ◽

Molten Pool ◽

High Speed ◽

Vision System ◽

Detection Algorithm ◽

Pass Filter ◽

Low Pass Filter ◽

Low Pass ◽

Defect Recognition ◽

Pool Image

This paper use the passive vision system through high-speed camera collects molten pool images; and then according to the frequency domain characteristics of the weld pool image Butterworth low-pass filter; gradient method for image enhancement obtained after pretreatment. Research Roberts, Sobel, Prewitt, Log, Zerocross, and Canny 6 both traditional differential operator edge detection processing results. Through comparison and analysis of choosing threshold for [0.1, 0. Canny operator can get the ideal molten pool edge character, for subsequent welding molten pool defect recognition provides favorable conditions.

Download Full-text

Text Sentiment Classification Based on Feature Fusion

Revue d intelligence artificielle ◽

10.18280/ria.340418 ◽

2020 ◽

Vol 34 (4) ◽

pp. 515-520

Author(s):

Chen Zhang ◽

Qingxu Li ◽

Xue Cheng

Keyword(s):

Hybrid Model ◽

Short Term Memory ◽

Feature Fusion ◽

Contextual Information ◽

Sentiment Classification ◽

Global Features ◽

Softmax Classifier ◽

Parallel Hybrid ◽

Local And Global Features ◽

Lstm Network

The convolutional neural network (CNN) and long short-term memory (LSTM) network are adept at extracting local and global features, respectively. Both can achieve excellent classification effects. However, the CNN performs poorly in extracting the global contextual information of the text, while LSTM often overlooks the features hidden between words. For text sentiment classification, this paper combines the CNN with bidirectional LSTM (BiLSTM) into a parallel hybrid model called CNN_BiLSTM. Firstly, the CNN was adopted to extract the local features of the text quickly. Next, the BiLSTM was employed to obtain the global text features containing contextual semantics. After that, the features extracted by the two neural networks (NNs) were fused, and processed by Softmax classifier for text sentiment classification. To verify its performance, the CNN_BiLSTM was compared with single NNs like CNN and LSTM, as well as other deep learning (DL) NNs through experiments. The experimental results show that the proposed parallel hybrid model outperformed the contrastive methods in F1-score and accuracy. Therefore, our model can solve text sentiment classification tasks effectively, and boast better practical value than other NNs.

Download Full-text

Periodic Surface Defect Detection in Steel Plates Based on Deep Learning

Applied Sciences ◽

10.3390/app9153127 ◽

2019 ◽

Vol 9 (15) ◽

pp. 3127 ◽

Cited By ~ 6

Author(s):

Yang Liu ◽

Ke Xu ◽

Jinwu Xu

Keyword(s):

Defect Detection ◽

Detection Rate ◽

Short Term Memory ◽

Steel Plates ◽

Improved Method ◽

Low Contrast ◽

Defect Image ◽

Surface Defect Detection ◽

Lstm Network ◽

Defect Recognition

It is difficult to detect roll marks on hot-rolled steel plates as they have a low contrast in the images. A periodical defect detection method based on a convolutional neural network (CNN) and long short-term memory (LSTM) is proposed to detect periodic defects, such as roll marks, according to the strong time-sequenced characteristics of such defects. Firstly, the features of the defect image are extracted through a CNN network, and then the extracted feature vectors are inputted into an LSTM network for defect recognition. The experiment shows that the detection rate of this method is 81.9%, which is 10.2% higher than a CNN method. In order to make more accurate use of the previous information, the method is improved with the attention mechanism. The improved method specifies the importance of inputted information at each previous moment, and gives the quantitative weight according to the importance. The experiment shows that the detection rate of the improved method is increased to 86.2%.

Download Full-text

Power System Transient Stability Assessment Based on Snapshot Ensemble LSTM Network

Sustainability ◽

10.3390/su13126953 ◽

2021 ◽

Vol 13 (12) ◽

pp. 6953

Author(s):

Yixing Du ◽

Zhijian Hu

Keyword(s):

Power Systems ◽

Power System ◽

Transient Stability ◽

Short Term Memory ◽

Risk Function ◽

Stability Margin ◽

Risk Level ◽

Stability Assessment ◽

Lstm Network ◽

Hierarchical Assessment

Data-driven methods using synchrophasor measurements have a broad application prospect in Transient Stability Assessment (TSA). Most previous studies only focused on predicting whether the power system is stable or not after disturbance, which lacked a quantitative analysis of the risk of transient stability. Therefore, this paper proposes a two-stage power system TSA method based on snapshot ensemble long short-term memory (LSTM) network. This method can efficiently build an ensemble model through a single training process, and employ the disturbed trajectory measurements as the inputs, which can realize rapid end-to-end TSA. In the first stage, dynamic hierarchical assessment is carried out through the classifier, so as to screen out credible samples step by step. In the second stage, the regressor is used to predict the transient stability margin of the credible stable samples and the undetermined samples, and combined with the built risk function to realize the risk quantification of transient angle stability. Furthermore, by modifying the loss function of the model, it effectively overcomes sample imbalance and overlapping. The simulation results show that the proposed method can not only accurately predict binary information representing transient stability status of samples, but also reasonably reflect the transient safety risk level of power systems, providing reliable reference for the subsequent control.

Download Full-text

Driver Drowsiness Estimation Based on Factorized Bilinear Feature Fusion and a Long-Short-Term Recurrent Convolutional Network

Information ◽

10.3390/info12010003 ◽

2020 ◽

Vol 12 (1) ◽

pp. 3

Author(s):

Shuang Chen ◽

Zengcai Wang ◽

Wenxin Chen

Keyword(s):

Short Term Memory ◽

Feature Fusion ◽

Detection Methods ◽

Video Frame ◽

Estimation Model ◽

Short Term ◽

Convolutional Network ◽

Drowsiness Detection ◽

Driver Drowsiness ◽

Time Information

The effective detection of driver drowsiness is an important measure to prevent traffic accidents. Most existing drowsiness detection methods only use a single facial feature to identify fatigue status, ignoring the complex correlation between fatigue features and the time information of fatigue features, and this reduces the recognition accuracy. To solve these problems, we propose a driver sleepiness estimation model based on factorized bilinear feature fusion and a long- short-term recurrent convolutional network to detect driver drowsiness efficiently and accurately. The proposed framework includes three models: fatigue feature extraction, fatigue feature fusion, and driver drowsiness detection. First, we used a convolutional neural network (CNN) to effectively extract the deep representation of eye and mouth-related fatigue features from the face area detected in each video frame. Then, based on the factorized bilinear feature fusion model, we performed a nonlinear fusion of the deep feature representations of the eyes and mouth. Finally, we input a series of fused frame-level features into a long-short-term memory (LSTM) unit to obtain the time information of the features and used the softmax classifier to detect sleepiness. The proposed framework was evaluated with the National Tsing Hua University drowsy driver detection (NTHU-DDD) video dataset. The experimental results showed that this method had better stability and robustness compared with other methods.

Download Full-text

A Combined Method for MEMS Gyroscope Error Compensation Using a Long Short-Term Memory Network and Kalman Filter in Random Vibration Environments

Sensors ◽

10.3390/s21041181 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1181

Author(s):

Chenhao Zhu ◽

Sheng Cai ◽

Yifan Yang ◽

Wei Xu ◽

Honghai Shen ◽

...

Keyword(s):

Kalman Filter ◽

Standard Deviation ◽

Error Compensation ◽

Random Vibration ◽

Short Term Memory ◽

Combined Method ◽

Short Term ◽

Mems Gyroscope ◽

Long Short Term Memory ◽

Lstm Network

In applications such as carrier attitude control and mobile device navigation, a micro-electro-mechanical-system (MEMS) gyroscope will inevitably be affected by random vibration, which significantly affects the performance of the MEMS gyroscope. In order to solve the degradation of MEMS gyroscope performance in random vibration environments, in this paper, a combined method of a long short-term memory (LSTM) network and Kalman filter (KF) is proposed for error compensation, where Kalman filter parameters are iteratively optimized using the Kalman smoother and expectation-maximization (EM) algorithm. In order to verify the effectiveness of the proposed method, we performed a linear random vibration test to acquire MEMS gyroscope data. Subsequently, an analysis of the effects of input data step size and network topology on gyroscope error compensation performance is presented. Furthermore, the autoregressive moving average-Kalman filter (ARMA-KF) model, which is commonly used in gyroscope error compensation, was also combined with the LSTM network as a comparison method. The results show that, for the x-axis data, the proposed combined method reduces the standard deviation (STD) by 51.58% and 31.92% compared to the bidirectional LSTM (BiLSTM) network, and EM-KF method, respectively. For the z-axis data, the proposed combined method reduces the standard deviation by 29.19% and 12.75% compared to the BiLSTM network and EM-KF method, respectively. Furthermore, for x-axis data and z-axis data, the proposed combined method reduces the standard deviation by 46.54% and 22.30% compared to the BiLSTM-ARMA-KF method, respectively, and the output is smoother, proving the effectiveness of the proposed method.

Download Full-text

Real-Time Detection of Dictionary DGA Network Traffic Using Deep Learning

SN Computer Science ◽

10.1007/s42979-021-00507-w ◽

2021 ◽

Vol 2 (2) ◽

Author(s):

Kate Highnam ◽

Domenic Puzio ◽

Song Luo ◽

Nicholas R. Jennings

Keyword(s):

Neural Network ◽

Deep Learning ◽

Real Time ◽

Network Traffic ◽

Short Term Memory ◽

Domain Names ◽

Control Networks ◽

Detection Techniques ◽

Lstm Network ◽

And Control

AbstractBotnets and malware continue to avoid detection by static rule engines when using domain generation algorithms (DGAs) for callouts to unique, dynamically generated web addresses. Common DGA detection techniques fail to reliably detect DGA variants that combine random dictionary words to create domain names that closely mirror legitimate domains. To combat this, we created a novel hybrid neural network, Bilbo the “bagging” model, that analyses domains and scores the likelihood they are generated by such algorithms and therefore are potentially malicious. Bilbo is the first parallel usage of a convolutional neural network (CNN) and a long short-term memory (LSTM) network for DGA detection. Our unique architecture is found to be the most consistent in performance in terms of AUC, $$F_1$$ F 1 score, and accuracy when generalising across different dictionary DGA classification tasks compared to current state-of-the-art deep learning architectures. We validate using reverse-engineered dictionary DGA domains and detail our real-time implementation strategy for scoring real-world network logs within a large enterprise. In 4 h of actual network traffic, the model discovered at least five potential command-and-control networks that commercial vendor tools did not flag.

Download Full-text

Extraction of local and global features by a convolutional neural network–long short-term memory network for diagnosing bearing faults

Proceedings of the Institution of Mechanical Engineers Part C Journal of Mechanical Engineering Science ◽

10.1177/09544062211016505 ◽

2021 ◽

pp. 095440622110165

Author(s):

Zhang Chao ◽

Wang Wei-zhi ◽

Zhang Chen ◽

Fan Bin ◽

Wang Jian-guo ◽

...

Keyword(s):

Neural Network ◽

Fault Diagnosis ◽

Condition Monitoring ◽

Short Term Memory ◽

Vibration Signal ◽

Short Term ◽

Global Features ◽

Term Memory ◽

Long Short Term Memory ◽

Lstm Network

Accurate and reliable fault diagnosis is one of the key and difficult issues in mechanical condition monitoring. In recent years, Convolutional Neural Network (CNN) has been widely used in mechanical condition monitoring, which is also a great breakthrough in the field of bearing fault diagnosis. However, CNN can only extract local features of signals. The model accuracy and generalization of the original vibration signals are very low in the process of vibration signal processing only by CNN. Based on the above problems, this paper improves the traditional convolution layer of CNN, and builds the learning module (local feature learning block, LFLB) of the local characteristics. At the same time, the Long Short-Term Memory (LSTM) is introduced into the network, which is used to extract the global features. This paper proposes the new neural network—improved CNN-LSTM network. The extracted deep feature is used for fault classification. The improved CNN-LSTM network is applied to the processing of the vibration signal of the faulty bearing collected by the bearing failure laboratory of Inner Mongolia University of science and technology. The results show that the accuracy of the improved CNN-LSTM network on the same batch test set is 98.75%, which is about 24% higher than that of the traditional CNN. The proposed network is applied to the bearing data collection of Western Reserve University under the condition that the network parameters remain unchanged. The experiment shows that the improved CNN-LSTM network has better generalization than the traditional CNN.

Download Full-text