scholarly journals A Deep Learning Model for Predictive Maintenance in Cyber-Physical Production Systems Using LSTM Autoencoders

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 972
Author(s):  
Xanthi Bampoula ◽  
Georgios Siaterlis ◽  
Nikolaos Nikolakis ◽  
Kosmas Alexopoulos
Keyword(s):  
Deep Learning ◽  
Real World ◽  
Short Term Memory ◽  
Production Systems ◽  
Learning Algorithm ◽  
Remaining Useful Life ◽  
Machine Learning Algorithms ◽  
Sensor Data ◽  
Maintenance Activities ◽  
Cyber Physical Production Systems

Condition monitoring of industrial equipment, combined with machine learning algorithms, may significantly improve maintenance activities on modern cyber-physical production systems. However, data of proper quality and of adequate quantity, modeling both good operational conditions as well as abnormal situations throughout the operational lifecycle, are required. Nevertheless, this is difficult to acquire in a non-destructive approach. In this context, this study investigates an approach to enable a transition from preventive maintenance activities, that are scheduled at predetermined time intervals, into predictive ones. In order to enable such approaches in a cyber-physical production system, a deep learning algorithm is used, allowing for maintenance activities to be planned according to the actual operational status of the machine and not in advance. An autoencoder-based methodology is employed for classifying real-world machine and sensor data, into a set of condition-related labels. Real-world data collected from manufacturing operations are used for training and testing a prototype implementation of Long Short-Term Memory autoencoders for estimating the remaining useful life of the monitored equipment. Finally, the proposed approach is evaluated in a use case related to a steel industry production process.

scholarly journals A New Machine Learning Algorithm for Numerical Prediction of Near-Earth Environment Sensors along the Inland of East Antarctica

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 755
Author(s):  
Yuchen Wang ◽  
Yinke Dou ◽  
Wangxiao Yang ◽  
Jingxue Guo ◽  
Xiaomin Chang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Learning Algorithm ◽  
Coastal Region ◽  
Sensor Data ◽  
Learning Networks ◽  
Short Term ◽  
Term Memory ◽  
Memory Network ◽  
The Antarctic

Accurate short-term small-area meteorological forecasts are essential to ensure the safety of operations and equipment operations in the Antarctic interior. This study proposes a deep learning-based multi-input neural network model to address this problem. The newly proposed model is predicted by combining a stacked autoencoder and a long- and short-term memory network. The self-stacking autoencoder maximises the features and removes redundancy from the target weather station’s sensor data and extracts temporal features from the sensor data using a long- and short-term memory network. The proposed new model evaluates the prediction performance and generalisation capability at four observation sites at different East Antarctic latitudes (including the Antarctic maximum and the coastal region). The performance of five deep learning networks is compared through five evaluation metrics, and the optimal form of input combination is discussed. The results show that the prediction capability of the model outperforms the other models. It provides a new method for short-term meteorological prediction in a small inland Antarctic region.

scholarly journals Advances in Machine Learning Detecting Changeover Processes in Cyber Physical Production Systems

2020 ◽  
Vol 4 (4) ◽  
pp. 108
Author(s):  
Bastian Engelmann ◽  
Simon Schmitt ◽  
Eddi Miller ◽  
Volker Bräutigam ◽  
Jan Schmitt
Keyword(s):  
Machine Learning ◽  
Production Control ◽  
Production Systems ◽  
Heterogeneous Data ◽  
Machine Learning Algorithms ◽  
Sensor Data ◽  
Medium Size ◽  
Data Set ◽  
Set Up ◽  
Cyber Physical Production Systems

The performance indicator, Overall Equipment Effectiveness (OEE), is one of the most important ones for production control, as it merges information of equipment usage, process yield, and product quality. The determination of the OEE is oftentimes not transparent in companies, due to the heterogeneous data sources and manual interference. Furthermore, there is a difference in present guidelines to calculate the OEE. Due to a big amount of sensor data in Cyber Physical Production Systems, Machine Learning methods can be used in order to detect several elements of the OEE by a trained model. Changeover time is one crucial aspect influencing the OEE, as it adds no value to the product. Furthermore, changeover processes are fulfilled manually and vary from worker to worker. They always have their own procedure to conduct a changeover of a machine for a new product or production lot. Hence, the changeover time as well as the process itself vary. Thus, a new Machine Learning based concept for identification and characterization of machine set-up actions is presented. Here, the issue to be dealt with is the necessity of human and machine interaction to fulfill the entire machine set-up process. Because of this, the paper shows the use case in a real production scenario of a small to medium size company (SME), the derived data set, promising Machine Learning algorithms, as well as the results of the implemented Machine Learning model to classify machine set-up actions.

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

2019 ◽  
Vol 9 (1) ◽  
pp. 16-31
Author(s):  
Kyungkoo Jun
Keyword(s):  
Fourier Transform ◽  
Deep Learning ◽  
Short Term Memory ◽  
Window Size ◽  
Sensor Data ◽  
Data Sets ◽  
Data Set ◽  
Proposed Model ◽  
Testing Data ◽  
Labeling Scheme

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data. Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN). Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme. Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

scholarly journals Predicting Future Occurrence of Acute Hypotensive Episodes Using Noninvasive and Invasive Features

2021 ◽  
Vol 186 (Supplement_1) ◽  
pp. 445-451
Author(s):  
Yifei Sun ◽  
Navid Rashedi ◽  
Vikrant Vaze ◽  
Parikshit Shah ◽  
Ryan Halter ◽  
...  
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Real World ◽  
Short Term Memory ◽  
Model Performance ◽  
Learning Technologies ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
K Nearest Neighbor ◽  
Continuous Map

ABSTRACT Introduction Early prediction of the acute hypotensive episode (AHE) in critically ill patients has the potential to improve outcomes. In this study, we apply different machine learning algorithms to the MIMIC III Physionet dataset, containing more than 60,000 real-world intensive care unit records, to test commonly used machine learning technologies and compare their performances. Materials and Methods Five classification methods including K-nearest neighbor, logistic regression, support vector machine, random forest, and a deep learning method called long short-term memory are applied to predict an AHE 30 minutes in advance. An analysis comparing model performance when including versus excluding invasive features was conducted. To further study the pattern of the underlying mean arterial pressure (MAP), we apply a regression method to predict the continuous MAP values using linear regression over the next 60 minutes. Results Support vector machine yields the best performance in terms of recall (84%). Including the invasive features in the classification improves the performance significantly with both recall and precision increasing by more than 20 percentage points. We were able to predict the MAP with a root mean square error (a frequently used measure of the differences between the predicted values and the observed values) of 10 mmHg 60 minutes in the future. After converting continuous MAP predictions into AHE binary predictions, we achieve a 91% recall and 68% precision. In addition to predicting AHE, the MAP predictions provide clinically useful information regarding the timing and severity of the AHE occurrence. Conclusion We were able to predict AHE with precision and recall above 80% 30 minutes in advance with the large real-world dataset. The prediction of regression model can provide a more fine-grained, interpretable signal to practitioners. Model performance is improved by the inclusion of invasive features in predicting AHE, when compared to predicting the AHE based on only the available, restricted set of noninvasive technologies. This demonstrates the importance of exploring more noninvasive technologies for AHE prediction.

scholarly journals Deep Learning Classification of Canine Behavior Using a Single Collar-Mounted Accelerometer: Real-World Validation

Animals ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 1549
Author(s):  
Robert D. Chambers ◽  
Nathanael C. Yoder ◽  
Aletha B. Carson ◽  
Christian Junge ◽  
David E. Allen ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Real World ◽  
Learning Algorithm ◽  
Drinking Behavior ◽  
True Positive Rate ◽  
Training Dataset ◽  
Activity Levels ◽  
Accelerometer Data ◽  
Activity Monitors

Collar-mounted canine activity monitors can use accelerometer data to estimate dog activity levels, step counts, and distance traveled. With recent advances in machine learning and embedded computing, much more nuanced and accurate behavior classification has become possible, giving these affordable consumer devices the potential to improve the efficiency and effectiveness of pet healthcare. Here, we describe a novel deep learning algorithm that classifies dog behavior at sub-second resolution using commercial pet activity monitors. We built machine learning training databases from more than 5000 videos of more than 2500 dogs and ran the algorithms in production on more than 11 million days of device data. We then surveyed project participants representing 10,550 dogs, which provided 163,110 event responses to validate real-world detection of eating and drinking behavior. The resultant algorithm displayed a sensitivity and specificity for detecting drinking behavior (0.949 and 0.999, respectively) and eating behavior (0.988, 0.983). We also demonstrated detection of licking (0.772, 0.990), petting (0.305, 0.991), rubbing (0.729, 0.996), scratching (0.870, 0.997), and sniffing (0.610, 0.968). We show that the devices’ position on the collar had no measurable impact on performance. In production, users reported a true positive rate of 95.3% for eating (among 1514 users), and of 94.9% for drinking (among 1491 users). The study demonstrates the accurate detection of important health-related canine behaviors using a collar-mounted accelerometer. We trained and validated our algorithms on a large and realistic training dataset, and we assessed and confirmed accuracy in production via user validation.

A semiautomatic annotation approach for sentiment analysis

2021 ◽  
pp. 016555152110065
Author(s):  
Rahma Alahmary ◽  
Hmood Al-Dossari
Keyword(s):  
Deep Learning ◽  
Sentiment Analysis ◽  
Short Term Memory ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Short Term ◽  
Term Memory ◽  
Annotation Process ◽  
Learning Classifiers ◽  
Long Short Term Memory

Sentiment analysis (SA) aims to extract users’ opinions automatically from their posts and comments. Almost all prior works have used machine learning algorithms. Recently, SA research has shown promising performance in using the deep learning approach. However, deep learning is greedy and requires large datasets to learn, so it takes more time for data annotation. In this research, we proposed a semiautomatic approach using Naïve Bayes (NB) to annotate a new dataset in order to reduce the human effort and time spent on the annotation process. We created a dataset for the purpose of training and testing the classifier by collecting Saudi dialect tweets. The dataset produced from the semiautomatic model was then used to train and test deep learning classifiers to perform Saudi dialect SA. The accuracy achieved by the NB classifier was 83%. The trained semiautomatic model was used to annotate the new dataset before it was fed into the deep learning classifiers. The three deep learning classifiers tested in this research were convolutional neural network (CNN), long short-term memory (LSTM) and bidirectional long short-term memory (Bi-LSTM). Support vector machine (SVM) was used as the baseline for comparison. Overall, the performance of the deep learning classifiers exceeded that of SVM. The results showed that CNN reported the highest performance. On one hand, the performance of Bi-LSTM was higher than that of LSTM and SVM, and, on the other hand, the performance of LSTM was higher than that of SVM. The proposed semiautomatic annotation approach is usable and promising to increase speed and save time and effort in the annotation process.

Prognostics and RUL Estimations of SAC305, SAC105 and SnAg Solders Under Temperature and Vibration Using Long Short-Term Memory (LSTM) Deep Learning

2021 ◽  
Author(s):  
Pradeep Lall ◽  
Tony Thomas ◽  
Ken Blecker
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Short Term Memory ◽  
Remaining Useful Life ◽  
Operating Conditions ◽  
Short Term ◽  
Term Memory ◽  
Rectangular Pattern ◽  
Useful Life ◽  
Long Short Term Memory

Abstract Prognostics and Remaining Useful Life (RUL) estimations of complex systems are essential to operational safety, increased efficiency, and help to schedule maintenance proactively. Modeling the remaining useful life of a system with many complexities is possible with the rapid development in the field of deep learning as a computational technique for failure prediction. Deep learning can adapt to multivariate parameters complex and nonlinear behavior, which is difficult using traditional time-series models for forecasting and prediction purposes. In this paper, a deep learning approach based on Long Short-Term Memory (LSTM) network is used to predict the remaining useful life of the PCB at different conditions of temperature and vibration. This technique can identify the different underlying patterns in the time series that can predict the RUL. This study involves feature vector identification and RUL estimations for SAC305, SAC105, and Tin Lead solder PCBs under different vibration levels and temperature conditions. The acceleration levels of vibration are fixed at 5g and 10g, while the temperature levels are 55°C and 100°C. The test board is a multilayer FR4 configuration with JEDEC standard dimensions consists of twelve packages arranged in a rectangular pattern. Strain signals are acquired from the backside of the PCB at symmetric locations to identify the failure of all the packages during vibration. The strain signals are resistance values that are acquired simultaneously during the experiment until the failure of most of the packages on the board. The feature vectors are identified from statistical analysis on the strain signals frequency and instantaneous frequency components. The principal component analysis is used as a data reduction technique to identify the different patterns produced from the four strain signals with failures of the packages during vibration. LSTM deep learning method is used to model the RUL of the packages at different individual operating conditions of vibration for all three solder materials involved in this study. A combined model for RUL prediction for a material that can take care of the changes in the operating conditions is also modeled for each material.

A Hybrid Supervised Deep Learning and Nonlinear Finite Element Framework for Efficient Fatigue Life Predictions of Rotary Shouldered Threaded Connections

2020 ◽  
Author(s):  
Fei Song ◽  
Ke Li
Keyword(s):  
Finite Element ◽  
Deep Learning ◽  
Fatigue Life ◽  
Learning Algorithm ◽  
Fatigue Tests ◽  
Full Scale ◽  
Remaining Useful Life ◽  
Nonlinear Finite Element ◽  
Threaded Connections ◽  
Life Predictions

Abstract In this paper, a hybrid computational framework that combines the state-of-the art machine learning algorithm (i.e., deep neural network) and nonlinear finite element analysis for efficient and accurate fatigue life prediction of rotary shouldered threaded connections is presented. Specifically, a large set of simulation data from nonlinear FEA, along with a small set of experimental data from full-scale fatigue tests, constitutes the dataset required for training and testing of a fast-loop predictive model that could cover most commonly used rotary shouldered connections. Feature engineering was first performed to explore the compressed feature space to be used to represent the data. An ensemble deep learning algorithm was then developed to learn the underlying pattern, and hyperparameter tuning techniques were employed to select the learning model that provides the best mapping, between the features and the fatigue strength of the connections. The resulting fatigue life predictions were found to agree favorably well with the experimental results from full-scale bending fatigue tests and field operational data. This newly developed hybrid modeling framework paves a new way to realtime predicting the remaining useful life of rotary shouldered threaded connections for prognostic health management of the drilling equipment.

scholarly journals Deep learning-based anomaly-onset aware remaining useful life estimation of bearings

2021 ◽  
Vol 7 ◽  
pp. e795
Author(s):  
Pooja Vinayak Kamat ◽  
Rekha Sugandhi ◽  
Satish Kumar
Keyword(s):  
Deep Learning ◽  
Anomaly Detection ◽  
Short Term Memory ◽  
Rotating Machinery ◽  
Remaining Useful Life ◽  
Operating Conditions ◽  
Learning Models ◽  
Vibration Data ◽  
Degradation Data ◽  
Useful Life

Remaining Useful Life (RUL) estimation of rotating machinery based on their degradation data is vital for machine supervisors. Deep learning models are effective and popular methods for forecasting when rotating machinery such as bearings may malfunction and ultimately break down. During healthy functioning of the machinery, however, RUL is ill-defined. To address this issue, this study recommends using anomaly monitoring during both RUL estimator training and operation. Essential time-domain data is extracted from the raw bearing vibration data, and deep learning models are used to detect the onset of the anomaly. This further acts as a trigger for data-driven RUL estimation. The study employs an unsupervised clustering approach for anomaly trend analysis and a semi-supervised method for anomaly detection and RUL estimation. The novel combined deep learning-based anomaly-onset aware RUL estimation framework showed enhanced results on the benchmarked PRONOSTIA bearings dataset under non-varying operating conditions. The framework consisting of Autoencoder and Long Short Term Memory variants achieved an accuracy of over 90% in anomaly detection and RUL prediction. In the future, the framework can be deployed under varying operational situations using the transfer learning approach.

scholarly journals Context-Based Resilience in Cyber-Physical Production System

2021 ◽  
Author(s):  
Ada Bagozi ◽  
Devis Bianchini ◽  
Valeria De Antonellis
Keyword(s):  
Production Systems ◽  
Complex Structure ◽  
High Volume ◽  
Sensor Data ◽  
Shop Floor ◽  
Human In The Loop ◽  
Operator Interface ◽  
Physical Elements ◽  
Cyber Physical Production Systems ◽  
Smart Machine

AbstractCyber-physical systems are hybrid networked cyber and engineered physical elements that record data (e.g. using sensors), analyse them using connected services, influence physical processes and interact with human actors using multi-channel interfaces. Examples of CPS interacting with humans in industrial production environments are the so-called cyber-physical production systems (CPPS), where operators supervise the industrial machines, according to the human-in-the-loop paradigm. In this scenario, research challenges for implementing CPPS resilience, promptly reacting to faults, concern: (i) the complex structure of CPPS, which cannot be addressed as a monolithic system, but as a dynamic ecosystem of single CPS interacting and influencing each other; (ii) the volume, velocity and variety of data (Big Data) on which resilience is based, which call for novel methods and techniques to ensure recovery procedures; (iii) the involvement of human factors in these systems. In this paper, we address the design of resilient cyber-physical production systems (R-CPPS) in digital factories by facing these challenges. Specifically, each component of the R-CPPS is modelled as a smart machine, that is, a cyber-physical system equipped with a set of recovery services, a Sensor Data API used to collect sensor data acquired from the physical side for monitoring the component behaviour, and an operator interface for displaying detected anomalous conditions and notifying necessary recovery actions to on-field operators. A context-based mediator, at shop floor level, is in charge of ensuring resilience by gathering data from the CPPS, selecting the proper recovery actions and invoking corresponding recovery services on the target CPS. Finally, data summarisation and relevance evaluation techniques are used for supporting the identification of anomalous conditions in the presence of high volume and velocity of data collected through the Sensor Data API. The approach is validated in a food industry real case study.

Sign in / Sign up

Export Citation Format

Share Document