scholarly journals Attention Mechanisms and Their Applications to Complex Systems

Entropy ◽  
2021 ◽  
Vol 23 (3) ◽  
pp. 283
Author(s):  
Adrián Hernández ◽  
José M. Amigó
Keyword(s):  
Machine Learning ◽  
Complex Systems ◽  
Recurrent Neural Networks ◽  
Graphics Processing Units ◽  
Learning Models ◽  
Short Term ◽  
Promising Solution ◽  
Key Aspects ◽  
Graphics Processing ◽  
Remarkable Advance

Deep learning models and graphics processing units have completely transformed the field of machine learning. Recurrent neural networks and long short-term memories have been successfully used to model and predict complex systems. However, these classic models do not perform sequential reasoning, a process that guides a task based on perception and memory. In recent years, attention mechanisms have emerged as a promising solution to these problems. In this review, we describe the key aspects of attention mechanisms and some relevant attention techniques and point out why they are a remarkable advance in machine learning. Then, we illustrate some important applications of these techniques in the modeling of complex systems.

scholarly journals Analysis and Prediction of Unplanned Intensive Care Unit Readmission using Recurrent Neural Networks with Long Short-Term Memory

2018 ◽  
Author(s):  
Yu-Wei Lin ◽  
Yuqian Zhou ◽  
Faraz Faghri ◽  
Michael J. Shaw ◽  
Roy H. Campbell
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Unplanned Readmission ◽  
Learning Models ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory ◽  
Machine Learning Models

AbstractBackgroundUnplanned readmission of a hospitalized patient is an extremely undesirable outcome as the patient may have been exposed to additional risks. The rates of unplanned readmission are, therefore, regarded as an important performance indicator for the medical quality of a hospital and healthcare system. Identifying high-risk patients likely to suffer from readmission before release benefits both the patients and the medical providers. The emergence of machine learning to detect hidden patterns in complex, multi-dimensional datasets provides unparalleled opportunities to develop efficient discharge decision-making support system for physicians.Methods and FindingsWe used supervised machine learning approaches for ICU readmission prediction. We used machine learning methods on comprehensive, longitudinal clinical data from the MIMIC-III to predict the ICU readmission of patients within 30 days of their discharge. We have utilized recent machine learning techniques such as Recurrent Neural Networks (RNN) with Long Short-Term Memory (LSTM), by this we have been able incorporate the multivariate features of EHRs and capture sudden fluctuations in chart event features (e.g. glucose and heart rate) that are significant in time series with temporal dependencies, which cannot be properly captured by traditional static models, but can be captured by our proposed deep neural network based model. We incorporate multiple types of features including chart events, demographic, and ICD9 embeddings. Our machine learning models identifies ICU readmissions at a higher sensitivity rate (0.742) and an improved Area Under the Curve (0.791) compared with traditional methods. We also illustrate the importance of each portion of the features and different combinations of the models to verify the effectiveness of the proposed model.ConclusionOur manuscript highlights the ability of machine learning models to improve our ICU decision making accuracy, and is a real-world example of precision medicine in hospitals. These data-driven results enable clinicians to make assisted decisions within their patient cohorts. This knowledge could have immediate implications for hospitals by improving the detection of possible readmission. We anticipate that machine learning models will improve patient counseling, hospital administration, allocation of healthcare resources and ultimately individualized clinical care.

scholarly journals Short-Term Prediction of COVID-19 Cases Using Machine Learning Models

2021 ◽  
Vol 11 (9) ◽  
pp. 4266
Author(s):  
Md. Shahriare Satu ◽  
Koushik Chandra Howlader ◽  
Mufti Mahmud ◽  
M. Shamim Kaiser ◽  
Sheikh Mohammad Shariful Islam ◽  
...  
Keyword(s):  
Machine Learning ◽  
Web Application ◽  
Learning Models ◽  
Short Term ◽  
Infected People ◽  
Infected Case ◽  
First Case ◽  
Novel Coronavirus ◽  
Short Term Prediction ◽  
Machine Learning Models

The first case in Bangladesh of the novel coronavirus disease (COVID-19) was reported on 8 March 2020, with the number of confirmed cases rapidly rising to over 175,000 by July 2020. In the absence of effective treatment, an essential tool of health policy is the modeling and forecasting of the progress of the pandemic. We, therefore, developed a cloud-based machine learning short-term forecasting model for Bangladesh, in which several regression-based machine learning models were applied to infected case data to estimate the number of COVID-19-infected people over the following seven days. This approach can accurately forecast the number of infected cases daily by training the prior 25 days sample data recorded on our web application. The outcomes of these efforts could aid the development and assessment of prevention strategies and identify factors that most affect the spread of COVID-19 infection in Bangladesh.

scholarly journals Prediction of Head Movement in 360-Degree Videos Using Attention Model

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3678
Author(s):  
Dongwon Lee ◽  
Minji Choi ◽  
Joohyun Lee
Keyword(s):  
Machine Learning ◽  
Short Term Memory ◽  
Moving Average ◽  
The Other ◽  
Learning Models ◽  
Short Term ◽  
Term Memory ◽  
Attention Model ◽  
Long Short Term Memory ◽  
Machine Learning Models

In this paper, we propose a prediction algorithm, the combination of Long Short-Term Memory (LSTM) and attention model, based on machine learning models to predict the vision coordinates when watching 360-degree videos in a Virtual Reality (VR) or Augmented Reality (AR) system. Predicting the vision coordinates while video streaming is important when the network condition is degraded. However, the traditional prediction models such as Moving Average (MA) and Autoregression Moving Average (ARMA) are linear so they cannot consider the nonlinear relationship. Therefore, machine learning models based on deep learning are recently used for nonlinear predictions. We use the Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural network methods, originated in Recurrent Neural Networks (RNN), and predict the head position in the 360-degree videos. Therefore, we adopt the attention model to LSTM to make more accurate results. We also compare the performance of the proposed model with the other machine learning models such as Multi-Layer Perceptron (MLP) and RNN using the root mean squared error (RMSE) of predicted and real coordinates. We demonstrate that our model can predict the vision coordinates more accurately than the other models in various videos.

scholarly journals Ambient Monitoring of Gait and Machine Learning Models for Dynamic and Short-Term Falls Risk Assessment in People With Dementia

2021 ◽  
Author(s):  
Navid Korhani ◽  
Babak Taati ◽  
Andrea Iaboni ◽  
Andrea Sabo ◽  
Sina Mehdizadeh ◽  
...  
Keyword(s):  
Machine Learning ◽  
Risk Assessment ◽  
Fall Risk ◽  
Learning Models ◽  
Short Term ◽  
Ambient Monitoring ◽  
Clinical Assessments ◽  
People With Dementia ◽  
Time Of Admission ◽  
Machine Learning Models

Data consists of baseline clinical assessments of gait, mobility, and fall risk at the time of admission of 54 adults with dementia. Furthermore, it includes the participants' daily medication intake in three medication categories, and frequent assessments of gait performed via a computer vision-based ambient monitoring system.

scholarly journals "Smart Tanks": An Intelligent High-Throughput Intervention Testing Platform in Daphnia

2021 ◽  
Author(s):  
Yongmin Cho ◽  
Rachael A Jonas-Closs ◽  
Lev Y Yampolsky ◽  
Marc W Kirschner ◽  
Leonid Peshkin
Keyword(s):  
Machine Learning ◽  
Chronological Age ◽  
Complex Behavior ◽  
Learning Models ◽  
Short Term ◽  
Health Span ◽  
Response Profile ◽  
Automated Phenotyping ◽  
Machine Learning Models

We present a novel platform for testing the effect of interventions on life- and health-span of a short-lived semi transparent freshwater organism, sensitive to drugs with complex behavior and physiology - the planktonic crustacean Daphnia magna. Within this platform, dozens of complex behavioural features of both routine motion and response to stimuli are continuously accurately quantified for large homogeneous cohorts via an automated phenotyping pipeline. We build predictive machine learning models calibrated using chronological age and extrapolate onto phenotypic age. We further apply the model to estimate the phenotypic age under pharmacological perturbation. Our platform provides a scalable framework for drug screening and characterization in both life-long and instant assays as illustrated using long term dose response profile of metformin and short term assay of such well-studied substances as caffeine and alcohol.

Sign in / Sign up

Export Citation Format

Share Document