scholarly journals JAMPI: Efficient Matrix Multiplication in Spark Using Barrier Execution Mode

2020 ◽  
Vol 4 (4) ◽  
pp. 32
Author(s):  
Tamas Foldi ◽  
Chris von Csefalvay ◽  
Nicolas A. Perez
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Message Passing ◽  
Matrix Multiplication ◽  
Map Reduce ◽  
Distributed Training ◽  
Learning Tasks ◽  
Memory Footprint ◽  
Asynchronous Network ◽  
Execution Mode

The new barrier mode in Apache Spark allows for embedding distributed deep learning training as a Spark stage to simplify the distributed training workflow. In Spark, a task in a stage does not depend on any other tasks in the same stage, and hence it can be scheduled independently. However, several algorithms require more sophisticated inter-task communications, similar to the MPI paradigm. By combining distributed message passing (using asynchronous network IO), OpenJDK’s new auto-vectorization and Spark’s barrier execution mode, we can add non-map/reduce-based algorithms, such as Cannon’s distributed matrix multiplication to Spark. We document an efficient distributed matrix multiplication using Cannon’s algorithm, which significantly improves on the performance of the existing MLlib implementation. Used within a barrier task, the algorithm described herein results in an up to 24% performance increase on a 10,000 × 10,000 square matrix with a significantly lower memory footprint. Applications of efficient matrix multiplication include, among others, accelerating the training and implementation of deep convolutional neural network-based workloads, and thus such efficient algorithms can play a ground-breaking role in the faster and more efficient execution of even the most complicated machine learning tasks.

scholarly journals JAMPI: Efficient Matrix Multiplication in Spark Using Barrier Execution Mode

2020 ◽  
Author(s):  
Tamas Foldi ◽  
Chris von Csefalvay ◽  
Nicolas A. Perez
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Message Passing ◽  
Matrix Multiplication ◽  
Map Reduce ◽  
Distributed Training ◽  
Learning Tasks ◽  
Memory Footprint ◽  
Asynchronous Network ◽  
Execution Mode

The new barrier mode in Apache Spark allows embedding distributed deep learning training as a Spark stage to simplify the distributed training workflow. In Spark, a task in a stage doesn’t depend on any other tasks in the same stage, and hence it can be scheduled independently. However, several algorithms require more sophisticated inter-task communications, similar to the MPI paradigm. By combining distributed message passing (using asynchronous network IO), OpenJDK’s new auto-vectorization and Spark’s barrier execution mode, we can add non-map/reduce based algorithms, such as Cannon’s distributed matrix multiplication to Spark. We document an efficient distributed matrix multiplication using Cannon’s algorithm, which improves significantly on the performance of the existing MLlib implementation. Used within a barrier task, the algorithm described herein results in an up to 24% performance increase on a 10,000x10,000 square matrix with a significantly lower memory footprint. Applications of efficient matrix multiplication include, among others, accelerating the training and implementation of deep convolutional neural network based workloads, and thus such efficient algorithms can play a ground-breaking role in faster, more efficient execution of even the most complicated machine learning tasks

scholarly journals TPOT-NN: augmenting tree-based automated machine learning with neural network estimators

2021 ◽  
Author(s):  
Joseph D. Romano ◽  
Trang T. Le ◽  
Weixuan Fu ◽  
Jason H. Moore
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Binary Classification ◽  
Inductive Learning ◽  
Future Directions ◽  
High Performing ◽  
Learning Tasks ◽  
Benchmark Datasets ◽  
Automated Machine Learning ◽  
Standard Tree

AbstractAutomated machine learning (AutoML) and artificial neural networks (ANNs) have revolutionized the field of artificial intelligence by yielding incredibly high-performing models to solve a myriad of inductive learning tasks. In spite of their successes, little guidance exists on when to use one versus the other. Furthermore, relatively few tools exist that allow the integration of both AutoML and ANNs in the same analysis to yield results combining both of their strengths. Here, we present TPOT-NN—a new extension to the tree-based AutoML software TPOT—and use it to explore the behavior of automated machine learning augmented with neural network estimators (AutoML+NN), particularly when compared to non-NN AutoML in the context of simple binary classification on a number of public benchmark datasets. Our observations suggest that TPOT-NN is an effective tool that achieves greater classification accuracy than standard tree-based AutoML on some datasets, with no loss in accuracy on others. We also provide preliminary guidelines for performing AutoML+NN analyses, and recommend possible future directions for AutoML+NN methods research, especially in the context of TPOT.

scholarly journals Encoding Health Records into Pathway Representations for Deep Learning

2021 ◽  
Author(s):  
Marco Luca Sbodio ◽  
Natasha Mulligan ◽  
Stefanie Speichert ◽  
Vanessa Lopez ◽  
Joao Bettencourt-Silva
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Source Code ◽  
Training Dataset ◽  
Health Records ◽  
Learning Tasks ◽  
Patient Pathways ◽  
Computational Resources ◽  
The Impact

There is a growing trend in building deep learning patient representations from health records to obtain a comprehensive view of a patient’s data for machine learning tasks. This paper proposes a reproducible approach to generate patient pathways from health records and to transform them into a machine-processable image-like structure useful for deep learning tasks. Based on this approach, we generated over a million pathways from FAIR synthetic health records and used them to train a convolutional neural network. Our initial experiments show the accuracy of the CNN on a prediction task is comparable or better than other autoencoders trained on the same data, while requiring significantly less computational resources for training. We also assess the impact of the size of the training dataset on autoencoders performances. The source code for generating pathways from health records is provided as open source.

scholarly journals ToFU: Topology functional units for deep learning

2021 ◽  
Vol 0 (0) ◽  
pp. 0
Author(s):  
Christopher Oballe ◽  
David Boothe ◽  
Piotr J. Franaszczuk ◽  
Vasileios Maroulas
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Spectral Features ◽  
Learning Tasks ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Persistence Diagram ◽  
Topology Of Data ◽  
Dissimilarity Function

<p style='text-indent:20px;'>We propose ToFU, a new trainable neural network unit with a persistence diagram dissimilarity function as its activation. Since persistence diagrams are topological summaries of structures, this new activation measures and learns the topology of data to leverage it in machine learning tasks. We showcase the utility of ToFU in two experiments: one involving the classification of discrete-time autoregressive signals, and another involving a variational autoencoder. In the former, ToFU yields competitive results with networks that use spectral features while outperforming CNN architectures. In the latter, ToFU produces topologically-interpretable latent space representations of inputs without sacrificing reconstruction fidelity.</p>

scholarly journals Calibrated Uncertainty for Molecular Property Prediction using Ensembles of Message Passing Neural Networks

2021 ◽  
Author(s):  
Jonas Busk ◽  
Peter Bjørn Jørgensen ◽  
Arghya Bhowmik ◽  
Mikkel N. Schmidt ◽  
Ole Winther ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Message Passing ◽  
Epistemic Uncertainty ◽  
Computer Experiments ◽  
Predictive Distribution ◽  
Training Data ◽  
Unified Framework ◽  
Unseen Data ◽  
Uncertainty Estimates

Abstract Data-driven methods based on machine learning have the potential to accelerate computational analysis of atomic structures. In this context, reliable uncertainty estimates are important for assessing confidence in predictions and enabling decision making. However, machine learning models can produce badly calibrated uncertainty estimates and it is therefore crucial to detect and handle uncertainty carefully. In this work we extend a message passing neural network designed specifically for predicting properties of molecules and materials with a calibrated probabilistic predictive distribution. The method presented in this paper differs from previous work by considering both aleatoric and epistemic uncertainty in a unified framework, and by recalibrating the predictive distribution on unseen data. Through computer experiments, we show that our approach results in accurate models for predicting molecular formation energies with well calibrated uncertainty in and out of the training data distribution on two public molecular benchmark datasets, QM9 and PC9. The proposed method provides a general framework for training and evaluating neural network ensemble models that are able to produce accurate predictions of properties of molecules with well calibrated uncertainty estimates.

scholarly journals An Overview of Convolutional Neural Network: Its Architecture and Applications

2018 ◽  
Author(s):  
Shadman Sakib ◽  
Nazib Ahmed ◽  
Ahmed Jawad Kabir ◽  
Hridon Ahmed
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Artificial Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Learning Strategies ◽  
Learning Tasks ◽  
Artificial Neural ◽  
Artificial Neural Network Ann ◽  
The Many

With the increase of the Artificial Neural Network (ANN), machine learning has taken a forceful twist in recent times. One of the most spectacular kinds of ANN design is the Convolutional Neural Network (CNN). The Convolutional Neural Network (CNN) is a technology that mixes artificial neural networks and up to date deep learning strategies. In deep learning, Convolutional Neural Network is at the center of spectacular advances. This artificial neural network has been applied to several image recognition tasks for decades and attracted the eye of the researchers of the many countries in recent years as the CNN has shown promising performances in several computer vision and machine learning tasks. This paper describes the underlying architecture and various applications of Convolutional Neural Network.

scholarly journals Big Data Analysis of Facebook Users Personality Reconition using Map Reduce Back Propagation Neural Networks

2021 ◽  
Vol 6 (2) ◽  
Author(s):  
Solomon Akinboro ◽  
Isaac K. Ogundoyin ◽  
Ayobami T. Olusesi
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Back Propagation ◽  
Threshold Value ◽  
Back Propagation Neural Network ◽  
Classification Model ◽  
Map Reduce ◽  
Support Vector ◽  
Computationally Efficient ◽  
Personality Recognition

Machine learning has been an effective tool to connect networks of enormous information for predicting personality. Identification of personality-related indicators encrypted in Facebook profiles and activities are of special concern in most research efforts. This research modeled user personality based on set of features extracted from the Facebook data using Map-Reduce Back Propagation Neural Network (MRBPNN). The performance of the MRBPNN classification model was evaluated in terms of five basic personality dimensions: Extraversion (EXT), Agreeableness (AGR), Conscientiousness (CON), Neuroticism (NEU), and Openness to Experience (OPN) using True positive, False Positive, accuracy, precision and F-measure as metrics at the threshold value of 0.32. The experimental results reveal that MRBPNN model has accuracy of 91.40%, 93.89%, 91.33%, 90.43% and 89.13% CON, OPN, EXT, NEU and AGR respectively for personality recognition which is more computationally efficient than Back Propagation Neural Network (BPNN) and Support Vector Machine (SVM). Therefore, personality recognition based on MRBPNN would produce a reliable prediction system for various personality traits with data having a very large instance.  Keywords— Machine learning, Facebook, MRBPNN, Personality Recognition, Neuroticism, Agreeableness.

scholarly journals Multitask Learning over Shared Subspaces

2020 ◽  
Author(s):  
Nicholas Menghi ◽  
Kemal Kacar ◽  
Will Penny
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Transfer Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Human Performance ◽  
Human Subjects ◽  
Model Learning ◽  
Learning Tasks ◽  
Parameter Sharing

AbstractThis paper uses constructs from the field of multitask machine learning to define pairs of learning tasks that either shared or did not share a common subspace. Human subjects then learnt these tasks using a feedback-based approach. We found, as hypothesised, that subject performance was significantly higher on the second task if it shared the same subspace as the first, an advantage that played out most strongly at the beginning of the second task. Additionally, accuracy was positively correlated over subjects learning same-subspace tasks but was not correlated for those learning different-subspace tasks. These results, and other aspects of learning dynamics, were compared to the behaviour of a Neural Network model trained using sequential Bayesian inference. Human performance was found to be consistent with a Soft Parameter Sharing variant of this model that constrained representations to be similar among tasks but only when this aided learning. We propose that the concept of shared subspaces provides a useful framework for the experimental study of human multitask and transfer learning.Author summaryHow does knowledge gained from previous experience affect learning of new tasks ? This question of “Transfer Learning” has been addressed by teachers, psychologists, and more recently by researchers in the fields of neural networks and machine learning. Leveraging constructs from machine learning, we designed pairs of learning tasks that either shared or did not share a common subspace. We compared the dynamics of transfer learning in humans with those of a multitask neural network model, finding that human performance was consistent with a soft parameter sharing variant of the model. Learning was boosted in the early stages of the second task if the same subspace was shared between tasks. Additionally, accuracy between tasks was positively correlated but only when they shared the same subspace. Our results highlight the roles of subspaces, showing how they could act as a learning boost if shared, and be detrimental if not.

scholarly journals An Overview of Convolutional Neural Network: Its Architecture and Applications

2019 ◽  
Author(s):  
Shadman Sakib ◽  
Nazib Ahmed ◽  
Ahmed Jawad Kabir ◽  
Hridon Ahmed
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Artificial Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Learning Strategies ◽  
Learning Tasks ◽  
Artificial Neural ◽  
Artificial Neural Network Ann ◽  
The Many

With the increase of the Artificial Neural Network (ANN), machine learning has taken a forceful twist in recent times. One of the most spectacular kinds of ANN design is the Convolutional Neural Network (CNN). The Convolutional Neural Network (CNN) is a technology that mixes artificial neural networks and up to date deep learning strategies. In deep learning, Convolutional Neural Network is at the center of spectacular advances. This artificial neural network has been applied to several image recognition tasks for decades and attracted the eye of the researchers of the many countries in recent years as the CNN has shown promising performances in several computer vision and machine learning tasks. This paper describes the underlying architecture and various applications of Convolutional Neural Network.

Adsorption Isotherm Predictions for Multiple Molecules in MOFs Using the Same Deep Learning Model

2019 ◽  
Author(s):  
Ryther Anderson ◽  
Achay Biong ◽  
Diego Gómez-Gualdrón
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Molecular Simulation ◽  
Large Scale ◽  
Learning Model ◽  
Operating Conditions ◽  
Small Subset ◽  
Screening Methods ◽  
Large Set ◽  
Metal Organic

<div>Tailoring the structure and chemistry of metal-organic frameworks (MOFs) enables the manipulation of their adsorption properties to suit specific energy and environmental applications. As there are millions of possible MOFs (with tens of thousands already synthesized), molecular simulation, such as grand canonical Monte Carlo (GCMC), has frequently been used to rapidly evaluate the adsorption performance of a large set of MOFs. This allows subsequent experiments to focus only on a small subset of the most promising MOFs. In many instances, however, even molecular simulation becomes prohibitively time consuming, underscoring the need for alternative screening methods, such as machine learning, to precede molecular simulation efforts. In this study, as a proof of concept, we trained a neural network as the first example of a machine learning model capable of predicting full adsorption isotherms of different molecules not included in the training of the model. To achieve this, we trained our neural network only on alchemical species, represented only by their geometry and force field parameters, and used this neural network to predict the loadings of real adsorbates. We focused on predicting room temperature adsorption of small (one- and two-atom) molecules relevant to chemical separations. Namely, argon, krypton, xenon, methane, ethane, and nitrogen. However, we also observed surprisingly promising predictions for more complex molecules, whose properties are outside the range spanned by the alchemical adsorbates. Prediction accuracies suitable for large-scale screening were achieved using simple MOF (e.g. geometric properties and chemical moieties), and adsorbate (e.g. forcefield parameters and geometry) descriptors. Our results illustrate a new philosophy of training that opens the path towards development of machine learning models that can predict the adsorption loading of any new adsorbate at any new operating conditions in any new MOF.</div>

Sign in / Sign up

Export Citation Format

Share Document