scholarly journals On Training Neural Network Decoders of Rate Compatible Polar Codes via Transfer Learning

Entropy ◽  
2020 ◽  
Vol 22 (5) ◽  
pp. 496
Author(s):  
Hyunjae Lee ◽  
Eun Young Seo ◽  
Hyosang Ju ◽  
Sang-Hyo Kim
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Model Complexity ◽  
Polar Codes ◽  
Rate Code ◽  
Initial State ◽  
Inclusion Property ◽  
Polar Coding ◽  
Learning Techniques ◽  
Learning Technique

Neural network decoders (NNDs) for rate-compatible polar codes are studied in this paper. We consider a family of rate-compatible polar codes which are constructed from a single polar coding sequence as defined by 5G new radios. We propose a transfer learning technique for training multiple NNDs of the rate-compatible polar codes utilizing their inclusion property. The trained NND for a low rate code is taken as the initial state of NND training for the next smallest rate code. The proposed method provides quicker training as compared to separate learning of the NNDs according to numerical results. We additionally show that an underfitting problem of NND training due to low model complexity can be solved by transfer learning techniques.

scholarly journals Is One Teacher Model Enough to Transfer Knowledge to a Student Model?

Algorithms ◽  
2021 ◽  
Vol 14 (11) ◽  
pp. 334
Author(s):  
Nicola Landro ◽  
Ignazio Gallo ◽  
Riccardo La Grassa
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Learning Problem ◽  
Learning Techniques ◽  
Starting Point ◽  
Learning Technique ◽  
Interesting Approach ◽  
Classification Tasks ◽  
Learned Features ◽  
Fine Tune

Nowadays, the transfer learning technique can be successfully applied in the deep learning field through techniques that fine-tune the CNN’s starting point so it may learn over a huge dataset such as ImageNet and continue to learn on a fixed dataset to achieve better performance. In this paper, we designed a transfer learning methodology that combines the learned features of different teachers to a student network in an end-to-end model, improving the performance of the student network in classification tasks over different datasets. In addition to this, we tried to answer the following questions which are in any case directly related to the transfer learning problem addressed here. Is it possible to improve the performance of a small neural network by using the knowledge gained from a more powerful neural network? Can a deep neural network outperform the teacher using transfer learning? Experimental results suggest that neural networks can transfer their learning to student networks using our proposed architecture, designed to bring to light a new interesting approach for transfer learning techniques. Finally, we provide details of the code and the experimental settings.

scholarly journals Applying a Deep Q Network for OpenAIs Car Racing Game

2020 ◽  
Author(s):  
Ali Fakhry
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Transfer Learning ◽  
State Of The Art ◽  
Learning Techniques ◽  
Car Racing ◽  
Custom Made ◽  
Learning Technique ◽  
Reward Threshold

The applications of Deep Q-Networks are seen throughout the field of reinforcement learning, a large subsect of machine learning. Using a classic environment from OpenAI, CarRacing-v0, a 2D car racing environment, alongside a custom based modification of the environment, a DQN, Deep Q-Network, was created to solve both the classic and custom environments. The environments are tested using custom made CNN architectures and applying transfer learning from Resnet18. While DQNs were state of the art years ago, using it for CarRacing-v0 appears somewhat unappealing and not as effective as other reinforcement learning techniques. Overall, while the model did train and the agent learned various parts of the environment, attempting to reach the reward threshold for the environment with this reinforcement learning technique seems problematic and difficult as other techniques would be more useful.

scholarly journals A deep-learning technique for phase identification in multiphase inorganic compounds using synthetic XRD powder patterns

2020 ◽  
Vol 11 (1) ◽  
Author(s):  
Jin-Woong Lee ◽  
Woon Bae Park ◽  
Jin Hee Lee ◽  
Satendra Pal Singh ◽  
Kee-Sun Sohn
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Inorganic Compounds ◽  
Phase Identification ◽  
X Ray Diffraction ◽  
X Ray ◽  
Learning Techniques ◽  
Learning Technique ◽  
Xrd Patterns

AbstractHere we report a facile, prompt protocol based on deep-learning techniques to sort out intricate phase identification and quantification problems in complex multiphase inorganic compounds. We simulate plausible powder X-ray diffraction (XRD) patterns for 170 inorganic compounds in the Sr-Li-Al-O quaternary compositional pool, wherein promising LED phosphors have been recently discovered. Finally, 1,785,405 synthetic XRD patterns are prepared by combinatorically mixing the simulated powder XRD patterns of 170 inorganic compounds. Convolutional neural network (CNN) models are built and eventually trained using this large prepared dataset. The fully trained CNN model promptly and accurately identifies the constituent phases in complex multiphase inorganic compounds. Although the CNN is trained using the simulated XRD data, a test with real experimental XRD data returns an accuracy of nearly 100% for phase identification and 86% for three-step-phase-fraction quantification.

scholarly journals MRlogP: Transfer Learning Enables Accurate logP Prediction Using Small Experimental Training Datasets

Processes ◽  
2021 ◽  
Vol 9 (11) ◽  
pp. 2029
Author(s):  
Yan-Kai Chen ◽  
Steven Shave ◽  
Manfred Auer
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Mean Squared Error ◽  
Effective Property ◽  
Training Data ◽  
Limiting Factor ◽  
Attrition Rates ◽  
Learning Techniques ◽  
Trained Neural Network ◽  
Experimental Training

Small molecule lipophilicity is often included in generalized rules for medicinal chemistry. These rules aim to reduce time, effort, costs, and attrition rates in drug discovery, allowing the rejection or prioritization of compounds without the need for synthesis and testing. The availability of high quality, abundant training data for machine learning methods can be a major limiting factor in building effective property predictors. We utilize transfer learning techniques to get around this problem, first learning on a large amount of low accuracy predicted logP values before finally tuning our model using a small, accurate dataset of 244 druglike compounds to create MRlogP, a neural network-based predictor of logP capable of outperforming state of the art freely available logP prediction methods for druglike small molecules. MRlogP achieves an average root mean squared error of 0.988 and 0.715 against druglike molecules from Reaxys and PHYSPROP. We have made the trained neural network predictor and all associated code for descriptor generation freely available. In addition, MRlogP may be used online via a web interface.

scholarly journals Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning

2019 ◽  
Author(s):  
Justin S Smith ◽  
Benjamin T. Nebgen ◽  
Roman Zubatyuk ◽  
Nicholas Lubbers ◽  
Christian Devereux ◽  
...  
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Materials Science ◽  
Chemical Space ◽  
General Purpose ◽  
Learning Techniques ◽  
Mechanical Methods ◽  
Cluster Accuracy ◽  
Large Systems ◽  
Chemical And Biological Systems

<p>Computational modeling of chemical and biological systems at atomic resolution is a crucial tool in the chemist's toolset. The use of computer simulations requires a balance between cost and accuracy: quantum-mechanical methods provide high accuracy but are computationally expensive and scale poorly to large systems, while classical force fields are cheap and scalable, but lack transferability to new systems. Machine learning can be used to achieve the best of both approaches. Here we train a general-purpose neural network potential (ANI-1ccx) that approaches CCSD(T)/CBS accuracy on benchmarks for reaction thermochemistry, isomerization, and drug-like molecular torsions. This is achieved by training a network to DFT data then using transfer learning techniques to retrain on a dataset of gold standard QM calculations (CCSD(T)/CBS) that optimally spans chemical space. The resulting potential is broadly applicable to materials science, biology and chemistry, and billions of times faster<i></i>than CCSD(T)/CBS calculations. </p>

scholarly journals The Need for Transfer Learning in CRISPR-Cas Off-Target Scoring

2021 ◽  
Author(s):  
Pavan K Kota ◽  
Yidan Pan ◽  
Hoang-Anh Vu ◽  
Mingming Cao ◽  
Richard G Baraniuk ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Benchmark Dataset ◽  
Model Complexity ◽  
Improve Performance ◽  
Rna Sequences ◽  
Guide Rna ◽  
Learning Techniques ◽  
Scalable Design ◽  
Biological Differences

The scalable design of safe guide RNA sequences for CRISPR gene editing depends on the computational "scoring" of DNA locations that may be edited. As there is no widely accepted benchmark dataset to compare scoring models, we present a curated "TrueOT" dataset that contains thoroughly validated datapoints to best reflect the properties of in vivo editing. Many existing models are trained on data from high throughput assays. We hypothesize that such models may suboptimally transfer to the low throughput data in TrueOT due to fundamental biological differences between proxy assays and in vivo behavior. We developed new Siamese convolutional neural networks, trained them on a proxy dataset, and compared their performance against existing models on TrueOT. Our simplest model with a single convolutional and pooling layer surprisingly exhibits state-ofthe-art performance on TrueOT. Adding subsequent layers improves performance on the proxy dataset while compromising performance on TrueOT. We demonstrate that model complexity can only improve performance on TrueOT if transfer learning techniques are employed. These results suggest an urgent need for the CRISPR community to agree upon a benchmark dataset such as TrueOT and highlight that various sources of CRISPR data cannot be assumed to be equivalent. Our codebase and datasets are available on GitHub at github.com/baolab-rice/CRISPR_OT_scoring.

scholarly journals Convolutional neural network for image classification based on transfer learning technique

2021 ◽  
Author(s):  
Ghassan Mohammed Halawani
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Image Classification ◽  
Transfer Learning ◽  
Learning Networks ◽  
The Neural Network ◽  
Learning Technique ◽  
Common Architecture

The main purpose of this project is to modify a convolutional neural network for image classification, based on a deep-learning framework. A transfer learning technique is used by the MATLAB interface to Alex-Net to train and modify the parameters in the last two fully connected layers of Alex-Net with a new dataset to perform classifications of thousands of images. First, the general common architecture of most neural networks and their benefits are presented. The mathematical models and the role of each part in the neural network are explained in detail. Second, different neural networks are studied in terms of architecture, application, and the working method to highlight the strengths and weaknesses of each of neural network. The final part conducts a detailed study on one of the most powerful deep-learning networks in image classification – i.e. the convolutional neural network – and how it can be modified to suit different classification tasks by using transfer learning technique in MATLAB.

scholarly journals Fish species recognition using transfer learning techniques

2021 ◽  
Vol 7 (2) ◽  
pp. 188
Author(s):  
Jaisakthi Seetharani Murugaiyan ◽  
Mirunalini Palaniappan ◽  
Thenmozhi Durairaj ◽  
Vigneshkumar Muthukumar
Keyword(s):  
Transfer Learning ◽  
Fish Species ◽  
Species Recognition ◽  
Marine Species ◽  
Superior Performance ◽  
Marine Animals ◽  
Large Dataset ◽  
Underwater Environment ◽  
Learning Techniques ◽  
Learning Technique

Marine species recognition is the process of identifying various species that help in population estimation and identifying the endangered types for taking further remedies and actions. The superior performance of deep learning for classification is due to the property of estimating millions of parameters that have to be extracted from many annotated datasets. However, many types of fish species are becoming extinct, which may reduce the number of samples. The unavailability of a large dataset is a significant hurdle for applying a deep neural network that can be overcome using transfer learning techniques. To overcome this problem, we propose a transfer learning technique using a pre-trained model that uses underwater fish images as input and applies a transfer learning technique to detect the fish species using a pre-trained Google Inception-v3 model. We have evaluated our proposed method on the Fish4knowledge(F4K) dataset and obtained an accuracy of 95.37%. The research would be helpful to identify fish existence and quantity for marine biologists to understand the underwater environment to encourage its preservation and study the behavior and interactions of marine animals.

scholarly journals Corn Disease Classification Using Transfer Learning and Convolutional Neural Network

2021 ◽  
Vol 9 (2) ◽  
pp. 211
Author(s):  
Faisal Dharma Adhinata ◽  
Gita Fadila Fitriana ◽  
Aditya Wijayanto ◽  
Muhammad Pajar Kharisma Putra
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Disease Classification ◽  
Machine Learning Techniques ◽  
Early Detection Of Disease ◽  
Global Average ◽  
Learning Techniques ◽  
Corn Plants

Indonesia is an agricultural country with abundant agricultural products. One of the crops used as a staple food for Indonesians is corn. This corn plant must be protected from diseases so that the quality of corn harvest can be optimal. Early detection of disease in corn plants is needed so that farmers can provide treatment quickly and precisely. Previous research used machine learning techniques to solve this problem. The results of the previous research were not optimal because the amount of data used was slightly and less varied. Therefore, we propose a technique that can process lots and varied data, hoping that the resulting system is more accurate than the previous research. This research uses transfer learning techniques as feature extraction combined with Convolutional Neural Network as a classification. We analysed the combination of DenseNet201 with a Flatten or Global Average Pooling layer. The experimental results show that the accuracy produced by the combination of DenseNet201 with the Global Average Pooling layer is better than DenseNet201 with Flatten layer. The accuracy obtained is 93% which proves the proposed system is more accurate than previous studies.

scholarly journals Quantifying Seagrass Distribution in Coastal Water with Deep Learning Models

2020 ◽  
Vol 12 (10) ◽  
pp. 1581 ◽  
Author(s):  
Daniel Perez ◽  
Kazi Islam ◽  
Victoria Hill ◽  
Richard Zimmerman ◽  
Blake Schaeffer ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Transfer Learning ◽  
Satellite Images ◽  
Support Vector ◽  
Learning Approach ◽  
Learning Models ◽  
Learning Techniques ◽  
The World ◽  
New Locations

Coastal ecosystems are critically affected by seagrass, both economically and ecologically. However, reliable seagrass distribution information is lacking in nearly all parts of the world because of the excessive costs associated with its assessment. In this paper, we develop two deep learning models for automatic seagrass distribution quantification based on 8-band satellite imagery. Specifically, we implemented a deep capsule network (DCN) and a deep convolutional neural network (CNN) to assess seagrass distribution through regression. The DCN model first determines whether seagrass is presented in the image through classification. Second, if seagrass is presented in the image, it quantifies the seagrass through regression. During training, the regression and classification modules are jointly optimized to achieve end-to-end learning. The CNN model is strictly trained for regression in seagrass and non-seagrass patches. In addition, we propose a transfer learning approach to transfer knowledge in the trained deep models at one location to perform seagrass quantification at a different location. We evaluate the proposed methods in three WorldView-2 satellite images taken from the coastal area in Florida. Experimental results show that the proposed deep DCN and CNN models performed similarly and achieved much better results than a linear regression model and a support vector machine. We also demonstrate that using transfer learning techniques for the quantification of seagrass significantly improved the results as compared to directly applying the deep models to new locations.

Sign in / Sign up

Export Citation Format

Share Document