scholarly journals Gated Graph Attention Network for Cancer Prediction

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 1938
Author(s):  
Linling Qiu ◽  
Han Li ◽  
Meihong Wang ◽  
Xiaoli Wang
Keyword(s):  
Neural Network ◽  
Prediction Accuracy ◽  
State Of The Art ◽  
Network Models ◽  
The State ◽  
Neural Network Models ◽  
Attention Network ◽  
Training Time ◽  
Cancer Prediction ◽  
Gating Mechanism

With its increasing incidence, cancer has become one of the main causes of worldwide mortality. In this work, we mainly propose a novel attention-based neural network model named Gated Graph ATtention network (GGAT) for cancer prediction, where a gating mechanism (GM) is introduced to work with the attention mechanism (AM), to break through the previous work’s limitation of 1-hop neighbourhood reasoning. In this way, our GGAT is capable of fully mining the potential correlation between related samples, helping for improving the cancer prediction accuracy. Additionally, to simplify the datasets, we propose a hybrid feature selection algorithm to strictly select gene features, which significantly reduces training time without affecting prediction accuracy. To the best of our knowledge, our proposed GGAT achieves the state-of-the-art results in cancer prediction task on LIHC, LUAD, KIRC compared to other traditional machine learning methods and neural network models, and improves the accuracy by 1% to 2% on Cora dataset, compared to the state-of-the-art graph neural network methods.

scholarly journals A protection method of trained CNN model with a secret key from unauthorized access

2021 ◽  
Vol 10 ◽  
Author(s):  
AprilPyone Maungmaung ◽  
Hitoshi Kiya
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Network Models ◽  
The State ◽  
Secret Key ◽  
Neural Network Models ◽  
Unauthorized Access ◽  
Protection Method ◽  
Novel Method

In this paper, we propose a novel method for protecting convolutional neural network models with a secret key set so that unauthorized users without the correct key set cannot access trained models. The method enables us to protect not only from copyright infringement but also the functionality of a model from unauthorized access without any noticeable overhead. We introduce three block-wise transformations with a secret key set to generate learnable transformed images: pixel shuffling, negative/positive transformation, and format-preserving Feistel-based encryption. Protected models are trained by using transformed images. The results of experiments with the CIFAR and ImageNet datasets show that the performance of a protected model was close to that of non-protected models when the key set was correct, while the accuracy severely dropped when an incorrect key set was given. The protected model was also demonstrated to be robust against various attacks. Compared with the state-of-the-art model protection with passports, the proposed method does not have any additional layers in the network, and therefore, there is no overhead during training and inference processes.

Learning Chinese Word Segmentation Based on Bidirectional GRU-CRF and CNN Network Model

2019 ◽  
Vol 15 (3) ◽  
pp. 47-62 ◽  
Author(s):  
Chenghai Yu ◽  
Shupei Wang ◽  
Jiajun Guo
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Network Models ◽  
Word Segmentation ◽  
Chinese Word ◽  
Chinese Word Segmentation ◽  
Neural Network Models ◽  
Long Distance ◽  
Training Time

Chinese word segmentation is the basis of the Chinese natural language processing (NLP). With the development of the deep learning, various neural network models are applied to the Chinese word segmentation. However, current neural network models have the characteristics of artificial feature extraction, nonstandard word-weight, inability to effectively use long-distance information and long training time of models in Chinese word segmentation. To solve a series of problems, this article presents a CNN-Bidirectional GRU-CRF neural network model (CNN Bidirectional GRU CRF Network, CBiGCN), which breaks through the limit of conventional method window, truly realizes end-to-end processing and applies to the neural network model by the five-Tag set method, bias-variable-weight greedy strategy and supplements by Goldstein-Armijo guidelines. Besides, this model, with simple structure, is easy to be operated. And it can automatically learn features, reduces large amounts of tasks on specific knowledge in the form of handcrafted features and data pre-processing, makes use of context information effectively. The authors set an experiment with two data corpuses for Chinese word segmentation to evaluate their system. The experiment verified their new model can obtain better Chinese word segmentation results and greatly reduce training time.

Traffic Sign Classification Comparison Between Various Convolution Neural Network Models

2021 ◽  
Vol 12 (07) ◽  
pp. 165-171
Author(s):  
Jonah Sokipriala ◽  
Sunny Orike
Keyword(s):  
Neural Network ◽  
Network Models ◽  
Transport Systems ◽  
Test Accuracy ◽  
Layer Model ◽  
Traffic Sign ◽  
Neural Network Models ◽  
Training Time ◽  
Fast Detection ◽  
The Arts

Fast detection and accurate classification of traffic signs is one of the major aspects of advance driver assistance system (ADAS) and intelligent transport systems (ITS), this paper presents a comparison between an 8-Layer convolutional neural network (CNN), and some state of the Arts model such as VGG16 and Resnet50, for traffic sign classification on The GTSRB. using a GPU to increase processing time, the design showed that with various augmentation applied to the CNN, our 8-layer Model was able to outperform the State of the Arts models with a higher test Accuracy, 50 times lesser training parameters, and faster training time our 8 -layer model was able to achieve 96% test accuracy.

scholarly journals An Empirical Study on Deep Neural Network Models for Chinese Dialogue Generation

Symmetry ◽  
2020 ◽  
Vol 12 (11) ◽  
pp. 1756
Author(s):  
Zhe Li ◽  
Mieradilijiang Maimaiti ◽  
Jiabao Sheng ◽  
Zunwang Ke ◽  
Wushour Silamu ◽  
...  
Keyword(s):  
Neural Network ◽  
Empirical Study ◽  
Deep Neural Network ◽  
Question Answering ◽  
State Of The Art ◽  
Network Models ◽  
Superior Performance ◽  
Research Progress ◽  
Neural Network Models ◽  
Wide Range

The task of dialogue generation has attracted increasing attention due to its diverse downstream applications, such as question-answering systems and chatbots. Recently, the deep neural network (DNN)-based dialogue generation models have achieved superior performance against conventional models utilizing statistical machine learning methods. However, despite that an enormous number of state-of-the-art DNN-based models have been proposed, there lacks detailed empirical comparative analysis for them on the open Chinese corpus. As a result, relevant researchers and engineers might find it hard to get an intuitive understanding of the current research progress. To address this challenge, we conducted an empirical study for state-of-the-art DNN-based dialogue generation models in various Chinese corpora. Specifically, extensive experiments were performed on several well-known single-turn and multi-turn dialogue corpora, including KdConv, Weibo, and Douban, to evaluate a wide range of dialogue generation models that are based on the symmetrical architecture of Seq2Seq, RNNSearch, transformer, generative adversarial nets, and reinforcement learning respectively. Moreover, we paid special attention to the prevalent pre-trained model for the quality of dialogue generation. Their performances were evaluated by four widely-used metrics in this area: BLEU, pseudo, distinct, and rouge. Finally, we report a case study to show example responses generated by these models separately.

scholarly journals Time Series Data Prediction using Elman Recurrent Neural Network on Tourist Visits in Tanah Lot Tourism Object

2019 ◽  
Vol 9 (1) ◽  
pp. 314-320 ◽  
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Recurrent Neural Network ◽  
Time Series Data ◽  
Network Models ◽  
Series Data ◽  
Neural Network Models ◽  
Training Time ◽  
Sliding Windows ◽  
Learning Time

The prediction of time series data is a forecast using the analysis of a relationship pattern between what will be predicted (prediction) and the time variable. The prediction process using the recurrent neural network (RNN) model could recognize and learn the data pattern of time series, but the presence of fluctuations in data makes the introduction of data patterns difficult to be learned. The data used for forecasting are tourist visits to Tanah Lot Bali tourist attraction for 10 years (2008-2017). The training process uses the RNN method on high fluctuating data, which requires a relatively long time in recognizing and studying the data patterns. Modification of the RNN method on learning rate and momentum by using dynamic values, can shorten learning time. The results showed the learning time using the RNN dynamic value, smaller than the variants of the RNN method such as the RNN Elman, Jordan RNN, Fully RNN, LSTM and the feedforward method (Backpropagation). The resulting error value is 0,05105 MSE. This value is smaller than the Fully RNN, Jordan RNN, LSTM and Feedforward methods. The elman method has the shortest training time among other models. The purpose of this research is to make a prediction design consisting of sliding windows techniques, training with neural network models and validation of results with k-fold cross-validation.

scholarly journals Deep Neural Network Approach for Prediction of Heating Energy Consumption in Old Houses

Energies ◽  
2020 ◽  
Vol 14 (1) ◽  
pp. 122
Author(s):  
Sungjin Lee ◽  
Soo Cho ◽  
Seo-Hoon Kim ◽  
Jonghun Kim ◽  
Suyong Chae ◽  
...  
Keyword(s):  
Neural Network ◽  
Energy Consumption ◽  
Low Income ◽  
Prediction Accuracy ◽  
Deep Neural Network ◽  
Network Models ◽  
Neural Network Models ◽  
Neural Network Approach ◽  
Input Variables ◽  
Heating Energy Consumption

Neural network models are data-driven and are effective for predicting and interpreting nonlinear or unexplainable physical phenomena. This study collected building information and heating energy consumption data from 16,158 old houses, selected key input variables that affect the heating energy consumption based on the collected datasets, and developed a deep neural network (DNN) model that showed the highest accuracy for the prediction of heating energy consumption in an old house. As a result, 11 key input variables were selected, and an optimal DNN model was developed. This optimal DNN model showed the highest prediction accuracy (R2 = 0.961) when the number of hidden layers was five and the number of neurons was 22. When the optimal DNN model was applied for the standard model of low-income detached houses, the prediction accuracy (Cv(RMSE)) of the optimal DNN model, compared to the EnergyPlus calculation result, was 8.74%, which satisfied the ASHRAE standard sufficiently.

scholarly journals Model Jaringan Syaraf Tiruan dalam Peramalan Kasus Positif Covid-19 di Indonesia

2020 ◽  
Vol 3 (2) ◽  
pp. 53
Author(s):  
Wirawan Setialaksana ◽  
Dwi Reski Anandari Sulaiman ◽  
Shabrina Syntha Dewi ◽  
Chairunnisa Ar Lamasitudju ◽  
Nini Rahayu Ashadi ◽  
...  
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Prediction Accuracy ◽  
Network Models ◽  
Model Accuracy ◽  
Neural Network Models ◽  
Accuracy Measurement ◽  
Artificial Neural ◽  
Artificial Neural Network Models

Mitigation steps to control Covid-19 outbreak in Indonesia need to take. One of those step is forecasting the spread of the disease. This study compare two artificial neural network models in catching the pattern of Covid-19 positive total cases in Indonesia. Data Training used in this study is Indonesian total positive cases of Covid-19 from March 2 until May 26. The next 10 days data become data testing to show the model accuracy in predicting Covid-19 total cases. MLP shows a better prediction comparing to ELM.Three different prediction accuracy measurement is used – MAE, MAPE, and RMSE. All of them shows less value in MLP than in ELM.

scholarly journals Performance analysis of the TensorFlow library with different optimisation algorithms

2021 ◽  
Vol 21 ◽  
pp. 330-335
Author(s):  
Maciej Wadas ◽  
Jakub Smołka
Keyword(s):  
Neural Network ◽  
Performance Analysis ◽  
Network Model ◽  
Neural Network Model ◽  
Deep Neural Networks ◽  
Network Models ◽  
Neural Network Models ◽  
Training Time ◽  
The Neural Network ◽  
Training Efficiency

This paper presents the results of performance analysis of the Tensorflow library used in machine learning and deep neural networks. The analysis focuses on comparing the parameters obtained when training the neural network model for optimization algorithms: Adam, Nadam, AdaMax, AdaDelta, AdaGrad. Special attention has been paid to the differences between the training efficiency on tasks using microprocessor and graphics card. For the study, neural network models were created in order to recognise Polish handwritten characters. The results obtained showed that the most efficient algorithm is AdaMax, while the computer component used during the research only affects the training time of the neural network model used.

scholarly journals Comparison of State-Of-The-Art Neural Network Survival Models With The Pooled Cohort Equations for Cardiovascular Disease Risk Prediction

2021 ◽  
Author(s):  
Yu Deng ◽  
Lei Liu ◽  
Hongmei Jiang ◽  
Yifan Peng ◽  
Yishu wei ◽  
...  
Keyword(s):  
Neural Network ◽  
Risk Prediction ◽  
State Of The Art ◽  
White Male ◽  
External Validation ◽  
Network Models ◽  
Black Female ◽  
Survival Models ◽  
Neural Network Models ◽  
Ascvd Risk

Abstract Background: The Pooled Cohort Equations (PCEs) are race- and sex-specific Cox PH-based models used for 10-year atherosclerotic cardiovascular disease (ASCVD) risk prediction with acceptable discrimination. In recent years, neural network models have gained increasing popularity with their success in image recognition and text classification. Various survival neural network models have been proposed by combining survival analysis and neural network architecture to take advantage of the strengths from both. However, the performance of these survival neural network models compared to each other and to PCEs in ASCVD prediction is unknown. Methods: In this study, we used 6 cohorts from the Lifetime Risk Pooling Project and compared the performance of the PCEs in 10-year ASCVD risk prediction with an all two-way interactions Cox PH model (Cox PH-TWI) and three state-of-the-art neural network survival models including Nnet-survival, Deepsurv, and Cox-nnet. For all the models, we used the same 7 covariates as used in the PCEs. We fitted each of the aforementioned models in white females, white males, black females, and black males, respectively. We evaluated models’ internal and external discrimination power and calibration.Results: The training/internal validation sample comprised 23246 individuals. The average age at baseline was 57.8 years old (SD = 9.6); 16% developed ASCVD during average follow-up of 10.50 (SD = 3.02) years. Based on 10x10 cross-validation, the method that had the highest C-statistics was Cox PH-TWI (0.7372) for white males, PCE (0.7973) for white females, Cox PH-TWI (0.6989) for black males, and Deepsurv (0.7874) for black females. In the external validation dataset, PCE (0.7102), Deepsurv (0.7293), PCE (0.6907), and Nnet-survival (0.7243) had the highest C-statistics for white male, white female, black male, and black female population, respectively. Calibration plots showed that in 10x10 validation, PCE had good calibration in white male, white female, black male but was outperformed by Deepsurv in black female. In external validation, all models overestimated the risk for 10-year ASCVD except for Deepsurv in black female.Conclusions We demonstrated the use of the state-of-the-art neural network survival models in ASCVD risk prediction. Neural network survival models and PCEs have generally comparable discrimination and calibration.

Sign in / Sign up

Export Citation Format

Share Document