scholarly journals Research and Verification of Convolutional Neural Network Lightweight in BCI

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Shipu Xu ◽  
Runlong Li ◽  
Yunsheng Wang ◽  
Yong Liu ◽  
Wenwen Hu ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Network Structure ◽  
Classification Accuracy ◽  
Original Network ◽  
Network Parameter ◽  
Network Parameters ◽  
Network Operation ◽  
Depth And Complexity ◽  
Convergence Stability

With the increasing of depth and complexity of the convolutional neural network, parameter dimensionality and volume of computing have greatly restricted its applications. Based on the SqueezeNet network structure, this study introduces a block convolution and uses channel shuffle between blocks to alleviate the information jam. The method is aimed at reducing the dimensionality of parameters of in an original network structure and improving the efficiency of network operation. The verification performance of the ORL dataset shows that the classification accuracy and convergence efficiency are not reduced or even slightly improved when the network parameters are reduced, which supports the validity of block convolution in structure lightweight. Moreover, using a classic CIFAR-10 dataset, this network decreases parameter dimensionality while accelerating computational processing, with excellent convergence stability and efficiency when the network accuracy is only reduced by 1.3%.

Compressing convolutional neural networks via intermediate features

2021 ◽  
pp. 1-13
Author(s):  
Jingfei Chang ◽  
Yang Lu ◽  
Ping Xue ◽  
Xing Wei ◽  
Zhen Wei
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Network Structure ◽  
Source Code ◽  
Floating Point ◽  
Portable Devices ◽  
Original Network ◽  
Basic Blocks ◽  
Better Than

Deep convolutional neural network (CNN) is difficult to deploy to mobile and portable devices due to its large number of parameters and floating-point operations (FLOPs). To tackle this problem, we propose a novel channel pruning method. We use the modified squeeze-and-excitation blocks (MSEB) to measure the importance of the channels in the convolutional layers. The unimportant channels, including convolutional kernels related to them, are pruned directly, which greatly reduces the storage cost and the number of calculations. For ResNet with basic blocks, we propose an approach to consistently prune all residual blocks in the same stage to ensure that the compact network structure is dimensionally correct. After pruning we retrain the compact network from scratch to restore the accuracy. Finally, we verify our method on CIFAR-10, CIFAR-100 and ILSVRC-2012. The results indicate that the performance of the compact network is better than the original network when the pruning rate is small. Even when the pruning amplitude is large, the accuracy can also be maintained or decreased slightly. On the CIFAR-100, when reducing the parameters and FLOPs up to 82% and 62% respectively, the accuracy of VGG-19 even improve by 0.54% after retraining. The source code is available at https://github.com/JingfeiChang/UCP.

scholarly journals BBNet: A Novel Convolutional Neural Network Structure in Edge-Cloud Collaborative Inference

Sensors ◽  
2021 ◽  
Vol 21 (13) ◽  
pp. 4494
Author(s):  
Hongbo Zhou ◽  
Weiwei Zhang ◽  
Chengwei Wang ◽  
Xin Ma ◽  
Haoran Yu
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Network Structure ◽  
Original Data ◽  
Compression Rate ◽  
Original Network ◽  
Intermediate Data ◽  
Communication Time ◽  
Neural Network Structure ◽  
End To End

Edge-cloud collaborative inference can significantly reduce the delay of a deep neural network (DNN) by dividing the network between mobile edge and cloud. However, the in-layer data size of DNN is usually larger than the original data, so the communication time to send intermediate data to the cloud will also increase end-to-end latency. To cope with these challenges, this paper proposes a novel convolutional neural network structure—BBNet—that accelerates collaborative inference from two levels: (1) through channel-pruning: reducing the number of calculations and parameters of the original network; (2) through compressing the feature map at the split point to further reduce the size of the data transmitted. In addition, This paper implemented the BBNet structure based on NVIDIA Nano and the server. Compared with the original network, BBNet’s FLOPs and parameter achieve up to 5.67× and 11.57× on the compression rate, respectively. In the best case, the feature compression layer can reach a bit-compression rate of 512×. Compared with the better bandwidth conditions, BBNet has a more obvious inference delay when the network conditions are poor. For example, when the upload bandwidth is only 20 kb/s, the end-to-end latency of BBNet is increased by 38.89× compared with the cloud-only approach.

scholarly journals Estimation of the influence of spiking neural network parameters on classification accuracy using a genetic algorithm

2018 ◽  
Vol 145 ◽  
pp. 488-494 ◽  
Author(s):  
Aleksandr Sboev ◽  
Alexey Serenko ◽  
Roman Rybka ◽  
Danila Vlasov ◽  
Andrey Filchenkov
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Classification Accuracy ◽  
Spiking Neural Network ◽  
Network Parameters

A visual terrain classification method for mobile robots’ navigation based on convolutional neural network and support vector machine

2021 ◽  
pp. 014233122098791
Author(s):  
Wanli Wang ◽  
Botao Zhang ◽  
Kaiqi Wu ◽  
Sergey A Chepinskiy ◽  
Anton A Zhilenkov ◽  
...  
Keyword(s):  
Neural Network ◽  
Support Vector Machine ◽  
Mobile Robots ◽  
Convolutional Neural Network ◽  
Hybrid Method ◽  
Classification Accuracy ◽  
Support Vector ◽  
High Classification Accuracy ◽  
Enhance Efficiency ◽  
Multi Class Classification

In this paper, a hybrid method based on deep learning is proposed to visually classify terrains encountered by mobile robots. Considering the limited computing resource on mobile robots and the requirement for high classification accuracy, the proposed hybrid method combines a convolutional neural network with a support vector machine to keep a high classification accuracy while improve work efficiency. The key idea is that the convolutional neural network is used to finish a multi-class classification and simultaneously the support vector machine is used to make a two-class classification. The two-class classification performed by the support vector machine is aimed at one kind of terrain that users are mostly concerned with. Results of the two classifications will be consolidated to get the final classification result. The convolutional neural network used in this method is modified for the on-board usage of mobile robots. In order to enhance efficiency, the convolutional neural network has a simple architecture. The convolutional neural network and the support vector machine are trained and tested by using RGB images of six kinds of common terrains. Experimental results demonstrate that this method can help robots classify terrains accurately and efficiently. Therefore, the proposed method has a significant potential for being applied to the on-board usage of mobile robots.

scholarly journals Hyperspectral Image Classification Based on Multi-Scale Residual Network with Attention Mechanism

2021 ◽  
Vol 13 (3) ◽  
pp. 335
Author(s):  
Yuhao Qing ◽  
Wenyi Liu
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Image Classification ◽  
Classification Accuracy ◽  
Hyperspectral Image ◽  
Principal Component ◽  
Hyperspectral Image Classification ◽  
Deep Network ◽  
Multi Scale

In recent years, image classification on hyperspectral imagery utilizing deep learning algorithms has attained good results. Thus, spurred by that finding and to further improve the deep learning classification accuracy, we propose a multi-scale residual convolutional neural network model fused with an efficient channel attention network (MRA-NET) that is appropriate for hyperspectral image classification. The suggested technique comprises a multi-staged architecture, where initially the spectral information of the hyperspectral image is reduced into a two-dimensional tensor, utilizing a principal component analysis (PCA) scheme. Then, the constructed low-dimensional image is input to our proposed ECA-NET deep network, which exploits the advantages of its core components, i.e., multi-scale residual structure and attention mechanisms. We evaluate the performance of the proposed MRA-NET on three public available hyperspectral datasets and demonstrate that, overall, the classification accuracy of our method is 99.82 %, 99.81%, and 99.37, respectively, which is higher compared to the corresponding accuracy of current networks such as 3D convolutional neural network (CNN), three-dimensional residual convolution structure (RES-3D-CNN), and space–spectrum joint deep network (SSRN).

Application of convolution neural network in medical image processing

2020 ◽  
pp. 1-11
Author(s):  
Jie Liu ◽  
Hongbo Zhao
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Network Structure ◽  
Medical Image ◽  
Sampling Methods ◽  
Medical Image Processing ◽  
Convolution Neural Network ◽  
Data Set ◽  
Convolution Kernels ◽  
Neural Network Structure

BACKGROUND: Convolution neural network is often superior to other similar algorithms in image classification. Convolution layer and sub-sampling layer have the function of extracting sample features, and the feature of sharing weights greatly reduces the training parameters of the network. OBJECTIVE: This paper describes the improved convolution neural network structure, including convolution layer, sub-sampling layer and full connection layer. This paper also introduces five kinds of diseases and normal eye images reflected by the blood filament of the eyeball “yan.mat” data set, convenient to use MATLAB software for calculation. METHODSL: In this paper, we improve the structure of the classical LeNet-5 convolutional neural network, and design a network structure with different convolution kernels, different sub-sampling methods and different classifiers, and use this structure to solve the problem of ocular bloodstream disease recognition. RESULTS: The experimental results show that the improved convolutional neural network structure is ideal for the recognition of eye blood silk data set, which shows that the convolution neural network has the characteristics of strong classification and strong robustness. The improved structure can classify the diseases reflected by eyeball bloodstain well.

scholarly journals Social Touch Gesture Recognition Using Convolutional Neural Network

2018 ◽  
Vol 2018 ◽  
pp. 1-10 ◽  
Author(s):  
Saad Albawi ◽  
Oguz Bayat ◽  
Saad Al-Azawi ◽  
Osman N. Ucan
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Gesture Recognition ◽  
Classification Accuracy ◽  
Recognition System ◽  
Human Robot Interaction ◽  
Sensor Data ◽  
Social Touch ◽  
Minimum Number ◽  
Original Frame

Recently, social touch gesture recognition has been considered an important topic for touch modality, which can lead to highly efficient and realistic human-robot interaction. In this paper, a deep convolutional neural network is selected to implement a social touch recognition system for raw input samples (sensor data) only. The touch gesture recognition is performed using a dataset previously measured with numerous subjects that perform varying social gestures. This dataset is dubbed as the corpus of social touch, where touch was performed on a mannequin arm. A leave-one-subject-out cross-validation method is used to evaluate system performance. The proposed method can recognize gestures in nearly real time after acquiring a minimum number of frames (the average range of frame length was from 0.2% to 4.19% from the original frame lengths) with a classification accuracy of 63.7%. The achieved classification accuracy is competitive in terms of the performance of existing algorithms. Furthermore, the proposed system outperforms other classification algorithms in terms of classification ratio and touch recognition time without data preprocessing for the same dataset.

scholarly journals Forecast Model of TV Show Rating Based on Convolutional Neural Network

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Lingfeng Wang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Prediction Model ◽  
Convolutional Neural Network ◽  
Network Structure ◽  
Rating Data ◽  
Tv Program ◽  
Neural Network Structure ◽  
Rating Prediction ◽  
Tv Show

The TV show rating analysis and prediction system can collect and transmit information more quickly and quickly upload the information to the database. The convolutional neural network is a multilayer neural network structure that simulates the operating mechanism of biological vision systems. It is a neural network composed of multiple convolutional layers and downsampling layers sequentially connected. It can obtain useful feature descriptions from original data and is an effective method to extract features from data. At present, convolutional neural networks have become a research hotspot in speech recognition, image recognition and classification, natural language processing, and other fields and have been widely and successfully applied in these fields. Therefore, this paper introduces the convolutional neural network structure to predict the TV program rating data. First, it briefly introduces artificial neural networks and deep learning methods and focuses on the algorithm principles of convolutional neural networks and support vector machines. Then, we improve the convolutional neural network to fit the TV program rating data and finally apply the two prediction models to the TV program rating data prediction. We improve the convolutional neural network TV program rating prediction model and combine the advantages of the convolutional neural network to extract effective features and good classification and prediction capabilities to improve the prediction accuracy. Through simulation comparison, we verify the feasibility and effectiveness of the TV program rating prediction model given in this article.

Evaluation of Parameter Settings for Training Neural Networks Using Backpropagation Algorithms

2022 ◽  
pp. 202-226
Author(s):  
Leema N. ◽  
Khanna H. Nehemiah ◽  
Elgin Christo V. R. ◽  
Kannan A.
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Activation Function ◽  
Neural Network Training ◽  
Network Parameter ◽  
Network Parameters ◽  
Network Training ◽  
Rate Minimum ◽  
Hidden Layer ◽  
Function Number

Artificial neural networks (ANN) are widely used for classification, and the training algorithm commonly used is the backpropagation (BP) algorithm. The major bottleneck faced in the backpropagation neural network training is in fixing the appropriate values for network parameters. The network parameters are initial weights, biases, activation function, number of hidden layers and the number of neurons per hidden layer, number of training epochs, learning rate, minimum error, and momentum term for the classification task. The objective of this work is to investigate the performance of 12 different BP algorithms with the impact of variations in network parameter values for the neural network training. The algorithms were evaluated with different training and testing samples taken from the three benchmark clinical datasets, namely, Pima Indian Diabetes (PID), Hepatitis, and Wisconsin Breast Cancer (WBC) dataset obtained from the University of California Irvine (UCI) machine learning repository.

Sign in / Sign up

Export Citation Format

Share Document