An Energy-Efficient Inference Method in Convolutional Neural Networks Based on Dynamic Adjustment of the Pruning Level

2021 ◽  
Vol 26 (6) ◽  
pp. 1-20
Author(s):  
Mohammad-Ali Maleki ◽  
Alireza Nabipour-Meybodi ◽  
Mehdi Kamal ◽  
Ali Afzali-Kusha ◽  
Massoud Pedram
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Lower Energy ◽  
Dynamic Adjustment ◽  
Inference Method ◽  
System A ◽  
Online Management ◽  
The Cost ◽  
Selection Of

In this article, we present a low-energy inference method for convolutional neural networks in image classification applications. The lower energy consumption is achieved by using a highly pruned (lower-energy) network if the resulting network can provide a correct output. More specifically, the proposed inference method makes use of two pruned neural networks (NNs), namely mildly and aggressively pruned networks, which are both designed offline. In the system, a third NN makes use of the input data for the online selection of the appropriate pruned network. The third network, for its feature extraction, employs the same convolutional layers as those of the aggressively pruned NN, thereby reducing the overhead of the online management. There is some accuracy loss induced by the proposed method where, for a given level of accuracy, the energy gain of the proposed method is considerably larger than the case of employing any one pruning level. The proposed method is independent of both the pruning method and the network architecture. The efficacy of the proposed inference method is assessed on Eyeriss hardware accelerator platform for some of the state-of-the-art NN architectures. Our studies show that this method may provide, on average, 70% energy reduction compared to the original NN at the cost of about 3% accuracy loss on the CIFAR-10 dataset.

scholarly journals X-ray image based pneumonia classification using convolutional neural networks

2020 ◽  
Vol 6 (20) ◽  
pp. 54-67
Author(s):  
Sarah Badr AlSumairi ◽  
Mohamed Maher Ben Ismail
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Data Augmentation ◽  
Critical Factor ◽  
Classification Problem ◽  
X Rays ◽  
X Ray ◽  
Erroneous Decision ◽  
Short Period

Pneumonia is an infectious disease of the lungs. About one third to one half of pneumonia cases are caused by bacteria. Early diagnosis is a critical factor for a successful treatment process. Typically, the disease can be diagnosed by a radiologist using chest X-ray images. In fact, chest X-rays are currently the best available method for diagnosing pneumonia. However, the recognition of pneumonia symptoms is a challenging task that relies on the availability of expert radiologists. Such “human” diagnosis can be inaccurate and subjective due to lack of clarity and erroneous decision. Moreover, the error can increase more if the physician is requested to analyze tens of X-rays within a short period of time. Therefore, Computer-Aided Diagnosis (CAD) systems were introduced to support and assist physicians and make their efforts more productive. In this paper, we investigate, design, implement and assess customized Convolutional Neural Networks to overcome the image-based Pneumonia classification problem. Namely, ResNet-50 and DenseNet-161 models were inherited to design customized deep network architecture and improve the overall pneumonia classification accuracy. Moreover, data augmentation was deployed and associated with standard datasets to assess the proposed models. Besides, standard performance measures were used to validate and evaluate the proposed system.

scholarly journals A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

2020 ◽  
Author(s):  
B Wang ◽  
Y Sun ◽  
Bing Xue ◽  
Mengjie Zhang
Keyword(s):  
Neural Networks ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Learning Task ◽  
Fixed Number ◽  
Learning Rate ◽  
Current Layer ◽  
Training Process ◽  
Deep Convolutional Neural Networks

© 2019, Springer Nature Switzerland AG. Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the problem. In recent years, instead of the traditional way of only connecting the current layer with its next layer, shortcut connections have been proposed to connect the current layer with its forward layers apart from its next layer, which has been proved to be able to facilitate the training process of deep CNNs. However, there are various ways to build the shortcut connections, it is hard to manually design the best shortcut connections when solving a particular problem, especially given the design of the network architecture is already very challenging. In this paper, a hybrid evolutionary computation (EC) method is proposed to automatically evolve both the architecture of deep CNNs and the shortcut connections. Three major contributions of this work are: Firstly, a new encoding strategy is proposed to encode a CNN, where the architecture and the shortcut connections are encoded separately; Secondly, a hybrid two-level EC method, which combines particle swarm optimisation and genetic algorithms, is developed to search for the optimal CNNs; Lastly, an adjustable learning rate is introduced for the fitness evaluations, which provides a better learning rate for the training process given a fixed number of epochs. The proposed algorithm is evaluated on three widely used benchmark datasets of image classification and compared with 12 peer Non-EC based competitors and one EC based competitor. The experimental results demonstrate that the proposed method outperforms all of the peer competitors in terms of classification accuracy.

scholarly journals A Survey on Techniques used in Medical Imaging Processing

2021 ◽  
Vol 2089 (1) ◽  
pp. 012013
Author(s):  
Priyadarshini Chatterjee ◽  
Dutta Sushama Rani
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Analysis ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Medical Image Analysis ◽  
Artificial Neural ◽  
Automated Screening ◽  
To Come

Abstract Automated diagnosis of diseases in the recent years have gain lots of advantages and potential. Specially automated screening of cancers has helped the clinicians over the time. Sometimes it is seen that the diagnosis of the clinicians is biased but automated detection can help them to come to a proper conclusion. Automated screening is implemented using either artificial inter connected system or convolutional inter connected system. As Artificial neural network is slow in computation, so Convolutional Neural Network has achieved lots of importance in the recent years. It is also seen that Convolutional Neural Network architecture requires a smaller number of datasets. This also provides them an edge over Artificial Neural Networks. Convolutional Neural Networks is used for both segmentation and classification. Image dissection is one of the important steps in the model used for any kind of image analysis. This paper surveys various such Convolutional Neural Networks that are used for medical image analysis.

scholarly journals Using Convolutional Neural Networks to Automate Aircraft Maintenance Visual Inspection

Aerospace ◽  
2020 ◽  
Vol 7 (12) ◽  
pp. 171
Author(s):  
Anil Doğru ◽  
Soufiane Bouarfa ◽  
Ridwan Arizar ◽  
Reyhan Aydoğan
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Visual Inspection ◽  
Hybrid Approach ◽  
Lightning Strike ◽  
Aircraft Maintenance ◽  
Original Dataset ◽  
Wide Range ◽  
Augmentation Techniques

Convolutional Neural Networks combined with autonomous drones are increasingly seen as enablers of partially automating the aircraft maintenance visual inspection process. Such an innovative concept can have a significant impact on aircraft operations. Though supporting aircraft maintenance engineers detect and classify a wide range of defects, the time spent on inspection can significantly be reduced. Examples of defects that can be automatically detected include aircraft dents, paint defects, cracks and holes, and lightning strike damage. Additionally, this concept could also increase the accuracy of damage detection and reduce the number of aircraft inspection incidents related to human factors like fatigue and time pressure. In our previous work, we have applied a recent Convolutional Neural Network architecture known by MASK R-CNN to detect aircraft dents. MASK-RCNN was chosen because it enables the detection of multiple objects in an image while simultaneously generating a segmentation mask for each instance. The previously obtained F1 and F2 scores were 62.67% and 59.35%, respectively. This paper extends the previous work by applying different techniques to improve and evaluate prediction performance experimentally. The approach uses include (1) Balancing the original dataset by adding images without dents; (2) Increasing data homogeneity by focusing on wing images only; (3) Exploring the potential of three augmentation techniques in improving model performance namely flipping, rotating, and blurring; and (4) using a pre-classifier in combination with MASK R-CNN. The results show that a hybrid approach combining MASK R-CNN and augmentation techniques leads to an improved performance with an F1 score of (67.50%) and F2 score of (66.37%).

scholarly journals Exploiting Weak Ties in Incomplete Network Datasets Using Simplified Graph Convolutional Neural Networks

2020 ◽  
Vol 2 (2) ◽  
pp. 125-146 ◽  
Author(s):  
Neda H. Bidoki ◽  
Alexander V. Mantzaris ◽  
Gita Sukthankar
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Centrality Measures ◽  
Weak Ties ◽  
Original Graph ◽  
Convolutional Network ◽  
Strong Ties ◽  
Network Nodes ◽  
Node Removal ◽  
Selection Of

This paper explores the value of weak-ties in classifying academic literature with the use of graph convolutional neural networks. Our experiments look at the results of treating weak-ties as if they were strong-ties to determine if that assumption improves performance. This is done by applying the methodological framework of the Simplified Graph Convolutional Neural Network (SGC) to two academic publication datasets: Cora and Citeseer. The performance of SGC is compared to the original Graph Convolutional Network (GCN) framework. We also examine how node removal affects prediction accuracy by selecting nodes according to different centrality measures. These experiments provide insight for which nodes are most important for the performance of SGC. When removal is based on a more localized selection of nodes, augmenting the network with both strong-ties and weak-ties provides a benefit, indicating that SGC successfully leverages local information of network nodes.

scholarly journals Ear Detection and Localization with Convolutional Neural Networks in Natural Images and Videos

Processes ◽  
2019 ◽  
Vol 7 (7) ◽  
pp. 457 ◽  
Author(s):  
William Raveane ◽  
Pedro Luis Galdámez ◽  
María Angélica González Arrieta
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Detection System ◽  
Recognition System ◽  
Input Image ◽  
Natural Images ◽  
System A ◽  
Irregular Shapes ◽  
Ear Detection ◽  
Detection And Localization

The difficulty in precisely detecting and locating an ear within an image is the first step to tackle in an ear-based biometric recognition system, a challenge which increases in difficulty when working with variable photographic conditions. This is in part due to the irregular shapes of human ears, but also because of variable lighting conditions and the ever changing profile shape of an ear’s projection when photographed. An ear detection system involving multiple convolutional neural networks and a detection grouping algorithm is proposed to identify the presence and location of an ear in a given input image. The proposed method matches the performance of other methods when analyzed against clean and purpose-shot photographs, reaching an accuracy of upwards of 98%, but clearly outperforms them with a rate of over 86% when the system is subjected to non-cooperative natural images where the subject appears in challenging orientations and photographic conditions.

scholarly journals Understanding unconventional preprocessors in deep convolutional neural networks for face identification

2019 ◽  
Vol 1 (11) ◽  
Author(s):  
Chollette C. Olisah ◽  
Lyndon Smith
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Data Augmentation ◽  
Color Space ◽  
Back Propagation ◽  
Face Identification ◽  
Deep Convolutional Neural Networks ◽  
Softmax Classifier ◽  
Deep Network

Abstract Deep convolutional neural networks have achieved huge successes in application domains like object and face recognition. The performance gain is attributed to different facets of the network architecture such as: depth of the convolutional layers, activation function, pooling, batch normalization, forward and back propagation and many more. However, very little emphasis is made on the preprocessor’s module of the network. Therefore, in this paper, the network’s preprocessing module is varied across different preprocessing approaches while keeping constant other facets of the deep network architecture, to investigate the contribution preprocessing makes to the network. Commonly used preprocessors are the data augmentation and normalization and are termed conventional preprocessors. Others are termed the unconventional preprocessors, they are: color space converters; grey-level resolution preprocessors; full-based and plane-based image quantization, Gaussian blur, illumination normalization and insensitive feature preprocessors. To achieve fixed network parameters, CNNs with transfer learning is employed. The aim is to transfer knowledge from the high-level feature vectors of the Inception-V3 network to offline preprocessed LFW target data; and features is trained using the SoftMax classifier for face identification. The experiments show that the discriminative capability of the deep networks can be improved by preprocessing RGB data with some of the unconventional preprocessors before feeding it to the CNNs. However, for best performance, the right setup of preprocessed data with augmentation and/or normalization is required. Summarily, preprocessing data before it is fed to the deep network is found to increase the homogeneity of neighborhood pixels even at reduced bit depth which serves for better storage efficiency.

scholarly journals Fusing Convolutional Neural Networks to Improve the Accuracy of Plant Leaf Disease Classification

2020 ◽  
pp. 9-19
Author(s):  
Bhavana Nerkar ◽  
Sanjay Talbar
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Disease Classification ◽  
Disease Detection ◽  
Plant Leaf ◽  
Leaf Disease ◽  
Highly Effective ◽  
Effective Network ◽  
Number Of Classes

Aims: This text aims to improve the accuracy of plant leaf disease detection using a fused convolutional neural network architecture Study Design:  In this study, propose a hybrid CNN architecture, that adds a bio-inspired layer to the existing CNN architecture in order to improve the accuracy and reduce the delay needed for leaf disease classification. Place and Duration of Study: National institute of electronics and information technology Aurangabad, between June 2018 and September 2020. Methodology: Convolutional neural networks (CNNs) have become a de-facto technique for classification of multi-dimensional data. Activation functions like rectified linear unit (ReLU), softmax, sigmoid, etc. have proven to be highly effective when doing so. Moreover, standard CNN architectures like AlexNet, VGGNet, Google net, etc. further assist this process by providing standard and highly effective network layer arrangements. But these architectures are limited by the speed due to high number of calculations needed to train and test the network. Moreover, as the number of classes increase, there is a reduction in validation and testing accuracy for the networks. In order to remove these drawbacks, hybrid CNN architecture, that adds a bio-inspired layer to the existing CNN architecture in order to improve the accuracy and speed of leaf classification. Results: The developed system was tested on different kinds of leaf diseases, and it was observed that the proposed system obtains more than 98% accuracy for both testing and validation sets. Conclusion: It is observed that the delay is reduced, while the accuracy is improved by the most effective classifiers. This encourage us to use the proposed system for real-time leaf image disease detection.

scholarly journals Polymorphic Malware Detection by Image Conversion Technique

2020 ◽  
Vol 9 (3) ◽  
pp. 2898-2903
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Training Module ◽  
Dynamic Approach ◽  
System A ◽  
Image Conversion ◽  
Binary Format ◽  
Testing Data ◽  
Hash Codes ◽  
Polymorphic Malware

This model implements ways to detect polymorphic malware. This model uses a dynamic approach to detect the polymorphic malware. The objective is to increase the accuracy and efficiency of the detection as this malware can morph themselves, making it difficult to trace through anti-malware systems. As the tracing is going to be difficult the detection and classification system needs to be flexible that can able to detect the malware in every possible environment. This objective can be achieved by giving the system a superintelligence, this can be done by using the Convolutional Neural Networks (CNNs) in our system. This method records the pattern or the traces made by the polymorphic malware. The pattern is in the form of the image which is formed by converting the binary format of the hash codes. The generated images are then sent to the training module, based on this training module the Convolutional Neural Networks gives the result for any testing data.

Sign in / Sign up

Export Citation Format

Share Document