scholarly journals Using Convolutional Neural Networks for Sentiment Attitude Extraction from Analytical Texts

10.29007/26g7 ◽  
2019 ◽  
Author(s):  
Nicolay Rusnachenko ◽  
Natalia Loukachevitch
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Mass Media ◽  
International Relations ◽  
Convolutional Neural Networks ◽  
Neural Network Model ◽  
Learning Task ◽  
Named Entities ◽  
Attitude Extraction

In this paper we present an application of the specific neural network model for sentiment attitude extraction without handcrafted NLP features implementation. Given a mass-media article with the list of named entities mentioned in it, the task is to extract sentiment relations between these entities. We considered this problem for the whole documents as a three-class machine learning task. The modified architecture of the Convolutional Neural Networks were used and called as Piecewise Convolutional Neural Network (PCNN). The latter exploits positions of named entities in text to emphasize aspects for inner and outer contexts of relation between entities. For the experiments, the RuSentRel corpus was used, it contains Russian analytical texts in the domain of international relations.

Deriving Metamodels to Relate Machine Learning Quality to Design Repository Characteristics in the Context of Additive Manufacturing

2020 ◽  
Author(s):  
Glen Williams ◽  
Nicholas A. Meisel ◽  
Timothy W. Simpson ◽  
Christopher McComb
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Additive Manufacturing ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Spatial Diversity ◽  
Computational Effort ◽  
Training Dataset ◽  
The Creation

Abstract The widespread growth of additive manufacturing, a field with a complex informatic “digital thread”, has helped fuel the creation of design repositories, where multiple users can upload distribute, and download a variety of candidate designs for a variety of situations. Additionally, advancements in additive manufacturing process development, design frameworks, and simulation are increasing what is possible to fabricate with AM, further growing the richness of such repositories. Machine learning offers new opportunities to combine these design repository components’ rich geometric data with their associated process and performance data to train predictive models capable of automatically assessing build metrics related to AM part manufacturability. Although design repositories that can be used to train these machine learning constructs are expanding, our understanding of what makes a particular design repository useful as a machine learning training dataset is minimal. In this study we use a metamodel to predict the extent to which individual design repositories can train accurate convolutional neural networks. To facilitate the creation and refinement of this metamodel, we constructed a large artificial design repository, and subsequently split it into sub-repositories. We then analyzed metadata regarding the size, complexity, and diversity of the sub-repositories for use as independent variables predicting accuracy and the required training computational effort for training convolutional neural networks. The networks each predict one of three additive manufacturing build metrics: (1) part mass, (2) support material mass, and (3) build time. Our results suggest that metamodels predicting the convolutional neural network coefficient of determination, as opposed to computational effort, were most accurate. Moreover, the size of a design repository, the average complexity of its constituent designs, and the average and spread of design spatial diversity were the best predictors of convolutional neural network accuracy.

METHOD FOR COHERECE EVALUATION OF UKRAINIAN TEXTS USING CONVO-LUTIONAL NEURAL NETWORK

2019 ◽  
pp. 64-71
Author(s):  
S.D. Pogorilyy ◽  
A.A. Kramov ◽  
P.V. Biletskyi
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Experimental Verification ◽  
Suggested Method ◽  
Text Coherence ◽  
Analytic Review ◽  
Automated Methods

The estimation of text coherence is one of the most actual tasks of computer linguistics. Analysis of text coherence is widely used for writing and selection of documents. It allows clearly conveying the idea of an author to a reader. The importance of this task can be confirmed by the availability of actual works that are dedicated to solving it. Different automated methods for the estimation of text coherence are based on the methodology of machine learning. Corresponding methods are based on of formal text representation and following detection of regularities for the generation of an output result. The purpose of this work is to perform the analytic review of different automated methods for the estimation of text coherence; to justify method selection and adapt it due to the features of the Ukrainian language; to perform the experimental verification of the effectiveness of the suggested method for a Ukrainian corpus. In this paper, the comparative analysis of the methods for the estimation of coherence of English texts basing on a machine learning methodology has been performed. The expediency of application of methods that are based on trained universal models for the formalized representation of text components has been justified. The following models using neural networks with different architecture can be considered: recurrent and convolutional networks. These types of networks are widely used for text processing because they allow processing input data with an unfixed structure like sentences or words. Despite the ability of recurrent neural networks to take into account previous data (this behavior is similar to text perception by the reader), the convolutional neural network for conducting experimental research has been chosen. Such choice has been made due to the ability of convolutional neural networks to detect relations between entities regardless of the distance between them. In this paper, the principle of the method basing on the convolutional neural network and the corresponding architecture has been described. Program application for the verification of the suggested method effectiveness has been created. Formalized representation of text elements has been performed using a previously trained model for the semantic representation of words; the training process of this model has been implemented on the corpus of Ukrainian scientific abstracts. The training of the formed networks using pre-trained model has been performed. Experimental verification of method effectiveness for solving of document discrimination task and insert task has been made on the set of scientific articles. The results obtained may indicate that the method using convolutional neural networks can be used for further estimation of coherence of Ukrainian texts.

scholarly journals A Study on the Effect of DropConnect to Control Overfitting in Designing Neural Networks

2020 ◽  
Author(s):  
Hyun-il Lim
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Neural Network Model ◽  
Network Models ◽  
Training Data ◽  
Neural Network Models ◽  
Structural Problems ◽  
The Neural Network ◽  
Training Neural Network

The neural network is an approach of machine learning by training the connected nodes of a model to predict the results of specific problems. The prediction model is trained by using previously collected training data. In training neural network models, overfitting problems can occur from the excessively dependent training of data and the structural problems of the models. In this paper, we analyze the effect of DropConnect for controlling overfitting in neural networks. It is analyzed according to the DropConnect rates and the number of nodes in designing neural networks. The analysis results of this study help to understand the effect of DropConnect in neural networks. To design an effective neural network model, the DropConnect can be applied with appropriate parameters from the understanding of the effect of the DropConnect in neural network models.

scholarly journals Steganalisis Blind dengan Metode Convolutional Neural Network (CNN) Yedroudj- Net terhadap Tools Steganografi

2020 ◽  
Vol 7 (4) ◽  
pp. 787
Author(s):  
Nurmi Hidayasari ◽  
Imam Riadi ◽  
Yudi Prayudi
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Error Probability ◽  
Learning Performance ◽  
Net Method ◽  
Blind Steganalysis ◽  
Steganalysis Method

<p>Steganalisis digunakan untuk mendeteksi ada atau tidaknya file steganografi. Salah satu kategori steganalisis adalah blind steganalisis, yaitu cara untuk mendeteksi file rahasia tanpa mengetahui metode steganografi apa yang digunakan. Sebuah penelitian mengusulkan bahwa metode Convolutional Neural Networks (CNN) dapat mendeteksi file steganografi menggunakan metode terbaru dengan nilai probabilitas kesalahan rendah dibandingkan metode lain, yaitu CNN Yedroudj-net. Sebagai metode steganalisis Machine Learning terbaru, diperlukan eksperimen untuk mengetahui apakah Yedroudj-net dapat menjadi steganalisis untuk keluaran dari tools steganografi yang biasa digunakan. Mengetahui kinerja CNN Yedroudj-net sangat penting, untuk mengukur tingkat kemampuannya dalam hal steganalisis dari beberapa tools. Apalagi sejauh ini, kinerja Machine Learning masih diragukan dalam blind steganalisis. Ditambah beberapa penelitian sebelumnya hanya berfokus pada metode tertentu untuk membuktikan kinerja teknik yang diusulkan, termasuk Yedroudj-net. Penelitian ini akan menggunakan lima alat yang cukup baik dalam hal steganografi, yaitu Hide In Picture (HIP), OpenStego, SilentEye, Steg dan S-Tools, yang tidak diketahui secara pasti metode steganografi apa yang digunakan pada alat tersebut. Metode Yedroudj-net akan diimplementasikan dalam file steganografi dari output lima alat. Kemudian perbandingan dengan tools steganalisis lain, yaitu StegSpy. Hasil penelitian menunjukkan bahwa Yedroudj-net bisa mendeteksi keberadaan file steganografi. Namun, jika dibandingkan dengan StegSpy hasil gambar yang tidak terdeteksi lebih tinggi.</p><p><em><strong><br /></strong></em></p><p><em><strong>Abstract</strong></em></p><p><em>Steganalysis is used to detect the presence or absence of steganograpy files. One category of steganalysis is blind steganalysis, which is a way to detect secret files without knowing what steganography method is used. A study proposes that the Convolutional Neural Networks (CNN) method can detect steganographic files using the latest method with a low error probability value compared to other methods, namely CNN Yedroudj-net. As the latest Machine Learning steganalysis method, an experiment is needed to find out whether Yedroudj-net can be a steganalysis for the output of commonly used steganography tools. Knowing the performance of CNN Yedroudj-net is very important, to measure the level of ability in terms of steganalysis from several tools. Especially so far, Machine Learning performance is still doubtful in blind steganalysis. Plus some previous research only focused on certain methods to prove the performance of the proposed technique, including Yedroudj-net. This research will use five tools that are good enough in terms of steganography, namely Hide In Picture (HIP), OpenStego, SilentEye, Steg and S-Tools, which is not known exactly what steganography methods are used on the tool. The Yedroudj-net method will be implemented in a steganographic file from the output of five tools. Then compare with other steganalysis tools, namely StegSpy. The results showed that Yedroudj-net could detect the presence of steganographic files. However, when compared with StegSpy the results of undetected images are higher.</em></p>

scholarly journals ENNGene: an Easy Neural Network model building tool for Genomics

2021 ◽  
Author(s):  
Eliska Chalupova ◽  
Ondrej Vaculik ◽  
Filip Jozefov ◽  
Jakub Polacek ◽  
Tomas Majtner ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Big Data ◽  
Deep Learning ◽  
Network Model ◽  
Neural Network Model ◽  
Model Building ◽  
Evolutionary Conservation ◽  
Learning Methods

Background: The recent big data revolution in Genomics, coupled with the emergence of Deep Learning as a set of powerful machine learning methods, has shifted the standard practices of machine learning for Genomics. Even though Deep Learning methods such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are becoming widespread in Genomics, developing and training such models is outside the ability of most researchers in the field. Results: Here we present ENNGene - Easy Neural Network model building tool for Genomics. This tool simplifies training of custom CNN or hybrid CNN-RNN models on genomic data via an easy-to-use Graphical User Interface. ENNGene allows multiple input branches, including sequence, evolutionary conservation, and secondary structure, and performs all the necessary preprocessing steps, allowing simple input such as genomic coordinates. The network architecture is selected and fully customized by the user, from the number and types of the layers to each layer's precise set-up. ENNGene then deals with all steps of training and evaluation of the model, exporting valuable metrics such as multi-class ROC and precision-recall curve plots or TensorBoard log files. To facilitate interpretation of the predicted results, we deploy Integrated Gradients, providing the user with a graphical representation of an attribution level of each input position. To showcase the usage of ENNGene, we train multiple models on the RBP24 dataset, quickly reaching the state of the art while improving the performance on more than half of the proteins by including the evolutionary conservation score and tuning the network per protein. Conclusions: As the role of DL in big data analysis in the near future is indisputable, it is important to make it available for a broader range of researchers. We believe that an easy-to-use tool such as ENNGene can allow Genomics researchers without a background in Computational Sciences to harness the power of DL to gain better insights into and extract important information from the large amounts of data available in the field.

scholarly journals Toward Audio Beehive Monitoring: Deep Learning vs. Standard Machine Learning in Classifying Beehive Audio Samples

2018 ◽  
Vol 8 (9) ◽  
pp. 1573 ◽  
Author(s):  
Vladimir Kulyukin ◽  
Sarbajit Mukherjee ◽  
Prakhar Amlathe
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Feature Engineering ◽  
Learning Methods ◽  
Machine Learning Methods

Electronic beehive monitoring extracts critical information on colony behavior and phenology without invasive beehive inspections and transportation costs. As an integral component of electronic beehive monitoring, audio beehive monitoring has the potential to automate the identification of various stressors for honeybee colonies from beehive audio samples. In this investigation, we designed several convolutional neural networks and compared their performance with four standard machine learning methods (logistic regression, k-nearest neighbors, support vector machines, and random forests) in classifying audio samples from microphones deployed above landing pads of Langstroth beehives. On a dataset of 10,260 audio samples where the training and testing samples were separated from the validation samples by beehive and location, a shallower raw audio convolutional neural network with a custom layer outperformed three deeper raw audio convolutional neural networks without custom layers and performed on par with the four machine learning methods trained to classify feature vectors extracted from raw audio samples. On a more challenging dataset of 12,914 audio samples where the training and testing samples were separated from the validation samples by beehive, location, time, and bee race, all raw audio convolutional neural networks performed better than the four machine learning methods and a convolutional neural network trained to classify spectrogram images of audio samples. A trained raw audio convolutional neural network was successfully tested in situ on a low voltage Raspberry Pi computer, which indicates that convolutional neural networks can be added to a repertoire of in situ audio classification algorithms for electronic beehive monitoring. The main trade-off between deep learning and standard machine learning is between feature engineering and training time: while the convolutional neural networks required no feature engineering and generalized better on the second, more challenging dataset, they took considerably more time to train than the machine learning methods. To ensure the replicability of our findings and to provide performance benchmarks for interested research and citizen science communities, we have made public our source code and our curated datasets.

Real-time Detection of Aortic Valve in Echocardiography using Convolutional Neural Networks

2020 ◽  
Vol 16 (5) ◽  
pp. 584-591 ◽  
Author(s):  
Muhammad Hanif Ahmad Nizar ◽  
Chow Khuen Chan ◽  
Azira Khalil ◽  
Ahmad Khairuddin Mohamed Yusof ◽  
Khin Wee Lai
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Heart Disease ◽  
Aortic Valve ◽  
Real Time ◽  
Convolutional Neural Networks ◽  
Valvular Heart Disease ◽  
Detection System ◽  
Processing Unit ◽  
Real Time Detection

Background: Valvular heart disease is a serious disease leading to mortality and increasing medical care cost. The aortic valve is the most common valve affected by this disease. Doctors rely on echocardiogram for diagnosing and evaluating valvular heart disease. However, the images from echocardiogram are poor in comparison to Computerized Tomography and Magnetic Resonance Imaging scan. This study proposes the development of Convolutional Neural Networks (CNN) that can function optimally during a live echocardiographic examination for detection of the aortic valve. An automated detection system in an echocardiogram will improve the accuracy of medical diagnosis and can provide further medical analysis from the resulting detection. Methods: Two detection architectures, Single Shot Multibox Detector (SSD) and Faster Regional based Convolutional Neural Network (R-CNN) with various feature extractors were trained on echocardiography images from 33 patients. Thereafter, the models were tested on 10 echocardiography videos. Results: Faster R-CNN Inception v2 had shown the highest accuracy (98.6%) followed closely by SSD Mobilenet v2. In terms of speed, SSD Mobilenet v2 resulted in a loss of 46.81% in framesper- second (fps) during real-time detection but managed to perform better than the other neural network models. Additionally, SSD Mobilenet v2 used the least amount of Graphic Processing Unit (GPU) but the Central Processing Unit (CPU) usage was relatively similar throughout all models. Conclusion: Our findings provide a foundation for implementing a convolutional detection system to echocardiography for medical purposes.

scholarly journals Discretization and machine learning approximation of BSDEs with a constraint on the Gains-process

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Idris Kharroubi ◽  
Thomas Lim ◽  
Xavier Warin
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Differential Equations ◽  
Numerical Experiments ◽  
Optimization Problem ◽  
Learning Approach ◽  
The Neural Network ◽  
Machine Learning Approach ◽  
Mesh Grid

AbstractWe study the approximation of backward stochastic differential equations (BSDEs for short) with a constraint on the gains process. We first discretize the constraint by applying a so-called facelift operator at times of a grid. We show that this discretely constrained BSDE converges to the continuously constrained one as the mesh grid converges to zero. We then focus on the approximation of the discretely constrained BSDE. For that we adopt a machine learning approach. We show that the facelift can be approximated by an optimization problem over a class of neural networks under constraints on the neural network and its derivative. We then derive an algorithm converging to the discretely constrained BSDE as the number of neurons goes to infinity. We end by numerical experiments.

Sign in / Sign up

Export Citation Format

Share Document