Chapter 12. Learning Reasoning Strategies in End-to-End Differentiable Proving

2021 ◽  
Author(s):  
Pasquale Minervini ◽  
Sebastian Riedel ◽  
Pontus Stenetorp ◽  
Edward Grefenstette ◽  
Tim Rocktäschel
Keyword(s):  
Large Scale ◽  
Source Code ◽  
Back Propagation ◽  
Selection Strategy ◽  
Rule Based ◽  
Reasoning Strategies ◽  
Theorem Provers ◽  
Gradient Based ◽  
Learning To Reason ◽  
Yield State

Attempts to render deep learning models interpretable, data-efficient, and robust have seen some success through hybridisation with rule-based systems, for example, in Neural Theorem Provers (NTPs). These neuro-symbolic models can induce interpretable rules and learn representations from data via back-propagation, while providing logical explanations for their predictions. However, they are restricted by their computational complexity, as they need to consider all possible proof paths for explaining a goal, thus rendering them unfit for large-scale applications. We present Conditional Theorem Provers (CTPs), an extension to NTPs that learns an optimal rule selection strategy via gradient-based optimisation. We show that CTPs are scalable and yield state-of-the-art results on the CLUTRR dataset, which tests systematic generalisation of neural models by learning to reason over smaller graphs and evaluating on larger ones. Finally, CTPs show better link prediction results on standard benchmarks in comparison with other neural-symbolic models, while being explainable. All source code and datasets are available online. (At https://github.com/uclnlp/ctp)

scholarly journals Tandem Inference: An Out-of-Core Streaming Algorithm for Very Large-Scale Relational Inference

2020 ◽  
Vol 34 (06) ◽  
pp. 10259-10266
Author(s):  
Sriram Srinivasan ◽  
Eriq Augustine ◽  
Lise Getoor
Keyword(s):  
Graphical Models ◽  
Large Scale ◽  
Relational Learning ◽  
Statistical Relational Learning ◽  
Rule Based ◽  
Novel Technique ◽  
Gradient Based ◽  
Streaming Algorithm ◽  
Soft Logic ◽  
Machine Memory

Statistical relational learning (SRL) frameworks allow users to create large, complex graphical models using a compact, rule-based representation. However, these models can quickly become prohibitively large and not fit into machine memory. In this work we address this issue by introducing a novel technique called tandem inference (ti). The primary idea of ti is to combine grounding and inference such that both processes happen in tandem. ti uses an out-of-core streaming approach to overcome memory limitations. Even when memory is not an issue, we show that our proposed approach is able to do inference faster while using less memory than existing approaches. To show the effectiveness of ti, we use a popular SRL framework called Probabilistic Soft Logic (PSL). We implement ti for PSL by proposing a gradient-based inference engine and a streaming approach to grounding. We show that we are able to run an SRL model with over 1B cliques in under nine hours and using only 10 GB of RAM; previous approaches required more than 800 GB for this model and are infeasible on common hardware. To the best of our knowledge, this is the largest SRL model ever run.

Economic benefit of shale gas exploitation based on back propagation neural network

2020 ◽  
Vol 39 (6) ◽  
pp. 8823-8830
Author(s):  
Jiafeng Li ◽  
Hui Hu ◽  
Xiang Li ◽  
Qian Jin ◽  
Tianhao Huang
Keyword(s):  
Neural Network ◽  
Shale Gas ◽  
Bp Neural Network ◽  
Large Scale ◽  
Linear Prediction ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Economic Benefits ◽  
Gas Well ◽  
Well Production

Under the influence of COVID-19, the economic benefits of shale gas development are greatly affected. With the large-scale development and utilization of shale gas in China, it is increasingly important to assess the economic impact of shale gas development. Therefore, this paper proposes a method for predicting the production of shale gas reservoirs, and uses back propagation (BP) neural network to nonlinearly fit reservoir reconstruction data to obtain shale gas well production forecasting models. Experiments show that compared with the traditional BP neural network, the proposed method can effectively improve the accuracy and stability of the prediction. There is a nonlinear correlation between reservoir reconstruction data and gas well production, which does not apply to traditional linear prediction methods

scholarly journals Image Restoration by Learning Morphological Opening-Closing Network

2020 ◽  
Vol 4 (1) ◽  
pp. 87-107
Author(s):  
Ranjan Mondal ◽  
Moni Shankar Dey ◽  
Bhabatosh Chanda
Keyword(s):  
Neural Network ◽  
Image Restoration ◽  
State Of The Art ◽  
Source Code ◽  
Back Propagation ◽  
Image Features ◽  
Main Difficulty ◽  
The Right ◽  
Right Order ◽  
Morphological Opening

AbstractMathematical morphology is a powerful tool for image processing tasks. The main difficulty in designing mathematical morphological algorithm is deciding the order of operators/filters and the corresponding structuring elements (SEs). In this work, we develop morphological network composed of alternate sequences of dilation and erosion layers, which depending on learned SEs, may form opening or closing layers. These layers in the right order along with linear combination (of their outputs) are useful in extracting image features and processing them. Structuring elements in the network are learned by back-propagation method guided by minimization of the loss function. Efficacy of the proposed network is established by applying it to two interesting image restoration problems, namely de-raining and de-hazing. Results are comparable to that of many state-of-the-art algorithms for most of the images. It is also worth mentioning that the number of network parameters to handle is much less than that of popular convolutional neural network for similar tasks. The source code can be found here https://github.com/ranjanZ/Mophological-Opening-Closing-Net

scholarly journals A networked voting rule for democratic representation

2018 ◽  
Vol 5 (3) ◽  
pp. 172265 ◽  
Author(s):  
Alexis R. Hernández ◽  
Carlos Gracia-Lázaro ◽  
Edgardo Brigatti ◽  
Yamir Moreno
Keyword(s):  
Large Scale ◽  
Selection Process ◽  
General Relation ◽  
Community Size ◽  
Rule Based ◽  
Voting Rule ◽  
Large Populations ◽  
Negative Effect ◽  
High Connectivity ◽  
Better Than

We introduce a general framework for exploring the problem of selecting a committee of representatives with the aim of studying a networked voting rule based on a decentralized large-scale platform, which can assure a strong accountability of the elected. The results of our simulations suggest that this algorithm-based approach is able to obtain a high representativeness for relatively small committees, performing even better than a classical voting rule based on a closed list of candidates. We show that a general relation between committee size and representatives exists in the form of an inverse square root law and that the normalized committee size approximately scales with the inverse of the community size, allowing the scalability to very large populations. These findings are not strongly influenced by the different networks used to describe the individuals’ interactions, except for the presence of few individuals with very high connectivity which can have a marginal negative effect in the committee selection process.

scholarly journals Segmentation of Database Images Using Gradient Based Multiscale Morphological Reconstruction

2015 ◽  
pp. 271-278
Author(s):  
A. Nithya ◽  
R. Kayalvizhi
Keyword(s):  
Large Scale ◽  
Object Segmentation ◽  
Object Identification ◽  
Segmentation Algorithm ◽  
Phase Object ◽  
Morphological Reconstruction ◽  
Object Based ◽  
Gradient Based ◽  
Object Region ◽  
Identification Phase

The main purpose of this research is to improve the accuracy of object segmentation in database images by constructing an object segmentation algorithm. Image segmentation is a crucial step in the field of image processing and pattern recognition. Segmentation allows the identification of structures in an image which can be utilized for further processing. Both region-based and object-based segmentation are utilized for large-scale database images in a robust and principled manner. Gradient based MultiScalE Graylevel mOrphological recoNstructions (G-SEGON) is used for segmenting an image. SEGON roughly identifies the background and object regions in the image. This proposed method comprises of four phases namely pre-processing phase, object identification phase, object region segmentation phase, majority selection and refinement phase. After developing the grey level mesh the resultant image is converted into gradient and K-means clustering segmentation algorithm is used to segment the object from the gradient image. After implementation the accuracy of the proposed G-SEGON technique is compared with the existing method to prove its efficiency.

scholarly journals Automatic detection of Long Method and God Class code smells through neural source code embeddings

2021 ◽  
Author(s):  
Aleksandar Kovačević ◽  
Jelena Slivka ◽  
Dragan Vidaković ◽  
Katarina-Glorija Grujić ◽  
Nikola Luburić ◽  
...  
Keyword(s):  
Machine Learning ◽  
Large Scale ◽  
Negative Impact ◽  
Source Code ◽  
Systematic Evaluation ◽  
Small Scale ◽  
Code Smells ◽  
Code Metrics ◽  
Code Smell ◽  
F Measure

<p>Code smells are structures in code that often have a negative impact on its quality. Manually detecting code smells is challenging and researchers proposed many automatic code smell detectors. Most of the studies propose detectors based on code metrics and heuristics. However, these studies have several limitations, including evaluating the detectors using small-scale case studies and an inconsistent experimental setting. Furthermore, heuristic-based detectors suffer from limitations that hinder their adoption in practice. Thus, researchers have recently started experimenting with machine learning (ML) based code smell detection. </p><p>This paper compares the performance of multiple ML-based code smell detection models against multiple traditionally employed metric-based heuristics for detection of God Class and Long Method code smells. We evaluate the effectiveness of different source code representations for machine learning: traditionally used code metrics and code embeddings (code2vec, code2seq, and CuBERT).<br></p><p>We perform our experiments on the large-scale, manually labeled MLCQ dataset. We consider the binary classification problem – we classify the code samples as smelly or non-smelly and use the F1-measure of the minority (smell) class as a measure of performance. In our experiments, the ML classifier trained using CuBERT source code embeddings achieved the best performance for both God Class (F-measure of 0.53) and Long Method detection (F-measure of 0.75). With the help of a domain expert, we perform the error analysis to discuss the advantages of the CuBERT approach.<br></p><p>This study is the first to evaluate the effectiveness of pre-trained neural source code embeddings for code smell detection to the best of our knowledge. A secondary contribution of our study is the systematic evaluation of the effectiveness of multiple heuristic-based approaches on the same large-scale, manually labeled MLCQ dataset.<br></p>

scholarly journals Paddy Rice Mapping in Thailand Using Time-Series Sentinel-1 Data and Deep Learning Model

2021 ◽  
Vol 13 (19) ◽  
pp. 3994
Author(s):  
Lu Xu ◽  
Hong Zhang ◽  
Chao Wang ◽  
Sisi Wei ◽  
Bo Zhang ◽  
...  
Keyword(s):  
Time Series ◽  
Sustainable Development ◽  
Large Scale ◽  
Management Practices ◽  
Conditional Random Field ◽  
Paddy Rice ◽  
Growth Patterns ◽  
Selection Strategy ◽  
Svm Classifier ◽  
Humid Climate

The elimination of hunger is the top concern for developing countries and is the key to maintain national stability and security. Paddy rice occupies an essential status in food supply, whose accurate monitoring is of great importance for human sustainable development. As one of the most important paddy rice production countries in the world, Thailand has a favorable hot and humid climate for paddy rice growing, but the growth patterns of paddy rice are too complicated to construct promising growth models for paddy rice discrimination. To solve this problem, this study proposes a large-scale paddy rice mapping scheme, which uses time-series Sentinel-1 data to generate a convincing annual paddy rice map of Thailand. The proposed method extracts temporal statistical features of the time-series SAR images to overcome the intra-class variability due to different management practices and modifies the U-Net model with the fully connected Conditional Random Field (CRF) to maintain the edge of the fields. In this study, 758 Sentinel-1 images that covered the whole country from the end of 2018 to 2019 were acquired to generate the annual paddy rice map. The accuracy, precision, and recall of the resultant paddy rice map reached 91%, 87%, and 95%, respectively. Compared to SVM classifier and the U-Net model based on feature selection strategy (FS-U-Net), the proposed scheme achieved the best overall performance, which demonstrated the capability of overcoming the complex cultivation conditions and accurately identifying the fragmented paddy rice fields in Thailand. This study provides a promising tool for large-scale paddy rice monitoring in tropical production regions and has great potential in the global sustainable development of food and environment management.

scholarly journals Analysis and Implementation  of Reinforcement Learning  on a GNU Radio Cognitive  Radio Platform

2021 ◽  
Author(s):  
◽  
Yu Ren
Keyword(s):  
Cognitive Radio ◽  
Success Rate ◽  
Large Scale ◽  
Channel Selection ◽  
Packet Transmission ◽  
Selection Strategy ◽  
Gnu Radio ◽  
Selection Scheme ◽  
Q Learning ◽  
Transmission Success

<p>Spectrum today is regulated based on fixed licensees. In the past radio operators have been allocated a frequency band for exclusive use. This has become problem for new users and the modern explosion in wireless services that, having arrived late find there is a scarcity in the remaining available spectrum. Cognitive radio (CR) presents a solution. CRs combine intelligence, spectrum sensing and software reconfigurable radio capabilities. This allows them to opportunistically transmit among several licensed bands for seamless communications, switching to another channel when a licensee is sensed in the original band without causing interference. Enabling this is an intelligent dynamic channel selection strategy capable of finding the best quality channel to transmit on that suffers from the least licensee interruption. This thesis evaluates a Q-learning channel selection scheme using an experimental approach. A cognitive radio deploying the scheme is implemented on GNU Radio and its performance is measured among channels with different utilizations in terms of its packet transmission success rate, goodput and interference caused. We derive similar analytical expressions in the general case of large-scale networks. Our results show that using the Q-learning scheme for channel selection significantly improves the goodput and packet transmission success rate of the system.</p>

Sign in / Sign up

Export Citation Format

Share Document