Dual-objective fine-tuning of BERT for entity matching

An increasing number of data providers have adopted shared numbering schemes such as GTIN, ISBN, DUNS, or ORCID numbers for identifying entities in the respective domain. This means for data integration that shared identifiers are often available for a subset of the entity descriptions to be integrated while such identifiers are not available for others. The challenge in these settings is to learn a matcher for entity descriptions without identifiers using the entity descriptions containing identifiers as training data. The task can be approached by learning a binary classifier which distinguishes pairs of entity descriptions for the same real-world entity from descriptions of different entities. The task can also be modeled as a multi-class classification problem by learning classifiers for identifying descriptions of individual entities. We present a dual-objective training method for BERT, called JointBERT, which combines binary matching and multi-class classification, forcing the model to predict the entity identifier for each entity description in a training pair in addition to the match/non-match decision. Our evaluation across five entity matching benchmark datasets shows that dual-objective training can increase the matching performance for seen products by 1% to 5% F1 compared to single-objective Transformer-based methods, given that enough training data is available for both objectives. In order to gain a deeper understanding of the strengths and weaknesses of the proposed method, we compare JointBERT to several other BERT-based matching methods as well as baseline systems along a set of specific matching challenges. This evaluation shows that JointBERT, given enough training data for both objectives, outperforms the other methods on tasks involving seen products, while it underperforms for unseen products. Using a combination of LIME explanations and domain-specific word classes, we analyze the matching decisions of the different deep learning models and conclude that BERT-based models are better at focusing on relevant word classes compared to RNN-based models.

Learning Deep Relations to Promote Saliency Detection

Though saliency detectors has made stunning progress recently. The performances of the state-of-the-art saliency detectors are not acceptable in some confusing areas, e.g., object boundary. We argue that the feature spatial independence should be one of the root cause. This paper explores the ubiquitous relations on the deep features to promote the existing saliency detectors efficiently. We establish the relation by maximizing the mutual information of the deep features of the same category via deep neural networks to break this independence. We introduce a threshold-constrained training pair construction strategy to ensure that we can accurately estimate the relations between different image parts in a self-supervised way. The relation can be utilized to further excavate the salient areas and inhibit confusing backgrounds. The experiments demonstrate that our method can significantly boost the performance of the state-of-the-art saliency detectors on various benchmark datasets. Besides, our model is label-free and extremely efficient. The inference speed is 140 FPS on a single GTX1080 GPU.

PD-GAN: Adversarial Learning for Personalized Diversity-Promoting Recommendation

This paper proposes Personalized Diversity-promoting GAN (PD-GAN), a novel recommendation model to generate diverse, yet relevant recommendations. Specifically, for each user, a generator recommends a set of diverse and relevant items by sequentially sampling from a personalized Determinantal Point Process (DPP) kernel matrix. This kernel matrix is constructed by two learnable components: the general co-occurrence of diverse items and the user's personal preference to items. To learn the first component, we propose a novel pairwise learning paradigm using training pairs, and each training pair consists of a set of diverse items and a set of similar items randomly sampled from the observed data of all users. The second component is learnt through adversarial training against a discriminator which strives to distinguish between recommended items and the ground-truth sets randomly sampled from the observed data of the target user. Experimental results show that PD-GAN is superior to generate recommendations that are both diverse and relevant.

Insufficient Data Can Also Rock! Learning to Converse Using Smaller Data with Augmentation

Recent successes of open-domain dialogue generation mainly rely on the advances of deep neural networks. The effectiveness of deep neural network models depends on the amount of training data. As it is laboursome and expensive to acquire a huge amount of data in most scenarios, how to effectively utilize existing data is the crux of this issue. In this paper, we use data augmentation techniques to improve the performance of neural dialogue models on the condition of insufficient data. Specifically, we propose a novel generative model to augment existing data, where the conditional variational autoencoder (CVAE) is employed as the generator to output more training data with diversified expressions. To improve the correlation of each augmented training pair, we design a discriminator with adversarial training to supervise the augmentation process. Moreover, we thoroughly investigate various data augmentation schemes for neural dialogue system with generative models, both GAN and CVAE. Experimental results on two open corpora, Weibo and Twitter, demonstrate the superiority of our proposed data augmentation model.

A Novel Fuzzy ARTMAP Architecture with Adaptive FeatureWeights based on Onicescu’s Informational Energy

<p>Fuzzy ARTMAP with Relevance factor (FAMR) is a Fuzzy ARTMAP (FAM) neural architecture with the following property: Each training pair has a relevance factor assigned to it, proportional to the importance of that pair during the learning phase. Using a relevance factor adds more flexibility to the training phase, allowing ranking of sample pairs according to the confidence we have in the information source or in the pattern itself. <br />We introduce a novel FAMR architecture: FAMR with Feature Weighting (FAMRFW). In the first stage, the training data features are weighted. In our experiments, we use a feature weighting method based on Onicescu’s informational energy (IE). In the second stage, the obtained weights are used to improve FAMRFW training. The effect of this approach is that category dimensions in the direction of relevant features are decreased, whereas category dimensions in the direction of non-relevant feature are increased. Experimental results, performed on several benchmarks, show that feature weighting can improve the classification performance of the general FAMR algorithm.</p>

Study of Optimized Steel Truss Design Using Neural Network to Resist Lateral Loads

Structural optimization is widely adopted in the design of structures with the development of computer aided design (CAD) and the development of computer technique recently. By applying the artificial neural network to structural optimization, designers can gain the design scheme of structures more feasibly and easily. In this paper, the genetic algorithm (GA) used in the error back-propagation (BP) network is applied to get the optimization result of the structural system. And the training pair of BP neural network is obtained from the structural analysis using a finite element program. The case study of 10 member truss structure using GA and BP will be helpful to reduce the cost of structures which is related to weight and the dynamic performance of optimization under the lateral load.

EQUILIBRIUM AND ATTRACTIVITY ANALYSIS FOR A CLASS OF HETERO-ASSOCIATIVE NEURAL MEMORIES

Based on the natural structure of Kosko’s Bidirectional Associative Memories (BAM), a high-performance, high-capacity associative neural model is proposed which is capable of simultaneous hetero-associative recall. The proposed model, Modified Bidirectional Decoding Strategy (MBDS), improves the recall rate by adding some association fascicles to Kosko’s BAM. The association fascicles are sparse coding neuron structures that provide activating strengths between two neuron fields (say, field X and field Y). The sufficient conditions for a state to become an equilibrium state of the MBDS network is derived. Based on these results, we discuss the basins of attraction of the training pairs in one iteration. The upper bound of the number of error bits which can be tolerated by MBDS is also derived. Because the attractivity of a stored training pair can be increased markedly with the aid of its corresponding association fascicles, we recommend a high capacity realization of MBDS, Bidirectional Holographic Memory (BHM), so that each training pair is stored uniquely and directly in the connection weights rather than encoded in a correlation matrix. Finally, computer simulations demonstrate the attractiveness of three different realizations of MBDS to verify our results.

A Comparison of Three Strategies of Reinforcement on Two-Choice Learning Problems with Severely Retarded Children

Three severely retarded children were given training on several two-choice discrimination problems. Each discrimination problem consisted of displaying two ordinary children's toys and asking the child to point to the toy named by the experimenter. The names consisted of three-letter (consonant-vowel-consonant) nonsense syllables. Training on each problem was conducted under one of three different reinforcement conditions. In the condition labeled “specific reinforcement,” correct choices were followed by the opportunity to play with the toy to which the child pointed. Under the “nonspecific reinforcement” condition, correct responses were followed by the opportunity to play with a toy offered by the experimenter, but which was not a part of the training pair. In the “variable reinforcement” condition, the child was offered, on correct pointing occasions, either the toy to which he or she pointed, or the toy which was not one of the pair being trained, in random order. The results indicated the level of correct responding was higher under the “specific reinforcement” condition than under either of the two other conditions, even when the latter were “weighted” for strength of reinforcement potential. It was concluded that a strategy of reinforcement which includes cue values of the discriminative stimulus may strengthen the learning process.