scholarly journals Textual Adversarial Attacking with Limited Queries

Electronics ◽  
2021 ◽  
Vol 10 (21) ◽  
pp. 2671
Author(s):  
Yu Zhang ◽  
Junan Yang ◽  
Xiaoshuai Li ◽  
Hui Liu ◽  
Kun Shao
Keyword(s):  
Language Processing ◽  
Main Idea ◽  
Local Model ◽  
Small Perturbations ◽  
Target Model ◽  
Word Level ◽  
Sentence Level ◽  
Adversarial Examples ◽  
Reducing Costs ◽  
The Cost

Recent studies have shown that natural language processing (NLP) models are vulnerable to adversarial examples, which are maliciously designed by adding small perturbations to benign inputs that are imperceptible to the human eye, leading to false predictions by the target model. Compared to character- and sentence-level textual adversarial attacks, word-level attack can generate higher-quality adversarial examples, especially in a black-box setting. However, existing attack methods usually require a huge number of queries to successfully deceive the target model, which is costly in a real adversarial scenario. Hence, finding appropriate models is difficult. Therefore, we propose a novel attack method, the main idea of which is to fully utilize the adversarial examples generated by the local model and transfer part of the attack to the local model to complete ahead of time, thereby reducing costs related to attacking the target model. Extensive experiments conducted on three public benchmarks show that our attack method can not only improve the success rate but also reduce the cost, while outperforming the baselines by a significant margin.

scholarly journals Textual Backdoor Defense via Poisoned Sample Recognition

2021 ◽  
Vol 11 (21) ◽  
pp. 9938
Author(s):  
Kun Shao ◽  
Yu Zhang ◽  
Junan Yang ◽  
Hui Liu
Keyword(s):  
Success Rate ◽  
Language Processing ◽  
Training Data ◽  
Infection Model ◽  
Search Range ◽  
Word Level ◽  
Sentence Level ◽  
Preliminary Model ◽  
Sample Recognition ◽  
Better Than

Deep learning models are vulnerable to backdoor attacks. The success rate of textual backdoor attacks based on data poisoning in existing research is as high as 100%. In order to enhance the natural language processing model’s defense against backdoor attacks, we propose a textual backdoor defense method via poisoned sample recognition. Our method consists of two parts: the first step is to add a controlled noise layer after the model embedding layer, and to train a preliminary model with incomplete or no backdoor embedding, which reduces the effectiveness of poisoned samples. Then, we use the model to initially identify the poisoned samples in the training set so as to narrow the search range of the poisoned samples. The second step uses all the training data to train an infection model embedded in the backdoor, which is used to reclassify the samples selected in the first step, and finally identify the poisoned samples. Through detailed experiments, we have proved that our defense method can effectively defend against a variety of backdoor attacks (character-level, word-level and sentence-level backdoor attacks), and the experimental effect is better than the baseline method. For the BERT model trained by the IMDB dataset, this method can even reduce the success rate of word-level backdoor attacks to 0%.

Chronological Ordering Based on Context Overlap Detection

2012 ◽  
Vol 2 (4) ◽  
pp. 31-44
Author(s):  
Mohamed H. Haggag ◽  
Bassma M. Othman
Keyword(s):  
Language Processing ◽  
Linguistic Knowledge ◽  
Syntactic Analysis ◽  
Text Generation ◽  
Context Processing ◽  
Domain Specific ◽  
Word Level ◽  
Sentence Level ◽  
Proposed Model ◽  
Chronological Sequence

Context processing plays an important role in different Natural Language Processing applications. Sentence ordering is one of critical tasks in text generation. Following the same order of sentences in the row sources of text is not necessarily to be applied for the resulted text. Accordingly, a need for chronological sentence ordering is of high importance in this regard. Some researches followed linguistic syntactic analysis and others used statistical approaches. This paper proposes a new model for sentence ordering based on sematic analysis. Word level semantics forms a seed to sentence level sematic relations. The model introduces a clustering technique based on sentences senses relatedness. Following to this, sentences are chronologically ordered through two main steps; overlap detection and chronological cause-effect rules. Overlap detection drills down into each cluster to step through its sentences in chronological sequence. Cause-effect rules forms the linguistic knowledge controlling sentences relations. Evaluation of the proposed algorithm showed the capability of the proposed model to process size free texts, non-domain specific and open to extend the cause-effect rules for specific ordering needs.

A longitudinal neuroimaging dataset on language processing in children ages 5, 7, and 9 years old.

2021 ◽  
Author(s):  
Jin Wang ◽  
Marisa N. Lytle ◽  
Yael Weiss ◽  
Brianna L. Yamasaki ◽  
James R. Booth
Keyword(s):  
Language Processing ◽  
Semantic Processing ◽  
Longitudinal Design ◽  
Word Level ◽  
Sentence Level ◽  
Multiple Imaging ◽  
Educational Assessments ◽  
Magnetic Resonance Imaging Mri ◽  
Changes Over Time ◽  
Brain Behavior

This dataset examines language development with a longitudinal design and includes diffusion- and T1-weighted structural magnetic resonance imaging (MRI), task-based functional MRI (fMRI), and a battery of psycho-educational assessments and parental questionnaires. We collected data from 5.5-6.5-year-old children (ses-5) and followed them up when they were 7-8 years old (ses-7) and then again at 8.5-10 years old (ses-9). To increase the sample size at the older time points, another cohort of 7-8-year-old children (ses-7) were recruited and followed up when they were 8.5-10 years old (ses-9). In total, 322 children who completed at least one structural and functional scan were included. Children performed four fMRI tasks consisting of two word-level tasks examining phonological and semantic processing and two sentence-level tasks investigating semantic and syntactic processing. The MRI data is valuable for examining changes over time in interactive specialization due to the use of multiple imaging modalities and tasks in this longitudinal design. In addition, the extensive psycho-educational assessments and questionnaires provide opportunities to explore brain-behavior and brain-environment associations.

A Fast Histogram-Based Postprocessor That Improves Posterior Probability Estimates

1999 ◽  
Vol 11 (5) ◽  
pp. 1235-1248 ◽  
Author(s):  
Wei Wei ◽  
Todd K. Leen ◽  
Etienne Barnard
Keyword(s):  
Recognition System ◽  
Error Rates ◽  
Training Data ◽  
Word Level ◽  
Probability Estimates ◽  
Sentence Level ◽  
Optimization Routine ◽  
Calibration Accuracy ◽  
Neural Network Classifiers ◽  
The Cost

Although the outputs of neural network classifiers are often considered to be estimates of posterior class probabilities, the literature that assesses the calibration accuracy of these estimates illustrates that practical networks often fall far short of being ideal estimators. The theorems used to justify treating network outputs as good posterior estimates are based on several assumptions: that the network is sufficiently complex to model the posterior distribution accurately, that there are sufficient training data to specify the network, and that the optimization routine is capable of finding the global minimum of the cost function. Any or all of these assumptions may be violated in practice. This article does three things. First, we apply a simple, previously used histogram technique to assess graphically the accuracy of posterior estimates with respect to individual classes. Second, we introduce a simple and fast remapping procedure that transforms network outputs to provide better estimates of posteriors. Third, we use the remapping in a real-world telephone speech recognition system. The remapping results in a 10% reduction of both word-level error rates (from 4.53% to 4.06%) and sentence-level error rates (from 16.38% to 14.69%) on one corpus, and a 29% reduction at sentence-level error (from 6.3% to 4.5%) on another. The remapping required negligible additional overhead (in terms of both parameters and calculations). McNemar's test shows that these levels of improvement are statistically significant.

scholarly journals Big Data Management and Analytics in Scientific Programming: A Deep Learning-Based Method for Aspect Category Classification of Question-Answering-Style Reviews

2020 ◽  
Vol 2020 ◽  
pp. 1-10 ◽  
Author(s):  
Hanqian Wu ◽  
Mumu Liu ◽  
Shangbin Zhang ◽  
Zhike Wang ◽  
Siliang Cheng
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Question Answering ◽  
Product Information ◽  
Empirical Studies ◽  
Product Reviews ◽  
Related Information ◽  
Basic Task ◽  
Word Level ◽  
Sentence Level

Online product reviews are exploring on e-commerce platforms, and mining aspect-level product information contained in those reviews has great economic benefit. The aspect category classification task is a basic task for aspect-level sentiment analysis which has become a hot research topic in the natural language processing (NLP) field during the last decades. In various e-commerce platforms, there emerge various user-generated question-answering (QA) reviews which generally contain much aspect-related information of products. Although some researchers have devoted their efforts on the aspect category classification for traditional product reviews, the existing deep learning-based approaches cannot be well applied to represent the QA-style reviews. Thus, we propose a 4-dimension (4D) textual representation model based on QA interaction-level and hyperinteraction-level by modeling with different levels of the text representation, i.e., word-level, sentence-level, QA interaction-level, and hyperinteraction-level. In our experiments, the empirical studies on datasets from three domains demonstrate that our proposals perform better than traditional sentence-level representation approaches, especially in the Digit domain.

scholarly journals A longitudinal neuroimaging dataset on language processing in children ages 5, 7, and 9 years old

2022 ◽  
Vol 9 (1) ◽  
Author(s):  
Jin Wang ◽  
Marisa N. Lytle ◽  
Yael Weiss ◽  
Brianna L. Yamasaki ◽  
James R. Booth
Keyword(s):  
Language Processing ◽  
Semantic Processing ◽  
Longitudinal Design ◽  
Word Level ◽  
Sentence Level ◽  
Multiple Imaging ◽  
Educational Assessments ◽  
Magnetic Resonance Imaging Mri ◽  
Changes Over Time ◽  
Brain Behavior

AbstractThis dataset examines language development with a longitudinal design and includes diffusion- and T1-weighted structural magnetic resonance imaging (MRI), task-based functional MRI (fMRI), and a battery of psycho-educational assessments and parental questionnaires. We collected data from 5.5-6.5-year-old children (ses-5) and followed them up when they were 7-8 years old (ses-7) and then again at 8.5-10 years old (ses-9). To increase the sample size at the older time points, another cohort of 7-8-year-old children (ses-7) were recruited and followed up when they were 8.5–10 years old (ses-9). In total, 322 children who completed at least one structural and functional scan were included. Children performed four fMRI tasks consisting of two word-level tasks examining phonological and semantic processing and two sentence-level tasks investigating semantic and syntactic processing. The MRI data is valuable for examining changes over time in interactive specialization due to the use of multiple imaging modalities and tasks in this longitudinal design. In addition, the extensive psycho-educational assessments and questionnaires provide opportunities to explore brain-behavior and brain-environment associations.

scholarly journals A Black-Box Attack Method against Machine-Learning-Based Anomaly Network Flow Detection Models

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Sensen Guo ◽  
Jinxiong Zhao ◽  
Xiaoyu Li ◽  
Junhong Duan ◽  
Dejun Mu ◽  
...  
Keyword(s):  
Machine Learning ◽  
Language Processing ◽  
Network Flow ◽  
Black Box ◽  
Target Model ◽  
Detection Algorithms ◽  
Machine Learning Model ◽  
Tremendous Progress ◽  
Adversarial Examples ◽  
Flow Detection

In recent years, machine learning has made tremendous progress in the fields of computer vision, natural language processing, and cybersecurity; however, we cannot ignore that machine learning models are vulnerable to adversarial examples, with some minor malicious input modifications, while appearing unmodified to human observers, the outputs of machine learning-based model can be misled easily. Likewise, attackers can bypass machine-learning-based security defenses model to attack systems in real time by generating adversarial examples. In this paper, we propose a black-box attack method against machine-learning-based anomaly network flow detection algorithms. Our attack strategy consists in training another model to substitute for the target machine learning model. Based on the overall understanding of the substitute model and the migration of the adversarial examples, we use the substitute model to craft adversarial examples. The experiment has shown that our method can attack the target model effectively. We attack several kinds of network flow detection models, which are based on different kinds of machine learning methods, and we find that the adversarial examples crafted by our method can bypass the detection of the target model with high probability.

scholarly journals CRank: Reusable Word Importance Ranking for Text Adversarial Attack

2021 ◽  
Vol 11 (20) ◽  
pp. 9570
Author(s):  
Xinyi Chen ◽  
Bo Liu
Keyword(s):  
Language Processing ◽  
Network Models ◽  
Neural Network Models ◽  
Ranking Strategy ◽  
Adversarial Examples ◽  
Importance Ranking ◽  
Adversarial Attack ◽  
Box Method ◽  
Low Efficiency ◽  
The Cost

Deep learning models have been widely used in natural language processing tasks, yet researchers have recently proposed several methods to fool the state-of-the-art neural network models. Among these methods, word importance ranking is an essential part that generates text adversarial examples, but suffers from low efficiency for practical attacks. To address this issue, we aim to improve the efficiency of word importance ranking, making steps towards realistic text adversarial attacks. In this paper, we propose CRank, a black box method utilized by our innovated masking and ranking strategy. CRank improves efficiency by 75% at the ’cost’ of only a 1% drop of the success rate when compared to the classic method. Moreover, we explore a new greedy search strategy and Unicode perturbation methods.

scholarly journals Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input

2019 ◽  
Vol 33 ◽  
pp. 3723-3730 ◽  
Author(s):  
Junliang Guo ◽  
Xu Tan ◽  
Di He ◽  
Tao Qin ◽  
Linli Xu ◽  
...  
Keyword(s):  
Machine Translation ◽  
Experimental Results ◽  
Word Embeddings ◽  
Model Accuracy ◽  
Neural Machine Translation ◽  
Word Level ◽  
Sentence Level ◽  
The Cost ◽  
Target Side

Non-autoregressive translation (NAT) models, which remove the dependence on previous target tokens from the inputs of the decoder, achieve significantly inference speedup but at the cost of inferior accuracy compared to autoregressive translation (AT) models. Previous work shows that the quality of the inputs of the decoder is important and largely impacts the model accuracy. In this paper, we propose two methods to enhance the decoder inputs so as to improve NAT models. The first one directly leverages a phrase table generated by conventional SMT approaches to translate source tokens to target tokens, which are then fed into the decoder as inputs. The second one transforms source-side word embeddings to target-side word embeddings through sentence-level alignment and word-level adversary learning, and then feeds the transformed word embeddings into the decoder as inputs. Experimental results show our method largely outperforms the NAT baseline (Gu et al. 2017) by 5.11 BLEU scores on WMT14 English-German task and 4.72 BLEU scores on WMT16 English-Romanian task.

scholarly journals BESA: BERT-based Simulated Annealing for Adversarial Text Attacks

2021 ◽  
Author(s):  
Xinghao Yang ◽  
Weifeng Liu ◽  
Dacheng Tao ◽  
Wei Liu
Keyword(s):  
Simulated Annealing ◽  
Language Processing ◽  
Simulated Annealing Algorithm ◽  
Language Model ◽  
Optimization Methods ◽  
Local Optima ◽  
Word Level ◽  
Modern Natural ◽  
Global Optima ◽  
Adversarial Examples

Modern Natural Language Processing (NLP) models are known immensely brittle towards text adversarial examples. Recent attack algorithms usually adopt word-level substitution strategies following a pre-computed word replacement mechanism. However, their resultant adversarial examples are still imperfect in achieving grammar correctness and semantic similarities, which is largely because of their unsuitable candidate word selections and static optimization methods. In this research, we propose BESA, a BERT-based Simulated Annealing algorithm, to address these two problems. Firstly, we leverage the BERT Masked Language Model (MLM) to generate contextual-aware candidate words to produce fluent adversarial text and avoid grammar errors. Secondly, we employ Simulated Annealing (SA) to adaptively determine the word substitution order. The SA provides sufficient word replacement options via internal simulations, with an objective to obtain both a high attack success rate and a low word substitution rate. Besides, our algorithm is able to jump out of local optima with a controlled probability, making it closer to achieve the best possible attack (i.e., the global optima). Experiments on five popular datasets manifest the superiority of BESA compared with existing methods, including TextFooler, BAE, BERT-Attack, PWWS, and PSO.

Sign in / Sign up

Export Citation Format

Share Document