scholarly journals CRank: Reusable Word Importance Ranking for Text Adversarial Attack

2021 ◽  
Vol 11 (20) ◽  
pp. 9570
Author(s):  
Xinyi Chen ◽  
Bo Liu
Keyword(s):  
Language Processing ◽  
Network Models ◽  
Neural Network Models ◽  
Ranking Strategy ◽  
Adversarial Examples ◽  
Importance Ranking ◽  
Adversarial Attack ◽  
Box Method ◽  
Low Efficiency ◽  
The Cost

Deep learning models have been widely used in natural language processing tasks, yet researchers have recently proposed several methods to fool the state-of-the-art neural network models. Among these methods, word importance ranking is an essential part that generates text adversarial examples, but suffers from low efficiency for practical attacks. To address this issue, we aim to improve the efficiency of word importance ranking, making steps towards realistic text adversarial attacks. In this paper, we propose CRank, a black box method utilized by our innovated masking and ranking strategy. CRank improves efficiency by 75% at the ’cost’ of only a 1% drop of the success rate when compared to the classic method. Moreover, we explore a new greedy search strategy and Unicode perturbation methods.

scholarly journals Analysis Methods in Neural Language Processing: A Survey

2019 ◽  
Vol 7 ◽  
pp. 49-72 ◽  
Author(s):  
Yonatan Belinkov ◽  
James Glass
Keyword(s):  
Language Processing ◽  
Network Models ◽  
Research Trends ◽  
Neural Network Models ◽  
Fine Grained ◽  
Survey Paper ◽  
Review Analysis ◽  
Analysis Methods ◽  
New Models ◽  
Future Work

The field of natural language processing has seen impressive progress in recent years, with neural network models replacing many of the traditional systems. A plethora of new models have been proposed, many of which are thought to be opaque compared to their feature-rich counterparts. This has led researchers to analyze, interpret, and evaluate neural networks in novel and more fine-grained ways. In this survey paper, we review analysis methods in neural language processing, categorize them according to prominent research trends, highlight existing limitations, and point to potential directions for future work.

scholarly journals Comparison of rule-based and neural network models for negation detection in radiology reports

2020 ◽  
pp. 1-22 ◽  
Author(s):  
D. Sykes ◽  
A. Grivas ◽  
C. Grover ◽  
R. Tobin ◽  
C. Sudlow ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Language Processing ◽  
Network Models ◽  
Neural Network Models ◽  
Test Set ◽  
Rule Based ◽  
Radiology Reports ◽  
The Neural Network ◽  
Negation Detection

Abstract Using natural language processing, it is possible to extract structured information from raw text in the electronic health record (EHR) at reasonably high accuracy. However, the accurate distinction between negated and non-negated mentions of clinical terms remains a challenge. EHR text includes cases where diseases are stated not to be present or only hypothesised, meaning a disease can be mentioned in a report when it is not being reported as present. This makes tasks such as document classification and summarisation more difficult. We have developed the rule-based EdIE-R-Neg, part of an existing text mining pipeline called EdIE-R (Edinburgh Information Extraction for Radiology reports), developed to process brain imaging reports, (https://www.ltg.ed.ac.uk/software/edie-r/) and two machine learning approaches; one using a bidirectional long short-term memory network and another using a feedforward neural network. These were developed on data from the Edinburgh Stroke Study (ESS) and tested on data from routine reports from NHS Tayside (Tayside). Both datasets consist of written reports from medical scans. These models are compared with two existing rule-based models: pyConText (Harkema et al. 2009. Journal of Biomedical Informatics42(5), 839–851), a python implementation of a generalisation of NegEx, and NegBio (Peng et al. 2017. NegBio: A high-performance tool for negation and uncertainty detection in radiology reports. arXiv e-prints, p. arXiv:1712.05898), which identifies negation scopes through patterns applied to a syntactic representation of the sentence. On both the test set of the dataset from which our models were developed, as well as the largely similar Tayside test set, the neural network models and our custom-built rule-based system outperformed the existing methods. EdIE-R-Neg scored highest on F1 score, particularly on the test set of the Tayside dataset, from which no development data were used in these experiments, showing the power of custom-built rule-based systems for negation detection on datasets of this size. The performance gap of the machine learning models to EdIE-R-Neg on the Tayside test set was reduced through adding development Tayside data into the ESS training set, demonstrating the adaptability of the neural network models.

scholarly journals Textual Adversarial Attacking with Limited Queries

Electronics ◽  
2021 ◽  
Vol 10 (21) ◽  
pp. 2671
Author(s):  
Yu Zhang ◽  
Junan Yang ◽  
Xiaoshuai Li ◽  
Hui Liu ◽  
Kun Shao
Keyword(s):  
Language Processing ◽  
Main Idea ◽  
Local Model ◽  
Small Perturbations ◽  
Target Model ◽  
Word Level ◽  
Sentence Level ◽  
Adversarial Examples ◽  
Reducing Costs ◽  
The Cost

Recent studies have shown that natural language processing (NLP) models are vulnerable to adversarial examples, which are maliciously designed by adding small perturbations to benign inputs that are imperceptible to the human eye, leading to false predictions by the target model. Compared to character- and sentence-level textual adversarial attacks, word-level attack can generate higher-quality adversarial examples, especially in a black-box setting. However, existing attack methods usually require a huge number of queries to successfully deceive the target model, which is costly in a real adversarial scenario. Hence, finding appropriate models is difficult. Therefore, we propose a novel attack method, the main idea of which is to fully utilize the adversarial examples generated by the local model and transfer part of the attack to the local model to complete ahead of time, thereby reducing costs related to attacking the target model. Extensive experiments conducted on three public benchmarks show that our attack method can not only improve the success rate but also reduce the cost, while outperforming the baselines by a significant margin.

scholarly journals Image Caption Generator

2021 ◽  
Vol 10 (3) ◽  
pp. 87-92
Author(s):  
Megha J Panicker ◽  
Vikas Upadhayay ◽  
Gunjan Sethi ◽  
Vrinda Mathur
Keyword(s):  
Computer Vision ◽  
Language Processing ◽  
Network Models ◽  
Image Captioning ◽  
Neural Network Models ◽  
Visually Impaired People ◽  
Video Frames ◽  
Modern Era ◽  
Deep Learning Model ◽  
Image Caption

In the modern era, image captioning has become one of the most widely required tools. Moreover, there are inbuilt applications that generate and provide a caption for a certain image, all these things are done with the help of deep neural network models. The process of generating a description of an image is called image captioning. It requires recognizing the important objects, their attributes, and the relationships among the objects in an image. It generates syntactically and semantically correct sentences.In this paper, we present a deep learning model to describe images and generate captions using computer vision and machine translation. This paper aims to detect different objects found in an image, recognize the relationships between those objects and generate captions. The dataset used is Flickr8k and the programming language used was Python3, and an ML technique called Transfer Learning will be implemented with the help of the Xception model, to demonstrate the proposed experiment. This paper will also elaborate on the functions and structure of the various Neural networks involved. Generating image captions is an important aspect of Computer Vision and Natural language processing. Image caption generators can find applications in Image segmentation as used by Facebook and Google Photos, and even more so, its use can be extended to video frames. They will easily automate the job of a person who has to interpret images. Not to mention it has immense scope in helping visually impaired people.

scholarly journals A Primer on Neural Network Models for Natural Language Processing

2016 ◽  
Vol 57 ◽  
pp. 345-420 ◽  
Author(s):  
Yoav Goldberg
Keyword(s):  
Neural Network ◽  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Speech Processing ◽  
Network Models ◽  
Neural Network Models ◽  
Convolutional Networks ◽  
The Past ◽  
Gradient Computation

Over the past few years, neural networks have re-emerged as powerful machine-learning models, yielding state-of-the-art results in fields such as image recognition and speech processing. More recently, neural network models started to be applied also to textual natural language signals, again with very promising results. This tutorial surveys neural network models from the perspective of natural language processing research, in an attempt to bring natural-language researchers up to speed with the neural techniques. The tutorial covers input encoding for natural language tasks, feed-forward networks, convolutional networks, recurrent networks and recursive networks, as well as the computation graph abstraction for automatic gradient computation.

A Novel Hybrid Model Using RBF and PSO for Net Asset Value Prediction

2018 ◽  
pp. 54-72
Author(s):  
C. M. Anish ◽  
Babita Majhi ◽  
Ritanjali Majhi
Keyword(s):  
Neural Network ◽  
Network Models ◽  
Performance Measure ◽  
Neural Network Models ◽  
Proposed Model ◽  
Derivative Free ◽  
Asset Value ◽  
Net Asset Value ◽  
Small Investors ◽  
The Cost

Net asset value (NAV) prediction is an important area of research as small investors are doing investment in there, Literature survey reveals that very little work has been done in this field. The reported literature mainly used various neural network models for NAV prediction. But the derivative based learning algorithms of these reported models have the problem of trapping into the local solution. Hence in chapter derivative free algorithm, particle swarm optimization is used to update the parameters of radial basis function neural network for prediction of NAV. The positions of particles represent the centers, spreads and weights of the RBF model and the minimum MSE is used as the cost function. The convergence characteristics are obtained to show the performance of the model during training phase. The MAPE and RMSE value are calculated during testing phase to show the performance of the proposed RBF-PSO model. These performance measure exhibits that the proposed model is better model in comparison to MLANN, FLANN and RBFNN models.

A Novel Hybrid Model Using RBF and PSO for Net Asset Value Prediction

2018 ◽  
pp. 1031-1049 ◽  
Author(s):  
C. M. Anish ◽  
Babita Majhi ◽  
Ritanjali Majhi
Keyword(s):  
Neural Network ◽  
Network Models ◽  
Performance Measure ◽  
Neural Network Models ◽  
Proposed Model ◽  
Derivative Free ◽  
Asset Value ◽  
Net Asset Value ◽  
Small Investors ◽  
The Cost

Net asset value (NAV) prediction is an important area of research as small investors are doing investment in there, Literature survey reveals that very little work has been done in this field. The reported literature mainly used various neural network models for NAV prediction. But the derivative based learning algorithms of these reported models have the problem of trapping into the local solution. Hence in chapter derivative free algorithm, particle swarm optimization is used to update the parameters of radial basis function neural network for prediction of NAV. The positions of particles represent the centers, spreads and weights of the RBF model and the minimum MSE is used as the cost function. The convergence characteristics are obtained to show the performance of the model during training phase. The MAPE and RMSE value are calculated during testing phase to show the performance of the proposed RBF-PSO model. These performance measure exhibits that the proposed model is better model in comparison to MLANN, FLANN and RBFNN models.

Sign in / Sign up

Export Citation Format

Share Document