scholarly journals BESA: BERT-based Simulated Annealing for Adversarial Text Attacks

2021 ◽  
Author(s):  
Xinghao Yang ◽  
Weifeng Liu ◽  
Dacheng Tao ◽  
Wei Liu
Keyword(s):  
Simulated Annealing ◽  
Language Processing ◽  
Simulated Annealing Algorithm ◽  
Language Model ◽  
Optimization Methods ◽  
Local Optima ◽  
Word Level ◽  
Modern Natural ◽  
Global Optima ◽  
Adversarial Examples

Modern Natural Language Processing (NLP) models are known immensely brittle towards text adversarial examples. Recent attack algorithms usually adopt word-level substitution strategies following a pre-computed word replacement mechanism. However, their resultant adversarial examples are still imperfect in achieving grammar correctness and semantic similarities, which is largely because of their unsuitable candidate word selections and static optimization methods. In this research, we propose BESA, a BERT-based Simulated Annealing algorithm, to address these two problems. Firstly, we leverage the BERT Masked Language Model (MLM) to generate contextual-aware candidate words to produce fluent adversarial text and avoid grammar errors. Secondly, we employ Simulated Annealing (SA) to adaptively determine the word substitution order. The SA provides sufficient word replacement options via internal simulations, with an objective to obtain both a high attack success rate and a low word substitution rate. Besides, our algorithm is able to jump out of local optima with a controlled probability, making it closer to achieve the best possible attack (i.e., the global optima). Experiments on five popular datasets manifest the superiority of BESA compared with existing methods, including TextFooler, BAE, BERT-Attack, PWWS, and PSO.

scholarly journals Multi Objective Simulated Annealing Approach for Facility Layout Design

2018 ◽  
Vol 3 (4) ◽  
pp. 365-380 ◽  
Author(s):  
Safiye Turgay
Keyword(s):  
Simulated Annealing ◽  
Objective Function ◽  
Material Handling ◽  
Simulated Annealing Algorithm ◽  
Facility Layout ◽  
Layout Design ◽  
Local Optima ◽  
Facility Layout Design ◽  
Multi Objective ◽  
Annealing Algorithm

Facility layout design problem considers the departments’ physcial layout design with area requirements in some restrictions such as material handling costs, remoteness and distance requests. Briefly, facility layout problem related to optimization of the layout costs and working conditions. This paper proposes a new multi objective simulated annealing algorithm for solving of the unequal area in layout design. Using of the different objective weights are generated with entropy approach and used in the alternative layout design. Multi objective function takes into the objective function and constraints. The suggested heuristic algorithm used the multi-objective parameters for initialization. Then prefered the entropy approach determines the weight of the objective functions. After the suggested improved simulated annealing approach applied to whole developed model. A multi-objective simulated annealing algorithm is implemented to increase the diversity and reduce the chance of getting layout conditions in local optima.

An Improved ICA Algorithm Based on the Negative Entropy and Simulated Annealing Algorithm

2013 ◽  
Vol 411-414 ◽  
pp. 1125-1128 ◽  
Author(s):  
Hong Yi Li ◽  
Meng Ye ◽  
Di Zhao
Keyword(s):  
Simulated Annealing ◽  
Independent Component Analysis ◽  
Blind Source Separation ◽  
Simulated Annealing Algorithm ◽  
Local Optima ◽  
Negative Entropy ◽  
Wide Range ◽  
Annealing Algorithm ◽  
Non Gaussian ◽  
Ica Algorithm

The Independent Component Analysis (ICA) is a classical algorithm for exploring statistically independent non-Gaussian signals from multi-dimensional data, which has a wide range of applications in engineering, for instance, the blind source separation. The classical ICA measures the Gaussian characteristic by kurtosis, which has the following two disadvantages. Firstly, the kurtosis relies on the value of samples, and is not robust to outliers. Secondly, the algorithm often falls into local optima. To address these drawbacks, we replace the kurtosis by negative entropy, utilize the simulated annealing algorithm for optimization, and finally propose an improved ICA algorithm. Experimental results demonstrate that the proposed algorithm outperforms the classical ICA in its robustness to outliers and convergent rate.

scholarly journals Textual Adversarial Attacking with Limited Queries

Electronics ◽  
2021 ◽  
Vol 10 (21) ◽  
pp. 2671
Author(s):  
Yu Zhang ◽  
Junan Yang ◽  
Xiaoshuai Li ◽  
Hui Liu ◽  
Kun Shao
Keyword(s):  
Language Processing ◽  
Main Idea ◽  
Local Model ◽  
Small Perturbations ◽  
Target Model ◽  
Word Level ◽  
Sentence Level ◽  
Adversarial Examples ◽  
Reducing Costs ◽  
The Cost

Recent studies have shown that natural language processing (NLP) models are vulnerable to adversarial examples, which are maliciously designed by adding small perturbations to benign inputs that are imperceptible to the human eye, leading to false predictions by the target model. Compared to character- and sentence-level textual adversarial attacks, word-level attack can generate higher-quality adversarial examples, especially in a black-box setting. However, existing attack methods usually require a huge number of queries to successfully deceive the target model, which is costly in a real adversarial scenario. Hence, finding appropriate models is difficult. Therefore, we propose a novel attack method, the main idea of which is to fully utilize the adversarial examples generated by the local model and transfer part of the attack to the local model to complete ahead of time, thereby reducing costs related to attacking the target model. Extensive experiments conducted on three public benchmarks show that our attack method can not only improve the success rate but also reduce the cost, while outperforming the baselines by a significant margin.

Evolutionary Optimization on Turning Linkage of Automobile in Simulated Annealing

2010 ◽  
Vol 34-35 ◽  
pp. 317-321
Author(s):  
Feng Chen ◽  
Jiang Zhu
Keyword(s):  
Simulated Annealing ◽  
Simulated Annealing Algorithm ◽  
Optimal Solution ◽  
Optimization Methods ◽  
Main Function ◽  
Turn Angle ◽  
Boundary Constraints ◽  
Steering Characteristics ◽  
Relationship Of ◽  
The Ideal

The main function of turning linkage of automobile is to realize the ideal relations of turn angle of the internal and external wheels when vehicles steering. At present the main methods on design computing and verifying turning linkage have still been the planar graphing and analysis method, therefore it is very important to adopt optimization methods to design the steering linkage. Being satisfied with the Ackerman theory steering characteristics and boundary constraints, considering the ideal relationship of steering angles between external and internal wheels in steering linkage to ensure motion accuracy of automobile, optimization model of turning linkage is established. Then the objective function with penalty terms is built by penalty strategy with addition type, so the constrained optimization is transformed into the unconstrained optimization. The simulated annealing algorithm is adopted to optimize turning linkage of automobile, so that optimization process was simplified and the global optimal solution is ensured reliably.

Material Parameter Identification of Cazacu’s Model for Ti6Al4V Using the Simulated Annealing Algorithm

2010 ◽  
Vol 636-637 ◽  
pp. 1125-1130 ◽  
Author(s):  
Gaëtan Gilles ◽  
Anne Marie Habraken ◽  
Laurent Duchêne
Keyword(s):  
Simulated Annealing ◽  
Parameter Identification ◽  
Simulated Annealing Algorithm ◽  
Optimization Method ◽  
Mechanical Tests ◽  
Global Optimum ◽  
Material Parameter Identification ◽  
Local Optima ◽  
Material Parameters ◽  
The One

Phenomenological yield criteria are generally described by many material parameters. A technique to identify these parameters is required to find the best fit to the results of the mechanical tests. The parameter identification by the classical simulated annealing technique is presented in this paper. This algorithm, based on works by Metropolis et al, is a global optimization method that distinguishes between different local optima to reach the global optimum. The anisotropic model used in this study is the one proposed by Cazacu et al. To prove the efficiency of the proposed algorithm, the material parameters of Ti6Al4V titanium alloy are identified and compared with those obtained using different identification procedures and the same experimental data.

The Merits of a Parallel Genetic Algorithm in Solving Hard Optimization Problems

2003 ◽  
Vol 125 (1) ◽  
pp. 141-146 ◽  
Author(s):  
A. J. Knoek van Soest ◽  
L. J. R. Richard Casius
Keyword(s):  
Genetic Algorithm ◽  
Simulated Annealing ◽  
Simulated Annealing Algorithm ◽  
Optimization Problems ◽  
Simplex Algorithm ◽  
Programming Algorithm ◽  
Parallel Genetic Algorithm ◽  
Local Optima ◽  
Optimal Region ◽  
Downhill Simplex

A parallel genetic algorithm for optimization is outlined, and its performance on both mathematical and biomechanical optimization problems is compared to a sequential quadratic programming algorithm, a downhill simplex algorithm and a simulated annealing algorithm. When high-dimensional non-smooth or discontinuous problems with numerous local optima are considered, only the simulated annealing and the genetic algorithm, which are both characterized by a weak search heuristic, are successful in finding the optimal region in parameter space. The key advantage of the genetic algorithm is that it can easily be parallelized at negligible overhead.

The Elastic Suspension Design Based on Optimized Design of Simulated Annealing Complex Method

2012 ◽  
Vol 538-541 ◽  
pp. 792-796
Author(s):  
Wei Hua Huang ◽  
Yu Peng He
Keyword(s):  
Simulated Annealing ◽  
Optimization Design ◽  
Simulated Annealing Algorithm ◽  
Optimization Methods ◽  
Complex Method ◽  
Optimized Design ◽  
Elastic Suspension ◽  
Compound Method ◽  
Annealing Algorithm ◽  
Searching Ability

As the global properties of traditional optimization algorithm was weak, the local solution of it appeared easily.The simulated annealing algorithm’s cooling rate was slower and he local searching ability was insufficient. Use the simulated annealing compound method that is combined of the simulated annealing algorithm and compound method. Research the optimization design of elastic suspension with the genetic compound method and compare with the traditional optimization methods. The results show that the algorithm is correct and effective in solving multi variable and constraint optimization problem of elastic suspension.

scholarly journals Towards Enhancement of Performance of K-Means Clustering Using Nature-Inspired Optimization Algorithms

2014 ◽  
Vol 2014 ◽  
pp. 1-16 ◽  
Author(s):  
Simon Fong ◽  
Suash Deb ◽  
Xin-She Yang ◽  
Yan Zhuang
Keyword(s):  
Clustering Algorithms ◽  
Optimization Algorithms ◽  
Optimization Methods ◽  
Global Optimum ◽  
Local Optima ◽  
Clustering Quality ◽  
Global Optima ◽  
Speed Up ◽  
Pros And Cons ◽  
Swarming Behavior

Traditional K-means clustering algorithms have the drawback of getting stuck at local optima that depend on the random values of initial centroids. Optimization algorithms have their advantages in guiding iterative computation to search for global optima while avoiding local optima. The algorithms help speed up the clustering process by converging into a global optimum early with multiple search agents in action. Inspired by nature, some contemporary optimization algorithms which include Ant, Bat, Cuckoo, Firefly, and Wolf search algorithms mimic the swarming behavior allowing them to cooperatively steer towards an optimal objective within a reasonable time. It is known that these so-called nature-inspired optimization algorithms have their own characteristics as well as pros and cons in different applications. When these algorithms are combined with K-means clustering mechanism for the sake of enhancing its clustering quality by avoiding local optima and finding global optima, the new hybrids are anticipated to produce unprecedented performance. In this paper, we report the results of our evaluation experiments on the integration of nature-inspired optimization methods into K-means algorithms. In addition to the standard evaluation metrics in evaluating clustering quality, the extended K-means algorithms that are empowered by nature-inspired optimization methods are applied on image segmentation as a case study of application scenario.

Simulated Annealing Optimization of Belt Conveyor Transmission

2010 ◽  
Vol 113-116 ◽  
pp. 2373-2378
Author(s):  
Ji Bin Ding
Keyword(s):  
Simulated Annealing ◽  
Objective Function ◽  
Optimization Model ◽  
Simulated Annealing Algorithm ◽  
Optimization Methods ◽  
Global Optimum ◽  
Local Optimum ◽  
Belt Conveyor ◽  
Boundary Constraints ◽  
Low Efficiency

The belt conveyor is a transporting machine by friction in a continuous manner. The two order helical gearing reducer may be generally used as conveyor transmission, and can reduce speed and increase torque of belt. The objective function may be specified that that total center distance of the reducer incline to minimum, so the optimization model including the property and boundary constraints is created. Then the objective function with penalty terms is converted by penalty strategy with addition type, so as to transform the constrained optimization into the unconstrained optimization model. Considering the problem of low efficiency and local optimum caused by standard optimization methods, the simulated annealing algorithm is adopted to solve the optimization model of Belt Conveyor Transmission, and neural network method is applied to fit relative coefficient, then BFGS Quasi-Newton method is recalled automatically when the setting working precision is achieved again. So that the optimization process is simplified and global optimum is acquired reliably.

Genetic Optimization Using Derivatives

1998 ◽  
Vol 7 ◽  
pp. 187-210 ◽  
Author(s):  
Jasjeet S. Sekhon ◽  
Walter R. Mebane
Keyword(s):  
Optimization Problems ◽  
Optimization Methods ◽  
Global Optimum ◽  
Genetic Optimization ◽  
Normal Mixture ◽  
Local Optima ◽  
Monte Carlo Experiments ◽  
Unconstrained Optimization Problems ◽  
Global Optima ◽  
Quasi Newton

We describe a new computer program that combines evolutionary algorithm methods with a derivative-based, quasi-Newton method to solve difficult unconstrained optimization problems. The program, called GENOUD (GENetic Optimization Using Derivatives), effectively solves problems that are nonlinear or perhaps even discontinuous in the parameters of the function to be optimized. When a statistical model's estimating function (for example, a log-likelihood) is nonlinear in the model's parameters, the function to be optimized will usually not be globally concave and may contain irregularities such as saddlepoints or discontinuous jumps. Optimization methods that rely on derivatives of the objective function may be unable to find any optimum at all. Or multiple local optima may exist, so that there is no guarantee that a derivative-based method will converge to the global optimum. We discuss the theoretical basis for expecting GENOUD to have a high probability of finding global optima. We conduct Monte Carlo experiments using scalar Normal mixture densities to illustrate this capability. We also use a system of four simultaneous nonlinear equations that has many parameters and multiple local optima to compare the performance of GENOUD to that of the Gauss-Newton algorithm in SAS's PROC MODEL.

Sign in / Sign up

Export Citation Format

Share Document