scholarly journals Word-Level Uncertainty Estimation for Black-Box Text Classifiers using RNNs

2020 ◽  
Author(s):  
Jakob Smedegaard Andersen ◽  
Tom Schöner ◽  
Walid Maalej
Keyword(s):  
Black Box ◽  
Uncertainty Estimation ◽  
Word Level ◽  
Text Classifiers

scholarly journals Deep Text Classification Can be Fooled

2018 ◽  
Author(s):  
Bin Liang ◽  
Hongcheng Li ◽  
Miaoqiang Su ◽  
Pan Bian ◽  
Xirong Li ◽  
...  
Keyword(s):  
Text Classification ◽  
State Of The Art ◽  
Black Box ◽  
Word Level ◽  
Text Classifiers ◽  
The Cost

In this paper, we present an effective method to craft text adversarial samples, revealing one important yet underestimated fact that DNN-based text classifiers are also prone to adversarial sample attack. Specifically, confronted with different adversarial scenarios, the text items that are important for classification are identified by computing the cost gradients of the input (white-box attack) or generating a series of occluded test samples (black-box attack). Based on these items, we design three perturbation strategies, namely insertion, modification, and removal, to generate adversarial samples. The experiment results show that the adversarial samples generated by our method can successfully fool both state-of-the-art character-level and word-level DNN-based text classifiers. The adversarial samples can be perturbed to any desirable classes without compromising their utilities. At the same time, the introduced perturbation is difficult to be perceived.

scholarly journals Sparsity Increases Uncertainty Estimation in Deep Ensemble

Computers ◽  
2021 ◽  
Vol 10 (4) ◽  
pp. 54
Author(s):  
Uyanga Dorjsembe ◽  
Ju Hong Lee ◽  
Bumghi Choi ◽  
Jae Won Song
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Deep Learning ◽  
Energy Efficient ◽  
Deep Neural Networks ◽  
Black Box ◽  
Uncertainty Estimation ◽  
Point Estimation ◽  
On Demand

Deep neural networks have achieved almost human-level results in various tasks and have become popular in the broad artificial intelligence domains. Uncertainty estimation is an on-demand task caused by the black-box point estimation behavior of deep learning. The deep ensemble provides increased accuracy and estimated uncertainty; however, linearly increasing the size makes the deep ensemble unfeasible for memory-intensive tasks. To address this problem, we used model pruning and quantization with a deep ensemble and analyzed the effect in the context of uncertainty metrics. We empirically showed that the ensemble members’ disagreement increases with pruning, making models sparser by zeroing irrelevant parameters. Increased disagreement im-plies increased uncertainty, which helps in making more robust predictions. Accordingly, an energy-efficient compressed deep ensemble is appropriate for memory-intensive and uncertainty-aware tasks.

scholarly journals A Hybrid Adversarial Attack for Different Application Scenarios

2020 ◽  
Vol 10 (10) ◽  
pp. 3559 ◽  
Author(s):  
Xiaohu Du ◽  
Jie Yu ◽  
Zibo Yi ◽  
Shasha Li ◽  
Jun Ma ◽  
...  
Keyword(s):  
Deep Learning ◽  
Success Rate ◽  
Black Box ◽  
De Algorithm ◽  
Word Level ◽  
Text Readability ◽  
Gradient Based ◽  
Adversarial Examples ◽  
Adversarial Attack ◽  
Cosine Distance

Adversarial attack against natural language has been a hot topic in the field of artificial intelligence security in recent years. It is mainly to study the methods and implementation of generating adversarial examples. The purpose is to better deal with the vulnerability and security of deep learning systems. According to whether the attacker understands the deep learning model structure, the adversarial attack is divided into black-box attack and white-box attack. In this paper, we propose a hybrid adversarial attack for different application scenarios. Firstly, we propose a novel black-box attack method of generating adversarial examples to trick the word-level sentiment classifier, which is based on differential evolution (DE) algorithm to generate semantically and syntactically similar adversarial examples. Compared with existing genetic algorithm based adversarial attacks, our algorithm can achieve a higher attack success rate while maintaining a lower word replacement rate. At the 10% word substitution threshold, we have increased the attack success rate from 58.5% to 63%. Secondly, when we understand the model architecture and parameters, etc., we propose a white-box attack with gradient-based perturbation against the same sentiment classifier. In this attack, we use a Euclidean distance and cosine distance combined metric to find the most semantically and syntactically similar substitution, and we introduce the coefficient of variation (CV) factor to control the dispersion of the modified words in the adversarial examples. More dispersed modifications can increase human imperceptibility and text readability. Compared with the existing global attack, our attack can increase the attack success rate and make modification positions in generated examples more dispersed. We’ve increased the global search success rate from 75.8% to 85.8%. Finally, we can deal with different application scenarios by using these two attack methods, that is, whether we understand the internal structure and parameters of the model, we can all generate good adversarial examples.

scholarly journals Evaluating and Enhancing the Robustness of Sustainable Neural Relationship Classifiers Using Query-Efficient Black-Box Adversarial Attacks

2021 ◽  
Vol 13 (11) ◽  
pp. 5892
Author(s):  
Ijaz Ul Haq ◽  
Zahid Younas Khan ◽  
Arshad Ahmad ◽  
Bashir Hayat ◽  
Asif Khan ◽  
...  
Keyword(s):  
Question Answering ◽  
Relation Extraction ◽  
Black Box ◽  
Inverse Document Frequency ◽  
Defense Strategies ◽  
Word Level ◽  
Learning Tasks ◽  
Document Frequency ◽  
High Concern ◽  
Adversarial Examples

Neural relation extraction (NRE) models are the backbone of various machine learning tasks, including knowledge base enrichment, information extraction, and document summarization. Despite the vast popularity of these models, their vulnerabilities remain unknown; this is of high concern given their growing use in security-sensitive applications such as question answering and machine translation in the aspects of sustainability. In this study, we demonstrate that NRE models are inherently vulnerable to adversarially crafted text that contains imperceptible modifications of the original but can mislead the target NRE model. Specifically, we propose a novel sustainable term frequency-inverse document frequency (TFIDF) based black-box adversarial attack to evaluate the robustness of state-of-the-art CNN, CGN, LSTM, and BERT-based models on two benchmark RE datasets. Compared with white-box adversarial attacks, black-box attacks impose further constraints on the query budget; thus, efficient black-box attacks remain an open problem. By applying TFIDF to the correctly classified sentences of each class label in the test set, the proposed query-efficient method achieves a reduction of up to 70% in the number of queries to the target model for identifying important text items. Based on these items, we design both character- and word-level perturbations to generate adversarial examples. The proposed attack successfully reduces the accuracy of six representative models from an average F1 score of 80% to below 20%. The generated adversarial examples were evaluated by humans and are considered semantically similar. Moreover, we discuss defense strategies that mitigate such attacks, and the potential countermeasures that could be deployed in order to improve sustainability of the proposed scheme.

Collecting Words: A Clinical Example of a Morphology-Focused Orthographic Intervention

2020 ◽  
Vol 51 (3) ◽  
pp. 544-560 ◽  
Author(s):  
Kimberly A. Murphy ◽  
Emily A. Diehm
Keyword(s):  
Written Language ◽  
Morphological Knowledge ◽  
English Orthography ◽  
Word Level ◽  
Novel Approach ◽  
Language Pathology ◽  
The One ◽  
Critical Intervention ◽  
Reading And Spelling ◽  
Reading And Spelling Difficulties

Purpose Morphological interventions promote gains in morphological knowledge and in other oral and written language skills (e.g., phonological awareness, vocabulary, reading, and spelling), yet we have a limited understanding of critical intervention features. In this clinical focus article, we describe a relatively novel approach to teaching morphology that considers its role as the key organizing principle of English orthography. We also present a clinical example of such an intervention delivered during a summer camp at a university speech and hearing clinic. Method Graduate speech-language pathology students provided a 6-week morphology-focused orthographic intervention to children in first through fourth grade ( n = 10) who demonstrated word-level reading and spelling difficulties. The intervention focused children's attention on morphological families, teaching how morphology is interrelated with phonology and etymology in English orthography. Results Comparing pre- and posttest scores, children demonstrated improvement in reading and/or spelling abilities, with the largest gains observed in spelling affixes within polymorphemic words. Children and their caregivers reacted positively to the intervention. Therefore, data from the camp offer preliminary support for teaching morphology within the context of written words, and the intervention appears to be a feasible approach for simultaneously increasing morphological knowledge, reading, and spelling. Conclusion Children with word-level reading and spelling difficulties may benefit from a morphology-focused orthographic intervention, such as the one described here. Research on the approach is warranted, and clinicians are encouraged to explore its possible effectiveness in their practice. Supplemental Material https://doi.org/10.23641/asha.12290687

Factors Influencing Cognate Performance for Young Multilingual Children's Vocabulary: A Research Synthesis

2020 ◽  
Vol 29 (4) ◽  
pp. 2170-2188
Author(s):  
Lindsey R. Squires ◽  
Sara J. Ohlfest ◽  
Kristen E. Santoro ◽  
Jennifer L. Roberts
Keyword(s):  
Test Validity ◽  
Reaction Times ◽  
Bilingual Children ◽  
Second Grade ◽  
High Quality Research ◽  
Word Level ◽  
Research Questions ◽  
Word Characteristics ◽  
Study Inclusion ◽  
Task Level

Purpose The purpose of this systematic review was to determine evidence of a cognate effect for young multilingual children (ages 3;0–8;11 [years;months], preschool to second grade) in terms of task-level and child-level factors that may influence cognate performance. Cognates are pairs of vocabulary words that share meaning with similar phonology and/or orthography in more than one language, such as rose – rosa (English–Spanish) or carrot – carotte (English–French). Despite the cognate advantage noted with older bilingual children and bilingual adults, there has been no systematic examination of the cognate research in young multilingual children. Method We conducted searches of multiple electronic databases and hand-searched article bibliographies for studies that examined young multilingual children's performance with cognates based on study inclusion criteria aligned to the research questions. Results The review yielded 16 articles. The majority of the studies (12/16, 75%) demonstrated a positive cognate effect for young multilingual children (measured in higher accuracy, faster reaction times, and doublet translation equivalents on cognates as compared to noncognates). However, not all bilingual children demonstrated a cognate effect. Both task-level factors (cognate definition, type of cognate task, word characteristics) and child-level factors (level of bilingualism, age) appear to influence young bilingual children's performance on cognates. Conclusions Contrary to early 1990s research, current researchers suggest that even young multilingual children may demonstrate sensitivity to cognate vocabulary words. Given the limits in study quality, more high-quality research is needed, particularly to address test validity in cognate assessments, to develop appropriate cognate definitions for children, and to refine word-level features. Only one study included a brief instruction prior to assessment, warranting cognate treatment studies as an area of future need. Supplemental Material https://doi.org/10.23641/asha.12753179

Morphological Awareness Skills of Second- and Third-Grade Students With and Without Speech Sound Disorders

2020 ◽  
Vol 51 (3) ◽  
pp. 603-616
Author(s):  
Kenn Apel ◽  
Victoria S. Henbest
Keyword(s):  
Word Reading ◽  
Speech Sound ◽  
Morphological Awareness ◽  
Literacy Skills ◽  
Receptive Vocabulary ◽  
Real Word ◽  
Speech Sound Disorders ◽  
Word Level ◽  
Third Grade Students ◽  
Pseudoword Reading

Purpose Morphological awareness is the ability to consciously manipulate the smallest units of meaning in language. Morphological awareness contributes to success with literacy skills for children with typical language and those with language impairment. However, little research has focused on the morphological awareness skills of children with speech sound disorders (SSD), who may be at risk for literacy impairments. No researcher has examined the morphological awareness skills of children with SSD and compared their skills to children with typical speech using tasks representing a comprehensive definition of morphological awareness, which was the main purpose of this study. Method Thirty second- and third-grade students with SSD and 30 with typical speech skills, matched on age and receptive vocabulary, completed four morphological awareness tasks and measures of receptive vocabulary, real-word reading, pseudoword reading, and word-level spelling. Results Results indicated there was no difference between the morphological awareness skills of students with and without SSD. Although morphological awareness was moderately to strongly related to the students' literacy skills, performance on the morphological awareness tasks contributed little to no additional variance to the children's real-word reading and spelling skills beyond what was accounted for by pseudoword reading. Conclusions Findings suggest that early elementary-age students with SSD may not present with concomitant morphological awareness difficulties and that the morphological awareness skills of these students may not play a unique role in their word-level literacy skills. Limitations and suggestions for future research on the morphological awareness skills of children with SSD are discussed.

Sign in / Sign up

Export Citation Format

Share Document