scholarly journals Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment

2020 ◽  
Vol 34 (05) ◽  
pp. 8018-8025 ◽  
Author(s):  
Di Jin ◽  
Zhijing Jin ◽  
Joey Tianyi Zhou ◽  
Peter Szolovits
Keyword(s):  
Machine Learning ◽  
Natural Language ◽  
Text Classification ◽  
Recurrent Neural Networks ◽  
State Of The Art ◽  
Semantic Content ◽  
Machine Learning Algorithms ◽  
Textual Entailment ◽  
Text Length ◽  
Adversarial Examples

Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models. It is helpful to evaluate or even improve the robustness of these models by exposing the maliciously crafted adversarial examples. In this paper, we present TextFooler, a simple but strong baseline to generate adversarial text. By applying it to two fundamental natural language tasks, text classification and textual entailment, we successfully attacked three target models, including the powerful pre-trained BERT, and the widely used convolutional and recurrent neural networks. We demonstrate three advantages of this framework: (1) effective—it outperforms previous attacks by success rate and perturbation rate, (2) utility-preserving—it preserves semantic content, grammaticality, and correct types classified by humans, and (3) efficient—it generates adversarial text with computational complexity linear to the text length.1

Deep Learning for text in limted data settings

2020 ◽  
Author(s):  
Pathikkumar Patel ◽  
Bhargav Lad ◽  
Jinan Fiaidhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
Time Series Forecasting ◽  
Text Data ◽  
Performance Levels

During the last few years, RNN models have been extensively used and they have proven to be better for sequence and text data. RNNs have achieved state-of-the-art performance levels in several applications such as text classification, sequence to sequence modelling and time series forecasting. In this article we will review different Machine Learning and Deep Learning based approaches for text data and look at the results obtained from these methods. This work also explores the use of transfer learning in NLP and how it affects the performance of models on a specific application of sentiment analysis.

scholarly journals Accelerating organic solar cell material's discovery: high-throughput screening and big data

2021 ◽  
Author(s):  
Xabier Rodríguez-Martínez ◽  
Enrique Pascual-San-José ◽  
Mariano Campoy-Quiles
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
High Throughput ◽  
Organic Solar Cells ◽  
High Throughput Screening ◽  
Organic Solar Cell ◽  
State Of The Art ◽  
Review Article ◽  
Machine Learning Algorithms ◽  
Device Optimization

This review article presents the state-of-the-art in high-throughput computational and experimental screening routines with application in organic solar cells, including materials discovery, device optimization and machine-learning algorithms.

scholarly journals Teleconsultations between Patients and Healthcare Professionals in Primary Care in Catalonia: the Evaluation of Text Classification Algorithms Using Machine Learning

2019 ◽  
Author(s):  
Francesc López Seguí ◽  
Ricardo Ander Egg Aguilar ◽  
Gabriel de Maeztu ◽  
Anna García-Altés ◽  
Francesc García Cuyàs ◽  
...  
Keyword(s):  
Machine Learning ◽  
Primary Care ◽  
Text Classification ◽  
Learning Strategy ◽  
Care Service ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Face To Face ◽  
Classification Tool ◽  
The Impact

Background: the primary care service in Catalonia has operated an asynchronous teleconsulting service between GPs and patients since 2015 (eConsulta), which has generated some 500,000 messages. New developments in big data analysis tools, particularly those involving natural language, can be used to accurately and systematically evaluate the impact of the service. Objective: the study was intended to examine the predictive potential of eConsulta messages through different combinations of vector representation of text and machine learning algorithms and to evaluate their performance. Methodology: 20 machine learning algorithms (based on 5 types of algorithms and 4 text representation techniques)were trained using a sample of 3,559 messages (169,102 words) corresponding to 2,268 teleconsultations (1.57 messages per teleconsultation) in order to predict the three variables of interest (avoiding the need for a face-to-face visit, increased demand and type of use of the teleconsultation). The performance of the various combinations was measured in terms of precision, sensitivity, F-value and the ROC curve. Results: the best-trained algorithms are generally effective, proving themselves to be more robust when approximating the two binary variables "avoiding the need of a face-to-face visit" and "increased demand" (precision = 0.98 and 0.97, respectively) rather than the variable "type of query"(precision = 0.48). Conclusion: to the best of our knowledge, this study is the first to investigate a machine learning strategy for text classification using primary care teleconsultation datasets. The study illustrates the possible capacities of text analysis using artificial intelligence. The development of a robust text classification tool could be feasible by validating it with more data, making it potentially more useful for decision support for health professionals.

scholarly journals Cyber Bullying Detection for Twitter Using ML Classification Algorithms

2021 ◽  
Vol 9 (11) ◽  
pp. 24-29
Author(s):  
Muskan Patidar
Keyword(s):  
Machine Learning ◽  
Social Media ◽  
Natural Language ◽  
Naive Bayes ◽  
Learning Algorithms ◽  
Naïve Bayes ◽  
Cyber Bullying ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Classification Algorithms

Abstract: Social networking platforms have given us incalculable opportunities than ever before, and its benefits are undeniable. Despite benefits, people may be humiliated, insulted, bullied, and harassed by anonymous users, strangers, or peers. Cyberbullying refers to the use of technology to humiliate and slander other people. It takes form of hate messages sent through social media and emails. With the exponential increase of social media users, cyberbullying has been emerged as a form of bullying through electronic messages. We have tried to propose a possible solution for the above problem, our project aims to detect cyberbullying in tweets using ML Classification algorithms like Naïve Bayes, KNN, Decision Tree, Random Forest, Support Vector etc. and also we will apply the NLTK (Natural language toolkit) which consist of bigram, trigram, n-gram and unigram on Naïve Bayes to check its accuracy. Finally, we will compare the results of proposed and baseline features with other machine learning algorithms. Findings of the comparison indicate the significance of the proposed features in cyberbullying detection. Keywords: Cyber bullying, Machine Learning Algorithms, Twitter, Natural Language Toolkit

scholarly journals A Comparative Analysis of Machine Learning Techniques for Spam Detection

2021 ◽  
pp. 657-661
Author(s):  
Rashida Ali ◽  
Ibrahim Rampurawala ◽  
Mayuri Wandhe ◽  
Ruchika Shrikhande ◽  
Arpita Bhatkar
Keyword(s):  
Machine Learning ◽  
Natural Language Processing ◽  
Comparative Analysis ◽  
Natural Language ◽  
Language Processing ◽  
High Volume ◽  
Machine Learning Algorithms ◽  
Machine Learning Techniques ◽  
Spam Detection ◽  
Learning Techniques

Internet provides a medium to connect with individuals of similar or different interests creating a hub. Since a huge hub participates on these platforms, the user can receive a high volume of messages from different individuals creating a chaos and unwanted messages. These messages sometimes contain a true information and sometimes false, which leads to a state of confusion in the minds of the users and leads to first step towards spam messaging. Spam messages means an irrelevant and unsolicited message sent by a known/unknown user which may lead to a sense of insecurity among users. In this paper, the different machine learning algorithms were trained and tested with natural language processing (NLP) to classify whether the messages are spam or ham.

scholarly journals Machine Learning for Landslides Prevention: A Survey

2020 ◽  
Author(s):  
Zhengjing Ma ◽  
Gang Mei
Keyword(s):  
Machine Learning ◽  
State Of The Art ◽  
Human Life ◽  
Machine Learning Algorithms ◽  
Warning Systems ◽  
Negative Effects ◽  
Relevant Research ◽  
Related Information ◽  
Urgent Task ◽  
Comprehensive Survey

Landslides are one of the most critical categories of natural disasters worldwide and induce severely destructive outcomes to human life and the overall economic system. To reduce its negative effects, landslides prevention has become an urgent task, which includes investigating landslide-related information and predicting potential landslides. Machine learning is a state-of-the-art analytics tool that has been widely used in landslides prevention. This paper presents a comprehensive survey of relevant research on machine learning applied in landslides prevention, mainly focusing on (1) landslides detection based on images, (2) landslides susceptibility assessment, and (3) the development of landslide warning systems. Moreover, this paper discusses the current challenges and potential opportunities in the application of machine learning algorithms for landslides prevention.

Personalized Transcriptomic Analyses Identify Unique Signatures That Correlate with Genomic Subtypes in Acute Myeloid Leukemia (AML) Using Explainable Artificial Intelligence

Blood ◽  
2020 ◽  
Vol 136 (Supplement 1) ◽  
pp. 33-34 ◽  
Author(s):  
Yazan Rouphail ◽  
Nathan Radakovich ◽  
Jacob Shreve ◽  
Sudipto Mukherjee ◽  
Babal K. Jha ◽  
...  
Keyword(s):  
Machine Learning ◽  
Board Of Directors ◽  
Research Funding ◽  
State Of The Art ◽  
Data Monitoring Committee ◽  
Biological Pathways ◽  
Machine Learning Algorithms ◽  
Advisory Committees ◽  
Altered Expression ◽  
Individual Level

Background Multi-omic analysis can identify unique signatures that correlate with cancer subtypes. While clinically meaningful molecular subtypes of AML have been defined based on the status of single genes such as NPM1 and FLT3, such categories remain heterogeneous and further work is needed to characterize their genetic and transcriptomic diversity on a truly individualized basis. Further, patients (pts) with NPM1+/FLT3-ITD- AML have a better overall survival compared to patients with NPM1-/FLT3-ITD+, suggesting that these pts could have different transcriptomic signature that impact phenotype, pathophysiology, and outcomes. Many current transcriptome analytic techniques use clustering analysis to aggregate samples and look at relationships on a cohort-wide basis to build transcriptomic signatures that correlate with phenotype or outcome. Such approaches can undermine the heterogeneity of the gene expression in pts with the same signatures. In this study, we took advantage of state of the art machine learning algorithms to identify unique transcriptomic signatures that correlate with AML genomic phenotype. Methods Genomic (whole exome sequencing and targeted deep sequencing) and transcriptomic data from 451 AML pts included in the Beat AML study (publicly available data) were used to build transcriptomic signatures that are specific for AML patients with NPM1+/FLT3-ITD+ compared to NPM1+/FLT3-ITD, and NPM1-/FLT3-ITD-. We chose these AML phenotypes as they have been described extensively and they correlate with clinical outcomes. Results A total of 242 patients (54%) had NPM1-/FLT3-, 35 (8%) were NPM1+/FLT3-, and 47 (10%) were NPM1+/FLT3+. Our algorithm identified 20 genes that are highly specific for NPM1/FLT3ITD phenotype: HOXB-AS3, SCRN1, LMX1B, PCBD1, DNAJC15, HOXA3, NPTXq, RP11-1055B8, ABDH128, HOXB8, SOCS2, HOXB3, HOXB9, MIR503HG, FAM221B, NRP1, NDUFAF3, MEG3, CCDC136, and HIST1H2BC. Interestingly, several of those genes were overexpressed or underexpressed in specific phenotypes. For example, SCRN1, LMX1B, RP11-1055B8, ABDH128, HOXB8, MIR503HG, NRP1 are only overexpressed or underexpressed in patients with NPM1-/FLT3-, while PCBD1, NDUFAF3, FAM221B are overexpressed or underexpressed in pts with NPM1+/FLT3+. These genes affect several important pathways that regulate cell differentiation, proliferation, mitochondrial oxidative phosphorylation, histone modification and lipid metabolism. All these genes had previously been reported as having altered expression in genomic studies of AML, confirming our approach's ability to identify biologically meaningful relationships. Further, our algorithm can provide a personalized explanation of overexpressed and underexpressed genes specific for a given patient, thus identifying targetable pathways for each pt. Figure 1 below shows three pts with the same genotype (NPM1+/FLT3-ITD+) but demonstrate different transcriptomic patterns of overexpression or underexpression that affect different biological pathways. Conclusions We describe the use of a state of the art explainable machine learning approach to define transcriptomic signatures that are specific for individual pts. In addition to correctly distinguishing AML subtype based on specific transcriptomic signatures, our model was able to accurately identify upregulated and downregulated genes that affecte several important biological pathways in AML and can summarize these pathways at an individual level. Such an approach can be used to provide personalized treatment options that can target the activated pathways at an individual level. Disclosures Mukherjee: Partnership for Health Analytic Research, LLC (PHAR, LLC): Honoraria; Novartis: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; EUSA Pharma: Consultancy; Celgene/Acceleron: Membership on an entity's Board of Directors or advisory committees; Bristol Myers Squib: Honoraria; Aplastic Anemia and MDS International Foundation: Honoraria; Celgene: Consultancy, Honoraria, Research Funding. Maciejewski:Alexion, BMS: Speakers Bureau; Novartis, Roche: Consultancy, Honoraria. Sekeres:BMS: Consultancy; Takeda/Millenium: Consultancy; Pfizer: Consultancy. Nazha:Jazz: Research Funding; Incyte: Speakers Bureau; Novartis: Speakers Bureau; MEI: Other: Data monitoring Committee.

SANTM: Efficient Self-attention-driven Network for Text Matching

2022 ◽  
Vol 22 (3) ◽  
pp. 1-21
Author(s):  
Prayag Tiwari ◽  
Amit Kumar Jaiswal ◽  
Sahil Garg ◽  
Ilsun You
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
The State ◽  
Attention Mechanism ◽  
Matching Problems ◽  
Attention Model ◽  
Extra Information ◽  
Textual Entailment ◽  
Benchmark Datasets ◽  
Text Matching

Self-attention mechanisms have recently been embraced for a broad range of text-matching applications. Self-attention model takes only one sentence as an input with no extra information, i.e., one can utilize the final hidden state or pooling. However, text-matching problems can be interpreted either in symmetrical or asymmetrical scopes. For instance, paraphrase detection is an asymmetrical task, while textual entailment classification and question-answer matching are considered asymmetrical tasks. In this article, we leverage attractive properties of self-attention mechanism and proposes an attention-based network that incorporates three key components for inter-sequence attention: global pointwise features, preceding attentive features, and contextual features while updating the rest of the components. Our model follows evaluation on two benchmark datasets cover tasks of textual entailment and question-answer matching. The proposed efficient Self-attention-driven Network for Text Matching outperforms the state of the art on the Stanford Natural Language Inference and WikiQA datasets with much fewer parameters.

scholarly journals Combining Axiom Injection and Knowledge Base Completion for Efficient Natural Language Inference

2019 ◽  
Vol 33 ◽  
pp. 7410-7417 ◽  
Author(s):  
Masashi Yoshikawa ◽  
Koji Mineshima ◽  
Hiroshi Noji ◽  
Daisuke Bekki
Keyword(s):  
Natural Language ◽  
Knowledge Base ◽  
Processing Speed ◽  
Processing Time ◽  
State Of The Art ◽  
Proof Automation ◽  
New Knowledge ◽  
Textual Entailment ◽  
Amount Of Knowledge ◽  
Recognizing Textual Entailment

In logic-based approaches to reasoning tasks such as Recognizing Textual Entailment (RTE), it is important for a system to have a large amount of knowledge data. However, there is a tradeoff between adding more knowledge data for improved RTE performance and maintaining an efficient RTE system, as such a big database is problematic in terms of the memory usage and computational complexity. In this work, we show the processing time of a state-of-the-art logic-based RTE system can be significantly reduced by replacing its search-based axiom injection (abduction) mechanism by that based on Knowledge Base Completion (KBC). We integrate this mechanism in a Coq plugin that provides a proof automation tactic for natural language inference. Additionally, we show empirically that adding new knowledge data contributes to better RTE performance while not harming the processing speed in this framework.

Sign in / Sign up

Export Citation Format

Share Document