scholarly journals Neighborhood Attentional Memory Networks for Recommendation Systems

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Tianlong Gu ◽  
Hongliang Chen ◽  
Chenzhong Bin ◽  
Liang Chang ◽  
Wei Chen
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
State Of The Art ◽  
Implicit Feedback ◽  
Learning Techniques ◽  
Ranking Score ◽  
Memory Modules ◽  
Output Module ◽  
Neighborhood Relations ◽  
Real World Datasets

Deep learning systems have been phenomenally successful in the fields of computer vision, speech recognition, and natural language processing. Recently, researchers have adopted deep learning techniques to tackle collaborative filtering with implicit feedback. However, the existing methods generally profile both users and items directly, while neglecting the similarities between users’ and items’ neighborhoods. To this end, we propose the neighborhood attentional memory networks (NAMN), a deep learning recommendation model applying two dedicated memory networks to capture users’ neighborhood relations and items’ neighborhood relations respectively. Specifically, we first design the user neighborhood component and the item neighborhood component based on memory networks and attention mechanisms. Then, by the associative addressing scheme with the user and item memories in the neighborhood components, we capture the complex user-item neighborhood relations. Stacking multiple memory modules together yields deeper architectures exploring higher-order complex user-item neighborhood relations. Finally, the output module jointly exploits the user and item neighborhood information with the user and item memories to obtain the ranking score. Extensive experiments on three real-world datasets demonstrate significant improvements of the proposed NAMN method over the state-of-the-art methods.

scholarly journals Representation Learning for Fine-Grained Change Detection

Sensors ◽  
2021 ◽  
Vol 21 (13) ◽  
pp. 4486
Author(s):  
Niall O’Mahony ◽  
Sean Campbell ◽  
Lenka Krpalkova ◽  
Anderson Carvalho ◽  
Joseph Walsh ◽  
...  
Keyword(s):  
Deep Learning ◽  
Change Detection ◽  
Model Calibration ◽  
State Of The Art ◽  
Representation Learning ◽  
Machine Intelligence ◽  
The State ◽  
Sensor Data ◽  
Fine Grained ◽  
Learning Techniques

Fine-grained change detection in sensor data is very challenging for artificial intelligence though it is critically important in practice. It is the process of identifying differences in the state of an object or phenomenon where the differences are class-specific and are difficult to generalise. As a result, many recent technologies that leverage big data and deep learning struggle with this task. This review focuses on the state-of-the-art methods, applications, and challenges of representation learning for fine-grained change detection. Our research focuses on methods of harnessing the latent metric space of representation learning techniques as an interim output for hybrid human-machine intelligence. We review methods for transforming and projecting embedding space such that significant changes can be communicated more effectively and a more comprehensive interpretation of underlying relationships in sensor data is facilitated. We conduct this research in our work towards developing a method for aligning the axes of latent embedding space with meaningful real-world metrics so that the reasoning behind the detection of change in relation to past observations may be revealed and adjusted. This is an important topic in many fields concerned with producing more meaningful and explainable outputs from deep learning and also for providing means for knowledge injection and model calibration in order to maintain user confidence.

scholarly journals An Efficient Method for Detection of DDoS Attacks on the Web Using Deep Learning Algorithms

2021 ◽  
Vol 10 (4) ◽  
pp. 2821-2829
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Ddos Attacks ◽  
Problem Statement ◽  
Neural Network Approach ◽  
Learning Techniques ◽  
Attack Data ◽  
Deep Learning Neural Network

Recently, DDoS attacks is the most significant threat in network security. Both industry and academia are currently debating how to detect and protect against DDoS attacks. Many studies are provided to detect these types of attacks. Deep learning techniques are the most suitable and efficient algorithm for categorizing normal and attack data. Hence, a deep neural network approach is proposed in this study to mitigate DDoS attacks effectively. We used a deep learning neural network to identify and classify traffic as benign or one of four different DDoS attacks. We will concentrate on four different DDoS types: Slowloris, Slowhttptest, DDoS Hulk, and GoldenEye. The rest of the paper is organized as follow: Firstly, we introduce the work, Section 2 defines the related works, Section 3 presents the problem statement, Section 4 describes the proposed methodology, Section 5 illustrate the results of the proposed methodology and shows how the proposed methodology outperforms state-of-the-art work and finally Section VI concludes the paper.

scholarly journals SetRank: A Setwise Bayesian Approach for Collaborative Ranking from Implicit Feedback

2020 ◽  
Vol 34 (04) ◽  
pp. 6127-6136
Author(s):  
Chao Wang ◽  
Hengshu Zhu ◽  
Chen Zhu ◽  
Chuan Qin ◽  
Hui Xiong
Keyword(s):  
Bayesian Approach ◽  
Posterior Probability ◽  
State Of The Art ◽  
User Preferences ◽  
Implicit Feedback ◽  
Pairwise Preference ◽  
Entire List ◽  
Collaborative Ranking ◽  
Real World Datasets ◽  
Made In

The recent development of online recommender systems has a focus on collaborative ranking from implicit feedback, such as user clicks and purchases. Different from explicit ratings, which reflect graded user preferences, the implicit feedback only generates positive and unobserved labels. While considerable efforts have been made in this direction, the well-known pairwise and listwise approaches have still been limited by various challenges. Specifically, for the pairwise approaches, the assumption of independent pairwise preference is not always held in practice. Also, the listwise approaches cannot efficiently accommodate “ties” due to the precondition of the entire list permutation. To this end, in this paper, we propose a novel setwise Bayesian approach for collaborative ranking, namely SetRank, to inherently accommodate the characteristics of implicit feedback in recommender system. Specifically, SetRank aims at maximizing the posterior probability of novel setwise preference comparisons and can be implemented with matrix factorization and neural networks. Meanwhile, we also present the theoretical analysis of SetRank to show that the bound of excess risk can be proportional to √M/N, where M and N are the numbers of items and users, respectively. Finally, extensive experiments on four real-world datasets clearly validate the superiority of SetRank compared with various state-of-the-art baselines.

scholarly journals Mol-BERT: An Effective Molecular Representation with BERT for Molecular Property Prediction

2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Juncai Li ◽  
Xiaofei Jiang
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Large Scale ◽  
Molecular Data ◽  
Molecular Property ◽  
Property Prediction ◽  
Learning Framework ◽  
Learning Techniques ◽  
Potential Benefits ◽  
Current Sequence

Molecular property prediction is an essential task in drug discovery. Most computational approaches with deep learning techniques either focus on designing novel molecular representation or combining with some advanced models together. However, researchers pay fewer attention to the potential benefits in massive unlabeled molecular data (e.g., ZINC). This task becomes increasingly challenging owing to the limitation of the scale of labeled data. Motivated by the recent advancements of pretrained models in natural language processing, the drug molecule can be naturally viewed as language to some extent. In this paper, we investigate how to develop the pretrained model BERT to extract useful molecular substructure information for molecular property prediction. We present a novel end-to-end deep learning framework, named Mol-BERT, that combines an effective molecular representation with pretrained BERT model tailored for molecular property prediction. Specifically, a large-scale prediction BERT model is pretrained to generate the embedding of molecular substructures, by using four million unlabeled drug SMILES (i.e., ZINC 15 and ChEMBL 27). Then, the pretrained BERT model can be fine-tuned on various molecular property prediction tasks. To examine the performance of our proposed Mol-BERT, we conduct several experiments on 4 widely used molecular datasets. In comparison to the traditional and state-of-the-art baselines, the results illustrate that our proposed Mol-BERT can outperform the current sequence-based methods and achieve at least 2% improvement on ROC-AUC score on Tox21, SIDER, and ClinTox dataset.

scholarly journals SENTIMENT ANALYSIS FOR SARCASTIC MESSAGES IN SOCIAL MEDIA USING DEEP LEARNING TECHNIQUES - AN EMPIRICAL STUDY

2021 ◽  
Vol 9 (2) ◽  
pp. 1051-1052
Author(s):  
K. Kavitha, Et. al.
Keyword(s):  
Social Media ◽  
Deep Learning ◽  
Empirical Study ◽  
Sentiment Analysis ◽  
Learning Strategies ◽  
Language Processing ◽  
The People ◽  
Learning Techniques ◽  
Key Terms ◽  
Learning Frameworks

Sentiments is the term of opinion or views about any topic expressed by the people through a source of communication. Nowadays social media is an effective platform for people to communicate and it generates huge amount of unstructured details every day. It is essential for any business organization in the current era to process and analyse the sentiments by using machine learning and Natural Language Processing (NLP) strategies. Even though in recent times the deep learning strategies are becoming more familiar due to higher capabilities of performance. This paper represents an empirical study of an application of deep learning techniques in Sentiment Analysis (SA) for sarcastic messages and their increasing scope in real time. Taxonomy of the sentiment analysis in recent times and their key terms are also been highlighted in the manuscript. The survey concludes the recent datasets considered, their key contributions and the performance of deep learning model applied with its primary purpose like sarcasm detection in order to describe the efficiency of deep learning frameworks in the domain of sentimental analysis.

scholarly journals Reviewing Autoencoders for Missing Data Imputation: Technical Trends, Applications and Outcomes

2020 ◽  
Vol 69 ◽  
pp. 1255-1285
Author(s):  
Ricardo Cardoso Pereira ◽  
Miriam Seoane Santos ◽  
Pedro Pereira Rodrigues ◽  
Pedro Henriques Abreu
Keyword(s):  
Missing Data ◽  
Missing Values ◽  
State Of The Art ◽  
Data Imputation ◽  
Tabular Data ◽  
Missing Data Imputation ◽  
Learning Techniques ◽  
Real World Datasets ◽  
And Training ◽  
Machine Learning Models

Missing data is a problem often found in real-world datasets and it can degrade the performance of most machine learning models. Several deep learning techniques have been used to address this issue, and one of them is the Autoencoder and its Denoising and Variational variants. These models are able to learn a representation of the data with missing values and generate plausible new ones to replace them. This study surveys the use of Autoencoders for the imputation of tabular data and considers 26 works published between 2014 and 2020. The analysis is mainly focused on discussing patterns and recommendations for the architecture, hyperparameters and training settings of the network, while providing a detailed discussion of the results obtained by Autoencoders when compared to other state-of-the-art methods, and of the data contexts where they have been applied. The conclusions include a set of recommendations for the technical settings of the network, and show that Denoising Autoencoders outperform their competitors, particularly the often used statistical methods.

scholarly journals Deep Learning-Based Sentiment Analysis of COVID-19 Vaccination Responses from Twitter Data

2021 ◽  
Vol 2021 ◽  
pp. 1-15
Author(s):  
Kazi Nabiul Alam ◽  
Md Shakib Khan ◽  
Abdur Rab Dhruba ◽  
Mohammad Monirujjaman Khan ◽  
Jehad F. Al-Amri ◽  
...  
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Performance Metrics ◽  
Short Term Memory ◽  
Confusion Matrix ◽  
Short Term ◽  
Learning Techniques ◽  
The World ◽  
Long Short Term Memory ◽  
Severe Anxiety

The COVID-19 pandemic has had a devastating effect on many people, creating severe anxiety, fear, and complicated feelings or emotions. After the initiation of vaccinations against coronavirus, people’s feelings have become more diverse and complex. Our aim is to understand and unravel their sentiments in this research using deep learning techniques. Social media is currently the best way to express feelings and emotions, and with the help of Twitter, one can have a better idea of what is trending and going on in people’s minds. Our motivation for this research was to understand the diverse sentiments of people regarding the vaccination process. In this research, the timeline of the collected tweets was from December 21 to July21. The tweets contained information about the most common vaccines available recently from across the world. The sentiments of people regarding vaccines of all sorts were assessed using the natural language processing (NLP) tool, Valence Aware Dictionary for sEntiment Reasoner (VADER). Initializing the polarities of the obtained sentiments into three groups (positive, negative, and neutral) helped us visualize the overall scenario; our findings included 33.96% positive, 17.55% negative, and 48.49% neutral responses. In addition, we included our analysis of the timeline of the tweets in this research, as sentiments fluctuated over time. A recurrent neural network- (RNN-) oriented architecture, including long short-term memory (LSTM) and bidirectional LSTM (Bi-LSTM), was used to assess the performance of the predictive models, with LSTM achieving an accuracy of 90.59% and Bi-LSTM achieving 90.83%. Other performance metrics such as precision,, F1-score, and a confusion matrix were also used to validate our models and findings more effectively. This study improves understanding of the public’s opinion on COVID-19 vaccines and supports the aim of eradicating coronavirus from the world.

Advancement in Data Engineering and Feature Processing Workflow by Using Deep Learning Techniques for the Automation of ESP Failure Root Cause Analyses

2021 ◽  
Author(s):  
Saniya Karnik ◽  
Navya Yenuganti ◽  
Bonang Firmansyah Jusri ◽  
Supriya Gupta ◽  
Prasanna Nirgudkar ◽  
...  
Keyword(s):  
Deep Learning ◽  
Failure Analysis ◽  
Language Processing ◽  
Turnaround Time ◽  
Process Efficiency ◽  
Root Cause ◽  
Feature Processing ◽  
Learning Techniques ◽  
Electrical Submersible Pump ◽  
Root Cause Identification

Abstract Today, Electrical Submersible Pump (ESP) failure analysis is a tedious, human-intensive, and time-consuming activity involving dismantle, inspection, and failure analysis (DIFA) for each failure. This paper presents a novel artificial intelligence workflow using an ensemble of machine learning (ML) algorithms coupled with natural language processing (NLP) and deep learning (DL). The algorithms outlined in this paper bring together structured and unstructured data across equipment, production, operations, and failure reports to automate root cause identification and analysis post breakdown. This process will result in reduced turnaround time (TAT) and human effort thus drastically improving process efficiency.

Efficient Large-Scale Stance Detection in Tweets

2020 ◽  
pp. 667-683
Author(s):  
Yilin Yan ◽  
Jonathan Chen ◽  
Mei-Ling Shyu
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Large Scale ◽  
Research Direction ◽  
Detection Methods ◽  
Use Case ◽  
Learning Techniques ◽  
Test Use ◽  
Presidential Election Campaign ◽  
Important Research Direction

Stance detection is an important research direction which attempts to automatically determine the attitude (positive, negative, or neutral) of the author of text (such as tweets), towards a target. Nowadays, a number of frameworks have been proposed using deep learning techniques that show promising results in application domains such as automatic speech recognition and computer vision, as well as natural language processing (NLP). This article shows a novel deep learning-based fast stance detection framework in bipolar affinities on Twitter. It is noted that millions of tweets regarding Clinton and Trump were produced per day on Twitter during the 2016 United States presidential election campaign, and thus it is used as a test use case because of its significant and unique counter-factual properties. In addition, stance detection can be utilized to imply the political tendency of the general public. Experimental results show that the proposed framework achieves high accuracy results when compared to several existing stance detection methods.

Sign in / Sign up

Export Citation Format

Share Document