scholarly journals Spam Detection Based on Feature Evolution to Deal with Concept Drift

2021 ◽  
Vol 27 (4) ◽  
pp. 364-386
Author(s):  
Marcia Henke ◽  
Eulanda Santos ◽  
Eduardo Souto ◽  
Altair O Santin
Keyword(s):  
Knowledge Transfer ◽  
Detection System ◽  
Concept Drift ◽  
Low Cost ◽  
Training Data ◽  
Classification Model ◽  
Easy Access ◽  
Spam Detection ◽  
Concept Drift Detection ◽  
E Mail

Electronic messages are still considered the most significant tools in business and personal applications due to their low cost and easy access. However, e-mails have become a major problem owing to the high amount of junk mail, named spam, which fill the e-mail boxes of users. Several approaches have been proposed to detect spam, such as filters implemented in e-mail servers and user-based spam message classification mechanisms. A major problem with these approaches is spam detection in the presence of concept drift, especially as a result of changes in features over time. To overcome this problem, this work proposes a new spam detection system based on analyzing the evolution of features. The proposed method is divided into three steps: 1) spam classification model training; 2) concept drift detection; and 3) knowledge transfer learning. The first step generates classification models, as commonly conducted in machine learning. The second step introduces a new strategy to avoid concept drift: SFS (Similarity-based Features Se- lection) that analyzes the evolution of the features taking into account similarity obtained between the feature vectors extracted from training data and test data. Finally, the third step focuses on the following questions: what, how, and when to transfer acquired knowledge? The proposed method is evaluated using two public datasets. The results of the experiments show that it is possible to infer a threshold to detect changes (drift) in order to ensure that the spam classification model is updated through knowledge transfer. Moreover, our anomaly detection system is able to perform spam classification and concept drift detection as two parallel and independent tasks.

scholarly journals Rapid and sensitive mycoplasma detection system using image-based deep learning

2021 ◽  
Author(s):  
Hiroko Iseoka ◽  
Masao Sasai ◽  
Shigeru Miyagawa ◽  
Kazuhiro Takekita ◽  
Satoshi Date ◽  
...  
Keyword(s):  
Cell Therapy ◽  
Detection System ◽  
Low Cost ◽  
Negative Control ◽  
Training Data ◽  
Cell Counting ◽  
Test Time ◽  
Manufacturing Cost ◽  
Mycoplasma Detection ◽  
Manual Counting

AbstractA major concern in the clinical application of cell therapy is the manufacturing cost of cell products, which mainly depends on quality control. The mycoplasma test, an important biological test in cell therapy, takes several weeks to detect a microorganism and is extremely expensive. Furthermore, the manual detection of mycoplasma from images requires high-level expertise. We hypothesized that a mycoplasma identification program using a convolutional neural network could reduce the test time and improve sensitivity. To this end, we developed a program comprising three parts (mycoplasma detection, prediction, and cell counting) that allows users to evaluate the sample and verify infected/non-infected cells identified by the program. In experiments conducted, stained DNA images of positive and negative control using mycoplasma-infected and non-infected Vero cells, respectively, were used as training data, and the program results were compared with those of conventional methods, such as manual counting based on visual observation. The minimum detectable mycoplasma contaminations for manual counting and the proposed program were 10 and 5 CFU (colony-forming unit), respectively, and the test time for manual counting was 20 times that for the proposed program. These results suggest that the proposed system can realize a low-cost and streamlined manufacturing process for cellular products in cell-based research and clinical applications.

scholarly journals IMPROVED NEAR DUPLICATE MATCHING SCHEME FOR E-MAIL SPAM DETECTION

2016 ◽  
pp. 259-263
Author(s):  
M. SIVA KUMAR REDDY ◽  
B. KRISHNA SAGAR
Keyword(s):  
Detection System ◽  
Spam Detection ◽  
Similarity Matching ◽  
New Approach ◽  
The People ◽  
Layout Structure ◽  
E Mail

Today the major problem that the people are facing is spam mails or e-mail spam. In recent years there are so many schemes are developed to detect the spam emails. Here the primary idea of the similarity matching scheme for spam detection is to maintain a known spam database, formed by user’s feedback, to block the subsequent near-duplicate spam’s. We propose a novel e-mail abstraction scheme, which considers e-mail layout structure to represent e-mails. We present a procedure to generate the e-mail abstraction using HTML content in e-mail, and this newly devised abstraction can more effectively capture the near-duplicate phenomenon of spams. Moreover, we design a complete spam detection system Cosdes (standing for Collaborative Spam Detection System), which possesses an efficient near-duplicate matching scheme and a progressive update scheme. To detect fastly near duplicates and duplicate spam mails in Cosdes, we propose a new approach SimHash.

scholarly journals CDDM: Concept Drift Detection Model for Data Stream

2020 ◽  
Vol 14 (10) ◽  
pp. 90
Author(s):  
Mashail Shaeel Althabiti ◽  
Manal Abdullah
Keyword(s):  
Data Stream ◽  
Concept Drift ◽  
Dynamic Environment ◽  
Classification Model ◽  
Behavior Changes ◽  
Weighted Version ◽  
Detection Model ◽  
Data Mining Algorithms ◽  
Concept Drift Detection ◽  
Mining Algorithms

<p>Data stream is the huge amount of data generated in various fields, including financial processes, social media activities, Internet of Things applications, and many others. Such data cannot be processed through traditional data mining algorithms due to several constraints, including limited memory, data speed, and dynamic environment. Concept Drift is known as the main constraint of data stream mining, mainly in the classification task. It refers to the change in the data stream underlining distribution over time. Thus, it results in accuracy deterioration of classification models and wrong predictions. Spam emails, consumer behavior changes, and adversary activates, are examples of Concept Drift. In this paper, a Concept Drift detection model is introduced, Concept Drift Detection Model (CDDM). It monitors the accuracy of the classification model over a sliding window, assuming the decline in accuracy indicates a drift occurrence. A modification over CDDM is a weighted version of the CDDM as W-CDDM.</p><p>Both models have evaluated against two real datasets and four artificial datasets. The experimental results of abrupt drift show that CDDM, W-CDDM outperforms the other models in the dataset of 100K and 1M instances, respectively. Regarding gradual drift, the W-CDDM overtook the rest in terms of accuracy, run time, and detection delays in the dataset of 100 K instances. While in the dataset of 1M instances, CDDM has got the highest accuracy using the NB classifier. Moreover, W-CDDM achieves the highest accuracy on real datasets.</p>

Cosdes: A Collaborative Spam Detection System with a Novel E-Mail Abstraction Scheme

2011 ◽  
Vol 23 (5) ◽  
pp. 669-682 ◽  
Author(s):  
Chi-Yao Tseng ◽  
Pin-Chieh Sung ◽  
Ming-Syan Chen
Keyword(s):  
Detection System ◽  
Spam Detection ◽  
E Mail

Drug Target Group Prediction with Multiple Drug Networks

2020 ◽  
Vol 23 (4) ◽  
pp. 274-284 ◽  
Author(s):  
Jingang Che ◽  
Lei Chen ◽  
Zi-Han Guo ◽  
Shuaiqun Wang ◽  
Aorigele
Keyword(s):  
Drug Target ◽  
Low Cost ◽  
Machine Learning Algorithms ◽  
Classification Model ◽  
Support Vector ◽  
Multiple Drug ◽  
Property A ◽  
Multiple Networks ◽  
Proposed Model ◽  
The One

Background: Identification of drug-target interaction is essential in drug discovery. It is beneficial to predict unexpected therapeutic or adverse side effects of drugs. To date, several computational methods have been proposed to predict drug-target interactions because they are prompt and low-cost compared with traditional wet experiments. Methods: In this study, we investigated this problem in a different way. According to KEGG, drugs were classified into several groups based on their target proteins. A multi-label classification model was presented to assign drugs into correct target groups. To make full use of the known drug properties, five networks were constructed, each of which represented drug associations in one property. A powerful network embedding method, Mashup, was adopted to extract drug features from above-mentioned networks, based on which several machine learning algorithms, including RAndom k-labELsets (RAKEL) algorithm, Label Powerset (LP) algorithm and Support Vector Machine (SVM), were used to build the classification model. Results and Conclusion: Tenfold cross-validation yielded the accuracy of 0.839, exact match of 0.816 and hamming loss of 0.037, indicating good performance of the model. The contribution of each network was also analyzed. Furthermore, the network model with multiple networks was found to be superior to the one with a single network and classic model, indicating the superiority of the proposed model.

Concept drift detection and localization in process mining

2021 ◽  
Author(s):  
Rafael Gaspar de Sousa ◽  
Sarajane Marques Peres ◽  
Marcelo Fantinato ◽  
Hajo Alexander Reijers
Keyword(s):  
Process Mining ◽  
Concept Drift ◽  
Concept Drift Detection ◽  
Detection And Localization

scholarly journals Experiments with Neural Networks in the Identification and Control of a Magnetic Levitation System Using a Low-Cost Platform

2021 ◽  
Vol 11 (6) ◽  
pp. 2535
Author(s):  
Bruno E. Silva ◽  
Ramiro S. Barbosa
Keyword(s):  
Neural Networks ◽  
Neural Control ◽  
Magnetic Levitation ◽  
Low Cost ◽  
Training Data ◽  
Neural Controllers ◽  
Levitation System ◽  
Internal Model Controller ◽  
Magnetic Levitation System ◽  
And Performance

In this article, we designed and implemented neural controllers to control a nonlinear and unstable magnetic levitation system composed of an electromagnet and a magnetic disk. The objective was to evaluate the implementation and performance of neural control algorithms in a low-cost hardware. In a first phase, we designed two classical controllers with the objective to provide the training data for the neural controllers. After, we identified several neural models of the levitation system using Nonlinear AutoRegressive eXogenous (NARX)-type neural networks that were used to emulate the forward dynamics of the system. Finally, we designed and implemented three neural control structures: the inverse controller, the internal model controller, and the model reference controller for the control of the levitation system. The neural controllers were tested on a low-cost Arduino control platform through MATLAB/Simulink. The experimental results proved the good performance of the neural controllers.

scholarly journals Transcription Alignment of Historical Vietnamese Manuscripts without Human-Annotated Learning Samples

2021 ◽  
Vol 11 (11) ◽  
pp. 4894
Author(s):  
Anna Scius-Bertrand ◽  
Michael Jungo ◽  
Beat Wolf ◽  
Andreas Fischer ◽  
Marc Bui
Keyword(s):  
Object Detection ◽  
State Of The Art ◽  
Positive Impact ◽  
Detection System ◽  
Training Data ◽  
Detection Accuracy ◽  
Current State ◽  
Alignment Task ◽  
Scanned Image ◽  
Automatic Transcription

The current state of the art for automatic transcription of historical manuscripts is typically limited by the requirement of human-annotated learning samples, which are are necessary to train specific machine learning models for specific languages and scripts. Transcription alignment is a simpler task that aims to find a correspondence between text in the scanned image and its existing Unicode counterpart, a correspondence which can then be used as training data. The alignment task can be approached with heuristic methods dedicated to certain types of manuscripts, or with weakly trained systems reducing the required amount of annotations. In this article, we propose a novel learning-based alignment method based on fully convolutional object detection that does not require any human annotation at all. Instead, the object detection system is initially trained on synthetic printed pages using a font and then adapted to the real manuscripts by means of self-training. On a dataset of historical Vietnamese handwriting, we demonstrate the feasibility of annotation-free alignment as well as the positive impact of self-training on the character detection accuracy, reaching a detection accuracy of 96.4% with a YOLOv5m model without using any human annotation.

Fully automated contrast and non-contrast cardiac view detection in echocardiography a multi-centre, multi-vendor study

2020 ◽  
Vol 41 (Supplement_2) ◽  
Author(s):  
S Gao ◽  
D Stojanovski ◽  
A Parker ◽  
P Marques ◽  
S Heitner ◽  
...  
Keyword(s):  
Neural Network ◽  
Training Data ◽  
Classification Model ◽  
Validation Dataset ◽  
Funding Source ◽  
Private Company ◽  
Validation Data ◽  
Independent Test ◽  
Model Training ◽  
Confusion Matrices

Abstract Background Correctly identifying views acquired in a 2D echocardiographic examination is paramount to post-processing and quantification steps often performed as part of most clinical workflows. In many exams, particularly in stress echocardiography, microbubble contrast is used which greatly affects the appearance of the cardiac views. Here we present a bespoke, fully automated convolutional neural network (CNN) which identifies apical 2, 3, and 4 chamber, and short axis (SAX) views acquired with and without contrast. The CNN was tested in a completely independent, external dataset with the data acquired in a different country than that used to train the neural network. Methods Training data comprised of 2D echocardiograms was taken from 1014 subjects from a prospective multisite, multi-vendor, UK trial with the number of frames in each view greater than 17,500. Prior to view classification model training, images were processed using standard techniques to ensure homogenous and normalised image inputs to the training pipeline. A bespoke CNN was built using the minimum number of convolutional layers required with batch normalisation, and including dropout for reducing overfitting. Before processing, the data was split into 90% for model training (211,958 frames), and 10% used as a validation dataset (23,946 frames). Image frames from different subjects were separated out entirely amongst the training and validation datasets. Further, a separate trial dataset of 240 studies acquired in the USA was used as an independent test dataset (39,401 frames). Results Figure 1 shows the confusion matrices for both validation data (left) and independent test data (right), with an overall accuracy of 96% and 95% for the validation and test datasets respectively. The accuracy for the non-contrast cardiac views of &gt;99% exceeds that seen in other works. The combined datasets included images acquired across ultrasound manufacturers and models from 12 clinical sites. Conclusion We have developed a CNN capable of automatically accurately identifying all relevant cardiac views used in “real world” echo exams, including views acquired with contrast. Use of the CNN in a routine clinical workflow could improve efficiency of quantification steps performed after image acquisition. This was tested on an independent dataset acquired in a different country to that used to train the model and was found to perform similarly thus indicating the generalisability of the model. Figure 1. Confusion matrices Funding Acknowledgement Type of funding source: Private company. Main funding source(s): Ultromics Ltd.

Sign in / Sign up

Export Citation Format

Share Document