scholarly journals Unsupervised Anomaly Detection for Glaucoma Diagnosis

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Wei Zhou ◽  
Yuan Gao ◽  
Jianhang Ji ◽  
Shicheng Li ◽  
Yugen Yi
Keyword(s):  
Anomaly Detection ◽  
Large Scale ◽  
Rapid Development ◽  
Class Imbalance ◽  
Principal Component ◽  
High Tech ◽  
Class Imbalance Problem ◽  
Image Domain ◽  
Glaucoma Diagnosis ◽  
Unsupervised Anomaly Detection

With the rapid development of high tech, Internet of Things (IoT) and artificial intelligence (AI) achieve a series of achievements in the healthcare industry. Among them, automatic glaucoma diagnosis is one of them. Glaucoma is second leading cause of blindness in the world. Although many automatic glaucoma diagnosis approaches have been proposed, they still face the following two challenges. First, the data acquisition of diseased images is extremely expensive, especially for disease with low occurrence, leading to the class imbalance. Second, large-scale labeled data are hard to obtain in medical image domain. The aforementioned challenges limit the practical application of these approaches in glaucoma diagnosis. To address these disadvantages, this paper proposes an unsupervised anomaly detection framework based on sparse principal component analysis (SPCA) for glaucoma diagnosis. In the proposed approach, we just employ the one-class normal (nonglaucoma) images for training, so the class imbalance problem can be avoided. Then, to distinguish the glaucoma (abnormal) images from the normal images, a feature set consisting of segmentation-based features and image-based features is extracted, which can capture the shape and textural changes. Next, SPCA is adopted to select the effective features from the feature set. Finally, with the usage of the extracted effective features, glaucoma diagnosis can be automatically accomplished via introducing the T 2 statistic and the control limit, overcoming the issue of insufficient labeled samples. Extensive experiments are carried out on the two public databases, and the experimental results verify the effectiveness of the proposed approach.

Generative and Autoencoder Models for Large-Scale Mutivariate Unsupervised Anomaly Detection

2021 ◽  
pp. 45-58
Author(s):  
Nabila Ounasser ◽  
Maryem Rhanoui ◽  
Mounia Mikram ◽  
Bouchra El Asri
Keyword(s):  
Anomaly Detection ◽  
Large Scale ◽  
Unsupervised Anomaly Detection

scholarly journals An Ensemble-Based Approach to Anomaly Detection in Marine Engine Sensor Streams for Efficient Condition Monitoring and Analysis

Sensors ◽  
2020 ◽  
Vol 20 (24) ◽  
pp. 7285
Author(s):  
Donghyun Kim ◽  
Sangbong Lee ◽  
Jihwan Lee
Keyword(s):  
Diesel Engine ◽  
Anomaly Detection ◽  
Large Scale ◽  
Abnormal Behavior ◽  
High Dimensional ◽  
Statistical Control ◽  
Marine Engine ◽  
Large Scale Dataset ◽  
Unsupervised Anomaly Detection ◽  
Sensor Streams

This study proposes an unsupervised anomaly detection method using sensor streams from the marine engine to detect the anomalous system behavior, which may be a possible sign of system failure. Previous works on marine engine anomaly detection proposed a clustering-based or statistical control chart-based approach that is unstable according to the choice of hyperparameters, or cannot fit well to the high-dimensional dataset. As a remedy to this limitation, this study adopts an ensemble-based approach to anomaly detection. The idea is to train several anomaly detectors with varying hyperparameters in parallel and then combine its result in the anomaly detection phase. Because the anomaly is detected by the combination of different detectors, it is robust to the choice of hyperparameters without loss of accuracy. To demonstrate our methodology, an actual dataset obtained from a 200,000-ton cargo vessel from a Korean shipping company that uses two-stroke diesel engine is analyzed. As a result, anomalies were successfully detected from the high-dimensional and large-scale dataset. After detecting the anomaly, clustering analysis was conducted to the anomalous observation to examine anomaly patterns. By investigating each cluster’s feature distribution, several common patterns of abnormal behavior were successfully visualized. Although we analyzed the data from two-stroke diesel engine, our method can be applied to various types of marine engine.

scholarly journals Using Principal Component Analysis to Solve a Class Imbalance Problem in Traffic Incident Detection

2013 ◽  
Vol 2013 ◽  
pp. 1-8 ◽  
Author(s):  
Changjiang Zheng ◽  
Shuyan Chen ◽  
Wei Wang ◽  
Jian Lu
Keyword(s):  
Principal Component Analysis ◽  
Detection System ◽  
Class Imbalance ◽  
Principal Component ◽  
Component Analysis ◽  
Incident Detection ◽  
Least Squares Regression ◽  
Class Imbalance Problem ◽  
Significance Level ◽  
Traffic Incident

High imbalances occur in real-world situations when a detection system needs to identify the rare but important event of a traffic incident. Traffic incident detection can be treated as a task of learning classifiers from imbalanced or skewed datasets. Using principal component analysis (PCA), a one-class classifier for incident detection is constructed from the major and minor principal components of normal instances. Experiments are conducted with a real traffic dataset collected from the A12 highway in The Netherlands. The parameters setting, including the significance level, the percentage of the total variation explained, and the upper bound of the eigenvalues for the minor components, is discussed. The test results demonstrate that this method achieves better performance than partial least squares regression. The method is shown to be promising for traffic incident detection.

scholarly journals Unsupervised Anomaly Detection for Network Data Streams in Industrial Control Systems

Information ◽  
2020 ◽  
Vol 11 (2) ◽  
pp. 105 ◽  
Author(s):  
Limengwei Liu ◽  
Modi Hu ◽  
Chaoqun Kang ◽  
Xiaoyong Li
Keyword(s):  
Anomaly Detection ◽  
Control Systems ◽  
Data Streams ◽  
Large Scale ◽  
Stable Operation ◽  
Time Data ◽  
Industrial Control ◽  
Industrial Control Systems ◽  
Unsupervised Anomaly Detection ◽  
Industrial Control Networks

The development and integration of information technology and industrial control networks have expanded the magnitude of new data; detecting anomalies or discovering other valid information from them is of vital importance to the stable operation of industrial control systems. This paper proposes an incremental unsupervised anomaly detection method that can quickly analyze and process large-scale real-time data. Our evaluation on the Secure Water Treatment dataset shows that the method is converging to its offline counterpart for infinitely growing data streams.

scholarly journals A Novel Encoder-Decoder Knowledge Graph Completion Model for Robot Brain

2021 ◽  
Vol 15 ◽  
Author(s):  
Yichen Song ◽  
Aiping Li ◽  
Hongkui Tu ◽  
Kai Chen ◽  
Chenchen Li
Keyword(s):  
Link Prediction ◽  
Large Scale ◽  
Rapid Development ◽  
Representation Learning ◽  
High Tech ◽  
Knowledge Graph ◽  
World Knowledge ◽  
Human Beings ◽  
Benchmark Datasets ◽  
The Brain

With the rapid development of artificial intelligence, Cybernetics, and other High-tech subject technology, robots have been made and used in increasing fields. And studies on robots have attracted growing research interests from different communities. The knowledge graph can act as the brain of a robot and provide intelligence, to support the interaction between the robot and the human beings. Although the large-scale knowledge graphs contain a large amount of information, they are still incomplete compared with real-world knowledge. Most existing methods for knowledge graph completion focus on entity representation learning. However, the importance of relation representation learning is ignored, as well as the cross-interaction between entities and relations. In this paper, we propose an encoder-decoder model which embeds the interaction between entities and relations, and adds a gate mechanism to control the attention mechanism. Experimental results show that our method achieves better link prediction performance than state-of-the-art embedding models on two benchmark datasets, WN18RR and FB15k-237.

scholarly journals An Efficient Cost-Sensitive Feature Selection Using Chaos Genetic Algorithm for Class Imbalance Problem

2016 ◽  
Vol 2016 ◽  
pp. 1-9 ◽  
Author(s):  
Jing Bian ◽  
Xin-guang Peng ◽  
Ying Wang ◽  
Hai Zhang
Keyword(s):  
Genetic Algorithm ◽  
Feature Selection ◽  
Network Security ◽  
Classification Accuracy ◽  
Large Scale ◽  
Class Imbalance ◽  
Evaluation Function ◽  
Class Imbalance Problem ◽  
Imbalance Problem ◽  
Chaos Genetic Algorithm

In the era of big data, feature selection is an essential process in machine learning. Although the class imbalance problem has recently attracted a great deal of attention, little effort has been undertaken to develop feature selection techniques. In addition, most applications involving feature selection focus on classification accuracy but not cost, although costs are important. To cope with imbalance problems, we developed a cost-sensitive feature selection algorithm that adds the cost-based evaluation function of a filter feature selection using a chaos genetic algorithm, referred to as CSFSG. The evaluation function considers both feature-acquiring costs (test costs) and misclassification costs in the field of network security, thereby weakening the influence of many instances from the majority of classes in large-scale datasets. The CSFSG algorithm reduces the total cost of feature selection and trades off both factors. The behavior of the CSFSG algorithm is tested on a large-scale dataset of network security, using two kinds of classifiers: C4.5 andk-nearest neighbor (KNN). The results of the experimental research show that the approach is efficient and able to effectively improve classification accuracy and to decrease classification time. In addition, the results of our method are more promising than the results of other cost-sensitive feature selection algorithms.

A novel unsupervised anomaly detection based on robust principal component classifier

2006 ◽  
Author(s):  
Wenbin Qiu ◽  
Yu Wu ◽  
Guoyin Wang ◽  
Simon X. Yang ◽  
Jie Bai ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Principal Component ◽  
Unsupervised Anomaly Detection ◽  
Component Classifier

scholarly journals Graduate Employment Prediction with Bias

2020 ◽  
Vol 34 (01) ◽  
pp. 670-677
Author(s):  
Teng Guo ◽  
Feng Xia ◽  
Shihao Zhen ◽  
Xiaomei Bai ◽  
Dongyu Zhang ◽  
...  
Keyword(s):  
Academic Performance ◽  
Employment Status ◽  
Large Scale ◽  
Short Term Memory ◽  
Early Stage ◽  
Class Imbalance ◽  
Social Consequences ◽  
Class Imbalance Problem ◽  
Generative Adversarial Network ◽  
Adversarial Network

The failure of landing a job for college students could cause serious social consequences such as drunkenness and suicide. In addition to academic performance, unconscious biases can become one key obstacle for hunting jobs for graduating students. Thus, it is necessary to understand these unconscious biases so that we can help these students at an early stage with more personalized intervention. In this paper, we develop a framework, i.e., MAYA (Multi-mAjor emploYment stAtus) to predict students' employment status while considering biases. The framework consists of four major components. Firstly, we solve the heterogeneity of student courses by embedding academic performance into a unified space. Then, we apply a generative adversarial network (GAN) to overcome the class imbalance problem. Thirdly, we adopt Long Short-Term Memory (LSTM) with a novel dropout mechanism to comprehensively capture sequential information among semesters. Finally, we design a bias-based regularization to capture the job market biases. We conduct extensive experiments on a large-scale educational dataset and the results demonstrate the effectiveness of our prediction framework.

Sign in / Sign up

Export Citation Format

Share Document