scholarly journals Reducing Objective Function Mismatch in Deep Clustering with the Unsupervised Companion Objective

2021 ◽  
Vol 2 ◽  
Author(s):  
Daniel J. Trosten ◽  
Robert Jenssen ◽  
Michael C. Kampffmeyer
Keyword(s):  
Neural Network ◽  
Objective Function ◽  
Recent Work ◽  
Local Similarity ◽  
Clustering Methods ◽  
Input Space ◽  
Clustering Model ◽  
Space Geometry ◽  
Intermediate Layers ◽  
Similarity Structure

Preservation of local similarity structure is a key challenge in deep clustering. Many recent deep clustering methods therefore use autoencoders to help guide the model's neural network towards an embedding which is more reflective of the input space geometry. However, recent work has shown that autoencoder-based deep clustering models can suffer from objective function mismatch (OFM). In order to improve the preservation of local similarity structure, while simultaneously having a low OFM, we develop a new auxiliary objective function for deep clustering. Our Unsupervised Companion Objective (UCO) encourages a consistent clustering structure at intermediate layers in the network -- helping the network learn an embedding which is more reflective of the similarity structure in the input space. Since a clustering-based auxiliary objective has the same goal as the main clustering objective, it is less prone to introduce objective function mismatch between itself and the main objective. Our experiments show that attaching the UCO to a deep clustering model improves the performance of the model, and exhibits a lower OFM, compared to an analogous autoencoder-based model.

Valid Signature Detection and Verification for Security of Individual Person

2016 ◽  
Vol 1 (1) ◽  
pp. 50-53 ◽  
Author(s):  
Varun Sharma ◽  
Narpat Singh
Keyword(s):  
Neural Network ◽  
Comparative Analysis ◽  
Recent Work ◽  
Research Work ◽  
Early Research ◽  
Individual Person ◽  
Signature Detection ◽  
Online Signature ◽  
The Right ◽  
Handwritten Signature

In the recent research work, the handwritten signature is a suitable field to detection of valid signature from different environment such online signature and offline signature. In early research work, a lot of unauthorized person put the signature and theft the data in illegal manner from organization or industries. So we have to need identify, the right person on the basis of various parameters that can be detected. In this paper, we have proposed two methods namely LDA and Neural Network for the offline signature from the scan signature image. For efficient research, we have focused the comparative analysis in terms of FRR, SSIM, MSE, and PSNR. These parameters are compared with the early work and the recent work. Our proposed work is more effective and provides the suitable result through our method which leads to existing work. Our method will help to find legal signature of authorized use for security and avoid illegal work.

scholarly journals Order Selection in Unsupervised Learning and Clustering for Arbitrary and Non-Arbitrary Shaped Data

2021 ◽  
Author(s):  
Mahdi Shahbaba
Keyword(s):  
Objective Function ◽  
Unsupervised Learning ◽  
Minimum Spanning Tree ◽  
Statistical Testing ◽  
Adjusted Rand Index ◽  
Order Selection ◽  
Clustering Methods ◽  
Conventional Methods ◽  
Anderson Darling ◽  
Statistical Testing Method

This thesis focuses on clustering for the purpose of unsupervised learning. One topic of our interest is on estimating the correct number of clusters (CNC). In conventional clustering approaches, such as X-means, G-means, PG-means and Dip-means, estimating the CNC is a preprocessing step prior to finding the centers and clusters. In another word, the first step estimates the CNC and the second step finds the clusters. Each step having different objective function to minimize. Here, we propose minimum averaged central error (MACE)-means clustering and use one objective function to simultaneously estimate the CNC and provide the cluster centers. We have shown superiority of MACEmeans over the conventional methods in term of estimating the CNC with comparable complexity. In addition, on average MACE-means results in better values for adjusted rand index (ARI) and variation of information (VI). Next topic of our interest is order selection step of the conventional methods which is usually a statistical testing method such as Kolmogrov-Smrinov test, Anderson-Darling test, and Hartigan's Dip test. We propose a new statistical test denoted by Sigtest (signature testing). The conventional statistical testing approaches rely on a particular assumption on the probability distribution of each cluster. Sigtest on the other hand can be used with any prior distribution assumption on the clusters. By replacing the statistical testing of the mentioned conventional approaches with Sigtest, we have shown that the clustering methods are improved in terms of having more accurate CNC as well as ARI and VI. Conventional clustering approaches fail in arbitrary shaped clustering. Our last contribution of the thesis is in arbitrary shaped clustering. The proposed method denoted by minimum Pathways is Arbitrary Shaped (minPAS) clustering is proposed based on a unique minimum spanning tree structure of the data. Our simulation results show advantage of minPAS over the state-of-the-art arbitrary shaped clustering methods such as DBSCAN and Affinity Propagation in terms of accuracy, ARI and VI indexes.

Development of Surrogate Models of Orthopedic Screws to Improve Biomechanical Performance

2012 ◽  
pp. 138-159
Author(s):  
Ching-Chi Hsu
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Finite Element ◽  
Objective Function ◽  
Bending Strength ◽  
Pullout Strength ◽  
Neural Network Method ◽  
Network Method ◽  
Artificial Neural Network Method ◽  
Artificial Neural

An optimization approach was applied to improve the design of the lag screws used in double screw nails. However, finite element analyses with an optimal algorithm may take a long time to find the best design. Thus, surrogate methods, either artificial neural networks or multiple linear regressions, were used to substitute for the finite element models. The results showed that an artificial neural network method can accurately develop the objective functions of the lag screws for both the bending strength and the pullout strength. A multiple linear regression method can successfully develop the objective function of the lag screws for the pullout strength, but it failed to construct the objective function for the bending strength. The optimal design of the lag screws could be obtained using the artificial neural network method and genetic algorithms.

scholarly journals A Deep Recurrent Neural Network for Non-Intrusive Load Monitoring Based on Multi-Feature Input Space and Post-Processing

Energies ◽  
2020 ◽  
Vol 13 (9) ◽  
pp. 2195
Author(s):  
Hasan Rafiq ◽  
Xiaohan Shi ◽  
Hengxu Zhang ◽  
Huimin Li ◽  
Manesh Kumar Ochani
Keyword(s):  
Neural Network ◽  
Power Consumption ◽  
Real Time ◽  
Recurrent Neural Network ◽  
Short Term Memory ◽  
Estimation Accuracy ◽  
Post Processing ◽  
Input Space ◽  
Deep Recurrent Neural Network ◽  
Load Monitoring

Non-intrusive load monitoring (NILM) is a process of estimating operational states and power consumption of individual appliances, which if implemented in real-time, can provide actionable feedback in terms of energy usage and personalized recommendations to consumers. Intelligent disaggregation algorithms such as deep neural networks can fulfill this objective if they possess high estimation accuracy and lowest generalization error. In order to achieve these two goals, this paper presents a disaggregation algorithm based on a deep recurrent neural network using multi-feature input space and post-processing. First, the mutual information method was used to select electrical parameters that had the most influence on the power consumption of each target appliance. Second, selected steady-state parameters based multi-feature input space (MFS) was used to train the 4-layered bidirectional long short-term memory (LSTM) model for each target appliance. Finally, a post-processing technique was used at the disaggregation stage to eliminate irrelevant predicted sequences, enhancing the classification and estimation accuracy of the algorithm. A comprehensive evaluation was conducted on 1-Hz sampled UKDALE and ECO datasets in a noised scenario with seen and unseen test cases. Performance evaluation showed that the MFS-LSTM algorithm is computationally efficient, scalable, and possesses better estimation accuracy in a noised scenario, and generalized to unseen loads as compared to benchmark algorithms. Presented results proved that the proposed algorithm fulfills practical application requirements and can be deployed in real-time.

A Framework for Self-Realizing Process Models for Additive Manufacturing

2011 ◽  
Author(s):  
Sungshik Yim ◽  
David W. Rosen
Keyword(s):  
Neural Network ◽  
Additive Manufacturing ◽  
Process Planning ◽  
Process Model ◽  
Selective Laser Sintering ◽  
Process Models ◽  
Clustering Methods ◽  
In Series ◽  
Design Requirements ◽  
Planning Problems

This research discusses a framework for automating process model realization for additive manufacturing. The models map relationships from design requirements to process variables and can be utilized for future process planning. A repository is employed to collect data and contains previous process plans and corresponding design requirements. The framework organizes data through a statistical clustering method and builds regression models using a multi-layer neural network. Hierarchical and k-means clustering methods are employed in series to manage the data. A two layer neural network and augmented training algorithm are employed to build process models. The framework has been tested with Stereolithography and Selective Laser Sintering process planning problems to demonstrate its usefulness.

GMM with parameters initialization based on SVD for network threat detection

2021 ◽  
Vol 40 (1) ◽  
pp. 477-490
Author(s):  
Yanping Xu ◽  
Tingcong Ye ◽  
Xin Wang ◽  
Yuping Lai ◽  
Jian Qiu ◽  
...  
Keyword(s):  
Gaussian Mixture Models ◽  
Singular Values ◽  
Gaussian Mixture ◽  
Singular Value ◽  
Optimal Parameters ◽  
Clustering Methods ◽  
Data Set ◽  
Detection Model ◽  
Clustering Model ◽  
Threat Behavior

In the field of security, the data labels are unknown or the labels are too expensive to label, so that clustering methods are used to detect the threat behavior contained in the big data. The most widely used probabilistic clustering model is Gaussian Mixture Models(GMM), which is flexible and powerful to apply prior knowledge for modelling the uncertainty of the data. Therefore, in this paper, we use GMM to build the threat behavior detection model. Commonly, Expectation Maximization (EM) and Variational Inference (VI) are used to estimate the optimal parameters of GMM. However, both EM and VI are quite sensitive to the initial values of the parameters. Therefore, we propose to use Singular Value Decomposition (SVD) to initialize the parameters. Firstly, SVD is used to factorize the data set matrix to get the singular value matrix and singular matrices. Then we calculate the number of the components of GMM by the first two singular values in the singular value matrix and the dimension of the data. Next, other parameters of GMM, such as the mixing coefficients, the mean and the covariance, are calculated based on the number of the components. After that, the initialization values of the parameters are input into EM and VI to estimate the optimal parameters of GMM. The experiment results indicate that our proposed method performs well on the parameters initialization of GMM clustering using EM and VI for estimating parameters.

Sign in / Sign up

Export Citation Format

Share Document