scholarly journals Missing Data Reconstruction Based on Spectral k-Support Norm Minimization for NB-IoT Data

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Luo Xuegang ◽  
Lv Junrui ◽  
Wang Juan
Keyword(s):  
Relative Density ◽  
Missing Values ◽  
Clustering Algorithm ◽  
Optimal Solution ◽  
Sensor Data ◽  
Low Rank ◽  
Data Reconstruction ◽  
Reconstruction Method ◽  
Norm Minimization ◽  
Density Based Clustering

An effective fraction of data with missing values from various physiochemical sensors in the Internet of Things is still emerging owing to unreliable links and accidental damage. This phenomenon will limit the predicative ability and performance for supporting data analyses by IoT-based platforms. Therefore, it is necessary to exploit a way to reconstruct these lost data with high accuracy. A new data reconstruction method based on spectral k-support norm minimization (DR-SKSNM) is proposed for NB-IoT data, and a relative density-based clustering algorithm is embedded into model processing for improving the accuracy of reconstruction. First, sensors are grouped by similar patterns of measurement. A relative density-based clustering, which can effectively identify clusters in data sets with different densities, is applied to separate sensors into different groups. Second, based on the correlations of sensor data and its joint low rank, an algorithm based on the matrix spectral k-support norm minimization with automatic weight is developed. Moreover, the alternating direction method of multipliers (ADMM) is used to obtain its optimal solution. Finally, the proposed method is evaluated by using two simulated and real sensor data sources from Panzhihua environmental monitoring station with random missing patterns and consecutive missing patterns. From the simulation results, it is proved that our algorithm performs well, and it can propagate through low-rank characteristics to estimate a large missing region’s value.

DBSCANI: Noise-Resistant Method for Missing Value Imputation

2016 ◽  
Vol 25 (3) ◽  
pp. 431-440 ◽  
Author(s):  
Archana Purwar ◽  
Sandeep Kumar Singh
Keyword(s):  
Spatial Data ◽  
Missing Values ◽  
Clustering Algorithm ◽  
Spatial Clustering ◽  
Data Sets ◽  
Quality Of Data ◽  
Data Set ◽  
Dbscan Clustering ◽  
Density Based Clustering

AbstractThe quality of data is an important task in the data mining. The validity of mining algorithms is reduced if data is not of good quality. The quality of data can be assessed in terms of missing values (MV) as well as noise present in the data set. Various imputation techniques have been studied in MV study, but little attention has been given on noise in earlier work. Moreover, to the best of knowledge, no one has used density-based spatial clustering of applications with noise (DBSCAN) clustering for MV imputation. This paper proposes a novel technique density-based imputation (DBSCANI) built on density-based clustering to deal with incomplete values in the presence of noise. Density-based clustering algorithm proposed by Kriegal groups the objects according to their density in spatial data bases. The high-density regions are known as clusters, and the low-density regions refer to the noise objects in the data set. A lot of experiments have been performed on the Iris data set from life science domain and Jain’s (2D) data set from shape data sets. The performance of the proposed method is evaluated using root mean square error (RMSE) as well as it is compared with existing K-means imputation (KMI). Results show that our method is more noise resistant than KMI on data sets used under study.

scholarly journals Weighted Nuclear Norm Minimization on Multimodality Clustering

2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Lei Du ◽  
Songsong Dai ◽  
Haifeng Song ◽  
Yuelong Chuang ◽  
Yingying Xu
Keyword(s):  
Spectral Clustering ◽  
Clustering Algorithm ◽  
Transition Probability ◽  
Nuclear Norm ◽  
Low Rank ◽  
Nuclear Norm Minimization ◽  
Norm Minimization ◽  
Rank Matrix ◽  
Low Rank Matrix ◽  
Weighted Nuclear Norm Minimization

Generally, multimodality data contain different potential information available and are capable of providing an enhanced analytical result compared to monosource data. The way to combine the data plays a crucial role in multimodality data analysis which is worth investigating. Multimodality clustering, which seeks a partition of the data in multiple views, has attracted considerable attention, for example, robust multiview spectral clustering (RMSC) explicitly handles the possible noise in the transition probability matrices associated with different views. Spectral clustering algorithm embeds the input data into a low-dimensional representation by dividing the clustering problem into k subproblems, and the corresponding eigenvalue reflects the loss of each subproblem. So, the eigenvalues of the Laplacian matrix should be treated differently, while RMSC regularizes each singular value equally when recovering the low-rank matrix. In this paper, we propose a multimodality clustering algorithm which recovers the low-rank matrix by weighted nuclear norm minimization. We also propose a method to evaluate the weight vector by learning a shared low-rank matrix. In our experiments, we use several real-world datasets to test our method, and experimental results show that the proposed method has a better performance than other baselines.

Prestack seismic data reconstruction and denoising by orientation-dependent tensor decomposition

Geophysics ◽  
2020 ◽  
pp. 1-66
Author(s):  
Quézia Cavalcante ◽  
Milton J. Porsani
Keyword(s):  
Covariance Matrix ◽  
Seismic Data ◽  
Missing Values ◽  
Tensor Decomposition ◽  
Low Rank ◽  
Eigenvalue Decomposition ◽  
Data Reconstruction ◽  
Frequency Space ◽  
Multidimensional Signals ◽  
Seismic Data Reconstruction

Multidimensional seismic data reconstruction and denoising can be achieved by assuming noiseless and complete data as low-rank matrices or tensors in the frequency-space domain. We propose a simple and effective approach to interpolate prestack seismic data that explores the low-rank property of multidimensional signals. The orientation-dependent tensor decomposition represents an alternative to multilinear algebraic schemes. Our method does not need to perform any explicit matricization, only requiring to calculate the so-called covariance matrix for one of the spatial dimensions. The elements of such a matrix are the inner products between the lower-dimensional tensors in a convenient direction. The eigenvalue decomposition of the covariance matrix provides the eigenvectors for the reduced-rank approximation of the data tensor. This approximation is used for recovery and denoising, iteratively replacing the missing values. We present synthetic and field data examples to illustrate the method's effectiveness for denoising and interpolating 4D and 5D seismic data with randomly missing traces.

Sparse and Low-Rank Coupling Image Segmentation Model Via Nonconvex Regularization

2015 ◽  
Vol 29 (02) ◽  
pp. 1555004 ◽  
Author(s):  
Xiujun Zhang ◽  
Chen Xu ◽  
Min Li ◽  
Xiaoli Sun
Keyword(s):  
Image Segmentation ◽  
Sparse Representation ◽  
Optimal Solution ◽  
Low Rank ◽  
Public Image ◽  
Representation Problem ◽  
Norm Minimization ◽  
Augmented Lagrange Multiplier ◽  
Nonconvex Regularization ◽  
Low Rank Representation

This paper investigates how to boost region-based image segmentation by inheriting the advantages of sparse representation and low-rank representation. A novel image segmentation model, called nonconvex regularization based sparse and low-rank coupling model, is presented for such a purpose. We aim at finding the optimal solution which is provided with sparse and low-rank simultaneously. This is achieved by relaxing sparse representation problem as L1/2 norm minimization other than the L1 norm minimization, while relaxing low-rank representation problem as the S1/2 norm minimization other than the nuclear norm minimization. This coupled model can be solved efficiently through the Augmented Lagrange Multiplier (ALM) method and half-threshold operator. Compared to the other state-of-the-art methods, the new method is better at capturing the global structure of the whole data, the robustness is better and the segmentation accuracy is also competitive. Experiments on two public image segmentation databases well validate the superiority of our method.

scholarly journals Canopy Parameter Estimation of Citrus grandis var. Longanyou Based on LiDAR 3D Point Clouds

2021 ◽  
Vol 13 (9) ◽  
pp. 1859
Author(s):  
Xiangyang Liu ◽  
Yaxiong Wang ◽  
Feng Kang ◽  
Yang Yue ◽  
Yongjun Zheng
Keyword(s):  
Convex Hull ◽  
Clustering Algorithm ◽  
Plant Protection ◽  
Point Clouds ◽  
Biomass Estimation ◽  
Reconstruction Method ◽  
Volume Calculation ◽  
Alpha Shape ◽  
3D Point Clouds ◽  
Canopy Volume

The characteristic parameters of Citrus grandis var. Longanyou canopies are important when measuring yield and spraying pesticides. However, the feasibility of the canopy reconstruction method based on point clouds has not been confirmed with these canopies. Therefore, LiDAR point cloud data for C. grandis var. Longanyou were obtained to facilitate the management of groves of this species. Then, a cloth simulation filter and European clustering algorithm were used to realize individual canopy extraction. After calculating canopy height and width, canopy reconstruction and volume calculation were realized using six approaches: by a manual method and using five algorithms based on point clouds (convex hull, CH; convex hull by slices; voxel-based, VB; alpha-shape, AS; alpha-shape by slices, ASBS). ASBS is an innovative algorithm that combines AS with slices optimization, and can best approximate the actual canopy shape. Moreover, the CH algorithm had the shortest run time, and the R2 values of VCH, VVB, VAS, and VASBS algorithms were above 0.87. The volume with the highest accuracy was obtained from the ASBS algorithm, and the CH algorithm had the shortest computation time. In addition, a theoretical but preliminarily system suitable for the calculation of the canopy volume of C. grandis var. Longanyou was developed, which provides a theoretical reference for the efficient and accurate realization of future functional modules such as accurate plant protection, orchard obstacle avoidance, and biomass estimation.

scholarly journals Clipped Matrix Completion: A Remedy for Ceiling Effects

2019 ◽  
Vol 33 ◽  
pp. 5151-5158
Author(s):  
Takeshi Teshima ◽  
Miao Xu ◽  
Issei Sato ◽  
Masashi Sugiyama
Keyword(s):  
Matrix Completion ◽  
Low Rank ◽  
Trace Norm ◽  
Minimization Algorithm ◽  
Benchmark Data ◽  
Hinge Loss ◽  
Norm Minimization ◽  
Rank Matrix ◽  
Recovery Error ◽  
Low Rank Matrix

We consider the problem of recovering a low-rank matrix from its clipped observations. Clipping is conceivable in many scientific areas that obstructs statistical analyses. On the other hand, matrix completion (MC) methods can recover a low-rank matrix from various information deficits by using the principle of low-rank completion. However, the current theoretical guarantees for low-rank MC do not apply to clipped matrices, as the deficit depends on the underlying values. Therefore, the feasibility of clipped matrix completion (CMC) is not trivial. In this paper, we first provide a theoretical guarantee for the exact recovery of CMC by using a trace-norm minimization algorithm. Furthermore, we propose practical CMC algorithms by extending ordinary MC methods. Our extension is to use the squared hinge loss in place of the squared loss for reducing the penalty of overestimation on clipped entries. We also propose a novel regularization term tailored for CMC. It is a combination of two trace-norm terms, and we theoretically bound the recovery error under the regularization. We demonstrate the effectiveness of the proposed methods through experiments using both synthetic and benchmark data for recommendation systems.

Distance Density Based Clustering Algorithm in Wireless Sensor Network

2011 ◽  
Vol 291-294 ◽  
pp. 344-348
Author(s):  
Lin Lin ◽  
Shu Yan ◽  
Yi Nian
Keyword(s):  
Clustering Algorithm ◽  
Distribution Density ◽  
Simulation Experiment ◽  
Clustering Algorithms ◽  
Wireless Sensor ◽  
Energy Usage ◽  
Cluster Heads ◽  
Hierarchical Topology ◽  
Energy Factors ◽  
Density Based Clustering

The hierarchical topology of wireless sensor networks can effectively reduce the consumption in communication. Clustering algorithm is the foundation to realize herarchical structure, so it has been extensive researched. On the basis of Leach algorithm, a distance density based clustering algorithm (DDBC) is proposed, considering synthetically the distribution density of around nodes and the remaining energy factors of the node to dynamically banlance energy usage of nodes when selecting cluster heads. We analyzed the performance of DDBC through compared with the existing other clustering algorithms in simulation experiment. Results show that the proposed method can generare stable quantity cluster heads and banlance the energy load effectively.

Sign in / Sign up

Export Citation Format

Share Document