Survey of Clustering Methods for Large Scale Dataset

2019 ◽  
Vol 7 (5) ◽  
pp. 1338-1344
Author(s):  
Anupama Jawale ◽  
Ganesh Magar
Keyword(s):  
Large Scale ◽  
Clustering Methods ◽  
Large Scale Dataset

scholarly journals Discovering Distinct Patterns in Gene Expression Profiles

2008 ◽  
Vol 5 (2) ◽  
Author(s):  
Li Teng ◽  
Laiwan Chan
Keyword(s):  
Gene Expression ◽  
Large Scale ◽  
Expression Profiles ◽  
Expression Patterns ◽  
Gene Expression Profiles ◽  
Clustering Methods ◽  
Gene Expressions ◽  
Real Gene ◽  
Large Scale Dataset ◽  
Coexpressed Genes

SummaryTraditional analysis of gene expression profiles use clustering to find groups of coexpressed genes which have similar expression patterns. However clustering is time consuming and could be diffcult for very large scale dataset. We proposed the idea of Discovering Distinct Patterns (DDP) in gene expression profiles. Since patterns showing by the gene expressions reveal their regulate mechanisms. It is significant to find all different patterns existing in the dataset when there is little prior knowledge. It is also a helpful start before taking on further analysis. We propose an algorithm for DDP by iteratively picking out pairs of gene expression patterns which have the largest dissimilarities. This method can also be used as preprocessing to initialize centers for clustering methods, like K-means. Experiments on both synthetic dataset and real gene expression datasets show our method is very effective in finding distinct patterns which have gene functional significance and is also effcient.

scholarly journals Joint regression and learning from pairwise rankings for personalized image aesthetic assessment

2021 ◽  
Author(s):  
Jin Zhou ◽  
Qing Zhang ◽  
Jian-Hao Fan ◽  
Wei Sun ◽  
Wei-Shi Zheng
Keyword(s):  
Large Scale ◽  
Assessment Model ◽  
Generic Model ◽  
Small Subset ◽  
Deep Convolutional Neural Networks ◽  
Personal Taste ◽  
Hinge Loss ◽  
Novel Approach ◽  
Large Scale Dataset ◽  
Image Pairs

AbstractRecent image aesthetic assessment methods have achieved remarkable progress due to the emergence of deep convolutional neural networks (CNNs). However, these methods focus primarily on predicting generally perceived preference of an image, making them usually have limited practicability, since each user may have completely different preferences for the same image. To address this problem, this paper presents a novel approach for predicting personalized image aesthetics that fit an individual user’s personal taste. We achieve this in a coarse to fine manner, by joint regression and learning from pairwise rankings. Specifically, we first collect a small subset of personal images from a user and invite him/her to rank the preference of some randomly sampled image pairs. We then search for the K-nearest neighbors of the personal images within a large-scale dataset labeled with average human aesthetic scores, and use these images as well as the associated scores to train a generic aesthetic assessment model by CNN-based regression. Next, we fine-tune the generic model to accommodate the personal preference by training over the rankings with a pairwise hinge loss. Experiments demonstrate that our method can effectively learn personalized image aesthetic preferences, clearly outperforming state-of-the-art methods. Moreover, we show that the learned personalized image aesthetic benefits a wide variety of applications.

scholarly journals VIPPrint: Validating Synthetic Image Detection and Source Linking Methods on a Large Scale Dataset of Printed Documents

2021 ◽  
Vol 7 (3) ◽  
pp. 50
Author(s):  
Anselmo Ferreira ◽  
Ehsan Nowroozi ◽  
Mauro Barni
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
Child Pornography ◽  
Forensic Analysis ◽  
Synthetic Image ◽  
Image Detection ◽  
Face Images ◽  
Large Scale Dataset ◽  
Scanned Images ◽  
Analysis Of The Images

The possibility of carrying out a meaningful forensic analysis on printed and scanned images plays a major role in many applications. First of all, printed documents are often associated with criminal activities, such as terrorist plans, child pornography, and even fake packages. Additionally, printing and scanning can be used to hide the traces of image manipulation or the synthetic nature of images, since the artifacts commonly found in manipulated and synthetic images are gone after the images are printed and scanned. A problem hindering research in this area is the lack of large scale reference datasets to be used for algorithm development and benchmarking. Motivated by this issue, we present a new dataset composed of a large number of synthetic and natural printed face images. To highlight the difficulties associated with the analysis of the images of the dataset, we carried out an extensive set of experiments comparing several printer attribution methods. We also verified that state-of-the-art methods to distinguish natural and synthetic face images fail when applied to print and scanned images. We envision that the availability of the new dataset and the preliminary experiments we carried out will motivate and facilitate further research in this area.

scholarly journals ShadingNet: Image Intrinsics by Fine-Grained Shading Decomposition

2021 ◽  
Author(s):  
Anil S. Baslamisli ◽  
Partha Das ◽  
Hoang-An Le ◽  
Sezer Karaoglu ◽  
Theo Gevers
Keyword(s):  
Neural Network ◽  
Large Scale ◽  
State Of The Art ◽  
Image Decomposition ◽  
Natural Environments ◽  
Decomposition Algorithms ◽  
Ambient Light ◽  
Fine Grained ◽  
Large Scale Dataset ◽  
Direct Illumination

AbstractIn general, intrinsic image decomposition algorithms interpret shading as one unified component including all photometric effects. As shading transitions are generally smoother than reflectance (albedo) changes, these methods may fail in distinguishing strong photometric effects from reflectance variations. Therefore, in this paper, we propose to decompose the shading component into direct (illumination) and indirect shading (ambient light and shadows) subcomponents. The aim is to distinguish strong photometric effects from reflectance variations. An end-to-end deep convolutional neural network (ShadingNet) is proposed that operates in a fine-to-coarse manner with a specialized fusion and refinement unit exploiting the fine-grained shading model. It is designed to learn specific reflectance cues separated from specific photometric effects to analyze the disentanglement capability. A large-scale dataset of scene-level synthetic images of outdoor natural environments is provided with fine-grained intrinsic image ground-truths. Large scale experiments show that our approach using fine-grained shading decompositions outperforms state-of-the-art algorithms utilizing unified shading on NED, MPI Sintel, GTA V, IIW, MIT Intrinsic Images, 3DRMS and SRD datasets.

scholarly journals Building Damage Detection Using U-Net with Attention Mechanism from Pre- and Post-Disaster Remote Sensing Datasets

2021 ◽  
Vol 13 (5) ◽  
pp. 905
Author(s):  
Chuyi Wu ◽  
Feng Zhang ◽  
Junshi Xia ◽  
Yichen Xu ◽  
Guoqing Li ◽  
...  
Keyword(s):  
Damage Assessment ◽  
Large Scale ◽  
Binary Classification ◽  
Open Data ◽  
Building Damage ◽  
Attention Mechanism ◽  
Large Scale Dataset ◽  
Data Program ◽  
The Impact ◽  
Post Disaster

The building damage status is vital to plan rescue and reconstruction after a disaster and is also hard to detect and judge its level. Most existing studies focus on binary classification, and the attention of the model is distracted. In this study, we proposed a Siamese neural network that can localize and classify damaged buildings at one time. The main parts of this network are a variety of attention U-Nets using different backbones. The attention mechanism enables the network to pay more attention to the effective features and channels, so as to reduce the impact of useless features. We train them using the xBD dataset, which is a large-scale dataset for the advancement of building damage assessment, and compare their result balanced F (F1) scores. The score demonstrates that the performance of SEresNeXt with an attention mechanism gives the best performance, with the F1 score reaching 0.787. To improve the accuracy, we fused the results and got the best overall F1 score of 0.792. To verify the transferability and robustness of the model, we selected the dataset on the Maxar Open Data Program of two recent disasters to investigate the performance. By visual comparison, the results show that our model is robust and transferable.

Large-scale dataset from China gives new insights into leaf margin–temperature relationships

2014 ◽  
Vol 402 ◽  
pp. 73-80 ◽  
Author(s):  
Wen-Yun Chen ◽  
Tao Su ◽  
Jonathan M. Adams ◽  
Frédéric M.B. Jacques ◽  
David K. Ferguson ◽  
...  
Keyword(s):  
Large Scale ◽  
Leaf Margin ◽  
Large Scale Dataset

scholarly journals TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild

2018 ◽  
pp. 310-327 ◽  
Author(s):  
Matthias Müller ◽  
Adel Bibi ◽  
Silvio Giancola ◽  
Salman Alsubaihi ◽  
Bernard Ghanem
Keyword(s):  
Object Tracking ◽  
Large Scale ◽  
Large Scale Dataset ◽  
In The Wild

scholarly journals Bayesian estimation of the seroprevalence of antibodies to SARS-CoV-2

JAMIA Open ◽  
2020 ◽  
Author(s):  
Qunfeng Dong ◽  
Xiang Gao
Keyword(s):  
Probability Distribution ◽  
Large Scale ◽  
Prior Probability ◽  
Probability Distributions ◽  
Single Point ◽  
Point Estimates ◽  
Specificity And Sensitivity ◽  
Large Scale Dataset ◽  
The Us ◽  
Antibody Tests

Abstract Accurate estimations of the seroprevalence of antibodies to severe acute respiratory syndrome coronavirus 2 need to properly consider the specificity and sensitivity of the antibody tests. In addition, prior knowledge of the extent of viral infection in a population may also be important for adjusting the estimation of seroprevalence. For this purpose, we have developed a Bayesian approach that can incorporate the variabilities of specificity and sensitivity of the antibody tests, as well as the prior probability distribution of seroprevalence. We have demonstrated the utility of our approach by applying it to a recently published large-scale dataset from the US CDC, with our results providing entire probability distributions of seroprevalence instead of single-point estimates. Our Bayesian code is freely available at https://github.com/qunfengdong/AntibodyTest.

Sign in / Sign up

Export Citation Format

Share Document