scholarly journals Supernova Host Galaxy Association and Photometric Classification of over 10,000 Light Curves from the Zwicky Transient Facility

2021 ◽  
Vol 5 (12) ◽  
pp. 283
Author(s):  
Braden Garretson ◽  
Dan Milisavljevic ◽  
Jack Reynolds ◽  
Kathryn E. Weil ◽  
Bhagya Subrayan ◽  
...  
Keyword(s):  
Value Added ◽  
Light Curves ◽  
Host Galaxy ◽  
Massive Data ◽  
Data Sets ◽  
Data Set ◽  
Scale Modeling ◽  
Final Data ◽  
Type Ia

Abstract Here we present a catalog of 12,993 photometrically-classified supernova-like light curves from the Zwicky Transient Facility, along with candidate host galaxy associations. By training a random forest classifier on spectroscopically classified supernovae from the Bright Transient Survey, we achieve an accuracy of 80% across four supernova classes resulting in a final data set of 8208 Type Ia, 2080 Type II, 1985 Type Ib/c, and 720 SLSN. Our work represents a pathfinder effort to supply massive data sets of supernova light curves with value-added information that can be used to enable population-scale modeling of explosion parameters and investigate host galaxy environments.

scholarly journals ParSNIP: Generative Models of Transient Light Curves with Physics-enabled Deep Learning

2021 ◽  
Vol 162 (6) ◽  
pp. 275
Author(s):  
Kyle Boone
Keyword(s):  
Three Dimensional ◽  
Generative Models ◽  
Light Curves ◽  
Type Ia Supernovae ◽  
Data Sets ◽  
Data Set ◽  
Out Of Sample ◽  
Type Ia ◽  
Ia Supernovae

Abstract We present a novel method to produce empirical generative models of all kinds of astronomical transients from data sets of unlabeled light curves. Our hybrid model, which we call ParSNIP, uses a neural network to model the unknown intrinsic diversity of different transients and an explicit physics-based model of how light from the transient propagates through the universe and is observed. The ParSNIP model predicts the time-varying spectra of transients despite only being trained on photometric observations. With a three-dimensional intrinsic model, we are able to fit out-of-sample multiband light curves of many different kinds of transients with model uncertainties of 0.04–0.06 mag. The representation learned by the ParSNIP model is invariant to redshift, so it can be used to perform photometric classification of transients even with heavily biased training sets. Our classification techniques significantly outperform state-of-the-art methods on both simulated (PLAsTiCC) and real (PS1) data sets with 2.3× and 2× less contamination, respectively, for classification of Type Ia supernovae. We demonstrate how our model can identify previously unobserved kinds of transients and produce a sample that is 90% pure. The ParSNIP model can also estimate distances to Type Ia supernovae in the PS1 data set with an rms of 0.150 ± 0.007 mag compared to 0.155 ± 0.008 mag for the SALT2 model on the same sample. We discuss how our model could be used to produce distance estimates for supernova cosmology without the need for explicit classification.

scholarly journals Classification of jujube defects in small data sets based on transfer learning

2021 ◽  
Author(s):  
Jianping Ju ◽  
Hong Zheng ◽  
Xiaohang Xu ◽  
Zhongyuan Guo ◽  
Zhaohui Zheng ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Loss Function ◽  
Training Model ◽  
Parameter Distribution ◽  
Test Accuracy ◽  
Small Data ◽  
Data Sets ◽  
Data Set ◽  
Small Data Sets

AbstractAlthough convolutional neural networks have achieved success in the field of image classification, there are still challenges in the field of agricultural product quality sorting such as machine vision-based jujube defects detection. The performance of jujube defect detection mainly depends on the feature extraction and the classifier used. Due to the diversity of the jujube materials and the variability of the testing environment, the traditional method of manually extracting the features often fails to meet the requirements of practical application. In this paper, a jujube sorting model in small data sets based on convolutional neural network and transfer learning is proposed to meet the actual demand of jujube defects detection. Firstly, the original images collected from the actual jujube sorting production line were pre-processed, and the data were augmented to establish a data set of five categories of jujube defects. The original CNN model is then improved by embedding the SE module and using the triplet loss function and the center loss function to replace the softmax loss function. Finally, the depth pre-training model on the ImageNet image data set was used to conduct training on the jujube defects data set, so that the parameters of the pre-training model could fit the parameter distribution of the jujube defects image, and the parameter distribution was transferred to the jujube defects data set to complete the transfer of the model and realize the detection and classification of the jujube defects. The classification results are visualized by heatmap through the analysis of classification accuracy and confusion matrix compared with the comparison models. The experimental results show that the SE-ResNet50-CL model optimizes the fine-grained classification problem of jujube defect recognition, and the test accuracy reaches 94.15%. The model has good stability and high recognition accuracy in complex environments.

scholarly journals Do galactic bars depend on environment?: an information theoretic analysis of Galaxy Zoo 2

2020 ◽  
Vol 501 (1) ◽  
pp. 994-1001
Author(s):  
Suman Sarkar ◽  
Biswajit Pandey ◽  
Snehasish Bhattacharjee
Keyword(s):  
Spatial Distribution ◽  
Mutual Information ◽  
Local Density ◽  
Statistical Significance ◽  
Distribution Functions ◽  
Cumulative Distribution ◽  
Host Galaxy ◽  
Data Sets ◽  
Data Set ◽  
Information Theoretic

ABSTRACT We use an information theoretic framework to analyse data from the Galaxy Zoo 2 project and study if there are any statistically significant correlations between the presence of bars in spiral galaxies and their environment. We measure the mutual information between the barredness of galaxies and their environments in a volume limited sample (Mr ≤ −21) and compare it with the same in data sets where (i) the bar/unbar classifications are randomized and (ii) the spatial distribution of galaxies are shuffled on different length scales. We assess the statistical significance of the differences in the mutual information using a t-test and find that both randomization of morphological classifications and shuffling of spatial distribution do not alter the mutual information in a statistically significant way. The non-zero mutual information between the barredness and environment arises due to the finite and discrete nature of the data set that can be entirely explained by mock Poisson distributions. We also separately compare the cumulative distribution functions of the barred and unbarred galaxies as a function of their local density. Using a Kolmogorov–Smirnov test, we find that the null hypothesis cannot be rejected even at $75{{\ \rm per\ cent}}$ confidence level. Our analysis indicates that environments do not play a significant role in the formation of a bar, which is largely determined by the internal processes of the host galaxy.

scholarly journals Fundamental resource trade-offs for encoded distributed optimization

2020 ◽  
Author(s):  
A Salman Avestimehr ◽  
Seyed Mohammadreza Mousavi Kalan ◽  
Mahdi Soltanolkotabi
Keyword(s):  
Computational Time ◽  
Massive Data ◽  
Data Sets ◽  
Massive Data Sets ◽  
Computational Framework ◽  
Data Set ◽  
Trade Offs ◽  
Major Bottleneck ◽  
Computing Environments ◽  
Analyze Data

Abstract Dealing with the shear size and complexity of today’s massive data sets requires computational platforms that can analyze data in a parallelized and distributed fashion. A major bottleneck that arises in such modern distributed computing environments is that some of the worker nodes may run slow. These nodes a.k.a. stragglers can significantly slow down computation as the slowest node may dictate the overall computational time. A recent computational framework, called encoded optimization, creates redundancy in the data to mitigate the effect of stragglers. In this paper, we develop novel mathematical understanding for this framework demonstrating its effectiveness in much broader settings than was previously understood. We also analyze the convergence behavior of iterative encoded optimization algorithms, allowing us to characterize fundamental trade-offs between convergence rate, size of data set, accuracy, computational load (or data redundancy) and straggler toleration in this framework.

scholarly journals Sequential Sampling for Estimation and Classification of the Incidence of Hop Powdery Mildew II: Cone Sampling

Plant Disease ◽  
2007 ◽  
Vol 91 (8) ◽  
pp. 1013-1020 ◽  
Author(s):  
David H. Gent ◽  
William W. Turechek ◽  
Walter F. Mahaffee
Keyword(s):  
Powdery Mildew ◽  
Binomial Distribution ◽  
Disease Incidence ◽  
Sequential Sampling ◽  
Model Construction ◽  
Data Sets ◽  
Data Set ◽  
Sampling Plans ◽  
Simulated Sampling

Sequential sampling models for estimation and classification of the incidence of powdery mildew (caused by Podosphaera macularis) on hop (Humulus lupulus) cones were developed using parameter estimates of the binary power law derived from the analysis of 221 transect data sets (model construction data set) collected from 41 hop yards sampled in Oregon and Washington from 2000 to 2005. Stop lines, models that determine when sufficient information has been collected to estimate mean disease incidence and stop sampling, for sequential estimation were validated by bootstrap simulation using a subset of 21 model construction data sets and simulated sampling of an additional 13 model construction data sets. Achieved coefficient of variation (C) approached the prespecified C as the estimated disease incidence, [Formula: see text], increased, although achieving a C of 0.1 was not possible for data sets in which [Formula: see text] < 0.03 with the number of sampling units evaluated in this study. The 95% confidence interval of the median difference between [Formula: see text] of each yard (achieved by sequential sampling) and the true p of the original data set included 0 for all 21 data sets evaluated at levels of C of 0.1 and 0.2. For sequential classification, operating characteristic (OC) and average sample number (ASN) curves of the sequential sampling plans obtained by bootstrap analysis and simulated sampling were similar to the OC and ASN values determined by Monte Carlo simulation. Correct decisions of whether disease incidence was above or below prespecified thresholds (pt) were made for 84.6 or 100% of the data sets during simulated sampling when stop lines were determined assuming a binomial or beta-binomial distribution of disease incidence, respectively. However, the higher proportion of correct decisions obtained by assuming a beta-binomial distribution of disease incidence required, on average, sampling 3.9 more plants per sampling round to classify disease incidence compared with the binomial distribution. Use of these sequential sampling plans may aid growers in deciding the order in which to harvest hop yards to minimize the risk of a condition called “cone early maturity” caused by late-season infection of cones by P. macularis. Also, sequential sampling could aid in research efforts, such as efficacy trials, where many hop cones are assessed to determine disease incidence.

scholarly journals The GTC exoplanet transit spectroscopy survey

2019 ◽  
Vol 622 ◽  
pp. A172 ◽  
Author(s):  
F. Murgas ◽  
G. Chen ◽  
E. Pallé ◽  
L. Nortmann ◽  
G. Nowak
Keyword(s):  
Transmission Spectrum ◽  
Rayleigh Scattering ◽  
Extrasolar Planets ◽  
Light Curves ◽  
Data Sets ◽  
Transmission Spectra ◽  
Data Set ◽  
Systematic Effects ◽  
The Individual ◽  
Spectrum Slope

Context. Rayleigh scattering in a hydrogen-dominated exoplanet atmosphere can be detected using ground- or space-based telescopes. However, stellar activity in the form of spots can mimic Rayleigh scattering in the observed transmission spectrum. Quantifying this phenomena is key to our correct interpretation of exoplanet atmospheric properties. Aims. We use the ten-meter Gran Telescopio Canarias (GTC) telescope to carry out a ground-based transmission spectra survey of extrasolar planets to characterize their atmospheres. In this paper we investigate the exoplanet HAT-P-11b, a Neptune-sized planet orbiting an active K-type star. Methods. We obtained long-slit optical spectroscopy of two transits of HAT-P-11b with the Optical System for Imaging and low-Intermediate-Resolution Integrated Spectroscopy (OSIRIS) on August 30, 2016 and September 25, 2017. We integrated the spectrum of HAT-P-11 and one reference star in several spectroscopic channels across the λ ~ 400–785 nm region, creating numerous light curves of the transits. We fit analytic transit curves to the data taking into account the systematic effects and red noise present in the time series in an effort to measure the change of the planet-to-star radius ratio (Rp∕Rs) across wavelength. Results. By fitting both transits together, we find a slope in the transmission spectrum showing an increase of the planetary radius towards blue wavelengths. Closer inspection of the transmission spectrum of the individual data sets reveals that the first transit presents this slope while the transmission spectrum of the second data set is flat. Additionally, we detect hints of Na absorption on the first night, but not on the second. We conclude that the transmission spectrum slope and Na absorption excess found in the first transit observation are caused by unocculted stellar spots. Modeling the contribution of unocculted spots to reproduce the results of the first night we find a spot filling factor of δ = 0.62−0.17+0.20 and a spot-to-photosphere temperature difference of ΔT = 429−299+184 K.

Massive Data Classification of Neural Responses

2010 ◽  
pp. 278-298
Author(s):  
Pedro Tomás ◽  
IST TU Lisbon ◽  
Aleksandar Ilic ◽  
Leonel Sousa
Keyword(s):  
Execution Time ◽  
Data Parallelism ◽  
Data Sets ◽  
Neural Responses ◽  
Neuronal Responses ◽  
Data Set ◽  
Web Interfaces ◽  
Mass Classification ◽  
Neuronal Code

When analyzing the neuronal code, neuroscientists usually perform extra-cellular recordings of neuronal responses (spikes). Since the size of the microelectrodes used to perform these recordings is much larger than the size of the cells, responses from multiple neurons are recorded by each micro-electrode. Thus, the obtained response must be classified and evaluated, in order to identify how many neurons were recorded, and to assess which neuron generated each spike. A platform for the mass-classification of neuronal responses is proposed in this chapter, employing data-parallelism for speeding up the classification of neuronal responses. The platform is built in a modular way, supporting multiple web-interfaces, different back-end environments for parallel computing or different algorithms for spike classification. Experimental results on the proposed platform show that even for an unbalanced data set of neuronal responses the execution time was reduced of about 45%. For balanced data sets, the platform may achieve a reduction in execution time equal to the inverse of the number of back-end computational elements.

scholarly journals On the cosmological performance of photometrically classified supernovae with machine learning

2020 ◽  
Vol 497 (3) ◽  
pp. 2974-2991
Author(s):  
Marcelo Vargas dos Santos ◽  
Miguel Quartin ◽  
Ribamar R R Reis
Keyword(s):  
Machine Learning ◽  
Mean Squared Error ◽  
Light Curves ◽  
Observational Cosmology ◽  
Automated Classification ◽  
Cosmological Constraints ◽  
Search Processes ◽  
Type Ia ◽  
Ia Supernovae

ABSTRACT The efficient classification of different types of supernovae is one of the most important problems for observational cosmology. However, spectroscopic confirmation of most objects in upcoming photometric surveys, such as the the Rubin Observatory Legacy Survey of Space and Time, will be unfeasible. The development of automated classification processes based on photometry has thus become crucial. In this paper, we investigate the performance of machine learning (ML) classification on the final cosmological constraints using simulated light-curves from the Supernova Photometric Classification Challenge, released in 2010. We study the use of different feature sets for the light-curves and many different ML pipelines based on either decision-tree ensembles or automated search processes. To construct the final catalogues we propose a threshold selection method, by employing a bias-variance tradeoff. This is a very robust and efficient way to minimize the mean squared error. With this method, we were able to obtain very strong cosmological constraints, which allowed us to keep $\sim 75{{\ \rm per\ cent}}$ of the total information in the Type Ia supernovae when using the SALT2 feature set, and $\sim 33{{\ \rm per\ cent}}$ for the other cases (based either on the Newling model or on standard wavelet decomposition).

Bagging Approach for Medical Plants Recognition Based on Their DNA Sequences

2018 ◽  
Vol 9 (4) ◽  
pp. 45-60
Author(s):  
Mohamed Elhadi Rahmani ◽  
Abdelmalek Amine ◽  
Reda Mohamed Hamou
Keyword(s):  
Dna Sequences ◽  
Majority Vote ◽  
Data Sets ◽  
Data Set ◽  
Drug Production ◽  
Medical Plants

Many drugs in modern medicines originate from plants and the first step in drug production, is the recognition of plants needed for this purpose. This article presents a bagging approach for medical plants recognition based on their DNA sequences. In this work, the authors have developed a system that recognize DNA sequences of 14 medical plants, first they divided the 14-class data set into bi class sub-data sets, then instead of using an algorithm to classify the 14-class data set, they used the same algorithm to classify the sub-data sets. By doing so, they have simplified the problem of classification of 14 plants into sub-problems of bi class classification. To construct the subsets, the authors extracted all possible pairs of the 14 classes, so they gave each class more chances to be well predicted. This approach allows the study of the similarity between DNA sequences of a plant with each other plants. In terms of results, the authors have obtained very good results in which the accuracy has been doubled (from 45% to almost 80%). Classification of a new sequence was completed according to majority vote.

An Improved Classification Analysis on Utility Aware K-Anonymized Dataset

2019 ◽  
Vol 16 (2) ◽  
pp. 445-452
Author(s):  
Kishore S. Verma ◽  
A. Rajesh ◽  
Adeline J. S. Johnsana
Keyword(s):  
Data Mining ◽  
Analytical Approach ◽  
Value Added ◽  
Data Sets ◽  
Data Set ◽  
Privacy Preserving Data Mining ◽  
Privacy Leakage ◽  
Anonymized Data ◽  
Null Values ◽  
The Individual

K anonymization is one of the worldwide used approaches to protect the individual records from the privacy leakage attack of Privacy Preserving Data Mining (PPDM) arena. Typically anonymized dataset will impact the effectiveness of data mining results. Anyhow, currently researchers of PPDM progress in driving their efforts in finding out the optimum trade-off between privacy and utility. This work tends in bringing out the optimum classifier from a set of best classifiers of data mining approaches that are capable enough in generating value-added classifying results on utility aware k-anonymized data set. We performed the analytical approach on the data set that are anonymized in sense of accompanying the anonymity utility factors like null values count and transformation pattern loss. The experimentation is done with three widely used classifiers HNB, PART and J48 and these classifiers are analysed with Accuracy, F-measure, and ROC-AUC which are literately proved to be the perfect measures of classification. Our experimental analysis reveals the best classifiers on the utility aware anonymized data sets of Cell oriented Anonymization (CoA), Attribute oriented Anonymization (AoA) and Record oriented Anonymization (RoA).

Sign in / Sign up

Export Citation Format

Share Document