Automatic classification of sources in large astronomical catalogs

In this paper we address two questions related to data analysis in large astronomical datasets, and we demonstrate how they can be answered making use of machine learning techniques. The first question is: how to efficiently find previously unknown or rare objects which can be expected to exist in big data samples? Using the largest existing extragalactic all-sky survey, provided by the WISE satellite, we demonstrate that, surprisingly, supervised classification methods can come to aid. The second question is: having a sufficiently large data sample, how can we look for new optimal classification schemes, possibly finding new and previously unknown classes and subclasses of sources? Based on the VIPERS cutting-edge galaxy catalog at redshift z > 0.5, we demonstrate that unsupervised classification methods can give unexpected but physically well-motivated results.

Mathematical Problems in Creating Large Astronomical Catalogs

The next stage after performing observations and their primary reduction is to transform the set of observations into a catalog. To this end, objects that are irrelevant to the catalog should be excluded from observations and gross errors should be discarded. To transform such a prepared data set into a high-precision catalog, we need to identify and correct systematic errors. Therefore, each object of the survey should be observed several, preferably many, times. The problem formally reduces to solving an overdetermined set of equations. However, in the case of catalogs this system of equations has a very specific form: it is extremely sparse, and its sparseness increases rapidly with the number of objects in the catalog. Such equation systems require special methods for storing data on disks and in RAM, and for the choice of the techniques for their solving. Another specific feature of such systems is their high “stiffiness”, which also increases with the volume of a catalog. Special stable mathematical methods should be used in order not to lose precision when solving such systems of equations. We illustrate the problem by the example of photometric star catalogs, although similar problems arise in the case of positional, radial-velocity, and parallax catalogs.