Data modelling approaches to astronomical data: Mapping large spectral line data cubes to dimensional data models

Data held in information systems is modelled using a variety of languages, where the choice of language may be decided by functional concerns as well as non-technical concerns. This chapter focuses on data modelling languages, and the challenges faced in mapping schemas in one data modelling language into another data modelling language. We review the ER, relational and UML modelling languages (the later being representative of object oriented programming languages), highlighting aspects of each modelling language that are not representable in the others. We describe how a nested hypergraph data model may be used as an underlying representation of data models, and hence present the differences between the modelling languages in a more precise manner. Finally, we propose a platform for the future building of an automated procedure for translating schemas from one modelling language to another.

Download Full-text

Bibliography, catalogs, pixel data: Management of heterogeneous Big Data at CDS by the documentalists

EPJ Web of Conferences ◽

10.1051/epjconf/201818602001 ◽

2018 ◽

Vol 186 ◽

pp. 02001 ◽

Cited By ~ 1

Author(s):

M. Buga ◽

P. Fernique ◽

C. Bot ◽

M. G. Allen ◽

F. Bonnarel ◽

...

Keyword(s):

Data Storage ◽

High Speed ◽

Three Dimensional ◽

Numerical Data ◽

Large Data ◽

Heterogeneous Data ◽

Data Sets ◽

Astronomical Data ◽

Data Cubes

High speed Internet and the evolution of data storage space in terms of cost-effectiveness has changed the way data are managed today. Large amounts of heterogeneous data can now be visualized easily and rapidly using interactive applications such as “Google Maps”. In this respect, the Hierarchical Progressive Survey (HiPS) method has been developed by the Centre de Données astronomiques de Strasbourg (CDS) since 2009. HiPS uses the hierarchical sky tessellation called HEALPix to describe and organize images, data cubes or source catalogs. These HiPS can be accessed and visualized using applications such as Aladin. We show that structuring the data using HiPS enables easy and quick access to large and complex sets of astronomical data. As with bibliographic and catalog data, full documentation and comprehensive metadata are absolutely required for pertinent usage of these data. Hence the role of documentalists in the process of producing HiPS is essential. We present the interaction between documentalists and other specialists who are all part of the CDS team and support this process. More precisely, we describe the tools used by the documentalists to generate HiPS or to update the Virtual Observatory standardized descriptive information (the “metadata”). We also present the challenges faced by the documentalists processing such heterogeneous data on the scales of megabytes up to petabytes. On one hand, documentalists at CDS manage small size textual or numerical data for one or few astronomical objects. On the other hand, they process large data sets such as big catalogs containing heterogeneous data like spectra, images or data cubes, for millions of astronomical objects. Finally, by participating in the development of an interactive visualization of images or three-dimensional data cubes using the HiPS method, documentalists contribute to a long-term management of complex, large astronomical data.

Download Full-text

Spectral Line Identification and Modelling (SLIM) in the MAdrid Data CUBe Analysis (MADCUBA) package

Astronomy and Astrophysics ◽

10.1051/0004-6361/201936144 ◽

2019 ◽

Vol 631 ◽

pp. A159 ◽

Cited By ~ 12

Author(s):

S. Martín ◽

J. Martín-Pintado ◽

C. Blanco-Sánchez ◽

V. M. Rivilla ◽

A. Rodríguez-Franco ◽

...

Keyword(s):

Radiative Transfer ◽

Spectral Line ◽

Transfer Equation ◽

Spectroscopic Data ◽

Radiative Transfer Equation ◽

Data Cube ◽

Physical Parameters ◽

Data Cubes ◽

Line Identification ◽

Wide Range

Context. The increase in bandwidth and sensitivity of state-of-the-art radio observatories is providing a wealth of molecular data from nearby star-forming regions up to high-z galaxies. Analysing large data sets of spectral cubes requires efficient and user-friendly tools optimised for astronomers with a wide range of backgrounds. Aims. In this paper we present the detailed formalism at the core of Spectral Line Identification and Modelling (SLIM) within the MAdrid Data CUBe Analysis (MADCUBA) package and their main data-handling functionalities. These tools have been developed to visualise, analyse, and model large spectroscopic data cubes. Methods. We present the highly interactive on-the-fly visualisation and modelling tools of MADCUBA and SLIM, which includes a stand-alone spectroscopic database. The parameters stored therein are used to solve the full radiative transfer equation under local thermodynamic equilibrium (LTE). The SLIM package provides tools to generate synthetic LTE model spectra based on input physical parameters of column density, excitation temperature, velocity, line width, and source size. It also provides an automatic fitting algorithm to obtain the physical parameters (with their associated errors) better fitting the observations. Synthetic spectra can be overlayed in the data cubes/spectra to ease the task of multi-molecular line identification and modelling. Results. We present the Java-based MADCUBA and its internal module SLIM packages which provide all the necessary tools for manipulation and analysis of spectroscopic data cubes. We describe in detail the spectroscopic fitting equations and make use of this tool to explore the breaking conditions and implicit errors of commonly used approximations in the literature. Conclusions. Easy-to-use tools like MADCUBA allow users to derive physical information from spectroscopic data without the need for simple approximations. The SLIM tool allows the full radiative transfer equation to be used, and to interactively explore the space of physical parameters and associated uncertainties from observational data.

Download Full-text

Potential integration of multi-fitting, inverse problem and mechanistic modelling approaches to applied research in animal science: a review

Animal Production Science ◽

10.1071/an14568 ◽

2014 ◽

Vol 54 (12) ◽

pp. 1905 ◽

Cited By ~ 3

Author(s):

L. M. Vargas-Villamil ◽

L. O. Tedeschi

Keyword(s):

Applied Research ◽

Integrated Approach ◽

Integral Approach ◽

Biological Processes ◽

Data Modelling ◽

Animal Science ◽

Potential Capacity ◽

Complex Models ◽

Number Of Publications ◽

Modelling Approaches

Modern researchers working in applied animal science systems have faced issues with modelling huge quantities of data. Modelling approaches that have previously been used to model biological systems are having problems to adapt to increased number of publications and research. So as to develop new approaches that have the potential to deal with these fast-changing complex conditions, it is relevant to review modern modelling approaches that have been used successfully in other fields. Therefore, this paper reviews the potential capacity of new integrated applied animal-science approaches to discriminate parameters, interpret data and understand biological processes. The analysis shows that the principal challenge is handling ill-conditioned complex models, but an integrated approach can obtain meaningful information from complementary data that cannot be obtained from present applied animal-science approaches. Furthermore, it is shown that parameter sloppiness and data complementarity are key concepts during system behaviour restrictions and parameter discrimination. Additionally, model evaluation and implementation of the potential integrated approach are reviewed. Finally, the objective of an integral approach is discussed. Our conclusion is that these approaches have the potential to be used to deepen the understanding of applied animal systems, and that exist enough developed resources and methodologies to deal with the huge quantities of data associated with this science.

Download Full-text

Basic Testing of the duchamp Source Finder

Publications of the Astronomical Society of Australia ◽

10.1071/as11041 ◽

2012 ◽

Vol 29 (3) ◽

pp. 276-295 ◽

Cited By ~ 9

Author(s):

T. Westmeier ◽

A. Popping ◽

P. Serra

Keyword(s):

Spectral Line ◽

Spectroscopic Data ◽

Source Parameters ◽

Systematic Errors ◽

Three Dimensions ◽

Second Step ◽

Signal To Noise ◽

Powerful Source ◽

Data Cubes ◽

Basic Source

AbstractThis paper presents and discusses the results of basic source finding tests in three dimensions (using spectroscopic data cubes) with duchamp, the standard source finder for the Australian Square Kilometre Array Pathfinder. For this purpose, we generated different sets of unresolved and extended Hi model sources. These models were then fed into duchamp, using a range of different parameters and methods provided by the software. The main aim of the tests was to study the performance of duchamp on sources with different parameters and morphologies and assess the accuracy of duchamp's source parametrisation. Overall, we find duchamp to be a powerful source finder capable of reliably detecting sources down to low signal-to-noise ratios and accurately measuring their position and velocity. In the presence of noise in the data, duchamp's measurements of basic source parameters, such as spectral line width and integrated flux, are affected by systematic errors. These errors are a consequence of the effect of noise on the specific algorithms used by duchamp for measuring source parameters in combination with the fact that the software only takes into account pixels above a given flux threshold and hence misses part of the flux. In scientific applications of duchamp these systematic errors would have to be corrected for. Alternatively, duchamp could be used as a source finder only, and source parametrisation could be done in a second step using more sophisticated parametrisation algorithms.

Download Full-text

A Distributed Data Implementation of the Perspective Shear-Warp Volume Rendering Algorithm for Visualisation of Large Astronomical Cubes

Publications of the Astronomical Society of Australia ◽

10.1071/as03039 ◽

2003 ◽

Vol 20 (3) ◽

pp. 300-313 ◽

Cited By ~ 8

Author(s):

Brett Beeson ◽

David G. Barnes ◽

Paul D. Bourke

Keyword(s):

Volume Rendering ◽

Distributed Data ◽

Stereoscopic Display ◽

Worst Case ◽

Astronomical Data ◽

3 Dimensional ◽

Data Cubes ◽

Commodity Hardware ◽

Data Volume ◽

Combined Images

AbstractWe describe the first distributed data implementation of the perspective shear-warp volume rendering algorithm and explore its applications to large astronomical data cubes and simulation realisations. Our system distributes sub-volumes of 3-dimensional images to leaf nodes of a Beowulf-class cluster, where the rendering takes place. Junction nodes composite the sub-volume renderings together and pass the combined images upwards for further compositing or display. We demonstrate that our system out-performs other software solutions and can render a 'worst-case' 512 × 512 × 512 data volume in less than four seconds using 16 rendering and 15 compositing nodes. Our system also performs very well compared with much more expensive hardware systems. With appropriate commodity hardware, such as Swinburne's Virtual Reality Theatre or a 3Dlabs Wildcat graphics card, stereoscopic display is possible.

Download Full-text