Supporting computational data model representation with high-performance I/O in parallel netCDF

Recent achievements in high-performance computing significantly narrow the performance gap between single and multi-node computing, and open up opportunities for systems with remote shared memory. The combination of in-memory storage, remote direct memory access and remote calls requires rethinking how data organized, protected and queried in distributed systems. Reviewed models let us implement new interpretations of distributed algorithms allowing us to validate different approaches to avoid race conditions, decrease resource acquisition or synchronization time. In this paper, we describe the data model for mixed memory access with analysis of optimized data structures. We also provide the result of experiments, which contain a performance comparison of data structures, operating with different approaches, evaluate the limitations of these models, and show that the model does not always meet expectations. The purpose of this paper to assist developers in designing data structures that will help to achieve architectural benefits or improve the design of existing distributed system.

Download Full-text

Comparative Evaluation of Large Data Model Representation Methods: The Analyst’s Perspective

Conceptual Modeling — ER 2002 - Lecture Notes in Computer Science ◽

10.1007/3-540-45816-6_25 ◽

2002 ◽

pp. 214-231 ◽

Cited By ~ 11

Author(s):

Daniel L. Moody

Keyword(s):

Data Model ◽

Comparative Evaluation ◽

Large Data ◽

Model Representation

Download Full-text

A Hybrid Framework for High-Performance Modeling of Three-Dimensional Pipe Networks

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi8100441 ◽

2019 ◽

Vol 8 (10) ◽

pp. 441 ◽

Cited By ~ 2

Author(s):

Shaohua Wang ◽

Yeran Sun ◽

Yinle Sun ◽

Yong Guan ◽

Zhenhua Feng ◽

...

Keyword(s):

Spatial Data ◽

Data Model ◽

High Performance ◽

3D Model ◽

Performance Modeling ◽

Three Dimensional ◽

Network Data ◽

Pipe Network ◽

Pipe Networks ◽

Hybrid Framework

Three-dimensional (3D) pipe network modeling plays an essential part in high performance-based smart city applications. Given that massive 3D pipe networks tend to be difficult to manage and to visualize, we propose in this study a hybrid framework for high-performance modeling of a 3D pipe network, including pipe network data model and high-performance modeling. The pipe network data model is devoted to three-dimensional pipe network construction based on network topology and building information models (BIMs). According to the topological relationships of the pipe point pipelines, the pipe network is decomposed into multiple pipe segment units. The high-performance modeling of 3D pipe network contains a spatial 3D model, the instantiation, adaptive rendering, and combination parallel computing. Spatial 3D model (S3M) is proposed for spatial data transmission, exchange, and visualization of massive and multi-source 3D spatial data. The combination parallel computing framework with GPU and OpenMP was developed to reduce the processing time for pipe networks. The results of the experiments showed that the hybrid framework achieves a high efficiency and the hardware resource occupation is reduced.

Download Full-text

ACDS: Adapting computational data streams for high performance

Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000 ◽

10.1109/ipdps.2000.846046 ◽

2002 ◽

Cited By ~ 39

Author(s):

C. Isert ◽

K. Schwan

Keyword(s):

Data Streams ◽

High Performance ◽

Computational Data

Download Full-text

End-to-End provenance representation for the understandability and reproducibility of scientific experiments using a semantic approach

Journal of Biomedical Semantics ◽

10.1186/s13326-021-00253-1 ◽

2022 ◽

Vol 13 (1) ◽

Author(s):

Sheeba Samuel ◽

Birgitta König-Ries

Keyword(s):

Data Model ◽

Biological Imaging ◽

Computational Science ◽

Semantic Approach ◽

Computational Data ◽

Scientific Experiments ◽

End To End ◽

Computational Processes ◽

Processing Steps ◽

Complete Path

Abstract Background The advancement of science and technologies play an immense role in the way scientific experiments are being conducted. Understanding how experiments are performed and how results are derived has become significantly more complex with the recent explosive growth of heterogeneous research data and methods. Therefore, it is important that the provenance of results is tracked, described, and managed throughout the research lifecycle starting from the beginning of an experiment to its end to ensure reproducibility of results described in publications. However, there is a lack of interoperable representation of end-to-end provenance of scientific experiments that interlinks data, processing steps, and results from an experiment’s computational and non-computational processes. Results We present the “REPRODUCE-ME” data model and ontology to describe the end-to-end provenance of scientific experiments by extending existing standards in the semantic web. The ontology brings together different aspects of the provenance of scientific studies by interlinking non-computational data and steps with computational data and steps to achieve understandability and reproducibility. We explain the important classes and properties of the ontology and how they are mapped to existing ontologies like PROV-O and P-Plan. The ontology is evaluated by answering competency questions over the knowledge base of scientific experiments consisting of computational and non-computational data and steps. Conclusion We have designed and developed an interoperable way to represent the complete path of a scientific experiment consisting of computational and non-computational steps. We have applied and evaluated our approach to a set of scientific experiments in different subject domains like computational science, biological imaging, and microscopy.

Download Full-text