Support for Computing in Distributed Environments Based on Continuous Integration

2021 ◽  
Vol 27 (12) ◽  
pp. 619-625
Author(s):  
I. V. Bychkov ◽  
◽  
S. A. Gorsky ◽  
A. G. Feoktistov ◽  
R. O. Kostromin ◽  
...  
Keyword(s):  
Substantial Reduction ◽  
Practical Significance ◽  
Scientific Applications ◽  
Application Development ◽  
Modular Architecture ◽  
Continuous Integration ◽  
Software Packages ◽  
Applied Software ◽  
Use Efficiency ◽  
Computational Resources

Nowadays, tools for designing scientific applications often do not implement the required continuous integration capabilities of the applied software. Therefore, such overheads as the application development time and experiment execution makespan are substantially increased. In this regard, we propose a new approach to developing scientific applications and carrying out experiments with them. It is based on applying continuous integration to both the applied and system software in developing distributed applied software packages with a modular architecture using the Orlando Tools framework. Within the proposed approach, we provide integrating the Orlando Tools subsystems with the GitLab system and automating the development of package modules. At the same time, Orlando Tools fully support constructing and testing problem-solving schemes (workflows) that combine package modules located on environment resources with different computational characteristics. To this end, Orlando Tools provides the necessary configuring and setting up of computational resources. The practical significance of our study is substantial reduction overheads needed to experiment fulfillments and increase of the resource use efficiency.

scholarly journals Framework for preparing subject data in testing modules of scientific applications

2019 ◽  
Author(s):  
E.S. Fereferov ◽  
A.G. Feoktistov ◽  
I.V. Bychkov
Keyword(s):  
Information Resources ◽  
Structured Data ◽  
Data Sources ◽  
Subject Domain ◽  
Data Preparation ◽  
Scientific Applications ◽  
Software Packages ◽  
Applied Software ◽  
Source Data ◽  
Other Information

The paper addresses the relevant problem of data preparation for testing modules of scientific applications. Such testing requires the multiple executions of modules with different parameters for various scenarios of solving problems in applications. Often, data sources for parameters used for problem-solving are subject data (experimental results, reports, statistical forms and other information resources) created earlier as a result of functioning various objects of a subject domain. Usually, such data are heterogeneous and weakly structured. The developer of scientific applications has to make additional efforts in extracting, cleaning, integrating, and formatting data in order to achieve the correctness and efficiency of their use in applications. The aim of the study is the development of a framework for automating the description of semi-structured data and their transformation into target structures used by scientific applications. We proposed a conceptual model that allows us to represent knowledge about the structure of the source data, determine their relations with the target structures and set the rules for data transformation. Additionally, we developed a framework prototype. It is integrated into the technological scheme of continuous integration for modules of scientific applications (distributed applied software packages) that are developed with the help of Orlando Tools. The effectiveness of this prototype functioning is confirmed by the results of experimental analysis.

scholarly journals Continuous integration, delivery, and deployment for scientific workflows in Orlando Tools

2020 ◽  
Author(s):  
S.A. Gorsky
Keyword(s):  
Heterogeneous Computing ◽  
Scientific Workflow ◽  
Scientific Workflows ◽  
Continuous Integration ◽  
Software Packages ◽  
Applied Software ◽  
Computing Environments ◽  
Heterogeneous Nodes ◽  
Deep Integration ◽  
Special Case

The paper addresses issues of continuous integration in the development of scientific applications based on workflows (special case of distributed applied software packages) for heterogeneous computing environments. The preparation and carrying out of scientific computational experiments are often accompanied by intensive software modification. Thus, there is a need for the following stages: building, testing, debugging, and installation new versions of software in heterogeneous nodes of environment. These stages may take longer time overheads than computations themselves. The solution to this challenge lies in the use of tools for continuous integration of software. However, such tools require deep integration with the tools for the workflow development because of scientific workflow specifics. To this end, the paper describes the combination of the authors Orlando Tools framework for the development and use packages with the GitLab system that is widely used for continuous integration. Such combination significantly reduces the complexity of software continuous when developing and using packages.

scholarly journals Integração, Relacionamento e Representação de Dados em Cidades Inteligentes: Uma Revisão de Literatura

2018 ◽  
Author(s):  
Larysse Silva ◽  
José Alex Lima ◽  
Nélio Cacho ◽  
Eiji Adachi ◽  
Frederico Lopes ◽  
...  
Keyword(s):  
Decision Making ◽  
Literature Review ◽  
Data Integration ◽  
Smart Cities ◽  
Heterogeneous Data ◽  
Data Sources ◽  
Application Development ◽  
Continuous Integration ◽  
Heterogeneous Data Sources ◽  
Computational Systems

A notable characteristic of smart cities is the increase in the amount of available data generated by several devices and computational systems, thus augmenting the challenges related to the development of software that involves the integration of larges volumes of data. In this context, this paper presents a literature review aimed to identify the main strategies used in the development of solutions for data integration, relationship, and representation in smart cities. This study systematically selected and analyzed eleven studies published from 2015 to 2017. The achieved results reveal gaps regarding solutions for the continuous integration of heterogeneous data sources towards supporting application development and decision-making.

AUTOMATED CONTINUOUS INTEGRATION USING CIRCLECI AND FIREBASE FOR ANDROID APPLICATION DEVELOPMENT

2021 ◽  
Vol 47 (05) ◽  
Author(s):  
NGUYỄN THỊ PHƯƠNG GIANG ◽  
TRẦN THỊ MINH KHOA
Keyword(s):  
Software Development ◽  
Industry 4.0 ◽  
Software Evolution ◽  
Android Application ◽  
Application Development ◽  
Software Developers ◽  
Continuous Integration ◽  
Complete Process ◽  
Automation Testing

Continuous Integration (CI) is the most common practice among software developers where they integrate their work into a frequent baseline. The industry 4.0 is facing huge challenges while developing Software at multiple sites and tested on multiple platforms. Today, so many CI tools widely used for software development as CircleCI, Jenkins, Travis. CircleCI is one of the CI tools that can helps in automating the complete process, reducing the works of a developer and check the development at each and every step of Software evolution. In this paper, we discuss the implementation of CircleCI for android application development. Firebase Test Lab will be used for some additional automation testing.

scholarly journals Service for parallel applications based on JINR cloud and HybriLIT resources

2019 ◽  
Vol 214 ◽  
pp. 07012 ◽  
Author(s):  
Nikita Balashov ◽  
Maxim Bashashin ◽  
Pavel Goncharov ◽  
Ruslan Kuchumov ◽  
Nikolay Kutovskiy ◽  
...  
Keyword(s):  
High Performance ◽  
Cloud Service ◽  
Parallel Applications ◽  
Cloud Infrastructure ◽  
Modular Architecture ◽  
Practical Applications ◽  
Speed Up ◽  
Scientific Results ◽  
Computational Resources ◽  
Performance Computing

Cloud computing has become a routine tool for scientists in many fields. The JINR cloud infrastructure provides JINR users with computational resources to perform various scientific calculations. In order to speed up achievements of scientific results the JINR cloud service for parallel applications has been developed. It consists of several components and implements a flexible and modular architecture which allows to utilize both more applications and various types of resources as computational backends. An example of using the Cloud&HybriLIT resources in scientific computing is the study of superconducting processes in the stacked long Josephson junctions (LJJ). The LJJ systems have undergone intensive research because of the perspective of practical applications in nano-electronics and quantum computing. In this contribution we generalize the experience in application of the Cloud&HybriLIT resources for high performance computing of physical characteristics in the LJJ system.

scholarly journals An Architecture for Developing Cyber Environments using Multiple HPC Infrastructures for e-Science Applications

2015 ◽  
Author(s):  
Felipe Maciel ◽  
Carina Oliveira ◽  
Renato Juaçaba Neto ◽  
João Alencar ◽  
Paulo Rego ◽  
...  
Keyword(s):  
High Performance Computing ◽  
High Performance ◽  
Scientific Applications ◽  
Computational Resources ◽  
Performance Computing

In this paper, we propose a novel architecture to allow the implementation of a cyber environment composed of different High Performance Computing (HPC) infrastructures (i.e., clusters, grids and clouds). To access this cyber environment, scientific researchers do not have to become computer experts. In particular, we assume that scientific researchers provide a description of the problem as an input to the cyber environment and then get their results without being responsible for managing the computational resources. We provide a prototype of the architecture and introduce an evaluation which studies a real workload of scientific applications executions. The results show the advantages of the proposed architecture. Besides, we highlight this work provides guidelines for developing cyber environments focused on e-Science.

scholarly journals Continuous integrating modules of distributed applied software packages in Orlando Tools

2019 ◽  
Vol 31 (2) ◽  
pp. 83-96
Author(s):  
A.G. Feoktistov ◽  
S.A. Gorsky ◽  
I.A. Sidorov ◽  
R.O. Kostromin ◽  
E.S. Fereferov ◽  
...  
Keyword(s):  
Software Packages ◽  
Applied Software

scholarly journals Application of data hashing tools to accelerate classification decisions in structural image recognition methods

2021 ◽  
Vol 5 (2) ◽  
pp. 13-20
Author(s):  
Volodymyr Gorokhovatsky ◽  
Nataliia Vlasenko ◽  
Mykhailo Rybalka
Keyword(s):  
Image Classification ◽  
High Speed ◽  
Data Representation ◽  
Practical Significance ◽  
Classification Methods ◽  
Software Modeling ◽  
Space Efficiency ◽  
Software Models ◽  
Applied Software ◽  
Structural Image

The subject of this research is the image classification methods based on a set of key points descriptors. The goal is to increase the performance of classification methods, in particular, to improve the time characteristics of classification by introducing hashing tools for reference data representation. Methods used: ORB detector and descriptors, data hashing tools, search methods in data arrays, metrics-based apparatus for determining the relevance of vectors, software modeling. The obtained results: developed an effective method of image classification based on the introduction of high-speed search using hash structures, which speeds up the calculation dozens of times; the classification time for the considered experimental descriptions increases linearly with decreasing number of hashes; the minimum metric value limit choice on setting the class for object descriptors significantly affects the accuracy of classification; the choice of such limit can be optimized for fixed samples databases; the experimentally achieved accuracy of classification indicates the efficiency of the proposed method based on data hashing. The practical significance of the work is - the classification model’s synthesis in the hash data representations space, efficiency proof of the proposed classifiers modifications on image examples, development of applied software models implementing the proposed classification methods in computer vision systems.

Sign in / Sign up

Export Citation Format

Share Document