AN ARCHITECTURE FOR DISTRIBUTED DATA ANALYSIS PROBLEM SOLVING IN NEUROPHYSIOLOGY

CMS distributed data analysis with CRAB3

Journal of Physics Conference Series ◽

10.1088/1742-6596/664/6/062038 ◽

2015 ◽

Vol 664 (6) ◽

pp. 062038 ◽

Cited By ~ 5

Author(s):

M Mascheroni ◽

J Balcas ◽

S Belforte ◽

B P Bockelman ◽

J M Hernandez ◽

...

Keyword(s):

Data Analysis ◽

Distributed Data ◽

Distributed Data Analysis

Get full-text (via PubEx)

Luiza: Analysis Framework for GLORIA

Acta Polytechnica ◽

10.14311/1718 ◽

2013 ◽

Vol 53 (1) ◽

Author(s):

Aleksander Filip Żarnecki ◽

Lech Wiktor Piotrowski ◽

Lech Mankiewicz ◽

Sebastian Małek

Keyword(s):

Data Analysis ◽

High Energy Physics ◽

Linear Collider ◽

International Linear Collider ◽

High Energy ◽

Distributed Data ◽

Analysis Framework ◽

Full Analysis ◽

Distributed Data Analysis ◽

Energy Physics

The Luiza analysis framework for GLORIA is based on the Marlin package, which was originally developed for data analysis in the new High Energy Physics (HEP) project, International Linear Collider (ILC). The HEP experiments have to deal with enormous amounts of data and distributed data analysis is therefore essential. The Marlin framework concept seems to be well suited for the needs of GLORIA. The idea (and large parts of the code) taken from Marlin is that every computing task is implemented as a processor (module) that analyzes the data stored in an internal data structure, and the additional output is also added to that collection. The advantage of this modular approach is that it keeps things as simple as possible. Each step of the full analysis chain, e.g. from raw images to light curves, can be processed step-by-step, and the output of each step is still self consistent and can be fed in to the next step without any manipulation.

Get full-text (via PubEx)

Cloud for Distributed Data Analysis Based on the Actor Model

Scientific Programming ◽

10.1155/2016/1050293 ◽

2016 ◽

Vol 2016 ◽

pp. 1-11 ◽

Cited By ~ 2

Author(s):

Ivan Kholod ◽

Ilya Petukhov ◽

Andrey Shorov

Keyword(s):

Data Mining ◽

Data Analysis ◽

Data Sets ◽

Distributed Data ◽

Actor Model ◽

Confidential Information ◽

Data Mining Algorithms ◽

Functional Blocks ◽

Mining Algorithms ◽

Distributed Data Analysis

This paper describes the construction of a Cloud for Distributed Data Analysis (CDDA) based on the actor model. The design uses an approach to map the data mining algorithms on decomposed functional blocks, which are assigned to actors. Using actors allows users to move the computation closely towards the stored data. The process does not require loading data sets into the cloud and allows users to analyze confidential information locally. The results of experiments show that the efficiency of the proposed approach outperforms established solutions.

Get full-text (via PubEx)