Distributed data processing requirements engineering: high level DDP design

Author(s):  
D.F. Palmer ◽  
W.M. Denny
2020 ◽  
Vol 38 (3-4) ◽  
pp. 1-31
Author(s):  
Won Wook Song ◽  
Youngseok Yang ◽  
Jeongyoon Eo ◽  
Jangho Seo ◽  
Joo Yeon Kim ◽  
...  

Optimizing scheduling and communication of distributed data processing for resource and data characteristics is crucial for achieving high performance. Existing approaches to such optimizations largely fall into two categories. First, distributed runtimes provide low-level policy interfaces to apply the optimizations, but do not ensure the maintenance of correct application semantics and thus often require significant effort to use. Second, policy interfaces that extend a high-level application programming model ensure correctness, but do not provide sufficient fine control. We describe Apache Nemo, an optimization framework for distributed dataflow processing that provides fine control for high performance and also ensures correctness for ease of use. We combine several techniques to achieve this, including an intermediate representation of dataflow, compiler optimization passes, and runtime extensions. Our evaluation results show that Nemo enables composable and reusable optimizations that bring performance improvements on par with existing specialized runtimes tailored for a specific deployment scenario. Apache Nemo is open-sourced at https://nemo.apache.org as an Apache incubator project.


BMC Genomics ◽  
2020 ◽  
Vol 21 (1) ◽  
Author(s):  
Onur Yukselen ◽  
Osman Turkyilmaz ◽  
Ahmet Rasit Ozturk ◽  
Manuel Garber ◽  
Alper Kucukural

1979 ◽  
Vol 21 (2) ◽  
Author(s):  
L. J. Heinrich

Der Beitrag erläutert das subjektive Verständnis des Begriffes ,,Computerleistung am Arbeitsplatz" als Schlagwort für eine progressive Gestaltungsphilosophie computergestützter Informationssysteme. Sie impliziert sowohl die Anwendung moderner Hard- und Softwaretechnologien, wie sie für die 80er Jahre bestimmend sein werden, als auch die in den Vordergrund rückende Berücksichtigung der sowohl von der Arbeitsaufgabe bestimmten als auch der subjektiven Benutzerbedürfnisse. Sie verbindet damit ,, Distributed Data Processing" als ein technologisches Konzept mit ..Benutzerorientierung". Die Gestaltungsbereiche der Benutzerorientierung - Arbeitsmittel und Arbeitsumwelt, Mensch- Computer-Interaktionsschnittstelle sowie die Arbeitsorganisation - werden erläutert. Gestaltungsmaßnahmen werden beispielhaft angegeben, und es wird auf die weiterführende Literatur verwiesen; dabei steht das im Oldenbourg- Verlag erschienene Buch ,,Computerleistung am Arbeitsplatz - benutzerorientiertes Distributed Data Processing" im Vordergrund.


Author(s):  
V.G. Belenkov ◽  
V.I. Korolev ◽  
V.I. Budzko ◽  
D.A. Melnikov

The article discusses the features of the use of the cryptographic information protection means (CIPM)in the environment of distributed processing and storage of data of large information and telecommunication systems (LITS).A brief characteristic is given of the properties of the cryptographic protection control subsystem - the key system (CS). A description is given of symmetric and asymmetric cryptographic systems, required to describe the problem of using KS in LITS.Functional and structural models of the use of KS and CIPM in LITS, are described. Generalized information about the features of using KS in LITS is given. The obtained results form the basis for further work on the development of the architecture and principles of KS construction in LITS that implement distributed data processing and storage technologies. They can be used both as a methodological guide, and when carrying out specific work on the creation and development of systems that implement these technologies, as well as when forming technical specifications for the implementation of work on the creation of such systems.


Sign in / Sign up

Export Citation Format

Share Document