Handling Iterations in Distributed Dataflow Systems

2022 ◽  
Vol 54 (9) ◽  
pp. 1-38
Author(s):  
Gábor E. Gévay ◽  
Juan Soto ◽  
Volker Markl

Over the past decade, distributed dataflow systems (DDS) have become a standard technology. In these systems, users write programs in restricted dataflow programming models, such as MapReduce, which enable them to scale out program execution to a shared-nothing cluster of machines. Yet, there is no established consensus that prescribes how to extend these programming models to support iterative algorithms. In this survey, we review the research literature and identify how DDS handle control flow, such as iteration, from both the programming model and execution level perspectives. This survey will be of interest for both users and designers of DDS.

2021 ◽  
Vol 43 (1) ◽  
pp. 1-73
Author(s):  
David J. Pearce

Rust is a relatively new programming language that has gained significant traction since its v1.0 release in 2015. Rust aims to be a systems language that competes with C/C++. A claimed advantage of Rust is a strong focus on memory safety without garbage collection. This is primarily achieved through two concepts, namely, reference lifetimes and borrowing . Both of these are well-known ideas stemming from the literature on region-based memory management and linearity / uniqueness . Rust brings both of these ideas together to form a coherent programming model. Furthermore, Rust has a strong focus on stack-allocated data and, like C/C++ but unlike Java, permits references to local variables. Type checking in Rust can be viewed as a two-phase process: First, a traditional type checker operates in a flow-insensitive fashion; second, a borrow checker enforces an ownership invariant using a flow-sensitive analysis. In this article, we present a lightweight formalism that captures these two phases using a flow-sensitive type system that enforces “ type and borrow safety .” In particular, programs that are type and borrow safe will not attempt to dereference dangling pointers. Our calculus core captures many aspects of Rust, including copy- and move-semantics, mutable borrowing, reborrowing, partial moves, and lifetimes. In particular, it remains sufficiently lightweight to be easily digested and understood and, we argue, still captures the salient aspects of reference lifetimes and borrowing. Furthermore, extensions to the core can easily add more complex features (e.g., control-flow, tuples, method invocation). We provide a soundness proof to verify our key claims of the calculus. We also provide a reference implementation in Java with which we have model checked our calculus using over 500B input programs. We have also fuzz tested the Rust compiler using our calculus against 2B programs and, to date, found one confirmed compiler bug and several other possible issues.


Human Affairs ◽  
2013 ◽  
Vol 23 (2) ◽  
Author(s):  
Filippo Zerilli

AbstractIn the past two decades academic and research literature on “corruption” has flourished. During the same period organizations and initiatives fighting against corruption have also significantly expanded, turning “anti-corruption” into a new research subject. However, despite a few exceptions there is a division of labor between scholars who study corruption itself and those who study the global anti-corruption industry. Juxtaposing corruption’s local discourses and anti-corruption international practices, this article is an attempt to bring together these two intertwined research dimensions and explore how an ethnographic approach might contribute to framing them together. Firstly, it describes how corruption in Romania is often conceptualized and explained in terms of national heritage, something related to old and recent cultural history, including traditional folklore. Secondly, it explores how anti-corruption works in practice, focusing on international legal cooperation projects monitoring the progress and shortcomings both prior to and post Romania’s accession to the European Union. Finally, revealing the articulations of these two apparently unrelated research fields, the article argues that corruption’s local explanations and the circular logic of auditing observed within the anti-corruption industry share a common developmental ideology mirroring the crypto-colonialist structure of power relations and dependency among European nation-states emerging out of the Cold War.


2018 ◽  
Vol 2018 ◽  
pp. 1-7 ◽  
Author(s):  
Syaripuddin ◽  
Herry Suprajitno ◽  
Fatmawati

Quadratic programming with interval variables is developed from quadratic programming with interval coefficients to obtain optimum solution in interval form, both the optimum point and optimum value. In this paper, a two-level programming approach is used to solve quadratic programming with interval variables. Procedure of two-level programming is transforming the quadratic programming model with interval variables into a pair of classical quadratic programming models, namely, the best optimum and worst optimum problems. The procedure to solve the best and worst optimum problems is also constructed to obtain optimum solution in interval form.


2019 ◽  
Vol 75 (2) ◽  
pp. 230-246 ◽  
Author(s):  
Sonia Yaco ◽  
Arkalgud Ramaprasad

PurposeThe purpose of this paper is to suggest a framework that creates a common language to enhance the connection between the domains of cultural heritage (CH) artifacts and instruction.Design/methodology/approachThe CH and instruction domains are logically deconstructed into dimensions of functions, semiotics, CH, teaching/instructional materials, agents and outcomes. The elements within those dimensions can be concatenated to create natural-English sentences that describe aspects of the problem domain.FindingsThe framework is valid using traditional social sciences content, semantic, practical and systemic validity constructs.Research limitations/implicationsThe framework can be used to map current research literature to discover areas of heavy, light and no research.Originality/valueThe framework provides a new way for CH and education stakeholders to describe and visualize the problem domain, which could allow for significant enhancements of each. Better understanding the problem domain would serve to enhance instruction informed from collections and vice versa. The educational process would have more depth due to better access to primary sources. Increased use of collections would reveal more ways through which they could be used in instruction. The framework can help visualize the past and present of the domain, and envisage its future.


2021 ◽  
Vol 15 (4) ◽  
pp. 518-523
Author(s):  
Ratko Stanković ◽  
Diana Božić

Improvements achieved by applying linear programming models in solving optimization problems in logistics cannot always be expressed by physically measurable values (dimensions), but in non-dimensional values. Therefore, it may be difficult to present the actual benefits of the improvements to the stake holders of the system being optimized. In this article, a possibility of applying simulation modelling in quantifying results of optimizing cross dock terminal gates allocation is outlined. Optimal solution is obtained on the linear programming model by using MS Excel spreadsheet optimizer, while the results are quantified on the simulation model, by using Rockwell Automation simulation software. Input data are collected from a freight forwarding company in Zagreb, specialized in groupage transport (Less Than Truckload - LTL).


2020 ◽  
Vol 179 ◽  
pp. 02016
Author(s):  
Yue Zhu ◽  
Cheng-lein TENG

A design method of one cultural product based on the meme of artifacts was constructed and applied to the development and design of cultural products. Firstly, the paper puts forward that the outer “tangible” level of the “spatial perspective” of culture corresponds to the dominant meme, as well as the mid “behavioral” level and Inner “intangible” level correspond to the hidden meme according to the theories of meme and the “spatial perspective” of culture. What’s more, the meme was extracted according to the appearance of Liao dynasty ceramics and related research literature. Secondly, based on the extracted meme of Liao dynasty ceramics, the design method of cultural products was constructed by applying the theoretical framework two orders of the signification of Roland Barthes Semiotics to explore the extended cultural meaning of meme and apply its translation to the design of cultural products. The design method of cultural products constructed by this Institute is more inclusive of the intrinsic level of artifacts than that in the past, which only created cultural products that focus on the physical representation of the visual elements. It is an attempt to provide designers of cultural products and students with design background with a design method of cultural products for reference.


1981 ◽  
Vol 10 (137) ◽  
Author(s):  
Neil D. Jones ◽  
Henning Christiansen

<p>A simple algebra-based algorithm for compiler generation is described. Its input is a semantic definition of a programming language, and its output is a ''compiling semantics'' which maps each source program into a sequence of compile-time actions whose net effect on execution is the production of a semantically equivalent target program. The method does not require individual compiler correctness proofs or the construction of specialized target algebras.</p><p>Source program execution is assumed to proceed by performing a series of elementary actions on a runtime state. A semantic algebra is introduced to represent and manipulate possible execution sequences. A source semantic definition has two parts: A set of semantic equations mapping source programs into terms of the algebra, and an interpretation which gives concrete definitions of the state and the elementary actions on it.</p>


Author(s):  
Veljko Milutinovic ◽  
Nemanja Trifunovic ◽  
Nenad Korolija ◽  
Jovan Popovic ◽  
Dragan Bojic

Author(s):  
Javier Conejero ◽  
Sandra Corella ◽  
Rosa M Badia ◽  
Jesus Labarta

Task-based programming has proven to be a suitable model for high-performance computing (HPC) applications. Different implementations have been good demonstrators of this fact and have promoted the acceptance of task-based programming in the OpenMP standard. Furthermore, in recent years, Apache Spark has gained wide popularity in business and research environments as a programming model for addressing emerging big data problems. COMP Superscalar (COMPSs) is a task-based environment that tackles distributed computing (including Clouds) and is a good alternative for a task-based programming model for big data applications. This article describes why we consider that task-based programming models are a good approach for big data applications. The article includes a comparison of Spark and COMPSs in terms of architecture, programming model, and performance. It focuses on the differences that both frameworks have in structural terms, on their programmability interface, and in terms of their efficiency by means of three widely known benchmarking kernels: Wordcount, Kmeans, and Terasort. These kernels enable the evaluation of the more important functionalities of both programming models and analyze different work flows and conditions. The main results achieved from this comparison are (1) COMPSs is able to extract the inherent parallelism from the user code with minimal coding effort as opposed to Spark, which requires the existing algorithms to be adapted and rewritten by explicitly using their predefined functions, (2) it is an improvement in terms of performance when compared with Spark, and (3) COMPSs has shown to scale better than Spark in most cases. Finally, we discuss the advantages and disadvantages of both frameworks, highlighting the differences that make them unique, thereby helping to choose the right framework for each particular objective.


Sign in / Sign up

Export Citation Format

Share Document