Bio-Inspired Techniques for Resources State Prediction in Large Scale Distributed Systems

2012 ◽  
pp. 18-36
Author(s):  
Andreea Visan ◽  
Mihai Istin ◽  
Florin Pop ◽  
Valentin Cristea
Keyword(s):  
Distributed Systems ◽  
Prediction Error ◽  
Large Scale ◽  
Fitness Function ◽  
Advance Reservation ◽  
Alice Experiment ◽  
State Prediction ◽  
And Performance ◽  
Prediction Techniques ◽  
History Database

The state prediction of resources in large scale distributed systems represents an important aspect for resources allocations, systems evaluation, and autonomic control. The paper presents advanced techniques for resources state prediction in Large Scale Distributed Systems, which include techniques based on bio-inspired algorithms like neural network improved with genetic algorithms. The approach adopted in this paper consists of a new fitness function, having prediction error minimization as the main scope. The proposed prediction techniques are based on monitoring data, aggregated in a history database. The experimental scenarios consider the ALICE experiment, active at the CERN institute. Compared with classical predicted algorithms based on average or random methods, the authors obtain an improved prediction error of 73%. This improvement is important for functionalities and performance of resource management systems in large scale distributed systems in the case of remote control ore advance reservation and allocation.

Bio-Inspired Techniques for Resources State Prediction in Large Scale Distributed Systems

2011 ◽  
Vol 2 (3) ◽  
pp. 1-18 ◽  
Author(s):  
Andreea Visan ◽  
Mihai Istin ◽  
Florin Pop ◽  
Valentin Cristea
Keyword(s):  
Distributed Systems ◽  
Prediction Error ◽  
Large Scale ◽  
Fitness Function ◽  
Advance Reservation ◽  
Alice Experiment ◽  
State Prediction ◽  
And Performance ◽  
Prediction Techniques ◽  
History Database

The state prediction of resources in large scale distributed systems represents an important aspect for resources allocations, systems evaluation, and autonomic control. The paper presents advanced techniques for resources state prediction in Large Scale Distributed Systems, which include techniques based on bio-inspired algorithms like neural network improved with genetic algorithms. The approach adopted in this paper consists of a new fitness function, having prediction error minimization as the main scope. The proposed prediction techniques are based on monitoring data, aggregated in a history database. The experimental scenarios consider the ALICE experiment, active at the CERN institute. Compared with classical predicted algorithms based on average or random methods, the authors obtain an improved prediction error of 73%. This improvement is important for functionalities and performance of resource management systems in large scale distributed systems in the case of remote control ore advance reservation and allocation.

UNIVERSAL ROUTING AND PERFORMANCE ASSURANCE FOR DISTRIBUTED NETWORKS

2007 ◽  
Vol 08 (01) ◽  
pp. 1-28
Author(s):  
KEVIN F. CHEN ◽  
EDWIN H.-M. SHA
Keyword(s):  
Distributed Systems ◽  
Large Scale ◽  
Distributed Networks ◽  
Distributed Network ◽  
Graph Layout ◽  
Universal Network ◽  
Wire Delay ◽  
And Performance ◽  
The Way

We show that universal routing can be achieved with low overhead in distributed networks. The validity of our results rests on a new network called the fat-stack. We show that from a routing perspective the fat-stack is efficient and is suitable for use as a baseline distributed network and as a crucial benchmark architecture for evaluating the performance of specific distributed networks. We show that the fat-stack is efficient by proving it is universal. A requirement for the fat-stack to be universal is that link capacities double up the levels of the network. We use methods developed in the areas of VLSI and processor interconnect for much of our analysis. We then show how to scale the fat-stack from a VLSI graph layout to a large-scale distributed topology and how the network can be an effective benchmark architecture. Our universality proofs show that a fat-stack of area Θ(A) can simulate any competing network of area A with [Formula: see text] overhead independently of wire delay. The universality result implies that the fat-stack of a given size is nearly the best routing network of that size. The fat-stack is also the minimal universal network for an [Formula: see text] overhead in terms of number of links. Actual simulations show that the fat-stack outperforms a mesh-based distributed network of comparable hardware usage. Our work helps explain why some deployed networks function in the way they do in terms of routing. It also provides an exemplary network of proven efficiency and scalability for building new distributed systems.

Verification of Super-Peer Model for Query Processing in Peer-to-Peer Networks

2016 ◽  
pp. 306-332
Author(s):  
J. Pourqasem ◽  
S.A. Edalatpanah
Keyword(s):  
Distributed Systems ◽  
Query Processing ◽  
Large Scale ◽  
Peer To Peer ◽  
P2p Networks ◽  
Processing Methods ◽  
Super Node ◽  
And Performance ◽  
Processing Techniques ◽  
Super Peer

Equal peers in peer-to-peer (P2P) networks are the drawbacks of system in term of bandwidth, scalability and efficiency. The super-peer model is based on heterogeneity and different characteristics of peers in P2P networks. The P2P networks and large- scale distributed systems based on P2P networks use the super-peer model to design the query processing mechanism. This chapter first reviews the query processing methods in P2P networks, in which the authors classify theses query processing approaches in Unstructured and Structured mechanisms. Furthermore, the query processing techniques in distributed systems based on P2P networks are discussed. Afterward, authors concentrate on super-peer model to process the query of peers in P2P networks. Authors present the query processing methods in P2P-based distributed systems using the super node. Finally, the chapter provides some examples of each of the presented query processing techniques, and then illustrates the properties of each of them in terms of scalability and performance issues.

Scheduling

2010 ◽  
pp. 91-110
Author(s):  
Valentin Cristea ◽  
Ciprian Dobre ◽  
Corina Stratan ◽  
Florin Pop
Keyword(s):  
Distributed Systems ◽  
Large Scale ◽  
Dynamic Scheduling ◽  
Scheduling Algorithms ◽  
Scheduling Problem ◽  
Scheduling Problems ◽  
Advance Reservation ◽  
Systems Architecture ◽  
Static Versus Dynamic ◽  
Scheduling Models

This chapter presents the scheduling problem in large scale distributed systems. Most parts of the chapter are devoted to discussion of scheduling algorithms and models. The main challenges of scheduling problem are approached here. The implementation issues are also covered. The chapter has three parts. The first part covers basics like scheduling models, scheduling algorithms for independent tasks and DAG scheduling Algorithms for tasks with dependencies. The first part of the chapter presents a classification of scheduling problems, methods that are relevant for the solution procedures, and computational complexity. The scheduling models are presented based on systems architecture described in Resource Management chapter. This firs part also provides a critical analysis of most important algorithms from different points of view, such as static versus dynamic policies, objective functions, applications models, adaptation, QoS constraints and strategies dealing with dynamic behavior of resources. The second part covers new scheduling mechanism like resources co-allocation and advance reservation. Multi-criteria optimization mechanisms for users and systems constrain (e.g. load-balancing, minimization of execution time) are described and analyzed in this chapter. This part uses algorithm and methods to highlight the importance of these topics. The dynamic scheduling is also the subject of this part. It is also presented the implementation issues for scheduler tools. Since it is not possible to cover the whole area of scheduling in one chapter, some restrictions are imposed. Firstly, the chapter presents only Scheduling for Large Scale Distributed Systems (LSDS), without single system scheduling. Secondly, some interesting topics of fault tolerance (re-scheduling) are not analyzed in this chapter.

Sign in / Sign up

Export Citation Format

Share Document