scholarly journals Optimization Techniques for a Distributed In-Memory Computing Platform by Leveraging SSD

2021 ◽  
Vol 11 (18) ◽  
pp. 8476
Author(s):  
June Choi ◽  
Jaehyun Lee ◽  
Jik-Soo Kim ◽  
Jaehwan Lee
Keyword(s):  
Memory Management ◽  
Computing System ◽  
Optimization Techniques ◽  
Main Memory ◽  
Apache Spark ◽  
Computing Platform ◽  
Intermediate Data ◽  
Management Capability ◽  
Overall Performance ◽  
Optimization Methodology

In this paper, we present several optimization strategies that can improve the overall performance of the distributed in-memory computing system, “Apache Spark”. Despite its distributed memory management capability for iterative jobs and intermediate data, Spark has a significant performance degradation problem when the available amount of main memory (DRAM, typically used for data caching) is limited. To address this problem, we leverage an SSD (solid-state drive) to supplement the lack of main memory bandwidth. Specifically, we present an effective optimization methodology for Apache Spark by collectively investigating the effects of changing the capacity fraction ratios of the shuffle and storage spaces in the “Spark JVM Heap Configuration” and applying different “RDD Caching Policies” (e.g., SSD-backed memory caching). Our extensive experimental results show that by utilizing the proposed optimization techniques, we can improve the overall performance by up to 42%.

scholarly journals Optimal Page Allocation of Hybrid Main Memory using Page Caching Algorithm

2019 ◽  
Vol 8 (3) ◽  
pp. 522-526
Keyword(s):  
Memory Management ◽  
Reducing Power ◽  
Main Memory ◽  
Optimal Performance ◽  
Evaluation Framework ◽  
Total Power ◽  
Scalable Algorithm ◽  
Hybrid Memory ◽  
Overall Performance ◽  
Allocation Strategies

In memory management system, DRAM has long write endurance and PRAM has shorter write latency. To get the advantages of both memories, a hybrid memory is constructed using DRAM and PRAM. In this article, the task allocation strategies on hybrid memory were carried out to achieve multiple targets such as, extending the lifetime, reducing power consumption and reducing the memory size. Different types of algorithm such as, genetic algorithm, scalable algorithm and distributed algorithms are proposed by many researchers. Here, page caching algorithm is proposed to achieve an optimal performance. The Page caching algorithm includes the following steps: 1.Getting instructions from CPU and finding address in DRAM, 2. Extracting page in PRAM, 3.Reallocating the page in DRAM and 4. Updating in PRAM. The work is designed and implemented using an evaluation framework and the hybrid memories optimal performance is calculated. The user provides workload, PRAM and DRAM parameters and environmental characteristics of hybrid memory as inputs to the system. It also determines the overall performance by considering several limitations. This architecture first optimizes the page allocation and then the task write in the memory was allocated. Comparing to the existing system, the page caching algorithm reduces upto 32.8% of total power.

scholarly journals Hybrid Optimization Based Mathematical Procedure for Dimensional Synthesis of Slider-Crank Linkage

Mathematics ◽  
2021 ◽  
Vol 9 (13) ◽  
pp. 1581
Author(s):  
Alfonso Hernández ◽  
Aitor Muñoyerro ◽  
Mónica Urízar ◽  
Enrique Amezua
Keyword(s):  
Genetic Algorithm ◽  
Fitness Function ◽  
Linear Equations ◽  
Optimization Procedure ◽  
Optimization Techniques ◽  
Dimensional Synthesis ◽  
Hybrid Strategy ◽  
Dimensional Parameters ◽  
Optimization Methodology ◽  
Crank Mechanism

In this paper, an optimization procedure for path generation synthesis of the slider-crank mechanism will be presented. The proposed approach is based on a hybrid strategy, mixing local and global optimization techniques. Regarding the local optimization scheme, based on the null gradient condition, a novel methodology to solve the resulting non-linear equations is developed. The solving procedure consists of decoupling two subsystems of equations which can be solved separately and following an iterative process. In relation to the global technique, a multi-start method based on a genetic algorithm is implemented. The fitness function incorporated in the genetic algorithm will take as arguments the set of dimensional parameters of the slider-crank mechanism. Several illustrative examples will prove the validity of the proposed optimization methodology, in some cases achieving an even better result compared to mechanisms with a higher number of dimensional parameters, such as the four-bar mechanism or the Watt’s mechanism.

scholarly journals Vision based hardware-software real-time control system for autonomous landing of an UAV

2020 ◽  
Author(s):  
Krzysztof Blachut ◽  
Hubert Szolc ◽  
Mateusz Wasala ◽  
Tomasz Kryjak ◽  
Marek Gorgon
Keyword(s):  
Control System ◽  
Real Time ◽  
Heterogeneous Computing ◽  
Computing System ◽  
Video Stream ◽  
Real Time Control ◽  
Time Data ◽  
Autonomous Landing ◽  
Computing Platform ◽  
Time Control

In this paper we present a vision based hardware-software control system enabling autonomous landing of a mul-tirotor unmanned aerial vehicle (UAV). It allows the detection of a marked landing pad in real-time for a 1280 x 720 @ 60 fps video stream. In addition, a LiDAR sensor is used to measure the altitude above ground. A heterogeneous Zynq SoC device is used as the computing platform. The solution was tested on a number of sequences and the landing pad was detected with 96% accuracy. This research shows that a reprogrammable heterogeneous computing system is a good solution for UAVs because it enables real-time data stream processing with relatively low energy consumption.

A Dynamic Data Fragmentation and Distribution Strategy for Main-Memory Database Cluster

2012 ◽  
Vol 490-495 ◽  
pp. 1231-1236 ◽  
Author(s):  
Tran Van Hung ◽  
Chuan He Huang
Keyword(s):  
Response Time ◽  
Cluster Computing ◽  
Data Access ◽  
Fast Response ◽  
Main Memory ◽  
Relation Database ◽  
Computing Platform ◽  
Wide Range ◽  
Cluster Environment ◽  
Remote Sites

MMDB cluster system is a memory optimized relation database that implements on cluster computing platform, provides applications with extremely fast response time and very high throughput as required by many applications in a wide range of industries. Here, a new dynamic fragment allocation algorithm (DFAPR) in Partially Replicated allocation scenario is proposed. This algorithm reallocates data with respect to changing data access pattern for each fragment in which data is maintained in current site, migrated or created new replicas on remote sites depend on accessing frequency and average response time. At last, the simulation results show that the DFAPR is suitable for MMDB cluster because it provides a better response time and maximize the locality of processing so it could be developed parallel processing of MMDB in cluster environment.

Grid Scheduling

2011 ◽  
pp. 86-111
Author(s):  
Florin Pop
Keyword(s):  
Distributed System ◽  
Critical Analysis ◽  
Heterogeneous Computing ◽  
Computing System ◽  
Optimization Techniques ◽  
Grid Scheduling ◽  
Grid Systems ◽  
Grid Environments ◽  
Scheduling Models ◽  
Heterogeneous Computing System

This chapter will present the scheduling mechanism in distributed systems with direct application in grids. The resource heterogeneity, the size and number of tasks, the variety of policies, and the high number of constraints are some of the main characteristics that contribute to this complexity. The necessity of scheduling in grid is sustained by the increasing of number of users and applications. The design of scheduling algorithms for a heterogeneous computing system interconnected with an arbitrary communication network is one of the actual concerns in distributed system research. The main concerns presented in the chapter refers to general presentation of scheduling for grid systems, specific requirements of scheduling in grids, critical analysis of existing methods and algorithms for grid schedulers, scheduling policies, fault tolerance in scheduling process in grid environments, scheduling models and algorithms and optimization techniques for grid scheduling.

Performance Evaluation of Geolocation Based Opportunistic Spectrum Access in Cloud-Assisted Cognitive Radio Networks

2016 ◽  
Vol 4 (1) ◽  
pp. 24-41
Author(s):  
Swetha Reddy ◽  
Isaac Cushman ◽  
Danda B. Rawat ◽  
Min Song
Keyword(s):  
Cognitive Radio ◽  
False Alarm ◽  
Estimation Error ◽  
Time Of Arrival ◽  
Spectrum Access ◽  
Angle Of Arrival ◽  
Secondary Users ◽  
Probability Of False Alarm ◽  
Computing Platform ◽  
Overall Performance

The popularity of cloud-assisted database-driven cognitive radio network (CRN) has increased significantly due to three main reasons; reduced sensing uncertainties (caused by the use of spectrum scanning and sensing techniques), FCC mandated use of a database for storing and utilizing idle channels, and leveraging cloud computing platform to process big data generated by wideband sensing and analyzing. In database-driven CRN, secondary users periodically query the database to find idle channels for opportunistic communications where secondary users use their geolocation (with the help of Global Positioning System - GPS) to find idle channels for given location and time. Use of GPS makes the overall CRN vulnerable where malicious users falsify their geolocations through GPS spoofing to find more channels. The other main drawback of GPS is estimation error while finding location of users and idle bands. Due to this there will be probability of misdetection and false alarm which will have its effect on overall performance and efficiency of the system. In this paper, the authors present a three-stage mechanism for detecting GPS spoofing attacks using angle of arrival, received signal strength and time of arrival. They also evaluate the probability of misdetection and probability of false alarm in this system while detecting location of secondary users. The authors evaluate the performance of the proposed approach using numerical results.

Study and Optimization of T-Tree Index in Main Memory Database

2013 ◽  
Vol 427-429 ◽  
pp. 2531-2535 ◽  
Author(s):  
Feng Dong Sun ◽  
Quan Guo ◽  
Lan Wang
Keyword(s):  
High Performance ◽  
Main Memory ◽  
Memory Access ◽  
Index Structure ◽  
Index Structures ◽  
Clock Speed ◽  
Main Memory Database ◽  
Tree Index ◽  
Overall Performance

The bottleneck is not the disk I/O but CUP clock speed faster than the memory speed in main memory database .In order to achieve high performance in main memory database ,it is a good approach to design new index structures to improve the memory access speed .This chapter presents a T-tree index structure and its algorithms in main memory database firstly .Then presents two results on Optimization of T-tree index ,including T-tail tree and TTB-tree. Our results indicate that the T-Tree provides good overall performance in main memory.

Sign in / Sign up

Export Citation Format

Share Document