scholarly journals Taming Performance Variability of Healthcare Data Service Frameworks with Proactive and Coarse-Grained Memory Cleaning

2019 ◽  
Vol 16 (17) ◽  
pp. 3096
Author(s):  
Eunji Lee
Keyword(s):  
Data Storage ◽  
Memory Management ◽  
Buffer Management ◽  
Coarse Grained ◽  
Healthcare Data ◽  
Management Scheme ◽  
High Responsiveness ◽  
Performance Optimizations ◽  
Memory Buffer ◽  
Embedded Database

This article explores the performance optimizations of an embedded database memory management system to ensure high responsiveness of real-time healthcare data frameworks. SQLite is a popular embedded database engine extensively used in medical and healthcare data storage systems. However, SQLite is essentially built around lightweight applications in mobile devices, and it significantly deteriorates when a large transaction is issued such as high resolution medical images or massive health dataset, which is unlikely to occur in embedded systems but is quite common in other systems. Such transactions do not fit in the in-memory buffer of SQLite, and SQLite enforces memory reclamation as they are processed. The problem is that the current SQLite buffer management scheme does not effectively manage these cases, and the naïve reclamation scheme used significantly increases the user-perceived latency. Motivated by this limitation, this paper identifies the causes of high latency during processing of a large transaction, and overcomes the limitation via proactive and coarse-grained memory cleaning in SQLite.The proposed memory reclamation scheme was implemented in SQLite 3.29, and measurement studies with a prototype implementation demonstrated that the SQLite operation latency decreases by 13% on an average and up to 17.3% with our memory reclamation scheme as compared to that of the original version.

A New FTL-Based Flash Memory Management Scheme for Flash-Based Storage Systems

2014 ◽  
Vol 651-653 ◽  
pp. 1000-1003
Author(s):  
Yin Yang ◽  
Wen Yi Li ◽  
Kai Wang
Keyword(s):  
Service Life ◽  
Data Storage ◽  
Flash Memory ◽  
Memory Management ◽  
Storage Systems ◽  
Management Scheme ◽  
Flash Translation Layer ◽  
Link Table

In this paper, we propose a novel and efficient flash translation layer scheme called BLTF: Block Link-Table FTL. In this proposed scheme, all blocks can be used for servicing update requests, so updates operation can be performed on any of the physical blocks, through uniting log blocks and physical blocks, it can avoid uneven erasing and low block utilization. The invalid blocks, in BLTF scheme, could be reclaimed properly and intensively, it can avoid merging log blocks with physical blocks. At last, the BLTF is tested by simulation, which demonstrates the BLTF can effectively solve data storage problems. Through comparison with other algorithms, we can know that the proposed BLTF greatly prolongs service life of flash devices and improves efficiency of blocks erasing operation.

scholarly journals s2p: Provenance Research for Stream Processing System

2021 ◽  
Vol 11 (12) ◽  
pp. 5523
Author(s):  
Qian Ye ◽  
Minyan Lu
Keyword(s):  
Data Storage ◽  
Stream Processing ◽  
Processing System ◽  
Coarse Grained ◽  
Stream Data ◽  
Related Data ◽  
Dsp System ◽  
Provenance Research ◽  
Dsp Systems ◽  
Abnormal Results

The main purpose of our provenance research for DSP (distributed stream processing) systems is to analyze abnormal results. Provenance for these systems is not nontrivial because of the ephemerality of stream data and instant data processing mode in modern DSP systems. Challenges include but are not limited to an optimization solution for avoiding excessive runtime overhead, reducing provenance-related data storage, and providing it in an easy-to-use fashion. Without any prior knowledge about which kinds of data may finally lead to the abnormal, we have to track all transformations in detail, which potentially causes hard system burden. This paper proposes s2p (Stream Process Provenance), which mainly consists of online provenance and offline provenance, to provide fine- and coarse-grained provenance in different precision. We base our design of s2p on the fact that, for a mature online DSP system, the abnormal results are rare, and the results that require a detailed analysis are even rarer. We also consider state transition in our provenance explanation. We implement s2p on Apache Flink named as s2p-flink and conduct three experiments to evaluate its scalability, efficiency, and overhead from end-to-end cost, throughput, and space overhead. Our evaluation shows that s2p-flink incurs a 13% to 32% cost overhead, 11% to 24% decline in throughput, and few additional space costs in the online provenance phase. Experiments also demonstrates the s2p-flink can scale well. A case study is presented to demonstrate the feasibility of the whole s2p solution.

scholarly journals AOIO: Application Oriented I/O Optimization for Buffer Management

Symmetry ◽  
2021 ◽  
Vol 13 (4) ◽  
pp. 573
Author(s):  
Xiaochang Li ◽  
Zhengjun Zhai ◽  
Xin Ye
Keyword(s):  
System Performance ◽  
Simulation Experiment ◽  
Buffer Management ◽  
Input Output ◽  
Output Data ◽  
Program Counter ◽  
Efficient Management ◽  
Management Scheme ◽  
Buffer Cache ◽  
Management Schemes

Emerging scale-out I/O intensive applications are broadly used now, which process a large amount of data in buffer/cache for reorganization or analysis and their performances are greatly affected by the speed of the I/O system. Efficient management scheme of the limited kernel buffer plays a key role in improving I/O system performance, such as caching hinted data for reuse in future, prefetching hinted data, and expelling data not to be accessed again from a buffer, which are called proactive mechanisms in buffer management. However, most of the existing buffer management schemes cannot identify data reference regularities (i.e., sequential or looping patterns) that can benefit proactive mechanisms, and they also cannot perform in the application level for managing specified applications. In this paper, we present an A pplication Oriented I/O Optimization (AOIO) technique automatically benefiting the kernel buffer/cache by exploring the I/O regularities of applications based on program counter technique. In our design, the input/output data and the looping pattern are in strict symmetry. According to AOIO, each application can provide more appropriate predictions to operating system which achieve significantly better accuracy than other buffer management schemes. The trace-driven simulation experiment results show that the hit ratios are improved by an average of 25.9% and the execution times are reduced by as much as 20.2% compared to other schemes for the workloads we used.

scholarly journals An Efficient Log Buffer Management Scheme of Flash Memory Through Delay of Merging Hot Data Blocks

2010 ◽  
Vol 10 (1) ◽  
pp. 68-77
Author(s):  
Hak-Chul Kim ◽  
Yong-Hun Park ◽  
Jong-Hyeong Yun ◽  
Dong-Min Seo ◽  
Suk-Il Song ◽  
...  
Keyword(s):  
Flash Memory ◽  
Buffer Management ◽  
Management Scheme

scholarly journals Dynamic Resource Allocation and Memory Management using Deep Convolutional Neural Network

2019 ◽  
Vol 9 (2) ◽  
pp. 608-612
Keyword(s):  
Neural Network ◽  
Resource Allocation ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Data Storage ◽  
Memory Management ◽  
Research Work ◽  
Dynamic Resource Allocation ◽  
Mobile Platform ◽  
Dynamic Resource

Memory management is very essential task for large-scale storage systems; in mobile platform generate storage errors due to insufficient memory as well as additional task overhead. Many existing systems have illustrated different solution for such issues, like load balancing and load rebalancing. Different unusable applications which are already installed in mobile platform user never access frequently but it allocates some memory space on hard device storage. In the proposed research work we describe dynamic resource allocation for mobile platforms using deep learning approach. In Real world mobile systems users may install different kind of applications which required ad-hoc basis. Such applications may be affect to execution performance of system as well space complexity, sometime they also affect another runnable applications performance. To eliminate of such issues, we carried out an approach to allocate runtime resources for data storage for mobile platform. When system connected with cloud data server it store complete file system on remote Virtual Machine (VM) and whenever a single application required which immediately install beginning as remote server to local device. For developed of proposed system we implemented deep learning base Convolutional Neural Network (CNN), algorithm has used with tensorflow environment which reduces the time complexity for data storage as well as extraction respectively.

Sign in / Sign up

Export Citation Format

Share Document