Research and Design of the Distributed Mass Small File Storage System Based on WCF

2013 ◽  
Vol 765-767 ◽  
pp. 1087-1091
Author(s):  
Hong Lin ◽  
Shou Gang Chen ◽  
Bao Hui Wang
Keyword(s):  
Fault Tolerance ◽  
Distributed Computing ◽  
Data Storage ◽  
Large Scale ◽  
Storage System ◽  
Hardware Platform ◽  
File Storage ◽  
Computing Framework ◽  
Research And Design ◽  
Small File

Recently, with the development of Internet and the coming of new application modes, data storage has some new characters and new requirements. In this paper, a Distributed Computing Framework Mass Small File storage System (For short:Dnet FS) based on Windows Communication Foundation in .Net platform is presented, which is lightweight, good-expansibility, running in cheap hardware platform, supporting Large-scale concurrent access, and having certain fault-tolerance. The framework of this system is analyzed and the performance of this system is tested and compared. All of these prove this system meet requirements.

Development and Evaluation of Local Area Network Based Archiving System

2021 ◽  
Vol 5 (2) ◽  
pp. 286-296
Author(s):  
Reymon M Santiañez ◽  
Benedict M Sollano
Keyword(s):  
Data Storage ◽  
Local Area Network ◽  
Storage System ◽  
Local Area ◽  
Information Storage ◽  
Area Network ◽  
Efficient Operation ◽  
File Storage ◽  
Electronic File ◽  
Work Cycle

The goal of this study was to create the Local Area Network Based Archiving System, a cross-platform development system for electronic information storage, security, preservation, and retention. The system incorporates capabilities such as data storage for long-term preservation and retrieval, file searching and retrieval, security features such as user account information system and account access privilege levels, and an email-like messaging system. The researchers developed the Local Area Network Based Archiving System using the Agile Software Development Methodology to keep up with the stakeholders' ever-changing needs. After each iteration of the work cycle, this methodology employs a process of frequent feedback. Features are added or refined in each iteration to ensure that the study meets its goals and expectations. The developed system received an overall average weighted mean of 4.53 in the evaluation summary, which is considered excellent. The strongest point of the system, according to the respondents' responses, was its content, which received the highest average mean among the five major categories in the system evaluation. The system's mobile responsiveness was a huge plus, as it considerably aided accessibility. The system should also be deployed, according to the respondents, because it will provide a powerful answer to the ongoing challenges with storing, managing, securing, and retrieving electronic files. As a result, the researchers concluded that a Local Area Network Based Archiving System is required for the efficient operation of an electronic  file storage system. Having centralized electronic file storage and retrieval system not only saves time and money in the long run but also allows for disaster recovery and business continuity.

scholarly journals New Algorithms for Secure Outsourcing of Large-Scale Systems of Linear Equations

2020 ◽  
Vol 9 (5) ◽  
pp. 545-550
Keyword(s):  
Cloud Computing ◽  
Distributed Computing ◽  
Data Storage ◽  
Large Scale ◽  
Homomorphic Encryption ◽  
Linear Equations ◽  
Force Transmission ◽  
Large Scale Systems ◽  
Secure Outsourcing ◽  
Systems Of Linear Equations

Cloud computing is the on-request accessibility of computer system resources, specially data storage and computing power, without direct dynamic management by the client. In the simplest terms, cloud computing means storing and accessing data and programs over the Internet instead of your computer’s hard drive. Along the improvement of cloud computing, more and more applications are migrated into the cloud. A significant element of distributed computing is pay-more only as costs arise. Distributed computing gives strong computational capacity to the general public at diminished cost that empowers clients with least computational assets to redistribute their huge calculation outstanding burdens to the cloud, and monetarily appreciate the monstrous computational force, transmission capacity, stockpiling, and even reasonable programming that can be partaken in a compensation for each utilization way Tremendous bit of leeway is the essential objective that forestalls the wide scope of registering model for clients when their secret information are expended during the figuring procedure. Critical thinking is a system to arrive at the pragmatic objective of specific instruments that tackles the issues as well as shield from pernicious practices.. In this paper, we examine secure outsourcing for large-scale systems of linear equations, which are the most popular problems in various engineering disciplines. Linear programming is an operation research technique formulates private data by the customer for LP problem as a set of matrices and vectors, to develop a set of efficient privacypreserving problem transformation techniques, which allow customers to transform original LP problem into some arbitrary one while protecting sensitive input/output information. Identify that LP problem solving in Cloud component is efficient extra cost on cloud server. In this paper we are utilizing Homomorphic encryption system to increase the performance and time efficiency

Grid Data Handling

2013 ◽  
pp. 294-321
Author(s):  
Alexandru Costan
Keyword(s):  
Fault Tolerance ◽  
Data Storage ◽  
Large Scale ◽  
File Systems ◽  
Future Research ◽  
Distributed Data ◽  
Data Handling ◽  
Grid Data ◽  
Distributed Data Storage ◽  
Grid Environments

To accommodate the needs of large-scale distributed systems, scalable data storage and management strategies are required, allowing applications to efficiently cope with continuously growing, highly distributed data. This chapter addresses the key issues of data handling in grid environments focusing on storing, accessing, managing and processing data. We start by providing the background for the data storage issue in grid environments. We outline the main challenges addressed by distributed storage systems: high availability which translates into high resilience and consistency, corruption handling regarding arbitrary faults, fault tolerance, asynchrony, fairness, access control and transparency. The core part of the chapter presents how existing solutions cope with these high requirements. The most important research results are organized along several themes: grid data storage, distributed file systems, data transfer and retrieval and data management. Important characteristics such as performance, efficient use of resources, fault tolerance, security, and others are strongly determined by the adopted system architectures and the technologies behind them. For each topic, we shortly present previous work, describe the most recent achievements, highlight their advantages and limitations, and indicate future research trends in distributed data storage and management.

Monitoring of a Grid Storage Virtualization Service

2013 ◽  
Vol 5 (1) ◽  
pp. 53-69
Author(s):  
Jacques Jorda ◽  
Aurélien Ortiz ◽  
Abdelaziz M’zoughi ◽  
Salam Traboulsi
Keyword(s):  
Monitoring System ◽  
Data Storage ◽  
Large Scale ◽  
Distributed Storage ◽  
Storage System ◽  
Data Access ◽  
Data Placement ◽  
Workload Prediction ◽  
Storage Virtualization

Grid computing is commonly used for large scale application requiring huge computation capabilities. In such distributed architectures, the data storage on the distributed storage resources must be handled by a dedicated storage system to ensure the required quality of service. In order to simplify the data placement on nodes and to increase the performance of applications, a storage virtualization layer can be used. This layer can be a single parallel filesystem (like GPFS) or a more complex middleware. The latter is preferred as it allows the data placement on the nodes to be tuned to increase both the reliability and the performance of data access. Thus, in such a middleware, a dedicated monitoring system must be used to ensure optimal performance. In this paper, the authors briefly introduce the Visage middleware – a middleware for storage virtualization. They present the most broadly used grid monitoring systems, and explain why they are not adequate for virtualized storage monitoring. The authors then present the architecture of their monitoring system dedicated to storage virtualization. We introduce the workload prediction model used to define the best node for data placement, and show on a simple experiment its accuracy.

A Review: Map Reduce Framework for Cloud Computing

2018 ◽  
Vol 7 (4.6) ◽  
pp. 13
Author(s):  
Mekala Sandhya ◽  
Ashish Ladda ◽  
Dr. Uma N Dulhare ◽  
. . ◽  
. .
Keyword(s):  
Data Mining ◽  
Cloud Computing ◽  
Distributed Computing ◽  
Data Storage ◽  
High Performance ◽  
Large Scale ◽  
Distributed Storage ◽  
Large Data ◽  
Mass Data ◽  
Internet Information

In this generation of Internet, information and data are growing continuously. Even though various Internet services and applications. The amount of information is increasing rapidly. Hundred billions even trillions of web indexes exist. Such large data brings people a mass of information and more difficulty discovering useful knowledge in these huge amounts of data at the same time. Cloud computing can provide infrastructure for large data. Cloud computing has two significant characteristics of distributed computing i.e. scalability, high availability. The scalability can seamlessly extend to large-scale clusters. Availability says that cloud computing can bear node errors. Node failures will not affect the program to run correctly. Cloud computing with data mining does significant data processing through high-performance machine. Mass data storage and distributed computing provide a new method for mass data mining and become an effective solution to the distributed storage and efficient computing in data mining. 

scholarly journals Collaborative Learning Based Straggler Prevention in Large-Scale Distributed Computing Framework

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Shyam Deshmukh ◽  
Komati Thirupathi Rao ◽  
Mohammad Shabaz
Keyword(s):  
Collaborative Learning ◽  
Distributed Computing ◽  
Large Scale ◽  
Cluster Computing ◽  
Data Transfer ◽  
Training Data ◽  
Shared Resources ◽  
Training Time ◽  
Big Data Applications ◽  
Computing Framework

Modern big data applications tend to prefer a cluster computing approach as they are linked to the distributed computing framework that serves users jobs as per demand. It performs rapid processing of tasks by subdividing them into tasks that execute in parallel. Because of the complex environment, hardware and software issues, tasks might run slowly leading to delayed job completion, and such phenomena are also known as stragglers. The performance improvement of distributed computing framework is a bottleneck by straggling nodes due to various factors like shared resources, heavy system load, or hardware issues leading to the prolonged job execution time. Many state-of-the-art approaches use independent models per node and workload. With increased nodes and workloads, the number of models would increase, and even with large numbers of nodes. Not every node would be able to capture the stragglers as there might not be sufficient training data available of straggler patterns, yielding suboptimal straggler prediction. To alleviate such problems, we propose a novel collaborative learning-based approach for straggler prediction, the alternate direction method of multipliers (ADMM), which is resource-efficient and learns how to efficiently deal with mitigating stragglers without moving data to a centralized location. The proposed framework shares information among the various models, allowing us to use larger training data and bring training time down by avoiding data transfer. We rigorously evaluate the proposed method on various datasets with high accuracy results.

Geo-Location-Based File Security System for Healthcare Data

2018 ◽  
pp. 125-135
Author(s):  
Govinda K.
Keyword(s):  
Data Storage ◽  
Credit Card ◽  
Medical Information ◽  
Storage Systems ◽  
Storage System ◽  
Security System ◽  
Sensitive Data ◽  
Healthcare Data ◽  
File Storage ◽  
Cryptographic Techniques

Nowadays, a person's medical information is just as important as their financial records as they may include not only names and addresses but also various sensitive data such as their employee details, bank account/credit card information, insurance details, etc. However, this fact is often overlooked when designing a file storage system for storing healthcare data. Storage systems are increasingly subject to attacks, so the security system is quickly becoming a mandatory feature of the data storage systems. For the purpose of security, we are dependent on various methods such as cryptographic techniques, two-step verification, and even biometric scanners. This chapter provides a mechanism to create a secure file storage system that provides two-layer security. The first layer is in the form of a password, through which the file is encrypted at the time of storage, and second is the locations at which the user wants the files to be accessed. Thus, this system would allow a user to access a file only at the locations specified by him/her. Therefore, the objective is to create a system that provides secure file storage based on geo-location information.

Sign in / Sign up

Export Citation Format

Share Document