scholarly journals A Distributed File-Based Storage System for Improving High Availability of Space Weather Data

2019 ◽  
Vol 9 (23) ◽  
pp. 5024
Author(s):  
Andrian ◽  
Kim ◽  
Ju
Keyword(s):  
Data Storage ◽  
Space Weather ◽  
File System ◽  
Single Point ◽  
Storage System ◽  
Science Research ◽  
Data Availability ◽  
Weather Data ◽  
Time Data ◽  
Replication Time

In space science research, the Indonesia National Institute of Aeronautics and Space (LAPAN) is concerned with the development of a system that provides actual information and predictions called the Space Weather Information and Forecast Services (SWIFtS). SWIFtS is supported by a data storage system that serves data, implementing a centralized storage model. This has some problems that impact to researchers as the primary users. The single point of failure and also the delay in data updating on the server is a significant issue when researchers need the latest data, but the server is unable to provide it. To overcome these problems, we proposed a new system that utilized a decentralized model for storing data, leveraging the InterPlanetary File System (IPFS) file system. Our proposed method focused on the automated background process, and its scheme would increase the data availability and throughput by spreading it into nodes through a peer-to-peer connection. Moreover, we also included system monitoring for real-time data flow from each node and information of node status that combines active and passive approaches. For system evaluation, the experiment was performed to determine the performance of the proposed system compared to the existing system by calculating mean replication time and the mean throughput of a node. As expected, performance evaluations showed that our proposed scheme had faster file replication time and supported high throughput.

The Design and Implementation of Intelligent High-Speed Disk Storage Protocols

2014 ◽  
Vol 513-517 ◽  
pp. 889-892
Author(s):  
Jin Xing Shen
Keyword(s):  
Data Storage ◽  
High Speed ◽  
Storage System ◽  
Area Network ◽  
Time Data ◽  
Storage Area Network ◽  
Disk Storage ◽  
Network Storage System ◽  
Memory Buffer ◽  
Storage Area

In this paper, through the analysis of the redundant array of independent disks system (RAID), storage area network system (SAN), network storage system (NAS), based on RAID on the system FPGA is used to design a set of intelligent high speed disk storage protocol, through the test and analysis can meet the continuous data acquisition, the real-time data storage needs, in the software through providing high access speed increases memory buffer, large storage capacity and higher data security.

A High Speed and Compact Data Acquisition and Storage System

2013 ◽  
Vol 753-755 ◽  
pp. 3125-3128
Author(s):  
Kai Bu ◽  
Hui Xu ◽  
Zhao Lin Sun ◽  
Jian Bin Liu
Keyword(s):  
Data Acquisition ◽  
Data Storage ◽  
Software Defined Radio ◽  
High Speed ◽  
Storage System ◽  
Base Stations ◽  
Time Data ◽  
Processing Application ◽  
Wideband Signal ◽  
And Storage

Data acquisition and record system plays an important role in signal processing. For the wideband signal acquisition and processing application, such as 3G/4G Base Stations, Wideband Microwave Backhaul, Military Communications, and RADAR, the need for high speed and smart software-defined radio (SDR) test platform is growing. In this paper, we implement a high compact form factor data acquisition and storage system used for the wideband signal application development. In order to implement high compact, low power and high speed real-time data storage, we use a PCIe-based switch architecture and leverage the Flash array as the storage part. The system supports RF sampling directly and offers a 16TB solid state storage module which could be read and write at the sustained speed of 12.8GB/s.

Real-Time Weather Analytics

2020 ◽  
Vol 17 (4) ◽  
pp. 15-31
Author(s):  
Lavanya K. ◽  
Sathyan Venkatanarayanan ◽  
Anay Anand Bhoraskar
Keyword(s):  
Data Storage ◽  
Weather Forecasting ◽  
Modern Science ◽  
Weather Forecast ◽  
Forecast Model ◽  
Data Availability ◽  
Third Party ◽  
Weather Data ◽  
Storage Devices ◽  
Weather Forecast Model

Weather forecasting is one of the biggest challenges that modern science is still contending with. The advent of high-power computing, technical advancement of data storage devices, and incumbent reduction in the storage cost have accelerated data collection to turmoil. In this background, many artificial intelligence techniques have been developed and opened interesting window of opportunity in hitherto difficult areas. India is on the cusp of a major technology overhaul with millions of people's data availability who were earlier unconnected with the internet. The country needs to fast forward the innovative use of available data. The proposed model endeavors to forecast temperature, precipitation, and other vital information for usability in the agrarian sector. This project intends to develop a robust weather forecast model that learns automatically from the daily feed of weather data that is input through a third-party API source. The weather feed is sourced from openweathermap, an online service that provides weather data, and is streamed into the forecast model through Kafka components. The LSTM neural network used by the forecast model is designed to continuously learn from predictions and perform actual analysis. The model can be architected to be implemented across very large applications having the capability to process large volumes of streamed or stored data.

Information Security in Data and Storage Grids through GS3

2013 ◽  
pp. 542-558
Author(s):  
Vincenzo Daniele Cunsolo ◽  
Salvatore Distefano ◽  
Antonio Puliafito ◽  
Marco Scarpa
Keyword(s):  
Data Storage ◽  
File System ◽  
Storage System ◽  
System Structure ◽  
Advanced Encryption Standard ◽  
Symmetric Cryptography ◽  
Secure Storage ◽  
File Access ◽  
System Service ◽  
And Storage

In grid computing infrastructures, the data storage subsystem is physically distributed among several nodes and logically shared among several users. This highlights the necessity of: (i) Availability for authorized users only, (ii) Confidentiality, and (iii) Integrity of information and data: in one term security. In this work we face the problem of data security in grid, by proposing a lightweight cryptography algorithm combining the strong and highly secure asymmetric cryptography technique (RSA) with the symmetric cryptography (Advanced Encryption Standard, AES). The proposed algorithm, we named Grid Secure Storage System (GS3), has been implemented on top of the Grid File Access Library (GFAL) of the gLite middleware, in order to provide a file system service with cryptography capability and POSIX interface. The choice of implementing GS3 as a file system allows to protect also the file system structure, and moreover to overcome the well-known problem of file rewriting in gLite/GFAL environments. This chapter describes and details both the GS3 algorithm and its implementation, also evaluating the performance of such implementation and discussing the obtained results.

TREAD: Privacy Preserving Incentivized Connected Vehicle Mobility Data Storage on InterPlanetary-File-System-Enabled Blockchain

2021 ◽  
pp. 036119812110450
Author(s):  
Junaid Ahmed Khan ◽  
Kavyashree Umesh Bangalore ◽  
Abdullah Kurkcu ◽  
Kaan Ozbay
Keyword(s):  
New York ◽  
Data Storage ◽  
File System ◽  
Technology Development ◽  
Personal Information ◽  
Storage System ◽  
Connected Vehicle ◽  
Test Bed ◽  
Trajectory Data ◽  
Mobility Data

Trajectory data from connected vehicles (CVs) and other micromobility sources such as e-scooters, bikes, and pedestrians is important for researchers, policy makers, and other stakeholders for leveraging the location, speed, and heading, along with other mobility data, to improve safety and bolster technology development toward innovative location-based applications for citizens. Such raw data needs to be stored and accessed from a non-proprietary database while the obfuscation and encryption techniques on current cloud-based proprietary solutions incur data losses that are deemed inefficient for accurate usage, particularly in time-sensitive real-time operations. In this paper, we target the problem of scalably storing and retrieving potentially sensitive data generated by vehicles and propose TREAD, a blockchain-based system comprising smart contracts to store this mobility data on a distributed ledger such that multiple peers can access and utilize it in different location-based applications while not revealing users’ sensitive personal information. It is, however, challenging to scalably store large amounts of constantly generated trajectories, and to achieve scalability we leverage InterPlanetary File System (IPFS), a scalable distributed peer-to-peer data storage system. To avoid users injecting malicious/fake trajectories into the ledger, we develop efficient consensus algorithms for the stakeholders to validate the storage and retrieval process in a distributed manner. We implemented TREAD on the open-source Hyperledger Fabric blockchain platform using trajectory data generated for 700 vehicles in a simulation environment well calibrated with vehicle trajectories from a real-world test-bed in New York City. Results show that TREAD scalably stores trajectory data with lower delay and overhead.

Research and Design on Intranet Security File System

2012 ◽  
Vol 433-440 ◽  
pp. 4704-4709
Author(s):  
Yan Shen Chen ◽  
De Zhi Han
Keyword(s):  
Data Storage ◽  
Data Security ◽  
High Performance ◽  
File System ◽  
Storage System ◽  
Data Encryption ◽  
Security Issue ◽  
Storage And Retrieval ◽  
System A ◽  
Secure File System

To solve the data security issue in intranet massive storage system, a Multi-Protocol Secure File System ( for short MPSFS) is designed. Firstly, the MPSFS supports the access of users with different protocols, and provides the unified access interface, so can achieve high performance in data storage and retrieval; secondly, with the help of other technologies such as identity authentication, access control and data encryption, the MPSFS can effectively ensure the data security in the intranet storage system. By the experiment, the MPSFS can provide good security and scalability for intranet massive storage system, and has less effect to the network I/O performance.

Data Storage Technology and its Development Based on Cloud Computing

2013 ◽  
Vol 756-759 ◽  
pp. 1275-1279
Author(s):  
Lin Na Huang ◽  
Feng Hua Liu
Keyword(s):  
Cloud Computing ◽  
Data Storage ◽  
Cloud Storage ◽  
High Performance ◽  
File System ◽  
Storage System ◽  
Distributed File System ◽  
Cloud Data ◽  
Storage Technology ◽  
Cloud Data Storage

Cloud storage of high performance is the basic condition for cloud computing. This article introduces the concept and advantage of cloud storage, discusses the infrastructure of cloud storage system as well as the architecture of cloud data storage, researches the details about the design of Distributed File System within cloud data storage, at the same time, puts forward different developing strategies for the enterprises according to the different roles that the enterprises are acting as during the developing process of cloud computing.

scholarly journals A Survey on Data Security in Cloud Computing Using Blockchain: Challenges, Existing-State-Of-The-Art Methods, And Future Directions

2021 ◽  
Vol 5 (3) ◽  
pp. 15-30
Author(s):  
Muhammad Usman Ashraf
Keyword(s):  
Cloud Computing ◽  
Data Storage ◽  
Data Security ◽  
Data Dissemination ◽  
State Of The Art ◽  
Storage System ◽  
Research Work ◽  
Detailed Comparison ◽  
Data Availability ◽  
Art Methods

Cloud computing is one of the ruling storage solutions. However, the cloud computing centralized storage method is not stable. Blockchain, on the other hand, is a decentralized cloud storage system that ensures data security. Cloud environments are vulnerable to several attacks which compromise the basic confidentiality, integrity, availability, and security of the network. This research focus on decentralized, safe data storage, high data availability, and effective use of storage resources. To properly respond to the situation of the blockchain method, we have conducted a comprehensive survey of the most recent and promising blockchain state-of-the-art methods, the P2P network for data dissemination, hash functions for data authentication, and IPFS (InterPlanetary File System) protocol for data integrity. Furthermore, we have discussed a detailed comparison of consensus algorithms of Blockchain concerning security. Also, we have discussed the future of blockchain and cloud computing. The major focus of this study is to secure the data in Cloud computing using blockchain and ease for researchers for further research work.

scholarly journals A Benchmark for Suitability of Alluxio over Spark

2020 ◽  
Vol 10 (1) ◽  
pp. 245-250
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Storage ◽  
Storage Systems ◽  
Distributed Storage ◽  
Storage System ◽  
Large Data ◽  
Time Data ◽  
Big Data Applications ◽  
Access To Data

Big data applications play an important role in real time data processing. Apache Spark is a data processing framework with in-memory data engine that quickly processes large data sets. It can also distribute data processing tasks across multiple computers, either on its own or in tandem with other distributed computing tools. Spark’s in-memory processing cannot share data between the applications and hence, the RAM memory will be insufficient for storing petabytes of data. Alluxio is a virtual distributed storage system that leverages memory for data storage and provides faster access to data in different storage systems. Alluxio helps to speed up data intensive Spark applications, with various storage systems. In this work, the performance of applications on Spark as well as Spark running over Alluxio have been studied with respect to several storage formats such as Parquet, ORC, CSV, and JSON; and four types of queries from Star Schema Benchmark (SSB). A benchmark is evolved to suggest the suitability of Spark Alluxio combination for big data applications. It is found that Alluxio is suitable for applications that use databases of size more than 2.6 GB storing data in JSON and CSV formats. Spark is found suitable for applications that use storage formats such as parquet and ORC with database sizes less than 2.6GB.

Sign in / Sign up

Export Citation Format

Share Document