scholarly journals Reliable and High Performance Flash Storage Support for Big Data

2017 ◽  
Author(s):  
Xubin He
Keyword(s):  
Big Data ◽  
High Performance ◽  
Flash Storage

scholarly journals Computational storage: an efficient and scalable platform for big data and HPC applications

2019 ◽  
Vol 6 (1) ◽  
Author(s):  
Mahdi Torabzadehkashi ◽  
Siavash Rezaei ◽  
Ali HeydariGorji ◽  
Hosein Bobarshad ◽  
Vladimir Alves ◽  
...  
Keyword(s):  
Big Data ◽  
High Performance ◽  
Distributed Processing ◽  
Data Access ◽  
Distributed Applications ◽  
Process Data ◽  
Storage Devices ◽  
Hadoop Mapreduce ◽  
Big Data Applications ◽  
Application Processor

AbstractIn the era of big data applications, the demand for more sophisticated data centers and high-performance data processing mechanisms is increasing drastically. Data are originally stored in storage systems. To process data, application servers need to fetch them from storage devices, which imposes the cost of moving data to the system. This cost has a direct relation with the distance of processing engines from the data. This is the key motivation for the emergence of distributed processing platforms such as Hadoop, which move process closer to data. Computational storage devices (CSDs) push the “move process to data” paradigm to its ultimate boundaries by deploying embedded processing engines inside storage devices to process data. In this paper, we introduce Catalina, an efficient and flexible computational storage platform, that provides a seamless environment to process data in-place. Catalina is the first CSD equipped with a dedicated application processor running a full-fledged operating system that provides filesystem-level data access for the applications. Thus, a vast spectrum of applications can be ported for running on Catalina CSDs. Due to these unique features, to the best of our knowledge, Catalina CSD is the only in-storage processing platform that can be seamlessly deployed in clusters to run distributed applications such as Hadoop MapReduce and HPC applications in-place without any modifications on the underlying distributed processing framework. For the proof of concept, we build a fully functional Catalina prototype and a CSD-equipped platform using 16 Catalina CSDs to run Intel HiBench Hadoop and HPC benchmarks to investigate the benefits of deploying Catalina CSDs in the distributed processing environments. The experimental results show up to 2.2× improvement in performance and 4.3× reduction in energy consumption, respectively, for running Hadoop MapReduce benchmarks. Additionally, thanks to the Neon SIMD engines, the performance and energy efficiency of DFT algorithms are improved up to 5.4× and 8.9×, respectively.

High-Performance Computing for Big Data Processing

2018 ◽  
Vol 88 ◽  
pp. 693-695 ◽  
Author(s):  
Yulei Wu ◽  
Yang Xiang ◽  
Jingguo Ge ◽  
Peter Muller
Keyword(s):  
Big Data ◽  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Big Data Processing ◽  
Performance Computing

scholarly journals Intrusion Detection Based on Big Data Fuzzy Analytics

2021 ◽  
Author(s):  
Farah Jemili ◽  
Hajer Bouras
Keyword(s):  
Big Data ◽  
Network Security ◽  
Intrusion Detection ◽  
Intrusion Detection System ◽  
High Performance ◽  
Detection System ◽  
Training Dataset ◽  
False Alarms ◽  
Massive Datasets ◽  
Detection Rates

In today’s world, Intrusion Detection System (IDS) is one of the significant tools used to the improvement of network security, by detecting attacks or abnormal data accesses. Most of existing IDS have many disadvantages such as high false alarm rates and low detection rates. For the IDS, dealing with distributed and massive data constitutes a challenge. Besides, dealing with imprecise data is another challenge. This paper proposes an Intrusion Detection System based on big data fuzzy analytics; Fuzzy C-Means (FCM) method is used to cluster and classify the pre-processed training dataset. The CTU-13 and the UNSW-NB15 are used as distributed and massive datasets to prove the feasibility of the method. The proposed system shows high performance in terms of accuracy, precision, detection rates, and false alarms.

scholarly journals Big Data and IT Network Data Visualization

2018 ◽  
Vol 3 (1) ◽  
pp. 9-16 ◽  
Author(s):  
Lidong Wang
Keyword(s):  
Big Data ◽  
Network Analysis ◽  
Graphics Processing Units ◽  
Data Analytics ◽  
High Performance ◽  
Big Data Analytics ◽  
Network Visualization ◽  
Network Data ◽  
Graphics Processing ◽  
Performance Computing

Visualization with graphs is popular in the data analysis of Information Technology (IT) networks or computer networks. An IT network is often modelled as a graph with hosts being nodes and traffic being flows on many edges. General visualization methods are introduced in this paper. Applications and technology progress of visualization in IT network analysis and big data in IT network visualization are presented. The challenges of visualization and Big Data analytics in IT network visualization are also discussed. Big Data analytics with High Performance Computing (HPC) techniques, especially Graphics Processing Units (GPUs) helps accelerate IT network analysis and visualization.

Sign in / Sign up

Export Citation Format

Share Document