The High-Performance Technologies for Big and Fast Data Analytics

Author(s):  
Pethuru Raj ◽  
Anupama Raman ◽  
Dhivya Nagaraj ◽  
Siddhartha Duggirala
2021 ◽  
Vol 13 (9) ◽  
pp. 4640
Author(s):  
Seung-Yeoun Choi ◽  
Sean-Hay Kim

New functions and requirements of high performance building (HPB) being added and several regulations and certification conditions being reinforced steadily make it harder for designers to decide HPB designs alone. Although many designers wish to rely on HPB consultants for advice, not all projects can afford consultants. We expect that, in the near future, computer aids such as design expert systems can help designers by providing the role of HPB consultants. The effectiveness and success or failure of the solution offered by the expert system must be affected by the quality, systemic structure, resilience, and applicability of expert knowledge. This study aims to set the problem definition and category required for existing HPB designs, and to find the knowledge acquisition and representation methods that are the most suitable to the design expert system based on the literature review. The HPB design literature from the past 10 years revealed that the greatest features of knowledge acquisition and representation are the increasing proportion of computer-based data analytics using machine learning algorithms, whereas rules, frames, and cognitive maps that are derived from heuristics are conventional representation formalisms of traditional expert systems. Moreover, data analytics are applied to not only literally raw data from observations and measurement, but also discrete processed data as the results of simulations or composite rules in order to derive latent rule, hidden pattern, and trends. Furthermore, there is a clear trend that designers prefer the method that decision support tools propose a solution directly as optimizer does. This is due to the lack of resources and time for designers to execute performance evaluation and analysis of alternatives by themselves, even if they have sufficient experience on the HPB. However, because the risk and responsibility for the final design should be taken by designers solely, they are afraid of convenient black box decision making provided by machines. If the process of using the primary knowledge in which frame to reach the solution and how the solution is derived are transparently open to the designers, the solution made by the design expert system will be able to obtain more trust from designers. This transparent decision support process would comply with the requirement specified in a recent design study that designers prefer flexible design environments that give more creative control and freedom over design options, when compared to an automated optimization approach.


Author(s):  
Lidong Wang

Visualization with graphs is popular in the data analysis of Information Technology (IT) networks or computer networks. An IT network is often modelled as a graph with hosts being nodes and traffic being flows on many edges. General visualization methods are introduced in this paper. Applications and technology progress of visualization in IT network analysis and big data in IT network visualization are presented. The challenges of visualization and Big Data analytics in IT network visualization are also discussed. Big Data analytics with High Performance Computing (HPC) techniques, especially Graphics Processing Units (GPUs) helps accelerate IT network analysis and visualization.


Author(s):  
Herbert Cornelius

For decades, HPC has established itself as an essential tool for discoveries, innovations and new insights in science, research and development, engineering and business across a wide range of application areas in academia and industry. Today High-Performance Computing is also well recognized to be of strategic and economic value – HPC matters and is transforming industries. This article will discuss new emerging technologies that are being developed for all areas of HPC: compute/processing, memory and storage, interconnect fabric, I/O and software to address the ongoing challenges in HPC such as balanced architecture, energy efficient high-performance, density, reliability, sustainability, and last but not least ease-of-use. Of specific interest are the challenges and opportunities for the next frontier in HPC envisioned around the 2020 timeframe: ExaFlops computing. We will also outline the new and emerging area of High Performance Data Analytics, Big Data Analytics using HPC, and discuss the emerging new delivery mechanism for HPC - HPC in the Cloud.


Big Data ◽  
2016 ◽  
pp. 1555-1581
Author(s):  
Gueyoung Jung ◽  
Tridib Mukherjee

In the modern information era, the amount of data has exploded. Current trends further indicate exponential growth of data in the future. This prevalent humungous amount of data—referred to as big data—has given rise to the problem of finding the “needle in the haystack” (i.e., extracting meaningful information from big data). Many researchers and practitioners are focusing on big data analytics to address the problem. One of the major issues in this regard is the computation requirement of big data analytics. In recent years, the proliferation of many loosely coupled distributed computing infrastructures (e.g., modern public, private, and hybrid clouds, high performance computing clusters, and grids) have enabled high computing capability to be offered for large-scale computation. This has allowed the execution of the big data analytics to gather pace in recent years across organizations and enterprises. However, even with the high computing capability, it is a big challenge to efficiently extract valuable information from vast astronomical data. Hence, we require unforeseen scalability of performance to deal with the execution of big data analytics. A big question in this regard is how to maximally leverage the high computing capabilities from the aforementioned loosely coupled distributed infrastructure to ensure fast and accurate execution of big data analytics. In this regard, this chapter focuses on synchronous parallelization of big data analytics over a distributed system environment to optimize performance.


2020 ◽  
Vol 35 (1) ◽  
pp. 194-208
Author(s):  
Zheng-Hao Jin ◽  
Haiyang Shi ◽  
Ying-Xin Hu ◽  
Li Zha ◽  
Xiaoyi Lu

2019 ◽  
Vol 35 (4) ◽  
pp. 893-903 ◽  
Author(s):  
Seemu Sharma ◽  
Seema Bawa

Abstract Cultural data and information on the web are continuously increasing, evolving, and reshaping in the form of big data due to globalization, digitization, and its vast exploration, with common people realizing the importance of ancient values. Therefore, before it becomes unwieldy and too complex to manage, its integration in the form of big data repositories is essential. This article analyzes the complexity of the growing cultural data and presents a Cultural Big Data Repository as an efficient way to store and retrieve cultural big data. The repository is highly scalable and provides integrated high-performance methods for big data analytics in cultural heritage. Experimental results demonstrate that the proposed repository outperforms in terms of space as well as storage and retrieval time of Cultural Big Data.


2013 ◽  
Author(s):  
E. Wes Bethel ◽  
Prabhat Prabhat ◽  
Suren Byna ◽  
Oliver Rübel ◽  
K. John Wu ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document