Big Data management and data analysis: Applied solutions in view of the spheres of the modern economy

Author(s):  
Larisa A. Ilyina ◽  
Ekaterina V. Lyubimova ◽  
Darya A. Prosvirina ◽  
Anton N. Sunteev
Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Zhongbo Bai ◽  
Xiaomei Bai

With the rapid growth of information technology and sports, analyzing sports information has become an increasingly challenging issue. Sports big data come from the Internet and show a rapid growth trend. Sports big data contain rich information such as athletes, coaches, athletics, and swimming. Nowadays, various sports data can be easily accessed, and amazing data analysis technologies have been developed, which enable us to further explore the value behind these data. In this paper, we first introduce the background of sports big data. Secondly, we review sports big data management such as sports big data acquisition, sports big data labeling, and improvement of existing data. Thirdly, we show sports data analysis methods, including statistical analysis, sports social network analysis, and sports big data analysis service platform. Furthermore, we describe the sports big data applications such as evaluation and prediction. Finally, we investigate representative research issues in sports big data areas, including predicting the athletes’ performance in the knowledge graph, finding a rising star of sports, unified sports big data platform, open sports big data, and privacy protections. This paper should help the researchers obtaining a broader understanding of sports big data and provide some potential research directions.


Author(s):  
Rajanala Vijaya Prakash

The data management industry has matured over the last three decades, primarily based on Relational Data Base Management Systems (RDBMS) technology. The amount of data collected and analyzed in enterprises has increased several folds in volume, variety and velocity of generation and consumption, organizations have started struggling with architectural limitations of traditional RDBMS architecture. As a result a new class of systems had to be designed and implemented, giving rise to the new phenomenon of “Big Data”. The data-driven world has the potential to improve the efficiencies of enterprises and improve the quality of our lives. There are a number of challenges that must be addressed to allow us to exploit the full potential of Big Data. This article highlights the key technical challenges of Big Data.


Author(s):  
Aqeel-ur Rehman ◽  
Rafi Ullah ◽  
Faisal Abdullah

In IoT, data management is a big problem due to the connectivity of billions of devices, objects, processes generating big data. Since the Things are not following any specific (common) standard, so analysis of such data becomes a big challenge. There is a need to elaborate about the characteristics of IoT based data to find out the available and applicable solutions. Such kind of study also directs to realize the need of new techniques to cope up with such challenges. Due to the heterogeneity of connected nodes, different data rates and formats it is getting a huge challenge to deal with such variety of data. As IoT is providing processing nodes in quantity in form of smart nodes, it is presenting itself a good platform for big data analysis. In this chapter, characteristics of big data and requirements for big data analysis are highlighted. Considering the big source of data generation as well as the plausible suitable platform of such huge data analysis, the associated challenges are also underlined.


Author(s):  
Anushree Raj ◽  
Rio D’Souza

In this era of information age, a huge amount of data generates every moment through various sources. This enormous data is beyond the processing capability of traditional data management system to manage and analyse the data in a specified time span. This huge amount of data refers to Big Data. Big Data faces numerous challenges in various operations on data such as capturing data, data analysis, data searching, data sharing, data filtering etc. HADOOP has showed a big way of various enterprises for big data management. Big data hadoop deals with the implementation of various industry use cases. To master the Apache Hadoop, we need to understand the hadoop eco system and hadoop architecture. In this paper we brief on the Hadoop architecture and hadoop eco system.


2017 ◽  
pp. 383-397
Author(s):  
Aqeel-ur Rehman ◽  
Rafi Ullah ◽  
Faisal Abdullah

In IoT, data management is a big problem due to the connectivity of billions of devices, objects, processes generating big data. Since the Things are not following any specific (common) standard, so analysis of such data becomes a big challenge. There is a need to elaborate about the characteristics of IoT based data to find out the available and applicable solutions. Such kind of study also directs to realize the need of new techniques to cope up with such challenges. Due to the heterogeneity of connected nodes, different data rates and formats it is getting a huge challenge to deal with such variety of data. As IoT is providing processing nodes in quantity in form of smart nodes, it is presenting itself a good platform for big data analysis. In this chapter, characteristics of big data and requirements for big data analysis are highlighted. Considering the big source of data generation as well as the plausible suitable platform of such huge data analysis, the associated challenges are also underlined.


2020 ◽  
Vol 4 (2) ◽  
pp. 4 ◽  
Author(s):  
Hafiz Suliman Munawar ◽  
Siddra Qayyum ◽  
Fahim Ullah ◽  
Samad Sepasgozar

Big data is the concept of enormous amounts of data being generated daily in different fields due to the increased use of technology and internet sources. Despite the various advancements and the hopes of better understanding, big data management and analysis remain a challenge, calling for more rigorous and detailed research, as well as the identifications of methods and ways in which big data could be tackled and put to good use. The existing research lacks in discussing and evaluating the pertinent tools and technologies to analyze big data in an efficient manner which calls for a comprehensive and holistic analysis of the published articles to summarize the concept of big data and see field-specific applications. To address this gap and keep a recent focus, research articles published in last decade, belonging to top-tier and high-impact journals, were retrieved using the search engines of Google Scholar, Scopus, and Web of Science that were narrowed down to a set of 139 relevant research articles. Different analyses were conducted on the retrieved papers including bibliometric analysis, keywords analysis, big data search trends, and authors’ names, countries, and affiliated institutes contributing the most to the field of big data. The comparative analyses show that, conceptually, big data lies at the intersection of the storage, statistics, technology, and research fields and emerged as an amalgam of these four fields with interlinked aspects such as data hosting and computing, data management, data refining, data patterns, and machine learning. The results further show that major characteristics of big data can be summarized using the seven Vs, which include variety, volume, variability, value, visualization, veracity, and velocity. Furthermore, the existing methods for big data analysis, their shortcomings, and the possible directions were also explored that could be taken for harnessing technology to ensure data analysis tools could be upgraded to be fast and efficient. The major challenges in handling big data include efficient storage, retrieval, analysis, and visualization of the large heterogeneous data, which can be tackled through authentication such as Kerberos and encrypted files, logging of attacks, secure communication through Secure Sockets Layer (SSL) and Transport Layer Security (TLS), data imputation, building learning models, dividing computations into sub-tasks, checkpoint applications for recursive tasks, and using Solid State Drives (SDD) and Phase Change Material (PCM) for storage. In terms of frameworks for big data management, two frameworks exist including Hadoop and Apache Spark, which must be used simultaneously to capture the holistic essence of the data and make the analyses meaningful, swift, and speedy. Further field-specific applications of big data in two promising and integrated fields, i.e., smart real estate and disaster management, were investigated, and a framework for field-specific applications, as well as a merger of the two areas through big data, was highlighted. The proposed frameworks show that big data can tackle the ever-present issues of customer regrets related to poor quality of information or lack of information in smart real estate to increase the customer satisfaction using an intermediate organization that can process and keep a check on the data being provided to the customers by the sellers and real estate managers. Similarly, for disaster and its risk management, data from social media, drones, multimedia, and search engines can be used to tackle natural disasters such as floods, bushfires, and earthquakes, as well as plan emergency responses. In addition, a merger framework for smart real estate and disaster risk management show that big data generated from the smart real estate in the form of occupant data, facilities management, and building integration and maintenance can be shared with the disaster risk management and emergency response teams to help prevent, prepare, respond to, or recover from the disasters.


Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Yu Jiang ◽  
Hang Yu ◽  
Jun Jiang

Schizophrenia is a serious mental disease whose pathogenesis has not been fully elucidated. Its clinical evaluation and diagnosis still highly depend on the clinical experience of doctors. It is of great scientific value and clinical significance to study the inducing factors and neuropathological mechanism of schizophrenia. Based on the four research problems of schizophrenia, this paper analyzes the data types that need to be stored in clinical trials and scientific research, including basic information, case report data, neuropsychological and cognitive function evaluation, magnetic resonance data, electroencephalogram (EEG) data, and intestinal flora data. Through the demand analysis of the system, including the data management part, data analysis part, the functional demand of the system management part, and the overall nonfunctional demand of the system, the overall architecture design, functional module division, and database table structure design of the system are completed. Adopting Browser/Server (B/S) architecture and front-end and back-end separation mode and applying Java and Python programming language, based on spring framework and database, a multidimensional information management system for schizophrenia is designed and implemented, which includes four modules: data analysis, data management, system management, and security control. In addition, each functional module of the system is designed and implemented in detail, and the software operation flow of each module is illustrated with the sequence diagram. Finally, the multidimensional data of schizophrenia collected in our laboratory were used for system test to verify whether the system can meet the needs of clinical big data management of schizophrenia and the multidimensional information management system of schizophrenia can meet the needs of clinical big data management. The information management system helps schizophrenic researchers to carry out data management and data analysis. It also has advantages that are easy to use, safe, and efficient and has strong scalability in data management, data analysis, and scalability. It reflects the innovation of the system and provides a good platform for the management, research, and analysis of clinical big data of schizophrenia.


2019 ◽  
Vol 9 (1) ◽  
pp. 01-12 ◽  
Author(s):  
Kristy F. Tiampo ◽  
Javad Kazemian ◽  
Hadi Ghofrani ◽  
Yelena Kropivnitskaya ◽  
Gero Michel

Sign in / Sign up

Export Citation Format

Share Document