scholarly journals Efficient IoT Data Management for Geological Disasters Based on Big Data-Turbocharged Data Lake Architecture

2021 ◽  
Vol 10 (11) ◽  
pp. 743
Author(s):  
Xiaohui Huang ◽  
Junqing Fan ◽  
Ze Deng ◽  
Jining Yan ◽  
Jiabao Li ◽  
...  
Keyword(s):  
Big Data ◽  
Data Management ◽  
Decision Makers ◽  
Prototype System ◽  
Data Repositories ◽  
Metadata Model ◽  
Parallel Data ◽  
Interactive Environment ◽  
Data Ingestion ◽  
Distributed Cache

Multi-source Internet of Things (IoT) data, archived in institutions’ repositories, are becoming more and more widely open-sourced to make them publicly accessed by scientists, developers, and decision makers via web services to promote researches on geohazards prevention. In this paper, we design and implement a big data-turbocharged system for effective IoT data management following the data lake architecture. We first propose a multi-threading parallel data ingestion method to ingest IoT data from institutions’ data repositories in parallel. Next, we design storage strategies for both ingested IoT data and processed IoT data to store them in a scalable, reliable storage environment. We also build a distributed cache layer to enable fast access to IoT data. Then, we provide users with a unified, SQL-based interactive environment to enable IoT data exploration by leveraging the processing ability of Apache Spark. In addition, we design a standard-based metadata model to describe ingested IoT data and thus support IoT dataset discovery. Finally, we implement a prototype system and conduct experiments on real IoT data repositories to evaluate the efficiency of the proposed system.

Emerging Trends and Techniques in Cloud-Based Data Analytics

2021 ◽  
pp. 37-47
Author(s):  
Shaila S. G. ◽  
Monish L. ◽  
Lavanya S. ◽  
Sowmya H. D. ◽  
Divya K.
Keyword(s):  
Social Media ◽  
Cloud Computing ◽  
Big Data ◽  
Data Management ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Decision Makers ◽  
Huge Data ◽  
Emerging Trends ◽  
Media Applications

The new trending technologies such as big data and cloud computing are in line with social media applications due to their fast growth and usage. The big data characteristic makes data management challenging. The term big data refers to an immense collection of both organised and unorganised data from various sources, and nowadays, cloud computing supports in storing and processing such a huge data. Analytics are done on huge data that helps decision makers to take decisions. However, merging two conflicting design principles brings a challenge, but it has its own advantage in business and various fields. Big data analytics in the cloud places rigorous demands on networks, storage, and servers. The chapter discusses the importance of cloud platform for big data, importance of analytics in cloud and gives detail insight about the trends and techniques adopted for cloud analytics.

Providing Successful Scat Data Management Support During Spill Response

1999 ◽  
Vol 1999 (1) ◽  
pp. 943-945 ◽  
Author(s):  
Alain Lamarche ◽  
Jack Ion ◽  
Edward H. Owens ◽  
Peter Rubec
Keyword(s):  
Data Management ◽  
Electronic System ◽  
Turnover Time ◽  
Decision Makers ◽  
Prototype System ◽  
Management Support ◽  
Speed Up ◽  
Oil Spill Response ◽  
The Cost ◽  
Computerized System

ABSTRACT Shoreline Cleanup Assessment Teams (SCAT) are now used worldwide to assess oiled shorelines as part of response cleanup activities. The amount of SCAT information gathered during surveys can be very large, with the possibility of overwhelming decision makers. New tools are now available to automate the processing of SCAT information. For example, dedicated computerized SCAT data management systems have been used during the Iron Baron (Tasmania) and Kure (California) incidents. More recently, a prototype system was developed by the State of Florida to electronically support all the steps involved in the cleanup phase of an oil spill response. Given this, when should computerized SCAT data management be used and at what level? An analysis of the work performed during recent spills involving SCAT activity provided answers to these questions. Some of the main findings include the following: (1) computerized systems can decrease the time necessary to gather data and increase the accuracy of the captured data; (2) computerized systems decrease the data turnover time and speed up the decision-making cycle; (3) an all-electronic computerized system can become essential in cases where the length of oiled shoreline is very large with respect to the number of SCAT survey teams; (4) for large spills, the increased cost of an all-electronic system may outweigh the cost of not being prepared.

A Framework for Managing Big Data in Enterprise Organizations

2020 ◽  
Vol 12 (1) ◽  
pp. 84-97
Author(s):  
Youssef Ahmed ◽  
Walaa Medhat ◽  
Tarek El Shishtawi
Keyword(s):  
Big Data ◽  
Data Management ◽  
Rapid Growth ◽  
Decision Makers ◽  
Data Service ◽  
Correct Decision ◽  
Management Framework ◽  
Critical Problems ◽  
Business Needs ◽  
Main Components

Big Data management is trending research that seeks to find a framework that will give support to decision makers in governments and enterprises organizations. For the rapid growth of data, dealing with Big Data with respect to management and finding new values has drawn attention recently. Strategies should be established together with the goals, vision, and objectives of an organization to manage Big Data. Big data management frameworks are the main components for the implementation of Big Data service. Many organizations that deals with Big Data have three critical problems, how to manage Big Data, how can Big Data create new values reference to its strategies and business needs, and how it can take the correct decision in the correct time. In this article, the authors propose a Big Data management framework that will handle all Big Data operation beginning with collecting data until making analysis and how new value can be created. The proposed framework also takes care of other factors such as organization strategies, governance, and security.

Analytical Study on Big Data

2018 ◽  
Vol 8 (5) ◽  
pp. 75
Author(s):  
Vivek Raich ◽  
Pankaj Maurya
Keyword(s):  
Information Technology ◽  
Big Data ◽  
Decision Maker ◽  
Analytical Study ◽  
Large Data ◽  
Decision Makers ◽  
Continuous Increase ◽  
Analytic Methods ◽  
Data Store ◽  
Business Engineering

in the time of the Information Technology, the big data store is going on. Due to which, Huge amounts of data are available for decision makers, and this has resulted in the progress of information technology and its wide growth in many areas of business, engineering, medical, and scientific studies. Big data means that the size which is bigger in size, but there are several types, which are not easy to handle, technology is required to handle it. Due to continuous increase in the data in this way, it is important to study and manage these datasets by adjusting the requirements so that the necessary information can be obtained.The aim of this paper is to analyze some of the analytic methods and tools. Which can be applied to large data. In addition, the application of Big Data has been analyzed, using the Decision Maker working on big data and using enlightened information for different applications.

scholarly journals DaLiF: a data lifecycle framework for data-driven governments

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Syed Iftikhar Hussain Shah ◽  
Vassilios Peristeras ◽  
Ioannis Magnisalis
Keyword(s):  
Big Data ◽  
Data Management ◽  
Good Governance ◽  
Business Community ◽  
Data Driven ◽  
Governmental Organizations ◽  
Data Lifecycle ◽  
The Government ◽  
Data Ecosystem ◽  
Governance Transparency

AbstractThe public sector, private firms, business community, and civil society are generating data that is high in volume, veracity, velocity and comes from a diversity of sources. This kind of data is known as big data. Public Administrations (PAs) pursue big data as “new oil” and implement data-centric policies to transform data into knowledge, to promote good governance, transparency, innovative digital services, and citizens’ engagement in public policy. From the above, the Government Big Data Ecosystem (GBDE) emerges. Managing big data throughout its lifecycle becomes a challenging task for governmental organizations. Despite the vast interest in this ecosystem, appropriate big data management is still a challenge. This study intends to fill the above-mentioned gap by proposing a data lifecycle framework for data-driven governments. Through a Systematic Literature Review, we identified and analysed 76 data lifecycles models to propose a data lifecycle framework for data-driven governments (DaliF). In this way, we contribute to the ongoing discussion around big data management, which attracts researchers’ and practitioners’ interest.

Sign in / Sign up

Export Citation Format

Share Document