scholarly journals Research and Application of Massive Data Processing in Oil Services

2012 ◽  
Vol 6-7 ◽  
pp. 1036-1040
Author(s):  
Bao An Li
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Internet Of Things ◽  
Data Processing ◽  
Rapid Growth ◽  
Massive Data ◽  
Oil Drilling ◽  
Data Problem ◽  
Traditional Industry ◽  
Massive Data Processing

Big data problem has caused widespread concern from industry to academia in recent years. As the amount of data produced by various industries and sectors of rapid growth, increasing demands on data processing and analysis capabilities, how to face the challenges of data, discover new opportunities, the issue has received wide attention. As a traditional industry, the oil drilling or refinery enterprise is facing the operational status of the system to produce large amounts of data. This text introduced an approach to massive data processing for oil enterprise based on cloud computing and Internet of Things.

scholarly journals Uma análise do consumo de energia de ambientes de processamento de dados massivos em nuvem

2018 ◽  
Author(s):  
Nestor D. O. Volpini ◽  
Vinicius S. Conceição ◽  
Raphael L. Pontes ◽  
Dorgival Guedes
Keyword(s):  
Data Mining ◽  
Cloud Computing ◽  
Big Data ◽  
Data Processing ◽  
Performance Metrics ◽  
Massive Data ◽  
Distributed Environments ◽  
Big Data Applications ◽  
Massive Data Processing ◽  
Data Mining Application

Massive data processing (big-data) related fields and cloud computing have been growing conjointly. Thus, data processing is among the largest resource consumers in datacenters, consuming around 2% of global energy. Comprehension of how elements such as virtualized environments and applications' parallelization degree affect such consumption is therefore an urgent need. This article relies on a monitoring solution that provides performance metrics, data mining application logs, and data produced in distributed environments to assess how power consumption of virtualized big-data applications varies on allocated resources.

Trends and Technologies in Big Data Processing

2020 ◽  
pp. 17-42
Author(s):  
Amitava Choudhury ◽  
Kalpana Rangra
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Internet Of Things ◽  
Data Processing ◽  
Real Time ◽  
Computing Technology ◽  
Time Data ◽  
Big Data Processing ◽  
Real Time Data ◽  
Real Time Data Processing

Data type and amount in human society is growing at an amazing speed, which is caused by emerging new services such as cloud computing, internet of things, and location-based services. The era of big data has arrived. As data has been a fundamental resource, how to manage and utilize big data better has attracted much attention. Especially with the development of the internet of things, how to process a large amount of real-time data has become a great challenge in research and applications. Recently, cloud computing technology has attracted much attention to high performance, but how to use cloud computing technology for large-scale real-time data processing has not been studied. In this chapter, various big data processing techniques are discussed.

scholarly journals Upgrading a high performance computing environment for massive data processing

2019 ◽  
Vol 10 (1) ◽  
Author(s):  
Lucas M. Ponce ◽  
Walter dos Santos ◽  
Wagner Meira ◽  
Dorgival Guedes ◽  
Daniele Lezzi ◽  
...  
Keyword(s):  
Big Data ◽  
Data Processing ◽  
High Performance Computing ◽  
High Performance ◽  
Data Access ◽  
Massive Data ◽  
Analysis Tool ◽  
Data Framework ◽  
Performance Computing ◽  
Massive Data Processing

Abstract High-performance computing (HPC) and massive data processing (Big Data) are two trends that are beginning to converge. In that process, aspects of hardware architectures, systems support and programming paradigms are being revisited from both perspectives. This paper presents our experience on this path of convergence with the proposal of a framework that addresses some of the programming issues derived from such integration. Our contribution is the development of an integrated environment that integretes (i) COMPSs, a programming framework for the development and execution of parallel applications for distributed infrastructures; (ii) Lemonade, a data mining and analysis tool; and (iii) HDFS, the most widely used distributed file system for Big Data systems. To validate our framework, we used Lemonade to create COMPSs applications that access data through HDFS, and compared them with equivalent applications built with Spark, a popular Big Data framework. The results show that the HDFS integration benefits COMPSs by simplifying data access and by rearranging data transfer, reducing execution time. The integration with Lemonade facilitates COMPSs’s use and may help its popularization in the Data Science community, by providing efficient algorithm implementations for experts from the data domain that want to develop applications with a higher level abstraction.

scholarly journals Aproximación al grado de conocimiento y aplicación de Big Data en las bibliotecas universitarias españolas

2020 ◽  
Vol 23 (1) ◽  
Author(s):  
Ana Belén Ríos Hilario ◽  
Alberto Fraile Sastre
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Massive Data ◽  
University Libraries ◽  
Internal Sources ◽  
Massive Data Processing ◽  
Correct Implementation ◽  
Big Data Technology

Se analiza el grado de conocimiento e implantación de la tecnología Big Data y sus características principales en las bibliotecas universitarias españolas inscritas en REBIUN con el objetivo de observar si estas instituciones se encuentran capacitadas para la utilización y aprovechamiento de las ventajas del tratamiento masivo de datos. Los datos son obtenidos mediante un cuestionario cuya respuesta proviene de fuentes internas de las bibliotecas, a partir de los cuales se establecen una serie de conclusiones junto a unas propuestas de mejora y líneas de trabajo futuras que permitan la correcta implantación, uso y aprovechamiento del Big Data en la oferta de servicios y funciones de las bibliotecas universitarias españolas. It is analyzed the degree of knowledge and implementation of Big Data technology and its main characteristics in the Spanish university libraries registered in REBIUN, with the objective of observing if these institutions are qualified for the use of the advantages of the massive data processing. The data is obtained by means of a questionnaire, whose response comes from internal sources of the libraries, from which a series of conclusions are established together with proposals for improvement and future lines of work that allow the correct implementation, use and exploitation of the Big Data in the offer of services and functions of the Spanish university libraries.

Trends and Technologies in Big Data Processing

2021 ◽  
pp. 42-67
Author(s):  
Amitava Choudhury ◽  
Kalpana Rangra
Keyword(s):  
Cloud Computing ◽  
Big Data ◽  
Internet Of Things ◽  
Data Processing ◽  
Real Time ◽  
Computing Technology ◽  
Time Data ◽  
Big Data Processing ◽  
Real Time Data ◽  
Real Time Data Processing

Data type and amount in human society is growing at an amazing speed, which is caused by emerging new services such as cloud computing, internet of things, and location-based services. The era of big data has arrived. As data has been a fundamental resource, how to manage and utilize big data better has attracted much attention. Especially with the development of the internet of things, how to process a large amount of real-time data has become a great challenge in research and applications. Recently, cloud computing technology has attracted much attention to high performance, but how to use cloud computing technology for large-scale real-time data processing has not been studied. In this chapter, various big data processing techniques are discussed.

scholarly journals SMART RESOURCE USAGE PREDICTION USING CLOUD COMPUTING FOR MASSIVE DATA PROCESSING SYSTEMS

2019 ◽  
Vol 01 (02) ◽  
pp. 108-118 ◽  
Author(s):  
Abraham Chandy
Keyword(s):  
Resource Allocation ◽  
Cloud Computing ◽  
Data Processing ◽  
Processing System ◽  
Massive Data ◽  
Network Simulator ◽  
Learning Approaches ◽  
Memory Usage ◽  
Data Processing System ◽  
Massive Data Processing

Resource management plays the vital role in the cloud computing as the requirement for the massive data processing system such as heath sectors, business solutions and the internet of things keeps on increasing in at an exponential range. Allocation of proper and perfect resources remains as the mains reasons for the successful computation of the applications. However the conventional resources management methodologies, that totally depends on the simple heuristic based methods fails to accomplish a performance that is predictable. The appropriate resource allocation is directly related to the workload demand prediction as the would help to bring down the cost, time and power and the memory usage. The proposed method in the paper leverages the machine learning approaches to manage the resource allocation in the cloud computing for the massive data processing system, the simulation of the proposed model using the network simulator -2 enables to achieve a better performance and resources utilization at a decreased cost, time, power and memory usage.

Sign in / Sign up

Export Citation Format

Share Document