scholarly journals Requirements Engineering for Large-Scale Big Data Applications

2020 ◽  
pp. 51-84
Author(s):  
Thalita Vergilio ◽  
Muthu Ramachandran ◽  
Duncan Mullier
Keyword(s):  
Big Data ◽  
Requirements Engineering ◽  
Large Scale ◽  
Big Data Applications

Large-Scale Sensor Network Analysis

2013 ◽  
pp. 314-347 ◽  
Author(s):  
Joaquin Vanschoren ◽  
Ugo Vespier ◽  
Shengfa Miao ◽  
Marvin Meeng ◽  
Ricardo Cachucho ◽  
...  
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Large Scale ◽  
Vital Signs ◽  
Sensor Data ◽  
Atmospheric Conditions ◽  
Big Data Applications ◽  
The World ◽  
Sheer Size ◽  
Effective Use

Sensors are increasingly being used to monitor the world around us. They measure movements of structures such as bridges, windmills, and plane wings, human’s vital signs, atmospheric conditions, and fluctuations in power and water networks. In many cases, this results in large networks with different types of sensors, generating impressive amounts of data. As the volume and complexity of data increases, their effective use becomes more challenging, and novel solutions are needed both on a technical as well as a scientific level. Founded on several real-world applications, this chapter discusses the challenges involved in large-scale sensor data analysis and describes practical solutions to address them. Due to the sheer size of the data and the large amount of computation involved, these are clearly “Big Data” applications.

Affordances of Data Science in Agriculture, Manufacturing, and Education

Web Services ◽  
2019 ◽  
pp. 953-978
Author(s):  
Krishnan Umachandran ◽  
Debra Sharon Ferdinand-James
Keyword(s):  
Big Data ◽  
Large Scale ◽  
Data Science ◽  
Data Generation ◽  
Large Scale Data ◽  
Big Data Applications ◽  
Effective Decision ◽  
Effective Decision Making ◽  
Text Images ◽  
Scale Data

Continued technological advancements of the 21st Century afford massive data generation in sectors of our economy to include the domains of agriculture, manufacturing, and education. However, harnessing such large-scale data, using modern technologies for effective decision-making appears to be an evolving science that requires knowledge of Big Data management and analytics. Big data in agriculture, manufacturing, and education are varied such as voluminous text, images, and graphs. Applying Big data science techniques (e.g., functional algorithms) for extracting intelligence data affords decision markers quick response to productivity, market resilience, and student enrollment challenges in today's unpredictable markets. This chapter serves to employ data science for potential solutions to Big Data applications in the sectors of agriculture, manufacturing and education to a lesser extent, using modern technological tools such as Hadoop, Hive, Sqoop, and MongoDB.

Affordances of Data Science in Agriculture, Manufacturing, and Education

2017 ◽  
pp. 14-40 ◽  
Author(s):  
Krishnan Umachandran ◽  
Debra Sharon Ferdinand-James
Keyword(s):  
Big Data ◽  
Large Scale ◽  
Data Science ◽  
Data Generation ◽  
Large Scale Data ◽  
Big Data Applications ◽  
Effective Decision ◽  
Effective Decision Making ◽  
Text Images ◽  
Scale Data

Continued technological advancements of the 21st Century afford massive data generation in sectors of our economy to include the domains of agriculture, manufacturing, and education. However, harnessing such large-scale data, using modern technologies for effective decision-making appears to be an evolving science that requires knowledge of Big Data management and analytics. Big data in agriculture, manufacturing, and education are varied such as voluminous text, images, and graphs. Applying Big data science techniques (e.g., functional algorithms) for extracting intelligence data affords decision markers quick response to productivity, market resilience, and student enrollment challenges in today's unpredictable markets. This chapter serves to employ data science for potential solutions to Big Data applications in the sectors of agriculture, manufacturing and education to a lesser extent, using modern technological tools such as Hadoop, Hive, Sqoop, and MongoDB.

On the Dynamic Shifting of the MapReduce Timeout

2016 ◽  
pp. 1-22
Author(s):  
Bunjamin Memishi ◽  
Shadi Ibrahim ◽  
Maria S. Perez ◽  
Gabriel Antoniu
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Response Time ◽  
Case Studies ◽  
Large Scale ◽  
Failure Detection ◽  
Mapreduce Framework ◽  
Big Data Applications ◽  
Design Ideas

MapReduce has become a relevant framework for Big Data processing in the cloud. At large-scale clouds, failures do occur and may incur unwanted performance degradation to Big Data applications. As the reliability of MapReduce depends on how well they detect and handle failures, this book chapter investigates the problem of failure detection in the MapReduce framework. The case studies of this contribution reveal that the current static timeout value is not adequate and demonstrate significant variations in the application's response time with different timeout values. While arguing that comparatively little attention has been devoted to the failure detection in the framework, the chapter presents design ideas for a new adaptive timeout.

scholarly journals Analyzing Big Data with the Hybrid Interval Regression Methods

2014 ◽  
Vol 2014 ◽  
pp. 1-8 ◽  
Author(s):  
Chia-Hui Huang ◽  
Keng-Chieh Yang ◽  
Han-Ying Kao
Keyword(s):  
Support Vector Machine ◽  
Big Data ◽  
Information Technologies ◽  
Large Scale ◽  
Cloud Services ◽  
Support Vector ◽  
Large Scale Data ◽  
Big Data Applications ◽  
Smooth Support Vector Machine ◽  
Scale Data

Big data is a new trend at present, forcing the significant impacts on information technologies. In big data applications, one of the most concerned issues is dealing with large-scale data sets that often require computation resources provided by public cloud services. How to analyze big data efficiently becomes a big challenge. In this paper, we collaborate interval regression with the smooth support vector machine (SSVM) to analyze big data. Recently, the smooth support vector machine (SSVM) was proposed as an alternative of the standard SVM that has been proved more efficient than the traditional SVM in processing large-scale data. In addition the soft margin method is proposed to modify the excursion of separation margin and to be effective in the gray zone that the distribution of data becomes hard to be described and the separation margin between classes.

scholarly journals Emerging Databases for Next Generation Big Data Applications

2019 ◽  
Vol 9 (1S6) ◽  
pp. 182-189
Keyword(s):  
Big Data ◽  
Real Time ◽  
Mobile Application ◽  
Large Scale ◽  
Management Systems ◽  
Next Generation ◽  
Analytical Process ◽  
Big Data Applications ◽  
Different Types

This raised as quality into large scale periods into analytic application as real time inventor of pricing, mobile application in that offers we suggestion, fraud detections, risks as analyzed, etc. emphasis in the requirements with distributes knowledge’s management systems in which is into handle quickly transactions & analytics simultaneous. Efficiently into processes into transactions & analytically as requested, though, needed as complete into various optimization & branching into knowledge selection as a systems. In the paper presented in the wildfire systems in that target Hybrid Transactional & Analytical process (HTAP). This wildfire leverage in the Spark systems into modified as large scale process for different types as the difficult analytical request, & columnar process to modify as quick transaction & analytics simultaneous.

scholarly journals MapReduce Based Parallel Neural Networks in Enabling Large Scale Machine Learning

2015 ◽  
Vol 2015 ◽  
pp. 1-13 ◽  
Author(s):  
Yang Liu ◽  
Jie Yang ◽  
Yuan Huang ◽  
Lixiong Xu ◽  
Siguang Li ◽  
...  
Keyword(s):  
Neural Networks ◽  
Big Data ◽  
Large Scale ◽  
Training Data ◽  
Computer Cluster ◽  
Data Intensive ◽  
Big Data Applications ◽  
The Neural Network ◽  
Computation Process ◽  
Data Intensive Applications

Artificial neural networks (ANNs) have been widely used in pattern recognition and classification applications. However, ANNs are notably slow in computation especially when the size of data is large. Nowadays, big data has received a momentum from both industry and academia. To fulfill the potentials of ANNs for big data applications, the computation process must be speeded up. For this purpose, this paper parallelizes neural networks based on MapReduce, which has become a major computing model to facilitate data intensive applications. Three data intensive scenarios are considered in the parallelization process in terms of the volume of classification data, the size of the training data, and the number of neurons in the neural network. The performance of the parallelized neural networks is evaluated in an experimental MapReduce computer cluster from the aspects of accuracy in classification and efficiency in computation.

A Survey of Cloud-Based Services Leveraged by Big Data Applications

Web Services ◽  
2019 ◽  
pp. 1706-1716
Author(s):  
S. ZerAfshan Goher ◽  
Barkha Javed ◽  
Peter Bloodsworth
Keyword(s):  
Big Data ◽  
Data Storage ◽  
Data Analytics ◽  
Large Scale ◽  
Future Trends ◽  
Advantages And Disadvantages ◽  
Large Scale Data ◽  
Big Data Applications ◽  
Big Data Storage ◽  
Scale Data

Due to the growing interest in harnessing the hidden significance of data, more and more enterprises are moving to data analytics. Data analytics require the analysis and management of large-scale data to find the hidden patterns among various data components to gain useful insight. The derived information is then used to predict the future trends that can be advantageous for a business to flourish such as customers' likes/dislikes, reasons behind customers' churn and more. In this paper, several techniques for the big data analysis have been investigated along with their advantages and disadvantages. The significance of cloud computing for big data storage has also been discussed. Finally, the techniques to make the robust and efficient usage of big data have also been discussed.

Sign in / Sign up

Export Citation Format

Share Document