scholarly journals Macroeconomic Nowcasting and Forecasting with Big Data

2018 ◽  
Vol 10 (1) ◽  
pp. 615-643 ◽  
Author(s):  
Brandyn Bok ◽  
Daniele Caratelli ◽  
Domenico Giannone ◽  
Argia M. Sbordone ◽  
Andrea Tambalotti
Keyword(s):  
New York ◽  
Big Data ◽  
Data Sets ◽  
Complex Data ◽  
Time Analysis ◽  
Macroeconomic Conditions ◽  
Real Time Analysis ◽  
Wide Range ◽  
Complex Data Sets ◽  
Innovative Techniques

Data, data, data…. Economists know their importance well, especially when it comes to monitoring macroeconomic conditions—the basis for making informed economic and policy decisions. Handling large and complex data sets was a challenge that macroeconomists engaged in real-time analysis faced long before so-called big data became pervasive in other disciplines. We review how methods for tracking economic conditions using big data have evolved over time and explain how econometric techniques have advanced to mimic and automate best practices of forecasters on trading desks, at central banks, and in other market-monitoring roles. We present in detail the methodology underlying the New York Fed Staff Nowcast, which employs these innovative techniques to produce early estimates of GDP growth, synthesizing a wide range of macroeconomic data as they become available.

scholarly journals Using Hadoop Technology to Overcome Big Data Problems by Choosing Proposed Cost-efficient Scheduler Algorithm for Heterogeneous Hadoop System (BD3)

2020 ◽  
pp. 58-84
Author(s):  
Abou_el_ela Abdou Hussein
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Storage ◽  
Database Management System ◽  
Data Sets ◽  
Complex Data ◽  
Daily Data ◽  
Complex Data Sets ◽  
Cost Efficient ◽  
Hadoop System

Day by day advanced web technologies have led to tremendous growth amount of daily data generated volumes. This mountain of huge and spread data sets leads to phenomenon that called big data which is a collection of massive, heterogeneous, unstructured, enormous and complex data sets. Big Data life cycle could be represented as, Collecting (capture), storing, distribute, manipulating, interpreting, analyzing, investigate and visualizing big data. Traditional techniques as Relational Database Management System (RDBMS) couldn’t handle big data because it has its own limitations, so Advancement in computing architecture is required to handle both the data storage requisites and the weighty processing needed to analyze huge volumes and variety of data economically. There are many technologies manipulating a big data, one of them is hadoop. Hadoop could be understand as an open source spread data processing that is one of the prominent and well known solutions to overcome handling big data problem. Apache Hadoop was based on Google File System and Map Reduce programming paradigm. Through this paper we dived to search for all big data characteristics starting from first three V's that have been extended during time through researches to be more than fifty six V's and making comparisons between researchers to reach to best representation and the precise clarification of all big data V’s characteristics. We highlight the challenges that face big data processing and how to overcome these challenges using Hadoop and its use in processing big data sets as a solution for resolving various problems in a distributed cloud based environment. This paper mainly focuses on different components of hadoop like Hive, Pig, and Hbase, etc. Also we institutes absolute description of Hadoop Pros and cons and improvements to face hadoop problems by choosing proposed Cost-efficient Scheduler Algorithm for heterogeneous Hadoop system.

Introduction to Big Data and Business Analytics

2022 ◽  
pp. 67-76
Author(s):  
Dineshkumar Bhagwandas Vaghela
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Data Sets ◽  
Complex Data ◽  
Business Analytics ◽  
Database Applications ◽  
Complex Data Sets ◽  
History Of ◽  
Rapid Generation

The term big data has come due to rapid generation of data in various organizations. In big data, the big is the buzzword. Here the data are so large and complex that the traditional database applications are not able to process (i.e., they are inadequate to deal with such volume of data). Usually the big data are described by 5Vs (volume, velocity, variety, variability, veracity). The big data can be structured, semi-structured, or unstructured. Big data analytics is the process to uncover hidden patterns, unknown correlations, predict the future values from large and complex data sets. In this chapter, the following topics will be covered more in detail. History of big data and business analytics, big data analytics technologies and tools, and big data analytics uses and challenges.

scholarly journals Application of Multivariate-Rank-Based Techniques in Clustering of Big Data

2018 ◽  
Vol 43 (4) ◽  
pp. 179-190
Author(s):  
Pritha Guha
Keyword(s):  
Big Data ◽  
Dna Analysis ◽  
Retail Banking ◽  
Data Sets ◽  
Complex Data ◽  
Data Handling ◽  
Credit Risk Management ◽  
Statistical Tools ◽  
Executive Summary ◽  
Complex Data Sets

Executive Summary Very large or complex data sets, which are difficult to process or analyse using traditional data handling techniques, are usually referred to as big data. The idea of big data is characterized by the three ‘v’s which are volume, velocity, and variety ( Liu, McGree, Ge, & Xie, 2015 ) referring respectively to the volume of data, the velocity at which the data are processed and the wide varieties in which big data are available. Every single day, different sectors such as credit risk management, healthcare, media, retail, retail banking, climate prediction, DNA analysis and, sports generate petabytes of data (1 petabyte = 250 bytes). Even basic handling of big data, therefore, poses significant challenges, one of them being organizing the data in such a way that it can give better insights into analysing and decision-making. With the explosion of data in our life, it has become very important to use statistical tools to analyse them.

scholarly journals Using visual analytics to make sense of railway Close Calls

2016 ◽  
Vol 231 (10) ◽  
pp. 1107-1114
Author(s):  
Miguel Figueres-Esteban ◽  
Peter Hughes ◽  
Coen van Gulijk
Keyword(s):  
Big Data ◽  
Data Analysis ◽  
Visual Analytics ◽  
Near Miss ◽  
Test Case ◽  
Data Sets ◽  
Complex Data ◽  
Network Text Analysis ◽  
Complex Data Sets ◽  
Close Call

In the big data era, large and complex data sets will exceed scientists’ capacity to make sense of them in the traditional way. New approaches in data analysis, supported by computer science, will be necessary to address the problems that emerge with the rise of big data. The analysis of the Close Call database, which is a text-based database for near-miss reporting on the GB railways, provides a test case. The traditional analysis of Close Calls is time consuming and prone to differences in interpretation. This paper investigates the use of visual analytics techniques, based on network text analysis, to conduct data analysis and extract safety knowledge from 500 randomly selected Close Call records relating to worker slips, trips and falls. The results demonstrate a straightforward, yet effective, way to identify hazardous conditions without having to read each report individually. This opens up new ways to perform data analysis in safety science.

Facies classification using machine learning

2016 ◽  
Vol 35 (10) ◽  
pp. 906-909 ◽  
Author(s):  
Brendon Hall
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Open Source ◽  
Data Sets ◽  
Complex Data ◽  
Facies Classification ◽  
Large Complex ◽  
Complex Data Sets ◽  
Software Platforms ◽  
Tools And Techniques

There has been much excitement recently about big data and the dire need for data scientists who possess the ability to extract meaning from it. Geoscientists, meanwhile, have been doing science with voluminous data for years, without needing to brag about how big it is. But now that large, complex data sets are widely available, there has been a proliferation of tools and techniques for analyzing them. Many free and open-source packages now exist that provide powerful additions to the geoscientist's toolbox, much of which used to be only available in proprietary (and expensive) software platforms.

scholarly journals Use of Big-Data Analytics with the Interactive Advertisement for Product/Service Representation towards its Customers

2019 ◽  
Vol 4 (2) ◽  
Author(s):  
HarshmitKaur Saluja ◽  
Vinod Kumar Yadav ◽  
K.M. Mohapatra
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Service Performance ◽  
Data Sets ◽  
Complex Data ◽  
Product Service ◽  
Literature Reviews ◽  
Complex Data Sets ◽  
The One

On the one hand, big-data analytics has brought revolution in the predictive modeler by enabling the complex data sets getting structured. On the other hand, the interactive advertisement has changed the complete scenario of the advertising sector by making advertisements content structured in such a way that it is customer-centric. The paper helps to widen the view to explore the growing urge of customization technique in advertising sector with interactive enablers. The paper further examines that how interactive advertisement and big-data has helped to represent product/service from the view of a customer and also improved the product/service performance. In order of study, exhaustive literature reviews resulting in three hypothesis are developed to take on the above-mentioned concerns.

scholarly journals From Trees to Barcodes and Back Again: Theoretical and Statistical Perspectives

Algorithms ◽  
2020 ◽  
Vol 13 (12) ◽  
pp. 335
Author(s):  
Lida Kanari ◽  
Adélie Garin ◽  
Kathryn Hess
Keyword(s):  
Symmetric Groups ◽  
Topological Data Analysis ◽  
Topological Descriptors ◽  
Data Sets ◽  
Complex Data ◽  
Wide Range ◽  
Persistence Diagram ◽  
Complex Data Sets ◽  
Concrete Language ◽  
Persistence Diagrams

Methods of topological data analysis have been successfully applied in a wide range of fields to provide useful summaries of the structure of complex data sets in terms of topological descriptors, such as persistence diagrams. While there are many powerful techniques for computing topological descriptors, the inverse problem, i.e., recovering the input data from topological descriptors, has proved to be challenging. In this article, we study in detail the Topological Morphology Descriptor (TMD), which assigns a persistence diagram to any tree embedded in Euclidean space, and a sort of stochastic inverse to the TMD, the Topological Neuron Synthesis (TNS) algorithm, gaining both theoretical and computational insights into the relation between the two. We propose a new approach to classify barcodes using symmetric groups, which provides a concrete language to formulate our results. We investigate to what extent the TNS recovers a geometric tree from its TMD and describe the effect of different types of noise on the process of tree generation from persistence diagrams. We prove moreover that the TNS algorithm is stable with respect to specific types of noise.

Introduction to Big Data and Business Analytics

2019 ◽  
pp. 279-288
Author(s):  
Dineshkumar Bhagwandas Vaghela
Keyword(s):  
Big Data ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Data Sets ◽  
Complex Data ◽  
Business Analytics ◽  
Database Applications ◽  
Complex Data Sets ◽  
History Of ◽  
Rapid Generation

The term big data has come due to rapid generation of data in various organizations. In big data, the big is the buzzword. Here the data are so large and complex that the traditional database applications are not able to process (i.e., they are inadequate to deal with such volume of data). Usually the big data are described by 5Vs (volume, velocity, variety, variability, veracity). The big data can be structured, semi-structured, or unstructured. Big data analytics is the process to uncover hidden patterns, unknown correlations, predict the future values from large and complex data sets. In this chapter, the following topics will be covered more in detail. History of big data and business analytics, big data analytics technologies and tools, and big data analytics uses and challenges.

scholarly journals Extracting meaning from biological imaging data

2014 ◽  
Vol 25 (22) ◽  
pp. 3470-3473 ◽  
Author(s):  
Andrew R. Cohen
Keyword(s):  
Image Data ◽  
Biological Imaging ◽  
Data Sets ◽  
Complex Data ◽  
Imaging Data ◽  
Specific Knowledge ◽  
Wide Range ◽  
Complex Data Sets ◽  
Working Together ◽  
Application Specific

Biological imaging continues to improve, capturing continually longer-term, richer, and more complex data, penetrating deeper into live tissue. How do we gain insight into the dynamic processes of disease and development from terabytes of multidimensional image data? Here I describe a collaborative approach to extracting meaning from biological imaging data. The collaboration consists of teams of biologists and engineers working together. Custom computational tools are built to best exploit application-specific knowledge in order to visualize and analyze large and complex data sets. The image data are summarized, extracting and modeling the features that capture the objects and relationships in the data. The summarization is validated, the results visualized, and errors corrected as needed. Finally, the customized analysis and visualization tools together with the image data and the summarization results are shared. This Perspective provides a brief guide to the mathematical ideas that rigorously quantify the notion of extracting meaning from biological image, and to the practical approaches that have been used to apply these ideas to a wide range of applications in cell and tissue optical imaging.

Facial Nerve EMG: Low-Tech Monitoring with a Stopwatch

2021 ◽  
Author(s):  
Julian Prell ◽  
Christian Scheller ◽  
Sebastian Simmermacher ◽  
Christian Strauss ◽  
Stefan Rampp
Keyword(s):  
Facial Nerve ◽  
Real Time ◽  
Random Order ◽  
Automated Analysis ◽  
Data Sets ◽  
Time Analysis ◽  
Frequency Pattern ◽  
Real Time Analysis ◽  
Offline Analysis ◽  
Automated Algorithms

Abstract Objective The quantity of A-trains, a high-frequency pattern of free-running facial nerve electromyography, is correlated with the risk for postoperative high-grade facial nerve paresis. This correlation has been confirmed by automated analysis with dedicated algorithms and by visual offline analysis but not by audiovisual real-time analysis. Methods An investigator was presented with 29 complete data sets measured during actual surgeries in real time and without breaks in a random order. Data were presented either strictly via loudspeaker (audio) or simultaneously by loudspeaker and computer screen (audiovisual). Visible and/or audible A-train activity was then quantified by the investigator with the computerized equivalent of a stopwatch. The same data were also analyzed with quantification of A-trains by automated algorithms. Results Automated (auto) traintime (TT), known to be a small, yet highly representative fraction of overall A-train activity, ranged from 0.01 to 10.86 s (median: 0.58 s). In contrast, audio-TT ranged from 0 to 1,357.44 s (median: 29.69 s), and audiovisual-TT ranged from 0 to 786.57 s (median: 46.19 s). All three modalities were correlated to each other in a highly significant way. Likewise, all three modalities correlated significantly with the extent of postoperative facial paresis. As a rule of thumb, patients with visible/audible A-train activity < 1 minute presented with a more favorable clinical outcome than patients with > 1 minute of A-train activity. Conclusion Detection and even quantification of A-trains is technically possible not only with intraoperative automated real-time calculation or postoperative visual offline analysis, but also with very basic monitoring equipment and real-time good quality audiovisual analysis. However, the investigator found audiovisual real-time-analysis to be very demanding; thus tools for automated quantification can be very helpful in this respect.

Sign in / Sign up

Export Citation Format

Share Document