METHODS FOR COMBINING NEUMANN–PEARSON STATISTICS AND BAYESIAN STATISTICS IN OBSERVATIONAL ASTROPHYSICS (REVIEW)

2021 ◽  
Vol 1 (1) ◽  
pp. 28-59
Author(s):  
V. L. Gorokhov ◽  
◽  
Yu. V. Baryshev ◽  
Pekka Teerikorpi ◽  
V. V. Vitkovsky ◽  
...  
Keyword(s):  
Big Data ◽  
Complex Systems ◽  
Bayesian Statistics ◽  
Temporal Structure ◽  
Multidimensional Data ◽  
Astronomical Observations ◽  
Emergency Situations ◽  
Bayesian Integration ◽  
Wide Range

The article offers an overview and methodology for combining Neumann–Pearson statistics and Bayesian statistics with integrated visualization of cognitive images for processing multidimensional data of astronomical observations. These methods are very successfully applied in astrophysics and can be used for a wide range of problems in BIG DATA. The technique of such a combination can be oriented towards identifying and forecasting emergency situations in complex systems. In the proposed approach, Bayesian integration and visualization of cognitive images is based on the statistical capabilities of algorithms and programs to identify and objectify in cognitive probabilistic images signs of differences in the spatial or temporal structure of objects of observation.

ExpanDrogram: Dynamic Visualization of Big Data Segmentation over Time

2021 ◽  
Vol 13 (2) ◽  
pp. 1-27
Author(s):  
A. Khalemsky ◽  
R. Gelbard
Keyword(s):  
Big Data ◽  
Visual Presentation ◽  
Trend Detection ◽  
Dynamic Visualization ◽  
Segmentation Process ◽  
Key Points ◽  
Wide Range ◽  
History Of ◽  
The Individual ◽  
Over Time

In dynamic and big data environments the visualization of a segmentation process over time often does not enable the user to simultaneously track entire pieces. The key points are sometimes incomparable, and the user is limited to a static visual presentation of a certain point. The proposed visualization concept, called ExpanDrogram, is designed to support dynamic classifiers that run in a big data environment subject to changes in data characteristics. It offers a wide range of features that seek to maximize the customization of a segmentation problem. The main goal of the ExpanDrogram visualization is to improve comprehensiveness by combining both the individual and segment levels, illustrating the dynamics of the segmentation process over time, providing “version control” that enables the user to observe the history of changes, and more. The method is illustrated using different datasets, with which we demonstrate multiple segmentation parameters, as well as multiple display layers, to highlight points such as new trend detection, outlier detection, tracking changes in original segments, and zoom in/out for more/less detail. The datasets vary in size from a small one to one of more than 12 million records.

RELATIONSHIP IN IOT AND BIG DATA ANALYTICS: THE OPPORTUNITIES AND OPEN CHALLENGES

2021 ◽  
Vol 83 (4) ◽  
pp. 100-111
Author(s):  
Ahmad Anwar Zainuddin ◽  
Keyword(s):  
Big Data ◽  
Internet Of Things ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Physical World ◽  
Future Directions ◽  
Wide Range ◽  
Time Utilization ◽  
Relationship Of ◽  
The Internet Of Things

Internet of Things (IoT) is an up-and-coming technology that has a wide variety of applications. It empowers physical objects to be organized in a specialized framework to grow its convenience in terms of ease and time utilization. It is to convert the thought of bridging the crevice between the physical world and the machine world. It is also being use in the wide range of the technology in this current situation. One of its applications is to monitor and store data over time from numerous devices allows for easy analysis of the dataset. This analysis can then be the basis of decisions made on the same. In this study, the concept, architecture, and relationship of IoT and Big Data are described. Next, several use cases in IoT and big data in the research methodology are studied. The opportunities and open challenges which including the future directions are described. Furthermore, by proposing a new architecture for big data analytics in the Internet of Things, this paper adds value. Overall, the various types of big IoT data analytics, their methods, and associated big data mining technologies are discussed.

Big Data Ethics and Politics: Toward New Understandings

2018 ◽  
Vol 38 (1) ◽  
pp. 3-9 ◽  
Author(s):  
Wenhong Chen ◽  
Anabel Quan-Haase
Keyword(s):  
Big Data ◽  
Mobile Apps ◽  
Data Ethics ◽  
The Social ◽  
Wide Range ◽  
Health Institutions ◽  
Methodological Approaches ◽  
Privacy Breaches ◽  
The Many ◽  
Ethics And Politics

The hype around big data does not seem to abate nor do the scandals. Privacy breaches in the collection, use, and sharing of big data have affected all the major tech players, be it Facebook, Google, Apple, or Uber, and go beyond the corporate world including governments, municipalities, and educational and health institutions. What has come to light is that enabled by the rapid growth of social media and mobile apps, various stakeholders collect and use large amounts of data, disregarding the ethics and politics. As big data touch on many realms of daily life and have profound impacts in the social world, the scrutiny around big data practice becomes increasingly relevant. This special issue investigates the ethics and politics of big data using a wide range of theoretical and methodological approaches. Together, the articles provide new understandings of the many dimensions of big data ethics and politics, showing it is important to understand and increase awareness of the biases and limitations inherent in big data analysis and practices.

scholarly journals Rainfall spatio-temporal correlation and intermittency structure from micro-γ to meso-β scale in the Netherlands

2021 ◽  
Author(s):  
Thomas C. van Leth ◽  
Hidde Leijnse ◽  
Aart Overeem ◽  
Remko Uijlenhoet
Keyword(s):  
The Netherlands ◽  
Spatial Scales ◽  
Rainfall Variability ◽  
Temporal Structure ◽  
Temporal Correlation ◽  
Exponential Model ◽  
Rain Gauges ◽  
Wide Range ◽  
Spatio Temporal ◽  
Two Parameter

AbstractWe investigate the spatio-temporal structure of rainfall at spatial scales from 7m to over 200 km in the Netherlands. We used data from two networks of laser disdrometers with complementary interstation distances in two Dutch cities (comprising five and six disdrometers, respectively) and a Dutch nationwide network of 31 automatic rain gauges. The smallest aggregation interval for which raindrop size distributions were collected by the disdrometers was 30 s, while the automatic rain gauges provided 10-min rainfall sums. This study aims to supplement other micro-γ investigations (usually performed in the context of spatial rainfall variability within a weather radar pixel) with new data, while characterizing the correlation structure across an extended range of scales. To quantify the spatio-temporal variability, we employ a two-parameter exponential model fitted to the spatial correlograms and characterize the parameters of the model as a function of the temporal aggregation interval. This widely used method allows for a meaningful comparison with seven other studies across contrasting climatic settings all around the world. We also separately analyzed the intermittency of the rainfall observations. We show that a single parameterization, consisting of a two-parameter exponential spatial model as a function of interstation distance combined with a power-law model for decorrelation distance as a function of aggregation interval, can coherently describe rainfall variability (both spatial correlation and intermittency) across a wide range of scales. Limiting the range of scales to those typically found in micro-γ variability studies (including four of the seven studies to which we compare our results) skews the parameterization and reduces its applicability to larger scales.

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 1.91% to 6.69%. <div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals BIG DATA FOR RISK ANALYSIS: THE FUTURE OF SAFE RAILWAYS

2016 ◽  
Author(s):  
Miguel Figueres Esteban
Keyword(s):  
Big Data ◽  
Risk Analysis ◽  
Safety Management ◽  
New Technology ◽  
Data Sources ◽  
Transport Systems ◽  
Text Documents ◽  
Railway Systems ◽  
Wide Range ◽  
Three Components

New technology brings ever more data to support decision-making for intelligent transport systems. Big Data is no longer a futuristic challenge, it is happening right now: modern railway systems have countless sources of data providing a massive quantity of diverse information on every aspect of operations such as train position and speed, brake applications, passenger numbers, status of the signaling system or reported incidents.The traditional approaches to safety management on the railways have relied on static data sources to populate traditional safety tools such as bow-tie models and fault trees. The Big Data Risk Analysis (BDRA) program for Railways at the University of Huddersfield is investigating how the many Big Data sources from the railway can be combined in a meaningful way to provide a better understanding about the GB railway systems and the environment within which they operate.Moving to BDRA is not simply a matter of scaling-up existing analysis techniques. BDRA has to coordinate and combine a wide range of sources with different types of data and accuracy, and that is not straight-forward. BDRA is structured around three components: data, ontology and visualisation. Each of these components is critical to support the overall framework. This paper describes how these three components are used to get safety knowledge from two data sources by means of ontologies from text documents. This is a part of the ongoing BDRA research that is looking at integrating many large and varied data sources to support railway safety and decision-makers.DOI: http://dx.doi.org/10.4995/CIT2016.2016.1825

scholarly journals DICTION as a Tool for Studying the Mass Media

2014 ◽  
pp. 462-476 ◽  
Author(s):  
R. Lance Holbert
Keyword(s):  
Big Data ◽  
Mass Media ◽  
Practical Knowledge ◽  
Media Content ◽  
Diverse Range ◽  
Systematic Assessment ◽  
Message Tailoring ◽  
Wide Range ◽  
The Media ◽  
The Mass Media

This chapter offers a systematic assessment of DICTION’s ability to address a wide range of media content. Each of the media-related works in this volume reflects a unique mix of communication inputs, and DICTION proves itself able to generate valid and reliable insights on a diverse range of material. In addition, the chapter focuses on a series of challenges (e.g., Message Tailoring, Hypertext, Interactivity) and opportunities (e.g., big data) for DICTION in relation to the study of media content. The program and the researchers who utilize it need to continue to evolve with the changing media landscape in order to generate practical knowledge that is relevant to improving communication.

Big Data Models and the Public Sector

Web Services ◽  
2019 ◽  
pp. 105-126
Author(s):  
N. Nawin Sona
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Big Data ◽  
Public Sector ◽  
Predictive Analytics ◽  
Data Types ◽  
The Public ◽  
Emerging Trends ◽  
Wide Range ◽  
Learning Data

This chapter aims to give an overview of the wide range of Big Data approaches and technologies today. The data features of Volume, Velocity, and Variety are examined against new database technologies. It explores the complexity of data types, methodologies of storage, access and computation, current and emerging trends of data analysis, and methods of extracting value from data. It aims to address the need for clarity regarding the future of RDBMS and the newer systems. And it highlights the methods in which Actionable Insights can be built into public sector domains, such as Machine Learning, Data Mining, Predictive Analytics and others.

Big Data, Artificial Intelligence, and the Internet of Things in Cross-Border E-Commerce

2021 ◽  
pp. 257-272
Author(s):  
Aboobucker Ilmudeen
Keyword(s):  
Artificial Intelligence ◽  
Big Data ◽  
Internet Of Things ◽  
Economic Globalization ◽  
The Internet ◽  
Technological Revolution ◽  
Cross Border ◽  
Wide Range ◽  
The Internet Of Things

Today, the terms big data, artificial intelligence, and internet of things (IoT) are many-fold as these are linked with various applications, technologies, eco-systems, and services in the business domain. The recent industrial and technological revolution have become popular ever before, and the cross-border e-commerce activities are emerging very rapidly. As a result, it supports to the growth of economic globalization that has strategic importance for the advancement of e-commerce activities across the globe. In the business industry, the wide range applications of technologies like big data, artificial intelligence, and internet of things in cross-border e-commerce have grown exponential. This chapter systematically reviews the role of big data, artificial intelligence, and IoT in cross-border e-commerce and proposes a conceptually-designed smart-integrated cross-border e-commerce platform.

Sign in / Sign up

Export Citation Format

Share Document