Fast algorithms using minimal data structures for common topological relationships in large, irregularly spaced topographic data sets

The publication of semantic web data, commonly represented in Resource Description Framework (RDF), has experienced outstanding growth over the last few years. Data from all fields of knowledge are shared publicly and interconnected in active initiatives such as Linked Open Data. However, despite the increasing availability of applications managing large-scale RDF information such as RDF stores and reasoning tools, little attention has been given to the structural features emerging in real-world RDF data. Our work addresses this issue by proposing specific metrics to characterise RDF data. We specifically focus on revealing the redundancy of each data set, as well as common structural patterns. We evaluate the proposed metrics on several data sets, which cover a wide range of designs and models. Our findings provide a basis for more efficient RDF data structures, indexes and compressors.

Download Full-text

Fast algorithms for nonparametric population modeling of large data sets

Automatica ◽

10.1016/j.automatica.2008.06.003 ◽

2009 ◽

Vol 45 (1) ◽

pp. 173-179 ◽

Cited By ~ 12

Author(s):

Gianluigi Pillonetto ◽

Giuseppe De Nicolao ◽

Marco Chierici ◽

Claudio Cobelli

Keyword(s):

Fast Algorithms ◽

Population Modeling ◽

Large Data ◽

Large Data Sets ◽

Data Sets

Download Full-text

Rapid multi-scale analysis of near-surface geophysical anomaly maps: Application to an archaeo-geophysical data set

Geophysics ◽

10.1190/geo2019-0564.1 ◽

2020 ◽

pp. 1-41 ◽

Cited By ~ 1

Author(s):

Jens Tronicke ◽

Niklas Allroggen ◽

Felix Biermann ◽

Florian Fanselow ◽

Julien Guillemoteau ◽

...

Keyword(s):

Random Noise ◽

Geophysical Data ◽

Magnetic Data ◽

Engineering Practice ◽

Data Sets ◽

Stripe Pattern ◽

Data Set ◽

Near Surface ◽

Multi Scale ◽

Topographic Data

In near-surface geophysics, ground-based mapping surveys are routinely employed in a variety of applications including those from archaeology, civil engineering, hydrology, and soil science. The resulting geophysical anomaly maps of, for example, magnetic or electrical parameters are usually interpreted to laterally delineate subsurface structures such as those related to the remains of past human activities, subsurface utilities and other installations, hydrological properties, or different soil types. To ease the interpretation of such data sets, we propose a multi-scale processing, analysis, and visualization strategy. Our approach relies on a discrete redundant wavelet transform (RWT) implemented using cubic-spline filters and the à trous algorithm, which allows to efficiently compute a multi-scale decomposition of 2D data using a series of 1D convolutions. The basic idea of the approach is presented using a synthetic test image, while our archaeo-geophysical case study from North-East Germany demonstrates its potential to analyze and process rather typical geophysical anomaly maps including magnetic and topographic data. Our vertical-gradient magnetic data show amplitude variations over several orders of magnitude, complex anomaly patterns at various spatial scales, and typical noise patterns, while our topographic data show a distinct hill structure superimposed by a microtopographic stripe pattern and random noise. Our results demonstrate that the RWT approach is capable to successfully separate these components and that selected wavelet planes can be scaled and combined so that the reconstructed images allow for a detailed, multi-scale structural interpretation also using integrated visualizations of magnetic and topographic data. Because our analysis approach is straightforward to implement without laborious parameter testing and tuning, computationally efficient, and easily adaptable to other geophysical data sets, we believe that it can help to rapidly analyze and interpret different geophysical mapping data collected to address a variety of near-surface applications from engineering practice and research.

Download Full-text

CoGAPS 3: Bayesian non-negative matrix factorization for single-cell analysis with asynchronous updates and sparse data structures

10.1101/699041 ◽

2019 ◽

Cited By ~ 3

Author(s):

Thomas D. Sherman ◽

Tiger Gao ◽

Elana J. Fertig

Keyword(s):

Single Cell ◽

Data Structures ◽

Computational Efficiency ◽

Matrix Factorization ◽

Single Cell Analysis ◽

Sparse Data ◽

Data Sets ◽

Cell Analysis ◽

Gradient Based ◽

Cell Data

AbstractMotivationBayesian factorization methods, including Coordinated Gene Activity in Pattern Sets (CoGAPS), are emerging as powerful analysis tools for single cell data. However, these methods have greater computational costs than their gradient-based counterparts. These costs are often prohibitive for analysis of large single-cell datasets. Many such methods can be run in parallel which enables this limitation to be overcome by running on more powerful hardware. However, the constraints imposed by the prior distributions in CoGAPS limit the applicability of parallelization methods to enhance computational efficiency for single-cell analysis.ResultsWe upgraded CoGAPS in Version 3 to overcome the computational limitations of Bayesian matrix factorization for single cell data analysis. This software includes a new parallelization framework that is designed around the sequential updating steps of the algorithm to enhance computational efficiency. These algorithmic advances were coupled with new software architecture and sparse data structures to reduce the memory overhead for single-cell data. Altogether, these updates to CoGAPS enhance the efficiency of the algorithm so that it can analyze 1000 times more cells, enabling factorization of large single-cell data sets.AvailabilityCoGAPS is available as a Bioconductor package and the source code is provided at github.com/FertigLab/CoGAPS. All efficiency updates to enable single-cell analysis available as of version [email protected]

Download Full-text

Generalized Odd Power Cauchy Family and Its Associated Heteroscedastic Regression Model

Statistics Optimization & Information Computing ◽

10.19139/soic-2310-5070-765 ◽

2021 ◽

Vol 9 (3) ◽

pp. 516-528

Author(s):

Emrah Altun EA ◽

Morad Alizadeh ◽

Thiago Ramires ◽

Edwin Ortega

Keyword(s):

Monte Carlo ◽

Regression Model ◽

Data Structures ◽

Generating Functions ◽

Shape Parameter ◽

Model Parameters ◽

Data Sets ◽

Complex Data ◽

Linear Representations ◽

Heteroscedastic Regression

This study introduces a generalization of the odd power Cauchy family by adding one more shape parameter togain more flexibility modeling the complex data structures. The linear representations for the density, moments, quantile,and generating functions are derived. The model parameters are estimated employing the maximum likelihood estimationmethod. The Monte Carlo simulations are performed under different parameter settings and sample sizes for the proposedmodels. In addition, we introduce a new heteroscedastic regression model based on the special member of the proposedfamily. Three data sets are analyzed with competitive and proposed models.

Download Full-text

PeaGlyph: Glyph design for investigation of balanced data structures

Information Visualization ◽

10.1177/14738716211050602 ◽

2021 ◽

pp. 147387162110506

Author(s):

Kenan Koc ◽

Andrew Stephen McGough ◽

Sara Johansson Fernstad

Keyword(s):

Data Structures ◽

Group Formation ◽

Multivariate Data ◽

Good Alternative ◽

Initial Study ◽

Data Sets ◽

The Novel ◽

Domain Expertise ◽

Shape Characteristics ◽

Initial Results

For many data analysis tasks, such as the formation of well-balanced groups for a fair race or collaboration in learning settings, the balancing between data attributes is at least as important as the actual values of items. At the same time, comparison of values is implicitly desired for these tasks. Even with statistical methods available to measure the level of balance, human judgment, and domain expertise plays an important role in judging the level of balance, and whether the level of unbalance is acceptable or not. Accordingly, there is a need for techniques that improve decision-making in the context of group formation that can be used as a visual complement to statistical analysis. This paper introduces a novel glyph-based visualization, PeaGlyph, which aims to support the understanding of balanced and unbalanced data structures, for instance by using a frequency format through countable marks and salient shape characteristics. The glyph was designed particularly for tasks of relevance for investigation of properties of balanced and unbalanced groups, such as looking-up and comparing values. Glyph-based visualization methods provide flexible and useful abstractions for exploring and analyzing multivariate data sets. The PeaGlyph design was based on an initial study that compared four glyph visualization methods in a joint study, including two base glyphs and their variations. The performance of the novel PeaGlyph was then compared to the best “performers” of the first study through evaluation. The initial results from the study are encouraging, and the proposed design may be a good alternative to the traditional glyphs for depicting multivariate data and allowing viewers to form an intuitive impression as to how balanced or unbalanced a set of objects are. Furthermore, a set of design considerations is discussed in context of the design of the glyphs.

Download Full-text

Fast Algorithms for Unsupervised Learning in Large Data Sets

Computer Science & Information Technology (CS & IT) ◽

10.5121/csit.2017.70207 ◽

2017 ◽

Cited By ~ 1

Author(s):

Syed Quddus

Keyword(s):

Unsupervised Learning ◽

Fast Algorithms ◽

Large Data ◽

Large Data Sets ◽

Data Sets

Download Full-text

Integrating Coral Restoration Data With a Novel Coral Sample Registry

Frontiers in Marine Science ◽

10.3389/fmars.2021.700172 ◽

2021 ◽

Vol 8 ◽

Author(s):

Amelia Moura ◽

Brian Beck ◽

Renee Duffey ◽

Lucas McEachron ◽

Margaret Miller ◽

...

Keyword(s):

Data Structures ◽

Current Data ◽

South Florida ◽

Coral Tissue ◽

Data Sets ◽

Accession Number ◽

Sample Collection ◽

Data Set ◽

Coral Sample ◽

And Performance

In the past decade, the field of coral reef restoration has experienced a proliferation of data detailing the source, genetics, and performance of coral strains used in research and restoration. Resource managers track the multitude of permits, species, restoration locations, and performance across multiple stakeholders while researchers generate large data sets and data pipelines detailing the genetic, genomic, and phenotypic variants of corals. Restoration practitioners, in turn, maintain records on fragment collection, genet performance, outplanting location and survivorship. While each data set is important in its own right, collectively they can provide deeper insights into coral biology and better guide coral restoration endeavors – unfortunately, current data sets are siloed with limited ability to cross-mine information for deeper insights and hypothesis testing. Herein we present the Coral Sample Registry (CSR), an online resource that establishes the first step in integrating diverse coral restoration data sets. Developed in collaboration with academia, management agencies, and restoration practitioners in the South Florida area, the CSR centralizes information on sample collection events by issuing a unique accession number to each entry. Accession numbers can then be incorporated into existing and future data structures. Each accession number is unique and corresponds to a specific collection event of coral tissue, whether for research, archiving, or restoration purposes. As such the accession number can serve as the key to unlock the diversity of information related to that sample’s provenance and characteristics across any and all data structures that include the accession number field. The CSR is open-source and freely available to users, designed to be suitable for all coral species in all geographic regions. Our goal is that this resource will be adopted by researchers, restoration practitioners, and managers to efficiently track coral samples through all data structures and thus enable the unlocking of a broader array of insights.

Download Full-text