Suffix Tree Constructing Algorithm for Datasets with Discrete Contents

2015 ◽  
Vol 733 ◽  
pp. 867-870
Author(s):  
Zhen Zhong Jin ◽  
Zheng Huang ◽  
Hua Zhang
Keyword(s):  
Data Structure ◽  
Sensor Network ◽  
Association Analysis ◽  
Data Structures ◽  
Suffix Tree ◽  
Analysis Data ◽  
Large Datasets ◽  
Intermediate Data ◽  
Input Strings

The suffix tree is a useful data structure constructed for indexing strings. However, when it comes to large datasets of discrete contents, most existing algorithms become very inefficient. Discrete datasets are need to be indexed in many fields like record analysis, data analyze in sensor network, association analysis etc. This paper presents an algorithm, STD, which stands for Suffix Tree for Discrete contents, that performs very efficiently with discrete input datasets. It imports several wonderful intermediate data structures for discrete strings; we also take care of the situation that the discrete input strings have similar characteristics. Moreover, STD keeps the advantages of existing implementations which are for successive input strings. Experiments were taken to evaluate the performance and shown that the method works well.

scholarly journals Reflections on the Intermediate Data Structure (IDS)

2021 ◽  
Vol 10 ◽  
pp. 71-75
Author(s):  
George Alter
Keyword(s):  
Data Structure ◽  
Life Course ◽  
Data Structures ◽  
Relational Database ◽  
Life Histories ◽  
Social Scientists ◽  
Database Model ◽  
Intermediate Data ◽  
Value Model ◽  
Historical Life

The Intermediate Data Structure (IDS) encourages sharing historical life course data by storing data in a common format. To encompass the complexity of life histories, IDS relies on data structures that are unfamiliar to most social scientists. This article examines four features of IDS that make it flexible and expandable: the Entity-Attribute-Value model, the relational database model, embedded metadata, and the Chronicle file. I also consider IDS from the perspective of current discussions about sharing data across scientific domains. We can find parallels to IDS in other fields that may lead to future innovations.

scholarly journals There and Back Again

2005 ◽  
Vol 12 (3) ◽  
Author(s):  
Olivier Danvy ◽  
Mayer Goldberg
Keyword(s):  
Dynamic Programming ◽  
Data Structure ◽  
Data Structures ◽  
Recursive Function ◽  
Return Time ◽  
Catalan Numbers ◽  
Intermediate Data ◽  
Programming Patterns ◽  
Tree Traversals

We present a programming pattern where a recursive function defined over a data structure traverses another data structure at return time. The idea is that the recursive calls get us `there' by traversing the first data structure and the returns get us `back again' while traversing the second data structure. We name this programming pattern of traversing a data structure at call time and another data structure at return time ``There And Back Again'' (TABA).<br /> <br />The TABA pattern directly applies to computing symbolic convolutions and to multiplying polynomials. It also blends well with other programming patterns such as dynamic programming and traversing a list at double speed. We illustrate TABA and dynamic programming with Catalan numbers. We illustrate TABA and traversing a list at double speed with palindromes and we obtain a novel solution to this traditional exercise. Finally, through a variety of tree traversals, we show how to apply TABA to other data structures than lists.<br /> <br />A TABA-based function written in direct style makes full use of an ALGOL-like control stack and needs no heap allocation. Conversely, in a TABA-based function written in continuation-passing style and recursively defined over a data structure (traversed at call time), the continuation acts as an iterator over a second data structure (traversed at return time). In general, the TABA pattern saves one from accumulating intermediate data structures at call time.

PARALLEL CREATION OF LINEAR OCTREES FROM QUADTREE SLICES

1994 ◽  
Vol 04 (04) ◽  
pp. 447-453
Author(s):  
L.K. SWIFT ◽  
T. JOHNSON ◽  
P.E. LIVADAS
Keyword(s):  
Data Structure ◽  
Parallel Algorithm ◽  
Data Structures ◽  
Linear Form ◽  
Image Data ◽  
Three Dimensions ◽  
Two Dimensional ◽  
Hierarchical Data ◽  
Intermediate Data ◽  
Linear Octree

Quadtrees and octrees are hierarchical data structures for efficiently storing image data. Quadtrees represent two dimensional images, while octrees are a generalization to three dimensions. The linear form of each is an abstraction of the tree structure to reduce storage requirements. We have developed a parallel algorithm to efficiently create a linear octree from quadtree slices of an object without the use of an intermediate data structure. We also propose the d-slice, which is a generalization of an octree, and which efficiently represents non-cubic volumes.

scholarly journals A Comparative Study about Data Structures Used for Efficient Management of Voxelised Full-Waveform Airborne LiDAR Data during 3D Polygonal Model Creation

2021 ◽  
Vol 13 (4) ◽  
pp. 559
Author(s):  
Milto Miltiadou ◽  
Neill D. F. Campbell ◽  
Darren Cosker ◽  
Michael G. Grant
Keyword(s):  
Data Structure ◽  
Data Structures ◽  
Airborne Lidar ◽  
Memory Allocation ◽  
Lidar Data ◽  
Full Waveform ◽  
Waveform Lidar ◽  
Airborne Lidar Data ◽  
Full Waveform Lidar ◽  
Polygonal Model

In this paper, we investigate the performance of six data structures for managing voxelised full-waveform airborne LiDAR data during 3D polygonal model creation. While full-waveform LiDAR data has been available for over a decade, extraction of peak points is the most widely used approach of interpreting them. The increased information stored within the waveform data makes interpretation and handling difficult. It is, therefore, important to research which data structures are more appropriate for storing and interpreting the data. In this paper, we investigate the performance of six data structures while voxelising and interpreting full-waveform LiDAR data for 3D polygonal model creation. The data structures are tested in terms of time efficiency and memory consumption during run-time and are the following: (1) 1D-Array that guarantees coherent memory allocation, (2) Voxel Hashing, which uses a hash table for storing the intensity values (3) Octree (4) Integral Volumes that allows finding the sum of any cuboid area in constant time, (5) Octree Max/Min, which is an upgraded octree and (6) Integral Octree, which is proposed here and it is an attempt to combine the benefits of octrees and Integral Volumes. In this paper, it is shown that Integral Volumes is the more time efficient data structure but it requires the most memory allocation. Furthermore, 1D-Array and Integral Volumes require the allocation of coherent space in memory including the empty voxels, while Voxel Hashing and the octree related data structures do not require to allocate memory for empty voxels. These data structures, therefore, and as shown in the test conducted, allocate less memory. To sum up, there is a need to investigate how the LiDAR data are stored in memory. Each tested data structure has different benefits and downsides; therefore, each application should be examined individually.

scholarly journals Shape Neutral Analysis of Graph-based Data-structures

2018 ◽  
Vol 18 (3-4) ◽  
pp. 470-483 ◽  
Author(s):  
GREGORY J. DUCK ◽  
JOXAN JAFFAR ◽  
ROLAND H. C. YAP
Keyword(s):  
Data Structure ◽  
Data Structures ◽  
Program Analysis ◽  
Constraint Handling ◽  
C Programs ◽  
Wide Range ◽  
Target Program ◽  
Target Data ◽  
Structure Graph ◽  
Structure Properties

AbstractMalformed data-structures can lead to runtime errors such as arbitrary memory access or corruption. Despite this, reasoning over data-structure properties for low-level heap manipulating programs remains challenging. In this paper we present a constraint-based program analysis that checks data-structure integrity, w.r.t. given target data-structure properties, as the heap is manipulated by the program. Our approach is to automatically generate a solver for properties using the type definitions from the target program. The generated solver is implemented using a Constraint Handling Rules (CHR) extension of built-in heap, integer and equality solvers. A key property of our program analysis is that the target data-structure properties are shape neutral, i.e., the analysis does not check for properties relating to a given data-structure graph shape, such as doubly-linked-lists versus trees. Nevertheless, the analysis can detect errors in a wide range of data-structure manipulating programs, including those that use lists, trees, DAGs, graphs, etc. We present an implementation that uses the Satisfiability Modulo Constraint Handling Rules (SMCHR) system. Experimental results show that our approach works well for real-world C programs.

Integrating Image Computation in Undergraduate Level Data-Structure Education

1998 ◽  
Vol 12 (08) ◽  
pp. 1071-1080 ◽  
Author(s):  
Sudeep Sarkar ◽  
Dmitry Goldgof
Keyword(s):  
Image Processing ◽  
Image Analysis ◽  
Data Structure ◽  
Data Structures ◽  
Science Curriculum ◽  
Early Stage ◽  
Real Life ◽  
South Florida ◽  
The University ◽  
Structure Education

There is a growing need for expertise both in image analysis and in software engineering. To date, these two areas have been taught separately in an undergraduate computer and information science curriculum. However, we have found that introduction to image analysis can be easily integrated in data-structure courses without detracting from the original goal of teaching data structures. Some of the image processing tasks offer a natural way to introduce basic data structures such as arrays, queues, stacks, trees and hash tables. Not only does this integrated strategy expose the students to image related manipulations at an early stage of the curriculum but it also imparts cohesiveness to the data-structure assignments and brings them closer to real life. In this paper we present a set of programming assignments that integrates undergraduate data-structure education with image processing tasks. These assignments can be incorporated in existing data-structure courses with low time and software overheads. We have used these assignment sets thrice: once in a 10-week duration data-structure course at the University of California, Santa Barbara and the other two times in 15-week duration courses at the University of South Florida, Tampa.

scholarly journals DenseZDD: A Compact and Fast Index for Families of Sets

Algorithms ◽  
2018 ◽  
Vol 11 (8) ◽  
pp. 128 ◽  
Author(s):  
Shuhei Denzumi ◽  
Jun Kawahara ◽  
Koji Tsuda ◽  
Hiroki Arimura ◽  
Shin-ichi Minato ◽  
...  
Keyword(s):  
Data Structure ◽  
Data Structures ◽  
Information Integration ◽  
Decision Diagrams ◽  
Web Information Retrieval ◽  
Binary Decision ◽  
Web Information ◽  
Set Operations ◽  
Succinct Data Structure ◽  
The Family

In this article, we propose a succinct data structure of zero-suppressed binary decision diagrams (ZDDs). A ZDD represents sets of combinations efficiently and we can perform various set operations on the ZDD without explicitly extracting combinations. Thanks to these features, ZDDs have been applied to web information retrieval, information integration, and data mining. However, to support rich manipulation of sets of combinations and update ZDDs in the future, ZDDs need too much space, which means that there is still room to be compressed. The paper introduces a new succinct data structure, called DenseZDD, for further compressing a ZDD when we do not need to conduct set operations on the ZDD but want to examine whether a given set is included in the family represented by the ZDD, and count the number of elements in the family. We also propose a hybrid method, which combines DenseZDDs with ordinary ZDDs. By numerical experiments, we show that the sizes of our data structures are three times smaller than those of ordinary ZDDs, and membership operations and random sampling on DenseZDDs are about ten times and three times faster than those on ordinary ZDDs for some datasets, respectively.

scholarly journals Spaced Seed Data Structures forDe NovoAssembly

2015 ◽  
Vol 2015 ◽  
pp. 1-8 ◽  
Author(s):  
Inanç Birol ◽  
Justin Chu ◽  
Hamid Mohamadi ◽  
Shaun D. Jackman ◽  
Karthika Raghavan ◽  
...  
Keyword(s):  
Data Structure ◽  
Data Structures ◽  
De Novo ◽  
Bloom Filters ◽  
De Bruijn Graph ◽  
Sequence Specificity ◽  
Sequencing Errors ◽  
Spaced Seeds ◽  
Read Error Correction ◽  
Seed Data

De novoassembly of the genome of a species is essential in the absence of a reference genome sequence. Many scalable assembly algorithms use the de Bruijn graph (DBG) paradigm to reconstruct genomes, where a table of subsequences of a certain length is derived from the reads, and their overlaps are analyzed to assemble sequences. Despite longer subsequences unlocking longer genomic features for assembly, associated increase in compute resources limits the practicability of DBG over other assembly archetypes already designed for longer reads. Here, we revisit the DBG paradigm to adapt it to the changing sequencing technology landscape and introduce three data structure designs for spaced seeds in the form of paired subsequences. These data structures address memory and run time constraints imposed by longer reads. We observe that when a fixed distance separates seed pairs, it provides increased sequence specificity with increased gap length. Further, we note that Bloom filters would be suitable to implicitly store spaced seeds and be tolerant to sequencing errors. Building on this concept, we describe a data structure for tracking the frequencies of observed spaced seeds. These data structure designs will have applications in genome, transcriptome and metagenome assemblies, and read error correction.

scholarly journals Historical Databases Now and in the Future

2021 ◽  
Vol 10 ◽  
pp. 9-12
Author(s):  
Kris Inwood ◽  
Hamish Maxwell-Stewart
Keyword(s):  
Data Structure ◽  
The Netherlands ◽  
Longitudinal Data ◽  
Practical Implementation ◽  
High Quality ◽  
Rule Based ◽  
Intermediate Data ◽  
The Future ◽  
Historical Databases ◽  
Historical Sample

Kees Mandemakers has enriched historical databases in the Netherlands and internationally through the development of the Historical Sample of the Netherlands, the Intermediate Data Structure, a practical implementation of rule-based record linking (LINKS) and personal encouragement of high quality longitudinal data in a number of countries.

Sign in / Sign up

Export Citation Format

Share Document