Resonator Networks, 1: An Efficient Solution for Factoring High-Dimensional, Distributed Representations of Data Structures

2020 ◽  
Vol 32 (12) ◽  
pp. 2311-2331
Author(s):  
E. Paxon Frady ◽  
Spencer J. Kent ◽  
Bruno A. Olshausen ◽  
Friedrich T. Sommer
Keyword(s):  
Data Structure ◽  
Data Structures ◽  
Efficient Solution ◽  
High Dimensional ◽  
Search Problem ◽  
Combinatorial Search ◽  
Distributed Representations ◽  
Factorization Problem ◽  
Analysis And Evaluation ◽  
New Type

The ability to encode and manipulate data structures with distributed neural representations could qualitatively enhance the capabilities of traditional neural networks by supporting rule-based symbolic reasoning, a central property of cognition. Here we show how this may be accomplished within the framework of Vector Symbolic Architectures (VSAs) (Plate, 1991 ; Gayler, 1998 ; Kanerva, 1996 ), whereby data structures are encoded by combining high-dimensional vectors with operations that together form an algebra on the space of distributed representations. In particular, we propose an efficient solution to a hard combinatorial search problem that arises when decoding elements of a VSA data structure: the factorization of products of multiple codevectors. Our proposed algorithm, called a resonator network, is a new type of recurrent neural network that interleaves VSA multiplication operations and pattern completion. We show in two examples—parsing of a tree-like data structure and parsing of a visual scene—how the factorization problem arises and how the resonator network can solve it. More broadly, resonator networks open the possibility of applying VSAs to myriad artificial intelligence problems in real-world domains. The companion article in this issue (Kent, Frady, Sommer, & Olshausen, 2020 ) presents a rigorous analysis and evaluation of the performance of resonator networks, showing it outperforms alternative approaches.

scholarly journals Resonator Networks, 2: Factorization Performance and Capacity Compared to Optimization-Based Methods

2020 ◽  
Vol 32 (12) ◽  
pp. 2332-2388 ◽  
Author(s):  
Spencer J. Kent ◽  
E. Paxon Frady ◽  
Friedrich T. Sommer ◽  
Bruno A. Olshausen
Keyword(s):  
Hadamard Product ◽  
Solution Space ◽  
Local Information ◽  
Alternative Methods ◽  
High Dimensional ◽  
Factorization Problem ◽  
Gradient Based ◽  
New Type ◽  
Alternative Approaches ◽  
Theoretical Foundations

We develop theoretical foundations of resonator networks, a new type of recurrent neural network introduced in Frady, Kent, Olshausen, and Sommer ( 2020 ), a companion article in this issue, to solve a high-dimensional vector factorization problem arising in Vector Symbolic Architectures. Given a composite vector formed by the Hadamard product between a discrete set of high-dimensional vectors, a resonator network can efficiently decompose the composite into these factors. We compare the performance of resonator networks against optimization-based methods, including Alternating Least Squares and several gradient-based algorithms, showing that resonator networks are superior in several important ways. This advantage is achieved by leveraging a combination of nonlinear dynamics and searching in superposition, by which estimates of the correct solution are formed from a weighted superposition of all possible solutions. While the alternative methods also search in superposition, the dynamics of resonator networks allow them to strike a more effective balance between exploring the solution space and exploiting local information to drive the network toward probable solutions. Resonator networks are not guaranteed to converge, but within a particular regime they almost always do. In exchange for relaxing the guarantee of global convergence, resonator networks are dramatically more effective at finding factorizations than all alternative approaches considered.

scholarly journals A Comparative Study about Data Structures Used for Efficient Management of Voxelised Full-Waveform Airborne LiDAR Data during 3D Polygonal Model Creation

2021 ◽  
Vol 13 (4) ◽  
pp. 559
Author(s):  
Milto Miltiadou ◽  
Neill D. F. Campbell ◽  
Darren Cosker ◽  
Michael G. Grant
Keyword(s):  
Data Structure ◽  
Data Structures ◽  
Airborne Lidar ◽  
Memory Allocation ◽  
Lidar Data ◽  
Full Waveform ◽  
Waveform Lidar ◽  
Airborne Lidar Data ◽  
Full Waveform Lidar ◽  
Polygonal Model

In this paper, we investigate the performance of six data structures for managing voxelised full-waveform airborne LiDAR data during 3D polygonal model creation. While full-waveform LiDAR data has been available for over a decade, extraction of peak points is the most widely used approach of interpreting them. The increased information stored within the waveform data makes interpretation and handling difficult. It is, therefore, important to research which data structures are more appropriate for storing and interpreting the data. In this paper, we investigate the performance of six data structures while voxelising and interpreting full-waveform LiDAR data for 3D polygonal model creation. The data structures are tested in terms of time efficiency and memory consumption during run-time and are the following: (1) 1D-Array that guarantees coherent memory allocation, (2) Voxel Hashing, which uses a hash table for storing the intensity values (3) Octree (4) Integral Volumes that allows finding the sum of any cuboid area in constant time, (5) Octree Max/Min, which is an upgraded octree and (6) Integral Octree, which is proposed here and it is an attempt to combine the benefits of octrees and Integral Volumes. In this paper, it is shown that Integral Volumes is the more time efficient data structure but it requires the most memory allocation. Furthermore, 1D-Array and Integral Volumes require the allocation of coherent space in memory including the empty voxels, while Voxel Hashing and the octree related data structures do not require to allocate memory for empty voxels. These data structures, therefore, and as shown in the test conducted, allocate less memory. To sum up, there is a need to investigate how the LiDAR data are stored in memory. Each tested data structure has different benefits and downsides; therefore, each application should be examined individually.

scholarly journals Shape Neutral Analysis of Graph-based Data-structures

2018 ◽  
Vol 18 (3-4) ◽  
pp. 470-483 ◽  
Author(s):  
GREGORY J. DUCK ◽  
JOXAN JAFFAR ◽  
ROLAND H. C. YAP
Keyword(s):  
Data Structure ◽  
Data Structures ◽  
Program Analysis ◽  
Constraint Handling ◽  
C Programs ◽  
Wide Range ◽  
Target Program ◽  
Target Data ◽  
Structure Graph ◽  
Structure Properties

AbstractMalformed data-structures can lead to runtime errors such as arbitrary memory access or corruption. Despite this, reasoning over data-structure properties for low-level heap manipulating programs remains challenging. In this paper we present a constraint-based program analysis that checks data-structure integrity, w.r.t. given target data-structure properties, as the heap is manipulated by the program. Our approach is to automatically generate a solver for properties using the type definitions from the target program. The generated solver is implemented using a Constraint Handling Rules (CHR) extension of built-in heap, integer and equality solvers. A key property of our program analysis is that the target data-structure properties are shape neutral, i.e., the analysis does not check for properties relating to a given data-structure graph shape, such as doubly-linked-lists versus trees. Nevertheless, the analysis can detect errors in a wide range of data-structure manipulating programs, including those that use lists, trees, DAGs, graphs, etc. We present an implementation that uses the Satisfiability Modulo Constraint Handling Rules (SMCHR) system. Experimental results show that our approach works well for real-world C programs.

Integrating Image Computation in Undergraduate Level Data-Structure Education

1998 ◽  
Vol 12 (08) ◽  
pp. 1071-1080 ◽  
Author(s):  
Sudeep Sarkar ◽  
Dmitry Goldgof
Keyword(s):  
Image Processing ◽  
Image Analysis ◽  
Data Structure ◽  
Data Structures ◽  
Science Curriculum ◽  
Early Stage ◽  
Real Life ◽  
South Florida ◽  
The University ◽  
Structure Education

There is a growing need for expertise both in image analysis and in software engineering. To date, these two areas have been taught separately in an undergraduate computer and information science curriculum. However, we have found that introduction to image analysis can be easily integrated in data-structure courses without detracting from the original goal of teaching data structures. Some of the image processing tasks offer a natural way to introduce basic data structures such as arrays, queues, stacks, trees and hash tables. Not only does this integrated strategy expose the students to image related manipulations at an early stage of the curriculum but it also imparts cohesiveness to the data-structure assignments and brings them closer to real life. In this paper we present a set of programming assignments that integrates undergraduate data-structure education with image processing tasks. These assignments can be incorporated in existing data-structure courses with low time and software overheads. We have used these assignment sets thrice: once in a 10-week duration data-structure course at the University of California, Santa Barbara and the other two times in 15-week duration courses at the University of South Florida, Tampa.

scholarly journals DenseZDD: A Compact and Fast Index for Families of Sets

Algorithms ◽  
2018 ◽  
Vol 11 (8) ◽  
pp. 128 ◽  
Author(s):  
Shuhei Denzumi ◽  
Jun Kawahara ◽  
Koji Tsuda ◽  
Hiroki Arimura ◽  
Shin-ichi Minato ◽  
...  
Keyword(s):  
Data Structure ◽  
Data Structures ◽  
Information Integration ◽  
Decision Diagrams ◽  
Web Information Retrieval ◽  
Binary Decision ◽  
Web Information ◽  
Set Operations ◽  
Succinct Data Structure ◽  
The Family

In this article, we propose a succinct data structure of zero-suppressed binary decision diagrams (ZDDs). A ZDD represents sets of combinations efficiently and we can perform various set operations on the ZDD without explicitly extracting combinations. Thanks to these features, ZDDs have been applied to web information retrieval, information integration, and data mining. However, to support rich manipulation of sets of combinations and update ZDDs in the future, ZDDs need too much space, which means that there is still room to be compressed. The paper introduces a new succinct data structure, called DenseZDD, for further compressing a ZDD when we do not need to conduct set operations on the ZDD but want to examine whether a given set is included in the family represented by the ZDD, and count the number of elements in the family. We also propose a hybrid method, which combines DenseZDDs with ordinary ZDDs. By numerical experiments, we show that the sizes of our data structures are three times smaller than those of ordinary ZDDs, and membership operations and random sampling on DenseZDDs are about ten times and three times faster than those on ordinary ZDDs for some datasets, respectively.

scholarly journals Spaced Seed Data Structures forDe NovoAssembly

2015 ◽  
Vol 2015 ◽  
pp. 1-8 ◽  
Author(s):  
Inanç Birol ◽  
Justin Chu ◽  
Hamid Mohamadi ◽  
Shaun D. Jackman ◽  
Karthika Raghavan ◽  
...  
Keyword(s):  
Data Structure ◽  
Data Structures ◽  
De Novo ◽  
Bloom Filters ◽  
De Bruijn Graph ◽  
Sequence Specificity ◽  
Sequencing Errors ◽  
Spaced Seeds ◽  
Read Error Correction ◽  
Seed Data

De novoassembly of the genome of a species is essential in the absence of a reference genome sequence. Many scalable assembly algorithms use the de Bruijn graph (DBG) paradigm to reconstruct genomes, where a table of subsequences of a certain length is derived from the reads, and their overlaps are analyzed to assemble sequences. Despite longer subsequences unlocking longer genomic features for assembly, associated increase in compute resources limits the practicability of DBG over other assembly archetypes already designed for longer reads. Here, we revisit the DBG paradigm to adapt it to the changing sequencing technology landscape and introduce three data structure designs for spaced seeds in the form of paired subsequences. These data structures address memory and run time constraints imposed by longer reads. We observe that when a fixed distance separates seed pairs, it provides increased sequence specificity with increased gap length. Further, we note that Bloom filters would be suitable to implicitly store spaced seeds and be tolerant to sequencing errors. Building on this concept, we describe a data structure for tracking the frequencies of observed spaced seeds. These data structure designs will have applications in genome, transcriptome and metagenome assemblies, and read error correction.

scholarly journals Depicting More Information in Enriched Squarified Treemaps with Layered Glyphs

Information ◽  
2020 ◽  
Vol 11 (2) ◽  
pp. 123
Author(s):  
Anderson Gregório Marques Soares ◽  
Elvis Thermo Carvalho Miranda ◽  
Rodrigo Santos do Amor Divino Lima ◽  
Carlos Gustavo Resque dos Santos ◽  
Bianchi Serique Meiguins
Keyword(s):  
Information Visualization ◽  
Data Structures ◽  
User Study ◽  
Relevant Information ◽  
Data Representation ◽  
High Dimensional ◽  
Hierarchical Data ◽  
Visual Data ◽  
Visual Variable ◽  
Data Clusters

The Treemap is one of the most relevant information visualization (InfoVis) techniques to support the analysis of large hierarchical data structures or data clusters. Despite that, Treemap still presents some challenges for data representation, such as the few options for visual data mappings and the inability to represent zero and negative values. Additionally, visualizing high dimensional data requires many hierarchies, which can impair data visualization. Thus, this paper proposes to add layered glyphs to Treemap’s items to mitigate these issues. Layered glyphs are composed of N partially visible layers, and each layer maps one data dimension to a visual variable. Since the area of the upper layers is always smaller than the bottom ones, the layers can be stacked to compose a multidimensional glyph. To validate this proposal, we conducted a user study to compare three scenarios of visual data mappings for Treemaps: only Glyphs (G), Glyphs and Hierarchy (GH), and only Hierarchy (H). Thirty-six volunteers with a background in InfoVis techniques, organized into three groups of twelve (one group per scenario), performed 8 InfoVis tasks using only one of the proposed scenarios. The results point that scenario GH presented the best accuracy while having a task-solving time similar to scenario H, which suggests that representing more data in Treemaps with layered glyphs enriched the Treemap visualization capabilities without impairing the data readability.

scholarly journals Maximum Likelihood Angle-Range Estimation for Monostatic FDA-MIMO Radar with Extended Range Ambiguity Using Subarrays

2020 ◽  
Vol 2020 ◽  
pp. 1-10
Author(s):  
Kaikai Yang ◽  
Sheng Hong ◽  
Qi Zhu ◽  
Yanheng Ye
Keyword(s):  
Maximum Likelihood ◽  
Joint Angle ◽  
Likelihood Estimation ◽  
Mimo Radar ◽  
Estimation Accuracy ◽  
High Dimensional ◽  
Search Problem ◽  
Range Estimation ◽  
Detection Range ◽  
Range Ambiguity

In this paper, we consider the joint angle-range estimation in monostatic FDA-MIMO radar. The transmit subarrays are first utilized to expand the range ambiguity, and the maximum likelihood estimation (MLE) algorithm is first proposed to improve the estimation performance. The range ambiguity is a serious problem in monostatic FDA-MIMO radar, which can reduce the detection range of targets. To extend the unambiguous range, we propose to divide the transmitting array into subarrays. Then, within the unambiguous range, the maximum likelihood (ML) algorithm is proposed to estimate the angle and range with high accuracy and high resolution. In the ML algorithm, the joint angle-range estimation problem becomes a high-dimensional search problem; thus, it is computationally expensive. To reduce the computation load, the alternating projection ML (AP-ML) algorithm is proposed by transforming the high-dimensional search into a series of one-dimensional search iteratively. With the proposed AP-ML algorithm, the angle and range are automatically paired. Simulation results show that transmitting subarray can extend the range ambiguity of monostatic FDA-MIMO radar and obtain a lower cramer-rao low bound (CRLB) for range estimation. Moreover, the proposed AP-ML algorithm is superior over the traditional estimation algorithms in terms of the estimation accuracy and resolution.

scholarly journals Human Search in a Fitness Landscape: How to Assess the Difficulty of a Search Problem

Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-11 ◽  
Author(s):  
Oana Vuculescu ◽  
Mads Kock Pedersen ◽  
Jacob F. Sherson ◽  
Carsten Bergenholtz
Keyword(s):  
Problem Solving ◽  
Computational Modeling ◽  
Cultural Evolution ◽  
Fitness Landscape ◽  
Search Behavior ◽  
Search Problem ◽  
Distinct Component ◽  
New Type ◽  
Human Problem ◽  
The Continuum

Computational modeling is widely used to study how humans and organizations search and solve problems in fields such as economics, management, cultural evolution, and computer science. We argue that current computational modeling research on human problem-solving needs to address several fundamental issues in order to generate more meaningful and falsifiable contributions. Based on comparative simulations and a new type of visualization of how to assess the nature of the fitness landscape, we address two key assumptions that approaches such as the NK framework rely on: that the NK captures the continuum of the complexity of empirical fitness landscapes and that search behavior is a distinct component, independent from the topology of the fitness landscape. We show the limitations of the most common approach to conceptualize how complex, or rugged, a landscape is, as well as how the nature of the fitness landscape is fundamentally intertwined with search behavior. Finally, we outline broader implications for how to simulate problem-solving.

Suffix Tree Constructing Algorithm for Datasets with Discrete Contents

2015 ◽  
Vol 733 ◽  
pp. 867-870
Author(s):  
Zhen Zhong Jin ◽  
Zheng Huang ◽  
Hua Zhang
Keyword(s):  
Data Structure ◽  
Sensor Network ◽  
Association Analysis ◽  
Data Structures ◽  
Suffix Tree ◽  
Analysis Data ◽  
Large Datasets ◽  
Intermediate Data ◽  
Input Strings

The suffix tree is a useful data structure constructed for indexing strings. However, when it comes to large datasets of discrete contents, most existing algorithms become very inefficient. Discrete datasets are need to be indexed in many fields like record analysis, data analyze in sensor network, association analysis etc. This paper presents an algorithm, STD, which stands for Suffix Tree for Discrete contents, that performs very efficiently with discrete input datasets. It imports several wonderful intermediate data structures for discrete strings; we also take care of the situation that the discrete input strings have similar characteristics. Moreover, STD keeps the advantages of existing implementations which are for successive input strings. Experiments were taken to evaluate the performance and shown that the method works well.

Sign in / Sign up

Export Citation Format

Share Document