scholarly journals How Good Are Modern Spatial Libraries?

2020 ◽  
Author(s):  
Varun Pandey ◽  
Alexander van Renen ◽  
Andreas Kipf ◽  
Alfons Kemper
Keyword(s):  
Spatial Data ◽  
High Performance ◽  
Nearest Neighbor ◽  
Vantage Point ◽  
Nearest Neighbor Search ◽  
Index Structure ◽  
Spatial Query ◽  
Neighbor Search ◽  
Spatial Data Management ◽  
Real World Datasets

Abstract Many applications today like Uber, Yelp, Tinder, etc. rely on spatial data or locations from its users. These applications and services either build their own spatial data management systems or rely on existing solutions. JTS Topology Suite (JTS), its C++ port GEOS, Google S2, ESRI Geometry API, and Java Spatial Index (JSI) are some of the spatial processing libraries that these systems build upon. These applications and services depend on indexing capabilities available in these libraries for high-performance spatial query processing. In this work, we compare these libraries qualitatively and quantitatively based on four different spatial queries using two real world datasets. We also compare these libraries with an open-source implementation of the Vantage Point Tree—an index structure that has been well studied in image retrieval and nearest-neighbor search algorithms for high-dimensional data. We found that Vantage Point Trees are very competitive and even outperform the aforementioned libraries in two queries.

scholarly journals A Hybrid Spatial Indexing Structure of Massive Point Cloud Based on Octree and 3D R*-Tree

2021 ◽  
Vol 11 (20) ◽  
pp. 9581
Author(s):  
Wei Wang ◽  
Yi Zhang ◽  
Genyu Ge ◽  
Qin Jiang ◽  
Yang Wang ◽  
...  
Keyword(s):  
Point Cloud ◽  
Nearest Neighbor ◽  
Nearest Neighbor Search ◽  
Index Structure ◽  
Spatial Indexing ◽  
Cloud Data ◽  
Neighbor Search ◽  
Nearest Neighbor Searching ◽  
Indexing Structure ◽  
Balanced Tree

The spatial index structure is one of the most important research topics for organizing and managing massive 3D Point Cloud. As a point in Point Cloud consists of Cartesian coordinates (x,y,z), the common method to explore geometric information and features is nearest neighbor searching. An efficient spatial indexing structure directly affects the speed of the nearest neighbor search. Octree and kd-tree are the most used for Point Cloud data. However, Octree or KD-tree do not perform best in nearest neighbor searching. A highly balanced tree, 3D R*-tree is considered the most effective method so far. So, a hybrid spatial indexing structure is proposed based on Octree and 3D R*-tree. In this paper, we discussed how thresholds influence the performance of nearest neighbor searching and constructing the tree. Finally, an adaptive way method adopted to set thresholds. Furthermore, we obtained a better performance in tree construction and nearest neighbor searching than Octree and 3D R*-tree.

scholarly journals Mobile Location Indexing Based On Synthetic Moving Objects

2019 ◽  
Vol 9 (4) ◽  
pp. 2556
Author(s):  
Thu Thu Zan ◽  
Sabai Phyu
Keyword(s):  
Performance Evaluation ◽  
Location Privacy ◽  
Moving Objects ◽  
Nearest Neighbor ◽  
Data Access ◽  
Nearest Neighbor Search ◽  
Index Structure ◽  
Mobile Objects ◽  
Neighbor Search ◽  
Privacy And Confidentiality

Today, the number of researches based on the data they move known as mobile objects indexing came out from the traditional static one. There are some indexing approaches to handle the complicated moving positions. One of the suitable ideas is pre-ordering these objects before building index structure. In this paper, a structure, a presorted-nearest index tree algorithm is proposed that allowed maintaining, updating, and range querying mobile objects within the desired period. Besides, it gives the advantage of an index structure to easy data access and fast query along with the retrieving nearest locations from a location point in the index structure. A synthetic mobile position dataset is also proposed for performance evaluation so that it is free from location privacy and confidentiality. The detail experimental results are discussed together with the performance evaluation of KDtree-based index structure. Both approaches are similarly efficient in range searching. However, the proposed approach is especially much more save time for the nearest neighbor search within a range than KD tree-based calculation.

scholarly journals The Chemfp Project

2019 ◽  
Author(s):  
Andrew Dalke
Keyword(s):  
Similarity Search ◽  
High Performance ◽  
Nearest Neighbor ◽  
Main Memory ◽  
Nearest Neighbor Search ◽  
Search Performance ◽  
Command Line ◽  
Data Set ◽  
Neighbor Search ◽  
Order Of Magnitude

<div>This paper describes the 10 years of work and research results of the chemfp project, available from http://chemfp.com/ . The project started as a way to promote the FPS format for cheminformatics fingerprint exchange. This is a line-oriented text format meant to be easy to read and write. It supports metadata such as the fingerprint type and data provenance.The chemfp package for Python was developed to provide the basic command-line tools and Python API for working with fingerprint data, because a format without useful tools will not be used. The similarity search performance improved by an order of magnitude over the decade, due to careful implementation and effective use of CPU hardware, including AVX2 support for faster popcount calculations than the built-in POPCNT instruction. The implementation details for high-performance search have rarely been discussed in the literature. As a result, many tools and published papers use implementations which are not close to the machine's capabilities. This paper describes those details to help with future optimization efforts. The most advanced version of chemfp evaluates about 130 million 1024-bit fingerprint Tanimotos per second on a single core of a standard x86-64 server machine. When combined with the BitBound algorithm, a k=1000 nearest-neighbor search of the 1.8 million 2048-bit Morgan fingerprints of ChEMBL 24 averages 27 ms/query and the same search of the 970 million PubChem fingerprints averages 220 ms/query, making chemfp one of the fastest similarity search tools available for CPUs. This appears to be several times faster than previously published work in the field, including in papers which use much more sophisticated data structures. A close analysis shows that nearly all earlier work assumes that the intersection popcount was the limiting performance factor, while on modern hardware uncompressed search is effectively memory bandwidth limited. For example, AVX2 search is 10% faster when memory prefetching, and the popcount evaluation time is far faster than fetching a random location in main memory. It proved difficult to evaluate existing tool performance because in the few cases where the tools were available, each used its own format, data sets, and search tasks. This paper introduces the chemfp benchmark data set to help make head-to-head comparisons easier in the future, and to help promote the FPS format. The FPS format is slow for tasks like web server reloads and command-line scripting. This paper also describes the FPB format, which is a binary application format for fast loads. </div>

Bregman Hyperplane Trees for Fast Approximate Nearest Neighbor Search

2012 ◽  
Vol 3 (4) ◽  
pp. 75-87
Author(s):  
Bilegsaikhan Naidan ◽  
Magnus Lie Hetland
Keyword(s):  
Query Processing ◽  
Nearest Neighbor ◽  
Nearest Neighbor Search ◽  
Index Structure ◽  
Data Sets ◽  
Search Performance ◽  
Space Partitioning ◽  
Neighbor Search ◽  
Order Of Magnitude ◽  
Exact Nearest Neighbor

This article presents a new approximate index structure, the Bregman hyperplane tree, for indexing the Bregman divergence, aiming to decrease the number of distance computations required at query processing time, by sacrificing some accuracy in the result. The experimental results on various high-dimensional data sets demonstrate that the proposed index structure performs comparably to the state-of-the-art Bregman ball tree in terms of search performance and result quality. Moreover, this method results in a speedup of well over an order of magnitude for index construction. The authors also apply their space partitioning principle to the Bregman ball tree and obtain a new index structure for exact nearest neighbor search that is faster to build and a slightly slower at query processing than the original.

scholarly journals The chemfp Project

2019 ◽  
Author(s):  
Andrew Dalke
Keyword(s):  
Similarity Search ◽  
High Performance ◽  
Nearest Neighbor ◽  
Main Memory ◽  
Nearest Neighbor Search ◽  
Search Performance ◽  
Command Line ◽  
Data Set ◽  
Neighbor Search ◽  
Order Of Magnitude

<div>This paper describes the 10 years of work and research results of the chemfp project, available from http://chemfp.com/ . The project started as a way to promote the FPS format for cheminformatics fingerprint exchange. This is a line-oriented text format meant to be easy to read and write. It supports metadata such as the fingerprint type and data provenance.The chemfp package for Python was developed to provide the basic command-line tools and Python API for working with fingerprint data, because a format without useful tools will not be used. <br></div><div><br></div><div>The similarity search performance improved by an order of magnitude over the decade, due to careful implementation and effective use of CPU hardware, including AVX2 support for faster popcount calculations than the built-in POPCNT instruction. The implementation details for high-performance search have rarely been discussed in the literature. As a result, many tools and published papers use implementations which are not close to the machine's capabilities.</div><div><br></div><div>This paper describes those details to help with future optimization efforts.</div><div><br></div><div>The most advanced version of chemfp evaluates about 130 million 1024-bit fingerprint Tanimotos per second on a single core of a standard x86-64 server machine. When combined with the BitBound algorithm, a k=1000 nearest-neighbor search of the 1.8 million 2048-bit Morgan fingerprints of ChEMBL 24 averages 27 ms/query and the same search of the 970 million PubChem fingerprints averages 220 ms/query, making chemfp one of the fastest similarity search tools available for CPUs. This appears to be several times faster than previously published work in the field, including in papers which use much more sophisticated data structures.</div><div><br></div><div>A close analysis shows that nearly all earlier work assumes that the intersection popcount was the limiting performance factor, while on modern hardware uncompressed search is effectively memory bandwidth limited. For example, AVX2 search is 10% faster when memory prefetching, and the popcount evaluation time is far faster than fetching a random location in main memory. It proved difficult to evaluate existing tool performance because in the few cases where the tools were available, each used its own format, data sets, and search tasks.</div><div><br></div><div>This paper introduces the chemfp benchmark data set to help make head-to-head comparisons easier in the future, and to help promote the FPS format. The FPS format is slow for tasks like web server reloads and command-line scripting. This paper also describes the FPB format, which is a binary application format for fast loads. </div>

scholarly journals Tree Index Nearest Neighbor Search of Moving Objects along a Road Network

2021 ◽  
Vol 2021 ◽  
pp. 1-18
Author(s):  
Wei Jiang ◽  
Fangliang Wei ◽  
Guanyu Li ◽  
Mei Bai ◽  
Yongqiang Ren ◽  
...  
Keyword(s):  
Road Network ◽  
Moving Objects ◽  
Nearest Neighbor ◽  
Search Algorithm ◽  
Urban Transportation ◽  
Road Networks ◽  
Nearest Neighbor Search ◽  
Index Structure ◽  
Widespread Application ◽  
Neighbor Search

With the widespread application of location-based service (LBS) technology in the urban Internet of Things, urban transportation has become a research hotspot. One key issue of urban transportation is the nearest neighbor search of moving objects along a road network. The fast-updating operations of moving objects along a road network suppress the query response time of urban services. Thus, a tree-indexed searching method is proposed to quickly find the answers to user-defined queries on frequently updating road networks. First, a novel index structure, called the double tree-hash index, is designed to reorganize the corresponding relationships of moving objects and road networks. Second, an index-enhanced search algorithm is proposed to quickly find the k -nearest neighbors of moving objects along the road network. Finally, an experiment shows that compared with state-of-the-art algorithms, our algorithm shows a significant improvement in search efficiency on frequently updating road networks.

scholarly journals The chemfp Project

2019 ◽  
Author(s):  
Andrew Dalke
Keyword(s):  
Similarity Search ◽  
High Performance ◽  
Nearest Neighbor ◽  
Main Memory ◽  
Nearest Neighbor Search ◽  
Search Performance ◽  
Command Line ◽  
Data Set ◽  
Neighbor Search ◽  
Order Of Magnitude

<div>This paper describes the 10 years of work and research results of the chemfp project, available from http://chemfp.com/ . The project started as a way to promote the FPS format for cheminformatics fingerprint exchange. This is a line-oriented text format meant to be easy to read and write. It supports metadata such as the fingerprint type and data provenance.The chemfp package for Python was developed to provide the basic command-line tools and Python API for working with fingerprint data, because a format without useful tools will not be used. <br></div><div><br></div><div>The similarity search performance improved by an order of magnitude over the decade, due to careful implementation and effective use of CPU hardware, including AVX2 support for faster popcount calculations than the built-in POPCNT instruction. The implementation details for high-performance search have rarely been discussed in the literature. As a result, many tools and published papers use implementations which are not close to the machine's capabilities.</div><div><br></div><div>This paper describes those details to help with future optimization efforts.</div><div><br></div><div>The most advanced version of chemfp evaluates about 130 million 1024-bit fingerprint Tanimotos per second on a single core of a standard x86-64 server machine. When combined with the BitBound algorithm, a k=1000 nearest-neighbor search of the 1.8 million 2048-bit Morgan fingerprints of ChEMBL 24 averages 27 ms/query and the same search of the 970 million PubChem fingerprints averages 220 ms/query, making chemfp one of the fastest similarity search tools available for CPUs. This appears to be several times faster than previously published work in the field, including in papers which use much more sophisticated data structures.</div><div><br></div><div>A close analysis shows that nearly all earlier work assumes that the intersection popcount was the limiting performance factor, while on modern hardware uncompressed search is effectively memory bandwidth limited. For example, AVX2 search is 10% faster when memory prefetching, and the popcount evaluation time is far faster than fetching a random location in main memory. It proved difficult to evaluate existing tool performance because in the few cases where the tools were available, each used its own format, data sets, and search tasks.</div><div><br></div><div>This paper introduces the chemfp benchmark data set to help make head-to-head comparisons easier in the future, and to help promote the FPS format. The FPS format is slow for tasks like web server reloads and command-line scripting. This paper also describes the FPB format, which is a binary application format for fast loads. </div>

scholarly journals CONNEKT: Co-Located Nearest Neighbor Search using KNN Querying with K-D Tree

2019 ◽  
Vol 8 (2) ◽  
pp. 1164-1171
Keyword(s):  
Spatial Data ◽  
Pattern Mining ◽  
Nearest Neighbor ◽  
Search Algorithm ◽  
Nearest Neighbor Search ◽  
Location Based Services ◽  
Context Aware ◽  
K Nearest Neighbor ◽  
Data Set ◽  
Neighbor Search

Data about entities or objects associated with geographical or location information could be called as spatial data. Spatial data helps in identifying and positioning anyone or anything globally anywhere across the world. Instances of various spatial features that are closely found together are called as spatial co-located patterns. So far, the spatial co-located patterns have been used only for knowledge discovery process but it would serve a wide variety of applications if analyzed intensively. One such application is to use co-location pattern mining for a context aware based search. Hence the main aim of this work is to extend the K-Nearest Neighbor (KNN) querying to co-located instances for context aware based querying or location-based services (LBS). For the above-said purpose, co-located nearest neighbor search algorithm namely “CONNEKT” is proposed. The co-located instances are mapped onto a K-dimensional tree (K-d tree) inorder to make the querying process efficient. The algorithm is analyzed using a hypothetical data set generated through QGIS

Sign in / Sign up

Export Citation Format

Share Document