Strategies for Document Management

2010 ◽  
Vol 1 (1) ◽  
pp. 64-83 ◽  
Author(s):  
Karen Corral ◽  
David Schuff ◽  
Gregory Schymik ◽  
Robert St. Louis
Keyword(s):  
Keyword Search ◽  
Performance Measure ◽  
Document Management ◽  
Research Database ◽  
Search Performance ◽  
Search Queries ◽  
Order Of Magnitude ◽  
Subject Indexing ◽  
Large Research ◽  
Document Stores

Keyword search has failed to adequately meet the needs of enterprise users. This is largely due to the size of document stores, the distribution of word frequencies, and the indeterminate nature of languages. The authors argue a different approach needs to be taken, and draw on the successes of dimensional data modeling and subject indexing to propose a solution. They test our solution by performing search queries on a large research database. By incorporating readily available subject indexes into the search process, they obtain order of magnitude improvements in the performance of search queries. Their performance measure is the ratio of the number of documents returned without using subject indexes to the number of documents returned when subject indexes are used. The authors explain why the observed tenfold improvement in search performance on our research database can be expected to occur for searches on a wide variety of enterprise document stores.

Strategies for Document Management

2012 ◽  
pp. 96-114
Author(s):  
Karen Corral ◽  
David Schuff ◽  
Gregory Schymik ◽  
Robert St. Louis
Keyword(s):  
Keyword Search ◽  
Performance Measure ◽  
Document Management ◽  
Research Database ◽  
Search Performance ◽  
Search Queries ◽  
Order Of Magnitude ◽  
Subject Indexing ◽  
Word Frequencies ◽  
Document Stores

Keyword search has failed to adequately meet the needs of enterprise users. This is largely due to the size of document stores, the distribution of word frequencies, and the indeterminate nature of languages. The authors argue a different approach needs to be taken, and draw on the successes of dimensional data modeling and subject indexing to propose a solution. They test our solution by performing search queries on a large research database. By incorporating readily available subject indexes into the search process, they obtain order of magnitude improvements in the performance of search queries. Their performance measure is the ratio of the number of documents returned without using subject indexes to the number of documents returned when subject indexes are used. The authors explain why the observed tenfold improvement in search performance on our research database can be expected to occur for searches on a wide variety of enterprise document stores.

scholarly journals Development and Implementation of a Registry of Patients Attending Multidisciplinary Pain Treatment Clinics: The Quebec Pain Registry

2017 ◽  
Vol 2017 ◽  
pp. 1-16 ◽  
Author(s):  
M. Choinière ◽  
M. A. Ware ◽  
M. G. Pagé ◽  
A. Lacasse ◽  
H. Lanctôt ◽  
...  
Keyword(s):  
Pain Treatment ◽  
Tertiary Care ◽  
Pain Clinic ◽  
Research Database ◽  
Pain Syndromes ◽  
Medical Disorders ◽  
Research Questions ◽  
Conducting Research ◽  
Multiple Domains ◽  
Large Research

The Quebec Pain Registry (QPR) is a large research database of patients suffering from various chronic pain (CP) syndromes who were referred to one of five tertiary care centres in the province of Quebec (Canada). Patients were monitored using common demographics, identical clinical descriptors, and uniform validated outcomes. This paper describes the development, implementation, and research potential of the QPR. Between 2008 and 2013, 6902 patients were enrolled in the QPR, and data were collected prior to their first visit at the pain clinic and six months later. More than 90% of them (mean age ± SD: 52.76 ± 4.60, females: 59.1%) consented that their QPR data be used for research purposes. The results suggest that, compared to patients with serious chronic medical disorders, CP patients referred to tertiary care clinics are more severely impaired in multiple domains including emotional and physical functioning. The QPR is also a powerful and comprehensive tool for conducting research in a “real-world” context with 27 observational studies and satellite research projects which have been completed or are underway. It contains data on the clinical evolution of thousands of patients and provides the opportunity of answering important research questions on various aspects of CP (or specific pain syndromes) and its management.

Graph Encryption for Top-K Nearest Keyword Search Queries on Cloud

2017 ◽  
Vol 2 (4) ◽  
pp. 371-381 ◽  
Author(s):  
Chang Liu ◽  
Liehuang Zhu ◽  
Jinjun Chen
Keyword(s):  
Keyword Search ◽  
Search Queries

scholarly journals The Chemfp Project

2019 ◽  
Author(s):  
Andrew Dalke
Keyword(s):  
Similarity Search ◽  
High Performance ◽  
Nearest Neighbor ◽  
Main Memory ◽  
Nearest Neighbor Search ◽  
Search Performance ◽  
Command Line ◽  
Data Set ◽  
Neighbor Search ◽  
Order Of Magnitude

<div>This paper describes the 10 years of work and research results of the chemfp project, available from http://chemfp.com/ . The project started as a way to promote the FPS format for cheminformatics fingerprint exchange. This is a line-oriented text format meant to be easy to read and write. It supports metadata such as the fingerprint type and data provenance.The chemfp package for Python was developed to provide the basic command-line tools and Python API for working with fingerprint data, because a format without useful tools will not be used. The similarity search performance improved by an order of magnitude over the decade, due to careful implementation and effective use of CPU hardware, including AVX2 support for faster popcount calculations than the built-in POPCNT instruction. The implementation details for high-performance search have rarely been discussed in the literature. As a result, many tools and published papers use implementations which are not close to the machine's capabilities. This paper describes those details to help with future optimization efforts. The most advanced version of chemfp evaluates about 130 million 1024-bit fingerprint Tanimotos per second on a single core of a standard x86-64 server machine. When combined with the BitBound algorithm, a k=1000 nearest-neighbor search of the 1.8 million 2048-bit Morgan fingerprints of ChEMBL 24 averages 27 ms/query and the same search of the 970 million PubChem fingerprints averages 220 ms/query, making chemfp one of the fastest similarity search tools available for CPUs. This appears to be several times faster than previously published work in the field, including in papers which use much more sophisticated data structures. A close analysis shows that nearly all earlier work assumes that the intersection popcount was the limiting performance factor, while on modern hardware uncompressed search is effectively memory bandwidth limited. For example, AVX2 search is 10% faster when memory prefetching, and the popcount evaluation time is far faster than fetching a random location in main memory. It proved difficult to evaluate existing tool performance because in the few cases where the tools were available, each used its own format, data sets, and search tasks. This paper introduces the chemfp benchmark data set to help make head-to-head comparisons easier in the future, and to help promote the FPS format. The FPS format is slow for tasks like web server reloads and command-line scripting. This paper also describes the FPB format, which is a binary application format for fast loads. </div>

Bregman Hyperplane Trees for Fast Approximate Nearest Neighbor Search

2012 ◽  
Vol 3 (4) ◽  
pp. 75-87
Author(s):  
Bilegsaikhan Naidan ◽  
Magnus Lie Hetland
Keyword(s):  
Query Processing ◽  
Nearest Neighbor ◽  
Nearest Neighbor Search ◽  
Index Structure ◽  
Data Sets ◽  
Search Performance ◽  
Space Partitioning ◽  
Neighbor Search ◽  
Order Of Magnitude ◽  
Exact Nearest Neighbor

This article presents a new approximate index structure, the Bregman hyperplane tree, for indexing the Bregman divergence, aiming to decrease the number of distance computations required at query processing time, by sacrificing some accuracy in the result. The experimental results on various high-dimensional data sets demonstrate that the proposed index structure performs comparably to the state-of-the-art Bregman ball tree in terms of search performance and result quality. Moreover, this method results in a speedup of well over an order of magnitude for index construction. The authors also apply their space partitioning principle to the Bregman ball tree and obtain a new index structure for exact nearest neighbor search that is faster to build and a slightly slower at query processing than the original.

The TAO of ATWV: Probing the mysteries of keyword search performance

2013 ◽  
Author(s):  
Steven Wegmann ◽  
Arlo Faria ◽  
Adam Janin ◽  
Korbinian Riedhammer ◽  
Nelson Morgan
Keyword(s):  
Keyword Search ◽  
Search Performance

scholarly journals Policy issues on covid-19 waste: comparing Indonesia and Taiwan

2021 ◽  
Vol 905 (1) ◽  
pp. 012113
Author(s):  
R A Nugroho ◽  
A A Rahmawati ◽  
S G Prakoso ◽  
I D A Nurhaeni ◽  
A T Kartinawanty ◽  
...  
Keyword(s):  
Government Policy ◽  
Environmental Sustainability ◽  
Keyword Search ◽  
Medical Waste ◽  
High Rate ◽  
Research Database ◽  
Literature Analysis ◽  
Policy Issues ◽  
Significant Difference ◽  
The Government

Abstract During the covid-19 pandemic, medical waste has been a concern to the sustainability issues. Comparing government awareness is critical to portray the government policy on combating covid-19 and maintaining environmental sustainability at the same time. This paper discussed how the covid-19 waste is managed between two countries: Indonesia and Taiwan. The two countries are chosen because of their contrasting condition where the prior has a high rate of infection while, on the other hand, the latter has a relatively low rate of infection. This study focuses on literature analysis that is available on the research database. Specific keyword search such as “environmental policy and covid-19 and Indonesia and Taiwan” is used in the search engine. The results indicated the significant difference in both countries in managing covid-19 waste. Further results are discussed in the paper.

scholarly journals The chemfp Project

2019 ◽  
Author(s):  
Andrew Dalke
Keyword(s):  
Similarity Search ◽  
High Performance ◽  
Nearest Neighbor ◽  
Main Memory ◽  
Nearest Neighbor Search ◽  
Search Performance ◽  
Command Line ◽  
Data Set ◽  
Neighbor Search ◽  
Order Of Magnitude

<div>This paper describes the 10 years of work and research results of the chemfp project, available from http://chemfp.com/ . The project started as a way to promote the FPS format for cheminformatics fingerprint exchange. This is a line-oriented text format meant to be easy to read and write. It supports metadata such as the fingerprint type and data provenance.The chemfp package for Python was developed to provide the basic command-line tools and Python API for working with fingerprint data, because a format without useful tools will not be used. <br></div><div><br></div><div>The similarity search performance improved by an order of magnitude over the decade, due to careful implementation and effective use of CPU hardware, including AVX2 support for faster popcount calculations than the built-in POPCNT instruction. The implementation details for high-performance search have rarely been discussed in the literature. As a result, many tools and published papers use implementations which are not close to the machine's capabilities.</div><div><br></div><div>This paper describes those details to help with future optimization efforts.</div><div><br></div><div>The most advanced version of chemfp evaluates about 130 million 1024-bit fingerprint Tanimotos per second on a single core of a standard x86-64 server machine. When combined with the BitBound algorithm, a k=1000 nearest-neighbor search of the 1.8 million 2048-bit Morgan fingerprints of ChEMBL 24 averages 27 ms/query and the same search of the 970 million PubChem fingerprints averages 220 ms/query, making chemfp one of the fastest similarity search tools available for CPUs. This appears to be several times faster than previously published work in the field, including in papers which use much more sophisticated data structures.</div><div><br></div><div>A close analysis shows that nearly all earlier work assumes that the intersection popcount was the limiting performance factor, while on modern hardware uncompressed search is effectively memory bandwidth limited. For example, AVX2 search is 10% faster when memory prefetching, and the popcount evaluation time is far faster than fetching a random location in main memory. It proved difficult to evaluate existing tool performance because in the few cases where the tools were available, each used its own format, data sets, and search tasks.</div><div><br></div><div>This paper introduces the chemfp benchmark data set to help make head-to-head comparisons easier in the future, and to help promote the FPS format. The FPS format is slow for tasks like web server reloads and command-line scripting. This paper also describes the FPB format, which is a binary application format for fast loads. </div>

Sign in / Sign up

Export Citation Format

Share Document