Quantifying test collection quality based on the consistency of relevance judgements

2011 ◽  
Author(s):  
Falk Scholer ◽  
Andrew Turpin ◽  
Mark Sanderson
Keyword(s):  
Test Collection

The CSIRO enterprise search test collection

2007 ◽  
Vol 41 (2) ◽  
pp. 42-45 ◽  
Author(s):  
Peter Bailey ◽  
Nick Craswell ◽  
Ian Soboroff ◽  
Arjen P. de Vries
Keyword(s):  
Test Collection ◽  
Enterprise Search

A Test Collection for Ad-hoc Dataset Retrieval

2021 ◽  
Author(s):  
Makoto P. Kato ◽  
Hiroaki Ohshima ◽  
Ying-Hsang Liu ◽  
Hsin-Liang Chen
Keyword(s):  
Ad Hoc ◽  
Test Collection

scholarly journals A Test Collection for Interactive Lifelog Retrieval

2018 ◽  
pp. 312-324 ◽  
Author(s):  
Cathal Gurrin ◽  
Klaus Schoeffmann ◽  
Hideo Joho ◽  
Bernd Munzer ◽  
Rami Albatal ◽  
...  
Keyword(s):  
Test Collection

Test Collection

2009 ◽  
pp. 3040-3041
Author(s):  
Ben Carterette
Keyword(s):  
Test Collection

scholarly journals Neural Embedding-Based Metrics for Pre-retrieval Query Performance Prediction

2021 ◽  
Author(s):  
Arabzadehghahyazi Negar
Keyword(s):  
Performance Prediction ◽  
State Of The Art ◽  
Learning To Rank ◽  
The State ◽  
Test Collection ◽  
Query Performance ◽  
Performance Predictors ◽  
Level Statistics ◽  
Ablation Study ◽  
Individual Specificity

file:///C:/Users/MWF/Downloads/Arabzadehghahyazi, Negar.Pre-retrieval Query Performance Prediction (QPP) methods are oblivious to the performance of the retrieval model as they predict query difficulty prior to observing the set of documents retrieved for the query. Among pre-retrieval query performance predictors, specificity-based metrics investigate how corpus, query and corpus-query level statistics can be used to predict the performance of the query. In this thesis, we explore how neural embeddings can be utilized to define corpus-independent and semantics-aware specificity metrics. Our metrics are based on the intuition that a term that is closely surrounded by other terms in the embedding space is more likely to be specific while a term surrounded by less closely related terms is more likely to be generic. On this basis, we leverage geometric properties between embedded terms to define four groups of metrics: (1) neighborhood-based, (2) graph-based, (3) cluster-based and (4) vector-based metrics. Moreover, we employ learning-to-rank techniques to analyze the importance of individual specificity metrics. To evaluate the proposed metrics, we have curated and publicly share a test collection of term specificity measurements defined based on Wikipedia category hierarchy and DMOZ taxonomy. We report on our extensive experiments on the effectiveness of our metrics through metric comparison, ablation study and comparison against the state-of-the-art baselines. We have shown that our proposed set of pre-retrieval QPP metrics based on the properties of pre-trained neural embeddings are more effective for performance prediction compared to the state-of-the-art methods. We report our findings based on Robust04, ClueWeb09 and Gov2 corpora and their associated TREC topics.

scholarly journals Effectiveness of Acceptance and Commitment Therapy on Resilience, Psychological Well-Being, and Life Expectancy in Asthmatic Patients

2021 ◽  
Author(s):  
Shaghayegh Bahramiabdolmalaki ◽  
Alireza Homayouni ◽  
Masoud Aliyali
Keyword(s):  
Life Expectancy ◽  
Acceptance And Commitment Therapy ◽  
Well Being ◽  
Test Collection ◽  
Psychological Well Being ◽  
Asthmatic Patients ◽  
Commitment Therapy ◽  
Acceptance And Commitment ◽  
Significant Difference ◽  
Post Test

Introduction: Psychosomatic experts have tried to associate mental disorders to physical illnesses. The vulnerability of different parts of the body is thought to depend on fundamental differences between individuals. One of the methods that seems to affect the psychological problems of asthma patients is acceptance and commitment therapy. Therefore, the aim of this study was to evaluate the effectiveness of acceptance- and commitment-based therapy on resilience, psychological well-being, and life expectancy in asthmatic patients. Methods: This quasi-experimental pre-test and post-test study was conducted on 30 asthmatic patients who were randomly assigned to the experimental (n = 15) and control (n = 15) groups according to the inclusion criteria. Acceptance and commitment therapy sessions were based on the treatment package of Hayes et al. in 8 sessions of 60 minutes on the experimental group and no intervention was performed on the control group. All participants took part in the pre-test and post-test. Collection tools included Conner-Davidson Resilience Questionnaire, Schneider Life expectancy, and Ryf Psychological Well-being. Results: The results showed a significant difference in the components of resilience, psychological well-being, and life expectancy in asthmatic patients before and after the experiment (p <0.05). In other words, acceptance and commitment-based therapy had a positive effect on resilience, psychological well-being and life expectancy in asthmatic patients and these components have increased in patients. Conclusion: Findings showed that acceptance- and commitment-based therapy was effective on resilience, psychological well-being, and life expectancy of asthmatic patients. This treatment is suggested to be used in conjunction with drug therapy to improve the psychological symptoms of asthmatic patients.

Modeling the co-citation dependence on semantic layers of co-cited documents

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Maryam Yaghtin ◽  
Hajar Sotudeh ◽  
Alireza Nikseresht ◽  
Mahdieh Mirzabeigi
Keyword(s):  
Peer Review ◽  
Language Processing ◽  
Opinion Mining ◽  
Citation Network ◽  
Test Collection ◽  
Medical Subject Headings ◽  
Content Type ◽  
Citation Frequency ◽  
Mesh Terms ◽  
Citation Measures

PurposeCo-citation frequency, defined as the number of documents co-citing two articles, is considered as a quantitative, and thus, an efficient proxy of subject relatedness or prestige of the co-cited articles. Despite its quantitative nature, it is found effective in retrieving and evaluating documents, signifying its linkage with the related documents' contents. To better understand the dynamism of the citation network, the present study aims to investigate various content features giving rise to the measure.Design/methodology/approachThe present study examined the interaction of different co-citation features in explaining the co-citation frequency. The features include the co-cited works' similarities in their full-texts, Medical Subject Headings (MeSH) terms, co-citation proximity, opinions and co-citances. A test collection is built using the CITREC dataset. The data were analyzed using natural language processing (NLP) and opinion mining techniques. A linear model was developed to regress the objective and subjective content-based co-citation measures against the natural log of the co-citation frequency.FindingsThe dimensions of co-citation similarity, either subjective or objective, play significant roles in predicting co-citation frequency. The model can predict about half of the co-citation variance. The interaction of co-opinionatedness and non-co-opinionatedness is the strongest factor in the model.Originality/valueIt is the first study in revealing that both the objective and subjective similarities could significantly predict the co-citation frequency. The findings re-confirm the citation analysis assumption claiming the connection between the cognitive layers of cited documents and citation measures in general and the co-citation frequency in particular.Peer reviewThe peer review history for this article is available at https://publons.com/publon/10.1108/OIR-04-2020-0126.

Sign in / Sign up

Export Citation Format

Share Document