scholarly journals Understanding Bias and Variance of Learning-to-Rank Algorithms: An Empirical Framework

Author(s):  
Muhammad Ibrahim
Author(s):  
Kris Ferreira ◽  
Sunanda Parthasarathy ◽  
Shreyas Sekar
Keyword(s):  

Technovation ◽  
1994 ◽  
Vol 14 (5) ◽  
pp. 311-321 ◽  
Author(s):  
Vinod Kumar ◽  
Aditha N.S. Persaud ◽  
Uma Kumar

2021 ◽  
Vol 39 (2) ◽  
pp. 1-29
Author(s):  
Qingyao Ai ◽  
Tao Yang ◽  
Huazheng Wang ◽  
Jiaxin Mao

How to obtain an unbiased ranking model by learning to rank with biased user feedback is an important research question for IR. Existing work on unbiased learning to rank (ULTR) can be broadly categorized into two groups—the studies on unbiased learning algorithms with logged data, namely, the offline unbiased learning, and the studies on unbiased parameters estimation with real-time user interactions, namely, the online learning to rank. While their definitions of unbiasness are different, these two types of ULTR algorithms share the same goal—to find the best models that rank documents based on their intrinsic relevance or utility. However, most studies on offline and online unbiased learning to rank are carried in parallel without detailed comparisons on their background theories and empirical performance. In this article, we formalize the task of unbiased learning to rank and show that existing algorithms for offline unbiased learning and online learning to rank are just the two sides of the same coin. We evaluate eight state-of-the-art ULTR algorithms and find that many of them can be used in both offline settings and online environments with or without minor modifications. Further, we analyze how different offline and online learning paradigms would affect the theoretical foundation and empirical effectiveness of each algorithm on both synthetic and real search data. Our findings provide important insights and guidelines for choosing and deploying ULTR algorithms in practice.


2021 ◽  
pp. 016555152199804
Author(s):  
Qian Geng ◽  
Ziang Chuai ◽  
Jian Jin

To provide junior researchers with domain-specific concepts efficiently, an automatic approach for academic profiling is needed. First, to obtain personal records of a given scholar, typical supervised approaches often utilise structured data like infobox in Wikipedia as training dataset, but it may lead to a severe mis-labelling problem when they are utilised to train a model directly. To address this problem, a new relation embedding method is proposed for fine-grained entity typing, in which the initial vector of entities and a new penalty scheme are considered, based on the semantic distance of entities and relations. Also, to highlight critical concepts relevant to renowned scholars, scholars’ selective bibliographies which contain massive academic terms are analysed by a newly proposed extraction method based on logistic regression, AdaBoost algorithm and learning-to-rank techniques. It bridges the gap that conventional supervised methods only return binary classification results and fail to help researchers understand the relative importance of selected concepts. Categories of experiments on academic profiling and corresponding benchmark datasets demonstrate that proposed approaches outperform existing methods notably. The proposed techniques provide an automatic way for junior researchers to obtain organised knowledge in a specific domain, including scholars’ background information and domain-specific concepts.


Sign in / Sign up

Export Citation Format

Share Document