Examination of work task and criteria choices for the relevance judgment process

2013 ◽  
Vol 69 (4) ◽  
pp. 523-544 ◽  
Author(s):  
Arthur Taylor
Keyword(s):  
Work Task ◽  
Relevance Judgment ◽  
Judgment Process

Ranking retrieval systems using pseudo relevance judgments

2015 ◽  
Vol 67 (6) ◽  
pp. 700-714 ◽  
Author(s):  
Sri Devi Ravana ◽  
Prabha Rajagopal ◽  
Vimala Balakrishnan
Keyword(s):  
Correlation Coefficient ◽  
Original Text ◽  
Key Factors ◽  
Document Ranking ◽  
Content Type ◽  
Relevance Judgment ◽  
Information Retrieval Evaluation ◽  
Retrieval Systems ◽  
Relevance Judgments ◽  
Judgment Process

Purpose – In a system-based approach, replicating the web would require large test collections, and judging the relevancy of all documents per topic in creating relevance judgment through human assessors is infeasible. Due to the large amount of documents that requires judgment, there are possible errors introduced by human assessors because of disagreements. The paper aims to discuss these issues. Design/methodology/approach – This study explores exponential variation and document ranking methods that generate a reliable set of relevance judgments (pseudo relevance judgments) to reduce human efforts. These methods overcome problems with large amounts of documents for judgment while avoiding human disagreement errors during the judgment process. This study utilizes two key factors: number of occurrences of each document per topic from all the system runs; and document rankings to generate the alternate methods. Findings – The effectiveness of the proposed method is evaluated using the correlation coefficient of ranked systems using mean average precision scores between the original Text REtrieval Conference (TREC) relevance judgments and pseudo relevance judgments. The results suggest that the proposed document ranking method with a pool depth of 100 could be a reliable alternative to reduce human effort and disagreement errors involved in generating TREC-like relevance judgments. Originality/value – Simple methods proposed in this study show improvement in the correlation coefficient in generating alternate relevance judgment without human assessors while contributing to information retrieval evaluation.

Research on Serendipity: The Methodological Challenges of Time and Language

2016 ◽  
Author(s):  
Lori McCay-Peet ◽  
Anabel Quan-Haase ◽  
Dagmar Kern ◽  
Peter Mutschke
Keyword(s):  
Work Task ◽  
Methodological Challenges

We explore the words academics (N = 26) in Germanyuse to describe unexpected and useful experienceswith information. We further report on the perceptionsof a portion of the participants’ experiences during anexploratory work task and a follow-up survey designedto capture reports of serendipity several days later.Nous explorons les mots que les universitaires (N =26) utilisent en Allemagne pour décrire desexpériences inattendues et utiles lors demanipulations d’information. Par ailleurs, nousrendons compte des perceptions d’une partie desexpériences des participants au cours d’une sessionde travaux exploratoires et une enquête de suiviconçue pour enregistrer les mentions de sérendipitéplusieurs jours plus tard.

Task Experience as a Predictor of Superior Loan Loss Judgments

2001 ◽  
Vol 20 (1) ◽  
pp. 147-155 ◽  
Author(s):  
William F. Wright
Keyword(s):  
Financial Institution ◽  
Loan Loss ◽  
Senior Managers ◽  
Performance Improvements ◽  
Outcome Feedback ◽  
Timely Feedback ◽  
Judgment Process ◽  
Task Experience ◽  
Biased Judgments

An area of significant importance and risk exposure during an audit of a financial institution is assessing the uncollectible portion of the client's loan portfolio. Auditing the collectibility of a commercial loan can be difficult because this complex judgment is semi-structured and many kinds of information can be relevant. However, timely judgment process and outcome feedback are available and may improve the quality of an auditor's conclusions over time. Therefore, to test for the benefits of task-specific experience, I compare loan judgments provided by inexperienced seniors, experienced managers, and more experienced junior partners and senior managers to a criterion based on the conclusions of senior audit partners. While previous research usually does not indicate performance improvements beyond the level of an audit senior (e.g., Tan and Libby 1997) for this complex task with timely feedback, consistent and substantial performance improvements are reported here. Auditors provided increasingly more appropriate and less biased judgments, and they achieved greater judgment consensus.

Institutions, uncertainty, and entrepreneurial judgment

2021 ◽  
pp. 1-11
Author(s):  
Claus Wiemann Frølund
Keyword(s):  
Human Interaction ◽  
Knightian Uncertainty ◽  
Entrepreneurial Process ◽  
Course Of Action ◽  
Judgment Process ◽  
Entrepreneurial Action ◽  
The Way

Abstract Entrepreneurial action takes place in a context of Knightian uncertainty. In order to overcome this uncertainty, entrepreneurs engage in a process of judgment resulting in a decision about the course of action. Institutions arise mainly to reduce economic friction by providing structure to human interaction and thus reducing uncertainty. However, institutions may also introduce further uncertainty and thus disrupt the judgment process preceding entrepreneurial action. The present paper builds upon recent efforts to integrate the concepts of uncertainty and institutions within the entrepreneurial context. Drawing on Frank H. Knight's seminal insight, the judgment-based view of entrepreneurship, and relevant concepts of entrepreneurial outcomes, the main contribution of the paper lies in the development of a model offering a coherent description of the way institutions affect uncertainty and the entrepreneurial process.

scholarly journals Crowd Worker Strategies in Relevance Judgment Tasks

2020 ◽  
Author(s):  
Lei Han ◽  
Eddy Maddalena ◽  
Alessandro Checco ◽  
Cristina Sarasua ◽  
Ujwal Gadiraju ◽  
...  
Keyword(s):  
Relevance Judgment

Being moral and doing good to others

2017 ◽  
Vol 35 (2) ◽  
pp. 180-191 ◽  
Author(s):  
Felix Septianto ◽  
Bambang Soegianto
Keyword(s):  
Prosocial Behavior ◽  
Moral Judgment ◽  
Cultural Context ◽  
Moral Identity ◽  
Qualitative Comparative Analysis ◽  
Marketing Strategies ◽  
Moral Emotion ◽  
Content Type ◽  
Judgment Process ◽  
Nuanced Understanding

Purpose Although previous research has established that moral emotion, moral judgment, and moral identity influence consumer intention to engage in prosocial behavior (e.g. donating, volunteering) under some circumstances, these factors, in reality, can concurrently influence judgment process. Therefore, it is important to get a more nuanced understanding of how the combinations of each factor can lead to a high intention to engage in prosocial behavior. The paper aims to discuss these issues. Design/methodology/approach This research employs fuzzy-set qualitative comparative analysis to explore different configurations of moral emotion, judgment, and identity that lead to a high consumer intention to engage in prosocial behavior. Findings Findings indicate four configurations of moral emotion, moral judgment, and moral identity that lead to a high intention to engage in prosocial behavior. Research limitations/implications This research focuses on the case of a hospital in Indonesia; thus, it is important not to overgeneralize the findings. Nonetheless, from a methodological standpoint, opportunity emerges to broaden the examinations in other service and cultural contexts. Practical implications The findings of this research can help the hospital to develop effective combinations of advertising and marketing strategies to promote prosocial behavior among its customers. Originality/value This paper provides the first empirical evidence on the existence of multiple pathways of moral emotion, judgment, and identity that lead to a high consumer intention to engage in prosocial behavior. The implications of this research also highlight the importance of cultural context in understanding consumer behavior.

scholarly journals Interrelation between the types of self-attitude and self-regulation of laziness in young individuals with different degree of self-regulation of behavior

2021 ◽  
pp. 28-39
Author(s):  
Alina Eduardovna Kim
Keyword(s):  
Young People ◽  
Self Regulation ◽  
Learning Task ◽  
Learning Activity ◽  
Work Process ◽  
Work Task ◽  
Work Tasks ◽  
The Subject ◽  
Theoretical Overview

The subject of this research is self-attitude and self-regulation of laziness in young individuals, who combine work and study. The article provides a brief theoretical overview of the research that prove interrelation between self-attitude and self-regulation of behavior and laziness. Using the quartilization procedure of the values of individual indicators, the author determined the groups with different degree of self-regulation of behavior; established the leading types of self-attitude of young individuals with different level of self-regulation of behavior. The presence and specificity of true links between the types of self-attitude with external and internal evaluative grounds and the severity of self-regulation of laziness in different contexts that provoke manifestations of laziness in young people with different level of self-regulation of behavior. Young individuals with high self-regulation of behavior demonstrate interconnectedness between self-regulation of laziness and types of self-attitude with internal evaluative grounds in execution of learning task, with external and internal evaluative grounds in execution of work task. The author underlines the importance of positive self-attitude for maintaining self-regulation of laziness. Interrelation between the types of self-attitude with both, external and internal evaluative grounds in execution of learning or work tasks are detected among the respondents with pronounced self-regulation above and below the average. Among young people with low self-regulation of behavior, the types of self-attitude with external evaluative grounds in conducting learning activity, the types of self-attitude with external and internal evaluative grounds in execution of work task, are interconnected with self-regulation of laziness. The reveled peculiarities should be taken into account in planning the educational and work process.

Sign in / Sign up

Export Citation Format

Share Document