Binary and graded relevance in IR evaluations--Comparison of the effects on ranking of IR systems

Keywords

Weighting, Relevance (law), Binary number, Ranking (information retrieval), Rank (graph theory)

References

• Overview of the Eighth Text REtrieval Conference (TREC-8)Ellen M. Voorhees, Donna Harman — 1999
• Relevance and Information Behavior.Linda Schamber — 1994
• Measures of relative relevance and ranked half-life: performance indicators for interactive IRPia Borlund, Peter Ingwersen — 1998
• A definition of relevance for information retrievalWilliam S. Cooper — 1971
• Average gain ratio: a simple retrieval performance measure for evaluation with multiple relevance levelsTetsuya Sakai — 2003
• Using statistical testing in the evaluation of retrieval experimentsDavid A. Hull — 1993
• Practical Nonparametric Statistics (2nd ed.)Thomas E. Obremski — 1981
• Towards the Identification of the Optimal Number of Relevance CategoriesRong Tang, W. M. Shaw, Jack L. Vevea — 1999
• Experimental components for the evaluation of interactive information retrieval systemsPia Borlund — 2000
• Ranking in PrincipleStephen Robertson, Nicholas J. Belkin — 1978
• The probability ranking principle in IRStephen Robertson — 1997
• Dimensions of relevanceErica Cosijn, Peter Ingwersen — 2000
• Relevance assessments and retrieval system evaluationMichael Lesk, G. Salton — 1968
• Liberal relevance criteria of TREC -: counting on negligible documents?Eero Sormunen — 2002
• Evaluation by highly relevant documentsEllen M. Voorhees — 2001
• A re-examination of relevance: toward a dynamic, situational definitionLinda Schamber, Michael B. Eisenberg, Michael S. Nilan — 1990
• Practical Nonparametric Statistics (2nd ed).Richard A. Groeneveld, W. J. Conover — 1981
• Cumulated gain-based evaluation of IR techniquesKalervo Järvelin, Jaana Kekäläinen — 2002
• Variations in Relevance Judgments and the Evaluation of Retrieval PerformanceRobert Burgin — 1992
• Presenting Results of Experimental Retrieval ComparisonsE. Michael Keen — 1997

Cited by

• On text document classification and retrieval using self-organising maps ; Tekstidokumenttien luokittelusta ja tiedonhausta itseorganisoituvilla kartoillaJyri Saarikoski — 2014
• Task-based Information Seeking and Retrieval in the Patent Domain: Processes and RelationshipsPreben Hansen — 2011
• A Survey of Faceted SearchBifan Wei, Jun Liu, Qinghua Zheng — 2013
• Development of job matching algorithm with collective learningKam Ching Cheng — 2013
• Leveraging social relevance : using social networks to enhance literature access and microblog searchLamjed Ben Jabeur — 2013
• Property of average precision and its generalization: An examination of evaluation indicator for information retrieval experimentsKazuaki Kishida — 2005
• Latent variable modelling of user interaction in image retrievalD. A. Morrison — 2011
• Relevance-based Evaluation of Alignment Approaches: The OAEI 2007 Food Task RevisitedWillem Robert van Hage, Hap Kolb, Guus Schreiber — 2008
• Trademark image retrieval by local featuresPaijit Kochakornjarupong — 2011
• On Search Engine Evaluation MetricsPavel Sirotkin — 2013
• Diversity-aware Evaluation for Paraphrase PatternsHideki Shima, Teruko Mitamura — 2011
• Controlling the Penalty on Late Arrival of Relevant Documents in Information Retrieval Evalution with Graded RelevanceTetsuya Sakai — 2006
• The twist measure for IR evaluation: Taking user's effort into accountNicola Ferro, Gianmaria Silvello, Heikki Keskustalo — 2015
• Novel methods for semantic and aesthetic multimedia retrievalМириам Реди — 2013
• Una teoría cognitiva integral para la recuperación de información: saliendo del entorno del laboratorioPeter Ingwersen, Kalervo Järvelin — 2009
• Rank-biased precision for measurement of retrieval effectivenessAlistair Moffat, Justin Zobel — 2008
• Assessors' search result satisfaction associated with relevance in a scientific domainPeter Ingwersen, Marianne Lykke, Toine Bogers — 2010
• VIRTUE: A visual tool for information retrieval performance evaluation and failure analysisMarco Angelini, Nicola Ferro, Giuseppe Santucci — 2014
• On the Properties of Evaluation Metrics for Finding One Highly Relevant DocumentTetsuya Sakai — 2007
• Evaluation in Music Information RetrievalJulián Urbano, Markus Schedl, Xavier Serra — 2013

Related papers

• Towards automatic generation of relevance judgments for a test collectionMireille Makary, Michael Oakes, Fadi Yamout — 2016
• Group-enhanced rankingYuan Lin, Hongfei Lin, Kan Xu — 2015
• Fine-Grained Relevance Annotations for Multi-Task Document Ranking and Question AnsweringSebastian Hofstätter, Markus Zlabinger, Mete Sertkan — 2020
• Chinese document re-ranking based on automatically acquired term resourceDonghong Ji, Shiju Zhao, Guozheng Xiao — 2009
• Learning To Rank Relevant Documents for Information Retrieval in Bioengineering Text CorporaKowk Sun Cheng, Myoungkyu Song — 2021
• Building a framework for the probability ranking principle by a family of expected weighted rankEdward Kai Fung Dang, Jack Wu, Robert W. P. Luk — 2009