A Comparative Study of Performance Measures for Information Retrieval Systems

2006 ◽  
Author(s):  
Xiannong Meng
Keyword(s):  
Information Retrieval ◽  
Comparative Study ◽  
Performance Measures ◽  
Retrieval Systems ◽  
Information Retrieval Systems

How many performance measures to evaluate information retrieval systems?

2011 ◽  
Vol 30 (3) ◽  
pp. 693-713 ◽  
Author(s):  
Alain Baccini ◽  
Sébastien Déjean ◽  
Laetitia Lafage ◽  
Josiane Mothe
Keyword(s):  
Information Retrieval ◽  
Performance Measures ◽  
Retrieval Systems ◽  
Information Retrieval Systems

Relevance Predictability in Information Retrieval Systems

1967 ◽  
Vol 06 (02) ◽  
pp. 45-51 ◽  
Author(s):  
A. Kent ◽  
J. Belzer ◽  
M. Kuhfeerst ◽  
E. D. Dym ◽  
D. L. Shirey ◽  
...  
Keyword(s):  
Information Retrieval ◽  
Experimental Conditions ◽  
Treatment Groups ◽  
Retrieval Systems ◽  
Significant Difference ◽  
Information Retrieval Systems ◽  
High Predictability ◽  
Intermediate Response ◽  
Quantitative Indicators ◽  
Level Of Processing

An experiment is described which attempts to derive quantitative indicators regarding the potential relevance predictability of the intermediate stimuli used to represent documents in information retrieval systems. In effect, since the decision to peruse an entire document is often predicated upon the examination of one »level of processing« of the document (e.g., the citation and/or abstract), it became interesting to analyze the properties of what constitutes »relevance«. However, prior to such an analysis, an even more elementary step had to be made, namely, to determine what portions of a document should be examined.An evaluation of the ability of intermediate response products (IRPs), functioning as cues to the information content of full documents, to predict the relevance determination that would be subsequently made on these documents by motivated users of information retrieval systems, was made under controlled experimental conditions. The hypothesis that there might be other intermediate response products (selected extracts from the document, i.e., first paragraph, last paragraph, and the combination of first and last paragraph), that would be as representative of the full document as the traditional IRPs (citation and abstract) was tested systematically. The results showed that:1. there is no significant difference among the several IRP treatment groups on the number of cue evaluations of relevancy which match the subsequent user relevancy decision on the document;2. first and last paragraph combinations have consistently predicted relevancy to a higher degree than the other IRPs;3. abstracts were undistinguished as predictors; and4. the apparent high predictability rating for citations was not substantive.Some of these results are quite different than would be expected from previous work with unmotivated subjects.

Performance of eight information retrieval systems in the Spanish web

2005 ◽  
Vol 14 (5) ◽  
pp. 335-346
Author(s):  
Por Carlos Benito Amat ◽  
Por Carlos Benito Amat
Keyword(s):  
Information Retrieval ◽  
Retrieval Systems ◽  
Information Retrieval Systems

A Dialectical Approach to Search Engine Evaluation

Libri ◽  
2020 ◽  
Vol 70 (3) ◽  
pp. 227-237
Author(s):  
Mahdi Zeynali-Tazehkandi ◽  
Mohsen Nowkarizi
Keyword(s):  
Information Retrieval ◽  
Information Science ◽  
Library And Information Science ◽  
Related Literature ◽  
Philosophical Foundations ◽  
Dialectical Approach ◽  
Retrieval Systems ◽  
The World ◽  
Information Retrieval Systems ◽  
Oriented Approach

AbstractEvaluation of information retrieval systems is a fundamental topic in Library and Information Science. The aim of this paper is to connect the system-oriented and the user-oriented approaches to relevant philosophical schools. By reviewing the related literature, it was found that the evaluation of information retrieval systems is successful if it benefits from both system-oriented and user-oriented approaches (composite). The system-oriented approach is rooted in Parmenides’ philosophy of stability (immovable) which Plato accepts and attributes to the world of forms; the user-oriented approach is rooted in Heraclitus’ flux philosophy (motion) which Plato defers and attributes to the tangible world. Thus, using Plato’s theory is a comprehensive approach for recognizing the concept of relevance. The theoretical and philosophical foundations determine the type of research methods and techniques. Therefore, Plato’s dialectical method is an appropriate composite method for evaluating information retrieval systems.

Sign in / Sign up

Export Citation Format

Share Document