Enhancing Clustering and Labeling for Large-Scale Information Retrieval Systems

2021 ◽  
Author(s):  
Xuemei Gong
Keyword(s):  
Information Retrieval ◽  
Large Scale ◽  
Retrieval Systems ◽  
Information Retrieval Systems

IR Research and Innovation in Commercial Online Systems: An Exploratory Survey

2013 ◽  
Author(s):  
Sherry Koshman ◽  
Edie Rasmussen
Keyword(s):  
Information Retrieval ◽  
Large Scale ◽  
Interactive Systems ◽  
Information Industry ◽  
Online Systems ◽  
Research And Innovation ◽  
Retrieval Systems ◽  
Information Retrieval Systems ◽  
Operational Systems ◽  
Mcgill University

From the 1994 CAIS Conference: The Information Industry in Transition McGill University, Montreal, Quebec. May 25 - 27, 1994."Conventional" information retrieval systems (IRS), originating in the research of the 11950s and 1960s, are based on keyword matching and the application of Boolean operators to produce a set of retrieved documents from a database. In the ensuing years, research in information retrieval has identified a number of innovations (for example, automatic weighting of terms, ranked output, and relevance feedback) which have the potential to significantly enhance the performance of IRS, though commercial vendors have been slow to incorporate these changes into their systems. This was the situation in 1988 which led Radecki, in a special issue of Information Processing & Management, to examine the potential for improvements in conventional Boolean retrieval systems, and explore the reasons why these improvements had not been implemented in operational systems. Over the last five years, this position has begun to change as commercial vendors such as Dialog, Dow Jones, West Publishing, and Mead have implemented new, non-Boolean features in their systems, including natural language input, weighted keyword terms, and document ranking. This paper identifies some of the significant findings of IR research and compares them to the implementation of non-Boolean features in such systems. The preliminary survey of new features in commercial systems suggests the need for new methods of evaluation, including the development of evalutation measures appropriate to large-scale, interactive systems.

scholarly journals Large-scale image search with text for information retrieval

2021 ◽  
Vol 4 (1) ◽  
pp. 87-89
Author(s):  
Janardan Bhatta
Keyword(s):  
Information Retrieval ◽  
Language Processing ◽  
Large Scale ◽  
Image Feature ◽  
Image Search ◽  
Search Results ◽  
Retrieval Systems ◽  
Information Retrieval Systems ◽  
Text Features ◽  
Text Query

Searching images in a large database is a major requirement in Information Retrieval Systems. Expecting image search results based on a text query is a challenging task. In this paper, we leverage the power of Computer Vision and Natural Language Processing in Distributed Machines to lower the latency of search results. Image pixel features are computed based on contrastive loss function for image search. Text features are computed based on the Attention Mechanism for text search. These features are aligned together preserving the information in each text and image feature. Previously, the approach was tested only in multilingual models. However, we have tested it in image-text dataset and it enabled us to search in any form of text or images with high accuracy.

scholarly journals Managing tail latency in large scale information retrieval systems

2020 ◽  
Vol 54 (1) ◽  
pp. 1-2
Author(s):  
Joel M. Mackenzie
Keyword(s):  
Information Retrieval ◽  
User Experience ◽  
Large Scale ◽  
Response Times ◽  
Smart Devices ◽  
Worst Case ◽  
Retrieval Systems ◽  
Trade Offs ◽  
Efficiency And Effectiveness ◽  
Information Retrieval Systems

As both the availability of internet access and the prominence of smart devices continue to increase, data is being generated at a rate faster than ever before. This massive increase in data production comes with many challenges, including efficiency concerns for the storage and retrieval of such large-scale data. However, users have grown to expect the sub-second response times that are common in most modern search engines, creating a problem --- how can such large amounts of data continue to be served efficiently enough to satisfy end users? This dissertation investigates several issues regarding tail latency in large-scale information retrieval systems. Tail latency corresponds to the high percentile latency that is observed from a system --- in the case of search, this latency typically corresponds to how long it takes for a query to be processed. In particular, keeping tail latency as low as possible translates to a good experience for all users, as tail latency is directly related to the worst-case latency and hence, the worst possible user experience. The key idea in targeting tail latency is to move from questions such as "what is the median latency of our search engine?" to questions which more accurately capture user experience such as "how many queries take more than 200 ms to return answers?" or "what is the worst case latency that a user may be subject to, and how often might it occur?" While various strategies exist for efficiently processing queries over large textual corpora, prior research has focused almost entirely on improvements to the average processing time or cost of search systems. As a first contribution, we examine some state-of-the-art retrieval algorithms for two popular index organizations, and discuss the trade-offs between them, paying special attention to the notion of tail latency. This research uncovers a number of observations that are subsequently leveraged for improved search efficiency and effectiveness. We then propose and solve a new problem, which involves processing a number of related query variations together, known as multi-queries , to yield higher quality search results. We experiment with a number of algorithmic approaches to efficiently process these multi-queries, and report on the cost, efficiency, and effectiveness trade-offs present with each. Finally, we examine how predictive models can be used to improve the tail latency and end-to-end cost of a commonly used multi-stage retrieval architecture without impacting result effectiveness. By combining ideas from numerous areas of information retrieval, we propose a prediction framework which can be used for training and evaluating several efficiency/effectiveness trade-off parameters, resulting in improved trade-offs between cost, result quality, and tail latency.

Relevance Predictability in Information Retrieval Systems

1967 ◽  
Vol 06 (02) ◽  
pp. 45-51 ◽  
Author(s):  
A. Kent ◽  
J. Belzer ◽  
M. Kuhfeerst ◽  
E. D. Dym ◽  
D. L. Shirey ◽  
...  
Keyword(s):  
Information Retrieval ◽  
Experimental Conditions ◽  
Treatment Groups ◽  
Retrieval Systems ◽  
Significant Difference ◽  
Information Retrieval Systems ◽  
High Predictability ◽  
Intermediate Response ◽  
Quantitative Indicators ◽  
Level Of Processing

An experiment is described which attempts to derive quantitative indicators regarding the potential relevance predictability of the intermediate stimuli used to represent documents in information retrieval systems. In effect, since the decision to peruse an entire document is often predicated upon the examination of one »level of processing« of the document (e.g., the citation and/or abstract), it became interesting to analyze the properties of what constitutes »relevance«. However, prior to such an analysis, an even more elementary step had to be made, namely, to determine what portions of a document should be examined.An evaluation of the ability of intermediate response products (IRPs), functioning as cues to the information content of full documents, to predict the relevance determination that would be subsequently made on these documents by motivated users of information retrieval systems, was made under controlled experimental conditions. The hypothesis that there might be other intermediate response products (selected extracts from the document, i.e., first paragraph, last paragraph, and the combination of first and last paragraph), that would be as representative of the full document as the traditional IRPs (citation and abstract) was tested systematically. The results showed that:1. there is no significant difference among the several IRP treatment groups on the number of cue evaluations of relevancy which match the subsequent user relevancy decision on the document;2. first and last paragraph combinations have consistently predicted relevancy to a higher degree than the other IRPs;3. abstracts were undistinguished as predictors; and4. the apparent high predictability rating for citations was not substantive.Some of these results are quite different than would be expected from previous work with unmotivated subjects.

Performance of eight information retrieval systems in the Spanish web

2005 ◽  
Vol 14 (5) ◽  
pp. 335-346
Author(s):  
Por Carlos Benito Amat ◽  
Por Carlos Benito Amat
Keyword(s):  
Information Retrieval ◽  
Retrieval Systems ◽  
Information Retrieval Systems

A Dialectical Approach to Search Engine Evaluation

Libri ◽  
2020 ◽  
Vol 70 (3) ◽  
pp. 227-237
Author(s):  
Mahdi Zeynali-Tazehkandi ◽  
Mohsen Nowkarizi
Keyword(s):  
Information Retrieval ◽  
Information Science ◽  
Library And Information Science ◽  
Related Literature ◽  
Philosophical Foundations ◽  
Dialectical Approach ◽  
Retrieval Systems ◽  
The World ◽  
Information Retrieval Systems ◽  
Oriented Approach

AbstractEvaluation of information retrieval systems is a fundamental topic in Library and Information Science. The aim of this paper is to connect the system-oriented and the user-oriented approaches to relevant philosophical schools. By reviewing the related literature, it was found that the evaluation of information retrieval systems is successful if it benefits from both system-oriented and user-oriented approaches (composite). The system-oriented approach is rooted in Parmenides’ philosophy of stability (immovable) which Plato accepts and attributes to the world of forms; the user-oriented approach is rooted in Heraclitus’ flux philosophy (motion) which Plato defers and attributes to the tangible world. Thus, using Plato’s theory is a comprehensive approach for recognizing the concept of relevance. The theoretical and philosophical foundations determine the type of research methods and techniques. Therefore, Plato’s dialectical method is an appropriate composite method for evaluating information retrieval systems.

Sign in / Sign up

Export Citation Format

Share Document