scholarly journals PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them

2021 ◽  
Vol 9 ◽  
pp. 1098-1115
Author(s):  
Patrick Lewis ◽  
Yuxiang Wu ◽  
Linqing Liu ◽  
Pasquale Minervini ◽  
Heinrich Küttler ◽  
...  
Keyword(s):  
Question Answering ◽  
State Of The Art ◽  
High Accuracy ◽  
Test Time ◽  
Open Domain ◽  
Combined System ◽  
Text Corpora ◽  
Degree Of Control ◽  
Conventional Models ◽  
High Degree

Abstract Open-domain Question Answering models that directly leverage question-answer (QA) pairs, such as closed-book QA (CBQA) models and QA-pair retrievers, show promise in terms of speed and memory compared with conventional models which retrieve and read from text corpora. QA-pair retrievers also offer interpretable answers, a high degree of control, and are trivial to update at test time with new knowledge. However, these models fall short of the accuracy of retrieve-and-read systems, as substantially less knowledge is covered by the available QA-pairs relative to text corpora like Wikipedia. To facilitate improved QA-pair models, we introduce Probably Asked Questions (PAQ), a very large resource of 65M automatically generated QA-pairs. We introduce a new QA-pair retriever, RePAQ, to complement PAQ. We find that PAQ preempts and caches test questions, enabling RePAQ to match the accuracy of recent retrieve-and-read models, whilst being significantly faster. Using PAQ, we train CBQA models which outperform comparable baselines by 5%, but trail RePAQ by over 15%, indicating the effectiveness of explicit retrieval. RePAQ can be configured for size (under 500MB) or speed (over 1K questions per second) while retaining high accuracy. Lastly, we demonstrate RePAQ’s strength at selective QA, abstaining from answering when it is likely to be incorrect. This enables RePAQ to “back-off” to a more expensive state-of-the-art model, leading to a combined system which is both more accurate and 2x faster than the state-of-the-art model alone.

scholarly journals Relevance-guided Supervision for OpenQA with ColBERT

2021 ◽  
Vol 9 ◽  
pp. 929-944
Author(s):  
Omar Khattab ◽  
Christopher Potts ◽  
Matei Zaharia
Keyword(s):  
Question Answering ◽  
State Of The Art ◽  
Training Data ◽  
Coarse Grained ◽  
Retrieval Model ◽  
Open Domain ◽  
Weak Supervision ◽  
Fine Grained ◽  
Vector Representations ◽  
Large Corpus

Abstract Systems for Open-Domain Question Answering (OpenQA) generally depend on a retriever for finding candidate passages in a large corpus and a reader for extracting answers from those passages. In much recent work, the retriever is a learned component that uses coarse-grained vector representations of questions and passages. We argue that this modeling choice is insufficiently expressive for dealing with the complexity of natural language questions. To address this, we define ColBERT-QA, which adapts the scalable neural retrieval model ColBERT to OpenQA. ColBERT creates fine-grained interactions between questions and passages. We propose an efficient weak supervision strategy that iteratively uses ColBERT to create its own training data. This greatly improves OpenQA retrieval on Natural Questions, SQuAD, and TriviaQA, and the resulting system attains state-of-the-art extractive OpenQA performance on all three datasets.

You Need Teachers

PEDIATRICS ◽  
1979 ◽  
Vol 64 (1) ◽  
pp. 125-125
Author(s):  
Miles Weinberger
Keyword(s):  
State Of The Art ◽  
Review Article ◽  
Current State ◽  
Children With Asthma ◽  
Degree Of Control ◽  
Considerable Morbidity ◽  
High Degree ◽  
Changing Role ◽  
Excellent Review Article

The excellent review article by Leffert1 and the accompanying commentary by Bergner2 made important points regarding the changing role of the pediatric allergist and the broad requirements for knowledge of any physicians who are to provide specialty care for children with asthma. While the current state of the art allows a high degree of control for this disease,3 considerable morbidity from inadequately treated asthma persists. This situation is unlikely to change rapidly unless departments of pediatrics place a high priority on ensuring that the modern allergist described by Dr. Bergner is on their faculty to teach the current housestaff and provide continuing education for the practitioner; only then will most general pediatricians be able to assume the role envisioned by Dr. Leffert.

scholarly journals Statistical Metaphor Processing

2013 ◽  
Vol 39 (2) ◽  
pp. 301-353 ◽  
Author(s):  
Ekaterina Shutova ◽  
Simone Teufel ◽  
Anna Korhonen
Keyword(s):  
Statistical Model ◽  
Real World ◽  
State Of The Art ◽  
High Accuracy ◽  
Lexical Acquisition ◽  
Open Domain ◽  
Text Model ◽  
Metaphor Interpretation ◽  
Metaphor Processing ◽  
Minimally Supervised

Metaphor is highly frequent in language, which makes its computational processing indispensable for real-world NLP applications addressing semantic tasks. Previous approaches to metaphor modeling rely on task-specific hand-coded knowledge and operate on a limited domain or a subset of phenomena. We present the first integrated open-domain statistical model of metaphor processing in unrestricted text. Our method first identifies metaphorical expressions in running text and then paraphrases them with their literal paraphrases. Such a text-to-text model of metaphor interpretation is compatible with other NLP applications that can benefit from metaphor resolution. Our approach is minimally supervised, relies on the state-of-the-art parsing and lexical acquisition technologies (distributional clustering and selectional preference induction), and operates with a high accuracy.

scholarly journals Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study

2021 ◽  
Vol 9 ◽  
pp. 1032-1046
Author(s):  
Xiangyang Mou ◽  
Chenghao Yang ◽  
Mo Yu ◽  
Bingsheng Yao ◽  
Xiaoxiao Guo ◽  
...  
Keyword(s):  
Quantitative Analysis ◽  
Question Answering ◽  
State Of The Art ◽  
Cutting Edge ◽  
Human Studies ◽  
Open Domain ◽  
Level Performance ◽  
Similar Task ◽  
Comprehensive Study

Abstract Recent advancements in open-domain question answering (ODQA), that is, finding answers from large open-domain corpus like Wikipedia, have led to human-level performance on many datasets. However, progress in QA over book stories (Book QA) lags despite its similar task formulation to ODQA. This work provides a comprehensive and quantitative analysis about the difficulty of Book QA: (1) We benchmark the research on the NarrativeQA dataset with extensive experiments with cutting-edge ODQA techniques. This quantifies the challenges Book QA poses, as well as advances the published state-of-the-art with a ∼7% absolute improvement on ROUGE-L. (2) We further analyze the detailed challenges in Book QA through human studies.1 Our findings indicate that the event-centric questions dominate this task, which exemplifies the inability of existing QA models to handle event-oriented scenarios.

scholarly journals MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering

2021 ◽  
Vol 9 ◽  
pp. 1389-1406
Author(s):  
Shayne Longpre ◽  
Yi Lu ◽  
Joachim Daiber
Keyword(s):  
Question Answering ◽  
State Of The Art ◽  
Linguistically Diverse ◽  
Data Representation ◽  
Independent Data ◽  
Open Domain ◽  
Low Resource ◽  
Art Methods ◽  
Questions And Answers ◽  
Cross Lingual

Abstract Progress in cross-lingual modeling depends on challenging, realistic, and diverse evaluation sets. We introduce Multilingual Knowledge Questions and Answers (MKQA), an open- domain question answering evaluation set comprising 10k question-answer pairs aligned across 26 typologically diverse languages (260k question-answer pairs in total). Answers are based on heavily curated, language- independent data representation, making results comparable across languages and independent of language-specific passages. With 26 languages, this dataset supplies the widest range of languages to-date for evaluating question answering. We benchmark a variety of state- of-the-art methods and baselines for generative and extractive question answering, trained on Natural Questions, in zero shot and translation settings. Results indicate this dataset is challenging even in English, but especially in low-resource languages.1

A Yes/No Answer Generator Based on Sentiment-Word Scores in Biomedical Question Answering

2020 ◽  
pp. 103-116
Author(s):  
Mourad Sarrouti ◽  
Said Ouatik El Alaoui
Keyword(s):  
Question Answering ◽  
State Of The Art ◽  
Biomedical Domain ◽  
Open Domain ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Current State ◽  
Sentiment Score ◽  
Speech Tagging ◽  
Sentiment Word

Background and Objective: Yes/no question answering (QA) in open-domain is a longstanding challenge widely studied over the last decades. However, it still requires further efforts in the biomedical domain. Yes/no QA aims at answering yes/no questions, which are seeking for a clear “yes” or “no” answer. In this paper, we present a novel yes/no answer generator based on sentiment-word scores in biomedical QA. Methods: In the proposed method, we first use the Stanford CoreNLP for tokenization and part-of-speech tagging all relevant passages to a given yes/no question. We then assign a sentiment score based on SentiWordNet to each word of the passages. Finally, the decision on either the answers “yes” or “no” is based on the obtained sentiment-passages score: “yes” for a positive final sentiment-passages score and “no” for a negative one. Results: Experimental evaluations performed on BioASQ collections show that the proposed method is more effective as compared with the current state-of-the-art method, and significantly outperforms it by an average of 15.68% in terms of accuracy.

scholarly journals Using Social Signals to Predict Shoplifting: A Transparent Approach to a Sensitive Activity Analysis Problem

Sensors ◽  
2021 ◽  
Vol 21 (20) ◽  
pp. 6812
Author(s):  
Shane Reid ◽  
Sonya Coleman ◽  
Philip Vance ◽  
Dermot Kerr ◽  
Siobhan O’Neill
Keyword(s):  
Signal Processing ◽  
State Of The Art ◽  
High Accuracy ◽  
Black Box ◽  
Social Signals ◽  
Social Signal Processing ◽  
Legal Cases ◽  
Current State ◽  
Social Signal ◽  
High Degree

Retail shoplifting is one of the most prevalent forms of theft and has accounted for over one billion GBP in losses for UK retailers in 2018. An automated approach to detecting behaviours associated with shoplifting using surveillance footage could help reduce these losses. Until recently, most state-of-the-art vision-based approaches to this problem have relied heavily on the use of black box deep learning models. While these models have been shown to achieve very high accuracy, this lack of understanding on how decisions are made raises concerns about potential bias in the models. This limits the ability of retailers to implement these solutions, as several high-profile legal cases have recently ruled that evidence taken from these black box methods is inadmissible in court. There is an urgent need to develop models which can achieve high accuracy while providing the necessary transparency. One way to alleviate this problem is through the use of social signal processing to add a layer of understanding in the development of transparent models for this task. To this end, we present a social signal processing model for the problem of shoplifting prediction which has been trained and validated using a novel dataset of manually annotated shoplifting videos. The resulting model provides a high degree of understanding and achieves accuracy comparable with current state of the art black box methods.

A Yes/No Answer Generator Based on Sentiment-Word Scores in Biomedical Question Answering

2017 ◽  
Vol 12 (3) ◽  
pp. 62-74 ◽  
Author(s):  
Mourad Sarrouti ◽  
Said Ouatik El Alaoui
Keyword(s):  
Question Answering ◽  
State Of The Art ◽  
Biomedical Domain ◽  
Open Domain ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
Current State ◽  
Sentiment Score ◽  
Speech Tagging ◽  
Sentiment Word

Background and Objective: Yes/no question answering (QA) in open-domain is a longstanding challenge widely studied over the last decades. However, it still requires further efforts in the biomedical domain. Yes/no QA aims at answering yes/no questions, which are seeking for a clear “yes” or “no” answer. In this paper, we present a novel yes/no answer generator based on sentiment-word scores in biomedical QA. Methods: In the proposed method, we first use the Stanford CoreNLP for tokenization and part-of-speech tagging all relevant passages to a given yes/no question. We then assign a sentiment score based on SentiWordNet to each word of the passages. Finally, the decision on either the answers “yes” or “no” is based on the obtained sentiment-passages score: “yes” for a positive final sentiment-passages score and “no” for a negative one. Results: Experimental evaluations performed on BioASQ collections show that the proposed method is more effective as compared with the current state-of-the-art method, and significantly outperforms it by an average of 15.68% in terms of accuracy.

JUST.ASK — A MULTI-PRONGED APPROACH TO QUESTION ANSWERING

2013 ◽  
Vol 22 (01) ◽  
pp. 1250036 ◽  
Author(s):  
ANA CRISTINA MENDES ◽  
LUÍSA COHEUR ◽  
JOÃO SILVA ◽  
HUGO RODRIGUES
Keyword(s):  
Scientific Community ◽  
Question Answering ◽  
State Of The Art ◽  
Information Source ◽  
Test Collection ◽  
Open Domain ◽  
Research Areas ◽  
Detailed Evaluation ◽  
Flexible Architecture ◽  
Recent Developments

In the last decades, several research areas experienced key improvements due to the appearance of numerous tools made available to the scientific community. For instance, Moses plays an important role in recent developments in machine translation and Lucene is, with no doubt, a widespread tool in information retrieval. The existence of these systems allows an easy development of baselines and, therefore, researchers can focus on improving preliminary results, instead of spending time in developing software from scratch. In addition, the existence of appropriate test collections leads to a straightforward comparison of systems and of their specific components. In this paper we describe Just.Ask, a multi-pronged approach to open-domain question answering. Just.Ask combines rule- with machine learning-based components and implements several state-of-the-art strategies in question answering. Also, it has a flexible architecture that allows for further extensions. Moreover, in this paper we report a detailed evaluation of each one of Just.Ask components. The evaluation is split into two parts: in the first one, we use a manually built test collection — the GoldWebQA — that intends to evaluate Just.Ask performance when the information source in use is the Web, without having to deal with its constant changes; in the second one, we use a set of questions gathered from the TREC evaluation forum, having a closed text collection, locally indexed and stored, as information source. Therefore, this paper contributes with a benchmark for research on question answering, since both Just.Ask and the GoldWebQA corpus are freely available for the scientific community.

Sign in / Sign up

Export Citation Format

Share Document