Language modeling approaches to question answering

2021 ◽  
Author(s):  
Protima Banerjee
Keyword(s):  
Question Answering ◽  
Language Modeling ◽  
Modeling Approaches

Effects of language modeling on speech-driven question answering

2004 ◽  
Author(s):  
Katsunobu Itou ◽  
Atsushi Fujii ◽  
Tomoyosi Akiba
Keyword(s):  
Question Answering ◽  
Language Modeling

scholarly journals Improving Sentence Retrieval Using Sequence Similarity

2020 ◽  
Vol 10 (12) ◽  
pp. 4316 ◽  
Author(s):  
Ivan Boban ◽  
Alen Doko ◽  
Sven Gotovac
Keyword(s):  
Question Answering ◽  
Sequence Similarity ◽  
Novelty Detection ◽  
Document Retrieval ◽  
Language Modeling ◽  
Information Need ◽  
Partial Matching ◽  
Retrieval Technique ◽  
Sentence Retrieval ◽  
Using Data

Sentence retrieval is an information retrieval technique that aims to find sentences corresponding to an information need. It is used for tasks like question answering (QA) or novelty detection. Since it is similar to document retrieval but with a smaller unit of retrieval, methods for document retrieval are also used for sentence retrieval like term frequency—inverse document frequency (TF-IDF), BM 25 , and language modeling-based methods. The effect of partial matching of words to sentence retrieval is an issue that has not been analyzed. We think that there is a substantial potential for the improvement of sentence retrieval methods if we consider this approach. We adapted TF-ISF, BM 25 , and language modeling-based methods to test the partial matching of terms through combining sentence retrieval with sequence similarity, which allows matching of words that are similar but not identical. All tests were conducted using data from the novelty tracks of the Text Retrieval Conference (TREC). The scope of this paper was to find out if such approach is generally beneficial to sentence retrieval. However, we did not examine in depth how partial matching helps or hinders the finding of relevant sentences.

scholarly journals Identification of Code-Switched Sentences and Words Using Language Modeling Approaches

2013 ◽  
Vol 2013 ◽  
pp. 1-7 ◽  
Author(s):  
Liang-Chih Yu ◽  
Wei-Cheng He ◽  
Wei-Nan Chien ◽  
Yuen-Hsien Tseng
Keyword(s):  
Second Language ◽  
Language Processing ◽  
Intelligent Systems ◽  
Language Modeling ◽  
Code Switching ◽  
Experimental Results ◽  
Second Language Processing ◽  
Significant Challenge ◽  
Modeling Approaches ◽  
Modeling Approach

Globalization and multilingualism contribute to code-switching—the phenomenon in which speakers produce utterances containing words or expressions from a second language. Processing code-switched sentences is a significant challenge for multilingual intelligent systems. This study proposes a language modeling approach to the problem of code-switching language processing, dividing the problem into two subtasks: the detection of code-switched sentences and the identification of code-switched words in sentences. A code-switched sentence is detected on the basis of whether it contains words or phrases from another language. Once the code-switched sentences are identified, the positions of the code-switched words in the sentences are then identified. Experimental results show that the language modeling approach achieved anF-measure of 80.43% and an accuracy of 79.01% for detecting Mandarin-Taiwanese code-switched sentences. For the identification of code-switched words, the word-based and POS-based models, respectively, achievedF-measures of 41.09% and 53.08%.

Sign in / Sign up

Export Citation Format

Share Document