An empirical study of financial incentivized question answering in social websites

An Empirical Study of Content Understanding in Conversational Question Answering

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6257 ◽

2020 ◽

Vol 34 (05) ◽

pp. 7578-7585

Author(s):

Ting-Rui Chiang ◽

Hao-Tong Ye ◽

Yun-Nung Chen

Keyword(s):

Natural Language Processing ◽

Empirical Study ◽

Language Processing ◽

Question Answering ◽

Source Code ◽

Content Understanding ◽

Question Answering Systems ◽

Benchmark Datasets ◽

Context Free ◽

Answering Questions

With a lot of work about context-free question answering systems, there is an emerging trend of conversational question answering models in the natural language processing field. Thanks to the recently collected datasets, including QuAC and CoQA, there has been more work on conversational question answering, and recent work has achieved competitive performance on both datasets. However, to best of our knowledge, two important questions for conversational comprehension research have not been well studied: 1) How well can the benchmark dataset reflect models' content understanding? 2) Do the models well utilize the conversation content when answering questions? To investigate these questions, we design different training settings, testing settings, as well as an attack to verify the models' capability of content understanding on QuAC and CoQA. The experimental results indicate some potential hazards in the benchmark datasets, QuAC and CoQA, for conversational comprehension research. Our analysis also sheds light on both what models may learn and how datasets may bias the models. With deep investigation of the task, it is believed that this work can benefit the future progress of conversation comprehension. The source code is available at https://github.com/MiuLab/CQA-Study.

Download Full-text

An Empirical Study on Deep Neural Network Models for Chinese Dialogue Generation

Symmetry ◽

10.3390/sym12111756 ◽

2020 ◽

Vol 12 (11) ◽

pp. 1756

Author(s):

Zhe Li ◽

Mieradilijiang Maimaiti ◽

Jiabao Sheng ◽

Zunwang Ke ◽

Wushour Silamu ◽

...

Keyword(s):

Neural Network ◽

Empirical Study ◽

Deep Neural Network ◽

Question Answering ◽

State Of The Art ◽

Network Models ◽

Superior Performance ◽

Research Progress ◽

Neural Network Models ◽

Wide Range

The task of dialogue generation has attracted increasing attention due to its diverse downstream applications, such as question-answering systems and chatbots. Recently, the deep neural network (DNN)-based dialogue generation models have achieved superior performance against conventional models utilizing statistical machine learning methods. However, despite that an enormous number of state-of-the-art DNN-based models have been proposed, there lacks detailed empirical comparative analysis for them on the open Chinese corpus. As a result, relevant researchers and engineers might find it hard to get an intuitive understanding of the current research progress. To address this challenge, we conducted an empirical study for state-of-the-art DNN-based dialogue generation models in various Chinese corpora. Specifically, extensive experiments were performed on several well-known single-turn and multi-turn dialogue corpora, including KdConv, Weibo, and Douban, to evaluate a wide range of dialogue generation models that are based on the symmetrical architecture of Seq2Seq, RNNSearch, transformer, generative adversarial nets, and reinforcement learning respectively. Moreover, we paid special attention to the prevalent pre-trained model for the quality of dialogue generation. Their performances were evaluated by four widely-used metrics in this area: BLEU, pseudo, distinct, and rouge. Finally, we report a case study to show example responses generated by these models separately.

Download Full-text