Multimodal Natural Language Query System For Processing And Analyzing Voice And Proximity-Based Queries

A study of the use of and and or for specifying intersection and union relationships between conjoined qualifiers of varying characteristics was conducted using a simulated natural language query system. Subjects always used or correctly to indicate union but and was used to indicate both union and intersection. Interpretation rules were defined that could be used to clarify the intended meaning for and for some but not all of the cases. The results indicated subjects could implicitly learn to be more precise. These results suggest that natural language interfaces can be built to recognize ambiguous input and should prompt users for clarification. Simple syntactic elements that would distinguish the meaning of and can be defined and taught to users.

Download Full-text

User oriented data base query with the ROBOT natural language query system

International Journal of Man-Machine Studies ◽

10.1016/s0020-7373(77)80037-0 ◽

1977 ◽

Vol 9 (6) ◽

pp. 697-713 ◽

Cited By ~ 59

Author(s):

L.R. Harris

Keyword(s):

Natural Language ◽

Data Base ◽

Natural Language Query ◽

Query System

Download Full-text

Single-shot Semantic Matching Network for Moment Localization in Videos

ACM Transactions on Multimedia Computing Communications and Applications ◽

10.1145/3441577 ◽

2021 ◽

Vol 17 (3) ◽

pp. 1-14

Author(s):

Xinfang Liu ◽

Xiushan Nie ◽

Junya Teng ◽

Li Lian ◽

Yilong Yin

Keyword(s):

Natural Language ◽

Fixed Number ◽

Single Shot ◽

Semantic Matching ◽

Matching Network ◽

Attention Model ◽

Semantic Relationships ◽

Natural Language Query ◽

Benchmark Datasets ◽

Short Memory

Moment localization in videos using natural language refers to finding the most relevant segment from videos given a natural language query. Most of the existing methods require video segment candidates for further matching with the query, which leads to extra computational costs, and they may also not locate the relevant moments under any length evaluated. To address these issues, we present a lightweight single-shot semantic matching network (SSMN) to avoid the complex computations required to match the query and the segment candidates, and the proposed SSMN can locate moments of any length theoretically. Using the proposed SSMN, video features are first uniformly sampled to a fixed number, while the query sentence features are generated and enhanced by GloVe, long-term short memory (LSTM), and soft-attention modules. Subsequently, the video features and sentence features are fed to an enhanced cross-modal attention model to mine the semantic relationships between vision and language. Finally, a score predictor and a location predictor are designed to locate the start and stop indexes of the query moment. We evaluate the proposed method on two benchmark datasets and the experimental results demonstrate that SSMN outperforms state-of-the-art methods in both precision and efficiency.

Download Full-text