Practical Language Processing for Virtual Humans

NPCEditor is a system for building a natural language processing component for virtual humans capable of engaging a user in spoken dialog on a limited domain. It uses a statistical language classification technology for mapping from user's text input to system responses. NPCEditor provides a user-friendly editor for creating effective virtual humans quickly. It has been deployed as a part of various virtual human systems in several applications.

The Dialect Features and Distinguish Approach of Scottish English (Note 1)

If standard official language is a glass of water, the dialect is like soup with a flavor of your hometown. The locals in Scotland pride themselves on speaking English with a Scottish accent, but its obscurity always leaves us at a loss. In order to understand Scottish English dialects better, this article first briefly analyzes the language classification in Scotland. Then, using empirical research methods, interviews with the 10 most representative speakers of Scottish English dialects are selected from the eight regions of Scotland. The audio is used as a research corpus. The corpus is 49 minutes and 17 seconds long, with a total number of 9293 words. It focuses on the analysis of the accent, vocabulary, and grammatical structure of the Scottish English dialect. Finally, suggestions are made on Scottish English listening and discerning ability training.

SwitchNet: Learning to switch for word-level language identification in code-mixed social media text

Word-level language identification is an essential prerequisite for extracting useful information from code-mixed social media content. Previous studies in word-level language identification show two important observations. First, the local context is an important indicator of the language of a word when a word is valid in multiple languages. Second, considering the word in isolation from its context leads to more effective language classification when a word is borrowed or embedded into sentences of other languages. In this paper, we propose a framework for language identification that makes use of a dynamic switching mechanism for effective language classification of both words that are borrowed or embedded from other languages as well as words that are valid in multiple languages. For a given input, the proposed switching mechanism makes a dynamic decision to bias its prediction either towards the prediction obtained by the contextual information or that obtained by the word in isolation. In contrast to existing studies that rely upon large amounts of annotated data for robust performance in a multilingual environment, the proposed approach uses minimal annotated resources and no external resources, making it easily extendible to newer languages. Evaluation over a corpus of transliterated Facebook comments shows that the proposed approach outperforms its baseline counterparts: classification based on the contextual information, classification based on the word in isolation, as well as an ensemble of the two classifiers.