scholarly journals Developing Language-Specific Models Using a Neural Architecture Search

2021 ◽  
Vol 11 (21) ◽  
pp. 10324
Author(s):  
YongSuk Yoo ◽  
Kang-moon Park

This paper applies the neural architecture search (NAS) method to Korean and English grammaticality judgment tasks. Based on the previous research, which only discusses the application of NAS on a Korean dataset, we extend the method to English grammatical tasks and compare the resulting two architectures from Korean and English. Since complex syntactic operations exist beneath the word order that is computed, the two different resulting architectures out of the automated NAS language modeling provide an interesting testbed for future research. To the extent of our knowledge, the methodology adopted here has not been tested in the literature. Crucially, the resulting structure of the NAS application shows an unexpected design for human experts. Furthermore, NAS has generated different models for Korean and English, which have different syntactic operations.

2020 ◽  
Vol 10 (10) ◽  
pp. 3457
Author(s):  
Kang-moon Park ◽  
Donghoon Shin ◽  
Yongsuk Yoo

In this paper, we apply the neural architecture search (NAS) method to Korean grammaticality judgment tasks. Since the word order of a language is the final result of complex syntactic operations, a successful neural architecture search in linguistic data suggests that NAS can automate language model designing. Although NAS application to language has been suggested in the literature, we add a novel dataset that contains Korean-specific linguistic operations, which adds great complexity in the patterns. The result of the experiment suggests that NAS provides an architecture for the language. Interestingly, NAS has suggested an unprecedented structure that would not be designed manually. Research on the final topology of the architecture is the topic of our future research.


2021 ◽  
Vol 54 (4) ◽  
pp. 1-34
Author(s):  
Pengzhen Ren ◽  
Yun Xiao ◽  
Xiaojun Chang ◽  
Po-yao Huang ◽  
Zhihui Li ◽  
...  

Deep learning has made substantial breakthroughs in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers’ prior knowledge and experience. And due to the limitations of humans’ inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as much as possible and let the algorithm automatically design the neural architecture. Neural Architecture Search ( NAS ) is just such a revolutionary algorithm, and the related research work is complicated and rich. Therefore, a comprehensive and systematic survey on the NAS is essential. Previously related surveys have begun to classify existing work mainly based on the key components of NAS: search space, search strategy, and evaluation strategy. While this classification method is more intuitive, it is difficult for readers to grasp the challenges and the landmark work involved. Therefore, in this survey, we provide a new perspective: beginning with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then providing solutions for subsequent related research work. In addition, we conduct a detailed and comprehensive analysis, comparison, and summary of these works. Finally, we provide some possible future research directions.


2004 ◽  
Vol 31 (3) ◽  
pp. 713-737 ◽  
Author(s):  
MICHELE KAIL

This study examined the on-line processing of French sentences in a grammaticality judgment experiment. Three age groups of French children (mean age: 6;8, 8;6 and 10;10 years) and a group of adults were asked to detect grammatical violations as quickly as possible. Three factors were studied: the violation type: agreement violations (number and gender) vs. word order violations; the violation position: early vs. late in the sentence; the target type of the violations: intra vs. interphrasal. An example of an early interphrasal verbal agreement violation follows: ‘Chaque semaine la voisine remplissent le frigo après avoir fait les courses au marché’ (Every week the neighbour fill the fridge after shopping at the market). The main developmental results were the following: not surprisingly, children were always slower than adults in the detection of grammatical violations. At each age level, morphological violations were more rapidly detected than word order violations. Each age group was faster at judging sentences with later occurring violations and the position effect was especially strong in the youngest groups. Finally, intraphrasal violations were more rapidly detected than interphrasal ones, this effect being observed only in the oldest groups (i.e. 10;10 years and adults). The results were compared to previous on-line data obtained in modern Greek (Kail & Diakogiorgi, 1998) showing strong similarities, even though Greek is a very rich morphological language. These results are discussed within the framework of the Competition Model, outlining the necessity to incorporate new processing constraints into the model.


2009 ◽  
Vol 31 (1) ◽  
pp. 167-207 ◽  
Author(s):  
SILVINA MONTRUL

ABSTRACTRecent studies of heritage speakers, many of whom possess incomplete knowledge of their family language, suggest that these speakers may be linguistically superior to second language (L2) learners only in phonology but not in morphosyntax. This study reexamines this claim by focusing on knowledge of clitic pronouns and word order in 24 L2 learners and 24 Spanish heritage speakers. Results of an oral production task, a written grammaticality judgment task, and a speeded comprehension task showed that, overall, heritage speakers seem to possess more nativelike knowledge of Spanish than their L2 counterparts. Implications for theories that stress the role of age and experience in L2 ultimate attainment and for the field of heritage language acquisition and teaching are discussed.


2020 ◽  
Vol 24 (1) ◽  
pp. 96-116
Author(s):  
Sergei Monakhov

There is little doubt that one of the most important areas of future research within the framework of Construction Grammar will be the comparative study of constructions in different languages of the world. One significant gain that modern Construction Grammar can make thanks to the cross-linguistic perspective is finding a clue to some contradictory cases of construction alternation. The aim of the present paper is to communicate the results of a case study of two pairs of alternating constructions in English and Russian: s-genitive (SG) and of-genitive (OG) in English and noun + noun in genitive case (NNG) and relative adjective derived from noun + noun (ANG) in Russian. It is evident that the long years of elaborate scientific analysis have not yielded any universally accepted view on the problem of English genitive alternation. There are at least five different accounts of this problem: the hypotheses of the animacy hierarchy, given-new hierarchy, topic-focus hierarchy, end-weight principle, and two semantically distinct constructions. We hypothesised that in this case the comparison of the distribution of two English and two Russian genitives could be insightful. The analysis presupposed two consecutive steps. First, we established an inter-language comparability of two pairs of constructions in English and Russian. Second, we tested the similarity of intra-language distribution of each pair of constructions from the perspective of the animacy hierarchy. For these two purposes, two types of corpora were used: (1) a translation corpus consisting of original texts in one language and their translations into one or more languages; and (2) national corpora consisting of original texts in two respective languages. It was established that in both languages, the choice between members of an alternating pair is governed by the rules of animacy hierarchisation. Additionally, it was possible to disprove the idea that the animacy hierarchy is necessarily based on the linearisation hierarchy. Two Russian constructions are typologically aligned with their English counterparts, not on the grounds of the linear order of head and modifier but on the grounds of structural similarity. The English SG and Russian NNG construction are diametrically opposed in terms of word order. However, they reveal the same underlying structure of the inflectional genitive as contrasted with the analytical genitive of the Russian ANG and the English OG. These findings speak strongly in favour of the animacy hierarchy account of English genitive alternation.


Author(s):  
Ziwei Zhang ◽  
Xin Wang ◽  
Wenwu Zhu

Machine learning on graphs has been extensively studied in both academic and industry. However, as the literature on graph learning booms with a vast number of emerging methods and techniques, it becomes increasingly difficult to manually design the optimal machine learning algorithm for different graph-related tasks. To solve this critical challenge, automated machine learning (AutoML) on graphs which combines the strength of graph machine learning and AutoML together, is gaining attention from the research community. Therefore, we comprehensively survey AutoML on graphs in this paper, primarily focusing on hyper-parameter optimization (HPO) and neural architecture search (NAS) for graph machine learning. We further overview libraries related to automated graph machine learning and in-depth discuss AutoGL, the first dedicated open-source library for AutoML on graphs. In the end, we share our insights on future research directions for automated graph machine learning. This paper is the first systematic and comprehensive review of automated machine learning on graphs to the best of our knowledge.


Author(s):  
Heejeong Ko

Scrambling is one of the most widely discussed and prominent factors affecting word order variation in Korean. Scrambling in Korean exhibits various syntactic and semantic properties that cannot be subsumed under the standard A/A'-movement. Clause-external scrambling as well as clause-internal scrambling in Korean show mixed A/A'-effects in a range of tests such as anaphor binding, weak crossover, Condition C, negative polarity item licensing, wh-licensing, and scopal interpretation. VP-internal scrambling, by contrast, is known to be lack of reconstruction effects conforming to the claim that short scrambling is A-movement. Clausal scrambling, on the other hand, shows total reconstructions effects, unlike phrasal scrambling. The diverse properties of Korean scrambling have received extensive attention in the literature. Some studies argue that scrambling is a type of feature-driven A-movement with special reconstruction effects. Others argue that scrambling can be A-movement or A'-movement depending on the landing site. Yet others claim that scrambling is not standard A/A'-movement, but must be treated as cost-free movement with optional reconstruction effects. Each approach, however, faces non-trivial empirical and theoretical challenges, and further study is needed to understand the complex nature of scrambling. As the theory develops in the Minimalist Program, a variety of proposals have also been advanced to capture properties of scrambling without resorting to A/A'-distinctions. Scrambling in Korean applies optionally but not randomly. It may be blocked due to various factors in syntax and its interfaces in the grammar. At the syntax proper, scrambling obeys general constraints on movement (e.g., island conditions, left branch condition, coordinate structure condition, proper binding condition, ban on string vacuous movement). Various semantic and pragmatic factors (e.g., specificity, presuppositionality, topic, focus) also play a crucial role in acceptability of sentences with scrambling. Moreover, current studies show that certain instances of scrambling are filtered out at the interface due to cyclic Spell-out and linearization, which strengthens the claim that scrambling is not a free option. Data from Korean pose important challenges against base-generation approaches to scrambling, and lend further credence to the view that scrambling is an instance of movement. The exact nature of scrambling in Korean—whether it is cost-free or feature-driven—must be further investigated in future research, however. The research on Korean scrambling leads us to the pursuit of a general theory, which covers obligatory A/A'-movement as well as optional displacement with mixed semantic effects in languages with free word order.


2020 ◽  
Vol 41 (5) ◽  
pp. 1223-1247
Author(s):  
Myeongeun Son

AbstractThis study investigates whether L2 learners develop and share an abstract syntactic representation between an L1 and L2 with different word orders and, if so, whether one language’s unique syntactic features affect the shared representation. Korean (SOV) and English (SVO) have equivalent dative alternations; however, because Korean allows word-order scrambling, several dative structures are available in Korean that do not have English counterparts. In this study’s cross-linguistic syntactic priming experiment, intermediate and advanced Korean learners of English described pictures in English after reading various types of Korean dative sentences. The study found evidence of cross-linguistic syntactic priming between Korean and English, regardless of L2 proficiency, but only when prime and target structures shared identical functional assignments, information structures, and order of thematic roles. These results suggest that, within limits created by language-specific features, L2 learners can develop and share abstract representations between two languages with different word orders.


Author(s):  
Adriana Cardoso

Chapter 5 outlines the main findings of the book. It summarizes the three studies offered in the book and suggests some lines that are worth exploring in future research. It also presents some final remarks on the methodology adopted and on the articulation of the different changes investigated. In this respect, it is shown that comparative syntax has provided precious insights into the diachrony of Portuguese and that the changes investigated in the book might be integrated in a cluster of phenomena changing at the same time in the history of Portuguese. Concretely, it is proposed that the loss of middle scrambling originates in a series of changes whose major superficial effect is the reduction of the word-order patterns available in Portuguese.


2016 ◽  
Vol 21 (4) ◽  
pp. 454-473 ◽  
Author(s):  
Emma Vanden Wyngaerd

Aims and Objectives: This paper investigates the word order and adjectival agreement patterns in French-Dutch codeswitched Determiner Phrases (DPs). It examines the predictions made by two theoretical points of view: the approach by MacSwan (2009) within the Minimalist Program (MP) and the Matrix Language Framework (MLF) (Myers-Scotton & Jake, 2009). Methodology: The predictions of these frameworks are compared to data gathered in a grammaticality judgment task. In total, 120 codeswitched sentences were presented aurally to participants, who were asked to rate the sentences on a three-point scale. Originality: While some previous work on word order within codeswitched DP’s exists, this paper is the first study investigating the adjectival agreement patterns in codeswitching. Data and Analysis: Statistical analysis of the data showed that the MacSwan approach is a better predictor for the grammaticality judgments, as sentences predicted to be grammatical by the MP were rated higher than sentences predicted to be ungrammatical by the same model. This difference was statistically significant. There was no significant difference in rating for the predictions of the MLF. Conclusions: The results of the judgment task in combination with the results of previous research on codeswitching highlight the importance of a combination of data from both naturalistic and experimental settings. Implications: The predictions of the Minimalist approach have the upper hand over the predictions of the MLF. However, it remains is important to integrate results from other experimental methodologies, such as naturalistic data and results from psycholinguistic and neurolinguistic studies.


Sign in / Sign up

Export Citation Format

Share Document