Automatic profiling of L2-simplified texts: Identifying discriminate features of linguistic proficiency

We describe the first wide results of the linguistic profiling of the Common European Framework of Reference (CEFR)-levelled English Corpus (CLEC), a corpus built up for Natural Language Processing purposes. The CLEC is a proficiency-levelled English corpus that covers A1, A2, B1, B2, and C1 CEFR levels and that has been built up to train statistic models for automatic proficiency assessment. We describe not only the main aspects of the corpus development but also display the linguistic features and the statistic results for levels A2, B1, and B2 written examples, carried out automatically. We show how raw text, lexical, morphosyntactic, or syntactic statistic outcomes can help to identify levels of proficiency, to test teaching materials accurate proficiency classification, to provide computable support to new text proficiency validation, and to specify level boundaries. In fact, upper levels strengthen proficiency by showing higher outcomes of lexical and syntactic complexity. This analysis validates the use of automatic tools for proficiency level identification based on lexical and syntactic data, whereas morphosyntactic features strengthen competence-level distinctions. Finally, we suggest that these results are a first step onto the CEFR-levelled automatic assessment of new texts.

Eye movements and human face perception: An holistic analysis and proficiency classification based on frontal 2D face images

Human faces convey a collection of information, such as gender, identity, and emotional states. Therefore, understanding the differences between volunteers’ eye movements on benchmark tests of face recognition and perception can explicitly indicate the most discriminating regions to improve performance in this visual cognitive task. The aim of this work is to qualify and classify these eye strategies using multivariate statistics and machine learning techniques, achieving up to 94.8% accuracy. Our experimental results show that volunteers have focused their visual attention, on average, at the eyes, but those with superior performance in the tests carried out have looked at the nose region more closely.

Motor Skill Proficiency Among Homeschooled Children

Almost 2 million American children are homeschooled but no information is currently available regarding motor skill proficiency within this population. The purpose of this research was to describe motor skill proficiency among homeschooled children and assess differences in homeschooled subgroups. This crosssectional study screened 73 homeschooled children aged 5–8 years for overall motor skill proficiency using the Bruininks-Oseretsky Test of Motor Proficiency, Second Edition, Short Form (BOT-2 SF). Independent t tests examined differences in motor skill proficiency within the homeschooled population. Mann-Whitney U tests examined differences in motor skill proficiency classification within significantly different subgroups. Homeschooled children demonstrated average motor proficiency. Significantly different motor proficiency was seen among homeschooled children participating in 3 or more hours of organized sports per week, t(71) = 2.805, p = .006, 95% CI = 1.77, 10.49, and whose primary caregiver was employed versus unemployed, t(71) = –3.875, p < .001, 95% CI = –13.29, –4.26. Mann-Whitney U tests revealed significantly different motor skill proficiency classification in these same subgroups. Overall, homeschooling showed no detrimental effect on motor skill proficiency. Participation in 3 or more hours of organized sports per week or having an unemployed primary caregiver may improve motor skill proficiency among this population.

Investigating the Effects of Differential Item Functioning on Proficiency Classification

This study provides a comprehensive evaluation of the effects of differential item functioning (DIF) on proficiency classification. Using Monte Carlo simulation, item- and test-level DIF magnitudes were varied systematically to investigate their impact on proficiency classification at multiple decision points. Findings from this study clearly show that the presence of DIF affects proficiency classification not by lowering the overall correct classification rates but by affecting classification error rates differently for reference and focal group members. The study also reveals that multiple items with low levels of DIF can be particularly problematic. They can do similar damage to proficiency classification as high-level DIF items with the same cumulative magnitudes but are much harder to detect with current DIF and differential bundle functioning (DBF) techniques. Finally, how DIF affects proficiency classification errors at multiple cut scores is fully described and discussed.

Analysing corpus-based criterial conjunctions for automatic proficiency classification

The linguistic profiling of L2 learning texts can be taken as a model for automatic proficiency assessment of new texts. But proficiency levels are distinguished by many different linguistic features among which the use of cohesive devices can be a criterial element for level distinctions, either in the number of conjunctions used (quantitative) and/or in the type and variety of them (qualitative). We have carried such an analysis with a subgroup of the CLEC (CEFR-levelled English Corpus) using Coh-Metrix, a tool for computing computational cohesion and coherence metrics for written and spoken texts, but our results suggest that automatic proficiency level assessment needs a deeper examination of conjunctions that should rely on the analysis of conjunction-types use and conjunction varieties, with an analysis of lexical choice. A variable based on familiarity ranks could help to predict cohesive levels proficiencyoriented.

Defining proficiency

In this study, various proficiency classification methods are explored in order to describe the relevant levels on the Common European Framework of Reference for Languages (CEF) that are represented by a group of 127 incoming English students at a Dutch university with respect to academic writing. The weakness of the widely-used group-based institutional status approach is demonstrated with two distinct student-centered approaches, self-assessment and test scores, both of which highlight the within-groups variation that is hidden in group-based approaches. Between-texts variation is further explored through the comparison of self-assessment and text-centered approaches to classification such as test item (response) scores, and widely used measures of lexical variation and syntactic complexity. Findings demonstrate the potential variation in the understanding of academic writing development depending on the the methods of proficiency classification used.