Automated Assessment of Free Text Questions for MOOC Using Regular Expressions

2014 ◽  
Vol 27 (2) ◽  
pp. 1-13 ◽  
Author(s):  
Enrique Sánchez Acosta ◽  
Juan José Escribano Otero
Keyword(s):  
Online Education ◽  
Assessment Tool ◽  
Assessment System ◽  
Question Type ◽  
Assessment Instruments ◽  
Free Text ◽  
Regular Expressions ◽  
Automatic Assessment ◽  
Use Of Technology ◽  
Short Answer Question

Presently, most platforms used on the selection of Massive Open Online Courses (MOOC) available online have various automated methods of assessment. These type of tools are based on applications that analyze the answers using a pre-correction algorithm. Most of these programs run several types of automatic assessment, but the possible use of the technology for each of them differs with respect to the kind of automation applied. The role of technology in the objective test for online education has become extremely common, so it can be found in various MOOC platforms with this type of questions or quizzes, because the assessment system can be fully computerized (from the test design to its correction and reporting). However, not all of the assessment instruments can be easily implemented in automatic mode with the use of technology. This paper seeks to research and clarify a type of assessment tool in which the use of technologies is quite low, namely, the essay question type and within them, the short answer question type or free text question type, using regular expressions. The large number of students who would be in MOOC prevents a teacher from assessing responses of thousands of students in a finite time without the aid of technology. This research analyzes the results of an MOOC from hundreds of students to verify that the use of regular expressions in an MOOC platform is not only recommended but also necessary.

scholarly journals Investigating the Feasibility of Automatic Assessment of Programming Tasks

10.28945/4150 ◽  
2018 ◽  
Vol 17 ◽  
pp. 201-223
Author(s):  
Janet Liebenberg ◽  
Vreda Pieterse
Keyword(s):  
Assessment Tool ◽  
Assessment Methods ◽  
Future Research ◽  
Automatic Assessment ◽  
First Year Students ◽  
Java Programming ◽  
High Achieving Students ◽  
Use Of Technology ◽  
Functional Correctness ◽  
High Achieving

Aim/Purpose: The aims of this study were to investigate the feasibility of automatic assessment of programming tasks and to compare manual assessment with automatic assessment in terms of the effect of the different assessment methods on the marks of the students. Background: Manual assessment of programs written by students can be tedious. The assistance of automatic assessment methods might possibly assist in reducing the assessment burden, but there may be drawbacks diminishing the benefits of applying automatic assessment. The paper reports on the experience of a lecturer trying to introduce automated grading. Students’ solutions to a practical Java programming test were assessed both manually and automatically and the lecturer tied the experience to the unified theory of acceptance and use of technology (UTAUT). Methodology: The participants were 226 first-year students registered for a Java programming course. Of the tests the participants submitted, 214 were assessed both manually and automatically. Various statistical methods were used to compare the manual assessment of student’s solutions with the automatic assessment of the same solutions. A detailed investigation of reasons for differences was also carried out. A further data collection method was the lecturer’s reflection on the feasibility of automatic assessment of programming tasks based on the UTAUT. Contribution: This study enhances the knowledge regarding benefits and drawbacks of automatic assessment of students’ programming tasks. The research contributes to the UTAUT by applying it in a context where it has hardly been used. Furthermore, the study is a confirmation of previous work stating that automatic assessment may be less reliable for students with lower marks, but more trustworthy for the high achieving students. Findings: An automatic assessment tool verifying functional correctness might be feasible for assessment of programs written during practical lab sessions but could be less useful for practical tests and exams where functional, conceptual and structural correctness should be evaluated. In addition, the researchers found that automatic assessment seemed to be more suitable for assessing high achieving students. Recommendations for Practitioners: This paper makes it clear that lecturers should know what assessment goals they want to achieve. The appropriate method of assessment should be chosen wisely. In addition, practitioners should be aware of the drawbacks of automatic assessment before choosing it. Recommendation for Researchers: This work serves as an example of how researchers can apply the UTAUT theory when conducting qualitative research in different contexts. Impact on Society: The study would be of interest to lecturers considering automated assessment. The two assessments used in the study are typical of the way grading takes place in practice and may help lecturers understand what could happen if they switch from manual to automatic assessment. Future Research: Investigate the feasibility of automatic assessment of students’ programming tasks in a practical lab environment while accounting for structural, functional and conceptual assessment goals.

Automatic and Objective Assessment of Motor Skills Performance in Flexible Bronchoscopy

Respiration ◽  
2021 ◽  
pp. 1-9
Author(s):  
Kristoffer Mazanti Cold ◽  
Morten Bo Søndergaard Svendsen ◽  
Uffe Bodtger ◽  
Leizl Joy Nayahangan ◽  
Paul Frost Clementsen ◽  
...  
Keyword(s):  
Motor Skills ◽  
Assessment Tool ◽  
Objective Assessment ◽  
Flexible Bronchoscopy ◽  
Assessment System ◽  
Measurement Unit ◽  
Automatic Assessment ◽  
Expert Performance ◽  
Pearson’S Correlation ◽  
Pearson's Correlation

<b><i>Background:</i></b> Motor skills have been identified as a useful measure to evaluate competency in bronchoscopy. However, no automatic assessment system of motor skills with a clear pass/fail criterion in flexible bronchoscopy exists. <b><i>Objectives:</i></b> The objective of the study was to develop an objective and automatic measure of motor skills in bronchoscopy and set a pass/fail criterion. <b><i>Methods:</i></b> Participants conducted 3 bronchoscopies each in a simulated setting. They were equipped with a Myo Armband that measured lower arm movements through an inertial measurement unit, and hand and finger motions through electromyography sensors. These measures were composed into an objective and automatic composite score of motor skills, the motor bronchoscopy skills score (MoBSS). <b><i>Results:</i></b> Twelve novices, eleven intermediates, and ten expert bronchoscopy operators participated, resulting in 99 procedures available for assessment. MoBSS was correlated with a higher diagnostic completeness (Pearson’s correlation, <i>r</i> = 0.43, <i>p</i> &#x3c; 0.001) and a lower procedure time (Pearson’s correlation, <i>r</i> = −0.90, <i>p</i> &#x3c; 0.001). MoBSS was able to differentiate operator performance based on the experience level (one-way ANOVA, <i>p</i> &#x3c; 0.001). Using the contrasting groups’ method, a passing score of −0.08 MoBSS was defined that failed 30/36 (83%) novice, 5/33 (15%) intermediate, and 1/30 (3%) expert procedures. <b><i>Conclusions:</i></b> MoBSS can be used as an automatic and unbiased assessment tool for motor skills performance in flexible bronchoscopy. MoBSS has the potential to generate automatic feedback to help guide trainees toward expert performance.

scholarly journals A comparison of three interactive examination designs in active learning classrooms for nursing students

BMC Nursing ◽  
2021 ◽  
Vol 20 (1) ◽  
Author(s):  
Linda Ahlstrom ◽  
Christopher Holmberg
Keyword(s):  
Nursing Education ◽  
Mixed Methods ◽  
Nursing Students ◽  
Active Learning ◽  
Online Education ◽  
Learning Strategies ◽  
Free Text ◽  
Course Content ◽  
Undergraduate Nursing ◽  
Chi Squared

Abstract Background Despite the advantages of using active learning strategies in nursing education, researchers have rarely investigated how such pedagogic approaches can be used to assess students or how interactive examinations can be modified depending on circumstances of practice (e.g., in online education). Aims The aim was to compare three interactive examination designs, all based on active learning pedagogy, in terms of nursing students’ engagement and preparedness, their learning achievement, and instructional aspects. Methods A comparative research design was used including final-year undergraduate nursing students. All students were enrolled in a quality improvement course at a metropolitan university in Sweden. In this comparative study to evaluate three course layouts, participants (Cohort 1, n = 89; Cohort 2, n = 97; Cohort 3, n = 60) completed different examinations assessing the same course content and learning objectives, after which they evaluated the examinations on a questionnaire in numerical and free-text responses. Chi-squared tests were conducted to compare background variables between the cohorts and Kruskal–Wallis H tests to assess numerical differences in experiences between cohorts. Following the guidelines of the Good Reporting of a Mixed Methods Study (GRAMMS), a sequential mixed-methods analysis was performed on the quantitative findings, and the qualitative findings were used complementary to support the interpretation of the quantitative results. Results The 246 students who completed the questionnaire generally appreciated the interactive examination in active learning classrooms. Among significant differences in the results, Cohort 2 (e.g., conducted the examination on campus) scored highest for overall positive experience and engagement, whereas Cohort 3 (e.g., conducted the examination online) scored the lowest. Students in Cohort 3 generally commended the online examination’s chat function available for use during the examination. Conclusions Interactive examinations for nursing students succeed when they are campus-based, focus on student preparation, and provide the necessary time to be completed.

scholarly journals Analysis of Educational Data in the Current State of University Learning for the Transition to a Hybrid Education Model

2021 ◽  
Vol 11 (5) ◽  
pp. 2068
Author(s):  
William Villegas-Ch. ◽  
Xavier Palacios-Pacheco ◽  
Milton Roman-Cañizares ◽  
Sergio Luján-Mora
Keyword(s):  
Online Education ◽  
Educational Model ◽  
Hybrid Education ◽  
Face To Face ◽  
Current State ◽  
Use Of Technology ◽  
Technological Advances ◽  
The World ◽  
The Face ◽  
Education Model

Currently, the 2019 Coronavirus Disease pandemic has caused serious damage to health throughout the world. Its contagious capacity has forced the governments of the world to decree isolation and quarantine to try to control the pandemic. The consequences that it leaves in all sectors of society have been disastrous. However, technological advances have allowed people to continue their different activities to some extent while maintaining isolation. Universities have great penetration in the use of technology, but they have also been severely affected. To give continuity to education, universities have been forced to move to an educational model based on synchronous encounters, but they have maintained the methodology of a face-to-face educational model, what has caused several problems in the learning of students. This work proposes the transition to a hybrid educational model, provided that this transition is supported by data analysis to identify the new needs of students. The knowledge obtained is contrasted with the performance presented by the students in the face-to-face modality and the necessary parameters for the transition to this modality are clearly established. In addition, the guidelines and methodology of online education are considered in order to take advantage of the best of both modalities and guarantee learning.

scholarly journals PENILAIAN AUTENTIK MATA PELAJARAN MATEMATIKA KURIKULUM 2013 GURU KELAS IV KOTA SEMARANG

2015 ◽  
Vol 4 (2) ◽  
Author(s):  
Fajar Cahyadi ◽  
Apriliana Purwandari
Keyword(s):  
Elementary Schools ◽  
Sampling Technique ◽  
Authentic Assessment ◽  
Assessment System ◽  
Assessment Instrument ◽  
Data Card ◽  
Assessment Instruments ◽  
Knowledge And Skills ◽  
Curriculum Assessment ◽  
Qualitative Descriptive

The change of KTSP curriculum into 2013 curriculum?é?á lead to changes in the assessment system. With the change in valuation system, teachers should be able to adapt and develop appropriate assessment with the curriculum has changed. In this 2013 curriculum using authentic assessment system that can assess the competence of the three domains, namely the domain of attitudes, knowledge, and skills. The problems revealed in this study are Is authentic assessment instrument in the subjects of mathematics which has been prepared by the teacher in accordance with the characteristics of the curriculum in 2013? Goals to be achieved in this study was to determine the suitability of authentic assessment instrument in the subjects of mathematics that had been developed by teachers with the characteristics of curriculum assessment in 2013, this is a type of qualitative research that uses qualitative descriptive approach. The population in this study is a Primary School in Hyderabad who have applied in the process of learning curriculum was radically 2013, amounting to 12 elementary schools. Samples taken are 5 elementary schools in Semarang by using purposive sampling technique. The data in this study was obtained through the analysis of the research instruments and text data card interview. Based on the analysis of experimental data showed that the instrument of authentic assessment in mathematics is organized by fourth grade teachers from five elementary schools in Semarang 85.2% in accordance with the characteristics of curriculum assessment in 2013, where the authentic assessment instruments assessment covers the realm of attitudes, knowledge, and skills. Therefore it can be concluded that the teacher has been able to make an authentic assessment instrument based on the characteristics of the curriculum in 2013.

scholarly journals Researching online communities of inquiry through digital ethnography

2021 ◽  
Vol 102 ◽  
pp. 01006
Author(s):  
Neil Johnson
Keyword(s):  
Online Learning ◽  
Online Education ◽  
Online Communities ◽  
Online Community ◽  
Study Data ◽  
Digital Ethnography ◽  
Use Of Technology ◽  
Ethnographic Data ◽  
Communities Of Inquiry

Research into providing effective online education has suggested an important goal for instructors is the creation of an online community of inquiry (CoI) where social, cognitive, and teacher presence are all important aspects of successful online learning. With reference to a recent reflective practice case study, this paper describes ways that the research on online communities of inquiry may be enriched through the use of digital ethnography. In the target reflective case study, data analysis tasks were designed and presented in an online VoiceThread site, promoting dialogic and multimodal engagement with data from actual research studies that are central to the module theme in teacher education. Interaction around these tasks is coded using the CoI framework. Ethnographic data from the participants was collected and coded using qualitative research protocols to contextualise the interaction data and provide a clearer understanding of how participants had come together throughout the module. The ethnographic data revealed some interesting concerns with online learning, including the use of technology as a barrier to participation.

scholarly journals Developing Web-Application’s Automated Testing Technique

2019 ◽  
Vol 17 (3) ◽  
pp. 93-110
Author(s):  
A. V. Tkachev ◽  
D. V. Irtegov
Keyword(s):  
Web Application ◽  
Development Process ◽  
Web Applications ◽  
Assessment System ◽  
Automated Testing ◽  
Automatic Assessment ◽  
Testing Technique ◽  
Code Coverage ◽  
Client Side ◽  
User Actions

The article is devoted to the technique of automated testing of NSUts – automatic assessment system for programming tasks developed at NSU. The main priority for this technique is to test both the old and the new versions of the application, so that the same or minimally modified tests could be executed on two versions of the system with different architectures. This could be useful while organizing the development process for other applications with a long life cycle. To test not only the server but also the client side of the web application, we suggest using tools like Selenium WebDriver to simulate user actions by sending commands to real browsers. We use the well-known Page Object design pattern to handle differences in HTML layout and functionality, and describe a number of ways to make developed tests less fragile and easily adapt those to work with the new version of the system. The article also describes the use of this technique to organize automated testing of the NSUts system and analyzes its effectiveness. The analysis shows that the estimated code coverage by these tests is quite high, and therefore the technique can be considered effective and applied to other similar web applications.

scholarly journals One score to rule them all: severity assessment in laboratory mice

2020 ◽  
Author(s):  
Steven R. Talbot ◽  
Birgitta Struve ◽  
Laura Wassermann ◽  
Miriam Heider ◽  
Nora Weegh ◽  
...  
Keyword(s):  
Heart Rate ◽  
Animal Welfare ◽  
Biomedical Research ◽  
Restraint Stress ◽  
Assessment Tool ◽  
Postoperative Recovery ◽  
Assessment System ◽  
Severity Assessment ◽  
Severity Levels ◽  
Individual Variables

AbstractAnimal welfare and the refinement of experimental procedures are fundamental aspects of biomedical research. They provide the basis for robust experimental designs and reproducibility of results. In many countries, the determination of welfare is a mandatory legal requirement and implies the assessment of the degree of the severity that an animal experiences during an experiment. However, for an effective severity assessment, an objective and exact approach/system/strategy is needed. In light of these demands, we have developed the Relative Severity Assessment (RELSA) score.This comprehensive composite score was established on the basis of physiological and behavioral data from a surgical mouse study. Body weight, the Mouse Grimace Scale score, burrowing behavior, and the telemetry-derived parameters heart rate, heart rate variability, temperature, and general activity were used to investigate the quality of indicating severity during postoperative recovery. The RELSA scores not only revealed individual severity levels but also allowed a comparison of severity in distinct mouse models addressing colitis, sepsis, and restraint stress using a k-means clustering approach with the maximum achieved RELSA scores.We discriminated and classified data from sepsis nonsurvivors into the highest relative severity level. Data from mice after intraperitoneal transmitter implantation and sepsis survivor al were located in the next lower cluster, while data from mice subjected to colitis and restraint stress were placed in the lowest severity cluster. Analysis of individual variables and their combinations revealed model- and time-dependent contributions to severity levels.In conclusion, we propose the RELSA score as a validated tool for objective real-time applicability in severity assessment and as a first step towards a unified and accessible risk assessment tool in biomedical research. As an effective severity assessment system, it will fundamentally improve animal welfare, as well as data quality and reproducibility.

Sign in / Sign up

Export Citation Format

Share Document