Learner Analytics Applied to the Clinical Diagnostic Reasoning with Hepius Cognitive Tutor and Simulator (Preprint)

BACKGROUND Virtual Patient Simulator is a tool that may generate a multi-dimensional representation of the student’s medical knowledge by analyzing the recordings of the user’s actions during a clinical simulation. Adequate metrics may provide teachers with valuable learning information. OBJECTIVE To describe the analytic metrics we used to analyze the clinical diagnostic reasoning of medical students obtained by a novel Cognitive Tutor and Simulator named Hepius embedding Natural Language Processing (NLP) techniques. METHODS Two clinical case simulations (Tests) were created to tune our metrics. During each simulation, students’ actions were logged into the program data base for off-line analysis. Twenty-six students, attending the 5th year of the School of Medicine at Humanitas University, underwent Test 1 (April 12th 2019) which simulated a patient suffering from dyspnea. Test 2 (May 21st 2019) dealt with abdominal pain and was attended by 36 students. Overall students’ performance was split into 7 issues: 1) the identification of relevant information in the given clinical scenario (SC); 2) history taking (AN); 3) physical exam (PE); 4) medical tests (MT) ordering; 5) diagnostic hypotheses (HY) setting; 6) binary analysis fulfillment (BA); 7) final diagnosis (RS) setting. Sensitivity (percentage of relevant information found) and precision (percentage of correct actions performed) metrics were computed for each issue and combined into a harmonic (F1), thereby obtaining a single score (1= maximal sensitivity and precision) evaluating the student’s performances. The seven F1-metric scores were further combined to obtained a convenient index assessing the student’s overall performances.The seven metrics were further grouped to reflect the student’s capability to collect (SC, AN, PE and MT) and to analyze (HY, BA and RS) information. A methodological score was computed on the basis of the discordance between the diagnostic pathway followed by the student and a reference one, previously defined by the teacher. RESULTS Mean overall scores were consistent between the two tests (0.6.±0.05 for Test 1 and 0.5±0.05 for Test 2). For each student, overall performance was achieved by a different contribution in collecting and analyzing information. Methodological scores highlighted some discordance between the reference diagnostic pattern previously set by the teachear and the one pursued by the student. CONCLUSIONS Different components of the student’s diagnostic process may be disentangled and quantified by appropriate metrics applied on students’ actions recorded while addressing a virtual case. Such an approach may help teachers in giving students individualized feedbacks aimed at filling up knowledge drawbacks and methodological inconsistencies.

The Effect of Assistance on Learning and Affect in an Algebra Tutor

We investigated the impact of assistance on learning and affect during problem-solving activities with a computer tutor we built using the Cognitive Tutor Authoring Tools framework. The tutor delivered its primary form of assistance in the form of worked-out examples. We manipulated the level of assistance the examples in the tutor provided, by having similar problem-example pairs in one version of the tutor (high-assistance condition) and reduced similarity problem-example pairs in the other version (reduced-assistance condition). The reduced-assistance condition resulted in significantly higher learning, without increasing negative affect like frustration.

Information Technologies for Learning Principles of Fault-Tolerant Systems

In this chapter, the authors present a methodology for developing a model-tracing cognitive tutor. The methodology is based on Bayesian probabilistic networks for generating pedagogical interventions. The presented probabilistic model increases fidelity assessment due to its ability of independently diagnosing the degree of mastery for every knowledge component involved in students' actions; fidelity assessment in education is the ability to represent students' cognitive states as close as possible for analysis and evaluation. The cognitive tutor was developed to promote a self-regulated learning approach with an open learner model. The open learner model let students change the learning flow by changing the assigned tasks. The authors explain in detail the structural construction and employed algorithms for developing a model-tracing cognitive tutor in the domain of fault-tolerant systems. Preliminary results and future work are also discussed to assess effectiveness of the proposed approach and its implication in actual educational programs.

A Cognitive Tutor of Arabic Word Root Extraction Using Artificial Word Generation, Scaffolding and Self-Explanation

Extracting the roots (stemming) of Arabic words is one of the most challenging skills taught to Arabic language learners. To address this challenge, this paper proposes the Arabic word Root extraction Tutor (ART). ART is a cognitive tutor intended to teach students production rules needed for Arabic word root extraction. In the passive mode, ART accepts an input word and generates its root with explanation. In the active mode, on the other hand, words are generated by ART and the student is prompted to provide the correct roots. ART integrates several techniques for enhanced tutoring. It provides a positive feedback for a correct answer and a negative one otherwise. In the latter case, Prompting Answer Strategy (PAS) is employed, where the student is guided to detect the error by integrating scaffolding and self-explanation. Scaffolding prompts the student to apply the relevant production rule step by step. In each step, a number of options are given to the student to select the correct one via self-explanation. If the error persists, the correct answer is generated with explanation. In addition to generating real words, artificial words are generated using the production rules. This novel technique is intended to ensure that the student applies the production rules rather than memorizes the roots of common words. Evaluation has shown the effectiveness of ART tutoring process and suggests artificial word generation as a promising technique in language tutoring.