Cognitive vs. Social Constructivist Learning for Research and Training on the Angoff Method

This chapter aims to revitalize the use of the Angoff method in measuring students' performance in the educational contexts by offering guidance on the constructivist learning perspective that is more appropriate for training K-12 teachers. Specifically, it compares the cognitive and social constructivist theories and the Completely Structured Training (CST) and Partially Structured Training (PST) designs for conducting training on the Angoff method. The analysis argues for the relative efficacy of the cognitive constructivist perspective of the CST based on a breakdown of the cognitive strategies of the Angoff method judgments over the social constructivist perspective of the PST that emphasizes interpersonal interactions. The chapter concludes with recommendations for empirical comparisons of the quality of judgments based on the CST and PST models.

Chinese ICU Physicians’ Knowledge of Antibiotic Pharmacokinetics/Pharmacodynamics (PK/PD): A Cross-Sectional Survey

Background: Patients with sepsis have a high mortality rate, accumulated evidences suggest that an optimal antibiotic administration strategy based on pharmacokinetics/pharmacodynamics（PK/PD）can improve the prognosis of septic patients. Therefore, we assessed Chinese intensive care unit (ICU) physicians’ knowledge about PK/PD.Methods: In December 2019, we designed a questionnaire focused on Chinese ICU physicians’ knowledge about PK/PD and collected the questionnaires after 3 months. The questionnaire was distributed via e-mail and WeChat, and was distributed to ICU doctors in 31 administrative regions of China except Hong Kong, Macao and Taiwan. The passing score was corrected by the Angoff method, and the ICU physicians’ knowledge about PK/PD was analysed accordingly.Results: We received a total of 1,309 questionnaires and retained 1,240 valid questionnaires. The passing score was 90.8, and the overall pass rate was 56.94%. The pass rate for tertiary and secondary hospitals was 59.07% and 37.19%, respectively. ICU physicians with less than 5 years of work experience and resident physician accounted for the highest pass rate, while those with between 5 to 10 years of work experience and attending accounted for the lowest pass rate. The majority of participants in the Chinese Critical Care Certified Course (5C) were from Jiangsu and Henan provinces, and they had the highest average scores (125.8 and 126.5, respectively). For Beijing and Shanghai, the average score was only 79.4 and 90.9, respectively.Conclusions: Chinese ICU physicians’ knowledge about PK/PD is unsatisfactory. Therefore, it is essential to strengthen ICU physicians’ knowledge about PK/PD.

Implementation and students’ perception of a criterion-referenced standard setting in a therapeutics course

Objective: To implement a criterion-referenced method to set standards for grading written tests in a didactic course and to assess students’ perceptions of the implementation. Methods: The Angoff method, a criterion-referenced method, was implemented in a therapeutics course with a letter grading system. Students were surveyed on their perceptions of the method including stress and test anxiety level after the course. Results: Of 122 students enrolled, 118 responded. More than 60% of respondents felt that the criterion-referenced method was fairer and reflected competency better than a norm-referenced method. The percent of respondents who felt that the new method increased the level of stress and test anxiety was higher than that of those who did not. Conclusions: A criterion-referenced method was successfully implemented in a pharmacy didactic course with a letter grading system. The implementation was overall favourably received by students although it may have increased the level of stress and test anxiety.

The impact of using virtual patients in preclinical pharmacology teaching

To reduce medication errors, medical educators must nurture the early development of rational and safe prescribing. Teaching pharmacology is challenging because it requires knowledge integration across disciplines, including physiology and pathology. Traditionally, pharmacology has been taught using lecture-based learning, which conveys consistent information but may promote passive learning. Virtual patients (VPs) have been used more recently to promote active learning, mainly in clinical years. Conversely, the use of VPs in preclinical disciplines, including pharmacology, is not well described. The objective was to investigate the potential benefits of combining traditional teaching with VPs in preclinical pharmacology teaching. All year 3 medical students (2 cohorts), enrolled in the Systematic Pharmacology I course (lectures: 3 h weekly; tutorial: 1 h weekly), were invited to participate in this naturalistic, prospective study. During tutorials, students were taught using case-based discussion and single-best-answer questions (control) in four tutorials and VPs (experimental) in the remaining six tutorials. The impact of VPs was assessed by 1) performance in written examinations, and 2) student satisfaction/perceptions, using a validated, modified questionnaire. Examination performance related to teaching in VP-based tutorials was significantly improved, compared with traditional tutorials. The level of difficulty of control and experimental assessment items was comparable, as determined by the Angoff method. Facilitation of learning was higher in VP tutorials, while a no-harm effect was noted on knowledge acquisition/maintenance, authenticity of learning, and disadvantages of learning. VPs may be effectively integrated in preclinical pharmacology teaching, with benefits on pharmacological knowledge and facilitation of learning.

Using the Angoff method to set a standard on mock exams for the Korean Nursing Licensing Examination

Purpose: This study explored the possibility of using the Angoff method, in which panel experts determine the cut score of an exam, for the Korean Nursing Licensing Examination (KNLE). Two mock exams for the KNLE were analyzed. The Angoff standard setting procedure was conducted and the results were analyzed. We also aimed to examine the procedural validity of applying the Angoff method in this context.Methods: For both mock exams, we set a pass-fail cut score using the Angoff method. The standard setting panel consisted of 16 nursing professors. After the Angoff procedure, the procedural validity of establishing the standard was evaluated by investigating the responses of the standard setters.Results: The descriptions of the minimally competent person for the KNLE were presented at the levels of general and subject performance. The cut scores of first and second mock exams were 74.4 and 76.8, respectively. These were higher than the traditional cut score (60% of the total score of the KNLE). The panel survey showed very positive responses, with scores higher than 4 out of 5 points on a Likert scale.Conclusion: The scores calculated for both mock tests were similar, and were much higher than the existing cut scores. In the second simulation, the standard deviation of the Angoff rating was lower than in the first simulation. According to the survey results, procedural validity was acceptable, as shown by a high level of confidence. The results show that determining cut scores by an expert panel is an applicable method.

Setting a Minimum Passing Standard for the Uncertainty Communication Checklist Through Patient and Physician Engagement

ABSTRACT Background Historically, medically trained experts have served as judges to establish a minimum passing standard (MPS) for mastery learning. As mastery learning expands from procedure-based skills to patient-centered domains, such as communication, there is an opportunity to incorporate patients as judges in setting the MPS. Objective We described our process of incorporating patients as judges to set the MPS and compared the MPS set by patients and emergency medicine residency program directors (PDs). Methods Patient and physician panels were convened to determine an MPS for a 21-item Uncertainty Communication Checklist. The MPS for both panels were independently calculated using the Mastery Angoff method. Mean scores on individual checklist items with corresponding 95% confidence intervals were also calculated for both panels and differences analyzed using a t test. Results Of 240 eligible patients and 42 eligible PDs, 25 patients and 13 PDs (26% and 65% cooperation rates, respectively) completed MPS-setting procedures. The patient-generated MPS was 84.0% (range 45.2–96.2, SD 10.2) and the physician-generated MPS was 88.2% (range 79.7–98.1, SD 5.5). The overall MPS, calculated as an average of these 2 results, was 86.1% (range 45.2–98.1, SD 9.0), or 19 of 21 checklist items. Conclusions Patients are able to serve as judges to establish an MPS using the Mastery Angoff method for a task performed by resident physicians. The patient-established MPS was nearly identical to that generated by a panel of residency PDs, indicating similar expectations of proficiency for residents to achieve skill “mastery.”