Development of a Formative Assessment Rubric for Peer Evaluation of Teaching (FARPET) and Pilot Use in Veterinary Online Teaching

Peer evaluation of teaching (PET) serves an important role as a component of faculty development in the medical education field. With the emergence of COVID-19, the authors recognized the need for a flexible tool that could be used for a variety of lecture formats, including virtual instruction, and that could provide a framework for consistent and meaningful PET feedback. This teaching tip describes the creation and pilot use of a PET rubric, which includes six fixed core items (lesson structure, content organization, audiovisual facilitation, concept development, enthusiasm, and relevance) and items to be assessed separately for asynchronous lectures (cognitive engagement—asynchronous) and synchronous lectures (cognitive engagement—synchronous, discourse quality, collaborative learning, and check for understanding). The instrument packet comprises the rubric, instructions for use, definitions, and examples of each item, plus three training videos for users to compare with authors’ consensus training scores; these serve as frame-of-reference training. The instrument was piloted among veterinary educators, and feedback was sought in a focus group setting. The instrument was well received, and training and use required a minimum time commitment. Inter-rater reliability within 1 Likert scale point (adjacent agreement) was assessed for each of the training videos, and consistency of scoring was demonstrated between focus group members using percent agreement (0.82, 0.85, 0.88) and between focus members and the authors’ consensus training scores (all videos: 0.91). This instrument may serve as a helpful resource for institutions looking for a framework for PET. We intend to continually adjust the instrument in response to feedback from wider use.

Development of a New Personality-Oriented Work Analysis Questionnaire: First Steps Towards Validation

In this paper we detail the construction and validation process for a new personality-oriented work analysis instrument, in the form of a standardized questionnaire, based on extant research that shows that personality traits are good predictors of job performance. We present the process of item development, frame of reference training, rating scale creation, and the selection of subject matter experts. By administering the instrument to three distinct positions, the interrater reliability coefficients resulted between .80 and .94. We also investigated the instrument’s ability to discriminate between the same rated positions, and the results for this indicator were quite low. Conclusions provide some possible explanations for the lower resulted discriminability. Practical and theoretical implications are discussed as well as other future research for general improvement of data quality.

Ethnic and Gender Biases in Clinical Performance Assessment (CPA) in Healthcare Education: A Systematic Review

Background: Several forms of bias, including ethnic and gender bias, are thought to impact evaluations on Clinical Performance Assessments (CPAs). Unfairness may influence student learning attitudes if a loss of trust causes a lack of engagement in learning. Understanding the biases occurring in CPAs can lead to well-designed examiner training to ensure equality and fairness. The purpose of this systematic review is to determine the current evidence in the literature for ethnic and/or gender bias by examiners evaluating pre-licensure healthcare students in CPAs using standardized patients (SPs). Methods: Literature was systematically searched in CINAHL, PubMed and Medline from inception to February 2019, and no date range was set. Studies related to the investigation of ethnic and/or gender biases occurring in CPAs using SPs for examining health professions students were selected. A systematic review was conducted to assess the methodological quality and strength of evidence of relevant research and to identify if any potential ethnic and/or gender bias occurred in CPAs. The Guidelines for Critical Review were used to appraise the selected studies. Results: Nine studies published from 2003 to 2017 were retrieved for review. Three studies met all the Guidelines for Critical Review quality criteria, indicating stronger evidence of their outcomes, two of the studies reported ethnic and/or gender bias existing in the CPAs. Overall, four studies found ethnic and/or gender bias in CPAs, but all study results had small effect sizes. Conclusions: No systematic and consistent bias was found across the studies; nonetheless, the possibility of ethnic or gender bias by some examiners cannot be ignored. To minimize potential examiner bias, the investigation of Frame of Reference training, multiple examiners per station, and combination assessments in CPAs is recommended.

Converting values awareness to values enactment through frame-of-reference training

Purpose This paper aims to investigate whether values enactment could be increased through frame-of-reference (FOR) training configured around values prototyping and behavioural domain training for managers within an Australian public sector organisation. Design/methodology/approach Employees from an Australian public sector organisation were studied to ascertain the effect of values training and development via a three-way longitudinal design with a control group. Findings The findings indicate that FOR training can increase employee values enactment clarity and, thereby, have a positive impact upon organisational values enactment. Practical implications The application of FOR training constitutes a new approach to supporting the development of employee values clarity, which, in turn, can support the achievement of organisational values enactment. Through FOR training, employees can learn to apply organisational values in their decision-making and other behaviours irrespective of whether they are highly congruent with their personal values. Originality/value Empirical research into values management is limited and there is a lack of consensus to what is needed to create a values-driven organisation. The article shows that FOR training can be a beneficial component of a broader human resource strategy aimed at increasing organisational values enactment. With reference to the resource-based view of the firm, it is argued that values enactment constitutes a distinctive capability that may confer sustained organisational advantage.