Reinventing Performance Management

Performance appraisals have traditionally been conducted annually or semi-annually. Recently, many companies are transitioning to ongoing feedback and coaching, either in addition to periodic appraisals, or lieu of them. There have also been calls for completely reinventing performance management systems, as the result of an abundance of research that indicates performance processes are over-engineered and time-consuming, and they tend to demotivate employees while hindering candid and honest conversations. This paper examines the common problems with attribution error and rater bias in traditional performance appraisal systems. Five mini-case studies are then presented by exploring how five large companies (Netflix, Adobe, Deloitte, IBM, GE), have reinvented the way in which performance management is implemented. The paper examines why these companies moved away from traditional performance appraisal and what processes replaced it. The paper also identifies emerging trends that will impact the future of performance management and offers suggestions for the road ahead.

Examining the accuracy of residents’ self-assessments and faculty assessment behaviours in anesthesiology

Background: Residents’ accurate self-assessment and clinical judgment are essential for optimizing their clinical skills development. Evidence from the medical literature suggests that residents generally do poorly at self-assessing their performance, often due to factors relating to learners’ personal backgrounds, cultures, the specific contexts of the learning environment and rater bias or inaccuracies. We evaluated the accuracy of anesthesiology residents’ self-assessed Global Entrustment scores and determined whether differences between faculty and resident scores varied by resident seniority, faculty leniency, and/or year of assessment. Methods: We employed variance components modeling techniques and analyzed 329 pairs of faculty and self-assessed entrustment scores among 43 faculty assessors and 15 residents. Using faculty scores as the gold standard, we compared faculty scores with residents’ scores (xi(faculty)–xi(resident)), and determined residents’ accuracy, including over- and under-confidence. Results: The results indicate that residents were respectively over- and under-confident in 10.9% and 54.4% of the assessments but more consistent in their individual self-assessments (rho = 0.70) than faculty assessors. Faculty scores were significantly higher (α = 0.396; z = 4.39; p < 0.001) than residents’ self-assessed scores. Being a lenient/dovish (β = 0.121, z = 3.16, p < 0.01) and a neutral (β = 0.137, z = 3.57, p < 0.001) faculty assessor predicted a higher likelihood of resident under-confidence. Senior residents were significantly less likely to be under-confident compared to junior residents (β = -0.182, z =-2.45, p < 0.05). The accuracy of self-assessments did not significantly vary during the two years of the study period. Conclusions: The majority of residents’ self-assessments were inaccurate. Our findings may help identify the sources of such inaccuracies.

A fully automated method for late ventricular diastole frame selection in post-dive echocardiography without ECG gating

Venous gas emboli (VGE) are often quantified as a marker of decompression stress on echocardiograms. Bubble-counting has been proposed as an easy to learn method, but remains time-consuming, rendering large dataset analysis impractical. Computer automation of VGE counting following this method has therefore been suggested as a means to eliminate rater bias and save time. A necessary step for this automation relies on the selection of a frame during late ventricular diastole (LVD) for each cardiac cycle of the recording. Since electrocardiograms (ECG) are not always recorded in field experiments, here we propose a fully automated method for LVD frame selection based on regional intensity minimization. The algorithm is tested on 20 previously acquired echocardiography recordings (from the original bubble-counting publication), half of which were acquired at rest (Rest) and the other half after leg flexions (Flex). From the 7,140 frames analyzed, sensitivity was found to be 0.913 [95% CI: 0.875-0.940] and specificity 0.997 [95% CI: 0.996-0.998]. The method’s performance is also compared to that of random chance selection and found to perform significantly better (p<0.0001). No trend in algorithm performance was found with respect to VGE counts, and no significant difference was found between Flex and Rest (p>0.05). In conclusion, full automation of LVD frame selection for the purpose of bubble counting in post-dive echocardiography has been established with excellent accuracy, although we caution that high quality acquisitions remain paramount in retaining high reliability.

Leader Identification through Networks of Conversational Interruptions

Leaders are often identified in empirical studies by either their position in an organizationally defined hierarchy or by survey responses, yet such methods conflate behavioral antecedents and outcomes with behaviors themselves. Furthermore, without an external standard for comparison, it cannot be known to what extent differences in leader assignment or emergence between demographic or other categories are due to behavioral differences or biases in the assessment, selection, training, or rating processes. In this study, we propose the ``interruption network'' as a model of small group structure that is (a) grounded in social status theory, (b) definable in both lab and field groups, (c) minimally impacted by rater bias, and (d) based on assessed behaviors rather than antecedents or outcomes. We show that analysis of interruption networks suggests that the often-reported male bias in leadership attributions is found in the ratings but not in the behavior of the observed individuals: males and females may engage in leadership behaviors equally as often, but males are attributed leadership significantly more. Use of the interruption network to represent small group social status therefore extends well-established research on non-verbal behaviors to an explicitly group-oriented context, supporting theory-based unobtrusive assessment and new ways to address important questions in applied psychology.

Manifestation of idiosyncratic rater effect in employee performance appraisal

Performance appraisal is the bedrock of talent management and has received much attention from scholars and researchers alike in their pursuit to develop accurate, objective, and robust Performance Management Systems (PMS). Through survey questionnaire the present study examines the prevalence of idiosyncratic rater biases on the performance appraisal systems and evaluates the measure of its impact. The correlations between the personality traits and the similarities of the raters’ workplace characteristics with the raters’ performance ratings are also determined. The study has provided empirical evidence of the manifestation of idiosyncratic rater bias in the company under study. The idiosyncratic rater tendencies showed a significant impact on performance ratings. It was seen that about one-third of the variations in the ratings were resultant of the idiosyncratic factors, such as similarities in the personality traits and workplace identities. It is also found that there exists a positive correlation between the similarities in the identities, as well as the personality traits of the raters and the ratees, and the way the rating awarded by the rater.

Peer-Rated Organizational Citizenship Behavior

Organizational Citizenship Behavior (OCB) is valuable to organizations and has become an important focus of employee performance evaluation. Employees’ peers may be particularly well-situated to rate their OCB. We investigated the proportion of variance in peer-rated OCB attributable to the ratee (true score) versus the rater (rater bias). Furthermore, we investigated whether these proportions were affected by the familiarity of the peer with the ratee. We found that high familiarity was associated with a greater proportion of ratee variance (.43 vs. .18), and a lower proportion of rater bias (.30 vs. .51), than was the case with low-to-moderate familiarity. Thus, when choosing peers as raters of OCB, there may be value in carefully considering the peers’ familiarity with the ratees.