The effects of self-other rating agreement on ethical behavior: an experimental investigation

PurposeThe purpose of this paper is to examine the impact of self-other rating agreement (SOA) on ethical decisions and behavior.Design/methodology/approachA sample of 169 students (60 women and 109 men) enrolled in a part-time, regional MBA program was divided into three SOA categories: (1) Self-aware, individuals whose self-ratings matched observer ratings, (2) Underraters, those whose self-ratings were lower than observer ratings and (3) Overraters, individuals whose self-ratings were higher than other ratings. Ethical behavior was evaluated with the completion of a managerial in-basket assessment.FindingsThe results revealed that ethical behavior varied by SOA, with underraters exhibiting the highest levels of ethical behavior, followed by self-aware (i.e. accurate) and then overraters. One of the intriguing results is that underraters displayed more ethical behaviors than accurate raters, raising questions about the use of accurate self-assessments as an indicator of personal and professional effectiveness.Originality/valueThe results indicate that organizations should consider SOA in their human resource processes because it has important implications for employee training, selection and promotion. While prior research has examined the effect of SOA on performance, commitment and leadership perceptions, the authors contribute to the literature by examining whether SOA influences actual ethical decisions and actions.

Inter-rater reliability of a newly developed gait analysis and motion score

Background: Gait analysis constitutes an essential part of orthopedic rehabilitation assessment. Previous studies indicate that observational-based gait analysis lacks reliability and requires extensive clinical training. Therefore, gait analysis in the clinical practice heavily relies on technical aids. The aim of the present study is to develop a reliable gait analysis assessment tool that can accurately assess clinically relevant gait cycle parameters in daily clinical practice. Methods: In this pilot study, a new gait analysis and motion score (GAMS), comprising 10 observational and 5 technically measured (e.g. pressure plate) gait parameters, was developed. The parameters were dichotomously operationalized, reflecting pathological versus physiological manifestations of the parameters. The rating algorithm was administered by 12 raters using videotaped treadmill sessions of 10 orthopedic subjects ( n = 120 ratings). Inter-rater reliability was calculated using the intraclass correlation coefficient (ICC) and the percentage of rating agreement. Results: The mean (standard deviation (SD)) GAMS ratings ranged from 10.0 (1.1) to 21.5 (1.3) points. The overall GAMS ICC was 0.98 (95% confidence interval (CI) 0.96–1.00), whereas the ICC of observational parameters alone was 0.97 (95% CI 0.93–0.99). The mean (SD) percentage of rating agreement was 86.1% (3.3%). For the observational parameters, the mean (SD) rating agreement was 82.5% (4.5%). Conclusion: This new GAMS shows excellent overall inter-rater reliability for a continuum of functional gait statuses. The new score may be an appropriate clinical tool to objectively evaluate patients’ gait patterns. Furthermore, the GAMS may find application as a clinician-reported outcome measure in orthopedic rehabilitation. Further studies are required to verify the validity and accuracy of the new GAMS and its functionality in assessing clinical changes in gait patterns.

Rater Congruency

This chapter explores the meaning of, and implications for, differences in 360 Feedback ratings among individual raters (e.g., self, peers, direct reports). This chapter looks at possible causes and consequences for high and low agreement and why it is important. A detailed discussion of the history of rating difference research is followed by an analysis of these differences. This chapter addresses a number of issues related to congruity in feedback ratings: What is rater congruence? Why does it matter? Is it mainly due to people overrating themselves? What are the business implications for high or low congruence? What are the recommendations to increase it? The importance of self-other rating agreement in 360 Feedback is discussed.

The Handbook of Strategic 360 Feedback

In the final chapter, personal observations of the handbook editors are given, including key themes, predictions, career highlights, and reflections on editing this volume. As noted in the case of 360, there have been so many permutations of the method over the years that very few implementations will look similar; as a result, there can be confusion about what exactly is 360 Feedback. By focusing on Strategic 360 Feedback as an organizing framework, practitioners can now examine a cohesive set of guidelines to shape the direction of this growing practice. Successfully leveraging 360 Feedback to align talent with strategy requires attention to all facets of the process. One of the themes that emerges from this volume is the potential use of self–other rating agreement as a measure of self-awareness and leader effectiveness. Another emerging theme from this volume is the effect that technology is having on the 360 Feedback process. In a different direction, vendors and practitioners should not claim the benefits of collecting feedback under the label of “360 degree” and then not follow the practices that make it effective and appropriate for the advertised use. The bottom line remains that for 360 Feedback to be effective and ethically defensible it must meet the basic criteria presented. Each of the levers of strategic intent, measurement quality, integration, and inclusion must be amped up or modified to achieve the best results in the right contexts and with the fully aligned purpose of the effort in mind.

Agreement on service performance ratings between frontline employees and their supervisor

Purpose The purpose of this paper is to examine whether agreement between frontline employee self-ratings and supervisory ratings of service performance functions as an indicator of healthy supervisor-subordination relationships above and beyond what might be indicated simply by either supervisory ratings or self-ratings. Design/methodology/approach Research hypotheses were tested using a sample of 220 matched pairs of frontline service workers and their immediate supervisors from nine full service hotels in the USA. Findings The results show that higher levels of agreement in service performance ratings between employees and supervisors is associated with higher levels of leader-member exchange (LMX) and organizational commitment. Practical implications Senior managers can refer to the level of performance rating agreement between customer service employees and their supervisors in assessing supervisors’ competency to manage their work relationship with their subordinates. Originality/value This study examined rating agreement in a service performance context and found rating agreement between subordinates and their supervisor may have a unique effect on service worker effectiveness, producing a unique incremental effect on LMX and organizational commitment. This is important given that few attempts have been made to examine service performance from both subordinates’ and supervisors’ perspectives and the implication that rating agreement may have for improving employee service performance.