Reliability of Aggregated Organizational Data: An Evaluation of Five Empirical Indices

1994 ◽  
Vol 2 (1) ◽  
pp. 37-48 ◽  
Author(s):  
Sarah Forbes ◽  
Roma L. Taunton
Keyword(s):  
Leadership Style ◽  
Effect Size ◽  
Work Group ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Time Interval ◽  
Aggregated Data ◽  
Individual Level ◽  
Systems Research ◽  
The Individual

Nursing systems research has included many levels of measurement such as individuals, work groups, organizations, and systems of organizations. Variables important to organizational survival often are measured at the individual level, with inferences made to the work group or the organization. This study demonstrated a process for determining the reliability of individual level data aggregated to the work group and organization levels. Data were analyzed on four variables: job enjoyment; manager’s leadership style in terms of structuring expectations and consideration; and control over nursing practice. Job enjoyment was assessed over time. Registered staff nurses (n = 632), representing 54 nursing units from 4 large acute care hospitals, comprised the sample. The results indicated the assessment of reliability and validity is important at the level of inference and at each time interval. Aggregated data were deemed reliable when the following criteria were met: Cronbach’s alpha >.60; intraclass correlation (1, k) >.60; and a significant F ratio. The interpretation of ω2 as an indicator of effect size suggested that the validity of inference at the aggregated level is in question when effect size is negligible or small.

Conceptual and Scientific Foundations

2010 ◽  
pp. 14-20
Author(s):  
Jay C. Thomas
Keyword(s):  
Work Group ◽  
Professional Standards ◽  
Field Analysis ◽  
Work Analysis ◽  
Individual Level ◽  
Force Field Analysis ◽  
The Individual ◽  
Work Done ◽  
Culture And Climate ◽  
Small Work

Chapter 2 discusses how organizational theory has used force field analysis and systems theory as conceptual frameworks for understanding the development, evolution, and effects of changes in organizations. It also covers how theories originating in social psychology have influenced concepts of small work group and team dynamics, and how, at the individual level, several theories of personality, cognition, human abilities, and attribution are influential. It does address that there are no universally recognized conceptual framework for groups and individuals, and that the OBC psychologist must be an expert at job or work analysis and in comprehending organizational culture and climate. It also covers how some forms of work done by OBC psychologists are constrained by legal and professional standards.

scholarly journals Transformational Leadership, Task-Involving Climate, and Their Implications in Male Junior Soccer Players: A Multilevel Approach

2019 ◽  
Vol 16 (19) ◽  
pp. 3649 ◽  
Author(s):  
Octavio Álvarez ◽  
Isabel Castillo ◽  
Vladimir Molina-García ◽  
Inés Tomás
Keyword(s):  
Transformational Leadership ◽  
Leadership Style ◽  
Soccer Players ◽  
Mediation Effects ◽  
Individual Level ◽  
Extra Effort ◽  
Transformational Leadership Style ◽  
The Individual ◽  
The Relationship ◽  
Outcome Criteria

Despite the well-known positive consequences of transformational coaches in sport, there is still little research exploring the mechanisms through which coaches’ transformational leadership exerts its impact on athletes. Multilevel SEM was used to examine the relationship between coaches’ transformational leadership style, a task-involving climate, and leadership effectiveness outcome criteria (i.e., players’ extra effort, coach effectiveness, and satisfaction with their coach), separately estimating between and within effects. A representative sample of 625 Spanish male soccer players ranging from 16 to 18 years old and nested in 50 teams completed a questionnaire package tapping the variables of interest. Results confirmed that at the team level, team perceptions of transformational leadership positively predicted teams’ perceptions of task climate, which in turn positively predicted the three outcome criteria. At the individual level, players’ perceptions of transformational leadership positively predicted teams’ perceptions of task climate, which in turn positively predicted teams’ extra effort and coach effectiveness. Mediation effects appeared at the team level for all the outcome criteria, and at the individual only for extra effort. Transformational leadership is recommended to enhance task climate, in order to increase players’ extra effort, their perceptions of the effectiveness of their coach, and their satisfaction with his/her leadership style.

scholarly journals Psychometric Properties of the Polish Version of Task and Ego Orientation in Sport Questionnaire (TEOSQ)

2020 ◽  
Vol 17 (10) ◽  
pp. 3593 ◽  
Author(s):  
Maciej Tomczak ◽  
Małgorzata Walczak ◽  
Paweł Kleka ◽  
Aleksandra Walczak ◽  
Łukasz Bojkowski
Keyword(s):  
Psychometric Properties ◽  
High Performance ◽  
Factor Model ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Task Orientation ◽  
Correlated Errors ◽  
Polish Version ◽  
Ego Orientation ◽  
The Individual

The main aim of the study was to assess the psychometric properties of the Polish version of the task and ego orientation in sport questionnaire (TEOSQ). The study covered 651 athletes aged 19.2 years, SD (Standard deviation) = 2.21. The task and ego orientation in sport questionnaire (TEOSQ) and sport motivation scale (SMS-28) were used. Cronbach’s Alpha for the ego subscale was 0.84, and for the task subscale 0.81 (McDonald’s omega was 0.84, 0.82 respectively). The reliability of the test-retest with two weeks interval was ICC (Intraclass correlation coefficient) = 0.86 for ego and ICC = 0.86 for task. Initially, the two-factor model was not fully fitted (CFI (Comparative fit index) = 0.84), however the model with correlated errors for selected test items was well fitted to data (CFI = 0.95). Statistically significant, positive correlations between the task orientation and the intrinsic motivation components were obtained. Additionally, individual athletes had higher scores on the ego factor and lower scores on the task factor than the team athletes. These effects were moderated by the level of participation and occurred among high-performance athletes. Due to satisfactory reliability and validity indicators the Polish version of the task and ego orientation in sport questionnaire (TEOSQ) can be used both for scientific research and in the individual diagnostics of athletes.

Reliability and Validity of the Posture and Fine Motor Assessment of Infants

1989 ◽  
Vol 9 (5) ◽  
pp. 259-272 ◽  
Author(s):  
Jane Case-Smith
Keyword(s):  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Fine Motor ◽  
Motor Assessment ◽  
Intraclass Correlation Coefficients ◽  
Highly Correlated ◽  
Peabody Developmental Motor Scales ◽  
The Individual ◽  
Test Retest Reliability

The Posture and Fine Motor Assessment of Infants (PFMAI) (Case-Smith, 1987) is a newly developed instrument for assessing the quality of motor function in infants. The test measures components of posture and fine motor control as they first develop. The purpose of this study was to support the test's reliability and validity. Interrater reliability, analyzed with intraclass correlation coefficients (ICCs), was high (.989 for total scores). Test-retest reliability, measured by ICCs, was .853 and .913 for the two test sections. The PFMAI demonstrated concurrent validity with the Peabody Developmental Motor Scales, Revised (Folio & Fewell, 1983) (correlations were .673 and .829 for the individual sections). Scores on the PFMAI were highly correlated with the infant's ages (.892 to .941); this finding provided one indication of construct validity.

Gamifying with badges: A big data natural experiment on Stack Exchange

First Monday ◽  
2017 ◽  
Author(s):  
Benny Bornfeld ◽  
Sheizaf Rafaeli
Keyword(s):  
Big Data ◽  
Effect Size ◽  
Natural Experiment ◽  
User Behavior ◽  
Online Systems ◽  
Individual Level ◽  
The Individual

Badges are a common gamification mechanism used by many crowd-sourced online systems. This study provides evidence to their effectiveness and measures their effect size using a big data natural experiment in three large Stack Exchange online Q&A sites. We analyze the introduction of 22 different badge-launch events and the resulting changes in user behavior. Consistent with earlier studies, we report that most badge introductions have the desired effect. Going beyond traditional findings on the individual level, this study measures overall badge effect size on the service.

COMPARING COMMUNITY-PREFERENCE–BASED AND DIRECT STANDARD GAMBLE UTILITY SCORES: EVIDENCE FROM ELECTIVE TOTAL HIP ARTHROPLASTY

2003 ◽  
Vol 19 (2) ◽  
pp. 362-372 ◽  
Author(s):  
David Feeny ◽  
Christopher Blanchard ◽  
Jeffrey L. Mahon ◽  
Robert Bourne ◽  
Cecil Rorabeck ◽  
...  
Keyword(s):  
Total Hip Arthroplasty ◽  
Hip Arthroplasty ◽  
Standard Gamble ◽  
Intraclass Correlation ◽  
Individual Level ◽  
Total Hip ◽  
The Mean ◽  
The Individual ◽  
Community Preferences ◽  
Utility Scores

Objectives: Do utility scores based on patient preferences and scores based on community preferences agree? The purpose is to assess agreement between directly measured standard gamble (SG) utility scores and utility scores from the Health Utilities Index Mark 2 (HUI2) and Mark 3 (HUI3) systems.Methods: Patients were assessed repeatedly throughout the process of waiting to see a surgeon, waiting for surgery, and recovery after total hip arthroplasty (THA). Group mean scores are compared using paired t-tests. Agreement is assessed using the intraclass correlation coefficient (ICC).Results: The mean SG, HUI2, and HUI3 (SD) scores at assessment 1 are 0.62 (0.31), 0.62 (0.19), and 0.52 (0.21); n=103. At assessment 2, the means are 0.67 (0.30), 0.68 (0.30), and 0.58 (0.22); n=84. There are no statistically significant differences between group mean SG and HUI2 scores. Mean SG and HUI3 scores are significantly different. ICCs are low.Conclusions: At the mean level for the group, SG and HUI2 scores match closely. At the individual level, agreement is poor. HUI2 scores were greater than HUI3 scores. HUI2 and HUI3 are appropriate for group level analyses relying on community preferences but are not a good substitute for directly measured utility scores at the individual leve.

Measuring Physical Activity in Juvenile Idiopathic Arthritis: Activity Diary Versus Accelerometer

2017 ◽  
Vol 44 (8) ◽  
pp. 1249-1256 ◽  
Author(s):  
Wineke Armbrust ◽  
G.J.F. Joyce Bos ◽  
Jan H.B. Geertzen ◽  
Pieter J.J. Sauer ◽  
Pieter U. Dijkstra ◽  
...  
Keyword(s):  
Physical Activity ◽  
Juvenile Idiopathic Arthritis ◽  
Effect Size ◽  
Convergent Validity ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Accelerometer Data ◽  
Individual Level ◽  
Intraclass Correlation Coefficients ◽  
Activity Diary

Objective.(1) To determine convergent validity of an activity diary (AD) and accelerometer (Actical brand/Phillips-Respironics) in measuring physical activity (PA) in children with juvenile idiopathic arthritis (JIA). (2) To determine how many days give reliable results. (3) To analyze effects of correcting accelerometer data for non-wear.Methods.Patients with JIA (8–13 yrs) were recruited from 3 Dutch pediatric rheumatology centers. PA was assessed for 7 days with an AD and accelerometer, and was expressed as mean min/day of rest, light PA (LPA), moderate to vigorous PA (MVPA), and PA level (PAL). To analyze convergent validity, intraclass correlation coefficients (ICC) were calculated and paired sample Student t tests were performed. The required number of days to achieve reliable results was calculated using the Spearman-Brown prophecy formula.Results.Convergent validity between AD and accelerometer was moderate for rest and PAL (ICC 0.41). ICC for LPA and MVPA were < 0.24. AD overestimated PAL and MVPA compared with the accelerometer. Wearing the accelerometer 7–19 days gave reliable PA estimates on group and individual levels. For the AD, 13–36 days were needed. Adjusting accelerometer data for non-wear resulted in a clinically relevant higher mean number of min/day spent in LPA (effect size 1.12), but not in MVPA (effect size 0.44).Conclusion.Convergent validity between AD and accelerometer is moderate to poor. In children with JIA, 1-week assessment with an accelerometer is sufficient to measure PA (all levels) reliably. On an individual level and for clinical use, 3 weeks are required. Additional use of AD enables correction for non-wear of accelerometer data.

The Role of Work Group in Individual Sickness Absence Behavior

2008 ◽  
Vol 49 (4) ◽  
pp. 452-467 ◽  
Author(s):  
Ari Väänänen ◽  
Nuria Tordera ◽  
Mika Kivimäki ◽  
Anne Kouvonen ◽  
Jaana Pentti ◽  
...  
Keyword(s):  
Sickness Absence ◽  
Work Group ◽  
Individual Level ◽  
Municipal Employees ◽  
The Social ◽  
Liberal Attitude ◽  
Study Population ◽  
The Individual ◽  
Social Components

The purpose of our two-year follow-up study was to examine the effect of the social components of the work group, such as group absence norms and cohesion, on sickness absence behavior among individuals with varying attitudes toward work attendance. The social components were measured using a questionnaire survey, and data on sickness absence behavior were collected from the employers' records. The study population consisted of 19,306 Finnish municipal employees working in 1,847 groups (78% women). Multilevel Poisson regression modeling was applied. The direct effects of work group characteristics on sickness absence were mostly insignificant. In contrast, both of the social components of a work group had an indirect impact: The more tolerant the group absence norms (at both individual- and cross-level) and the lower the group cohesion (at the individual level), the more the absence behavior of an individual was influenced by his or her attitude toward work attendance. We conclude that work group moderates the extent to which individuals with a liberal attitude toward work attendance actually engage in sickness absence behavior.

Person-Fit to the Five Factor Model of Personality

2012 ◽  
Vol 71 (1) ◽  
pp. 35-45 ◽  
Author(s):  
Jüri Allik ◽  
Anu Realo ◽  
René Mõttus ◽  
Peter Borkenau ◽  
Peter Kuppens ◽  
...  
Keyword(s):  
Factor Model ◽  
Five Factor Model ◽  
Intraclass Correlation ◽  
Individual Response ◽  
Person Fit ◽  
Intraindividual Variation ◽  
Adequate Model ◽  
Interindividual Differences ◽  
Individual Level ◽  
The Individual

The Five Factor Model (FFM), a valid model of interindividual differences in the personality of a group of people, reportedly does not always provide a good fit for the individuals of that group. In addition to intraindividual variation across a considerable period of time, meaningful intraindividual variation can be observed within a single test administration. Two person-fit indices showed that the FFM is an adequate model for 95% of the 1,765 target-judge pairs in four different countries (Belgium, the Czech Republic, Estonia, and Germany): the double-entry intraclass correlation (ICCDE), which indicated that the 30 NEO PI-R scores on scales measuring the same personality trait are more similar and certainly less different than scores measuring different traits, and the individual contribution to the extracted eigenvalues (Zeig). The individual response pattern to the personality questionnaire characterized by the ICCDE and Zeig strongly determined the percentage of explained variance for the group-level factor structure of interindividual differences and the mean self-observer profile agreement. We demonstrate that, if the percentage of variance explained by the first five principal components is high enough, the FFM also provides an adequate fit at the individual level for most people.

Public Service Motivation in a Work Group: Role of Ethical Climate and Servant Leadership

2018 ◽  
Vol 48 (2) ◽  
pp. 203-225 ◽  
Author(s):  
Dong Chul Shim ◽  
Hyun Hee Park
Keyword(s):  
Servant Leadership ◽  
Public Service ◽  
Structural Equation ◽  
Ethical Climate ◽  
Structural Equation Models ◽  
Work Group ◽  
Public Service Motivation ◽  
Government Employees ◽  
Individual Level ◽  
The Individual

The current study attempts to examine Perry’s assertion that the public service motivation (PSM) of government employees may be influenced by the logic of appropriateness. Based on a survey of 596 Korean local government employees in 110 work groups, this study investigated the associations between ethical climate, servant leadership, and PSM. Multilevel structural equation models (SEM) were employed. At the individual level, ethical climate (i.e., efficiency, rule/law, independence) was significantly associated with PSM. However, at the work group level, ethical climate did not demonstrate a significant association with PSM. In addition, this study found that servant leadership is effective in helping government employees develop PSM at both the individual and work group levels.

Sign in / Sign up

Export Citation Format

Share Document