Crowdsourcing Job Satisfaction Data: Examining the Construct Validity of Glassdoor.com Ratings

Researchers, practitioners, and job seekers now routinely use crowdsourced data about organizations for both decision-making and research purposes. Despite the popularity of such websites, empirical evidence regarding their validity is generally absent. In this study, we tackled this problem by combining two curated datasets: (a) the results of the 2017 Federal Employee Viewpoint Survey (FEVS), which contains facet-level job satisfaction ratings from 407,789 US federal employees, and which we aggregated to the agency level, and (b) current overall and facet ratings of job satisfaction of the federal agencies contained within FEVS from Glassdoor.com as scraped from the Glassdoor application programming interface (API) within a month of the FEVS survey’s administration. Using these data, we examined convergent validity, discriminant validity, and methods effects for the measurement of both overall and facet-level job satisfaction by analyzing a multitrait-multimethod matrix (MTMM). Most centrally, we provide evidence that overall Glassdoor ratings of satisfaction within US federal agencies correlate moderately with aggregated FEVS overall ratings (r = .516), supporting the validity of the overall Glassdoor rating as a measure of overall job satisfaction aggregated to the organizational level. In contrast, the validity of facet-level measurement was not well-supported. Overall, given varying strengths and weaknesses with both Glassdoor and survey data, we recommend the combined use of both traditional and crowdsourced data on organizational characteristics for both research and practice.

Download Full-text

Construct Validity of the French Shirom-Melamed Vigor Measure (F-SMVM)

European Journal of Psychological Assessment ◽

10.1027/1015-5759/a000518 ◽

2020 ◽

Vol 36 (2) ◽

pp. 372-386 ◽

Cited By ~ 2

Author(s):

Sandrine Isoard-Gautheur ◽

Clément Ginoux ◽

Jean-Philippe Heuzé ◽

Damien Tessier ◽

David Trouilloud ◽

...

Keyword(s):

Job Satisfaction ◽

Construct Validity ◽

Work Engagement ◽

Concurrent Validity ◽

Convergent Validity ◽

Discriminant Validity ◽

Work Motivation ◽

Convergent And Discriminant Validity ◽

Confirmatory Factor ◽

Abstract Work

Abstract. Work vigor is defined by cognitive, affective, and behavioral components. The purpose of this multi-study paper was to develop a French version of the Shirom-Melamed Vigor Measure (F-SMVM), and test its (a) construct validity (i.e., factorial structure and internal consistency), (b) convergent and discriminant validity with the Utrecht Work Engagement Scale (UWES-9), and (c) concurrent validity with work motivation and job satisfaction. Six hundred and forty (61.72% female) employees participated in the three present studies. Convergent and discriminant construct validity of the F-SMVM were examined with a confirmatory factor analysis (CFA) and a multitrait-multimethod (MTMM) analysis. Concurrent validity was examined through correlations and regressions with work motivation and job satisfaction. Overall, the results supported the reliability and the construct validity of the F-SMVM. They also showed evidence of convergent validity between the F-SMVM and the UWES-9, as well as evidence of concurrent validity with work motivation and job satisfaction. The F-SMVM represents a valid measure assessing three interrelated dimensions representing physical, cognitive, and emotional components of work engagement.

Download Full-text

Validity of Single-item Versus Multiple-item Job Satisfaction Measures in Predicting Life: Satisfaction and Turnover Intention

Asia-Pacific Journal of Management Research and Innovation ◽

10.1177/2319510x21997724 ◽

2020 ◽

Vol 16 (3) ◽

pp. 210-228

Author(s):

Sunday B. Fakunmoju

Keyword(s):

Job Satisfaction ◽

Life Satisfaction ◽

Turnover Intention ◽

Convergent Validity ◽

Discriminant Validity ◽

Cross Sectional ◽

Single Item ◽

Human Service Professionals ◽

Multiple Item ◽

Satisfaction Measures

The use of single-item (SI) measures to operationalise a construct has endured extensive methodological critique, and its practical advantages over multiple-item (MI) measures pale in comparisons to the theoretical advantages of MI measures. Among constructs that have been operationalised with a single item, job satisfaction remains one of the favourites, although empirical knowledge about its validity is limited to data derived from management, marketing and human resources fields. Little is known about its validity among human service professionals. Using two cross-sectional surveys, the present article describes validity of SI versus MI measures of job satisfaction in predicting life satisfaction and turnover intention among social workers across organisations and professional specialisations, including supervisors, managers and administrators. Results in both studies suggested that SI measure of job satisfaction was methodologically and analytically suitable for examining job-related outcomes. It established convergent validity with MI job satisfaction measures and discriminant validity with job dissatisfaction measures. It demonstrated comparable demographic outcomes, association and predictive relationship with life satisfaction and turnover intention in the same magnitude as MI job satisfaction. It accounted for comparable variance in life satisfaction and turnover intention and generated bivariate, multivariate and mediation model outcomes that are systematically similar to those of MI job satisfaction measures. The article stipulates conditions for the use of SI job satisfaction measures, offers suggestions about how to resolve methodological impasse in choosing between SI and MI measures, and concludes with recommendations that include criteria for choosing between SI and MI measures for research.

Download Full-text

CODEM Instrument

GeroPsych ◽

10.1024/1662-9647/a000100 ◽

2014 ◽

Vol 27 (1) ◽

pp. 23-31 ◽

Cited By ~ 4

Author(s):

Anne Kuemmel (This author contributed eq ◽

Julia Haberstroh (This author contributed ◽

Johannes Pantel

Keyword(s):

Convergent Validity ◽

Interrater Reliability ◽

Discriminant Validity ◽

Assessment Tool ◽

Intraclass Correlation ◽

Well Being ◽

Communication Behavior ◽

People With Dementia ◽

Pearson's R ◽

Pearson’S R

Communication and communication behaviors in situational contexts are essential conditions for well-being and quality of life in people with dementia. Measuring methods, however, are limited. The CODEM instrument, a standardized observational communication behavior assessment tool, was developed and evaluated on the basis of the current state of research in dementia care and social-communicative behavior. Initially, interrater reliability was examined by means of videoratings (N = 10 people with dementia). Thereupon, six caregivers in six German nursing homes observed 69 residents suffering from dementia and used CODEM to rate their communication behavior. The interrater reliability of CODEM was excellent (mean κ = .79; intraclass correlation = .91). Statistical analysis indicated that CODEM had excellent internal consistency (Cronbach’s α = .95). CODEM also showed excellent convergent validity (Pearson’s R = .88) as well as discriminant validity (Pearson’s R = .63). Confirmatory factor analysis verified the two-factor solution of verbal/content aspects and nonverbal/relationship aspects. With regard to the severity of the disease, the content and relational aspects of communication exhibited different trends. CODEM proved to be a reliable, valid, and sensitive assessment tool for examining communication behavior in the field of dementia. CODEM also provides researchers a feasible examination tool for measuring effects of psychosocial intervention studies that strive to improve communication behavior and well-being in dementia.

Download Full-text

Validation of the German Version of the Exercise Dependence Scale

European Journal of Psychological Assessment ◽

10.1027/1015-5759/a000144 ◽

2013 ◽

Vol 29 (3) ◽

pp. 213-219 ◽

Cited By ~ 13

Author(s):

Astrid Müller ◽

Laurence Claes ◽

Dirk Smits ◽

Olaf Gefeller ◽

Anja Hilbert ◽

...

Keyword(s):

Convergent Validity ◽

Discriminant Validity ◽

Factor Model ◽

Population Based ◽

Exercise Dependence ◽

German Population ◽

German Version ◽

Internal Reliability ◽

Eating Disorder Examination ◽

Dependence Scale

This study evaluated the psychometric properties of the German version of the 21-item Exercise Dependence Scale (EDS-G), which was administered in a large representative German sample (N = 1,611). In order to examine convergent validity, we also asked participants to answer the Eating Disorder Examination-Questionnaire (EDE-Q). To investigate the discriminant validity, clients of fitness centers (N = 129), sport students (N = 85), and medical students (N = 129) filled out the EDS-G. In this German population-based sample, the seven-factor structure suggested by the original authors as well as a higher-order model (7 + 1 factor model) were supported. The total EDS-G scale showed an excellent internal reliability and was positively related to the EDE-Q total and subscale scores. The EDS-G scores differentiated between samples with varying degrees of exercise. In summary, the results indicate that the EDS-G is a psychometrically sound and potentially valid brief measure for the assessment of exercise dependence symptoms.

Download Full-text

Development and Construct Validation of the German Workplace Trust Survey (G-WTS)

European Journal of Psychological Assessment ◽

10.1027/1015-5759/a000002 ◽

2010 ◽

Vol 26 (1) ◽

pp. 3-10 ◽

Cited By ~ 17

Author(s):

Nale Lehmann-Willenbrock ◽

Simone Kauffeld

Keyword(s):

Convergent Validity ◽

Affective Commitment ◽

Discriminant Validity ◽

Organizational Context ◽

Organizational Trust ◽

Three Dimensional ◽

Future Research ◽

Work Related ◽

Reliable Assessment ◽

Workplace Trust

In research on trust in the organizational context, there is some agreement evolving that trust should be measured with respect to various foci. The Workplace Trust Survey (WTS) by Ferres (2002) provides reliable assessment of coworker, supervisor, and organizational trust. By means of a functionally equivalent translation, we developed a German version of the questionnaire (G-WTS) comprising 21 items. A total of 427 employees were surveyed with the G-WTS and questionnaires concerning several work-related attitudes and behaviors and 92 of these completed the survey twice. The hypothesized three-dimensional conceptualization of organizational trust was confirmed by confirmatory factor analysis. The G-WTS showed good internal consistency and retest reliability values. Concerning convergent validity, all of the three G-WTS dimensions positively predicted job satisfaction. In terms of discriminant validity, Coworker Trust enhanced group cohesion; Supervisor Trust fostered innovative behavior, while Organizational Trust was associated with affective commitment. Theoretical and practical contributions as well as opportunities for future research with the G-WTS are discussed.

Download Full-text

Development of the Fear of Intimacy Components Questionnaire (FICQ)

European Journal of Psychological Assessment ◽

10.1027/1015-5759/a000238 ◽

2015 ◽

Vol 31 (4) ◽

pp. 302-309 ◽

Cited By ~ 3

Author(s):

Maria Pedro Sobral ◽

Maria Emília Costa

Keyword(s):

Convergent Validity ◽

Discriminant Validity ◽

Two Dimensions ◽

Dimensional Structure ◽

Future Research ◽

Fear Of Intimacy ◽

Confirmatory Factor Analyses ◽

New Instrument ◽

Confirmatory Factor ◽

Test Retest Reliability

Abstract. We developed a new instrument designed to measure fear of intimacy in romantic relationships. We suggest assessing fear of intimacy through two dimensions: self-revelation and dependence. The Fear of Intimacy Components Questionnaire (FICQ) was validated across three studies in which a 10-item solution systematically emerged. Consistently with a two component perspective, a two-factor solution fitted data the best: fear of losing the self (FLS) and fear of losing the other (FLO). Qualitative analyses verified content validity. Exploratory and confirmatory factor analyses tested the factor structure. Multigroup analyses supported the structural invariance across gender, age, and relationship status. Both factors showed adequate discriminant validity and internal consistency, and good 3-week period test-retest reliability. Associations between the FICQ and insecure attachment orientations demonstrated convergent validity. The association between the FICQ and relationship satisfaction above and beyond a preexisting measure offered criterion validity. By going beyond traditional self-revelation-focused conception of fear of intimacy, that is, by proposing a bi-dimensional structure to fear of intimacy, we believe that this new measure will contribute to future research on fear of intimacy.

Download Full-text

PENGEMBANGAN SKALA LOCUS OF CONTROL

Jurnal Riset Manajemen dan Bisnis (JRMB) Fakultas Ekonomi UNIAT ◽

10.36226/jrmb.v2i3.76 ◽

2017 ◽

Vol 2 (3) ◽

pp. 417-424

Author(s):

Hendryadi Hendryadi

Keyword(s):

Locus Of Control ◽

Internal Consistency ◽

Convergent Validity ◽

Discriminant Validity ◽

Short Form ◽

Employee Selection ◽

Future Research ◽

Validity And Reliability ◽

Control Scale ◽

Two Stages

This article aims to develop a short form of the locus of control scale. The study was conducted in two stages: a study of 66 respondents as pilot testing which aims to test content validity, structure validity, and internal consistency. Study 2 was conducted on 328 respondents used to test the validity and reliability of the scale evaluated by the PLS-SEM method (such as internal consistency, convergent validity, and discriminant validity). The analysis concludes that the 8-item locus of control scales tested have adequate validity and reliability. A short form locus of control scale was developed and validated in this study, so it can be used in future research and evaluation for HR management practitioners in employee selection Keywords: locus of control, EFA, CFA, scale construction

Download Full-text

An empirically verified project risk maturity model

International Journal of Managing Projects in Business ◽

10.1108/ijmpb-03-2013-0015 ◽

2014 ◽

Vol 7 (2) ◽

pp. 263-284 ◽

Cited By ~ 17

Author(s):

Budi Hartono ◽

Deo F.N. Wijaya ◽

Hilya M. Arini

Keyword(s):

Construction Industry ◽

Convergent Validity ◽

Discriminant Validity ◽

Face Validity ◽

Maturity Model ◽

Maturity Level ◽

Cross Sectional Survey ◽

Project Risk ◽

Content Type ◽

Risk Maturity

Purpose – The purpose of this paper is to develop and to empirically verify a model of project risk management maturity (PRMM). Design/methodology/approach – Theoretical work to develop the initial model of risk maturity. Empirical study by a cross-sectional survey to the Indonesian construction industry. Findings – A new model of PRMM is developed and empirically tested. The model is valid (face validity, content validity, discriminant validity, convergent validity, and criterion validity) and reliable. Research limitations/implications – A more comprehensive, follow-up study is required to gain more insights on the actual maturity level of Indonesian construction industry. Practical implications – The model is applicable to assess the organizational maturity level which in turn could be used for improving organization performance. Originality/value – The work demonstrates a novel approach in developing models by emphasizing on the empirical verification.

Download Full-text

Development of the Adult Pandemic Attitude Scale (A-PAS)

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph18126311 ◽

2021 ◽

Vol 18 (12) ◽

pp. 6311

Author(s):

Mihyeon Seong ◽

Juyoung Park ◽

Soojin Chung ◽

Sohyune Sok

Keyword(s):

Construct Validity ◽

Convergent Validity ◽

Discriminant Validity ◽

Item Analysis ◽

Preliminary Investigation ◽

Attitude Scale ◽

Face Validity ◽

Validity And Reliability ◽

Five Dimensions ◽

Development Step

This study aimed to develop an instrument for measuring the attitudes that reflect the characteristics of the pandemic (Adult Pandemic Attitude Scale (A-PAS)) and verifying its validity and reliability. This study used a methodological research design and was conducted with a development step and an evaluation step. The development step included development of preliminary items, content validity, face validity, and preliminary investigation. The evaluation step included item analysis, construct validity, convergent validity, discriminant validity, criterion validity, factor naming, reliability, and completion of the final instrument. The A-PAS developed in this study consisted of a total of 20 items in five dimensions. The internal consistency of 20 items of the A-PAS, Cronbach’s α was 0.92 for 20 items, Cronbach’s α for each factor, a subscale of instrument, was 0.61~0.87 and Raykov’s p coefficient of each factor, which is a subscale of the tool, was found to be 0.60 to 0.88. Analysis of construct validity showed the results as follows: χ2 (p) = 134.05 (p < 0.001), RMSEA = 0.02, RMR = 0.02, GFI = 0.94, CFI = 0.99. The study findings suggest that the developed instrument can be utilized to measure the attitudes of adults toward pandemics, and reflect the reality of the pandemic situation. The outcomes can be used as valuable data for intervention, prevention activities, and policy preparation. The instrument will be applied in the event of a pandemic, such as COVID-19, and will be helpful in promoting the health of the people.

Download Full-text

Unpacking the Quantifying and Qualifying Potential of Semi-Open Job Satisfaction Questions through Computer-Aided Sentiment Analysis

Journal of Well-Being Assessment ◽

10.1007/s41543-021-00040-w ◽

2021 ◽

Cited By ~ 1

Author(s):

Indy Wijngaards ◽

Martijn Burger ◽

Job van Exel

Keyword(s):

Job Satisfaction ◽

Sentiment Analysis ◽

Text Analysis ◽

Discriminant Validity ◽

Added Value ◽

Information Richness ◽

Convergent And Discriminant Validity ◽

Computer Aided ◽

Satisfaction Question ◽

Open Question

AbstractDespite their suitability for mitigating survey biases and their potential for enhancing information richness, open and semi-open job satisfaction questions are rarely used in surveys. This is mostly due to the high costs associated with manual coding and difficulties that arise when validating text measures. Recently, advances in computer-aided text analysis have enabled researchers to rely less on manual coding to construct text measures. Yet, little is known about the validity of text measures generated by computer-aided text analysis software and only a handful of studies have attempted to demonstrate their added value. In light of this gap, drawing on a sample of 395 employees, we showed that the responses to a semi-open job satisfaction question can reliably and conveniently be converted into a text measure using two types of computer-aided sentiment analysis: SentimentR, and Linguistic Inquiry and Word Count (LIWC) 2015. Furthermore, the substantial convergence between the LIWC2015 and, in particular, SentimentR measure with a closed question measure of job satisfaction and logical associations with closed question measures of constructs that fall within and outside job satisfaction’s nomological network, suggest that a semi-open question has adequate convergent and discriminant validity. Finally, we illustrated that the responses to our semi-open question can be used to fine-tune the computer-aided sentiment analysis dictionaries and unravel antecedents of job satisfaction.

Download Full-text