Is It Biased? Empirical Analysis of Various Phenomena That Affect Survey Results

It is often assumed that survey results reflect only the quality of the sample and the underlying measuring instruments used in the survey. However, various phenomena can affect the results, but these influences are often neglected when conducting surveys. This study aimed to test the influences of various method effects on survey results. We tested the influences of the following method effects: item wording, confirmatory bias, careless responding, and acquiescence bias. Using a split-ballot survey design with online questionnaires, we collected data from 791 participants. We tested if these method effects had an influence on mean values, item correlations, construct correlations, model fits, and construct measurement invariance. The instruments used to test these influences were from the domain of personality and gender inequality, and their items were adapted based on the method effect tested. All tested method effects, except careless responding, had a statistically significant effect on at least one component of the analysis. Item wording and confirmatory bias affected mean values, model fit, and measurement invariance. Controlling for acquiescence bias improved the fit of the model. This paper confirms that the tested method effects should be carefully considered when using surveys in research, and suggests some guidelines on how to do so.

Psychometric evaluation of the Clinical Outcome in Routine Evaluation – General Population: Czech version

Objectives. This study aimed to assess psychometric properties, such as reliability, construct validity, and cut-off scores, for the Czech version of the Clinical Outcome in Routine Evaluation – General Population (GP-CORE) questionnaire, a tool usable for repeated measurement of psychological distress within routine clinical settings. Participants and setting. Two general populations and one clinical sample were used with N values of 420, 394, and 345, respectively. Hypotheses. One of the competing theoretical factor solutions will demonstrate the best fit. Statistical analysis. To examine the factor structure of the GP-CORE, a confirmatory multidimensional item response theory analysis (graded response model) was employed. Results. The best fitting model was a bifactor solution representing one content domain of overall distress and two item wording domains (positively and negatively worded items). Clinical cut-off scores were determined to be 1.85 (men) and 1.90 (women). Study limitations. The GP-CORE can be used as an unidimensional measure of overall distress, but users have to be aware of the influence of positive vs. negative item wording on the responses.

人々がマスクを着用する理由とは ――国内研究の追試とリサーチクエスチョンの検証――

Wearing masks is an easy and effective way to prevent infection by COVID-19. In Japan, two studies investigated the reasons why people wear a mask; however, the results were inconsistent. Therefore, the present study reexamined the association between mask wearing and the reasons given for doing so, focusing on the differences in and problems of item wording. The results of both studies were found to be almost reproducible, and there were few problems regarding item wording. Furthermore, it was found that people wear a mask to prevent themselves and others from infection and to conform to other people wearing a mask. We suggest that inconsistencies in the results were due to differences in item wording, and discuss how future research should be conducted.

An Examination of the Spanish Translation of the 50-item International Personality Item Pool Big-five Inventory in a Spanish Speaking Peruvian Sample

The International Personality Item Pool (IPIP) five-factor model inventories are widely used for personality research and have been translated into multiple languages. However, the extent of the psychometric assessment of translated scales is variable, often minimal. The lack of psychometric scrutiny is particularly problematic because translation is an inherently complex process. Here, we present a structural analysis of one Spanish translation of the 50-item IPIP five-factor inventory in a sample of Peruvian, non-university educated, working adults (n = 778). A global confirmatory factor analytic (CFA) model of the a priori five factors failed to fit. So too did single factor models for four of the five factors, the exception being Neuroticism. Fit was improved via use of an exploratory structural equation measurement model, but the resultant solution showed very poor theoretical coherence. So, we explored the data for systematic measurement artefacts and sought to model them to improve the psychometric properties of the scale. Specifically, the pattern of factor loadings suggested that the lack of coherence might be due to the effects of the valence of item wording (i.e., positively or negatively worded items). CFA models including five substantive factors and a series of method factors modelling shared covariance based on item wording, improved fit and coherence. This investigation suggests that unless method factors are explicitly modelled the tested Spanish translation may not be suitable for use in certain Spanish-speaking countries or samples composed of non-university educated participants. More broadly, the study has implications for many translated scales, especially when used without thorough psychometric evaluation.

Development of Patient-Reported Outcome Measures (Symptoms and Impacts) in Adults with Pyruvate Kinase Deficiency

Introduction: Pyruvate kinase (PK) deficiency is a rare, congenital autosomal recessive hemolytic anemia managed with supportive treatments, including transfusion, splenectomy, and iron chelation. Disease-directed treatments, including a small molecule PK activator and gene therapy, are currently in development. No disease-specific patient-reported outcome (PRO) measures have been validated for use in this patient population. The objective of this initiative was to develop PRO measures for assessing symptoms and impacts of PK deficiency and compare them to existing, non-disease-specific measures previously recommended for use in this disease area. Methods: A targeted literature review was conducted to inform the development of a preliminary hypothesized conceptual framework to identify signs, symptoms, and impacts commonly experienced by patients with PK deficiency and to inform the direction and content of interviews with such patients. Concept elicitation interviews were conducted with 21 adults with PK deficiency from the US, Netherlands, and Germany. Draft items were then tested in cognitive interviews with 20 adults with PK deficiency to further establish content validity and revised based on the results. A comparison was conducted between concepts included in the newly developed PK deficiency disease-specific measures and the domain structure and item concepts included in the EORTC QLQ-C30 and SF-36v2 to evaluate the extent of differences and conceptual overlap with instruments that had previously been recommended in this population. Specific attributes compared included face validity (i.e., conceptual coverage and inclusion of proximal symptoms and/or impacts) and measurement characteristics (i.e., item wording, recall, and response options). Results: Two measures, the PK Deficiency Diary (PKDD), a 7-item measure of the core signs and symptoms of PK deficiency, and PK Deficiency Impact Assessment (PKDIA), a 14-item measure of the impacts of PK deficiency on patients' HRQoL, were developed. A comparison of the newly drafted measures to the EORTC QLQ-C30 and SF-36v2 demonstrated minimal similarities in concepts, domains, item wording, and recall period. Of the 7 concepts in the PKDD, only 3 were common to the EORTC QLQ-C30, 4 were common to the SF36v2, and 2 were related but did not match exactly (i.e., "bone pain" and "pain"). Of the 12 distinct concepts in the PKDIA, only 5 were common to the EORTC QLQ-C30, 2 were common to the SF-36v2, and 3 were related but did not match exactly (i.e., difficulty starting things, difficulty finishing things, and difficulty performing moderate physical activity). Conclusions: This research demonstrates that the EORTC-QLQ-C30 and SF-36v2 lack the appropriate conceptual relevance and coverage of disease-specific signs, symptoms, and impacts most relevant and burdensome to patients with PK deficiency. The newly developed PKDD and PKDIA may be useful tools in clinical trials in patients with PK deficiency. Psychometric validation of these measures is currently underway. Disclosures Salek: Pfizer: Honoraria, Speakers Bureau; Merck: Consultancy; Agios Pharmaceuticals, Inc.: Consultancy, Honoraria. Boscoe:Agios Pharmaceuticals, Inc.: Employment, Equity Ownership. Evans:Agios Pharmaceuticals, Inc.: Consultancy, Research Funding. Egan:Agios Pharmaceuticals, Inc.: Consultancy, Research Funding. Wells:Agios Pharmaceuticals, Inc.: Consultancy, Research Funding. Piantedosi:Agios Pharmaceuticals, Inc.: Employment. Grace:Agios Pharmaceuticals, Inc: Consultancy, Membership on an entity's Board of Directors or advisory committees, Research Funding; Novartis: Research Funding. Storm:Agios: Employment.