Reliability of Nominal Data Based on Qualitative Judgments

1989 ◽  
Vol 26 (2) ◽  
pp. 135-148 ◽  
Author(s):  
William D. Perreault ◽  
Laurence E. Leigh
Keyword(s):  
Content Analysis ◽  
Meta Analysis ◽  
Marketing Research ◽  
Reliability And Validity ◽  
Cognitive Response ◽  
Nominal Data ◽  
Consumer Responses ◽  
Nominal Scale ◽  
Scale Data

Most research related to the reliability and validity of marketing measures has focused on multi-item quantitative scales. In contrast, little attention has been given to the quality of nominal scale data developed from qualitative judgments. Judgment-based (“coded”) nominal scale data are important and frequently used in marketing research-for example, in analysis of consumer responses to open-ended survey questions, in cognitive response research, in meta-analysis, and in content analysis. The authors address opportunities and challenges involved in evaluating and improving the quality of judgment-based nominal scale data, with specific emphasis on the use of multiple judges. They review approaches commonly used in other disciplines, then develop a new index of reliability that is more appropriate for the type of interjudge data typically found in marketing studies. Data from a cognitive response experiment are used to illustrate the new index and compare it with other common measures. The authors conclude with suggestions on how to improve the design of studies that rely on judgment-coded data.

Is the empirical validity of prior experience for performance affected by rating approach?

2014 ◽  
Vol 43 (3) ◽  
pp. 438-463 ◽  
Author(s):  
Dan Baugher ◽  
Ellen Weisbord ◽  
Chris Ramos
Keyword(s):  
Prior Experience ◽  
Meta Analysis ◽  
Reliability And Validity ◽  
Job Applicants ◽  
Validity Data ◽  
Content Type ◽  
The Public ◽  
Supervisory Ratings ◽  
The One

Purpose – In the public sector, Training and Experience (T & E) exams assess prior experience and are one of the most often used methods for selecting job applicants. This study uses a KSA approach, where raters judge the quality of job relevant prior experience, not its duration or quantity. It was hypothesized that an additional rater and a consensus meeting between raters would increase reliability and validity. Design/methodology/approach – T & E and supervisory ratings were obtained over a 12-year period for 166 candidates seeking promotion to a budget analyst position. Validity was measured by the correlation between T & E scores and supervisory ratings. Consensus was required only for T & E scores differing by a specific amount (hybrid consensus). Findings – Intraclass reliability was 0.73, 0.84, and 0.95 in the one-rater, two-rater, and hybrid consensus conditions with each coefficient greater than the next (p < 0.05) showing the benefit of multiple raters and consensus for reliability. Validity was significant at 0.21, 0.26, and 0.251 for each rating condition, respectively (two-tail test; p < 0.01). Validity was greater in the two-rater condition than in the one-rater condition (one-tail test; p < 0.05). Consensus did not improve validity beyond that of two raters. For consensus T & Es (n=76), two raters improved validity (one-tail test; p < 0.05), moving from 0.112 to 0.231 but not reliability; consensus improved reliability (two-tail test; p < 0.05) but not validity. Originality/value – There has been a vacuum in T & E research for close to 20 years. Validity data are difficult to obtain but critical for meta-analysis. T & Es showed validity. Use of two raters improved validity but consensus did not increase the gain.

Introduction to the Special Section: Measurement and Marketing Research—Is the Flirtation Going to Lead to a Romance?

1979 ◽  
Vol 16 (1) ◽  
pp. 1-5 ◽  
Author(s):  
Michael L. Ray ◽  
Gilbert A. Churchill
Keyword(s):  
Content Analysis ◽  
Special Section ◽  
Marketing Research ◽  
Reliability And Validity ◽  
The Past ◽  
Section Measurement ◽  
Practical Guidelines

The articles in this special section provide practical guidelines and examples for carrying out reliability and validity measurement testing, which is essential to the advancement of marketing research. A content analysis of JMR articles shows increasing concern with these issues in the past five years. A special AMA committee and a new JMR section are proposed to move the field beyond concern (flirtation) and into action (romance).

scholarly journals Serology for the diagnosis of human hepatic cystic echinococcosis and its relation with cyst staging: A systematic review of the literature with meta-analysis

2021 ◽  
Vol 15 (4) ◽  
pp. e0009370
Author(s):  
Francesca Tamarozzi ◽  
Ronaldo Silva ◽  
Veronica Andrea Fittipaldo ◽  
Dora Buonfrate ◽  
Bruno Gottstein ◽  
...  
Keyword(s):  
Systematic Review ◽  
Cystic Echinococcosis ◽  
Meta Analysis ◽  
Review Of The Literature ◽  
Serological Tests ◽  
Consistent Finding ◽  
Newcastle Ottawa Scale ◽  
Scale Data ◽  
Ottawa Scale

Background The diagnosis of cystic echinococcosis (CE) is primarily based on imaging, while serology should be applied when imaging is inconclusive. CE cyst stage has been reported among the most important factors influencing the outcome of serodiagnosis. We performed a systematic review and meta-analysis of the relation between cyst stage of hepatic CE and diagnostic sensitivity of serological tests, to evaluate whether their relation is a consistent finding and provide guidance for the interpretation of results of serological tests. Methodology/Principal findings MEDLINE, EMBASE, CENTRAL, and Lilacs databases were searched on December 1st 2019. Original studies published after 2003 (year of publication of the CE cyst classification), reporting sensitivity of serological tests applied to the diagnosis of human hepatic CE, as diagnosed and staged by imaging, were included. The quality of studies was assessed using the Newcastle-Ottawa Scale. Data from 14 studies were included in the meta-analysis. Summary estimates of sensitivities and 95% confidence intervals were obtained using random effects meta-analysis. Overall, test sensitivity was highest in the presence of CE2 and CE3 (CE3a and/or CE3b), and lowest in the presence of CE5 and CE4 cysts. ELISA, ICT and WB showed the highest sensitivities, while IHA performed worst. Conclusions/Significance The results of our study confirm the presence of a clear and consistent relation between cyst stage and serological tests results. Limitations of evidence included the heterogeneity of the antigenic preparations used, which prevented to determine whether the relation between cyst stage and sensitivity was influenced by the type of antigenic preparation, the paucity of studies testing the same panel of sera with different assays, and the lack of studies assessing the performance of the same assay in both field and hospital-based settings. Our results indicate the absolute need to consider cyst staging when evaluating serological results of patients with hepatic CE.

scholarly journals The Influence of Competence Potential of Changes in Higher Education on the Training of Specialists for Entrepreneurship

2020 ◽  
Vol 38 (4) ◽  
pp. 49-68
Author(s):  
Nataliya Vnukova ◽  
Svitlana Achkasova ◽  
Maryna Us ◽  
Vasyl Pyvovarov
Keyword(s):  
Higher Education ◽  
Content Analysis ◽  
Education System ◽  
Marketing Research ◽  
International Standards ◽  
Approach To Learning ◽  
Higher Education System ◽  
Key Competences

AbstractThe paper presents the substantiation of the need to reform the higher education system of Ukraine in line with European and international standards and requirements, providing a full transition to competentive approach to learning. The essence and advantages of using competency potential for improvement of the higher education system in Ukraine are determined. A comprehensive marketing research are conducted, using the method of content analysis that helped to reveal the modern competencies needed for students and graduates of financial and legal specialties. In addition, a survey was conducted which showed that there were problems with the level and quality of the formation of professional and key competences.

Self-efficacy and quality of life among people with cardiovascular diseases: A meta-analysis.

2018 ◽  
Vol 63 (2) ◽  
pp. 295-312 ◽  
Author(s):  
Anna Banik ◽  
Ralf Schwarzer ◽  
Nina Knoll ◽  
Katarzyna Czekierda ◽  
Aleksandra Luszczynska
Keyword(s):  
Quality Of Life ◽  
Cardiovascular Diseases ◽  
Self Efficacy ◽  
Meta Analysis

META-ANALYSIS ON AMERICAN PRESIDENTIAL ELECTION - 2016 AND AMERICAN STOCK MARKET

2020 ◽  
Vol 38 (1) ◽  
Author(s):  
Farhan Ahmed ◽  
Salman Bahoo ◽  
Sohail Aslam ◽  
Muhammad Asif Qureshi
Keyword(s):  
Content Analysis ◽  
Stock Market ◽  
Event Study ◽  
Presidential Election ◽  
Republican Party ◽  
Stock Exchange ◽  
Meta Analysis ◽  
Composite Index ◽  
Stock Index ◽  
Political Event

This paper aims to analyze the efficient stock market hypothesis as responsive to American Presidential Election, 2016. The meta-analysis has been done combining content analysis and event study methodology. The all major newspapers, news channels, public polls, literature and five important indices as Dow Jones Industrial Average (DJIA), NASDAQ Stock Market Composit Indexe (NASDAQ-COMP), Standard & Poor's 500 Index (SPX-500), New York Stock Exchange Composite Index (NYSE-COMP) and Other U.S Indexes-Russell 2000 (RUT-2000) are critically examined and empirically analyzed. The findings from content analysis reflect that stunned winning of Mr Trump from Republican Party worked as shock for American stock market. From event study, findings confirmed that all the major indices reflected a decline on winning of Trump and losing of Ms. Clinton from Democratic. The results are supported empirically and practically through the political event like BREXIT that resulted in shock to Global stock index and loss of $2 Trillion.

Sign in / Sign up

Export Citation Format

Share Document