Generalizability theory

2015 ◽  
Author(s):  
David L. Streiner ◽  
Geoffrey R. Norman ◽  
John Cairney
Keyword(s):  
Basic Concept ◽  
Generalizability Theory ◽  
Optimal Strategies ◽  
Relative Magnitude ◽  
Reliability Theory ◽  
Fixed Number ◽  
Multiple Sources ◽  
Detailed Review ◽  
Sources Of Error ◽  
Different Sources

This chapter is a detailed review of generalizability theory (G theory), an extension of classical reliability theory originally devised by Cronbach. The basic concept is that any measurement contains multiple sources of error, and through the use of G theory these various sources can be calculated in a single study. This permits the researcher to examine the relative magnitude of different sources of error and the relation among them. The power of the method rests in its ability to use these estimates to devise optimal strategies for distributing observations. That is, G theory can be used to determine how a fixed number of observations should be distributed across raters, occasions, or cases to optimize reliability.

scholarly journals Estimating Measurement Errors and Score Dependability of NECO 2019 Mathematics Examination

2021 ◽  
Vol 2 (3) ◽  
pp. 50-55
Author(s):  
James Uzomba Okeaba ◽  
Nkechi Patricia-Mary Esomonu
Keyword(s):  
Measurement Error ◽  
Measurement Errors ◽  
Rank Order ◽  
Generalizability Theory ◽  
Simple Random Sampling ◽  
Error Variance ◽  
Multiple Sources ◽  
Sources Of Error ◽  
Mathematics Examination ◽  
Research Questions

This study estimated measurement error and score dependability in examinations using the Generalizability Theory. Scores obtained by the students (object of measurements) in examinations are affected by multiple sources of error (facets), and these scores are used in taking relative and absolute decisions about the students. There is, therefore, needed to estimate measurement error and score dependability to find the extent of the contributions of the facets to error in examination scores. Three research questions and one hypothesis were used to guide the study. The study population comprised 5,085 SS3 students of the 34 Government-owned senior secondary schools in Yenagoa LGA of Bayelsa State 2019/2020 academic session. 10 schools were selected using simple random sampling technic and the 1,525 SS3 students of the selected schools formed the sample. section A of the 2018 NECO Mathematics main paper and 2018 NECO Mathematics Marking Scheme were used to collect the data. EduG version 6.0-e based on ANOVA and Generalizability theory was used to answer the three research questions. A 95% confidence interval was computed using the standard error variance components to test the hypothesis. The findings of the study revealed that some hidden sources of error were at play in the study. the students’ facets (σ2S) made the highest contribution to measurement error in examination scores followed by the residual (σ2SIM). Also, the Students’ facet was significantly different (p< 0.05) in its contributions to measurement error, while the other facets and their interactions were not significantly different in their contribution to measurement error. Hence, Ho1 was not accepted for the students’ facet but accepted for other facets. An increase in the level of markers from 1 to 4 with level of items 5 yielded an outcome of 0.84 to 0.91 respectively, a generalizability coefficient of 0.94 high enough to rank order students according to their relative abilities in examinations was obtained when the level of markers was at 2 with an increment in level of items to 10. An index of dependability of 0.93 that is high enough to maximize reliability was obtained when we have level of markers at 2 and the items at 10.

Dependability of Measurement in Counseling Psychology

1999 ◽  
Vol 27 (3) ◽  
pp. 325-352 ◽  
Author(s):  
William T. Hoyt ◽  
Janet N. Melby
Keyword(s):  
Statistical Power ◽  
Generalizability Theory ◽  
Future Research ◽  
Multiple Sources ◽  
Sources Of Error ◽  
Total Proportion ◽  
Observer Ratings ◽  
Flexible Framework ◽  
Flexible Application ◽  
Estimation Of Variance

Classical approaches to the assessment of reliability neglect to take into account multiple sources of error and to consider diverse measurement contexts. Generalizability theory (GT) offers a flexible framework for assessing dependability of measurement. With GT, investigators can estimate the total proportion of variance in ratings that is due to error rather than focusing on one source of error at a time. Simultaneous consideration of multiple sources of error allows investigators to assess the overall impact of measurement error in terms of attenuation of study findings and reduction of statistical power. Estimation of variance components allows for flexible application of findings to a variety of possible future research designs. Illustrative analyses demonstrate the special advantages of GT for planning studies in which observer ratings will be used.

Revision of a Criterion-Referenced Vocabulary Test Using Generalizability Theory

2009 ◽  
Vol 31 (1) ◽  
pp. 81
Author(s):  
Takeaki Kumazawa
Keyword(s):  
Generalizability Theory ◽  
Classical Test Theory ◽  
Performance Testing ◽  
Error Variance ◽  
Test Theory ◽  
Vocabulary Test ◽  
Multiple Sources ◽  
Generalizability Coefficient ◽  
Criterion Referenced ◽  
Generalizability Study

Classical test theory (CTT) has been widely used to estimate the reliability of measurements. Generalizability theory (G theory), an extension of CTT, is a powerful statistical procedure, particularly useful for performance testing, because it enables estimating the percentages of persons variance and multiple sources of error variance. This study focuses on a generalizability study (G study) conducted to investigate such variance components for a paper-pencil multiple-choice vocabulary test used as a diagnostic pretest. Further, a decision study (D study) was conducted to compute the generalizability coefficient (G coefficient) for absolute decisions. The results of the G and D studies indicated that 46% of the total variance was due to the items effect; further, the G coefficient for absolute decisions was low. 古典的テスト理論は尺度の信頼性を測定するため広く用いられている。古典的テスト理論の応用である一般化可能性理論（G理論）は特にパフォーマンステストにおいて有効な分析手法であり、受験者と誤差の要因となる分散成分の割合を測定することができる。本研究では診断テストとして用いられた多岐選択式語彙テストの分散成分を測定するため一般化可能性研究（G研究）を行った。さらに、決定研究（D研究）では絶対評価に用いる一般化可能性係数を算出した。G研究とD研究の結果、項目の分散成分が全体の分散の46%を占め、また信頼度指数は高くなかった。

XML Data Integration

2010 ◽  
pp. 333-360 ◽  
Author(s):  
Yan Qi ◽  
Huiping Cao ◽  
K. Selçuk Candan ◽  
Maria Luisa Sapino
Keyword(s):  
Data Integration ◽  
Query Processing ◽  
Multiple Sources ◽  
Data Types ◽  
Xml Data ◽  
Data Schema ◽  
Integration Data ◽  
Feedback Techniques ◽  
Different Sources ◽  
Xml Data Integration

In XML Data Integration, data/metadata merging and query processing are indispensable. Specifically, merging integrates multiple disparate (heterogeneous and autonomous) input data sources together for further usage, while query processing is one main reason why the data need to be integrated in the first place. Besides, when supported with appropriate user feedback techniques, queries can also provide contexts in which conflicts among the input sources can be interpreted and resolved. The flexibility of XML structure provides opportunities for alleviating some of the difficulties that other less flexible data types face in the presence of uncertainty; yet, this flexibility also introduces new challenges in merging multiple sources and query processing over integrated data. In this chapter, the authors discuss two alternative ways XML data/schema can be integrated: conflict-eliminating (where the result is cleaned from any conflicts that the different sources might have with each other) and conflict-preserving (where the resulting XML data or XML schema captures the alternative interpretations of the data). They also present techniques for query processing over integrated, possibly imprecise, XML data, and cover strategies that can be used for resolving underlying conflicts.

scholarly journals Human responses to multiple sources of directional information in virtual crowd evacuations

2014 ◽  
Vol 11 (91) ◽  
pp. 20130904 ◽  
Author(s):  
Nikolai W. F. Bode ◽  
Armel U. Kemloh Wagoum ◽  
Edward A. Codling
Keyword(s):  
Multiple Sources ◽  
Directional Information ◽  
Individual Level ◽  
Exit Route ◽  
Static Information ◽  
Alternative Sources ◽  
And Gender ◽  
Crowd Movement ◽  
Interactive Virtual Environment ◽  
Different Sources

The evacuation of crowds from buildings or vehicles is one example that highlights the importance of understanding how individual-level interactions and decision-making combine and lead to the overall behaviour of crowds. In particular, to make evacuations safer, we need to understand how individuals make movement decisions in crowds. Here, we present an evacuation experiment with over 500 participants testing individual behaviour in an interactive virtual environment. Participants had to choose between different exit routes under the influence of three different types of directional information: static information (signs), dynamic information (movement of simulated crowd) and memorized information, as well as the combined effect of these different sources of directional information. In contrast to signs, crowd movement and memorized information did not have a significant effect on human exit route choice in isolation. However, when we combined the latter two treatments with additional directly conflicting sources of directional information, for example signs, they showed a clear effect by reducing the number of participants that followed the opposing directional information. This suggests that the signals participants observe more closely in isolation do not simply overrule alternative sources of directional information. Age and gender did not consistently explain differences in behaviour in our experiments.

scholarly journals Scaling Data from Multiple Sources

2020 ◽  
pp. 1-24
Author(s):  
Ted Enamorado ◽  
Gabriel López-Moctezuma ◽  
Marc Ratkovic
Keyword(s):  
National Security ◽  
The Other ◽  
Tea Party ◽  
Multiple Sources ◽  
Latent Factors ◽  
Flexible Modeling ◽  
Speech Data ◽  
Ideological Dimension ◽  
Modeling Strategy ◽  
Different Sources

Abstract We introduce a method for scaling two datasets from different sources. The proposed method estimates a latent factor common to both datasets as well as an idiosyncratic factor unique to each. In addition, it offers a flexible modeling strategy that permits the scaled locations to be a function of covariates, and efficient implementation allows for inference through resampling. A simulation study shows that our proposed method improves over existing alternatives in capturing the variation common to both datasets, as well as the latent factors specific to each. We apply our proposed method to vote and speech data from the 112th U.S. Senate. We recover a shared subspace that aligns with a standard ideological dimension running from liberals to conservatives, while recovering the words most associated with each senator’s location. In addition, we estimate a word-specific subspace that ranges from national security to budget concerns, and a vote-specific subspace with Tea Party senators on one extreme and senior committee leaders on the other.

Combined Multisensor Displays

1980 ◽  
Vol 24 (1) ◽  
pp. 17-21 ◽  
Author(s):  
A.J. Josefowitz ◽  
R.A. North ◽  
Joel Trimble
Keyword(s):  
High Performance ◽  
Multiple Sources ◽  
Different Sources ◽  
High Performance Aircraft

Interest in multi-sensor combined displays has arisen from a variety of different sources. Because cockpit space is limited and mission profiles for high-performance aircraft continue to become more complex, multi-sensor combined displays provide the potential for both conserving cockpit space and reducing pilot workload demands by consolidating and integrating information from multiple sources onto one display. An experiment was conducted to compare multi-sensor display concepts against more conventional multi-display and multi-function displays. Results from the experiment were analyzed and recommendations are offered for further developmental work in this area.

The calculation of deflexions of the vertical from gravity anomalies

1950 ◽  
Vol 204 (1078) ◽  
pp. 374-395 ◽  
Keyword(s):  
Gravity Data ◽  
Gravity Anomalies ◽  
Sources Of Error ◽  
Free Air ◽  
Gravity Measurements ◽  
Standard Deviations ◽  
The Difference ◽  
Different Sources

The theory of the application of gravity measurements to geodetic calculations is discussed, and the errors involved in calculating deflexions of the vertical are estimated. If the gravity data are given as free air anomalies from Jeffreys’s (1948) formula, so thdt the second and third harmonics of gravity are assumed known, the orders of magnitude of the standard deviations of the different sources of error are the following: Single deflexion: neglect of gravity outside 20° 1" Difference of deflexions: neglect of gravity outside 5° 0"·5 Calculation of effects of gravity from 0º·05 to 5° 0"·1 Calculation of effects of gravity within 0º·05 between 0"·1 and 0"·5 Estimates of the deflexions are made for Greenwich, Herstmonceux, Southampton and Bayeux, and the difference between Greenwich and Southampton is compared with the astronomical and geodetic amplitudes.

BLOCKING MULTIPLE SOURCES OF ERROR IN SMALL ANALYTIC STUDIES

2000 ◽  
Vol 12 (4) ◽  
pp. 497-502 ◽  
Author(s):  
Ramón V. León ◽  
Robert W. Mee
Keyword(s):  
Multiple Sources ◽  
Sources Of Error

The Effects of Consistency among Simultaneous Forecasts on Weather-Related Decisions

2021 ◽  
Vol 13 (1) ◽  
pp. 3-10
Author(s):  
Chen Su ◽  
Jessica N. Burgeno ◽  
Susan Joslyn
Keyword(s):  
Mobile Telephone ◽  
School Closure ◽  
Decision Makers ◽  
Multiple Sources ◽  
Weather Event ◽  
Weather Forecasts ◽  
Computerized Task ◽  
Closure Decisions ◽  
Different Sources ◽  
User Trust

AbstractPeople access weather forecasts from multiple sources [mobile telephone applications (“apps”), newspapers, and television] that are not always in agreement for a particular weather event. The experiment reported here investigated the effects of inconsistency among forecasts on user trust, weather-related decisions, and confidence in user decisions. In a computerized task, participants made school-closure decisions on the basis of snow forecasts from different sources and answered a series of questions about each forecast. Inconsistency among simultaneous forecasts did not significantly reduce trust, although inaccuracy did. Moreover, inconsistency may convey useful information to decision-makers. Not only do participants appear to incorporate the information provided by all forecasts into their own estimates of the outcome, but our results also suggest that inconsistency gives rise to the impression of greater uncertainty, which leads to more cautious decisions. The implications for decisions in a variety of domains are discussed.

Sign in / Sign up

Export Citation Format

Share Document