Evaluation of Agreement between Measurement Methods from Data with Matched Repeated Measurements via the Coefficient of Individual Agreement

Recently, a new estimation procedure has been developed to assess bias and precision of a new measurement method, relative to a reference standard. However, the author did not develop confidence bands around the bias and standard deviation curves. Therefore, the goal in this paper is to extend this methodology in several important directions. First, by developing simultaneous confidence bands for the various parameters estimated to allow formal comparisons between different measurement methods. Second, by proposing a new index of agreement. Third, by providing a series of new graphs to help the investigator to assess bias, precision, and agreement between the two measurement methods. The methodology requires repeated measurements on each individual for at least one of the two measurement methods. It works very well to estimate the differential and proportional biases, even with as few as two to three measurements by one of the two methods and only one by the other. The repeated measurements need not come from the reference standard but from either measurement methods. This is a great advantage as it may sometimes be more feasible to gather repeated measurements with the new measurement method.

Download Full-text

Concordance rate of a four-quadrant plot for repeated measurements

BMC Medical Research Methodology ◽

10.1186/s12874-021-01461-0 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Mayu Hiraishi ◽

Kensuke Tanioka ◽

Toshio Shimokawa

Keyword(s):

Numerical Simulations ◽

Measurement Methods ◽

Repeated Measurements ◽

Concordance Rate ◽

Multivariate Normal ◽

Altman Analysis ◽

Individual Subject ◽

Clinical Measurement ◽

Auc Value ◽

Four Quadrant

Abstract Background To assure the equivalence between new clinical measurement methods and the standard methods, the four-quadrant plot and the plot’s concordance rate is used in clinical practice, along with Bland-Altman analysis. The conventional concordance rate does not consider the correlation among the data on individual subjects, which may affect its proper evaluation. Methods We propose a new concordance rate for the four-quadrant plot based on multivariate normal distribution to take into account the covariance within each individual subject. The proposed concordance rate is formulated as the conditional probability of the agreement. It contains a parameter to set the minimum concordant number between two measurement methods, which is regarded as agreement. This parameter allows flexibility in the interpretation of the results. Results Through numerical simulations, the AUC value of the proposed method was 0.967, while that of the conventional concordance rate was 0.938. In the application to a real example, the AUC value of the proposed method was 0.999 and that of the conventional concordance rate was 0.964. Conclusion From the results of numerical simulations and a real example, the proposed concordance rate showed better accuracy and higher diagnosability than the conventional approaches.

Download Full-text

Modeling Intraindividual Variability in Three-Level Multilevel Models

Methodology ◽

10.1027/1614-2241/a000150 ◽

2018 ◽

Vol 14 (3) ◽

pp. 95-108 ◽

Cited By ~ 4

Author(s):

Steffen Nestler ◽

Katharina Geukes ◽

Mitja D. Back

Keyword(s):

Multilevel Model ◽

Multilevel Models ◽

Mixed Effects ◽

Repeated Measurements ◽

Scale Model ◽

Suggested Approach ◽

Psychological Study ◽

Level Data ◽

Ecological Momentary ◽

Momentary Assessment

Abstract. The mixed-effects location scale model is an extension of a multilevel model for longitudinal data. It allows covariates to affect both the within-subject variance and the between-subject variance (i.e., the intercept variance) beyond their influence on the means. Typically, the model is applied to two-level data (e.g., the repeated measurements of persons), although researchers are often faced with three-level data (e.g., the repeated measurements of persons within specific situations). Here, we describe an extension of the two-level mixed-effects location scale model to such three-level data. Furthermore, we show how the suggested model can be estimated with Bayesian software, and we present the results of a small simulation study that was conducted to investigate the statistical properties of the suggested approach. Finally, we illustrate the approach by presenting an example from a psychological study that employed ecological momentary assessment.

Download Full-text

A critical comparison of selected implicit measurement methods.

Journal of Neuroscience Psychology and Economics ◽

10.1037/npe0000086 ◽

2018 ◽

Vol 11 (4) ◽

pp. 249-266 ◽

Cited By ~ 6

Author(s):

Judith Znanewitz ◽

Lisa Braun ◽

David Hensel ◽

Claudia Fantapié Altobelli ◽

Fabian Hattke

Keyword(s):

Measurement Methods ◽

Critical Comparison ◽

Implicit Measurement

Download Full-text

Use of a Lyophilized Reference Plasma to Compare Coagulation Test Procedures

Thrombosis and Haemostasis ◽

10.1055/s-0038-1651402 ◽

1975 ◽

Vol 34 (02) ◽

pp. 426-444 ◽

Cited By ~ 3

Author(s):

J Kahan ◽

I Nohén

Keyword(s):

Oral Anticoagulant Therapy ◽

Repeated Measurements ◽

Test Procedures ◽

Coagulation Activity ◽

Close Relationship ◽

Measured Activity ◽

Test Materials ◽

The Difference ◽

The Stability ◽

The One

SummaryIn 4 collaborative trials, involving a varying number of hospital laboratories in the Stockholm area, the coagulation activity of different test materials was estimated with the one-stage prothrombin tests routinely used in the laboratories, viz. Normotest, Simplastin-A and Thrombotest. The test materials included different batches of a lyophilized reference plasma, deep-frozen specimens of diluted and undiluted normal plasmas, and fresh and deep-frozen specimens from patients on long-term oral anticoagulant therapy.Although a close relationship was found between different methods, Simplastin-A gave consistently lower values than Normotest, the difference being proportional to the estimated activity. The discrepancy was of about the same magnitude on all the test materials, and was probably due to a divergence between the manufacturers’ procedures used to set “normal percentage activity”, as well as to a varying ratio of measured activity to plasma concentration. The extent of discrepancy may vary with the batch-to-batch variation of thromboplastin reagents.The close agreement between results obtained on different test materials suggests that the investigated reference plasma could be used to calibrate the examined thromboplastin reagents, and to compare the degree of hypocoagulability estimated by the examined PIVKA-insensitive thromboplastin reagents.The assigned coagulation activity of different batches of the reference plasma agreed closely with experimentally obtained values. The stability of supplied batches was satisfactory as judged from the reproducibility of repeated measurements. The variability of test procedures was approximately the same on different test materials.

Download Full-text

Calibration of a Lyophilized Pooled Plasma as Candidate Reference Plasma for Standardization of the Prothrombin Time Ratio

Hämostaseologie ◽

10.1055/s-0038-1655219 ◽

1993 ◽

Vol 13 (02) ◽

pp. 96-105 ◽

Cited By ~ 2

Author(s):

H. Beeser ◽

U. Becker ◽

H. J. Kolde ◽

E. Spanuth ◽

P. Witt ◽

...

Keyword(s):

Prothrombin Time ◽

International Study ◽

International Normalized Ratio ◽

Measurement Methods ◽

Factor V ◽

Diagnostic Instruments ◽

Plasma Pool ◽

Normal Plasma ◽

German Association ◽

Prothrombin Time Ratio

SummaryThe prothrombin time (PT), obtained from a fresh normal plasma pool (FPP), is the basis both for the establishment of the 100% activity (normal plasma) and for the ratio calculation used in the International Normalized Ratio (INR) according to the recommendations of the ICSH/ICTH (6). Today the PT of lyophilized normal plasma pools are successfully used as reference for the assessment of samples in proficiency studies. However, a lack of comparability is to be recognized. Therefore the Committee of Hematology of the German Association of Diagnostics’ and Diagnostic Instruments’ Manufacturers (VDGH) decided to produce a candidate reference plasma (VDGH Reference Plasma) which was calibrated against fresh normal plasma pools in an international study.The basic calibration was performed by using the same certified BCR thromboplastin (BCT/099) by all participants. The endpoint was determined manually and by using the coagulometer Schnitger-Gross. In additional testings each participant used his own routine thromboplastins and methods. Calculating the ratio [PT VDGH Reference Plasma (sec)/PT fresh normal plasma pool (sec)] the VDGH Reference Plasma showed a deviation from the average fresh normal plasma pool of 1.05 both with the BCT/099 and with all thromboplastins. There were obtained some statistical differences between “plain” and “combined’’ (added factor V and fibrinogen) thromboplastins. No statistical difference was found between the different endpoint measurement methods (manual, mechanical, optical).In spite of these statistical deviations the VDGH Reference Plasma can be used for the standardization of the PT-normal (100%) value with different ratios for plain (1.06) and combined (1.02) thromboplastins. The manufacturers will use this VDGH Reference Plasma for the calibration of their commercially available calibration plasmas, which allows the user of such a material to calculate a calibrated 100% PT value.

Download Full-text