Comparing the Power of Quality-Control Rules to Detect Persistent Increases In Random Error

1992 ◽  
Vol 38 (3) ◽  
pp. 364-369 ◽  
Author(s):  
C A Parvin
Keyword(s):  
Quality Control ◽  
Error Detection ◽  
Statistical Evaluation ◽  
Poor Performance ◽  
Operating Conditions ◽  
Evaluation Approach ◽  
Control Rules ◽  
Stable Operating ◽  
Alternative Approach ◽  
Analytical Imprecision

Abstract This paper continues an investigation into the merits of an alternative approach to the statistical evaluation of quality-control rules. In this report, computer simulation is used to evaluate and compare quality-control rules designed to detect increases in within-run or between-run imprecision. When out-of-control conditions are evaluated in terms of their impact on total analytical imprecision, the error detection ability of a rule depends on the relative magnitudes of the between-run and within-run error components under stable operating conditions. A recently proposed rule based on the F-test, designed to detect increases in between-run imprecision, is shown to have relatively poor performance characteristics. Additionally, several issues are examined that have been difficult to address with the traditional evaluation approach.

Quality control failures exceeding the weekly limit (QC FEWL): a simple tool to improve assay error detection

2019 ◽  
Vol 56 (6) ◽  
pp. 668-673
Author(s):  
Eric S Kilpatrick
Keyword(s):  
Quality Control ◽  
Error Detection ◽  
Data Extraction ◽  
Complex Data ◽  
Systematic Bias ◽  
Detection Rates ◽  
Control Rules ◽  
Quality Control Testing ◽  
Binomial Statistics ◽  
Control Failures

Background Even when a laboratory analyte testing process is in control, routine quality control testing will fail with a frequency that can be predicted by the number of quality control levels used, the run frequency and the control rule employed. We explored whether simply counting the number of assay quality control run failures during a running week, and then objectively determining if there was an excess, could complement daily quality control processes in identifying an out-of-control assay. Methods Binomial statistics were used to determine the threshold number of quality control run failures in any rolling week which would statistically exceed that expected for a particular test. Power function graphs were used to establish error detection (Ped) and false rejection rates compared with popular control rules. Results Identifying quality control failures exceeding the weekly limit (QC FEWL) is a more powerful means of detecting smaller systematic (bias) errors than traditional daily control rules (12s, 13s or 13s/22s/R4s) and markedly superior in detecting smaller random (imprecision) errors while maintaining false identification rates below 2%. Error detection rates also exceeded those using a within- and between-run Westgard multirule (13s/22s/41s/10x). Conclusions Daily review of tests shown to statistically exceed their rolling week limit of expected quality control run failures is more powerful than traditional quality control tools at identifying potential systematic and random test errors and so offers a supplement to daily quality control practices that has no requirement for complex data extraction or manipulation.

Performance characteristics of rules for internal quality control: probabilities for false rejection and error detection.

1977 ◽  
Vol 23 (10) ◽  
pp. 1857-1867 ◽  
Author(s):  
J O Westgard ◽  
T Groth ◽  
T Aronsson ◽  
H Falk ◽  
C H de Verdier
Keyword(s):  
Quality Control ◽  
Error Detection ◽  
Control Charts ◽  
Performance Characteristics ◽  
Internal Quality Control ◽  
Internal Quality ◽  
Sum Rule ◽  
Control Rules ◽  
Systematic Shift ◽  
False Rejection

Abstract When assessing the performance of an internal quality control system, it is useful to determine the probability for false rejections (pfr) and the probability for error detection (ped). These performance characteristics are estimated here by use of a computer stimulation procedure. The control rules studied include those commonly employed with Shewhart-type control charts, a cumulative sum rule, and rules applicable when a series of control measurements are treated as a single control observation. The error situations studied include an increase in random error, a systematic shift, a systematic drift, and mixtures of these. The probability for error detection is very dependent on the number of control observations and the choice of control rules. No one rule is best for detecting all errors, thus combinations of rules are desirable. Some appropriate combinations are suggested and their performance characteristics are presented.

Comparing the Power of Quality-Control Rules to Detect Persistent Systematic Error

1992 ◽  
Vol 38 (3) ◽  
pp. 358-363 ◽  
Author(s):  
C A Parvin
Keyword(s):  
Quality Control ◽  
Error Detection ◽  
Response Rate ◽  
Linear Trend ◽  
Detection Capability ◽  
Control Rule ◽  
Control Rules ◽  
Detection Capabilities ◽  
Detection Characteristics ◽  
Quality Control Test

Abstract A simulation approach that allows direct estimation of the power of a quality-control rule to detect error that persists until detection is used to compare and evaluate the error detection capabilities of a group of quality-control rules. Two persistent error situations are considered: a constant shift and a linear trend in the quality-control mean. A recently proposed "moving slope" quality-control test for the detection of linear trends is shown to have poor error detection characteristics. A multimean quality-control rule is introduced to illustrate the strategy underlying multirule procedures, which is to increase power without sacrificing response rate. This strategy is shown to provide superior error detection capability when compared with other rules evaluated under both error situations.

Estimating the performance characteristics of quality-control procedures when error persists until detection

1991 ◽  
Vol 37 (10) ◽  
pp. 1720-1724 ◽  
Author(s):  
C A Parvin
Keyword(s):  
Quality Control ◽  
Error Detection ◽  
Rejection Rate ◽  
Performance Characteristics ◽  
Simulation Methods ◽  
Control Rule ◽  
Control Rules ◽  
Control Procedures ◽  
Error Detection Rate ◽  
False Rejection

Abstract The concepts of the power function for a quality-control rule, the error detection rate, and the false rejection rate were major advances in evaluating the performance characteristics of quality-control procedures. Most early articles published in this area evaluated the performance characteristics of quality-control rules with the assumption that an intermittent error condition occurred only within the current run, as opposed to a persistent error that continued until detection. Difficulties occur when current simulation methods are applied to the persistent error case. Here, I examine these difficulties and propose an alternative method that handles persistent error conditions effectively when evaluating and quantifying the performance characteristics of a quality-control rule.

Influence of a between-run component of variation, choice of control limits, and shape of error distribution on the performance characteristics of rules for internal quality control.

1979 ◽  
Vol 25 (3) ◽  
pp. 394-400 ◽  
Author(s):  
J O Westgard ◽  
H Falk ◽  
T Groth
Keyword(s):  
Quality Control ◽  
Standard Deviation ◽  
Error Detection ◽  
Error Distribution ◽  
Performance Characteristics ◽  
Internal Quality ◽  
Chi Square ◽  
Control Rules ◽  
Non Gaussian ◽  
Control Limits

Abstract A computer-stimulation study has been performed to determine how the performance characteristics of quality-control rules are affected by the presence of a between-run component of variation, the choice of control limits (calculated from within-run vs. total standard deviations), and the shape of the error distribution. When a between-run standard deviation (Sb) exists and control limits are calculated from the total standard deviation (St, which includes Sb as well as the within-run standard deviation, Sw), there is generally a loss in ability to detect analytical disturbances or errors. With control limits calculated from Sw, there is generally an increase in the level of false rejections. The presence of non-gaussian error distribution appears to have considerably less effect. It can be recommended that random error be controlled by use of a chi-square or range-control rule, with control limits calculated from Sw. Optimal control of systematic errors is difficult when Sb exists. An effort should be made to reduce Sb, and this will lead to increased ability to detect analytical errors. When Sb is tolerated or accepted as part of the baseline state of operation for the analytical method, then further increases in the number of control observations will be necessary to achieve a given probability for error detection.

Establishing Evidence-Based Statistical Quality Control Practices

2018 ◽  
Vol 151 (4) ◽  
pp. 364-370 ◽  
Author(s):  
James O Westgard ◽  
Sten A Westgard
Keyword(s):  
Quality Control ◽  
Error Detection ◽  
Scientific Evidence ◽  
Measurement Procedure ◽  
Statistical Quality Control ◽  
Evidence Based ◽  
Statistical Quality ◽  
Medical Laboratories ◽  
Control Rules ◽  
Graphical Summary

AbstractObjectivesTo establish an objective, scientific, evidence-based process for planning statistical quality control (SQC) procedures based on quality required for a test, precision and bias observed for a measurement procedure, probabilities of error detection and false rejection for different control rules and numbers of control measurements, and frequency of QC events (or run size) to minimize patient risk.MethodsA Sigma-Metric Run Size Nomogram and Power Function Graphs have been used to guide the selection of control rules, numbers of control measurements, and frequency of QC events (or patient run size).ResultsA tabular summary is provided by a Sigma-Metric Run Size Matrix, with a graphical summary of Westgard Sigma Rules with Run Sizes.ConclusionMedical laboratories can plan evidence-based SQC practices using simple tools that relate the Sigma-Metric of a testing process to the control rules, number of control measurements, and run size (or frequency of QC events).

New insight into the comparative power of quality-control rules that use control observations within a single analytical run

1993 ◽  
Vol 39 (3) ◽  
pp. 440-447 ◽  
Author(s):  
C A Parvin
Keyword(s):  
Quality Control ◽  
Statistical Theory ◽  
Standard Deviation ◽  
Error Detection ◽  
Graphical Display ◽  
Simulation Studies ◽  
Control Rules ◽  
The Mean ◽  
Detection Characteristics ◽  
Insight Into

Abstract The error detection characteristics of quality-control (QC) rules that use control observations within a single analytical run are investigated. Unlike the evaluation of QC rules that span multiple analytical runs, most of the fundamental results regarding the performance of QC rules applied within a single analytical run can be obtained from statistical theory, without the need for simulation studies. The case of two control observations per run is investigated for ease of graphical display, but the conclusions can be extended to more than two control observations per run. Results are summarized in a graphical format that offers many interesting insights into the relations among the various QC rules. The graphs provide heuristic support to the theoretical conclusions that no QC rule is best under all error conditions, but the multirule that combines the mean rule and a within-run standard deviation rule offers an attractive compromise.

New Insight into the Comparative Power Quality-Control Rules That Use Control Observations within a Single Analytical Run

1993 ◽  
Vol 39 (8) ◽  
pp. 1589-1589 ◽  
Author(s):  
Curtis A Parvin
Keyword(s):  
Quality Control ◽  
Operating Characteristic ◽  
Clinical Medicine ◽  
Quantitative Measure ◽  
Evaluation Tool ◽  
True Negative ◽  
Control Value ◽  
Control Rules ◽  
Analytical Imprecision ◽  
Insight Into

Abstract Vol. 39: p. 441 In the article by C.A. Parvin Entitled "New Insight into the comparative power of quality-control rules that use control observations within a single analytical run," 1993;39;440-7, the means in the first line on page 441 should have been µ1 + σb1E1 and µ2 + σb2E1. In the third paragraph on page 441, in the Sentence "Finally, I assume that total analytical imprecision increases from its stable value, σtj, to REw = 1.5σtj, because of an increase in the between-run component of imprecision from its stable value, σwj, to an out-of-control value, σwj," the word "between" should have been "within." pp. 565-572. In the review by M.H. zweig and G. Campbell entitled "Receiver operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine," 1993;39:561-77, the label for the x-axis at the bottom of each of Figures 4 through 12 should read "False-positive fraction (1 - specificity)." Correspondingly, the label at the top of each Figure should read "True-negative fraction (specificity)." Note that the (corect) dual labeling of the x-axis solves the problem of whether to plot specificity or 1 - specificity on the x-axis. pp. 767-769. In the article by K. Emancipator and M.H. Kroll entitled "A quantitative measure of nonlinearity," 1993;39:766-772, three equations were printed incorrectly. They should have read as follows: Equation 6a (p. 767) Equation 8b (p. 768) and on p. 769, the last unnumbered equation

Choosing quality-control systems to detect maximum clinically allowable analytical errors.

1989 ◽  
Vol 35 (2) ◽  
pp. 284-288 ◽  
Author(s):  
K Linnet
Keyword(s):  
Quality Control ◽  
Control Systems ◽  
Chemical Components ◽  
Minimal Requirement ◽  
Control Rules ◽  
Clinical Chemical ◽  
Critical Error ◽  
Analytical Imprecision ◽  
Standard Deviations ◽  
Analytical Errors

Abstract Critical systematic and random analytical errors for 17 common clinical chemical components were estimated from published values for analytical imprecision, biological variation, and "medically important changes." Appropriate quality-control systems for these analytes are discussed on the basis of power considerations. The simple rule 1(3)s, with one control per run, is minimally sufficient for the analytes (about one quarter of those considered here) for which the magnitude of critical error is at least 3 analytical standard deviations. The more powerful rule 1(2)s, with one control per run, is the minimal requirement for analytes for which critical errors are about 2 analytical standard deviations; these are about half the remaining analytes. Greater power values are achieved by using multiple rules based on several controls per run. In general, this study does not support the view put forward by some authors that the quality-control rules in use today are too restrictive.

Animal Models and Alternatives in the Quality Control of Vaccines: Are In Vitro Methods or In Vivo Methods the Scientific Equivalent of the Emperor's New Clothes?

1995 ◽  
Vol 23 (1) ◽  
pp. 61-73
Author(s):  
Coenraad Hendriksen ◽  
Johan van der Gun
Keyword(s):  
Quality Control ◽  
Test Methods ◽  
In Vitro Test ◽  
Large Numbers ◽  
Alternative Approach ◽  
Inactivated Vaccines ◽  
Vitro Test

In the quality control of vaccine batches, the potency testing of inactivated vaccines is one of the areas requiring very large numbers of animals, which usually suffer significant distress as a result of the experimental procedures employed. This article deals with the potency testing of diphtheria and tetanus toxoids, two vaccines which are used extensively throughout the world. The relevance of the potency test prescribed by the European Pharmacopoeia monographs is questioned. The validity of the potency test as a model for the human response, the ability of the test to be standardised, and the relevance of the test in relation to the quality of the product are discussed. It is concluded that the potency test has only limited predictive value for the antitoxin responses to be expected in recipients of these toxoids. An alternative approach for estimating the potency of toxoid batches is discussed, in which a distinction is made between estimation of the immunogenic potency of the first few batches obtained from a seed lot and monitoring the consistency of the quality of subsequent batches. The use of animals is limited to the first few batches. Monitoring the consistency of the quality of subsequent batches is based on in vitro test methods. Factors which hamper the introduction and acceptance of the alternative approach are considered. Finally, proposals are made for replacement, reduction and/or refinement (the Three Rs) in the use of animals in the routine potency testing of toxoids.

Sign in / Sign up

Export Citation Format

Share Document