Measurement Uncertainty

One of the great revolutions in metrology in chemistry has been the understanding of the need to quote an appropriate measurement uncertainty with a result. For some time, a standard deviation determined under not particularly well-defined conditions was considered a reasonable adjunct to a measurement result, and multiplying by the appropriate Student’s t value gave the 95% confidence interval. But knowing that in a long run of experiments repeated under identical conditions 95% of the 95% confidence intervals would include the population mean did not answer the fundamental question of how good the result was. This became evident as international trade burgeoned and more and more discrepancies in measurement results and disagreements between trading partners came to light. To determine if two measurements of ostensibly the same measurand on the same material give results that are equivalent, they must be traceable to the same metrological reference and have stated measurement uncertainties. How to achieve that comparability is the subject of this chapter and the next. When making a chemical measurement by taking a certain amount of the test material, working it up in a form that can be analyzed, calibrating the instrument, and performing the measurement, analysts understand that there will be some doubt about the result. Contributions to uncertainty derive from each step in the analysis, and even from the basis on which the analysis is carried out. An uncertainty budget documents the history of the assessment of the measurement uncertainty of a result, and it is the outcome of the process of identifying and quantifying uncertainty. Although the client may only receive the fruits of this process as (value ± expanded uncertainty), accreditation to ISO/IEC 17025 requires the laboratory to document how the uncertainty is estimated. Estimates of plutonium sources highlight the importance of uncertainty. The International Atomic Energy Agency (IAEA) estimates there are about 700 tonnes of plutonium in the world. The uncertainty of measurement of plutonium is of the order of 0.1%, so even if all the plutonium were in one place, when analyzed the uncertainty would be 700 kg (1000 kg = 1 tonne). Seven kilograms of plutonium makes a reasonable bomb.

Conclusions: Bringing It All Together

If you have read this book, whether a few pages at a time, by jumping back and forth, or meticulously from beginning to end, the aim of this chapter is to draw together the methods, concepts, and ideas to help you answer the question, how do I make a good analytical measurement? If nothing else, you will have discovered, like the answers to the greater questions of life, that there is not a simple prescription for quality assurance that if followed leads to success. Even knowing if you have the right answer is not always vouchsafed; is customer satisfaction sufficient? Does the continuing solvency of your business say that something must be going well? Does staying within ± 2σ in interlaboratory studies cause you happiness? The best laboratories do all of this and more. At the heart of a good laboratory is an excellent manager who has recruited good staff, set up a culture of quality, and who understands the science and business of chemical analysis and the requirements of his or her clients. I do not believe laboratories can be run by people with only managerial skills; at some point a chemical analyst is going to have to take responsibility for the product. In this reprise of the book’s contents I revisit the six principles of valid analytical measurement (VAM) so cleverly enunciated by the Laboratory of the Government Chemist. But first some words about clients and samples. As has been stressed throughout this book, many problems can be solved by chemical analysis, and the point of chemical analysis is therefore not to do chemistry for its own sake, but to contribute to the solution of those problems. Clients, or customers as now found in ISO/IEC 17025, come in many shapes and sizes, from people who gladly admit no scientific knowledge at all to fellow professionals who can discuss the analysis as equals. The first kind are more difficult to work with than the latter, although colleagues who meddle are never totally welcome. An apparently simple request to analyze something might require extensive negotiation about exactly what is needed.

Accreditation

Accreditation is the procedure by which the competence of a laboratory to perform a specified range of tests or measurements is assessed against a national or international standard. The accreditation covers the kinds of materials tested or measured, the procedures or methods used, the equipment and personnel used in those procedures, and all relevant systems that the laboratory has in place. Once accredited, the laboratory is entitled to endorse test results with their accreditation status which, if it has any validity, is an imprimatur of some degree of quality and gives the client added confidence in the results. Accreditation therefore benefits the laboratory, by allowing the laboratory to demonstrate competence in particular tests, and the client, by providing a choice of accredited laboratories that are deemed competent. Accreditation is part of conformity assessment in international trade. Conformity assessment leads to the acceptance of the goods of one country by another, with confidence borne of mutual recognition of manufacturing and testing procedures. Figure 9.1 shows the relation between accreditation and the goal of conformity in trade. For accreditation to be a cornerstones of conformity in trade, each laboratory that is assessed, in whatever country, must be judged against the same standard (e.g., ISO/IEC 17025), and the assessment process must be essentially the same from one country to another. The standards are indeed international, through the International Organization for Standardization (ISO), and the accreditation bodies themselves are scrutinized under the auspices of the International Laboratory Accreditation Co-operation (ILAC), being accredited to the ISO/IEC Standard 17011 (ISO/IEC 2004a). Full membership in ILAC is open to recognized bodies that operate accreditation schemes for testing laboratories, calibration laboratories, and inspection bodies that have been accepted as signatories to the ILAC Mutual Recognition Arrangement. They must maintain conformance with appropriate international standards such as ISO/IEC 17011 and ILAC guidance documents, and the must ensure that all their accredited laboratories comply with ISO/IEC 17025 and related ILAC guidance documents. Table 9.1 lists the full members and signatories of the ILAC Mutual Recognition Arrangement. The National Association of Testing Authorities (NATA) of Australia has the distinction of being the first accreditation body in the world (founded in 1947), and has long been in the vanguard of the approach to quality through accreditation.

Method Validation

Many aspects of a chemical analysis must be scrutinized to ensure that the product, a report containing the results of the analysis, fulfills the expectations of the client. One of the more fundamental factors is the analytical method itself. How was it chosen? Where does it come from? When a laboratory is faced with a problem requiring chemical analysis, there may be set methods described in a standard operating procedure, but often the analyst might have to make a choice among methods. For the majority of analytical methods used in field laboratories, there is neither the expertise nor the inclination to start from scratch and reinvent the wheel. The analyst wants a method that can be implemented in his or her laboratory. Compilations of methods that have been evaluated do exist and have the imprimatur of international organizations such as the International Organization for Standardization (ISO) or the American Society for Testing and Materials (ASTM). Failing this, the scientific literature abounds in potential methods that have the recommendation of the authors, but may not always be as suitable as claimed. This chapter has two aims: to demonstrate the necessity of using properly validated and verified methods and to explain what constitutes a validated method, and to provide an introduction to method validation for in-house methods. There is an abundance of published material that defines, describes, and generally assists with method validation, some of which is referenced here (Burgess 2000; Christensen et al. 1995; EURACHEM 1998; Fajgelj and Ambrus 2000; Green 1996; Hibbert 2005; ICH 1995, 1996; LGC 2003; Thompson et al. 2002; USP 1999; Wood 1999). “Method validation” is a term used for the suite of procedures to which an analytical method is subjected to provide objective evidence that the method, if used in the manner specified, will produce results that conform to the statement of the method validation parameters. Like many aspects quality assurance, method validation is of a relative nature. As with the concept of fitness for purpose, a method is validated for a particular use under particular circumstances. If those circumstances vary, then the method would need to be re-validated at least for the differences.

Statistics for the Quality Control Laboratory

Because volumes are devoted to the statistics of data analysis in the analytical laboratory (indeed, I recently authored one [Hibbert and Gooding 2005]), I will not rehearse the entire subject here. Instead, I present in this chapter a brief overview of the statistics I consider important to a quality manager. It is unlikely that someone who has never been exposed to the concepts of statistics will find themselves in a position of QA manager with only this book as a guide; if that is your situation, I am sorry. Here I review the basics of the normal distribution and how replicated measurements lead to statements about precision, which are so important for measurement uncertainty. Hypothesis and significance testing are described, allowing testing of hypotheses such as “there is no significant bias” in a measurement. The workhorse analysis of variance (ANOVA), which is the foundational statistical method for elucidating the effects of factors on experiments, is also described. Finally, you will discover the statistics of linear calibration, giving you tools other than the correlation coefficient to assess a straight line (or other linear) graph. The material in this chapter underpins the concept of a system being in “statistical control,”which is discussed in chapter 4. Extensive coverage of statistics is given in Massart et al.’s (1997) two-volume handbook. Mullins’ (2003) text is devoted to the statistics of quality assurance. Berzelius (1779–1848) was remarkably farsighted when he wrote about measurement in chemistry: “not to obtain results that are absolutely exact— which I consider only to be obtained by accident—but to approach as near accuracy as chemical analysis can go.” Did Berzelius have in mind a “true” value? Perhaps not, and in this he was being very up to date. The concept of a true value is somewhat infra dig, and is now consigned to late-night philosophical discussions. The modern approach to measurement, articulated in the latest, but yet-to-be-published International Vocabulary of Basic and General Terms in Metrology (VIM; Joint Committee for Guides in Metrology 2007), considers measurement as a number of actions that improve knowledge about the unknown value of a measurand (the quantity being measured). That knowledge of the value includes an estimate of the uncertainty of the measurement result.

Quality Control Tools

Although thoroughly convinced that no laboratory can function without proper regard to quality in all its myriad forms, the question remains, What do we do? As a quality control manager with a budget and the best aspirations possible, what are the first steps in providing your laboratory or company with an appropriate system (other than buying this book, of course)? Each laboratory is unique, and what is important for one may be less important for another. So before buying software or nailing control charts to your laboratory door, sit down and think about what you hope to achieve. Consider how many different analyses are done, the volume of test items, the size of the operation, what level of training your staff have, whether the laboratory is accredited or seeking accreditation, specific quality targets agreed upon with a client, and any particular problems. This chapter explains how to use some of the standard quality tools, including ways to describe your present system and methods and ongoing statistical methods to chart progress to quality. Being in “statistical control” in an analytical laboratory is a state in which the results are without uncorrected bias and vary randomly with a known and acceptable standard deviation. Statistical control is held to be a good and proper state because once we are dealing with a random variable, future behavior can be predicted and therefore risk is controlled. Having results that conform to the normal or Gaussian distribution (see chapter 2) means that about 5 in every 100 results will fall outside ± 2 standard deviations of the population mean, and 3 in 1000 will fall outside ± 3 standard deviations. By monitoring results to discover if this state is violated, something can be done about the situation before the effects become serious (i.e., expensive). If you are in charge of quality control laboratories in manufacturing companies, it is important to distinguish between the variability of a product and the variability of the analysis. When analyzing tablets on a pharmaceutical production line, variability in the results of an analysis has two contributions: from the product itself and from the analytical procedure.

Modeling and Optimizing Analytical Methods

I asked a professor, visiting from a nation well regarded for its hardworking ethos, whether in his search for ever better catalysts for some synthesis or other, he used experimental design. His answer was, “I have many research students. They work very hard!” Many people believe that an infinite number of monkeys and typewriters would produce the works of Shakespeare, but these days few organizations have the luxury of great numbers of researchers tweaking processes at random in order to make them ever more efficient. The approach of experimental scientists is to systematically change aspects of a process until the results improve. In this chapter I look at this approach from a statistical viewpoint and show how a structured methodology, called experimental design, can save time and effort and arrive at the best (statistically defined) result. It may be a revelation to some readers that the tried-and-trusted “change one factor at a time” approach might yield incorrect results, after requiring more experiments than is necessary. In the sections that follow, I explain how experimental design entails more than just having an idea of what you are going to do before beginning an experiment. Optimization is the maximizing or minimizing a response by changing one or more input variables. In this chapter optimization is synonymous with maximization, as any minimization can be turned into a maximization by a straightforward transformation: Minimization of cost can be seen as maximization of profit; minimization of waste turns into maximization of production; minimization of f(x) is maximization of 1/f(x) or -f(x). Before describing methods of effecting such an optimization, the term optimization must be carefully defined, and what is being optimized must be clearly understood. There are some texts on experimental design available for chemists, although often the subject is treated, as it is here, within a broader context. A good starter for the basics of factorial designs is the Analytical Chemistry Open Learning series (Morgan 1991). Reasonably comprehensive coverage is given in Massart et al.’s (1997) two-volume series, and also in a book from the Royal Society of Chemistry (Mullins 2003).

Metrological Traceability

The ability to trace a measurement result to a reference value lies at the heart of any measurement. Traceability is part of standards governing laboratory practice, such as ISO/IEC 17025 and Good Laboratory Practice (see chapter 9), as a mandatory property of a measurement result, yet as a concept, traceability of a chemical measurement result is poorly understood. It is either taken for granted, often without much foundation, or ignored altogether. Why is traceability so important? How have we been able to ignore it for so long? The International Union of Pure and Applied Chemistry (IUPAC) has applied itself to this problem and a definitive discussion on metrological traceability in chemistry will be published. In this chapter I use the term “metrological traceability” to refer to the property of a measurement result that relates the result to a metrological reference. The word “metrological” is used to distinguish the concept from other kinds of traceability, such as the paper trail of documentation, or the physical trail of the chain of custody of a forensic sample. When the term “traceable standard” is used to refer to a calibration material, for example, the provenance of the material is not at issue, but the quantity value embodied in the standard. In explaining the importance of metrological traceability, I return to the discussions about chemical measurement (chapter 1). The concentration of a chemical is never measured for its own sake, but for a purpose, which often involves trade, health, environmental, or legal matters. The ultimate goal is achieved by comparing the measurement result with another measurement result, with a prescribed value, a legal or regulatory limit, or with values amassed from the experience of the analyst or client. In trading grain, for example, if exported wheat is analyzed by both buyer and seller for protein content, they should be confident that they will obtain comparable measurement results; in other words, results for the same sample of wheat should agree within the stated measurement uncertainties. If the results do not agree, then one party or the other will be disadvantaged, the samples will have to be remeasured, perhaps by a third-party referee, at cost of time and money.

Interlaboratory Studies

No matter how carefully a laboratory scrutinizes its performance with internal quality control procedures, testing against other laboratories increases confidence in a laboratory’s results and among all the laboratories involved in comparison testing. Although without independent knowledge of the value of the measurand it is possible that all the laboratories involved are producing erroneous results, it is also comforting to know that your laboratory is not too different from its peers. An interlaboratory study is a planned series of analyses of a common test material performed by a number of laboratories, with the goal of evaluating the relative performances of the laboratories, the appropriateness and accuracy of the method used, or the composition and identity of the material being tested. The exact details of the study depend on the nature of the test, but all studies have a common pattern: an organizing laboratory creates and distributes a test material that is to be analyzed to the participants in the study, and the results communicated back to the organizing laboratory. The results are statistically analyzed and a report of the findings circulated. Interlaboratory studies are increasingly popular. Ongoing rounds of interlaboratory studies are conducted by most accreditation bodies; the Key Comparison program of the Consultative Committee of the Amount of Substance (CCQM) is one such interlaboratory study (BIPM 2006). There is a great deal of literature on interlaboratory studies (Hibbert 2005; Horwitz 1995; Hund et al. 2000; Lawn et al. 1997; Maier et al. 1993; Thompson and Wood 1993), and an ISO/IEC guide for the conduct of proficiency testing studies is available (ISO/IEC 1997). There are three principal groups of studies: studies that test laboratories (proficiency tests), studies that test methods, and studies that test materials (table 5.1). Laboratories that participate in method and material studies are chosen for their ability to analyze the particular material using the given method. It is not desirable to discover any lacunae in the participating laboratories, and outliers cause lots of problems. The aim of the study is to obtain information about the method or material, so confidence in the results is of the greatest importance.

Introduction to Quality in the Analytical Chemistry Laboratory

To understand quality of chemical measurements, one needs to understand something about measurement itself. The present edition of the International Vocabulary of Basic and General Terms in Metrology (ISO 1993, term 2.1) defines a measurement as a “set of operations having the object of determining a value of a quantity.” Quantity is defined as an “attribute of a phenomenon, body or substance that may be distinguished qualitatively and determined quantitatively” (ISO 1993, term 1.1). Typical quantities that a chemist might be interested in are mass (not weight), length, volume, concentration, amount of substance (not number of moles), current, and voltage. A curse of chemistry is that there is only one unit for amount of substance, the mole, and perhaps because “amount of substance” is verbally unwieldy and its contraction “amount” is in common nonscientific usage, the solecism “number of moles” is ubiquitous and has led to general confusion between quantities and units. The term “measurand,” which might be new to some readers, is the quantity intended to be measured, so it is correct to say of a numerical result that it is the value of the measurand. Do not confuse measurand with analyte. A test material is composed of the analyte and the matrix, and so the measurand is physically embodied in the analyte. For example, if the measurand is the mass fraction of dioxin in a sample of pig liver, the dioxin is the analyte and the liver is the matrix. A more rigorous approach of defining a quantity in terms of, System – Component; kind of quantity, has been under discussion in clinical medicine for some time. This concept of specifying a quantity has recently been put on a sound ontological footing by Dybkaer (2004). A measurement result typically has three components: a number and an uncertainty with appropriate units (which may be 1 and therefore conventionally omitted). For example, an amount concentration of copper might be 3.2 ± 0.4 μmol L-1. Chapter 6 explains the need to qualify an uncertainty statement to describe what is meant by plus or minus (e.g., a 95% confidence interval), and the measurand must also be clearly defined, including speciation, or isomeric form. Sometimes the measurement is defined by the procedure, such as “pH 8 extractable organics.”