Reviews of evidence

The preceding chapters have focused on the generation of scientific evidence using epidemiological studies. Here the emphasis is on bringing together evidence (evidence synthesis) to inform future research and health policies. The chapter considers levels and quality of scientific evidence and describes in detail how to conduct a systematic review. It reviews the ways of summarizing and evaluating evidence from epidemiological studies. Summary of evidence is needed in everyday clinical practice and for public health. We live in a time of information overload, and it is impossible to read all the available scientific journals, even on a narrow scientific topic. Simply using search terms such as ‘cancer’ will result in millions of results in Google Scholar or PubMed (a service of the US National Library of Medicine®) database. Given the ever-increasing volume of medical literature and time constraints, summary of evidence plays a big role in decision-making.

Association or causation

Epidemiological studies can only show associations they cannot prove that a link is causative. Even in the bias free study with minimal confounding, a strong association does not mean that, for example, the presence of the risk factor has a direct biological link to the disease in question. There are several tests that can be used to increase the confidence that an association has biological meaning and needs to be considered. It is interesting always to differentiate factors that are on the ‘path’ to disease but not the underlying cause. The appropriateness of all these steps is dependent on the validity of the association.

Use of secondary data

Primary data collection is challenging and with increasing electronic data capture in routine healthcare and other aspects of life, it is possible to address several epidemiological questions by robust analysis of such ‘secondary data’. There are considerable advantages in terms of scope, size, and speed of study to be balanced against the quality and depth of using primary data. Even when such direct contact is not required, there is often the need to extract necessary information from individual subject records such as medical files. There is often no alternative source of information, although the greater digitization of information is changing that scenario with the potential that the availability of such information might preclude the need for primary data.

Comparing rates between and within populations

In comparing rates between populations, it is important that one is comparing ‘like with like’. One population may be considerably older than a population to which it is compared and therefore it would not be surprising that mortality rates were higher. Instead it is more useful to make comparisons taking account of differences in characteristics such as age or gender. The same considerations apply to examining disease rates over time in a given population. If the characteristics of the population change over time (e.g. the population gets older), this needs to be considered. To formulate hypotheses, the rate of a disease under study in a population may be compared with the rate in other populations, or in the same population at difierent time points. If the rates vary significantly between populations or are changing within a population, then this provides impetus for investigating the reasons underlying these differences or changes.

Ethical issues in epidemiology

There are several major ethical issues that face an epidemiological study. There is always the challenge in studying free-living individuals in the modern society, of balancing the needs for robust methods with individual freedoms. The key concerns reflect ensuring an appropriate consent process, maintaining confidentiality, and minimizing any negative consequences for a participant. The most commonly collected information for an epidemiological study come either from material already available in databases, material such as hard copies of records that can have key data items extracted, or data that is gathered directly from the subject. Occasionally a limited physical examination is undertaken. Much less often, there is a requirement to take samples of biological fluid such as blood and urine, or to undergo simple investigations such as electrocardiography or plain radiography, but even such investigations are typically associated with trivial risk to health.

Meta-analysis

The previous chapter has discussed how to gather and evaluate existing evidence from epidemiological studies. Here further consideration is given to summarizing the identified evidence in such a way that it can be used for decision-making, including approaches to control for chance and potential bias. Meta-analysis refers to the statistical analysis of results from individual studies for integrating the findings. There are other terms related to meta-analysis such as quantitative review, combined analysis, pooled analysis, or quantitative synthesis. Some of them use different methods, for example, meta-analysis of published data considers each study as a unit of analysis while individual patient data analysis includes the original data from each study on a participant level. This chapter describes how to numerically summarize data through performing a meta-analysis using data from a systematic review of epidemiological studies. It also considers possible bias, reporting guidelines, and statistical software available for meta-analysis.

Introductory data analysis

This chapter builds on the previous one on the analysis of descriptive epidemiological studies and illustrates statistical methods appropriate for analysis of analytical epidemiological studies. It mainly focuses on data obtained from case–control and cohort studies, but also considers other study designs presented in Chapter 6. There are also several practical examples to help with the analysis and interpretation of the results of analytical epidemiological studies. In practice, relatively little mathematical calculation is done without computers. In this chapter, however, formulae are presented for the main measures of effect together with worked examples. Indeed, when data are available in tabulated form, as opposed to raw data files, it is frequently an easy task to calculate the important measures ‘by hand’. The formulae presented will permit the reader, for example, to check or further explore data published by others.

Preparation of collected primary data for statistical analysis

Although epidemiological studies are increasingly based on the analysis of existing data sets (including linked data sets), many studies still require primary data collection. Such data may come from patient questionnaires, interviews, abstraction from records, and/or the results of tests and measures such as weight or blood test results. The next stage is to analyse the data gathered from individual subjects to provide the answers required. Before commencing with the statistical analysis of any data set, the data themselves must be prepared in a format so that the detailed statistical analysis can achieve its goals. Items to be considered include the format the data are initially collected in and how they are transferred to an appropriate electronic form. This chapter explores how errors are minimized and the quality of the data set ensured. These tasks are not trivial and need to be planned as part of a detailed study methodology.

Collecting information

Collecting accurate and comprehensive information both direct from the participants, or indirectly from records or other data is one of the major challenges to a successful epidemiological study. Epidemiological information comes from a variety of sources. These may be conveniently divided into those that are available from previously documented data and those that require the gathering of new information. Examples of the former include extracting information about individuals from their medical records, occupational records, and similar data sources. Design and choice of delivery of patient data capture forms by direct interview or telephone, by post, email, or other electronic means all require considerable thought and pilot testing. Attention to the specific wording of certain questions is crucial. This chapter therefore focuses on the issues surrounding the collection of information that otherwise would not be available: primary data collection.

The costs of an epidemiological study

Epidemiological studies can be very expensive, especially from large populations with multicentre recruitment. The researcher will need to ensure that there are adequate resources, allowing for the fact that things will not always go to plan, but making sure that the research is value for money. What is considered a reasonable cost will also depend on how strong the rationale is for conducting the study. Although in theory the study design influences the costs, in practice the resources available will often constrain the methodological choices. Costing an epidemiological study accurately at the start is vital. There are several ways to maximize the use of resources to ensure the study is efficient.