Clinical – Page 3 – Dr Chris Cates' EBM Website

The PROGRESS trial: which drug works? (Lancet 2001)

The Lancet reported the results of the Progress trial in which Perindopril was used to lower the blood pressure of patients following strokes in 2001. There were some problems in the analysis of the results from this trial because the investigators were allowed to choose one of two alternative regimens for the patients. In one group the patients were randomised to Perindopril or placebo, and in the other group where the doctors felt it was appropriate to use a thiazide diuretic as well as Perindopril the patients were randomised to Perindopril and Indapamide or double placebo.

In the analysis of the trial results the combination of Perindopril and Indapamide resulted in statistically significant benefits to the patients in terms of prevention of stroke and of major vascular events, whereas Perindopril alone did not reach statistical significance. It is already known that Indapamide improves clinical outcomes so it is possible that the main benefit in the combined drug group comes mostly from the Indapamide, and it is not possible to separately assess the added benefit of Perindopril.

If the second group had been randomised to all get Indapamide and then either Perindopril or Placebo on top it would have been reasonable to combine the results from all the included patients to make an overall assessment of the benefit attributable to Perindopril. The combined results presented in the paper are seriously confounded by the addition of Indapamide in some of the patients and are therefore uninterpretable.

I am grateful to the GP Trainers in Kingston (London) who suggested looking at this paper in a Critical Appraisal Workshop. When we asked what question this study was trying to answer it became clear that the two regimens were addressing different questions: the first is assessing the benefit of Perindopril against placebo, the second Perindopril and Indapamide against double placebo. These are quite distinct and if these were two separate studies I would not be happy to combine them into a single result in a Meta-analysis (as the trialists have done in the paper). What is more they carry out a test for heterogeneity (difference) between the two regimens and find highly significant differences in outcome for both stroke and all adverse events (p< 0.001 in Figure 5 of the original paper).

We concluded that a thiazide diuretic is worth considering to lower the blood pressure of all patients following a stroke, but the case for Perindopril is unproven. The editorial team at the Lancet seem to think this is a fair criticism as they have published my letter making this point, and if you want to look for yourself at the paper or the letters they are both available on the Lancet Website.

The lowering of blood pressure after stroke. Cates C. The Lancet – Vol.358, Issue9297, 08 December 2001,Page1993

“Sir

In the study of the perindopril protection against recurrent stroke study (PROGRESS) Collaborative Group (Sept 29, p 1033), the first group were given perindopril alone and did not differ significantly from the combination therapy group in rate of stroke or major vascular events by comparison with placebo. The second group were given perindopril and indap amide, but this treatment was compared with double placebo.

If all patients in the second group were given indapamide and randomly assigned perindopril or placebo, it would make sense to combine the results from both groups, on the basis that both were placebo comparisons of perindopril, but with different co-interventions. Use of the second placebo means that the two groups are actually answering separate questions of the efficacy of perindopril’s alone (group one) and in combined treatment (group two).

Pooling the results of the two groups makes little sense to me under these circumstances, since the known efficacy of indapamide is a serious confounding factor. Moreover, the question arises of how much benefit perindopril adds to use of indapamide alone. In view of the 10% of patients taking perindopril who withdrew in the run-in period and the surprising lack of efficacy in relation to the average fall in blood pressure noted by Jan Staessen and Jiguang Wang, my take-home message from this report would be to start patients on a thiazide diuretic after a stroke or transient ischaemic attack. Addition of perindopril might be beneficial, but, unfortunately, the double placebo in this study makes the study design unsuitable to address that question directly.”

Response to: PROGRESS Collaborative Group. Randomised trial of a perindopril-based blood-pressure-lowering regimen among 6105 individuals with previous stroke or transient ischaemic attack.Lancet 2001;358:1033-1111

Clopidogrel in addition to aspirin in unstable angina (NEJM 2001)

Two articles were published over the same weekend in August 2001 reporting the results of the CURE study investigating the use of clopidogrel in addition to aspirin in patients with unstable angina (1,2). The Lancet published the results from a sub-group of 2658 patients who were treated with percutaneous coronary intervention, in other words they had coronary angioplasty or stents (1). The full study of 12,562 patients was reported in the New England Journal of Medicine, so this article will concentrate on the NEJM paper (2).
Let us first notice how the newspapers reported the results of this trial. One of the Medical Newspapers for General Practitioners in the UK reported that “Treating unstable angina and non-Q wave MI with clopidogrel and aspirin resulted in a 20% reduction in the risk of MI stroke and vascular death. The risk of major bleeding increased by 1% using both anti-platelet drugs”. Similarly Medical Monitor reported that “Clopidogrel cuts MI deaths by 23% and that for every 1000 patients treated with clopidogrel, six will require a blood transfusion.”

These reports are quite misleading because they have used the Relative Risk Reduction to report benefit but the absolute difference for the adverse effects (using percentages to report the outcome in each case). The NEJM paper itself is much more balanced and gives the proportion of patients who have cardiovascular death, MI or stroke as 9.3% in the clopidogrel group and 11.4% in the placebo group. This is illustrated below using Visual Rx (4) to display these results as a picture of 100 patients at risk:

Figure 1: Cates plot showing two people out of 100 on clopidogrel are prevented from having a stroke, MI or vascular death over nine months in comparison to placebo

Over the course of 9 months, treating 100 patients with clopidogrel will prevent 2 cardiovascular deaths, MI or strokes, which gives a Number Needed to Treat of 48. This is shown above as two yellow faces for those who are saved by clopidogrel, whilst the nine red faces are for those who suffer these events on clopidogrel or placebo. In practice we do not know which two will benefit so all 100 patients need to be given the clopidogrel.
In the same way 2.7% of patients suffered a major bleed on placebo compared to 3.7% on Clopidogrel, a Number Needed to Harm of 100. This is illustrated below, where the 3 red faces are the 2.7% who bleed given placebo or clopidogrel, and the single crossed-out green face is the extra patient who bleeds due to the clopidogrel who would not have done so on placebo.

Figure 2: Cates plot showing one additional person out of 100 has a major bleed on clopidogrel over nine months in comparison to placebo

So overall after 9 months of clopidogrel treatment there is one patient suffering a major bleed for two patients saved from strokes, heart attack or cardiovascular death for each 100 patients treated. This strikes me as a more balanced summary of the benefits and risks of treatment, using the SAME statistic to describe both outcomes.This example also illustrates the importance of reporting the rates of events in both groups (as shown on the pictures), and using the differences not the relative risks. In fact the relative risk of death, MI and Stroke is 0.80 (a drop of 20%) whilst the relative risk of major bleeding is 1.38 (a rise of 38%)! These relative risks by themselves are meaningless unless the event rates on placebo are also presented. The 20% reduction reported is from a baseline rate of 11.4% in the placebo group, so the absolute difference is only around 2% of all those treated with clopidogrel. Likewise the 38% increase in bleeding is from a baseline rate of 2.7% on placebo and translates into 1% of all the patients given clopidogrel.
Finally these patients had unstable angina and would have higher cardiovascular risks than patients with stable angina. It may be therefore that in stable angina clopidogrel could cause more bleeding events than heart attacks and strokes prevented.

10/9/2001
References:
1. Mehta S, Yusuf S, Peters R et al. Effects of pretreatment with clopidogrel and aspirin followed by long-term therapy in patients undergoing percutaneous coronary intervention: the PCI-CURE study. Lancet 2001;358(9281):527-33.
2. The clopidogrel in unstable angina to prevent recurrent events trial investigators. Effects of Clopidogrel in addition to aspirin in patients with acute coronary syndromes without ST-segment elevation. N Engl J Med 2001;345(7):494-502.
3. Visual Rx. http://www.nntonline.net

Warfarin or Aspirin to prevent stroke in Atrial Fibrillation? (BMJ 2001)

A BMJ review in 2001 sought to persuade us that there is little to choose between antiplatelet treatment and anticoagulation in atrial fibrillation except cost. (1) Rather different conclusions were drawn from analysis of a systematic review of a very similar data set published on the Cochrane library in the same year. In the latter case the reviewers concluded that ‘the evidence strongly supports warfarin in AF for patients at average or greater risk of stroke, although clearly there is a risk of haemorrhage. Although not definitively supported by the evidence, aspirin may prove to be useful for stroke prevention in sub-groups with a low risk of stroke, with less risk of haemorrhage than with warfarin.’ (2)
In these reviews it has not been possible to prove beyond reasonable doubt that aspirin is more efficacious than placebo or that aspirin is less efficacious than anticoagulation. The disadvantage of using a 5% significance level to decide if we can be sure about results was highlighted earlier this year in the BMJ (3). Non-significant trends are open to subjective interpretation when results are handled dichotomously in this way. Moreover whilst aspirin is certainly more convenient than anticoagulation, the cost argument employed by Taylor et al is flawed as the costs of caring for stroke sufferers (or those with major bleeds) has not been considered (4).

The directions of the differences found in trials randomising patients to warfarin or aspirin are the same as those found in the placebo-controlled trials. If non-fatal strokes are compared to major bleeds the pooled odds ratios are almost reciprocal from the meta-analysis of the head to head trials. In practice therefore the trade off for an individual patient depends on their assessed risk of having a stroke or a major bleed. In the majority of trials included non-fatal strokes are more than twice as common as bleeds, and therefore since both outcomes are rare the odds ratio behaves like a risk ratio. This means that in comparison with antiplatelet treatment, if 100 such patients are given anticoagulation for three years, roughly two non-fatal strokes will be prevented and one extra major bleed will occur. This is illustrated in the two Cates plots calculated using Visual Rx (version 4) and the point estimates for non fatal stroke and major bleed from Table 3 in the BMJ review with event rates of 7% for stroke and 2% for bleeds over 3 years with aspirin treatment.

In practice therefore the decision to prescribe anticoagulation or antiplatelet treatment needs to be individually assessed and discussed with each patient. Some may well choose aspirin, but this needs to be on the basis of the risks that they face of having a stroke or bleeding, not on whether the pooled results of a meta-analysis reach 5% significance.

Figure 1: Cates plot for warfarin versus aspirin in AF – impact on stroke over 3 years

Figure 2: Cates plot for warfarin versus aspirin in AF – impact on bleeding over 3 years

References:
Taylor FC, Cohen H, Ebrahim S. Systematic review of long term anticoagulation or antiplatelet treatment in patients with non-rheumatic atrial fibrillation. BMJ. 2001;322:321-6.http://www.bmj.com/cgi/content/full/322/7282/321
Segal JB, McNamara RL, Miller MR, Powe NR, Goodman SN, Robinson KA, Bass EB. Anticoagulants or antiplatelet therapy for non-rheumatic atrial fibrillation and flutter (Cochrane Review). In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software.
Sterne JA, Smith GD. Sifting the evidence-what’s wrong with significance tests? BMJ 2001;322(7280):226-231 http://www.bmj.com/cgi/content/full/322/7280/226
Cates CJ. Care is required with cost effectiveness approach. BMJ. 2000;321:449. http://www.bmj.com/cgi/content/full/321/7258/449

Do I need to change my practice (Pulse Article 2001)?

This article is part of a series on Critical Reading.

When speaking to registrars about critical appraisal, one of the commonest question is “How do I decide whether the paper is good enough to warrant a change in my current practice?” In the article on asking a good question I described how to break down the question addressed by a research paper into its four components, and having done this you next have to decide whether the findings of the paper are likely to be important to you and especially to your patients.

Is it valid?

In particular is the approach being described in the paper worth trying on the next patient who presents with the relevant condition. To answer this we need to look at issues relating to the validity of the paper in question. Two types of validity have been described: internal validity which relates to the mechanisms of the study itself and external validity which is more to do whether the results of the paper can be extrapolated to the patient in our own practice. In the rest of this article I will concentrate on issues of internal validity using as an example an imaginary study of olive oil for children with acute otitis media.

Choosing controls

The key issue to think about in relation to internal validity is to look at how a comparison group is chosen in relation to the patients who are given the experimental treatment. In a case-series (for example a set of 6 patients who are given a new treatment in routine practice) there may be no comparison group at all, so the immediate concern is that they might have achieved a good result anyway. For example I might tell you that I have treated a series of 100 children with acute otitis media with warm olive oil and that 85 were better in a few days. This sounds impressive until you look at the results of placebo treatment in antibiotic trials for this condition and find a similar recovery rate.

Better than a case series would be a case-control study in which the records of patients who had prolonged pain following ear infections were checked to see how many had been given olive oil; this proportion receiving olive oil could then be compared to the proportion of olive oil use in other patients who did not have prolonged pain. The problem now is being sure that the children do not have other differences influencing the olive oil usage, and this is rarely possible.

Better still a group of children could be compared by offering parents the choice of whether they use the oil or not; this would constitute a prospective cohort study but uncertainty remains about possible important differences between those who chose to have the oil and those who refuse it.

Overcoming Bias

In both the case-control study and the cohort study design the threat to internal validity is related to bias in the choice of the comparison group (selection bias), as well as other possible biases which may be present because both the patient and the doctor are well aware of the treatment that they have received. It will be no surprise to you that the only secure way around these biases is to use a randomised controlled trial that is preferably double-blind, and these will be addressed in the next article.

HRT and heart disease

So are any of these biases important. They certainly can be and a couple of examples may help to show how. In the early non-randomised studies of Hormone Replacement Therapy the results suggested that women on HRT had lower rates of heart disease, and HRT has therefore been advocated as a measure to reduce risks of Ischaemic heart disease(1). Some of the authors of these early studies did point out that there were some problems, particularly as the rates of road traffic accident deaths were also lower in the group receiving HRT. The more recent evidence from randomised controlled trials (such as the HERS study[2]) has not confirmed the protective effect and it is probable that the women who opted for HRT had other differences from the control group and may have had generally lower risk factors for heart disease.

Preventing Teenage Pregnancy

Another example of this was a cross-sectional survey in the BMJ reporting the association between teenage pregnancies and practice characteristics in different areas (3). The results include this statement “On multivariate analysis, practices with at least one female doctor, a young doctor, or more practice nurse time had significantly lower teenage pregnancy rates. Deprivation and fundholding remained significantly associated with higher teenage pregnancy rates.” The problem here is that we have no evidence that the age or sex of the doctors caused the lower rates of pregnancy, and the unexplained association with fund-holding practices having higher pregnancy rates should perhaps ring some alarm bells. No one suggested that the end of fundholding would solve the teenage pregnancy problem!

A fuller discussion of association and causation can be found in Follies and Fallacies of Medicine (Tarragon Press) [4] which I would recommend as both amusing and informative background reading for all registrars.

References:

1. Barrett-Connor E, Grady D. Hormone replacement therapy, heart disease and other considerations. Annu Rev Public Health 1998;19:55-72

2. Hulley S, Grady D, Bush T et al. Randomised trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA 1998;280:605-133.

3. Association between teenage pregnancy rates and the age and sex of general practitioners: cross sectional survey in Trent 1994-7. Julia Hippisley-Cox, Jane Allen, Mike Pringle, Dave Ebdon, Marion McPhearson, Dick Churchill, and Sue Bradley. BMJ 2000; 320: 842-845.

4. Follies and Fallacies in Medicine. Skrabanek and McCormick. Tarragon Press.

NICE 2000 guidance on the use of zanamivir for influenza

Purpose of the NICE guidance: to target zanamivir to at-risk patients with a high likelihood of having influenza. Hence the restriction to use in such patients and only when the level of circulating influenza-like illness has been confirmed to be above 50/100,000. Fever of over 38°C and a clinical picture of flu (sudden onset of illness with muscle pains and dry cough) are also a requirement before treatment with zanamivir is considered as many people think they have flu when they have much milder viral illnesses.

Benefits of the treatment: very modest with only a single day reduction in duration of illness and 7% reduction in complications requiring antibiotics. In other words 14 patients need to be treated with zanamivir for one patient to avoid the need for antibiotics. No proven benefit in terms of reducing hospital admission or mortality. (See picture below.)

Side effects: zanamivir can cause wheezing in asthmatic and COPD patients, so such patients are advised to have their reliever inhaler to hand when they take the treatment! In a study in healthy asthmatics one in 13 developed wheezing.

Children: zanamivir is not licensed for children under 12

Workload: NICE recognise that there could be a considerable extra workload caused by this guidance (in terms of telephone calls and home visits). Practices may wish to have a plan prepared to deal with this eventuality. A possible scenario would be 2 extra visits per GP per day with an unknown extra number of telephone calls for patients enquiring about their suitability for treatment. A questionnaire has been prepared for nurses to use in triaging telephone queries from patients and presumably this will be used by NHS direct, but could also be implemented at practice level. However issuing of prescriptions for zanamivir without seeing the patient seems unwise, in view of the possibility of complications (such as pneumonia), and the fact that it was a new ‘black triangle’ medication.

Cates plot on preventing complications of flu by using zanamivir

If 100 patients are all given zanamivir for a flu-like illness 74 will not suffer a complication requiring antibiotics anyway (shown as green smiling faces below); 20 will still need antibiotics (shown as red faces) and 6 (shown as yellow faces) will be saved from having antibiotics by the use of zanamivir.

Appendix 1

Summary of Nice Guidance on the Use of Zanamivir (Relenza) in the treatment of Influenza

Issue date: November 2000

Review date : June 2002

1. Guidance

1.1 For otherwise healthy adults with influenza, the use of zanamivir is not recommended.

1.2 Zanamivir is recommended, when influenza is circulating in the community, for the treatment of at-risk adults, who present within 36 hours of the onset of influenza like illness (ILI) and who are able to commence treatment within 48 hours of the onset of these symptoms.

1.2.1 Based on the evidence from clinical trials, at-risk adults are individuals falling into one or more of the following categories:

age 65 years or over

chronic respiratory disease (including chronic obstructive

pulmonary disease and asthma) requiring regular medication

significant cardiovascular disease (excluding individuals with hypertension)

immunocompromised

diabetes mellitus

1.2.2 Community based virological surveillance schemes should be used to indicate when influenza is circulating in the community (see paragraph 5.4).

1.2.3 Effective targeting of zanamivir for the at-risk adult population with a high incidence of true influenza is essential to maximise both the clinical and cost effectiveness of this therapy.

1.3 The guidance does not cover the circumstances of a pandemic or a widespread epidemic of a new strain of influenza to which there is little

or no community resistance. In such circumstances, the Department of Health and the National Assembly for Wales might wish to consult the Institute on the need for supplementary guidance.

Choosing controls in non-randomised studies (Lancet 2000 DVT and flying)

I wonder what your views are in relation to the risks of deep vein thrombosis (DVT) and long-haul flights? If you have the opportunity to travel by air do you take an aspirin before you go and perhaps even wear support stockings for the journey (one of my senior colleagues does) as well as getting up and walking about on the flight.

As far as I know none of these approaches has been tested in randomised trials on air passengers, so we have to rely on other types of study such as the research letter published in the Lancet recently (Kraaijenhagen RA, Haverkamp D, Koopman MMW, Prandoni P, Piovella F, Büller HR. Travel and risk of venous thrombosis. Lancet 2000;356:1492).

The authors of this study decided that the ideal control group for patients with deep vein thrombosis, which they could confirm on ultrasound or venogram, was the 75% of patients who presented to hospital with clinical signs of a DVT but tested negatively. On the basis of this control group they found that there was no association between any of the forms of recent travel that they asked the patients about (plane, rail or car travel) and whether their swollen leg was shown to have a clot in the deep veins.

They concluded that this was of some reassurance that travel was not associated with DVT, but there is a major flaw in their reasoning. They have made an implicit assumption (which is not discussed in their research letter) that there is NO association between travelling and swollen legs that do not contain a clot.

I am not sure that a great deal is known about the aetiology of clinically suspicious leg swelling that is found negative on ultrasound or venography, and it is at the least plausible that flying could increase the likelihood of this condition occurring. If for example there was a five-fold increase in both types of swollen leg after flying (that is those that are venogram positive and negative), the odds ratio of having previously flown in the past 4 weeks would still be one when the two groups were compared. For this reason I am not personally reassured by the findings of this study and plan to take whatever precautions I can when I am next on a long-haul flight.

When randomisation is possible the comparability of controls and cases should be less of a problem, but in non-randomised studies the assumptions arising from the choice of controls have to be examined carefully!

Reporting Results of Studies: can passive smoking really be good for you? (Pulse Article 1999)

Passive smoking and health risks.

“Passive smoking may be good for you” or so the tobacco companies would like us to believe! This idea arose from a misrepresentation of the confidence interval for data on passive smoking, and provides a good example of why we need a working knowledge of some statistics to deal with the propaganda that comes our way in General Practice. Sadly statistics is reported to be one of the subjects least liked by medical students, and those of us who have been in practice for more than a few years may be unfamiliar with some of the ways that results of studies are now reported. There has been a shift away from the use of p values towards Confidence Intervals (CI) in many medical journals, and the British Medical Journal now expects authors of papers to present data in this way.

Don’t forget common sense

Before going into more detail about the use of Confidence Intervals the example quoted for passive smoking above may be swallowed by the public, and even in some cases by journalists, but hopefully most GPs would be suspicious that such a finding just does not make sense. It does not fit with all the other data that has emerged in the past 20 years, and therefore needs some further looking at. Never leave common sense behind when looking at statistical reports!

Confidence Intervals or P values

So what are Confidence Intervals all about and how did they get misused in this example? In general when research is undertaken the results are analysed with two separate questions in mind. The first is how big is the effect being studied (in this case how big is the risk of lung cancer for passive smokers)? The second question is how likely is it that the result is due to chance alone? The two issues are connected, because a very large effect is much less likely to have arisen purely by chance, but the statistical approach used is different depending on which question you are trying answer. The “p” value will only answer the question “what is the chance that the study could show its result if the true effect was no different from placebo”? The Confidence Interval describes how sure we are about the accuracy of the trial in predicting the true size of the effect.

Both questions relate to the fact that we cannot know what the effect would be of a treatment or risk factor on everyone in the world; any study can only look at a sample of people who are treated or exposed to the risk. We then have to assume that if, say, one hundred identical studies were carried out in the same way on different groups of patients the results found would be normally distributed around the average effect size of the treatment. The larger the number of patients included in the trial the closer the result of that trial are likely to be to the true effect in the whole population. The result of any particular trial can therefore be presented as showing an effect of a certain size, and the Confidence Interval describes the range of values between which you can be 95% certain that the true value lies.

The data on Passive Smoking

Perhaps this can be illustrated with the passive smoking data. The results were that the on passive smoking study in seven European countries showed that there was an extra risk of developing lung cancer of around 16% for non-smokers who were exposed to smoke in the workplace or who had a spouse who smoked. This was comparing 650 lung cancer cases with 1542 controls in Europe and was accompanied by an estimate that 1100 deaths occurred each year in the European Union as a result of passive smoking.

The 95% Confidence Interval associated with this data is shown in the diagram and the tobacco industry had just chosen to highlight the lower end of the Confidence Interval, which shows a small chance that passive smoking could be associated with a 7% lower rate of lung cancer! Unsurprisingly they did not report the equal chance that the risk may be as high as 44% more lung cancer in passive smokers, and the Sunday Telegraph swallowed the story whole. More details are provided in the excellent article by Simon Chapman in the BMJ 1998;316:945.

Gardner and Altman mention this danger in their book “Statistics with Confidence”, and they suggest that results should be presented with the effect size, confidence interval and p value to prevent this kind of misunderstanding. The first two chapters are well worth reading if you want a fuller understanding of the rationale behind the use of Confidence Intervals. A final point about the Confidence Interval is that when it crosses the no-difference line (as shown in the diagram above) then the results do not reach significance at the level chosen (usually 5%).

Simon Chapman points out however that a meta analysis in the BMJ in the 18 October 1997 issue compared 4626 cases with 477924 controls and showed a 24% excess risk of lung cancer in non-smokers living with smokers. The 95% Confidence Interval was 13%to 36% which is well clear of the no-difference line and hence highly statistically significant, with a p value of >0.001. Again this data was conveniently ignored.

The moral of the story is that you cannot believe it just because you read it in the Newspaper. As far as the advantages of passive smoking are concerned, they can join the other myths and misunderstandings documented in one of my favourite books Follies and Fallacies in Medicine by Skrabanek and McCormick.

Statistics with Confidence MJ Gardner and D Altman BMJ Publishing 1989

Follies and fallacies in Medicine Skrabanek and McCormick Tarragon 1998

Can you trust what you read? Why we need Randomised Trials (Pulse Article 1999)

How can you tell if a paper is reliable? This was the question that many of the registrars wanted to have answered at a recent half-day release session on critical reading.

The Challenge of Archie Cochrane

Before he died Archie Cochrane expressed his sadness that no-one had gathered together the most reliable data available so that it could be used a basis for practice and research in Health Care. In response to this challenge the Cochrane Collaboration has emerged as a group of dedicated individual doctors and health care professionals who have set out to collect together data from controlled clinical trials, and summarise what they have found in the form of systematic reviews. The Collaboration is an international organisation and is structured by health problem areas to avoid duplication of effort. Many Journals have been hand-searched to identify controlled trials, and the reviews are structured so as to reduce bias at each stage of the process (which includes a ban on drug companies sponsoring individual reviews). In the UK many of the editorial bases are funded through the NHS Research and Development Programme.

The output of the Collaboration which includes a database of over 250,000 controlled trials identified and over 600 systematic reviews, is published in electronic form in the Cochrane Library, and a future article in this series will give an example of how it can be used.

The place of RCTs

There is considerable misunderstanding at this point about the place of Randomised Controlled Trials (RCTs). It would be unfair to say that you should not bother to read anything that is not an RCT, but it is also true to say that the most reliable way to study causation is with a systematic review of randomised controlled trials.

The way I like to look at the issue of randomisation is as follows; ask yourself the question “Could this trial have been randomised?” If you decide that it could have been randomised but it was not, then a large question mark should be placed over conclusions about whether the paper can reliably answer any questions related to the intervention causing good or bad outcomes.

Evidence for HRT

Hormone Replacement therapy is a good current example of this. Most of the current evidence relating to the purported benefits of HRT comes from non-randomised studies, and the results are therefore likely to be biased by differences between the type of women who opt for HRT and those who do not. An excellent editorial in the British Journal of General Practice in 1998 presents the current state of play in this area is recommended reading. Randomised Controlled Trials are currently under way to assess the effects of using HRT, but these will not report findings for a few years yet.

Sometimes you cannot randomise

There are of course some areas in which Randomisation is either impossible or unethical; you could not carry out a trial in which patients were randomised into cigarette-smoking or not! The very strong evidence on the dangers of smoking comes from large well conducted cohort studies, which are quite enough to leave little doubt about the size of the dangers involved.

Does it matter how you do it?

Whilst on the subject of Randomisation, how it is done matters too. The technical term to describe the actual randomisation is “allocation concealment” and if you read reports of older trials this often used to be done by using the patient’s hospital number to decide which treatment type they should receive or even alternate between treatments. It has been shown that trials with inadequate allocation concealment of this sort tend to show larger benefits of the intervention under study and it is not too difficult to imagine why.

Magic cure for warts?

Imagine you have developed a new treatment for removing warts and you arrange a trial to test it against one of the current methods. A patient walks into your surgery with a whole mass of horrible looking large warts, which you think that no treatment on earth will remove, and you can tell from the unconcealed alternated or random allocation that they would be in turn to receive your new technique. What will you do? Human nature is such that you will find some reason that this person will not quite fit into the trial and you will move on to another patient who has a nice small wart to treat next time. Obviously in this instance the advantage of randomisation in removing bias in the allocation process has been lost.

A better way to do it

When assessing the allocation concealment in Randomised trials I would look for at least opaque sealed envelopes which contain a random sequence of numbers to determine the next patient’s treatment. Even better would be a separate centre (such as the hospital pharmacy) to randomly allocate the treatment to the patient after the decision has been made to include the patient in the trial.

Try it yourself

So next time you are reading a paper, after you have asked yourself what is the question the paper was trying to answer, just pause to consider whether the trial could have been randomised. If it was randomised how easy would it have been to tell which treatment the next patient was getting? If you are satisfied on both of these fronts then read on, and if not perhaps move on to another paper.

References

1) Schulz KF, Chalmers I, Hayes RJ, Altman DJ. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 119;273:408-12

2) Hannaford PC. Is there sufficient evidence for us to encourage the widespread use of hormone replacement therapy to prevent disease? BJGP 48;427:951-2

Antibiotics and Common Infections (1999)

Introduction

Many common infections will resolve without the need for antibiotics, so when do we need to prescribe antibiotics and which ones should we use? In 1999, children antibiotics were given for acute otitis media more often than for any other condition seen in General Practice. Sore throat was also a common reason for prescribing in adults and children, and bronchitis may have been top of the list for prescribing antibiotics in adults.

This information page will look at these conditions and examine whether antibiotics change the course of the illness.

Bacterial resistance in the population is an increasing cause for concern in the UK and worldwide, and excessive use of antibiotics is believed to be an important factor in this process. Antibiotics also carry a risk to the individual patient of immediate side effects, such as rashes, diarrhoea and thrush.

In 2006 MEREC published a set of bulletins on upper respiratory infections, including pictures generated by Visual Rx in the section on Sinusitis.

2009 UPDATE: The limited benefit of antibiotics in preventing rare but serious complications (such as quinsy and mastoiditis) was highlighted in a National Prescribing Centre bulletin.

Acute Otitis Media

The peak incidence for acute ear infections is between 6 and 15 months, and there is no consensus on the best treatment; the rate of antibiotic usage in 1999 varied from 31% in the Netherlands to 98% in the USA and Australia.

Two reviews were published in the BMJ in 1997 examining the trials that had compared an antibiotic with placebo for treatment of acute otitis media (Del Mar 1997, Froom 1997). Both reviews concluded that most children would get better without antibiotics. If 20 children are given antibiotics for acute otitis media, 18 would be free from pain after two to seven days in comparison with 17 who would have been pain-free without antibiotics in the same time period. Moreover one extra child would also suffer a rash or diarrhoea if antibiotics were used.

Very few cases of mastoiditis were reported in children in either the antibiotic treated groups or the controls.

There was no clear cut picture as to which children would be more likely to benefit from antibiotic treatment, but one study suggested that younger children and those with previous episodes or bilateral acute otitis media were less likely to get better quickly with placebo treatment. (Burke P, Bain J, Robinson D, et al. Acute red ear in children: controlled trial of non-antibiotic treatment in general practice. BMJ 303:558-562, 1991). This is now further supported by more recent studies detailed on this site in the Ear infections section.

A possible approach to prescribing for acute otitis media is to discuss the evidence with parents (see the handout that we used) and to check that the child is being given a full dose of Paracetamol for their age. In those children who are not particularly ill and who are not prone to recurrent infections many parents may be happy not to use antibiotics. If the parents are keen for a prescription it can be given with the advice to defer cashing it at the chemist for 24 to 48 hours as the child may well be better by this time.

This approach was successfully adopted in our practice and it has been possible to achieve a considerable reduction in antibiotic usage for acute otitis media as a consequence. [Cates CJ. An evidence based approach to reducing antibiotic use in children with acute otitis media: controlled before and after study. BMJ 1999;318:715-716].

The results of five years of follow-up after our change in practice are documented in the Evidence Based Medicine Journal and a pdf can be found here.

Sore Throat

Many patients who present with sore throat are suffering from viral rather than bacterial infections and the worst looking purulent tonsillitis may be due to Glandular Fever. For this reason Amoxycillin should NOT be used for throat infections as it may cause an unpleasant rash if the patients’ throat infection is due to Glandular Fever. Penicillin V is still the first choice if an antibiotic is to be used for tonsillitis, as so far the Streptococcus remains fully sensitive to penicillin.

A summary of the evidence relating to antibiotics and sore throat in 2006 (which included data from 27 randomised controlled trials) came to the following conclusions:

“Antibiotics confer relative benefits in the treatment of sore throat. However, the absolute benefits are modest. Protecting sore throat sufferers against suppurative and non-suppurative complications in modern Western society can only be achieved by treating many with antibiotics, most of whom will derive no benefit. In emerging economies (where rates of acute rheumatic fever are high, for example), the number needed to treat may be much lower for antibiotics to be considered effective. Antibiotics shorten the duration of symptoms by about sixteen hours overall.”

(Del Mar CB, Glasziou PP, Spinks AB. Antibiotics for sore throat. Cochrane Database of Systematic Reviews 2006, Issue 4.)

Acute Bronchitis

An article in the Drugs and Therapeutics Bulletin has suggested that patients with signs of lower respiratory tract infection should be treated aggressively with antibiotics. (Antibiotic treatment of adults with chest infection in general practice. DTB 1998;36:68-72)

Should antibiotics be given to other patients with acute bronchitis? A systematic review seeking to answer this question identified 750 patients (aged 8 to 65) included in randomised controlled trials to compare antibiotic treatment with placebo. The antibiotics used were erythromycin, co-trimoxazole and doxycycline. The benefit in terms of average duration of symptoms is about half a day less cough or sputum production if antibiotics are given, and in some studies the cough continued for more than five days whether antibiotics were given or not. On average patients given antibiotics tended to return to work 0.75 days earlier than those on placebo.

The number needed to harm with adverse effects was 15 patients which is close to the number needed to treat of 13 to avoid one case of no improvement at follow-up when assessed by the physician. (Becker L, Glazier R, McIsaac W, Smucny J. Antibiotics for Acute Bronchitis (Cochrane Review). In: The Cochrane Library, Issue 3, 1998. Oxford: Update Software.)

Similarly balanced figures were obtained from another systematic review on this subject in the BMJ this summer, with estimates of 11 for the number needed to treat and 13 for the number needed to harm with an antibiotic prescription.

(Fahey T, Stocks N, Thomas T. Quantitative systematic review of randomised controlled trials comparing antibiotic with placebo for acute cough in adults. BMJ 1998; 316: 906-910)

Conclusion

The available data shows that the benefit of antibiotics in many patients presenting with sore throat, earache, cough and sinus pain is very limited. This presents two challenges for us in general practice. How can we target treatment to those few patients who will benefit most from an antibiotic, and how can we reduce our prescribing for those who do not.

There is precious little evidence to help us decide which patients will benefit from antibiotic prescriptions so at present we have to continue to use clinical judgement (such as whether the patient has a high fever and looks toxic). Throat swabs are of limited use due to the background carriage rate of Streptococcus in healthy individuals and the time taken to receive the results of the test.

It is certainly worth checking what the patient is expecting, as in general we tend to assume that the patient wants an antibiotic when this may not be the case. It is also helpful to have written information explaining the limited benefits of antibiotic prescriptions and the problems of side effects and increasing antibiotic resistance that can be given to the patient.

A copy of the otitis media handout is included on this site, and you are welcome to adapt this for you own use (tailored to your practice population) and perhaps ask some of your patients to comment on the wording before putting it out for general use. The data presented in this paper may be used as a basis for handouts.

Deferred prescriptions have been found to be useful in acute otitis media which resolves quickly (often within 48 hours) as the parents can be involved in the decision as to whether the child is given the antibiotic, depending on how quickly the symptoms resolve. It is not known how successful the same strategy would be in bronchitis, since the resolution of cough if often fairly slow.

Final Points

Discuss in advance any changes that you plan to make with the whole practice; patients are very quick to exploit differences between partners.
Start with one area (such as acute otitis media in children) and see if you can demonstrate a change; the Medicines Management department at your Health Authority may be able to help with the collection and analysis of PACT data for you, or you can use your own level 3 PACT data.
In the long run not giving an antibiotic prescribing can decrease the reattendance rate, as shown by the study in Southampton. (Little P et al. Reattendance and complications in a randomised controlled trial of prescribing strategies for sore throat: the medicalising effect of prescribing antibiotics. BMJ 1997;315:350-353)
Finally don’t accept uncritically everything that drug reps tell you; their job is to persuade you to use their product and there is sometimes another side to the story!

Category: Clinical