Category: Evidence & Practice

Evidence & Practice Articles Overview

Network Meta-analysis comparing treatments to prevent asthma attacks in adults (BMJ 2014)

There is an interesting example of a network meta-analysis that was published by Loymans and colleagues in the BMJ in May 2014 (Loymans RJB, Gemperli A, Cohen J, Rubinstein SM, Sterk PJ, Reddel HK, et al. Comparative effectiveness of long term drug treatment strategies to prevent asthma exacerbations: network meta-analysis. BMJ2014; 348: g3009). http://www.bmj.com/content/348/bmj.g3009

In an accompanying editorial at http://www.bmj.com/content/348/bmj.g3148, I have outlined how the network meta-analysis combines evidence from the included randomised trials of maintenance treatments for adults with asthma.

The strength (and weakness) of the network approach is that treatment effects estimated from the direct randomised comparisons within each trial are combined with indirect non-randomised comparisons of the treatment effects between the trials. Although the authors tested the consistency of the direct and indirect comparisons, the power to find inconsistency is low when there is only a small amount of direct evidence. This is pointed out by the authors and demonstrated by the wide confidence intervals in Figures 5 and 6. Note that these figures are on a log scale, so 2 on the scale represents a Rate Ratio(RR) of e2, which is equivalent to RR = 7.39!

The problem is that indirect comparisons are subject to potential confounding by differences between the participants, outcome measurements and trial designs between the individual trials. This assumption is that the patients are similar enough that it would be reasonable to assume that they could been found with equal likelihood in any of the trials? The technical term for this is a “transitivity assumption”.

One way to assess whether the network meets this transitivity assumption is to look at the asthma exacerbation (attack) rates across both arms of each of the trials to see if these are broadly similar. The Loymans network is very well reported and includes full documentation of the exacerbation rates in each trial in supplementary table S1. This shows considerable discrepancy between the attack rates in the different trials.

The variation in the frequency of the asthma attacks between studies causes two problems. Firstly the results cannot readily be applied in clinical practice, as the treatments have been compared in a wide variety of different severities of asthma. Secondly, since the frequency of asthma attacks is likely to be an important effect modifier when comparing the effects of the different maintenance treatments, the indirect comparisons may be confounded by this.

One example of the possible impact of such confounding is explored further in my editorial, and may explain why combination with lower dose of inhaled corticosteroids (ICS) is ranked in the network as likely to be more effective than combination therapy with higher dose ICS. In general the trials using higher dose ICS were on people who had more severe asthma and more frequent attacks (as you might expect), so this is not a fair indirect comparison of the higher and lower dose combination therapy. There was very little direct evidence comparing the high and low dose ICS combination treatments (i.e. randomised within the same trial).

Finally, it is also not too surprising that current best practice comes out as least likely to lead to withdrawal of treatment. The authors suggest that this outcome gives an indication of the safety of the asthma treatments. However, it may simply be that all the other treatments required some change for the participants. When treatments are changed we would expect some people to dislike the new inhaler for one reason or another. Again this is not really a fair comparison.

It is also a shame that the safety of the different treatments is discussed in the text without much mention of serious adverse events, which are in relegated to a supplementary table!

For a more in-depth discussion of the methodology of the network approach, you might find the following helpful: Cipriani A, Higgins JPT, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis.Ann Intern Med 2013; 159: 130-7.

How to make sense of a Cochrane systematic review (BREATHE 2014)

This free access 2014 article in BREATHE unpacks what goes into Systematic reviews and gives pointers about how to appraise them. We use an example of the Cochrane systematic review comparing spacers with nebulisers to deliver salbutamol in acute asthma in adults and children.

Full PDF Article available here

Communicating Risk (BMJ 2012)

“Communicating Risk” is the title of a clinical review in the BMJ on June 23rd 2012. The article summarises some of the different ways in which evidence can be presented, and discusses the relative merits of using Relative Risk Reduction (RRR), Absolute Risk Reduction (ARR) and Number Needed to Treat (NNT). Within Cochrane Systematic reviews the new Summary of Findings tables are designed to take relative measures (such as Odds Ratios and Relative Risks) from systematic reviews and turn them into Absolute Risk Reductions, as the benefits and risks of a treatment can then be directly compared with each other. Visual Rx will achieve the same thing and generate a Cates plot to use pictorial evidence alongside the numbers (such as Figure 2 in the BMJ article). The Cates plots from the NPC decision aid on statins is shown for a patient with a 20% risk of a cardiovascular event over 10 years, but most Primary care computer systems will now generate a personalised risk for the patient. This is why the simple data entry screen (labelled statin calculator) has been put in the left margin of our website, so clinicians or patients can enter their own 10 year CVD risk and see how much difference taking a statin would make to them.

The BMJ article also discusses the use of natural frequencies and the evidence from Akl and colleagues that clinicians and patients find natural frequencies easier to understand than probabilities. Finally they also point out that those patients who use decision aids, such as those provided by the NHS National Prescribing Centre are “consistently more ready to make a decision than those receiving usual care”.

The rapid responses also make interesting reading. Akl wonders if it is time for RRR to be abandoned, and Miriam Pryke questions whether it is ever right to “endeavour to change beliefs or promote behavioural change”, rather than just present the evidence.

An article in the BMJ in February 2012 on “Shared decision making” also makes interesting reading and covers many of the same issues from a slightly different perspective, as well as featuring a Cates plot used by the Mayo clinic to explain the benefits of statins.

Follies and Fallacies in Medicine

One of my favourite books is “Follies and Fallacies in Medicine”.  It has been out of print for some time but Lucy Skrabanek has kindly made the book available as a pdf file. You can find it at http://chagall.med.cornell.edu/Skrabanek/ If you have not read it I would recommend downloading the file and browsing through. It will not take too long and I would be very surprised if you do not find it interesting and relevant

On the same lines there is an interesting BMJ Blog by Peter Arnold entitled “I don’t know”. There is and interesting overlap between this article and the forward to Follies and Fallacies in Medicine in which Skrabanek and McCormick refer to being infected with “skepticaemia”. The very first step towards using Evidence in Patient-care is recognising that there is much we do not know.

“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” 
Mark Twain 

No evidence is without ideology (Julian Tudor Hart BMJ 2005)

This is the title of a very sensible rapid response from Julian Tudor Hart in the BMJ in October 2005, and he has given permission for me to quote him in this newsletter, (see below).

I entirely agree that it is not possible to consider bald evidence without considering the ‘story’ or ideology that has inspired the work that produced the evidence. This story may have a major bearing on the interpretation of the results of any study, or for that matter systematic review.

In the same way that ideology always interacts with evidence, it is a false division to try to look at evidence based medicine and narrative based medicine as if they are opposite methods of practice. As I have tried to point out in the recent interview for Doctor Magazine, consulting in medicine needs to consider the patient’s agenda, their ‘story’. Patients do not present complaints alone, but usually have interpreted them after discussion with friends or family members. If we never dig out their story the patient is unlikely to be satisfied with the consultation, and dysfunctional consultations may result when the patient’s story has a completely different set of assumptions to the doctor’s story of what the symptoms might mean.

The publication of a trial testing a new treatment is designed to tell a story from the sponsors which might read along these lines: “this is an excellent new product that your patients will benefit from.” Inevitably this will influence the way results are presented, and which of the huge quantity of outcome data collected is presented to the readers. Registration of clinical trials and publication of trial protocols should help to reduce publication bias of this kind, but one of the fascinations of reading reports of trials is to look underneath the results to the underlying ‘story’, and see how this colours the way the data is presented.

As you read you may wish to reflect on your own pre-existing ideas about the clinical question being asked. What are your assumptions? How do these differ from the authors’ assumptions? Is there a bigger picture? Are the outcomes presented the ones that the treatment has the best chance of influencing? How does this relate to the outcomes that really matter to the patient? Have adverse advents been considered, and have enough patients been studied to detect serious but rarer events?

Evidence should be considered with its ideological framework, just as symptoms need the context of the patient’s story so that we can see how they make sense to their owner!

Reference:

Hart JT. Evidence not ideology: No evidence is without ideology. BMJ 2005; 331:964

“No evidence is without ideology

EDITOR—Godlee asks for evidence without ideology, as though it were possible either to discover or use evidence without ideology of some kind. Scientific evidence is derived from hypotheses conceived within an ideology—that is, a set of prior assumptions about the real world, established by previous evidence, by faith, or by both. New evidence can then be produced by testing hypotheses derived from those assumptions against reality. The validity of competing hypotheses, including those macro-hypotheses about the world or society we call ideologies, depends on their explanatory and predictive power in the real world.

About the private finance initiative and the Blair government’s disintegration of the NHS into a competitive market led by consumer wants rather than by national health needs, nobody has published more evidence than Pollock. For the editor of the BMJ to dismiss this as led by ideology is an impertinence. Without exception, every paper published by the BMJ starts from ideological assumptions of some kind. That the editor’s assumptions apparently coincide with those of currently fashionable and conventional opinion does not change their ideological nature. Readers can make their own judgments as to which ideology has most explanatory and predictive power, either experimentally or in the more chaotic real world of practice, which in the absence of pilot projects is all we have to go on in assessing the consequences of marketisation.

This is a deadly serious business. Asked to describe the nature of the corporate state in the 1920s, before the full consequences of fascism were understood by comfortable people outside Italy, Mussolini answered that in his state the worlds of government and business would become one and indivisible. Godlee should consider how far we have already travelled along that road, and then reconsider the ethics of neutrality in such a situation. At the birth of the NHS, the BMJ had a role of which its later editors were frankly ashamed. Today, when the NHS is being buried alive, has it lost the power of speech?

Julian Tudor Hart, retired general practitioner

Primary Care Group, Swansea University Clinical School, Swansea SA2 8PP julian@tudorhart.freeserve.co.uk”

“The influence of the Pharmaceutical Industry” House of Commons Health Committee report 2004

“Why don’t you see Drug Reps?” This was the question posed to me a few years ago when I attended an educational meeting at our local post-graduate centre. I was collared trying to get past the stands of promotional glossy brochures and free gifts by a new representative to our area. I muttered something about not finding the information that reps gave very useful to me, and he retorted that his company had just released a new tablet for diabetes. “The results were very encouraging”, he said, and “I was sure to hear more about it soon”.

I did. A few months later the drug in question was withdrawn because it caused unexpected serious liver problems in some patients.

Last week saw the voluntary suspension of another Cox-2 inhibitor, valdecoxib (Bextra) by Pfizer. The memo from Professor Duff at the Committee of Safety of Medicines states that “valdecoxib is associated with a higher rate of serious, potentially fatal skin reactions, …… ,in addition to the established class evidence of cardiovascular risk with selective COX-2 inhibitors.”

It has been our policy not to see drug reps in the practice for the past 20 years, and I think if I was asked today why we do not see drug reps, I would use the COX-2 story as a good example. The BMJ theme issue in May 2003 included several papers on this topic, including a cross-sectional survey that showed an association between frequent contact with a drug representative and greater willingness to prescribe new drugs(1).

This is hardly earth-shattering news – pharmaceutical companies would not continue to invest in their representatives if they did not boost the sales of their newest products. The same BMJ issue also contains a reference to a confidential survey of 117 junior hospital doctors in the USA(2). Of the 117 who were sent the survey an impressive 105 replied. They describe an interesting perceived personal impermeability to the influences of the pharmaceutical industry: “Most respondents (61%) stated that industry promotions and contacts did not influence their own prescribing, but only 16% believed other physicians were similarly unaffected.”

So why do you see drug reps? Isn’t it better to obtain information on new drugs from parties who do not have such a vested interest in them? I will continue to wait for assessments in the Drugs and Therapeutics Bulletin before rushing to prescribe the latest drugs on the market.

In the next newsletter I will reflect on the recently published House of Commons Health Committee report “The Influence of the Pharmaceutical Industry”(3). It makes fascinating reading, particularly in relation to the recent problems with SSRI and COX-2 inhibitors, and the hidden ways in which the industry seeks to increase the sales of its products. It is available free online athttp://www.publications.parliament.uk/pa/cm200405/cmselect/cmhealth/42/42.pdf I will just have to finish reading it first!

References:

1.Watkins C, Moore L, Harvey I, Carthy P, Robinson E, Brawn R. Characteristics of general practitioners who frequently see drug industry representatives: national cross sectional study. BMJ 2003;326(7400):1178-1179.

2.Steinman MA, Shlipak MG, McPhee SJ. Of principles and pens: attitudes and practices of medicine housestaff toward pharmaceutical industry promotions. Am J Med 2001;110(7):551-7.

3.HouseofCommonsHealthCommittee. The Influence of the Pharmaceutical Industry. London: The Stationery Office Ltd; 2005.