Visual Rx Help File
What's New?
In the 2004 Updated version of Visual Rx the Confidence Interval (CI) for the calculated Number Needed to Treat (NNT) can be generated by entering the upper and lower CI for the Odds Ratio or Relative Risk on the data entry screen. The CI for the NNT will then be displayed on the second smiley face screen. The rounding is always up to the next integer when calculating NNT (so 3.1 will round up to 4).
Using Visual Rx
This programme can change pooled data from Meta-analysis into Number Needed to Treat. It is designed primarily for use with The Cochrane Database of Systematic reviews, but can also be used with pooled data from any meta-analysis. The data you will need to input into Visual Rx is found in the Forest plots in a particular Cochrane review. This screen can be reached by clicking on the Outline button for the individual review and selecting "Graphs and tables". Follow the blue hypertext link to the outcome that you want to use and the pooled result (with CI in brackets) is usually shown at the bottom of the screen.
TIP: NNT can only be calculated from outcomes that have been analysed as data that has counted the number of people who experience the event of interest. NNT cannot be calculated from data measuring the average size of events. For example "Hospital Admission" can be used, but not length of stay in hospital.
Choice of Statistic method from the Cochrane Review
The writer of the review determines the statistical method displayed on the Cochrane Database. Other methods of analysing the data in the review can be selected using the Statistics drop-down menu at the top of the display screen in Metaview (follow the link at the top left of the Forest plot screen). If outcomes are recorded as continuous data then it is impossible to calculate NNT for this outcome.
Ideally the Statistical Method for a particular review should be determined in advance for all reviews in the protocol (to avoid post-hoc decisions), but it is worth inspecting the Heterogeneity figures at the bottom of the Metaview screen for the outcome you have selected.
The Chi-squared total for the outcome is displayed at the bottom of the list of trials reporting the outcome, and is separately reported for each sub-group (if any are present).
Please check whether data are recorded as adverse events (i.e. as mortality rather than survival) and see the section "how events are recorded" before proceeding to enter data into Visual Rx.
WARNING: If all the Chi-squared results are much higher than the number of trials that have been pooled then there is probably significant Heterogeneity and none of the Statistical methods fits the data in this outcome. This can be checked using a Chi-squared table. In this case caution should be exercised before calculating NNT from the data.
How events are recorded in Cochrane Reviews
Visual Rx can be used to calculate NNT from any of the summary statistics on the Cochrane Database, but it is designed to analyse data as ADVERSE EVENTS. Most (but not all) Cochrane reviews enter data about bad things that happen to people, but some report how well patients respond,
EXAMPLE: Reviews on smoking tend to report how many patients stop smoking (smoking cessation). This is a good event as smoking is damaging to health.
WARNNG: Visual Rx can convert data from Reviews reporting beneficial events such as smoking cessation, but ONLY USING THE ODDS RATIO. Relative Risks will give different answers when beneficial events are used and the baseline risk varies.
Entering Data into Visual Rx
When you have checked the Heterogeneity for the Statistical method and checked that the data from the outcome are recorded as adverse events, then you can proceed to enter the data.
Intervention title
This is a free text field that you can use to label the graphical display of NNT generated by Visual Rx. Keep the title short, as only fifty characters will be displayed.
Baseline Risk (Previously CER)
The next data field is for the baseline risk (previously referred to as Control Event Rate, or CER). This can be entered directly from the data displayed on the Forest plot. Initially you may wish to use the pooled CER which is the sum of all the events in the control groups (in all trials) divided by the total patient numbers in control groups in all trials. This is displayed as a fraction at the bottom of the column of CER results for each trial. However this may not be the best way to assess the baseline risk of a particular type of patient, as it depends on the size of the included trials and will be heavily influenced by the largest trials.
The percentage risk should be entered into the first box on Visual Rx (e.g. a risk of 18% is entered as 18 in the first box), and the second box is set at a default at 100. If you wish to enter the baseline risk as a fraction put the numerator (top of the fraction) in the first box and the denominator (bottom of the fraction) in the second box.
Note: This pooled CER is dependent on the characteristics of the patients entered in each trial (amongst other things) and you may need to adjust this using other information to estimate the baseline risk of a particular patient group before calculating NNT for that group. If you want to change the baseline risk maximise the data entry screen and put in a new baseline level (the Odds Ratio and CI will stay in the boxes), and then click ‘Calculate’ again.
Statistical Method
The Odds Ratio is the default method for data entry into Visual Rx, but other methods can be selected on screen if desired. Click on the arrow to the right of the box on Visual Rx and then click on your chosen method to select it.
See section "Choice of Statistical Methods from a Cochrane Review" for more details on which method to use.
Odds Ratio
The pooled Odds Ratio may already be displayed on the Forest Plot on the Cochrane Database at the bottom of the column of results for each trial. You may need to enlarge the Metaview screen or use the scroll bars to find it. If another statistical method has been used to pool the results you can follow the top left link to show Metaview and choose Odds Ratio from there. It usually makes little difference whether the Peto Odds Ratio or the MH Odds ratio is used for calculation.
The pooled Odds ratio should be entered into the box on the Visual Rx data entry screen as a decimal (not as a percentage). For example 0.78 should be typed directly into the box.
The pooled result on the Forest Plot is presented with the 95% confidence limits in brackets and these should be entered into the boxes on the data entry screen for Visual Rx, again using a decimal point as above.
If you now click on the Calculate button Visual Rx will show first the baseline risk for the patients when no treatment is given. If you click on the NEXT button the programme will calculate the NNT (rounded up to the nearest whole number) and display the NNT, CI and a graphical display of the result on screen.
When the Odds Ratio is displayed there is a further check box to show that the data has been entered as adverse events. If the data is recorded as beneficial events (such as cure) uncheck this box before proceeding to calculate the NNT.
If you wish to save the pictures it is best to hold down the ALT button and press print screen. This will copy the contents of the open window to the clipboard and it can then be pasted into a picture-editing programme (such as PAINT).
Relative Risk
The option to use Relative Risk can be selected by clicking on the arrow to the right of the Statistical Method box, and a drop-down menu allows Relative Risk to be chosen. The pooled effect can be entered in the same was as for the Odds ratio, as can the upper and lower confidence intervals.
If beneficial events are recorded this statistic should not be used as the NNT will vary with baseline risk in the opposite direction to the relative risk for adverse events.
Risk Difference
This method can also be entered into Visual Rx using the method as described for the Odds ratio. Although the pooled risk difference is often used in this way it has several disadvantages. The first is that this method usually has the highest Heterogeneity and therefore fits the data least well. Moreover the pooled risk difference cannot be used across a specified range of controlled event rates as it is assumed to be constant across all of these. This may not be a good assumption when the CER varies widely, as it often does in Meta-analysis.
The point estimate of the Risk Difference is dependent upon the pooled CER and this is at the mercy of the size of the different trials included, and the baseline risk of the patients in each trial. For this reason the NNT calculated from the Risk Difference may not be stable when new trials are added to a Cochrane review.
This statistic is therefore not generally recommended for use in Visual Rx, but has been included for comparative purposes.
Experimental Event Rate
Some advocate this method, (including Bandolier), but you will need to calculate your own Confidence Interval figures and this is not recommended for pooled results – it can be used for single trials. The boxes are available to enter the total n/N in the Experimental Group. If the trials have unbalanced control and experimental groups Simpson's paradox may give the wrong result. An example of this is found in the review of Nursing intervention for smoking cessation on the Cochrane library.
As the results are beneficial in this review the CER and EER need to be recalculated for Adverse events so for High Intensity Intervention to totals to be entered are not
CER 356/2301 and EER 546/3820
but instead CER 1765/2301 and EER 3274/3820
Compare the display for these lumped totals and the Peto Odds Ratio of 1.39 with the CER of 356/2301, but uncheck the adverse event box. You will then see the results of Simpson's paradox for yourself.
Again this method is included only for comparative purposes. It is not possible to calculate the confidence intervals using Visual Rx for this method but the formula to do so is included in Statistics with Confidence by Gardner and Altman for interested readers.
The Graphical Display
This display represents a total of 100 faces that are divided into three categories. These can be thought of as if you were performing triage. The first screen shows faces that are either red or green. The smiling green faces are those patients who are free from harm (such as pain) with the control treatment. The red faces are those in pain with the control treatment. The baseline risk is shown in the box at the top left.
If you click the next button you will see the second smiley face screen demonstrating the effects of treatment. The smiling green faces are those patients who are free from harm (such as pain) with the active treatment. The red faces are those still in pain even with the active treatment. The middle group are the only ones that benefit from the treatment (and are shown as yellow faces). Since it is not possible which patients will benefit, all 100 have to be given active treatment for this group to benefit. The NNT is the inverse of this proportion and is displayed at the top left of the screen.
If the treatment is harmful over all then there are no yellow faces but instead the crossed-out green faces are those harmed by the treatment for the outcome under consideration. The NNH is the inverse of the proportion of these faces to the hundred total that are treated.
If you wish to assess the impact of treatment on patients at different baseline risks maximise the data entry screen (by clicking on the icon on the toolbar) and put in a new baseline level. The results in the OR or RR boxes will stay as previously entered (including the upper and lower confidence intervals).
Updates
It is envisaged that the programme will be regularly updated so please give us an email address to let you know when a free update is available.
Suggestions for improvements are welcomed. We are currently working on a modification to allow easier printing and saving of the pictures.