Evidence Based Medicine and Virtual Journal Club

Hi, I’m Tom, currently a third year at Birmingham and have been asked to explain the premise of evidence-based medicine as well as appraise an article from an online session recently.

Evidence-based medicine

The current model of medicine is founded upon the idea of evidence-based medicine (EBM), utilising research to optimise decision making as opposed to opinion-based medicine whereby a doctor’s opinion governs decision making. The power of different pieces of research forms the evidence hierarchy.

Evidence hierarchy

The pyramid shown on the right (1) is a visual representation of the evidence hierarchy. Moving up the pyramid leads to a higher quality of evidence with a lower risk of bias - ‘a systematic error, where a particular research finding deviates from a ‘true’ finding’ (2). As such, systematic reviews (of randomised control trials) are intrinsically of far higher quality and at less intrinsic risk of bias than opinions of individuals (a qualitative piece of research).

CASP

CASP stands for Critical Appraisal Skills Programme and is a method of assessing the quality of a piece of research. It involves checklists of yes/no/partial yes and through some methods can quantify (assess through a number) how good the research is. Other methods such as AMSTAR exist for systematic reviews.

Birmingham

At Birmingham, we learn about evidence-based medicine, the evidence hierarchy and CASP through lectures during the pre-clinical year modules. In our third year (our first clinical year), there is a specific module (EBM) dedicated to the discussion of this topic and specific study designs. During this year, the understanding of EBM becomes far more relevant and relatable to understand why specific treatments are used in medicine and the impact, both positive and negative, of these treatments on patients.

Political and public acceptability of a sugar-sweetened beverages tax: a mixed-method systematic review and meta-analysis by Eykelenboom et al. (3)

The review assesses a clearly focussed question, that of public and political acceptability of a ‘sugar tax’. The question is further split into qualitative and quantitative acceptability. Quantitative acceptability would be useful in drawing statistical conclusions whereas qualitative assessment allows for real-world impact/ opinions. A focused population and intervention are noted, though the particular quantification of ‘acceptability’ was not.

The authors looked for qualitative and quantitative studies though any study design assessing quantitative impact of the ‘sugar tax’ was included, without the authors deciding upon an ideal study design prior, where results would be more comparable. The authors note that inclusion was ‘hierarchical’, one can presume based upon the evidence hierarchy.

Four databases were used (PubMed, EMBASE, Scopus and Web of Science) but further databases such as Medline could have been utilised to maximise search results. Examination of references of identified articles did find further search results, though no table pertaining to such studies is listed. There is no mention of searches for unpublished articles, a potential source of studies particularly due to the new nature of the topic. The authors set no original language limit but excluded non-English studies later - a flawed methodology, introducing bias, significantly as multiple countries worldwide have introduced sugar taxes, many non-English speaking. A list of excluded studies was not included, to review detriment. Using two reviewers to assess database results and extract data were positives.

Using a standardised tool (MMAT) to assess included articles reduced bias. However, individual tools for each study design may have been more appropriate to fully compare evidence quality.

Results have been combined, particularly for those presenting quantitative impact, where they were combined to form a meta-analysis, though this was not warranted (see later). Results were combined into themes, an area of subjectivity, a review weakness. Understanding the combination of studies is aided by the characteristics of studies table. A positive is that no competing interests were noted, nor was there private finding.

Overall, the review showed public support for the ‘sugar tax’ of 42% with a confidence interval from 38-47%, specifically as a strategy to reduce obesity of 39% (29-50%) and if revenue from such a tax was appropriately used of 66% (60-72%). The above results are from the meta-analysis, where heterogeneity had an I2 value of 99.07%, extremely high, meaning that there are extensive differences between studies’ methodologies and results. One should be sceptical as to why the results were originally combined. 50% (48-52%) believe the ‘sugar tax’ would have negative impact upon socioeconomic equality though many other subthemes are lacking in quantitative evidence, as the authors note. Confidence intervals, specifically for ‘as a strategy to reduce obesity’ were relatively broad, reducing precision. The review did not estimate political acceptability as no studies fulfilled the inclusion criteria.

Without all studies commenting on every topic, understanding the power of each statement is difficult, such as where only one study discussed mistrust of public health experts.

Many outcomes are considered but whether studies have been included in the correct outcome/ theme is subjective and further questioned by meta-analysis heterogeneity. The researchers provide future research direction, a benefit, specifically in comparing pre- and post-tax acceptability.

The power of qualitative and quantitative research together is useful for the ‘sugar tax’ as it assessed effectiveness and opinions on the matter. As the majority of studies were conducted in economically developed countries, the review is broadly applicable to the UK. One cannot assess how appropriate the results are for Birmingham specifically and some results may be impacted by responder bias due to how questions may have been asked. This can reduce how valid results are to the reviews question and therefore their applicability. With some UK studies in the review conducted through online forums, there may be a predisposition for participation bias, whereby those with stronger opinions may selectively involve themselves. Furthermore, studies conducted through online parental forums may select for a higher socio-economic class, which is often associated with increased health literacy (how people understand, promote and maintain good health (4)), so inflating the perceived acceptability of the ‘sugar tax’. A positive of the review focuses upon its recommendations in order to change the balance of benefits and harms resulting from the ‘sugar tax’.

References

1. Hattis D. Evidence Hierarchy [Internet]. 2016 [cited 5 April 2020]. Available from: https://www.researchgate.net/publication/311504831_Options_for_basing_Dietary_Reference_Intakes_DRIs_on_chronic_disease_endpoints_report_from_a_joint_US-Canadian-sponsored_working_group

2. Skill Zone Ltd h. Definition: Bias [Internet]. AQR.org.uk. 2020 [cited 7 April 2020]. Available from: https://www.aqr.org.uk/glossary/bias

3. Eykelenboom M, van Stralen M, Olthof M, Schoonmade L, Steenhuis I, Renders C. Political and public acceptability of a sugar-sweetened beverages tax: a mixed-method systematic review and meta-analysis. Int J Behav Nutr Phys Act [Internet]. 2019 [cited 4 April 2020]; 16(1). Available from: https://ijbnpa.biomedcentral.com/track/pdf/10.1186/s12966-019-0843-0

4. WHO | Track 2: Health literacy and health behaviour [Internet]. Who.int. 2020 [cited 7 April 2020]. Available from: https://www.who.int/healthpromotion/conferences/7gchp/track2/en/