SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review*_Chinese Journal of Evidence-Based Medicine

Authors：

Simon Lewin ¹ , Andrew D Oxman ² , John N Lavis ³ , Atle Fretheim ⁴

1. Norwegian Knowledge Centre for the Health Services; 2. Norwegian Knowledge Centre for the Health Services; 3. Centre for Health Economics and Policy Analysis; 4. Norwegian Knowledge Centre for the Health Services;

Corresponding author：

Keywords：

DOI：

10.7507/1672-2531.20100416

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

健康干预效果系统评价的可靠性参差不齐。因此，政策制定者及支持者需评价此类证据的可信度。采用系统透明的评价过程可避免判断时出现误差和偏倚。在确定干预效果系统评价结果的可信度时，本文建议考虑以下五个问题：① 这篇评价是否明确提出了一个恰当的决策或管理问题？② 是否使用了恰当的标准去评价系统评价中的研究？③ 对相关研究的检索是否详尽且较全面？④ 对纳入研究与评价主题相关性及其偏倚风险所展开的评估是否具有可重复性？⑤ 不同研究之间的结果是否相似？

Citation： Simon Lewin,Andrew D Oxman,John N Lavis,Atle Fretheim. SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review*. Chinese Journal of Evidence-Based Medicine, 2010, 10(4): 388-397. doi: 10.7507/1672-2531.20100416 Copy

1.	Lavis JN, Oxman AD, Lewin S, et al. SUPPORT Tools for evidenceinformed health Policymaking (STP). Introduction. Health Res Policy Syst, 2009, 7(Suppl 1): I1.
2.	Lavis JN, Posada FB, Haines A, et al. Use of research to inform public policymaking. Lancet, 2004, 364: 1615-1621.
3.	Oxman AD, Lavis JN, Lewin S, et al. SUPPORT Tools for evidenceinformed health Policymaking (STP). 1. What is evidence-informed policymaking. Health Res Policy Syst, 2009, 7(Suppl 1): S1.
4.	Oxman AD, Fretheim A, Lavis JN, et al. SUPPORT Tools for evidence- informed health Policymaking (STP). 12. Finding and using research evidence about resource use and costs. Health Res Policy Syst, 2009, 7(Suppl 1): S12.
5.	Renfrew MJ, Craig D, Dyson L, et al. Breastfeeding promotion for infants in neonatal units: a systematic review and economic analysis. Health Technol Assess, 2009, 13: 1-iv.
6.	Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess, 2004, 8: iii-72.
7.	Carlsen B, Glenton C, Pope C. Thou shalt versus thou shalt not: a meta-synthesis of GPs’ attitudes to clinical practice guidelines. Br J Gen Pract, 2007, 57: 971-978.
8.	Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy, 2005, 10(Suppl 1):6-20.
9.	Munro SA, Lewin SA, Smith HJ, et al. Patient adherence to tuberculosis treatment: a systematic review of qualitative research. PLoS Med, 2007, 4: e238.
10.	Pound P, Britten N, Morgan M, et al. Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med, 2005, 61: 133-155.
11.	Laurant M, Reeves D, Hermens R, et al. Substitution of doctors by nurses in primary care. Cochrane Database Syst Rev, 2005, 2: CD001271.
12.	Forsetlund L, Bjorndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev, 2009, 2: CD003030.
13.	Mulrow CD. Rationale for systematic reviews. BMJ, 1994, 309: 597-599.
14.	Oxman AD, Schunemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence. Health Res Policy Syst, 2006, 4: 20.
15.	Oxman AD, Glasziou P, Williams JW Jr. What should clinicians do when faced with conflicting recommendations? BMJ, 2008, 337: a2530.
16.	Jadad AR, Cook DJ, Jones A, et al. Methodology and reports of systematic reviews and metaanalyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA, 1998, 280: 278-280.
17.	Jorgensen AW, Hilden J, Gotzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ, 2006, 333: 782.
18.	Dixon E, Hameed M, Sutherland F, et al. Evaluating meta-analyses in the general surgical literature: a critical appraisal. Ann Surg, 2005, 241: 450-459.
19.	Jadad AR, Cook DJ, Browman GP. A guide to interpreting discordant systematic reviews. CMAJ, 1997, 156: 1411-1416.
20.	Jadad AR, Moher M, Browman GP, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ, 2000, 320: 537-540.
21.	Linde K, Willich SN. How objective are systematic reviews? Differences between reviews on complementary medicine. J R Soc Med, 2003, 96: 17-22.
22.	Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol, 2007, 7: 10.
23.	Critical Appraisal Skills Programme: 10 questions to help you make sense of reviews 2006 [http://www.phru.nhs.uk/Doc_Links/S.Rev iews%20Appraisal%20Tool.pdf]. United Kingdom, Public Health Resource Unit.
24.	Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol, 1991, 44: 1271-1278.
25.	Canadian Coordinating Office for Health Technology Assessment: Proposed Evaluation Tools for COMPUS 2005 [http://www.cadth. ca/media/compus/pdf/COMPUS_Evaluation_Methodology_final_e. pdf]. Ottawa, Ottawa: Canadian Coordinating Office for Health Technology Assessment.
26.	West S, King V, Carey TS, et al. Systems to rate the strength of scientific evidence [Evidence report/technology assessment no 47] 2002 [http://www.ncbi.nlm.nih.gov/bookshelf/br.fcgi?book=hserta? =A73054 ]. Publication No. 02-E016. Rockville, MD, USA, Agency for Healthcare Research and Quality.
27.	Higgins JPT, Altman DF. Chapter 8: Assessing risk of bias in included studies. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 (updated September 2008) 2008 [http://www. cochrane-handbook.org/]. The Cochrane Collaboration.
28.	Lavis JN, Oxman AD, Souza NM, et al. SUPPORT Tools for evidence- informed health Policymaking (STP). 9. Assessing the applicability of the findings of a systematic review. Health Res Policy Syst, 2009, 7(Suppl 1): S9.
29.	Counsell C. Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews. Ann Intern Med, 1997, 127: 380-387.
30.	Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [updated September 2008] 2008 [http://www.cochrane-handbook.org/]. The Cochrane Collaboration.
31.	Meremikwu MM, Donegan S, Esu E. Chemoprophylaxis and intermittent treatment for preventing malaria in children. Cochrane Database Syst Rev, 2008, 2: CD003756.
32.	Mulrow CD. The medical review article: state of the science. Ann Intern Med, 1987, 106: 485-488.
33.	Oxman AD. Checklists for review articles. BMJ, 1994, 309: 648-651.
34.	Grobler LA, Marais BJ, Mabunda S, et al. Interventions for increasing the proportion of health professionals practicing in underserved communities. Cochrane Database Syst Rev, 2009, 1: CD005314.
35.	Lefebvre C, Manheimer E, Glanville J, on behalf of the Cochrane Information Retrieval Methods Group. Searching for studies. Cochrane Handbook for systematic reviews of interventions. Version 5.0.1 [updated September 2008] 2008 [http://www.cochranehandbook. org/]. The Cochrane Collaboration.
36.	Dickersin K, Min YI. Publication bias: the problem that won’t go away. Ann N Y Acad Sci, 1993, 703: 135-146.
37.	Hopewell S, Loudon K, Clarke MJ, et al. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev, 2009, 21(1): MR000006.
38.	Hopewell S, Clarke M, Stewart L, et al. Time to publication for results of clinical trials. Cochrane Database Syst Rev, 2007, 2: MR000011.
39.	Lavis JN, Wilson M, Oxman AD, et al. SUPPORT Tools for evidence- informed health Policymaking (STP). 4. Using research evidence to clarify a problem. Health Res Policy Syst, 2009, 7(Suppl 1): S4.
40.	Lavis JN, Wilson MG, Oxman AD, et al. SUPPORT Tools for evidence-informed health Policymaking (STP). 5. Using research evidence to frame options to address a problem. Health Res Policy Syst, 2009, 7(Suppl 1): S5.
41.	Katrak P, Bialocerkowski A, Massy-Westropp N, et al. A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology, 2004, 4: 22.
42.	Moher D, Jadad AR, Nichol G, et al. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials, 1995, 16: 62-73.
43.	Shepperd S, Doll H, Broad J, Gladman J, et al. Early discharge hospital at home. Cochrane Database Syst Rev, 2009, 1: CD000356.
44.	Lewin SA, Dick J, Pond P, et al. Lay health workers in primary and community health care. Cochrane Database Syst Rev, 2005, 1: CD004015.
45.	Jamtvedt G, Young JM, Kristoffersen DT, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev, 2006, 2: CD000259.
46.	Arblaster L, Lambert M, Entwistle V, et al. A systematic review of the effectiveness of health service interventions aimed at reducing inequalities in health. J Health Serv Res Policy, 1996, 1: 93-103.
47.	Moher D, Jadad AR, Klassen TP. Guides for reading and interpreting systematic reviews: III. How did the authors synthesize the data and make their conclusions? Arch Pediatr Adolesc Med, 1998, 152(9): 915-20.
48.	Oxman AD, Cook DJ, Guyatt GH. Users’ guides to the medical literature. VI. How to use an overview. Evidence-Based Medicine Working Group. JAMA, 1994, 272: 1367-1371.
49.	Oxman AD, Lavis JN, Lewin S, et al. SUPPORT Tools for evidenceinformed health Policymaking (STP). 10. Taking equity into consideration when assessing the findings of a systematic review. Health Res Policy Syst, 2009, 7(Suppl 1): S10.
50.	Oxman AD, Lavis JN, Fretheim A, et al. SUPPORT Tools for evidence-informed health Policymaking (STP). 17. Dealing with insufficient research evidence. Health Res Policy Syst, 2009, 7(Suppl 1): S17.
51.	Dixon-Woods M, Agarwal S, Jones D, et al. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy, 2005, 10: 45-53.
52.	Noyes J, Popay J, Pearson A, et al. Chapter 20: Qualitative research and Cochrane reviews. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 (updated September 2008) 2008 [http://www.cochrane-handbook.org/]. The Cochrane Collaboration.
53.	Lavis JN. Supporting the Use of Systematic Reviews in Policymaking. PLoS Med, 6(11): e1000141.
54.	Jefferson T, Demicheli V, Vale L. Quality of systematic reviews of economic evaluations in health care. JAMA, 2002, 287: 2809-2812.
55.	Siegfried N, Muller M, Deeks JJ, et al. Male circumcision for prevention of heterosexual acquisition of HIV in men. Cochrane Database Syst Rev, 2009, 2: CD003362.
56.	McGuinness B, Craig D, Bullock R, et al. Statins for the prevention of dementia. Cochrane Database Syst Rev, 2009, 2: CD003160.

1. Lavis JN, Oxman AD, Lewin S, et al. SUPPORT Tools for evidenceinformed health Policymaking (STP). Introduction. Health Res Policy Syst, 2009, 7(Suppl 1): I1.
2. Lavis JN, Posada FB, Haines A, et al. Use of research to inform public policymaking. Lancet, 2004, 364: 1615-1621.
3. Oxman AD, Lavis JN, Lewin S, et al. SUPPORT Tools for evidenceinformed health Policymaking (STP). 1. What is evidence-informed policymaking. Health Res Policy Syst, 2009, 7(Suppl 1): S1.
4. Oxman AD, Fretheim A, Lavis JN, et al. SUPPORT Tools for evidence- informed health Policymaking (STP). 12. Finding and using research evidence about resource use and costs. Health Res Policy Syst, 2009, 7(Suppl 1): S12.
5. Renfrew MJ, Craig D, Dyson L, et al. Breastfeeding promotion for infants in neonatal units: a systematic review and economic analysis. Health Technol Assess, 2009, 13: 1-iv.
6. Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess, 2004, 8: iii-72.
7. Carlsen B, Glenton C, Pope C. Thou shalt versus thou shalt not: a meta-synthesis of GPs’ attitudes to clinical practice guidelines. Br J Gen Pract, 2007, 57: 971-978.
8. Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy, 2005, 10(Suppl 1):6-20.
9. Munro SA, Lewin SA, Smith HJ, et al. Patient adherence to tuberculosis treatment: a systematic review of qualitative research. PLoS Med, 2007, 4: e238.
10. Pound P, Britten N, Morgan M, et al. Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med, 2005, 61: 133-155.
11. Laurant M, Reeves D, Hermens R, et al. Substitution of doctors by nurses in primary care. Cochrane Database Syst Rev, 2005, 2: CD001271.
12. Forsetlund L, Bjorndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev, 2009, 2: CD003030.
13. Mulrow CD. Rationale for systematic reviews. BMJ, 1994, 309: 597-599.
14. Oxman AD, Schunemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 8. Synthesis and presentation of evidence. Health Res Policy Syst, 2006, 4: 20.
15. Oxman AD, Glasziou P, Williams JW Jr. What should clinicians do when faced with conflicting recommendations? BMJ, 2008, 337: a2530.
16. Jadad AR, Cook DJ, Jones A, et al. Methodology and reports of systematic reviews and metaanalyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA, 1998, 280: 278-280.
17. Jorgensen AW, Hilden J, Gotzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ, 2006, 333: 782.
18. Dixon E, Hameed M, Sutherland F, et al. Evaluating meta-analyses in the general surgical literature: a critical appraisal. Ann Surg, 2005, 241: 450-459.
19. Jadad AR, Cook DJ, Browman GP. A guide to interpreting discordant systematic reviews. CMAJ, 1997, 156: 1411-1416.
20. Jadad AR, Moher M, Browman GP, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ, 2000, 320: 537-540.
21. Linde K, Willich SN. How objective are systematic reviews? Differences between reviews on complementary medicine. J R Soc Med, 2003, 96: 17-22.
22. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol, 2007, 7: 10.
23. Critical Appraisal Skills Programme: 10 questions to help you make sense of reviews 2006 [http://www.phru.nhs.uk/Doc_Links/S.Rev iews%20Appraisal%20Tool.pdf]. United Kingdom, Public Health Resource Unit.
24. Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol, 1991, 44: 1271-1278.
25. Canadian Coordinating Office for Health Technology Assessment: Proposed Evaluation Tools for COMPUS 2005 [http://www.cadth. ca/media/compus/pdf/COMPUS_Evaluation_Methodology_final_e. pdf]. Ottawa, Ottawa: Canadian Coordinating Office for Health Technology Assessment.
26. West S, King V, Carey TS, et al. Systems to rate the strength of scientific evidence [Evidence report/technology assessment no 47] 2002 [http://www.ncbi.nlm.nih.gov/bookshelf/br.fcgi?book=hserta? =A73054 ]. Publication No. 02-E016. Rockville, MD, USA, Agency for Healthcare Research and Quality.
27. Higgins JPT, Altman DF. Chapter 8: Assessing risk of bias in included studies. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 (updated September 2008) 2008 [http://www. cochrane-handbook.org/]. The Cochrane Collaboration.
28. Lavis JN, Oxman AD, Souza NM, et al. SUPPORT Tools for evidence- informed health Policymaking (STP). 9. Assessing the applicability of the findings of a systematic review. Health Res Policy Syst, 2009, 7(Suppl 1): S9.
29. Counsell C. Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews. Ann Intern Med, 1997, 127: 380-387.
30. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [updated September 2008] 2008 [http://www.cochrane-handbook.org/]. The Cochrane Collaboration.
31. Meremikwu MM, Donegan S, Esu E. Chemoprophylaxis and intermittent treatment for preventing malaria in children. Cochrane Database Syst Rev, 2008, 2: CD003756.
32. Mulrow CD. The medical review article: state of the science. Ann Intern Med, 1987, 106: 485-488.
33. Oxman AD. Checklists for review articles. BMJ, 1994, 309: 648-651.
34. Grobler LA, Marais BJ, Mabunda S, et al. Interventions for increasing the proportion of health professionals practicing in underserved communities. Cochrane Database Syst Rev, 2009, 1: CD005314.
35. Lefebvre C, Manheimer E, Glanville J, on behalf of the Cochrane Information Retrieval Methods Group. Searching for studies. Cochrane Handbook for systematic reviews of interventions. Version 5.0.1 [updated September 2008] 2008 [http://www.cochranehandbook. org/]. The Cochrane Collaboration.
36. Dickersin K, Min YI. Publication bias: the problem that won’t go away. Ann N Y Acad Sci, 1993, 703: 135-146.
37. Hopewell S, Loudon K, Clarke MJ, et al. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev, 2009, 21(1): MR000006.
38. Hopewell S, Clarke M, Stewart L, et al. Time to publication for results of clinical trials. Cochrane Database Syst Rev, 2007, 2: MR000011.
39. Lavis JN, Wilson M, Oxman AD, et al. SUPPORT Tools for evidence- informed health Policymaking (STP). 4. Using research evidence to clarify a problem. Health Res Policy Syst, 2009, 7(Suppl 1): S4.
40. Lavis JN, Wilson MG, Oxman AD, et al. SUPPORT Tools for evidence-informed health Policymaking (STP). 5. Using research evidence to frame options to address a problem. Health Res Policy Syst, 2009, 7(Suppl 1): S5.
41. Katrak P, Bialocerkowski A, Massy-Westropp N, et al. A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology, 2004, 4: 22.
42. Moher D, Jadad AR, Nichol G, et al. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials, 1995, 16: 62-73.
43. Shepperd S, Doll H, Broad J, Gladman J, et al. Early discharge hospital at home. Cochrane Database Syst Rev, 2009, 1: CD000356.
44. Lewin SA, Dick J, Pond P, et al. Lay health workers in primary and community health care. Cochrane Database Syst Rev, 2005, 1: CD004015.
45. Jamtvedt G, Young JM, Kristoffersen DT, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev, 2006, 2: CD000259.
46. Arblaster L, Lambert M, Entwistle V, et al. A systematic review of the effectiveness of health service interventions aimed at reducing inequalities in health. J Health Serv Res Policy, 1996, 1: 93-103.
47. Moher D, Jadad AR, Klassen TP. Guides for reading and interpreting systematic reviews: III. How did the authors synthesize the data and make their conclusions? Arch Pediatr Adolesc Med, 1998, 152(9): 915-20.
48. Oxman AD, Cook DJ, Guyatt GH. Users’ guides to the medical literature. VI. How to use an overview. Evidence-Based Medicine Working Group. JAMA, 1994, 272: 1367-1371.
49. Oxman AD, Lavis JN, Lewin S, et al. SUPPORT Tools for evidenceinformed health Policymaking (STP). 10. Taking equity into consideration when assessing the findings of a systematic review. Health Res Policy Syst, 2009, 7(Suppl 1): S10.
50. Oxman AD, Lavis JN, Fretheim A, et al. SUPPORT Tools for evidence-informed health Policymaking (STP). 17. Dealing with insufficient research evidence. Health Res Policy Syst, 2009, 7(Suppl 1): S17.
51. Dixon-Woods M, Agarwal S, Jones D, et al. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy, 2005, 10: 45-53.
52. Noyes J, Popay J, Pearson A, et al. Chapter 20: Qualitative research and Cochrane reviews. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 (updated September 2008) 2008 [http://www.cochrane-handbook.org/]. The Cochrane Collaboration.
53. Lavis JN. Supporting the Use of Systematic Reviews in Policymaking. PLoS Med, 6(11): e1000141.
54. Jefferson T, Demicheli V, Vale L. Quality of systematic reviews of economic evaluations in health care. JAMA, 2002, 287: 2809-2812.
55. Siegfried N, Muller M, Deeks JJ, et al. Male circumcision for prevention of heterosexual acquisition of HIV in men. Cochrane Database Syst Rev, 2009, 2: CD003362.
56. McGuinness B, Craig D, Bullock R, et al. Statins for the prevention of dementia. Cochrane Database Syst Rev, 2009, 2: CD003160.

Chinese Journal of Evidence-Based Medicine

SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review*

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content