How to integrate randomized and non-randomized studies of interventions_Chinese Journal of Evidence-Based Medicine

Authors：

LAI Honghao ^1,2 , TIAN Chen ^1,2 , ZHU Hongfei ^1,2 , YANG Qiuyu ³ , SUN Mingyao ³ , TIAN Jinhui ^4,5 ,  GE Long ^1,2,4 ,  YIN Min ⁶

1. Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
2. Department of Social Science and Health Management, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
3. Evidence Based Nursing Centre, School of Nursing, Lanzhou University, Lanzhou 730000, P. R. China;
4. Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou 730000, P. R. China;
5. Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
6. Health Examination Center, The First Hospital of Lanzhou University, Lanzhou 730000, P. R. China;

Corresponding author：

GE Long, Email: ldyyym@126.com; YIN Min, Email: ldyyym@126.com

Keywords：

GRADE method; Quality of evidence; Certainty of evidence; Non-randomized study; Risk of bias

DOI：

10.7507/1672-2531.202201089

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

High-quality randomized controlled trials are the best source of evidence to explain the relationship between health interventions and outcomes. However, in cases where they are insufficient, indirect, or inappropriate, researchers may need to include non-randomized studies of interventions to strengthen the evidence body and improve the certainty (quality) of evidence. The latest research from the GRADE working group provides a way for researchers to integrate randomized and non-randomized evidence. The present paper introduced the relevant methods to provide guidance for systematic reviewers, health technology assessors, and guideline developers.

Citation： LAI Honghao, TIAN Chen, ZHU Hongfei, YANG Qiuyu, SUN Mingyao, TIAN Jinhui, GE Long, YIN Min. How to integrate randomized and non-randomized studies of interventions. Chinese Journal of Evidence-Based Medicine, 2022, 22(7): 836-841. doi: 10.7507/1672-2531.202201089 Copy

1.	Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol, 2011, 64(4): 383-394.
2.	陈耀龙, 李幼平, 杜亮, 等. 医学研究中证据分级和推荐强度的演进. 中国循证医学杂志, 2008, 8(2): 127-133.
3.	Iorio A, Spencer FA, Falavigna M, et al. Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients. BMJ, 2015, 350: h870.
4.	Zhang Y, Alonso-Coello P, Guyatt GH, et al. GRADE guidelines: 19. assessing the certainty of evidence in the importance of outcomes or values and preferences-risk of bias and indirectness. J Clin Epidemiol, 2019, 111: 94-104.
5.	Zhang Y, Coello PA, Guyatt GH, et al. GRADE guidelines: 20. Assessing the certainty of evidence in the importance of outcomes or values and preferences-inconsistency, imprecision, and other domains. J Clin Epidemiol, 2019, 111: 83-93.
6.	Cumpston M, Li T, Page MJ, et al. Updated guidance for trusted systematic reviews: a new edition of the Cochrane handbook for systematic reviews of interventions. Cochrane Database Syst Rev, 2019, (10): ED000142.
7.	Schünemann HJ, Tugwell P, Reeves BC, et al. Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions. Res Synth Methods, 2013, 4(1): 49-62.
8.	Cuello-Garcia CA, Santesso N, Morgan RL, et al. GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol, 2022, 142: 200-208.
9.	张俊华, 孙鑫, 李幼平, 等. 循证中医药学的现在和未来. 中国循证医学杂志, 2019, 19(5): 515-520.
10.	陈薇, 方赛男, 刘建平, 等. 国内循证医学证据分级体系的引入及其在中医药领域面对的挑战. 中国中西医结合杂志, 2017, 37(11): 1285-1288.
11.	Hampson G, Towse A, Dreitlein WB, et al. Real-world evidence for coverage decisions: opportunities and challenges. J Comp Eff Res, 2018, 7(12): 1133-1143.
12.	Deverka PA, Douglas MP, Phillips KA. Use of real-world evidence in US payer coverage decision-making for next-generation sequencing-based tests: challenges, opportunities, and potential solutions. Value Health, 2020, 23(5): 540-550.
13.	Reeves BC, Higgins JP, Ramsay C, et al. An introduction to methodological issues when including non-randomised studies in systematic reviews on the effects of interventions. Res Synth Methods, 2013, 4(1): 1-11.
14.	Koene J, Zyto S, van der Stel J, et al. The relations between executive functions and occupational functioning in individuals with bipolar disorder: a scoping review. Int J Bipolar Disord, 2022, 10(1): 8.
15.	高亚, 葛龙, 李伦, 等. Scoping Review撰写方法简介. 中国药物评价, 2018, 35(5): 321-324.
16.	Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ, 2011, 343: d5928.
17.	Faber T, Ravaud P, Riveros C, et al. Meta-analyses including non-randomized studies of therapeutic interventions: a methodological review. BMC Med Res Methodol, 2016, 16: 35.
18.	Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol, 2007, 36(3): 666-676.
19.	Deeks JJ, Dinnes J, D'Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess, 2003, 7(27): iii-x,1i-173i.
20.	王浩, 唐晓宇, 王和平, 等. ROBINS-I: 评估非随机干预性研究偏倚风险的新工具. 中国循证心血管医学杂志, 2018, 10(7): 789-793.
21.	曾宪涛, 刘慧, 陈曦, 等. Meta分析系列之四: 观察性研究的质量评价工具. 中国循证心血管医学杂志, 2012, 4(4): 297-299.
22.	Cuello-Garcia CA, Morgan RL, Brozek J, et al. A scoping review and survey provides the rationale, perceptions, and preferences for the integration of randomized and nonrandomized studies in evidence syntheses and GRADE assessments. J Clin Epidemiol, 2018, 98: 33-40.
23.	Borenstein M, Hedges LV, Higgins JP, et al. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods, 2010, 1(2): 97-111.
24.	Reeves BC, Wells GA, Waddington H. Quasi-experimental study designs series-paper 5: a checklist for classifying studies evaluating the effects on health interventions-a taxonomy without labels. J Clin Epidemiol, 2017, 89: 30-42.
25.	Mueller M, D'Addario M, Egger M, et al. Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations. BMC Med Res Methodol, 2018, 18(1): 44.
26.	Schwingshackl L, Balduzzi S, Beyerbach J, et al. Evaluating agreement between bodies of evidence from randomised controlled trials and cohort studies in nutrition research: meta-epidemiological study. BMJ, 2021, 374: n1864.
27.	Beyerbach J, Stadelmaier J, Hoffmann G, et al. Evaluating concordance of bodies of evidence from randomized controlled trials, dietary intake, and biomarkers of intake in cohort studies: a meta-epidemiological study. Adv Nutr, 2022, 13(1): 48-65.
28.	Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev, 2014, (4): MR000034.
29.	Rockers PC, Røttingen JA, Shemilt I, et al. Inclusion of quasi-experimental studies in systematic reviews of health systems research. Health Policy, 2015, 119(4): 511-521.

1. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol, 2011, 64(4): 383-394.
2. 陈耀龙, 李幼平, 杜亮, 等. 医学研究中证据分级和推荐强度的演进. 中国循证医学杂志, 2008, 8(2): 127-133.
3. Iorio A, Spencer FA, Falavigna M, et al. Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients. BMJ, 2015, 350: h870.
4. Zhang Y, Alonso-Coello P, Guyatt GH, et al. GRADE guidelines: 19. assessing the certainty of evidence in the importance of outcomes or values and preferences-risk of bias and indirectness. J Clin Epidemiol, 2019, 111: 94-104.
5. Zhang Y, Coello PA, Guyatt GH, et al. GRADE guidelines: 20. Assessing the certainty of evidence in the importance of outcomes or values and preferences-inconsistency, imprecision, and other domains. J Clin Epidemiol, 2019, 111: 83-93.
6. Cumpston M, Li T, Page MJ, et al. Updated guidance for trusted systematic reviews: a new edition of the Cochrane handbook for systematic reviews of interventions. Cochrane Database Syst Rev, 2019, (10): ED000142.
7. Schünemann HJ, Tugwell P, Reeves BC, et al. Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions. Res Synth Methods, 2013, 4(1): 49-62.
8. Cuello-Garcia CA, Santesso N, Morgan RL, et al. GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol, 2022, 142: 200-208.
9. 张俊华, 孙鑫, 李幼平, 等. 循证中医药学的现在和未来. 中国循证医学杂志, 2019, 19(5): 515-520.
10. 陈薇, 方赛男, 刘建平, 等. 国内循证医学证据分级体系的引入及其在中医药领域面对的挑战. 中国中西医结合杂志, 2017, 37(11): 1285-1288.
11. Hampson G, Towse A, Dreitlein WB, et al. Real-world evidence for coverage decisions: opportunities and challenges. J Comp Eff Res, 2018, 7(12): 1133-1143.
12. Deverka PA, Douglas MP, Phillips KA. Use of real-world evidence in US payer coverage decision-making for next-generation sequencing-based tests: challenges, opportunities, and potential solutions. Value Health, 2020, 23(5): 540-550.
13. Reeves BC, Higgins JP, Ramsay C, et al. An introduction to methodological issues when including non-randomised studies in systematic reviews on the effects of interventions. Res Synth Methods, 2013, 4(1): 1-11.
14. Koene J, Zyto S, van der Stel J, et al. The relations between executive functions and occupational functioning in individuals with bipolar disorder: a scoping review. Int J Bipolar Disord, 2022, 10(1): 8.
15. 高亚, 葛龙, 李伦, 等. Scoping Review撰写方法简介. 中国药物评价, 2018, 35(5): 321-324.
16. Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ, 2011, 343: d5928.
17. Faber T, Ravaud P, Riveros C, et al. Meta-analyses including non-randomized studies of therapeutic interventions: a methodological review. BMC Med Res Methodol, 2016, 16: 35.
18. Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol, 2007, 36(3): 666-676.
19. Deeks JJ, Dinnes J, D'Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess, 2003, 7(27): iii-x,1i-173i.
20. 王浩, 唐晓宇, 王和平, 等. ROBINS-I: 评估非随机干预性研究偏倚风险的新工具. 中国循证心血管医学杂志, 2018, 10(7): 789-793.
21. 曾宪涛, 刘慧, 陈曦, 等. Meta分析系列之四: 观察性研究的质量评价工具. 中国循证心血管医学杂志, 2012, 4(4): 297-299.
22. Cuello-Garcia CA, Morgan RL, Brozek J, et al. A scoping review and survey provides the rationale, perceptions, and preferences for the integration of randomized and nonrandomized studies in evidence syntheses and GRADE assessments. J Clin Epidemiol, 2018, 98: 33-40.
23. Borenstein M, Hedges LV, Higgins JP, et al. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods, 2010, 1(2): 97-111.
24. Reeves BC, Wells GA, Waddington H. Quasi-experimental study designs series-paper 5: a checklist for classifying studies evaluating the effects on health interventions-a taxonomy without labels. J Clin Epidemiol, 2017, 89: 30-42.
25. Mueller M, D'Addario M, Egger M, et al. Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations. BMC Med Res Methodol, 2018, 18(1): 44.
26. Schwingshackl L, Balduzzi S, Beyerbach J, et al. Evaluating agreement between bodies of evidence from randomised controlled trials and cohort studies in nutrition research: meta-epidemiological study. BMJ, 2021, 374: n1864.
27. Beyerbach J, Stadelmaier J, Hoffmann G, et al. Evaluating concordance of bodies of evidence from randomized controlled trials, dietary intake, and biomarkers of intake in cohort studies: a meta-epidemiological study. Adv Nutr, 2022, 13(1): 48-65.
28. Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev, 2014, (4): MR000034.
29. Rockers PC, Røttingen JA, Shemilt I, et al. Inclusion of quasi-experimental studies in systematic reviews of health systems research. Health Policy, 2015, 119(4): 511-521.

Chinese Journal of Evidence-Based Medicine

How to integrate randomized and non-randomized studies of interventions

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content