Evidence certainty grading of network meta-analysis: method update and case application_Chinese Journal of Evidence-Based Medicine

Authors：

HUANG Jiajie ¹ , LAI Honghao ^2,3 , LIU Jianing ¹ , ZHAO Weilong ^2,3 , SUN Mingyao ⁴ , YE Ziying ^2,3 , LI Ying ^2,3 , PAN Bei ^5,6 , TIAN Jinhui ^5,6 ,  LI Mengting ⁷ ,  GE Long ^2,3,5

1. College of Nursing, Gansu University of Chinese Medicine, Lanzhou 730000, P. R. China;
2. Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
3. Center of Health Management and Health Development, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
4. Evidence Based Nursing Centre, School of Nursing, Lanzhou University, Lanzhou 730000, P. R. China;
5. Key Laboratory of Evidence Based Medicine of Gansu Province, Lanzhou 730000, P. R. China;
6. Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
7. College of Social Science and Health Management, Gansu University of Chinese Medicine, Lanzhou 730000, P. R. China;

Corresponding author：

LI Mengting, Email: limengting1997@163.com; GE Long, Email: gelong2009@163.com

Keywords：

Network meta-analysis; GRADE; Credibility of evidence

DOI：

10.7507/1672-2531.202310039

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Network meta-analysis (NMA) is a method that can compare and rank the effects of different interventions, which plays an important role in evidence translation and evidence-based decision-making. In 2014, the GRADE working group first introduced the GRADE method for NMA evidence certainty grading. Since then, its method system has been gradually supplemented and improved. In recent years, the GRADE working group has further improved the methods for evaluating intransitivity and imprecision in NMA, and has made recommendations for the presentation and interpretation of NMA results, forming a complete methodological chain of NMA evidence certainty grading and result interpretation consisting of 6 steps. Our team updated the method system of GRADE applied in NMA with specific cases to provide references for relevant researchers.

Citation： HUANG Jiajie, LAI Honghao, LIU Jianing, ZHAO Weilong, SUN Mingyao, YE Ziying, LI Ying, PAN Bei, TIAN Jinhui, LI Mengting, GE Long. Evidence certainty grading of network meta-analysis: method update and case application. Chinese Journal of Evidence-Based Medicine, 2024, 24(10): 1231-1240. doi: 10.7507/1672-2531.202310039 Copy

1.	Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med, 2004, 23(20): 3105-3124.
2.	田金徽, 李伦, 赵晔, 等. 网状Meta分析现状. 中国药物评价, 2014, 31(3): 129-133.
3.	Lee A. The development of network meta-analysis. J R Soc Med, 2022, 115(8): 313-321.
4.	Efthimiou O, Debray TP, van Valkenhoef G, et al. GetReal in network meta-analysis: a review of the methodology. Res Synth Methods, 2016, 7(3): 236-263.
5.	Puhan MA, Schünemann HJ, Murad MH, et al. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 2014, 349: g5630.
6.	Brignardello-Petersen R, Bonner A, Alexander PE, et al. Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis. J Clin Epidemiol, 2018, 93: 36-44.
7.	Brignardello-Petersen R, Mustafa RA, Siemieniuk RAC, et al. GRADE approach to rate the certainty from a network meta-analysis: addressing incoherence. J Clin Epidemiol, 2019, 108: 77-85.
8.	Brignardello-Petersen R, Florez ID, Izcovich A, et al. GRADE approach to drawing conclusions from a network meta-analysis using a minimally contextualised framework. BMJ, 2020, 371: m3900.
9.	Brignardello-Petersen R, Izcovich A, Rochwerg B, et al. GRADE approach to drawing conclusions from a network meta-analysis using a partially contextualised framework. BMJ, 2020, 371: m3907.
10.	Brignardello-Petersen R, Tomlinson G, Florez I, et al. Grading of recommendations assessment, development, and evaluation concept article 5: addressing intransitivity in a network meta-analysis. J Clin Epidemiol, 2023, 160: 151-159.
11.	Brignardello-Petersen R, Murad MH, Walter SD, et al. GRADE approach to rate the certainty from a network meta-analysis: avoiding spurious judgments of imprecision in sparse networks. J Clin Epidemiol, 2019, 105: 60-67.
12.	Wang Q, Xiao Y, Guo T, et al. The use of GRADE approach in Cochrane reviews of TCM was insufficient: a cross-sectional survey. J Clin Epidemiol, 2022, 142: 1-9.
13.	Xun Y, Zhou Q, Yang N, et al. Clinical epidemiology in China series. Paper 2: promoting GRADE at the national level: The experience from China. J Clin Epidemiol, 2021, 140: 172-177.
14.	Gao Y, Ge L, Ma X, et al. Improvement needed in the network geometry and inconsistency of Cochrane network meta-analyses: a cross-sectional survey. J Clin Epidemiol, 2019, 113: 214-227.
15.	Brignardello-Petersen R, Guyatt GH, Mustafa RA, et al. GRADE guidelines 33: addressing imprecision in a network meta-analysis. J Clin Epidemiol, 2021, 139: 49-56.
16.	Izcovich A, Chu DK, Mustafa RA, et al. A guide and pragmatic considerations for applying GRADE to network meta-analysis. BMJ, 2023, 381: e074495.
17.	王琪, 王建成, 潘蓓, 等. 网状Meta分析证据质量分级: GRADE方法学前沿与进展. 中国循证医学杂志, 2020, 20(8): 979-985.
18.	栗梦婷, 朱鸿飞, 后亮瑛, 等. 网状Meta分析结论形成的方法: 最小背景化框架. 中国循证医学杂志, 2021, 21(9): 1102-1109.
19.	朱鸿飞, 栗梦婷, 后亮瑛, 等. 网状Meta分析结论形成的方法: 部分背景化框架. 中国循证医学杂志, 2021, 21(9): 1110-1116.
20.	王琪, 王韵华, 赖鸿皓, 等. 网状Meta分析证据质量分级: CINeMA在线应用程序简介. 中国循证医学杂志, 2020, 20(9): 1111-1116.
21.	Pan B, Ge L, Lai H, et al. The comparative effectiveness and safety of insomnia drugs: a systematic review and network meta-analysis of 153 randomized trials. Drugs, 2023, 83(7): 587-619.
22.	李伦, 田金徽, 姚亮, 等. 网状Meta分析的统计学基础、假设和证据质量评估. 循证医学, 2015, 15(3): 180-183.
23.	刘凤琪, 石舒原, 杨智荣, 等. 网络Meta分析研究进展系列(七): 多水平网状Meta回归. 中国循证心血管医学杂志, 2020, 12(12): 1409-1412.
24.	Schandelmaier S, Chang Y, Devasenapathy N, et al. A systematic survey identified 36 criteria for assessing effect modification claims in randomized trials or meta-analyses. J Clin Epidemiol, 2019, 113: 159-167.
25.	Schandelmaier S, Briel M, Varadhan R, et al. Development of the instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ, 2020, 192(32): E901-E906.
26.	李戈, 杨智荣, 王巍巍, 等. 《GRADE在网状Meta分析不精确性的处理》文献解读. 中国循证儿科杂志, 2021, 16(6): 487-489.
27.	Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines 6. Rating the quality of evidence--imprecision. J Clin Epidemiol, 2011, 64(12): 1283-1293.
28.	Schünemann HJ. Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision. J Clin Epidemiol, 2016, 75: 6-15.
29.	GRADE Working Group. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 2015, 350: h3326.
30.	Papakonstantinou T, Nikolakopoulou A, Rücker G, et al. Estimating the contribution of studies in network meta-analysis: paths, flows and streams. F1000Res, 2018, 7: 610.
31.	Brignardello-Petersen R, Guyatt G H. β-Blockers in heart failure--are all created equal? Pol Arch Med Wewn, 2013, 123(5): 204-205.
32.	Schünemann HJ, Wiercioch W, Brozek J, et al. GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT. J Clin Epidemiol, 2017, 81: 101-110.
33.	黄嘉杰, 赖鸿皓, 孙铭谣, 等. 实施性研究综合框架(CFIR)更新解读. 中国全科医学, 2023, 26(31): 3863-3871, 3876.
34.	赖鸿皓, 孙铭谣, 黄嘉杰, 等. 中西医结合临床实践指南实施框架构建的思考. 协和医学杂志, 2023, 14(1): 31-38.
35.	葛龙, 秦钰, 宋忠阳, 等. 新型冠状病毒感染恢复期常见症中西医结合管理循证实践指南. 兰州大学学报(医学版), 2023, 49(5): 28-40,46.
36.	田晨, 孙铭谣, 李颖, 等. 快速动态循证要览制订方法与流程. 中国循证医学杂志, 2023, 23(4): 450-456.
37.	孙铭谣, 田晨, 李颖, 等. 药物治疗新型冠状病毒感染快速动态循证要览. 中国循证医学杂志, 2023, 23(7): 745-754.
38.	田晨, 杨秋玉, 孙铭谣, 等. 快速动态循证指南制订方法. 协和医学杂志, 2024, 15(1): 183-191.
39.	孙铭谣, 曾茵茹, 赖鸿皓, 等. 动态指南发展现状与挑战. 中国循证医学杂志, 2023, 23(8): 970-977.
40.	Institute of Social and Preventive Medicine. CINeMA: confidence in network meta-analysis. 2017.
41.	田晨, 晏毅龙, 王勇, 等. 证据智能合成与分级: AutoMeta平台开发与验证. 中国循证医学杂志, 2024, 24(4): 459-465.
42.	赖鸿皓, 王浙, 李滢, 等. 中医药多元证据整合方法学MERGE框架. 协和医学杂志, 2024, 15(1): 172-182.
43.	陈昊, 王艳, 陈耀龙, 等. 从GRADE系统方法学看中医药临床证据体系的构建. 中华中医药杂志, 2017, 32(1): 25-28.
44.	张薇, 李小娟, 邓宏勇. 中医临床证据分级和推荐体系发展现状. 中国中医药信息杂志, 2020, 27(5): 133-136.
45.	赖鸿皓, 田晨, 朱鸿飞, 等. 如何整合随机与非随机干预研究. 中国循证医学杂志, 2022, 22(7): 836-841.
46.	Cuello-Garcia CA, Santesso N, Morgan RL, et al. GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol, 2022, 142: 200-208.

1. Lu G, Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med, 2004, 23(20): 3105-3124.
2. 田金徽, 李伦, 赵晔, 等. 网状Meta分析现状. 中国药物评价, 2014, 31(3): 129-133.
3. Lee A. The development of network meta-analysis. J R Soc Med, 2022, 115(8): 313-321.
4. Efthimiou O, Debray TP, van Valkenhoef G, et al. GetReal in network meta-analysis: a review of the methodology. Res Synth Methods, 2016, 7(3): 236-263.
5. Puhan MA, Schünemann HJ, Murad MH, et al. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 2014, 349: g5630.
6. Brignardello-Petersen R, Bonner A, Alexander PE, et al. Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis. J Clin Epidemiol, 2018, 93: 36-44.
7. Brignardello-Petersen R, Mustafa RA, Siemieniuk RAC, et al. GRADE approach to rate the certainty from a network meta-analysis: addressing incoherence. J Clin Epidemiol, 2019, 108: 77-85.
8. Brignardello-Petersen R, Florez ID, Izcovich A, et al. GRADE approach to drawing conclusions from a network meta-analysis using a minimally contextualised framework. BMJ, 2020, 371: m3900.
9. Brignardello-Petersen R, Izcovich A, Rochwerg B, et al. GRADE approach to drawing conclusions from a network meta-analysis using a partially contextualised framework. BMJ, 2020, 371: m3907.
10. Brignardello-Petersen R, Tomlinson G, Florez I, et al. Grading of recommendations assessment, development, and evaluation concept article 5: addressing intransitivity in a network meta-analysis. J Clin Epidemiol, 2023, 160: 151-159.
11. Brignardello-Petersen R, Murad MH, Walter SD, et al. GRADE approach to rate the certainty from a network meta-analysis: avoiding spurious judgments of imprecision in sparse networks. J Clin Epidemiol, 2019, 105: 60-67.
12. Wang Q, Xiao Y, Guo T, et al. The use of GRADE approach in Cochrane reviews of TCM was insufficient: a cross-sectional survey. J Clin Epidemiol, 2022, 142: 1-9.
13. Xun Y, Zhou Q, Yang N, et al. Clinical epidemiology in China series. Paper 2: promoting GRADE at the national level: The experience from China. J Clin Epidemiol, 2021, 140: 172-177.
14. Gao Y, Ge L, Ma X, et al. Improvement needed in the network geometry and inconsistency of Cochrane network meta-analyses: a cross-sectional survey. J Clin Epidemiol, 2019, 113: 214-227.
15. Brignardello-Petersen R, Guyatt GH, Mustafa RA, et al. GRADE guidelines 33: addressing imprecision in a network meta-analysis. J Clin Epidemiol, 2021, 139: 49-56.
16. Izcovich A, Chu DK, Mustafa RA, et al. A guide and pragmatic considerations for applying GRADE to network meta-analysis. BMJ, 2023, 381: e074495.
17. 王琪, 王建成, 潘蓓, 等. 网状Meta分析证据质量分级: GRADE方法学前沿与进展. 中国循证医学杂志, 2020, 20(8): 979-985.
18. 栗梦婷, 朱鸿飞, 后亮瑛, 等. 网状Meta分析结论形成的方法: 最小背景化框架. 中国循证医学杂志, 2021, 21(9): 1102-1109.
19. 朱鸿飞, 栗梦婷, 后亮瑛, 等. 网状Meta分析结论形成的方法: 部分背景化框架. 中国循证医学杂志, 2021, 21(9): 1110-1116.
20. 王琪, 王韵华, 赖鸿皓, 等. 网状Meta分析证据质量分级: CINeMA在线应用程序简介. 中国循证医学杂志, 2020, 20(9): 1111-1116.
21. Pan B, Ge L, Lai H, et al. The comparative effectiveness and safety of insomnia drugs: a systematic review and network meta-analysis of 153 randomized trials. Drugs, 2023, 83(7): 587-619.
22. 李伦, 田金徽, 姚亮, 等. 网状Meta分析的统计学基础、假设和证据质量评估. 循证医学, 2015, 15(3): 180-183.
23. 刘凤琪, 石舒原, 杨智荣, 等. 网络Meta分析研究进展系列(七): 多水平网状Meta回归. 中国循证心血管医学杂志, 2020, 12(12): 1409-1412.
24. Schandelmaier S, Chang Y, Devasenapathy N, et al. A systematic survey identified 36 criteria for assessing effect modification claims in randomized trials or meta-analyses. J Clin Epidemiol, 2019, 113: 159-167.
25. Schandelmaier S, Briel M, Varadhan R, et al. Development of the instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ, 2020, 192(32): E901-E906.
26. 李戈, 杨智荣, 王巍巍, 等. 《GRADE在网状Meta分析不精确性的处理》文献解读. 中国循证儿科杂志, 2021, 16(6): 487-489.
27. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines 6. Rating the quality of evidence--imprecision. J Clin Epidemiol, 2011, 64(12): 1283-1293.
28. Schünemann HJ. Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision. J Clin Epidemiol, 2016, 75: 6-15.
29. GRADE Working Group. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ, 2015, 350: h3326.
30. Papakonstantinou T, Nikolakopoulou A, Rücker G, et al. Estimating the contribution of studies in network meta-analysis: paths, flows and streams. F1000Res, 2018, 7: 610.
31. Brignardello-Petersen R, Guyatt G H. β-Blockers in heart failure--are all created equal? Pol Arch Med Wewn, 2013, 123(5): 204-205.
32. Schünemann HJ, Wiercioch W, Brozek J, et al. GRADE Evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT. J Clin Epidemiol, 2017, 81: 101-110.
33. 黄嘉杰, 赖鸿皓, 孙铭谣, 等. 实施性研究综合框架(CFIR)更新解读. 中国全科医学, 2023, 26(31): 3863-3871, 3876.
34. 赖鸿皓, 孙铭谣, 黄嘉杰, 等. 中西医结合临床实践指南实施框架构建的思考. 协和医学杂志, 2023, 14(1): 31-38.
35. 葛龙, 秦钰, 宋忠阳, 等. 新型冠状病毒感染恢复期常见症中西医结合管理循证实践指南. 兰州大学学报(医学版), 2023, 49(5): 28-40,46.
36. 田晨, 孙铭谣, 李颖, 等. 快速动态循证要览制订方法与流程. 中国循证医学杂志, 2023, 23(4): 450-456.
37. 孙铭谣, 田晨, 李颖, 等. 药物治疗新型冠状病毒感染快速动态循证要览. 中国循证医学杂志, 2023, 23(7): 745-754.
38. 田晨, 杨秋玉, 孙铭谣, 等. 快速动态循证指南制订方法. 协和医学杂志, 2024, 15(1): 183-191.
39. 孙铭谣, 曾茵茹, 赖鸿皓, 等. 动态指南发展现状与挑战. 中国循证医学杂志, 2023, 23(8): 970-977.
40. Institute of Social and Preventive Medicine. CINeMA: confidence in network meta-analysis. 2017.
41. 田晨, 晏毅龙, 王勇, 等. 证据智能合成与分级: AutoMeta平台开发与验证. 中国循证医学杂志, 2024, 24(4): 459-465.
42. 赖鸿皓, 王浙, 李滢, 等. 中医药多元证据整合方法学MERGE框架. 协和医学杂志, 2024, 15(1): 172-182.
43. 陈昊, 王艳, 陈耀龙, 等. 从GRADE系统方法学看中医药临床证据体系的构建. 中华中医药杂志, 2017, 32(1): 25-28.
44. 张薇, 李小娟, 邓宏勇. 中医临床证据分级和推荐体系发展现状. 中国中医药信息杂志, 2020, 27(5): 133-136.
45. 赖鸿皓, 田晨, 朱鸿飞, 等. 如何整合随机与非随机干预研究. 中国循证医学杂志, 2022, 22(7): 836-841.
46. Cuello-Garcia CA, Santesso N, Morgan RL, et al. GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol, 2022, 142: 200-208.

Previous Article
Principles of latent variable mixture modeling and its value in clinical research applications

Chinese Journal of Evidence-Based Medicine

Evidence certainty grading of network meta-analysis: method update and case application

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Format

Content