A guideline for determining the effect size decision threshold of health outcomes_Chinese Journal of Evidence-Based Medicine

Authors：

LIU Yafei ^1,2 , ZHU Hongfei ^3,4,5 , LI Ying ^1,2 , LAI Honghao ^1,2 , ZHOU Qi ⁶ , LUO Xufei ⁶ , CHEN Yaolong ^6,7,8,9,10 , DENG Hongyong ¹¹ , DU Liang ¹² , JIN Yinghui ¹³ , KONG Dezhao ¹⁴ , LI Guowei ¹⁵ , LI Jiang ¹⁶ , LI Ling ¹² , LIAO Xing ¹⁷ , LIU Jie ¹⁸ , SHI Nannan ^17,19 , SHANG Hongcai ²⁰ , TIAN Jinhui ^6,7,8,9,10 , WANG Xiaohui ¹ , WU Darong ²¹ , YANG Fengwen ²² , YANG Kehu ^6,7,8,9,10 , YU Changhe ²⁰ , ZHANG Jiaxing ²³ , ZHAO Chen ¹⁷ ,  GE Long ^1,2,7,8,9,10

1. Department of Social Science and Health Management, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
2. Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
3. School of Public Health, Fudan University, Shanghai 200032, P. R. China;
4. Key Laboratory of Public Health Safety, Fudan University, Shanghai 200032, P. R. China;
5. National Health Commission Key Laboratory of Health Technology Assessment, Fudan University, Shanghai 200032, P. R. China;
6. Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
7. Institute of Health Data Science, Lanzhou University, Lanzhou 730000, P. R. China;
8. Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou 730000, P. R. China;
9. WHO Collaborating Center for Guideline Implementation and Knowledge Translation, Lanzhou 730000, P. R. China;
10. Chinese GRADE Center, Lanzhou 73000, P. R. China;
11. TCM Health Collaborative Innovation Center, Shanghai University of Traditional Chinese Medicine, Shanghai 201203, P. R. China;
12. Chinese Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, P. R. China;
13. Center for Evidence-Based and Translational Medicine, Zhongnan Hospital of Wuhan University, Wuhan 430071, P. R. China;
14. Affiliated Hospital of Liaoning University of Traditional Chinese Medicine, Shenyang 110032, P. R. China;
15. Guangdong Second Provincial General Hospital Affiliated to Jinan University, Guangzhou 510317, P. R. China;
16. Cancer Hospital Chinese Academy of Medical Sciences, Beijing 100021, P. R. China;
17. Institute of Basic Research in Clinical, China Academy of Chinese Medical Sciences, Beijing 100700, P. R. China;
18. Guang'anmen Hospital, China Academy of Chinese Medical Sciences, Beijing 100700, P. R. China;
19. China Center for Evidence-Based Traditional Chinese Medicine, Beijing 100700, P. R. China;
20. Dongzhimen Hospital, Beijing University of Traditional Chinese Medicine, Beijing 100700, P. R. China;
21. The Second Affiliated Hospital of Guangzhou University of Traditional Chinese Medicine (Guangdong Hospital of Traditional Chinese Medicine), Guangzhou 510120, P. R. China;
22. Evidence-Based Medicine Center, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, P. R. China;
23. Department of Pharmacy, Guizhou Provincial People’s Hospital, Guiyang 550002, P. R. China;

Corresponding author：

GE Long, Email: gelong2009@163.com

Keywords：

Decision threshold of effect size; Core outcomes; MERGE; Methodology

DOI：

10.7507/1672-2531.202308133

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In the clinical evidence-based decision-making process, factors including benefits and harms, evidence certainty, cost and feasibility of interventions should be considered based on the best evidence. The development of decision threshold of effect size (DTES) for patient health outcomes can help stakeholders understand the benefits and harms of interventions, assess evidence certainty, and interpret research results. Based on international experience, the MERGE Working Group through group discussions, semi-structured interviews, expert consensus, and pilot application developed a set of 8-step guidelines for the development of DTES for health outcomes. These steps included necessity assessment, formation of working groups, selection of patient important outcomes, creation of scenarios based on the best evidence, design of expert consultation questionnaires, analysis of expert consultation results, face-to-face expert consensus, dissemination of the application and reevaluation. The DTES guideline development is intended to provide methodological guidance for stakeholders to develop DTES for health outcome indicators in different domains.

Citation： LIU Yafei, ZHU Hongfei, LI Ying, LAI Honghao, ZHOU Qi, LUO Xufei, CHEN Yaolong, DENG Hongyong, DU Liang, JIN Yinghui, KONG Dezhao, LI Guowei, LI Jiang, LI Ling, LIAO Xing, LIU Jie, SHI Nannan, SHANG Hongcai, TIAN Jinhui, WANG Xiaohui, WU Darong, YANG Fengwen, YANG Kehu, YU Changhe, ZHANG Jiaxing, ZHAO Chen, GE Long. A guideline for determining the effect size decision threshold of health outcomes. Chinese Journal of Evidence-Based Medicine, 2024, 24(4): 405-413. doi: 10.7507/1672-2531.202308133 Copy

1.	Alonso-Coello P, Schünemann HJ, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ, 2016, 353: i2016.
2.	国家卫生健康委员会. 药品临床综合评价管理指南. 2021.
3.	Kantorová L, Friessová T, Slezáková S, et al. Addressing challenges when applying GRADE to public health guidelines: a scoping review protocol and pilot analysis. Int J Environ Res Public Health, 2022, 19(2): 992.
4.	孙铭谣, 田晨, 李颖, 等. 药物治疗新型冠状病毒感染快速动态循证要览. 中国循证医学杂志, 2023, 23(7): 745-754.
5.	孙晓娥. 扎根理论在深度访谈研究中的实例探析. 西安交通大学学报(社会科学版), 2011, 31: 87-92.
6.	Neumann I, Quiñelen E, Nahuelhual P, et al. Using Explicit Thresholds were valuable for judging benefits and harms in partially contextualized GRADE Guidelines. J Clin Epidemiol, 2022, 147: 69-75.
7.	Helsingen LM, Zeng L, Siemieniuk RA, et al. Establishing thresholds for important benefits considering the harms of screening interventions. BMJ Open, 2020, 10(12): e037854.
8.	Morgano GP, Mbuagbaw L, Santesso N, et al. Defining decision thresholds for judgments on health benefits and harms using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks: a protocol for a randomised methodological study (GRADE-THRESHOLD). BMJ Open, 2022, 12(3): e053246.
9.	Hultcrantz M, Rind D, Akl EA, et al. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol, 2017, 87: 4-13.
10.	Zeng L, Brignardello-Petersen R, Guyatt G. When applying GRADE, how do we decide the target of certainty of evidence rating. Evid Based Ment Health, 2021, 24(3): 121-123.
11.	Zeng L, Brignardello-Petersen R, Hultcrantz M, et al. GRADE guidelines 32: GRADE offers guidance on choosing targets of GRADE certainty of evidence ratings. J Clin Epidemiol, 2021, 137: 163-175.
12.	Schünemann HJ, Neumann I, Hultcrantz M, et al. GRADE guidance 35: update on rating imprecision for assessing contextualized certainty of evidence and making decisions. J Clin Epidemiol, 2022, 150: 225-242.
13.	Guyatt GH, Osoba D, Wu AW, et al. Methods to explain the clinical significance of health status measures. Mayo Clin Proc, 2002, 77(4): 371-383.
14.	Qaseem A, Forland F, Macbeth F, et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med, 2012, 156(7): 525-531.
15.	ECBC. Evidence-based methodologies for public health. 2011.
16.	WHO. WHO handbook for guideline development. 2012.
17.	Schünemann HJ, Wiercioch W, Etxeandia I, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ, 2014, 186(3): E123-E142.
18.	Williamson PR, Altman DG, Bagley H, et al. The COMET handbook: version 1. 0. Trials, 2017, 18(Suppl 3): 280.
19.	Kirkham JJ, Davis K, Altman DG, et al. Core outcome set-standards for development: the COS-STAD recommendations. PLoS Med, 2017, 14(11): e1002447.
20.	WHO. WHO handbook of guideline development (2nd edition). 2014.
21.	丁泓帆, 杨楠, 邓围, 等. WHO指南制定的基本原则和方法. 中国循证医学杂志, 2016, 16(4): 471-477.
22.	Schünemann H, Brożek J, Guyatt G, et al. GRADE handbook for selecting and rating the importance of outcomes (Updated October 2013). 2013.
23.	Schünemann H, Brożek J, Guyatt G, et al. GRADE handbook for summarizing the evidence (Updated October 2013). 2013.
24.	Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 2017, 358: j4008.
25.	陶欢, 杨乐天, 平安, 等. 随机或非随机防治性研究系统评价的质量评价工具AMSTAR 2解读. 中国循证医学杂志, 2018, 18(1): 101-108.
26.	Higgins JPT, Thomas J, Chandler J, et al. Cochrane handbook for systematic reviews of interventions version 6.3 (updated February 2022). 2022.
27.	Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol, 2011, 64(4): 383-394.
28.	Siemieniuk RA, Bartoszko JJ, Zeraatkar D, et al. Drug treatments for COVID-19: living systematic review and network meta-analysis. BMJ, 2020, 370: m2980.
29.	Guyatt GH, Oxman AD, Kunz R, et al. Going from evidence to recommendations. BMJ, 2008, 336(7652): 1049-1051.
30.	Puhan MA, Singh S, Weiss CO, et al. A framework for organizing and selecting quantitative approaches for benefit-harm assessment. BMC Med Res Methodol, 2012, 12: 173.
31.	Yu T, Fain K, Boyd CM, et al. Benefits and harms of roflumilast in moderate to severe COPD. Thorax, 2014, 69(7): 616-622.
32.	Santesso N, Schünemann H, Blumenthal P, et al. World Health Organization guidelines: use of cryotherapy for cervical intraepithelial neoplasia. Int J Gynaecol Obstet, 2012, 118(2): 97-102.

1. Alonso-Coello P, Schünemann HJ, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ, 2016, 353: i2016.
2. 国家卫生健康委员会. 药品临床综合评价管理指南. 2021.
3. Kantorová L, Friessová T, Slezáková S, et al. Addressing challenges when applying GRADE to public health guidelines: a scoping review protocol and pilot analysis. Int J Environ Res Public Health, 2022, 19(2): 992.
4. 孙铭谣, 田晨, 李颖, 等. 药物治疗新型冠状病毒感染快速动态循证要览. 中国循证医学杂志, 2023, 23(7): 745-754.
5. 孙晓娥. 扎根理论在深度访谈研究中的实例探析. 西安交通大学学报(社会科学版), 2011, 31: 87-92.
6. Neumann I, Quiñelen E, Nahuelhual P, et al. Using Explicit Thresholds were valuable for judging benefits and harms in partially contextualized GRADE Guidelines. J Clin Epidemiol, 2022, 147: 69-75.
7. Helsingen LM, Zeng L, Siemieniuk RA, et al. Establishing thresholds for important benefits considering the harms of screening interventions. BMJ Open, 2020, 10(12): e037854.
8. Morgano GP, Mbuagbaw L, Santesso N, et al. Defining decision thresholds for judgments on health benefits and harms using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks: a protocol for a randomised methodological study (GRADE-THRESHOLD). BMJ Open, 2022, 12(3): e053246.
9. Hultcrantz M, Rind D, Akl EA, et al. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol, 2017, 87: 4-13.
10. Zeng L, Brignardello-Petersen R, Guyatt G. When applying GRADE, how do we decide the target of certainty of evidence rating. Evid Based Ment Health, 2021, 24(3): 121-123.
11. Zeng L, Brignardello-Petersen R, Hultcrantz M, et al. GRADE guidelines 32: GRADE offers guidance on choosing targets of GRADE certainty of evidence ratings. J Clin Epidemiol, 2021, 137: 163-175.
12. Schünemann HJ, Neumann I, Hultcrantz M, et al. GRADE guidance 35: update on rating imprecision for assessing contextualized certainty of evidence and making decisions. J Clin Epidemiol, 2022, 150: 225-242.
13. Guyatt GH, Osoba D, Wu AW, et al. Methods to explain the clinical significance of health status measures. Mayo Clin Proc, 2002, 77(4): 371-383.
14. Qaseem A, Forland F, Macbeth F, et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med, 2012, 156(7): 525-531.
15. ECBC. Evidence-based methodologies for public health. 2011.
16. WHO. WHO handbook for guideline development. 2012.
17. Schünemann HJ, Wiercioch W, Etxeandia I, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ, 2014, 186(3): E123-E142.
18. Williamson PR, Altman DG, Bagley H, et al. The COMET handbook: version 1. 0. Trials, 2017, 18(Suppl 3): 280.
19. Kirkham JJ, Davis K, Altman DG, et al. Core outcome set-standards for development: the COS-STAD recommendations. PLoS Med, 2017, 14(11): e1002447.
20. WHO. WHO handbook of guideline development (2nd edition). 2014.
21. 丁泓帆, 杨楠, 邓围, 等. WHO指南制定的基本原则和方法. 中国循证医学杂志, 2016, 16(4): 471-477.
22. Schünemann H, Brożek J, Guyatt G, et al. GRADE handbook for selecting and rating the importance of outcomes (Updated October 2013). 2013.
23. Schünemann H, Brożek J, Guyatt G, et al. GRADE handbook for summarizing the evidence (Updated October 2013). 2013.
24. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 2017, 358: j4008.
25. 陶欢, 杨乐天, 平安, 等. 随机或非随机防治性研究系统评价的质量评价工具AMSTAR 2解读. 中国循证医学杂志, 2018, 18(1): 101-108.
26. Higgins JPT, Thomas J, Chandler J, et al. Cochrane handbook for systematic reviews of interventions version 6.3 (updated February 2022). 2022.
27. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol, 2011, 64(4): 383-394.
28. Siemieniuk RA, Bartoszko JJ, Zeraatkar D, et al. Drug treatments for COVID-19: living systematic review and network meta-analysis. BMJ, 2020, 370: m2980.
29. Guyatt GH, Oxman AD, Kunz R, et al. Going from evidence to recommendations. BMJ, 2008, 336(7652): 1049-1051.
30. Puhan MA, Singh S, Weiss CO, et al. A framework for organizing and selecting quantitative approaches for benefit-harm assessment. BMC Med Res Methodol, 2012, 12: 173.
31. Yu T, Fain K, Boyd CM, et al. Benefits and harms of roflumilast in moderate to severe COPD. Thorax, 2014, 69(7): 616-622.
32. Santesso N, Schünemann H, Blumenthal P, et al. World Health Organization guidelines: use of cryotherapy for cervical intraepithelial neoplasia. Int J Gynaecol Obstet, 2012, 118(2): 97-102.

Chinese Journal of Evidence-Based Medicine

A guideline for determining the effect size decision threshold of health outcomes

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content