Variability evaluation and empirical study in the importance rating of clinical questions and outcomes in the development of guidelines_Chinese Journal of Evidence-Based Medicine

Authors：

GAO Yicheng ^1,2,3 , YU Zijin ⁴ , WANG Yaqi ^1,2,3 , FANG Rui ⁵ , WANG Cheng ⁵ , LI Yuanyuan ⁵ , DENG Yingjie ⁵ , XIANG Wenyuan ⁵ ,  FEI Yutong ^1,2,3

1. Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing 100029, P. R. China;
2. Institute for Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing 100029, P. R. China;
3. Beijing GRADE Center, Beijing University of Chinese Medicine, Beijing 100029, P. R. China;
4. School of Acupuncture-Moxibustion and Tuina, Beijing University of Chinese Medicine, Beijing 100029, P. R. China;
5. Affiliated Hospital of Traditional Chinese Medicine, Xinjiang Medical University, Urumqi 830000, P. R. China;

Corresponding author：

FEI Yutong, Email: yutong_fei@163.com

Keywords：

Clinical question; Variability; Outcome; Methodology; Clinical practice guideline

DOI：

10.7507/1672-2531.202402106

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective The application of the coefficient of variation (CV) in the development of clinical practice guidelines is limited to evaluating the consistency of the consensus panel in clinical questions rating, and the application of variability was limited. This study presents the application and results of variability evaluation in the development of guidelines. Methods We conducted a large-scale clinical survey through questionnaire survey, and conducted two rounds of questionnaire survey and face-to-face consensus meeting for the consensus group. Means and CV were calculated for clinical questions and outcome importance ratings. We performed the summary and analysis by SPSS and Microsoft Excel. Results A total of 356 clinical survey questionnaires and two rounds survey in consensus panel were collected. We found that in the clinical survey and the first-round of the consensus panel, the CV was greater than 25% for all clinical questions regardless of the overall importance score. In the consensus panel, the results of the second-round were greatly changed. On the one hand, compared with the first-round, the CV of almost all clinical questions was smaller in the second-round, and the CV of high-priority clinical questions was less than 25%, while the clinical questions with a CV greater than 25% were of low-priority. In view of the CV of outcome importance, the clinical survey was similar to the results of the first-round of consensus panel. The CV of very important outcomes was less than 30%. In the second-round of consensus panel, the variability of very important outcomes was less than 20%. The higher the importance level of the outcome was, the smaller the CV was. Conclusion The study of variability evaluation has practical methodological value, which can assist clinical questions and outcomes priority selection, and help to fully consider the influence of different factors and values, and develop high-quality guidelines.

Citation： GAO Yicheng, YU Zijin, WANG Yaqi, FANG Rui, WANG Cheng, LI Yuanyuan, DENG Yingjie, XIANG Wenyuan, FEI Yutong. Variability evaluation and empirical study in the importance rating of clinical questions and outcomes in the development of guidelines. Chinese Journal of Evidence-Based Medicine, 2024, 24(10): 1198-1204. doi: 10.7507/1672-2531.202402106 Copy

1.	Institute of Medicine. Clinical practice guidelines we can trust. Washington DC: The National Academies Press, 2011, 16-25.
2.	Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol, 2011, 64(4): 395-400.
3.	El-Harakeh A, Morsi RZ, Fadlallah R, et al. Prioritization approaches in the development of health practice guidelines: a systematic review. BMC Health Serv Res, 2019, 19(1): 692.
4.	Wiercioch W, Nieuwlaat R, Dahm P, et al. Development and application of health outcome descriptors facilitated decision-making in the production of practice guidelines. J Clin Epidemiol, 2021, 138: 115-127.
5.	Wiercioch W, Nieuwlaat R, Zhang Y, et al. New methods facilitated the process of prioritizing questions and health outcomes in guideline development. J Clin Epidemiol, 2022, 143: 91-104.
6.	Zaror C, Deana NF, Espinoza-Espinoza G, et al. Questions and health outcomes prioritization for the development of a COVID-19 dental clinical practice guideline: a case study. J Eval Clin Pract, 2022, 28(3): 404-410.
7.	Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Tech Assess, 1998, 2(3): 1-88.
8.	Chapple M, Murphy R. The nominal group technique: extending the evaluation of students' teaching and learning experiences. Assess Eval High Edu, 1996, 21(2): 147-160.
9.	高一城, 夏如玉, 柴倩云, 等. 组建患者小组: 将患者价值观与偏好整合融入指南制订过程的建议. 协和医学杂志, 2023, (6): 1-13.
10.	肖晶旻, 谢倩文, 刘少南, 等. 循证中医药实践中患者价值观及偏好研究的概括性评价. 中医杂志, 2019, 60(24): 2089-2094.
11.	李元, 韩学杰, 王丽颖. 中医临床实践指南共识专家遴选方法. 北京中医药, 2021, 40(9): 1002-1004.
12.	周殷华, 程瑜. 国际干细胞医疗市场治理与医生的职业价值观. 广西民族大学学报(哲学社会科学版), 2017, 39(1): 20-26.
13.	王伟, 陈军. 生命价值观视角下医生对患者的态度调查. 中国医学伦理学, 2016, 29(6): 963-965.
14.	Xing D, Wang Q, Yang Z, et al. Evidence-based guidelines for intra-articular injection in knee osteoarthritis: formulating and evaluating research questions. Int J Rheum Dis, 2018, 21(8): 1533-1542.
15.	高一城, 刘美君, 苏澄元, 等. 变异性量化应用在指南临床问题构建中的方法学建议. 中国循证医学杂志, 2023, 23(11): 1329-1333.
16.	《中成药治疗优势病种临床应用指南》标准化项目组. 中成药治疗更年期综合征临床应用指南(2020年). 中国中西医结合杂志, 2021, 41(4): 418-426.
17.	杨超, 杜世豪, 王舢泽, 等. 基于改良德尔菲法构建《女性尿失禁针灸临床实践指南》临床问题. 中国针灸, 2022, 42(8): 927-931.
18.	Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn't. BMJ, 1996, 312(7023): 71-72.
19.	高一城, 夏如玉, 王雅琪, 等. 循证临床实践指南中临床问题结构完整性及表述规范性的建议. 中国循证医学杂志, 2023, 23(1): 120-124.
20.	Chakraborty S, Brijnath B, Dermentzis J, et al. Defining key questions for clinical practice guidelines: a novel approach for developing clinically relevant questions. Health Res Policy Syst, 2020, 18(1): 113.
21.	陈薇, 郭宇博, 方赛男, 等. 中医药临床实践指南中临床问题的确定. 中国中西医结合杂志, 2020, 40(2): 220-223.
22.	李慧, 陈耀龙, 韦当, 等. 中医(中西医结合)临床实践指南制修订方法—临床问题的构建. 中华中医药杂志, 2016, 31(6): 2202-2205.
23.	Sanabria AJ, Pardo-Hernandez H, Ballesteros M, et al. The UpPriority tool was developed to guide the prioritization of clinical guideline questions for updating. J Clin Epidemiol, 2020, 126: 80-92.
24.	Speckman RA, Friedly JL. Asking structured, answerable clinical questions using the population, intervention/comparator, outcome (pico) framework. PM R, 2019, 11(5): 548-553.

1. Institute of Medicine. Clinical practice guidelines we can trust. Washington DC: The National Academies Press, 2011, 16-25.
2. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol, 2011, 64(4): 395-400.
3. El-Harakeh A, Morsi RZ, Fadlallah R, et al. Prioritization approaches in the development of health practice guidelines: a systematic review. BMC Health Serv Res, 2019, 19(1): 692.
4. Wiercioch W, Nieuwlaat R, Dahm P, et al. Development and application of health outcome descriptors facilitated decision-making in the production of practice guidelines. J Clin Epidemiol, 2021, 138: 115-127.
5. Wiercioch W, Nieuwlaat R, Zhang Y, et al. New methods facilitated the process of prioritizing questions and health outcomes in guideline development. J Clin Epidemiol, 2022, 143: 91-104.
6. Zaror C, Deana NF, Espinoza-Espinoza G, et al. Questions and health outcomes prioritization for the development of a COVID-19 dental clinical practice guideline: a case study. J Eval Clin Pract, 2022, 28(3): 404-410.
7. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Tech Assess, 1998, 2(3): 1-88.
8. Chapple M, Murphy R. The nominal group technique: extending the evaluation of students' teaching and learning experiences. Assess Eval High Edu, 1996, 21(2): 147-160.
9. 高一城, 夏如玉, 柴倩云, 等. 组建患者小组: 将患者价值观与偏好整合融入指南制订过程的建议. 协和医学杂志, 2023, (6): 1-13.
10. 肖晶旻, 谢倩文, 刘少南, 等. 循证中医药实践中患者价值观及偏好研究的概括性评价. 中医杂志, 2019, 60(24): 2089-2094.
11. 李元, 韩学杰, 王丽颖. 中医临床实践指南共识专家遴选方法. 北京中医药, 2021, 40(9): 1002-1004.
12. 周殷华, 程瑜. 国际干细胞医疗市场治理与医生的职业价值观. 广西民族大学学报(哲学社会科学版), 2017, 39(1): 20-26.
13. 王伟, 陈军. 生命价值观视角下医生对患者的态度调查. 中国医学伦理学, 2016, 29(6): 963-965.
14. Xing D, Wang Q, Yang Z, et al. Evidence-based guidelines for intra-articular injection in knee osteoarthritis: formulating and evaluating research questions. Int J Rheum Dis, 2018, 21(8): 1533-1542.
15. 高一城, 刘美君, 苏澄元, 等. 变异性量化应用在指南临床问题构建中的方法学建议. 中国循证医学杂志, 2023, 23(11): 1329-1333.
16. 《中成药治疗优势病种临床应用指南》标准化项目组. 中成药治疗更年期综合征临床应用指南(2020年). 中国中西医结合杂志, 2021, 41(4): 418-426.
17. 杨超, 杜世豪, 王舢泽, 等. 基于改良德尔菲法构建《女性尿失禁针灸临床实践指南》临床问题. 中国针灸, 2022, 42(8): 927-931.
18. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn't. BMJ, 1996, 312(7023): 71-72.
19. 高一城, 夏如玉, 王雅琪, 等. 循证临床实践指南中临床问题结构完整性及表述规范性的建议. 中国循证医学杂志, 2023, 23(1): 120-124.
20. Chakraborty S, Brijnath B, Dermentzis J, et al. Defining key questions for clinical practice guidelines: a novel approach for developing clinically relevant questions. Health Res Policy Syst, 2020, 18(1): 113.
21. 陈薇, 郭宇博, 方赛男, 等. 中医药临床实践指南中临床问题的确定. 中国中西医结合杂志, 2020, 40(2): 220-223.
22. 李慧, 陈耀龙, 韦当, 等. 中医(中西医结合)临床实践指南制修订方法—临床问题的构建. 中华中医药杂志, 2016, 31(6): 2202-2205.
23. Sanabria AJ, Pardo-Hernandez H, Ballesteros M, et al. The UpPriority tool was developed to guide the prioritization of clinical guideline questions for updating. J Clin Epidemiol, 2020, 126: 80-92.
24. Speckman RA, Friedly JL. Asking structured, answerable clinical questions using the population, intervention/comparator, outcome (pico) framework. PM R, 2019, 11(5): 548-553.

Chinese Journal of Evidence-Based Medicine

Variability evaluation and empirical study in the importance rating of clinical questions and outcomes in the development of guidelines

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content