A tool for assessing methodological limitations of qualitative research for inclusion in qualitative evidence synthesis—CAMELOT_Chinese Journal of Evidence-Based Medicine

Authors：

GAI Qiongyan ¹ , ZHONG Weiqin ^2,3 , LIU Yuping ² ,  BAI Zhenggang ⁴

1. Nanjing Drum Tower Hospital, the Affiliated Hospital of Nanjing University Medical School, Nanjing 210008, P. R. China;
2. Nanjing Drum Tower Hospital Clinical College of Jiangsu University, Nanjing 210008, P. R. China;
3. Yi jishan Hospital, the First Affiliated Hospital of Wannan Medical College, Wuhu 241000, P. R. China;
4. Evidence-based Research Center of Social Science & Health, Nanjing University of Science and Technology, Nanjing 210094, P. R. China;

Corresponding author：

BAI Zhenggang, Email: baizhenggang@126.com

Keywords：

Qualitative evidence synthesis; Qualitative research; Methodological limitations; Assessment tool; CAMELOT

DOI：

10.7507/1672-2531.202406177

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In recent years, the application of qualitative evidence synthesis results in health and social care policy has increased, and assessment of the limitations of qualitative research methodologies is critical in ensuring the credibility of qualitative evidence synthesis results. However, currently commonly used assessment tools are not designed specifically for Cochrane systematic reviews or qualitative evidence synthesis, and most lack evidence-based development background. To address this gap, the Cochrane qualitative methodological limitations tool (CAMELOT) was created to provide authors with a standardized tool specifically designed to assess the limitations of qualitative research methodologies. Based on the operation guide of evidence-based evaluation tool and Delphi consensus survey method, this paper introduces the development process of CAMELOT and expounds its field contents in detail.

Citation： GAI Qiongyan, ZHONG Weiqin, LIU Yuping, BAI Zhenggang. A tool for assessing methodological limitations of qualitative research for inclusion in qualitative evidence synthesis—CAMELOT. Chinese Journal of Evidence-Based Medicine, 2025, 25(2): 239-243. doi: 10.7507/1672-2531.202406177 Copy

1.	Flemming K, Booth A, Garside R, et al. Qualitative evidence synthesis for complex interventions and guideline development: clarification of the purpose, designs and relevant methods. BMJ Glob Health, 2019, 4(Suppl 1): e000882.
2.	王云云, 申泉, 田国祥, 等. 定性研究在临床实践指南制订过程中的应用方法解析. 中国循证心血管医学杂志, 2021, 13(10): 1156-1161.
3.	McArthur C, Bai Y, Hewston P, et al. Barriers and facilitators to implementing evidence-based guidelines in long-term care: a qualitative evidence synthesis. Implement Sci, 2021, 16(1): 70.
4.	Harvey CM, Smith H, Portela A, et al. Stakeholder's perspectives of postnatal discharge: a qualitative evidence synthesis. BMJ Glob Health, 2023, 8(Suppl 2): e011766.
5.	Kpokiri EE, Sri-Pathmanathan C, Shrestha P, et al. Crowdfunding for health research: a qualitative evidence synthesis and a pilot programme. BMJ Glob Health, 2022, 7(7): e009110.
6.	Taylor M, Garner P, Oliver S, et al. Use of qualitative research in World Health Organisation guidelines: a document analysis. Health Res Policy Syst, 2024, 22(1): 44.
7.	王渝强, 郭应强, 黄维, 等. 留置股血管导管的危重症患者早期活动安全性与可行性的定性系统评价. 中国胸心血管外科临床杂志, 2023, 30(3): 434-439.
8.	Glenton C, Colvin C, Carlsen B, et al. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Chichester: John Wiley & Sons, 2013: CD010414.
9.	Carmona C, Baxter S, Carroll C. Systematic review of the methodological literature for integrating qualitative evidence syntheses into health guideline development. Res Synth Methods, 2021, 12(4): 491-505.
10.	刘雪寒, 鲁春丽, 郑偌祥, 等. 定性研究的综合方法概述及其在医学中的应用进展. 世界中医药, 2021, 16(20): 3093-3099.
11.	Booth A, Mshelia S, Analo CV, et al. Qualitative evidence syntheses: assessing the relative contributions of multi-context and single-context reviews. J Adv Nurs, 2019, 75(12): 3812-3822.
12.	杨超群, 刘文迪, 王子怡, 等. 公共卫生决策证据质量分级方法问题的定性系统评价. 中国循证医学杂志, 2023, 23(12): 1413-1418.
13.	拜争刚, 刘少堃, 黄崇斐, 等. 定性系统评价证据分级工具—CERQual简介. 中国循证医学杂志, 2015, 15(12): 1465-1470.
14.	Wainwright M, Zahroh RI, Tunçalp Ö, et al. The use of GRADE-CERQual in qualitative evidence synthesis: an evaluation of fidelity and reporting. Health Res Policy Syst, 2023, 21(1): 77.
15.	Munthe-Kaas HM, Glenton C, Booth A, et al. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol, 2019, 19(1): 113.
16.	Nøkleby H, Ames HMR, Langøien LJ, et al. Tools for assessing the methodological limitations of a QES-a short note. Syst Rev, 2024, 13(1): 103.
17.	Munthe‐Kaas HM, Booth A, Sommer I, et al. Developing CAMELOT for assessing methodological limitations of qualitative research for inclusion in qualitative evidence syntheses. Cochrane Evid Synth Method, 2024, 2(6): e12058.
18.	Maeda Y, Caskurlu S, Kozan K, et al. Development of a critical appraisal tool for assessing the reporting quality of qualitative studies: a worked example. Qual Quant, 2023, 57(2): 1011-1031.
19.	Cooper C, Booth A, Varley-Campbell J, et al. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol, 2018, 18(1): 85.
20.	Small SP. Reflections on critical appraisal of research for qualitative evidence synthesis. JBI Evid Synth, 2023, 21(6): 1064-1065.
21.	Moser A, Korstjens I. Series: practical guidance to qualitative research. Part 7: qualitative evidence synthesis for emerging themes in primary care research: scoping review, meta-ethnography and rapid realist review. Eur J Gen Pract, 2023, 29(1): 2274467.
22.	周静, 杨男, 施李正, 等. 罕见病用药医保准入价值评估准则研究的定性系统评价. 中国循证医学杂志, 2019, 19(11): 1292-1299.
23.	Noushad B, Van Gerven PWM, de Bruin ABH. Twelve tips for applying the think-aloud method to capture cognitive processes. Med Teach, 2024, 46(7): 892-897.
24.	Glenton C, Lewin S, Lawrie TA, et al. Qualitative evidence synthesis (QES) for guidelines: paper 3 - using qualitative evidence syntheses to develop implementation considerations and inform implementation processes. Health Res Policy Syst, 2019, 17(1): 74.
25.	Booth A, Sommer I, Noyes J, et al. Rapid reviews methods series: guidance on rapid qualitative evidence synthesis. BMJ Evid Based Med, 2024, 29(3): 194-200.
26.	D'angelo D, Coclite D, Napoletano A, et al. The international guideline evaluation screening tool (IGEST): development and validation. BMC Med Res Methodol, 2022, 22(1): 134.
27.	Hasson F. Research guidelines for the Delphi survey technique. J Adv Nurs, 2000, 32(4): 1008-1015.

1. Flemming K, Booth A, Garside R, et al. Qualitative evidence synthesis for complex interventions and guideline development: clarification of the purpose, designs and relevant methods. BMJ Glob Health, 2019, 4(Suppl 1): e000882.
2. 王云云, 申泉, 田国祥, 等. 定性研究在临床实践指南制订过程中的应用方法解析. 中国循证心血管医学杂志, 2021, 13(10): 1156-1161.
3. McArthur C, Bai Y, Hewston P, et al. Barriers and facilitators to implementing evidence-based guidelines in long-term care: a qualitative evidence synthesis. Implement Sci, 2021, 16(1): 70.
4. Harvey CM, Smith H, Portela A, et al. Stakeholder's perspectives of postnatal discharge: a qualitative evidence synthesis. BMJ Glob Health, 2023, 8(Suppl 2): e011766.
5. Kpokiri EE, Sri-Pathmanathan C, Shrestha P, et al. Crowdfunding for health research: a qualitative evidence synthesis and a pilot programme. BMJ Glob Health, 2022, 7(7): e009110.
6. Taylor M, Garner P, Oliver S, et al. Use of qualitative research in World Health Organisation guidelines: a document analysis. Health Res Policy Syst, 2024, 22(1): 44.
7. 王渝强, 郭应强, 黄维, 等. 留置股血管导管的危重症患者早期活动安全性与可行性的定性系统评价. 中国胸心血管外科临床杂志, 2023, 30(3): 434-439.
8. Glenton C, Colvin C, Carlsen B, et al. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Chichester: John Wiley & Sons, 2013: CD010414.
9. Carmona C, Baxter S, Carroll C. Systematic review of the methodological literature for integrating qualitative evidence syntheses into health guideline development. Res Synth Methods, 2021, 12(4): 491-505.
10. 刘雪寒, 鲁春丽, 郑偌祥, 等. 定性研究的综合方法概述及其在医学中的应用进展. 世界中医药, 2021, 16(20): 3093-3099.
11. Booth A, Mshelia S, Analo CV, et al. Qualitative evidence syntheses: assessing the relative contributions of multi-context and single-context reviews. J Adv Nurs, 2019, 75(12): 3812-3822.
12. 杨超群, 刘文迪, 王子怡, 等. 公共卫生决策证据质量分级方法问题的定性系统评价. 中国循证医学杂志, 2023, 23(12): 1413-1418.
13. 拜争刚, 刘少堃, 黄崇斐, 等. 定性系统评价证据分级工具—CERQual简介. 中国循证医学杂志, 2015, 15(12): 1465-1470.
14. Wainwright M, Zahroh RI, Tunçalp Ö, et al. The use of GRADE-CERQual in qualitative evidence synthesis: an evaluation of fidelity and reporting. Health Res Policy Syst, 2023, 21(1): 77.
15. Munthe-Kaas HM, Glenton C, Booth A, et al. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol, 2019, 19(1): 113.
16. Nøkleby H, Ames HMR, Langøien LJ, et al. Tools for assessing the methodological limitations of a QES-a short note. Syst Rev, 2024, 13(1): 103.
17. Munthe‐Kaas HM, Booth A, Sommer I, et al. Developing CAMELOT for assessing methodological limitations of qualitative research for inclusion in qualitative evidence syntheses. Cochrane Evid Synth Method, 2024, 2(6): e12058.
18. Maeda Y, Caskurlu S, Kozan K, et al. Development of a critical appraisal tool for assessing the reporting quality of qualitative studies: a worked example. Qual Quant, 2023, 57(2): 1011-1031.
19. Cooper C, Booth A, Varley-Campbell J, et al. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol, 2018, 18(1): 85.
20. Small SP. Reflections on critical appraisal of research for qualitative evidence synthesis. JBI Evid Synth, 2023, 21(6): 1064-1065.
21. Moser A, Korstjens I. Series: practical guidance to qualitative research. Part 7: qualitative evidence synthesis for emerging themes in primary care research: scoping review, meta-ethnography and rapid realist review. Eur J Gen Pract, 2023, 29(1): 2274467.
22. 周静, 杨男, 施李正, 等. 罕见病用药医保准入价值评估准则研究的定性系统评价. 中国循证医学杂志, 2019, 19(11): 1292-1299.
23. Noushad B, Van Gerven PWM, de Bruin ABH. Twelve tips for applying the think-aloud method to capture cognitive processes. Med Teach, 2024, 46(7): 892-897.
24. Glenton C, Lewin S, Lawrie TA, et al. Qualitative evidence synthesis (QES) for guidelines: paper 3 - using qualitative evidence syntheses to develop implementation considerations and inform implementation processes. Health Res Policy Syst, 2019, 17(1): 74.
25. Booth A, Sommer I, Noyes J, et al. Rapid reviews methods series: guidance on rapid qualitative evidence synthesis. BMJ Evid Based Med, 2024, 29(3): 194-200.
26. D'angelo D, Coclite D, Napoletano A, et al. The international guideline evaluation screening tool (IGEST): development and validation. BMC Med Res Methodol, 2022, 22(1): 134.
27. Hasson F. Research guidelines for the Delphi survey technique. J Adv Nurs, 2000, 32(4): 1008-1015.

Chinese Journal of Evidence-Based Medicine

A tool for assessing methodological limitations of qualitative research for inclusion in qualitative evidence synthesis—CAMELOT

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content