Assessment tool ArRoWS for the quality of real world observational studies: an interpretation_Chinese Journal of Evidence-Based Medicine

Authors：

CAO Xue ¹ , MENG Xiangran ¹ , WANG Xin ¹ , SUN Minglin ¹ , DENG Hui ² , AI Yanke ¹ ,  LIU Jia ¹

1. Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, P. R. China;
2. College of Acupuncture and Moxibustion, Shanxi University of Chinese Medicine, Xianyang 712046, P. R. China;

Corresponding author：

LIU Jia, Email: marie_liujia@sina.cn

Keywords：

Observational studies; Real world evidence; Appraisal tool; ArRoWS

DOI：

10.7507/1672-2531.202206043

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Assessment of Real World Observational Studies (ArRoWS) is a tool developed by the Leicester Real World Evidence (LRWE) Unit of the Diabetes Research Centre of the University of Leicester in the United Kingdom to assess the quality of real world evidence research, and has been reported to have good practicability. ArRoWS can be used to quickly and specifically assess the quality of real world evidence research that uses electronic health record information. The tool contains 16 items, nine of which are common items, and seven of which are related to specific research designs. The current study introduces the development background, development process, assessment items, assessment criteria, and application methods of ArRoWS and other related aspects, to provide references for real world researchers in China.

Citation： CAO Xue, MENG Xiangran, WANG Xin, SUN Minglin, DENG Hui, AI Yanke, LIU Jia. Assessment tool ArRoWS for the quality of real world observational studies: an interpretation. Chinese Journal of Evidence-Based Medicine, 2023, 23(2): 227-232. doi: 10.7507/1672-2531.202206043 Copy

1.	孙鑫, 谭婧, 唐立, 等. 重新认识真实世界研究. 中国循证医学杂志, 2017, 17(2): 126-130.
2.	国家药品监督管理局. 用于产生真实世界证据的真实世界数据指导原则(试行). Available at: https://www.cde.org.cn/main/news/viewInfoCommon/2a1c437ed54e7b838a7e86f4ac21c539.
3.	王雯, 谭婧, 任燕, 等. 重新认识真实世界数据研究: 更新与展望. 中国循证医学杂志, 2020, 20(11): 1241-1246.
4.	孙鑫, 谭婧, 王雯, 等. 建立真实世界数据与研究技术规范, 促进中国真实世界证据的生产与使用. 中国循证医学杂志, 2019, 19(7): 755-762.
5.	王雪, 乌兰其其格, 杨悦. 美国应用真实世界证据辅助医药产品监管决策的研究. 中国药事, 2020, 34(9): 1057-1062.
6.	王丽茹, 刘东辉, 邢杰, 等. 基于真实世界数据的结直肠癌预后特征识别. 国际遗传学杂志, 2021, 44(6): 401-408.
7.	孟若谷, 卓琳, 乔瑞, 等. 真实世界证据在上市后药品安全性监测与评价中的应用价值. 中国药物警戒, 2021, 18(7): 624-627.
8.	任静, 王永乐, 刘亭亭, 等. 双抗治疗轻型缺血性卒中或短暂性脑缺血发作——从临床指南到真实世界研究. 中国全科医学, 2021, 24(21): 2696-2700, 2706.
9.	明丹丹, 李军, 许璇, 等. 真实世界研究的方法及其在临床研究中的应用. 中国药房, 2018, 29(15): 2138-2143.
10.	中华中医药学会《中医药真实世界研究技术规范》制订组. 中医药真实世界研究技术规范——证据质量评价与报告. 中医杂志, 2022, 63(3): 293-300.
11.	Ma LL, Wang YY, Yang ZH, et al. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better. Mil Med Res, 2020, 7(1): 7.
12.	Zeng X, Zhang Y, Kwong JS, et al. The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. J Evid Based Med, 2015, 8(1): 2-10.
13.	Wells G A, Shea B, O'Connell D. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Available at: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.
14.	Critical Appraisal Skills Programme. CASP (case control study checklist). Available at: https://casp-uk.net/casp-tools-checklists/.
15.	廖星, 章轶立, 谢雁鸣. 真实世界研究标准: RECORD清单和GRACE清单的解读. 中国中药杂志, 2015, 40(24): 4734-4738.
16.	Coles B, Tyrer F, Hussein H, et al. Development, content validation, and reliability of the assessment of real-world observational studies (ArRoWS) critical appraisal tool. Ann Epidemiol, 2021, 55: 57-63.
17.	Coles B, Tyrer F, Khunti K. Improving quality in cardiometabolic medicine real-world evidence. Prim Care Diabetes, 2019, 13(2): 188.
18.	Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol, 2007, 36(3): 666-676.
19.	陈耀龙, 李幼平, 杜亮, 等. 医学研究中证据分级和推荐强度的演进. 中国循证医学杂志, 2008, 8(2): 127-133.
20.	Waldram A, McKerr C, Gobin M, et al. Control selection methods in recent case-control studies conducted as part of infectious disease outbreaks. Eur J Epidemiol, 2015, 30(6): 465-471.
21.	Rose S, Laan MJ. Why match? Investigating matched case-control study designs with causal effect estimation. Int J Biostat, 2009, 5(1): Article 1.
22.	Pocock S, Kleinbaum D, Kupper L, et al. Epidemiologic research: principles and quantitative methods. Biometrics, 1983, 39(3): 822.
23.	Breslow NE, Day NE. Statistical methods in cancer research. Volume I - The analysis of case-control studies. IARC Sci Publ, 1980, (32): 5-338.
24.	Pearce N. Analysis of matched case-control studies. BMJ, 2016, 352: i969.
25.	Hsu CW, Tseng WT, Wang LJ, et al. Comparative effectiveness of antidepressants on geriatric depression: real-world evidence from a population-based study. J Affect Disord, 2022, 296: 609-615.
26.	Schechter M, Melzer-Cohen C, Rozenberg A, et al. Cardiorenal outcomes with sodium/glucose cotransporter-2 inhibitors in patients with type 2 diabetes and low kidney risk: real world evidence. Cardiovasc Diabetol, 2021, 20(1): 169.
27.	艾飞玲, 胡葵茹, 石钰霖, 等. 基于纽卡斯尔-渥太华量表对中国吸烟队列研究文献的质量评价. 中华疾病控制杂志, 2021, 25(6): 722-729.
28.	张蒙, 张雪梅, 杨璐, 等. 我国老年谵妄危险因素的队列研究文献的质量评价. 护理研究, 2016, 30(33): 4206-4207.
29.	杨璐, 杨佳, 张泽梅, 等. 我国老年谵妄预后队列研究的文献质量评价. 护理研究, 2016, (1): 80-82.
30.	何琼, 黄渊秀, 康文婧, 等. 2001-2010年我国伤害预防病例对照研究及队列研究文献质量评价. 中华疾病控制杂志, 2014, 18(10): 913-916.
31.	王梅, 王建华, 张抗, 等. 中医药疗效评价队列研究的方法学质量评价. 中医杂志, 2016, 57(16): 1379-1383.
32.	张学成, 刘岩, 张晶晶, 等. 急性心力衰竭患者30天再住院率: 基于真实世界研究与随机对照试验比较的Meta分析. 中国循证医学杂志, 2021, 21(10): 1203-1210.
33.	曾宪涛, 刘慧, 陈曦, 等. Meta分析系列之四: 观察性研究的质量评价工具. 中国循证心血管医学杂志, 2012, 04(4): 297-299.
34.	中华中医药学会. 中医药真实世界研究技术规范证据质量评价与报告(试行). Available at: https://www.cacm.org.cn/2021/06/30/13874/.
35.	Khambholja K, Gehani M. Use of structured template and reporting tool for real-world evidence template for critical appraisal of the quality of reporting of real-world evidence studies: a systematic review. Value Health, 2022. DOI: 10.1016/j.jval.2022.09.003.

1. 孙鑫, 谭婧, 唐立, 等. 重新认识真实世界研究. 中国循证医学杂志, 2017, 17(2): 126-130.
2. 国家药品监督管理局. 用于产生真实世界证据的真实世界数据指导原则(试行). Available at: https://www.cde.org.cn/main/news/viewInfoCommon/2a1c437ed54e7b838a7e86f4ac21c539.
3. 王雯, 谭婧, 任燕, 等. 重新认识真实世界数据研究: 更新与展望. 中国循证医学杂志, 2020, 20(11): 1241-1246.
4. 孙鑫, 谭婧, 王雯, 等. 建立真实世界数据与研究技术规范, 促进中国真实世界证据的生产与使用. 中国循证医学杂志, 2019, 19(7): 755-762.
5. 王雪, 乌兰其其格, 杨悦. 美国应用真实世界证据辅助医药产品监管决策的研究. 中国药事, 2020, 34(9): 1057-1062.
6. 王丽茹, 刘东辉, 邢杰, 等. 基于真实世界数据的结直肠癌预后特征识别. 国际遗传学杂志, 2021, 44(6): 401-408.
7. 孟若谷, 卓琳, 乔瑞, 等. 真实世界证据在上市后药品安全性监测与评价中的应用价值. 中国药物警戒, 2021, 18(7): 624-627.
8. 任静, 王永乐, 刘亭亭, 等. 双抗治疗轻型缺血性卒中或短暂性脑缺血发作——从临床指南到真实世界研究. 中国全科医学, 2021, 24(21): 2696-2700, 2706.
9. 明丹丹, 李军, 许璇, 等. 真实世界研究的方法及其在临床研究中的应用. 中国药房, 2018, 29(15): 2138-2143.
10. 中华中医药学会《中医药真实世界研究技术规范》制订组. 中医药真实世界研究技术规范——证据质量评价与报告. 中医杂志, 2022, 63(3): 293-300.
11. Ma LL, Wang YY, Yang ZH, et al. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better. Mil Med Res, 2020, 7(1): 7.
12. Zeng X, Zhang Y, Kwong JS, et al. The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. J Evid Based Med, 2015, 8(1): 2-10.
13. Wells G A, Shea B, O'Connell D. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Available at: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.
14. Critical Appraisal Skills Programme. CASP (case control study checklist). Available at: https://casp-uk.net/casp-tools-checklists/.
15. 廖星, 章轶立, 谢雁鸣. 真实世界研究标准: RECORD清单和GRACE清单的解读. 中国中药杂志, 2015, 40(24): 4734-4738.
16. Coles B, Tyrer F, Hussein H, et al. Development, content validation, and reliability of the assessment of real-world observational studies (ArRoWS) critical appraisal tool. Ann Epidemiol, 2021, 55: 57-63.
17. Coles B, Tyrer F, Khunti K. Improving quality in cardiometabolic medicine real-world evidence. Prim Care Diabetes, 2019, 13(2): 188.
18. Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol, 2007, 36(3): 666-676.
19. 陈耀龙, 李幼平, 杜亮, 等. 医学研究中证据分级和推荐强度的演进. 中国循证医学杂志, 2008, 8(2): 127-133.
20. Waldram A, McKerr C, Gobin M, et al. Control selection methods in recent case-control studies conducted as part of infectious disease outbreaks. Eur J Epidemiol, 2015, 30(6): 465-471.
21. Rose S, Laan MJ. Why match? Investigating matched case-control study designs with causal effect estimation. Int J Biostat, 2009, 5(1): Article 1.
22. Pocock S, Kleinbaum D, Kupper L, et al. Epidemiologic research: principles and quantitative methods. Biometrics, 1983, 39(3): 822.
23. Breslow NE, Day NE. Statistical methods in cancer research. Volume I - The analysis of case-control studies. IARC Sci Publ, 1980, (32): 5-338.
24. Pearce N. Analysis of matched case-control studies. BMJ, 2016, 352: i969.
25. Hsu CW, Tseng WT, Wang LJ, et al. Comparative effectiveness of antidepressants on geriatric depression: real-world evidence from a population-based study. J Affect Disord, 2022, 296: 609-615.
26. Schechter M, Melzer-Cohen C, Rozenberg A, et al. Cardiorenal outcomes with sodium/glucose cotransporter-2 inhibitors in patients with type 2 diabetes and low kidney risk: real world evidence. Cardiovasc Diabetol, 2021, 20(1): 169.
27. 艾飞玲, 胡葵茹, 石钰霖, 等. 基于纽卡斯尔-渥太华量表对中国吸烟队列研究文献的质量评价. 中华疾病控制杂志, 2021, 25(6): 722-729.
28. 张蒙, 张雪梅, 杨璐, 等. 我国老年谵妄危险因素的队列研究文献的质量评价. 护理研究, 2016, 30(33): 4206-4207.
29. 杨璐, 杨佳, 张泽梅, 等. 我国老年谵妄预后队列研究的文献质量评价. 护理研究, 2016, (1): 80-82.
30. 何琼, 黄渊秀, 康文婧, 等. 2001-2010年我国伤害预防病例对照研究及队列研究文献质量评价. 中华疾病控制杂志, 2014, 18(10): 913-916.
31. 王梅, 王建华, 张抗, 等. 中医药疗效评价队列研究的方法学质量评价. 中医杂志, 2016, 57(16): 1379-1383.
32. 张学成, 刘岩, 张晶晶, 等. 急性心力衰竭患者30天再住院率: 基于真实世界研究与随机对照试验比较的Meta分析. 中国循证医学杂志, 2021, 21(10): 1203-1210.
33. 曾宪涛, 刘慧, 陈曦, 等. Meta分析系列之四: 观察性研究的质量评价工具. 中国循证心血管医学杂志, 2012, 04(4): 297-299.
34. 中华中医药学会. 中医药真实世界研究技术规范证据质量评价与报告(试行). Available at: https://www.cacm.org.cn/2021/06/30/13874/.
35. Khambholja K, Gehani M. Use of structured template and reporting tool for real-world evidence template for critical appraisal of the quality of reporting of real-world evidence studies: a systematic review. Value Health, 2022. DOI: 10.1016/j.jval.2022.09.003.

Previous Article
Patient version of guidelines: current practice and research gaps
Next Article
Interpretation of standards for conducting and reporting Delphi studies (CREDES) in healthcare research

Chinese Journal of Evidence-Based Medicine

Assessment tool ArRoWS for the quality of real world observational studies: an interpretation

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content