Research status and progress of intelligent screening for titles and abstracts in systematic reviews_Chinese Journal of Evidence-Based Medicine

Authors：

ZHOU Yanxi ¹ , QIN Xuan ^2,3,4 , YAO Minghong ^2,3,4 , MA Yu ^2,3,4 , MEI Fan ^2,3,4 ,  LI Ling ^2,3,4 ,  SUN Xin ^1,2,3,4

1. West China School of Public Health and West China Fourth Hospital, Sichuan University, Chengdu 610041, P. R. China;
2. Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, P. R. China;
3. NMPA Key Laboratory for Real World Data Research and Evaluation in Hainan, Chengdu 610041, P. R. China;
4. Sichuan Center for Evidence-Based Chinese Medicine, Chengdu 610041, P. R. China;

Corresponding author：

LI Ling, Email: liling@wchscu.cn; SUN Xin, Email: sunxin@wchscu.cn

Keywords：

Intelligent screening; Artificial intelligence; Algorithms; Systematic reviews; Literature screening; Titles and abstracts

DOI：

10.7507/1672-2531.202504157

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Systematic reviews (SRs) serve as a core methodology in evidence-based medicine (EBM), providing critical evidence for clinical practice and health decision-making. However, the manual screening of titles and abstracts in SRs is labor-intensive and time-consuming, becoming a major bottleneck in research efficiency. Recent advancements in artificial intelligence (AI), particularly large language models (LLMs), have introduced new opportunities and transformations in this field. This article provided an overview of the current status of intelligent screening for titles and abstracts in systematic reviews, with a focus on the application and effectiveness of LLMs. It aims to provide recommendations for users and developers, facilitating the better integration of automation algorithms into the SR process.

1.	杨丰春, 徐晓巍, 李姣. 循证医学研究中的证据自动更新方法研究. 中国数字医学, 2022, 17(1): 44-49.
2.	吕萌, 罗旭飞, 刘云兰, 等. 2019年期刊公开发表的中国临床实践指南文献调查与评价—传播与实施情况. 协和医学杂志, 2022, 13(4): 673-678.
3.	Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA, 1992, 268(17): 2420-2425.
4.	Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet, 2017, 390(10092): 415-423.
5.	Mulrow CD. Rationale for systematic reviews. BMJ, 1994, 309(6954): 597-599.
6.	Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using EndNote. J Med Libr Assoc, 2017, 105(1): 84-87.
7.	Haddaway NR, Westgate MJ. Predicting the time needed for environmental systematic reviews and systematic maps. Conserv Biol, 2019, 33(2): 434-443.
8.	Shemilt I, Khan N, Park S, et al. Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews. Syst Rev, 2016, 5(1): 140.
9.	Patel JJ, Hill A, Lee ZY, et al. Critical appraisal of a systematic review: a concise review. Crit Care Med, 2022, 50(9): 1371-1379.
10.	Borah R, Brown AW, Capers PL, et al. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open, 2017, 7(2): e012545.
11.	Tóth B, Berek L, Gulácsi L, et al. Automation of systematic reviews of biomedical literature: a scoping review of studies indexed in PubMed. Syst Rev, 2024, 13(1): 174.
12.	Akinseloyin O, Jiang X, Palade V. A question-answering framework for automated abstract screening using large language models. J Am Med Inform Assoc, 2024, 31(9): 1939-1952.
13.	Dennstädt F, Zink J, Putora PM, et al. Title and abstract screening for literature reviews using large language models: an exploratory study in the biomedical domain. Syst Rev, 2024, 13(1): 158.
14.	Gartlehner G, Affengruber L, Titscher V, et al. Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial. J Clin Epidemiol, 2020, 121: 20-28.
15.	Waffenschmidt S, Knelangen M, Sieben W, et al. Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review. BMC Med Res Methodol, 2019, 19(1): 132.
16.	Vaswani A, Shazeer NM, Parmar N, et al. Attention is all you need. Proceedings of the Neural Information Processing Systems, 2017.
17.	Touvron H, Lavril T, Izacard G, et al. LLaMA: open and efficient foundation language models. ArXiv, 2023, abs/2302.13971.
18.	Introducing the next generation of Claude. 2024.
19.	DeepSeek-V3 technical report. 2024.
20.	Naveed H, Khan AU, Qiu S, et al. A comprehensive overview of large language models. ArXiv, 2023, abs/2307.06435.
21.	Oami T, Okada Y, Nakada TA. Performance of a large language model in screening citations. JAMA Netw Open, 2024, 7(7): e2420496.
22.	Karystianis G, Thayer K, Wolfe M, et al. Evaluation of a rule-based method for epidemiological document classification towards the automation of systematic reviews. J Biomed Inform, 2017, 70: 27-34.
23.	Khalil H, Ameen D, Zarnegar A. Tools to support the automation of systematic reviews: a scoping review. J Clin Epidemiol, 2022, 144: 22-42.
24.	Yu W, Clyne M, Dolan SM, et al. GAPscreener: an automatic tool for screening human genetic association literature in PubMed using the support vector machine technique. BMC Bioinformatics, 2008, 9: 205.
25.	Shemilt I, Simon A, Hollands GJ, et al. Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. Res Synth Methods, 2014, 5(1): 31-49.
26.	Gates A, Johnson C, Hartling L. Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the Abstrackr machine learning tool. Syst Rev, 2018, 7(1): 45.
27.	Przybyła P, Brockmeier AJ, Kontonatsios G, et al. Prioritising references for systematic reviews with RobotAnalyst: a user study. Res Synth Methods, 2018, 9(3): 470-488.
28.	Bannach-Brown A, Przybyła P, Thomas J, et al. Machine learning algorithms for systematic review: reducing workload in a preclinical review of animal studies and reducing human screening error. Syst Rev, 2019, 8(1): 23.
29.	Currie GL, Angel-Scott HN, Colvin L, et al. Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis. PLoS Biol, 2019, 17(5): e3000243.
30.	Gates A, Guitard S, Pillay J, et al. Performance and usability of machine learning for screening in systematic reviews: a comparative evaluation of three tools. Syst Rev, 2019, 8(1): 278.
31.	Lam J, Howard BE, Thayer K, et al. Low-calorie sweeteners and health outcomes: a demonstration of rapid evidence mapping (rEM). Environ Int, 2019, 123: 451-458.
32.	Callaghan MW, Müller-Hansen F. Statistical stopping criteria for automated screening in systematic reviews. Syst Rev, 2020, 9(1): 273.
33.	Reddy SM, Patel S, Weyrich M, et al. Comparison of a traditional systematic review approach with review-of-reviews and semi-automation as strategies to update the evidence. Syst Rev, 2020, 9(1): 243.
34.	Cohen AM, Hersh WR, Peterson K, et al. Reducing workload in systematic review preparation using automated citation classification. J Am Med Inform Assoc, 2006, 13(2): 206-219.
35.	Xiong Z, Liu T, Tse G, et al. A machine learning aided systematic review and meta-analysis of the relative risk of atrial fibrillation in patients with diabetes mellitus. Front Physiol, 2018, 9: 835.
36.	Weißer T, Saßmannshausen T, Ohrndorf D, et al. A clustering approach for topic filtering within systematic literature reviews. MethodsX, 2020, 7: 100831.
37.	Joachims T. A probabilistic analysis of the Roccio algorithm with TFIDF for text categorization. Proceedings of the Fourteenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc, 1997: 143-151.
38.	Cristianini N, Shawe-Taylor J. An introduction to support vector machines: and other kernel-based learning methods. Cambridge University Press, 1999.
39.	Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. Arxiv, 2023: 150801991cs.
40.	Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation. J Mach Learn Res, 2003, 3(null): 993-1022.
41.	Cohen AM. Optimizing feature representation for automated systematic review work prioritization. AMIA Annual Symposium proceedings AMIA Symposium, 2008, 2008: 121-125.
42.	Cohen AM, Ambert K, McDonagh M. Cross-topic learning for work prioritization in systematic review creation and update. J Am Med Inform Assoc, 2009, 16(5): 690-704.
43.	Bekhuis T, Demner-Fushman D. Towards automating the initial screening phase of a systematic review. Stud Health Technol Inform, 2010, 160(Pt 1): 146-150.
44.	Cohen AM, Ambert K, Mcdonagh M. A prospective evaluation of an automated classification system to support evidence-based medicine and systematic review. AMIA Annual Symposium proceedings AMIA Symposium, 2010, 2010: 121-125.
45.	Wallace BC, Trikalinos TA, Lau J, et al. Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics, 2010, 11: 55.
46.	Wallace BC, Small K, Brodley CE, et al. Toward modernizing the systematic review pipeline in genetics: efficient updating via data mining. Genet Med, 2012, 14(7): 663-669.
47.	Miwa M, Thomas J, O'Mara-Eves A, et al. Reducing systematic review workload through certainty-based screening. J Biomed Inform, 2014, 51: 242-253.
48.	Cohen AM, Smalheiser NR, McDonagh MS, et al. Automated confidence ranked classification of randomized controlled trial articles: an aid to evidence-based medicine. J Am Med Inform Assoc, 2015, 22(3): 707-717.
49.	Dunn AG, Arachi D, Bourgeois FT. Identifying clinical study types from PubMed metadata: the active (machine) learning approach. Stud Health Technol Inform, 2015, 216: 867-71.
50.	Mo Y, Kontonatsios G, Ananiadou S. Supporting systematic reviews using LDA-based document representations. Syst Rev, 2015, 4: 172.
51.	Hashimoto K, Kontonatsios G, Miwa M, et al. Topic detection using paragraph vectors to support active learning in systematic reviews. J Biomed Inform, 2016, 62: 59-65.
52.	Ouzzani M, Hammady H, Fedorowicz Z, et al. Rayyan-a web and mobile app for systematic reviews. Syst Rev, 2016, 5(1): 210.
53.	Ajiji P, Cottin J, Picot C, et al. Feasibility study and evaluation of expert opinion on the semi-automated meta-analysis and the conventional meta-analysis. Eur J Clin Pharmacol, 2022, 78(7): 1177-1184.
54.	Bekhuis T, Tseytlin E, Mitchell KJ, et al. Feature engineering and a proposed decision-support system for systematic reviewers of medical evidence. PLoS One, 2014, 9(1): e86277.
55.	Frunza O, Inkpen D, Matwin S, et al. Exploiting the systematic review protocol for classification of medical abstracts. Artif Intell Med, 2011, 51(1): 17-25.
56.	Huang KC, Chiang IJ, Xiao F, et al. PICO element detection in medical text without metadata: are first sentences enough. J Biomed Inform, 2013, 46(5): 940-946.
57.	Matwin S, Kouznetsov A, Inkpen D, et al. A new algorithm for reducing the workload of experts in performing systematic reviews. J Am Med Inform Assoc, 2010, 17(4): 446-453.
58.	van den Bulk LM, Bouzembrak Y, Gavai A, et al. Automatic classification of literature in systematic reviews on food safety using machine learning. Curr Res Food Sci, 2021, 5: 84-95.
59.	van Lissa CJ. Mapping phenomena relevant to adolescent emotion regulation: a text-mining systematic review. Adolesc Res Rev, 2022, 7(1): 127-139.
60.	Lecun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553): 436-444.
61.	Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. Science, 2006, 313(5786): 504-507.
62.	Cohen JD, Servan-Schreiber D, Mcclelland JL. A parallel distributed processing approach to automaticity. Am J Psychol, 1992, 105(2): 239-269.
63.	Duponchel L, Rocha de Oliveira R, Motto-Ros V. Large language models (such as ChatGPT) as tools for machine learning-based data insights in analytical chemistry. Anal Chem, 2025, 97(13): 6956-6961.
64.	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, California, USA: Curran Associates Inc, 2017: 6000-6010.
65.	Guo E, Gupta M, Deng J, et al. Automated paper screening for clinical reviews using large language models: data analysis study. J Med Internet Res, 2024, 26: e48996.
66.	Issaiy M, Ghanaati H, Kolahi S, et al. Methodological insights into ChatGPT's screening performance in systematic reviews. BMC Med Res Methodol, 2024, 24(1): 78.
67.	Landschaft A, Antweiler D, Mackay S, et al. Implementation and evaluation of an additional GPT-4-based reviewer in PRISMA-based medical systematic literature reviews. Int J Med Inform, 2024, 189: 105531.
68.	Li M, Sun J, Tan X. Evaluating the effectiveness of large language models in abstract screening: a comparative analysis. Syst Rev, 2024, 13(1): 219.
69.	Matsui K, Utsumi T, Aoki Y, et al. Human-comparable sensitivity of large language models in identifying eligible studies through title and abstract screening: 3-layer strategy using GPT-3. 5 and GPT-4 for systematic reviews. J Med Internet Res, 2024, 26: e52758.
70.	White M. Sample size in quantitative instrument-based studies published in Scopus up to 2022: An artificial intelligence aided systematic review. Acta Psychol (Amst), 2023, 241: 104095.
71.	Lieberum JL, Toews M, Metzendorf MI, et al. Large language models for conducting systematic reviews: on the rise, but not yet ready for use-a scoping review. J Clin Epidemiol, 2025, 181: 111746.
72.	Nguyen M, Baker A, Kirsch A, et al. Min P sampling: balancing creativity and coherence at high temperature. Arxiv, 2023: 240701082cs.
73.	Salinas A, Morstatter F. The butterfly effect of altering prompts: how small changes and jailbreaks affect large language model performance. Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2024.
74.	Kataoka Y, So R, Banno M, et al. Development of meta-prompts for large language models to screen titles and abstracts for diagnostic test accuracy reviews. medRxiv, 2023: 2023.10. 31.23297818.
75.	Li J, Kabouji J, Bouhadoun S, et al. Sensitivity and specificity of alternative screening methods for systematic reviews using text mining tools. J Clin Epidemiol, 2023, 162: 72-80.
76.	Dobrinić D, Miler M, Medak D. Mapping the green urban: a comprehensive review of materials and learning methods for green infrastructure mapping. Sensors (Basel), 2025, 25(2): 464.
77.	Clark J, Glasziou P, Del Mar C, et al. A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol, 2020, 121: 81-90.
78.	Clark J, McFarlane C, Cleo G, et al. The impact of systematic review automation tools on methodological quality and time taken to complete systematic review tasks: case study. JMIR Med Educ, 2021, 7(2): e24418.
79.	Perlman-Arrow S, Loo N, Bobrovitz N, et al. A real-world evaluation of the implementation of NLP technology in abstract screening of a systematic review. Res Synth Methods, 2023, 14(4): 608-621.

1. 杨丰春, 徐晓巍, 李姣. 循证医学研究中的证据自动更新方法研究. 中国数字医学, 2022, 17(1): 44-49.
2. 吕萌, 罗旭飞, 刘云兰, 等. 2019年期刊公开发表的中国临床实践指南文献调查与评价—传播与实施情况. 协和医学杂志, 2022, 13(4): 673-678.
3. Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA, 1992, 268(17): 2420-2425.
4. Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet, 2017, 390(10092): 415-423.
5. Mulrow CD. Rationale for systematic reviews. BMJ, 1994, 309(6954): 597-599.
6. Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using EndNote. J Med Libr Assoc, 2017, 105(1): 84-87.
7. Haddaway NR, Westgate MJ. Predicting the time needed for environmental systematic reviews and systematic maps. Conserv Biol, 2019, 33(2): 434-443.
8. Shemilt I, Khan N, Park S, et al. Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews. Syst Rev, 2016, 5(1): 140.
9. Patel JJ, Hill A, Lee ZY, et al. Critical appraisal of a systematic review: a concise review. Crit Care Med, 2022, 50(9): 1371-1379.
10. Borah R, Brown AW, Capers PL, et al. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open, 2017, 7(2): e012545.
11. Tóth B, Berek L, Gulácsi L, et al. Automation of systematic reviews of biomedical literature: a scoping review of studies indexed in PubMed. Syst Rev, 2024, 13(1): 174.
12. Akinseloyin O, Jiang X, Palade V. A question-answering framework for automated abstract screening using large language models. J Am Med Inform Assoc, 2024, 31(9): 1939-1952.
13. Dennstädt F, Zink J, Putora PM, et al. Title and abstract screening for literature reviews using large language models: an exploratory study in the biomedical domain. Syst Rev, 2024, 13(1): 158.
14. Gartlehner G, Affengruber L, Titscher V, et al. Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial. J Clin Epidemiol, 2020, 121: 20-28.
15. Waffenschmidt S, Knelangen M, Sieben W, et al. Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review. BMC Med Res Methodol, 2019, 19(1): 132.
16. Vaswani A, Shazeer NM, Parmar N, et al. Attention is all you need. Proceedings of the Neural Information Processing Systems, 2017.
17. Touvron H, Lavril T, Izacard G, et al. LLaMA: open and efficient foundation language models. ArXiv, 2023, abs/2302.13971.
18. Introducing the next generation of Claude. 2024.
19. DeepSeek-V3 technical report. 2024.
20. Naveed H, Khan AU, Qiu S, et al. A comprehensive overview of large language models. ArXiv, 2023, abs/2307.06435.
21. Oami T, Okada Y, Nakada TA. Performance of a large language model in screening citations. JAMA Netw Open, 2024, 7(7): e2420496.
22. Karystianis G, Thayer K, Wolfe M, et al. Evaluation of a rule-based method for epidemiological document classification towards the automation of systematic reviews. J Biomed Inform, 2017, 70: 27-34.
23. Khalil H, Ameen D, Zarnegar A. Tools to support the automation of systematic reviews: a scoping review. J Clin Epidemiol, 2022, 144: 22-42.
24. Yu W, Clyne M, Dolan SM, et al. GAPscreener: an automatic tool for screening human genetic association literature in PubMed using the support vector machine technique. BMC Bioinformatics, 2008, 9: 205.
25. Shemilt I, Simon A, Hollands GJ, et al. Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. Res Synth Methods, 2014, 5(1): 31-49.
26. Gates A, Johnson C, Hartling L. Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the Abstrackr machine learning tool. Syst Rev, 2018, 7(1): 45.
27. Przybyła P, Brockmeier AJ, Kontonatsios G, et al. Prioritising references for systematic reviews with RobotAnalyst: a user study. Res Synth Methods, 2018, 9(3): 470-488.
28. Bannach-Brown A, Przybyła P, Thomas J, et al. Machine learning algorithms for systematic review: reducing workload in a preclinical review of animal studies and reducing human screening error. Syst Rev, 2019, 8(1): 23.
29. Currie GL, Angel-Scott HN, Colvin L, et al. Animal models of chemotherapy-induced peripheral neuropathy: A machine-assisted systematic review and meta-analysis. PLoS Biol, 2019, 17(5): e3000243.
30. Gates A, Guitard S, Pillay J, et al. Performance and usability of machine learning for screening in systematic reviews: a comparative evaluation of three tools. Syst Rev, 2019, 8(1): 278.
31. Lam J, Howard BE, Thayer K, et al. Low-calorie sweeteners and health outcomes: a demonstration of rapid evidence mapping (rEM). Environ Int, 2019, 123: 451-458.
32. Callaghan MW, Müller-Hansen F. Statistical stopping criteria for automated screening in systematic reviews. Syst Rev, 2020, 9(1): 273.
33. Reddy SM, Patel S, Weyrich M, et al. Comparison of a traditional systematic review approach with review-of-reviews and semi-automation as strategies to update the evidence. Syst Rev, 2020, 9(1): 243.
34. Cohen AM, Hersh WR, Peterson K, et al. Reducing workload in systematic review preparation using automated citation classification. J Am Med Inform Assoc, 2006, 13(2): 206-219.
35. Xiong Z, Liu T, Tse G, et al. A machine learning aided systematic review and meta-analysis of the relative risk of atrial fibrillation in patients with diabetes mellitus. Front Physiol, 2018, 9: 835.
36. Weißer T, Saßmannshausen T, Ohrndorf D, et al. A clustering approach for topic filtering within systematic literature reviews. MethodsX, 2020, 7: 100831.
37. Joachims T. A probabilistic analysis of the Roccio algorithm with TFIDF for text categorization. Proceedings of the Fourteenth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc, 1997: 143-151.
38. Cristianini N, Shawe-Taylor J. An introduction to support vector machines: and other kernel-based learning methods. Cambridge University Press, 1999.
39. Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. Arxiv, 2023: 150801991cs.
40. Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation. J Mach Learn Res, 2003, 3(null): 993-1022.
41. Cohen AM. Optimizing feature representation for automated systematic review work prioritization. AMIA Annual Symposium proceedings AMIA Symposium, 2008, 2008: 121-125.
42. Cohen AM, Ambert K, McDonagh M. Cross-topic learning for work prioritization in systematic review creation and update. J Am Med Inform Assoc, 2009, 16(5): 690-704.
43. Bekhuis T, Demner-Fushman D. Towards automating the initial screening phase of a systematic review. Stud Health Technol Inform, 2010, 160(Pt 1): 146-150.
44. Cohen AM, Ambert K, Mcdonagh M. A prospective evaluation of an automated classification system to support evidence-based medicine and systematic review. AMIA Annual Symposium proceedings AMIA Symposium, 2010, 2010: 121-125.
45. Wallace BC, Trikalinos TA, Lau J, et al. Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics, 2010, 11: 55.
46. Wallace BC, Small K, Brodley CE, et al. Toward modernizing the systematic review pipeline in genetics: efficient updating via data mining. Genet Med, 2012, 14(7): 663-669.
47. Miwa M, Thomas J, O'Mara-Eves A, et al. Reducing systematic review workload through certainty-based screening. J Biomed Inform, 2014, 51: 242-253.
48. Cohen AM, Smalheiser NR, McDonagh MS, et al. Automated confidence ranked classification of randomized controlled trial articles: an aid to evidence-based medicine. J Am Med Inform Assoc, 2015, 22(3): 707-717.
49. Dunn AG, Arachi D, Bourgeois FT. Identifying clinical study types from PubMed metadata: the active (machine) learning approach. Stud Health Technol Inform, 2015, 216: 867-71.
50. Mo Y, Kontonatsios G, Ananiadou S. Supporting systematic reviews using LDA-based document representations. Syst Rev, 2015, 4: 172.
51. Hashimoto K, Kontonatsios G, Miwa M, et al. Topic detection using paragraph vectors to support active learning in systematic reviews. J Biomed Inform, 2016, 62: 59-65.
52. Ouzzani M, Hammady H, Fedorowicz Z, et al. Rayyan-a web and mobile app for systematic reviews. Syst Rev, 2016, 5(1): 210.
53. Ajiji P, Cottin J, Picot C, et al. Feasibility study and evaluation of expert opinion on the semi-automated meta-analysis and the conventional meta-analysis. Eur J Clin Pharmacol, 2022, 78(7): 1177-1184.
54. Bekhuis T, Tseytlin E, Mitchell KJ, et al. Feature engineering and a proposed decision-support system for systematic reviewers of medical evidence. PLoS One, 2014, 9(1): e86277.
55. Frunza O, Inkpen D, Matwin S, et al. Exploiting the systematic review protocol for classification of medical abstracts. Artif Intell Med, 2011, 51(1): 17-25.
56. Huang KC, Chiang IJ, Xiao F, et al. PICO element detection in medical text without metadata: are first sentences enough. J Biomed Inform, 2013, 46(5): 940-946.
57. Matwin S, Kouznetsov A, Inkpen D, et al. A new algorithm for reducing the workload of experts in performing systematic reviews. J Am Med Inform Assoc, 2010, 17(4): 446-453.
58. van den Bulk LM, Bouzembrak Y, Gavai A, et al. Automatic classification of literature in systematic reviews on food safety using machine learning. Curr Res Food Sci, 2021, 5: 84-95.
59. van Lissa CJ. Mapping phenomena relevant to adolescent emotion regulation: a text-mining systematic review. Adolesc Res Rev, 2022, 7(1): 127-139.
60. Lecun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553): 436-444.
61. Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. Science, 2006, 313(5786): 504-507.
62. Cohen JD, Servan-Schreiber D, Mcclelland JL. A parallel distributed processing approach to automaticity. Am J Psychol, 1992, 105(2): 239-269.
63. Duponchel L, Rocha de Oliveira R, Motto-Ros V. Large language models (such as ChatGPT) as tools for machine learning-based data insights in analytical chemistry. Anal Chem, 2025, 97(13): 6956-6961.
64. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, California, USA: Curran Associates Inc, 2017: 6000-6010.
65. Guo E, Gupta M, Deng J, et al. Automated paper screening for clinical reviews using large language models: data analysis study. J Med Internet Res, 2024, 26: e48996.
66. Issaiy M, Ghanaati H, Kolahi S, et al. Methodological insights into ChatGPT's screening performance in systematic reviews. BMC Med Res Methodol, 2024, 24(1): 78.
67. Landschaft A, Antweiler D, Mackay S, et al. Implementation and evaluation of an additional GPT-4-based reviewer in PRISMA-based medical systematic literature reviews. Int J Med Inform, 2024, 189: 105531.
68. Li M, Sun J, Tan X. Evaluating the effectiveness of large language models in abstract screening: a comparative analysis. Syst Rev, 2024, 13(1): 219.
69. Matsui K, Utsumi T, Aoki Y, et al. Human-comparable sensitivity of large language models in identifying eligible studies through title and abstract screening: 3-layer strategy using GPT-3. 5 and GPT-4 for systematic reviews. J Med Internet Res, 2024, 26: e52758.
70. White M. Sample size in quantitative instrument-based studies published in Scopus up to 2022: An artificial intelligence aided systematic review. Acta Psychol (Amst), 2023, 241: 104095.
71. Lieberum JL, Toews M, Metzendorf MI, et al. Large language models for conducting systematic reviews: on the rise, but not yet ready for use-a scoping review. J Clin Epidemiol, 2025, 181: 111746.
72. Nguyen M, Baker A, Kirsch A, et al. Min P sampling: balancing creativity and coherence at high temperature. Arxiv, 2023: 240701082cs.
73. Salinas A, Morstatter F. The butterfly effect of altering prompts: how small changes and jailbreaks affect large language model performance. Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2024.
74. Kataoka Y, So R, Banno M, et al. Development of meta-prompts for large language models to screen titles and abstracts for diagnostic test accuracy reviews. medRxiv, 2023: 2023.10. 31.23297818.
75. Li J, Kabouji J, Bouhadoun S, et al. Sensitivity and specificity of alternative screening methods for systematic reviews using text mining tools. J Clin Epidemiol, 2023, 162: 72-80.
76. Dobrinić D, Miler M, Medak D. Mapping the green urban: a comprehensive review of materials and learning methods for green infrastructure mapping. Sensors (Basel), 2025, 25(2): 464.
77. Clark J, Glasziou P, Del Mar C, et al. A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol, 2020, 121: 81-90.
78. Clark J, McFarlane C, Cleo G, et al. The impact of systematic review automation tools on methodological quality and time taken to complete systematic review tasks: case study. JMIR Med Educ, 2021, 7(2): e24418.
79. Perlman-Arrow S, Loo N, Bobrovitz N, et al. A real-world evaluation of the implementation of NLP technology in abstract screening of a systematic review. Res Synth Methods, 2023, 14(4): 608-621.

Chinese Journal of Evidence-Based Medicine

Latest ArticlesResearch status and progress of intelligent screening for titles and abstracts in systematic reviews

Abstract Full text Figures/Tables Video References Cited by

Format

Content