ObjectivesTo explore the effect of the deep learning algorithm convolutional neural network (CNN) in screening of randomized controlled trials (RCTs) in Chinese medical literatures.MethodsLiterature with the topic " oral science” published in 2014 were retrieved from CNKI and exported citations containing title and abstract. RCTs screening was conducted by double independent screening, checking and peer discussion. The final results of the citations were used for CNN algorithm model training. After completing the algorithm model training, a prospective comparative trial was organized by searching all literature with the topic "oral science" published in CNKI from January to March 2018 to compare the sensitivity (SEN) and specificity (SPE) of algorithm with manual screening. The initial results of a single screener represented the performance of manual screening, and the final results after peer discussion were used as the gold standard. The best thresholds of algorithm were determined with the receptor operative characteristic (ROC) curve.ResultsA total of 1 246 RCTs and 4 754 non-RCTs were eventually included for training and testing of CNN algorithm model. 249 RCTs and 949 non-RCTs were included in the prospective trial. The SEN and SPE of manual screening were 98.01% and 98.82%. For the algorithm model, the SEN of RCTs screening decreased with the increase of threshold value while the SPE increased with the increase of threshold value. After 27 changes of threshold value, ROC curve were obtained. The area under the ROC curve was 0.9977, unveiling the optimal accuracy threshold (Threshold=0.4, SEN=98.39%, SPE=98.84%) and high sensitivity threshold (Threshold=0.06, SEN=99.60%, SPE=94.10%).ConclusionsA CNN algorithm model is trained with Chinese RCTs classification database established in this study and shows an excellent classification performance in screening RCTs of Chinese medical literature, which is proved to be comparable to the manual screening performance in the prospective controlled trial.
Systematic reviews can provide important evidence support for clinical practice and health decision-making. In this process, literature screening and data extraction are extensively time-consuming procedures. Natural language processing (NLP), as one of the research directions of computer science and artificial intelligence, can accelerate the process of literature screening and data extraction in systematic reviews. This paper introduced the requirements of systematic reviews for rapid literature screening and data extraction, the development of NLP and types of machine learning; and systematically collated the NLP tools for the title and abstract screening, full-text screening and data extraction in systematic reviews; and discussed the problems in the application of NLP tools in the field of systematic reviews and proposed a prospect for its future development.
ObjectiveTo evaluate the reporting quality of systematic reviews (SRs)/meta-analyses on acupuncture focusing on literature screening results and explore the influencing factors of the complete reporting.MethodsPubMed, EMbase, CNKI, WanFang Data, and VIP databases were searched to collect SRs/meta-analyses on acupuncture from inception to December 31st, 2019. Two reviewers independently screened literature, extracted data and evaluated the reporting quality of literature screening results of SRs/meta-analyses on acupuncture based on PRISMA statement. Logistic regression model analysis was applied to explore the influencing factors of the complete reporting rate of literature screening results. Statistical analysis was performed by using Excel 2016 and SPSS 16.0 software.ResultsA total of 1 227 SRs/meta-analyses were included. Only 62.3% SRs fully reported the four parts of literature screening results. The parts with a low reporting rate included the number of studies assessed for eligibility (73.2%) and the reasons for exclusions at each stage (67.0%). And the reporting rate of the literature screening flowchart was also low (63.6%). The reporting rate of literature screening results in Chinese SRs was lower than that in English SRs, and there was significantly statistical difference (P<0.001). Multivariate logistic regression analysis showed that the type of published journal, publication year, pages of article and the number of searched databases were correlated with the complete reporting rate of literature screening results (P<0.001).ConclusionsThe complete reporting rate of the literature screening results of SRs on acupuncture is low, especially in Chinese SRs. The complete reporting rate of literature screening results is significantly higher for SRs published after PRISMA statement, in SCI journals, with longer length and more searched databases.