Medical text classification model integrating medical entity label semantics_Journal of Biomedical Engineering

Authors：

WEI Li ,  ZHAO Dechun , QIN Lu , LIU Yanghuazi , SHEN Yuchen

School of Life Health Information Science and Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P. R. China;

Corresponding author：

ZHAO Dechun, Email: zhaodc@cqupt.edu.cn

Keywords：

Medical question classification; Medical entity label semantics; Adaptive fusion mechanism; Prior knowledge

DOI：

10.7507/1001-5515.202408001

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Automatic classification of medical questions is of great significance in improving the quality and efficiency of online medical services, and belongs to the task of intent recognition. Joint entity recognition and intent recognition perform better than single task models. Currently, most publicly available medical text intent recognition datasets lack entity annotation, and manual annotation of these entities requires a lot of time and manpower. To solve this problem, this paper proposes a medical text classification model, bidirectional encoder representation based on transformer-recurrent convolutional neural network-entity-label-semantics (BRELS), which integrates medical entity label semantics. This model firstly utilizes an adaptive fusion mechanism to absorb prior knowledge of medical entity labels, achieving local feature enhancement. Then in global feature extraction, a lightweight recurrent convolutional neural network (LRCNN) is used to suppress parameter growth while preserving the original semantics of the text. The ablation and comparison experiments are conducted on three public medical text intent recognition datasets to validate the performance of the model. The results show that F1 score reaches 87.34%, 81.71%, and 77.74% on each dataset, respectively. The results in this paper show that the BRELS model can effectively identify and understand medical terminology, thereby effectively identifying users’ intentions, which can improve the quality and efficiency of online medical services.

1.	中国互联网络信息中心. 第52次《中国互联网络发展状况统计报告》. (2023-08-28) [2024-07-30]. https://cnnic.cn/n4/2023/0828/c199-10830.html.
2.	黄涌, 葸娟霞, 关成斌. 基于BERT-BiGRU模型的智慧医疗问答系统. 软件工程, 2024, 27(3): 11-14,25.
3.	Cilar L B, Lucija G, Gregor S. Review of artificial intelligence-based question-answering systems in healthcare. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2023, 13(2): e1487.
4.	Hu Y, Han G, Liu X, et al. Design and implementation of a medical question and answer system based on deep learning. Mathematical Problems in Engineering, 2022, 2022(1): 4600404.
5.	郑承宇, 王新, 王婷, 等. 基于ALBERT-TextCNN模型的多标签医疗文本分类方法. 山东大学学报(理学版), 2022, 57(4): 21-29.
6.	Yao L, Mao C, Luo Y. Clinical text classification with rule-based features and knowledge-guided convolutional neural networks. BMC Medical Informatics and Decision Making, 2019, 19(S3): 71.
7.	Patrick J, Li M. An ontology for clinical questions about the contents of patient notes. Journal of Biomedical Informatics, 2012, 45(2): 292-306.
8.	Roberts K, Rodriguez L, Shooshan S E, et al. Resource classification for medical questions. AMIA Annual Symposium Proceedings. 2017, 2016: 1040-1049.
9.	Ibrahim M A, Ghani Khan M U, Mehmood F, et al. GHS-NET a generic hybridized shallow neural network for multi-label biomedical text classification. Journal of Biomedical Informatics, 2021, 116: 103699.
10.	杜永兴, 孙彤彤, 周李涌, 等. 基于Vocab-GCN的中文医疗文本分类方法. 传感器与微系统, 2023, 42(8): 152-156.
11.	Lu H, Ehwerhemuepha L, Rakovski C. A comparative study on deep learning models for text classification of unstructured medical notes with various levels of class imbalance. BMC Medical Research Methodology, 2022, 22(1): 181.
12.	许浪, 李代伟, 张海清, 等. 基于神经网络的医疗文本分类研究. 计算机工程与科学, 2023, 45(6): 1116-1122.
13.	Bagui S, Nandi D, Bagui S, et al. Machine learning and deep learning for phishing email classification using one-hot encoding. Journal of Computer Science, 2021, 17: 610-623.
14.	HaCohen-Kerner Y, Miller D, Yigal Y. The influence of preprocessing on text classification using a bag-of-words representation. PloS One, 2020, 15(5): e0232525.
15.	Chen K, Zhang Z, Long J, et al. Turning from TF-IDF to TF-IGM for term weighting in text classification. Expert Systems with Applications, 2016, 66: 245-260.
16.	Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 2013, 26: 3111-3119.
17.	Pennington J, Socher R, Manning C D. Glove: global vectors for word representation// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg: Association for Computational Linguistics, 2014: 1532-1543.
18.	Joulin A, Grave E, Bojanowski P, et al. Bag of tricks for efficient text classification. arXiv preprint, 2016, arXiv: 1607.01759.
19.	Devlin J, Chang M W, Lee K, et al. Bert: pre-training of deep bidirectional transformers for language understanding// Proceedings of NAACL-HLT 2019. Minneapolis: Association for Computational Linguistics, 2019: 4171-4186.
20.	Liu Y, Ott M, Goyal N, et al. Roberta: A robustly optimized bert pretraining approach. arXiv preprint, 2019, arXiv: 1907.11692.
21.	Lan Z, Chen M, Goodman S, et al. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint, 2019, arXiv: 1909.11942.
22.	Yang Z, Dai Z, Yang Y, et al. Xlnet: generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems, 2019, 32.
23.	郑承宇, 王新, 王婷, 等. 基于迁移学习和集成学习的医疗文本分类. 计算机技术与发展, 2022, 32(4): 28-33.
24.	李博涵, 向宇轩, 封顶, 等. 融合知识感知与双重注意力的短文本分类模型. 软件学报, 2022, 33(10): 3565-3581.
25.	王宇亮, 杨观赐, 罗可欣. 基于意图—槽位注意机制的医疗咨询意图理解与实体抽取算法. 计算机应用研究, 2023, 40(5): 1402-1409.
26.	魏鹏飞, 曾碧, 汪明慧, 等. 基于深度学习的口语理解联合建模算法综述. 软件学报, 2022, 33(11): 4192-4216.
27.	王鲁, 刘瑞麟, 黄敬中, 等. 基于CNN-Transformer的农作物病虫害知识问答意图识别与槽位填充联合模型. 农业工程学报, 2024, 40(13): 156-162.
28.	Zhangn N, Che M, Bi Z, et al. Cblue: A chinese biomedical language understanding evaluation benchmark// Annual Meeting of the Association for Computational Linguistics 2022 (ACL2022). Dublin: Association for Computational Linguistics, 2022: 7888-7915.
29.	尹宝生, 周澎. 融合标签知识的中文医学命名实体识别. 计算机科学, 2024, 51(S1): 140-146.
30.	韩世依, 张钰晖, 马云山, 等. 清华大学自然语言处理与社会人文计算实验室. THUOCL: 清华大学开放中文词. (2017-01-20) [2024-12-02]. https://github.com/thunlp/THUOCL.
31.	Lai S, Xu L, Liu K, et al. Recurrent convolutional neural networks for text classification// Proceedings of the AAAI conference on artificial intelligence 2015 (AAAI2015). Austin: Association for the Advancement of Artificial Intelligence, 2015, 29(1): 2267-2273.
32.	第四届神经计算在先进应用中的国际会议. 中文糖尿病数据集. (2023-04-04) [2024-12-02]. https://github.com/yuni-bobo/Chinese-DQC.
33.	尹康平, 阿里夸克. 医疗搜索检索词意图分类数据集. (2021-03-22) [2024-12-02]. https://tianchi.aliyun.com/dataset/95414.
34.	苏向东机器学习小组, 内蒙古大学. 中文医学意图数据集. (2021-02-26) [2024-12-02]. https://tianchi.aliyun.com/dataset/92109.
35.	王乾, 曾诚, 何鹏, 等. 基于RoBERTa-RCNN和注意力池化的新闻主题文本分类. 郑州大学学报(理学版), 2024, 56(2): 43-50.
36.	Xiong J Z, Feng M Y, Wang X S, et al. Decoding sentiments: Enhancing covid-19 tweet analysis through bert-rcnn fusion. Journal of Theory and Practice of Engineering Science, 2024, 4(1): 86-93.

1. 中国互联网络信息中心. 第52次《中国互联网络发展状况统计报告》. (2023-08-28) [2024-07-30]. https://cnnic.cn/n4/2023/0828/c199-10830.html.
2. 黄涌, 葸娟霞, 关成斌. 基于BERT-BiGRU模型的智慧医疗问答系统. 软件工程, 2024, 27(3): 11-14,25.
3. Cilar L B, Lucija G, Gregor S. Review of artificial intelligence-based question-answering systems in healthcare. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2023, 13(2): e1487.
4. Hu Y, Han G, Liu X, et al. Design and implementation of a medical question and answer system based on deep learning. Mathematical Problems in Engineering, 2022, 2022(1): 4600404.
5. 郑承宇, 王新, 王婷, 等. 基于ALBERT-TextCNN模型的多标签医疗文本分类方法. 山东大学学报(理学版), 2022, 57(4): 21-29.
6. Yao L, Mao C, Luo Y. Clinical text classification with rule-based features and knowledge-guided convolutional neural networks. BMC Medical Informatics and Decision Making, 2019, 19(S3): 71.
7. Patrick J, Li M. An ontology for clinical questions about the contents of patient notes. Journal of Biomedical Informatics, 2012, 45(2): 292-306.
8. Roberts K, Rodriguez L, Shooshan S E, et al. Resource classification for medical questions. AMIA Annual Symposium Proceedings. 2017, 2016: 1040-1049.
9. Ibrahim M A, Ghani Khan M U, Mehmood F, et al. GHS-NET a generic hybridized shallow neural network for multi-label biomedical text classification. Journal of Biomedical Informatics, 2021, 116: 103699.
10. 杜永兴, 孙彤彤, 周李涌, 等. 基于Vocab-GCN的中文医疗文本分类方法. 传感器与微系统, 2023, 42(8): 152-156.
11. Lu H, Ehwerhemuepha L, Rakovski C. A comparative study on deep learning models for text classification of unstructured medical notes with various levels of class imbalance. BMC Medical Research Methodology, 2022, 22(1): 181.
12. 许浪, 李代伟, 张海清, 等. 基于神经网络的医疗文本分类研究. 计算机工程与科学, 2023, 45(6): 1116-1122.
13. Bagui S, Nandi D, Bagui S, et al. Machine learning and deep learning for phishing email classification using one-hot encoding. Journal of Computer Science, 2021, 17: 610-623.
14. HaCohen-Kerner Y, Miller D, Yigal Y. The influence of preprocessing on text classification using a bag-of-words representation. PloS One, 2020, 15(5): e0232525.
15. Chen K, Zhang Z, Long J, et al. Turning from TF-IDF to TF-IGM for term weighting in text classification. Expert Systems with Applications, 2016, 66: 245-260.
16. Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, 2013, 26: 3111-3119.
17. Pennington J, Socher R, Manning C D. Glove: global vectors for word representation// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg: Association for Computational Linguistics, 2014: 1532-1543.
18. Joulin A, Grave E, Bojanowski P, et al. Bag of tricks for efficient text classification. arXiv preprint, 2016, arXiv: 1607.01759.
19. Devlin J, Chang M W, Lee K, et al. Bert: pre-training of deep bidirectional transformers for language understanding// Proceedings of NAACL-HLT 2019. Minneapolis: Association for Computational Linguistics, 2019: 4171-4186.
20. Liu Y, Ott M, Goyal N, et al. Roberta: A robustly optimized bert pretraining approach. arXiv preprint, 2019, arXiv: 1907.11692.
21. Lan Z, Chen M, Goodman S, et al. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint, 2019, arXiv: 1909.11942.
22. Yang Z, Dai Z, Yang Y, et al. Xlnet: generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems, 2019, 32.
23. 郑承宇, 王新, 王婷, 等. 基于迁移学习和集成学习的医疗文本分类. 计算机技术与发展, 2022, 32(4): 28-33.
24. 李博涵, 向宇轩, 封顶, 等. 融合知识感知与双重注意力的短文本分类模型. 软件学报, 2022, 33(10): 3565-3581.
25. 王宇亮, 杨观赐, 罗可欣. 基于意图—槽位注意机制的医疗咨询意图理解与实体抽取算法. 计算机应用研究, 2023, 40(5): 1402-1409.
26. 魏鹏飞, 曾碧, 汪明慧, 等. 基于深度学习的口语理解联合建模算法综述. 软件学报, 2022, 33(11): 4192-4216.
27. 王鲁, 刘瑞麟, 黄敬中, 等. 基于CNN-Transformer的农作物病虫害知识问答意图识别与槽位填充联合模型. 农业工程学报, 2024, 40(13): 156-162.
28. Zhangn N, Che M, Bi Z, et al. Cblue: A chinese biomedical language understanding evaluation benchmark// Annual Meeting of the Association for Computational Linguistics 2022 (ACL2022). Dublin: Association for Computational Linguistics, 2022: 7888-7915.
29. 尹宝生, 周澎. 融合标签知识的中文医学命名实体识别. 计算机科学, 2024, 51(S1): 140-146.
30. 韩世依, 张钰晖, 马云山, 等. 清华大学自然语言处理与社会人文计算实验室. THUOCL: 清华大学开放中文词. (2017-01-20) [2024-12-02]. https://github.com/thunlp/THUOCL.
31. Lai S, Xu L, Liu K, et al. Recurrent convolutional neural networks for text classification// Proceedings of the AAAI conference on artificial intelligence 2015 (AAAI2015). Austin: Association for the Advancement of Artificial Intelligence, 2015, 29(1): 2267-2273.
32. 第四届神经计算在先进应用中的国际会议. 中文糖尿病数据集. (2023-04-04) [2024-12-02]. https://github.com/yuni-bobo/Chinese-DQC.
33. 尹康平, 阿里夸克. 医疗搜索检索词意图分类数据集. (2021-03-22) [2024-12-02]. https://tianchi.aliyun.com/dataset/95414.
34. 苏向东机器学习小组, 内蒙古大学. 中文医学意图数据集. (2021-02-26) [2024-12-02]. https://tianchi.aliyun.com/dataset/92109.
35. 王乾, 曾诚, 何鹏, 等. 基于RoBERTa-RCNN和注意力池化的新闻主题文本分类. 郑州大学学报(理学版), 2024, 56(2): 43-50.
36. Xiong J Z, Feng M Y, Wang X S, et al. Decoding sentiments: Enhancing covid-19 tweet analysis through bert-rcnn fusion. Journal of Theory and Practice of Engineering Science, 2024, 4(1): 86-93.

Journal of Biomedical Engineering

Latest ArticlesMedical text classification model integrating medical entity label semantics

Abstract Full text Figures/Tables Video References Cited by

Format

Content