Research on exudate segmentation method for retinal fundus images based on deep learning_Chinese Journal of Ocular Fundus Diseases

Authors：

Deng Jianzhi ^1,2 , Guo Yongping ² , Zhou Yuehan ³ ,  Xiong Bin ¹

1. College of Earth Sciences, Guilin University of Technology, Guilin 541004, China;
2. College of Physics and Electronic Information Engineering, Guilin University of Technology, Guilin 541004, China;
3. College of Pharmacy, Guilin Medical University, Guilin 541004, China;

Corresponding author：

Xiong Bin, Email: xiongbin@glut.edu.cn

Keywords：

Segmentation of retinal exudation; U-shaped network; Residual structure; Context extraction; Convolutional triple attention

DOI：

10.3760/cma.j.cn511434-20231221-00500

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To automatically segment diabetic retinal exudation features from deep learning color fundus images. Methods An applied study. The method of this study is based on the U-shaped network model of the Indian Diabetic Retinopathy Image Dataset (IDRID) dataset, introduces deep residual convolution into the encoding and decoding stages, which can effectively extract seepage depth features, solve overfitting and feature interference problems, and improve the model's feature expression ability and lightweight performance. In addition, by introducing an improved context extraction module, the model can capture a wider range of feature information, enhance the perception ability of retinal lesions, and perform excellently in capturing small details and blurred edges. Finally, the introduction of convolutional triple attention mechanism allows the model to automatically learn feature weights, focus on important features, and extract useful information from multiple scales. Accuracy, recall, Dice coefficient, accuracy and sensitivity were used to evaluate the ability of the model to detect and segment the automatic retinal exudation features of diabetic patients in color fundus images. Results After applying this method, the accuracy, recall, dice coefficient, accuracy and sensitivity of the improved model on the IDRID dataset reached 81.56%, 99.54%, 69.32%, 65.36% and 78.33%, respectively. Compared with the original model, the accuracy and Dice index of the improved model are increased by 2.35% , 3.35% respectively. Conclusion The segmentation method based on U-shaped network can automatically detect and segment the retinal exudation features of fundus images of diabetic patients, which is of great significance for assisting doctors to diagnose diseases more accurately.

Citation： Deng Jianzhi, Guo Yongping, Zhou Yuehan, Xiong Bin. Research on exudate segmentation method for retinal fundus images based on deep learning. Chinese Journal of Ocular Fundus Diseases, 2024, 40(7): 518-525. doi: 10.3760/cma.j.cn511434-20231221-00500 Copy

1.	张健, 杨波. 糖尿病视网膜病变发病机制及其药物治疗研究的进展[J]. 心血管康复医学杂志, 2016, 25(3): 339-341. DOI: 10.3969/j.issn.1008-0074.2016.03.31.Zhang J, Yang B. Research progress for pathogenesis of diabetic retinopathy and its drug treatment[J]. Chin J Cardiovasc Rehabil Med, 2016, 25(3): 339-341. DOI: 10.3969/j.issn.1008-0074.2016.03.31.
2.	Bressler SB, Beaulieu WT, Glassman AR, et al. Factors associated with worsening proliferative diabetic retinopathy in eyes treated with panretinal photocoagulation or Ranibizumab[J]. Ophthalmology, 2017, 124(4): 431-439. DOI: 10.1016/j.ophtha.2016.12.005.
3.	袁明霞. 糖尿病视网膜病变的早期诊断与综合防治[J]. 中国医刊, 2018, 53(4): 358-361. DOI: 10.3969/j.issn.1008-1070.2018.04.004.Yuan MX. Early diagnosis and comprehensive prevention of diabetic retinopathy[J]. Chinese Journal of Medicine, 2018, 53(4): 358-361. DOI: 10.3969/j.issn.1008-1070.2018.04.004.
4.	张宝莹. 糖尿病视网膜病变的临床表现及其诊断和防治[J]. 糖尿病新世界, 2016, 19(22): 121-122. DOI: 10.16658/j.cnki.1672-4062.2016.22.121.Zhang BY. Clinical manifestations, diagnosis, prevention and treatment of diabetic retinopathy[J]. Diabetes New World, 2016, 19(22): 121-122. DOI: 10.16658/j.cnki.1672-4062.2016.22.121.
5.	马晓宇, 张力, 毕燕龙. 人工智能在糖尿病视网膜病变领域的研究进展[J]. 国际眼科杂志, 2022, 22(11): 1818-1821. DOI: 10.3980/j.issn.1672-5123.2022.11.11.Ma XY, Zhang L, Bi YL. Research progress on artificial intelligence in diabetic retinopathy[J]. Int Eye Sci, 2022, 22(11): 1818-1821. DOI: 10.3980/j.issn.1672-5123.2022.11.11.
6.	Farooq MS, Arooj A, Alroobaea R, et al. Untangling computer-aided diagnostic system for screening diabetic retinopathy based on deep learning techniques[J]. Sensors, 2022, 22(5): 1803. DOI: 10.3390/s22051803.
7.	Porwal P, Pachade S, Kamble R, et al. Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research[J]. Data, 2018, 3(3): 25. DOI: 10.3390/data3030025.
8.	Chidambaram N, Vijayan D. Detection of exudates in diabetic retinopathy[J]. IEEE, 2018, 2018: 660-664. DOI: 10.1109/icacci.2018.8554923.
9.	Wang H, Cao P, Yang J, et al. MCA-UNet: multi-scale cross co-attentional U-Net for automatic medical image segmentation[J]. Health Inf Sci Syst, 2023, 11(1): 10. DOI: 10.1007/s13755-022-00209-4.
10.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[J]. IEEE, 2016: 770-778. DOI: 10.1109/cvpr.2016.90.
11.	陈人和, 赖振意, 钱育蓉. 改进的生成对抗网络图像去噪算法[J]. 计算机工程与应用, 2021, 57(5): 168-172. DOI: 10.3778/j.issn.1002-8331.2003-0336.Chen RH, Lai ZY, Qian YR. Improved image denoising generative adversarial network algorithm[J]. Comput Engin Appl, 2021, 57(5): 168-172. DOI: 10.3778/j.issn.1002-8331.2003-0336.
12.	Howard AG, Zhu M, Chen B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications[J]. 2017, 27(7): 9. DOI: 10.48550/arXiv.1704.04861.
13.	Wang Q, Wu B, Zhu P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[J]. IEEE, 2020, 2020: 11534-11542. DOI: 10.1109/cvpr42600.2020.01155.
14.	Gu Z, Cheng J, Fu H, et al. CE-Net: context encoder network for 2d medical image segmentation[J]. IEEE transactions on medical imaging, 2019, 38(10): 2281-2292. DOI: 10.1109/tmi.2019.2903562.
15.	Misra D, Nalamada T, Arasanipalai AU, et al. Rotate to attend: convolutional triplet attention module[J]. IEEE, 2021, 2021: 3139-3148. DOI: 10.1109/wacv48630.2021.00318.
16.	Zhang F, Li S, Deng J. Unsupervised domain adaptation with shape constraint and triple attention for joint optic disc and cup segmentation[J/OL]. Sensors, 2022, 22(22): 8748[2022-11-12]. https://pubmed.ncbi.nlm.nih.gov/36433345/. DOI: 10.3390/s22228748.
17.	Oktay O, Schlemper J, Folgoc LL, et al. Attention U-net: learning where to look for the pancreas[J]. Comput Vis Pat Recog, 2018, 2018: 1. DOI: 10.48550/arXiv.1804.03999.
18.	Zhou Z, Siddiquee MMR, Tajbakhsh N, et al. Unet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE T Med Imaging, 2019, 39(6): 1856-1867. DOI: 10.1109/tmi.2019.2959609.
19.	Guo S, Li T, Kang H, et al. L-Seg: an end-to-end unified framework for multi-lesion segmentation of fundus images[J]. Neurocomputing, 2019, 349: 52-63. DOI: 10.1016/j.neucom.2019.04.019.
20.	Ibtehaz N, Rahman MS. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neur Net, 2020, 121: 74-87. DOI: 10.1016/j.neunet.2019.08.025.
21.	Cao H, Wang Y, Chen J, et al. Swin-unet: UNet-like pure transformer for medical image segmentation[J]. IEEE, 2022, 2022: 205-218. DOI: 10.1007/978-3-031-25066-8_9.
22.	龙胜春, 胡安特, 陈芝清. 基于混合多特征的微动脉瘤检测方法[J]. 中国生物医学工程学报, 2022, 41(5): 626-630. DOI: 10.3969/j.issn.0258-8021.2022.05.012.Long SC, Hu AT, Chen ZQ. Microaneurysm detection method based on mixed multi-features[J]. Chinese Journal of Biomedical Engineering, 2022, 41(5): 626-630. DOI: 10.3969/j.issn.0258-8021.2022.05.012.
23.	周梦颖, 杨晓宇, 邱媛, 等. 一种眼底图像出血点的检测算法[J]. 北京生物医学工程, 2022, 41(3): 255-259. DOI: 10.3969/j.issn.1002-3208.2022.03.006.Zhou MY, Yang XY, Qiu Y, et al. Algorithm of hemorrhages in fundus images[J]. Beijing Biomedical Engineering, 2022, 41(3): 255-259. DOI: 10.3969/j.issn.1002-3208.2022.03.006.
24.	Jaafar HF, Al-Libawy H, Al-Gayem QK. Automated approach for extraction of microaneurysms and hemorrhages in retinal fundus images[J]. Int J Intell Engineer Syst, 2020, 13(5): 210. DOI: 10.22266/ijies2020.1031.20.
25.	Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: a review[J/OL]. Med Image Anal, 2019, 58: 101552[2019-08-31]. https://pubmed.ncbi.nlm.nih.gov/31521965/. DOI: 10.1016/j.media.2019.101552.
26.	Liu Q, Liu H, Zhao Y, et al. Dual-branch network with dual-sampling modulated dice loss for hard exudate segmentation in color fundus images[J]. IEEE J Biomed Health Inform, 2021, 26(3): 1091-1102. DOI: 10.1109/jbhi.2021.3108169.
27.	Milletari F, Navab N, Ahmadi SA. V-net: fully convolutional neural networks for volumetric medical image segmentation[J]. IEEE, 2016: 565-571. DOI: 10.1109/3DV.2016.79.
28.	Shujaat M, Aslam N, Noreen I, et al. Intelligent and integrated framework for exudate detection in retinal fundus images[J]. Intell Autom Soft Co, 2021, 30(2): 663-672. DOI: 10.32604/iasc.2021.019194.
29.	Han X, Zhong Y, Cao L, et al. Pre-trained alexnet architecture with pyramid pooling and supervision for high spatial resolution remote sensing image scene classification[J]. Remote Sens, 2017, 9(8): 848. DOI: 10.3390/rs9080848.
30.	Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[J]. IEEE, 2015, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
31.	杨知桥, 张莹, 王新杰, 等. 改进U型网络在视网膜病变检测中的应用研究[J]. 计算机科学与探索, 2022, 16(8): 1877. DOI: 10.3778/j.issn.1673-9418.2012011.Yang ZQ, Zhang Y, Wang XJ, et al. Application research of improved U-shaped network in detection of retinopathy[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1877. DOI: 10.3778/j.issn.1673-9418.2012011.
32.	Fu Y, Zhang G, Lu X, et al. RMCA U-net: hard exudates segmentation for retinal fundus images[J/OL]. Expert Syst Appl, 2023, 234: 120987[2023-08-23]. https://www.sciencedirect.com/science/article/pii/S0957417423014896?via%3Dihub. DOI: 10.1016/j.eswa.2023.120987.

1. 张健, 杨波. 糖尿病视网膜病变发病机制及其药物治疗研究的进展[J]. 心血管康复医学杂志, 2016, 25(3): 339-341. DOI: 10.3969/j.issn.1008-0074.2016.03.31.Zhang J, Yang B. Research progress for pathogenesis of diabetic retinopathy and its drug treatment[J]. Chin J Cardiovasc Rehabil Med, 2016, 25(3): 339-341. DOI: 10.3969/j.issn.1008-0074.2016.03.31.
2. Bressler SB, Beaulieu WT, Glassman AR, et al. Factors associated with worsening proliferative diabetic retinopathy in eyes treated with panretinal photocoagulation or Ranibizumab[J]. Ophthalmology, 2017, 124(4): 431-439. DOI: 10.1016/j.ophtha.2016.12.005.
3. 袁明霞. 糖尿病视网膜病变的早期诊断与综合防治[J]. 中国医刊, 2018, 53(4): 358-361. DOI: 10.3969/j.issn.1008-1070.2018.04.004.Yuan MX. Early diagnosis and comprehensive prevention of diabetic retinopathy[J]. Chinese Journal of Medicine, 2018, 53(4): 358-361. DOI: 10.3969/j.issn.1008-1070.2018.04.004.
4. 张宝莹. 糖尿病视网膜病变的临床表现及其诊断和防治[J]. 糖尿病新世界, 2016, 19(22): 121-122. DOI: 10.16658/j.cnki.1672-4062.2016.22.121.Zhang BY. Clinical manifestations, diagnosis, prevention and treatment of diabetic retinopathy[J]. Diabetes New World, 2016, 19(22): 121-122. DOI: 10.16658/j.cnki.1672-4062.2016.22.121.
5. 马晓宇, 张力, 毕燕龙. 人工智能在糖尿病视网膜病变领域的研究进展[J]. 国际眼科杂志, 2022, 22(11): 1818-1821. DOI: 10.3980/j.issn.1672-5123.2022.11.11.Ma XY, Zhang L, Bi YL. Research progress on artificial intelligence in diabetic retinopathy[J]. Int Eye Sci, 2022, 22(11): 1818-1821. DOI: 10.3980/j.issn.1672-5123.2022.11.11.
6. Farooq MS, Arooj A, Alroobaea R, et al. Untangling computer-aided diagnostic system for screening diabetic retinopathy based on deep learning techniques[J]. Sensors, 2022, 22(5): 1803. DOI: 10.3390/s22051803.
7. Porwal P, Pachade S, Kamble R, et al. Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research[J]. Data, 2018, 3(3): 25. DOI: 10.3390/data3030025.
8. Chidambaram N, Vijayan D. Detection of exudates in diabetic retinopathy[J]. IEEE, 2018, 2018: 660-664. DOI: 10.1109/icacci.2018.8554923.
9. Wang H, Cao P, Yang J, et al. MCA-UNet: multi-scale cross co-attentional U-Net for automatic medical image segmentation[J]. Health Inf Sci Syst, 2023, 11(1): 10. DOI: 10.1007/s13755-022-00209-4.
10. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[J]. IEEE, 2016: 770-778. DOI: 10.1109/cvpr.2016.90.
11. 陈人和, 赖振意, 钱育蓉. 改进的生成对抗网络图像去噪算法[J]. 计算机工程与应用, 2021, 57(5): 168-172. DOI: 10.3778/j.issn.1002-8331.2003-0336.Chen RH, Lai ZY, Qian YR. Improved image denoising generative adversarial network algorithm[J]. Comput Engin Appl, 2021, 57(5): 168-172. DOI: 10.3778/j.issn.1002-8331.2003-0336.
12. Howard AG, Zhu M, Chen B, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications[J]. 2017, 27(7): 9. DOI: 10.48550/arXiv.1704.04861.
13. Wang Q, Wu B, Zhu P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[J]. IEEE, 2020, 2020: 11534-11542. DOI: 10.1109/cvpr42600.2020.01155.
14. Gu Z, Cheng J, Fu H, et al. CE-Net: context encoder network for 2d medical image segmentation[J]. IEEE transactions on medical imaging, 2019, 38(10): 2281-2292. DOI: 10.1109/tmi.2019.2903562.
15. Misra D, Nalamada T, Arasanipalai AU, et al. Rotate to attend: convolutional triplet attention module[J]. IEEE, 2021, 2021: 3139-3148. DOI: 10.1109/wacv48630.2021.00318.
16. Zhang F, Li S, Deng J. Unsupervised domain adaptation with shape constraint and triple attention for joint optic disc and cup segmentation[J/OL]. Sensors, 2022, 22(22): 8748[2022-11-12]. https://pubmed.ncbi.nlm.nih.gov/36433345/. DOI: 10.3390/s22228748.
17. Oktay O, Schlemper J, Folgoc LL, et al. Attention U-net: learning where to look for the pancreas[J]. Comput Vis Pat Recog, 2018, 2018: 1. DOI: 10.48550/arXiv.1804.03999.
18. Zhou Z, Siddiquee MMR, Tajbakhsh N, et al. Unet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE T Med Imaging, 2019, 39(6): 1856-1867. DOI: 10.1109/tmi.2019.2959609.
19. Guo S, Li T, Kang H, et al. L-Seg: an end-to-end unified framework for multi-lesion segmentation of fundus images[J]. Neurocomputing, 2019, 349: 52-63. DOI: 10.1016/j.neucom.2019.04.019.
20. Ibtehaz N, Rahman MS. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neur Net, 2020, 121: 74-87. DOI: 10.1016/j.neunet.2019.08.025.
21. Cao H, Wang Y, Chen J, et al. Swin-unet: UNet-like pure transformer for medical image segmentation[J]. IEEE, 2022, 2022: 205-218. DOI: 10.1007/978-3-031-25066-8_9.
22. 龙胜春, 胡安特, 陈芝清. 基于混合多特征的微动脉瘤检测方法[J]. 中国生物医学工程学报, 2022, 41(5): 626-630. DOI: 10.3969/j.issn.0258-8021.2022.05.012.Long SC, Hu AT, Chen ZQ. Microaneurysm detection method based on mixed multi-features[J]. Chinese Journal of Biomedical Engineering, 2022, 41(5): 626-630. DOI: 10.3969/j.issn.0258-8021.2022.05.012.
23. 周梦颖, 杨晓宇, 邱媛, 等. 一种眼底图像出血点的检测算法[J]. 北京生物医学工程, 2022, 41(3): 255-259. DOI: 10.3969/j.issn.1002-3208.2022.03.006.Zhou MY, Yang XY, Qiu Y, et al. Algorithm of hemorrhages in fundus images[J]. Beijing Biomedical Engineering, 2022, 41(3): 255-259. DOI: 10.3969/j.issn.1002-3208.2022.03.006.
24. Jaafar HF, Al-Libawy H, Al-Gayem QK. Automated approach for extraction of microaneurysms and hemorrhages in retinal fundus images[J]. Int J Intell Engineer Syst, 2020, 13(5): 210. DOI: 10.22266/ijies2020.1031.20.
25. Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: a review[J/OL]. Med Image Anal, 2019, 58: 101552[2019-08-31]. https://pubmed.ncbi.nlm.nih.gov/31521965/. DOI: 10.1016/j.media.2019.101552.
26. Liu Q, Liu H, Zhao Y, et al. Dual-branch network with dual-sampling modulated dice loss for hard exudate segmentation in color fundus images[J]. IEEE J Biomed Health Inform, 2021, 26(3): 1091-1102. DOI: 10.1109/jbhi.2021.3108169.
27. Milletari F, Navab N, Ahmadi SA. V-net: fully convolutional neural networks for volumetric medical image segmentation[J]. IEEE, 2016: 565-571. DOI: 10.1109/3DV.2016.79.
28. Shujaat M, Aslam N, Noreen I, et al. Intelligent and integrated framework for exudate detection in retinal fundus images[J]. Intell Autom Soft Co, 2021, 30(2): 663-672. DOI: 10.32604/iasc.2021.019194.
29. Han X, Zhong Y, Cao L, et al. Pre-trained alexnet architecture with pyramid pooling and supervision for high spatial resolution remote sensing image scene classification[J]. Remote Sens, 2017, 9(8): 848. DOI: 10.3390/rs9080848.
30. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[J]. IEEE, 2015, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
31. 杨知桥, 张莹, 王新杰, 等. 改进U型网络在视网膜病变检测中的应用研究[J]. 计算机科学与探索, 2022, 16(8): 1877. DOI: 10.3778/j.issn.1673-9418.2012011.Yang ZQ, Zhang Y, Wang XJ, et al. Application research of improved U-shaped network in detection of retinopathy[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1877. DOI: 10.3778/j.issn.1673-9418.2012011.
32. Fu Y, Zhang G, Lu X, et al. RMCA U-net: hard exudates segmentation for retinal fundus images[J/OL]. Expert Syst Appl, 2023, 234: 120987[2023-08-23]. https://www.sciencedirect.com/science/article/pii/S0957417423014896?via%3Dihub. DOI: 10.1016/j.eswa.2023.120987.

Chinese Journal of Ocular Fundus Diseases

Research on exudate segmentation method for retinal fundus images based on deep learning

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content