Research on exudate segmentation method for retinal fundus images based on deep learning_Chinese Journal of Ocular Fundus Diseases

Authors：

Deng Jianzhi ^1,3 , Guo Yongping ³ , Zhou Yuehan ² ,  Xiong Bin ¹

1. College of Earth Sciences, Guilin University of Technology, Guilin 541004, China;
2. College of Pharmacy, Guilin Medical University, Guilin 541004, China;
3. College of Physics and Electronic Information Engineering, Guilin University of Technology, Guilin 541004, China;

Corresponding author：

Xiong Bin, Email: xiongbin@glut.edu.cn

Keywords：

Segmentation of retinal exudation; Residual structure; Context extraction; Convolutional triple attention; Review

DOI：

10.3760/cma.j.cn511434-20231221-00500

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Fundus exudation, as one of the main characteristics of eye diseases, is closely related to various eye diseases. However, due to its low contrast and blurred boundaries in medical images, retinal exudation is easily overlooked or confused. This paper proposes an improved U-shaped network model for segmentation of diabetic retinal exudation. This model adopts deep residual convolution in the encoding and decoding stages, which can effectively extract deep seepage features, solve overfitting and feature interference problems, and improve the model's feature expression ability and lightweight performance. In addition, by introducing an improved context extraction module, the model can capture a wider range of feature information, enhance the perception ability of retinal lesions, and perform excellently in capturing small details and blurred edges. Finally, the introduction of Convolutional Triple Attention Mechanism allows the model to automatically learn feature weights, focus on important features, and extract useful information from multiple scales. This method improves the efficiency of computer-aided detection and segmentation of retinal exudates. It not only helps doctors overcome the subjectivity of diagnosis to a certain extent through quantitative indicators, but also provides useful reference and inspiration for the auxiliary diagnosis of other similar diseases, which has important research significance.

1.	张健, 杨波. 糖尿病视网膜病变发病机制及其药物治疗研究的进展[J]. 心血管康复医学杂志, 2016, 25(3): 339-341. DOI: 10.3969/j.issn.1008-0074.2016.03.31.Zhang J, Yang B. Research progress for pathogenesis of diabetic retinopathy and its drug treatment[J]. Chin J Cardiovasc Rehabil Med, 2016, 25(3): 339-341. DOI: 10.3969/j.issn.1008-0074.2016.03.31.
2.	Bressler SB, Beaulieu WT, Glassman AR, et al. Factors associated with worsening proliferative diabetic retinopathy in eyes treated with panretinal photocoagulation or Ranibizumab[J]. Ophthalmology, 2017, 124(4): 431-439. DOI: 10.1016/j.ophtha.2016.12.005.
3.	袁明霞. 糖尿病视网膜病变的早期诊断与综合防治[J]. 中国医刊, 2018, 53(4): 358-361. DOI: 10.3969/j.issn.1008-1070.2018.04.004.Yuan MX. Early diagnosis and comprehensive prevention of diabetic retinopathy[J]. Chinese Journal of Medicine, 2018, 53(4): 358-361. DOI: 10.3969/j.issn.1008-1070.2018.04.004.
4.	张宝莹. 糖尿病视网膜病变的临床表现及其诊断和防治[J]. 糖尿病新世界, 2016, 19(22): 121-122. DOI: 10.16658/j.cnki.1672-4062.2016.22.121.Zhang BY. Clinical manifestations, diagnosis, prevention and treatment of diabetic retinopathy[J]. Diabetes New World, 2016, 19(22): 121-122. DOI: 10.16658/j.cnki.1672-4062.2016.22.121.
5.	马晓宇, 张力, 毕燕龙. 人工智能在糖尿病视网膜病变领域的研究进展[J]. 国际眼科杂志, 2022, 22(11): 1818-1821. DOI: 10.3980/j.issn.1672-5123.2022.11.11.Ma XY, Zhang L, Bi YL. Research progress on artificial intelligence in diabetic retinopathy[J]. Int Eye Sci, 2022, 22(11): 1818-1821. DOI: 10.3980/j.issn.1672-5123.2022.11.11.
6.	Farooq M S, Arooj A, Alroobaea R, et al. Untangling computer-aided diagnostic system for screening diabetic retinopathy based on deep learning techniques[J]. Sensors, 2022, 22(5): 1803. DOI: 10.3390/s22051803.
7.	龙胜春, 胡安特, 陈芝清. 基于混合多特征的微动脉瘤检测方法[J]. 中国生物医学工程学报, 2022, 41(5): 626-630. DOI: 10.3969/j.issn.0258-8021.2022.05.012.Long SC, Hu AT, Chen ZQ. Microaneurysm detection method based on mixed multi-features[J]. Chinese Journal of Biomedical Engineering, 2022, 41(5): 626-630. DOI: 10.3969/j.issn.0258-8021.2022.05.012.
8.	周梦颖, 杨晓宇, 邱媛, 等. 一种眼底图像出血点的检测算法[J]. 北京生物医学工程, 2022, 41(3): 255-259. DOI: 10.3969/j.issn.1002-3208.2022.03.006.Zhou MY, Yang XY, Qiu Y, et al. Algorithm of hemorrhages in fundus images[J]. Beijing Biomedical Engineering, 2022, 41(3): 255-259. DOI: 10.3969/j.issn.1002-3208.2022.03.006.
9.	Jaafar HF, Al-Libawy H, Al-Gayem QK. Automated approach for extraction of microaneurysms and hemorrhages in retinal fundus images[J]. Int J Intell Engineer Syst, 2020, 13(5): 210. DOI: 10.22266/ijies2020.1031.20.
10.	Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: a review[J/OL]. Med Image Anal, 2019, 58: 101552[2019-08-31]. https://pubmed.ncbi.nlm.nih.gov/31521965/. DOI: 10.1016/j.media.2019.101552.
11.	Liu Q, Liu H, Zhao Y, et al. Dual-branch network with dual-sampling modulated dice loss for hard exudate segmentation in color fundus images[J]. IEEE J Biomed Health Inform, 2021, 26(3): 1091-1102. DOI: 10.1109/jbhi.2021.3108169.
12.	Milletari F, Navab N, Ahmadi SA. V-net: fully convolutional neural networks for volumetric medical image segmentation[J]. IEEE, 2016: 565-571. DOI: 10.1109/3DV.2016.79.
13.	Shujaat M, Aslam N, Noreen I, et al. Intelligent and integrated framework for exudate detection in retinal fundus images[J]. Intell Autom Soft Co, 2021, 30(2): 663-672. DOI: 10.32604/iasc.2021.019194.
14.	Han X, Zhong Y, Cao L, et al. Pre-trained alexnet architecture with pyramid pooling and supervision for high spatial resolution remote sensing image scene classification[J]. Remote Sens, 2017, 9(8): 848. DOI: 10.3390/rs9080848.
15.	杨知桥, 张莹, 王新杰, 等. 改进U型网络在视网膜病变检测中的应用研究[J]. 计算机科学与探索, 2022, 16(8): 1877. DOI: 10.3778/j.issn.1673-9418.2012011.Yang ZQ, Zhang Y, Wang XJ, et al. Application research of improved U-shaped network in detection of retinopathy[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1877. DOI: 10.3778/j.issn.1673-9418.2012011.
16.	Fu Y, Zhang G, Lu X, et al. RMCA U-net: hard exudates segmentation for retinal fundus images[J/OL]. Expert Syst Appl, 2023, 234: 120987[2023-08-23]. https://www.sciencedirect.com/science/article/pii/S0957417423014896?via%3Dihub. DOI: 10.1016/j.eswa.2023.120987.
17.	Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[J]. IEEE, 2015, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
18.	Wang H, Cao P, Yang J, et al. MCA-UNet: multi-scale cross co-attentional U-Net for automatic medical image segmentation[J]. Health Inf Sci Syst, 2023, 11(1): 10. DOI: 10.1007/s13755-022-00209-4.
19.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[J]. IEEE, 2016: 770-778. DOI: 10.1109/cvpr.2016.90.
20.	陈人和, 赖振意, 钱育蓉. 改进的生成对抗网络图像去噪算法[J]. 计算机工程与应用, 2021, 57(5): 168-172. DOI: 10.3778/j.issn.1002-8331.2003-0336.Chen RH, Lai ZY, Qian YR. Improved image denoising generative adversarial network algorithm[J]. Comput Engin Appl, 2021, 57(5): 168-172. DOI: 10.3778/j.issn.1002-8331.2003-0336.
21.	Howard AG, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. 2017, 27(7): 9. DOI: 10.48550/arXiv.1704.04861.
22.	Wang Q, Wu B, Zhu P, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks[J]. IEEE, 2020, 2020: 11534-11542. DOI: 10.1109/cvpr42600.2020.01155.
23.	Gu Z, Cheng J, Fu H, et al. CE-Net: context encoder network for 2d medical image segmentation[J]. IEEE T Med Imaging, 2019, 38(10): 2281-2292. DOI: 10.1109/tmi.2019.2903562.
24.	Misra D, Nalamada T, Arasanipalai AU, et al. Rotate to attend: convolutional triplet attention module[J]. IEEE, 2021, 2021: 3139-3148. DOI: 10.1109/wacv48630.2021.00318.
25.	Zhang F, Li S, Deng J. Unsupervised domain adaptation with shape constraint and triple attention for joint optic disc and cup segmentation[J/OL]. Sensors, 2022, 22(22): 8748[2022-11-12]. https://pubmed.ncbi.nlm.nih.gov/36433345/. DOI: 10.3390/s22228748.
26.	Porwal P, Pachade S, Kamble R, et al. Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research[J]. Data, 2018, 3(3): 25. DOI: 10.3390/data3030025.
27.	Chidambaram N, Vijayan D. Detection of exudates in diabetic retinopathy[J]. IEEE, 2018, 2018: 660-664. DOI: 10.1109/icacci.2018.8554923.
28.	Oktay O, Schlemper J, Folgoc LL, et al. Attention u-net: Learning where to look for the pancreas[J]. Comput Vis Pat Recog, 2018, 2018: 1. DOI: 10.48550/arXiv.1804.03999.
29.	Zhou Z, Siddiquee MMR, Tajbakhsh N, et al. Unet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE T Med Imaging, 2019, 39(6): 1856-1867. DOI: 10.1109/tmi.2019.2959609.
30.	Guo S, Li T, Kang H, et al. L-Seg: an end-to-end unified framework for multi-lesion segmentation of fundus images[J]. Neurocomputing, 2019, 349: 52-63. DOI: 10.1016/j.neucom.2019.04.019.
31.	Ibtehaz N, Rahman MS. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neur Net, 2020, 121: 74-87. DOI: 10.1016/j.neunet.2019.08.025.
32.	Cao H, Wang Y, Chen J, et al. Swin-unet: UNet-like pure transformer for medical image segmentation[J]. IEEE, 2022, 2022: 205-218. DOI: 10.1007/978-3-031-25066-8_9.

1. 张健, 杨波. 糖尿病视网膜病变发病机制及其药物治疗研究的进展[J]. 心血管康复医学杂志, 2016, 25(3): 339-341. DOI: 10.3969/j.issn.1008-0074.2016.03.31.Zhang J, Yang B. Research progress for pathogenesis of diabetic retinopathy and its drug treatment[J]. Chin J Cardiovasc Rehabil Med, 2016, 25(3): 339-341. DOI: 10.3969/j.issn.1008-0074.2016.03.31.
2. Bressler SB, Beaulieu WT, Glassman AR, et al. Factors associated with worsening proliferative diabetic retinopathy in eyes treated with panretinal photocoagulation or Ranibizumab[J]. Ophthalmology, 2017, 124(4): 431-439. DOI: 10.1016/j.ophtha.2016.12.005.
3. 袁明霞. 糖尿病视网膜病变的早期诊断与综合防治[J]. 中国医刊, 2018, 53(4): 358-361. DOI: 10.3969/j.issn.1008-1070.2018.04.004.Yuan MX. Early diagnosis and comprehensive prevention of diabetic retinopathy[J]. Chinese Journal of Medicine, 2018, 53(4): 358-361. DOI: 10.3969/j.issn.1008-1070.2018.04.004.
4. 张宝莹. 糖尿病视网膜病变的临床表现及其诊断和防治[J]. 糖尿病新世界, 2016, 19(22): 121-122. DOI: 10.16658/j.cnki.1672-4062.2016.22.121.Zhang BY. Clinical manifestations, diagnosis, prevention and treatment of diabetic retinopathy[J]. Diabetes New World, 2016, 19(22): 121-122. DOI: 10.16658/j.cnki.1672-4062.2016.22.121.
5. 马晓宇, 张力, 毕燕龙. 人工智能在糖尿病视网膜病变领域的研究进展[J]. 国际眼科杂志, 2022, 22(11): 1818-1821. DOI: 10.3980/j.issn.1672-5123.2022.11.11.Ma XY, Zhang L, Bi YL. Research progress on artificial intelligence in diabetic retinopathy[J]. Int Eye Sci, 2022, 22(11): 1818-1821. DOI: 10.3980/j.issn.1672-5123.2022.11.11.
6. Farooq M S, Arooj A, Alroobaea R, et al. Untangling computer-aided diagnostic system for screening diabetic retinopathy based on deep learning techniques[J]. Sensors, 2022, 22(5): 1803. DOI: 10.3390/s22051803.
7. 龙胜春, 胡安特, 陈芝清. 基于混合多特征的微动脉瘤检测方法[J]. 中国生物医学工程学报, 2022, 41(5): 626-630. DOI: 10.3969/j.issn.0258-8021.2022.05.012.Long SC, Hu AT, Chen ZQ. Microaneurysm detection method based on mixed multi-features[J]. Chinese Journal of Biomedical Engineering, 2022, 41(5): 626-630. DOI: 10.3969/j.issn.0258-8021.2022.05.012.
8. 周梦颖, 杨晓宇, 邱媛, 等. 一种眼底图像出血点的检测算法[J]. 北京生物医学工程, 2022, 41(3): 255-259. DOI: 10.3969/j.issn.1002-3208.2022.03.006.Zhou MY, Yang XY, Qiu Y, et al. Algorithm of hemorrhages in fundus images[J]. Beijing Biomedical Engineering, 2022, 41(3): 255-259. DOI: 10.3969/j.issn.1002-3208.2022.03.006.
9. Jaafar HF, Al-Libawy H, Al-Gayem QK. Automated approach for extraction of microaneurysms and hemorrhages in retinal fundus images[J]. Int J Intell Engineer Syst, 2020, 13(5): 210. DOI: 10.22266/ijies2020.1031.20.
10. Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: a review[J/OL]. Med Image Anal, 2019, 58: 101552[2019-08-31]. https://pubmed.ncbi.nlm.nih.gov/31521965/. DOI: 10.1016/j.media.2019.101552.
11. Liu Q, Liu H, Zhao Y, et al. Dual-branch network with dual-sampling modulated dice loss for hard exudate segmentation in color fundus images[J]. IEEE J Biomed Health Inform, 2021, 26(3): 1091-1102. DOI: 10.1109/jbhi.2021.3108169.
12. Milletari F, Navab N, Ahmadi SA. V-net: fully convolutional neural networks for volumetric medical image segmentation[J]. IEEE, 2016: 565-571. DOI: 10.1109/3DV.2016.79.
13. Shujaat M, Aslam N, Noreen I, et al. Intelligent and integrated framework for exudate detection in retinal fundus images[J]. Intell Autom Soft Co, 2021, 30(2): 663-672. DOI: 10.32604/iasc.2021.019194.
14. Han X, Zhong Y, Cao L, et al. Pre-trained alexnet architecture with pyramid pooling and supervision for high spatial resolution remote sensing image scene classification[J]. Remote Sens, 2017, 9(8): 848. DOI: 10.3390/rs9080848.
15. 杨知桥, 张莹, 王新杰, 等. 改进U型网络在视网膜病变检测中的应用研究[J]. 计算机科学与探索, 2022, 16(8): 1877. DOI: 10.3778/j.issn.1673-9418.2012011.Yang ZQ, Zhang Y, Wang XJ, et al. Application research of improved U-shaped network in detection of retinopathy[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1877. DOI: 10.3778/j.issn.1673-9418.2012011.
16. Fu Y, Zhang G, Lu X, et al. RMCA U-net: hard exudates segmentation for retinal fundus images[J/OL]. Expert Syst Appl, 2023, 234: 120987[2023-08-23]. https://www.sciencedirect.com/science/article/pii/S0957417423014896?via%3Dihub. DOI: 10.1016/j.eswa.2023.120987.
17. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation[J]. IEEE, 2015, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
18. Wang H, Cao P, Yang J, et al. MCA-UNet: multi-scale cross co-attentional U-Net for automatic medical image segmentation[J]. Health Inf Sci Syst, 2023, 11(1): 10. DOI: 10.1007/s13755-022-00209-4.
19. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[J]. IEEE, 2016: 770-778. DOI: 10.1109/cvpr.2016.90.
20. 陈人和, 赖振意, 钱育蓉. 改进的生成对抗网络图像去噪算法[J]. 计算机工程与应用, 2021, 57(5): 168-172. DOI: 10.3778/j.issn.1002-8331.2003-0336.Chen RH, Lai ZY, Qian YR. Improved image denoising generative adversarial network algorithm[J]. Comput Engin Appl, 2021, 57(5): 168-172. DOI: 10.3778/j.issn.1002-8331.2003-0336.
21. Howard AG, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. 2017, 27(7): 9. DOI: 10.48550/arXiv.1704.04861.
22. Wang Q, Wu B, Zhu P, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks[J]. IEEE, 2020, 2020: 11534-11542. DOI: 10.1109/cvpr42600.2020.01155.
23. Gu Z, Cheng J, Fu H, et al. CE-Net: context encoder network for 2d medical image segmentation[J]. IEEE T Med Imaging, 2019, 38(10): 2281-2292. DOI: 10.1109/tmi.2019.2903562.
24. Misra D, Nalamada T, Arasanipalai AU, et al. Rotate to attend: convolutional triplet attention module[J]. IEEE, 2021, 2021: 3139-3148. DOI: 10.1109/wacv48630.2021.00318.
25. Zhang F, Li S, Deng J. Unsupervised domain adaptation with shape constraint and triple attention for joint optic disc and cup segmentation[J/OL]. Sensors, 2022, 22(22): 8748[2022-11-12]. https://pubmed.ncbi.nlm.nih.gov/36433345/. DOI: 10.3390/s22228748.
26. Porwal P, Pachade S, Kamble R, et al. Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research[J]. Data, 2018, 3(3): 25. DOI: 10.3390/data3030025.
27. Chidambaram N, Vijayan D. Detection of exudates in diabetic retinopathy[J]. IEEE, 2018, 2018: 660-664. DOI: 10.1109/icacci.2018.8554923.
28. Oktay O, Schlemper J, Folgoc LL, et al. Attention u-net: Learning where to look for the pancreas[J]. Comput Vis Pat Recog, 2018, 2018: 1. DOI: 10.48550/arXiv.1804.03999.
29. Zhou Z, Siddiquee MMR, Tajbakhsh N, et al. Unet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE T Med Imaging, 2019, 39(6): 1856-1867. DOI: 10.1109/tmi.2019.2959609.
30. Guo S, Li T, Kang H, et al. L-Seg: an end-to-end unified framework for multi-lesion segmentation of fundus images[J]. Neurocomputing, 2019, 349: 52-63. DOI: 10.1016/j.neucom.2019.04.019.
31. Ibtehaz N, Rahman MS. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation[J]. Neur Net, 2020, 121: 74-87. DOI: 10.1016/j.neunet.2019.08.025.
32. Cao H, Wang Y, Chen J, et al. Swin-unet: UNet-like pure transformer for medical image segmentation[J]. IEEE, 2022, 2022: 205-218. DOI: 10.1007/978-3-031-25066-8_9.

Chinese Journal of Ocular Fundus Diseases

Latest ArticlesResearch on exudate segmentation method for retinal fundus images based on deep learning

Abstract Full text Figures/Tables Video References Cited by

Format

Content