Adaptive lesion-aware fusion network for joint grading of multiple fundus diseases_Journal of Biomedical Engineering

Authors：

ZENG Wei ,  GUO Shengwen

School of Automation Science and Engineering, South China University of Technology, Guangzhou 510641, P. R. China;

Corresponding author：

GUO Shengwen, Email: shwguo@scut.edu.cn

Keywords：

Diabetic retinopathy; Diabetic macular edema; Joint grading; Adaptive lesion attention; Deep feature fusion

DOI：

10.7507/1001-5515.202411032

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Diabetic retinopathy (DR) and its complication, diabetic macular edema (DME), are major causes of visual impairment and even blindness. The occurrence of DR and DME is pathologically interconnected, and their clinical diagnoses are closely related. Joint learning can help improve the accuracy of diagnosis. This paper proposed a novel adaptive lesion-aware fusion network (ALFNet) to facilitate the joint grading of DR and DME. ALFNet employed DenseNet-121 as the backbone and incorporated an adaptive lesion attention module (ALAM) to capture the distinct lesion characteristics of DR and DME. A deep feature fusion module (DFFM) with a shared-parameter local attention mechanism was designed to learn the correlation between the two diseases. Furthermore, a four-branch composite loss function was introduced to enhance the network’s multi-task learning capability. Experimental results demonstrated that ALFNet achieved superior joint grading performance on the Messidor dataset, with joint accuracy rates of 0.868 (DR 2 & DME 3), outperforming state-of-the-art methods. These results highlight the unique advantages of the proposed approach in the joint grading of DR and DME, thereby improving the efficiency and accuracy of clinical decision-making.

1.	Banday M Z, Sameer A S, Nissar S. Pathophysiology of diabetes: An overview. Avicenna J Med, 2020, 10(4): 174-188.
2.	Sun H, Saeedi P, Karuranga S, et al. IDF Diabetes Atlas: Global, regional and country-level diabetes prevalence estimates for 2021 and projections for 2045. Diabetes Res Clin Pract, 2022, 183: 109119.
3.	Lee R, Wong T Y, Sabanayagam C. Epidemiology of diabetic retinopathy, diabetic macular edema and related vision loss. Eye Vis, 2015, 2: 1-25.
4.	Lindner M, Arefnia B, Ivastinovic D, et al. Association of periodontitis and diabetic macular edema in various stages of diabetic retinopathy. Clin Oral Investig, 2022, 26(1): 505-512.
5.	Davidson J A, Ciulla T A, McGill J B, et al. How the diabetic eye loses vision. Endocrine, 2007, 32(1): 107-116.
6.	Ciulla T A, Amador A G, Zinman B. Diabetic retinopathy and diabetic macular edema: Pathophysiology, screening, and novel therapies. Diabetes Care, 2003, 26(9): 2653-2664.
7.	Asiri N, Hussain M, Al Adel F, et al. Deep learning based computer-aided diagnosis systems for diabetic retinopathy: A survey. Artif Intell in Med, 2019, 99: 101701.
8.	Silberman N, Ahrlich K, Fergus R, et al. Case for automated detection of diabetic retinopathy// AAAI Spring Symposium: Artificial Intelligence for Development. Stanford: AI Access Foundation, 2010: 85-90.
9.	Niemeijer M, Ginneken B V, Staal J, et al. Automatic detection of red lesions in digital color fundus photographs. IEEE Trans Med Imaging, 2005, 24(5): 584-592.
10.	Casanova R, Saldana S, Chew E Y, et al. Application of random forests methods to diabetic retinopathy classification analyses. PLOS One, 2014, 9(6): e98587.
11.	Tariq A, Akram M U, Shaukat A, et al. Automated detection and grading of diabetic maculopathy in digital retinal images. J Digit Imaging, 2013, 26(4): 803-812.
12.	Al-Bander B, Al-Nuaimy W, Al-Taee M A, et al. Diabetic macular edema grading based on deep neural networks// Ophthalmic Medical Image Analysis International Workshop. Athens: University of Iowa, 2016: 121-128.
13.	Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 2016, 316(22): 2402-2410.
14.	Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2818-2826.
15.	Krause J, Gulshan V, Rahimy E, et al. Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy. Ophthalmology, 2018, 125(8): 1264-1272.
16.	Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, Inception-ResNet and the impact of residual connections on learning// Thirty-First AAAI Conference on Artificial Intelligence. San Francisco: AAAI Press, 2017: 4278-4284.
17.	Li X, Hu X, Yu L, et al. CANet: Cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE Trans Med Imaging, 2020, 39(5): 1483-1493.
18.	Yue T, Yang W, Liao Q. CCNET: Cross coordinate network for joint diabetic retinopathy and diabetic macular edema grading// 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society. Glasgow: IEEE, 2022: 2062-2065.
19.	Guo X, Li X, Lin Q, et al. Joint grading of diabetic retinopathy and diabetic macular edema using an adaptive attention block and semisupervised learning. Appl Intell, 2023, 53(13): 16797-16812.
20.	Huang G, Liu Z, Maaten L V D, et al. Densely connected convolutional networks// IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261-2269.
21.	Decencière E, Zhang X, Cazuguel G, et al. Feedback on a publicly distributed image database: The Messidor database. Image Anal Stereol, 2014: 231-234.
22.	Deng J, Dong W, Socher R, et al. ImageNet: A large-scale hierarchical image database// 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 248-255.
23.	Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res, 2014, 15(1): 1929-1958.
24.	Sánchez C I, Niemeijer M, Dumitrescu A V, et al. Evaluation of a computer-aided diagnosis system for diabetic retinopathy screening on public data. Invest Ophthalmol Vis Sci, 2011, 52(7): 4866-4871.
25.	Pires R, Avila S, Jelinek H F, et al. Beyond lesion-based diabetic retinopathy: A direct approach for referral. IEEE J Biomed Health Inform, 2017, 21(1): 193-200.
26.	Vo H H, Verma A. New deep neural nets for fine-grained diabetic retinopathy recognition on hybrid color space// 2016 IEEE International Symposium on Multimedia. San Jose: IEEE, 2016: 209-215.
27.	Seoud L, Hurtut T, Chelbi J, et al. Red lesion detection using dynamic shape features for diabetic retinopathy screening. IEEE Trans Med Imaging, 2016, 35(4): 1116-1126.
28.	Wang Z, Yin Y, Shi J, et al. Zoom-in-Net: Deep mining lesions for diabetic retinopathy detection// Medical Image Computing and Computer Assisted Intervention −MICCAI 2017. Quebec City: Springer International Publishing, 2017: 267-275.
29.	Kingma D P, Ba J. Adam: A method for stochastic optimization// Proceedings of the 3rd International Conference on Learning Representations. San Diego: OpenReview.net, 2015: 1-15.
30.	Paszke A, Gross S, Chintala S, et al. Automatic differentiation in pytorch// NIPS 2017 Autodiff Workshop. Long Beach: OpenReview.net, 2017: 1-4.
31.	Sandler M, Howard A, Zhu M, et al. MobileNetV2: Inverted residuals and linear bottlenecks// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4510-4520.
32.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
33.	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. Int J Comput Vis, 2020, 128(2): 336-359.
34.	Chen Q, Peng Y, Keenan T, et al. A multi-task deep learning model for the classification of age-related macular degeneration. AMIA Jt Summits Transl Sci Proc, 2019, 2019: 505-514.
35.	Liu L, Dou Q, Chen H, et al. Multi-Task deep model with margin ranking loss for lung nodule analysis. IEEE Trans Med Imaging, 2020, 39(3): 718-728.

1. Banday M Z, Sameer A S, Nissar S. Pathophysiology of diabetes: An overview. Avicenna J Med, 2020, 10(4): 174-188.
2. Sun H, Saeedi P, Karuranga S, et al. IDF Diabetes Atlas: Global, regional and country-level diabetes prevalence estimates for 2021 and projections for 2045. Diabetes Res Clin Pract, 2022, 183: 109119.
3. Lee R, Wong T Y, Sabanayagam C. Epidemiology of diabetic retinopathy, diabetic macular edema and related vision loss. Eye Vis, 2015, 2: 1-25.
4. Lindner M, Arefnia B, Ivastinovic D, et al. Association of periodontitis and diabetic macular edema in various stages of diabetic retinopathy. Clin Oral Investig, 2022, 26(1): 505-512.
5. Davidson J A, Ciulla T A, McGill J B, et al. How the diabetic eye loses vision. Endocrine, 2007, 32(1): 107-116.
6. Ciulla T A, Amador A G, Zinman B. Diabetic retinopathy and diabetic macular edema: Pathophysiology, screening, and novel therapies. Diabetes Care, 2003, 26(9): 2653-2664.
7. Asiri N, Hussain M, Al Adel F, et al. Deep learning based computer-aided diagnosis systems for diabetic retinopathy: A survey. Artif Intell in Med, 2019, 99: 101701.
8. Silberman N, Ahrlich K, Fergus R, et al. Case for automated detection of diabetic retinopathy// AAAI Spring Symposium: Artificial Intelligence for Development. Stanford: AI Access Foundation, 2010: 85-90.
9. Niemeijer M, Ginneken B V, Staal J, et al. Automatic detection of red lesions in digital color fundus photographs. IEEE Trans Med Imaging, 2005, 24(5): 584-592.
10. Casanova R, Saldana S, Chew E Y, et al. Application of random forests methods to diabetic retinopathy classification analyses. PLOS One, 2014, 9(6): e98587.
11. Tariq A, Akram M U, Shaukat A, et al. Automated detection and grading of diabetic maculopathy in digital retinal images. J Digit Imaging, 2013, 26(4): 803-812.
12. Al-Bander B, Al-Nuaimy W, Al-Taee M A, et al. Diabetic macular edema grading based on deep neural networks// Ophthalmic Medical Image Analysis International Workshop. Athens: University of Iowa, 2016: 121-128.
13. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 2016, 316(22): 2402-2410.
14. Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2818-2826.
15. Krause J, Gulshan V, Rahimy E, et al. Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy. Ophthalmology, 2018, 125(8): 1264-1272.
16. Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, Inception-ResNet and the impact of residual connections on learning// Thirty-First AAAI Conference on Artificial Intelligence. San Francisco: AAAI Press, 2017: 4278-4284.
17. Li X, Hu X, Yu L, et al. CANet: Cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE Trans Med Imaging, 2020, 39(5): 1483-1493.
18. Yue T, Yang W, Liao Q. CCNET: Cross coordinate network for joint diabetic retinopathy and diabetic macular edema grading// 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society. Glasgow: IEEE, 2022: 2062-2065.
19. Guo X, Li X, Lin Q, et al. Joint grading of diabetic retinopathy and diabetic macular edema using an adaptive attention block and semisupervised learning. Appl Intell, 2023, 53(13): 16797-16812.
20. Huang G, Liu Z, Maaten L V D, et al. Densely connected convolutional networks// IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261-2269.
21. Decencière E, Zhang X, Cazuguel G, et al. Feedback on a publicly distributed image database: The Messidor database. Image Anal Stereol, 2014: 231-234.
22. Deng J, Dong W, Socher R, et al. ImageNet: A large-scale hierarchical image database// 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 248-255.
23. Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res, 2014, 15(1): 1929-1958.
24. Sánchez C I, Niemeijer M, Dumitrescu A V, et al. Evaluation of a computer-aided diagnosis system for diabetic retinopathy screening on public data. Invest Ophthalmol Vis Sci, 2011, 52(7): 4866-4871.
25. Pires R, Avila S, Jelinek H F, et al. Beyond lesion-based diabetic retinopathy: A direct approach for referral. IEEE J Biomed Health Inform, 2017, 21(1): 193-200.
26. Vo H H, Verma A. New deep neural nets for fine-grained diabetic retinopathy recognition on hybrid color space// 2016 IEEE International Symposium on Multimedia. San Jose: IEEE, 2016: 209-215.
27. Seoud L, Hurtut T, Chelbi J, et al. Red lesion detection using dynamic shape features for diabetic retinopathy screening. IEEE Trans Med Imaging, 2016, 35(4): 1116-1126.
28. Wang Z, Yin Y, Shi J, et al. Zoom-in-Net: Deep mining lesions for diabetic retinopathy detection// Medical Image Computing and Computer Assisted Intervention −MICCAI 2017. Quebec City: Springer International Publishing, 2017: 267-275.
29. Kingma D P, Ba J. Adam: A method for stochastic optimization// Proceedings of the 3rd International Conference on Learning Representations. San Diego: OpenReview.net, 2015: 1-15.
30. Paszke A, Gross S, Chintala S, et al. Automatic differentiation in pytorch// NIPS 2017 Autodiff Workshop. Long Beach: OpenReview.net, 2017: 1-4.
31. Sandler M, Howard A, Zhu M, et al. MobileNetV2: Inverted residuals and linear bottlenecks// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4510-4520.
32. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
33. Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization. Int J Comput Vis, 2020, 128(2): 336-359.
34. Chen Q, Peng Y, Keenan T, et al. A multi-task deep learning model for the classification of age-related macular degeneration. AMIA Jt Summits Transl Sci Proc, 2019, 2019: 505-514.
35. Liu L, Dou Q, Chen H, et al. Multi-Task deep model with margin ranking loss for lung nodule analysis. IEEE Trans Med Imaging, 2020, 39(3): 718-728.

Journal of Biomedical Engineering

Latest ArticlesAdaptive lesion-aware fusion network for joint grading of multiple fundus diseases

Abstract Full text Figures/Tables Video References Cited by

Format

Content