Medical image segmentation data augmentation method based on channel weight and data-efficient features_Journal of Biomedical Engineering

Authors：

 WU Xing ^1,2 , TAO Chenjie ¹ , LI Zhi ¹ , ZHANG Jian ^3,4 , SUN Qun ⁵ , HAN Xianhua ⁶ , CHEN Yanwei ⁷

1. School of Computer Engineering and Science, Shanghai University, Shanghai 200444, P. R. China;
2. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, P. R. China;
3. Medical College of Shanghai University, Shanghai University, Shanghai 200444, P. R. China;
4. Shanghai Universal Medical Imaging Diagnostic Center, Shanghai 200233, P. R. China;
5. Gastroenterology, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiaotong University, Shanghai 200233, P. R. China;
6. Faculty of Science, Yamaguchi University, Yamaguchi-Ken 753-8511, Japan;
7. College of Information Science and Engineering, Ritsumeikan University, Shiga-Ken 525-8577, Japan;

Corresponding author：

WU Xing, Email: xingwu@shu.edu.cn

Keywords：

Medical image segmentation; Data augmentation; Data efficiency; Deep learning; Interpretability

DOI：

10.7507/1001-5515.202302024

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In computer-aided medical diagnosis, obtaining labeled medical image data is expensive, while there is a high demand for model interpretability. However, most deep learning models currently require a large amount of data and lack interpretability. To address these challenges, this paper proposes a novel data augmentation method for medical image segmentation. The uniqueness and advantages of this method lie in the utilization of gradient-weighted class activation mapping to extract data efficient features, which are then fused with the original image. Subsequently, a new channel weight feature extractor is constructed to learn the weights between different channels. This approach achieves non-destructive data augmentation effects, enhancing the model's performance, data efficiency, and interpretability. Applying the method of this paper to the Hyper-Kvasir dataset, the intersection over union (IoU) and Dice of the U-net were improved, respectively; and on the ISIC-Archive dataset, the IoU and Dice of the DeepLabV3+ were also improved respectively. Furthermore, even when the training data is reduced to 70 %, the proposed method can still achieve performance that is 95 % of that achieved with the entire dataset, indicating its good data efficiency. Moreover, the data-efficient features used in the method have interpretable information built-in, which enhances the interpretability of the model. The method has excellent universality, is plug-and-play, applicable to various segmentation methods, and does not require modification of the network structure, thus it is easy to integrate into existing medical image segmentation method, enhancing the convenience of future research and applications.

Citation： WU Xing, TAO Chenjie, LI Zhi, ZHANG Jian, SUN Qun, HAN Xianhua, CHEN Yanwei. Medical image segmentation data augmentation method based on channel weight and data-efficient features. Journal of Biomedical Engineering, 2024, 41(2): 220-227, 236. doi: 10.7507/1001-5515.202302024 Copy

1.	Minaee S, Boykov Y, Porikli F, et al. Image segmentation using deep learning: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(7): 3523-3542..
2.	Khan A, Sohail A, Zahoora U, et al. A survey of the recent architectures of deep convolutional neural networks. Artificial Intelligence Review, 2020, 53: 5455-5516..
3.	Ulku I, Akagündüz E. A survey on deep learning-based architectures for semantic segmentation on 2D images. Applied Artificial Intelligence, 2022, 36(1): 2032924..
4.	Li Z, Wu X, Wang J, et al. Weather-degraded image semantic segmentation with multi-task knowledge distillation. Image and Vision Computing, 2022, 127: 104554..
5.	Xia H, Ma M, Li H, et al. MC-net: multi-scale context-attention network for medical CT image segmentation. Applied Intelligence, 2022, 52(2): 1508-1519..
6.	Ding Z, Wang T, Sun Q, et al. Adaptive fusion with multi-scale features for interactive image segmentation. Applied Intelligence, 2021, 51(8): 5610-5621..
7.	Chen W, Chen X, Lin Y. Homogeneous ensemble extreme learning machine autoencoder with mutual representation learning and manifold regularization for medical datasets. Applied Intelligence, 2023, 53(12): 15476-15495..
8.	Zhang Y, Wei Y, Wu Q, et al. Collaborative unsupervised domain adaptation for medical image diagnosis. IEEE Transactions on Image Processing, 2020, 29: 7834-7844..
9.	Batzner S, Musaelian A, Sun L, et al. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature Communications, 2022, 13(1): 2453..
10.	Chlap P, Min H, Vandenberg N, et al. A review of medical image data augmentation techniques for deep learning applications. Journal of Medical Imaging and Radiation Oncology, 2021, 65(5): 545-563..
11.	Sirazitdinov I, Kholiavchenko M, Kuleev R, et al. Data augmentation for chest pathologies classification//2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019: 1216-1219..
12.	Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Springer International Publishing, 2015: 234-241..
13.	Zhou Z, Siddiquee M M R, Tajbakhsh N, et al. Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging, 2019, 39(6): 1856-1867..
14.	Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017: 2881-2890..
15.	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation//Proceedings of the European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, 2018, 11211: 833-851..
16.	Guo M H, Xu T X, Liu J J, et al. Attention mechanisms in computer vision: a survey. Computational Visual Media, 2022, 8(3): 331-368..
17.	Lee H J, Kim H E, Nam H. SRM: a style-based recalibration module for convolutional neural networks//Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea: IEEE, 2019: 1854-1862..
18.	Hu J, Shen L, Sun G. Squeeze-and-excitation networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City: IEEE, 2018: 7132-7141..
19.	Hosseinzadeh Taher M R, Haghighi F, Feng R, et al. A systematic benchmarking analysis of transfer learning for medical image analysis//Domain Adaptation and Representation Transfer, and Affordable Healthcare and AI for Resource Diverse Global Health (2021), Strasbourg, France, Springer International Publishing, 2021, 12968: 3-13..
20.	Kolesnikov A, Beyer L, Zhai X, et al. Big transfer (BiT): general visual representation learning//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, Springer International Publishing, 2020: 491-507..
21.	Kowsher M, Sami A A, Prottasha N J, et al. Bangla-BERT: transformer-based efficient model for transfer learning and language understanding. IEEE Access, 2022, 10: 91855-91870..
22.	Wu X, Chen C, Zhong M, et al. COVID-AL: The diagnosis of COVID-19 with deep active learning. Medical Image Analysis, 2021, 68: 101913..
23.	Yang L, Zhang Y, Chen J, et al. Suggestive annotation: a deep active learning framework for biomedical image segmentation//Medical Image Computing and Computer Assisted Intervention− MICCAI 2017: 20th International Conference, Quebec City, Canada, Springer International Publishing, 2017: 399-407..
24.	Wu X, Chen C, Zhong M, et al. HAL: hybrid active learning for efficient labeling in medical domain. Neurocomputing, 2021, 456: 563-572..
25.	Moshkovitz M, Yang Y Y, Chaudhuri K. Connecting interpretability and robustness in decision trees through separation. arXiv Preprint, 2021, arXiv: 2102.07048..
26.	Stiglic G, Kocbek P, Fijacko N, et al. Interpretability of machine learning-based prediction models in healthcare. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2020, 10(5): e1379..
27.	Zhou B, Khosla A, Lapedriza A, et al. Learning deep features for discriminative localization//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016: 2921-2929..
28.	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization//2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy: IEEE, 2017: 618-626..
29.	Borgli H, Thambawita V, Smedsrud P H, et al. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Scientific Data, 2020, 7(1): 283..
30.	Oren T, Gal A. ISIC-Archive-Downloader. (2020-07-25) [2023-05-23]. https: //github.com/GalAvineri/ISIC-Archive-Downloader..

1. Minaee S, Boykov Y, Porikli F, et al. Image segmentation using deep learning: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(7): 3523-3542..
2. Khan A, Sohail A, Zahoora U, et al. A survey of the recent architectures of deep convolutional neural networks. Artificial Intelligence Review, 2020, 53: 5455-5516..
3. Ulku I, Akagündüz E. A survey on deep learning-based architectures for semantic segmentation on 2D images. Applied Artificial Intelligence, 2022, 36(1): 2032924..
4. Li Z, Wu X, Wang J, et al. Weather-degraded image semantic segmentation with multi-task knowledge distillation. Image and Vision Computing, 2022, 127: 104554..
5. Xia H, Ma M, Li H, et al. MC-net: multi-scale context-attention network for medical CT image segmentation. Applied Intelligence, 2022, 52(2): 1508-1519..
6. Ding Z, Wang T, Sun Q, et al. Adaptive fusion with multi-scale features for interactive image segmentation. Applied Intelligence, 2021, 51(8): 5610-5621..
7. Chen W, Chen X, Lin Y. Homogeneous ensemble extreme learning machine autoencoder with mutual representation learning and manifold regularization for medical datasets. Applied Intelligence, 2023, 53(12): 15476-15495..
8. Zhang Y, Wei Y, Wu Q, et al. Collaborative unsupervised domain adaptation for medical image diagnosis. IEEE Transactions on Image Processing, 2020, 29: 7834-7844..
9. Batzner S, Musaelian A, Sun L, et al. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature Communications, 2022, 13(1): 2453..
10. Chlap P, Min H, Vandenberg N, et al. A review of medical image data augmentation techniques for deep learning applications. Journal of Medical Imaging and Radiation Oncology, 2021, 65(5): 545-563..
11. Sirazitdinov I, Kholiavchenko M, Kuleev R, et al. Data augmentation for chest pathologies classification//2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 2019: 1216-1219..
12. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Springer International Publishing, 2015: 234-241..
13. Zhou Z, Siddiquee M M R, Tajbakhsh N, et al. Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging, 2019, 39(6): 1856-1867..
14. Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017: 2881-2890..
15. Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation//Proceedings of the European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, 2018, 11211: 833-851..
16. Guo M H, Xu T X, Liu J J, et al. Attention mechanisms in computer vision: a survey. Computational Visual Media, 2022, 8(3): 331-368..
17. Lee H J, Kim H E, Nam H. SRM: a style-based recalibration module for convolutional neural networks//Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea: IEEE, 2019: 1854-1862..
18. Hu J, Shen L, Sun G. Squeeze-and-excitation networks//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City: IEEE, 2018: 7132-7141..
19. Hosseinzadeh Taher M R, Haghighi F, Feng R, et al. A systematic benchmarking analysis of transfer learning for medical image analysis//Domain Adaptation and Representation Transfer, and Affordable Healthcare and AI for Resource Diverse Global Health (2021), Strasbourg, France, Springer International Publishing, 2021, 12968: 3-13..
20. Kolesnikov A, Beyer L, Zhai X, et al. Big transfer (BiT): general visual representation learning//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, Springer International Publishing, 2020: 491-507..
21. Kowsher M, Sami A A, Prottasha N J, et al. Bangla-BERT: transformer-based efficient model for transfer learning and language understanding. IEEE Access, 2022, 10: 91855-91870..
22. Wu X, Chen C, Zhong M, et al. COVID-AL: The diagnosis of COVID-19 with deep active learning. Medical Image Analysis, 2021, 68: 101913..
23. Yang L, Zhang Y, Chen J, et al. Suggestive annotation: a deep active learning framework for biomedical image segmentation//Medical Image Computing and Computer Assisted Intervention− MICCAI 2017: 20th International Conference, Quebec City, Canada, Springer International Publishing, 2017: 399-407..
24. Wu X, Chen C, Zhong M, et al. HAL: hybrid active learning for efficient labeling in medical domain. Neurocomputing, 2021, 456: 563-572..
25. Moshkovitz M, Yang Y Y, Chaudhuri K. Connecting interpretability and robustness in decision trees through separation. arXiv Preprint, 2021, arXiv: 2102.07048..
26. Stiglic G, Kocbek P, Fijacko N, et al. Interpretability of machine learning-based prediction models in healthcare. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2020, 10(5): e1379..
27. Zhou B, Khosla A, Lapedriza A, et al. Learning deep features for discriminative localization//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016: 2921-2929..
28. Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization//2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy: IEEE, 2017: 618-626..
29. Borgli H, Thambawita V, Smedsrud P H, et al. HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Scientific Data, 2020, 7(1): 283..
30. Oren T, Gal A. ISIC-Archive-Downloader. (2020-07-25) [2023-05-23]. https: //github.com/GalAvineri/ISIC-Archive-Downloader..

Journal of Biomedical Engineering

Medical image segmentation data augmentation method based on channel weight and data-efficient features

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content