Breast cancer lesion segmentation based on co-learning feature fusion and Transformer_Journal of Biomedical Engineering

Authors：

ZHAI Yuesong ¹ ,  CHEN Zhili ¹ ,  SHAO Dan ²

1. School of Computer Science and Engineering, Shenyang Jianzhu University, Shenyang 110168, P. R. China;
2. Department of Nuclear Medicine, Guangdong Academy of Medical Sciences, Guangdong Provincial People's Hospital, Guangzhou 519041, P. R. China;

Corresponding author：

CHEN Zhili, Email: zzc@sjzu.edu.cn; SHAO Dan, Email: shaodan@gdph.org.cns

Keywords：

Positron emission tomography and computed tomography; Breast cancer lesion segmentation; Dual-path U-Net; Co-learning feature fusion; Transformer

DOI：

10.7507/1001-5515.202306063

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

The PET/CT imaging technology combining positron emission tomography (PET) and computed tomography (CT) is the most advanced imaging examination method currently, and is mainly used for tumor screening, differential diagnosis of benign and malignant tumors, staging and grading. This paper proposes a method for breast cancer lesion segmentation based on PET/CT bimodal images, and designs a dual-path U-Net framework, which mainly includes three modules: encoder module, feature fusion module and decoder module. Among them, the encoder module uses traditional convolution for feature extraction of single mode image; The feature fusion module adopts collaborative learning feature fusion technology and uses Transformer to extract the global features of the fusion image; The decoder module mainly uses multi-layer perceptron to achieve lesion segmentation. This experiment uses actual clinical PET/CT data to evaluate the effectiveness of the algorithm. The experimental results show that the accuracy, recall and accuracy of breast cancer lesion segmentation are 95.67%, 97.58% and 96.16%, respectively, which are better than the baseline algorithm. Therefore, it proves the rationality of the single and bimodal feature extraction method combining convolution and Transformer in the experimental design of this article, and provides reference for feature extraction methods for tasks such as multimodal medical image segmentation or classification.

Citation： ZHAI Yuesong, CHEN Zhili, SHAO Dan. Breast cancer lesion segmentation based on co-learning feature fusion and Transformer. Journal of Biomedical Engineering, 2024, 41(2): 237-245. doi: 10.7507/1001-5515.202306063 Copy

1.	Sung H, Ferlay J, Siegel R L, et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2021, 71(3): 209-249.
2.	Lei S, Zheng R, Zhang S, et al. Breast cancer incidence and mortality in women in China: temporal trends and projections to 2030. Cancer Biol Med, 2021, 18(3): 900-909.
3.	Shi Z, Lin J, Wu Y, et al. Burden of cancer and changing cancer spectrum among older adults in China: Trends and projections to 2030. Cancer Epidemiol, 2022, 76: 102068.
4.	Zeng H, Chen W, Zheng R, et al. Changing cancer survival in China during 2003-15: a pooled analysis of 17 population-based cancer registries. Lancet Glob Health, 2018, 6(5): e555-e567.
5.	Hildebrandt M G, Naghavi-Behzad M, Vogsen M. A role of FDG-PET/CT for response evaluation in metastatic breast cancer?. Semin Nucl Med, 2022, 52(5): 520-530.
6.	Chen W. Clinical application of PET in pediatric brain tumors. PET Clin, 2008, 3(4): 517-529.
7.	辛阳. CT成像技术的发展及技术特点. 科技创新导报, 2018, 15(4): 132-133.
8.	Li Y, Dai Y, Guo Y, et al. Correlation analysis of sup18/sup F-FDG PET/CT for the staging and treatment effect assessment of breast cancer. Journal of X-Ray Science and Technology, 2019, 27(6): 1131-1144.
9.	Cruz-Roa A, Arevalo J, Judkins A, et al. A method for medulloblastoma tumor differentiation based on convolutional neural networks and transfer learning//International Symposium on Medical Information Processing and Analysis. International Society for Optics and Photonics, 2015, DOI: 10.1117/12.2208825.
10.	赵旭. 基于医学先验的多尺度乳腺超声肿瘤实例分割方法. 哈尔滨: 哈尔滨工业大学, 2019.
11.	徐胜舟, 程时宇. 基于全卷积神经网络迁移学习的乳腺肿块图像分割. 中南民族大学学报: 自然科学版, 2019, 38(2): 278-284.
12.	Al-Antari M A, Al-Masni M A, Choi M T, et al. A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification. Int J Med Inform, 2018, 117: 44-54.
13.	Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2015: 234–241.
14.	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation//European Conference on Computer Vision. Springer, Cham, 2018: 833-851.
15.	Zhuang J. LadderNet: multi-path networks based on U-Net for medical image segmentation. arXiv preprint, 2018, DOI: 10.48550/arXiv.1810.07810.
16.	Kumar A, Fulham M, Feng D, et al. Co-learning feature fusion maps from PET-CT images of lung cancer. IEEE Trans Med Imaging, 2019, 39(1): 204-217.
17.	Xiao X, Lian S, Luo Z, et al. Weighted Res-UNet for high-quality retina vessel segmentation//2018 9th International Conference on Information Technology in Medicine and Education (ITME). IEEE Computer Society, 2018: 327-331.
18.	Guan S, Khan A A, Sikdar S, et al. Fully dense UNet for 2-D sparse photoacoustic tomography artifact removal. IEEE J Biomed Health Inform, 2020, 24(2): 568-576.
19.	Isensee F, Jäger P F, Kohl S A A, et al. Automated design of deep learning methods for biomedical image segmentation. arXiv preprint, 2019, arXiv: 1904.08128.
20.	Chen J, Lu Y, Yu Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation. arXiv preprint, 2021, DOI: 10.48550/arXiv.2102.04306.
21.	Jain J, Li J, Chiu M T, et al. OneFormer: one transformer to rule universal image segmentation. arXiv preprint, 2022, DOI: 10.48550/arXiv.2211.06220.
22.	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint, 2020, DOI: 10.48550/arXiv.2010.11929.
23.	Xie E, Wang W, Yu Z, et al. SegFormer: simple and efficient design for semantic segmentation with transformers. arXiv preprint, 2021, DOI: 10.48550/arXiv.2105.15203.
24.	Liu Z, Lin Y, Cao Y, et al. Swin Transformer: hierarchical vision Transformer using shifted windows. arXiv preprint, 2021. DOI: 10.48550/arXiv.2103.14030.
25.	Ji S, Yang M, Yu K. 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell, 2013, 35(1): 221-231.
26.	Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need. arXiv preprint, 2017. DOI: 10.48550/arXiv.1706.03762.
27.	余辉, 张书旭. 4DCT图像二维配准与三维配准的区别. 医疗装备, 2011, 24(7): 6-8.
28.	胡德文, 陈广学, 朱剑铭, 等. 基于ROI的医学图像预处理技术研究. 医学影像学杂志, 2018, 28(4): 585-589.
29.	Fukuda T, Fernandez R, Rosenberg A, et al. Data augmentation improves recognition of foreign accented speech. Interspeech, 2018, 18(9): 2409-2413.
30.	侯向丹, 李紫宇, 牛敬钰, 等. 结合注意力机制和多路径U-Net的视网膜血管分割. 计算机辅助设计与图形学学报, 2023, 35(1): 55-65.

1. Sung H, Ferlay J, Siegel R L, et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2021, 71(3): 209-249.
2. Lei S, Zheng R, Zhang S, et al. Breast cancer incidence and mortality in women in China: temporal trends and projections to 2030. Cancer Biol Med, 2021, 18(3): 900-909.
3. Shi Z, Lin J, Wu Y, et al. Burden of cancer and changing cancer spectrum among older adults in China: Trends and projections to 2030. Cancer Epidemiol, 2022, 76: 102068.
4. Zeng H, Chen W, Zheng R, et al. Changing cancer survival in China during 2003-15: a pooled analysis of 17 population-based cancer registries. Lancet Glob Health, 2018, 6(5): e555-e567.
5. Hildebrandt M G, Naghavi-Behzad M, Vogsen M. A role of FDG-PET/CT for response evaluation in metastatic breast cancer?. Semin Nucl Med, 2022, 52(5): 520-530.
6. Chen W. Clinical application of PET in pediatric brain tumors. PET Clin, 2008, 3(4): 517-529.
7. 辛阳. CT成像技术的发展及技术特点. 科技创新导报, 2018, 15(4): 132-133.
8. Li Y, Dai Y, Guo Y, et al. Correlation analysis of sup18/sup F-FDG PET/CT for the staging and treatment effect assessment of breast cancer. Journal of X-Ray Science and Technology, 2019, 27(6): 1131-1144.
9. Cruz-Roa A, Arevalo J, Judkins A, et al. A method for medulloblastoma tumor differentiation based on convolutional neural networks and transfer learning//International Symposium on Medical Information Processing and Analysis. International Society for Optics and Photonics, 2015, DOI: 10.1117/12.2208825.
10. 赵旭. 基于医学先验的多尺度乳腺超声肿瘤实例分割方法. 哈尔滨: 哈尔滨工业大学, 2019.
11. 徐胜舟, 程时宇. 基于全卷积神经网络迁移学习的乳腺肿块图像分割. 中南民族大学学报: 自然科学版, 2019, 38(2): 278-284.
12. Al-Antari M A, Al-Masni M A, Choi M T, et al. A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification. Int J Med Inform, 2018, 117: 44-54.
13. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2015: 234–241.
14. Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation//European Conference on Computer Vision. Springer, Cham, 2018: 833-851.
15. Zhuang J. LadderNet: multi-path networks based on U-Net for medical image segmentation. arXiv preprint, 2018, DOI: 10.48550/arXiv.1810.07810.
16. Kumar A, Fulham M, Feng D, et al. Co-learning feature fusion maps from PET-CT images of lung cancer. IEEE Trans Med Imaging, 2019, 39(1): 204-217.
17. Xiao X, Lian S, Luo Z, et al. Weighted Res-UNet for high-quality retina vessel segmentation//2018 9th International Conference on Information Technology in Medicine and Education (ITME). IEEE Computer Society, 2018: 327-331.
18. Guan S, Khan A A, Sikdar S, et al. Fully dense UNet for 2-D sparse photoacoustic tomography artifact removal. IEEE J Biomed Health Inform, 2020, 24(2): 568-576.
19. Isensee F, Jäger P F, Kohl S A A, et al. Automated design of deep learning methods for biomedical image segmentation. arXiv preprint, 2019, arXiv: 1904.08128.
20. Chen J, Lu Y, Yu Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation. arXiv preprint, 2021, DOI: 10.48550/arXiv.2102.04306.
21. Jain J, Li J, Chiu M T, et al. OneFormer: one transformer to rule universal image segmentation. arXiv preprint, 2022, DOI: 10.48550/arXiv.2211.06220.
22. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint, 2020, DOI: 10.48550/arXiv.2010.11929.
23. Xie E, Wang W, Yu Z, et al. SegFormer: simple and efficient design for semantic segmentation with transformers. arXiv preprint, 2021, DOI: 10.48550/arXiv.2105.15203.
24. Liu Z, Lin Y, Cao Y, et al. Swin Transformer: hierarchical vision Transformer using shifted windows. arXiv preprint, 2021. DOI: 10.48550/arXiv.2103.14030.
25. Ji S, Yang M, Yu K. 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell, 2013, 35(1): 221-231.
26. Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need. arXiv preprint, 2017. DOI: 10.48550/arXiv.1706.03762.
27. 余辉, 张书旭. 4DCT图像二维配准与三维配准的区别. 医疗装备, 2011, 24(7): 6-8.
28. 胡德文, 陈广学, 朱剑铭, 等. 基于ROI的医学图像预处理技术研究. 医学影像学杂志, 2018, 28(4): 585-589.
29. Fukuda T, Fernandez R, Rosenberg A, et al. Data augmentation improves recognition of foreign accented speech. Interspeech, 2018, 18(9): 2409-2413.
30. 侯向丹, 李紫宇, 牛敬钰, 等. 结合注意力机制和多路径U-Net的视网膜血管分割. 计算机辅助设计与图形学学报, 2023, 35(1): 55-65.

Journal of Biomedical Engineering

Breast cancer lesion segmentation based on co-learning feature fusion and Transformer

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content