Research progress of breast pathology image diagnosis based on deep learning_Journal of Biomedical Engineering

Authors：

JIANG Liang , ZHANG Cheng ,  CAO Hui , JIANG Baihao

College of Intelligence and Information Engineering, Shandong University of Traditional Chinese Medicine, Jinan 250355, P. R. China;

Corresponding author：

Keywords：

Deep learning; Breast pathology images; Convolutional neural network; Multimodal data

DOI：

10.7507/1001-5515.202311061

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Breast cancer is a malignancy caused by the abnormal proliferation of breast epithelial cells, predominantly affecting female patients, and it is commonly diagnosed using histopathological images. Currently, deep learning techniques have made significant breakthroughs in medical image processing, outperforming traditional detection methods in breast cancer pathology classification tasks. This paper first reviewed the advances in applying deep learning to breast pathology images, focusing on three key areas: multi-scale feature extraction, cellular feature analysis, and classification. Next, it summarized the advantages of multimodal data fusion methods for breast pathology images. Finally, the study discussed the challenges and future prospects of deep learning in breast cancer pathology image diagnosis, providing important guidance for advancing the use of deep learning in breast diagnosis.

1.	Sung H, Ferlay J, Siegel R L, et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2021, 71(3): 209-249.
2.	Vuong D, Simpson P T, Green B, et al. Molecular classification of breast cancer. Virchows Archiv, 2014, 465: 1-14.
3.	Gruber I V, Rueckert M, Kagan K O, et al. Measurement of tumour size with mammography, sonography and magnetic resonance imaging as compared to histological tumour size in primary breast cancer. BMC Cancer, 2013, 13(1): 1-8.
4.	Assi H A, Khoury K E, Dbouk H, et al. Epidemiology and prognosis of breast cancer in young women. J Thorac Dis, 2013, 5(Suppl 1): S2-S8.
5.	Tan P H, Ellis I, Allison K, et al. The 2019 WHO classification of tumours of the breast. Histopathology, 2020, 77(2): 181-185.
6.	Rakha E A, Pareja F G. New advances in molecular breast cancer pathology. Semin Cancer Biol, 2021, 72: 102-113.
7.	Baxi V, Edwards R, Montalto M, et al. Digital pathology and artificial intelligence in translational medicine and clinical practice. Mod Pathol, 2022, 35(1): 23-32.
8.	Srinidhi C L, Ciga O, Martel A L. Deep neural network models for computational histopathology: A survey. Med Image Anal, 2021, 67: 101813.
9.	Krithiga R, Geetha P. Breast cancer detection, segmentation and classification on histopathology images analysis: a systematic review. Arch Computat Methods Eng, 2021, 28: 2607-2619.
10.	Rashmi R, Prasad K, Udupa C B K. Breast histopathological image analysis using image processing techniques for diagnostic purposes: A methodological review. J Med Syst, 2022, 46: 1-24.
11.	Li J, Mi W, Guo Y, et al. Artificial intelligence for histological subtype classification of breast cancer: combining multi‐scale feature maps and the recurrent attention model. Histopathology, 2022, 80(5): 836-846.
12.	Jiang H, Diao Z, Shi T, et al. A review of deep learning-based multiple-lesion recognition from medical images: classification, detection and segmentation. Comput Biol Med, 2023: 106726.
13.	Zhang J, Qiu S, Li Q, et al. Hepatocellular carcinoma histopathological images grading with a novel attention-sharing hybrid network based on multi-feature fusion. Biomed Signal Process Control, 2023, 86: 105126.
14.	Sheikh T S, Lee Y, Cho M. Histopathological classification of breast cancer images using a multi-scale input and multi-feature network. Cancers, 2020, 12(8): 2031.
15.	Amin M S, Ahn H. FabNet: A features agglomeration-based convolutional neural network for multiscale breast cancer histopathology images classification. Cancers, 2023, 15(4): 1013.
16.	Alqahtani Y, Mandawkar U, Sharma A, et al. Breast cancer pathological image classification based on the multiscale CNN squeeze model. Comput Intell Neurosci, 2022, 2022: 7075408.
17.	Xu C, Yi K, Jiang N, et al. MDFF-Net: A multi-dimensional feature fusion network for breast histopathology image classification. Comput Biol Med, 2023, 165: 107385.
18.	Li W, Long H, Zhan X, et al. MDAA: multi-scale and dual-adaptive attention network for breast cancer classification. Signal Image Video P, 2024, 18(4): 3133-3143.
19.	Karthik R, Menaka R, Siddharth M V. Classification of breast cancer from histopathology images using an ensemble of deep multiscale networks. Biocybern Biomed Eng, 2022, 42(3): 963-976.
20.	Wang L, Liu J, Jiang P, et al. LGViT: Local-global vision transformer for breast cancer histopathological image classification// IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Shenzhen: IEEE, 2023: 1-5.
21.	Sreelekshmi V, Pavithran K, Nair J J. SwinCNN: An Integrated Swin Trasformer and CNN for improved breast cancer grade classification. IEEE Access 2024, 12: 68697-68710.
22.	韩继能, 谢嘉伟, 顾松, 等. 基于全景病理图像细胞密度和异型特征的胶质瘤自动分级. 生物医学工程学杂志, 2021, 38(6): 1062-1071.
23.	Shihabuddin A R, Beevi S. Multi CNN based automatic detection of mitotic nuclei in breast histopathological images. Comput Biol Med, 2023, 158: 106815.
24.	Mathew T, Ajith B, Kini J R, et al. Deep learning-based automated mitosis detection in histopathology images for breast cancer grading. Int J Imag Syst Tech, 2022, 32(4): 1192-1208.
25.	Sohail A, Khan A, Wahab N, et al. A multi-phase deep CNN based mitosis detection framework for breast cancer histopathological images. Sci Rep, 2021, 11(1): 6215.
26.	Mahmood T, Arsalan M, Owais M, et al. Artificial intelligence-based mitosis detection in breast cancer histopathology images using faster R-CNN and deep CNNs. J Clin Med, 2020, 9(3): 749.
27.	Sigirci I O, Albayrak A, Bilgin G. Detection of mitotic cells in breast cancer histopathological images using deep versus handcrafted features. Multimed Tools Appl, 2022, 81(10): 13179-13202.
28.	刘月平. 乳腺癌组织病理诊断的几个热点问题. 临床与实验病理学杂志, 2023, 39(8): 897-900.
29.	Ibrahim A, Gamble P, Jaroensri R, et al. Artificial intelligence in digital breast pathology: techniques and applications. Breast, 2020, 49: 267-273.
30.	Umer M J, Sharif M, Kadry S, et al. Multi-class classification of breast cancer using 6B-Net with deep feature fusion and selection method. J Pers Med, 2022, 12(5): 683.
31.	He Z, Lin M, Xu Z, et al. Deconv-transformer (DecT): A histopathological image classification model for breast cancer based on color deconvolution and transformer architecture. Inform Sci, 2022, 608: 1093-1112.
32.	Jiang L, Zhang C, Zhang H, et al. A lightweight spatially-aware classification model for breast cancer pathology images. Biocybern Biomed Eng, 2024, 44(3): 586-608.
33.	Mi W, Li J, Guo Y, et al. Deep learning-based multi-class classification of breast digital pathology images. Cancer Manag Res, 2021: 4605-4617.
34.	Huang S K, Yu Y T, Huang C R, et al. Cross-scale fusion transformer for histopathological image classification. IEEE J Biomed Health Inform, 2023, 28(1): 297-308.
35.	Kausar T, Wang M J, Idrees M, et al. HWDCNN: Multi-class recognition in breast histopathology with Haar wavelet decomposed image based convolution neural network. Biocybern Biomed Eng, 2019, 39(4): 967-982.
36.	Luo N, Zhong X, Su L, et al. Artificial intelligence-assisted dermatology diagnosis: From unimodal to multimodal. Comput Biol Med, 2023: 107413.
37.	Roy V. Breast cancer classification with multi-fusion technique and correlation analysis. Fusion Prac Appl, 2022, 9(2): 48-61.
38.	Mobadersany P, Yousefi S, Amgad M, et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci, 2018, 115(13): E2970-E2979.
39.	Yao J, Zhu X, Zhu F, et al. Deep correlational learning for survival prediction from multi-modality data// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2017: 406-414.
40.	Liu T, Huang J, Liao T, et al. A hybrid deep learning model for predicting molecular subtypes of human breast cancer using multimodal data. IRBM, 2022, 43(1): 62-74.
41.	Yan R, Zhang F, Rao X, et al. Richer fusion network for breast cancer classification based on multimodal data. BMC Med Inform Decis Mak, 2021, 21(1): 1-15.
42.	Yang J, Ju J, Guo L, et al. Prediction of HER2-positive breast cancer recurrence and metastasis risk from histopathological images and clinical information via multimodal deep learning. Comput Struct Biotechnol J, 2022, 20: 333-342.
43.	Younis Y S, Ali A H, Alhafidhb O K S, et al. Early diagnosis of breast cancer using image processing techniques. J Nanomater, 2022, 2022: 1-6.
44.	Nasser M, Yusof U K. Deep learning based methods for breast cancer diagnosis: A systematic review and future direction. Diagnostics, 2023, 13(1): 161.
45.	Abdelsamea M M, Zidan U, Senousy Z, et al. A survey on artificial intelligence in histopathology image analysis. WIREs Data Mining Knowl Discov, 2022, 12(6): e1474.
46.	陈英, 林洪平, 张伟, 等. 医学图像数据集扩充方法研究进展. 生物医学工程学杂志, 2023, 40(1): 185-192.
47.	Ma Y, Huang Y, Yuan K, et al. Explainable stuttering recognition using axial attention// International Conference on Intelligent Computing. Singapore: Springer Nature Singapore, 2023: 209-220.
48.	James G, Witten D, Hastie T, et al. Unsupervised learning// An introduction to statistical learning: with applications in Python. Cham: Springer International Publishing, 2023: 503-556.
49.	Stahlschmidt S R, Ulfenborg B, Synnergren J. Multimodal deep learning for biomedical data fusion: a review. Brief Bioinform, 2022, 23(2): bbab569.
50.	Kokol P, Kokol M, Zagoranski S. Machine learning on small size samples: A synthetic knowledge synthesis. Sci Prog, 2022, 105(1): 00368504211029777.
51.	Fang F, Yao Y, Zhou T, et al. Self-supervised multi-modal hybrid fusion network for brain tumor segmentation. IEEE J Biomed Health Inform, 2021, 26(11): 5310-5320.

1. Sung H, Ferlay J, Siegel R L, et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2021, 71(3): 209-249.
2. Vuong D, Simpson P T, Green B, et al. Molecular classification of breast cancer. Virchows Archiv, 2014, 465: 1-14.
3. Gruber I V, Rueckert M, Kagan K O, et al. Measurement of tumour size with mammography, sonography and magnetic resonance imaging as compared to histological tumour size in primary breast cancer. BMC Cancer, 2013, 13(1): 1-8.
4. Assi H A, Khoury K E, Dbouk H, et al. Epidemiology and prognosis of breast cancer in young women. J Thorac Dis, 2013, 5(Suppl 1): S2-S8.
5. Tan P H, Ellis I, Allison K, et al. The 2019 WHO classification of tumours of the breast. Histopathology, 2020, 77(2): 181-185.
6. Rakha E A, Pareja F G. New advances in molecular breast cancer pathology. Semin Cancer Biol, 2021, 72: 102-113.
7. Baxi V, Edwards R, Montalto M, et al. Digital pathology and artificial intelligence in translational medicine and clinical practice. Mod Pathol, 2022, 35(1): 23-32.
8. Srinidhi C L, Ciga O, Martel A L. Deep neural network models for computational histopathology: A survey. Med Image Anal, 2021, 67: 101813.
9. Krithiga R, Geetha P. Breast cancer detection, segmentation and classification on histopathology images analysis: a systematic review. Arch Computat Methods Eng, 2021, 28: 2607-2619.
10. Rashmi R, Prasad K, Udupa C B K. Breast histopathological image analysis using image processing techniques for diagnostic purposes: A methodological review. J Med Syst, 2022, 46: 1-24.
11. Li J, Mi W, Guo Y, et al. Artificial intelligence for histological subtype classification of breast cancer: combining multi‐scale feature maps and the recurrent attention model. Histopathology, 2022, 80(5): 836-846.
12. Jiang H, Diao Z, Shi T, et al. A review of deep learning-based multiple-lesion recognition from medical images: classification, detection and segmentation. Comput Biol Med, 2023: 106726.
13. Zhang J, Qiu S, Li Q, et al. Hepatocellular carcinoma histopathological images grading with a novel attention-sharing hybrid network based on multi-feature fusion. Biomed Signal Process Control, 2023, 86: 105126.
14. Sheikh T S, Lee Y, Cho M. Histopathological classification of breast cancer images using a multi-scale input and multi-feature network. Cancers, 2020, 12(8): 2031.
15. Amin M S, Ahn H. FabNet: A features agglomeration-based convolutional neural network for multiscale breast cancer histopathology images classification. Cancers, 2023, 15(4): 1013.
16. Alqahtani Y, Mandawkar U, Sharma A, et al. Breast cancer pathological image classification based on the multiscale CNN squeeze model. Comput Intell Neurosci, 2022, 2022: 7075408.
17. Xu C, Yi K, Jiang N, et al. MDFF-Net: A multi-dimensional feature fusion network for breast histopathology image classification. Comput Biol Med, 2023, 165: 107385.
18. Li W, Long H, Zhan X, et al. MDAA: multi-scale and dual-adaptive attention network for breast cancer classification. Signal Image Video P, 2024, 18(4): 3133-3143.
19. Karthik R, Menaka R, Siddharth M V. Classification of breast cancer from histopathology images using an ensemble of deep multiscale networks. Biocybern Biomed Eng, 2022, 42(3): 963-976.
20. Wang L, Liu J, Jiang P, et al. LGViT: Local-global vision transformer for breast cancer histopathological image classification// IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Shenzhen: IEEE, 2023: 1-5.
21. Sreelekshmi V, Pavithran K, Nair J J. SwinCNN: An Integrated Swin Trasformer and CNN for improved breast cancer grade classification. IEEE Access 2024, 12: 68697-68710.
22. 韩继能, 谢嘉伟, 顾松, 等. 基于全景病理图像细胞密度和异型特征的胶质瘤自动分级. 生物医学工程学杂志, 2021, 38(6): 1062-1071.
23. Shihabuddin A R, Beevi S. Multi CNN based automatic detection of mitotic nuclei in breast histopathological images. Comput Biol Med, 2023, 158: 106815.
24. Mathew T, Ajith B, Kini J R, et al. Deep learning-based automated mitosis detection in histopathology images for breast cancer grading. Int J Imag Syst Tech, 2022, 32(4): 1192-1208.
25. Sohail A, Khan A, Wahab N, et al. A multi-phase deep CNN based mitosis detection framework for breast cancer histopathological images. Sci Rep, 2021, 11(1): 6215.
26. Mahmood T, Arsalan M, Owais M, et al. Artificial intelligence-based mitosis detection in breast cancer histopathology images using faster R-CNN and deep CNNs. J Clin Med, 2020, 9(3): 749.
27. Sigirci I O, Albayrak A, Bilgin G. Detection of mitotic cells in breast cancer histopathological images using deep versus handcrafted features. Multimed Tools Appl, 2022, 81(10): 13179-13202.
28. 刘月平. 乳腺癌组织病理诊断的几个热点问题. 临床与实验病理学杂志, 2023, 39(8): 897-900.
29. Ibrahim A, Gamble P, Jaroensri R, et al. Artificial intelligence in digital breast pathology: techniques and applications. Breast, 2020, 49: 267-273.
30. Umer M J, Sharif M, Kadry S, et al. Multi-class classification of breast cancer using 6B-Net with deep feature fusion and selection method. J Pers Med, 2022, 12(5): 683.
31. He Z, Lin M, Xu Z, et al. Deconv-transformer (DecT): A histopathological image classification model for breast cancer based on color deconvolution and transformer architecture. Inform Sci, 2022, 608: 1093-1112.
32. Jiang L, Zhang C, Zhang H, et al. A lightweight spatially-aware classification model for breast cancer pathology images. Biocybern Biomed Eng, 2024, 44(3): 586-608.
33. Mi W, Li J, Guo Y, et al. Deep learning-based multi-class classification of breast digital pathology images. Cancer Manag Res, 2021: 4605-4617.
34. Huang S K, Yu Y T, Huang C R, et al. Cross-scale fusion transformer for histopathological image classification. IEEE J Biomed Health Inform, 2023, 28(1): 297-308.
35. Kausar T, Wang M J, Idrees M, et al. HWDCNN: Multi-class recognition in breast histopathology with Haar wavelet decomposed image based convolution neural network. Biocybern Biomed Eng, 2019, 39(4): 967-982.
36. Luo N, Zhong X, Su L, et al. Artificial intelligence-assisted dermatology diagnosis: From unimodal to multimodal. Comput Biol Med, 2023: 107413.
37. Roy V. Breast cancer classification with multi-fusion technique and correlation analysis. Fusion Prac Appl, 2022, 9(2): 48-61.
38. Mobadersany P, Yousefi S, Amgad M, et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci, 2018, 115(13): E2970-E2979.
39. Yao J, Zhu X, Zhu F, et al. Deep correlational learning for survival prediction from multi-modality data// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2017: 406-414.
40. Liu T, Huang J, Liao T, et al. A hybrid deep learning model for predicting molecular subtypes of human breast cancer using multimodal data. IRBM, 2022, 43(1): 62-74.
41. Yan R, Zhang F, Rao X, et al. Richer fusion network for breast cancer classification based on multimodal data. BMC Med Inform Decis Mak, 2021, 21(1): 1-15.
42. Yang J, Ju J, Guo L, et al. Prediction of HER2-positive breast cancer recurrence and metastasis risk from histopathological images and clinical information via multimodal deep learning. Comput Struct Biotechnol J, 2022, 20: 333-342.
43. Younis Y S, Ali A H, Alhafidhb O K S, et al. Early diagnosis of breast cancer using image processing techniques. J Nanomater, 2022, 2022: 1-6.
44. Nasser M, Yusof U K. Deep learning based methods for breast cancer diagnosis: A systematic review and future direction. Diagnostics, 2023, 13(1): 161.
45. Abdelsamea M M, Zidan U, Senousy Z, et al. A survey on artificial intelligence in histopathology image analysis. WIREs Data Mining Knowl Discov, 2022, 12(6): e1474.
46. 陈英, 林洪平, 张伟, 等. 医学图像数据集扩充方法研究进展. 生物医学工程学杂志, 2023, 40(1): 185-192.
47. Ma Y, Huang Y, Yuan K, et al. Explainable stuttering recognition using axial attention// International Conference on Intelligent Computing. Singapore: Springer Nature Singapore, 2023: 209-220.
48. James G, Witten D, Hastie T, et al. Unsupervised learning// An introduction to statistical learning: with applications in Python. Cham: Springer International Publishing, 2023: 503-556.
49. Stahlschmidt S R, Ulfenborg B, Synnergren J. Multimodal deep learning for biomedical data fusion: a review. Brief Bioinform, 2022, 23(2): bbab569.
50. Kokol P, Kokol M, Zagoranski S. Machine learning on small size samples: A synthetic knowledge synthesis. Sci Prog, 2022, 105(1): 00368504211029777.
51. Fang F, Yao Y, Zhou T, et al. Self-supervised multi-modal hybrid fusion network for brain tumor segmentation. IEEE J Biomed Health Inform, 2021, 26(11): 5310-5320.

Journal of Biomedical Engineering

Latest ArticlesResearch progress of breast pathology image diagnosis based on deep learning

Abstract Full text Figures/Tables Video References Cited by

Format

Content