Deep learning-based fully automated intelligent and precise diagnosis for melanocytic lesions_Journal of Biomedical Engineering

Authors：

SHI Tianlei ^1,2 , ZHANG Jiayi ^1,2,3 , BAO Yongyang ⁴ ,  GAO Xin ^2,3,5

1. School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, P. R. China;
2. Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, Jiangsu 215163, P. R. China;
3. Jinan Guoke Medical Engineering and Technology Development Co., Ltd., Jinan 250101, P. R. China;
4. Shanghai Ninth People’s Hospital, Shanghai Jiaotong University School of Medicine, Shanghai 200011, P. R. China;
5. Jiangsu Province Engineering Research Center of Diagnosis and Treatment of Children’s Malignant Tumor, Suzhou, Jiangsu 215025, P. R. China;

Corresponding author：

GAO Xin, Email: xingaosam@163.com

Keywords：

Melanocytic lesions; Whole slide images; Intelligent and precise diagnosis; Deep learning; Color normalization

DOI：

10.7507/1001-5515.202203080

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Melanocytic lesions occur on the surface of the skin, in which the malignant type is melanoma with a high fatality rate, seriously endangering human health. The histopathological analysis is the gold standard for diagnosis of melanocytic lesions. In this study, a fully automated intelligent diagnosis method based on deep learning was proposed to classify the pathological whole slide images (WSI) of melanocytic lesions. Firstly, the color normalization based on CycleGAN neural network was performed on multi-center pathological WSI; Secondly, ResNet-152 neural network-based deep convolutional network prediction model was built using 745 WSI; Then, a decision fusion model was cascaded, which calculates the average prediction probability of each WSI; Finally, the diagnostic performance of the proposed method was verified by internal and external test sets containing 182 and 54 WSI, respectively. Experimental results showed that the overall diagnostic accuracy of the proposed method reached 94.12% in the internal test set and exceeded 90% in the external test set. Furthermore, the color normalization method adopted was superior to the traditional color statistics-based and staining separation-based methods in terms of structure preservation and artifact suppression. The results demonstrate that the proposed method can achieve high precision and strong robustness in pathological WSI classification of melanocytic lesions, which has the potential in promoting the clinical application of computer-aided pathological diagnosis.

Citation： SHI Tianlei, ZHANG Jiayi, BAO Yongyang, GAO Xin. Deep learning-based fully automated intelligent and precise diagnosis for melanocytic lesions. Journal of Biomedical Engineering, 2022, 39(5): 919-927. doi: 10.7507/1001-5515.202203080 Copy

1.	Swetter S M, Tsao H, Bichakjian C K, et al. Guidelines of care for the management of primary cutaneous melanoma. J Am Acad Dermatol, 2019, 80(1): 208-250.
2.	Schadendorf D, van Akkooi A C J, Berking C, et al. Melanoma. Lancet, 2018, 392(10151): 971-984.
3.	Piepkorn M W, Barnhill R L, Elder D E, et al. The MPATH-Dx reporting schema for melanocytic proliferations and melanoma. J Am Acad Dermatol, 2014, 70(1): 131-141.
4.	Davis L E, Shalin S C, Tackett A J. Current state of melanoma diagnosis and treatment. Cancer Biol Ther, 2019, 20(11): 1366-1379.
5.	The Skin Cancer Foundation. Skin cancer facts & statistics, 2021 (2022-05) [2022-07-04]. https: //www.skincancer.org/skin-cancer-information/skin-cancer-facts.
6.	Haggenmüller S, Maron R C, Hekler A, et al. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts. Eur J Cancer Care, 2021, 156: 202-216.
7.	Rocha L K F L, Vilain R E, Scolyer R A, et al. Confocal microscopy, dermoscopy, and histopathology features of atypical intraepidermal melanocytic proliferations associated with evolution to melanoma in situ. Int J Dermatol, 2022, 61(2): 167-174.
8.	Jitian Mihulecea C-R, Frățilă S, Rotaru M. Clinical-dermoscopic similarities between atypical nevi and early stage melanoma. Exp Ther Med, 2021, 22(2): 854-854.
9.	Lodha S, Saggar S, Celebi J T, et al. Discordance in the histopathologic diagnosis of difficult melanocytic neoplasms in the clinical setting. J Cutan Pathol, 2008, 35(4): 349-352.
10.	Ronen S, Al-Rohil R N, Keiser E, et al. Discordance in diagnosis of melanocytic lesions and its impact on clinical management. Arch Pathol Lab Med, 2021, 145(12): 1505-1515.
11.	Tolkach Y, Dohmgörgen T, Toma M, et al. High-accuracy prostate cancer pathology using deep learning. Nat Mach Intell, 2020, 2(7): 411-418.
12.	Coudray N, Ocampo P S, Sakellaropoulos T, et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med, 2018, 24(10): 1559-1567.
13.	Bera K, Schalper K A, Rimm D L, et al. Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology. Nat Rev Clin Oncol, 2019, 16(11): 703-715.
14.	Rakhlin A, Shvets A, Iglovikov V, et al. Deep convolutional neural networks for breast cancer histology image analysis// Campilho A, Karray F, ter Haar Romeny B. Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science. Cham: Springer, 2018, 10882: 737-744.
15.	韩继能, 谢嘉伟, 顾松, 等. 基于全景病理图像细胞密度和异型特征的胶质瘤自动分级. 生物医学工程学杂志, 2021, 38(6): 1062-1071.
16.	王荃, 沈勤, 张泽林, 等. 基于深度学习和组织形态分析的肺癌基因突变预测. 生物医学工程学杂志, 2020, 37(1): 10-18.
17.	Hekler A, Utikal J S, Enk A H, et al. Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images. Eur J Cancer Care, 2019, 118: 91-96.
18.	Hekler A, Utikal J S, Enk A H, et al. Pathologist-level classification of histopathological melanoma images with deep neural networks. Eur J Cancer Care, 2019, 115: 79-83.
19.	Brinker T J, Schmitt M, Krieghoff-Henning E I, et al. Diagnostic performance of artificial intelligence for histologic melanoma recognition compared to 18 international expert pathologists. J Am Acad Dermatol, 2022, 86(3): 640-642.
20.	Li T, Li F, Zuo K. Pathologist-Level classification of melanoma disease pathologies using a convolutional neural network: A retrospective study of Chinese// Yao J, Xiao Y, You P, et al. The International Conference on Image, Vision and Intelligent Systems (ICIVIS 2021). Lecture Notes in Electrical Engineering. Singapore: Springer, 2022, 813: 833-839.
21.	Bejnordi B E, Litjens G, Timofeeva N, et al. Stain specific standardization of whole-slide histopathological images. IEEE Trans on Med Imag, 2016, 35(2): 404-415.
22.	Howard F M, Dolezal J, Kochanny S, et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nat Commun, 2021, 12(1): 4423.
23.	Stiff K M, Franklin M J, Zhou Y, et al. Artificial intelligence and melanoma: A comprehensive review of clinical, dermoscopic, and histologic applications. Pigment Cell Melanoma Res, 2022, 35(2): 203-211.
24.	Ianni J D, Soans R E, Sankarapandian S, et al. Tailored for real-world: A whole slide image classification system validated on uncurated multi-site data emulating the prospective pathology workload. Sci Rep, 2020, 10(1): 3217.
25.	Shaban M T, Baur C, Navab N, et al. Staingan: Stain style transfer for digital histological images// 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI). Venice: IEEE, 2019: 953-956.
26.	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks// IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 2223-2232.
27.	Ruifrok A C, Johnston D A. Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol, 2001, 23(4): 291-299.
28.	Vahadane A, Peng T, Sethi A, et al. Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans on Med Imag, 2016, 35(8): 1962-1971.
29.	张术昌, 袁梓洋, 王红霞, 等. 面向组织病理学图像的颜色迁移算法. 计算机辅助设计与图形学学报, 2020, 32(12): 1890-1897.
30.	卞殷旭, 邢涛, 邓伟杰, 等. 基于深度学习的色彩迁移生物医学成像技术. 红外与激光工程, 2022, 51(2): 339-356.
31.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
32.	周涛, 刘赟璨, 陆惠玲, 等. ResNet及其在医学图像处理领域的应用:研究进展与挑战. 电子与信息学报, 2022, 44(1): 149-167.
33.	Höhn J, Krieghoff-Henning E, Jutzi T B, et al. Combining CNN-based histologic whole slide image analysis and patient data to improve skin cancer classification. Eur J Cancer Care, 2021, 149: 94-101.
34.	Reinhard E, Adhikhmin M, Gooch B, et al. Color transfer between images. IEEE Comput Graph Appl, 2001, 21(5): 34-41.

1. Swetter S M, Tsao H, Bichakjian C K, et al. Guidelines of care for the management of primary cutaneous melanoma. J Am Acad Dermatol, 2019, 80(1): 208-250.
2. Schadendorf D, van Akkooi A C J, Berking C, et al. Melanoma. Lancet, 2018, 392(10151): 971-984.
3. Piepkorn M W, Barnhill R L, Elder D E, et al. The MPATH-Dx reporting schema for melanocytic proliferations and melanoma. J Am Acad Dermatol, 2014, 70(1): 131-141.
4. Davis L E, Shalin S C, Tackett A J. Current state of melanoma diagnosis and treatment. Cancer Biol Ther, 2019, 20(11): 1366-1379.
5. The Skin Cancer Foundation. Skin cancer facts & statistics, 2021 (2022-05) [2022-07-04]. https: //www.skincancer.org/skin-cancer-information/skin-cancer-facts.
6. Haggenmüller S, Maron R C, Hekler A, et al. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts. Eur J Cancer Care, 2021, 156: 202-216.
7. Rocha L K F L, Vilain R E, Scolyer R A, et al. Confocal microscopy, dermoscopy, and histopathology features of atypical intraepidermal melanocytic proliferations associated with evolution to melanoma in situ. Int J Dermatol, 2022, 61(2): 167-174.
8. Jitian Mihulecea C-R, Frățilă S, Rotaru M. Clinical-dermoscopic similarities between atypical nevi and early stage melanoma. Exp Ther Med, 2021, 22(2): 854-854.
9. Lodha S, Saggar S, Celebi J T, et al. Discordance in the histopathologic diagnosis of difficult melanocytic neoplasms in the clinical setting. J Cutan Pathol, 2008, 35(4): 349-352.
10. Ronen S, Al-Rohil R N, Keiser E, et al. Discordance in diagnosis of melanocytic lesions and its impact on clinical management. Arch Pathol Lab Med, 2021, 145(12): 1505-1515.
11. Tolkach Y, Dohmgörgen T, Toma M, et al. High-accuracy prostate cancer pathology using deep learning. Nat Mach Intell, 2020, 2(7): 411-418.
12. Coudray N, Ocampo P S, Sakellaropoulos T, et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med, 2018, 24(10): 1559-1567.
13. Bera K, Schalper K A, Rimm D L, et al. Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology. Nat Rev Clin Oncol, 2019, 16(11): 703-715.
14. Rakhlin A, Shvets A, Iglovikov V, et al. Deep convolutional neural networks for breast cancer histology image analysis// Campilho A, Karray F, ter Haar Romeny B. Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science. Cham: Springer, 2018, 10882: 737-744.
15. 韩继能, 谢嘉伟, 顾松, 等. 基于全景病理图像细胞密度和异型特征的胶质瘤自动分级. 生物医学工程学杂志, 2021, 38(6): 1062-1071.
16. 王荃, 沈勤, 张泽林, 等. 基于深度学习和组织形态分析的肺癌基因突变预测. 生物医学工程学杂志, 2020, 37(1): 10-18.
17. Hekler A, Utikal J S, Enk A H, et al. Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images. Eur J Cancer Care, 2019, 118: 91-96.
18. Hekler A, Utikal J S, Enk A H, et al. Pathologist-level classification of histopathological melanoma images with deep neural networks. Eur J Cancer Care, 2019, 115: 79-83.
19. Brinker T J, Schmitt M, Krieghoff-Henning E I, et al. Diagnostic performance of artificial intelligence for histologic melanoma recognition compared to 18 international expert pathologists. J Am Acad Dermatol, 2022, 86(3): 640-642.
20. Li T, Li F, Zuo K. Pathologist-Level classification of melanoma disease pathologies using a convolutional neural network: A retrospective study of Chinese// Yao J, Xiao Y, You P, et al. The International Conference on Image, Vision and Intelligent Systems (ICIVIS 2021). Lecture Notes in Electrical Engineering. Singapore: Springer, 2022, 813: 833-839.
21. Bejnordi B E, Litjens G, Timofeeva N, et al. Stain specific standardization of whole-slide histopathological images. IEEE Trans on Med Imag, 2016, 35(2): 404-415.
22. Howard F M, Dolezal J, Kochanny S, et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nat Commun, 2021, 12(1): 4423.
23. Stiff K M, Franklin M J, Zhou Y, et al. Artificial intelligence and melanoma: A comprehensive review of clinical, dermoscopic, and histologic applications. Pigment Cell Melanoma Res, 2022, 35(2): 203-211.
24. Ianni J D, Soans R E, Sankarapandian S, et al. Tailored for real-world: A whole slide image classification system validated on uncurated multi-site data emulating the prospective pathology workload. Sci Rep, 2020, 10(1): 3217.
25. Shaban M T, Baur C, Navab N, et al. Staingan: Stain style transfer for digital histological images// 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI). Venice: IEEE, 2019: 953-956.
26. Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks// IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 2223-2232.
27. Ruifrok A C, Johnston D A. Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol, 2001, 23(4): 291-299.
28. Vahadane A, Peng T, Sethi A, et al. Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans on Med Imag, 2016, 35(8): 1962-1971.
29. 张术昌, 袁梓洋, 王红霞, 等. 面向组织病理学图像的颜色迁移算法. 计算机辅助设计与图形学学报, 2020, 32(12): 1890-1897.
30. 卞殷旭, 邢涛, 邓伟杰, 等. 基于深度学习的色彩迁移生物医学成像技术. 红外与激光工程, 2022, 51(2): 339-356.
31. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
32. 周涛, 刘赟璨, 陆惠玲, 等. ResNet及其在医学图像处理领域的应用:研究进展与挑战. 电子与信息学报, 2022, 44(1): 149-167.
33. Höhn J, Krieghoff-Henning E, Jutzi T B, et al. Combining CNN-based histologic whole slide image analysis and patient data to improve skin cancer classification. Eur J Cancer Care, 2021, 149: 94-101.
34. Reinhard E, Adhikhmin M, Gooch B, et al. Color transfer between images. IEEE Comput Graph Appl, 2001, 21(5): 34-41.

Journal of Biomedical Engineering

Deep learning-based fully automated intelligent and precise diagnosis for melanocytic lesions

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content