Automatic segmentation of kidney tumor based on cascaded multiscale convolutional neural networks_Journal of Biomedical Engineering

Authors：

JI Hong ¹ , QIAN Xusheng ² , ZHOU Zhiyong ^2,3 , ZHU Jianbing ⁴ , YE Lushuang ⁵ ,  WANG Feng ¹ ,  DAI Yakang ^2,3

1. School of Biological Science and Medical Engineering, Southeast University, Nanjing 210096, P.R.China;
2. Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, Jiangsu 215163, P.R.China;
3. Jinan Guoke Medical Technology Development Co., Ltd, Jinan 250101, P.R.China;
4. Suzhou Science and Technology Town Hospital, Suzhou, Jiangsu 215153, P.R.China;
5. Medical College of Shaoxing University, Shaoxing, Zhejiang 312000, P.R.China;

Corresponding author：

WANG Feng, Email: daiyk@sibet.ac.cn; DAI Yakang, Email: daiyk@sibet.ac.cn

Keywords：

kidney tumor; multi-scale; convolution neural network; automatic segmentation; cascade

DOI：

10.7507/1001-5515.202101044

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

The background of abdominal computed tomography (CT) images is complex, and kidney tumors have different shapes, sizes and unclear edges. Consequently, the segmentation methods applying to the whole CT images are often unable to effectively segment the kidney tumors. To solve these problems, this paper proposes a multi-scale network based on cascaded 3D U-Net and DeepLabV3+ for kidney tumor segmentation, which uses atrous convolution feature pyramid to adaptively control receptive field. Through the fusion of high-level and low-level features, the segmented edges of large tumors and the segmentation accuracies of small tumors are effectively improved. A total of 210 CT data published by Kits2019 were used for five-fold cross validation, and 30 CT volume data collected from Suzhou Science and Technology Town Hospital were independently tested by trained segmentation models. The results of five-fold cross validation experiments showed that the Dice coefficient, sensitivity and precision were 0.796 2 ± 0.274 1, 0.824 5 ± 0.276 3, and 0.805 1 ± 0.284 0, respectively. On the external test set, the Dice coefficient, sensitivity and precision were 0.817 2 ± 0.110 0, 0.829 6 ± 0.150 7, and 0.831 8 ± 0.116 8, respectively. The results show a great improvement in the segmentation accuracy compared with other semantic segmentation methods.

Citation： JI Hong, QIAN Xusheng, ZHOU Zhiyong, ZHU Jianbing, YE Lushuang, WANG Feng, DAI Yakang. Automatic segmentation of kidney tumor based on cascaded multiscale convolutional neural networks. Journal of Biomedical Engineering, 2021, 38(4): 722-731. doi: 10.7507/1001-5515.202101044 Copy

1.	Kuthi L, Jenei A, Hajdu A, et al. Prognostic factors for renal cell carcinoma subtypes diagnosed according to the 2016 WHO renal tumor classification: a study involving 928 patients. Pathol Oncol Res, 2017, 23(3): 689-698.
2.	Bray F, Ferlay J, Soerjomataram I, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA-Cancer J Clin, 2018, 68(6): 394-424.
3.	Kutikov A, Uzzo R G. The R. E. N. A. L. nephrometry score: a comprehensive standardized system for quantitating renal tumor size, location and depth. J Urol, 2009, 182(3): 844-853.
4.	Song L, Geoffrey K, Kaijian H. Bottleneck feature supervised U-Net for pixel-wise liver and tumor segmentation. Expert Syst Appl, 2020, 145: 113131.
5.	Li X, Chen H, Qi X, et al. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans Med Imaging, 2018, 37(12): 2663-2674.
6.	Jin Q, Meng Z, Sun C, et al. RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scans. Front Bioeng Biotechnol, 2020, 8: 605132.
7.	Seo H, Huang C, Bassenne M, et al. Modified U-Net (mU-Net) with incorporation of object-dependent high level features for improved liver and liver-tumor segmentation in CT images. IEEE Trans Med Imaging, 2019, 39(5): 1316-1325.
8.	Wang G, Li W, Ourselin S, et al. Automatic brain tumor segmentation based on cascaded convolutional neural networks with uncertainty estimation. Front Comput Neurosci, 2019, 13: 56.
9.	Havaei M, Davy A, Warde-Farley D, et al. Brain tumor segmentation with deep neural networks. Med Image Anal, 2017, 35: 18-31.
10.	Liu C, Ding W, Li L, et al. Brain tumor segmentation network using attention-based fusion and spatial relationship constraint. arXiv, 2020: 2010.156472.
11.	李肃义, 唐世杰, 李凤, 等. 基于深度学习的生物医学数据分析进展. 生物医学工程学杂志, 2020, 37(2): 349-357.
12.	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell, 2015, 39(4): 640-651.
13.	Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation// 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich: Medical Image Computing and Computer Assisted Intervention Society, 2015: 234-241.
14.	Zhang J, Jin Y, Xu J, et al. MDU-Net: Multi-scale densely connected U-Net for biomedical image segmentation. arXiv, 2018: 1812.00352.
15.	Zhang Z, Wu C, Coleman S, et al. DENSE-INception U-net for medical image segmentation. Comput Meth Programs Biomed, 2020, 192: 105395.
16.	Ibtehaz N, Rahman M S. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw, 2020, 121: 74-87.
17.	Çiçek Ö, Abdulkadir A, Lienkamp S S, et al. 3D U-Net: Learning dense volumetric segmentation from sparse annotation// 19th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Athens: Medical Image Computing and Computer Assisted Intervention Society, 2016: 424-432.
18.	Zhou Z, Siddiquee M, Tajbakhsh N, et al. UNet++: a nested U-Net architecture for medical image segmentation// 4th Deep Learning in Medical Image Analysis (DLMIA). Granada: International Workshop on Deep Learning in Medical Image Analysis, 2018: 3-11.
19.	Huang H, Lin L, Tong R, et al. UNet 3+: a full-scale connected UNet for medical image segmentation// IEEE International Conference on Acoustics, Speech, and Signal Processing. Barcelona: IEEE, 2020: 1055-1059.
20.	Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network// 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 6230-6239.
21.	Yang G, Li G, Pan T, et al. Automatic segmentation of kidney and renal tumor in CT images based on 3D fully convolutional neural network with pyramid pooling module// 24th International Conference on Pattern Recognition (ICPR). Beijing: IEEE, 2018: 3790-3795.
22.	Chen L, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation// 15th European Conference on Computer Vision (ECCV). Munich: ECCV, 2018: 833-851.
23.	Isensee F, Maier-Hein K J A. An attempt at beating the 3D U-Net. arXiv, 2019: 1908.02182.
24.	Buslaev A, Parinov A, Khvedchenya E, et al. Albumentations: fast and flexible image augmentations. arXiv, 2018: 1809.06839.
25.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2016: 770-778.
26.	Heller N, Sathianathen N, Kalapara A, et al. The KiTS19 challenge data: 300 kidney tumor cases with clinical context, CT semantic segmentations, and surgical outcomes. arXiv, 2019: 1904.00445.
27.	Paszke A, Gross S, Massa F, et al. Pytorch: An imperative style, high-performance deep learning library// 33rd Conference on Neural Information Processing Systems (NeurIPS). Vancouver: Neural Information Processing Systems Foundation, 2019: 1-12.
28.	Hu J, Shen L, Sun G. Squeeze-and-Excitation Networks// 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 7132-7141.
29.	Woo S, Park J, Lee J-Y, et al. Cbam: Convolutional block attention module// 15th European Conference on Computer Vision (ECCV). Munich: ECCV, 2018: 3-19.
30.	Gsa B, Li X A, Yang C B, et al. Marginal loss and exclusion loss for partially supervised multi-organ segmentation. Med Image Anal, 2021, 70: 101979.

1. Kuthi L, Jenei A, Hajdu A, et al. Prognostic factors for renal cell carcinoma subtypes diagnosed according to the 2016 WHO renal tumor classification: a study involving 928 patients. Pathol Oncol Res, 2017, 23(3): 689-698.
2. Bray F, Ferlay J, Soerjomataram I, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA-Cancer J Clin, 2018, 68(6): 394-424.
3. Kutikov A, Uzzo R G. The R. E. N. A. L. nephrometry score: a comprehensive standardized system for quantitating renal tumor size, location and depth. J Urol, 2009, 182(3): 844-853.
4. Song L, Geoffrey K, Kaijian H. Bottleneck feature supervised U-Net for pixel-wise liver and tumor segmentation. Expert Syst Appl, 2020, 145: 113131.
5. Li X, Chen H, Qi X, et al. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans Med Imaging, 2018, 37(12): 2663-2674.
6. Jin Q, Meng Z, Sun C, et al. RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scans. Front Bioeng Biotechnol, 2020, 8: 605132.
7. Seo H, Huang C, Bassenne M, et al. Modified U-Net (mU-Net) with incorporation of object-dependent high level features for improved liver and liver-tumor segmentation in CT images. IEEE Trans Med Imaging, 2019, 39(5): 1316-1325.
8. Wang G, Li W, Ourselin S, et al. Automatic brain tumor segmentation based on cascaded convolutional neural networks with uncertainty estimation. Front Comput Neurosci, 2019, 13: 56.
9. Havaei M, Davy A, Warde-Farley D, et al. Brain tumor segmentation with deep neural networks. Med Image Anal, 2017, 35: 18-31.
10. Liu C, Ding W, Li L, et al. Brain tumor segmentation network using attention-based fusion and spatial relationship constraint. arXiv, 2020: 2010.156472.
11. 李肃义, 唐世杰, 李凤, 等. 基于深度学习的生物医学数据分析进展. 生物医学工程学杂志, 2020, 37(2): 349-357.
12. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell, 2015, 39(4): 640-651.
13. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation// 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich: Medical Image Computing and Computer Assisted Intervention Society, 2015: 234-241.
14. Zhang J, Jin Y, Xu J, et al. MDU-Net: Multi-scale densely connected U-Net for biomedical image segmentation. arXiv, 2018: 1812.00352.
15. Zhang Z, Wu C, Coleman S, et al. DENSE-INception U-net for medical image segmentation. Comput Meth Programs Biomed, 2020, 192: 105395.
16. Ibtehaz N, Rahman M S. MultiResUNet: rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw, 2020, 121: 74-87.
17. Çiçek Ö, Abdulkadir A, Lienkamp S S, et al. 3D U-Net: Learning dense volumetric segmentation from sparse annotation// 19th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Athens: Medical Image Computing and Computer Assisted Intervention Society, 2016: 424-432.
18. Zhou Z, Siddiquee M, Tajbakhsh N, et al. UNet++: a nested U-Net architecture for medical image segmentation// 4th Deep Learning in Medical Image Analysis (DLMIA). Granada: International Workshop on Deep Learning in Medical Image Analysis, 2018: 3-11.
19. Huang H, Lin L, Tong R, et al. UNet 3+: a full-scale connected UNet for medical image segmentation// IEEE International Conference on Acoustics, Speech, and Signal Processing. Barcelona: IEEE, 2020: 1055-1059.
20. Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network// 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 6230-6239.
21. Yang G, Li G, Pan T, et al. Automatic segmentation of kidney and renal tumor in CT images based on 3D fully convolutional neural network with pyramid pooling module// 24th International Conference on Pattern Recognition (ICPR). Beijing: IEEE, 2018: 3790-3795.
22. Chen L, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation// 15th European Conference on Computer Vision (ECCV). Munich: ECCV, 2018: 833-851.
23. Isensee F, Maier-Hein K J A. An attempt at beating the 3D U-Net. arXiv, 2019: 1908.02182.
24. Buslaev A, Parinov A, Khvedchenya E, et al. Albumentations: fast and flexible image augmentations. arXiv, 2018: 1809.06839.
25. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE, 2016: 770-778.
26. Heller N, Sathianathen N, Kalapara A, et al. The KiTS19 challenge data: 300 kidney tumor cases with clinical context, CT semantic segmentations, and surgical outcomes. arXiv, 2019: 1904.00445.
27. Paszke A, Gross S, Massa F, et al. Pytorch: An imperative style, high-performance deep learning library// 33rd Conference on Neural Information Processing Systems (NeurIPS). Vancouver: Neural Information Processing Systems Foundation, 2019: 1-12.
28. Hu J, Shen L, Sun G. Squeeze-and-Excitation Networks// 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 7132-7141.
29. Woo S, Park J, Lee J-Y, et al. Cbam: Convolutional block attention module// 15th European Conference on Computer Vision (ECCV). Munich: ECCV, 2018: 3-19.
30. Gsa B, Li X A, Yang C B, et al. Marginal loss and exclusion loss for partially supervised multi-organ segmentation. Med Image Anal, 2021, 70: 101979.

Journal of Biomedical Engineering

Automatic segmentation of kidney tumor based on cascaded multiscale convolutional neural networks

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content