The background of abdominal computed tomography (CT) images is complex, and kidney tumors have different shapes, sizes and unclear edges. Consequently, the segmentation methods applying to the whole CT images are often unable to effectively segment the kidney tumors. To solve these problems, this paper proposes a multi-scale network based on cascaded 3D U-Net and DeepLabV3+ for kidney tumor segmentation, which uses atrous convolution feature pyramid to adaptively control receptive field. Through the fusion of high-level and low-level features, the segmented edges of large tumors and the segmentation accuracies of small tumors are effectively improved. A total of 210 CT data published by Kits2019 were used for five-fold cross validation, and 30 CT volume data collected from Suzhou Science and Technology Town Hospital were independently tested by trained segmentation models. The results of five-fold cross validation experiments showed that the Dice coefficient, sensitivity and precision were 0.796 2 ± 0.274 1, 0.824 5 ± 0.276 3, and 0.805 1 ± 0.284 0, respectively. On the external test set, the Dice coefficient, sensitivity and precision were 0.817 2 ± 0.110 0, 0.829 6 ± 0.150 7, and 0.831 8 ± 0.116 8, respectively. The results show a great improvement in the segmentation accuracy compared with other semantic segmentation methods.