In the extraction of fetal electrocardiogram (ECG) signal, due to the unicity of the scale of the U-Net same-level convolution encoder, the size and shape difference of the ECG characteristic wave between mother and fetus are ignored, and the time information of ECG signals is not used in the threshold learning process of the encoder’s residual shrinkage module. In this paper, a method of extracting fetal ECG signal based on multi-scale residual shrinkage U-Net model is proposed. First, the Inception and time domain attention were introduced into the residual shrinkage module to enhance the multi-scale feature extraction ability of the same level convolution encoder and the utilization of the time domain information of fetal ECG signal. In order to maintain more local details of ECG waveform, the maximum pooling in U-Net was replaced by Softpool. Finally, the decoder composed of the residual module and up-sampling gradually generated fetal ECG signals. In this paper, clinical ECG signals were used for experiments. The final results showed that compared with other fetal ECG extraction algorithms, the method proposed in this paper could extract clearer fetal ECG signals. The sensitivity, positive predictive value, and F1 scores in the 2013 competition data set reached 93.33%, 99.36%, and 96.09%, respectively, indicating that this method can effectively extract fetal ECG signals and has certain application values for perinatal fetal health monitoring.