Reinforcement learning-based method for type B aortic dissection localization_Journal of Biomedical Engineering

Authors：

ZENG An ¹ , LIN Xianyang ¹ , ZHAO Jingliang ¹ ,  PAN Dan ² , YANG Baoyao ¹ , LIU Xin ¹

1. School of Computers, Guangdong University of Technology, Guangzhou 510006, P. R. China;
2. School of Electronics and Information, Guangdong Polytechnic Normal University, Guangzhou 510665, P. R. China;

Corresponding author：

PAN Dan, Email: pandan@gpnu.edu.cn

Keywords：

Aortic dissection; Two-stage segmentation; Reinforcement learning; Reward function

DOI：

10.7507/1001-5515.202309047

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

In the segmentation of aortic dissection, there are issues such as low contrast between the aortic dissection and surrounding organs and vessels, significant differences in dissection morphology, and high background noise. To address these issues, this paper proposed a reinforcement learning-based method for type B aortic dissection localization. With the assistance of a two-stage segmentation model, the deep reinforcement learning was utilized to perform the first-stage aortic dissection localization task, ensuring the integrity of the localization target. In the second stage, the coarse segmentation results from the first stage were used as input to obtain refined segmentation results. To improve the recall rate of the first-stage segmentation results and include the segmentation target more completely in the localization results, this paper designed a reinforcement learning reward function based on the direction of recall changes. Additionally, the localization window was separated from the field of view window to reduce the occurrence of segmentation target loss. Unet, TransUnet, SwinUnet, and MT-Unet were selected as benchmark segmentation models. Through experiments, it was verified that the majority of the metrics in the two-stage segmentation process of this paper performed better than the benchmark results. Specifically, the Dice index improved by 1.34%, 0.89%, 27.66%, and 7.37% for each respective model. In conclusion, by incorporating the type B aortic dissection localization method proposed in this paper into the segmentation process, the overall segmentation accuracy is improved compared to the benchmark models. The improvement is particularly significant for models with poorer segmentation performance.

Citation： ZENG An, LIN Xianyang, ZHAO Jingliang, PAN Dan, YANG Baoyao, LIU Xin. Reinforcement learning-based method for type B aortic dissection localization. Journal of Biomedical Engineering, 2024, 41(5): 878-885. doi: 10.7507/1001-5515.202309047 Copy

1.	Sayed A, Munir M, Bahbah EI. Aortic dissection: A review of the pathophysiology, management and prospective advances. Curr Cardiol Rev, 2021, 17(4): 87-101.
2.	陈永昆, 张石龙, 张顺利, 等. 主动脉夹层发病的影响因素分析. 临床医学进展, 2023, 13(6): 9541-9550.
3.	Hahn L D, Baeumler K, Hsiao A. Artificial intelligence and machine learning in aortic disease. Curr Opin Cardiol, 2021, 36(6): 695-703.
4.	Fetnaci N, Łubniewski P, Miguel B, et al. 3D segmentation of the true and false lumens on CT aortic dissection images// Three-Dimensional Image Processing (3DIP) and Applications 2013. Burlingame: SPIE, 2013, 8650: 176-190.
5.	Duan Xiaojie, Shi Meichen, Wang Jianming, et al. Segmentation of the aortic dissection from CT images based on spatial continuity prior model// 2016 8th International Conference on Information Technology in Medicine and Education (ITME). Fuzhou: IEEE, 2016: 275-280.
6.	Lee N, Tek H, Laine A F. True-false lumen segmentation of aortic dissection using multi-scale wavelet analysis and generative-discriminative model matching// Medical Imaging 2008: Computer-Aided Diagnosis. San Diego: SPIE, 2008, 6915: 878-888.
7.	呼亚萍, 孔韦韦, 李萌, 等. 基于边缘检测全变分模型的图像去噪方法. 现代电子技术, 2021, 44(5): 52-56.
8.	Zhu Z, Xia Y, Shen W, et al. A 3D coarse-to-fine framework for automatic pancreas segmentation. arXiv preprint arXiv: 2017: 1712.00201.
9.	吴春彪. 基于两阶段全卷积神经网络的冠状动脉分割研究. 计算机科学与应用, 2022, 12(4): 828-834.
10.	Man Y, Huang Y, Feng J, et al. Deep Q learning driven CT pancreas segmentation with geometry-aware U-Net. IEEE Trans Med Imaging, 2019, 38(8): 1971-1980.
11.	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv, 2014: 1409.1556.
12.	Le N, Rathour V S, Yamazaki K, et al. Deep reinforcement learning in computer vision: a comprehensive survey. Artif Intell Rev, 2022, 55: 2733-2819.
13.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
14.	Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer International Publishing, 2015: 234-241.
15.	Chen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv, 2021: 2102.04306.
16.	Cao H, Wang Y, Chen J, et al. Swin-Unet: Unet-like pure transformer for medical image segmentation// European Conference on Computer Vision. Tel Aviv: Springer, Cham, 2022: 205-218.
17.	Wang H, Xie S, Lin L, et al. Mixed transformer U-net for medical image segmentation// ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Singapore: IEEE, 2022: 2390-2394.
18.	Wang X, Wang S, Liang X, et al. Deep reinforcement learning: A survey. IEEE Trans Neural Netw Learn Syst, 2022, 35(4): 5064-5078.
19.	Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature, 2015, 518(7540): 529-533.
20.	Zhao Z. Variants of Bellman equation on reinforcement learning problems// 2nd International Conference on Artificial Intelligence, Automation, and High-Performance Computing (AIAHPC 2022). Zhuhai: SPIE, 2022, 12348: 467-478.
21.	Hasselt H, Guez A, Silver D. Deep reinforcement learning with double Q-Learning// Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Phoenix: AAAI Press, 2016: 2094-2100.
22.	Wang Z, Schaul T, Hessel M, et al. Dueling network architectures for deep reinforcement learning// International Conference on Machine Learning. New York City: PMLR, 2016: 1995-2003.
23.	Hausknecht M, Stone P. Deep recurrent Q-learning for partially observable MDPs// 2015 AAAI Fall Symposium Series. Arlington: AAAI, 2015: 29-37.
24.	Xiong J, Po L M, Cheung K W, et al. Edge-sensitive left ventricle segmentation using deep reinforcement learning. Sensors, 2021, 21(7): 2375.
25.	Yao Z, Xie W, Zhang J, et al. ImageTBAD: A 3D computed tomography angiography image dataset for automatic segmentation of type-B aortic dissection. Front Physiol, 2021, 12: 732711.
26.	dos Santos Mignon A, da Rocha R L A. An adaptive implementation of ε-greedy in reinforcement learning. Procedia Comput Sci, 2017, 109: 1146-1151.

1. Sayed A, Munir M, Bahbah EI. Aortic dissection: A review of the pathophysiology, management and prospective advances. Curr Cardiol Rev, 2021, 17(4): 87-101.
2. 陈永昆, 张石龙, 张顺利, 等. 主动脉夹层发病的影响因素分析. 临床医学进展, 2023, 13(6): 9541-9550.
3. Hahn L D, Baeumler K, Hsiao A. Artificial intelligence and machine learning in aortic disease. Curr Opin Cardiol, 2021, 36(6): 695-703.
4. Fetnaci N, Łubniewski P, Miguel B, et al. 3D segmentation of the true and false lumens on CT aortic dissection images// Three-Dimensional Image Processing (3DIP) and Applications 2013. Burlingame: SPIE, 2013, 8650: 176-190.
5. Duan Xiaojie, Shi Meichen, Wang Jianming, et al. Segmentation of the aortic dissection from CT images based on spatial continuity prior model// 2016 8th International Conference on Information Technology in Medicine and Education (ITME). Fuzhou: IEEE, 2016: 275-280.
6. Lee N, Tek H, Laine A F. True-false lumen segmentation of aortic dissection using multi-scale wavelet analysis and generative-discriminative model matching// Medical Imaging 2008: Computer-Aided Diagnosis. San Diego: SPIE, 2008, 6915: 878-888.
7. 呼亚萍, 孔韦韦, 李萌, 等. 基于边缘检测全变分模型的图像去噪方法. 现代电子技术, 2021, 44(5): 52-56.
8. Zhu Z, Xia Y, Shen W, et al. A 3D coarse-to-fine framework for automatic pancreas segmentation. arXiv preprint arXiv: 2017: 1712.00201.
9. 吴春彪. 基于两阶段全卷积神经网络的冠状动脉分割研究. 计算机科学与应用, 2022, 12(4): 828-834.
10. Man Y, Huang Y, Feng J, et al. Deep Q learning driven CT pancreas segmentation with geometry-aware U-Net. IEEE Trans Med Imaging, 2019, 38(8): 1971-1980.
11. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv, 2014: 1409.1556.
12. Le N, Rathour V S, Yamazaki K, et al. Deep reinforcement learning in computer vision: a comprehensive survey. Artif Intell Rev, 2022, 55: 2733-2819.
13. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
14. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation// International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer International Publishing, 2015: 234-241.
15. Chen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv, 2021: 2102.04306.
16. Cao H, Wang Y, Chen J, et al. Swin-Unet: Unet-like pure transformer for medical image segmentation// European Conference on Computer Vision. Tel Aviv: Springer, Cham, 2022: 205-218.
17. Wang H, Xie S, Lin L, et al. Mixed transformer U-net for medical image segmentation// ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Singapore: IEEE, 2022: 2390-2394.
18. Wang X, Wang S, Liang X, et al. Deep reinforcement learning: A survey. IEEE Trans Neural Netw Learn Syst, 2022, 35(4): 5064-5078.
19. Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature, 2015, 518(7540): 529-533.
20. Zhao Z. Variants of Bellman equation on reinforcement learning problems// 2nd International Conference on Artificial Intelligence, Automation, and High-Performance Computing (AIAHPC 2022). Zhuhai: SPIE, 2022, 12348: 467-478.
21. Hasselt H, Guez A, Silver D. Deep reinforcement learning with double Q-Learning// Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Phoenix: AAAI Press, 2016: 2094-2100.
22. Wang Z, Schaul T, Hessel M, et al. Dueling network architectures for deep reinforcement learning// International Conference on Machine Learning. New York City: PMLR, 2016: 1995-2003.
23. Hausknecht M, Stone P. Deep recurrent Q-learning for partially observable MDPs// 2015 AAAI Fall Symposium Series. Arlington: AAAI, 2015: 29-37.
24. Xiong J, Po L M, Cheung K W, et al. Edge-sensitive left ventricle segmentation using deep reinforcement learning. Sensors, 2021, 21(7): 2375.
25. Yao Z, Xie W, Zhang J, et al. ImageTBAD: A 3D computed tomography angiography image dataset for automatic segmentation of type-B aortic dissection. Front Physiol, 2021, 12: 732711.
26. dos Santos Mignon A, da Rocha R L A. An adaptive implementation of ε-greedy in reinforcement learning. Procedia Comput Sci, 2017, 109: 1146-1151.

Journal of Biomedical Engineering

Reinforcement learning-based method for type B aortic dissection localization

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content