Application progress of mixed reality technology in hepatobiliary and pancreatic fields_CHINESE JOURNAL OF BASES AND CLINICS IN GENERAL SURGERY

Authors：

CAO Yuanyinuo ¹ , LILI Shiluo ² , XUANYUAN Yirui ¹ ,  CHEN Yonghua ^1,3

1. West China Hospital/West China Clinical Medical College, Sichuan University, Chengdu 610041, P. R. China;
2. West China Fourth Hospital/West China School of Public Health, Sichuan University, Chengdu 610041, P. R. China;
3. Division of Pancreatic Surgery, Department of General Surgery, West China Hospital, Sichuan University, Chengdu 610041, P. R. China;

Corresponding author：

CHEN Yonghua, Email: chenyonghua2007@163.com

Keywords：

mixed reality technology; liver neoplasm; pancreatic neoplasm; surgical navigation; teaching practice

DOI：

10.7507/1007-9424.202212022

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To investigate the application progress of mixed reality (MR) technology in hepatobiliary and pancreatic fields. Method The relevant literatures on the application of MR technology of the hepatobiliary and pancreatic field in recent years at home and abroad were reviewed. Results MR technology had been widely used in the hepatobiliary and pancreatic field, including preoperative diagnosis and evaluation, surgical plan formulation, doctor-patient communication, intraoperative navigation precision surgery, teaching practice and many other aspects, which had the advantages of shortening the operation time, reducing the difficulty of surgery and improving the success rate of surgery. To some extent, it had promoted the innovation of clinical diagnosis and treatment in the field of liver, gallbladder and pancreas. Conclusions The application and development of MR related techniques are of great significance to the operation and teaching in the hepatobiliary and pancreatic field. With the development and progress of MR technology and modern medicine, MR technology will give full play to its advantages in intelligent real-time navigation hepatobiliary and pancreatic surgery system and promote the further development of hepatobiliary and pancreatic surgery.

Citation： CAO Yuanyinuo, LILI Shiluo, XUANYUAN Yirui, CHEN Yonghua. Application progress of mixed reality technology in hepatobiliary and pancreatic fields. CHINESE JOURNAL OF BASES AND CLINICS IN GENERAL SURGERY, 2023, 30(5): 613-619. doi: 10.7507/1007-9424.202212022 Copy

1.	中华医学会外科学分会肝脏外科学组, 中华医学会数字医学分会, 中国医师协会肝癌专业委员会, 等. 增强与混合现实技术在腹腔镜复杂性肝切除术中应用的中国专家共识. 中华外科杂志, 2022, 60(6): 517-523.
2.	罗青林, 李生伟. 机器人手术系统在国内肝胆外科应用的历史、现状和展望. 中国普外基础与临床杂志, 2017, 24(11): 1419-1422.
3.	侯建存, 张雅敏. 混合现实技术在肝胆外科领域中的应用. 中华外科杂志, 2022, 60(1): 17-21.
4.	陈哲, 姜维良. 混合现实技术在血管外科中的应用进展. 中国普外基础与临床杂志, 2021, 28(6): 837-840.
5.	闫晓冬, 侯建存, 田青, 等. 混合现实技术在外科领域的应用研究进展. 山东医药, 2020, 60(35): 112-114.
6.	刘帆, 昃跃峰. 混合现实技术在教育领域的应用. 新课程教学(电子版). 2021, 5: 140-141.
7.	梅昆有, 李治, 王罡, 等. 混合现实技术在肝胆肿瘤手术中的应用. 中国介入影像与治疗学, 2019, 16(12): 745-748.
8.	杨威, 汤睿, 童翾, 等. 混合现实技术在精准肝胆外科手术中应用研究. 中国实用外科杂志, 2021, 41(3): 298-305.
9.	张隆鑫, 陈晓鑫, 姜开文, 等. ALPPS联合混合现实全息影像成像治疗巨块型肝癌1例. 中国普外基础与临床杂志, 2020, 27(10): 1284-1289.
10.	祝文, 曾小军, 项楠, 等. 增强现实与混合现实导航技术在三维腹腔镜缩小右半肝切除术中的应用. 中华外科杂志, 2022, 60(3): 249-256.
11.	冷超, 朱鹏, 张必翔. 混合现实全息影像重建技术在肝切除术中的初步应用. 腹部外科, 2020, 33(4): 311-313, 248.
12.	Andolfi C, Plana A, Kania P, et al. Usefulness of three-dimensional modeling in surgical planning, resident training, and patient education. J Laparoendosc Adv Surg Tech A, 2017, 27(5): 512-515.
13.	刘刚, 李晓莉, 龚连生, 等. 混合现实技术辅助全腹腔镜下胰十二指肠切除术. 中国内镜杂志, 2020, 26(7): 72-75.
14.	Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg, 2017, 266(5): 706-712.
15.	刘文瑛, 欧阳再兴, 朱剑华, 等. 多模态影像技术在巨块型肝癌精准手术治疗中的应用. 中国普外基础与临床杂志, 2022, 29(9): 1207-1212.
16.	Simone M, Galati R, Barile G, et al. Remote mentoring in laparotomic and laparoscopic cancer surgery during Covid-19 pandemic: an experimental setup based on mixed reality. Med Educ Online, 2021, 26(1): 1996923. doi: 10.1080/10872981.2021.1996923.
17.	Oldhafer KJ, Peterhans M, Kantas A, et al. Navigated liver surgery: current state and importance in the future. Chirurg, 2018, 89(10): 769-776.
18.	Heiselman JS, Clements LW, Collins JA, et al. Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery. J Med Imaging (Bellingham), 2018, 5(2): 021203. doi: 10.1117/1.JMI.5.2.021203.
19.	Aoki T, Koizumi T, Sugimoto M, et al. Holography-guided percutaneous puncture technique for selective near-infrared fluorescence-guided laparoscopic liver resection using mixed-reality wearable spatial computer. Surg Oncol, 2020, 35: 476-477.
20.	马春阳, 周益龙, 李怀亮, 等. 混合现实术中导航技术联合三维可视化技术在复杂性肝癌切除术中的应用. 中华医学杂志, 2019, 99(4): 279-283.
21.	Saito Y, Sugimoto M, Morine Y, et al. Intraoperative support with three-dimensional holographic cholangiography in hepatobiliary surgery. Langenbecks Arch Surg, 2022, 407(3): 1285-1289.
22.	祝文, 曾小军, 胡浩宇, 等. 应用增强现实与混合现实导航预防腹腔镜肝切除术中出血价值研究. 中国实用外科杂志, 2022, 42(3): 298-302,308.
23.	Zhu LY, Hou JC, Yang L, et al. Application value of mixed reality in hepatectomy for hepatocellular carcinoma. World J Gastrointest Surg, 2022, 14(1): 36-45.
24.	Balci D, Kirimker EO, Raptis DA, et al. Uses of a dedicated 3D reconstruction software with augmented and mixed reality in planning and performing advanced liver surgery and living donor liver transplantation (with videos). Hepatobiliary Pancreat Dis Int, 2022, 21(5): 455-461.
25.	王峻峰, 赞左, 宋正己, 等. 基于3D打印/混合现实技术在胰周感染性积液治疗的应用. 国际临床研究杂志, 2021, 5(1): 24-28.
26.	Wierzbicki R, Pawłowicz M, Job J, et al. 3D mixed-reality visualization of medical imaging data as a supporting tool for innovative, minimally invasive surgery for gastrointestinal tumors and systemic treatment as a new path in personalized treatment of advanced cancer diseases. J Cancer Res Clin Oncol, 2022, 148(1): 237-243.
27.	Saito Y, Sugimoto M, Imura S, et al. Intraoperative 3D hologram support with mixed reality techniques in liver surgery. Ann Surg, 2020, 271(1): e4-e7. doi: 10.1097/SLA.0000000000003552.
28.	Strickland A, Fairhurst K, Lauder C, et al. Development of an ex vivo simulated training model for laparoscopic liver resection. Surg Endosc, 2011, 25(5): 1677-1682.
29.	李鹏超, 袁文博, 万芬, 等. 混合现实技术在住院医师规范化培训外科手术教学中的应用研究. 创新创业理论研究与实践, 2022, 5(22): 179-184.
30.	吕秀琴, 张生海. 基于HoloLens2的DEM空间分析实验教学混合现实环境研究. 实验技术与管理. 2021, 38(10): 124-129.
31.	Pelanis E, Kumar RP, Aghayan DL, et al. Use of mixed reality for improved spatial understanding of liver anatomy. Minim Invasive Ther Allied Technol, 2020, 29(3): 154-160.
32.	景涛, 邱阳, 李静尧, 等. 混合现实技术在住院医师规范化培训临床手术能力教学中的应用. 重庆医学, 2021, 50(8): 1435-1437.
33.	李忠海, 徐钢, 高淑贤, 等. 混合现实技术在外科临床教学中的应用及效果评价. 中国高等医学教育, 2022, (9): 67-69.
34.	王宏, 欧晏娇, 胡鹏, 等. 混合现实技术在肝胆外科手术教学中的应用效果. 中华医学教育探索杂志, 2019, 18(12): 1230-1234.
35.	Bala L, Kinross J, Martin G, et al. A remote access mixed reality teaching ward round. Clin Teach, 2021, 18(4): 386-390.
36.	石小强, 马艳飞, 梁挺, 等. 混合现实技术在专科医师培训中辅助腹膜后肿瘤手术临床教学的应用研究. 中国医学教育技术, 2020, 34(4): 504-507.
37.	Liu S, Wang H, Wang C, et al. Intra-operative 3D hologram support with mixed reality technique based on CT-MRI fusion images: operation guidance for brain brachytherapy. J Contemp Brachytherapy, 2021, 13(2): 205-210.
38.	Zhou Z, Yang Z, Jiang S, et al. Design and validation of a surgical navigation system for brachytherapy based on mixed reality. Med Phys, 2019, 46(8): 3709-3718.
39.	van den Berg NS, Engelen T, Brouwer OR, et al. A pilot study of SPECT/CT-based mixed-reality navigation towards the sentinel node in patients with melanoma or Merkel cell carcinoma of a lower extremity. Nucl Med Commun, 2016, 37(8): 812-817.
40.	Davrieux CF, Giménez ME, González CA, et al. Mixed reality navigation system for ultrasound-guided percutaneous punctures: a pre-clinical evaluation. Surg Endosc, 2020, 34(1): 226-230.
41.	Tu P, Qin C, Guo Y, et al. Ultrasound image guided and mixed reality-based surgical system with real-time soft tissue deformation computing for robotic cervical pedicle screw placement. IEEE Trans Biomed Eng, 2022, 69(8): 2593-2603.
42.	Li R, Pan J, Yang Y, et al. Accurate and robust feature description and dense point-wise matching based on feature fusion for endoscopic images. Comput Med Imaging Graph, 2021, 94: 102007. doi: 10.1016/j.compmedimag.2021.102007.
43.	Bloom D, Colombo JN, Miller N, et al. Early preclinical experience of a mixed reality ultrasound system with active GUIDance for NEedle-based interventions: The GUIDE study. Cardiovasc Digit Health J, 2022, 3(5): 232-240.

1. 中华医学会外科学分会肝脏外科学组, 中华医学会数字医学分会, 中国医师协会肝癌专业委员会, 等. 增强与混合现实技术在腹腔镜复杂性肝切除术中应用的中国专家共识. 中华外科杂志, 2022, 60(6): 517-523.
2. 罗青林, 李生伟. 机器人手术系统在国内肝胆外科应用的历史、现状和展望. 中国普外基础与临床杂志, 2017, 24(11): 1419-1422.
3. 侯建存, 张雅敏. 混合现实技术在肝胆外科领域中的应用. 中华外科杂志, 2022, 60(1): 17-21.
4. 陈哲, 姜维良. 混合现实技术在血管外科中的应用进展. 中国普外基础与临床杂志, 2021, 28(6): 837-840.
5. 闫晓冬, 侯建存, 田青, 等. 混合现实技术在外科领域的应用研究进展. 山东医药, 2020, 60(35): 112-114.
6. 刘帆, 昃跃峰. 混合现实技术在教育领域的应用. 新课程教学(电子版). 2021, 5: 140-141.
7. 梅昆有, 李治, 王罡, 等. 混合现实技术在肝胆肿瘤手术中的应用. 中国介入影像与治疗学, 2019, 16(12): 745-748.
8. 杨威, 汤睿, 童翾, 等. 混合现实技术在精准肝胆外科手术中应用研究. 中国实用外科杂志, 2021, 41(3): 298-305.
9. 张隆鑫, 陈晓鑫, 姜开文, 等. ALPPS联合混合现实全息影像成像治疗巨块型肝癌1例. 中国普外基础与临床杂志, 2020, 27(10): 1284-1289.
10. 祝文, 曾小军, 项楠, 等. 增强现实与混合现实导航技术在三维腹腔镜缩小右半肝切除术中的应用. 中华外科杂志, 2022, 60(3): 249-256.
11. 冷超, 朱鹏, 张必翔. 混合现实全息影像重建技术在肝切除术中的初步应用. 腹部外科, 2020, 33(4): 311-313, 248.
12. Andolfi C, Plana A, Kania P, et al. Usefulness of three-dimensional modeling in surgical planning, resident training, and patient education. J Laparoendosc Adv Surg Tech A, 2017, 27(5): 512-515.
13. 刘刚, 李晓莉, 龚连生, 等. 混合现实技术辅助全腹腔镜下胰十二指肠切除术. 中国内镜杂志, 2020, 26(7): 72-75.
14. Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg, 2017, 266(5): 706-712.
15. 刘文瑛, 欧阳再兴, 朱剑华, 等. 多模态影像技术在巨块型肝癌精准手术治疗中的应用. 中国普外基础与临床杂志, 2022, 29(9): 1207-1212.
16. Simone M, Galati R, Barile G, et al. Remote mentoring in laparotomic and laparoscopic cancer surgery during Covid-19 pandemic: an experimental setup based on mixed reality. Med Educ Online, 2021, 26(1): 1996923. doi: 10.1080/10872981.2021.1996923.
17. Oldhafer KJ, Peterhans M, Kantas A, et al. Navigated liver surgery: current state and importance in the future. Chirurg, 2018, 89(10): 769-776.
18. Heiselman JS, Clements LW, Collins JA, et al. Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery. J Med Imaging (Bellingham), 2018, 5(2): 021203. doi: 10.1117/1.JMI.5.2.021203.
19. Aoki T, Koizumi T, Sugimoto M, et al. Holography-guided percutaneous puncture technique for selective near-infrared fluorescence-guided laparoscopic liver resection using mixed-reality wearable spatial computer. Surg Oncol, 2020, 35: 476-477.
20. 马春阳, 周益龙, 李怀亮, 等. 混合现实术中导航技术联合三维可视化技术在复杂性肝癌切除术中的应用. 中华医学杂志, 2019, 99(4): 279-283.
21. Saito Y, Sugimoto M, Morine Y, et al. Intraoperative support with three-dimensional holographic cholangiography in hepatobiliary surgery. Langenbecks Arch Surg, 2022, 407(3): 1285-1289.
22. 祝文, 曾小军, 胡浩宇, 等. 应用增强现实与混合现实导航预防腹腔镜肝切除术中出血价值研究. 中国实用外科杂志, 2022, 42(3): 298-302,308.
23. Zhu LY, Hou JC, Yang L, et al. Application value of mixed reality in hepatectomy for hepatocellular carcinoma. World J Gastrointest Surg, 2022, 14(1): 36-45.
24. Balci D, Kirimker EO, Raptis DA, et al. Uses of a dedicated 3D reconstruction software with augmented and mixed reality in planning and performing advanced liver surgery and living donor liver transplantation (with videos). Hepatobiliary Pancreat Dis Int, 2022, 21(5): 455-461.
25. 王峻峰, 赞左, 宋正己, 等. 基于3D打印/混合现实技术在胰周感染性积液治疗的应用. 国际临床研究杂志, 2021, 5(1): 24-28.
26. Wierzbicki R, Pawłowicz M, Job J, et al. 3D mixed-reality visualization of medical imaging data as a supporting tool for innovative, minimally invasive surgery for gastrointestinal tumors and systemic treatment as a new path in personalized treatment of advanced cancer diseases. J Cancer Res Clin Oncol, 2022, 148(1): 237-243.
27. Saito Y, Sugimoto M, Imura S, et al. Intraoperative 3D hologram support with mixed reality techniques in liver surgery. Ann Surg, 2020, 271(1): e4-e7. doi: 10.1097/SLA.0000000000003552.
28. Strickland A, Fairhurst K, Lauder C, et al. Development of an ex vivo simulated training model for laparoscopic liver resection. Surg Endosc, 2011, 25(5): 1677-1682.
29. 李鹏超, 袁文博, 万芬, 等. 混合现实技术在住院医师规范化培训外科手术教学中的应用研究. 创新创业理论研究与实践, 2022, 5(22): 179-184.
30. 吕秀琴, 张生海. 基于HoloLens2的DEM空间分析实验教学混合现实环境研究. 实验技术与管理. 2021, 38(10): 124-129.
31. Pelanis E, Kumar RP, Aghayan DL, et al. Use of mixed reality for improved spatial understanding of liver anatomy. Minim Invasive Ther Allied Technol, 2020, 29(3): 154-160.
32. 景涛, 邱阳, 李静尧, 等. 混合现实技术在住院医师规范化培训临床手术能力教学中的应用. 重庆医学, 2021, 50(8): 1435-1437.
33. 李忠海, 徐钢, 高淑贤, 等. 混合现实技术在外科临床教学中的应用及效果评价. 中国高等医学教育, 2022, (9): 67-69.
34. 王宏, 欧晏娇, 胡鹏, 等. 混合现实技术在肝胆外科手术教学中的应用效果. 中华医学教育探索杂志, 2019, 18(12): 1230-1234.
35. Bala L, Kinross J, Martin G, et al. A remote access mixed reality teaching ward round. Clin Teach, 2021, 18(4): 386-390.
36. 石小强, 马艳飞, 梁挺, 等. 混合现实技术在专科医师培训中辅助腹膜后肿瘤手术临床教学的应用研究. 中国医学教育技术, 2020, 34(4): 504-507.
37. Liu S, Wang H, Wang C, et al. Intra-operative 3D hologram support with mixed reality technique based on CT-MRI fusion images: operation guidance for brain brachytherapy. J Contemp Brachytherapy, 2021, 13(2): 205-210.
38. Zhou Z, Yang Z, Jiang S, et al. Design and validation of a surgical navigation system for brachytherapy based on mixed reality. Med Phys, 2019, 46(8): 3709-3718.
39. van den Berg NS, Engelen T, Brouwer OR, et al. A pilot study of SPECT/CT-based mixed-reality navigation towards the sentinel node in patients with melanoma or Merkel cell carcinoma of a lower extremity. Nucl Med Commun, 2016, 37(8): 812-817.
40. Davrieux CF, Giménez ME, González CA, et al. Mixed reality navigation system for ultrasound-guided percutaneous punctures: a pre-clinical evaluation. Surg Endosc, 2020, 34(1): 226-230.
41. Tu P, Qin C, Guo Y, et al. Ultrasound image guided and mixed reality-based surgical system with real-time soft tissue deformation computing for robotic cervical pedicle screw placement. IEEE Trans Biomed Eng, 2022, 69(8): 2593-2603.
42. Li R, Pan J, Yang Y, et al. Accurate and robust feature description and dense point-wise matching based on feature fusion for endoscopic images. Comput Med Imaging Graph, 2021, 94: 102007. doi: 10.1016/j.compmedimag.2021.102007.
43. Bloom D, Colombo JN, Miller N, et al. Early preclinical experience of a mixed reality ultrasound system with active GUIDance for NEedle-based interventions: The GUIDE study. Cardiovasc Digit Health J, 2022, 3(5): 232-240.

CHINESE JOURNAL OF BASES AND CLINICS IN GENERAL SURGERY

Application progress of mixed reality technology in hepatobiliary and pancreatic fields

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content