Neurosurgery navigation system, which is expensive and complicated to operate, has a low penetration rate, and is only found in some large medical institutions. In order to meet the needs of other small and medium-sized medical institutions for neurosurgical navigation systems, the scalp localization system of neurosurgery based on augmented reality (AR) theory was developed. AR technology is used to fuse virtual world images with real images. The system integrates computed tomography (CT) or magnetic resonance imaging (MRI) with the patient's head in real life to achieve the scalp positioning. This article focuses on the key points of Digital Imaging and Communications in Medicine (DICOM) standard, three-dimensional (3D) reconstruction, and AR image layer fusion in medical image visualization. This research shows that the system is suitable for a variety of mobile phones, can achieve two-dimensional (2D) image display, 3D rendering and clinical scalp positioning application, which has a certain significance for the auxiliary neurosurgical head surface positioning.
Brain-computer interface (BCI) has great potential to replace lost upper limb function. Thus, there has been great interest in the development of BCI-controlled robotic arm. However, few studies have attempted to use noninvasive electroencephalography (EEG)-based BCI to achieve high-level control of a robotic arm. In this paper, a high-level control architecture combining augmented reality (AR) BCI and computer vision was designed to control a robotic arm for performing a pick and place task. A steady-state visual evoked potential (SSVEP)-based BCI paradigm was adopted to realize the BCI system. Microsoft's HoloLens was used to build an AR environment and served as the visual stimulator for eliciting SSVEPs. The proposed AR-BCI was used to select the objects that need to be operated by the robotic arm. The computer vision was responsible for providing the location, color and shape information of the objects. According to the outputs of the AR-BCI and computer vision, the robotic arm could autonomously pick the object and place it to specific location. Online results of 11 healthy subjects showed that the average classification accuracy of the proposed system was 91.41%. These results verified the feasibility of combing AR, BCI and computer vision to control a robotic arm, and are expected to provide new ideas for innovative robotic arm control approaches.
ObjectiveTo study clinical practical value of multimode imaging technique in precise hepatectomy for huge hepatocellular carcinoma (HCC). MethodsThe clinicopathologic data of patients with huge HCC who underwent precise hepatectomy in Yuebei People’s Hospital from Jan. 2018 to Dec. 2020 were collected. The three-dimensional (3D) reconstruction, 3D visualization, 3D printing, and augmented reality (AR) were used to guide preoperative evaluation, surgical planning, and surgical navigation. The liver function indexes, surgical mode, operative time, intraoperative bleeding, volume of resected liver, postoperative hospitalization, and complications were analyzed. ResultsThere were 23 patients in this study, including 18 males and 5 females, with (56.8±8.1) years old. The virtual tumor volume assessed by multimodal imaging technology was (865.2±165.6) mL and the virtual resected liver volume was (1 628.8±144.4) mL. The planned operations were anatomical hepatectomy in 19 patients and non-anatomical hepatectomy in 4 patients. The actual operation included 17 cases of anatomical hepatectomy and 6 cases of non-anatomical hepatectomy, which was basically consistent with the results of AR. The operative time was (298.4±74.5) min, the median hepatic blood flow blocking time was 20 min, and the intraoperative bleeding was (330.4±152.8) mL. Compared with preoperative levels, the levels of hemoglobin and albumin decreased temporarily on the first day after operation (P<0.05), and then which began to rise on the third day and basically rose to the normal range; prothrombintime, total bilirubin, alanine aminotransferase, and aspartate aminotransferase increased transiently on the first day after operation (P<0.05), then which began to decline to the normal levels. There were no serious operative complications and no perioperative death. The median follow-up time was 18 months, the tumor recurrence and metastasis occurred in 3 cases. ConclusionFrom preliminary results of this study, it could improve surgical safety and precision of hepatectomy for huge HCC by preoperative precise assessment and operation navigation in good time of multimode imaging technology.