Recent studies have introduced attention models for medical visual question answering (MVQA). In medical research, not only is the modeling of “visual attention” crucial, but the modeling of “question attention” is equally significant. To facilitate bidirectional reasoning in the attention processes involving medical images and questions, a new MVQA architecture, named MCAN, has been proposed. This architecture incorporated a cross-modal co-attention network, FCAF, which identifies key words in questions and principal parts in images. Through a meta-learning channel attention module (MLCA), weights were adaptively assigned to each word and region, reflecting the model’s focus on specific words and regions during reasoning. Additionally, this study specially designed and developed a medical domain-specific word embedding model, Med-GloVe, to further enhance the model’s accuracy and practical value. Experimental results indicated that MCAN proposed in this study improved the accuracy by 7.7% on free-form questions in the Path-VQA dataset, and by 4.4% on closed-form questions in the VQA-RAD dataset, which effectively improves the accuracy of the medical vision question answer.
Objective To develop an innovative recognition algorithm that aids physicians in the identification of pulmonary nodules. MethodsPatients with pulmonary nodules who underwent thoracoscopic surgery at the Department of Thoracic Surgery, Affiliated Drum Tower Hospital of Nanjing University Medical School in December 2023, were enrolled in the study. Chest surface exploration data were collected at a rate of 60 frames per second and a resolution of 1 920×1 080. Frame images were saved at regular intervals for subsequent block processing. An algorithm database for lung nodule recognition was developed using the collected data. ResultsA total of 16 patients were enrolled, including 9 males and 7 females, with an average age of (54.9±14.9) years. In the optimized multi-topology convolutional network model, the test results demonstrated an accuracy rate of 94.39% for recognition tasks. Furthermore, the integration of micro-variation amplification technology into the convolutional network model enhanced the accuracy of lung nodule identification to 96.90%. A comprehensive evaluation of the performance of these two models yielded an overall recognition accuracy of 95.59%. Based on these findings, we conclude that the proposed network model is well-suited for the task of lung nodule recognition, with the convolutional network incorporating micro-variation amplification technology exhibiting superior accuracy. Conclusion Compared to traditional methods, our proposed technique significantly enhances the accuracy of lung nodule identification and localization, aiding surgeons in locating lung nodules during thoracoscopic surgery.
Objective To evaluate the visual function before and after photodynamic therapy(PDT) in patients with subfoveal choroidal neovascularization ( CNV) caused by age-related macular degeneration (AMD). Methods Twenty-five consecutive patients (34 eyes) treated with PDT (verteporfin) for subfoveal CNV in age-related macular degeneration diagnosed by fluorescein angiography (FFA), indocyanine green angiography (ICGA) and optical coherence tomography (OCT). Visual function including best corrected visual acuity, contrast sensitivity frequency, binocular function, confusion, stereo-vision, color vision, metamorphopsis and central scotoma were examed before photodynamic therapy and 1 week, 1 month, 3 month after photodynamic therapy. The follow-up time varied from 3 months to 2 years (mean 7.6 months). Results The changes of visual function at the 3rd month after photodynamic therapy revealed improving in 13 eyes (38.24%), without any change in 17 eyes (50.00%), and decreasing in 6 eyes (17.65%). Visual acuity with logMRA improved after photodynamic therapy, but without statistic difference. All spatial contrast sensitivity improved. Contrast sensitivity for spatial frequencies 5 cycles per degree (cpd) was better after photodynamic therapy with significant difference (P=0.045).Binocular function, fusion function, stereo function and color vision were slightly improved without statistic difference. Conclusion Damage of visual function in macular degeneration is many-sided. The treatment of PDT for exudative AMD can improve part visual function. (Chin J Ocul Fundus Dis，2004，20：289-291)
A new form of static random-dot stereograms free of monocular clues was designed as stimulus to elicit disparity evoked potentials in 40 normal subjects.A characteristic wide positive wave at about 250ms was consistently recorded in disparity stimulation,which may be regard as evdence of the presence of stereopsis.In constrast,recordings for monocular stimulus all demonstrated a relatively smaller and sharper positive wave with a markedly shortened latency. （Chin J Ocul Fundus Dis,1993,9:214-217）
Attention can concentrate our mental resources on processing certain interesting objects, which is an important mental behavior and cognitive process. Recognizing attentional states have great significance in improving human’s performance and reducing errors. However, it still lacks a direct and standardized way to monitor a person’s attentional states. Based on the fact that visual attention can modulate the steady-state visual evoked potential (SSVEP), we designed a go/no-go experimental paradigm with 10 Hz steady state visual stimulation in background to investigate the separability of SSVEP features modulated by different visual attentional states. The experiment recorded the EEG signals of 15 postgraduate volunteers under high and low visual attentional states. High and low visual attentional states are determined by behavioral responses. We analyzed the differences of SSVEP signals between the high and low attentional levels, and applied classification algorithms to recognize such differences. Results showed that the discriminant canonical pattern matching (DCPM) algorithm performed better compared with the linear discrimination analysis (LDA) algorithm and the canonical correlation analysis (CCA) algorithm, which achieved up to 76% in accuracy. Our results show that the SSVEP features modulated by different visual attentional states are separable, which provides a new way to monitor visual attentional states.
Objective To observe the functional state of the optic nerve and discover the injury of visual pathway function in time under general ane sthesia. The flash visual evoked potential (F-VEP) was used to monitor visual function during orbital surgery. Methods A total of 252 out of 282 patients undergoing orbital surgery under general anesthesia were successfully monitored by F-VEP during the surgery. All patients were monitored by this method under the following conditions:consious state before operation, under general anaesthesia, during and after dissection of orbital tumor and at the end of operation. Results ①There was no significant difference of wave amplitude and latency under general anesthesia and consciousness condition. ②The amplitude and latency of F-VEP were normal in the orbital surgery withou toptic nerve injury. ③Pulling and oppression of optic nerve could cause temporary wave loss, but the wave recovered after removal of the pull and oppression. ④ The wave loss of F-VEP would occur immedicately when optic nerve was severe injured and its blood supply was deficient. Since the application of the visual function monitoring, 24 cases were treated in time during disturbance of visual function and no patient has unexpected visual loss during orbital surgery. Conclusion The intraoperative monitoring of F-VEP during orbital surgery can decrease the proportion of permanent visual loss caused by orbit al surgery, and help the surgical procedures go to function-anatomy stage from experience-anatomy stage. (Chin J Ocul Fundus Dis, 2001,17:260-263)
Objective To assess the change of the motion perception of cones in patients with AMD. Methods Motion perceptions of B、G and R cones of 20 eyes with dry type AMD were isolated by blue,green and red random dots stimulus displayed on the yellow,purple and blue-green background respectively,and keeping equiluminance between stimulus and background. Results The detecting displacement threshold and identifying threshold of motion perception from B,G and R cones were all damaged in eyes with dry type AMD compared with the normal group. Conclusion The abnormal changes of motion perception may reveal preclinical damage of visual function damage in dry type AMD. (Chin J Ocul Fundus Dis, 1999, 15: 219-221)
With the development of science and technology, artificial intelligence is gradually integrated into every aspect of daily life and the medical field is no exception. Cardiovascular diseases, as the first killer to global health, is the focus of new technologies and methods. In this study, the application of computer vision, natural language processing, robotics and machine learning in cardiovascular disease studies were reviewed and prospected, in order to promote the development for new technologies and applications in the future.
In order to study the effect of light with different wavelengths on the motion behavior of carp robots, phototaxis experiment, anatomical experiment, light control experiment and speed measurement experiment were carried out in this study. Blue, green, yellow and red light with different wavelength were used to conduct phototaxis experiments on carp to observe their movement behavior. By dissecting the skull bones of the carp to determine the appropriate location to carry the light control device, we independently developed a light control carrying device which was suitable for any illumination intensity environment. The experiment of the light-controlled carp robots was carried out. The motion behavior of the carp robot was checked by using computer binocular stereo vision technology. The motion trajectory of the carp robot was tracked and obtained by applying kernel correlation filter (KCF) algorithm. The motion velocity of the carp robot at different wavelengths was calculated according to their motion trajectory. The results showed that carps’ sensitivity to different light changed from strong to weak in the order of blue, red, yellow and green, so that using light with different wavelengths to control the speed of the carp robot has certain laws to follow. A new method to avoid brain damage in carp robots control can be provided in this study.