{"title":"基于视觉和文本特征提取技术的医疗领域放射学图像骨骼图像视觉问答系统","authors":"Jinesh Melvin Y.I., Mukesh Shrimali, Sushopti Gawade","doi":"10.1007/s40745-024-00553-0","DOIUrl":null,"url":null,"abstract":"<div><p>The Medical Imaging Query Response System is among the most challenging concepts in the medical field. It requires a significant amount of effort to organize and comprehend the various representations of the human body. Additionally, the system needs to be verified by users in the healthcare industry. With the aid of various images, including MRI scans, CT scans, ultrasounds, X-rays, PET-CT scans, and more, it may be possible to identify human health issues. It is anticipated to encourage patient participation and support clinical decision-making. As a result of the use of a number of characteristics that are inadequately matched to medical images and questions, technically, the VQA system in the healthcare domain is more complicated than in the common domain. The challenges were caused by the datasets, approaches, and models used for both visual and textual aspects. This can sometimes make it harder for clinical assistance to provide relevant answers. The proposed system will analyze current models and diagnose the problem in order to improve the medical visual question-answering system for recent datasets. The models that were compared to the model were convolutional neural networks (CNN), deep belief networks (DBN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and bidirectional long short-term memory (BiLSTM). To assess the effectiveness of each model, the following measures should be used: Classification Accuracy, F-Classification, F-Measure, C-False Negative Rate (FNR), C-Positive Predictive Value, C-Precision, C-Recall, C-Sensitivity, and C-True Positive Rate (CTPR) With the objective of improving the performance of any dataset with accuracy and measures for both visual and textual features to get the right answers for given questions, the proposed system helps to recognize how ideal the existing models are and generates new models using the B12 FASTER Recurrent Neural Network (RNN) and Kai-Bi-LSTM. With questions and appropriate answers, the suggested model will assist in extracting the features of imported images and text.</p></div>","PeriodicalId":36280,"journal":{"name":"Annals of Data Science","volume":"12 3","pages":"969 - 990"},"PeriodicalIF":0.0000,"publicationDate":"2024-06-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Visual Question Answer System for Skeletal Image Using Radiology Images in the Healthcare Domain Based on Visual and Textual Feature Extraction Techniques\",\"authors\":\"Jinesh Melvin Y.I., Mukesh Shrimali, Sushopti Gawade\",\"doi\":\"10.1007/s40745-024-00553-0\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>The Medical Imaging Query Response System is among the most challenging concepts in the medical field. It requires a significant amount of effort to organize and comprehend the various representations of the human body. Additionally, the system needs to be verified by users in the healthcare industry. With the aid of various images, including MRI scans, CT scans, ultrasounds, X-rays, PET-CT scans, and more, it may be possible to identify human health issues. It is anticipated to encourage patient participation and support clinical decision-making. As a result of the use of a number of characteristics that are inadequately matched to medical images and questions, technically, the VQA system in the healthcare domain is more complicated than in the common domain. The challenges were caused by the datasets, approaches, and models used for both visual and textual aspects. This can sometimes make it harder for clinical assistance to provide relevant answers. The proposed system will analyze current models and diagnose the problem in order to improve the medical visual question-answering system for recent datasets. The models that were compared to the model were convolutional neural networks (CNN), deep belief networks (DBN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and bidirectional long short-term memory (BiLSTM). To assess the effectiveness of each model, the following measures should be used: Classification Accuracy, F-Classification, F-Measure, C-False Negative Rate (FNR), C-Positive Predictive Value, C-Precision, C-Recall, C-Sensitivity, and C-True Positive Rate (CTPR) With the objective of improving the performance of any dataset with accuracy and measures for both visual and textual features to get the right answers for given questions, the proposed system helps to recognize how ideal the existing models are and generates new models using the B12 FASTER Recurrent Neural Network (RNN) and Kai-Bi-LSTM. With questions and appropriate answers, the suggested model will assist in extracting the features of imported images and text.</p></div>\",\"PeriodicalId\":36280,\"journal\":{\"name\":\"Annals of Data Science\",\"volume\":\"12 3\",\"pages\":\"969 - 990\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-06-29\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Annals of Data Science\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://link.springer.com/article/10.1007/s40745-024-00553-0\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"Decision Sciences\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of Data Science","FirstCategoryId":"1085","ListUrlMain":"https://link.springer.com/article/10.1007/s40745-024-00553-0","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Decision Sciences","Score":null,"Total":0}
Visual Question Answer System for Skeletal Image Using Radiology Images in the Healthcare Domain Based on Visual and Textual Feature Extraction Techniques
The Medical Imaging Query Response System is among the most challenging concepts in the medical field. It requires a significant amount of effort to organize and comprehend the various representations of the human body. Additionally, the system needs to be verified by users in the healthcare industry. With the aid of various images, including MRI scans, CT scans, ultrasounds, X-rays, PET-CT scans, and more, it may be possible to identify human health issues. It is anticipated to encourage patient participation and support clinical decision-making. As a result of the use of a number of characteristics that are inadequately matched to medical images and questions, technically, the VQA system in the healthcare domain is more complicated than in the common domain. The challenges were caused by the datasets, approaches, and models used for both visual and textual aspects. This can sometimes make it harder for clinical assistance to provide relevant answers. The proposed system will analyze current models and diagnose the problem in order to improve the medical visual question-answering system for recent datasets. The models that were compared to the model were convolutional neural networks (CNN), deep belief networks (DBN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and bidirectional long short-term memory (BiLSTM). To assess the effectiveness of each model, the following measures should be used: Classification Accuracy, F-Classification, F-Measure, C-False Negative Rate (FNR), C-Positive Predictive Value, C-Precision, C-Recall, C-Sensitivity, and C-True Positive Rate (CTPR) With the objective of improving the performance of any dataset with accuracy and measures for both visual and textual features to get the right answers for given questions, the proposed system helps to recognize how ideal the existing models are and generates new models using the B12 FASTER Recurrent Neural Network (RNN) and Kai-Bi-LSTM. With questions and appropriate answers, the suggested model will assist in extracting the features of imported images and text.
期刊介绍:
Annals of Data Science (ADS) publishes cutting-edge research findings, experimental results and case studies of data science. Although Data Science is regarded as an interdisciplinary field of using mathematics, statistics, databases, data mining, high-performance computing, knowledge management and virtualization to discover knowledge from Big Data, it should have its own scientific contents, such as axioms, laws and rules, which are fundamentally important for experts in different fields to explore their own interests from Big Data. ADS encourages contributors to address such challenging problems at this exchange platform. At present, how to discover knowledge from heterogeneous data under Big Data environment needs to be addressed. ADS is a series of volumes edited by either the editorial office or guest editors. Guest editors will be responsible for call-for-papers and the review process for high-quality contributions in their volumes.