scholarly journals Medical domain knowledge in domain-agnostic generative AI

2022 ◽  
Author(s):  
Jakob Nikolas Kather ◽  
Narmin Ghaffari Laleh ◽  
Sebastian Foersch ◽  
Daniel Truhn
Keyword(s):  
Medical Image ◽  
Domain Knowledge ◽  
Medical Image Analysis ◽  
Medical Knowledge ◽  
Fine Tuning ◽  
Domain Specific ◽  
Additional Domain ◽  
Single Use ◽  
Explicit Training ◽  
Medical Concepts

The text-guided diffusion model GLIDE (Guided Language to Image Diffusion for Generation and Editing) is the state of the art in text-to-image generative artificial intelligence (AI). GLIDE has rich representations, but medical applications of this model have not been systematically explored. If GLIDE had useful medical knowledge, it could be used for medical image analysis tasks, a domain in which AI systems are still highly engineered towards a single use-case. Here we show that the publicly available GLIDE model has reasonably strong representations of key topics in cancer research and oncology, in particular the general style of histopathology images and multiple facets of diseases, pathological processes and laboratory assays. However, GLIDE seems to lack useful representations of the style and content of radiology data. Our findings demonstrate that domain-agnostic generative AI models can learn relevant medical concepts without explicit training. Thus, GLIDE and similar models might be useful for medical image processing tasks in the future - particularly with additional domain-specific fine-tuning.

scholarly journals Templated Text Synthesis for Expert-Guided Multi-Label Extraction from Radiology Reports

2021 ◽  
Vol 3 (2) ◽  
pp. 299-317
Author(s):  
Patrick Schrempf ◽  
Hannah Watson ◽  
Eunsoo Park ◽  
Maciej Pajak ◽  
Hamish MacKinnon ◽  
...  
Keyword(s):  
Domain Knowledge ◽  
Expert Knowledge ◽  
Medical Image Analysis ◽  
Imaging Data ◽  
Data Synthesis ◽  
Domain Specific ◽  
Radiology Reports ◽  
Training Examples ◽  
Analysis Models ◽  
Difficult Cases

Training medical image analysis models traditionally requires large amounts of expertly annotated imaging data which is time-consuming and expensive to obtain. One solution is to automatically extract scan-level labels from radiology reports. Previously, we showed that, by extending BERT with a per-label attention mechanism, we can train a single model to perform automatic extraction of many labels in parallel. However, if we rely on pure data-driven learning, the model sometimes fails to learn critical features or learns the correct answer via simplistic heuristics (e.g., that “likely” indicates positivity), and thus fails to generalise to rarer cases which have not been learned or where the heuristics break down (e.g., “likely represents prominent VR space or lacunar infarct” which indicates uncertainty over two differential diagnoses). In this work, we propose template creation for data synthesis, which enables us to inject expert knowledge about unseen entities from medical ontologies, and to teach the model rules on how to label difficult cases, by producing relevant training examples. Using this technique alongside domain-specific pre-training for our underlying BERT architecture i.e., PubMedBERT, we improve F1 micro from 0.903 to 0.939 and F1 macro from 0.512 to 0.737 on an independent test set for 33 labels in head CT reports for stroke patients. Our methodology offers a practical way to combine domain knowledge with machine learning for text classification tasks.

scholarly journals A survey on incorporating domain knowledge into deep learning for medical image analysis

2021 ◽  
Vol 69 ◽  
pp. 101985 ◽  
Author(s):  
Xiaozheng Xie ◽  
Jianwei Niu ◽  
Xuefeng Liu ◽  
Zhengsu Chen ◽  
Shaojie Tang ◽  
...  
Keyword(s):  
Image Analysis ◽  
Deep Learning ◽  
Medical Image ◽  
Domain Knowledge ◽  
Medical Image Analysis

scholarly journals Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?

2016 ◽  
Vol 35 (5) ◽  
pp. 1299-1312 ◽  
Author(s):  
Nima Tajbakhsh ◽  
Jae Y. Shin ◽  
Suryakanth R. Gurudu ◽  
R. Todd Hurst ◽  
Christopher B. Kendall ◽  
...  
Keyword(s):  
Neural Networks ◽  
Image Analysis ◽  
Convolutional Neural Networks ◽  
Medical Image ◽  
Medical Image Analysis ◽  
Fine Tuning

scholarly journals A review of transfer learning for medical image classification

2021 ◽  
Author(s):  
Hee E. Kim ◽  
Alejandro Cosa-Linan ◽  
Mate E. Maros ◽  
Nandhini Santhanam ◽  
Mahboubeh Jannesari ◽  
...  
Keyword(s):  
Image Classification ◽  
Transfer Learning ◽  
Medical Image ◽  
Medical Image Analysis ◽  
Fine Tuning ◽  
Data Sets ◽  
Learning Approaches ◽  
Limited Data ◽  
Feature Extractor ◽  
Learning Types

Abstract This review paper provides an overview of the peer-reviewed articles using transfer learning for medical image analysis, while also providing guidelines for selecting a convolutional neural network model and its configurations for the image classification task. The data characteristics and the trend of models and transfer learning types in the medical domain are additionally analyzed. Publications were retrieved from the databases PubMed and Web of Science of peer-reviewed articles published in English until December 31, 2020. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. With respect to the model, the majority of studies (n = 57) empirically evaluated numerous models followed by deep (n = 33) and shallow (n = 24) models. With respect to the transfer learning approaches, the majority of studies (n = 46) empirically searched for the optimal transfer learning configuration followed by feature extractor (n = 38) and fine-tuning scratch (n = 27), feature extractor hybrid (n = 7) and fine-tuning (n = 3). The investigated studies showed that transfer learning demonstrates either a better or at least a similar performance compared to medical experts despite the limited data sets. We hence encourage data scientists and practitioners to use models such as ResNet or Inception with a feature extractor approach, which saves computational costs and time without degrading the predictive power.

scholarly journals Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks

2020 ◽  
Vol 34 (05) ◽  
pp. 9233-9241
Author(s):  
Yong Wang ◽  
Longyue Wang ◽  
Shuming Shi ◽  
Victor O.K. Li ◽  
Zhaopeng Tu
Keyword(s):  
Unified Model ◽  
Fine Tuning ◽  
General Knowledge ◽  
Specific Knowledge ◽  
Adversarial Learning ◽  
Domain Specific ◽  
Domain Specific Knowledge ◽  
Additional Domain ◽  
Knowledge Distillation ◽  
Domain Transformation

The key challenge of multi-domain translation lies in simultaneously encoding both the general knowledge shared across domains and the particular knowledge distinctive to each domain in a unified model. Previous work shows that the standard neural machine translation (NMT) model, trained on mixed-domain data, generally captures the general knowledge, but misses the domain-specific knowledge. In response to this problem, we augment NMT model with additional domain transformation networks to transform the general representations to domain-specific representations, which are subsequently fed to the NMT decoder. To guarantee the knowledge transformation, we also propose two complementary supervision signals by leveraging the power of knowledge distillation and adversarial learning. Experimental results on several language pairs, covering both balanced and unbalanced multi-domain translation, demonstrate the effectiveness and universality of the proposed approach. Encouragingly, the proposed unified model achieves comparable results with the fine-tuning approach that requires multiple models to preserve the particular knowledge. Further analyses reveal that the domain transformation networks successfully capture the domain-specific knowledge as expected.1

scholarly journals Development of a Consumer Health Vocabulary by Mining Health Forum Texts Based on Word Embedding: Semiautomatic Approach (Preprint)

2018 ◽  
Author(s):  
Gen Gu ◽  
Xingting Zhang ◽  
Xingeng Zhu ◽  
Zhe Jian ◽  
Ken Chen ◽  
...  
Keyword(s):  
Vector Space ◽  
Information Seeking ◽  
Large Scale ◽  
Medical Knowledge ◽  
Semantic Distance ◽  
Word Embedding ◽  
Fine Tuning ◽  
Consumer Health ◽  
Text Information ◽  
Medical Concepts

BACKGROUND The vocabulary gap between consumers and professionals in the medical domain hinders information seeking and communication. Consumer health vocabularies have been developed to aid such informatics applications. This purpose is best served if the vocabulary evolves with consumers’ language. OBJECTIVE Our objective is to develop a method for identifying and adding new terms to consumer health vocabularies, so that it can keep up with the constantly evolving medical knowledge and language use. METHODS In this paper, we propose a consumer health term–finding framework based on a distributed word vector space model. We first learned word vectors from a large-scale text corpus and then adopted a supervised method with existing consumer health vocabularies for learning vector representation of words, which can provide additional supervised fine tuning after unsupervised word embedding learning. With a fine-tuned word vector space, we identified pairs of professional terms and their consumer variants by their semantic distance in the vector space. A subsequent manual review of the extracted and labeled pairs of entities was conducted to validate the results generated by the proposed approach. The results were evaluated using mean reciprocal rank (MRR). RESULTS Manual evaluation showed that it is feasible to identify alternative medical concepts by using professional or consumer concepts as queries in the word vector space without fine tuning, but the results are more promising in the final fine-tuned word vector space. The MRR values indicated that on an average, a professional or consumer concept is about 14th closest to its counterpart in the word vector space without fine tuning, and the MRR in the final fine-tuned word vector space is 8. Furthermore, the results demonstrate that our method can collect abbreviations and common typos frequently used by consumers. CONCLUSIONS By integrating a large amount of text information and existing consumer health vocabularies, our method outperformed several baseline ranking methods and is effective for generating a list of candidate terms for human review during consumer health vocabulary development.

Privacy-preserving Collaborative Training for Medical Image Analysis Based on Multi-Blockchain

2020 ◽  
Vol 23 ◽  
Author(s):  
Wanlu Zhang ◽  
Qigang Wang ◽  
Mei Li
Keyword(s):  
Medical Image ◽  
Data Privacy ◽  
Medical Image Analysis ◽  
Auxiliary Information ◽  
Training Process ◽  
Private Data ◽  
Medical Institutions ◽  
Model Training ◽  
Collaborative Training ◽  
Similar Task

Background: As artificial intelligence and big data analysis develop rapidly, data privacy, especially patient medical data privacy, is getting more and more attention. Objective: To strengthen the protection of private data while ensuring the model training process, this article introduces a multi-Blockchain-based decentralized collaborative machine learning training method for medical image analysis. In this way, researchers from different medical institutions are able to collaborate to train models without exchanging sensitive patient data. Method: Partial parameter update method is applied to prevent indirect privacy leakage during model propagation. With the peer-to-peer communication in the multi-Blockchain system, a machine learning task can leverage auxiliary information from another similar task in another Blockchain. In addition, after the collaborative training process, personalized models of different medical institutions will be trained. Results: The experimental results show that our method achieves similar performance with the centralized model-training method by collecting data sets of all participants and prevents private data leakage at the same time. Transferring auxiliary information from similar task on another Blockchain has also been proven to effectively accelerate model convergence and improve model accuracy, especially in the scenario of absence of data. Personalization training process further improves model performance. Conclusion: Our approach can effectively help researchers from different organizations to achieve collaborative training without disclosing their private data.

Combinatorial Double Auction Based Meta-scheduler for Medical Image Analysis Application in Grid Environment

2020 ◽  
Vol 13 (5) ◽  
pp. 999-1007
Author(s):  
Karthikeyan Periyasami ◽  
Arul Xavier Viswanathan Mariammal ◽  
Iwin Thanakumar Joseph ◽  
Velliangiri Sarveshwaran
Keyword(s):  
Image Analysis ◽  
Medical Image ◽  
Medical Image Analysis ◽  
Grid Resource ◽  
Resource Broker ◽  
Analysis Application ◽  
Local Grid ◽  
Grid Resource Broker ◽  
Global Grid ◽  
Grid Site

Background: Medical image analysis application has complex resource requirement. Scheduling Medical image analysis application is the complex task to the grid resources. It is necessary to develop a new model to improve the breast cancer screening process. Proposed novel Meta scheduler algorithm allocate the image analyse applications to the local schedulers and local scheduler submit the job to the grid node which analyses the medical image and generates the result sent back to Meta scheduler. Meta schedulers are distinct from the local scheduler. Meta scheduler and local scheduler have the aim at resource allocation and management. Objective: The main objective of the CDAM meta-scheduler is to maximize the number of jobs accepted. Methods: In the beginning, the user sends jobs with the deadline to the global grid resource broker. Resource providers sent information about the available resources connected in the network at a fixed interval of time to the global grid resource broker, the information such as valuation of the resource and number of an available free resource. CDAM requests the global grid resource broker for available resources details and user jobs. After receiving the information from the global grid resource broker, it matches the job with the resources. CDAM sends jobs to the local scheduler and local scheduler schedule the job to the local grid site. Local grid site executes the jobs and sends the result back to the CDAM. Success full completion of the job status and resource status are updated into the auction history database. CDAM collect the result from all local grid site and return to the grid users. Results: The CDAM was simulated using grid simulator. Number of jobs increases then the percentage of the jobs accepted also decrease due to the scarcity of resources. CDAM is providing 2% to 5% better result than Fair share Meta scheduling algorithm. CDAM algorithm bid density value is generated based on the user requirement and user history and ask value is generated from the resource details. Users who, having the most significant deadline are generated the highest bid value, grid resource which is having the fastest processor are generated lowest ask value. The highest bid is assigned to the lowest Ask it means that the user who is having the most significant deadline is assigned to the grid resource which is having the fastest processor. The deadline represents a time by which the user requires the result. The user can define the deadline by which the results are needed, and the CDAM will try to find the fastest resource available in order to meet the user-defined deadline. If the scheduler detects that the tasks cannot be completed before the deadline, then the scheduler abandons the current resource, tries to select the next fastest resource and tries until the completion of application meets the deadline. CDAM is providing 25% better result than grid way Meta scheduler this is because grid way Meta scheduler allocate jobs to the resource based on the first come first served policy. Conclusion: The proposed CDAM model was validated through simulation and was evaluated based on jobs accepted. The experimental results clearly show that the CDAM model maximizes the number of jobs accepted than conventional Meta scheduler. We conclude that a CDAM is highly effective meta-scheduler systems and can be used for an extraordinary situation where jobs have a combinatorial requirement.

Sign in / Sign up

Export Citation Format

Share Document