-
JMIR Medical Education Jun 2023The competence of ChatGPT (Chat Generative Pre-Trained Transformer) in non-English languages is not well studied.
BACKGROUND
The competence of ChatGPT (Chat Generative Pre-Trained Transformer) in non-English languages is not well studied.
OBJECTIVE
This study compared the performances of GPT-3.5 (Generative Pre-trained Transformer) and GPT-4 on the Japanese Medical Licensing Examination (JMLE) to evaluate the reliability of these models for clinical reasoning and medical knowledge in non-English languages.
METHODS
This study used the default mode of ChatGPT, which is based on GPT-3.5; the GPT-4 model of ChatGPT Plus; and the 117th JMLE in 2023. A total of 254 questions were included in the final analysis, which were categorized into 3 types, namely general, clinical, and clinical sentence questions.
RESULTS
The results indicated that GPT-4 outperformed GPT-3.5 in terms of accuracy, particularly for general, clinical, and clinical sentence questions. GPT-4 also performed better on difficult questions and specific disease questions. Furthermore, GPT-4 achieved the passing criteria for the JMLE, indicating its reliability for clinical reasoning and medical knowledge in non-English languages.
CONCLUSIONS
GPT-4 could become a valuable tool for medical education and clinical support in non-English-speaking regions, such as Japan.
PubMed: 37384388
DOI: 10.2196/48002 -
European Radiology Jun 2023To provide insights for on-site development of transformer-based structuring of free-text report databases by investigating different labeling and pre-training...
OBJECTIVES
To provide insights for on-site development of transformer-based structuring of free-text report databases by investigating different labeling and pre-training strategies.
METHODS
A total of 93,368 German chest X-ray reports from 20,912 intensive care unit (ICU) patients were included. Two labeling strategies were investigated to tag six findings of the attending radiologist. First, a system based on human-defined rules was applied for annotation of all reports (termed "silver labels"). Second, 18,000 reports were manually annotated in 197 h (termed "gold labels") of which 10% were used for testing. An on-site pre-trained model (T) using masked-language modeling (MLM) was compared to a public, medically pre-trained model (T). Both models were fine-tuned on silver labels only, gold labels only, and first with silver and then gold labels (hybrid training) for text classification, using varying numbers (N: 500, 1000, 2000, 3500, 7000, 14,580) of gold labels. Macro-averaged F1-scores (MAF1) in percent were calculated with 95% confidence intervals (CI).
RESULTS
T (95.5 [94.5-96.3]) showed significantly higher MAF1 than T (75.0 [73.4-76.5]) and T (75.2 [73.6-76.7]), but not significantly higher MAF1 than T (94.7 [93.6-95.6]), T (94.9 [93.9-95.8]), and T (95.2 [94.3-96.0]). When using 7000 or less gold-labeled reports, T (N: 7000, 94.7 [93.5-95.7]) showed significantly higher MAF1 than T (N: 7000, 91.5 [90.0-92.8]). With at least 2000 gold-labeled reports, utilizing silver labels did not lead to significant improvement of T (N: 2000, 91.8 [90.4-93.2]) over T (N: 2000, 91.4 [89.9-92.8]).
CONCLUSIONS
Custom pre-training of transformers and fine-tuning on manual annotations promises to be an efficient strategy to unlock report databases for data-driven medicine.
KEY POINTS
• On-site development of natural language processing methods that retrospectively unlock free-text databases of radiology clinics for data-driven medicine is of great interest. • For clinics seeking to develop methods on-site for retrospective structuring of a report database of a certain department, it remains unclear which of previously proposed strategies for labeling reports and pre-training models is the most appropriate in context of, e.g., available annotator time. • Using a custom pre-trained transformer model, along with a little annotation effort, promises to be an efficient way to retrospectively structure radiological databases, even if not millions of reports are available for pre-training.
Topics: Humans; Databases, Factual; Natural Language Processing; Radiology; Retrospective Studies; Color
PubMed: 36905469
DOI: 10.1007/s00330-023-09526-y -
Mathematical Biosciences and... Aug 2023Social media contains useful information about people and society that could help advance research in many different areas of health (e.g. by applying opinion mining,...
Social media contains useful information about people and society that could help advance research in many different areas of health (e.g. by applying opinion mining, emotion/sentiment analysis and statistical analysis) such as mental health, health surveillance, socio-economic inequality and gender vulnerability. User demographics provide rich information that could help study the subject further. However, user demographics such as gender are considered private and are not freely available. In this study, we propose a model based on transformers to predict the user's gender from their images and tweets. The image-based classification model is trained in two different methods: using the profile image of the user and using various image contents posted by the user on Twitter. For the first method a Twitter gender recognition dataset, publicly available on Kaggle and for the second method the PAN-18 dataset is used. Several transformer models, i.e. vision transformers (ViT), LeViT and Swin Transformer are fine-tuned for both of the image datasets and then compared. Next, different transformer models, namely, bidirectional encoders representations from transformers (BERT), RoBERTa and ELECTRA are fine-tuned to recognize the user's gender by their tweets. This is highly beneficial, because not all users provide an image that indicates their gender. The gender of such users could be detected from their tweets. The significance of the image and text classification models were evaluated using the Mann-Whitney U test. Finally, the combination model improved the accuracy of image and text classification models by 11.73 and 5.26% for the Kaggle dataset and by 8.55 and 9.8% for the PAN-18 dataset, respectively. This shows that the image and text classification models are capable of complementing each other by providing additional information to one another. Our overall multimodal method has an accuracy of 88.11% for the Kaggle and 89.24% for the PAN-18 dataset and outperforms state-of-the-art models. Our work benefits research that critically require user demographic information such as gender to further analyze and study social media content for health-related issues.
Topics: Humans; Social Media; Electric Power Supplies; Research Design
PubMed: 37919997
DOI: 10.3934/mbe.2023711 -
Scientific Reports Sep 2023Single image dehazing has received a lot of concern and achieved great success with the help of deep-learning models. Yet, the performance is limited by the local...
Single image dehazing has received a lot of concern and achieved great success with the help of deep-learning models. Yet, the performance is limited by the local limitation of convolution. To address such a limitation, we design a novel deep learning dehazing model by combining the transformer and guided filter, which is called as Deep Guided Transformer Dehazing Network. Specially, we address the limitation of convolution via a transformer-based subnetwork, which can capture long dependency. Haze is dependent on the depth, which needs global information to compute the density of haze, and removes haze from the input images correctly. To restore the details of dehazed result, we proposed a CNN sub-network to capture the local information. To overcome the slow speed of the transformer-based subnetwork, we improve the dehazing speed via a guided filter. Extensive experimental results show consistent improvement over the state-of-the-art dehazing on natural haze and simulated haze images.
PubMed: 37714880
DOI: 10.1038/s41598-023-41561-z -
Korean Journal of Radiology Jan 2024
PubMed: 38184774
DOI: 10.3348/kjr.2023.0948 -
Biology Jul 2023The emergence and rapid development of deep learning, specifically transformer-based architectures and attention mechanisms, have had transformative implications across... (Review)
Review
The emergence and rapid development of deep learning, specifically transformer-based architectures and attention mechanisms, have had transformative implications across several domains, including bioinformatics and genome data analysis. The analogous nature of genome sequences to language texts has enabled the application of techniques that have exhibited success in fields ranging from natural language processing to genomic data. This review provides a comprehensive analysis of the most recent advancements in the application of transformer architectures and attention mechanisms to genome and transcriptome data. The focus of this review is on the critical evaluation of these techniques, discussing their advantages and limitations in the context of genome data analysis. With the swift pace of development in deep learning methodologies, it becomes vital to continually assess and reflect on the current standing and future direction of the research. Therefore, this review aims to serve as a timely resource for both seasoned researchers and newcomers, offering a panoramic view of the recent advancements and elucidating the state-of-the-art applications in the field. Furthermore, this review paper serves to highlight potential areas of future investigation by critically evaluating studies from 2019 to 2023, thereby acting as a stepping-stone for further research endeavors.
PubMed: 37508462
DOI: 10.3390/biology12071033 -
Diagnostics (Basel, Switzerland) Mar 2024Ultrasound (US) has become a widely used imaging modality in clinical practice, characterized by its rapidly evolving technology, advantages, and unique challenges, such... (Review)
Review
Ultrasound (US) has become a widely used imaging modality in clinical practice, characterized by its rapidly evolving technology, advantages, and unique challenges, such as a low imaging quality and high variability. There is a need to develop advanced automatic US image analysis methods to enhance its diagnostic accuracy and objectivity. Vision transformers, a recent innovation in machine learning, have demonstrated significant potential in various research fields, including general image analysis and computer vision, due to their capacity to process large datasets and learn complex patterns. Their suitability for automatic US image analysis tasks, such as classification, detection, and segmentation, has been recognized. This review provides an introduction to vision transformers and discusses their applications in specific US image analysis tasks, while also addressing the open challenges and potential future trends in their application in medical US image analysis. Vision transformers have shown promise in enhancing the accuracy and efficiency of ultrasound image analysis and are expected to play an increasingly important role in the diagnosis and treatment of medical conditions using ultrasound imaging as technology progresses.
PubMed: 38473014
DOI: 10.3390/diagnostics14050542 -
MedRxiv : the Preprint Server For... Aug 2023Many clinical applications require medical image harmonization to combine and normalize images from different scanners or protocols. This paper introduces a...
Many clinical applications require medical image harmonization to combine and normalize images from different scanners or protocols. This paper introduces a Transformer-based MR image harmonization method. Our proposed method leverages the self-attention mechanism of the Transformer to learn the complex relationships between image patches and effectively transfer the imaging characteristics from a source image domain to a target image domain. We evaluate our approach to state-of-the-art methods using a publicly available dataset of brain MRI scans and show that it provides superior quantitative metrics and visual quality. Furthermore, we demonstrate that the proposed approach is highly resistant to fluctuations in image modality, resolution, and noise. Overall, the experiment results indicate that our approach is a promising method for medical image harmonization that can improve the accuracy and reliability of automated analysis and diagnosis in clinical settings.
PubMed: 37662360
DOI: 10.1101/2023.08.16.23294184 -
BMC Bioinformatics Nov 2023Galaxy is a web-based open-source platform for scientific analyses. Researchers use thousands of high-quality tools and workflows for their respective analyses in...
BACKGROUND
Galaxy is a web-based open-source platform for scientific analyses. Researchers use thousands of high-quality tools and workflows for their respective analyses in Galaxy. Tool recommender system predicts a collection of tools that can be used to extend an analysis. In this work, a tool recommender system is developed by training a transformer on workflows available on Galaxy Europe and its performance is compared to other neural networks such as recurrent, convolutional and dense neural networks.
RESULTS
The transformer neural network achieves two times faster convergence, has significantly lower model usage (model reconstruction and prediction) time and shows a better generalisation that goes beyond training workflows than the older tool recommender system created using RNN in Galaxy. In addition, the transformer also outperforms CNN and DNN on several key indicators. It achieves a faster convergence time, lower model usage time, and higher quality tool recommendations than CNN. Compared to DNN, it converges faster to a higher precision@k metric (approximately 0.98 by transformer compared to approximately 0.9 by DNN) and shows higher quality tool recommendations.
CONCLUSION
Our work shows a novel usage of transformers to recommend tools for extending scientific workflows. A more robust tool recommendation model, created using a transformer, having significantly lower usage time than RNN and CNN, higher precision@k than DNN, and higher quality tool recommendations than all three neural networks, will benefit researchers in creating scientifically significant workflows and exploratory data analysis in Galaxy. Additionally, the ability to train faster than all three neural networks imparts more scalability for training on larger datasets consisting of millions of tool sequences. Open-source scripts to create the recommendation model are available under MIT licence at https://github.com/anuprulez/galaxy_tool_recommendation_transformers.
Topics: Software; Neural Networks, Computer; Workflow; Data Analysis; Europe
PubMed: 38012574
DOI: 10.1186/s12859-023-05573-w