Chapter 8: Clinical Applications of
Generative AI in Radiology
Authors
ORCID
Abstract
This chapter discusses
Introduction
The adoption of artificial intelligence (AI) into healthcare has sparked substantial advances in
both clinical and administrative areas, with radiology at the forefront of this transition.
Radiology, a medical speciality that relies on modern imaging modalities including computed
tomography (CT), magnetic resonance imaging (MRI), and X-rays, is critical for diagnosing,
monitoring, and directing therapy for a variety of disorders. The exponential growth in
imaging needs, along with the complexity of current diagnostic procedures, has put an
unprecedented strain on radiology departments. Conventional radiology procedures are
labour-intensive, time-consuming, and prone to human mistake. A typical imaging pipeline
consists of several phases, including image capture, processing, interpretation, and
reporting. Each level creates potential for inefficiency. For example, obtaining high-quality
pictures frequently necessitates longer scan periods or more radiation exposure, both of
which have consequences for patient safety. This has prompted novel ways to improve
effectiveness and precision while maintaining high levels of patient care. Among recent
developments, generative AI has shown exceptional promise in resolving these difficulties,
ushering in a new age of AI-driven innovation in radiology. Key applications include image
synthesis, image generation and enhancement, image translation, report generation and
data augmentation and patient communication. Medical imaging not only helps detect and
assess diseases but it also informs clinical decision-making and analyses treatments
efficiency. This chapter will also discuss the principal generative models that are Variational
Autoencoders (VAE), Generative Adversarial Networks (GAN), Diffusion Models, Large
Language Models, and Vision-Language Models that enable these applications.
Generative AI Models
Generative AI models change the game of medical technologies, and the field of radiology in
particular, since it allows the generation of synthetic data that is close to actual medical
datasets. These models utilize a probabilistic approach to envisage and fabricate data
examples along with various applications such as images restoration and synthetic data
generation that does not compromise confidentiality.
Generative models can be classified based on the types of data they handle (vision,
language, or both) as well as their training and sample generation methods (explicit vs
implicit). Explicit models are those that specify the probability distribution over data
generated and therefore allow making predictions by evaluating likelihoods. On the other
hand, implicit models create some data through random procedures but do not attempt to
compute or specify an underlying probability distribution. Vision-based generative models,
such as Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and
diffusion models, are specifically designed for image processing (see Figure 1) (Kim et al.,
2024).
<Figure 1 here>
Figure 1: Typical architectures of generative models for medical images. A: Variational auto-encoder. B: Generative
adversarial network. C: Diffusion model. x = input image, z = latent space, N = normal distribution, ε = noise, x’ =
reconstructed image, T = timestep. (Figure - PMC, n.d.)
Generative AI in Radiology
Generative AI models represented a transformational scope in radiology by providing
solutions for image production and diagnostic accuracy. They are intended to produce
realistic medical images from real data with high accuracy. They are powerful because they
can learn complicated distributions from training data, allowing the model to generate new
images with the same statistical features as the original dataset. Generative Adversarial
Networks (GANs) use an iteratively competitive generator-discriminator network pair to
generate high-resolution images. GANs have been used to minimise radiation exposure by
denoising low-dose CT scans without sacrificing diagnostic quality, and they have expedited
MRI scans by reconstructing high-quality images from under sampled data, resulting in
shorter scan times. Unlike GANs that focus on realistic image synthesis, Variational
Autoencoders (VAEs) encode input images into latent representation through probabilistic
methods with a high capability, VAE not only for high-quality image synthesis with realistic
images, but also for effective anomaly detection. Diffusion models, an additional
development, gradually refine images from noise, resulting in higher image quality and
detail.
Additionally, Large Language Models (LLMs) such as GPT-4 and Vision-Language Models
(VLMs) like LLaVa-Med have expanded the horizon of AI in radiology by integrating image
interpretation with natural language understanding, offering radiologists enhanced diagnostic
support and patient-friendly reporting.
Generative models applied to medical imaging development require abundant and highly
varied datasets to capture the intrinsic variability present in various imaging modalities. In
order to enhance model performance and expand the generalizability of the model, data
augmentation and transfer learning are common techniques often used. Additionally, these
models are validated using a comprehensive set of evaluation protocols comparing not only
image fidelity but also clinical utility and diagnostic accuracy. The most commonly employed
metrics are Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and
Fréchet Inception Distance (FID), which measure the visual quality of the generated images
as well as correspondence between the generated and real medical data.
Image Generation
GANs and their variants have contributed significantly in medical imaging. These models aid in
overcoming the drawbacks of conventional imaging methods by providing two main objectives:
improving image resolution and producing synthetic images for certain clinical tasks.
High-resolution, noise-reduced images are produced using reconstruction models like AiCE and
TrueFidelity, although they frequently rely on proprietary, vendor-specific equipment. These
techniques concentrate on increasing contrast, decreasing noise, and boosting clarity in radiological
images. Sparse-view CT reconstruction and unified multimodal frameworks are recent developments
that improve resolution even in situations with sparse data. Advanced models are used in post-
reconstruction procedures to carry out tasks including super-resolution, artefact removal, and
denoising. Transformer-based models, for instance, overcome the drawbacks of convolutional neural
networks (CNNs) by allowing for simultaneous local and global learning, which produces visual
outputs that are more refined. Pix2Pix and CycleGAN, two GAN-based techniques, have
demonstrated considerable potential in resolving common problems such as image distortion,
enabling pixel-level precision, and creating images that preserve anatomical integrity. Image variety
and quality are significantly enhanced by diffusion models such as Denoising Diffusion Probabilistic
Models (DDPM) and Denoising Diffusion Implicit Models (DDIM), which capture complex, high-
dimensional distributions.
Task specific image synthesis, which aims to handle particular clinical issues or enhance datasets
for studies. In order to reduce radiation exposure and preserve diagnostic accuracy, GANs have
been used to create images for low-dose CT scans. One prominent example is the two-stage transfer
learning technique that allows low-dose CT data generation from standard scans by adapting models
to various noise styles. Additionally, StyleGAN and encoder-combined models, like pixel2style2pixel
(pSp), have shown remarkable ability to produce high-resolution synthetic images
with accurate feature control, improving the analysis of disease progression and patterns.
Image Translation
The practice of transforming one kind of medical image into another while keeping important
anatomical details is known as image translation. Given its potential to lower costs, simplify intricate
imaging operations, and increase accessibility to hard-to-obtain images, this technique has acquired
a lot of interest in radiology.
Inter and Intra Modal Image Translation
Advanced Brain Imaging Applications
MRI to CT Translation
Automated Radiology Reporting and Analysis
Patient Centred Applications
Ethical Considerations and Challenges
Case Study (Nehterlands)
Future Directions
Summary
Conclusion
References
Kim, K., Cho, K., Jang, R., Kyung, S., Lee, S., Ham, S., Choi, E., Hong, G., & Kim, N. (2024). Updated
primer on generative artificial intelligence and large language models in medical imaging for medical
professionals. Korean Journal of Radiology, 25(3), 224. https://doi.org/10.3348/kjr.2023.0818
Figure 1- PMC. (n.d.). https://pmc.ncbi.nlm.nih.gov/articles/PMC10912493/figure/F2/
Gadhiraju, A., & Prabhod, K. J. (2024, April 15). Generative AI in Radiology: Transforming image
analysis and diagnosis. https://aimlstudies.co.uk/index.php/jaira/article/view/191
Yoon, J. T., Lee, K. M., Oh, J. -H., Kim, H. -G., & Jeong, J. W. (2024). Insights and Considerations in
Development and Performance Evaluation of Generative Adversarial Networks (GANs): What
Radiologists Need to Know. Diagnostics, 14(16), 1756.
https://doi.org/10.3390/diagnostics141617564
Zhong, Z., & Xie, X. (2024). Clinical applications of generative artificial intelligence in radiology: image
translation, synthesis, and text generation. BJR| Artificial Intelligence, 1(1), ubae012.
van Leeuwen, K. G., de Rooij, M., Schalekamp, S., van Ginneken, B., & Rutten, M. J. (2024). Clinical
use of artificial intelligence products for radiology in the Netherlands between 2020 and
2022. European Radiology, 34(1), 348-354.
Buijs, E., Maggioni, E., Mazziotta, F., Lega, F., & Carrafiello, G. (2024). Clinical impact of AI in radiology
department management: a systematic review. La radiologia medica, 1-11.
Nakaura, T., Yoshida, N., Kobayashi, N., Shiraishi, K., Nagayama, Y., Uetani, H., ... & Hirai, T. (2024).
Preliminary assessment of automated radiology report generation with generative pre-trained
transformers: comparing results to radiologist-generated reports. Japanese Journal of
Radiology, 42(2), 190-200.
Park, J., Oh, K., Han, K., & Lee, Y. H. (2024). Patient-centered radiology reports with generative
artificial intelligence: adding value to radiology reporting. Scientific Reports, 14(1), 13218.
