^{Published June 28, 2021}

In the past decade, interest in artificial intelligence (AI) research in radiology has increased dramatically, as reflected by the tenfold increase in papers published in the subject over this period. This development is accompanied by fears from radiologists that AI will eventually replace human readers for the diagnostic interpretation of imaging studies. This article will review the current methods AI uses for diagnostic imaging tasks, the challenges AI algorithms have to overcome for their more widespread implementation, and how radiologist roles may shift to better accommodate the strengths of machine learning.

Deep Learning: An Introduction

The term “artificial intelligence,” in the most general sense, refers to the ability of algorithms to mimic human cognitive abilities. AI algorithms used for image analysis are typically designed using neural networks, which involves layers of processing nodes organized into an input, output, and multiple hidden layers. The architecture is feedforward; outputs of the previous layer are used as the inputs of the next. Each layer is composed of nodes, each of which has a weighted function that associates the inputs to outputs. The function weights are determined by backpropagation during training, an algorithm which updates function weights by minimizing a loss function.

Convolutional neural networks (CNNs) are neural networks specifically designed to handle image data and are the most common, though by no means only, deep learning architecture used for medical image applications. In diagnostic imaging studies, CNNs are usually trained with supervised learning by radiologist annotated data. The layers of a CNN process input images into an image classification output decided by weighted probabilities. For example, one hypothetical CNN’s classification output of a chest radiograph may be an 80% probability of pneumonia diagnosis, 15% pleural effusion, and 5% pneumothorax. The architecture of a CNN can be summarized as follows: first, convolutional layers extract features from an input image. The resulting feature map then passes through pooling layers, which decrease the number of trainable parameters by reducing the dimensions of the convolutional layer input [2]. Finally, a fully connected layer uses the inputs from the feature map and applies weights to parameters to classify the image, typically using rectified linear unit (ReLU) as its activation function.

CNNs have been developed for a wide range of diagnostic tasks based on modality image input data. They have shown comparable results to radiologists for identification of lung nodules and quantification of coronary artery calcium volume during low-dose CT screening, comparable performance to radiologists on the diagnosis of multiple thoracic pathologies from chest radiographs, and better diagnostic performance than radiologists for diagnosis of pneumonia from chest radiographs. Aside from applications for chest imaging, CNNs have achieved high accuracy (AUC) for liver fibrosis staging, accurate segmentation of clinical target volume (CTV) and organs at risk (OARs) in colorectal cancer films, and improved prediction of overall survival in glioblastoma patients from MRI data. However, while the results of CNNs are often comparable to diagnostic radiologists, it is important to note their detection tasks are narrow in scope, and like other AI methods, they are dependent on high quantities of quality training and validation data.

Challenges

There are several challenges to AI implementation in diagnostic radiology not readily solved with mere improvements in technology. The opaque nature of an AI algorithm’s functionality is one such challenge. The nature of backpropagation is such that algorithm developers are inherently unable to explain why parameters end up at their trained values, only that they were the values the algorithm discovered as optimal. Likewise, the precise features of an image that a neural network used to arrive at its classification output are impossible to know. In this regard, a neural network resembles the brain it was loosely inspired by—scientists understand that they work, but not how or why. This “black box” quality of machine learning has not, however, prevented the FDA from approving commercial AI-based medical devices and algorithms, and the history of medicine shows the field has accepted other black boxes for patient care. For example, the exact mechanisms by which many drugs work or produce side effects are poorly understood, but such drugs are used and trusted by physicians. Computer scientists are also attempting to make AI more transparent by developing processes to delineate the methods by which neural networks reach conclusions. This new field, called explainable artificial intelligence (XAI), is an active area of research.

Nevertheless, it is easy to imagine a future where AI becomes part of standard of care for diagnostic radiology, wherein health systems, and specifically the radiologist, would still be responsible for algorithm errors, even if an AI’s reasoning for the error is unexplainable. It is still unknown who would bear the liability for an algorithm misdiagnosis, nor whether the public and their primary physicians would accept machine diagnosis.

The training and validation of medical image algorithms is yet another challenge to their implementation. Deep learning algorithms such as CNNs often require huge amounts of data because of the high number of parameters that must be optimized during training. Large high-quality datasets, which must be annotated by radiologists, are expensive to procure and often several orders of magnitude smaller than the training datasets used for other deep learning applications. They usually number from the hundreds to the tens of thousands, compared to the 4 million labeled images Facebook’s DeepFace facial recognition neural network used as its training dataset. There has been recent progress on creating publicly available image datasets for algorithm development and on medical image analysis challenges that provide datasets for developers. Stanford’s CheXpert dataset is the largest of these, with 224,216 labeled chest radiographs available for public use. However, training datasets is still often proprietary due to privacy concerns or intended commercial use of the algorithms utilized. This dearth of training data and lack of transparency can create problems of algorithmic biases; homogenous training data can cause health care AI algorithms to weigh certain diagnoses unequally based on socioeconomic status, race, or gender, and properly diverse datasets are difficult to assemble.

AI and the Future of Radiology

However, the greatest challenge for AI in diagnostic radiology may be whether it replaces a diagnostic radiologist’s function entirely, and the resulting health care implications for patients. Medical experts have even raised the question if diagnostic radiologists should continue to be trained, given the fast-approaching integration of AI technologies as part of diagnostic algorithms, which will almost certainly include some form of initial automated machine-driven image analysis. The advantages of AI over human radiologists seem overwhelming at first. Computers can work 24/7 without signs of fatigue. While diagnostic radiologists typically read between 10,000–20,000 films annually, the number of images a neural network could read in the same time period is multiple orders of magnitudes greater, in the tens or hundreds of millions. As such, the economic reasoning for institutions to replace diagnostic radiologists seems obvious. However, currently developed algorithms only accomplish tasks narrow in scope (e.g., the binary classification of a lung nodule or the density analysis of a renal stone), whereas radiologists read studies holistically, looking for all possible abnormalities displayed in an imaging study. For a computer to read a diagnostic imaging data set in the same way a radiologist typically does would require the development, training, validation, and integration of thousands of discrete AI algorithms into one workflow. This is a difficult task, and though theoretically possible, a single generalized AI algorithm that can holistically analyze all features of an image for all possible diagnoses is even harder. For the foreseeable future, radiologists will still be required for accurately reading film, with AI packages offering decision support algorithms, and boosting reader efficiency by both providing second opinions on the primary detection tasks and simplifying workflow (e.g., prepopulating reports, automatically creating ROIs, etc.)

Even if AI technologies improve such that diagnostic AI application packages become capable of holistically reading modality studies, one must also remember that radiologists are not just image analysts, but fundamentally clinicians, trained to interpret and communicate imaging findings in the clinical context of the patient. Radiomics that uses AI generates complex data that must be interpreted by radiologists and linked to clinical uses; this is a need that will only grow in the future. AI-driven increased efficiency of image analysis may cause radiologists to pivot to more patient interaction and to become even more integrated in the clinical decision processes, in collaboration with a patient’s care team.

Given the uncertainties of the black box nature of AI, radiologists appear to be the best positioned medical professionals to elucidate a neural network’s probability outputs to a patient, along with the image features the machine is using to arrive at its diagnoses and their meaning. To best synergize with AI, radiologists of the future will have to improve care through knowledge of and empathy for their patients, the ability to identify which AI-produced data is relevant to meet diagnostic demands, and the interpretation of scans to elucidate the next steps following image findings to better advocate for their patients.

AI, and in particular convolutional neural networks, have seen success in narrow detection tasks for medical imaging diagnoses. While various challenges exist, such as their opaque nature and the cost and small scale of training datasets, they will likely be surmountable in time. AI will inevitably enter the diagnostic radiologists’ workflow. Although AI will not replace radiologists in the future, it is likely that radiologists who use AI will eventually replace radiologists who do not. The use of AI will lead to more efficient image diagnosis, in combination with optimized decision support algorithms, which will benefit patient care. In the not too distant future, AI could usher a potential realignment of radiologist duties towards a more interactive and patient-focused paradigm, in addition to traditional models centered around the reporting of imaging findings.