_{Published on January 22, 2021}

Although we have known for many decades that radiologists do make errors and the number of these can be significant, continuing educational solutions rarely recognize the practicalities facing modern-day clinicians. These practicalities include: proven effectiveness, methods of delivery that align to busy clinical workloads, modules that are compatible with any technological display, and tailoring education to reflect the unique errors each radiologist makes. Also, under the current climate of pandemic and travel restrictions, the tremendous difficulties of physically attending conferences and other educational venues must be recognized.

The current article focuses on education using clinical test sets, which contain data sets on normal and abnormal cases with known truth that the clinician must diagnose. We aim to summarize what busy radiologists in 2020 require from continuing educational solutions, illustrating how effective, relevant, available, and tailored education can be delivered to all using the DetectED-X platform (Sydney, Australia). 

So, What Is Needed?

High Quality Data Sets Combined With Performance Algorithms

Radiologists need effective and clinically relevant education that is available at all times. One solution that has been shown to work focuses on the provision of high-quality radiologic test sets that are specifically selected and designed to improve diagnostic efficacy. Via their local workstations, clinicians can view and judge sets of usually up to 60 DICOM cases at full native resolution, with higher than normal prevalence, and search for clinically relevant lesions validated by pathology findings using the DetectED-X platform. This way, it is possible to encounter in 1 hour the number of specific abnormalities only presented over periods of years. Radiologists should be able to judge these cases the same way they do clinically, so expert behaviours are relatively unaffected, and once all cases are judged and suspicious areas of interest are marked, clinicians should receive instant results on their performance using well-known and benchmarked metrics, such as sensitivity, specificity, lesion sensitivity, ROC, and jackknife alternative free-response ROC (JAFROC) (Fig. 1). Algorithms should then support these judgments by enabling the presentation of all errors made by each radiologist and identifying specific areas for immediate improvement (Fig. 2).   

• 

• 

Accessible and Recognised Education

To improve accessibility, it is essential that continuing medical education using test sets is available 24/7, regardless of clinicians’ location, must be completed in timeframes that align to busy clinical workloads, and should provide immediately much needed continuing medical education (CME) credits or continuing professional development (CPD) points. Also, to maximize ease of use, educational solutions should be compatible with all types of mobile devices, should involve test sets with smaller number of cases that can be completed in as little as 20 minutes, and must ensure that every activity is rewarded with CME or CPD—regardless of whether the clinician is located in the United States, Europe, or Asia.

Tailored Solutions

Tailoring to each clinician’s needs is required. No longer should clinicians receive test sets or other material that is common for everyone: Dr. Brady will make different mistakes from Dr. Cusack, so why should the two receive the same education? Artificial intelligence (AI) technology is now sufficiently mature to offer a robust educational solution. Elsewhere, in the entertainment industry for example, AI is now being utilized for the personalization of viewing platforms based on previously collected data to improve user experiences. Such advances can now be applied to online medical education, facilitating customized educational materials that accurately recognize each clinician’s strengths and weaknesses. By combining algorithms previously developed for the general-purpose recommender system, such as movie recommendation tools, with radiology-specific knowledge driven from research on factors defining case difficulty for specific clinicians, AI models can predict and access cases most suitable for each user.  

One Potential Solution

Test set technologies have been available over the last 3 decades, particularly in the United Kingdom, Australia, and New Zealand, and these have shown demonstrable mean improvements in diagnostic performance of 34%. However, these test sets can take up to 2 hours to complete and are usually only available via high-quality clinical workstations with images loaded locally. A new approach to address accessibility has been launched by DetectED-X, a University of Sydney startup, where high-quality cases are now available through modules (the smallest of which take only 20 minutes to complete), are available on any mobile device, and are immediately certified with CME credits or CPD points available for every activity. To ensure that this education is available at fast speeds, regardless of wherever in the world the radiologist is located, DetectED-X is working with GE Healthcare, Volpara Solutions, and Amazon Web Services to ensure robust and widespread distribution. Highly accurate tailoring based on tens of thousands of interactions with radiologic images is part of the DetectED-X approach, so that each clinician, regardless of training or experience, has an optimized educational experience. Its evidence-based AI algorithms for recognizing clinical error-making patterns have been tested using more than 600 clinicians reading mammograms or lung CT images, with an accuracy of 80% and 83% for predicting the positive and negative cases that each clinician has difficulty with, respectively. Once case difficulty for a clinician has been set, this is coupled to the clinician’s learning objective (e.g., whether they want to see a test data set, oversampled with calcifications or denser breasts), then a personalized test data set is presented to the clinician. Radiologists need a new approach so that effective, relevant, and tailored education is available everywhere and at all times. The COVID-19 pandemic has brought this need to the forefront.