_{Published January 4, 2022}

By the time you read this, results from the United States Medical Licensing Examination (USMLE) Step 1 exam will have changed from a numeric score to pass or fail. As students, medical schools, and residency programs anticipate this upcoming change, many are left wondering if and how this update will affect applicant selection for radiology residencies. Our study used an anonymous and voluntary 14-question online survey to assess program directors’ views on the scoring change of Step 1 from numeric to pass or fail, while assessing if other metrics, such as Step 2 Clinical Knowledge (CK) scores, may become more important during application review. Eighty-eight of 308 (29%) members of the Association of Program Directors in Radiology (APDR) completed the survey.

Our study found that more than two-thirds of survey respondents indicated their programs currently use a Step 1 screen before sending interview invitations. Specific to radiology, previous studies have demonstrated a correlation between Step 1 performance and both the number of interpretive errors made as a radiology resident and future success on the American Board of Radiology Core Exam. Our study also found that more than 90% of survey respondents anticipate their programs may or definitively will require Step 2 CK scores before application review, once Step 1 becomes pass or fail; in contrast, many radiology programs presently do not require Step 2 CK scores before reviewing applications. In addition, our study found that regardless of the current use of a Step 1 screen, survey respondents did not significantly differ in their anticipation of requiring Step 2 CK scores (p=0.71) or extending later interviews to accommodate Step 2 CK scores (p=0.64). This finding may reflect hesitancy from programs to place significant weight on Step 2 CK before more research is done. There may be increased utility of using Step 2 CK scores over Step 1 scores as a marker of clinical knowledge, as the material covered in Step 2 CK has greater clinical relevance than that of Step 1. Through faculty, peer, and patient-level evaluations, previous studies from internal medicine have shown that Step 2 CK scores correlate better to clinical performance both during and after residency than Step 1 scores. It could be informative to examine whether this correlation exists with Step 2 CK scores and performance in radiology residency, as more residency applicants may include Step 2 CK scores with their initial application in upcoming years.

Although making Step 1 pass or fail may help mitigate the significant distress surrounding the examination, there are valid concerns that the stress surrounding Step 1 will simply be transferred to Step 2 CK, when Step 2 CK becomes the only remaining standardized metric among applications. In an ideal world, all applications would receive a thorough and holistic review with consideration of both qualitative and quantitative characteristics. However, many selection committees do not have enough resources to do so, given the growing number of applications each program receives for the same number of residency spots. Some specialties have implemented preference signaling and supplemental applications to highlight an applicant’s top programs of interest and specific program characteristics (location, patient population, research, etc.) a particular applicant is seeking. Some studies have suggested more drastic measures, such as application caps, to reduce the usage of standardized test scores as a filter during application review and allow for more holistic application reviews. While changing Step 1 to pass or fail does not resolve all the challenges associated with the residency selection process, it does pave the way for additional meaningful changes to the system in the future.