1. INTRODUCTION

Radiologic examinations play an important role in modern medicine. Reporting the findings from radiologic images is an important method for radiologists to communicate with referring physicians.1,2 A radiologic report can consist of many parts, such as demographics (patient background), the main context or body of the report (procedure-related information, findings, and other issues), and the conclusion (impression or diagnosis).3 The American College of Radiology (ACR) Practice Parameters and Technical Standards encourage radiologists to provide recommendations for follow-up examinations or additional diagnostic studies when appropriate to clarify or confirm the impression.3 In addition, most clinicians prefer the inclusion of report recommendations when appropriate.4,5

Recommendations for additional imaging (RAIs) can be useful for clarifying an indeterminate imaging finding, confirming malignancy or benignity, assessing temporal changes of an abnormality, and providing follow-up guidance based on the findings.3,4,6,7 RAIs appear frequently in radiologic reports. Previous studies have shown that 8% to 69% of radiologic reports contain RAIs for a variety of examination modalities.6–12 In practice, an RAI is used not only to suggest “additional imaging” for further evaluation but also to suggest a more suitable imaging modality for future follow-up. This nuance can be seen through the different wording used in the report.

The aim of this study was to specifically measure the frequency of RAIs for abdominal magnetic resonance (MR) studies after abdominal computed tomography (CT) scans. The secondary aim was to measure the effectiveness (i.e., the ability to influence physicians to order a follow-up MR study) of the RAIs and to determine the extent to which different wording and locations of the RAIs affect their effectiveness.

2. METHODS 2.1. Study setting and approval

The Institutional Review Board approved this retrospective study, which was conducted at a tertiary referral medical center with 2800 beds located in Taipei, Taiwan. The radiology department generates more than 750 000 reports annually.

Data were obtained from the radiology information system (RIS, Taiwan Electronic Data Processing Co.) at our institution. All CT examination reports from 2020 were selected.

In this study, the term RAI refers specifically to a recommendation after an abdominal CT suggesting an abdominal magnetic resonance imaging (MRI) either as an additional examination or as a follow-up study. Recommendations for modalities other than MRI were excluded.

RAIs were classified into two groups based on the nuances of the wording. For example, an RAI for further evaluation (RAI-FE) could contain phrases such as “recommend MRI for correlation” or “suggest MRI correlation” in the report; an RAI for a follow-up study (RAI-FU) could contain phrases such as “suggest following up with MRI” in the reports.

A recommending CT was defined as an abdominal CT study that included an RAI in its report; a nonrecommending CT was defined as an abdominal CT study that did not include an RAI in its report. A recommended MR was defined as an abdominal MR study performed within a 120-day period from the report date of a preceding recommending CT.

Data were processed according to the following steps (Supplementary Figure 1, https://links.lww.com/JCMA/A167):

1.  All CT reports issued in 2020 were queried. 2.  Abdominal CT reports were selected. 3.  All recommending CTs were identified. 4. The RAIs in all recommending CTs were classified into six categories based on the wording and location of the recommendations: (a) Category 1: RAI-FUs only in the context section of the reports. (b) Category 2: RAI-FUs only in the conclusion section of the reports. (c) Category 3: RAI-FUs in both the context and the conclusion sections of the reports. (d) Category 4: RAI-FEs only in the context section of the reports. (e) Category 5: RAI-FEs only in the conclusion section of the reports. (f) Category 6: RAI-FEs in both the context and the conclusion sections of the reports.

Data were reviewed by K.-L.L, a board-certified radiologist who was blinded to the subjects’ clinical information.

The anatomical location of the lesions that prompted radiologists to make their recommendations was documented. All abdominal MRI studies (including pelvic MRI) performed at our institution between January 2020 and April 2021 were also extracted from the RIS. MR studies of body parts other than the abdomen were excluded. MR studies performed at other institutions were also excluded.

Using the Python programming language, we identified all pairs in which an abdominal CT was followed by an abdominal MRI within 120 days. A strict “CT–MRI” sequence was required to determine a valid pair. For example, in a hypothetical series of studies of CT/1–CT/2–MRI/1–MRI/2 examinations, only “CT/2–MRI/1” was counted as a valid pair. For each valid pair, the interval between the report date of a recommending CT and the date that a recommended MRI was performed was recorded.

2.2. Statistical analysis

The χ2 test with Yates’ correction was used to evaluate potential associations between categorical variables.

The Shapiro–Wilk test was performed and showed that the distribution of interval days between CT and MRI departed significantly from normality for recommending CTs, nonrecommending CTs, and recommending CTs with category 3 to 6 RAIs. Therefore, the Mann–Whitney U test and the Kruskal–Wallis test were performed to examine the differences in the intervals between CT and MRI.

p values less than 0.05 were considered statistically significant. Post hoc tests with χ2 tests and Bonferroni-adjusted p values were used when appropriate.

Statistical analyses were performed using the Python programming language and the SciPy (version 1.6.2) and statsmodels (version 0.12.2) statistical packages. The figures were created using the Python programming language and the matplotlib package (version 3.3.4).

3. RESULTS

In 2020, 102 937 CT reports were issued by the radiology department. A total of 32,371 abdominal CT reports were selected. Of all abdominal CTs, 871 (2.7%) were identified as recommending CTs. The top three reasons for recommending abdominal MRI were hepatic lesions (52.9%), pancreatic lesions (17.9%), and lesions in the female reproductive organs (9.8%) (Supplemental Table 1 https://links.lww.com/JCMA/A168).

Between January 2020 and April 2021, 9559 abdominal MR examinations were performed at our institution. There were 1572 pairs of “one abdominal CT followed by one abdominal MRI within 120 days”. That is, 4.9% (1572 out of 32 371) of abdominal CTs were followed by an abdominal MR study within 120 days; 16.4% (1572 out of 9559) of abdominal MR studies had a preceding abdominal CT within 120 days. Approximately 3.3% (320 out of 9559) of abdominal MR studies had an RAI in the prior abdominal CT report.

The χ2 test with Yates’ correction showed that recommending CTs (N = 320 of 871, 36.7%) were statistically more likely to lead to subsequent MR than nonrecommending CTs (N = 1252 of 31 500, 4.0%; χ2 = 1969.3, df=1, p < 0.001) (Fig. 1 and Table 1).

Table 1 - The association between a recommending CT and a subsequent abdominal MR study within 120 days Presence of a Subsequent MR Study Within 120 Days of the CT Report (%) No Subsequent MR Study Within 120 Days of the CT Report (%) χ2, p valuea Nonrecommending CT 1252 (4.0) 30248 (96.0) 1969.3, <0.001b Recommending CT 320 (36.7) 551 (63.3)

RAI = recommendation after an abdominal CT suggesting an abdominal MRI either as an additional examination or as a follow-up study; Recommending CT = an abdominal CT study that included an RAI in its report; Nonrecommending CT = an abdominal CT study that did not include RAI in its report.

aχ2 test was performed with Yates’ correction.

bRepresents statistical significance, p < 0.05.

Fig. 1:

Percentage of abdominal CT studies followed by an abdominal MR study within 120 days, comparing nonrecommending CT (1252 of 31 500, 4.0%) to recommending CT (320 of 871, 36.7%). Recommending CT = an abdominal CT study that included an RAI in its report. Non-recommending CT = an abdominal CT study that did not include RAI in its report. RAI = recommendation after an abdominal CT suggesting an abdominal MRI either as an additional examination or as a follow-up study.

The Mann–Whitney U test showed that the intervals in the recommending CT group (29.0, 39.25; median, IQR days) were significantly shorter than those in the nonrecommending CT group (39.0, 61.0; median, IQR days; U = 232745.5, p<0.001) (Fig. 2 and Table 2).

Table 2 - Interval between an abdominal CT report and subsequent abdominal MR study. Comparison between nonrecommending CTs and recommending CTs Interval Between the Date of an Abdominal CT Report and the Following Abdominal MR Study (median, IQR, mean ± SD days)a Nonrecommending CTs 39.0, 61.0, 45.9 ± 35.3 Recommending CTs 29.0, 39.25, 34.8 ± 27.9

aThe Mann–Whitney U test was performed to identify differences between the intervals of recommending CTs and non-recommending CTs with their subsequent abdominal MRI. The intervals in the recommending CT group (median=29.0 days) were significantly shorter than those in the non-recommending CT group (median=39.0 days), U = 232745.5, p < 0.001.

Fig. 2:

The distribution of intervals between studies in “abdominal CT-abdominal MR” pairs is shown by kernel density estimation. The curves for recommending CTs (solid line) and nonrecommending CTs (dotted line) are drawn separately. Recommending CTs (median = 29.0 days) led to shorter intervals with a curve that skewed more to the left than nonrecommending CTs (median = 39.0 days), U = 232745.5, p < 0.001.

In total, 27, 41, 155, 82, 88, and 478 RAIs from recommending CTs (N = 871) were classified 1, 2, 3, 4, 5, and 6 based on the wording (RAI-FU or RFA-FE) and location (in context only, in conclusion only, or both) of the recommendations, respectively. For each category from 1 to 6, there were 3, 9, 53, 14, 25, and 216 recommending CTs paired with recommended MRs, accounting for 11.1%, 22.0%, 34.2%, 17.1%, 28.4%, and 45.2% of recommending CTs in each group, respectively (Fig. 3 and Table 3). A χ2 test with Yates’ correction was performed. The Bonferroni-adjusted p value was 0.0083. Category 4 recommendations (RAI-FEs that were only in the context section of the reports) led to a significantly lower rate of subsequently recommended MR examinations (χ2 = 14.14, p < 0.001); category 6 recommendations (RAI-FEs that were in both the context and the conclusion sections of the reports) led to a significantly higher rate of subsequent examinations (χ2=31.74, p<0.001).

Table 3 - The association between RAI category 1-6 in recommending CTs and a subsequent abdominal MR study within 120 days Presence of a Subsequent MR Within 120 Days After the CT Report (%) No Subsequent MR Study Within 120 Days After the CT Report (%) χ2 value, p valuea Category 1 3 (11.1) 24 (88.9) 6.78, p = 0.0092 Category 2 9 (22.0) 32 (78.0) 3.41, p = 0.0649 Category 3 53 (34.2) 102 (65.8) 0.40, p = 0.5266 Category 4 14 (17.1) 68 (82.9) 14.14, p < 0.001b Category 5 25 (28.4) 63 (71.6) 2.54, p = 0.1112 Category 6 216 (45.2) 262 (54.8) 31.74, p<0.001b

Recommending CTs were separated into categories 1-6 based on the wording and location of the recommendations; their definitions are described in the Materials and Methods section of this article.

aχ2 test was performed with Yates’ corre.

bRepresents statistical significance. To reduce the probability of a type I error, Bonferroni adjustment was performed, and the threshold p value was 0.0083.

Fig. 3:

The recommending CTs were separated into six categories based on the wording and location of the recommendations. In each category from 1 to 6, the percentages of a recommending CT paired with a recommended MR were 11.1%, 22.0%, 34.2%, 17.1%, 28.4%, and 45.2%, respectively. In category 4, the percentage was significantly lower than that in the other categories; in category 6, the percentage was significantly higher than that in the other categories.

In each RAI category from 1 to 6, the median intervals between the reporting date of a recommending CT and the subsequent abdominal MRI were 25.0, 33.0, 43.0, 16.5, 23.0, and 26.5 days, respectively (Fig. 4 and Table 4). No significant differences were found among the 6 categories with respect to the interval days between a recommending CT and a recommended MRI, according to the Kruskal–Wallis test (χ2 = 10.95, p = 0.052, df = 5).

Table 4 - Interval days between recommending CT with category 1 to 6 RAIs and the following recommended MR study Interval Between the Date of an Abdominal CT Report and the Subsequent Abdominal MRI (median, IQR, mean ± SD days)a Category 1 25.0, 41.5, 36.7 ± 42.7 Category 2 33.0, 51.0, 40.9 ± 32.6 Category 3 43.0, 35.0, 44.0 ± 26.1 Category 4 16.5, 60.0, 31.9 ± 32.8 Category 5 23.0, 37.0, 29.0 ± 26.1 Category 6 26.5, 37.25, 33.2 ± 27.6

IQR = interquartile range; RAI = recommendation after an abdominal CT suggesting an abdominal MRI either as an additional examination or as a follow-up study; SD = standard deviation.

aThe Kruskal–Wallis test was performed to examine the differences in interval days between recommending CT and the subsequent recommended MR study in categories 1 to 6 RAIs. No statistically significant differences (χ2 = 10.95, p = 0.052, df = 5) were found among the six categories.

Fig. 4:

Distribution of interval days between CT and MRI among categories 1 to 6 of recommending CTs. No statistically significant differences (p = 0.052) were found by the Kruskal–Wallis test.

4. DISCUSSION

In this study, we were the first to specifically address the influence of RAIs for abdominal MR studies in abdominal CT reports. We successfully demonstrated how RAIs influence clinical decisions and how the wording and location of the RAIs affect their effectiveness.

4.1. Frequency of RAIs

In this study, recommending CTs were identified in only 2.7% (871 out of 32 371) of the abdominal CTs, which is relatively low compared with the results ranging from 8% to 69% of previous studies.6–12 This may be because RAIs in this study were limited to those for abdominal MRI, in contrast to the variety of examination modalities in previous work.

The most common reason for radiologists to make RAIs was hepatic lesions (52.9%), followed by pancreatic lesions (17.9%) and lesions in the female reproductive organs (9.8%). This result differed from a previous study by Blaivas and Lyon,8 which showed ovarian cysts, renal cysts, and gallbladder lesions to be the most common reasons for RAIs after abdominal CT. This discrepancy could also be due to the reasons mentioned above.

4.2. Effectiveness of RAIs

Previous works showed that 5.3% to 8% of high-cost examinations followed a recommendation from radiologic reports.13,14 Our data revealed that 16.4% (1572 of 9559) of the abdominal MR studies were performed following a prior abdominal CT within 120 days. However, only 3.3% (320 of 9559) of the abdominal MR studies were conducted following the RAIs from the previous CT reports. The percentage (3.3%) was even lower than the ones in the other similar studies (5.3-8%).13,14 This phenomenon indicated that the majority (96.7%, 9239 of 9559) of the abdominal MR studies after CT were ordered without RAIs from the radiologists. Although the clinical judgments to decide whether to order another imaging exam depend mostly on the clinical physicians and the reasons to order a further MR exam may not be limited to the contents of the previous CT reports, radiologists could discreetly participate in clinical decision making by adding the RAIs to the reports. Our results showed that the presence of RAIs in the radiologic report significantly increased the rate of a subsequent MR examination within 120 days from 4% to 36.7%, p < 0.001 (Fig. 1 and Table 1). This finding was comparable to a study by Baumgarten and Nelson9 that showed that 30.5% of the recommendations were subsequently followed.

Our data also showed that the presence of RAIs was significantly associated with a shorter median interval between the abdominal CT and subsequent abdominal MR (29.0 days vs. 39.0 days, p < 0.001) (Fig. 2 and Table 2), which is another manifestation of the influence of RAIs.

4.3. Factors that may affect the effectiveness of RAIs

Differences in wording in radiologic reports may affect the adherence rate to the recommendations.9,15,16 For example, a survey study by Gunn et al. showed that primary care physicians felt less medicolegally obliged when qualifying words such as “if clinically indicated” were included in the recommendation.16 In the same study, the authors found that clinical physicians may feel more obligated when the recommendation had a separate section. However, a study by Baumgarten and Nelson showed no statistically significant difference in effectiveness between strong wording (“study is recommended” or “follow-up is recommended”) and weak wording (“if clinically indicated”, “may be helpful”, or “attention to this region on follow-up”) in the recommendations.9

In our study, we separated the RAIs into six categories according to their wording and locations (Fig. 3 and Table 3) to measure effectiveness. The data showed that category 4 RAIs (RAI-FEs that were only in the context section of the reports) led to a significantly lower compliance rate (17.1%, p < 0.001). The maximum effectiveness of the recommendation was achieved when radiologists made category 6 RAIs (RAI-FEs that were in both the context and the conclusion sections of the reports) (45.2%, p < 0.001). However, no statistically significant difference was found among the six categories in terms of interval days between the abdominal CT and subsequent abdominal MR (Fig. 4 and Table 4).

This study has several potential limitations. Data were collected at a single institution; thus, it is unclear whether the results are generalizable. Additionally, the sample size is relatively small. More data are needed to detect statistically significant differences among RAI categories. It is difficult to assess the impact of RAIs on clinician decision-making. Because we did not retrospectively inquire all the clinicians, we are unable to elucidate the confounding factors that influence their ordering behavior. We did not evaluate the appropriateness of the recommendations, so we are unable to analyze the potential supplier-induced demands.

In conclusion, RAIs have a significant influence on clinical decisions. RAIs in abdominal CT reports increase the tendency to perform subsequent abdominal MR. These recommendations also shorten the time interval between the abdominal CT and the following abdominal MR. The most effective method of making an RAI is suggesting another examination for further evaluation that is mentioned in both the context and conclusion of the report. Radiologists should be aware of the power of RAIs and be prudent and conscientious when making any recommendations in radiologic reports.

Acknowledgments

This study was supported by grants from the Ministry of Science and Technology, Taiwan [110-2314-B-075-005 & 111-2314-B-075-025-MY3, to CHW; 110-2314-B-075-032, to FCC], Taipei Veterans General Hospital [V111B-032 to CHW, V110C-037, V111C-028 to FCC, V112B-007 to CHW and V112D67-002-MY3-1 to FCC], Yen Tjing Ling Medical Foundation [CI-111-2, to CHW], Veterans General Hospitals and University System of Taiwan Joint Research Program [VGHUST110-G1-5-2, to FCC], Vivian W. Yen Neurological Foundation [to CHW and FCC], and Prof. Tsuen CHANG’s Scholarship Program from Prof. Albert Ly-Young Shen’s Medical Education Foundation [to CHW].

References 1. Wallis A, McCoubrie P. The radiology report - are we getting the message across? Clin Radiol 2011;66:1015–22. 2. Goergen SK, Pool FJ, Turner TJ, Grimm JE, Appleyard MN, Crock C, et al. Evidence-based guideline for the written radiology report: methods, recommendations and implementation challenges. J Med Imaging Radiat Oncol 2013;57:1–7. 3. American College of Radiology. The ACR practice parameter for communication of diagnostic imaging findings. Published 2020. Available at https://www.acr.org/-/media/acr/files/practice-parameters/communicationdiag.pdf. Accessed July 12, 2022. 4. Arenson RL. Recommendations for additional imaging in radiology reports: radiologists’ self-referral or good clinical practice? Radiology 2009;253:291–2. 5. Grieve FM, Plumb AA, Khan SH. Radiology reporting: a general practitioner’s perspective. Br J Radiol 2010;83:17–22. 6. Shuaib W, Vijayasarathi A, Johnson JO, Salastekar N, He Q, Maddu KK, et al. Factors affecting patient compliance in the acute setting: an analysis of 20,000 imaging reports. Emerg Radiol 2014;21:373–9. 7. Sistrom CL, Dreyer KJ, Dang PP, Weilburg JB, Boland GW, Rosenthal DI, et al. Recommendations for additional imaging in radiology reports: multifactorial analysis of 5.9 million examinations. Radiology 2009;253:453–61. 8. Blaivas M, Lyon M. Frequency of radiology self-referral in abdominal computed tomographic scans and the implied cost. Am J Emerg Med 2007;25:396–9. 9. Baumgarten DA, Nelson RC. Outcome of examinations self-referred as a result of spiral CT of the abdomen. Acad Radiol 1997;4:802–5. 10. Furtado CD, Aguirre DA, Sirlin CB, Dang D, Stamato SK, Lee P, et al. Whole-body CT screening: spectrum of findings and recommendations in 1192 patients. Radiology 2005;237:385–94. 11. Tanpitukpongse TP, Grady AT, Sosa JA, Eastwood JD, Choudhury KR, Hoang JK. Incidental thyroid nodules on CT or MRI: discordance between what we report and what receives workup. Am J Roentgenol 2015;205:1281–7. 12. Squillaci E, Bolacchi F, Ricci F, De Stasio V, Pugliese L, Di Martino A, et al. Radiologists’ recommendations for additional imaging (RAI) in the inpatient setting. Radiol Medica 2019;124:432–7. 13. Lee SI, Krishnaraj A, Chatterji M, Dreyer KJ, Thrall JH, Hahn PF. When does a radiologist’s recommendation for follow-up result in high-cost imaging? Radiology 2012;262:544–9. 14. Lee SI, Saokar A, Dreyer KJ, Weilburg JB, Thrall JH, Hahn PF. Does radiologist recommendation for follow-up with the same imaging modality contribute substantially to high-cost imaging volume? Radiology 2007;242:857–64. 15. Lee B, Whitehead MT. Radiology reports: what you think you’re saying and what they think you’re saying. Curr Probl Diagn Radiol 2017;46:186–95. 16. Gunn AJ, Sahani DV, Bennett SE, Choy G. Recent measures to improve radiology reporting: perspectives from primary care physicians. J Am Coll Radiol 2013;10:122–7.