1 INTRODUCTION

Unexpected findings that may have clinical implications arise in both clinical care and research. In recent decades, a robust literature has emerged that explores the ethical obligations of clinicians and researchers regarding the management and disclosure of such unexpected findings and offers corresponding ethical guidance.1-4 Questions related to the management of unexpected findings in the context of pragmatic clinical trials (PCTs), including comparative effectiveness research embedded in clinical practice, however, have received far less consideration. These findings, which elsewhere we have termed PCT “collateral findings” (PCT-CFs), are findings that emerge during the course of a PCT but are unrelated to the primary research question(s), and may have implications for individual patients, clinicians, or health care systems from whom or within which research data were collected.5 For example, a PCT exploring strategies to increase colonoscopy screening rates revealed some screening tests had far higher rates of false positives than others, driving potentially unnecessary follow-up testing.6 In another PCT aimed at increasing the uptake of cardiac care guidelines, individuals with diagnoses indicating elevated risk for a cardiac event were identified, but there did not seem to be corresponding documentation that these diagnoses were communicated to the respective patients, suggesting a potential critical missed opportunity to reduce such risk.7

While PCT-CFs have some similarities to findings that arise in other contexts, including incidental and secondary findings (IFs/SFs) in explanatory research and clinical care, they also differ in ethically important ways, thereby limiting the applicability of existing scholarship and guidance to decision-making regarding their management.5 For example, prior guidance related to decision-making about management of IFs/SFs arising from clinical care or explanatory research has generally emphasized the role of informed consent, and the corresponding importance of prospective discussion with patient-participants about their preferences for receiving such findings. However, patient-participant consent might be ill-suited to play a central role in the ethical management of many PCT-CFs since PCTs do not always solicit prospective informed consent from individuals, and PCTs often study the behavior of clinicians or health systems, and thus might not directly study patients at all.5 Furthermore, the timeliness of detecting PCT-CFs might differ considerably from that of IF/SFs. For example, while a radiology report generated during explanatory research is likely to be read soon after it is generated, PCTs often involve extraction of data generated from routine health care operations at the end of a specified period of time or after a total target number of patients have been included in the PCT, meaning that the PCT-CF might not be identified until months or even years later.8

The need for ethically sound guidance regarding the management of PCT-CFs that is responsive to these and other contextual features of PCTs is particularly acute, as funders including both the National Institutes of Health (NIH) and the Patient Centered Outcomes Research Institute (PCORI) have invested substantial resources to enable a rapid expansion of these trial designs.9, 10 Developing such guidance requires taking into account the experiences and values of relevant stakeholders. To that end, this article reports on in-depth interviews with individuals working within institutions engaged in PCTs.

2 METHODS 2.1 Sample selection and recruitment

We recruited respondents using purposive sampling, targeting individuals with experience in the conduct or oversight of PCTs and those in health system leadership. We sought to include the perspectives of individuals with different institutional responsibilities related to PCTs, including principal investigators (PIs), clinicians, quality leaders, delivery system leaders, and those responsible for ethical and regulatory oversight, including Institutional Review Board (IRB) professionals and legal counsel.

Respondents were recruited from four populations: (a) NIH Collaboratory-funded PCTs; (b) other NIH-funded PCTs; (c) PCORI-funded PCTs; and (d) the Health Care Systems Research Network. Letters of invitation describing the project were sent via email.

This study was approved by the Institutional Review Boards at Johns Hopkins Medicine, Baylor College of Medicine, and Duke University Health Systems.

2.2 Study procedures

We conducted hour-long, semi-structured telephone interviews using detailed interview guides. The interview guide used with investigators is provided in the Appendix; the guides used with other types of stakeholders were similar, but tailored to their particular roles in PCTs (available from the authors on request). The guides focused on two broad areas: (a) experience with PCTs and PCT-CFs, and (b) PCT-CF management (actual or hypothetical). All interviews were conducted by one of two members of the research team (SM, DM). Respondents were offered $100 for completing the interview. All interviews were audiorecorded, professionally transcribed, reviewed for accuracy, and redacted for personally identifying information.

We took an iterative approach in our interviews, such that our interviews evolved in response to earlier interviews along two primary dimensions. First, while we initially characterized the findings of interest as “incidental findings,” we identified shortcomings with the language of IFs in this context and therefore transitioned to the terminology of “collateral findings.” Second, we elicited later respondents' reactions to PCT-CF examples offered by earlier interviewees.

2.3 Analysis

We used an integrated approach to developing the code structure, including both a priori codes drawn from our interview guide, and emergent, inductive codes.11 Three investigators (SM, DM, EM) reviewed a sample of transcripts to identify key themes and iteratively develop a codebook. Each transcript was then coded by one of two investigators (SM or DM) and reviewed by a third (EM), using the NVIVO12 software package (QSR International, Burlington, MA). Memos were written for each code, describing relevant themes and their frequency, and presenting exemplary quotations. Any differences of opinion about the meaning of specific quotations were discussed and resolved through an iterative process of discussion and comparison to the raw data.

3 RESULTS

We conducted 39 interviews between January and November 2019, involving a total of 41 respondents (one interview involved three respondents from the same institution; Table 1). The most common role types of respondents were researchers of funded PCTs (n = 24) and health care system leaders (n = 9).

TABLE 1. Interview respondent characteristics Role Number of respondents Researchers (Principal Investigators, Co-Investigators) 24 IRB members 5 Quality Assurance Leaders 1 Clinicians 1 Legal Counsel 1 Health Care System Leaders 9 Total 41a Affiliation Collaboratory 24 Non-Collaboratory (NIH) 5 Non-Collaboratory (PCORI) 3 The Health Care System Research Network (HCSRN) 9 Total 41 Sex Male 27 Female 14 Total 41 a One interview involved three respondents from the same institution.

We identified four major themes regarding management of PCT-CFs. First, discussions of PCT-CFs are complicated by layers of ambiguity. Second, management of PCT-CFs is context-specific, and not amenable to a “one-size-fits-all” approach. Third, there was a wide diversity of attitudes regarding the scope of researcher responsibilities in PCTs. Fourth, PCT-CFs had generally not been previously considered by respondents, but there was widespread belief in the importance of prospective planning to anticipate such issues in future PCTs.

3.1 Layers of ambiguity challenge classification of PCT-CFs

Discussions regarding PCT-CFs were complicated by layers of ambiguity, both with respect to the nature of PCTs themselves and the categorization of unanticipated results that emerge from them.

3.1.1 PCTs and blurry boundaries In many interviews, considerable discussion regarding the definition and scope of PCTs was required before the respondents' views about unanticipated findings in PCTs could be explored. Respondent descriptions of PCTs suggested uncertainty about the nature and scope of PCTs, and several referenced the “blurry” boundaries between activities that constitute research and those that constitute quality assurance or quality improvement (QI). Several PIs stated that they themselves had not considered their own trials as PCTs, although they recognized that others might do so. Similarly, others indicated that the same activity might be classified differently by different stakeholders. As one explained:

…I've been the PI for a lot of randomized controlled trials and I at the time did not necessarily categorize them or consider them pragmatic clinical trials but I have seen them described that way by others. (R19)

For another, “the whole research versus QI thing, I think, is poorly understood. And much blurrier than people would like to think…” (R31)

3.1.2 Categorization of unanticipated results

Relatedly, respondents offered distinct, and at times conflicting, accounts for how unanticipated results in PCTs should be characterized. For example, the same PCT-CF was characterized differently by different respondents, with one investigator involved with the trial from which the PCT-CF emanated describing it as an “incidental finding,” (R2) while another explicitly rejected the language of IFs as inappropriate because the finding resulted from analysis of clinical data rather than from interventional research, and had presumably been reviewed and addressed by the responsible physician. (R3) A third investigator deemed the same PCT-CF as something that became “closer to a quality improvement exercise, quality assurance… rather than part of the research that's being done” (R1); and an expert in implementation science classified the emergence of the PCT-CF as the identification of a “gap in care.” (R31) These differences in views about the nature of the finding, perhaps not surprisingly, in turn shaped attitudes regarding whether, how, and by whom it should be managed.

3.2 PCT-CF management is context-specific

Context was a central theme in views about decision-making regarding PCT-CF management. Comments akin to “I think it depends [on the finding]” (R1) were common across interviews. Respondents offered a variety of relevant contextual factors that influenced their views about decision-making, which we describe in turn.

3.2.1 Clinical relevance and modifying factors

The clinical relevance of the PCT-CF, including severity and medical actionability, were the factors most frequently referenced by respondents as guiding decisions about PCT-CF management. Respondents emphasized such factors as whether the PCT-CF was “intervenable and high-stakes,” (R22), the “severity…and meaningfulness to that person's clinical care,” (R14), and “the likelihood of benefit in the follow up on the information.” (R20)

Several respondents identified modifying factors that shaped decisions about clinical relevance, including both the timing of identification and whether the PCT-CF was uniquely known to the researcher(s) and otherwise unavailable to the patient or treating physician(s). With respect to timing, respondents noted that data in a PCT are often not analyzed in real-time, but instead there is, as one PI explained, some “intrinsic delay in terms of how that information gets to the team.” (R9) Consequently, by the time the PCT-CF is identified, it may no longer be clinically actionable, which may weigh against disclosure. Others thought that it was relevant whether information was uniquely known to researchers, or whether it was likely accessible to others, such as clinicians. As one PI stated:

Has a responsible provider already seen and made a decision whether or not to act on this information? If so, then we say, ‘There's nothing more to be done. That's not our job to go back and second-guess the decision of that provider who was on the spot.’ On the other hand, if we say, ‘No reasonably qualified and responsible provider has access to or knew about this, then we need to pass that information on….we're on duty. (R12)

3.2.2 System-level impact and opportunity costs Respondents also identified system-level impacts associated with PCT-CF management as relevant for decision-making, including the costs and burden associated with confirming them, notifying physicians and patients, and any subsequent clinical evaluation. As one PI explained,

That's really burdensome for us. We're not funded to do that. We don't have personnel to do that. That would take hours and hours and hours of my time to notify each provider about the clinical situation and then follow up with all of these people. From a practical point of view, I don't know that we can even really do that, because we never anticipated this. (R2)

A health system leader similarly acknowledged the burden, particularly in the context of large PCTs involving follow-up for many people, but rejected that it would ultimately change decision-making, noting “it's certainly different in how much work it would take, but I don't think it affects the general principles.” (R39)

Respondents weighed the opportunity costs associated with disclosing information about PCT-CFs, for both clinical and research activities. One PI described concerns about a potential PCT-CF being a “false alarm” for which management would result in “taking resources away” from other clinical priorities. (R1) Several others highlighted concerns about undermining institutional willingness to participate in PCTs if institutions are deemed to be responsible for PCT-CF management either due to concerns about the burden of follow-up, or to the risk that PCT-CF disclosure would cause reputational harm. (R2) Having less institutional participation could threaten what one PI deemed the “broader mission” (R25) of pragmatic research.

3.2.3 Consent

Several respondents deliberated about the challenges for PCT-CF management related to the absence of prospective informed consent for some PCTs, and correspondingly, the lack of evidence of patient-subject preferences regarding their disclosure to guide decision-making. For one PI the absence of consent meant there should be a “higher bar” (R27) for disclosure. Another PI offered similar observations, noting that the absence of consent would “influence the range of plausible solutions available” for management, as the patient-subjects would not be aware of research participation “but for” disclosure of the PCT-CF. (R22) However, the same respondent acknowledged that s/he could see consent “cutting in opposite directions,” arguing both for a higher and a lower bar for PCT-CF disclosure.

3.2.4 Other factors

Respondents mentioned a number of other considerations as potentially influential in decision-making regarding PCT-CF management. Several raised concerns related to liability, drawing analogies to clinical contexts in which the failure to notify a patient about a potentially preventable harm might lead to legal or professional sanctions. Others referenced patient or public expectations. One PI described considering how “a person in the street” would view obligations regarding PCT-CFs, and the expectation that laypersons would be unlikely to view the obligations of researchers as distinct from those of clinicians: “…anyone looking at this from the outside would say, ‘You're part of the healthcare system. You have absolutely every obligation that the healthcare system would have.” (R12)

3.3 Wide diversity of attitudes regarding researcher responsibilities for PCT-CFs

There was a wide diversity of attitudes regarding the scope of researcher responsibilities regarding the management of PCT-CFs, including whether researchers had obligations distinct from those of clinicians, and whether the source of data—from clinical care vs from research—affected the nature of those responsibilities.

For example, when asked whether the obligations of a researcher were distinct from those of a clinician with respect to the discovery of a finding with clinical relevance for an individual, a quality assurance leader stated: “I don't know the ethics around that one…I mean, I still think you're obligated to let them know.” In reflecting on this obligation, the respondent analogized to stopping after witnessing a car accident or to pick up trash observed along the ground, concluding there was a similar “moral obligation” to act upon information that could affect an individual's morbidity or mortality. (R10)

A PI expressed similar resistance to the moral relevance of the research-practice distinction when asked whether it was relevant if the finding resulted from a test done in the context of clinical care vs research:

Yeah, it's a great question…. Instinctually I don't think I would see it differently. I totally understand the distinction you're drawing, but I guess from where I stood the same criteria of magnitude of the stakes and plausibility of constructive intervention would be my guiding principles regardless of whether the source of the collateral finding was a protocolized ‘extra-usual-care phenomenon’ or merely part of usual care. (R22)

Others, however, relied heavily on the research-care distinction when assessing responsibilities for PCT-CF management. As one PI explained:

But even though [researchers] have the opportunity to [examine potential PCT-CFs], that's not the role of the people who are visualizing the data, that's the role of the clinicians taking care of the patient. I don't think it's the role of the research team who are being given access to this data, I don't think it's their role to be responding to it and changing care as a result… (R8)

Another PI offered similar observations in explaining the rationale for concluding that researchers did not have an obligation to communicate a PCT-CF to individuals, stating:

I was of the opinion that this was a clinical trial. This was a trial. And that patients were deidentified for a reason and that we shouldn't contact the patients and we shouldn't contact the providers. It should be just like any other study because I think that you could, whenever you have this much data that you could find all kinds of things…it's kind of a slippery slope…the health system had agreed to do this [study] and now you can't really go back and say ‘oh now we're going to dump all this other stuff on you that you weren't expecting and that you didn't agree to. (R4)

Relatedly, there was not shared agreement among researchers as to the appropriate scope of data collection and subsequent analysis within PCTs, which could, in turn, shape the likelihood of identifying a PCT-CF. For several respondents, minimizing the likelihood of identifying a PCT-CF was consistent with good research practice. As one PI said, this means, in part, “collecting exactly the data that you need,” which, in this PI's view, restricted the likelihood of encountering PCT-CFs. Collecting data elements beyond those which are “directly relevant to some element of your conceptual model or the outcomes of your study” ultimately represents a “misuse of system resources.” (R33) Another PI offered similar observations, describing that their team “tried to be really specific about the data that we requested from our site so as to really focus on the research question and nothing else,” something they characterized as consistent with “standard research practice.” (R5) Conversely, other researchers offered comments consistent with a view towards a more expansive scope for data collection and analyses in PCTs. In the words of one such respondent,

[a]s an investigator, I feel like we're obligated to use our federal resources to glean as much valuable information as possible in the context of the study. I mean beyond our primary and secondary aims… if we can address broader system level issues at the end of the day then all the better. (R19)

According to another,

We could think of probably 20 or 30 other analyses we can do with this existing data that could be used by the health care organization, by patients and providers to improve care. But the fact is, we're not. There's no funding source to actually use this information. (R2)

3.4 Prospective planning critical, but not widely undertaken Numerous respondents, including IRB professionals, quality assurance leaders, researchers, and health system leaders indicated that the issue of PCT-CFs was one they had not previously considered. Several, particularly those with affiliations to a trial that encountered a PCT-CF, also suggested that PCT-CFs, were likely to become more frequent. As one investigator explained

…honestly, I hadn't thought about [PCT-CFs] before, but I'm not even slightly surprised, and I'm sure this is the tip of the iceberg in terms of other potential [collateral] findings that may be of clinical relevance that were never considered in the original design. (R2)

A quality assurance leader had a similar observation:

I hadn't really thought about [PCT-CFs]…my take-home would be for me and from where I sit if this comes up…which it invariably will, and, as we talked about, our technology is certainly outstripping our ability to know what to do with this information that we're getting…I've not really thought about this until today—I don't think internally we do have any guidelines related to what do we do with this information and how do we disseminate it back out to patients and what's the messaging around that. (R10)

While no respondents described considering the potential for PCT-CFs during the trial-planning phase, many characterized proactive consideration as critically important. This view was represented across a range of roles, including IRB professionals, researchers, and health system leaders. As one PI explained, “more thought should go into [PCT-CFs] and a plan made and placed ahead of time…it's always good to have a plan ahead of time so people aren't just scrambling around, and you could just be more thoughtful about it.” (R18) A health care delivery system leader connected this insight to what he perceived as the value of a more general lesson, namely, the importance of involving “relevant stakeholders in the design and interpretation of the findings,” (R33), who can help anticipate potential PCT-CFs. Others observed that, even with prospective consideration, some unexpected PCT-CFs would likely arise. However, acknowledging that some surprising findings will occur despite planning did not, for these respondents, negate the value of planning. As one IRB professional explained:

…there's only so much that can be anticipated when you have a bunch of people sitting around in a room. When you actually go out into the real world…you will inevitably encounter things that you weren't expecting, maybe you should've expected, or maybe there's just such a novel finding that it has never been seen before….[but] even though you can't anticipate all the things you might find, anticipate that there's at least going to be something that you're going to find that you didn't anticipate. (R29)

4 DISCUSSION

This is the first qualitative study to explore the experiences and perspectives of diverse stakeholders involved in the design, conduct, and oversight of PCTs regarding the management of PCT-CFs. Our interviewees offered a wide range of experiences and insights related to PCT-CFs. Several themes that emerged from these interviews can help to inform the ethical management of PCT-CFs in future PCTs, while also suggesting additional avenues for research.

First, the liminal, boundary-spanning nature of PCTs complicates downstream issues related to PCT-CFs, including identifying and characterizing a PCT-CF, assessing whether a responsibility exists to manage it, and if so, by whom. Notably, no respondent had previously considered PCT-CFs and their appropriate management. Furthermore, even when an example PCT-CF was offered to them, respondents generally struggled with how best to categorize it, often seeking to analogize it to a more familiar context, such as an IF in research or something akin to a QI activity. It was not uncommon for the same PCT-CF to be viewed differently by different respondents. Perhaps not surprisingly, then, respondents often had quite different views about whether a particular PCT-CF required further management, such as communication to patients or their clinicians, and, if so, upon whom that responsibility should rest.

Second, our data suggest that there is likely no “one-size-fits-all” approach for PCT-CF management. Rather, the heterogenous nature of PCTs themselves, as well as the PCT-CFs that may arise in them, indicate the importance of considering context-specific features, including the clinical relevance of the finding, and the opportunity costs associated with subsequent management, such as communication of the PCT-CF to patients or clinicians, and any downstream effects on additional testing or other health system resources. While it is beyond the scope of this paper to address this complex issue in detail, it is critical to realize that there will inevitably be some PCT-CFs that will clearly need to be disclosed to patients due to factors such as patient well-being, actionability, and the likelihood that the finding would not otherwise be known if not for the PCT. Regardless, our findings lend support to proposals made elsewhere for the development of typologies and case studies to support decision-making regarding both whether and how PCT-CFs should be disclosed, and to whom.12

Third, like the broader discussions regarding PCTs and related efforts to integrate research into clinical care systems (eg, learning health systems), our data indicate a lack of agreement among key stakeholders about the continued relevance of the research-practice distinction.13, 14 Some interviewees described researchers, including those conducting PCTs, as having obligations that were distinct from those of clinicians, which in turn may imply a limited duty to undertake such activities as contacting patients regarding PCT-CFs with potential clinical relevance.5, 15 Others, however, seemed less inclined to perceive the distinction as relevant, focusing instead on such factors as the potential to avert health harms, or the expectation that patients or the public would not find such distinctions relevant, and thus a failure to take action would undermine patient trust. Our data also indicate disagreement as to the moral relevance of the data source, such as whether the PCT-CF arose through analyses of data collected from clinical care vs data generated specifically for research, and about the appropriate scope of data collection in PCTs more broadly.

Fourth, our data indicate the importance of prospective planning for PCT-CFs. As mentioned previously, no respondent described previously considering the possibility of PCT-CFs. However, there was widespread agreement about the importance of prospective planning as a means to guide CF management. For example, funders may wish to consider that study teams submit formal descriptions outlining the potential for PCT-CFs related to their data collection and analysis plans, and a corresponding process for their management.

Despite the importance of these findings for future practice and scholarship, our study had several potential limitations that should be considered. For instance, many respondents reported no prior experience with PCT-CFs. While we presented real cases drawn from their or other health systems as part of our inquiry, respondents' discussions related to how they or their systems might or should respond to a PCT-CF were often hypothetical. However, their attitudes and beliefs provide key insights into the relevant factors likely for future PCT-CF management. In addition, we spoke to a limited number of respondents, and thus their reactions may not be generalizable to other settings. Nevertheless, we identified few emergent themes from our later interviewees, which suggests that we achieved thematic saturation, at least within the stakeholder types we interviewed. Future research should explore the perspectives of other stakeholders who may reasonably be called upon to advise in the management of PCT-CFs, such as data safety monitoring boards. Relatedly, our qualitative study was not designed to assess whether there are any differences based upon the nature of the health system in which the PCT-CF arose or the role of the respondent. For example, it seems reasonable to hypothesize that PCT-CF management may proceed differently within integrated delivery systems with defined patient populations and expressed commitments to advance health through conducting pragmatic research targeted to the needs of their health plan members than within a traditional fee-for-service health system. Future work should investigate such variations.

Despite these limitations, it is clear that prospective planning regarding the possibilities of PCT-CFs is essential. Prospective planning is especially important for several reasons. First, since the context of a particular PCT will play a critical role in determining the appropriate management of a PCT-CF, simple guidelines are unlikely to be helpful. In addition, many of those who will be engaged in decision-making and disclosure of PCT-CFs are unlikely to have experience doing so, so planning should make those processes less ad hoc should they become necessary. Furthermore, the increased adoption of PCTs as a means of generating data to inform health care decision-making will undoubtedly be associated with an increased prevalence of PCT-CFs. Given that PCTs can involve the use of interventions and/or data use without explicit consent of patients and sometimes clinicians, it is essential that they are conducted in a trustworthy fashion, and this arguably includes the proper management of PCT-CFs. Regardless, further deliberation about the ethical obligations regarding PCT-CFs is needed. To enhance the likelihood of developing sound policies and practices, such deliberations should include the input and perspectives of key stakeholders in PCTs, including professionals, policy makers and patients.

ACKNOWLEDGEMENTS

This work was supported within the National Institutes of Health (NIH) Health Care Systems Research Collaboratory by the NIH Common Fund through cooperative agreement U24AT009676 from the Office of Strategic Coordination within the Office of the NIH Director. Supplemental funding for this work was provided by the National Center for Complementary and Integrative Health under award number U24AT009676. The views presented here are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health. The views presented here are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health.

CONFLICT OF INTEREST

Dr. Sugarman serves on Merck KGaA's Bioethics Advisory Panel and Stem Cell Research Oversight Committee; IQVIA's Ethics Advisory Panel; Apsen Neurosciences Scientific Advisory Board; and consulted with Biogen and Portola Pharmaceuticals. None of these are related to the material described in this manuscript. None of the other authors have relationships to disclose.