Background: Adaptive treatment strategies that can dynamically react to individual cancer progression can provide effective personalized care. Longitudinal multi-omics information, paired with an artificially intelligent clinical decision support system (AI-CDSS) can assist clinicians in determining optimal therapeutic options and treatment adaptations. However, AI-CDSS is not perfectly accurate, as such, clinicians' over/under reliance on AI may lead to unintended consequences, ultimately failing to develop optimal strategies. To investigate such collaborative decision-making process, we conducted a Human-AI interaction case study on response-adaptive radiotherapy (RT). Methods: We designed and conducted a two-phase study for two disease sites and two treatment modalities—adaptive RT for non-small cell lung cancer (NSCLC) and adaptive stereotactic body RT for hepatocellular carcinoma (HCC)—in which clinicians were asked to consider mid-treatment modification of the dose per fraction for a number of retrospective cancer patients without AI-support (Unassisted Phase) and with AI-assistance (AI-assisted Phase). The AI-CDSS graphically presented trade-offs in tumor control and the likelihood of toxicity to organs at risk, provided an optimal recommendation, and associated model uncertainties. In addition, we asked for clinicians' decision confidence level and trust level in individual AI recommendations and encouraged them to provide written remarks. We enrolled 13 evaluators (radiation oncology physicians and residents) from two medical institutions located in two different states, out of which, 4 evaluators volunteered in both NSCLC and HCC studies, resulting in a total of 17 completed evaluations (9 NSCLC, and 8 HCC). To limit the evaluation time to under an hour, we selected 8 treated patients for NSCLC and 9 for HCC, resulting in a total of 144 sets of evaluations (72 from NSCLC and 72 from HCC). Evaluation for each patient consisted of 8 required inputs and 2 optional remarks, resulting in up to a total of 1440 data points. Results: AI-assistance did not homogeneously influence all experts and clinical decisions. From NSCLC cohort, 41 (57%) decisions and from HCC cohort, 34 (47%) decisions were adjusted after AI assistance. Two evaluations (12%) from the NSCLC cohort had zero decision adjustments, while the remaining 15 (88%) evaluations resulted in at least two decision adjustments. Decision adjustment level positively correlated with dissimilarity in decision-making with AI [NSCLC: ρ = 0.53 (p < 0.001); HCC: ρ = 0.60 (p < 0.001)] indicating that evaluators adjusted their decision closer towards AI recommendation. Agreement with AI-recommendation positively correlated with AI Trust Level [NSCLC: ρ = 0.59 (p < 0.001); HCC: ρ = 0.7 (p < 0.001)] indicating that evaluators followed AI's recommendation if they agreed with that recommendation. The correlation between decision confidence changes and decision adjustment level showed an opposite trend [NSCLC: ρ = -0.24 (p = 0.045), HCC: ρ = 0.28 (p = 0.017)] reflecting the difference in behavior due to underlying differences in disease type and treatment modality. Decision confidence positively correlated with the closeness of decisions to the standard of care (NSCLC: 2 Gy/fx; HCC: 10 Gy/fx) indicating that evaluators were generally more confident in prescribing dose fractionations more similar to those used in standard clinical practice. Inter-evaluator agreement increased with AI-assistance indicating that AI-assistance can decrease inter-physician variability. The majority of decisions were adjusted to achieve higher tumor control in NSCLC and lower normal tissue complications in HCC. Analysis of evaluators' remarks indicated concerns for organs at risk and RT outcome estimates as important decision-making factors. Conclusions: Human-AI interaction depends on the complex interrelationship between expert's prior knowledge and preferences, patient's state, disease site, treatment modality, model transparency, and AI's learned behavior and biases. The collaborative decision-making process can be summarized as follows: (i) some clinicians may not believe in an AI system, completely disregarding its recommendation, (ii) some clinicians may believe in the AI system but will critically analyze its recommendations on a case-by-case basis; (iii) when a clinician finds that the AI recommendation indicates the possibility for better outcomes they will adjust their decisions accordingly; and (iv) When a clinician finds that the AI recommendation indicate a worse possible outcome they will disregard it and seek their own alternative approach.

D Niraula, KC Cuneo, ID Dinov, JB Jamaluddin, J Jin, Y Luo, RK Ten Haken, AK Bryant, MP Dykstra, JM Frakes, CL Liveringhouse, SR Miller, MN Mills, RF Palm, SN Regan, and A Rishi have no conflicting interests. BD Gonzalez reports fees unrelated to this work from Sure Med Compliance and Elly Health. MM Matuszak reports research funding from Varian, a licensing agreement with Fuse Oncology, and serves in the AAPM Board of Directors and is the Co-Director of MROQC, funded by BCBSM. TJ Dilling is a member of the National Comprehensive Cancer Network (NCCN) NSCLC panel. JF Torres-Roca reports stock ownership and leadership in Cvergenx, Inc. He reports IP and royalty rights in RSI, GARD, RxRSI. HHM Yu reports funding or fees unrelated to this work from the National Institute of Health, UpToDate, Novocure and Bristol-Myers Squib. I El Naqa is on the scientific advisory of Endectra, LLC., co-founder of iRAI LLC, deputy editor for the journal of Medical Physics, co-Chief editor of British Journal of Radiology (BJR)-AI and receives funding from the National Institute of Health (NIH), foundations, and Department of Defense (DoD). A PCT patent application for ARCliDS has been filed. Patent Applicant: H Lee Moffitt Cancer Center IP office in conjunction with University of Michigan IP office. Inventors: Dipesh Niraula, Issam El Naqa, Randall K. Ten Haken, Wenbo Sun, Judy Jin, Ivo Dinov, Kyle Cuneo, Martha M Matuszak, and Jamalina Jamaluddin. Application Number: US2023/075004. Status of Application: Pending. Specific aspect of manuscript covered in patent application: The patent covers the underlying model-based decision-making framework of ARCliDS.

This work was partly supported by the National Institute of Health (NIH) grant R01-CA233487 and its supplement.

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

Ethics committee/IRB of Moffitt Cancer Center and ethics committee/IRB of University of Michigan gave ethical approval for this work.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Part of the data produced in the present study are contained in the manuscript. All data produced in the present study will be published after peer-review process.