IntroductionBackground

Emotions lie at the core of human existence, emerging as products of evolutionary mechanisms designed to equip individuals to handle critical interpersonal interactions and life challenges [,]. Since the dawn of psychotherapy, the role of emotions, whether emanating from the patient or the therapist, has been recognized as crucial to the therapeutic journey [-].

Studies on emotional expression within the therapeutic realm emphasize their substantial influence on the outcomes of both single sessions and the entire treatment journey [,,]. Emotions intertwine with both specific and generic therapeutic methodologies that underpin clinical success [,]. Facilitating patients to gain emotional awareness and use these emotions constructively stands central to therapeutic transformation [,]. For this transformative effect, therapists must engage collaboratively with the patient’s in-session emotions, guiding them through recognition, acceptance, regulation, and management [-].

The therapeutic setting can illuminate suppressed emotions, allowing patients to confront neglected self-awareness and experience new emotional dimensions. Such insights present valuable data on patients’ inherent needs and reactions, offering a platform to confront past fears and integrate new experiences [,]. Emotions directed toward the therapist during sessions hold significant implications for the therapeutic journey’s success [,]. An environment characterized by empathy and affirmation determines if the emotional engagement will steer the therapy positively or detrimentally []. Furthermore, such an environment is both the catalyst for change and the foundation for effective psychotherapy [].

The dawn of the new millennium saw the advent of routine outcome monitoring and feedback []. These systems, which track and report the patient’s therapeutic progress, have seen widespread adoption and study in varied mental health contexts over recent years. These feedback mechanisms, unified by their use of standardized measures, monitor outcomes like well-being, symptomatology, and functional aspects, sometimes focusing on specific therapeutic processes like the working alliance []. Meta-analyses show that systematic feedback enhances therapy outcomes by reducing symptom intensity and attrition rates [,]. They also highlight reductions in treatment time frames and associated costs [-].

Given these insights, the American Psychological Association has been advocating for feedback and outcome monitoring since the early 2000s [,]. Feedback systems are now integral to mental health frameworks globally, especially in countries such as Australia, the Netherlands, and the United Kingdom, where monitoring outcomes is a statutory mandate for health service providers. With the capacity to tweak treatments based on progress or disruptions in the therapist-patient relationship, many experts regard progress monitoring as a pivotal strategy to enhance real-world therapy outcomes [,].

Study Objectives

In light of the above, the primary objective of this study is to examine the influence of a brief postsession battery, which is apt to foster patient self-monitoring and reflection on their emotional reactions toward their psychotherapist and on the quality of the therapeutic relationship and treatment outcomes.

Secondary objectives are to (1) explore whether and how the characteristics of the patient, the therapist, and the process moderate the effect of regular self-monitoring on the therapeutic relationship and outcomes; (2) examine the relationships between the affective response of the patient, the alliance, and the result of the therapy session outcome; and (3) explore how the affective responses of the patient unfold or change throughout the course of therapy.

MethodsEthical Considerations

The institutional review board of the University of North Carolina (UNC) at Chapel Hill approved this study (23-1067) on July 31, 2023. This research was designed and executed in accordance with ethical standards for studies involving human participants.

Before starting the baseline survey, participants will receive a detailed web-based consent form regarding the study. This form will include the purpose, rationale, and methodology of the study, as well as the contact information for both the principal investigator and the institutional review board. It will be explained to potential participants that to protect their identities, only relevant and minimally necessary information will be collected. Furthermore, it will be assured that any published results will report on groups, not individual participants. Participants will be prompted with a consent statement and given options: “I consent to participate in this study” or “I do not consent to participate in this study.” Those who choose not to consent will be immediately directed to a closing page. All participants are informed that they can opt out of the study at any time without providing a reason.

Only the central research team will have access to the data until the research results are published. Once the study concludes and the primary findings are released, external applications for access to deidentified data will be entertained. These applications must adhere to data privacy rules, and the intended research must be scientifically and ethically robust. Following the publication of the research findings, the data will be publicly shared through the Open Science Framework [].

No financial incentives were offered for participation in this trial. However, upon completing the trial, participants from both the intervention and control groups were given the opportunity to request a free copy of the scales and general feedback used in the intervention group by emailing the principal investigator.

Study Design

This study is a randomized controlled trial (NCT06038747) of 1:1 for individual psychotherapy versus individual psychotherapy plus self-monitoring. A predetermined computer-generated randomization list will be generated to assign patients to 1 of the 2 arms. Patients will not be blind to treatment allocation. Patients will be enrolled through the web-based recruitment platforms “ResearchMatch” and “Research for Me,” and data will be collected through web-based surveys facilitated by the Qualtrics software hosted by the UNC at Chapel Hill.

Intervention

Participants in the intervention group will continue their regular individual psychotherapy sessions. Additionally, after each session, they will complete a brief postsession battery consisting of 2 scales on the affective reactions of the patients toward their therapist during the session and read general feedback encouraging them to discuss their feelings and reflections with their therapist. This questionnaire aims to prompt reflection on one’s own experience of the therapeutic relationship.

Treatment as Usual

Patients in the control group will receive only their regular individual psychotherapy (treatment as usual) and will not complete postsession questionnaires. Additionally, they will not receive any communication encouraging them to discuss their emotional responses with their therapist during the session.

Measures

This study uses a comprehensive set of self-report instruments to collect extensive data on the therapeutic process and various patient-related factors. The data will cover 5 main domains. The sociodemographic and clinical domains capture the patient’s demographic, clinical, and treatment specifics. The mental health state domain focuses on the present symptoms of anxiety and depression that impact the daily lives of respondents. The therapeutic relationship domain evaluates the specific facets of the relationship between the patient and the therapist. The last domain evaluates session and therapy outcomes.

The primary outcome measure of the trial is the “Clinical Outcomes in Routine Evaluation-Outcome Measure” (CORE-OM) [], while the secondary outcomes are the “Real Relationship Inventory-Client-Short Form” (RRI-C-SF) [], the “Working Alliance Inventory-Short Revised” (WAI-SR) [], and the number of scheduled therapy sessions that have been missed or canceled by the patient.

Sociodemographic Domain

The “sociodemographic data form” collects self-reported details such as the patient’s age, gender, education level, and ethnicity.

Clinical Domain

The “clinical data form A” gathers patient-provided data on the duration and frequency of the ongoing psychotherapy, session attendance method (in-person, video call, telephone, and mixed), presence or absence of a diagnosed mental disorder, any prescribed psychotropic medications, and the therapist’s gender.

The “clinical data form B” records information on sessions attended, missed, and canceled over the last 5 weeks and any change in psychiatric medications during this time frame.

Mental Health State Domain

The Patient Health Questionnaire-9 (PHQ-9) [] is a 9-item self-report depression screening tool that captures the severity of depressive symptoms during the past week. It allows the evaluation of symptoms according to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria for major depression. The PHQ-9 administered in the primary care sample showed a Cronbach α of 0.89.

The Generalized Anxiety Disorder-7 (GAD-7) [] is a 7-item self-report anxiety screening tool with a focus on symptoms of generalized anxiety disorder. It assesses the severity and frequency of anxiety symptoms during the past week. GAD-7 showed a Cronbach α of 0.92.

Therapeutic Relationship Domain

The RRI-C-SF [] is an 8-item self-report tool measuring the “real relationship” between a therapist and patient from the patient’s viewpoint. It includes 2 subscales: “genuineness” and “realism,” both of which represent closely related dimensions of authenticity and perceptual accuracy within the therapeutic relationship. The RRI-C-SF has a McDonald’s ω total of 0.93.

The WAI-SR [] is a 12-item self-report measure of the quality of the working alliance between patient and therapist from the patient’s perspective. WAI-SR includes three 4-item subscales that focus on agreement on therapy tasks, goals, and the development of an emotional bond. Items are rated on a 6-point Likert scale, with higher scores indicating a more robust alliance. The Cronbach α for the total scale was 0.91, while the α coefficients for the subscales spanned a range from 0.85 to 0.90.

The “in-Session Patient Affective Reactions Questionnaire” (SPARQ) [,] is a patient-reported tool that comprises 8 items that explore the patterns of thought, feeling, and behavior activated and experienced by the patients toward their therapist during a session. It consists of 2 scales: “positive affect” and “negative affect.” The positive affect scale exhibited a Cronbach α coefficient of 0.84 and an average interitem correlation of 0.51. On the other hand, the negative affect scale demonstrated an α of 0.76 and an average interitem correlation of 0.39.

The “Rift In-Session Questionnaire” (RISQ) [] is a 4-item self-report questionnaire designed to measure the patient’s risk of experiencing ruptures in the therapeutic relationship. The RISQ measures feelings of belittlement, rejection, disparagement, and attack, as well as any tendencies toward disobedience or “naughtiness” toward the therapist. The RISQ demonstrated good internal consistency with a Cronbach α coefficient of 0.73 and an average interitem correlation of 0.35.

Session and Therapy Outcomes Domain

The Session Evaluation Scale (SES) [] assesses the quality of the therapy session from the perspective of the patient. The SES comprises 5 items rated on a 5-point Likert scale and showed Cronbach α coefficients ranging from 0.88 to 0.89.

The CORE-OM [] is a 34-item self-report measure of change in psychotherapy that comprises 4 domains: subjective well-being, symptoms, function, and risk. The CORE-OM demonstrated good internal and test-retest reliability between 0.75 and 0.95, and convergent validity against a battery of validated measures.

Efforts to Minimize Potential Sources of Bias

In our research, extensive steps will be taken to minimize potential biases. First, participants will be sourced from ResearchMatch, a national web-based registry that boasts a broad and varied volunteer pool from all over the United States, reducing selection bias and enhancing the external validity of our study. Second, our methodology is closely tuned to gender perspectives, giving weight to nonbinary gender as a key factor in both data evaluation and result interpretation, fortifying the scientific rigor and clinical significance of the study. Third, our comprehensive set of measures covers a spectrum of therapeutic and individual aspects, from sociodemographic details to personality characteristics, providing a comprehensive snapshot of the therapeutic landscape and limiting measurement bias. Data will be gathered through Qualtrics, a trusted web-based tool that protects respondent privacy and minimizes potential biases in responses. Furthermore, great care was taken in calculating the required sample size, accounting for potential dropouts, to ensure that we had ample statistical power, thereby sidestepping the pitfalls of weak analyses or insufficient representation.

Sample Size Calculation

Assuming a power of 0.80, a 2-tailed α level of .05, an effect size (d) of 0.50, and an allocation ratio of 1:1, we calculated a minimum sample size of 128 patients, or 64 participants per group, using the R package pwr (version 1.3-0; The Comprehensive R Archive Network). However, to enhance statistical power and precision, we aim for a target total sample size of 520 participants, with 260 participants per group. This adjustment also accounts for the possibility that only 50% of participants in the intervention group will complete all postsession scales.

Inclusion and Exclusion Criteria

Participants are eligible for the study if they are aged 18 years or older, fluent in English, and currently undergoing individual psychotherapy with a minimum frequency of 2 sessions per month. Individuals with a legal guardian will be excluded from the study.

Recruitment

Patients will be enrolled through ResearchMatch and Research for Me. ResearchMatch [] is a disease-neutral, institution-neutral, US national web-based registry designed to enroll volunteers for clinical studies. Established by various educational entities and supported in part by the National Institutes of Health’s National Center for Advancing Translational Sciences, this platform provides access to more than 155,000 volunteers across the United States and has proven effective. Research for Me is a community of volunteers acting as the primary gateway for patients and local residents who wish to participate in research at the UNC at Chapel Hill. This initiative was established by the North Carolina Translational and Clinical Sciences Institute, which is the central component of the National Institutes of Health’s Clinical and Translational Science Awards program at UNC at Chapel Hill. Data suggest that individuals enrolled through online research platforms reliably report their demographic and psychological details, especially when there is no monetary incentive involved [].

Randomization

Participants will be assigned to either the intervention or control group using a pre-established computer-generated randomization list. This assignment will occur after the baseline assessment is completed.

Data Collection Process

Participants in both the intervention and control groups will undergo assessments at baseline, midtrial (week 5), and end-of-trial (week 10). Additionally, those in the intervention group will have postsession evaluations. These assessments will be delivered through Qualtrics, a secure web-based survey platform hosted by the UNC at Chapel Hill.

Baseline assessment: All participants will complete the demographic and clinical data forms in addition to the measures for mental health state, therapeutic relationship, and session and therapy outcome domains ().

Table 1. Instruments and time line for assessment in the trial.Domains and measuresBaselineMidtrialaEnd-of-trialbDemographic domain
Sociodemographic data form✓

Clinical domain
Clinical data form A✓


Clinical data form B
✓✓Mental health state domain
Patient Health Questionnaire-9 []✓✓✓
Generalized Anxiety Disorder-7 []✓✓✓Therapeutic relationship domain
Real Relationship Inventory-Client-Short Form []✓✓✓
Working Alliance Inventory-Short Revised []✓✓✓
in-Session Patient Affective Reactions Questionnaire [,]✓✓✓
Rift In-Session Questionnaire []✓✓✓Session and therapy outcome domain
Session Evaluation Scale []✓✓✓
Clinical Outcomes in Routine Evaluation-Outcome Measures []✓✓✓

a5 weeks after the baseline assessment.

b10 weeks after the baseline assessment.

Postsession assessment (only for the intervention group): participants in the intervention group will be required to complete SPARQ and RISQ [] after each session throughout the 10-week duration of the trial. Immediately after completion of the postsession battery, participants will be presented with a general feedback statement emphasizing the importance of discussing session-related feelings with their therapist.

Mid-trial assessment: 5 weeks postbaseline evaluation, all participants will retake the scales used in the baseline assessment ().

End-of-trial assessment: 10 weeks after the baseline, all participants will again be asked to complete the initial assessment battery ().

Dealing With Missing Values

We will examine data for degree and patterns of missingness at the patient and scale level, use propensity scoring to model the pattern of missingness, and include the propensity score as a covariate in analyses if indicated []. Missing items will be prorated so long as at least 75% of the items are available, and otherwise the scale score will be treated as missing [].

Planned Analysis

A comprehensive statistical strategy will be formulated and documented before starting the data analysis. R (R Foundation for Statistical Computing) will be used for data cleaning, labeling, scale scoring, and subsequent analyses. The basic characteristics of the study participants will be clearly described for each arm while also being aggregated to a sufficient level to reduce the risk of deductive identification. Baseline categorical variables will undergo a comparative evaluation between the intervention and control groups using chi-square evaluations. Continuous variables will be compared using 2-tailed t tests and Mann-Whitney U tests. Outcome metrics will be presented for each research group at various intervals. These outcomes will be detailed based on cumulative scores and the percentage of participants that demonstrate improvement from the start point.

Statistical Analysis

Analysis of covariance will be used to evaluate the influence of treatment conditions on primary and secondary outcomes since it is the most appropriate statistical method for the analysis of continuous outcomes in RCTs [,]. Analysis of covariance is a linear regression in which treatment assignment and baseline scores are included as covariates. Patient clinical and demographic characteristics, as well as therapist demographic and professional characteristics, will be included as exploratory variables. A total of 2 analyses will be carried out: the intention-to-treat analysis to examine the general effect and the per protocol analysis as a sensitivity analysis. The time will be measured in weeks, and the baseline is coded with the value 0. Analyses will be performed using R statistical computing software (version 4.3.1 or higher; R Foundation for Statistical Computing).

Trial Status

At the time of submission of this protocol’s manuscript, recruitment is complete.

Dissemination Policy

The results of this research project will first appear as preprints and subsequently be shared through scholarly journals and presentations at conferences. The Open Science Framework will host a repository containing study tools and data, scoring guidelines, presentations, and preprint versions. Wikiversity pages, tailored to offer technical assets for both practitioners and researchers, will feature links directing to this repository. Furthermore, this study’s conclusions could be communicated to pertinent mental health associations, contributing to future studies and potential improvements in psychotherapeutic relationships. Our goal is to adopt a comprehensive and broad-based communication approach that engages scholars, health care professionals, and the general public.

Results

Participant recruitment started in September 2023. Baseline data collection was completed in December 2023, with a total of 520 recruited participants, 475 of whom completed the baseline assessment (243 participants were assigned to the intervention group and 232 were assigned to the control group). Data was completed February 2024. Data analysis has not begun as of the time of submission. The results are expected to be published in the autumn of 2024.

Discussion

This clinical study is designed to assess the impact of introducing a concise postsession questionnaire able to trigger the self-awareness and introspection of patients about their experience of the psychotherapeutic relationship. The primary goal is to understand its influence on the effectiveness of treatment, with the CORE-OM serving as the primary outcome measure.

Should our battery prove to increase symptom reduction rates or improve overall well-being, its ease of delivery directly to patients—requiring no additional therapist involvement—suggests potential for seamless adoption across therapeutic contexts without incurring extra expenses for patients or placing added demands on therapists. This could make the system highly cost-efficient. Furthermore, the insights from this study could shed light on the nuanced interaction between specific elements of the therapeutic bond and the results of sessions or general treatment. This knowledge would empower clinicians and policy makers to recognize and prioritize those aspects of the therapeutic relationship that strengthen or hinder the efficacy of psychotherapy.

One of the main strengths of this study is its ability to gauge the benefits of regular patient self-monitoring and subsequent reflection on their emotional reactions toward their therapists in a real-world setting. The study sample accurately represents the diverse patient population within the United States, further enhancing its relevance. Other key advantages include robust statistical power, the use of an external outcome measure not related to the intervention, and a comprehensive consideration of the patient, the therapist, and treatment variables. However, there are limitations, such as the lack of information from the clinician or observer and the extended duration of data collection. Furthermore, an important potential limitation inherent in our approach concerns the adherence of the participants to the research protocol, specifically the regular completion of the battery after each therapy session, and their compliance with the recommendation to share their emotional responses toward their therapist directly and openly with the therapist.

To this day, efforts to implement and test regular self-monitoring of patient affective experiences during sessions, with systematic feedback provided directly to patients without the therapist’s intervention, seem inadequate. The introduction of a newly developed systematic client feedback system for psychotherapy may offer improvements in treatment outcomes while also serving as a cost-effective tool in real-world clinical practice.

Data sharing is not applicable to this article, as no data sets were generated or analyzed during this study protocol. The data sets generated during the actual research study will be made available in the Open Science Framework repository [].

EV has received grants and served as consultant, advisor, or CME speaker for the following entities: AB-Biotics, AbbVie, Angelini, Biogen, Biohaven, Boehringer-Ingelheim, Celon Pharma, Compass, Dainippon Sumitomo Pharma, Ethypharm, Ferrer, Gedeon Richter, GH Research, Glaxo-Smith Kline, Idorsia, Janssen, Lundbeck, Medincell, Novartis, Orion Corporation, Organon, Otsuka, Rovi, Sage, Sanofi-Aventis, Sunovion, Takeda, and Viatris, outside the submitted work. EAY is the co-founder and Executive Director of Helping Give Away Psychological Science, a 501c3; he has consulted about psychological assessment with Signant Health and received royalties from the American Psychological Association and Guilford Press, and he holds equity in Joe Startup Technologies.

Edited by A Mavragani; The proposal for this study was peer reviewed by the European Commission.submitted 11.12.23; accepted 30.01.24; published 08.04.24.

©Alberto Stefana, Eduard Vieta, Paolo Fusar-Poli, Eric A Youngstrom. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 08.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.